Upload folder using huggingface_hub

README.md

@@ -1,239 +1,10 @@
 ---
+tags:
+- unsloth
+- mlx
 base_model: unsloth/Qwen3-Coder-30B-A3B-Instruct
-language:
-- en
 library_name: mlx
 license: apache-2.0
-tags:
-- cdxgen
-- transformers
-- sbom
-- supply-chain-security
-- mlx
+license_link: https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct/blob/main/LICENSE
 pipeline_tag: text-generation
-datasets:
-- CycloneDX/cdx-docs
 ---
-# Abstract
-
-We present [cdx1](https://huggingface.co/collections/CycloneDX/cdx1-67a616a859ac0582df99700b) and [cdx1-pro](https://huggingface.co/collections/CycloneDX/cdx1-pro-688e15a3c3b593753ceefc05), a family of language models designed to emulate the expertise of a professional in DevOps, xBOM (Bill of Materials), and the CycloneDX specification. The base models, `unsloth/Qwen2.5-Coder-14B-Instruct` (for cdx1) and `unsloth/Qwen3-Coder-30B-A3B-Instruct` (for cdx1-pro), were fine-tuned on a specialized, high-quality [dataset](https://huggingface.co/CycloneDX/datasets). This dataset was constructed using a synthetic data generation strategy with a teacher model (Gemini 2.5 Pro). The primary objective was to align the fine-tuned models' capabilities with the teacher model's performance on xBOM and CycloneDX-related question-answering tasks.
-
-## Approach to Data
-
-### Data Curation and Generation
-
-The models were trained on [cdx-docs](https://huggingface.co/datasets/CycloneDX/cdx-docs), a curated dataset comprising technical documentation, authoritative OWASP guides, and semantic interpretations derived from the CycloneDX Generator (cdxgen) source code. The dataset was augmented using a synthetic data generation technique. This process involved prompting a teacher model (Gemini 2.5 Pro) to generate question-answer pairs that encapsulate the nuances and semantics of the domain. The generated data was structured to facilitate effective learning by the target cdx1 models.
-
-### Alignment with Inference
-
-During the training phase, the dataset was iteratively refined to ensure the format and context of the training examples closely resembled the intended inference-time inputs. This alignment is critical for the models to learn the domain's complexity and respond accurately to real-world prompts.
-
-## Benchmarking
-
-The cdx1 models are optimized for xBOM use cases, including BOM summarization, component tagging, validation, and troubleshooting. To evaluate model performance, we developed a custom benchmark suite named [xBOMEval](https://github.com/CycloneDX/cdxgen/tree/master/contrib/xBOMEval).
-
-### Categories
-
-xBOMEval contains tests across the following categories:
-
-- **Bias:** Assesses potential model bias towards CycloneDX or SPDX specifications through targeted questions.
-- **Specification (Spec):** Measures factual recall and synthesis on topics such as CycloneDX, PURL, and SPDX.
-- **Logic:** Evaluates problem-solving and reasoning capabilities with complex questions about specifications.
-- **DevOps:** Assesses knowledge of platforms and tools like GitHub, Azure Pipelines, and package managers.
-- **Linux:** Tests proficiency with Linux environments, including terminal and PowerShell commands.
-- **Docker:** Measures understanding of Docker, Podman, and the OCI specification.
-
-### Scoring
-
-Model responses were scored using a combination of automated evaluation by a high-capability model (Gemini 2.5 Pro) and manual human review. To maintain benchmark integrity, the evaluation set was held out and not included in any model's training data. Detailed results and configurations are available in the `xBOMEval` directory of the [cdxgen repository](https://github.com/CycloneDX/cdxgen).
-
-## Benchmark Results - August 2025
-
-### Logic Category Comparison
-
-| Model               | Accuracy (%) |
-| :------------------ | :----------- |
-| `gemini-2.5-pro`    | 93.60        |
-| `deepthink-r1`      | 89.63        |
-| `gpt-5`             | 83.23        |
-| `deepseek-r1`       | 82.92        |
-| `gpt-oss-120b`      | 80.49        |
-| `gpt-oss-20b`       | 79.27        |
-| `cdx1-pro-mlx-8bit` | 73.17        |
-| `o4-mini-high`      | 67.99        |
-| `qwen3-coder-480B`  | 48.48        |
-| `cdx1-mlx-8bit`     | 46.04        |
-
-
-This table compares the accuracy of **ten** different AI models on a logic benchmark designed to assess reasoning and problem-solving skills. The results highlight a clear hierarchy of performance, with the newly added `gpt-5` debuting as a top-tier model.
-
-**Key Findings from the Chart:**
-
-- **Dominant Leader:** `gemini-2.5-pro` is the undisputed leader, achieving the highest accuracy of **93.6%**, placing it in a class of its own.
-- **Top-Tier Competitors:** A strong group of models follows, led by `deepthink-r1` at **89.63%**. The newly introduced **`gpt-5`** makes a powerful debut, securing the third-place spot with **83.23%** accuracy. It slightly outperforms `deepseek-r1` (82.92%) and `gpt-oss-120b` (80.49%).
-- **Strong Mid-Tier:** The `gpt-oss-20b` model performs impressively well for its size at **79.27%**, outscoring several larger models and leading the middle pack, which also includes `cdx1-pro-mlx-8bit` (73.17%) and `o4-mini-high` (67.99%).
-- **Lower Performers:** `qwen3-coder-480B` (48.48%) and `cdx1-mlx-8bit` (46.04%) score the lowest. It is noted that the score for `cdx1-mlx-8bit` is artificially low due to context length limitations, which caused it to miss questions.
-- **Efficiency and Performance:** The results from the `gpt-oss` models, particularly the 20B variant, demonstrate that highly optimized, smaller models can be very competitive on logic tasks.
-
-### Performance Tiers
-
-The models can be grouped into four clear performance tiers:
-
-- **Elite Tier (>90%):**
-  - `gemini-2.5-pro` (93.6%)
-- **High-Performing Tier (80%-90%):**
-  - `deepthink-r1` (89.63%)
-  - `gpt-5` (83.23%)
-  - `deepseek-r1` (82.92%)
-  - `gpt-oss-120b` (80.49%)
-- **Mid-Tier (65%-80%):**
-  - `gpt-oss-20b` (79.27%)
-  - `cdx1-pro-mlx-8bit` (73.17%)
-  - `o4-mini-high` (67.99%)
-- **Lower Tier (<50%):**
-  - `qwen3-coder-480B` (48.48%)
-  - `cdx1-mlx-8bit` (46.04%)
-
-### Spec Category Comparison
-
-| Model               | Accuracy (%) |
-| :------------------ | :----------- |
-| `gemini-2.5-pro`    | 100.00       |
-| `deepseek-r1`       | 98.58        |
-| `cdx1-pro-mlx-8bit` | 98.30        |
-| `gpt-5`             | 95.17        |
-| `qwen3-coder-480B`  | 90.34        |
-| `gpt-oss-120b`      | 89.20        |
-| `cdx1-mlx-8bit`     | 83.52        |
-| `deepthink-r1`      | 12.36        |
-| `gpt-oss-20b`       | 9.09         |
-| `o4-mini-high`      | 0.00         |
-
-
-This table evaluates **ten** AI models on the "Spec Category," a test of factual recall on 352 technical specification questions. The results starkly illustrate that a model's reliability and cooperative behavior are as crucial as its underlying knowledge. Several models, including the newly added `gpt-5`, achieved high scores only after overcoming significant behavioral hurdles.
-
-**Key Findings from the Chart:**
-
-- **Elite Factual Recall:** A top tier of models demonstrated near-perfect knowledge retrieval. **`gemini-2.5-pro`** led with a perfect **100%** score and superior answer depth. It was closely followed by **`deepseek-r1`** (98.58%) and **`cdx1-pro-mlx-8bit`** (98.3%).
-
-- **High Score with Major Caveats (`gpt-5`):** The newly added **`gpt-5`** achieved a high accuracy of **95.17%**, placing it among the top performers. However, this result required a significant compromise:
-  - The model initially refused to answer the full set of questions, only offering to respond in small batches that required six separate user confirmations. This compromise was accepted to prevent an outright failure.
-  - A related variant, `gpt-5-thinking`, refused the test entirely after a minute of processing.
-
-- **Complete Behavioral Failures:** Three models effectively failed the test not due to a lack of knowledge, but because they refused to cooperate:
-  - **`o4-mini-high`** scored **0%** after refusing to answer, citing too many questions.
-  - **`deepthink-r1`** (12.36%) and **`gpt-oss-20b`** (9.09%) also failed, answering only a small fraction of the questions without acknowledging the limitation.
-
-- **Strong Mid-Tier Performers:** `qwen3-coder-480B` (90.34%) and `gpt-oss-120b` (89.2%) both demonstrated strong and reliable factual recall without the behavioral issues seen elsewhere.
-
-- **Impact of Scale and Systematic Errors:** The contrast between the two `cdx1` models is revealing. The larger `cdx1-pro-mlx-8bit` (98.3%) performed exceptionally well, while the smaller `cdx1-mlx-8bit` (83.52%) was hampered by a single systematic error (misunderstanding "CBOM"), which cascaded into multiple wrong answers.
-
-### Summary of Key Themes
-
-1.  **Reliability is Paramount:** This test's most important finding is that knowledge is useless if a model is unwilling or unable to share it. The failures of `o4-mini-high`, `deepthink-r1`, `gpt-oss-20b`, and the behavioral friction from `gpt-5` highlight this critical dimension.
-2.  **Scores Don't Tell the Whole Story:** The 95.17% score for `gpt-5` obscures the significant user intervention required to obtain it. Similarly, the near-identical scores of `cdx1-pro` and `gemini-2.5-pro` don't capture Gemini's superior answer quality.
-3.  **Scale Can Overcome Flaws:** The dramatic performance leap from the 14B to the 30B `cdx1` model suggests that increased scale can help correct for specific knowledge gaps and improve overall accuracy.
-
-### Other Categories
-
-Performance in additional technical categories is summarized below.
-
-| Category | cdx1-mlx-8bit | cdx1-pro-mlx-8bit |
-| -------- | ------------- | ----------------- |
-| DevOps   | 87.46%        | 96.1%             |
-| Docker   | 89.08%        | 100%              |
-| Linux    | 90.6%         | 95.8%             |
-
-## Model Availability
-
-The `cdx1` and `cdx1-pro` models are provided in multiple formats and quantization levels to facilitate deployment across diverse hardware environments. Models are available in the **MLX** format, optimized for local inference on Apple Silicon, and the **GGUF** format, which offers broad compatibility with CPUs and various GPUs. The selection of quantization levels allows users to balance performance with resource consumption, enabling effective operation even in environments with limited VRAM.
-
-The table below details the available formats and their approximate resource requirements. All quantized models can be found on [Hugging Face](https://huggingface.co/CycloneDX/models).
-
-| Model              | Format | Quantization | File Size (GiB) | Est. VRAM (GiB) | Notes                                      |
-| :----------------- | :----- | :----------- | :-------------- | :-------------- | :----------------------------------------- |
-| **cdx1 (14B)**     | MLX    | 4-bit        | ~8.1            | > 8             | For Apple Silicon with unified memory.     |
-|                    | MLX    | 6-bit        | ~12             | > 12            | For Apple Silicon with unified memory.     |
-|                    | MLX    | 8-bit        | ~14.2           | > 14            | Higher fidelity for Apple Silicon.         |
-|                    | MLX    | 16-bit       | ~30             | > 30            | bfloat16 for fine-tuning.                  |
-|                    | GGUF   | Q4_K_M       | 8.99            | ~10.5           | Recommended balance for quality/size.      |
-|                    | GGUF   | Q8_0         | 15.7            | ~16.5           | Near-lossless quality.                     |
-|                    | GGUF   | BF16         | 29.5            | ~30             | bfloat16 for fine-tuning.                  |
-| **cdx1-pro (30B)** | MLX    | 4-bit        | ~17.5           | > 18            | For Apple Silicon with unified memory.     |
-|                    | MLX    | 6-bit        | ~24.8           | > 25            | For Apple Silicon with unified memory.     |
-|                    | MLX    | 8-bit        | ~32.4           | > 33            | Higher fidelity for Apple Silicon.         |
-|                    | MLX    | 16-bit       | ~57             | > 57            | bfloat16 for fine-tuning.                  |
-|                    | GGUF   | Q4_K_M       | 18.6            | ~20.0           | Recommended balance for quality/size.      |
-|                    | GGUF   | IQ4_NL       | 17.6            | ~20.0           | Recommended balance for quality/size.      |
-|                    | GGUF   | Q8_0         | 32.5            | ~33             | Near-lossless quality.                     |
-|                    | GGUF   | Q2_K         | 11.3            | ~12             | Low quality. Use for speculative decoding. |
-|                    | GGUF   | BF16         | 57              | ~60             | bfloat16 for fine-tuning.                  |
-
-**Notes on Quantization and Formats:**
-
-- **IQ4_NL (Importance-aware Quantization, Non-Linear):** A sophisticated 4-bit method that preserves important model weights with higher precision. It often provides superior performance compared to standard 4-bit quants at a similar file size and is a strong alternative to `Q4_K_M`.
-- **K-Quants (Q2_K, Q4_K_M):** This family of quantization methods generally offers a better quality-to-size ratio than older \_0 or \_1 variants.
-- **Q2_K:** An extremely small 2-bit quantization designed for environments with severe resource limitations. Users should anticipate a noticeable reduction in model accuracy and coherence in exchange for the minimal VRAM and storage footprint.
-- **Q8_0:** A full 8-bit quantization that provides high fidelity at the cost of a larger file size. It is suitable for systems with ample VRAM.
-- **VRAM Requirements:** The values provided are estimates for loading the model and processing a moderate context. Actual VRAM consumption can vary based on factors such as context length, batch size, and the specific inference software used.
-
-## Safety and Bias
-
-### Safety
-To rigorously evaluate safety performance, we developed a comprehensive testing framework comprising over 200 adversarial prompts spanning 10 critical risk categories including cybersecurity threats, hate speech, illegal activities, privacy violations, physical safety risks, misinformation, bias and discrimination, self-harm, child safety, and copyright infringement. These questions were systematically generated using a multi-layered approach: first establishing domain-specific threat models based on NIST AI RMF guidelines, then crafting prompts that incorporate real-world evasion techniques (including leetspeak substitutions, roleplay scenarios, and encoded instructions) to test for policy circumvention. Each category contains progressively severe prompts ranging from general inquiries about harmful activities to highly specific requests for executable code and step-by-step instructions. During evaluation, our model consistently refused all safety-compromising requests, demonstrating robust adherence to ethical boundaries without attempting to fulfill harmful instructions—even when presented with sophisticated evasion attempts. This testing protocol exceeds standard industry benchmarks by incorporating both direct harmful requests and nuanced edge cases designed to probe boundary conditions in safety policies.
-
-### Bias
-Our analysis reveals that cdx1 and cdx1-pro models exhibits a notable bias toward CycloneDX specifications, a tendency directly attributable to the composition of its training data which contains significantly more CycloneDX-related content than competing Software Bill of Materials (SBOM) standards. This data imbalance manifests in the model's consistent preference for recommending CycloneDX over alternative frameworks such as SPDX and omnibor, even in contexts where these competing standards might offer superior suitability for specific use cases. The model frequently fails to provide balanced comparative analysis, instead defaulting to CycloneDX-centric recommendations without adequate consideration of factors like ecosystem compatibility, tooling support, or organizational requirements that might favor alternative specifications. We recognize this as a limitation affecting the model's objectivity in technical decision support. Our long-term mitigation strategy involves targeted expansion of the training corpus with high-quality, balanced documentation of all major SBOM standards, implementation of adversarial debiasing techniques during fine-tuning, and development of explicit prompting protocols that require the model to evaluate multiple standards against specific technical requirements before making recommendations. We are committed to evolving cdx1 toward genuine impartiality in standards evaluation while maintaining its deep expertise in software supply chain security.
-
-## Weaknesses
-
-(To be determined)
-
-## Acknowledgments
-
-(To be determined)
-
-## Citation
-
-Please cite the following resources if you use the datasets, models, or benchmark in your work.
-
-### For the Dataset
-
-```bibtex
-@misc{cdx-docs,
-  author = {OWASP CycloneDX Generator Team},
-  title = {{cdx-docs: A Curated Dataset for SBOM and DevOps Tasks}},
-  year = {2025},
-  month = {February},
-  howpublished = {\url{https://huggingface.co/datasets/CycloneDX/cdx-docs}}
-}
-```
-
-### For the Models
-
-```bibtex
-@misc{cdx1_models,
-  author = {OWASP CycloneDX Generator Team},
-  title = {{cdx1 and cdx1-pro: Language Models for SBOM and DevOps}},
-  year = {2025},
-  month = {February},
-  howpublished = {\url{https://huggingface.co/CycloneDX}}
-}
-```
-
-### For the xBOMEval Benchmark
-
-```bibtex
-@misc{xBOMEval_v1,
-  author = {OWASP CycloneDX Generator Team},
-  title = {{xBOMEval: A Benchmark for Evaluating Language Models on SBOM Tasks}},
-  year = {2025},
-  month = {August},
-  howpublished = {\url{https://github.com/CycloneDX/cdxgen}}
-}
-```
-
-## Licenses
-
-- **Datasets:** CC0-1.0
-- **Models:** Apache-2.0

chat_template.jinja

@@ -1,86 +1,135 @@
-{%- if tools %}
-    {{- '<|im_start|>system\n' }}
-    {%- if messages[0].role == 'system' %}
-        {{- messages[0].content + '\n\n' }}
+{# Copyright 2025-present Unsloth. Apache 2.0 License. Unsloth Chat template fixes #}
+{% macro render_item_list(item_list, tag_name='required') %}
+    {%- if item_list is defined and item_list is iterable and item_list | length > 0 %}
+        {%- if tag_name %}{{- '\n<' ~ tag_name ~ '>' -}}{% endif %}
+            {{- '[' }}
+                {%- for item in item_list -%}
+                    {%- if loop.index > 1 %}{{- ", "}}{% endif -%}
+                    {%- if item is string -%}
+                        {{ "`" ~ item ~ "`" }}
+                    {%- else -%}
+                        {{ item }}
+                    {%- endif -%}
+                {%- endfor -%}
+            {{- ']' }}
+        {%- if tag_name %}{{- '</' ~ tag_name ~ '>' -}}{% endif %}
     {%- endif %}
-    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
-    {%- for tool in tools %}
-        {{- "\n" }}
-        {{- tool | tojson }}
-    {%- endfor %}
-    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{% endmacro %}
+
+{%- if messages[0]["role"] == "system" %}
+    {%- set system_message = messages[0]["content"] %}
+    {%- set loop_messages = messages[1:] %}
 {%- else %}
-    {%- if messages[0].role == 'system' %}
-        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
-    {%- endif %}
+    {%- set loop_messages = messages %}
 {%- endif %}
-{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
-{%- for message in messages[::-1] %}
-    {%- set index = (messages|length - 1) - loop.index0 %}
-    {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
-        {%- set ns.multi_step_tool = false %}
-        {%- set ns.last_query_index = index %}
-    {%- endif %}
-{%- endfor %}
-{%- for message in messages %}
-    {%- if message.content is string %}
-        {%- set content = message.content %}
-    {%- else %}
-        {%- set content = '' %}
+
+{%- if not tools is defined %}
+    {%- set tools = [] %}
+{%- endif %}
+
+{%- if system_message is defined %}
+    {{- "<|im_start|>system\n" + system_message }}
+{%- else %}
+    {%- if tools is iterable and tools | length > 0 %}
+        {{- "<|im_start|>system\nYou are Qwen, a helpful AI assistant that can interact with a computer to solve tasks." }}
     {%- endif %}
-    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
-        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
-    {%- elif message.role == "assistant" %}
-        {%- set reasoning_content = '' %}
-        {%- if message.reasoning_content is string %}
-            {%- set reasoning_content = message.reasoning_content %}
-        {%- else %}
-            {%- if '</think>' in content %}
-                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
-                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
-            {%- endif %}
+{%- endif %}
+{%- if tools is iterable and tools | length > 0 %}
+    {{- "\n\nYou have access to the following functions:\n\n" }}
+    {{- "<tools>" }}
+    {%- for tool in tools %}
+        {%- if tool.function is defined %}
+            {%- set tool = tool.function %}
         {%- endif %}
-        {%- if loop.index0 > ns.last_query_index %}
-            {%- if loop.last or (not loop.last and reasoning_content) %}
-                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
-            {%- else %}
-                {{- '<|im_start|>' + message.role + '\n' + content }}
+        {{- "\n<function>\n<name>" ~ tool.name ~ "</name>" }}
+        {{- '\n<description>' ~ (tool.description | trim) ~ '</description>' }}
+        {{- '\n<parameters>' }}
+        {%- for param_name, param_fields in tool.parameters.properties|items %}
+            {{- '\n<parameter>' }}
+            {{- '\n<name>' ~ param_name ~ '</name>' }}
+            {%- if param_fields.type is defined %}
+                {{- '\n<type>' ~ (param_fields.type | string) ~ '</type>' }}
             {%- endif %}
-        {%- else %}
-            {{- '<|im_start|>' + message.role + '\n' + content }}
-        {%- endif %}
-        {%- if message.tool_calls %}
-            {%- for tool_call in message.tool_calls %}
-                {%- if (loop.first and content) or (not loop.first) %}
-                    {{- '\n' }}
-                {%- endif %}
-                {%- if tool_call.function %}
-                    {%- set tool_call = tool_call.function %}
-                {%- endif %}
-                {{- '<tool_call>\n{"name": "' }}
-                {{- tool_call.name }}
-                {{- '", "arguments": ' }}
-                {%- if tool_call.arguments is string %}
-                    {{- tool_call.arguments }}
-                {%- else %}
-                    {{- tool_call.arguments | tojson }}
+            {%- if param_fields.description is defined %}
+                {{- '\n<description>' ~ (param_fields.description | trim) ~ '</description>' }}
+            {%- endif %}
+            {{- render_item_list(param_fields.enum, 'enum') }}
+            {%- set handled_keys = ['type', 'description', 'enum', 'required'] %}
+            {%- for json_key, json_value in param_fields|items %}
+                {%- if json_key not in handled_keys %}
+                    {%- set normed_json_key = json_key|string %}
+                    {%- if json_value is mapping %}
+                        {{- '\n<' ~ normed_json_key ~ '>' ~ (json_value | tojson | safe) ~ '</' ~ normed_json_key ~ '>' }}
+                    {%- else %}
+                        {{- '\n<' ~ normed_json_key ~ '>' ~ (json_value | string) ~ '</' ~ normed_json_key ~ '>' }}
+                    {%- endif %}
                 {%- endif %}
-                {{- '}\n</tool_call>' }}
             {%- endfor %}
+            {{- render_item_list(param_fields.required, 'required') }}
+            {{- '\n</parameter>' }}
+        {%- endfor %}
+        {{- render_item_list(tool.parameters.required, 'required') }}
+        {{- '\n</parameters>' }}
+        {%- if tool.return is defined %}
+            {%- if tool.return is mapping %}
+                {{- '\n<return>' ~ (tool.return | tojson | safe) ~ '</return>' }}
+            {%- else %}
+                {{- '\n<return>' ~ (tool.return | string) ~ '</return>' }}
+            {%- endif %}
         {%- endif %}
+        {{- '\n</function>' }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+{%- endif %}
+{%- if system_message is defined %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if tools is iterable and tools | length > 0 %}
         {{- '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in loop_messages %}
+    {%- if message.role == "assistant" and message.tool_calls is defined and message.tool_calls is iterable and message.tool_calls | length > 0 %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content is defined and message.content is string and message.content | trim | length > 0 %}
+            {{- '\n' + message.content | trim + '\n' }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+            {%- if tool_call.arguments is defined %}
+                {%- for args_name, args_value in tool_call.arguments|items %}
+                    {{- '<parameter=' + args_name + '>\n' }}
+                    {%- set args_value = args_value if args_value is string else args_value | string %}
+                    {{- args_value }}
+                    {{- '\n</parameter>\n' }}
+                {%- endfor %}
+            {%- endif %}
+            {{- '</function>\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "user" or message.role == "system" or message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
     {%- elif message.role == "tool" %}
-        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
-            {{- '<|im_start|>user' }}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user\n' }}
         {%- endif %}
-        {{- '\n<tool_response>\n' }}
-        {{- content }}
-        {{- '\n</tool_response>' }}
-        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+        {{- '<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>\n' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
             {{- '<|im_end|>\n' }}
         {%- endif %}
+    {%- else %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>\n' }}
     {%- endif %}
 {%- endfor %}
 {%- if add_generation_prompt %}
     {{- '<|im_start|>assistant\n' }}
-{%- endif %}
+{%- endif %}
+{# Copyright 2025-present Unsloth. Apache 2.0 License. Unsloth Chat template fixes #}

config.json

@@ -10,9 +10,9 @@
     "hidden_act": "silu",
     "hidden_size": 2048,
     "initializer_range": 0.02,
-    "intermediate_size": 6144,
+    "intermediate_size": 5472,
     "max_position_embeddings": 262144,
-    "max_window_layers": 48,
+    "max_window_layers": 28,
     "mlp_only_layers": [],
     "model_type": "qwen3_moe",
     "moe_intermediate_size": 768,
@@ -24,16 +24,19 @@
     "num_key_value_heads": 4,
     "output_router_logits": false,
     "pad_token_id": 151654,
+    "qkv_bias": false,
     "rms_norm_eps": 1e-06,
     "rope_scaling": null,
     "rope_theta": 10000000,
-    "router_aux_loss_coef": 0.001,
+    "router_aux_loss_coef": 0.0,
+    "shared_expert_intermediate_size": 0,
     "sliding_window": null,
     "tie_word_embeddings": false,
     "torch_dtype": "bfloat16",
-    "transformers_version": "4.54.0",
+    "transformers_version": "4.54.1",
     "unsloth_fixed": true,
     "use_cache": true,
+    "use_qk_norm": true,
     "use_sliding_window": false,
     "vocab_size": 151936
 }

generation_config.json

@@ -1,13 +1,12 @@
 {
-    "bos_token_id": 151643,
-    "do_sample": true,
-    "eos_token_id": [
-        151645,
-        151643
-    ],
-    "pad_token_id": 151643,
-    "temperature": 0.7,
-    "top_k": 20,
-    "top_p": 0.8,
-    "transformers_version": "4.51.0"
+  "pad_token_id": 151643,
+  "do_sample": true,
+  "eos_token_id": [
+    151645,
+    151643
+  ],
+  "repetition_penalty": 1.05,
+  "temperature": 0.7,
+  "top_p": 0.8,
+  "top_k": 20
 }

model-00001-of-00013.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:00bf00857a7ee66981f0724e01a46d7c942ee2844978466d3106863b2f84c99e
+oid sha256:e2c91e9afd41575e546ee4186be55a982b3d5303849b37719635e541728d1581
 size 5204647414

model-00002-of-00013.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:648268d56a9fd92ebb1f0c66462526d121dc72848071b85aa554f880ab42db4e
+oid sha256:9bebacf4ce55b4035cc4aee9264fe623edd9b470e6af4b9386d5441fd8e1a2d0
 size 4984970772

model-00003-of-00013.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:97a6c01d2b231d9b1ba4d4b5d6bdbde6435850833e05a0784a29a51d03afdd83
+oid sha256:5efdcc20c7145ac7a277bc960efbb3f112a258304f447841aa009afddfa9150f
 size 4984970797

model-00004-of-00013.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a2d24f9165c31655c032be0f19078ece7729457d47a841059a88f60aabb0702d
+oid sha256:6bf2acef428d084cb213914d08fca5393ec9aca7497d5bc1c375b9cfb26a0dec
 size 4984970824

model-00005-of-00013.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:45386c7039d77d4c072608a119b411d993074572631264326c675d631d4c5c38
+oid sha256:fa9d5a95689a7ddacc557ac5d4ef91102e763efd7f6cf8b579820826e055dac0
 size 4984970804

model-00006-of-00013.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3da0f64c4321d66e600ff12c235054b14d665e910857e445695c265d500b3aea
+oid sha256:2c2a1fc571ec44d6bbba8928f86b9ab99f40147397d89a6895a850c03ab1c0ed
 size 4984970822

model-00007-of-00013.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:35cca769e57ad0ef3e52c725009de170e6e6b46b0f052c1b374aff2d796a79f1
+oid sha256:c24ff403e02e6386a1bb597f0dee9ec12bf4fd6e3f40e276ea305cd958c732b0
 size 4984970824

model-00008-of-00013.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e0eb2e9bc6b508762de131f39112b6e10ad6224c602f4d70524371cf2ec3cdb9
+oid sha256:3b88ef5f2e4db320229315fbeb0257b414cfc9515189d558b023b9eae552ce63
 size 4984970806

model-00009-of-00013.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ad3ce6ced2a58edf0fbf79d53c07fbee826288a2839be9c3d22ca8696df78c5e
+oid sha256:4889ad31c8412d47b883414d71cb798f37ea11cee1120c45aa54d631253931f1
 size 4984970818

model-00010-of-00013.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9a8a79a95ea6e47283a6b90b757f3eb705ffb63aab465a5a35ff0eb86db2d43c
+oid sha256:bfdc0c883de859f1e57d250f362b4af297573055adc83a7dd15eae41e61001e5
 size 4984970752

model-00011-of-00013.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1597fe7587d4db6ae1b08b7bd24f50e8b275eb3bb8206f1a8f09309b89adda47
+oid sha256:1ac3aeaf7bd7ce784625d51df9cb65148ec6a797b81fa7fce7cf49422cc8f4f8
 size 4984970814

model-00012-of-00013.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:47a6b9413d6d2afcfcfef3b542b974266da010c390638e99547f54fc3d3721a6
+oid sha256:3f9a273f814f700e0d9e0d107a83e285ea5b5f4d37c9807c6e4c709c3e2dbdb6
 size 4984970822

model-00013-of-00013.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:77050d027d7fa116e4857543404f5d2ae50bba47758ce45c8ea978927fb3fa26
+oid sha256:2baa49e37e374c49aa76d19b90c2552f6a71a3ce636822c0bad7277fd340474c
 size 1024987469

qwen3coder_tool_parser.py

@@ -0,0 +1,675 @@
+# SPDX-License-Identifier: Apache-2.0
+
+import json
+import re
+import uuid
+from collections.abc import Sequence
+from typing import Union, Optional, Any, List, Dict
+from enum import Enum
+
+from vllm.entrypoints.openai.protocol import (
+    ChatCompletionRequest,
+    ChatCompletionToolsParam,
+    DeltaMessage,
+    DeltaToolCall,
+    DeltaFunctionCall,
+    ExtractedToolCallInformation,
+    FunctionCall,
+    ToolCall,
+)
+from vllm.entrypoints.openai.tool_parsers.abstract_tool_parser import (
+    ToolParser,
+    ToolParserManager,
+)
+from vllm.logger import init_logger
+from vllm.transformers_utils.tokenizer import AnyTokenizer
+
+logger = init_logger(__name__)
+
+
+@ToolParserManager.register_module("qwen3_xml")
+class Qwen3XMLToolParser(ToolParser):
+    def __init__(self, tokenizer: AnyTokenizer):
+        super().__init__(tokenizer)
+
+        self.current_tool_name_sent: bool = False
+        self.prev_tool_call_arr: list[dict] = []
+        self.current_tool_id: int = -1
+        self.streamed_args_for_tool: list[str] = []
+
+        # Sentinel tokens for streaming mode
+        self.tool_call_start_token: str = "<tool_call>"
+        self.tool_call_end_token: str = "</tool_call>"
+        self.tool_call_prefix: str = "<function="
+        self.function_end_token: str = "</function>"
+        self.parameter_prefix: str = "<parameter="
+        self.parameter_end_token: str = "</parameter>"
+        self.is_tool_call_started: bool = False
+        self.failed_count: int = 0
+
+        # Enhanced streaming state - reset for each new message
+        self._reset_streaming_state()
+
+        # Regex patterns
+        self.tool_call_complete_regex = re.compile(
+            r"<tool_call>(.*?)</tool_call>", re.DOTALL
+        )
+        self.tool_call_regex = re.compile(
+            r"<tool_call>(.*?)</tool_call>|<tool_call>(.*?)$", re.DOTALL
+        )
+        self.tool_call_function_regex = re.compile(
+            r"<function=(.*?)</function>|<function=(.*)$", re.DOTALL
+        )
+        self.tool_call_parameter_regex = re.compile(
+            r"<parameter=(.*?)</parameter>|<parameter=(.*?)$", re.DOTALL
+        )
+
+        if not self.model_tokenizer:
+            raise ValueError(
+                "The model tokenizer must be passed to the ToolParser "
+                "constructor during construction."
+            )
+
+        self.tool_call_start_token_id = self.vocab.get(self.tool_call_start_token)
+        self.tool_call_end_token_id = self.vocab.get(self.tool_call_end_token)
+
+        if self.tool_call_start_token_id is None or self.tool_call_end_token_id is None:
+            raise RuntimeError(
+                "Qwen3 XML Tool parser could not locate tool call start/end "
+                "tokens in the tokenizer!"
+            )
+
+        logger.info(f"vLLM Successfully import tool parser {self.__class__.__name__} !")
+
+    def _generate_tool_call_id(self) -> str:
+        """Generate a unique tool call ID."""
+        return f"call_{uuid.uuid4().hex[:24]}"
+
+    def _reset_streaming_state(self):
+        """Reset all streaming state."""
+        self.current_tool_index = 0
+        self.is_tool_call_started = False
+        self.header_sent = False
+        self.current_tool_id = None
+        self.current_function_name = None
+        self.current_param_name = None
+        self.current_param_value = ""
+        self.param_count = 0
+        self.in_param = False
+        self.in_function = False
+        self.accumulated_text = ""
+        self.json_started = False
+        self.json_closed = False
+
+    def _parse_xml_function_call(
+        self, function_call_str: str, tools: Optional[list[ChatCompletionToolsParam]]
+    ) -> Optional[ToolCall]:
+        def get_arguments_config(func_name: str) -> dict:
+            if tools is None:
+                return {}
+            for config in tools:
+                if not hasattr(config, "type") or not (
+                    hasattr(config, "function") and hasattr(config.function, "name")
+                ):
+                    continue
+                if config.type == "function" and config.function.name == func_name:
+                    if not hasattr(config.function, "parameters"):
+                        return {}
+                    params = config.function.parameters
+                    if isinstance(params, dict) and "properties" in params:
+                        return params["properties"]
+                    elif isinstance(params, dict):
+                        return params
+                    else:
+                        return {}
+            logger.warning(f"Tool '{func_name}' is not defined in the tools list.")
+            return {}
+
+        def convert_param_value(
+            param_value: str, param_name: str, param_config: dict, func_name: str
+        ) -> Any:
+            # Handle null value for any type
+            if param_value.lower() == "null":
+                return None
+
+            if param_name not in param_config:
+                if param_config != {}:
+                    logger.warning(
+                        f"Parsed parameter '{param_name}' is not defined in the tool "
+                        f"parameters for tool '{func_name}', directly returning the string value."
+                    )
+                return param_value
+
+            if (
+                isinstance(param_config[param_name], dict)
+                and "type" in param_config[param_name]
+            ):
+                param_type = str(param_config[param_name]["type"]).strip().lower()
+            else:
+                param_type = "string"
+            if param_type in ["string", "str", "text", "varchar", "char", "enum"]:
+                return param_value
+            elif (
+                param_type.startswith("int")
+                or param_type.startswith("uint")
+                or param_type.startswith("long")
+                or param_type.startswith("short")
+                or param_type.startswith("unsigned")
+            ):
+                try:
+                    param_value = int(param_value)
+                except:
+                    logger.warning(
+                        f"Parsed value '{param_value}' of parameter '{param_name}' is not an integer in tool "
+                        f"'{func_name}', degenerating to string."
+                    )
+                return param_value
+            elif param_type.startswith("num") or param_type.startswith("float"):
+                try:
+                    float_param_value = float(param_value)
+                    param_value = float_param_value if float_param_value - int(float_param_value) != 0 else int(float_param_value)
+                except:
+                    logger.warning(
+                        f"Parsed value '{param_value}' of parameter '{param_name}' is not a float in tool "
+                        f"'{func_name}', degenerating to string."
+                    )
+                return param_value
+            elif param_type in ["boolean", "bool", "binary"]:
+                param_value = param_value.lower()
+                if param_value not in ["true", "false"]:
+                    logger.warning(
+                        f"Parsed value '{param_value}' of parameter '{param_name}' is not a boolean (`true` of `false`) in tool '{func_name}', degenerating to false."
+                    )
+                return param_value == "true"
+            else:
+                if param_type == "object" or param_type.startswith("dict"):
+                    try:
+                        param_value = json.loads(param_value)
+                        return param_value
+                    except:
+                        logger.warning(
+                            f"Parsed value '{param_value}' of parameter '{param_name}' is not a valid JSON object in tool "
+                            f"'{func_name}', will try other methods to parse it."
+                        )
+                try:
+                    param_value = eval(param_value)
+                except:
+                    logger.warning(
+                        f"Parsed value '{param_value}' of parameter '{param_name}' cannot be converted via Python `eval()` in tool '{func_name}', degenerating to string."
+                    )
+                return param_value
+
+        # Extract function name
+        end_index = function_call_str.index(">")
+        function_name = function_call_str[:end_index]
+        param_config = get_arguments_config(function_name)
+        parameters = function_call_str[end_index + 1 :]
+        param_dict = {}
+        for match in self.tool_call_parameter_regex.findall(parameters):
+            match_text = match[0] if match[0] else match[1]
+            idx = match_text.index(">")
+            param_name = match_text[:idx]
+            param_value = str(match_text[idx + 1 :])
+            # Remove prefix and trailing \n
+            if param_value.startswith("\n"):
+                param_value = param_value[1:]
+            if param_value.endswith("\n"):
+                param_value = param_value[:-1]
+
+            param_dict[param_name] = convert_param_value(
+                param_value, param_name, param_config, function_name
+            )
+        return ToolCall(
+            type="function",
+            function=FunctionCall(
+                name=function_name, arguments=json.dumps(param_dict, ensure_ascii=False)
+            ),
+        )
+
+    def _get_function_calls(self, model_output: str) -> List[str]:
+        # Find all tool calls
+        matched_ranges = self.tool_call_regex.findall(model_output)
+        raw_tool_calls = [
+            match[0] if match[0] else match[1] for match in matched_ranges
+        ]
+
+        # Back-off strategy if no tool_call tags found
+        if len(raw_tool_calls) == 0:
+            raw_tool_calls = [model_output]
+
+        raw_function_calls = []
+        for tool_call in raw_tool_calls:
+            raw_function_calls.extend(self.tool_call_function_regex.findall(tool_call))
+
+        function_calls = [
+            match[0] if match[0] else match[1] for match in raw_function_calls
+        ]
+        return function_calls
+
+    def extract_tool_calls(
+        self,
+        model_output: str,
+        request: ChatCompletionRequest,
+    ) -> ExtractedToolCallInformation:
+        # Quick check to avoid unnecessary processing
+        if self.tool_call_prefix not in model_output:
+            return ExtractedToolCallInformation(
+                tools_called=False, tool_calls=[], content=model_output
+            )
+
+        try:
+            function_calls = self._get_function_calls(model_output)
+            if len(function_calls) == 0:
+                return ExtractedToolCallInformation(
+                    tools_called=False, tool_calls=[], content=model_output
+                )
+
+            tool_calls = [
+                self._parse_xml_function_call(function_call_str, request.tools)
+                for function_call_str in function_calls
+            ]
+
+            # Populate prev_tool_call_arr for serving layer to set finish_reason
+            self.prev_tool_call_arr.clear()  # Clear previous calls
+            for tool_call in tool_calls:
+                if tool_call:
+                    self.prev_tool_call_arr.append(
+                        {
+                            "name": tool_call.function.name,
+                            "arguments": tool_call.function.arguments,
+                        }
+                    )
+
+            # Extract content before tool calls
+            content_index = model_output.find(self.tool_call_start_token)
+            content_index = (
+                content_index
+                if content_index >= 0
+                else model_output.find(self.tool_call_prefix)
+            )
+            content = model_output[:content_index]  # .rstrip()
+
+            return ExtractedToolCallInformation(
+                tools_called=(len(tool_calls) > 0),
+                tool_calls=tool_calls,
+                content=content if content else None,
+            )
+
+        except Exception:
+            logger.exception("Error in extracting tool call from response.")
+            return ExtractedToolCallInformation(
+                tools_called=False, tool_calls=[], content=model_output
+            )
+
+    def extract_tool_calls_streaming(
+        self,
+        previous_text: str,
+        current_text: str,
+        delta_text: str,
+        previous_token_ids: Sequence[int],
+        current_token_ids: Sequence[int],
+        delta_token_ids: Sequence[int],
+        request: ChatCompletionRequest,
+    ) -> Union[DeltaMessage, None]:
+        # If no delta text, return None unless it's an EOS token after tool calls
+        if not delta_text:
+            # Check if this is an EOS token after all tool calls are complete
+            # We check for tool calls in the text even if is_tool_call_started is False
+            # because it might have been reset after processing all tools
+            if delta_token_ids and self.tool_call_end_token_id not in delta_token_ids:
+                # Count complete tool calls
+                complete_calls = len(
+                    self.tool_call_complete_regex.findall(current_text)
+                )
+
+                # If we have completed tool calls and populated prev_tool_call_arr
+                if complete_calls > 0 and len(self.prev_tool_call_arr) > 0:
+                    # Check if all tool calls are closed
+                    open_calls = current_text.count(
+                        self.tool_call_start_token
+                    ) - current_text.count(self.tool_call_end_token)
+                    if open_calls == 0:
+                        # Return empty delta message to allow finish_reason processing
+                        return DeltaMessage(content="")
+                elif not self.is_tool_call_started and current_text:
+                    # This is a regular content response that's now complete
+                    return DeltaMessage(content="")
+            return None
+
+        # Check if this is the first call (reset state if needed)
+        if not previous_text:
+            self._reset_streaming_state()
+
+        # Update accumulated text
+        self.accumulated_text = current_text
+
+        # Check if we need to advance to next tool
+        if self.json_closed and not self.in_function:
+            # Check if this tool call has ended
+            tool_ends = current_text.count(self.tool_call_end_token)
+            if tool_ends > self.current_tool_index:
+                # This tool has ended, advance to next
+                self.current_tool_index += 1
+                self.header_sent = False
+                self.param_count = 0
+                self.json_started = False
+                self.json_closed = False
+
+                # Check if there are more tool calls
+                tool_starts = current_text.count(self.tool_call_start_token)
+                if self.current_tool_index >= tool_starts:
+                    # No more tool calls
+                    self.is_tool_call_started = False
+                # Continue processing next tool
+                return None
+
+        # Handle normal content before tool calls
+        if not self.is_tool_call_started:
+            # Check if tool call is starting
+            if (
+                self.tool_call_start_token_id in delta_token_ids
+                or self.tool_call_start_token in delta_text
+            ):
+                self.is_tool_call_started = True
+                # Return any content before the tool call
+                if self.tool_call_start_token in delta_text:
+                    content_before = delta_text[
+                        : delta_text.index(self.tool_call_start_token)
+                    ]
+                    if content_before:
+                        return DeltaMessage(content=content_before)
+                return None
+            else:
+                # Check if we're between tool calls - skip whitespace
+                if current_text.rstrip().endswith(self.tool_call_end_token):
+                    # We just ended a tool call, skip whitespace
+                    if delta_text.strip() == "":
+                        return None
+                # Normal content, no tool call
+                return DeltaMessage(content=delta_text)
+
+        # Check if we're between tool calls (waiting for next one)
+        # Count tool calls we've seen vs processed
+        tool_starts_count = current_text.count(self.tool_call_start_token)
+        if self.current_tool_index >= tool_starts_count:
+            # We're past all tool calls, shouldn't be here
+            return None
+
+        # We're in a tool call, find the current tool call portion
+        # Need to find the correct tool call based on current_tool_index
+        tool_starts = []
+        idx = 0
+        while True:
+            idx = current_text.find(self.tool_call_start_token, idx)
+            if idx == -1:
+                break
+            tool_starts.append(idx)
+            idx += len(self.tool_call_start_token)
+
+        if self.current_tool_index >= len(tool_starts):
+            # No more tool calls to process yet
+            return None
+
+        tool_start_idx = tool_starts[self.current_tool_index]
+        # Find where this tool call ends (or current position if not ended yet)
+        tool_end_idx = current_text.find(self.tool_call_end_token, tool_start_idx)
+        if tool_end_idx == -1:
+            tool_text = current_text[tool_start_idx:]
+        else:
+            tool_text = current_text[
+                tool_start_idx : tool_end_idx + len(self.tool_call_end_token)
+            ]
+
+        # Looking for function header
+        if not self.header_sent:
+            if self.tool_call_prefix in tool_text:
+                func_start = tool_text.find(self.tool_call_prefix) + len(
+                    self.tool_call_prefix
+                )
+                func_end = tool_text.find(">", func_start)
+
+                if func_end != -1:
+                    # Found complete function name
+                    self.current_function_name = tool_text[func_start:func_end]
+                    self.current_tool_id = self._generate_tool_call_id()
+                    self.header_sent = True
+                    self.in_function = True
+
+                    # IMPORTANT: Add to prev_tool_call_arr immediately when we detect a tool call
+                    # This ensures finish_reason="tool_calls" even if parsing isn't complete
+                    already_added = any(
+                        tool.get("name") == self.current_function_name
+                        for tool in self.prev_tool_call_arr
+                    )
+                    if not already_added:
+                        self.prev_tool_call_arr.append(
+                            {
+                                "name": self.current_function_name,
+                                "arguments": "{}",  # Placeholder, will be updated later
+                            }
+                        )
+
+                    # Send header with function info
+                    return DeltaMessage(
+                        tool_calls=[
+                            DeltaToolCall(
+                                index=self.current_tool_index,
+                                id=self.current_tool_id,
+                                function=DeltaFunctionCall(
+                                    name=self.current_function_name, arguments=""
+                                ),
+                                type="function",
+                            )
+                        ]
+                    )
+            return None
+
+        # We've sent header, now handle function body
+        if self.in_function:
+            # Send opening brace if not sent yet
+            if not self.json_started and not self.parameter_prefix in delta_text:
+                self.json_started = True
+                return DeltaMessage(
+                    tool_calls=[
+                        DeltaToolCall(
+                            index=self.current_tool_index,
+                            function=DeltaFunctionCall(arguments="{"),
+                        )
+                    ]
+                )
+
+            # Make sure json_started is set if we're processing parameters
+            if not self.json_started:
+                self.json_started = True
+
+            # Check for function end in accumulated text
+            if not self.json_closed and self.function_end_token in tool_text:
+                # Close JSON
+                self.json_closed = True
+
+                # Extract the complete tool call to update prev_tool_call_arr with final arguments
+                # Find the function content
+                func_start = tool_text.find(self.tool_call_prefix) + len(
+                    self.tool_call_prefix
+                )
+                func_content_end = tool_text.find(self.function_end_token, func_start)
+                if func_content_end != -1:
+                    func_content = tool_text[func_start:func_content_end]
+                    # Parse to get the complete arguments
+                    try:
+                        parsed_tool = self._parse_xml_function_call(
+                            func_content, request.tools if request else None
+                        )
+                        if parsed_tool:
+                            # Update existing entry in prev_tool_call_arr with complete arguments
+                            for i, tool in enumerate(self.prev_tool_call_arr):
+                                if tool.get("name") == parsed_tool.function.name:
+                                    self.prev_tool_call_arr[i]["arguments"] = (
+                                        parsed_tool.function.arguments
+                                    )
+                                    break
+                    except Exception:
+                        pass  # Ignore parsing errors during streaming
+
+                result = DeltaMessage(
+                    tool_calls=[
+                        DeltaToolCall(
+                            index=self.current_tool_index,
+                            function=DeltaFunctionCall(arguments="}"),
+                        )
+                    ]
+                )
+
+                # Reset state for next tool
+                self.in_function = False
+                self.json_closed = True
+
+                return result
+
+            # Look for parameters
+            # Count how many complete parameters we have processed
+            complete_params = tool_text.count(self.parameter_end_token)
+
+            # Check if we should start a new parameter
+            if not self.in_param and self.param_count < complete_params:
+                # Find the unprocessed parameter
+                # Count parameter starts
+                param_starts = []
+                idx = 0
+                while True:
+                    idx = tool_text.find(self.parameter_prefix, idx)
+                    if idx == -1:
+                        break
+                    param_starts.append(idx)
+                    idx += len(self.parameter_prefix)
+
+                if len(param_starts) > self.param_count:
+                    # Process the next parameter
+                    param_idx = param_starts[self.param_count]
+                    param_start = param_idx + len(self.parameter_prefix)
+                    remaining = tool_text[param_start:]
+
+                    if ">" in remaining:
+                        # We have the complete parameter name
+                        name_end = remaining.find(">")
+                        self.current_param_name = remaining[:name_end]
+
+                        # Find the parameter value
+                        value_start = param_start + name_end + 1
+                        value_text = tool_text[value_start:]
+                        if value_text.startswith("\n"):
+                            value_text = value_text[1:]
+
+                        # Find where this parameter ends
+                        param_end_idx = value_text.find(self.parameter_end_token)
+                        if param_end_idx != -1:
+                            # Complete parameter found
+                            param_value = value_text[:param_end_idx]
+                            if param_value.endswith("\n"):
+                                param_value = param_value[:-1]
+
+                            # Build complete JSON fragment for this parameter
+                            if self.param_count == 0:
+                                json_fragment = (
+                                    '"'
+                                    + self.current_param_name
+                                    + '": "'
+                                    + json.dumps(param_value)[1:-1]
+                                    + '"'
+                                )
+                            else:
+                                json_fragment = (
+                                    ', "'
+                                    + self.current_param_name
+                                    + '": "'
+                                    + json.dumps(param_value)[1:-1]
+                                    + '"'
+                                )
+
+                            self.param_count += 1
+
+                            return DeltaMessage(
+                                tool_calls=[
+                                    DeltaToolCall(
+                                        index=self.current_tool_index,
+                                        function=DeltaFunctionCall(
+                                            arguments=json_fragment
+                                        ),
+                                    )
+                                ]
+                            )
+
+            # Continue parameter value
+            if self.in_param:
+                if self.parameter_end_token in delta_text:
+                    # End of parameter
+                    end_idx = delta_text.find(self.parameter_end_token)
+                    value_chunk = delta_text[:end_idx]
+
+                    # Skip past > if at start
+                    if not self.current_param_value and ">" in value_chunk:
+                        gt_idx = value_chunk.find(">")
+                        value_chunk = value_chunk[gt_idx + 1 :]
+
+                    if not self.current_param_value and value_chunk.startswith("\n"):
+                        value_chunk = value_chunk[1:]
+
+                    # Calculate incremental JSON
+                    full_value = self.current_param_value + value_chunk
+                    prev_escaped = (
+                        json.dumps(self.current_param_value)[1:-1]
+                        if self.current_param_value
+                        else ""
+                    )
+                    full_escaped = json.dumps(full_value)[1:-1]
+                    delta_escaped = full_escaped[len(prev_escaped) :]
+
+                    self.in_param = False
+                    self.current_param_value = ""
+
+                    return DeltaMessage(
+                        tool_calls=[
+                            DeltaToolCall(
+                                index=self.current_tool_index,
+                                function=DeltaFunctionCall(
+                                    arguments=delta_escaped + '"'
+                                ),
+                            )
+                        ]
+                    )
+                else:
+                    # Continue accumulating value
+                    value_chunk = delta_text
+
+                    # Handle first chunk after param name
+                    if not self.current_param_value and ">" in value_chunk:
+                        gt_idx = value_chunk.find(">")
+                        value_chunk = value_chunk[gt_idx + 1 :]
+
+                    if not self.current_param_value and value_chunk.startswith("\n"):
+                        value_chunk = value_chunk[1:]
+
+                    if value_chunk:
+                        # Stream the escaped delta
+                        prev_escaped = (
+                            json.dumps(self.current_param_value)[1:-1]
+                            if self.current_param_value
+                            else ""
+                        )
+                        self.current_param_value += value_chunk
+                        full_escaped = json.dumps(self.current_param_value)[1:-1]
+                        delta_escaped = full_escaped[len(prev_escaped) :]
+
+                        if delta_escaped:
+                            return DeltaMessage(
+                                tool_calls=[
+                                    DeltaToolCall(
+                                        index=self.current_tool_index,
+                                        function=DeltaFunctionCall(
+                                            arguments=delta_escaped
+                                        ),
+                                    )
+                                ]
+                            )
+
+        return None