reasoning_parser for TRT-LLM

#5
by thecollabagepatch - opened

I notice a discrepancy in the model card and the example notebook you guys are linking to around the reasoning_parser in the TensorRT-LLM command.

TRTLLM_ENABLE_PDL=1 trtllm-serve nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8
--host 0.0.0.0
--port 8000
--backend _autodeploy
--reasoning_parser nano-v3
--tool_parser qwen3_coder
--extra_llm_api_options nano_v3.yaml

When I try this example:

Error: Invalid value for '--reasoning_parser': 'nano-v3' is not one of 'deepseek-r1', 'qwen3'.

I assume the correct reasoning_parser is 'deepseek-r1' as that is the one specified in the cookbook, and I was able to get a test output that, for the most part, looked about right using it.

Just figured I'd point that out. Thanks for the new toy guys!

Sign up or log in to comment