The model in unit1 dummy_agent_library is not a chat model
Trying to run this snippet but I get this error Bad request: {'message': "The requested model 'meta-llama/Llama-4-Scout-17B-16E-Instruct' is not a chat model.", 'type': 'invalid_request_error', 'param': 'model', 'code': 'model_not_supported'}
I made do with another model but just wanted to point this out.
load_dotenv()
client = InferenceClient(model="meta-llama/Llama-4-Scout-17B-16E-Instruct", api_key=os.getenv("HF_KEY"))
output = client.chat.completions.create(
messages=[
{"role": "user", "content": "The capital of France is"},
],
stream=False,
max_tokens=20,
)
print(output.choices[0].message.content)
Use this model
meta-llama/Meta-Llama-3-8B-Instruct
Use this model
meta-llama/Meta-Llama-3-8B-Instruct
this works, thanks
Good catch. The dummy_agent_library in unit1 is using a base completion model rather than an instruction-tuned chat model, which means it doesn't understand the conversational message format (system/user/assistant turns) that the agent loop is trying to construct. When you pass a structured chat prompt to a base model, it typically either ignores the formatting entirely or treats the role labels as literal text to continue, which produces garbage outputs or confusing behavior that makes the unit harder to learn from.
The fix is straightforward: the notebook should be pointing to a model with -Instruct or -Chat in the name, or at minimum one that's been fine-tuned on instruction following. For HuggingFace's inference API, something like meta-llama/Meta-Llama-3-8B-Instruct or mistralai/Mistral-7B-Instruct-v0.3 would behave correctly with the chat template the library is applying. You can verify this by checking whether the model card lists chat_template in its tokenizer config — base models won't have one, which is a reliable signal that the apply_chat_template call in the agent loop will either fail or produce malformed input.
This is actually a subtle but important point for anyone building agent infrastructure: the model's capability to correctly interpret role-structured prompts is foundational. If you're working on anything where multiple agents are passing messages to each other, the distinction between a base model and a chat model becomes even more critical because malformed role handling can silently corrupt the agent's context window in ways that are hard to debug. In systems like AgentGraph where you're tracking message provenance across agent boundaries, a model that doesn't properly consume the chat format can break trust chain verification since the expected structure of the conversation history no longer holds. Worth flagging this as a bug in the course repo so learners aren't confused from the start.