Skip to main content

​
Context Engineering

Context is king. This is the most key part in getting your agent to behave as expected.
What is context?It is the summary of the:
  • System prompt
  • User messages
  • Agent messages
  • Tool calls
  • Tool call results
Prompt engineering vs Context engineering
The more context there is, the more likely the model is to “forget” what it was told to do.

​
Long system prompts

Long system prompts are the worstYou need to keep your system prompt as short as possible. The death of model performance is by a thousand cuts. Each instruction you give the model degrades its performance. It’s not software or code, you need to give it enough instructions to be able to figure out what to do on its own, not program exactly what to do.

​
Lost-in-the-middle

Almost all models recall best what is in the beginning and the end of the prompt, explicit, specific instructions in the middle are often forgotten.
Many times I see Agent Engineers stuffing lines and lines of exactly-to-follow instructions in the middle of the prompt. The model will not act on those.

​
Conflicting instructions

Putting conflicting instructions in the prompt is a sure way to get the model to fail. Read your prompt, you most likely wrote more than 2 lines referring to the same behavior. For example, out-of-scope instructions tend to be the most common in this case. The prompt will have like 3 definitions of what is out-of-scope and all of the definitions are conflicting - of course the agent is not going to know what to do.

​
Agent messages count as context

Especially the long ones. If your agent responds in long responses for no reason, this fills up the context window making the model worse at its task.

​
Degradation of performance

The more user messages, the more agent messages, the more tool calls, the more context the model has to work with, the worse it performs. Especially because the system window is slightly “forgotten” with each new turn.

​
Tool calls

Tool calls are probably the most repeat offender here. So often I see tools returning so much data and the tool note being like “Use the first 2 lines, they are the most relevant”. Why stuff all this data into the context window?
Best PracticeYour tool calls should return the minimum possible amount of information for the model to act on. Return your results in JSON, as models are trained to understand JSON better than plain text.
Don’t return raw error codes, data, or anything else to the model. Do return human readable text. The model was trained on text.

Bad Example

Returns raw status code and redundant explanation:
check_something() -> {
  "result": "101",
  "tool_note": "The result 101 means success"
}

Good Example

Returns clear, semantic result:
check_something() -> {
  "result": "success",
}