Skip to main content
Temperature is the primary parameter that controls how creative or deterministic your agent’s responses are. Understanding it helps you tune your agent for the right balance of accuracy and variety.

​
What Is Temperature?

Temperature is a number between 0 and 2 that controls the randomness of the AI model’s output:
  • Low temperature (0 - 0.3) — The model picks the most likely next word at each step. Responses are more focused, consistent, and predictable.
  • Medium temperature (0.4 - 0.7) — A balanced mix of predictability and variety. Good for most conversational use cases.
  • High temperature (0.8 - 2.0) — The model is more willing to pick less likely words. Responses are more creative, varied, and sometimes surprising.
Think of temperature as a dial between “factual assistant” (low) and “creative writer” (high). Most agents work best somewhere in the middle.

​
How Temperature Interacts with Knowledge Bases

When your agent has linked knowledge bases, temperature affects how the model uses retrieved content:
  • Low temperature — The model sticks closely to the retrieved source material. Responses are more faithful to your documents but may feel rigid.
  • Higher temperature — The model synthesizes and paraphrases more freely. Responses feel more natural but may drift further from source text.
For knowledge-grounded agents where accuracy matters, keep temperature at 0.3 or below. The RAG system provides the context; low temperature ensures the model uses it faithfully.

​
Setting Temperature

You can set the temperature when creating or editing an agent:
  1. Open your agent’s settings.
  2. Find the Temperature slider in the model configuration section.
  3. Drag to your desired value or type a specific number.
  4. Save your changes.
The change applies to all new messages in all conversations with this agent.

​
Other Generation Parameters

Beyond temperature, some models support additional parameters that affect output:

​
Max Tokens

Controls the maximum length of the AI’s response. Setting this prevents overly long answers but may cut off responses mid-thought if set too low.
  • Short responses — 256-512 tokens (quick answers, summaries)
  • Medium responses — 1024-2048 tokens (detailed explanations)
  • Long responses — 4096+ tokens (comprehensive analysis, long-form content)

​
Streaming

When streaming is enabled on your agent, responses are delivered word-by-word in real time rather than waiting for the full response to generate. This improves perceived speed and user experience. Streaming is controlled at two levels:
  • Agent level — streaming_enabled toggle in agent settings (default: on)
  • Request level — The client declares streaming capability per request
Effective streaming requires both the agent setting and the client request to have streaming enabled.
Web chat interfaces typically use streaming for a responsive feel. Channels like Telegram or WhatsApp receive the full response at once since they do not support real-time streaming.

​
Tips for Tuning

Set temperature to 0.5 and test a variety of questions. If responses feel too generic or repetitive, increase it. If they feel erratic or off-topic, decrease it.
A detailed, structured system prompt pairs well with lower temperature — the prompt constrains behavior while low temperature ensures compliance. A loose, personality-driven prompt can handle higher temperature.
Do not just test with “Hello.” Use the kinds of questions your actual users will ask, including edge cases and topics outside your knowledge base. This shows how temperature affects fallback behavior.
Different models respond to temperature differently. GPT-4 tends to be well-calibrated across the range. Smaller models may become incoherent at higher temperatures. Test with your specific model.