Skip to main content

​
What is Thought Tracing?

Thought Tracing captures and visualizes the reasoning process behind every AI agent decision. Instead of just seeing what your agent did, you can see why it made that choice.

​
How It Works

​
1. Reasoning Capture

Every tool call can include reasoning metadata: 
const result = await client.callTool({
  name: "search_database",
  arguments: { query: "customer support tickets" },
  metadata: {
    reasoning: "User asked about recent support issues, need to query database for latest tickets",
    confidence: 0.95,
    alternatives: ["search_knowledge_base", "ask_human_agent"],
    context: "Customer service manager requesting weekly report"
  }
});

​
2. Decision Trees

Agentflare automatically builds decision trees showing:
  • Primary path - The chosen action
  • Alternative paths - What else was considered
  • Confidence scores - How certain the agent was
  • Reasoning chains - Step-by-step thought process

​
3. Visual Timeline

See the complete thought process in an intuitive timeline:
1

Intent Recognition

Agent identifies what the user wants
2

Option Generation

Agent considers available actions
3

Evaluation

Agent weighs pros/cons of each option
4

Decision

Agent selects the best action
5

Execution

Agent performs the chosen action

​
Key Benefits

​
1. Debugging & Optimization

Identify Issues

Quickly spot where agents make wrong decisions

Optimize Prompts

Refine prompts based on reasoning patterns

Test Variations

A/B test different reasoning approaches

Validate Logic

Ensure agents think as expected

​
2. Transparency & Trust

Provide clear explanations for AI decisions to stakeholders:
  • Audit trails - Complete decision history
  • Compliance reports - Regulatory documentation
  • User explanations - Help users understand AI behavior
Identify potential biases in agent reasoning:
  • Pattern analysis - Detect recurring biases
  • Fairness metrics - Measure decision fairness
  • Corrective actions - Suggestions for improvement
Ensure consistent, high-quality decisions:
  • Reasoning standards - Define quality criteria
  • Deviation alerts - Flag unusual reasoning patterns
  • Continuous improvement - Learn from each decision

​
Implementation Guide

​
Step 1: Enable Thought Tracing

Thought tracing is automatically enabled for all tool calls through the Agentflare proxy. To capture richer reasoning data, add metadata to your tool calls: 
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { HTTPTransport } from "@modelcontextprotocol/sdk/client/http.js";

const client = new Client({
  transport: new HTTPTransport({
    baseUrl: "https://proxy.agentflare.com/<workspace>/<server-slug>",
    headers: {
      "Authorization": `Bearer ${process.env.AGENTFLARE_API_KEY}`
    }
  })
});

await client.connect();
// Thought tracing is now enabled - all tool calls are automatically tracked

​
Step 2: Add Reasoning Context

await client.callTool({
  name: "send_email",
  arguments: {
    to: "<customer@example.com>",
    subject: "Order Status Update",
    body: "Your order #12345 has been shipped"
  },
  metadata: {
    reasoning: "Customer inquired about order status. Found order #12345 is shipped. Sending proactive notification to improve satisfaction.",
    confidence: 0.88,
    alternatives: [
      "wait_for_customer_followup",
      "call_customer_directly",
      "update_order_status_only"
    ],
    context: {
      user_segment: "premium_customer",
      previous_interactions: 3,
      satisfaction_score: 4.2,
      order_value: 299.99
    },
    decision_factors: [
      "customer_premium_status",
      "order_shipping_delay",
      "proactive_communication_policy"
    ]
  }
});

​
Step 3: View in Dashboard

Navigate to Dashboard → Thought Tracing to see:
  • Live Feed
  • Decision Trees
  • Analytics
Real-time stream of agent decisions with reasoning

​
Advanced Features

​
1. Reasoning Patterns

Agentflare identifies common reasoning patterns:

Sequential Reasoning

Step-by-step logical progression

Comparative Analysis

Weighing multiple options

Contextual Decisions

Decisions based on situational context

Fallback Logic

Backup plans when primary options fail

​
2. Confidence Tracking

Monitor decision confidence over time: 
// Track confidence trends
const confidenceMetrics = {
  average: 0.87,
  trend: "+0.03 this week",
  distribution: {
    high: 0.75,    // > 0.8
    medium: 0.20,  // 0.5-0.8
    low: 0.05      // < 0.5
  }
};

​
3. Alternative Analysis

Understand what agents didn’t choose:
Analyze alternative actions that were considered but rejected
Measure the potential value of alternative decisions
Identify cases where alternatives might have been better

​
Best Practices

​
1. Reasoning Quality

  • Clear & Concise
  • Actionable Context
  • Appropriate Detail
// Good
reasoning: "User wants product info. Searching catalog for 'wireless headphones' to find relevant options."

// Avoid
reasoning: "The user has expressed interest in potentially purchasing audio equipment"

​
2. Confidence Calibration

Ensure confidence scores are well-calibrated. A confidence of 0.9 should mean the agent is correct 90% of the time.
{
  confidence: 0.95,  // Very certain
  reasoning: "Exact match found in knowledge base with verified answer"
}

​
3. Alternative Selection

Include meaningful alternatives that were genuinely considered: 
{
  alternatives: [
    "search_knowledge_base",    // Relevant alternative
    "escalate_to_human",       // Relevant alternative
    "ask_for_clarification"    // Relevant alternative
  ],
  // Avoid generic alternatives like:
  // ["do_nothing", "random_action", "give_up"]
}

​
Troubleshooting

If reasoning isn’t appearing:
  1. Check that thoughtTracing.enabled is true
  2. Verify API key has thought tracing permissions
  3. Ensure reasoning metadata is properly formatted
  4. Check for any middleware filtering
If reasoning is unclear or unhelpful:
  1. Review reasoning guidelines with your team
  2. Use reasoning templates for consistency
  3. Implement reasoning validation
  4. Train agents on better reasoning practices
If thought tracing affects performance:
  1. Use asynchronous reasoning capture
  2. Implement reasoning sampling
  3. Optimize reasoning length
  4. Consider batch processing

​
Next Steps

Memory Replay

See complete decision sequences

Performance Analytics

Monitor reasoning performance

Cost Tracking

Track reasoning costs

Tool Calls

Understand tool call patterns
Thought Tracing is most effective when reasoning is added consistently across all tool calls. Start simple and gradually add more context as you learn what’s most valuable.
⌘I