Skip to main content

​
Overview

Tool Calls are the fundamental building blocks of AI agent interactions. Every action your agent takes - from searching a database to sending an email - is captured as a tool call with complete context, timing, and reasoning.

​
What Gets Tracked

​
1. Complete Call Information

Every tool call includes comprehensive metadata: 
{
  id: "call_abc123",
  tool_name: "search_products",
  arguments: {
    query: "wireless headphones",
    filters: { price_max: 200, category: "electronics" }
  },
  timestamp: "2024-01-15T10:32:15Z",
  duration: 234,
  status: "success",
  result: {
    products: [...],
    count: 15,
    confidence: 0.92
  },
  cost: 0.023,
  session_id: "session_456def",
  user_id: "user_789ghi"
}

​
2. Contextual Metadata

Rich context about why the call was made:

Reasoning

Why the agent chose this tool

Confidence

How certain the agent was about the choice

Alternatives

What other tools were considered

Context

Situational information affecting the decision

​
3. Performance Metrics

Detailed timing and resource usage:
  • Response Times
  • Resource Usage
  • Cost Tracking
  • Call duration - Total time from invocation to completion
  • Network latency - Time spent in network requests
  • Processing time - Time spent on computation
  • Queue time - Time waiting in processing queue

​
Live Tool Call Feed

​
1. Real-time Monitoring

Watch tool calls as they happen:
Real-time feed of all tool calls across your agents:
  • Streaming updates - See calls as they happen
  • Filtering - Focus on specific tools or agents
  • Search - Find calls by content or metadata
  • Alerts - Get notified of important events
Click any call to see complete details:
  • Full request/response - Complete call data
  • Timing breakdown - Where time was spent
  • Cost analysis - Detailed cost breakdown
  • Related calls - Calls in the same session
Switch between live and historical views:
  • Time range selection - View calls from specific periods
  • Replay mode - Watch historical calls unfold
  • Comparison - Compare current vs. historical performance
  • Trends - See patterns over time
Find exactly what you’re looking for: 
// Filter by tool name
const searchCalls = await client.getToolCalls({
  tool_name: "search_products",
  timeRange: "last_24_hours"
});

// Filter by status
const failedCalls = await client.getToolCalls({
  status: "error",
  timeRange: "last_week"
});

​
Analytics & Insights

​
1. Performance Analytics

Response Time Trends

Monitor how tool response times change over time

Success Rates

Track success/failure rates by tool

Usage Patterns

See which tools are used most frequently

Cost Efficiency

Analyze cost per successful tool call

​
2. Tool Effectiveness Analysis

  • Success Metrics
  • User Impact
const toolMetrics = {
  search_products: {
    success_rate: 0.96,
    avg_duration: 234,
    avg_cost: 0.023,
    user_satisfaction: 0.87
  },
  send_email: {
    success_rate: 0.99,
    avg_duration: 156,
    avg_cost: 0.005,
    user_satisfaction: 0.94
  }
};

​
3. Error Analysis

Identify common failure modes:
  • Timeout errors - Tools taking too long to respond
  • Authentication failures - API key or permission issues
  • Rate limiting - Exceeding API limits
  • Data validation - Invalid arguments or responses
Understand the business impact of errors:
  • User experience - How errors affect users
  • Cost implications - Wasted resources on failed calls
  • Cascading effects - How errors propagate
  • Recovery patterns - How agents handle failures
Monitor error resolution:
  • Time to resolution - How quickly errors are fixed
  • Resolution effectiveness - Whether fixes work
  • Preventive measures - Steps to prevent recurrence
  • Learning opportunities - Insights from errors

​
Optimization Strategies

​
1. Performance Optimization

1

Identify Bottlenecks

Use tool call analytics to find slow or expensive tools
2

Optimize Arguments

Refine tool arguments based on success patterns
3

Improve Caching

Cache frequently used results to reduce calls
4

Batch Operations

Combine multiple small calls into batch operations

​
2. Cost Optimization

Tool Selection

Choose the most cost-effective tools for each task

Argument Optimization

Optimize tool arguments to reduce costs

Caching Strategy

Implement smart caching to avoid duplicate calls

Usage Monitoring

Set budgets and alerts for tool usage

​
3. User Experience Optimization

  • Response Time
  • Success Rate
// Optimize for faster responses
const optimizations = {
  parallel_calls: true,      // Run independent calls in parallel
  early_termination: true,   // Stop on first good result
  progressive_loading: true, // Show results as they arrive
  timeout_handling: true     // Graceful timeout management
};

​
Advanced Features

​
1. Tool Call Correlation

Understand relationships between tool calls: 
// Find related calls
const relatedCalls = await client.getRelatedToolCalls({
  callId: "call_abc123",
  relationship: "same_session",
  depth: 3
});

// Analyze call sequences
const sequences = await client.analyzeCallSequences({
  pattern: "search_products -> get_reviews -> add_to_cart",
  timeRange: "last_month"
});

​
2. Predictive Analytics

Anticipate future tool usage:
Predict future tool usage patterns:
  • Seasonal trends - Tools used more during certain periods
  • User behavior - How different users utilize tools
  • Capacity planning - Prepare for expected load
  • Cost forecasting - Predict future tool costs
Detect unusual patterns:
  • Sudden spikes - Unexpected increases in tool usage
  • Performance degradation - Tools becoming slower
  • Cost anomalies - Unusual increases in costs
  • Error clusters - Groups of related errors

​
3. Custom Metrics

Define your own tool call metrics: 
const customMetrics = {
  business_value: {
    name: "Business Value Score",
    calculation: (call) => {
      // Custom logic to calculate business value
      return call.result.revenue_impact * call.confidence;
    }
  },
  user_satisfaction: {
    name: "User Satisfaction",
    calculation: (call) => {
      // Based on user feedback and behavior
      return call.user_feedback?.rating || 0;
    }
  }
};

​
Best Practices

​
1. Effective Monitoring

Don’t just collect data - actively monitor and act on tool call insights to improve your agents continuously.
  • Set Up Alerts
  • Regular Review
const alerts = {
  slow_calls: {
    condition: "duration > 5000",
    notification: "slack"
  },
  high_error_rate: {
    condition: "error_rate > 0.05",
    notification: "email"
  },
  cost_spike: {
    condition: "hourly_cost > 10",
    notification: "pagerduty"
  }
};

​
2. Data-Driven Decisions

1

Establish Baselines

Measure current performance before making changes
2

A/B Testing

Test optimizations with controlled experiments
3

Impact Measurement

Measure the impact of changes on key metrics
4

Continuous Improvement

Iterate based on results and new insights

​
Integration & Export

​
1. API Access

Access tool call data programmatically: 
// Get live tool calls
const liveStream = client.streamToolCalls({
  filters: { tool_name: "search_products" },
  onCall: (call) => {
    console.log(`New call: ${call.id}`);
    // Process call data
  }
});

​
2. External Integrations

Send data to your existing tools:
  • Analytics Platforms
  • Monitoring Systems
// Send to Google Analytics
await analytics.track("tool_call", {
  tool_name: call.tool_name,
  duration: call.duration,
  success: call.status === "success"
});

​
Next Steps

Performance Analytics

Dive deeper into performance metrics

Cost Tracking

Monitor and optimize costs

Thought Tracing

Understand decision reasoning

Memory Replay

Analyze complete sessions
Tool calls are the foundation of agent observability. The more context you capture with each call, the more valuable insights you’ll gain about your agents’ behavior.
⌘I