RAAF Troubleshooting Guide

This guide helps you diagnose and resolve common issues when working with Ruby AI Agents Factory (RAAF). From setup problems to runtime errors, this guide provides systematic approaches to identify and fix issues.

After reading this guide, you will know:

How to diagnose common RAAF setup and runtime issues
Debugging techniques for agent behavior problems
Performance troubleshooting and optimization
Error handling best practices
When and how to seek additional help

1. Common Setup Issues

1.1. Installation Problems

1.1.1. Gem Installation Fails

Symptoms: ERROR: Failed building gem native extension

Solutions:

Update build tools: ```bash # macOS xcode-select --install

# Ubuntu/Debian sudo apt-get install build-essential

# CentOS/RHEL sudo yum groupinstall "Development Tools" ```

Install system dependencies: ```bash # For vector storage (if using) pip install numpy scipy

# For NLP features python -m spacy download en_core_web_sm ```

Use specific Ruby version: bash rbenv install 3.1.0 rbenv global 3.1.0 gem install raaf

1.1.2. Missing API Keys

Symptoms: RAAF::Errors::AuthenticationError: API key not configured

Solutions:

Set environment variables: bash export OPENAI_API_KEY="your-api-key" export ANTHROPIC_API_KEY="your-anthropic-key"
Check API key format:

   # OpenAI keys start with 'sk-'
   # Anthropic keys start with 'sk-ant-'
   # Groq keys start with 'gsk_'

Verify API key permissions: bash curl -H "Authorization: Bearer $OPENAI_API_KEY" \ https://api.openai.com/v1/models

1.2. Configuration Issues

1.2.1. Provider Not Found

Symptoms: RAAF::Errors::ProviderError: Unknown provider 'custom_provider'

Solutions:

Check provider spelling:

EXAMPLE VALIDATION FAILED - This example needs work and contributions are welcome! Please see Contributing to RAAF for guidance. Error: NameError: uninitialized constant RAAF::Models::OpenAiProvider /var/folders/r5/1t1h14ts04v5plm6tg1237pr0000gn/T/code_block20250725-12953-1yxsxb.rb:448:in '<main>'

   # Correct
   provider = RAAF::Models::ResponsesProvider.new

   # Incorrect
   provider = RAAF::Models::OpenAiProvider.new  # Wrong case

Ensure provider gem is installed:

EXAMPLE VALIDATION FAILED - This example needs work and contributions are welcome! Please see Contributing to RAAF for guidance. /Users/hajee/.rvm/rubies/ruby-3.4.5/lib/ruby/3.4.0/rubygems/dependency.rb:301:in 'Gem::Dependency#to_specs': Could not find 'anthropic-sdk' (>= 0) among 224 total gem(s) (Gem::MissingSpecError) Checked in 'GEM_PATH=/Users/hajee/.rvm/gems/ruby-3.4.5:/Users/hajee/.rvm/rubies/ruby-3.4.5/lib/ruby/gems/3.4.0' , execute `gem env` for more information from /Users/hajee/.rvm/rubies/ruby-3.4.5/lib/ruby/3.4.0/rubygems/dependency.rb:313:in 'Gem::Dependency#to_spec' from /Users/hajee/.rvm/rubies/ruby-3.4.5/lib/ruby/3.4.0/rubygems/core_ext/kernel_gem.rb:56:in 'Kernel#gem' from /var/folders/r5/1t1h14ts04v5plm6tg1237pr0000gn/T/code_block20250725-12953-zzv1if.rb:445:in '<main>'

   # For Anthropic
   gem 'anthropic-sdk'

   # For Groq
   gem 'groq-ruby'

Use full provider class name:

   runner = RAAF::Runner.new(
     agent: agent,
     provider: RAAF::Models::AnthropicProvider.new
   )

2. Runtime Issues

2.1. Agent Execution Problems

2.1.1. Agent Doesn't Respond

Symptoms:

Agent returns empty responses
Long delays without output
Silent failures

Debugging Steps:

Enable debug logging:

   RAAF.configure do |config|
     config.log_level = :debug
     config.debug_categories = [:api, :agents, :tools]
   end

Check API connectivity:

   # Test basic API call
   runner = RAAF::Runner.new(agent: agent, debug: true)
   result = runner.run("Hello")
   puts result.error if result.error

Verify model availability:

   provider = RAAF::Models::ResponsesProvider.new
   begin
     models = provider.list_models
     puts "Available models: #{models}"
   rescue => e
     puts "API Error: #{e.message}"
   end

2.1.2. Tool Execution Failures

Symptoms: RAAF::Errors::ToolError: Tool 'get_weather' failed to execute

Debugging Steps:

Test tool function directly:

   def get_weather(location:)
     puts "Called with location: #{location}"
     "Weather data for #{location}"
   end

   # Test directly
   result = get_weather(location: "San Francisco")
   puts result

Check tool parameter types:

   # Ensure parameters match expected types
   def calculate_total(price:, tax_rate:)
     # price should be Numeric, not String
     raise ArgumentError, "price must be numeric" unless price.is_a?(Numeric)
     price * (1 + tax_rate)
   end

Add error handling to tools:

   def robust_tool(param:)
     begin
       # Tool logic here
       perform_operation(param)
     rescue => e
       { error: "Tool failed: #{e.message}" }
     end
   end

2.2. Memory and Context Issues

2.2.1. Context Not Preserved

Symptoms:

Agent doesn't remember previous conversation
Context variables reset unexpectedly
Memory seems to be lost

Solutions:

Verify memory manager configuration:

   memory_manager = RAAF::Memory::MemoryManager.new(
     store: RAAF::Memory::InMemoryStore.new,
     max_tokens: 4000
   )

   runner = RAAF::Runner.new(
     agent: agent,
     memory_manager: memory_manager
   )

Check session ID consistency:

   # Use consistent session ID
   session_id = "user_#{user.id}_conversation"

   # All calls should use same session ID
   result1 = runner.run("Hello", session_id: session_id)
   result2 = runner.run("What did I just say?", session_id: session_id)

Debug memory storage:

   # Check what's in memory
   messages = memory_manager.get_messages(session_id)
   puts "Stored messages: #{messages.length}"
   messages.each { |msg| puts "#{msg[:role]}: #{msg[:content]}" }

2.2.2. Memory Pruning Too Aggressive

Symptoms:

Important context gets removed
Agent forgets recent interactions
Conversation becomes incoherent

Solutions:

Adjust pruning strategy:

   memory_manager = RAAF::Memory::MemoryManager.new(
     store: store,
     max_tokens: 8000,  # Increase token limit
     pruning_strategy: :semantic_similarity,  # More intelligent pruning
     preserve_recent_count: 10  # Keep last 10 messages
   )

Use context variables for persistent data:

   runner = RAAF::Runner.new(
     agent: agent,
     memory_manager: memory_manager,
     context_variables: {
       user_name: "Alice",
       preferences: { theme: "dark" },
       conversation_goal: "technical_support"
     }
   )

3. Performance Issues

3.1. Slow Response Times

Symptoms:

Agents take too long to respond
Timeout errors
Poor user experience

Debugging Steps:

Enable performance tracing:

   tracer = RAAF::Tracing::SpanTracer.new
   tracer.add_processor(RAAF::Tracing::ConsoleProcessor.new(
     include_timing: true
   ))

   runner = RAAF::Runner.new(agent: agent, tracer: tracer)

Profile different components:

   require 'benchmark'

   time = Benchmark.measure do
     result = runner.run("Test message")
   end

   puts "Total time: #{time.real}s"

Check model selection:

   # Fast models for simple tasks
   fast_agent = RAAF::Agent.new(
     name: "FastAgent",
     instructions: "Brief responses only",
     model: "gpt-4o-mini"  # Faster than gpt-4o
   )

Optimization Strategies:

Use appropriate models:

   # For simple tasks
   simple_agent.model = "gpt-4o-mini"

   # For complex reasoning
   complex_agent.model = "gpt-4o"

   # For speed
   speed_agent.model = "groq:llama-3.1-70b-versatile"

Optimize memory usage:

   # Reduce memory footprint
   memory_manager = RAAF::Memory::MemoryManager.new(
     store: store,
     max_tokens: 2000,  # Smaller context
     pruning_strategy: :sliding_window
   )

Enable parallel processing:

   agent = RAAF::Agent.new(
     name: "ParallelAgent",
     instructions: "Use tools efficiently",
     model: "gpt-4o",
     parallel_tool_calls: true  # Enable parallel tool execution
   )

3.2. High Token Usage

Symptoms:

Unexpected API costs
Token limit exceeded errors
Poor cost efficiency

Solutions:

Monitor token usage:

   result = runner.run("Test message")
   usage = result.usage

   puts "Prompt tokens: #{usage[:prompt_tokens]}"
   puts "Completion tokens: #{usage[:completion_tokens]}"
   puts "Total tokens: #{usage[:total_tokens]}"

Optimize prompt size:

   # Keep instructions concise
   agent = RAAF::Agent.new(
     name: "EfficientAgent",
     instructions: "Be helpful and concise.",  # Short instructions
     model: "gpt-4o-mini"
   )

Implement cost tracking:

EXAMPLE VALIDATION FAILED - This example needs work and contributions are welcome! Please see Contributing to RAAF for guidance. Error: NameError: uninitialized constant RAAF::Tracing::CostTracker /var/folders/r5/1t1h14ts04v5plm6tg1237pr0000gn/T/code_block20250725-12953-u2zet3.rb:444:in '<main>'

   cost_tracker = RAAF::Tracing::CostTracker.new(
     pricing: {
       'gpt-4o' => { input: 5.00, output: 15.00 },  # Per 1M tokens
       'gpt-4o-mini' => { input: 0.15, output: 0.60 }
     }
   )

4. Error Handling

4.1. API Errors

4.1.1. Rate Limiting

Symptoms: RAAF::Errors::RateLimitError: Rate limit exceeded

Solutions:

Implement exponential backoff:

   provider = RAAF::Models::ResponsesProvider.new(
     max_retries: 5,
     retry_backoff: :exponential,
     retry_jitter: true
   )

Use rate limiting middleware:

   class RateLimitedRunner
     def initialize(runner, requests_per_minute: 60)
       @runner = runner
       @rate_limiter = RateLimiter.new(requests_per_minute)
     end

     def run(message)
       @rate_limiter.wait_if_needed
       @runner.run(message)
     end
   end

4.1.2. Authentication Errors

Symptoms: RAAF::Errors::AuthenticationError: Invalid API key

Solutions:

Verify API key:

   api_key = ENV['OPENAI_API_KEY']
   puts "API key starts with: #{api_key[0..10]}..." if api_key
   puts "API key length: #{api_key&.length}"

Check permissions: bash # Test API access curl -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "Content-Type: application/json" \ -d '{"model":"gpt-3.5-turbo","messages":[{"role":"user","content":"test"}]}' \ https://api.openai.com/v1/chat/completions

4.2. Tool Errors

4.2.1. Function Not Found

Symptoms: RAAF::Errors::ToolError: Function 'undefined_tool' not found

Solutions:

List available tools:

   tools = agent.tools
   puts "Available tools: #{tools.map { |t| t[:function][:name] }}"

Check tool registration:

   def my_tool(param:)
     "Result: #{param}"
   end

   # Register tool
   agent.add_tool(method(:my_tool))

   # Verify registration
   puts agent.tools.inspect

4.2.2. Parameter Validation Errors

Symptoms: ArgumentError: wrong number of arguments

Solutions:

Add parameter validation:

   def validated_tool(required_param:, optional_param: nil)
     raise ArgumentError, "required_param cannot be nil" if required_param.nil?

     # Tool logic here
   end

Use JSON schema validation:

   def schema_validated_tool(params)
     schema = {
       type: "object",
       properties: {
         name: { type: "string" },
         age: { type: "integer", minimum: 0 }
       },
       required: ["name"]
     }

     # Validate params against schema
     JSON::Validator.validate!(schema, params)
   end

5. Debugging Techniques

5.1. Logging and Tracing

5.1.1. Enable Comprehensive Logging

EXAMPLE VALIDATION FAILED - This example needs work and contributions are welcome! Please see Contributing to RAAF for guidance. Error: NameError: uninitialized constant Logger /var/folders/r5/1t1h14ts04v5plm6tg1237pr0000gn/T/code_block20250725-12953-own3hw.rb:451:in '<main>'

# Configure detailed logging
RAAF.configure do |config|
  config.log_level = :debug
  config.debug_categories = [:api, :agents, :tools, :memory, :tracing]
end

# Custom logger
logger = Logger.new(STDOUT)
logger.level = Logger::DEBUG

RAAF.logger = logger

5.1.2. Trace Agent Execution

# Console tracing for debugging
tracer = RAAF::Tracing::SpanTracer.new
tracer.add_processor(RAAF::Tracing::ConsoleProcessor.new(
  log_level: :debug,
  include_payloads: true,
  include_timing: true
))

runner = RAAF::Runner.new(agent: agent, tracer: tracer)

5.1.3. Custom Debug Output

class DebugRunner < RAAF::Runner
  def run(message, **options)
    puts "🚀 Starting conversation with message: #{message}"

    result = super(message, **options)

    puts "✅ Conversation completed in #{result.duration_ms}ms"
    puts "💬 Messages: #{result.messages.length}"
    puts "🔧 Tool calls: #{result.tool_calls.length}"
    puts "📊 Token usage: #{result.usage}"

    result
  end
end

5.2. Testing and Isolation

5.2.1. Mock Providers for Testing

RSpec.describe 'Agent Debugging' do
  let(:mock_provider) { RAAF::Testing::MockProvider.new }
  let(:runner) { RAAF::Runner.new(agent: agent, provider: mock_provider) }

  it 'debugs tool execution' do
    mock_provider.add_tool_response(
      tool_calls: [{ tool_name: 'get_weather', parameters: { location: 'SF' } }],
      final_response: "Weather response"
    )

    result = runner.run("What's the weather?")

    # Debug output
    puts "Tool calls made: #{result.tool_calls}"
    puts "Final response: #{result.messages.last[:content]}"
  end
end

5.2.2. Isolated Component Testing

# Test memory in isolation
memory_store = RAAF::Memory::InMemoryStore.new
memory_manager = RAAF::Memory::MemoryManager.new(store: memory_store)

memory_manager.add_message("test_session", "user", "Hello")
messages = memory_manager.get_messages("test_session")

puts "Stored messages: #{messages}"

6. Performance Profiling

6.1. Memory Profiling

EXAMPLE VALIDATION FAILED - This example needs work and contributions are welcome! Please see Contributing to RAAF for guidance. Error: NameError: undefined local variable or method 'runner' for main /var/folders/r5/1t1h14ts04v5plm6tg1237pr0000gn/T/code_block20250725-12953-mn5ue0.rb:447:in 'block in <main>' /Users/hajee/.rvm/gems/ruby-3.4.5/gems/memory_profiler-1.1.0/lib/memory_profiler/reporter.rb:76:in 'MemoryProfiler::Reporter#run' /Users/hajee/.rvm/gems/ruby-3.4.5/gems/memory_profiler-1.1.0/lib/memory_profiler/reporter.rb:33:in 'MemoryProfiler::Reporter.report'

require 'memory_profiler'

report = MemoryProfiler.report do
  result = runner.run("Complex task")
end

report.pretty_print(to_file: 'memory_profile.txt')

6.2. CPU Profiling

EXAMPLE VALIDATION FAILED - This example needs work and contributions are welcome! Please see Contributing to RAAF for guidance. <internal:/Users/hajee/.rvm/rubies/ruby-3.4.5/lib/ruby/3.4.0/rubygems/core_ext/kernel_require.rb>:136:in 'Kernel#require': cannot load such file -- ruby-prof (LoadError) from <internal:/Users/hajee/.rvm/rubies/ruby-3.4.5/lib/ruby/3.4.0/rubygems/core_ext/kernel_require.rb>:136:in 'Kernel#require' from /var/folders/r5/1t1h14ts04v5plm6tg1237pr0000gn/T/code_block20250725-12953-kf4kbe.rb:444:in '<main>'

require 'ruby-prof'

RubyProf.start

result = runner.run("Performance test")

result = RubyProf.stop
printer = RubyProf::FlatPrinter.new(result)
printer.print(File.open('cpu_profile.txt', 'w'))

6.3. Network Profiling

require 'net/http'

# Monkey patch to track API calls
class Net::HTTP
  alias_method :original_request, :request

  def request(req)
    start_time = Time.now
    response = original_request(req)
    duration = Time.now - start_time

    puts "API Call: #{req.method} #{req.path} - #{duration}s"
    response
  end
end

7. Common Error Messages

7.1. "Agent not responding"

Possible Causes:

Network connectivity issues
API key problems
Model not available
Token limit exceeded

Quick Fix:

# Test with minimal example
agent = RAAF::Agent.new(
  name: "TestAgent",
  instructions: "Just say hello",
  model: "gpt-4o-mini"
)

result = RAAF::Runner.new(agent: agent).run("Hi")
puts result.error || result.messages.last[:content]

7.2. "Tool execution failed"

Possible Causes:

Tool function errors
Parameter mismatches
Network issues in tools
Tool not properly registered

Quick Fix:

# Test tool directly
def test_tool(param:)
  puts "Tool called with: #{param}"
  "Tool result: #{param.upcase}"
end

# Test outside agent
result = test_tool(param: "hello")
puts result

# Then add to agent
agent.add_tool(method(:test_tool))

7.3. "Memory not persisting"

Possible Causes:

Session ID inconsistency
Memory store configuration
Pruning too aggressive
Store not properly initialized

Quick Fix:

# Simple memory test
memory_manager = RAAF::Memory::MemoryManager.new(
  store: RAAF::Memory::InMemoryStore.new
)

session_id = "test_session"
memory_manager.add_message(session_id, "user", "Remember this")
messages = memory_manager.get_messages(session_id)

puts "Messages in memory: #{messages.length}"

8. When to Seek Help

8.1. Community Resources

GitHub Issues: Report bugs or ask questions
GitHub Discussions: Community support and questions
Stack Overflow: Tag questions with raaf and ruby
Documentation: Check latest docs for updates

8.2. Providing Information

When seeking help, include:

Ruby version: ruby --version
RAAF version: gem list raaf
Error message: Full stack trace
Minimal reproduction: Smallest possible example
Environment: OS, dependencies, configuration

8.3. Bug Report Template

# Minimal reproduction case
require 'raaf'

agent = RAAF::Agent.new(
  name: "BugAgent",
  instructions: "Reproduce the bug",
  model: "gpt-4o-mini"
)

runner = RAAF::Runner.new(agent: agent)

begin
  result = runner.run("Trigger the bug")
  puts result.messages.last[:content]
rescue => e
  puts "Error: #{e.class}: #{e.message}"
  puts e.backtrace.first(10)
end

9. Next Steps

For additional help:

API Reference - Complete API documentation
Performance Guide - Optimization techniques
Testing Guide - Testing strategies
Contributing - How to contribute fixes
GitHub Issues - Report problems