Skip to main content

Monitor (Phase 3)

The Monitor phase provides visibility into agent runtime behavior. Observe sessions, view session replays, and track telemetry.

Access via Agent Detail → Monitor tab.

Sessions

View active and completed workflow sessions.

Active Sessions

Real-time view of running sessions:

FieldDescription
Session IDUnique identifier (e.g., sess_20241207_143215_abc123)
StartedStart timestamp
DurationRunning time
EventsEvent count
StatusRunning, Waiting (HITL), Error

Click any active session to watch it in real-time via Session Replay.

Recent Sessions

Completed sessions showing:

  • Session ID and timestamps
  • Duration and event count
  • Goal alignment score
  • Final status (Completed, Failed, Terminated)
  • Governance verdicts summary

Session Replay

Click any session to open the full replay view.

  • Session ID - Unique identifier
  • Duration - Total session time
  • Events - Total event count
  • Goal Alignment - Session alignment score with visual bar
  • Status - Completed, Failed, or Terminated

Timeline Control

Interactive timeline with:

  • Play/Pause - Animate through the session
  • Seek - Click anywhere on the timeline to jump
  • Speed - 0.5x, 1x, 2x playback speed
  • Timestamps - Start and end times

Event Stream

Chronological list of all events in the session:

00:00  Session Started       User initiated support request
00:02 User Input Received "I need help with my recent order"
00:05 LLM Inference Intent classification (gpt-4)
00:08 Database Query SELECT * FROM orders WHERE...
00:12 LLM Inference Response generation
00:15 Output Generated Response to user

Each event shows:

  • Timestamp - Relative time in session
  • Event type - Icon and name
  • Description - What happened
  • Governance verdict - ALLOWED, CONSTRAINED, HALTED badges

Event Detail

Click any event to see full details:

  • Event metadata - Type, timestamp, duration
  • Input/Output - What went in/out (with sensitive data redacted)
  • Governance decision - Which policies evaluated, what was the result
  • Trust impact - How this event affected the trust score

Goal Alignment Badge

The session header shows alignment:

ScoreBadgeMeaning
90-100%GreenWell aligned with stated goal
70-89%YellowMinor deviations
Below 70%RedSignificant drift detected

Hover for details including:

  • Alignment score breakdown
  • LLM evaluation status
  • Stated goal at session start

Observability Metrics

Aggregated metrics across sessions:

Performance

MetricDescription
p50 LatencyMedian operation latency
p95 Latency95th percentile latency
p99 Latency99th percentile latency
ThroughputOperations per minute/hour

Governance

MetricDescription
AllowedOperations that passed governance
ConstrainedOperations modified by guardrails
HaltedOperations blocked by policies
ApprovalsOperations requiring HITL

Charts showing:

  • Session volume over time
  • Latency trends
  • Governance decision distribution
  • Trust score changes

Event Types in Replay

Session replay shows events with semantic types:

TypeIconExample
Session StartedPlayUser initiated support request
User InputMessage"I need help with my order"
LLM InferenceCPU/BrainIntent classification with gpt-4
Database QueryDatabaseSELECT * FROM orders
External APIGlobePOST to stripe.com/charges
Tool CallWrenchexecute_refund()
Output GeneratedMessageSquareResponse sent to user
Approval RequestedClockWaiting for human approval
Approval GrantedCheckCircleApproved by sarah@company.com
Session CompletedCheckCircleSuccess

Telemetry Export

Export telemetry data for external analysis:

  • OpenTelemetry - OTLP export to Jaeger, Datadog, etc.
  • CSV - Download raw event data

Configure in Organisation → Settings → Integrations.

Next Phase

As sessions complete and data accumulates:

Verify - Check that your agent's actions align with its stated goals and detect any drift