Runtime Observability
Runtime observability means more than pass/fail run status.
Inspect a NodeFox run
Use node state, outputs and logs in the application to identify where a run progressed or failed. Configure log depth and audit output in Settings before a diagnostic run. Use the replay viewer for captured execution evidence; replay inspection is distinct from rerunning live model calls or external writes. Headless records are retained on the runner machine according to its retention settings.
The checklist below is an operating design: decide which fields your workflow and surrounding systems must capture. Custom business rationale, external alerting and retention policies require configuration; they are not all automatic fields in every run.
What good observability includes
- branch decisions and reasons
- node-level payload transitions
- tool invocation traces
- approval and acceptance events
- route-level cost and retry behavior
- graph version and contract context for each run
- activation release events and originating conditions
- loop counters, timeout paths, and escalation outcomes
Why this matters
When incidents occur, teams need exact execution evidence. Observability is what turns incident response from guesswork into diagnosis.
Without run-level evidence, teams cannot answer core questions quickly:
- which branch fired the side effect?
- what payload shape was present at release time?
- did fallback execute because of model quality, dependency failure, or policy logic?
- was the run using the expected graph version?
Minimum run record for production workflows
Capture at least:
- run id, graph id/version, and start/end timestamps
- node execution sequence with eligibility decisions
- activation events and gate outcomes
- branch/decision rationale and matched rule metadata
- side-effect request/response status (sanitized)
- retry/fallback/escalation transitions
- per-branch cost and latency stats
Cost analytics and budget enforcement signals
Observability should support economics, not only debugging:
- token usage by Conversation branch and variant
- tool-call count and latency by node
- retry amplification factor by workflow
- budget threshold crossings and enforced throttles
- expensive-loop detection (iterations vs value delivered)
Implementation baseline
- capture branch outcome context
- capture side-effect result status
- classify retry/fallback reasons
- tie cost to route path
- preserve replayable snapshots at critical decision points
- expose anomaly alerts for sudden branch-distribution shifts
Anti-patterns to avoid
- logging only final outputs without transition context
- storing raw sensitive payloads instead of redacted evidence
- missing graph version in run logs
- monitoring latency only while ignoring branch-quality drift