Observability for Agent Orchestration, Pipelines and Swarms

Narrative

Independent commentary on agent observability clusters around three questions: whether OpenTelemetry's GenAI conventions are actually load-bearing yet, whether the "two-layer" durable-orchestration-plus-tracing pattern is real engineering practice or marketing, and whether anyone outside frontier labs can debug a non-deterministic multi-agent run at all. Simon Willison's newsletter is the most consistently cited independent voice: his piece on the "lethal trifecta" argues that effective observability requires characterising every agent action as read-only versus state-changing so humans and automated reviewers can catch prompt-injection-driven exfiltration, tying observability directly to security rather than treating it as a debugging convenience. His "Agentic Engineering Patterns" guide, expanded through 2026 with a subagents chapter, documents the parallel-agent workflows practitioners are actually running, including his own admission of losing track of which git worktree or cloud instance held a feature after a crash, a small but concrete illustration of the state-tracking problem that tooling vendors describe abstractly.

On the architecture debate, Patrick McGuinness's Substack post "The AI Agent Architecture Debate" is the clearest independent synthesis of the Anthropic-versus-Cognition split. He notes that Standard observability and logging tools were insufficient, so Anthropic built a custom system to monitor agent decisions and interactions to diagnose failures