Topics, agent versions and release comparison

After this page you can say what users actually ask, which topic fails most and what recurring problem sits behind it; tell which agent configuration is live without tagging releases; and answer whether the version shipped yesterday is worse than the one before.

Topics

Topics are a view over Traces, not a new nav item, and need the trace-optimization plan feature and a Model. Extract topics now starts a job over a time range and a sample of up to 500 traces (200 by default): the first user message of each trace is sent to the Model in batches, labelled, normalised and merged onto existing topics, so a trace belongs to exactly one topic and re-extraction reassigns it. The job reports traces scanned, traces labelled, skipped for no content or a fetch failure, and topics touched. The list shows each topic’s traces, error rate, average latency, first and last seen, a seven-day trend and its pattern count, sorted by volume. Error rate and latency read as unknown, not zero, when there is nothing to divide.

Find patterns in this topic digests up to twenty of the topic’s traces — errors first, then slowest — and asks the Model for up to eight recurring problems, each with a title, what happened, why, evidence trace ids drawn only from that sample, and a paste-ready fix prompt for a coding agent. Patterns are stored on the topic with the time they were generated.

• The trace list has no topic: filter; a topic’s traces are listed from the topic itself.
• There is no scheduled extraction, no cost estimate before a run and no alert on a pattern growing.
• Tokens and cost per topic are not aggregated; a topic carries counts, error rate and latency only.

Agent versions

At ingest every llm_call gets an agent version: the first 16 hex characters of a hash over the normalised system prompt, the sorted tool names, the model and the temperature / top-p values. The same configuration always gives the same id; a changed prompt, tool set, model or sampling parameter gives a new one. No tagging is needed, and when your SDK also sets app_version the two are shown side by side.

Traces → Agents lists the versions seen in the range, newest first: model, tool names, a preview of the system prompt, first and last seen, trace count, call count and error rate. agentVersion is a filter on the trace list, its facets and export, and on the timeseries and analytics rollups, so a version’s traces, error rate and latency are one click away. A per-version pass rate needs the scores that live in the control plane and is not served with the list. A monitor can break down by agent version and pin a baseline version — see Monitors.

Compare two releases

Compare releases (from Home’s header, or …/traces/compare) puts two values of one dimension — app version by default, or Environment, or agent version — side by side over one time range. The values offered are the ones seen in that range, busiest first, with their trace counts; the busiest is side B (the release most likely live now) until you pick. Each side shows spans, error rate, LLM p50 and p95, model calls and tokens, with a chart of spans, error spans and p95 per bucket; the two charts share one crosshair, and dragging on either zooms both. Below, a table gives each figure for A and B and the change from A to B, red for regressions and green for improvements. Open traces for A / B opens the trace list filtered to that value with the range kept. Cost is not compared, because the cost rollup has no version filter, and neither is pass rate, because scores are not yet queryable next to traces. The dimension and both sides live in the URL, so a comparison is a link.