Sessions and users

After this page you can read a whole conversation turn by turn instead of one request at a time, get from a failing end user to their sessions and traces, and know the three ids your code sets to make that possible.

Set the ids from the SDK

Every span can carry session_id (one conversation), user_id (the end user your app authenticated) and device_id (the installation or browser, for grouping before sign-in). Without them a trace is still recorded; it just cannot be grouped. Set them for one trace, or once for everything that follows in the current request or task:

trace_id, end, ctx = evalkit.start_trace(
    "chat-turn", user_id="u_8123", session_id="conv_55", device_id="ios-9f2",
)

# or once, for everything that follows in this request / task
evalkit.set_user(user_id="u_8123", session_id="conv_55")
...
evalkit.set_user()          # clears all three
evalkit.current_user()      # what is in effect right now

Python’s set_user() writes to a context variable, so a value set inside a request or an asyncio task never leaks into another one; fields merge, and clear_user(token) restores what was there before. The Django middleware fills user_id from request.user and the FastAPI / Starlette middleware from scope["user"] when the user is authenticated. In TypeScript, setUser inside an active trace binds to that trace only; with no active span it sets a process-wide default. Child spans inherit the trace’s ids, including spans from third-party OpenTelemetry instrumentation.

Sessions view

Traces → Sessions lists one row per session id in the time range: session · user · traces · turns · tool calls · errors · tokens · total latency · last seen. A turn is one llm_call. Simulations are excluded unless you include them. Filter by user or search a session or user id. The list loads 100 sessions at a time with Load more; every header sorts the loaded rows in the browser, and the footer says so. The time range is the same picker the trace list uses, so a link keeps it.

Opening a session shows the header (user, traces, turns, errors, tokens, ≈cost, duration, first seen) and its turns in order as the trace store rolls them up — each with its prompt, status, duration, LLM calls, tokens, cost and agent count. LLM calls, tokens and cost appear only when a turn in the session made a model call, and a turn without one is blank there. Cost is the figure your instrumentation reported when it reported one (gen_ai.usage.cost_usd), otherwise the price-sheet estimate; hover the cell to see which. Open a turn and use prev / next to step through the conversation; Back returns to the session, not to the list. Evaluate this session scores the whole conversation with conversation- or session-scope evaluators; an online rule with session scope does the same continuously once the session has been idle for a set number of minutes.

Users view

Traces → Users lists one row per user id: user · sessions · traces · errors · error % · tokens in · tokens out · first seen · last seen, aggregated in the trace store so counts match the list. Error % is error spans over the user’s traces — the rollup counts spans, so a trace with several failing spans can push it past 100 %; it turns red above 5 %, the same line every APM page draws. The two token columns appear only when a listed user has model-call tokens, and a user without any is blank there. Search a user id; the search and the time range (the shared picker: 15 minutes to 30 days, or custom) live in the URL so a link reproduces the segment. Users load 100 at a time with Load more and every header sorts the loaded rows in the browser. Row actions Sessions and Traces open those views scoped to the user (a removable chip) with the range kept. There is no cost column: the rollup returns tokens per user, not dollars and not tokens per model, so nothing could be priced honestly. End-user thumbs and ratings are stored as scores on traces and are not yet joined onto the user rollup, so there is no feedback column.

Add a session to a dataset

A whole conversation can become a row of a conversation dataset (type conversation, turns kept as context, the final reply as the expected output) — see Golden datasets for the multi-turn shape. Building such a row directly from the Sessions drawer is not wired yet; import the conversation as JSONL or add its traces individually.