Claude Agent SDK

The Claude Agent SDK runs Claude Code in a subprocess, so the model calls happen outside your process, where no client can be patched. init() patches the two places your code reads the run instead, claude_agent_sdk.query() and ClaudeSDKClient.receive_response(), and writes one span per run from the messages they return.

What you get

• One llm_call span per run, named claude_agent_sdk.query, with provider anthropic and the model the assistant messages name.
• The prompt you passed, and as the completion the text of every assistant message of the run, joined.
• Each tool Claude used, as a gen_ai.tool.call event on the span with its name, id and input. The tools run inside Claude Code, so they get no spans of their own.
• From the run's ResultMessage: input and output tokens, the cost Claude Code reported in llm.cost_usd, the number of turns in llm.num_turns, the API time in llm.api_latency_ms and the stop reason.
• A run that raises is an error span with the message.

Install

pip install syntropylabs-evalkit claude-agent-sdk

Set up and run

import asyncio
import os

import evalkit

evalkit.init(subscription_key=os.environ["EVALKIT_SUBSCRIPTION_KEY"], service_name="claude-agent")

# Imported after init(): the patch replaces claude_agent_sdk.query on the module, so a name
# imported before init() keeps the unpatched function.
from claude_agent_sdk import ClaudeAgentOptions, query  # noqa: E402


async def main() -> None:
    options = ClaudeAgentOptions(allowed_tools=["Read", "Grep"], max_turns=3)
    with evalkit.session("conv_claude_1", user_id="u_42"):  # groups the runs of one conversation
        async for message in query(prompt="Which files mention refunds?", options=options):
            pass  # the span is written when the ResultMessage arrives


asyncio.run(main())

Where init() goes

• Before claude_agent_sdk is imported, or at least before you import query from it by name. The patch replaces the module's query; from claude_agent_sdk import query above init() binds the original.
• init() imports and patches the package itself when it is installed. If you install it while the process runs, after init(), call evalkit.patch_claude_agent_sdk() once.

Sessions, users and agents

• Wrap the run in evalkit.session(...), as above, and name the agent with evalkit.set_agent(...) before it; both are read when the span is written.
• With ClaudeSDKClient, the span is written as receive_response() ends, so keep the session block around that loop.

Pitfalls

• No span after a run. The span is written when the run's ResultMessage arrives. Read the messages to the end; a loop left early writes no span.
• A ClaudeSDKClient span has no prompt. The span takes its prompt from what the client keeps of the message you sent, which may be empty. The completion, tools, tokens and cost are still recorded.
• Cache tokens are missing. The span records the input and output tokens of the ResultMessage; cache reads and writes are not read from it.
• Log lines become spans; your log output stays as it was. Each record that propagates to the root logger is recorded as a log span, as a handler there would record it: at the level of the logger that wrote it, which is the root level, WARNING, unless that logger or your app sets another. init() changes no handler, formatter or level, so your console output stays as it was. Every record carries trace_id and span_id, so a format of your own can show them, as in %(asctime)s %(levelname)s [%(trace_id)s] %(name)s: %(message)s. On 0.3.4 and earlier, init() changes the format of the root logger's console handlers and sets the root level to INFO; configure logging after init() there.

Safe by design

• Spans are queued in memory and sent from a background thread, so a request never waits for the trace service.
• If the trace service is down or unreachable, your app answers as before. The SDK retries each export 3 times, logs evalkit: export failed after 3 retries as a warning and drops those spans.
• If the key is wrong, the trace service refuses the spans. After 3 refusals in a row the SDK logs one warning and stops exporting until the process restarts; your app keeps answering.
• A streamed model reply reaches your code chunk by chunk, unchanged. The SDK reads the same chunks on the side and writes the span when the stream ends.
• Tracing never breaks the call it records. An error inside the SDK is logged at debug level and the call runs as if untraced, and an exception from the provider reaches your code unchanged.
• A failed export never raises in your code, and evalkit.flush() returns normally too. At exit the SDK sends what is queued within 3 seconds in all, whether or not the trace service answers, so an outage never holds up a deploy.

Next steps

• Choose what is recorded, from prompts to function arguments, and mask what must not leave the process: Configuration and privacy.
• Every library the SDK traces without code, and how streamed calls are counted: What is traced automatically.
• Follow one request across services, or run beside an OpenTelemetry setup you already have: OpenTelemetry and distributed tracing.
• Read the trace you just sent: Your first trace, and group turns into conversations: Sessions and users.