Docs

    Frameworks

    MCP clients

    Trace the tools your agent calls on Model Context Protocol servers, in Python and TypeScript: each call_tool or callTool becomes a tool span with its arguments and result, with a tested client and server tutorial.

    EvalKit traces the client side of the Model Context Protocol. init() patches ClientSession.call_tool in Python and Client.prototype.callTool in TypeScript, so every tool your agent calls on an MCP server is a span with its arguments and result. This page builds an MCP server with one tool and a client agent that uses it, in both languages.

    What you get

    • Each tool call is a tool_call span named after the tool, with mcp.tool.name, gen_ai.tool.call.arguments and gen_ai.tool.call.result. A result with isError set makes it an error span.
    • Only tool calls are recorded by EvalKit. Listing tools, reading resources and getting prompts are not.
    • Servers are not traced: the SDK patches the client class only.
    • The Python mcp package, from 2.x, records its own OpenTelemetry spans, which EvalKit sends as function_call spans: MCP send initialize, MCP send tools/list and MCP send tools/call lookup_order.
    support-turn                          root span from start_trace
  MCP send tools/list                 function_call   recorded by the mcp package
  chat.completions.create             llm_call
    POST                              http_call
  MCP send tools/call lookup_order    function_call   recorded by the mcp package
  lookup_order                        tool_call       recorded by EvalKit
  chat.completions.create             llm_call
    POST                              http_call

    Install

    pip install syntropylabs-evalkit mcp openai

    Set up

    import os

import evalkit

evalkit.init(subscription_key=os.environ["EVALKIT_SUBSCRIPTION_KEY"], service_name="my-agent")

# Import the MCP client after init(); every session.call_tool(...) is then a tool span.
from mcp import ClientSession

    Where init() goes

    • Call init() once in the client process, before the MCP client is imported. The patch is on the client class, so sessions and clients created later are covered.
    • TypeScript: call init() in tracing.ts and import it first, so it runs before @modelcontextprotocol/sdk and openai are evaluated.
    • The server process needs no SDK for the tool calls to be recorded.

    Tutorial: an agent that calls an MCP tool

    The server offers one tool, lookup_order, over stdio. The client starts the server as a subprocess, lists its tools, gives them to the model, calls the tool the model asks for and answers. One question makes two model calls and one tool call, inside one root span for the turn.

    1. 1

      Install the packages

      Run the install line above for your language.

    2. 2

      Set the keys

      Export the environment key of your Environment and your OpenAI key in the shell that runs the client.

    3. 3

      Save the code

      Python: server.py and client.py. TypeScript: src/server.ts, src/tracing.ts and src/client.ts.

    4. 4

      Run the client

      It starts the server itself and prints the model's answer.

    5. 5

      Open the trace

      In Traces, open the newest trace of the service mcp-support or mcp-ts-support.

    Shell
    export EVALKIT_SUBSCRIPTION_KEY=tk_live_...   # Settings → This project → Environments → Copy key
export OPENAI_API_KEY=sk-...
    server.py
    from mcp.server.mcpserver import MCPServer

server = MCPServer("orders")


@server.tool()
def lookup_order(query: str) -> dict:
    """Find an order from the customer's words."""
    return {"order_id": "1001", "status": "shipped", "arrives": "Thursday"}


if __name__ == "__main__":
    server.run()  # stdio, so the client below can start it as a subprocess
    client.py
    import asyncio
import json
import os
import sys

import evalkit

# init() runs before the MCP client is imported, so ClientSession.call_tool is patched.
evalkit.init(
    subscription_key=os.environ["EVALKIT_SUBSCRIPTION_KEY"],
    service_name="mcp-support",
    environment="development",
)

from mcp import ClientSession, StdioServerParameters  # noqa: E402
from mcp.client.stdio import stdio_client  # noqa: E402
from openai import AsyncOpenAI  # noqa: E402

client = AsyncOpenAI()  # reads OPENAI_API_KEY from the environment
SYSTEM = "You are a support agent for an online shop. Use lookup_order for questions about an order."


async def answer(session: ClientSession, message: str) -> str:
    listed = await session.list_tools()
    tools = [{"type": "function", "function": {"name": t.name, "description": t.description or "", "parameters": t.input_schema}}
             for t in listed.tools]
    messages = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": message}]
    reply = (await client.chat.completions.create(model="gpt-4o-mini", messages=messages, tools=tools)).choices[0].message
    if reply.tool_calls:
        messages.append(reply.model_dump(exclude_none=True))
        for call in reply.tool_calls:
            # A tool_call span named after the tool, with its arguments and result.
            result = await session.call_tool(call.function.name, json.loads(call.function.arguments))
            text = "\n".join(block.text for block in result.content if block.type == "text")
            messages.append({"role": "tool", "tool_call_id": call.id, "content": text})
        reply = (await client.chat.completions.create(model="gpt-4o-mini", messages=messages, tools=tools)).choices[0].message
    return reply.content


async def main() -> None:
    server = StdioServerParameters(command=sys.executable, args=["server.py"])
    async with stdio_client(server) as (read, write), ClientSession(read, write) as session:
        await session.initialize()
        evalkit.set_agent("support-bot")  # the trace lists under this agent in Traces > Agents
        with evalkit.session("conv_mcp_1", user_id="u_42"):  # groups the turns of one conversation
            _, end, ctx = evalkit.start_trace("support-turn")  # one root span for the turn
            with evalkit.with_trace(ctx):
                print(await answer(session, "Where is my order 1001?"))
            end()


asyncio.run(main())
    src/server.ts
    import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
import { z } from "zod";

const server = new McpServer({ name: "orders", version: "1.0.0" });

server.registerTool(
  "lookup_order",
  { description: "Find an order from the customer's words.", inputSchema: { query: z.string() } },
  async () => ({ content: [{ type: "text", text: JSON.stringify({ order_id: "1001", status: "shipped", arrives: "Thursday" }) }] }),
);

await server.connect(new StdioServerTransport()); // stdio, so the client can start it as a subprocess
    src/tracing.ts
    import evalkit from "syntropylabs-evalkit";

evalkit.init({
  subscriptionKey: process.env.EVALKIT_SUBSCRIPTION_KEY!,
  serviceName: "mcp-ts-support",
  environment: "development",
});
    src/client.ts
    import "./tracing.js"; // first import: init() runs before the MCP client and OpenAI load
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js";
import OpenAI from "openai";
import evalkit from "syntropylabs-evalkit";

const openai = new OpenAI(); // reads OPENAI_API_KEY from the environment
const SYSTEM = "You are a support agent for an online shop. Use lookup_order for questions about an order.";

const mcp = new Client({ name: "support-bot", version: "1.0.0" });
await mcp.connect(new StdioClientTransport({ command: "npx", args: ["tsx", "src/server.ts"] }));

async function answer(message: string): Promise<string | null> {
  const { tools } = await mcp.listTools();
  const offered: OpenAI.Chat.Completions.ChatCompletionTool[] = tools.map((t) => ({
    type: "function",
    function: { name: t.name, description: t.description ?? "", parameters: t.inputSchema },
  }));
  const messages: OpenAI.Chat.Completions.ChatCompletionMessageParam[] = [
    { role: "system", content: SYSTEM },
    { role: "user", content: message },
  ];
  let reply = (await openai.chat.completions.create({ model: "gpt-4o-mini", messages, tools: offered })).choices[0].message;
  if (reply.tool_calls?.length) {
    messages.push(reply);
    for (const call of reply.tool_calls) {
      if (call.type !== "function") continue;
      // A tool_call span named after the tool, with its arguments and result.
      const result = await mcp.callTool({ name: call.function.name, arguments: JSON.parse(call.function.arguments) });
      const text = (result.content as { type: string; text?: string }[]).map((block) => block.text ?? "").join("\n");
      messages.push({ role: "tool", tool_call_id: call.id, content: text });
    }
    reply = (await openai.chat.completions.create({ model: "gpt-4o-mini", messages, tools: offered })).choices[0].message;
  }
  return reply.content;
}

// One root span for the turn, so the model and tool calls land in one trace.
const turn = evalkit.startTrace("support-turn", { sessionId: "conv_mcp_ts_1", userId: "u_42", agentName: "support-bot" });
console.log(await evalkit.withTrace(turn.ctx, () => answer("Where is my order 1001?")));
turn.end();

await mcp.close();
await evalkit.flush(); // Node does not send queued spans on exit

    The TypeScript files are ES modules that use top-level await: give the project a package.json with "type": "module" (npm pkg set type=module) and run them with tsx.

    python client.py

    Both print the model's answer, for example:

    text
    Order 1001 shipped yesterday and arrives on Thursday.

    Tested with mcp 2.3.0 on syntropylabs-evalkit 0.3.4 (Python: 8 spans in the turn's trace, and MCP send initialize in a trace of its own), and @modelcontextprotocol/sdk 1.32.1 on 0.3.5 (TypeScript: 4 spans in the turn's trace, and the printed answer as a log span of its own).

    See the trace

    • The waterfall shows the trees under What you get. Click lookup_order to read the arguments the model chose and the text the server returned.
    • Traces → Sessions lists conv_mcp_1 (Python) or conv_mcp_ts_1 (TypeScript), and Traces → Agents lists support-bot.

    Sessions, users and agents

    • Python: name the agent with evalkit.set_agent("support-bot"), open the conversation with with evalkit.session(session_id, user_id=...), and start one root span per turn with evalkit.start_trace("support-turn") entered through evalkit.with_trace(ctx).
    • TypeScript: one evalkit.startTrace("support-turn", { sessionId, userId, agentName }) per turn, with the turn run inside evalkit.withTrace(turn.ctx, ...). Every span of the turn inherits the session, user and agent.
    • In a web app, call the tools in the request handler and set the session there, as the page of your framework shows.

    Pitfalls

    • MCP send initialize sits in a trace of its own. The session is initialized before the turn's root span starts, so that span is not part of the turn. Nothing is missing from the turn itself.
    • No spans for tool listing, resources or prompts. Only tool calls are patched. In Python, the mcp package's own MCP send ... spans cover the rest.
    • The server's own work is not traced. Tool calls are recorded on the client, with what the server returned. The work a tool does inside the server process is not part of the client's trace.
    • mcp 1.x and 2.x differ. The Python tutorial uses mcp 2.x, where the server class is MCPServer in mcp.server.mcpserver and a listed tool's schema is input_schema. On 1.x they are FastMCP in mcp.server.fastmcp and inputSchema; the tracing is the same.
    • TypeScript: a script ends before its spans are sent. End it with await evalkit.flush(), after closing the MCP client. It never rejects; on 0.3.4 and earlier, add .catch(() => {}).
    • Python: log lines become spans; your log output stays as it was. Each record that propagates to the root logger is recorded as a log span, as a handler there would record it: at the level of the logger that wrote it, which is the root level, WARNING, unless that logger or your app sets another. init() changes no handler, formatter or level, so your console output stays as it was. Every record carries trace_id and span_id, so a format of your own can show them, as in %(asctime)s %(levelname)s [%(trace_id)s] %(name)s: %(message)s. On 0.3.4 and earlier, init() changes the format of the root logger's console handlers and sets the root level to INFO; configure logging after init() there.

    Safe by design

    In Python:

    • Spans are queued in memory and sent from a background thread, so a request never waits for the trace service.
    • If the trace service is down or unreachable, your app answers as before. The SDK retries each export 3 times, logs evalkit: export failed after 3 retries as a warning and drops those spans.
    • If the key is wrong, the trace service refuses the spans. After 3 refusals in a row the SDK logs one warning and stops exporting until the process restarts; your app keeps answering.
    • A streamed model reply reaches your code chunk by chunk, unchanged. The SDK reads the same chunks on the side and writes the span when the stream ends.
    • Tracing never breaks the call it records. An error inside the SDK is logged at debug level and the call runs as if untraced, and an exception from the provider reaches your code unchanged.
    • A failed export never raises in your code, and evalkit.flush() returns normally too. At exit the SDK sends what is queued within 3 seconds in all, whether or not the trace service answers, so an outage never holds up a deploy.

    In TypeScript:

    • Spans are batched in memory and exported in the background, so a request never waits for the trace service.
    • If the trace service is down or unreachable, your app answers as before. The SDK writes [evalkit] export network error to the console and drops those spans.
    • If the key is wrong, the trace service refuses the spans and they are dropped; your app keeps answering.
    • A streamed model reply reaches your code chunk by chunk, unchanged, as the provider SDK's own stream object: tee(), withResponse(), finalMessage() and the other helpers work as without the SDK. The SDK reads the same chunks on the side and ends the span when the stream ends.
    • Tracing never breaks the call it records. An error inside the SDK's hooks is swallowed and the call goes on as if untraced, and the provider's own errors reach your code unchanged.
    • await evalkit.flush() never rejects: it resolves to true when the spans were sent and false when the export failed or init() has not run, so a trace service outage never fails a job or a serverless invocation. On 0.3.4 and earlier it rejects when the export fails; write await evalkit.flush().catch(() => {}) there.

    Next steps

    EvalKit is built by SyntropyLabs. Published on PyPI and npm.