Fastify

init() patches the Node.js HTTP server, so every request a Fastify app answers becomes one trace with the model, tool, HTTP and database calls it made inside. fastifyPlugin() adds the route name. This page sets that up, walks through a small support agent with one tool, and lists what to watch for.

What you get

• Without the plugin: one root span per request, named by the raw path, such as GET /orders/1001, with method, URL, status, headers and bodies.
• With fastifyPlugin(): the same span, named by the route, GET /orders/:id, or by the plugin's name option, so Services groups requests per endpoint.
• An llm_call span for each OpenAI call and a tool_call span for each tool your code runs through traceTool, inside the request.

The tutorial below, with the plugin, records one trace of 4 spans:

POST /chat                     http_call   the request, named by fastifyPlugin()
  chat.completions.create      llm_call    the model asks for lookup_order
  lookup_order                 tool_call   its arguments and result
  chat.completions.create      llm_call    the answer

Install

mkdir support-bot && cd support-bot
npm init -y && npm pkg set type=module   # ES modules, as the imports below are written
npm install syntropylabs-evalkit openai fastify
npm install -D tsx typescript @types/node

Set up

import evalkit from "syntropylabs-evalkit";

evalkit.init({
  subscriptionKey: process.env.EVALKIT_SUBSCRIPTION_KEY!,
  serviceName: "my-api",
  environment: "development",
});

The plugin opts out of Fastify's plugin encapsulation, so its hooks apply to the whole app; register it before your routes. Without it, each request is still one trace, named by its raw path.

Where init() goes

• Put init() in a module of its own, src/tracing.ts, and import it on the first line of your entry file. ES modules run their imports in the order they are written, so init() runs before Fastify, OpenAI and your own modules load.
• Writing evalkit.init() below the other imports of the entry file is too late: every import runs before any statement of the file, so a module that calls traceTool at its top level, such as src/agent.ts, would run before init() and throw.
• Create clients, tools and wrappers after that import. traceTool, traceFunction and the framework adapters need init() to have run, and throw [evalkit] call evalkit.init() before using the SDK when it has not.
• Call init() once per process, at startup. A second call applies its options, such as a new service name, to the spans that start after it, and patches nothing twice.

Tutorial: a support agent with one tool

1. 1

   Create the project

   Install the SDK, the framework and the OpenAI client, as in Install.

2. 2

   Add the files

   Copy src/tracing.ts, src/agent.ts, src/server.ts and tsconfig.json below. The agent offers the model one tool, lookup_order, runs it when the model asks, and sends the result back for the answer.

3. 3

   Set the keys

   Export your environment key and your OpenAI key in the shell. The code reads both from the environment and never writes them anywhere.

4. 4

   Start the server

   Run npx tsx src/server.ts. The server listens on port 3000.

5. 5

   Send one request

   Ask about an order. The model calls lookup_order, reads its result and answers.

6. 6

   Open the trace

   The trace appears in Traces within about 5 seconds, the export interval. See See the trace.

import evalkit from "syntropylabs-evalkit";

evalkit.init({
  subscriptionKey: process.env.EVALKIT_SUBSCRIPTION_KEY!,
  serviceName: "fastify-support",
  environment: "development",
});

import OpenAI from "openai";
import evalkit from "syntropylabs-evalkit";

const client = new OpenAI(); // reads OPENAI_API_KEY from the environment

const SYSTEM = "You are a support agent for an online shop. Use lookup_order for questions about an order.";
const TOOLS: OpenAI.Chat.Completions.ChatCompletionTool[] = [{
  type: "function",
  function: {
    name: "lookup_order",
    description: "Find an order from the customer's words.",
    parameters: { type: "object", properties: { query: { type: "string" } }, required: ["query"] },
  },
}];

// A tool_call span with the arguments and the result. traceTool needs init() to have run.
const lookupOrder = evalkit.traceTool("lookup_order", async (args: { query: string }) => {
  return { order_id: "1001", status: "shipped", arrives: "Thursday" };
});

export async function answer(message: string): Promise<string | null> {
  const messages: OpenAI.Chat.Completions.ChatCompletionMessageParam[] = [
    { role: "system", content: SYSTEM },
    { role: "user", content: message },
  ];
  let reply = (await client.chat.completions.create({ model: "gpt-4o-mini", messages, tools: TOOLS })).choices[0].message;
  if (reply.tool_calls?.length) {
    messages.push(reply);
    for (const call of reply.tool_calls) {
      if (call.type !== "function") continue;
      const result = await lookupOrder(JSON.parse(call.function.arguments));
      messages.push({ role: "tool", tool_call_id: call.id, content: JSON.stringify(result) });
    }
    reply = (await client.chat.completions.create({ model: "gpt-4o-mini", messages, tools: TOOLS })).choices[0].message;
  }
  return reply.content;
}

import "./tracing.js"; // first import: init() runs before Fastify, OpenAI and the agent load
import Fastify from "fastify";
import evalkit from "syntropylabs-evalkit";
import { answer } from "./agent.js";

const app = Fastify();
await app.register(evalkit.fastifyPlugin()); // names each request's span by its route

app.post<{ Body: { message: string; session_id: string; user_id?: string } }>("/chat", async (request) => {
  const { message, session_id, user_id } = request.body;
  evalkit.setAgent("support-bot"); // inside a request: names the agent of this trace only
  // Groups the turns of one conversation; the scope follows the async calls inside it.
  const reply = await evalkit.withSession({ sessionId: session_id, userId: user_id }, () => answer(message));
  return { reply };
});

await app.listen({ port: Number(process.env.PORT ?? 3000) });

{
  "compilerOptions": {
    "target": "ES2022",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "noEmit": true
  },
  "include": ["src"]
}

export EVALKIT_SUBSCRIPTION_KEY=tk_live_...   # Settings → This project → Environments → Copy key
export OPENAI_API_KEY=sk-...
npx tsx src/server.ts

curl -s -X POST localhost:3000/chat \
  -H 'content-type: application/json' \
  -d '{"message": "Where is my order 1001?", "session_id": "conv_1", "user_id": "u_42"}'

{"reply":"Order 1001 shipped yesterday and arrives on Thursday."}

Tested with syntropylabs-evalkit 0.3.5, Node.js 26, Fastify 5.12, openai 7.33 and tsx 4.23: one trace of 4 spans, with the session conv_1 and the agent support-bot.

See the trace

1. Open Traces in the Environment whose key you used. The newest row is POST /chat from the service fastify-support.
2. Open it. The waterfall shows the request span with the two model calls and the tool call inside it.
3. Click the first llm_call span: the request lists the lookup_order tool, and the reply asks for it. The tool_call span shows the arguments the model chose and the order your code returned. The second model call shows the answer, with tokens and cost.
4. Open Traces → Sessions and find conv_1: this request is its first turn. Traces → Agents lists support-bot.

Every response carries the trace id in an x-trace-id header, and a traceparent header for the request span. Paste the id into the search on Traces to open the trace of one request, for example from a support ticket.

Sessions, users and agents

• evalkit.setAgent("support-bot") inside a request names the agent of that request's trace, root span included. Outside a request it names the agent of everything the current async context does next.
• evalkit.withSession({ sessionId, userId }, fn) puts every span started inside fn in the session, following the async calls inside it, and restores the previous scope when fn settles. Concurrent requests never see each other's session. The request span itself started before withSession and carries none, but the trace still belongs to the session, because a trace belongs to every session any of its spans carries.
• evalkit.setUser({ sessionId, userId }) inside a request names the session and user of that request's whole trace, root span included. It binds to the trace, so the next request starts with no identity.

To name the session from headers for every route, call evalkit.setUser({ sessionId, userId }) in a preHandler hook. It runs inside the request, so it names the request's whole trace.

Pitfalls

• A POST never gets an answer, on 0.3.4 and earlier. There an async onRequest hook, such as an authentication check that awaits a token lookup, makes Fastify wait for a body the SDK had already read, and the request hangs; move the asynchronous work to a preValidation or preHandler hook. From 0.3.5 an async onRequest hook works: in a run with one, the request was answered and its body recorded.
• Operations named by raw path. Without the plugin the request span is named GET /orders/1001, one operation per id. Register fastifyPlugin() to name it by the route.
• Health probes on 0.3.4 and earlier. From 0.3.5 no health or readiness probe is recorded (/health, /healthz, /health/live, /readyz, /livez), with or without fastifyPlugin(). On 0.3.4 and earlier only /health and /healthz are skipped, and with fastifyPlugin() registered even those get the adapter's own span; drop them in a mask hook there.
• Streamed replies. A model call you stream back stays in the request's trace, and its span ends when the stream ends. Pass stream_options: { include_usage: true } so OpenAI sends token counts in the stream.
• A serverless function or a script loses its last spans. Node does not send queued spans when the process ends. Call await evalkit.flush() before a script ends or a serverless handler returns; it never rejects. Do not call it on every request of a long-running server: the batch is sent every 5 seconds anyway.
• Running without tracing. To run without tracing, for example in tests, guard init() on the key variable. Guard fastifyPlugin() and traceTool the same way: they need init() and fail with [evalkit] call evalkit.init() before using the SDK without it. evalkit.flush() needs no guard: before init() it resolves to false (on 0.3.4 and earlier it rejects).

Safe by design

• Spans are batched in memory and exported in the background, so a request never waits for the trace service.
• If the trace service is down or unreachable, your app answers as before. The SDK writes [evalkit] export network error to the console and drops those spans.
• If the key is wrong, the trace service refuses the spans and they are dropped; your app keeps answering.
• A streamed model reply reaches your code chunk by chunk, unchanged, as the provider SDK's own stream object: tee(), withResponse(), finalMessage() and the other helpers work as without the SDK. The SDK reads the same chunks on the side and ends the span when the stream ends.
• Tracing never breaks the call it records. An error inside the SDK's hooks is swallowed and the call goes on as if untraced, and the provider's own errors reach your code unchanged.
• await evalkit.flush() never rejects: it resolves to true when the spans were sent and false when the export failed or init() has not run, so a trace service outage never fails a job or a serverless invocation. On 0.3.4 and earlier it rejects when the export fails; write await evalkit.flush().catch(() => {}) there.

Next steps

• Choose what is recorded, from prompts to function arguments, and mask what must not leave the process: Configuration and privacy.
• Every library the SDK traces without code, and how streamed calls are counted: What is traced automatically.
• Follow one request across services, or run beside an OpenTelemetry setup you already have: OpenTelemetry and distributed tracing.
• Read the trace you just sent: Your first trace, and group turns into conversations: Sessions and users.