Koa

init() patches the Node.js HTTP server, so every request a Koa app answers becomes one trace with the model, tool, HTTP and database calls it made inside. This page sets that up, walks through a small support agent with one tool, and lists what to watch for.

What you get

• One root span per request, with method, URL, status, headers and bodies. Without koaMiddleware() it is named by the method and the raw path, such as GET /orders/1001.
• With koaMiddleware(): the same span, named by the route the router matched, such as GET /orders/:id, or by the middleware's name option.
• An llm_call span for each OpenAI call and a tool_call span for each tool your code runs through traceTool.

The tutorial below records one trace of 4 spans:

POST /chat                     http_call   the request, named by koaMiddleware()
  chat.completions.create      llm_call    the model asks for lookup_order
  lookup_order                 tool_call   its arguments and result
  chat.completions.create      llm_call    the answer

Install

mkdir support-bot && cd support-bot
npm init -y && npm pkg set type=module   # ES modules, as the imports below are written
npm install syntropylabs-evalkit openai koa @koa/router koa-bodyparser
npm install -D tsx typescript @types/node @types/koa @types/koa__router @types/koa-bodyparser

Set up

import evalkit from "syntropylabs-evalkit";

evalkit.init({
  subscriptionKey: process.env.EVALKIT_SUBSCRIPTION_KEY!,
  serviceName: "my-api",
  environment: "development",
});

Each request is traced without koaMiddleware() too, named by its raw path. The middleware names the request's span by the route @koa/router matched; add it before the router, so the route is known when the response is done.

Where init() goes

• Put init() in a module of its own, src/tracing.ts, and import it on the first line of your entry file. ES modules run their imports in the order they are written, so init() runs before Koa, OpenAI and your own modules load.
• Writing evalkit.init() below the other imports of the entry file is too late: every import runs before any statement of the file, so a module that calls traceTool at its top level, such as src/agent.ts, would run before init() and throw.
• Create clients, tools and wrappers after that import. traceTool, traceFunction and the framework adapters need init() to have run, and throw [evalkit] call evalkit.init() before using the SDK when it has not.
• Call init() once per process, at startup. A second call applies its options, such as a new service name, to the spans that start after it, and patches nothing twice.

Tutorial: a support agent with one tool

1. 1

   Create the project

   Install the SDK, the framework and the OpenAI client, as in Install.

2. 2

   Add the files

   Copy src/tracing.ts, src/agent.ts, src/server.ts and tsconfig.json below. The agent offers the model one tool, lookup_order, runs it when the model asks, and sends the result back for the answer.

3. 3

   Set the keys

   Export your environment key and your OpenAI key in the shell. The code reads both from the environment and never writes them anywhere.

4. 4

   Start the server

   Run npx tsx src/server.ts. The server listens on port 3000.

5. 5

   Send one request

   Ask about an order. The model calls lookup_order, reads its result and answers.

6. 6

   Open the trace

   The trace appears in Traces within about 5 seconds, the export interval. See See the trace.

import evalkit from "syntropylabs-evalkit";

evalkit.init({
  subscriptionKey: process.env.EVALKIT_SUBSCRIPTION_KEY!,
  serviceName: "koa-support",
  environment: "development",
});

import OpenAI from "openai";
import evalkit from "syntropylabs-evalkit";

const client = new OpenAI(); // reads OPENAI_API_KEY from the environment

const SYSTEM = "You are a support agent for an online shop. Use lookup_order for questions about an order.";
const TOOLS: OpenAI.Chat.Completions.ChatCompletionTool[] = [{
  type: "function",
  function: {
    name: "lookup_order",
    description: "Find an order from the customer's words.",
    parameters: { type: "object", properties: { query: { type: "string" } }, required: ["query"] },
  },
}];

// A tool_call span with the arguments and the result. traceTool needs init() to have run.
const lookupOrder = evalkit.traceTool("lookup_order", async (args: { query: string }) => {
  return { order_id: "1001", status: "shipped", arrives: "Thursday" };
});

export async function answer(message: string): Promise<string | null> {
  const messages: OpenAI.Chat.Completions.ChatCompletionMessageParam[] = [
    { role: "system", content: SYSTEM },
    { role: "user", content: message },
  ];
  let reply = (await client.chat.completions.create({ model: "gpt-4o-mini", messages, tools: TOOLS })).choices[0].message;
  if (reply.tool_calls?.length) {
    messages.push(reply);
    for (const call of reply.tool_calls) {
      if (call.type !== "function") continue;
      const result = await lookupOrder(JSON.parse(call.function.arguments));
      messages.push({ role: "tool", tool_call_id: call.id, content: JSON.stringify(result) });
    }
    reply = (await client.chat.completions.create({ model: "gpt-4o-mini", messages, tools: TOOLS })).choices[0].message;
  }
  return reply.content;
}

import "./tracing.js"; // first import: init() runs before Koa, OpenAI and the agent load
import Koa from "koa";
import Router from "@koa/router";
import bodyParser from "koa-bodyparser";
import evalkit from "syntropylabs-evalkit";
import { answer } from "./agent.js";

const app = new Koa();
const router = new Router();

router.post("/chat", async (ctx) => {
  const { message, session_id, user_id } = ctx.request.body as { message: string; session_id: string; user_id?: string };
  evalkit.setAgent("support-bot"); // inside a request: names the agent of this trace only
  // Groups the turns of one conversation; the scope follows the async calls inside it.
  const reply = await evalkit.withSession({ sessionId: session_id, userId: user_id }, () => answer(message));
  ctx.body = { reply };
});

app.use(evalkit.koaMiddleware()); // before the router: names each request's span by the route it matched
app.use(bodyParser());
app.use(router.routes());
app.listen(Number(process.env.PORT ?? 3000));

{
  "compilerOptions": {
    "target": "ES2022",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "noEmit": true
  },
  "include": ["src"]
}

export EVALKIT_SUBSCRIPTION_KEY=tk_live_...   # Settings → This project → Environments → Copy key
export OPENAI_API_KEY=sk-...
npx tsx src/server.ts

curl -s -X POST localhost:3000/chat \
  -H 'content-type: application/json' \
  -d '{"message": "Where is my order 1001?", "session_id": "conv_1", "user_id": "u_42"}'

{"reply":"Order 1001 shipped yesterday and arrives on Thursday."}

Tested with syntropylabs-evalkit 0.3.5, Node.js 26, Koa 3.2 with @koa/router and koa-bodyparser, openai 7.33 and tsx 4.23: one trace of 4 spans, with the session conv_1 and the agent support-bot.

See the trace

1. Open Traces in the Environment whose key you used. The newest row is POST /chat from the service koa-support.
2. Open it. The waterfall shows the request span with the two model calls and the tool call inside it.
3. Click the first llm_call span: the request lists the lookup_order tool, and the reply asks for it. The tool_call span shows the arguments the model chose and the order your code returned. The second model call shows the answer, with tokens and cost.
4. Open Traces → Sessions and find conv_1: this request is its first turn. Traces → Agents lists support-bot.

Every response carries the trace id in an x-trace-id header, and a traceparent header for the request span. Paste the id into the search on Traces to open the trace of one request, for example from a support ticket.

Sessions, users and agents

• evalkit.setAgent("support-bot") inside a request names the agent of that request's trace, root span included. Outside a request it names the agent of everything the current async context does next.
• evalkit.withSession({ sessionId, userId }, fn) puts every span started inside fn in the session, following the async calls inside it, and restores the previous scope when fn settles. Concurrent requests never see each other's session. The request span itself started before withSession and carries none, but the trace still belongs to the session, because a trace belongs to every session any of its spans carries.
• evalkit.setUser({ sessionId, userId }) inside a request names the session and user of that request's whole trace, root span included. It binds to the trace, so the next request starts with no identity.

To name the session from headers for every route, call evalkit.setUser({ sessionId, userId }) in a middleware before your router. It runs inside the request, so it names the request's whole trace.

Pitfalls

• Requests fail with 500 after an async middleware, on 0.3.4 and earlier. There a middleware that awaits something before bodyParser() leaves the parser nothing to read, and Koa answers 500; register bodyParser() first. From 0.3.5 the SDK records the body as the parser reads it.
• Operations named by raw path. Without koaMiddleware() the request spans are named GET /orders/1001, not by the router's pattern, so Services lists one operation per id. Add the middleware before the router.
• Health probes on 0.3.4 and earlier. From 0.3.5 no health or readiness probe is recorded (/health, /healthz, /health/live, /readyz, /livez), with or without koaMiddleware(). On 0.3.4 and earlier only /health and /healthz are skipped, and with koaMiddleware() registered even those get the adapter's own span; drop them in a mask hook there.
• HTTPS servers on 0.3.4 and earlier. There init() patches Node's HTTP server only, so a server created with https.createServer gets its request span from koaMiddleware(): keep it in. From 0.3.5 init() patches both.
• A serverless function or a script loses its last spans. Node does not send queued spans when the process ends. Call await evalkit.flush() before a script ends or a serverless handler returns; it never rejects. Do not call it on every request of a long-running server: the batch is sent every 5 seconds anyway.
• Running without tracing. To run without tracing, for example in tests, guard init() on the key variable. Guard koaMiddleware() and traceTool the same way: they need init() and fail with [evalkit] call evalkit.init() before using the SDK without it. evalkit.flush() needs no guard: before init() it resolves to false (on 0.3.4 and earlier it rejects).

Safe by design

• Spans are batched in memory and exported in the background, so a request never waits for the trace service.
• If the trace service is down or unreachable, your app answers as before. The SDK writes [evalkit] export network error to the console and drops those spans.
• If the key is wrong, the trace service refuses the spans and they are dropped; your app keeps answering.
• A streamed model reply reaches your code chunk by chunk, unchanged, as the provider SDK's own stream object: tee(), withResponse(), finalMessage() and the other helpers work as without the SDK. The SDK reads the same chunks on the side and ends the span when the stream ends.
• Tracing never breaks the call it records. An error inside the SDK's hooks is swallowed and the call goes on as if untraced, and the provider's own errors reach your code unchanged.
• await evalkit.flush() never rejects: it resolves to true when the spans were sent and false when the export failed or init() has not run, so a trace service outage never fails a job or a serverless invocation. On 0.3.4 and earlier it rejects when the export fails; write await evalkit.flush().catch(() => {}) there.

Next steps

• Choose what is recorded, from prompts to function arguments, and mask what must not leave the process: Configuration and privacy.
• Every library the SDK traces without code, and how streamed calls are counted: What is traced automatically.
• Follow one request across services, or run beside an OpenTelemetry setup you already have: OpenTelemetry and distributed tracing.
• Read the trace you just sent: Your first trace, and group turns into conversations: Sessions and users.