---
name: syntropylabs-evalkit
description: Integrate, configure, verify or debug the EvalKit SDK (syntropylabs-evalkit) in a Python or TypeScript service, so requests and their LLM, tool, HTTP and database calls are traced to SyntropyLabs. Use when asked to add SyntropyLabs or EvalKit tracing, when editing code that calls evalkit.init(), or when traces do not arrive.
---

# EvalKit SDK integration

Source: https://syntropylabs.ai/docs/evalkit-skill.md. Written for `syntropylabs-evalkit` 0.3.2 on PyPI and npm. To refresh it, fetch the URL and replace the saved copy.

EvalKit runs inside the user's server process. One `init()` call traces the installed LLM, HTTP and database clients, and one middleware line per web framework makes each request one trace.

## When to use this skill

- The user asks to add, check or fix SyntropyLabs or EvalKit tracing.
- You are editing code that imports `evalkit` or `syntropylabs-evalkit`.
- Traces from this project do not show up in SyntropyLabs.

Do not use it to trace the coding agent itself; that is `evalkit coding install` (https://syntropylabs.ai/docs/coding-agents).

## 1. Save this skill

Before changing code, save this document in the repository for later sessions. Pick the row for the agent you are:

| Agent | Save to | Notes |
| --- | --- | --- |
| Claude Code | `.claude/skills/syntropylabs-evalkit/SKILL.md` | Save as is. |
| Codex | `.agents/skills/syntropylabs-evalkit/SKILL.md` | Save as is. |
| Gemini CLI | `.gemini/skills/syntropylabs-evalkit/SKILL.md` | Save as is. It also reads `.agents/skills/`. |
| GitHub Copilot | `.github/skills/syntropylabs-evalkit/SKILL.md` | Save as is. It also reads `.claude/skills/` and `.agents/skills/`. |
| Cursor | `.cursor/rules/syntropylabs-evalkit.mdc` | Front matter: `description` (the same text) and `alwaysApply: false`. |
| Windsurf | `.windsurf/rules/syntropylabs-evalkit.md` | Front matter: `trigger: model_decision` and `description`. |
| Any other agent | `AGENTS.md` | Add an `## EvalKit` section holding everything below the front matter. |

Keep the text as is except the front matter the row names. Tell the user the path; committing it is their choice.

## 2. Detect the language and framework

- Python: `pyproject.toml`, `requirements*.txt`, `setup.py`, `Pipfile`, `uv.lock` or `poetry.lock`. Needs Python 3.9 or later.
- TypeScript or JavaScript on Node.js: `package.json`. Needs Node 18 or later.
- Framework: read the dependencies. Python: `fastapi`, `starlette`, `flask`, `django`, `litestar`. Node: `express`, `fastify`, `koa`, `hono`, `@hapi/hapi`, `@nestjs/core`.
- In a monorepo, integrate each server process the user wants traced, one at a time.
- Any other language: stop and tell the user. EvalKit has SDKs for Python and TypeScript only.

## 3. Install with the project's package manager

| Lockfile or manifest | Command |
| --- | --- |
| `uv.lock` | `uv add syntropylabs-evalkit` |
| `poetry.lock` | `poetry add syntropylabs-evalkit` |
| `Pipfile` | `pipenv install syntropylabs-evalkit` |
| `requirements.txt` | add a `syntropylabs-evalkit` line, then `pip install -r requirements.txt` |
| `pnpm-lock.yaml` | `pnpm add syntropylabs-evalkit` |
| `yarn.lock` | `yarn add syntropylabs-evalkit` |
| `bun.lock` or `bun.lockb` | `bun add syntropylabs-evalkit` |
| `package-lock.json` or none | `npm install syntropylabs-evalkit` |

The Python package imports as `evalkit`. In TypeScript, provider SDKs such as `openai` and `@anthropic-ai/sdk` are optional peer dependencies. Do not add ones the project does not use.

## 4. Read the key from an environment variable

The environment key (`tk_live_...`) belongs to one Environment of a SyntropyLabs project; the user copies it from Settings, This project, Environments. Neither SDK reads it from the environment, so the code passes it in.

- Use the project's existing variable for it. If there is none, ask the user what to call it and suggest `EVALKIT_SUBSCRIPTION_KEY`.
- Never write the key into a file, a commit, a test fixture, a log line or your reply. Never print it.
- Add the variable name, with an empty value, to `.env.example` (or `.env.sample`, `.env.template`) if the project has one. Do not create or edit `.env`. Tell the user to set the value themselves.
- Only call `init()` when the variable is set, so the app still starts without it.

## 5. Call init() once, at the entry point

Call it once per process, as early as possible: before the modules that create LLM, HTTP or database clients are imported. A second call returns the first client and ignores its arguments.

Python, at the top of the module that creates the app (`main.py`, `app.py`, `wsgi.py`, `asgi.py`, or `settings.py` for Django):

```python
import os
import evalkit

if os.environ.get("EVALKIT_SUBSCRIPTION_KEY"):
    evalkit.init(
        subscription_key=os.environ["EVALKIT_SUBSCRIPTION_KEY"],
        service_name="checkout-api",   # shown in Traces and Services
        environment="development",     # development, staging or production
    )
```

TypeScript: create `src/evalkit.ts` (`.js` in a JavaScript project) and make it the entry file's first import, in the project's import style, for example `import { evalkitEnabled } from "./evalkit.js";` in ESM:

```ts
import evalkit from "syntropylabs-evalkit";

const subscriptionKey = process.env.EVALKIT_SUBSCRIPTION_KEY;
export const evalkitEnabled = Boolean(subscriptionKey);

if (subscriptionKey) {
  evalkit.init({
    subscriptionKey,
    serviceName: "checkout-api",
    environment: "development",
  });
}
```

- Take `service_name` from the project's name. Map `environment` from an existing project setting, if any.
- `base_url` (TypeScript: `baseUrl`) is the trace ingest URL. Leave it out for the hosted service; pass it only for a self-hosted or local receiver, from a variable of the project's own (the SDK reads none).
- A short-lived script or job calls `evalkit.flush()` (TypeScript: `await evalkit.flush()`) before it exits.
- Python traces every public module-level function under the directory of the file that calls `init()`, except those defined after `init()` in that file. Other top-level packages need `trace_packages=["orders", "billing"]`. `function_tracing=False` turns it off. TypeScript traces only functions you wrap.

## 6. Add the framework's middleware line

Python:

```python
# FastAPI or Starlette
from evalkit import EvalKitMiddleware
app.add_middleware(EvalKitMiddleware)

# Flask, after init()
evalkit.instrument_flask(app)

# Django: add to MIDDLEWARE. With init() in settings.py, pass the app packages
# as trace_packages=[...], or their functions are not traced
"evalkit.EvalKitDjangoMiddleware"

# Litestar
from evalkit import create_litestar_middleware
app = Litestar(route_handlers=[...], middleware=[create_litestar_middleware()])
```

FastAPI versions whose `FastAPI()` takes a `telemetry` argument trace each request a second time through EvalKit's OpenTelemetry bridge. Create the app with `FastAPI(telemetry={"tracing": False})` there.

TypeScript, before the routes. Every adapter throws until `init()` runs, so guard it with the flag from `src/evalkit.ts`:

```ts
import evalkit from "syntropylabs-evalkit";
import { evalkitEnabled } from "./evalkit.js";

if (evalkitEnabled) app.use(evalkit.expressMiddleware());   // Express
// Fastify: await app.register(evalkit.fastifyPlugin())
// Koa: app.use(evalkit.koaMiddleware())
// Hono: app.use(evalkit.honoMiddleware())
// Hapi: await server.register(evalkit.hapiPlugin())
// NestJS: app.useGlobalInterceptors(evalkit.createNestjsInterceptor()) after NestFactory.create(),
//   or await evalkit.enableNestjsAutoTrace(app) to also trace every provider and controller method
```

When `init()` is guarded, guard `instrument_flask(app)` and the Litestar middleware too. The FastAPI, Starlette and Django middleware pass requests through untraced until `init()` runs. With no supported framework, skip this step: calls are still traced, each as its own trace.

## 7. Content capture and privacy

Prompts, completions, request bodies, tool arguments and SQL text are captured by default. A secret scrubber masks credentials (`Authorization`, `x-api-key` and similar keys) in every span whatever the setting. Identity such as user ids and emails is not masked.

If the app handles personal or regulated data, ask the user which they want:

- `capture_content=False` (TypeScript: `captureContent: false`), or `EVALKIT_CAPTURE_CONTENT=false` in the environment: keeps tokens, cost, latency, model and tool names, drops every payload.
- A `mask` hook passed to `init()`: it receives each span (fields such as `prompt`, `completion` and `attributes`) before export and returns it changed, or `None` (TypeScript: `null`) to drop it. If the hook raises, the span is dropped.

## 8. Verify

1. Start the app with the variable set, in the user's usual way.
2. Send one request, for example with `curl -i`. The response carries an `x-trace-id` header (Python: every framework integration above; TypeScript: every Node.js HTTP server).
3. In SyntropyLabs, open Traces for the key's Environment. Within about 5 seconds the request shows as one trace, such as `GET /orders/42`. LLM and database calls are child spans; outbound HTTP calls are too in Python, and events on the request span in TypeScript. Calls to `/health` paths and to the ingest URL are not recorded.
4. More than one trace for one request means a second tracer is active, such as FastAPI's own tracing (step 6).

If nothing arrives:

- `401` or `403` from the ingest endpoint: the key does not belong to this Environment. After 3 in a row the SDK stops exporting until the process restarts.
- The process cannot reach `https://api.syntropylabs.ai` (or the `base_url`). Check proxies and egress rules.
- The process exited before the batch was sent. Call `flush()` at the end.
- `init()` did not run: the variable is missing in that process, or the code that calls it is not reached.
- The user is looking at another Environment (the switcher is in the top bar).
- TypeScript `debug: true` prints each export. Python `debug=True` logs at DEBUG on the `evalkit` logger, shown only after `logging.basicConfig()` and `logging.getLogger("evalkit").setLevel(logging.DEBUG)`. Export and key failures are warnings either way.

## 9. Optional next steps

Offer these once traces arrive. Do not add them unasked.

- Sessions and users: `with evalkit.session("conv_55", user_id="u_8123"):` in Python, `withSession({ sessionId, userId }, fn)` in TypeScript. Agent names: `evalkit.set_agent("support-bot")` or `setAgent("support-bot")`.
- `Eval()` in CI with a gate (paid plan feature): `evalkit eval run eval.json` (TypeScript: `npx evalkit eval run eval.json`) exits 0 when the gate passes and non-zero otherwise. The CI key goes in a secret named `EVALKIT_API_KEY`.
- For agent metrics, an `Eval()` row needs `trace_id` or `session_id` (a recorded trace or session) or `messages` (OpenAI chat format; optional `tools`, `system_prompt`). With 0.3.2 put them in `metadata`; 0.3.3 also takes them top level (TypeScript: `traceId`, `sessionId`, `systemPrompt`).
- Prompts (paid plan feature): `evalkit.prompts.get("support_agent_system", label="production")` fetches a versioned prompt.

## Do not

- Do not look for or install a Go, Java or browser SDK; none exist.
- Do not wrap or patch what `init()` already instruments: OpenAI, Anthropic, Bedrock, Google, Vertex, Cohere, Mistral (Python), LangChain, LiteLLM (Python), `requests`, `httpx`, `aiohttp`, `fetch`, `axios` and the database drivers. Do not call the `patch_*` helpers on top of `init()`.
- Do not add an OpenTelemetry exporter for EvalKit. If the app has a tracer provider, `init()` adds its span processor to it.
- Do not call `init()` per request, or in code that also runs in the browser.
- Do not turn off secret masking (`capture_secrets`).
- Do not use an API that is not in this skill or in https://syntropylabs.ai/docs/llms.txt.