TypeScript SDK

Install, initialize, and instrument a Node.js app — auto-tracing, manual spans, and function-level APM.

Installation

npm install syntropylabs-evalkit
# or
yarn add syntropylabs-evalkit

Quick start

import * as evalkit from 'syntropylabs-evalkit';

const client = evalkit.init({
  subscriptionKey: 'tk_live_...',   // from Settings → Tracing
  serviceName: 'my-app',
  environment: 'production',
  debug: true,
});

// OpenAI is auto-patched — just use it normally
import OpenAI from 'openai';
const openai = new OpenAI();

const res = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'Hello!' }],
});
console.log(res.choices[0].message.content);

Manual spans

import { startSpan } from 'syntropylabs-evalkit';

const { end } = startSpan('my-operation', { 'custom.key': 'value' });
try {
  const data = await fetchData();
  end('OK', { 'result.count': data.length });
} catch (e) {
  end('ERROR', { 'error.message': String(e) });
  throw e;
}

Tracing your own functions & tools (APM)

Auto-instrumentation covers libraries (LLM / HTTP / DB). For your code, opt in — a function, a tool, a class, or a whole service object:

import * as evalkit from 'syntropylabs-evalkit';
import { Traced } from 'syntropylabs-evalkit';

// One function -> function_call span (input / output / latency)
const doWork = evalkit.traceFunction('doWork', (x: number) => x * 2);

// One tool -> tool_call span (Input/Output panels + tool metrics)
const searchWeb = evalkit.traceTool('search_web', (q: string) => runSearch(q));

// Every method of a class, APM-style
@Traced()
class OrderService {
  place(order: Order) { /* ... */ }
  cancel(id: string) { /* ... */ }
}

// Every function of a service object (parity with Python's trace_module)
export const orders = evalkit.traceObject({ place, cancel }, { prefix: 'orders' });

// NestJS — trace EVERY provider/controller method via the DI registry.
// One line in main.ts after create — pass the app, the SDK resolves
// DiscoveryService itself (no @nestjs/core import needed):
await evalkit.enableNestjsAutoTrace(app);

Client-side tools you run yourself only show their output if you wrap them with traceTool — the SDK sees the model’s request but never your function’s return value on its own. Server-side tools (e.g. OpenAI web_search) are captured automatically. Auto-discovery of all functions is only possible where the framework has a registry (NestJS DI) — for Express / Fastify / Koa / Hono / Hapi, use traceObject / traceFunction on your modules.

NestJS / Express middleware

The SDK auto-instruments all incoming HTTP requests — no manual middleware needed for the trace itself, but the middleware below turns each request into a proper root span per framework. Call evalkit.init() before your app bootstraps.

// main.ts (NestJS)
import * as evalkit from 'syntropylabs-evalkit';

evalkit.init({
  subscriptionKey: process.env.EVALKIT_SUBSCRIPTION_KEY!,
  serviceName: 'my-nestjs-app',
  environment: process.env.NODE_ENV ?? 'development',
});

const app = await NestFactory.create(AppModule);
await evalkit.enableNestjsAutoTrace(app);   // APM: trace every provider/controller
await app.listen(3000);

// Other frameworks — see Integrations → Web Frameworks:
// app.use(evalkit.expressMiddleware());
// await app.register(evalkit.fastifyPlugin());

Offline evaluation

const { scores } = evalkit.evaluate({
  output: agentReply,
  expectedTools: ["search", "summarize"],
  toolCalls: [{ name: "search" }, { name: "summarize" }],
  constraints: { requiredTerms: ["citation"], hasCitations: true },
});

Configuration

Full list of init() options — see the Configuration reference.

evalkit.init({
  subscriptionKey: 'tk_live_...',
  baseUrl: 'https://api.syntropylabs.ai',  // default
  serviceName: 'my-service',
  environment: 'production',
  debug: false,
  scheduledDelayMillis: 5000,              // batch export delay (ms)
  maxBodyBytes: 10 * 1024 * 1024,          // max captured HTTP body size (default 10 MB)
});