Python SDK

Install, initialize, and instrument a Python app — auto-tracing, manual spans, and function-level APM.

Installation

pip install syntropylabs-evalkit

The distribution installs as syntropylabs-evalkit but the import name stays evalkit.

Quick start

import evalkit

client = evalkit.init(
    subscription_key="tk_live_...",   # from Settings → Tracing
    service_name="my-app",
    environment="production",
    debug=True,
)

# All OpenAI / Anthropic calls are now traced automatically
from openai import OpenAI
openai_client = OpenAI()
response = openai_client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(response.choices[0].message.content)

Manual spans

end, ctx = evalkit.start_span("my-operation", {"custom.key": "value"})
try:
    result = do_something()
    end("OK")
except Exception:
    end("ERROR")
    raise

tid = evalkit.current_trace_id()   # active trace id

Tracing your own functions (APM) — on by default

Function tracing is on by default. init() traces every function in your app’s own source tree (the directory of the file that called init()) as each module imports — one function_call span per call with input/output/latency. Third-party libraries are never wrapped, only module-level functions are (class methods are left alone unless you opt in — see below), and signatures are preserved so framework introspection (FastAPI Depends, etc.) keeps working.

# Nothing to wire — just init():
evalkit.init(subscription_key="tk_live_...", service_name="my-api")
# every function your app defines is now traced as it imports.

# Disable it:
#   EVALKIT_FUNCTION_TRACE=false            (env)
#   evalkit.init(..., function_tracing=False)
# Trace sibling packages outside the caller's dir:
#   evalkit.init(..., trace_packages=["support_bot", "workers"])

For finer control, opt in explicitly — a function, a tool, a whole class, or a module/package:

# One function -> function_call span (input / output / latency)
@evalkit.trace_function()
def do_work(x):
    return x * 2

# One tool -> tool_call span (renders in Input/Output panels + tool metrics)
@evalkit.trace_tool()
def search_web(query: str):
    return run_search(query)

# Every method of a class, APM-style
@evalkit.traced
class OrderService:
    def place(self, order): ...
    def cancel(self, id): ...

# Every function defined in a module / whole package
import myapp.services as svc
evalkit.trace_module(svc)
evalkit.trace_package(myapp)

Client-side tools you run yourself (your own functions the model calls) only show their output if you wrap them with trace_tool — the SDK sees the model’s request but never your function’s return value on its own. Server-side tools (e.g. OpenAI web_search) are captured automatically.

Web framework middleware

Add the middleware so each incoming request becomes one root trace and everything inside nests under it. See Web Frameworks for the full list.

# FastAPI / Starlette
app.add_middleware(evalkit.EvalKitMiddleware)      # Starlette: EvalKitStarletteMiddleware
# Flask
evalkit.instrument_flask(app)
# Django — in settings.py MIDDLEWARE add:
#   "evalkit.EvalKitDjangoMiddleware"
# Litestar
app = Litestar(route_handlers=[...], middleware=[evalkit.create_litestar_middleware()])

Offline evaluation

Score an output locally, deterministically, with no LLM cost — the result is pushed as an evaluation span next to your trace. See Offline Evaluation for the full option list.

scores = evalkit.evaluate(
    output=agent_reply,
    expected_tools=["search", "summarize"],
    tool_calls=[{"name": "search"}, {"name": "summarize"}],
    constraints={"required_terms": ["citation"], "has_citations": True},
)   # → {"tool_trajectory": 1.0, "tool_f1": 1.0, "response_match": 1.0, ...}

Configuration

Full list of init() options — see the Configuration reference.

evalkit.init(
    subscription_key="tk_live_...",
    base_url="https://api.syntropylabs.ai",  # default
    service_name="my-service",
    environment="production",               # production | staging | development
    debug=False,                            # log exports to stdout
    scheduled_delay_millis=5000,            # batch export delay (ms)
    max_body_bytes=10 * 1024 * 1024,        # max captured HTTP body size (default 10 MB)
    function_tracing=True,                  # auto-trace your app's functions (default True)
    trace_packages=None,                    # extra sibling packages to auto-trace
)