Backend tracing · LLM evaluation · agent simulation

    Know what your agents did.

    Syntropylabs traces your backend end to end — your own functions, HTTP requests, database queries and every model or tool call your agents make — then evaluates the results with judges you control and simulates users before you ship.

    $ pip install syntropylabs-evalkit

    claude-code.turn

    Claude Code
    running
    —

    Spans

    —

    Tokens

    —

    ≈Cost

    $ —

    Sample run · a recorded Claude Code turn, replayed

    Instrument, observe, evaluate.

    Three steps, in the order a team actually does them. The same screens the product shows after sign-in.

    1. Step 1: Instrument

      Install the SDK and call init() once with an environment key. Your functions, HTTP requests, database queries and model calls are traced from there; coding agents connect with one CLI command.

      Install
      pip install syntropylabs-evalkit
npm install syntropylabs-evalkit

    2. Step 2: Observe

      Each request becomes one trace: the spans it ran, the models it called, what it cost and where it failed. Sessions group turns; flags mark tool loops and truncation.

      support_agent.turn

      ok1.84 s5 spans3,412 tokens$0.0006session sess_4b1e

      fig. 1 · trace header · sample data

    3. Step 3: Evaluate

      Run a dataset through evaluators, or score live traces with an online rule. Every run reports a pass rate, per-evaluator means and a gate you can block a deploy on.

      Pass rate

      87.5%

      Gate

      gate passed

      pass_rate ≥ 0.85 ✓

      fig. 2 · run summary · sample data

    Traces

    Every request your app made: the spans it ran, what it cost, where it failed, and the conversation behind it.

    Operation Service StatusModel LatencyTokens≈CostScore
    support_agent.turn8f3a1c0d94e2session sess_4b1esupport-apiokgpt-4o-mini1.84 s3,412$0.000692%auto
    refund_agent.turnc21d7e5a30b8session sess_9a02tool looprefund-workerokclaude-sonnet-44.31 s7,905$0.038968%auto
    claude-code.turn5be04f7d1a96session sess_f77cClaude Codeunsetclaude-sonnet-448.20 s61,208$0.19—
    rag.answere9a2b6c4d015errordocs-boterrorgpt-4o6.02 s9,880$0.031241%auto
    voice.turn17c8d3f2a4e0session sess_20d1ivr-agentokgpt-4o-realtime0.92 s1,104$0.0071—

    fig. 3 · traces list · sample data

    Evaluations

    Run your dataset through evaluators and compare runs against a baseline.

    support-agent · nightly

    completedofficial
    source uisnapshot 9f3c1a

    Pass rate

    87.5%

    Average score

    84%

    Rows

    240

    Judge cost

    $1.42

    318,402 in · 41,911 out tokens

    Per evaluator

    gate passedpass_rate ≥ 0.85 ✓ · groundedness ≥ 0.9 ✓

    Groundedness

    91%

    Answer relevance

    88%

    Tool-use correctness

    79%

    PII

    100%

    Online rulegroundednessjudge gpt-4o-minisample 20%budget $5 / dayactive

    fig. 4 · evaluation run · sample data

    Datasets

    Upload, import and curate the rows your evaluations run on.

    support-tickets

    single-turn240 rows
    snapshot 9f3c1apinned by 3 runs
    #InputExpected outputStatus Source
    1Refund order 48213, delivered 3 weeks agoRefund approved within 30-day windowgoldentrace 8f3a1c
    2Cancel my subscription but keep my dataCancel at period end, retain data 90 daysfinalizedcsv import
    3Why was I charged twice this month?Identify duplicate charge, offer reversaldraftsynthesized

    fig. 5 · dataset rows · sample data

    Simulations

    Drive your agent through generated scenarios in a sandbox and score how it did.

    Scenario Status TurnsTool callsTask completion Policy adherenceVerdict
    Refund after 30 days, customer insistsdone63100%100%passed
    Address change mid-orderdone42100%75%failed
    Ask for another customer’s invoicedone31100%100%passed
    Angry customer, three topics at oncetimeout129no verdictno verdictno verdict

    fig. 6 · simulation run · sample data

    8lines

    Eight lines in, one trace out. The snippet below produces the trace beside it, span for span. Hover, focus or tap a line to see the span it made.

    agent.pyHover, focus or tap a line to see its span
    1. 1import evalkit
    2. 2evalkit.init(subscription_key="tk_live_…", environment="production")

    support_agent.turn

    8f3a1c0d94e2ok1.8 s5 spans3,412 tokens$0.0006
    functionsupport_agent.turnfunction_call
    1.8 s
    llmllm.callgpt-4o-mini
    620 ms
    dbdb.querySELECT … FROM orders WHERE id = $1
    12 ms
    httphttp.requestPOST api.stripe.com/v1/refunds
    412 ms
    llmllm.callgpt-4o-mini
    540 ms

    fig. 7 · the trace this snippet produces · sample data

    • init() once with an environment key. Your own functions, HTTP and database clients and every model call are traced as they run: application tracing for the backend, not only for the agent.
    • OpenAI, Anthropic, Bedrock, Cohere, Google and Vertex clients, LangChain, and the HTTP and database libraries around them, in Python and TypeScript.
    • Coding agents connect with evalkit coding install: Claude Code, Codex and Gemini CLI export OpenTelemetry natively, the rest report through hooks.
    Install
    pip install syntropylabs-evalkit
    Coding agents
    pip install syntropylabs-evalkit
evalkit coding install --vendor claude-code --key tk_live_…

    $0

    Tracing is free for one project, with no card on file. Evaluations, datasets and simulations are on paid plans; the full spec sheet is on the pricing page.

    Plan summary
    PlanFreeEnterprise
    Price$0Custom
    Tracing and the trace waterfall
    Projects1Unlimited
    Trace retention7 days90 days
    Spans per month10,000 soft capNo cap
    Evaluations, datasets, simulations, playground, alerts—
    SupportCommunityDedicated, with SLA