1The problem

    You ship agents.
    But how do you
    really know
    they're working?

    LLMs don't fail loudly. They just… drift. Hallucinate. Miss context. Break at the worst time. And traditional testing can't keep up.

    Any LLM, Any FrameworkPrivacy FirstNo Vendor Lock-in
    Developer watching an AI evaluation run fail: why did it hallucinate, did the context change, is it getting worse?
    2The solution

    Syntropylabs. Your agent evaluation copilot.
    From one line of code to full-stack evals.

    pip install syntropylabs-evalkit
    01

    One line, fully instrumented.

    stl.init() patches every LLM client, HTTP layer, and DB adapter your agent touches. No manual spans.

    agent.py
    stl.init(subscription_key="sk_live_...")
    02

    Every call becomes a trace.

    Tool calls, LLM calls, DB reads — stitched into one waterfall per run, not fourteen disconnected logs.

    llm.generate
    980ms
    03

    Every output gets scored.

    LLM-as-judge evaluators run automatically on every trace — online in production, or offline in batch.

    Relevancy0.94
    Groundedness0.88
    Coherence0.91
    04

    Regressions get caught before ship.

    Compare pass rates across deploys. Know a prompt change hurt quality before your users do.

    v12
    62%
    v13
    94%
    3The platform

    One connected pipeline, not five tools

    The SDK feeds the harness. The harness feeds the dashboard. Nothing to stitch together yourself.

    SDK

    One init() call instruments your entire agent stack — LLM clients, HTTP, SQL, logging. Python and TypeScript, zero deps.

    • Auto-patches OpenAI, Anthropic, Gemini, Bedrock
    • W3C traceparent propagation
    • Session + device context

    Agent Harness evaluation

    The evaluation core. Run batch jobs, score traces online, compare models pairwise — text, voice, image, and video.

    • Online + batch evaluation
    • Voice agent eval via LiveKit
    • Image / video quality scoring

    Dashboard

    Trace waterfall, session explorer, eval results, cost breakdown, and model catalog — everything in one place.

    • Trace waterfall per agent run
    • Per-session cost analytics
    • Regression comparison across deploys
    4Why it's different

    Why it's different

    Deep evaluations

    Grounded tool-use checks catch fabricated actions and hallucinated calls an LLM judge alone would miss — not just pass/fail.

    Built for real agents

    Full parent/child span tree with self-time aware latency profiling — built for multi-step, multi-turn, tool-calling agents.

    Any LLM, any framework

    OpenAI, Anthropic, Gemini, Bedrock, and more — auto-patched. Zero vendor lock-in, zero manual instrumentation.

    Private by design

    Your traces stay in your project. On-premise deployment and SSO/SAML available for teams that need it.

    5Capabilities

    The complete evaluation stack

    Trace every span, score every output, catch every regression — from prototype to production.

    Agent Tracing

    Every LLM call, tool invocation, HTTP hop, and DB query lands as a typed span — one waterfall per run.

    support_agent
    3.4s
    db.get_order
    45ms
    llm.plan
    0.8s
    stripe.refund
    412ms
    llm.reply
    1.2s
    llm_calltool_callhttp_calldb_query

    Zero-Config SDK

    One init() call. All providers patched automatically — plus a central model catalog to manage access across all of them.

    $ pip install syntropylabs-evalkit
    $ npm i syntropylabs-evalkit

    Online Evaluation

    Every production trace scored automatically as it arrives. No cron jobs.

    Scoring live traces…

    Voice Agent Eval

    Generate audio, score naturalness + instruction-following, replay rows. Connect live in-browser.

    naturalness 0.91instruction 0.8712 / 14 passed

    Batch Evaluation

    Run a dataset through your agent, score per row, regenerate failures individually.

    row_001
    94%
    row_002
    72%
    row_003
    38%
    row_004
    91%

    Image & Video Eval

    Generate, then LLM-as-judge score across visual quality dimensions.

    Visual quality88%
    Prompt alignment74%
    Coherence91%

    Evaluator Collections

    Group LLM-as-judge prompts into reusable collections. Apply to any project or set globally. Three rule types: model-graded, custom prompt, statistical.

    RelevancyGroundednessSafetyPII DetectionCoherenceCustom Prompt
    6Built to move fast
    < 5ms
    Ingest p99 latency
    8
    LLM providers supported
    < 5 min
    From init() to first trace
    Zero
    Manual span instrumentation
    7Built for developers

    One call.
    Everything Tracked.

    init() patches every LLM client, HTTP layer, and database adapter your agent touches. Traces start flowing in seconds — no manual spans, no config files.

    • pip install syntropylabs-evalkit / npm install syntropylabs-evalkit
    • Auto-instruments OpenAI, Anthropic, Gemini, Bedrock, HTTP, SQL, Redis, Mongoose
    • Online eval: attach evaluators to a project, every trace gets scored automatically
    • W3C traceparent — frontend and backend stitched into one trace, no extra work
    agent_setup.py
    1import syntropylabs as stl
    2
    3stl.init(
    4 subscription_key="sk_live_...",
    5 service_name="my-agent",
    6 environment="production",
    7 session_id=user_session_id,
    8 device_id=device_fingerprint,
    9)
    10
    11# OpenAI, Anthropic, HTTP, SQL — all patched automatically
    12from openai import OpenAI
    13client = OpenAI()
    14
    15response = client.chat.completions.create(
    16 model="gpt-4o",
    17 messages=[{"role": "user", "content": prompt}],
    18)
    19# ↑ captured: model, prompt, completion, tokens, latency
    8Start free, scale smart

    Simple, transparent pricing

    Start free. Scale up when you need it.

    Free

    $0

    For developers getting started with agent evaluation. No credit card required.

    • Up to 10,000 spans / month
    • 7-day trace retention
    • 2 LLM providers
    • Batch evaluation (100 rows)
    • Community support

    Enterprise

    Custom

    For organizations with custom security, scale, and compliance requirements.

    • Unlimited spans
    • 90-day trace retention
    • All LLM providers
    • Online + batch evaluation
    • On-premise deployment
    • SSO / SAML
    • SLA & dedicated support
    9Ship agents you trust

    Ship AI agents
    you can trust.

    Instrument in minutes. Score every output. Catch failures before your users do.

    Trace the live pipeline·Read the Blog