Curate and maintain a golden dataset | Syntropylabs

SyntropyLabs/Docsv0.2.16

Get started

Introduction
Concepts
Create an environment and get its key
Install the SDK
Your first trace
Your first evaluation

Traces

Browse traces and the query bar
The trace page
Sessions and users
Errors, logs and services
Topics, agent versions and release comparison
Live tail
Saved views
Export traces

Coding agents

Trace a coding agent
What each vendor exports
Reading a turn and its subagents
Content capture, status and doctor
Troubleshooting

Evaluations

Create a run from a datasetRequires a paid plan
Read a runRequires a paid plan
Compare against a baseline and set a gateRequires a paid plan
Runs from codeRequires a paid plan
Online rulesRequires a paid plan
Calibration and disagreementsRequires a paid plan

Datasets

Upload a CSV and map columns
Rows, editing and conflicts
Build a dataset from traces
Generate outputs
Snapshots and versions
Golden datasets: import, export and evaluate
Curate and maintain a golden datasetRequires a paid plan
Synthesize rowsRequires a paid plan
Voice datasetsRequires a paid plan

Evaluators

The managed libraryRequires a paid plan
Custom LLM-judge evaluatorsRequires a paid plan
Code evaluators and assertionsRequires a paid plan
CollectionsRequires a paid plan
Edit, duplicate and deleteRequires a paid plan
Calibration detailRequires a paid plan

Simulations

AgentsRequires a paid plan
ScenariosRequires a paid plan
Launch and read a runRequires a paid plan
Score and compare runsRequires a paid plan
Red-team runsRequires a paid plan

Playground

ChatRequires a paid plan
Pointwise and pairwiseRequires a paid plan
Grid over a dataset and model compareRequires a paid plan

Prompts

PromptsRequires a paid plan

Annotate

Score types and review queuesRequires a paid plan
Work a queue and label from a traceRequires a paid plan
Export labels and build RL datasetsRequires a paid plan

Alerts and cost

AlertsRequires a paid plan
MonitorsRequires a paid plan
Notifications
Cost

Settings and organization

Project settings
Environments and keys
Models and providers
Members and roles
Plan, usage and locks

SDK reference

Python SDK
TypeScript SDK
Configuration and privacy
What is traced automatically
Web frameworks
OpenTelemetry and distributed tracing
Skill file for AI coding assistants

API reference

API overview
Trace read API
Environments, projects and organizations
Datasets
Evaluation runs
Evaluators, collections and calibration
Online rules and trace evaluation
Review queues, scores and RL datasets
Simulations and scenarios
Alerts, notifications and cost
Prompts

Help

FAQ

PreviousGolden datasets: import, export and evaluate NextSynthesize rowsRequires a paid plan

EvalKit is built by Syntropylabs. Published on PyPI and npm.

SyntropyLabs

© 2026 SyntropyLabs Inc.

Set in Bricolage Grotesque, Inter and JetBrains Mono.

Every figure on this page is sample data unless a caption says otherwise.

Product

Docs
Integrations
Compare
Pricing
Blog
Sign in

Company

About
Careers
Contact

Legal

Privacy
Terms