Flask

A Flask app is traced by one call, evalkit.instrument_flask(app). It registers request hooks on the app, and each request becomes one trace: a root span for the request, with every model call, tool call, HTTP call and database query it makes nested underneath.

What you get

• A root http_call span of kind SERVER per request, named by the method and the URL rule, such as POST /chat or GET /orders/<int:order_id>. A request that matched no rule keeps its raw path.
• On the root span: http.method, http.url, http.route, http.status_code, and the request and response headers and bodies, credentials masked. A status of 400 or above marks the span as an error.
• Every response carries X-Trace-Id and a traceparent header. An incoming traceparent is honoured.
• The model calls and tools nest under the root span. A streamed response, with stream_with_context or without it, keeps its model call in the request's trace with the session open in the generator.
• Lines written by the server's logger, such as werkzeug's request line, arrive as log spans: the request line in the request's trace, start-up lines in a trace of their own.

The tutorial below gives this trace, plus the werkzeug log line:

POST /chat                   http_call   the request, named by its route
  chat.completions.create    llm_call    the model asks for lookup_order
    POST                     http_call   the OpenAI client's own request
  lookup_order               tool_call   the tool's arguments and result
  chat.completions.create    llm_call    the model's answer
    POST                     http_call

Install

pip install syntropylabs-evalkit flask openai

Set up

import os

import evalkit

evalkit.init(subscription_key=os.environ["EVALKIT_SUBSCRIPTION_KEY"], service_name="my-api")

from flask import Flask  # noqa: E402

app = Flask(__name__)
evalkit.instrument_flask(app)  # each request becomes one trace

Where init() goes

• Call init() once per process, before the framework and the model clients are imported. It patches the OpenAI, Anthropic and other client classes, the HTTP clients and the database drivers it finds installed, so a client created afterwards is traced without any code of yours. A second call returns the first client and ignores its arguments.
• Create clients after init(). A function imported by name before it, such as from litellm import completion, keeps the unpatched function.
• Automatic function tracing covers the public module-level functions of the modules under the directory of the file that calls init(). In that file itself, only the functions defined above the call are wrapped, so in the single-file tutorials below no function of yours gets a span of its own. Add packages that live elsewhere with trace_packages, as in evalkit.init(..., trace_packages=["support"]).
• Put init() at the top of the module that creates the app. With an app factory, call init() at module level above create_app(), not inside it, so it runs once per process however many apps the factory builds.
• Call instrument_flask(app) right after creating the app, before you register other before_request hooks. Flask runs those hooks in the order they were registered, so hooks registered after it run inside the request span.

Tutorial: a support agent with one tool

1. 1

   Install

   Install the SDK, Flask and the OpenAI client, as in Install.

2. 2

   Write the app

   Save app.py below. It answers POST /chat with an agent that has one tool, lookup_order: the model asks for the tool, the app runs it, and the model answers with its result.

3. 3

   Set the keys

   Export your environment key and your OpenAI key.

4. 4

   Run it and send one request

   Start the development server, then send the request below. The spans follow within 5 seconds.

5. 5

   Open the trace

   Find POST /chat in Traces, as in See the trace.

import json
import os

import evalkit

# init() runs before Flask and OpenAI are imported, so every client created below is already patched.
evalkit.init(
    subscription_key=os.environ["EVALKIT_SUBSCRIPTION_KEY"],
    service_name="flask-support",
    environment="development",
)

from flask import Flask, request  # noqa: E402
from openai import OpenAI  # noqa: E402

app = Flask(__name__)
evalkit.instrument_flask(app)  # each request becomes one trace with its root span
client = OpenAI()  # reads OPENAI_API_KEY from the environment

SYSTEM = "You are a support agent for an online shop. Use lookup_order for questions about an order."
TOOLS = [{
    "type": "function",
    "function": {
        "name": "lookup_order",
        "description": "Find an order from the customer's words.",
        "parameters": {"type": "object", "properties": {"query": {"type": "string"}}, "required": ["query"]},
    },
}]


@evalkit.trace_tool(name="lookup_order")  # a tool_call span with the arguments and the result
def lookup_order(query: str) -> dict:
    return {"order_id": "1001", "status": "shipped", "arrives": "Thursday"}


def answer(message: str) -> str:
    messages = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": message}]
    reply = client.chat.completions.create(model="gpt-4o-mini", messages=messages, tools=TOOLS).choices[0].message
    if reply.tool_calls:
        messages.append(reply.model_dump(exclude_none=True))
        for call in reply.tool_calls:
            result = lookup_order(**json.loads(call.function.arguments))
            messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})
        reply = client.chat.completions.create(model="gpt-4o-mini", messages=messages, tools=TOOLS).choices[0].message
    return reply.content


@app.post("/chat")
def chat():
    body = request.get_json()
    evalkit.set_agent("support-bot")  # the trace lists under this agent in Traces > Agents
    with evalkit.session(body["session_id"], user_id=body.get("user_id")):  # groups the turns of one conversation
        return {"reply": answer(body["message"])}

export EVALKIT_SUBSCRIPTION_KEY=tk_live_...   # Settings → This project → Environments → Copy key
export OPENAI_API_KEY=sk-...

flask --app app run   # serves on port 5000

curl -s -X POST localhost:5000/chat \
  -H 'content-type: application/json' \
  -d '{"message": "Where is my order 1001?", "session_id": "conv_1", "user_id": "u_42"}'

{"reply":"Order 1001 shipped yesterday and arrives on Thursday."}

Tested with syntropylabs-evalkit 0.3.5, Python 3.13, Flask 3.1.3 and openai 3.31, on the development server and on gunicorn 26.2 with --preload -w 2.

See the trace

• Open Traces. The newest row is POST /chat from the service flask-support, with the tokens of both model calls.
• Click it. The waterfall shows the request span with the two llm_call spans, each with its POST, the lookup_order tool span, and the werkzeug request line as a log.
• To open one exact request, copy the X-Trace-Id response header, paste it into Search traces and click Open trace.
• Traces → Sessions lists conv_1 with user u_42, and Traces → Agents lists support-bot. The root span carries the agent but no session id, because the request span opened before the view named the session. The trace still belongs to conv_1: a trace belongs to every session that any of its spans carries.

Sessions, users and agents

instrument_flask does not fill a user id on its own. Open evalkit.session(...) in the view, as the tutorial does, or set the ids once per request in hooks. evalkit.set_user returns a token; reset it in teardown_request, because the ids stay in force on that thread after the request ends:

from flask import g, request


@app.before_request
def evalkit_identity():
    # Registered after instrument_flask, so it runs inside the request's trace.
    g.evalkit_identity = evalkit.set_user(
        user_id=request.headers.get("X-User-Id"),
        session_id=request.headers.get("X-Session-Id"),
    )


@app.teardown_request
def evalkit_identity_reset(exc):
    token = g.pop("evalkit_identity", None)
    if token is not None:
        evalkit.clear_user(token)  # the next request on this thread starts with no identity

With these hooks the root span and every span under it carry the session and user, and the next request served by the same worker starts clean. evalkit.set_agent(name) in a hook or a view names the agent the same way.

Pitfalls

• No spans from gunicorn with --preload, on 0.3.4 and earlier. With --preload, gunicorn runs init() in the master process and then forks the workers, and there a forked worker never sends its spans, because the export thread does not survive the fork. Run gunicorn without --preload there. From 0.3.5 a worker forked after init() starts an export thread of its own: in a run of Flask under gunicorn --preload -w 2, the workers' spans arrived while they ran and at shutdown. The same holds for multiprocessing with fork; see Workers and shutdown for Celery.
• A user id outlives its request. evalkit.set_user(user_id=...) in a view, with no reset, stays in force for the next request served by the same thread: under gunicorn's sync worker, a request without a user id was recorded with the previous request's user. Use with evalkit.session(...), or the hooks above with clear_user(token).
• Log lines become spans; your log output stays as it was. Each record that propagates to the root logger is recorded as a log span, as a handler there would record it: at the level of the logger that wrote it, which is the root level, WARNING, unless that logger or your app sets another. init() changes no handler, formatter or level, so your console output stays as it was. Every record carries trace_id and span_id, so a format of your own can show them, as in %(asctime)s %(levelname)s [%(trace_id)s] %(name)s: %(message)s. On 0.3.4 and earlier, init() changes the format of the root logger's console handlers and sets the root level to INFO; configure logging after init() there.
• The debug reloader runs init() twice. flask run --debug starts a watcher process and a serving process, and both import the app, so each calls init(). Requests are recorded by the process that serves them.
• The app does not start without the key. os.environ["EVALKIT_SUBSCRIPTION_KEY"] raises KeyError when the variable is unset, which is the point in a deployment. Where the app must also run without tracing, such as in tests, call init() only when the variable is set, and guard the code that needs it: @evalkit.trace_tool() and the other decorators, and instrument_flask, raise RuntimeError when init() never ran (read in the SDK source).
• A short script, a CLI command or a serverless handler loses its last spans. Spans go out every 5 seconds and when the process exits normally. A serverless runtime freezes the process between invocations instead of ending it, so call evalkit.flush() at the end of each invocation, and in a script before it exits early.

Safe by design

• Spans are queued in memory and sent from a background thread, so a request never waits for the trace service.
• If the trace service is down or unreachable, your app answers as before. The SDK retries each export 3 times, logs evalkit: export failed after 3 retries as a warning and drops those spans.
• If the key is wrong, the trace service refuses the spans. After 3 refusals in a row the SDK logs one warning and stops exporting until the process restarts; your app keeps answering.
• A streamed model reply reaches your code chunk by chunk, unchanged. The SDK reads the same chunks on the side and writes the span when the stream ends.
• Tracing never breaks the call it records. An error inside the SDK is logged at debug level and the call runs as if untraced, and an exception from the provider reaches your code unchanged.
• A failed export never raises in your code, and evalkit.flush() returns normally too. At exit the SDK sends what is queued within 3 seconds in all, whether or not the trace service answers, so an outage never holds up a deploy.

Verify it worked

• Each response has an X-Trace-Id header. No header means instrument_flask(app) did not run on the app that served the request.
• The trace appears in Traces within about 5 seconds. If it does not, look for evalkit: warnings in the app's log.
• Pass debug=True to init() and set the evalkit logger to DEBUG to log each export.

Next steps

• Choose what is recorded, from prompts to function arguments, and mask what must not leave the process: Configuration and privacy.
• Every library the SDK traces without code, and how streamed calls are counted: What is traced automatically.
• Follow one request across services, or run beside an OpenTelemetry setup you already have: OpenTelemetry and distributed tracing.
• Read the trace you just sent: Your first trace, and group turns into conversations: Sessions and users.