Errors and stack traces

From 0.3.6 both SDKs record an error one way wherever it happens, in a traced function, a model call, a request or a database query, by OpenTelemetry's exception conventions, and the receiver reads the same shape from either. Nothing of a frame's local variables or of a call's arguments is read, and the exception's text goes through the same secret masking as the rest of the span, credential parameters of a URL inside it included (?api_key=***). Failed spans are grouped on Errors, logs and services.

What a failed span records

• Python records the exception's own stack trace. On 0.3.5 and earlier it read the exception being handled, which inside an except block can be another one.
• A Python trace_function or trace_tool call that ends with a cancelled task, KeyboardInterrupt or SystemExit ends its span with status ERROR and re-raises; a cancelled task also carries evalkit.cancelled: true. On 0.3.5 and earlier such a call records no span.
• TypeScript holds exception events from other OpenTelemetry libraries to the same caps.
• On 0.3.5 and earlier, the status message holds the message alone, and there are no evalkit.exception.* attributes.

Where errors are recorded

• Traced functions and tools: the exception that escaped, re-raised or rethrown unchanged.
• Model calls: the exception the provider raised, a stream that failed part way included.
• Requests: from 0.3.6 the framework integrations record the exception that failed the request, with its frames and causes, not only the status code. In Python: FastAPI, Flask and Django. In TypeScript: the Fastify plugin, the Nest interceptor, and the Koa, Hono and hapi adapters. Express hands errors to error middleware, so register expressErrorHandler(), below. From 0.3.7 the exception is recorded only when the request is answered with a server error; see Which HTTP answers are errors.
• Python log records: a record with exception info, as from logger.exception(...), carries the exception with exception.escaped false. A record at ERROR or above has status ERROR; a warning logged with an exception keeps status OK.
• Python manual spans: end("ERROR", exception=exc) on the end that start_span returns.

import express from "express";
import evalkit from "syntropylabs-evalkit";

const app = express();
// ... your routes
app.use(evalkit.expressErrorHandler()); // after the routes: records the error on the request's span, then calls next(err)
app.use((err, req, res, next) => res.status(500).json({ error: "internal" })); // your own error handler

Which HTTP answers are errors

By default a request's server span is ERROR for every 4xx and 5xx answer, with the exception the request failed with, if any. From 0.3.7 you choose which answers count with server_error_status_codes (TypeScript serverErrorStatusCodes). ["500-599"] follows OpenTelemetry's HTTP conventions, where a 4xx is the caller's mistake, such as a scanner probing /.env, an expired token or a body that fails validation: its span keeps http.status_code, stays OK and records no exception, so bots' 404s stay off the Errors page. It suits a public API.

• Every server integration follows the same rule: in Python FastAPI, Starlette, Flask, Django and Litestar; in TypeScript Node's http server and the Express, Fastify, Koa, Hono, hapi and NestJS adapters.
• The setting never applies to outgoing calls: a 4xx an outgoing call gets back is always an error, since the call failed for your app.
• A traced function that raises an exception a framework answers a request with is judged by the status it carries and records it as http.response.status_code; with a status outside the list it ends OK with no exception, and the exception still reaches the caller. In Python: FastAPI and Starlette HTTPException, Werkzeug's and Litestar's HTTPException, and Django's Http404, PermissionDenied, SuspiciousOperation and BadRequest. In TypeScript: a NestJS HttpException (methods traced by enableNestjsAutoTrace() included), a Hono HTTPException, a hapi Boom error and an http-errors error.
• The setting takes codes and inclusive ranges: ["500-599"] leaves every 4xx to the caller, and [429, "500-599"] also counts the 429s your app answers. The default is ["400-599"].

Errors you handle yourself

An error your code catches is not recorded on its own. In TypeScript, captureException(error, { escaped, fingerprint }) records it on the active span and marks the span as errored; with no span active it becomes a span of its own. escaped is false by default. It never throws, and does nothing before init(). In Python, log it with logger.exception(...) inside the except block.

import logging

log = logging.getLogger(__name__)

try:
    charge_card(order)
except PaymentError:
    log.exception("charge failed")   # a log span at ERROR, with the exception and exception.escaped false
    queue_retry(order)

Your code and library code

Each frame says whether it is your own code, as in_app, and evalkit.exception.source points at the innermost frame that is. Python counts a frame as yours unless it is in the standard library, in site-packages, dist-packages or another install directory, frozen, or in EvalKit. TypeScript counts it as yours unless it is Node's own (node:, internal/) or a dependency's (node_modules).

evalkit.init(
    subscription_key=os.environ["EVALKIT_SUBSCRIPTION_KEY"],
    in_app_include=["acme"],        # your own packages, even installed in site-packages
    in_app_exclude=["generated"],   # code in your tree that is not yours to fix
)

• Python takes module prefixes: "acme" matches acme and acme.billing. An exclusion wins over an inclusion, so in_app_exclude=["acme.vendored"] leaves a vendored package inside acme out.
• TypeScript takes substrings or RegExps of the file path, and an exclusion wins over an inclusion.

Fingerprints

The receiver groups failed spans by itself. To choose the group, set a fingerprint: the span records it as evalkit.exception.fingerprint, and errors with the same fingerprint are grouped together.

token = evalkit.set_error_fingerprint("checkout-timeout")   # a string, or a list joined with "|"
try:
    checkout(cart)
finally:
    evalkit.clear_error_fingerprint(token)

@evalkit.trace_function(error_fingerprint="payments-declined")   # trace_tool takes it too
def charge(card): ...

• Python's set_error_fingerprint is scoped like set_user, per thread and per asyncio task, for the errors recorded from then on; None clears it. clear_error_fingerprint(token) restores the one before.
• TypeScript's setErrorFingerprint(error, fingerprint) marks the error object and returns it, so it fits in a throw.

Unhandled exceptions

capture_unhandled is new in 0.3.6. On 0.3.5 and earlier, captureGlobalErrors: true can change how the process ends.

Stack depth and source maps in TypeScript

• Frames are read from error.stack; Error.prepareStackTrace is never replaced.
• V8 collects 10 frames per error. stackTraceLimit, unset by default, raises Error.stackTraceLimit for deeper stacks. The limit is process-wide, so every error your app creates then collects more frames; it is raised only from V8's default, and a limit you set yourself is left alone.
• sourceMaps: true turns on Node's source map support with process.setSourceMapsEnabled(true), so frames of compiled or bundled code point at your TypeScript sources. Node's --enable-source-maps flag does the same.

Every option above is also listed with its default on Configuration and privacy.