Docs

    Alerts and cost

    Notifications

    The bell in the top bar, the ten kinds of notification it collects, where each one links, and how read and unread work.

    The bell collects what happened while you were away. Each item links to the run, trace, queue or rule it is about.

    The bell

    The bell is in the top bar of every project page. Its badge counts your unread notifications, up to 99+, and refreshes every minute while the tab is visible, and at once when you come back to the tab. The list shows the newest first, with Unread only on by default; turn it off to see read items. Each row has the kind, a title, a short body and the time. Clicking a row marks it read and opens its link. Mark all read clears the badge and deletes nothing. Read state is per person, so marking yours read changes nothing for anyone else.

    The ten kinds

    KindWritten whenWho receives itLinks to
    Mentiona comment mentions youthe members it mentions, except the authorthe trace, span, session, run or dataset row the comment is on
    Alert firedan alert rule firesOwners, Admins and members whose email is on the ruleAlerts, History tab
    Monitor signala monitor finds an anomaly or regressionOwners, Admins and members whose email is on the monitorthe monitor, on the Alerts Monitors tab
    Budget pausedan online rule reaches its daily judge budgetOwners and AdminsEvaluations, Online rules tab, for that Environment
    Run completedan evaluation run you started completesthe person who started itthe run
    Simulation finisheda simulation run you launched endsthe person who launched itthe simulation run
    Review assignedreview queue items are assigned to youthe assigneesthe queue, in Annotate
    Access requestedsomeone asks to join the organizationOwners and Adminsthe request, in organization settings, Members
    Usage thresholdan Environment passes 80% or 100% of its monthly span limitOwners and AdminsAlerts, History tab
    Plan changeda platform admin changes the organization's planthe organization Ownerorganization settings, Plan & usage

    Storing a notification never holds up the event behind it: a run completes even if its notification could not be saved. A pass rate with no value reads "no verdict", not 0%. Four kinds also arrive outside the bell. Alert rules and monitors send email and Slack messages to their own recipients, described in Alerts. Usage notices are emailed to the organization Owner, and a mention emails each member it names. The other kinds appear only in the bell.

    Limitations

    • The badge polls, so a new item can take up to 30 seconds to appear.
    • The list covers every organization and project you belong to and has no scope filter, so opening an item can take you into another project.
    • You cannot mute a kind, get a digest or mark an item unread again. To find a row you read by accident, turn off Unread only.

    EvalKit is built by SyntropyLabs. Published on PyPI and npm.