> ## Documentation Index
> Fetch the complete documentation index at: https://docs.useapgar.dev/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Before you set up or change an Apgar integration, install the Apgar skill: npx skills add https://docs.useapgar.dev
> Search these docs with the MCP server at https://docs.useapgar.dev/mcp. Setup for each agent: https://docs.useapgar.dev/ai-agents.md

# How it works

> From traces to turns, releases, verdicts, and messages.

Apgar turns the traces of an agent into turns, groups turns by release, and compares each new release with a baseline release until it can say whether the new release is worse.

![Traces arrive over OTLP/HTTP at POST /v1/traces or are pulled from Langfuse by the worker. Apgar normalizes them into spans, assembles spans into turns and tool calls, discovers releases, runs evaluations, reaches verdicts, and sends messages to Slack and webhooks.](https://mintcdn.com/apgar/CQ6o-L_OizzzJ47E/images/how-it-works-light.svg?fit=max\&auto=format\&n=CQ6o-L_OizzzJ47E\&q=85\&s=9e348629eab4882bad2d983ed5eede13)

## Ingest

Traces arrive in one of two ways:

* An OpenTelemetry SDK or collector pushes OTLP/HTTP to `/v1/traces` with an ingest token. See [OpenTelemetry](https://docs.useapgar.dev/send-traces/opentelemetry.md).
* The worker pulls observations from a Langfuse project. See [Langfuse](https://docs.useapgar.dev/send-traces/langfuse.md).

Both sources go through the same normalizer. It classifies each span (workflow, agent, chat, tool, retrieval, or other), keeps a small set of attributes, and drops message content after reducing it to a character count and a JSON validity flag. Tool arguments are replaced by a keyed hash. See [Data handling](https://docs.useapgar.dev/privacy/data-handling.md).

## Turns

A turn is one trace: one request to the agent and everything it did to answer. The worker assembles a trace into a turn 60 seconds after its last span once its span without a parent has arrived, or 10 minutes after its first span arrived. Spans that arrive later update the turn, but its release, start time, and agent stay as first assembled. A turn records its release, environment, session, surface, prompt version, models, LLM calls and errors, tool calls, errors, retries, loops, tokens, cost, output length, and latency.

A session is the conversation a turn belongs to. Statistics are computed over sessions, so a few long conversations cannot outweigh many short ones. See [Sessions](https://docs.useapgar.dev/concepts/sessions.md).

## Releases

When a turn carries a release identifier that the agent has not seen before, Apgar registers a new release, records when it was first seen, and sends a **New release** message. See [Releases and baselines](https://docs.useapgar.dev/concepts/releases.md).

## Evaluations

The worker evaluates a release every 5 minutes for the first 2 hours and every 15 minutes after that. Each evaluation:

1. Picks the baseline: by default the most recent earlier release that did not fail, over its own traffic in the 7 days before the new release appeared.
2. Computes each [metric](https://docs.useapgar.dev/concepts/metrics.md) on both sides, with an interval from a bootstrap over sessions.
3. Fails a metric when the whole interval lies past its tolerance on the bad side, and passes it when the interval lies inside the tolerance.
4. Breaks every metric down by tool, surface, session depth, response model, and prompt version, to say where a failing change is concentrated.

A result becomes a verdict only after it repeats on two evaluations at least 10 minutes apart. See [Verdicts](https://docs.useapgar.dev/concepts/verdicts.md).

## Messages

Apgar sends a message when a release is detected, when a regression is confirmed, when a release is confirmed healthy, and when monitoring ends without a decision because the monitoring window ran out or someone stopped it. Messages go to Slack, to signed webhooks, or both. See [Slack](https://docs.useapgar.dev/notifications/slack.md) and [Webhooks](https://docs.useapgar.dev/notifications/webhooks.md).

## End of monitoring

Monitoring of a release ends when a newer release of the same agent records its first turn, when 72 hours have passed since the release was first seen, or when someone stops it. A last check then decides the release. After that, only recording expected changes can change the result.

## Labels

The UI labels a release by its state: **First release**, **Collecting**, **Possible regression**, **Looks healthy**, **Not enough data**, **Regression**, **Healthy**, and **Unconfirmed**. Messages use only the four titles that matter in a channel: **New release**, **Regression**, **Healthy**, and **Unconfirmed**. The state table in [Verdicts](https://docs.useapgar.dev/concepts/verdicts.md#states) maps every label to its engine state.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.