> ## Documentation Index > Fetch the complete documentation index at: https://docs.useapgar.dev/llms.txt > Use this file to discover all available pages before exploring further. > ## Agent Instructions > Before you set up or change an Apgar integration, install the Apgar skill: npx skills add https://docs.useapgar.dev > Search these docs with the MCP server at https://docs.useapgar.dev/mcp. Setup for each agent: https://docs.useapgar.dev/ai-agents.md # Troubleshooting > What to check when turns, releases, verdicts, or messages do not show up. Each section starts with what you see and lists the causes in the order they are worth checking. The agent page, **Settings, Notifications**, and the [Prometheus metrics](https://docs.useapgar.dev/self-hosting/monitoring.md) are the places to look. ## No turns on the agent page * **The exporter does not reach Apgar.** Check the exporter logs for the status of `POST /v1/traces`, and `apgar_ingest_requests_total` by status. `401` is a missing, mistyped, or revoked token. `415` means the exporter sends gRPC or an unexpected content type; set `OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf`. * **Spans arrived, but no turn is assembled yet.** A trace becomes a turn 60 seconds after its last span once its span without a parent has arrived, and otherwise 10 minutes after its first span arrived. When the parent of your top span belongs to a service that does not send to Apgar, every turn waits the 10 minutes. `apgar_assembly_lag_seconds` shows how far assembly is behind. * **The trace belongs to another agent, or to none.** A token bound to an agent sends every trace to that agent, whatever the trace names. `apgar_turns_dropped_total` with reason `agent_not_found` means the bound agent was deleted, or an unbound token with **Create agents automatically** off received a trace that names an unknown agent; with reason `no_agent`, the trace carries no agent name and the token is unbound. See [Agent name](https://docs.useapgar.dev/send-traces/attributes.md#agent-name). * **Spans were dropped at ingest.** `apgar_ingest_spans_dropped_total` with reason `invalid_ids` or `invalid_timestamps` means the SDK sent spans without a valid trace id and span id (all-zero ids are invalid), with a malformed parent span id, or without a start time. * **The spans are older than 7 days** (`APGAR_LATE_SPAN_DAYS`). They are stored as late, never create releases, and are never evaluated. A backfill of more recent traces creates releases and sends messages like live traffic. * **Langfuse: the sync has not run.** The worker syncs each connection every minute. Only the first successful cycle of a connection reads the last 24 hours; an agent added to a connection that has already synced sees traces from about 30 minutes before it was added on. The agent page shows the sync status, and `apgar_langfuse_requests_total` by status shows whether Langfuse answers. A `404` on every request means a self hosted v3 server. ## Turns arrive, but no release appears The agent page says "No releases yet" where the release list would be. Without any turn in the last 24 hours it would show the **Getting started** steps instead. * **The environment does not match.** Only turns whose environment equals the agent's environment create releases and count in comparisons. The environment comes from `apgar.environment`, `deployment.environment.name`, `deployment.environment`, or `langfuse.environment`, and is `default` when none is set. **Add agent** fills in `production`. Compare the agent environment with the environment on the spans, or clear it to accept every environment. The agent page shows no warning for this. * **The turns are untagged.** See the next section. ## Turns are untagged * The release attribute is missing on the spans. Add it to `OTEL_RESOURCE_ATTRIBUTES`, or set it on the turn's root span, the outermost `invoke_agent` or `invoke_workflow` span. A release on a span above that root is not read. * The release is set only on the root span of a long run. A turn whose root span arrives more than 10 minutes after its first span is assembled without the root and keeps the release it had then. Set the release on the resource instead. * The agent reads a different attribute. **Agent settings, Release attribute** names the only attribute that is read; an agent without one reads `apgar.release`, `gen_ai.agent.version`, then `langfuse.release`. Spans stored before a custom attribute was chosen do not carry it. * Langfuse: the trace has no release. Set the release on the trace in the SDK. * Langfuse: the agent reads `apgar.release`, `gen_ai.agent.version`, or `service.version`. Langfuse traces do not carry them; use `langfuse.release` or `langfuse.trace.metadata.`. ## The release stays in Collecting * The release is below the early floor of 20 turns and 5 sessions, or below the minimum sample (**Minimum turns** and **Minimum sessions**) with no failing guardrail. The release page shows the counts. * The baseline has fewer turns or sessions than the minimum sample. The previous release may have been short lived, or its traffic is older than the 7 day baseline window. * Most sessions are mixed: they started on the previous release. Mixed turns leave both sides. See [Mixed sessions](https://docs.useapgar.dev/concepts/sessions.md#mixed-sessions). * No turns arrived in the last 24 hours, so the release is not checked again until its window ends. ## The release stays in Not enough data * The intervals are wider than the tolerances. The release page states the smallest change the sample can detect. More traffic narrows the intervals; so does a larger tolerance. * A guardrail has a previous value of 0 and a relative tolerance, so its change is undefined. Make that metric informational or turn it off. See [When a metric is unavailable](https://docs.useapgar.dev/concepts/metrics.md#when-a-metric-is-unavailable). * The agent has no applicable guardrail, for example an agent without tools whose remaining guardrails lack coverage. Such a release never reaches **Healthy**. ## A metric is unavailable The release page says what is missing. See [When a metric is unavailable](https://docs.useapgar.dev/concepts/metrics.md#when-a-metric-is-unavailable). The usual fixes: * **Retry rate** or **Loop rate**: tool spans lack arguments. Set `gen_ai.tool.call.arguments` on tool spans. * **Cost per turn**: chat spans lack a cost. Set `apgar.cost_usd` on each chat span, or let Langfuse price the model. * **Format failure rate**, or **Output length** without token counts: output content is not captured, or the Langfuse connection has `fetch_content` off. * **Turns per session**: fewer than 10 closed sessions, or most sessions are inferred because no session id is sent. See [Sessions](https://docs.useapgar.dev/concepts/sessions.md). ## No Slack message * The agent has no Slack connection. **Healthy** also needs **Send the healthy message** on in **Agent settings, Verdict rules**, and a release replaced by a newer one sends no **Unconfirmed** message. * Delivery failed. **Settings, Notifications** lists every delivery with its last error. Slack must answer `200`. * Links point at `localhost`: set `APGAR_PUBLIC_URL`. ## Webhook signatures do not verify * Compute the signature over the raw body bytes. Serializing the JSON again changes them. * Sign `X-Apgar-Timestamp`, a period, and the body. `sent_at` inside the body is the queue time and differs from the header on retries. * Use the secret stored on the connection. The connection test signs with the same one, so it is the quickest check. ## A process exits at startup | Exit code | Cause | | - | - | | `1` | ClickHouse is unreachable or its certificate does not verify, or a second worker saw a live heartbeat | | `2` | A required variable is missing or too short. The message names the field | | `3` | The database schema does not match this version. Run `apgar migrate` | On Kubernetes, the worker applies the migrations in an init container. See [When the worker does not start](https://docs.useapgar.dev/self-hosting/kubernetes.md#when-the-worker-does-not-start). This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.