Span kinds
Each span gets one kind:invoke_workflow, invoke_agent, chat, execute_tool, retrieval, or other. The first of these attributes that is present decides, and values are matched without regard to case:
When none of them is present, a span with
gen_ai.request.model, gen_ai.usage.input_tokens, llm.token_count.prompt, or llm.model_name is a chat span, and a span with gen_ai.tool.name or tool.name is a tool span. Span names are not used.
The root of a turn is the earliest workflow or agent span that has no workflow or agent ancestor. Without one, it is the earliest span without a parent. When every span has a parent, for example because the parent span was sampled away, the earliest span whose parent is missing becomes the root and the turn is marked partial.
Agent name
Root span attributesgen_ai.agent.name, agent.name, langfuse.trace.name, then resource attributes gen_ai.agent.name, service.name.
Release
Every agent created in the UI has a Release attribute (release.from), and only that attribute is read, on the root span and then on the resource. The choices are apgar.release, gen_ai.agent.version, langfuse.release, service.version, or any custom attribute. Apgar starts keeping a custom attribute within a minute of the setting being saved; spans stored before then do not carry it. An agent that an unbound ingest token creates has no Release attribute and reads apgar.release, gen_ai.agent.version, then langfuse.release. A Langfuse agent sees only the Langfuse fields of a trace, so its Release attribute should be langfuse.release or a langfuse.trace.metadata.<key> attribute. See Langfuse.
Environment
apgar.environment, deployment.environment.name, deployment.environment, langfuse.environment. Read on the root span, then on the resource. Missing means default.
An agent with an environment counts only turns from that environment. Leave the agent environment empty to accept every environment. Do not send production and staging traffic to one agent without an environment: both would mix in the same comparison, and a staging release would end the monitoring of the production release.
Session
apgar.session.id, gen_ai.conversation.id, session.id, langfuse.session.id, on the root span. Session ids change per conversation, so they are not read from the resource.
When none is present, the trace id is the session and the session is marked inferred. A turn is then its own session. See Sessions.
Surface
apgar.surface on the root span. Otherwise the agent’s Surface attribute (surface_mapping.from) decides:
The surface is truncated to 64 characters. Use it for the channel a turn came from, such as
slack, web, or api.
Prompt version
langfuse.prompt.name with langfuse.prompt.version as name@version, then apgar.prompt.version, then gen_ai.prompt.name. Read on the root span, then on the first chat span.
Models
On chat spans. Request model:gen_ai.request.model, llm.request.model_name, llm.model_name, langfuse.observation.model.name. Response model: gen_ai.response.model, llm.response.model_name, llm.model_name, langfuse.observation.model.name. The response model of the last chat span is the primary model of the turn.
Tools
On tool spans. Tool name:gen_ai.tool.name, tool.name, else the span name.
Tool arguments: gen_ai.tool.call.arguments, input.value, langfuse.observation.input. Apgar stores an HMAC of the arguments, never the arguments: of their canonical JSON when they are an object, a list, or a JSON string, else of the raw string. A tool call is a retry when an earlier call in the same turn had the same tool and arguments and failed. A turn has a loop when three consecutive calls have the same tool and arguments. A call without arguments has an unknown retry status; a turn is marked loop as soon as three hashed calls match, and has an unknown loop status only when no loop was found and some call lacks arguments.
Errors
A span is an error when its OTLP status code isERROR or its Langfuse level is ERROR. The error type comes from error.type, exception.type, the OTLP status message, or the Langfuse status message, truncated to 128 characters.
Tokens
Input:gen_ai.usage.input_tokens, llm.token_count.prompt, Langfuse usage details input. Output: gen_ai.usage.output_tokens, llm.token_count.completion, Langfuse usage details output. A turn sums its chat spans.
Cost
apgar.cost_usd, gen_ai.usage.cost, llm.cost.total, Langfuse cost details total or the observation totalCost. US dollars.
Apgar does not estimate cost from token counts. The GenAI semantic conventions define no cost attribute, so most OTLP instrumentations need the application or a span processor to set apgar.cost_usd on each chat span. Langfuse fills cost itself when it knows the model price.
The cost of a turn is the sum over its chat spans. If one chat span has token counts but no cost, the turn has no cost, because a partial sum would read as a lower cost.
Content
Output content is read fromgen_ai.output.messages, OpenInference output.value, and the Langfuse observation output, on the root span or else on the last chat span. When the value is a message list, the last assistant message counts; a plain string counts whole. Apgar keeps the character count and whether the text parses as JSON, then drops the content. Input and output content attributes are never stored.
OTLP logs and span events are not read. An instrumentation that records content as events, or not at all, leaves the character fallback of Output length and Format failure rate unavailable.
Late spans
A span that started more thanAPGAR_LATE_SPAN_DAYS (default 7) days before it arrived is marked late. Late turns are stored and excluded from evaluations.