There are two ways to get observability for an LLM application. You can adopt a vendor SDK and accept that your traces live in their schema, or you can emit OpenTelemetry and keep the option of moving.
This post is about the second one: what the GenAI semantic conventions actually specify, what belongs on a span, where the conventions are still moving, and the two mistakes that turn a trace into a log with extra steps.
Why OpenTelemetry rather than a vendor SDK
The honest argument is not technical elegance. It is that LLM observability vendors have been acquired, relicensed and repositioned repeatedly over the past two years, and instrumentation is the most expensive thing to redo. Every service you instrument is a code change, a review and a deploy. Doing that twice because your vendor changed is a bad use of a quarter.
OTel spans are portable. Emit them once and you can point them at a self-hosted collector, a commercial backend, or both during a migration. That is the whole argument, and it is a good one.
There is a real cost, and it is larger than it looks. The GenAI semantic conventions are younger than the rest of OpenTelemetry, every attribute in them is still at Development stability, and in May 2026 they moved out into a repository of their own. Plan for churn.
The semantic conventions, and where they moved
The gen_ai.* namespace is the OpenTelemetry semantic convention for generative AI. Worth knowing where it came from: Traceloop's OpenLLMetry project developed conventions that became the basis for the official OTel GenAI effort, and its README notes that its semantic conventions are now part of OpenTelemetry. OpenLIT states that it follows and maintains the conventions with the OTel community.
That matters practically. It means the attribute names below are not one vendor's invention, and a backend that claims OTel support should understand them.
Three things about their current state that a tutorial written even six months ago gets wrong.
They live in a different repository now. The split out of open-telemetry/semantic-conventions into open-telemetry/semantic-conventions-genai merged on 5 May 2026. The old paths are stubs pointing at the new repo. The last released tag containing real GenAI content was v1.41.1 on 11 May 2026, and v1.42.0 onwards are tombstones. The new repository has no releases and no tags at all, so the current specification is unversioned content on main. If you want to pin a version, what you are actually pinning is a commit.
Nothing in it is stable. The GenAI attribute registry carries 118 stability badges and every one of them reads Development. Not one Stable, not one Release Candidate. The only Stable badges anywhere in the GenAI docs belong to inherited core attributes such as error.type, server.address and server.port. This is not a reason to avoid the conventions, but it is a reason not to build a one-way schema migration on them.
gen_ai.system is gone. It was renamed to gen_ai.provider.name in semantic-conventions v1.37.0, published 25 August 2025, listed in the changelog under breaking changes as "Rename gen_ai.system to gen_ai.provider.name". It was not deprecated with a grace period; it simply does not appear in the current registry, and there is no deprecation entry for it. Current values include openai, anthropic, gcp.vertex_ai, gcp.gemini, aws.bedrock, azure.ai.openai, cohere, mistral_ai, groq, deepseek, perplexity and x_ai.
Emit gen_ai.provider.name. Plenty of backends still read gen_ai.system, so emitting both through a transition costs one attribute and avoids a blank provider column. There is a section below on which backends are on which generation.
The core attributes on a model call span:
gen_ai.provider.name the provider, e.g. "openai", "anthropic"
gen_ai.operation.name "chat", "text_completion", "embeddings"
gen_ai.request.model the model you asked for
gen_ai.response.model the model that actually served it
gen_ai.request.temperature sampling parameters
gen_ai.request.max_tokens
gen_ai.usage.input_tokens prompt tokens
gen_ai.usage.output_tokens completion tokens
gen_ai.response.finish_reasons why generation stopped
Three names that look right and are not in the spec: there is no gen_ai.usage.total_tokens, and there is no gen_ai.prompt or gen_ai.completion. Tools emitting those are using names of their own. All three are everywhere in older tutorials.
Token accounting has also become more granular than those two counters. The registry now separates cache reads from cache writes, splits input by audio, image and text, and carries reasoning output tokens on its own. If your provider reports them, there are named attributes to hold them. On a cache-heavy workload, the gap between input tokens and cache reads is the gap between a plausible bill and an accurate one.
gen_ai.request.model and gen_ai.response.model being separate is not pedantry. If you route through a gateway with fallback, or use a provider alias that resolves to a dated snapshot, these differ, and the difference is exactly what you need when a regression appears and nobody changed the code.
What a useful span actually looks like
Here is a minimal manual instrumentation in Python. Adapt the attribute set, not the structure.
from opentelemetry import trace
from opentelemetry.trace import SpanKind, Status, StatusCode
tracer = trace.get_tracer("my-app")
def call_model(client, messages, model="gpt-4o-mini"):
with tracer.start_as_current_span(
f"chat {model}",
kind=SpanKind.CLIENT,
) as span:
span.set_attribute("gen_ai.provider.name", "openai")
span.set_attribute("gen_ai.operation.name", "chat")
span.set_attribute("gen_ai.request.model", model)
try:
response = client.chat.completions.create(
model=model, messages=messages,
)
except Exception as error:
span.record_exception(error)
span.set_status(Status(StatusCode.ERROR))
raise
usage = response.usage
span.set_attribute("gen_ai.response.model", response.model)
span.set_attribute("gen_ai.usage.input_tokens", usage.prompt_tokens)
span.set_attribute("gen_ai.usage.output_tokens", usage.completion_tokens)
span.set_attribute(
"gen_ai.response.finish_reasons",
[c.finish_reason for c in response.choices],
)
return response
Three details worth copying.
SpanKind.CLIENT. A model call is an outbound request to a remote service. Marking it correctly means backends render it in the right place in a service map instead of treating it as internal work.
The span name is chat {model}, not the prompt. Naming is specified per operation rather than by one universal rule: inference and embeddings spans should be {gen_ai.operation.name} {gen_ai.request.model}, tool spans execute_tool {gen_ai.tool.name}, agent spans invoke_agent {gen_ai.agent.name}, and workflow spans invoke_workflow {gen_ai.workflow.name}. The reason is the same in every case: span names are a low-cardinality dimension. Putting anything user-derived in them will destroy your backend's aggregation and, on some pricing models, your bill.
Errors are recorded and re-raised. A span that swallows the exception is worse than no span, because it makes a failing system look healthy.
The two mistakes that make traces useless
Mistake one: putting prompts and completions on span attributes by default
It is tempting. It is also how teams end up with an observability backend full of customer PII, a compliance incident, and spans large enough to get dropped by collector limits.
The conventions treat prompt and completion content as opt-in, precisely because of this. The mechanism changed in v1.37.0: the per-message events (gen_ai.user.message and its siblings) were deprecated and replaced by span attributes, gen_ai.system_instructions, gen_ai.input.messages and gen_ai.output.messages, all at requirement level Opt-In.
Turning it on takes two environment variables, not one, and the second is not a boolean:
OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=SPAN_ONLY
The capture variable takes NO_CONTENT, which is the default, or SPAN_ONLY, EVENT_ONLY or SPAN_AND_EVENT. Setting one variable and not the other is the usual reason content never shows up. Treat content capture as a deliberate decision with a redaction step, not a default.
If you do capture content, capture it at a boundary you control, apply masking before it leaves the process, and be aware that on several platforms server-side data masking is a paid feature. Langfuse, for example, gates server-side data masking behind its enterprise licence when self-hosting, along with audit logs and retention policies. If you are self-hosting for compliance reasons, check that the compliance features are on the free side of the line before you plan around them.
Mistake two: instrumenting the model call and nothing else
A span around chat.completions.create tells you a model was slow. It does not tell you why the answer was wrong, which is the question you actually have.
The useful trace covers the whole unit of work:
POST /api/support-reply (SERVER, 4.2s)
├── retrieve context (INTERNAL, 180ms)
│ └── vector search (CLIENT, 140ms)
├── chat gpt-4o-mini (CLIENT, 3.1s)
│ └── gen_ai.usage.output_tokens = 512
├── execute generated code (INTERNAL, 620ms)
└── score response (INTERNAL, 90ms)
Now a slow request is attributable, and a wrong answer has the retrieval step sitting right next to the generation that used it. That is the difference between observability and logging.
The conventions now cover that wider unit of work too, which is the other thing older tutorials miss. Past chat there are namespaces for agents (gen_ai.agent.*), tool calls (gen_ai.tool.*), evaluations (gen_ai.evaluation.*), memory (gen_ai.memory.*), retrieval (gen_ai.retrieval.*), conversations (gen_ai.conversation.*) and workflows (gen_ai.workflow.name). If you are instrumenting an agent rather than a single model call, reach for those before inventing your own, with the caveat that all of them are at Development stability like everything else in the namespace.
Sessions and multi-turn conversations
A conversation is not a trace. It is a sequence of traces that need grouping.
Use a stable identifier propagated as an attribute across every trace in the conversation. Most backends expose this as a session concept and will group on it if you set it consistently. Set it once at the entry point and propagate it through baggage rather than threading it through every function signature.
The same applies to tenant identity. Set tenant.id or your equivalent as a span attribute, and be aware of a trap: on some pipelines the value ends up in resource attributes on some spans and span attributes on others, and querying the wrong map makes every tenant look inactive. Pick one and validate it with a real query before you build dashboards on it.
Which generation is your backend on
The rename split the ecosystem in half, and it is the sort of thing nobody writes down. As of August 2026:
- On the current spec, reading
gen_ai.provider.name: OpenLIT, Pydantic Logfire, Grafana Beyla, W&B Weave, and Arize Phoenix on its read path. - Still on the pre-v1.37
gen_ai.system: Langfuse, Helicone, LangSmith and Laminar.
Langfuse is the one worth checking yourself if you are planning around it: its ingestion code reads gen_ai.system and does not appear to read gen_ai.provider.name anywhere across its five ingest files.
The consequence is small and annoying. Emit only gen_ai.provider.name into a backend on the old generation and your provider dimension goes blank, so every chart grouped by provider quietly collapses into one bucket rather than erroring. Emitting both attributes through a transition removes the failure mode for the cost of one attribute. Either way, validate it with a real query before building dashboards on it, exactly as you would for tenant identity.
Choosing a backend
Anything that accepts OTLP will take these spans. The practical differences:
- OpenLIT: Apache-2.0, one datastore (ClickHouse), no paid tier today, explicitly maintains the
gen_aiconventions with the OTel community. The lightest thing to operate. - Laminar: Apache-2.0, accepts OTLP over gRPC, HTTP protobuf and HTTP JSON, documents full
gen_aicompatibility. Three datastores. - Langfuse: MIT core with an
ee/carve-out, well documented OTel ingestion, by far the largest community at 33,900+ stars. - Opik: Apache-2.0, 21,697 stars, first-party OTel support. Heaviest self-hosted stack at five datastores. Whether it maps
gen_ai.*specifically or a proprietary schema was not something we could confirm from its docs, so check that if portability matters. - Your existing backend: Datadog, Grafana, Honeycomb and others accept OTLP. If you already operate one, sending LLM spans there is less machinery than standing up a second platform.
That last option is underrated. A separate LLM observability tool means a second place to look during an incident.
Where the gateway approach differs
Everything above assumes you instrument the application. There is another option: route model calls through a gateway that emits the spans itself.
The trade is real in both directions. Instrumentation gives you spans around your own logic, which a gateway cannot see. A gateway gives you complete coverage of the model call path with no per-service rollout, including services you cannot easily change: a vendor integration, a coding agent, something in a language you have no SDK for.
Most teams end up wanting both, and they compose: emit OTLP from your application for your own logic, route model calls through a gateway for the call path, and correlate on trace ID. Everstack is OpenTelemetry-native on both sides, so gateway-produced spans and your own SDK spans land in the same trace rather than in two systems you join by hand.
A checklist
- Use the
gen_ai.*conventions rather than inventing attributes. - Emit
gen_ai.provider.name, notgen_ai.system, and emit both while your backend catches up. - Do not expect
gen_ai.usage.total_tokens,gen_ai.promptorgen_ai.completionto exist. They are not in the spec. - Mark model calls
SpanKind.CLIENT. - Keep span names low-cardinality. Never put prompts in them.
- Record both
gen_ai.request.modelandgen_ai.response.model. - Treat prompt and completion capture as opt-in with redaction.
- Instrument the whole unit of work, not just the model call.
- Propagate session and tenant identifiers consistently, and validate with a real query.
- Verify your backend understands
gen_ai.*rather than assuming OTLP support means semantic support, and check which generation of the attribute names it reads. - Expect churn. Every GenAI attribute is at Development stability, and the conventions now live in their own repository with no releases.
Get those right and your traces survive a change of backend. Get the first two wrong and you will be re-instrumenting either way.
Everstack's tracing documentation covers the emitter side, and the Langfuse comparison goes deeper on the instrumentation versus gateway trade.