Record LLM calls

Capture model, token, latency, error, and cost data and attach it to the customer who caused it.

Teml represents an LLM call as a trace span using OpenTelemetry gen_ai.* attributes. When customer identity is active, the call and its cost appear in the same customer story as errors and application activity.

Use an SDK wrapper

The JavaScript, Python, and Go SDKs provide wrappers for non-streaming OpenAI and Anthropic calls.

JavaScript / TypeScript

import OpenAI from 'openai';
import { wrapOpenAI } from '@teml/sdk/openai';

const openai = wrapOpenAI(new OpenAI(), teml);
const response = await openai.chat.completions.create({
  model: process.env.OPENAI_MODEL!,
  messages,
});

Python

import os
from openai import OpenAI
from teml.integrations.openai import wrap_openai

openai = wrap_openai(OpenAI(), client)
response = openai.chat.completions.create(
    model=os.environ['OPENAI_MODEL'],
    messages=messages,
)

import (
	"github.com/openai/openai-go"
	"github.com/openai/openai-go/option"
	temlopenai "github.com/getteml/teml/sdks/go/instrumentation/openai"
)

real := openai.NewClient(option.WithAPIKey(os.Getenv("OPENAI_API_KEY")))
openaiClient := temlopenai.WrapClient(temlClient, &real)
response, err := openaiClient.Chat.Completions.New(
	ctx,
	openai.ChatCompletionNewParams{
		Model:    os.Getenv("OPENAI_MODEL"),
		Messages: messages,
	},
)

Call identify or establish request-scoped identity before the model request.

Use existing OpenTelemetry instrumentation

Teml reads standard gen_ai span attributes from any OTLP source. At minimum, record the provider, model, input tokens, output tokens, duration, and error status. Teml can calculate cost for recognized models when token counts are present.

Prompt and completion content

Content capture is off by default. Leave it off unless the debugging value outweighs the privacy and security cost. If you enable it, review retention, user consent, PII handling, and access controls before production rollout.

Streaming wrapper calls currently pass through without automatic LLM-call instrumentation. Add an explicit span around a streaming request if you need its latency and token data.

Verify

Open Traces → LLM Costs, then open the customer who made the request. The same call should appear in both views with matching model, token, and cost values.