Quickstart - LLM tracing
LLM tracing "Hello world."
This quickstart shows how to instrument a simple LLM app to send inputs and outputs to Evidently Cloud. You will use the open-source Tracely library.
You will need an OpenAI key to create a toy LLM app.
Need help? Ask on Discord.
1. Set up Evidently Cloud
Set up your Evidently Cloud workspace:
Sign up. If you do not have one yet, sign up for a free Evidently Cloud account.
Create an Organization. When you log in the first time, create and name your Organization.
Create a Project. Click + button under Project List. Create a Project, copy and save the Project ID. (Projects page)
Get your API token. Click the Key icon in the left menu. Generate and save the token. (Token page).
You can now go to your Python environment.
2. Installation
Install the Tracely library to instrument your app:
!pip install tracely
Install the Evidently library to interact with Evidently Cloud:
!pip install evidently
Install the OpenAI library to create a toy app:
!pip install openai
Imports:
import os
import openai
import time
from tracely import init_tracing
from tracely import trace_event
2. Initialize Tracing
Initialize the OpenAI client. Pass the token as an environment variable:
# os.environ["OPENAI_API_KEY"] = "YOUR_KEY"
client = openai.OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
Set up tracing parameters. Give it a name to identify your tracing dataset.
init_tracing(
address="https://app.evidently.cloud/",
api_key="EVIDENTLY_API_KEY",
project_id="YOUR_PROJECT_ID"
export_name="LLM tracing example"
)
3. Trace a simple function
Create a simple function to send questions to Open AI API and receive a completion. Set the questions list:
question_list = [
"What is Evidently Python library?",
"What is LLM observability?",
"How is MLOps different from LLMOps?"
]
Create a function and use the trace_event()
decorator to trace it:
@trace_event()
def pseudo_assistant(question):
system_prompt = "You are a helpful assistant. Please answer the following question concisely."
messages = [
{"role": "system", "content": system_prompt},
{"role": "user", "content": question},
]
return client.chat.completions.create(model="gpt-4o-mini", messages=messages).choices[0].message.content
# Iterate over the list of questions and pass each to the assistant
for question in question_list:
response = pseudo_assistant(question=question)
time.sleep(1)
4. View Traces
Go to the Evidently Cloud, open Datasets in the left menu (Datasets Page), and view your Traces.

What's next?
Want to run evaluations over this data? See a Quickstart.
Quickstart - LLM evaluationsCheck out a more in-depth tutorial to learn more about tracing:
Tutorial - TracingLast updated