sia.hackernoon.com

Do you know that LLM service providers can see the interactions that your AI agents have when you use their APIs? This may be common knowledge to those who are technical and knows how APIs work, but non-technical people may not have any idea that this goes on behind the scene.

To get started, what is an API? An API also known as Application Programming Interface is a way for two software components to communicate with each other using a set of definitions and protocols. In the case of LLMs, it allows your query, tools, and functions to interact with the provider’s model.

To illustrate what I mean, I built a simple agent that works with my command line, used the observability tool I built for my agents to monitor the interactions. Here’s what actually gets sent to OpenAI when you use their API with an AI agent framework. This isn’t just a theory but a real trace from a simple conversation…

In the images attached, you can see my system prompts and some logs that I added for the purpose of this demonstration. Then below is contents of the trace from my tool:


Thought:
      gen_ai.completion.0.content: Action: conversation_tool
Action Input: {"user_message": "Pick three countries from each continent and give what they are known for. Example: Asia - UAE, tourism"}
      gen_ai.completion.0.role: assistant
      gen_ai.usage.completion_tokens: 96
      gen_ai.usage.prompt_tokens: 1652
      gen_ai.usage.total_tokens: 1748
📊 Real Span: conversation_tool.tool_usage
   ID: 7be6961316e3c158
   Trace: b9c96c0989ca1ff44359dd42262fa618
   Start: 2025-08-06T13:32:43.900102
   Attributes: ['agentsso.span.kind', 'tool.name', 'telemetry.sdk.name', 'service.name', 'deployment.environment', 'tool.parameters', 'tool.result', 'tool.status']
📝 Set current trace ID: b9c96c0989ca1ff44359dd42262fa618
✅ Stored 2 REAL OpenTelemetry spans!
🆔 Real trace ID: b9c96c0989ca1ff44359dd42262fa618
127.0.0.1 - - [06/Aug/2025 13:32:44] "POST /v1/traces HTTP/1.1" 200 -
📨 Received traces request with Content-Type: application/x-protobuf
📦 Received protobuf data (12063 bytes)
🔍 Parsing 12063 bytes of real OpenTelemetry data...
✅ Successfully parsed as OpenTelemetry protobuf!
📊 Real Span: openai/gpt-3.5-turbo.llm
   ID: 02328b84cc2cdda8
   Trace: b9c96c0989ca1ff44359dd42262fa618
   Start: 2025-08-06T13:32:43.909294
   Attributes: ['telemetry.sdk.name', 'service.name', 'deployment.environment', 'gen_ai.system', 'agentsso.entity.name', 'agentsso.span.kind', 'gen_ai.request.model', 'gen_ai.request.max_tokens', 'gen_ai.prompt.0.role', 'gen_ai.prompt.0.content', 'gen_ai.prompt.1.role', 'gen_ai.prompt.1.content', 'gen_ai.prompt.2.role', 'gen_ai.prompt.2.content', 'gen_ai.completion.0.content', 'gen_ai.completion.0.role', 'gen_ai.usage.completion_tokens', 'gen_ai.usage.prompt_tokens', 'gen_ai.usage.total_tokens']
   🧠 LLM SPAN DETECTED!
      gen_ai.system: crewai
      gen_ai.request.model: openai/gpt-3.5-turbo
      gen_ai.request.max_tokens: 2000
      gen_ai.prompt.0.role: system
      gen_ai.prompt.0.content: You are Friendly AI Assistant. You are a helpful and friendly AI assistant who loves to have conversations 
                with users. You're knowledgeable, empathetic, and always try to provide 
                useful information while maintaining a warm, conversational tone.
Your personal goal is: Engage in meaningful conversations with users, answer their questions 
             accurately, and provide helpful assistance in a friendly manner.
             Use the conversation_tool to analyze user messages and provide 
             contextual, engaging responses.
You ONLY have access to the following tools, and should NEVER make up tools that are not listed here:

Tool Name: conversation_tool
Tool Arguments: {'user_message': {'description': None, 'type': 'str'}}
Tool Description: 
A tool to process and analyze user messages for better response generation.

Args:
    user_message: The message from the user

Returns:
    processed_message: Analysis of the user's message with context and intent


IMPORTANT: Use the following format in your response:

Thought: you should always think about what to do Action: the action to take, only one name of [conversation_tool], just the name, exactly as it's written. Action Input: the input to the action, just a simple JSON object, enclosed in curly braces, using " to wrap keys and values. Observation: the result of the action


Once all necessary information is gathered, return the following format: