Add initial instrumentation of OpenAI client #14221

anuraaga · 2025-07-10T04:54:16Z

This is initial instrumentation for OpenAI SDK. It is very similar in concept to the previous Bedrock instrumentation.

Future PRs will add support for streaming, async client, embeddings, and responses.

This is adapted from the instrumentation released in elastic's EDOT

anuraaga · 2025-07-10T04:54:40Z

conventions/src/main/kotlin/otel.scala-conventions.gradle.kts

@@ -5,6 +5,7 @@ plugins {
 }

 dependencies {
+  compileOnly("org.scala-lang:scala-library:2.11.12")


IntelliJ was having trouble syncing the project without this. It seemed harmless so hopefully it's ok.

otelbot-java-instrumentation · 2025-07-10T05:00:51Z

🔧 The result from spotlessApply was committed to the PR branch.

anuraaga · 2025-07-10T05:25:03Z

...java/io/opentelemetry/javaagent/instrumentation/openai/v1_1/OpenAIInstrumentationModule.java

+@SuppressWarnings("IdentifierName") // Want to match library's convention
+public class OpenAIInstrumentationModule extends InstrumentationModule {
+  public OpenAIInstrumentationModule() {
+    super("openai", "openai-java", "openai-java-1.1");


Got this lint error though it looks matching to me

Expected to find super\(\n? *"openai-java",\n? *"openai-java-1.1" in instrumentation/openai/openai-java-1.1/javaagent/src/main/java/io/opentelemetry/javaagent/instrumentation/openai/v1_1/OpenAIInstrumentationModule.java

I'd guess that it wants to see openai-java as the first instrumentation name to match with the directory name.

…metry-java-instrumentation into openai-instrumentation

testing-common/build.gradle.kts

anuraaga

Was a bit scared but could somehow get the shading working

anuraaga · 2025-07-11T06:31:43Z

testing/wiremock-shaded-for-testing/build.gradle.kts

+      exclude(dependency("org.junit.platform:junit-platform-commons"))
+      exclude(dependency("org.ow2.asm:asm"))
+      // Exclude dependencies bundled during Armeria shading
+      exclude(dependency("com.fasterxml.jackson.core:jackson-annotations"))


This is a bit weird, I wonder if instead of x-shaded-for-testing we should have a single shaded-test-deps project combining them? For this PR keeping changes to existing at a minimal though

i agree that having a single shaded artifact might be better

anuraaga · 2025-07-11T06:32:49Z

testing/wiremock-shaded-for-testing/build.gradle.kts

+
+    // Ensures tests are not affected by wiremock dependencies. Wiremock itself is
+    // safe with its original name.
+    relocate("com.google.common", "io.opentelemetry.testing.internal.guava")


It was tedious to find all these, but I couldn't find a better way. I thought it could be nice to just relocate everything, but I couldn't see any opt-out in that case to not relocate slf4j / junit. So opted-in with a big list instead.

Any notes to add on regenerate/check assumptions on this when libs update?

codefromthecrypt

Worth citing where this is coming from (albeit adapted?) Can help build confidence vs bedrock which was built from scratch and not formerly released elsewhere.

codefromthecrypt · 2025-07-11T09:55:57Z

...y/src/main/java/io/opentelemetry/instrumentation/openai/v1_1/ChatCompletionEventsHelper.java

+    } else if (content.isArrayOfContentParts()) {
+      return joinContentParts(content.asArrayOfContentParts());
+    } else {
+      throw new IllegalStateException("Unhandled content type for " + content);


I assume this wont crash the actual request..

Made it safer

codefromthecrypt · 2025-07-11T09:56:48Z

...y/src/main/java/io/opentelemetry/instrumentation/openai/v1_1/ChatCompletionEventsHelper.java

+      throw new IllegalStateException("Unhandled content type for " + content);
+    }
+  }
+


Guessing we aren't lucky enough to have base types to address the duplication that way.. and as the client is generated typical issue I suppose

codefromthecrypt · 2025-07-11T10:03:23Z

testing/wiremock-shaded-for-testing/build.gradle.kts

+
+    // Ensures tests are not affected by wiremock dependencies. Wiremock itself is
+    // safe with its original name.
+    relocate("com.google.common", "io.opentelemetry.testing.internal.guava")


Any notes to add on regenerate/check assumptions on this when libs update?

laurit · 2025-07-14T17:39:29Z

closing and reopening to re-trigger checks

instrumentation/openai/openai-java-1.1/javaagent/build.gradle.kts

...java/io/opentelemetry/javaagent/instrumentation/openai/v1_1/OpenAIClientInstrumentation.java

instrumentation/openai/openai-java-1.1/library/build.gradle.kts

...y/src/main/java/io/opentelemetry/instrumentation/openai/v1_1/ChatCompletionEventsHelper.java

...-1.1/library/src/main/java/io/opentelemetry/instrumentation/openai/v1_1/OpenAITelemetry.java

laurit · 2025-07-15T15:28:47Z

testing/wiremock-shaded-for-testing/build.gradle.kts

+      exclude(dependency("org.junit.platform:junit-platform-commons"))
+      exclude(dependency("org.ow2.asm:asm"))
+      // Exclude dependencies bundled during Armeria shading
+      exclude(dependency("com.fasterxml.jackson.core:jackson-annotations"))


i agree that having a single shaded artifact might be better

anuraaga

Thanks @laurit applied the comments

instrumentation/openai/openai-java-1.1/javaagent/build.gradle.kts

…nstrumentation into openai-instrumentation

laurit · 2025-07-16T10:07:47Z

...t/src/main/java/io/opentelemetry/javaagent/instrumentation/openai/v1_1/OpenAiSingletons.java

+      OpenAITelemetry.builder(GlobalOpenTelemetry.get())
+          .setCaptureMessageContent(
+              AgentInstrumentationConfig.get()
+                  .getBoolean("otel.instrumentation.genai.capture-message-content", false))


I think we should document this similarly to

opentelemetry-java-instrumentation/instrumentation/aws-sdk/README.md

Line 13 in eb8720f

| `otel.instrumentation.genai.capture-message-content` | Boolean | `false` | v2 only, record content of user and LLM messages when using Bedrock. |

An alternative would be to document in on the opentelemetry.io website.

Thanks, added a readme

Add initial instrumentation of OpenAI client

3cbd01c

anuraaga requested a review from a team as a code owner July 10, 2025 04:54

anuraaga commented Jul 10, 2025

View reviewed changes

./gradlew spotlessApply

f69d5a2

testing-common

e4a8f66

anuraaga commented Jul 10, 2025

View reviewed changes

Merge branch 'openai-instrumentation' of github.com:anuraaga/opentele…

4610cb5

…metry-java-instrumentation into openai-instrumentation

anuraaga commented Jul 10, 2025

View reviewed changes

testing-common/build.gradle.kts Outdated Show resolved Hide resolved

anuraaga added 8 commits July 10, 2025 16:31

Shade wiremock

39e8eff

apache

8b65535

wrong package

811985c

Actually migrate aws-sdk recording extension

41d944a

Move suppression

c357fe2

Relocate servlet

c1b346a

FOOOOSA

4b44332

Relocate yaml

1a238e0

anuraaga force-pushed the openai-instrumentation branch from a302aef to 1a238e0 Compare July 11, 2025 01:32

anuraaga added 5 commits July 11, 2025 12:04

Guava

6ab669b

asm

9912b77

jsonpath

b62cabf

apache

a787c10

Don't invert for now

2fa61f5

anuraaga commented Jul 11, 2025

View reviewed changes

codefromthecrypt approved these changes Jul 11, 2025

View reviewed changes

Cleanup

bcd60a9

laurit closed this Jul 14, 2025

laurit reopened this Jul 14, 2025

laurit reviewed Jul 15, 2025

View reviewed changes

anuraaga added 2 commits July 16, 2025 11:22

Cleanup

21673fd

More cleanup

fd4734e

anuraaga commented Jul 16, 2025

View reviewed changes

instrumentation/openai/openai-java-1.1/javaagent/build.gradle.kts Show resolved Hide resolved

Merge branch 'main' of github.com:open-telemetry/opentelemetry-java-i…

a86427e

…nstrumentation into openai-instrumentation

laurit reviewed Jul 16, 2025

View reviewed changes

laurit approved these changes Jul 16, 2025

View reviewed changes

trask added this to the v2.18.0 milestone Jul 16, 2025

Add README

193461c

trask merged commit 9fc8b20 into open-telemetry:main Jul 17, 2025
88 of 89 checks passed

Add initial instrumentation of OpenAI client #14221

Add initial instrumentation of OpenAI client #14221

Uh oh!

Conversation

anuraaga commented Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

otelbot-java-instrumentation bot commented Jul 10, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

anuraaga left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codefromthecrypt left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

laurit commented Jul 14, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anuraaga left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

anuraaga commented Jul 10, 2025 •

edited

Loading