Unable to View Traces on Laminar Traces Page Due to **StatusCode.DEADLINE_EXCEEDED** Error

[AswiniIAI]

When attempting to view traces on the Laminar traces page, no traces are displayed. After refreshing the page, the following error messages appear in the terminal:

textWARNING [opentelemetry.exporter.otlp.proto.grpc.exporter] Transient error StatusCode.DEADLINE_EXCEEDED encountered while exporting traces to {host_ip_address}, retrying in 4s. WARNING [opentelemetry.exporter.otlp.proto.grpc.exporter] Transient error StatusCode.DEADLINE_EXCEEDED encountered while exporting traces to {host_ip_address}, retrying in 8s.

The frontend generates output correctly, but the traces are not visible on the traces page, and the StatusCode.DEADLINE_EXCEEDED error persists in the terminal.

Steps to Reproduce:

• Navigate to the Laminar traces page.
• Attempt to view traces.
• Refresh the page and check the terminal for errors.

Expected Behavior:
Traces should be visible on the Laminar traces page without any errors.
Actual Behavior:
No traces are displayed, and the terminal logs show StatusCode.DEADLINE_EXCEEDED errors when exporting traces to {host_ip_address}.
Environment:

Additional Context:

• The frontend output appears to be working as expected.
• The issue occurs consistently when checking traces.

Question:
How can I resolve the StatusCode.DEADLINE_EXCEEDED error and ensure traces are displayed correctly on the Laminar traces page?

[skull8888888]

@AswiniIAI how do you initialize Laminar? Can you please share some code.

[dinmukhamedm]

It looks like a connection issue. Let's start with the simplest connectivity check. On the same machine as you are running your instrumented service, can you try running

curl 20.41.235.189:8001

and paste the output?

DEADLINE_EXCEEDED indicates that there is a server running on the address and port, but it either responds incorrectly or, more likely, does not respond within the timeout value

[AswiniIAI]

It looks like a connection issue. Let's start with the simplest connectivity check. On the same machine as you are running your instrumented service, can you try running

curl 20.41.235.189:8001

and paste the output?

DEADLINE_EXCEEDED indicates that there is a server running on the address and port, but it either responds incorrectly or, more likely, does not respond within the timeout value

Hi @dinmukhamedm ,

Thank you for the suggestion. I ran the curl command on the same machine as my instrumented service, and here’s the output:

(venv) username:~$ curl {host_ip_address} curl: (1) Received HTTP/0.9 when not allowed

I also tried adjusting the timeout value in my code. The default timeout was 10 seconds, which I increased to 30 seconds initially and then to 60 seconds. However, I’m still encountering the same StatusCode.DEADLINE_EXCEEDED error when exporting traces to {host_ip_address}.
Could you suggest any further steps to diagnose the connection issue or potential misconfigurations in the OpenTelemetry exporter setup?

[AswiniIAI]

@AswiniIAI how do you initialize Laminar? Can you please share some code.

Hi @skull8888888 ,

Thank you for your response. Below is how I initialize Laminar in my code:
pythonfrom lmnr import Laminar
Laminar.initialize( project_api_key=os.getenv("LMNR_PROJECT_API_KEY"), base_url="{base_url}", http_port={http_port}, grpc_port={grpc_port}, )

This is the complete initialization code for Laminar in my application. I’m using the endpoint {ip_address} for gRPC, as specified in the grpc_port parameter. However, I’m still encountering the StatusCode.DEADLINE_EXCEEDED error when exporting traces, as mentioned in the issue.
Could you confirm if this initialization looks correct or suggest any adjustments (e.g., to the gRPC configuration or timeout settings) to resolve the issue?

[dinmukhamedm]

The initialization code looks correct. The http0.9 response is also expected. This may indicate an issue with either configuration or our code (app-server otel ingester side). Let's dive in deeper.

1. Which docker compose file are you using?
2. Could you paste the full logs of app-server container?
3. Do you see anything unusual in clickhouse or postgres logs?

[AswiniIAI]

Hi @dinmukhamedm ,

Here’s the Docker Compose file I’m using:
`services:
app:
build: .
restart: unless-stopped
ports:
- "127.0.0.1:8000:8000" # bound locally for verification (using some other ports)
command: python3 main.py

celery-worker:
build: .
restart: unless-stopped
command: celery -A celery_app worker --loglevel=info

celery-beat:
build: .
restart: unless-stopped
command: celery -A celery_app beat --loglevel=info`

I also tested a small code snippet with the same Laminar URL and API key that I’m using in my project. Here is the snippet:

`import os
from openai import OpenAI
from dotenv import load_dotenv

load_dotenv()

from lmnr import observe, Laminar
Laminar.initialize(project_api_key=os.getenv("LMNR_PROJECT_API_KEY"),
base_url="",
http_port=,
grpc_port=,
)

client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

@observe() # annotate all functions you want to trace
def poem_writer(topic):
response = client.chat.completions.create(
model="", # i tried with gpt-4.1 model.
messages=[
{"role": "user", "content": f"write a poem about {topic}"},
],
)
poem = response.choices[0].message.content
return poem

if name == "main":
print(poem_writer(topic="laminar flow"))`

For the above code snippet, I’m able to get the expected output, but at the end I also see this error message:
Failed to export traces to {ip_address}, error code: StatusCode.DEADLINE_EXCEEDED
(This is the same behavior I observe in my project traces scenario.)

Below I’m attaching the log files for app-server, clickhouse, and postgres:

postgre.log
clickhouse.log
app_server.log

One more detail: Until August 13, traces were being ingested into Laminar successfully. After that date, traces stopped working.

[dinmukhamedm]

@AswiniIAI thanks for all the details. The app-server.log indicates there is an issue with message queues.

• Which docker-compose file are you using to run Laminar? It looks like app-server fails to connect to RabbitMQ service.
• Do you have rabbitmq-0 (or rabbitmq-1) container spun up? Are there any interesting logs inside it?
• What is the RABBITMQ_URL inside your app-server configuration? This is read in the docker-compose file using the contents of your .env file at the root of lmnr-ai/lmnr