[connector/elasticapm] Add ability configure metric aggregation limit configs

[isaacaflores2]

Exposed aggregation limit configs that are propagated to the lsmintervalprocessor.

Note: I opted to keep the configuration names ( resource_limit ) the same as the lsmintervalprocessor mostly so they are consistent for both components. Another option would be to restructure everything to omit the _limit suffix so elasticapm::aggregation::limit::resource and elasticapm::aggregation::limit::scope. Let me know what you think.

Checklist

• Update connector/elasticapmconnector/README.md to includes the limits in the example config.yml
• Add test cases to assert overflow behavior
• Run collector locally to verify overflow behavior
• Double check the default values are appropriate and match MIS
  • I found the default values under apm-managed-service/checkout/apm-aggregation-service/config/config.go

                        // Defaults to apm-server 8GB equivalent limits.
   		MaxSpanGroups:                         40000,
   		MaxSpanGroupsPerService:               4000,
   		MaxTransactionGroups:                  40000,
   		MaxTransactionGroupsPerService:        4000,
   		MaxServiceTransactionGroups:           8000,
   		MaxServiceTransactionGroupsPerService: 800,
   		MaxServices:                           8000,

Local Testing

1. Build collector make genelasticcol
2. Run the collector with the below config

receivers:
 otlp:
   protocols:
     grpc:
       endpoint: 0.0.0.0:4317
       include_metadata: true
     http:
       endpoint: 0.0.0.0:4318
       include_metadata: true

exporters:
 otlp:
   endpoint: "${env:ELASTIC_APM_SERVER_URL:-localhost:8200}"
   headers:
     Authorization: "Bearer ${env:ELASTIC_APM_TOKEN}"
   sending_queue:
     enabled: false
   timeout: 60s
 debug:

connectors:
 elasticapm:
   aggregation:
     limit:
       resource:
         max_cardinality: 1
       scope:
         max_cardinality: 1
       metric:
         max_cardinality: 1
       datapoint:
         max_cardinality: 1

service:
 pipelines:
   logs:
     receivers: [otlp]
     exporters: [elasticapm, otlp]
   metrics:
     receivers: [otlp]
     exporters: [elasticapm, otlp]
   traces:
     receivers: [otlp]
     exporters: [elasticapm, otlp]
   metrics/apm:
     receivers: [elasticapm]
     exporters: [otlp, debug]

3. Send data with different service names (local_test_1, local_test_2)

• telemetrygen traces --duration 1s --otlp-insecure --service "local_test_1"

4. Search for overflow metric documents with query labels.overflow:true to validate overflow behavior

   Example Overflow metric

      {
       "_index": ".ds-metrics-service_summary.1m.otel-default-2025.07.31-000001",
       "_id": "RO44WNIpbtQmrnRmAAABmGKSIzM",
       "_version": 1,
       "_source": {
         "@timestamp": "2025-07-31T22:20:01.203Z",
         "_metric_names_hash": "fd91fda6720b28bc",
         "attributes": {
           "metricset.interval": "1m",
           "metricset.name": "service_summary",
           "processor.event": "metric"
         },
         "data_stream": {
           "dataset": "service_summary.1m.otel",
           "namespace": "default",
           "type": "metrics"
         },
         "metrics": {
           "service_summary": 1
         },
         "resource": {
           "attributes": {
             "agent.name": "unknown",
             "service.name": "_other",
             "signaltometrics.service.instance.id": "11c21f6a-98d8-44a6-ab32-4574bca301c1",
             "signaltometrics.service.name": "motel-aggregation-collector"
           }
         },
         "scope": {
           "name": "github.com/open-telemetry/opentelemetry-collector-contrib/connector/signaltometricsconnector"
         }
       },
       "fields": {
         "signaltometrics.service.instance.id": [
           "11c21f6a-98d8-44a6-ab32-4574bca301c1"
         ],
         "scope.name": [
           "github.com/open-telemetry/opentelemetry-collector-contrib/connector/signaltometricsconnector"
         ],
         "service.name": [
           "_other"
         ],
         "data_stream.namespace": [
           "default"
         ],
         "resource.attributes.agent.name": [
           "unknown"
         ],
         "_metric_names_hash": [
           "fd91fda6720b28bc"
         ],
         "service_summary": [
           1
         ],
         "metricset.interval": [
           "1m"
         ],
         "data_stream.type": [
           "metrics"
         ],
         "resource.attributes.signaltometrics.service.name": [
           "motel-aggregation-collector"
         ],
         "metricset.name": [
           "service_summary"
         ],
         "signaltometrics.service.name": [
           "motel-aggregation-collector"
         ],
         "attributes.processor.event": [
           "metric"
         ],
         "@timestamp": [
           "2025-07-31T22:20:01.203Z"
         ],
         "data_stream.dataset": [
           "service_summary.1m.otel"
         ],
         "processor.event": [
           "metric"
         ],
         "metrics.service_summary": [
           1
         ],
         "agent.name": [
           "unknown"
         ],
         "attributes.metricset.interval": [
           "1m"
         ],
         "resource.attributes.signaltometrics.service.instance.id": [
           "11c21f6a-98d8-44a6-ab32-4574bca301c1"
         ],
         "attributes.metricset.name": [
           "service_summary"
         ],
         "event.dataset": [
           "service_summary.1m.otel"
         ],
         "resource.attributes.service.name": [
           "_other"
         ]
       }
     }

service map overflow

Reference

Closes : #465

[lahsivjar]

A few comments on the approach for the configurations.

[lahsivjar]

LGTM! Just one minor comment about putting configs in files in testadata directory which can be addressed in #698.

Thanks for sticking with this and apologies about the late review.