> ## Content Index > Fetch the complete content index at: https://stalin-pereira.com/llms.txt > Use this file to discover other available public pages before exploring further. # Instrumenting a Node.js Service with OpenTelemetry Auto-Instrumentation - URL: https://stalin-pereira.com/instrumenting-a-node-js-service-with-opentelemetry-auto-instrumentation/ - Published: 2026-01-07T04:36:00.000Z - Updated: 2026-10-16T19:23:21.000Z - Description: Get traces and metrics from a Node.js service without vendor lock-in: zero-code setup, the ESM loader trap that silently drops spans, a production-ready SDK config, surprising defaults, and a Collector pipeline that keeps your backend a config choice. - Author: Stalin Pereira - Tags: Observability Proprietary APM agents are easy to install and hard to leave. You add a vendor's package to every service, their data format becomes your data format, and your dashboards, alerts and on-call habits all grow around it. When pricing changes or you want a second backend, switching means re-instrumenting every service. That cost is the lock-in. OpenTelemetry (OTel) separates two jobs that agents bundle together: *generating* telemetry in your process and *analysing* it in a backend. Your service emits traces, metrics and logs in the open OTLP format. Where they end up becomes a configuration decision rather than a code change. For Node.js, auto-instrumentation gets you most of the way without touching application code. This post walks through instrumenting a Node.js service properly: the zero-code route, the ESM trap that silently drops spans, a programmatic setup worth running in production, the defaults that surprise people, and the Collector configuration that turns all of this into something you control. Package names and APIs are current as of `@opentelemetry/sdk-node` 0.2xx and `@opentelemetry/auto-instrumentations-node` 0.8x. The JS SDK moves quickly, so check the [official docs](https://opentelemetry.io/docs/zero-code/js/?ref=stalin-pereira.com) when you upgrade. ## How auto-instrumentation actually works There's no magic here, just monkey-patching. When the SDK starts, each instrumentation package registers a hook on the module loader. When your code (or a dependency) loads `http`, `express`, `pg` or `ioredis`, the hook wraps the relevant functions before your code gets the module back. From then on, an incoming request starts a server span, a database query starts a client span, and the active context flows through async calls, so child spans attach to the right parent. Two consequences follow: - **The SDK must load before anything it instruments.** If `express` is imported before the hook is installed, it isn't patched, and nothing warns you. That's why setup happens on the Node command line (`--require` or `--import`), not inside `app.js`. - **CommonJS and ESM are patched by different mechanisms.** `require()` is intercepted with `require-in-the-middle`. Native `import` needs a loader hook built on `import-in-the-middle`. Forget the loader hook and ESM-imported libraries are silently left uninstrumented. The meta-package `@opentelemetry/auto-instrumentations-node` bundles instrumentations for the common stack: `http`/`https`, `undici` (Node's built-in `fetch`), Express, Koa, NestJS, GraphQL, gRPC, `pg`, `mysql2`, MongoDB, Mongoose, Redis and ioredis, several messaging clients, the AWS SDK, and more. Check its README for the current list. Fastify, for example, now ships its own `@fastify/otel` plugin instead of a bundled instrumentation. ## The zero-code route (CommonJS) If your service is CommonJS, the fastest path is two packages and a flag: ```bash npm install --save @opentelemetry/api @opentelemetry/auto-instrumentations-node export OTEL_SERVICE_NAME="checkout-api" export OTEL_EXPORTER_OTLP_ENDPOINT="http://otel-collector:4318" export OTEL_TRACES_EXPORTER="otlp" export OTEL_METRICS_EXPORTER="otlp" export OTEL_LOGS_EXPORTER="none" export OTEL_NODE_RESOURCE_DETECTORS="env,host,os,process" export OTEL_NODE_DISABLED_INSTRUMENTATIONS="net,dns" node --require @opentelemetry/auto-instrumentations-node/register app.js ``` If you can't change the start command (a `npm start` script or a base image entrypoint, say), set `NODE_OPTIONS="--require @opentelemetry/auto-instrumentations-node/register"` instead. Everything is configured through standard `OTEL_*` environment variables, which is a big part of the appeal. The same variables work in Java, Python and Go, so a platform team can set them once in a Helm chart. Instrumentation names in `OTEL_NODE_ENABLED_INSTRUMENTATIONS` and `OTEL_NODE_DISABLED_INSTRUMENTATIONS` drop the `@opentelemetry/instrumentation-` prefix, so `net,dns` means `@opentelemetry/instrumentation-net` and `@opentelemetry/instrumentation-dns`. The limitation: you can't pass per-instrumentation options (such as "ignore health checks") through environment variables. Once you need those, move to a programmatic setup. ## ESM: the trap that silently drops spans If your `package.json` has `"type": "module"` or your entry point is `.mjs`, swapping `--require` for `--import` isn't enough. Without the loader hook, the result is confusing: you still get HTTP server spans, because `http` is patched another way, but they're named just `GET` with no route, and the Express middleware and handler spans are missing entirely. Nothing errors. The trace just looks thin. The fix is to install the OTel loader hook. With the zero-code entry point: ```bash node --experimental-loader=@opentelemetry/instrumentation/hook.mjs \ --import @opentelemetry/auto-instrumentations-node/register \ app.js ``` This works, but recent Node versions print a warning that `--experimental-loader` may be removed in favour of `module.register()`. The cleaner approach is to call `module.register()` yourself at the top of a small instrumentation file, which you'll want anyway for configuration. ## A programmatic setup for production Install the SDK alongside the auto-instrumentations: ```bash npm install --save @opentelemetry/api @opentelemetry/sdk-node \ @opentelemetry/auto-instrumentations-node \ @opentelemetry/resources @opentelemetry/semantic-conventions ``` Then create `instrumentation.mjs`: ```javascript import { register } from 'node:module'; import { NodeSDK } from '@opentelemetry/sdk-node'; import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node'; import { resourceFromAttributes } from '@opentelemetry/resources'; import { ATTR_SERVICE_NAME, ATTR_SERVICE_VERSION } from '@opentelemetry/semantic-conventions'; // Install the ESM loader hook so `import` statements can be patched. register('@opentelemetry/instrumentation/hook.mjs', import.meta.url); const sdk = new NodeSDK({ resource: resourceFromAttributes({ [ATTR_SERVICE_NAME]: 'checkout-api', [ATTR_SERVICE_VERSION]: process.env.APP_VERSION ?? 'dev', }), instrumentations: [ getNodeAutoInstrumentations({ '@opentelemetry/instrumentation-http': { ignoreIncomingRequestHook: (req) => req.url === '/healthz' || req.url === '/readyz', }, '@opentelemetry/instrumentation-net': { enabled: false }, '@opentelemetry/instrumentation-dns': { enabled: false }, }), ], }); sdk.start(); process.once('SIGTERM', async () => { try { await sdk.shutdown(); // flush buffered spans and metrics } finally { process.exit(0); } }); ``` Run it with `node --import ./instrumentation.mjs app.js`. Exporters, endpoint, sampling and protocol still come from the same `OTEL_*` variables. Only what genuinely needs code lives in the file. Some notes on the choices above: - **Health checks are ignored at the source.** Kubernetes probes can easily be the most frequent request a service receives. They tell you nothing in a trace view, and you pay to store them. - **`net` and `dns` are disabled.** They add a `tcp.connect` or lookup span under almost every outgoing call. That's occasionally useful when debugging connection problems, but usually noise. `fs` is already off by default for the same reason. - **Shutdown flushes buffers.** Spans are batched in memory. Exit without `sdk.shutdown()` and you lose the last few seconds of telemetry, which are often the seconds you care about during a crash or deploy. If your app already handles `SIGTERM` (to drain HTTP connections, say), call `sdk.shutdown()` from that handler as the last step instead of adding a second listener. ### Adding the spans auto-instrumentation can't see Auto-instrumentation covers I/O at library boundaries. It can't see that a slow request spent most of its time in your pricing logic. Add manual spans where the business logic matters. They nest under the auto-generated spans automatically: ```javascript import { trace, SpanStatusCode } from '@opentelemetry/api'; const tracer = trace.getTracer('checkout-api'); export async function priceCart(cart) { return tracer.startActiveSpan('priceCart', async (span) => { try { span.setAttribute('cart.items', cart.items.length); return await computeTotals(cart); } catch (err) { span.recordException(err); span.setStatus({ code: SpanStatusCode.ERROR }); throw err; } finally { span.end(); } }); } ``` Application code depends only on `@opentelemetry/api`. If the SDK isn't loaded (in unit tests, for example), these calls are cheap no-ops. ## Defaults that surprise people Turn on `OTEL_LOG_LEVEL=debug` once in a non-production environment and read the output. It shows which instrumentations were applied, which exporters were chosen, and what's failing. Three things tend to come up. **Metrics and logs exporters default to OTLP.** If `OTEL_METRICS_EXPORTER` and `OTEL_LOGS_EXPORTER` are unset, `NodeSDK` creates OTLP exporters for them too. With no collector listening, export attempts fail and retry, and shutdown can stall for several seconds while it tries to flush. Set every signal explicitly, using `none` for the ones you don't collect. **Cloud resource detectors make network calls.** By default, all resource detectors run, including AWS, GCP, Azure and Alibaba. On a laptop or in a different cloud, they probe metadata endpoints such as `169.254.169.254` and `metadata.google.internal` and log the failures. Set `OTEL_NODE_RESOURCE_DETECTORS` to what you actually run on, for example `env,host,os,process,container,aws`. **The zero-code entry point installs a `SIGTERM` listener.** The `register` module shuts the SDK down on `SIGTERM`, but it doesn't exit the process. Node drops its default exit-on-`SIGTERM` behaviour as soon as any listener exists, so a service with no shutdown handler of its own can keep running after `SIGTERM` until the orchestrator sends `SIGKILL`. Make sure your app handles `SIGTERM` and exits. ## Traces that cross service boundaries Single-service traces are useful. Distributed traces are where OTel pays off. The HTTP and `undici` instrumentations automatically inject a `traceparent` header into outgoing requests and extract it from incoming ones, so a request that passes through a gateway, this service and a downstream API shows up as one trace with one trace ID. By default the SDK uses the W3C Trace Context and Baggage propagators. That matters when services in other languages, or a vendor's load balancer, need to join the same trace. For how the headers work and where propagation breaks (message queues, async jobs, proxies that strip headers), see [implementing W3C Trace Context across distributed services](https://stalin-pereira.com/implementing-w3c-trace-context-across-distributed-services/). ## The Collector: where you actually take control You can point the SDK straight at a vendor's OTLP endpoint. For anything beyond a single service, put an [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/?ref=stalin-pereira.com) in between. Your services always talk to `otel-collector:4318`, and everything else (filtering, sampling, enrichment, which backends receive data) becomes collector configuration your platform team owns. ```yaml receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318 processors: memory_limiter: check_interval: 1s limit_percentage: 80 spike_limit_percentage: 15 filter/noise: error_mode: ignore trace_conditions: - span.attributes["url.path"] == "/healthz" - span.attributes["url.path"] == "/metrics" batch: {} exporters: otlp_http/primary: endpoint: https://otlp.vendor-a.example.com headers: api-key: ${env:VENDOR_A_API_KEY} otlp_http/evaluation: endpoint: https://otlp.vendor-b.example.com headers: authorization: Bearer ${env:VENDOR_B_TOKEN} service: pipelines: traces: receivers: [otlp] processors: [memory_limiter, filter/noise, batch] exporters: [otlp_http/primary, otlp_http/evaluation] metrics: receivers: [otlp] processors: [memory_limiter, batch] exporters: [otlp_http/primary] ``` That config sends every trace to two backends at once. This is how you evaluate a new vendor, or migrate to one, without touching a single service. It also drops noise centrally, for services you can't easily redeploy. The `filter` processor is in the contrib distribution (`otelcol-contrib`). Older Collector releases use `otlphttp` instead of `otlp_http`, and `traces: span:` instead of `trace_conditions`, so check the README for the version you deploy. Sampling deserves a deliberate decision too. The simplest lever is head sampling in the SDK, set through environment variables: ```bash export OTEL_TRACES_SAMPLER="parentbased_traceidratio" export OTEL_TRACES_SAMPLER_ARG="0.1" # keep 10% of new traces; follow the parent's decision otherwise ``` The `parentbased_` prefix matters: downstream services respect the upstream decision, so you don't end up with traces that have holes in them. If you need to keep every error and every slow request, which head sampling can't guarantee, look at tail sampling in the Collector. It buffers whole traces before deciding, which needs more memory and a load-balancing tier in front of it. ## What auto-instrumentation won't do for you - **It doesn't make your telemetry meaningful.** You get spans for every query and request, but not "which customer tier", "which feature flag" or "which pricing rule". Add business attributes and a handful of manual spans. - **It isn't free.** Patching, context propagation and span export cost some CPU and memory per request. The cost is usually modest for I/O-bound services, but measure it under your own load before and after, especially on hot paths with many small spans. - **It can capture things you didn't mean to.** URLs with tokens in query strings, database statements with literal values, and headers can all end up in attributes. Review what the database and HTTP instrumentations record, and scrub at the Collector if needed. - **It only covers supported libraries.** An in-house RPC client or an unusual driver will show up as a gap in the trace. That's your cue for a manual span or a custom instrumentation. ## Rollout checklist - Load the SDK before the app: `--require` for CommonJS, `--import` plus the loader hook (or `module.register()`) for ESM. - Check that framework spans actually appear: in an Express app, route-named server spans and handler spans. A trace of bare `GET` spans means the hook didn't load. - Set `OTEL_SERVICE_NAME` and a version attribute on every service. - Set `OTEL_TRACES_EXPORTER`, `OTEL_METRICS_EXPORTER` and `OTEL_LOGS_EXPORTER` explicitly, using `none` where you don't collect a signal. - Limit `OTEL_NODE_RESOURCE_DETECTORS` to the platforms you actually run on. - Disable noisy instrumentations (`net`, `dns`), and ignore health and readiness probes. - Flush on shutdown, and make sure the process still exits on `SIGTERM`. - Send data to a Collector, not straight to a vendor, so filtering, sampling and backend choice stay in configuration. - Choose a sampling strategy before the bill chooses one for you. When not to bother: a short-lived script, a single-process tool with no downstream calls, or a service whose vendor agent already gives you everything and that you have no plans to move. Everywhere else, the case for OTel is the one this post started with. The instrumentation belongs to you, and your backend is just a configuration setting.