Instrument backend requests with private optional telemetry #11

New issue

Open

opened 2026-11-01 20:11:31 +00:00 by viacheslavshuka · 0 comments

viacheslavshuka commented

2026-11-01 20:11:31 +00:00

Owner

Operators need request diagnostics without exposing participant data or making application availability depend on a collector.

Scope

Use native FastAPI OpenTelemetry traces and metrics. Correlate existing structured application logs through trace and span identifiers. Accept a configurable OTLP endpoint without bundling storage or dashboards.

Acceptance criteria

With no configured destination, external telemetry export stays disabled and application JSON logging remains available.
With a configured endpoint, representative request traces and metrics reach that endpoint. Application logs include correlation identifiers when a trace exists.
Logging uses structlog with cache_logger_on_first_use=True, early level filtering, orjson.dumps, and structlog.BytesLoggerFactory.
Sampling, batching, queue limits, and exporter timeouts are explicit. A slow or unavailable collector cannot fail an ordinary application request or create an unbounded queue.
Attributes omit credentials, cookies, tokens, secret links, raw request bodies, rejected raw input, and free-text material metadata. Question text, answers, and revealing fragments cannot appear in logs or telemetry.
Route labels use stable route patterns rather than unbounded user-specific URLs. Database instrumentation, if added, applies equivalent SQL and attribute sanitization.
Shutdown handles pending telemetry within bounded resource lifetimes. Diagnostics report exporter problems without repeatedly logging sensitive payloads.
Measurements report instrumentation overhead under a documented load. No unsupported negligible-overhead claim or fixed performance gate is added.

Validation

Use a local disposable OTLP receiver to inspect exported attributes. Inject secrets and spoiler-like test strings into requests and verify their absence. Exercise no destination, unavailable receiver, full queue, and shutdown while requests continue.

Dependencies

#6: Start the FastAPI runtime with owned database resources. Instrumentation needs the actual runtime and resource lifecycle.

#16: Run durable PostgreSQL jobs with atomic effects

Operators need request diagnostics without exposing participant data or making application availability depend on a collector. ## Scope Use native FastAPI OpenTelemetry traces and metrics. Correlate existing structured application logs through trace and span identifiers. Accept a configurable OTLP endpoint without bundling storage or dashboards. ## Acceptance criteria - With no configured destination, external telemetry export stays disabled and application JSON logging remains available. - With a configured endpoint, representative request traces and metrics reach that endpoint. Application logs include correlation identifiers when a trace exists. - Logging uses structlog with `cache_logger_on_first_use=True`, early level filtering, `orjson.dumps`, and `structlog.BytesLoggerFactory`. - Sampling, batching, queue limits, and exporter timeouts are explicit. A slow or unavailable collector cannot fail an ordinary application request or create an unbounded queue. - Attributes omit credentials, cookies, tokens, secret links, raw request bodies, rejected raw input, and free-text material metadata. Question text, answers, and revealing fragments cannot appear in logs or telemetry. - Route labels use stable route patterns rather than unbounded user-specific URLs. Database instrumentation, if added, applies equivalent SQL and attribute sanitization. - Shutdown handles pending telemetry within bounded resource lifetimes. Diagnostics report exporter problems without repeatedly logging sensitive payloads. - Measurements report instrumentation overhead under a documented load. No unsupported negligible-overhead claim or fixed performance gate is added. ## Validation Use a local disposable OTLP receiver to inspect exported attributes. Inject secrets and spoiler-like test strings into requests and verify their absence. Exercise no destination, unavailable receiver, full queue, and shutdown while requests continue. ## Dependencies - [#6: Start the FastAPI runtime with owned database resources](https://code.viacheslavshuka.com/viacheslavshuka/quorum/issues/6). Instrumentation needs the actual runtime and resource lifecycle. ## Related work - [#16: Run durable PostgreSQL jobs with atomic effects](https://code.viacheslavshuka.com/viacheslavshuka/quorum/issues/16)

viacheslavshuka added the

labels

2026-11-01 20:13:07 +00:00

viacheslavshuka added a new dependency

2026-11-01 20:15:35 +00:00

#6 Start the FastAPI runtime with owned database resources

viacheslavshuka referenced this issue

2026-11-01 20:15:37 +00:00