Observability¶
Metrics, tracing, and error tracking each reach the kit through a small
structural protocol instead of a concrete SDK import.
AsyncMetricsInterceptor, AsyncTracingInterceptor, and
AsyncSentryInterceptor (see Interceptors) call these
protocols and nothing else, so the core package never depends on
prometheus-client, opentelemetry, or sentry-sdk — you pay only for what
you install, and you can swap in your own adapter, or a test double, without
subclassing anything.
The protocols¶
| Protocol | Shape | Ships as |
|---|---|---|
GrpcServerMetricsProtocol |
record_request(service, method, status, grpc_code, duration) -> None |
GrpcServerMetrics ([metrics]) |
TracerProtocol |
start_as_current_span(name) -> SpanProtocol |
An adapted OTel tracer |
SpanProtocol |
set_attribute(key, value), record_exception(exc), context manager |
An OTel span |
ErrorReporterProtocol |
isolate(), capture_exception(exc), add_breadcrumb(...), set_tags(**tags) |
SentrySdkAdapter ([sentry]) |
All four import straight from the interceptors package:
from grpc_server_kit.aio.interceptors import (
ErrorReporterProtocol, GrpcServerMetricsProtocol, SpanProtocol, TracerProtocol,
)
SpanProtocol and TracerProtocol are @runtime_checkable, so a custom
adapter can be sanity-checked with isinstance(adapter, TracerProtocol).
Pass None for any of them — the default — and the corresponding
interceptor no-ops. That's exactly how each interceptor behaves when its
extra isn't installed or observability is disabled, so "no backend
configured" is a first-class, always-safe state rather than an error path.
Prometheus metrics¶
GrpcServerMetrics (grpc_server_kit.observability.metrics, [metrics]
extra) defines exactly two, frozen metrics:
| Metric | Type | Labels |
|---|---|---|
grpc_requests_total |
Counter | service, method, status (success/error), grpc_code |
grpc_request_duration_seconds |
Histogram | service, method |
from grpc_server_kit.observability.metrics import get_grpc_server_metrics
metrics = get_grpc_server_metrics(prefix="my_service") # my_service_grpc_requests_total, ...
get_grpc_server_metrics() caches by prefix — registering the same
Prometheus metric name twice raises inside prometheus_client, so repeated
calls for the same prefix hand back the existing instance — and raises
ValueError if you ask for that prefix again with different histogram
buckets, since silently keeping the old buckets would record real
latencies into the wrong histogram with no warning at all.
AsyncMetricsInterceptor catches and logs any exception record_request
itself raises, so a metrics backend hiccup never fails the RPC being
measured.
Tracing attributes and propagation¶
AsyncTracingInterceptor opens one span per RPC, named after the method
(e.g. pkg.Service/Method), and sets the OpenTelemetry semantic-convention
attributes:
rpc.system="grpc"rpc.service,rpc.method— parsed from the full method namerpc.grpc.status_code— the numeric status code, set once the RPC (or the full response stream) finishesrequest.id— thex-request-idmetadata value, when present
It records exceptions selectively. For an RPC that ends with a server-class
status, the interceptor looks for an original exception chained onto the
AbortError (via __cause__ or __context__) and records that on the
span — recovering the real bug even though, by the time the outcome reaches
this interceptor, an inner AsyncExceptionHandlerInterceptor has already
turned it into a clean abort. A deliberate abort with a client-class status,
or one raised with nothing being handled, has no exception to recover and is
not recorded.
This interceptor is a lightweight seam: it does not extract an incoming
traceparent header, so a span it opens starts a new trace rather than
continuing the caller's. For full distributed-tracing propagation, call
instrument_aio_server() once at startup — a thin wrapper over
OpenTelemetry's own GrpcAioInstrumentorServer ([tracing] extra):
from grpc_server_kit.aio.observability import instrument_aio_server
instrument_aio_server() # extracts incoming trace context; call once, before serving
The two compose: instrument_aio_server() handles context propagation at the
transport level, and AsyncTracingInterceptor adds the kit's semconv
attributes and selective exception recording on top.
Error tracking¶
AsyncSentryInterceptor runs every RPC inside reporter.isolate() — a
forked, isolated scope — before setting tags (grpc_method, request_id)
and a breadcrumb. Isolation matters because asyncio tasks otherwise share
process-global SDK state: without a fresh scope per RPC, tags and breadcrumbs
from concurrent calls would bleed into each other's error reports.
It only captures raw Exceptions that reach it — grpc.aio.AbortError and
grpc.RpcError are re-raised untouched, never captured. Combined with the
canonical chain order, this is why
Sentry's position matters: placed inside (closer to the handler than)
AsyncExceptionHandlerInterceptor, it sees the exact exception the handler
raised; placed outside, it would only ever see the AbortError the
exception handler already converted that exception into, and would capture
nothing, ever. An optional capture_filter narrows which exceptions get
reported — the Dishka bundle wires one that reuses the exception handler's
own status map, so only exceptions mapping to a server-class status
(INTERNAL, UNAVAILABLE, ...) are captured, keeping expected client errors
like a mapped ValueError out of the error tracker.
SentrySdkAdapter (grpc_server_kit.observability.sentry, [sentry]
extra) adapts the global sentry_sdk module API to ErrorReporterProtocol;
it assumes sentry_sdk.init(...) has already run.
SDK initialization is the application's job¶
None of the three seams initializes its own backend, and that's deliberate:
sentry_sdk.init(...), OpenTelemetry TracerProvider setup, and starting a
Prometheus /metrics HTTP server are process-wide concerns with their own
lifecycle, unrelated to any single gRPC server instance. Your application
does this once at startup, then hands the resulting adapters to the
interceptors — directly, or via the
Dishka providers:
import sentry_sdk
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from prometheus_client import start_http_server
sentry_sdk.init(dsn="...") # before building any interceptor
trace.set_tracer_provider(TracerProvider())
start_http_server(9090) # scrape target for grpc_requests_total etc.
Health checks are excluded by default¶
All four observability interceptors — metrics, request logger, tracing,
sentry — default their skip_methods to the gRPC Health Check/Watch
methods, so routine liveness/readiness polling doesn't inflate request logs,
latency histograms, or trace and error volume. Pass your own skip_methods=
if you'd rather observe health traffic too — see
Interceptors for exactly what skipping does
under the hood.