Building HTTP and gRPC clients for production¶

An orders service calls a warehouse and a payment service, and either can be slow or not answer at all. The first instinct is to retry.

But a retried stock read is harmless, while a retried payment can charge the buyer twice, and the client cannot tell the two apart.

We will configure its HTTP and gRPC clients, break their dependencies and count the requests that actually reach them.

Create clients when the application starts¶

clientwright attaches time limits and a retry policy to HTTPX, and build('httpx', ...) still returns a native httpx.AsyncClient:

from clientwright import (
    AdapterDeps, ClientConfig, RetryConfig, TimeoutConfig, build,
)


def http_client(name, base_url, *, metrics=None):
    return build(
        'httpx',
        ClientConfig(
            service_name=name,
            base_url=base_url,
            timeout=TimeoutConfig(total=2, connect=0.3, read=0.5),
            retry=RetryConfig(),
            circuit_breaker=None,
            on_unsupported='strict',
        ),
        AdapterDeps(metrics=metrics),
    )

service_name labels the client's metrics and logs. RetryConfig() already means three attempts, the first included, with 100 ms × 2 backoff, ±20% jitter, Retry-After support and a retry budget of 0.1. A circuit breaker is on by default (five failures, 60 seconds); circuit_breaker=None turns it off until we configure it below. on_unsupported='strict' makes a setting the adapter cannot apply fail at construction.

Create both clients in the application's startup scope and close them after active handlers finish:

from contextlib import asynccontextmanager


@asynccontextmanager
async def outbound_clients(inventory_url, payments_url):
    async with (
        http_client('inventory', inventory_url) as inventory,
        http_client('payments', payments_url) as payments,
    ):
        yield inventory, payments

Opening them per request would throw away connection reuse, the retry budget and breaker state.

Give a stock read a total time limit¶

Inventory answers GET /stock/{sku} with {"sku": "sku-42", "available": 7}:

async def fetch_stock(client, sku):
    response = await client.get(f'/stock/{sku}')
    response.raise_for_status()
    return response.json()['available']

The client allows 0.3 seconds to connect, 0.5 seconds between response chunks and two seconds for the whole call, attempts and backoff included. The HTTPX read timeout limits a pause, not the response: in the deadline lab, a server sending a byte every 80 ms completes under a 300 ms read timeout, while TimeoutConfig(total=0.35, read=0.3) stops it with DeadlineExceededError.

To share the incoming request's time between inventory and payments, give the clients AdapterDeps(deadline_source=AmbientDeadlineSource()) and set a BudgetContext with use_budget(), as in the deadline article. Two fresh seconds per client are not a two-second budget for the order.

Retry a payment only when the server can deduplicate it¶

Payments saves a charge and then returns 503. Retrying is safe only if the payments API returns the saved result for the same Idempotency-Key and payload, and rejects the key with a different payload. An order has one charge, so the key comes from the order ID:

from clientwright.adapters.httpx import IDEMPOTENT_EXTENSION


async def charge(client, order_id, amount_minor):
    response = await client.post(
        '/payments',
        json={'order_id': order_id, 'amount_minor': amount_minor},
        headers={'Idempotency-Key': f'charge-{order_id}'},
        extensions={IDEMPOTENT_EXTENSION: True},
    )
    response.raise_for_status()
    return response.json()['payment_id']

IDEMPOTENT_EXTENSION lets clientwright retry the POST; the header lets the server recognize the repeat. The retry lab checks three versions against a local service:

A second charge() for the same order returns the same payment ID. The lab deduplicates in memory and sequentially; a real API needs durable storage and atomic handling of concurrent duplicates.

Respect the requested delay and limit extra traffic¶

With the default policy, the first retry waits 80 to 120 ms and the second 160 to 240 ms. Retry-After: 1 replaces that with one second. If the requested delay cannot fit the remaining deadline, as with Retry-After: 60 in the lab, the client returns the 503 at once after a single request.

The retry budget lives per origin (scheme, host and port) inside the client. A fresh origin starts with ten tokens, the bucket holds at most ten, and every call adds 0.1. The budget lab makes 50 sequential calls to an endpoint that always returns 503, with the breaker off:

That is 50 first attempts, 10 retries from the initial tokens and 4 earned by the calls. Each replica and each new client starts with a full bucket, and an outer retry loop around this client multiplies attempts again: the adapter lab turns three outer calls into nine requests. Retry in one layer only.

Keep a payments outage from blocking inventory¶

Separate clients already have separate breakers. A shared client that calls several origins by full URL needs its breaker keyed by origin, which is the default:

from clientwright import CircuitBreakerConfig


def shared_http_client():
    return build('httpx', ClientConfig(
        service_name='shop-backends',
        timeout=TimeoutConfig(total=2),
        retry=RetryConfig(),
        circuit_breaker=CircuitBreakerConfig(
            fail_threshold=3,
            recovery_timeout=10,
            half_open_max_calls=1,
        ),
    ))

In the breaker lab, three failed calls to payments make nine attempts, and the fourth raises CircuitOpenError without a request, while a stock read on another origin still returns 200. The breaker counts consecutive failed calls with status 500 or above; any other outcome, 400 and 429 included, resets the count. With a test clock, the lab also checks recovery from open through half-open to closed. One breaker check wraps the whole retry loop:

---
config:
  theme: default
  look: classic
  flowchart:
    useMaxWidth: false
    wrappingWidth: 150
    padding: 12
    nodeSpacing: 24
    rankSpacing: 32
---
flowchart TD
    accTitle: One HTTP call, several possible attempts
    accDescr: The breaker is checked once per call and records the final outcome. Every attempt shares the call deadline and needs a retry token.
    A["Call deadline"]
    B["Circuit breaker check"]
    C["Request attempt"]
    D["Assess result"]
    E["Retryable, attempts left, fits deadline, token available?"]
    F["Delay with jitter"]
    G["Record final outcome; return"]
    A --> B --> C --> D --> E
    E -->|"retry"| F --> C
    E -->|"no retry"| G

Two failing backends behind one gateway origin need a finer key, such as CircuitKey.ORIGIN_ROUTE with ROUTE_EXTENSION, or separate clients.

Give gRPC an explicit list of repeatable methods¶

The gRPC inventory has GetStock (a read), Reserve (a write) and Export (a stream). The gRPC lab binds them in InventoryStub over raw bytes; generate yours from protobuf. grpc-client-kit allows retries only for GetStock:

import grpc
from grpc_client_kit import (
    RetryConfig as GrpcRetry,
    TimeoutConfig as GrpcTimeout,
    build_interceptors,
)


def inventory_chain():
    return build_interceptors(
        timeout=GrpcTimeout(default=1),
        retry=GrpcRetry(
            max_attempts=3,
            initial_backoff=0.05,
            retryable_codes={grpc.StatusCode.UNAVAILABLE},
            idempotent_methods={'/shop.Inventory/GetStock'},
        ),
    )

Without an allowlist, the kit's default RetryConfig retries every unary method on UNAVAILABLE and RESOURCE_EXHAUSTED. But UNAVAILABLE does not prove the handler did nothing: the lab's Reserve changes stock and then returns it, and the default policy records three reservations. With the allowlist there is one, and the caller gets the error, while GetStock still retries and returns b'7'. We also leave out RESOURCE_EXHAUSTED: a server out of capacity should not get extra traffic. The one-second timeout covers the whole RPC; given 300 ms, a slow first attempt leaves the second only the remainder, and it ends with DEADLINE_EXCEEDED.

Share channels only between compatible clients¶

An audit reader uses the same address but must make one attempt:

from contextlib import asynccontextmanager
from grpc_client_kit import ChannelPool, GrpcClient, GrpcClientConfig


@asynccontextmanager
async def grpc_clients(target):
    config = GrpcClientConfig(
        target=target,
        insecure=True,  # Local lab; configure credentials for a TLS deployment.
        options=[('grpc.enable_retries', 0)],
    )
    async with ChannelPool() as pool:
        orders = GrpcClient(
            InventoryStub, config, pool,
            interceptor_factory=lambda target: inventory_chain(),
        )
        audit = GrpcClient(
            InventoryStub, config, pool,
            interceptor_factory=lambda target: build_interceptors(
                timeout=GrpcTimeout(default=1), retry=None,
            ),
        )
        yield orders, audit

interceptor_factory creates and caches a chain per target, which keeps channel identity stable. The pool separates channels by security, credentials, options, compression and interceptor chain, not by address alone; the identity lab checks reuse and separation. ChannelPool owns the channels and closes them on exit; leaving a GrpcClient context does not touch them. grpc.enable_retries=0 leaves retries to the interceptor, so native gRPC retries do not multiply them. For the audit client, retry=None removes the retry interceptor; an empty idempotent_methods would not mean "retry nothing".

The gRPC client differs from the HTTP one in three ways:

• grpc-client-kit 0.4.0 has no retry budget.
• Its breaker is kept per method and sits inside the retry loop, counting attempts: with a threshold of two, a breaker in the lab blocks the third attempt of the first failing call.
• Restarting Export after items 1, 2 can produce 1, 2, 1, 2, 3. The lab checks both this opt-in behavior and the default failure; keep stream retries off until the protocol can resume or deduplicate. How interceptors see a stream's outcome is covered in a separate article.

Count attempts as well as successful calls¶

For a stock endpoint that returns 503 twice and then succeeds, this returns (7, 1, 3): quantity, logical calls and attempts.

from clientwright.core.testing import RecordingMetrics


async def observe_stock(base_url):
    metrics = RecordingMetrics()
    async with http_client('inventory', base_url, metrics=metrics) as client:
        available = await fetch_stock(client, 'sku-42')
    return available, len(metrics.calls), len(metrics.attempts)

RecordingMetrics is the library's test recorder; production passes its own through AdapterDeps(metrics=...). One successful call can hide three requests. The call duration ends at the response headers, and reading the body is recorded as a separate metric.

Choose policies for the operations you actually call¶

Start with a total time limit and the list of operations that are safe to repeat, then tune the retry budget and the breaker against a dependency that fails on purpose. clientwright does this for HTTP while keeping native clients, and grpc-client-kit does it for gRPC channels and interceptors.

Examples and labs¶

The snippets come from the labs and were checked with Python 3.13, clientwright==0.5.0, httpx==0.28.1, grpc-client-kit==0.4.0 and grpcio==1.84.0 against local HTTP and gRPC servers. They test client behavior, not performance, TLS, proxies or payment storage.

• Native HTTP clients, metrics and nested retries
• Read timeout versus total deadline
• Retry-After, payment deduplication and body replay
• Retry budget and server request counts
• Circuit breakers per origin
• gRPC retry safety, deadlines and streams
• gRPC channel reuse and isolation