HTTP range requests: resume transfers without stitching different files

First principles

HTTP range requests let a client ask for part of a selected representation, commonly to resume a large download. A request such as Range: bytes=1000-1999 asks for a byte interval. A server that supports the range may return 206 Partial Content with a Content-Range describing which bytes of the complete representation were sent. A server can ignore the range and return the full representation with 200 OK; clients must not assume every origin supports ranges.

The subtle problem is representation identity. Suppose a client downloads the first megabyte of a file, the file changes, then the client asks for the next megabyte by offset alone. Appending those bytes can produce a corrupt hybrid that never existed on the server. The range offset is meaningful only relative to the same selected representation. Use a strong validator such as a strong ETag with If-Range, or an appropriate last-modified validator when the server's semantics make it safe. If the validator still matches, the server may send the requested range. If it does not, the server sends the full current representation rather than a partial continuation.

Validators and range support are separate from integrity. A strong ETag is an opaque validator under HTTP semantics; it is not necessarily a cryptographic digest and should not be presented as one. For end-to-end integrity, compare a trusted checksum or signature for the completed artifact. Do not trust a checksum that arrives through the same compromised channel as the file unless its authenticity is established independently.

Range handling also needs boundaries. A response may use a different content encoding or multipart range format than the client expects. A 416 Range Not Satisfiable response can include Content-Range: bytes */length, which helps the client determine the current length. A resumed client must validate status, returned range, representation validator, and total length before appending. If any check fails, restart or preserve the partial file for diagnosis rather than silently combining bytes.

Runnable example

This curl example demonstrates the shape of a conditional range request. Replace the URL with a resource you are allowed to download. The first command stores headers separately; inspect the returned strong ETag before using it in the second request. Do not assume the example server supports ranges.

curl --fail --dump-header first.headers --output first.part \
  'https://example.invalid/archive.bin'

grep -i '^etag:' first.headers
curl --fail --range 1048576-2097151 \
  --header 'If-Range: "replace-with-the-strong-etag"' \
  --dump-header next.headers --output next.part \
  'https://example.invalid/archive.bin'

The placeholder host intentionally cannot serve a real artifact. Replace it only with a trusted test endpoint. Before appending next.part, confirm the response is 206, its Content-Range begins at the requested offset, and the validator corresponds to the saved prefix. If the server returns 200, it supplied a full representation: replace the partial file rather than append it. Production clients should parse headers structurally, not rely on case-sensitive grep expressions.

Contextual failure case

A desktop updater downloads an executable, stores one megabyte, loses its connection, and resumes at the next byte range. The release is published while the transfer is interrupted. The server returns bytes from the new release and the client appends them to the old prefix. The resulting file may be rejected by a signature check, or worse, a weak updater may execute a malformed artifact. The resume code tracked progress but not which immutable representation the progress belonged to.

Another failure occurs when a proxy transforms content encoding. Byte offsets refer to a representation; if one response is compressed and a later response is not, the client's offsets and local byte count may refer to different sequences. Clients that support ranges should define their accepted content coding and ensure validators identify the same selected representation. Intermediaries can affect what is selected, which is why cache keys, Vary, and validator semantics matter.

Range requests are not a substitute for retry safety on arbitrary actions. A range GET asks for a representation slice; retrying a payment POST is a separate semantic problem. Keep transfer continuation state separate from mutation idempotency keys. A corrupted or stale partial file should be discarded or quarantined, not “repaired” by guessing missing offsets.

Verification

Test a stable resource, a resource that changes between requests, a server that ignores Range, an invalid offset, and a response whose Content-Range does not match the requested bytes. Assert that the client appends only after validating 206, start/end bounds, total length, validator, and content coding. Test a 200 restart and a 416 response. Verify the final file against an authenticated checksum or signature where the artifact format provides one.

When diagnosing a failed download, capture status, range, validator, and transfer length without logging authorization headers or signed URLs. Reliable API clients covers validators and retries at the request level; resumable transfer adds a byte-identity invariant that must hold across every segment.

Exercise

Implement a downloader state record containing partial length, strong ETag, expected content coding, and destination path. Define the checks required before append. Simulate a server that changes ETag and returns a full 200, then prove the old partial bytes are replaced. Add a malformed 206 with the wrong start offset and assert that no bytes are appended. Explain why the ETag alone is not proof of cryptographic integrity.

Safety and caveats

Range support, validators, content negotiation, and proxy behavior vary by server and representation. Strong ETags are opaque validators, not necessarily hashes. A server's byte offsets may not match a client-side transformed representation. Treat partial artifacts as untrusted until complete integrity checks pass, and do not use test or production download commands against resources without authorization.