feat: Initial support for HTTP range requests by lcian · Pull Request #481 · getsentry/objectstore

lcian · 2026-05-26T14:47:49Z

Adds standard HTTP range requests (RFC) support for single-object GET operations.

Spec vs implementation:

This implementation only supports byte ranges. The spec doesn't specifically talk about bytes, so in theory it admits ranges of other "types", which just don't make sense for us.
This implementation only supports Single part ranges, meaning stuff like Range: bytes=0-1023. The spec allows for Multipart ranges too, i.e. fetching multiple ranges in a single request, which we don't support. We can add it in the future if needed.
We should probably implement Conditional range requests in a follow-up PR, as they're pretty much essential to guarantee that clients read consistent data (if the object is swapped between range reads, we would otherwise serve chunks from 2 potentially different objects). Note that Symbolicator doesn't use these, so we should also update Symbolicator and any other user that uses range reads to actually use these to prevent random failures/data inconsistency. (maybe in practice this is fine as data that Symbolicator fetches cannot really change due to the way it's written, I don't know yet)
Looking for feedback on whether we think we need this now or not.

Implementation details:

All get_object paths now take Option<ByteRange> and return ContentRange alongside the payload. This seems to be the way to implement this with the least code duplication. Otherwise we would have to introduce a get_object_range, implement it for every backend and wrappers of the backends, etc.
Bigtable: we know that the objects there are small, so we just return the full payload. This behavior is compliant with the spec.
GCS/S3: objectstore-service reserializes the Range header and forwards it to the backend. On the response path, we proxy back the content-range provided by the backend. TODO:
Not supported for batch operations.
Not implemented in clients for now because Symbolicator (which would be the only user atm) doesn't use our client. We can easily implement it when needed.

Close FS-363

Thread `Option<ByteRange>` through all `get_object` paths and return `ContentRange` in every response so the HTTP handler can choose between 200, 206, and 416 with correct headers. Backends that store objects remotely (GCS, S3) forward the Range header and parse the response. Local-filesystem uses seek-based partial reads. In-memory and Bigtable ignore the range and always return the full payload with 200, which is RFC-compliant.

- Parse Range header unit case-insensitively per RFC 9110 (e.g. `Bytes=0-4` now works instead of being ignored) - Multi-range requests fall back to full 200 response instead of returning 400, matching RFC semantics for unsupported features

- Only set explicit Content-Length when content_range.total > 0, avoiding a Content-Length: 0 regression when metadata.size is unknown. - Use response Content-Length (not metadata.size) in 206 fallback paths for GCS and S3 so Content-Length matches the body if Content-Range is missing from a 206 response.

Reduce test boilerplate by consolidating 20 unit tests into 7 with identical coverage. Remove redundant integration test already covered by unit tests. Use HeaderValue::from_static, delegate to Display impl, and pass ByteRange directly instead of an intermediate HeaderMap.

Tests added to main after this branch diverged still used the old get_object signature (1 arg, 2-tuple return). Update them to pass None for the range parameter and destructure the 3-tuple.

…r S3 partial size bytes=-0 (last 0 bytes) is semantically meaningless and should be rejected with RangeError::Invalid rather than parsed as Last(0). For S3 206 responses, Content-Length reflects the partial body length, not the full object size. Read metadata.size from the Content-Range total field instead so the metadata accurately represents the full object. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

lcian · 2026-06-01T15:16:47Z

+                    match total {
+                        Some(total) => return Err(Error::RangeNotSatisfiable { total }),
+                        None => {
+                            return Err(Error::Generic {


Seems like the GCS emulator was buggy and not returning the header for 406 responses.
Fixed it here getsentry/google-cloud-storage-testbench#3 and upstreaming it here googleapis/storage-testbench#802.

lcian · 2026-06-01T15:17:57Z

The S3 backend doesn't really work right now, we need #442, that's why I didn't add tests.

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 0aa575c. Configure here.}

matt-codecov

now that i'm seeing it, i don't love how inlining range reads in our regular GET operations looks, but i also don't love duplicating all the GET code and just handling this one header and two response codes differently.

also, this is elaborated more in another comment, but it would be nice to add Accept-Ranges: bytes headers only for objects that live in backends that support ranges. but i don't have a clear idea of how to do that cleanly today

i thought about actionable feedback for about an hour and didn't come up with anything so i'm accepting to unblock, but please give them some thought on your own. re-request review if you think you have good solutions

matt-codecov · 2026-06-03T00:41:46Z

+                // We are free to decide whether we want to fall back to the whole object or error.
+                // Per RFC 9110:
+                // > A server that supports range requests MAY ignore or reject a Range header
+                //   field that contains an invalid ranges-specifier [...]


to me, this RFC excerpt sounds a bit narrower in what it actually allows. if we advertise support for range reads and a client sends us a well-formed range specifier, it doesn't permit us to decide not to honor that range. if we want full spec compliance, i think we have to impl multi-range support and also build range reads ourselves atop backends that don't support it natively

a workable middle ground would be to only add Accept-Ranges: bytes to responses for objects stored in backends that support it. clients can HEAD req a specific object to learn in advance whether range reads are supported for it. any scenario i can think of can do what it needs to do that way:

a client that wants to download a huge file can check whether to download in parallel chunks (if range-reads are supported) or in a single request (otherwise)

a client that wants to resume an interrupted download can check whether to append (if range-reads are supported) or replace (otherwise) the incomplete download with the retry response.

a client that wants to download a specific file from within an archive (based on some out-of-band index query) can blindly attempt a range-read and just deal with it locally if the range wasn't respected

if they want multiple files within the archive, they may want to check with HEAD req and DL in parallel, or maybe they will try multi-range and can similarly just deal with it locally if the range wasn't respected. idk

unfortunately i'm not sure if there is a great way to do that. it's breaking abstraction to give the service impl a way to manipulate HTTP headers. an ugly kludge would be to add a range_read bool to Metadata or to return it on the side

if we did the backend refactor idea i wrote about, it might be cleaner. StorageService would absorb TieredBackend and could expose a BackendDescriptor or something that can carry this information. idk

matt-codecov · 2026-06-03T01:32:18Z

    Xt(id): Xt<ObjectId>,
+    headers: HeaderMap,
 ) -> ApiResult<Response> {
+    let byte_range = match headers.get(http::header::RANGE) {


could this all be an extractor?

This comment was marked as resolved.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as outdated.

Sign in to view

This comment was marked as spam.

Sign in to view

lcian changed the title ~~feat: Support HTTP Range requests for single-object GETs~~ feat: Range requests May 27, 2026

lcian changed the title ~~feat: Range requests~~ feat(server+service): Support HTTP range requests May 27, 2026

lcian added 9 commits May 29, 2026 10:38

improve

e1fa227

improve

d0fbeb6

improve

1c5f6cb

improve

e8dd7f7

fix: Update new tiered.rs tests for range request API changes

d12a378

Tests added to main after this branch diverged still used the old get_object signature (1 arg, 2-tuple return). Update them to pass None for the range parameter and destructure the 3-tuple.

lcian force-pushed the lcian/feat/range-requests branch from 85de81c to d12a378 Compare May 29, 2026 08:41

lcian and others added 4 commits May 29, 2026 13:37

Merge branch 'main' into lcian/feat/range-requests

a379844

improve

d03594c

improve

a1c11fd

lcian changed the title ~~feat(server+service): Support HTTP range requests~~ feat: Support HTTP range requests May 29, 2026

lcian changed the title ~~feat: Support HTTP range requests~~ feat: Basic support for HTTP range requests May 29, 2026

lcian changed the title ~~feat: Basic support for HTTP range requests~~ feat: Initial support for HTTP range requests May 29, 2026

lcian added 6 commits May 29, 2026 15:57

improve

1f6caf0

improve

02743ad

improve

7d629c8

improve range types

3fb6975

improve range types a bit more

4025c2a

improve endpoint

676cf89

lcian added 2 commits June 1, 2026 15:41

improve s3

83a1899

fix

61e7cec

lcian force-pushed the lcian/feat/range-requests branch from 5e153aa to 61e7cec Compare June 1, 2026 14:07

lcian added 2 commits June 1, 2026 16:08

fix

ee6fa4a

add a test

8d40bb4

lcian marked this pull request as ready for review June 1, 2026 14:24

lcian requested a review from a team as a code owner June 1, 2026 14:24

sentry Bot reviewed Jun 1, 2026

View reviewed changes

Comment thread objectstore-service/src/backend/s3_compatible.rs Outdated

cursor Bot reviewed Jun 1, 2026

View reviewed changes

Comment thread objectstore-service/src/backend/s3_compatible.rs Outdated

sentry Bot reviewed Jun 1, 2026

View reviewed changes

Comment thread objectstore-service/src/backend/gcs.rs

sentry Bot reviewed Jun 1, 2026

View reviewed changes

Comment thread objectstore-service/src/backend/bigtable.rs

lcian force-pushed the lcian/feat/range-requests branch 2 times, most recently from 93e1429 to 8d40bb4 Compare June 1, 2026 14:50

fix

a47e329

sentry Bot reviewed Jun 1, 2026

View reviewed changes

Comment thread objectstore-service/src/backend/local_fs.rs Outdated

add tests

5eb797c

This comment was marked as outdated.

Sign in to view

reject invalid bounds

302b1c4

lcian commented Jun 1, 2026

View reviewed changes

sentry Bot reviewed Jun 2, 2026

View reviewed changes

Comment thread objectstore-service/src/backend/tiered.rs Outdated

improve

0e21c59

lcian force-pushed the lcian/feat/range-requests branch from 2b78ff4 to 0e21c59 Compare June 2, 2026 11:03

lcian added 3 commits June 2, 2026 13:04

improve

20034e1

clippy

271d62c

improve

aaa8221

This comment was marked as outdated.

Sign in to view

improve

0aa575c

cursor Bot reviewed Jun 2, 2026

View reviewed changes

Comment thread objectstore-service/src/backend/in_memory.rs

matt-codecov approved these changes Jun 3, 2026

View reviewed changes

Uh oh!

Conversation

lcian commented May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as resolved.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as outdated.

Uh oh!

This comment was marked as spam.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

lcian Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

lcian Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

matt-codecov left a comment

Choose a reason for hiding this comment

Uh oh!

matt-codecov Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

matt-codecov Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

lcian commented May 26, 2026 •

edited

Loading