Skip to content

Commit dc1ca93

Browse files
cmdcolinclaude
andcommitted
docs: point at @gmod/range-cache-filehandle for HTTP reads
A whole-reference query on ce#1000 issues 545 filehandle reads over a 141 KB file, and the README said nothing about what that costs against a bare RemoteFile. docs/api.md already assumed a byte-range-caching filehandle without naming one that exists. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent eb529bc commit dc1ca93

2 files changed

Lines changed: 27 additions & 3 deletions

File tree

README.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -137,6 +137,28 @@ reason to reach for either is a host that runs several worker contexts, since
137137
one pool serves a context rather than a machine.
138138
[docs/workers.md](docs/workers.md) has the measurements.
139139

140+
## Reading over HTTP
141+
142+
An indexed query reads a file as many small byte ranges — a whole-reference
143+
query on a 141 KB test file issues 545 of them. A bare `RemoteFile` turns each
144+
one into its own range request, so put a byte-range cache underneath:
145+
146+
```js
147+
import { RemoteFileWithRangeCache } from '@gmod/range-cache-filehandle'
148+
149+
const cram = new IndexedCramFile({
150+
cramFilehandle: new RemoteFileWithRangeCache(url),
151+
index: new CraiIndex({
152+
filehandle: new RemoteFileWithRangeCache(`${url}.crai`),
153+
}),
154+
})
155+
```
156+
157+
It serves reads from a 256 KiB chunk grid and coalesces the chunks a read is
158+
missing into one request per contiguous run, so those 545 reads become a handful
159+
of requests. It also threads the `AbortSignal` below, which is what makes the
160+
next section worth anything over a network.
161+
140162
## Cancelling a query
141163

142164
Pass an `AbortSignal` and the query stops decoding and drops the fetch it has in

docs/api.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -105,9 +105,11 @@ controller.abort() // `records` rejects with an AbortError
105105
```
106106

107107
This is worth more than "index reads are short" suggests. A byte-range-caching
108-
filehandle coalesces adjacent reads into one request, so a small viewport over
109-
deep data becomes a single multi-megabyte fetch — exactly the thing you want to
110-
drop when the user pans away.
108+
filehandle such as
109+
[@gmod/range-cache-filehandle](https://github.com/GMOD/range-cache-filehandle)
110+
coalesces adjacent reads into one request, so a small viewport over deep data
111+
becomes a single multi-megabyte fetch — exactly the thing you want to drop when
112+
the user pans away.
111113

112114
**Aborting your query never fails anyone else's.** Concurrent queries share two
113115
things in a `CramFile`: the parsed `.crai`, and each decoded slice in the record

0 commit comments

Comments
 (0)