perf(blockstm): batch MVData index updates

[songgaoye]

Description

Optimize BlockSTM MVData index updates by batching copy-on-write B-tree insertions.

Previously, each new key called GetOrDefault independently, cloning, freezing, and publishing the B-tree once per key. This PR adds BatchGetOrDefault, which inserts all missing entries through a single copy-on-write update and CAS publication.

The MVData consolidation and pre-estimate paths now use the batch operation when adding multiple indexes. The original single-key path is retained because it is faster for one item.

Additional changes:

• Handle duplicate and concurrently overlapping batch keys consistently.
• Avoid publishing a new snapshot when every key already exists.
• Add a dedicated batch_get_or_default latency metric.
• Add unit tests, concurrency tests, and comparative benchmarks.

Benchmarks

Apple M4, five samples per case with -benchtime=500ms. “Individual” represents the previous per-key algorithm.

Keys	Individual median	Batch median	Speedup
10	1,681 ns/op	938 ns/op	1.79x
100	34,731 ns/op	16,155 ns/op	2.15x
1,000	538,921 ns/op	227,531 ns/op	2.37x

Keys	Individual B/op	Batch B/op	Individual allocs/op	Batch allocs/op
10	4,752	1,936	70	27
100	96,384	18,752	1,211	197
1,000	1,556,300	183,377	18,688	1,853

Benchmark command:

go test ./internal/blockstm/tree \
  -run '^$' \
  -bench '^BenchmarkBTreeBatchGetOrDefault$' \
  -benchmem \
  -benchtime=500ms \
  -count=5

Testing

go test ./internal/blockstm/tree ./internal/blockstm -count=1
go test -race ./internal/blockstm/tree ./internal/blockstm -count=1

Closes: #XXXX

[codecov]

Codecov Report

❌ Patch coverage is 98.30508% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 65.05%. Comparing base (dce173a) to head (c5ac0f3).

Files with missing lines	Patch %	Lines
internal/blockstm/tree/metrics.go	85.71%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #26773      +/-   ##
==========================================
+ Coverage   64.96%   65.05%   +0.08%     
==========================================
  Files         785      785              
  Lines       55404    55454      +50     
==========================================
+ Hits        35994    36073      +79     
+ Misses      19410    19381      -29     

Files with missing lines	Coverage Δ
internal/blockstm/mvdata.go	95.67% <100.00%> (-0.70%)	⬇️
internal/blockstm/tree/btree.go	67.44% <100.00%> (+15.71%)	⬆️
internal/blockstm/tree/metrics.go	86.56% <85.71%> (+71.56%)	⬆️

... and 8 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[dianab-cl]

@greptile review

[greptile-apps]

Greptile Summary

The PR optimizes BlockSTM index maintenance by adding atomic batched copy-on-write B-tree insertion.

• Consolidation and pre-estimation now collect index entries and update them in batches, retaining the single-entry fast path.
• BatchGetOrDefault handles duplicates, concurrent overlap, CAS retries, and batches containing only existing keys.
• Adds latency instrumentation, correctness and concurrency tests, comparative benchmarks, and a changelog entry.

Confidence Score: 5/5

The PR appears safe to merge; no actionable correctness, concurrency, security, or repository-rule issue was identified.

The batched operation preserves stored-value identity across duplicates and concurrent CAS retries, while BlockSTM readers already tolerate the temporary separation between data publication and bitmap-index updates.

Important Files Changed

Filename	Overview
internal/blockstm/tree/btree.go	Adds a CAS-based batched get-or-insert operation that publishes one copy-on-write snapshot and recomputes results correctly after contention.
internal/blockstm/mvdata.go	Replaces repeated per-key index insertion with collected batch updates while preserving bitmap updates and the single-key fast path.
internal/blockstm/tree/btree_test.go	Covers missing and existing entries, duplicate keys, overlapping concurrent batches, snapshot reuse, and comparative benchmarks.
internal/blockstm/tree/metrics.go	Registers dedicated latency instrumentation for the new batch operation.
internal/blockstm/tree/metrics_test.go	Verifies successful initialization of the new histogram.

Sequence Diagram

sequenceDiagram
    participant M as MVData
    participant B as BTree
    participant S as Snapshot
    M->>B: BatchGetOrDefault(entries)
    loop Until CAS succeeds
        B->>S: Load current snapshot
        B->>B: Copy once on first missing key
        B->>B: Resolve existing and duplicate keys
        B->>B: Initialize and insert missing entries
        alt Every key exists
            B-->>M: Return stored entries
        else Snapshot updated
            B->>B: Freeze updated tree
            B->>B: CompareAndSwap(snapshot, updated)
        end
    end
    M->>M: Set transaction bit on returned indexes

Loading

_{Reviews (1): Last reviewed commit: "add metrics test" | Re-trigger Greptile}