Skip to content

GH-47417: [C++] Fix JSON parser losing nulls in a null-typed list - #51108

Open
advitrocks9 wants to merge 1 commit into
apache:mainfrom
advitrocks9:fix-gh-47417
Open

GH-47417: [C++] Fix JSON parser losing nulls in a null-typed list#51108
advitrocks9 wants to merge 1 commit into
apache:mainfrom
advitrocks9:fix-gh-47417

Conversation

@advitrocks9

Copy link
Copy Markdown
Contributor

Rationale for this change

{"a": [null, 1]} parses to [[1]]. Nothing raises, and validate(full=True) is the only thing
that notices:

Offset invariant failure: offset for slot 1 out of bounds: 2 > 1

Debug builds abort on a DCHECK instead, at parser.cc:540 if the second element is null and
parser.cc:1151 if it is typed.

What changes are included in this PR?

RawBuilderSet::AppendNull takes the child builder by value. Builders of Kind::kNull have no
arena, their null count is packed into BuilderPtr::index, so the increment updates the parent
but is lost to the caller. The next null then re-increments the same stale value, which leaves an
all-null list as NullArray(1) and a promoted list with its leading nulls missing.

It now takes a BuilderPtr*, matching MakeBuilder in the same class. The regression came in
with b7054c21aab and has shipped since 0.14.0.

Are these changes tested?

BlockParser.NullsInList covers both arms. arrow-json-test passes 261/261 in Debug and
Release. Reverting parser.cc with the test in place aborts in Debug and fails both columns in
Release.

Are there any user-facing changes?

Those inputs parse correctly now. This is not inference-only: an explicit list(null()) schema
reaches the same arm under both Error and Ignore.

This PR contains a "Critical Fix". Well-formed JSON silently loses list elements in release
builds and aborts on a DCHECK in debug builds.

@advitrocks9
advitrocks9 requested a review from pitrou as a code owner August 31, 2026 18:32
Copilot AI lite review requested due to automatic review settings August 31, 2026 18:32
@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #47417 has been automatically assigned in GitHub to PR creator.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a long-standing JSON parsing bug in the C++ BlockParser where null elements could be silently dropped (or trigger DCHECKs in debug) when appending nulls to a null-typed list, by ensuring the inline null-count state in BuilderPtr is updated in-place.

Changes:

  • Change RawBuilderSet::AppendNull to take a BuilderPtr* so increments to the inline null-count for Kind::kNull persist to the caller.
  • Update all AppendNull call sites to pass builder pointers accordingly (including nested object-field null propagation).
  • Add a regression test covering nulls inside lists (null-only and promoted-from-null cases).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
cpp/src/arrow/json/parser.cc Fixes null appends by mutating the BuilderPtr in-place, preventing lost null counts in null-typed list builders.
cpp/src/arrow/json/parser_test.cc Adds a regression test to ensure nulls inside lists are preserved and don’t break list offsets.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +270 to +276
TEST(BlockParser, NullsInList) {
auto options = ParseOptions::Defaults();
options.unexpected_field_behavior = UnexpectedFieldBehavior::InferType;
AssertParseColumns(options, R"({"a": [null, null], "b": [null, "hi", null]})",
{field("a", list(null())), field("b", list(utf8()))},
{"[[null, null]]", R"([[null, "hi", null]])"});
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants