Skip to content

Handle out-of-range array index in nested JSON syntax#1854

Open
ATOM00blue wants to merge 1 commit into
httpie:masterfrom
ATOM00blue:fix-nested-json-large-index
Open

Handle out-of-range array index in nested JSON syntax#1854
ATOM00blue wants to merge 1 commit into
httpie:masterfrom
ATOM00blue:fix-nested-json-large-index

Conversation

@ATOM00blue
Copy link
Copy Markdown

Summary

Using a very large array index in the nested JSON request item syntax makes HTTPie crash with an unhandled exception instead of reporting a syntax error.

$ http --offline --print=B pie.dev/post 'item[99999999999999999999999]:=1'
...
  File ".../httpie/cli/nested_json/interpret.py", line 87, in interpret
    cursor.extend([None] * (path.accessor - len(cursor) + 1))
OverflowError: cannot fit 'int' into an index-sized integer

A slightly smaller (but still huge) index hits MemoryError instead:

$ http --offline --print=B pie.dev/post 'item[1000000000000000000]:=1'
...
MemoryError

The root cause is that the index is used to eagerly pre-fill a list with index + 1 elements, which either overflows the platform index size or tries to allocate an enormous list.

Fix

Wrap the list expansion and turn OverflowError/MemoryError into a NestedJSONSyntaxError, mirroring the existing handling of negative indexes. The user now gets a clear message pointing at the offending value:

$ http --offline --print=B pie.dev/post 'item[99999999999999999999999]:=1'
HTTPie Value Error: The index is too large.
item[99999999999999999999999]
     ^^^^^^^^^^^^^^^^^^^^^^^

Valid sparse arrays (e.g. item[100]:=1) are unaffected.

Test plan

  • Added two cases to test_nested_json_errors covering both the overflow and the large-allocation paths; they fail before the change and pass after.
  • pytest tests/test_json.py passes (296 passed).
  • flake8 httpie/ tests/ is clean on the changed files.

Using a very large array index in the nested JSON request item syntax
(e.g. `item[99999999999999999999999]:=1`) crashed HTTPie with an
unhandled `OverflowError` or `MemoryError`, because the index was used
to eagerly pre-fill a list with that many elements.

Catch those errors during the list expansion and raise a
`NestedJSONSyntaxError` instead, mirroring how negative indexes are
already handled, so the user gets a clear message pointing at the
offending value.
Copilot AI review requested due to automatic review settings May 22, 2026 02:31
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR prevents crashes in nested JSON parsing when users provide extremely large array indexes, replacing them with a user-friendly syntax error.

Changes:

  • Catch OverflowError/MemoryError when expanding arrays for large indexes and raise NestedJSONSyntaxError instead.
  • Add regression tests for very large index inputs and expected error rendering.
  • Document the fix in the changelog under the unreleased section.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
tests/test_json.py Adds regression cases asserting a clear error for out-of-range nested JSON array indexes.
httpie/cli/nested_json/interpret.py Wraps array extension to convert OverflowError/MemoryError into NestedJSONSyntaxError.
CHANGELOG.md Documents the crash fix and resulting improved error reporting.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +87 to +95
try:
cursor.extend([None] * (path.accessor - len(cursor) + 1))
except (OverflowError, MemoryError):
raise NestedJSONSyntaxError(
source=key,
token=path.tokens[1],
message='The index is too large.',
message_kind='Value',
) from None
raise NestedJSONSyntaxError(
source=key,
token=path.tokens[1],
message='The index is too large.',
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants