Skip to content

Support running test262 using boa cli#5234

Open
ivankra wants to merge 4 commits intoboa-dev:mainfrom
ivankra:test262
Open

Support running test262 using boa cli#5234
ivankra wants to merge 4 commits intoboa-dev:mainfrom
ivankra:test262

Conversation

@ivankra
Copy link

@ivankra ivankra commented Mar 23, 2026 •
edited
Loading

This Pull Request fixes/closes #5054

It changes the following:

  • Move tests/tester/src/exec/js262.rs to core/runtime/src/test262.rs so it can be reused by both cli and boa_tester
  • Add --test262-object flag to CLI
  • Add print() to CLI with --test262-object
  • Enable [[CanBlock]] by default in CLI to match boa_tester as well as d8/jsc/sm shells default behavior
  • Add --no-can-block flag to CLI to support Atomics CanBlockIsFalse tests (same flag name as d8 has)
  • Auto-detect .mjs files as modules in CLI (a small optional quality-of-life improvement; matches behavior of most other major JS engine shells)

These changes make it possible to reproduce boa_tester's test262 results using external test262 harnesses with boa cli binary.

@ivankra ivankra requested a review from a team as a code owner March 23, 2026 09:16
@github-actions github-actions bot added C-Dependencies Pull requests that update a dependency file C-Documentation update documentation C-CLI Issues and PRs related to the Boa command line interface. C-Tests Issues and PRs related to the tests. C-Runtime Issues and PRs related to Boa's runtime features Waiting On Review Waiting on reviews from the maintainers labels Mar 23, 2026
@github-actions github-actions bot added this to the v1.0.0 milestone Mar 23, 2026
@ivankra ivankra mentioned this pull request Mar 23, 2026
Open
jedel1043
jedel1043 reviewed Mar 23, 2026
View reviewed changes
core/runtime/src/test262.rs Outdated
let clock = START
.get()
.ok_or_else(|| JsNativeError::typ().with_message("could not get the monotonic clock"))?;
let clock = START.get_or_init(Instant::now);
Copy link
Member

@jedel1043 jedel1043 Mar 23, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is correct. This will always return 0 or something close to 0 for the first call to monotonic_now, which could fail some tests that assume some amount of time already passed

Copy link
Author

@ivankra ivankra Mar 23, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I think you're right -- looking at the spec for DOMHighResTimeStamp, it does suggest to use worker/binary's start time as the time origin.

I moved initialization of START to the beginning of register_js262() (diff) and tested:

$ ./target/release/boa --test262-object -e '$262.agent.sleep(100); print($262.agent.monotonicNow());'
100
undefined

(before it printed 0 as clock got initialized on first use)

test262 tests aren't affected, no regressions either way.

@github-actions
Copy link

github-actions bot commented Mar 24, 2026

Test262 conformance changes

Test result main count PR count difference
Total 52,963 52,963 0
Passed 50,732 50,545 -187
Ignored 1,426 1,426 0
Failed 805 992 +187
Panics 0 2 +2
Conformance 95.79% 95.43% -0.35%
Broken tests (185): 
test/built-ins/Iterator/prototype/some/iterator-already-exhausted.js (previously Passed)
test/built-ins/Iterator/prototype/some/predicate-returns-non-boolean.js (previously Passed)
test/built-ins/Iterator/prototype/some/predicate-this.js (previously Passed)
test/built-ins/Iterator/prototype/some/predicate-returns-falsey.js (previously Passed)
test/built-ins/Iterator/prototype/some/predicate-returns-truthy.js (previously Passed)
test/built-ins/Iterator/prototype/some/predicate-args.js (previously Passed)
test/built-ins/Iterator/prototype/some/predicate-returns-falsey-then-truthy.js (previously Passed)
test/built-ins/Iterator/prototype/some/iterator-has-no-return.js (previously Passed)
test/built-ins/Iterator/prototype/some/argument-validation-failure-closes-underlying.js (previously Passed)
test/built-ins/Iterator/prototype/some/argument-effect-order.js (previously Passed)
test/built-ins/Iterator/prototype/some/callable.js (previously Passed)
test/built-ins/Iterator/prototype/some/result-is-boolean.js (previously Passed)
test/built-ins/Iterator/prototype/reduce/reducer-args-no-initial-value.js (previously Passed)
test/built-ins/Iterator/prototype/reduce/reducer-args-initial-value.js (previously Passed)
test/built-ins/Iterator/prototype/reduce/reducer-this.js (previously Passed)
test/built-ins/Iterator/prototype/reduce/iterator-yields-once-initial-value.js (previously Passed)
test/built-ins/Iterator/prototype/reduce/iterator-already-exhausted-initial-value.js (previously Passed)
test/built-ins/Iterator/prototype/reduce/argument-validation-failure-closes-underlying.js (previously Passed)
test/built-ins/Iterator/prototype/reduce/argument-effect-order.js (previously Passed)
test/built-ins/Iterator/prototype/reduce/callable.js (previously Passed)
test/built-ins/Iterator/prototype/reduce/non-callable-reducer.js (previously Passed)
test/built-ins/Iterator/prototype/reduce/iterator-already-exhausted-no-initial-value.js (previously Passed)
test/built-ins/Iterator/prototype/reduce/reducer-memo-can-be-any-type.js (previously Passed)
test/built-ins/Iterator/prototype/reduce/iterator-yields-once-no-initial-value.js (previously Passed)
test/built-ins/Iterator/prototype/constructor/weird-setter.js (previously Passed)
test/built-ins/Iterator/prototype/find/iterator-already-exhausted.js (previously Passed)
test/built-ins/Iterator/prototype/find/predicate-returns-non-boolean.js (previously Passed)
test/built-ins/Iterator/prototype/find/predicate-this.js (previously Passed)
test/built-ins/Iterator/prototype/find/predicate-returns-falsey.js (previously Passed)
test/built-ins/Iterator/prototype/find/predicate-returns-truthy.js (previously Passed)
test/built-ins/Iterator/prototype/find/predicate-args.js (previously Passed)
test/built-ins/Iterator/prototype/find/predicate-returns-falsey-then-truthy.js (previously Passed)
test/built-ins/Iterator/prototype/find/iterator-has-no-return.js (previously Passed)
test/built-ins/Iterator/prototype/find/argument-validation-failure-closes-underlying.js (previously Passed)
test/built-ins/Iterator/prototype/find/argument-effect-order.js (previously Passed)
test/built-ins/Iterator/prototype/find/callable.js (previously Passed)
test/built-ins/Iterator/prototype/map/returned-iterator-yields-mapper-return-values.js (previously Passed)
test/built-ins/Iterator/prototype/map/iterator-already-exhausted.js (previously Passed)
test/built-ins/Iterator/prototype/map/throws-typeerror-when-generator-is-running.js (previously Passed)
test/built-ins/Iterator/prototype/map/mapper-args.js (previously Passed)
test/built-ins/Iterator/prototype/map/mapper-this.js (previously Passed)
test/built-ins/Iterator/prototype/map/result-is-iterator.js (previously Passed)
test/built-ins/Iterator/prototype/map/underlying-iterator-advanced-in-parallel.js (previously Passed)
test/built-ins/Iterator/prototype/map/argument-validation-failure-closes-underlying.js (previously Passed)
test/built-ins/Iterator/prototype/map/argument-effect-order.js (previously Passed)
test/built-ins/Iterator/prototype/map/underlying-iterator-closed.js (previously Passed)
test/built-ins/Iterator/prototype/map/underlying-iterator-closed-in-parallel.js (previously Passed)
test/built-ins/Iterator/prototype/map/callable.js (previously Passed)
test/built-ins/Iterator/prototype/flatMap/flattens-iterator.js (previously Passed)
test/built-ins/Iterator/prototype/flatMap/iterator-already-exhausted.js (previously Passed)
test/built-ins/Iterator/prototype/flatMap/flattens-iterable.js (previously Passed)
test/built-ins/Iterator/prototype/flatMap/iterable-to-iterator-fallback.js (previously Passed)
test/built-ins/Iterator/prototype/flatMap/throws-typeerror-when-generator-is-running.js (previously Passed)
test/built-ins/Iterator/prototype/flatMap/mapper-args.js (previously Passed)
test/built-ins/Iterator/prototype/flatMap/flattens-only-depth-1.js (previously Passed)
test/built-ins/Iterator/prototype/flatMap/mapper-this.js (previously Passed)
test/built-ins/Iterator/prototype/flatMap/result-is-iterator.js (previously Passed)
test/built-ins/Iterator/prototype/flatMap/underlying-iterator-advanced-in-parallel.js (previously Passed)
test/built-ins/Iterator/prototype/flatMap/return-is-forwarded-to-mapper-result.js (previously Passed)
test/built-ins/Iterator/prototype/flatMap/argument-validation-failure-closes-underlying.js (previously Passed)
test/built-ins/Iterator/prototype/flatMap/argument-effect-order.js (previously Passed)
test/built-ins/Iterator/prototype/flatMap/mapper-returns-closed-iterator.js (previously Passed)
test/built-ins/Iterator/prototype/flatMap/mapper-returns-non-object.js (previously Passed)
test/built-ins/Iterator/prototype/flatMap/underlying-iterator-closed.js (previously Passed)
test/built-ins/Iterator/prototype/flatMap/underlying-iterator-closed-in-parallel.js (previously Passed)
test/built-ins/Iterator/prototype/flatMap/callable.js (previously Passed)
test/built-ins/Iterator/prototype/flatMap/iterable-primitives-are-not-flattened.js (previously Passed)
test/built-ins/Iterator/prototype/flatMap/strings-are-not-flattened.js (previously Passed)
test/built-ins/Iterator/prototype/every/iterator-already-exhausted.js (previously Passed)
test/built-ins/Iterator/prototype/every/predicate-returns-non-boolean.js (previously Passed)
test/built-ins/Iterator/prototype/every/predicate-this.js (previously Passed)
test/built-ins/Iterator/prototype/every/predicate-returns-falsey.js (previously Passed)
test/built-ins/Iterator/prototype/every/predicate-returns-truthy.js (previously Passed)
test/built-ins/Iterator/prototype/every/predicate-args.js (previously Passed)
test/built-ins/Iterator/prototype/every/iterator-has-no-return.js (previously Passed)
test/built-ins/Iterator/prototype/every/argument-validation-failure-closes-underlying.js (previously Passed)
test/built-ins/Iterator/prototype/every/argument-effect-order.js (previously Passed)
test/built-ins/Iterator/prototype/every/callable.js (previously Passed)
test/built-ins/Iterator/prototype/every/result-is-boolean.js (previously Passed)
test/built-ins/Iterator/prototype/every/predicate-returns-truthy-then-falsey.js (previously Passed)
test/built-ins/Iterator/prototype/forEach/fn-called-for-each-yielded-value.js (previously Passed)
test/built-ins/Iterator/prototype/forEach/iterator-already-exhausted.js (previously Passed)
test/built-ins/Iterator/prototype/forEach/fn-this.js (previously Passed)
test/built-ins/Iterator/prototype/forEach/result-is-undefined.js (previously Passed)
test/built-ins/Iterator/prototype/forEach/argument-validation-failure-closes-underlying.js (previously Passed)
test/built-ins/Iterator/prototype/forEach/argument-effect-order.js (previously Passed)
test/built-ins/Iterator/prototype/forEach/callable.js (previously Passed)
test/built-ins/Iterator/prototype/forEach/fn-args.js (previously Passed)
test/built-ins/Iterator/prototype/filter/iterator-already-exhausted.js (previously Passed)
test/built-ins/Iterator/prototype/filter/predicate-returns-non-boolean.js (previously Passed)
test/built-ins/Iterator/prototype/filter/predicate-this.js (previously Passed)
test/built-ins/Iterator/prototype/filter/predicate-args.js (previously Passed)
test/built-ins/Iterator/prototype/filter/throws-typeerror-when-generator-is-running.js (previously Passed)
test/built-ins/Iterator/prototype/filter/result-is-iterator.js (previously Passed)
test/built-ins/Iterator/prototype/filter/underlying-iterator-advanced-in-parallel.js (previously Passed)
test/built-ins/Iterator/prototype/filter/predicate-filters.js (previously Passed)
test/built-ins/Iterator/prototype/filter/argument-validation-failure-closes-underlying.js (previously Passed)
test/built-ins/Iterator/prototype/filter/argument-effect-order.js (previously Passed)
test/built-ins/Iterator/prototype/filter/underlying-iterator-closed.js (previously Passed)
test/built-ins/Iterator/prototype/filter/underlying-iterator-closed-in-parallel.js (previously Passed)
test/built-ins/Iterator/prototype/filter/callable.js (previously Passed)
test/built-ins/Iterator/prototype/drop/limit-greater-than-total.js (previously Passed)
test/built-ins/Iterator/prototype/drop/limit-rangeerror.js (previously Passed)
test/built-ins/Iterator/prototype/drop/limit-equals-total.js (previously Passed)
test/built-ins/Iterator/prototype/drop/result-is-iterator.js (previously Passed)
test/built-ins/Iterator/prototype/drop/underlying-iterator-advanced-in-parallel.js (previously Passed)
test/built-ins/Iterator/prototype/drop/argument-validation-failure-closes-underlying.js (previously Passed)
test/built-ins/Iterator/prototype/drop/argument-effect-order.js (previously Passed)
test/built-ins/Iterator/prototype/drop/limit-tonumber-throws.js (previously Passed)
test/built-ins/Iterator/prototype/drop/underlying-iterator-closed.js (previously Passed)
test/built-ins/Iterator/prototype/drop/underlying-iterator-closed-in-parallel.js (previously Passed)
test/built-ins/Iterator/prototype/drop/callable.js (previously Passed)
test/built-ins/Iterator/prototype/drop/limit-tonumber.js (previously Passed)
test/built-ins/Iterator/prototype/drop/limit-less-than-total.js (previously Passed)
test/built-ins/Iterator/prototype/Symbol.toStringTag/weird-setter.js (previously Passed)
test/built-ins/Iterator/prototype/take/limit-rangeerror.js (previously Passed)
test/built-ins/Iterator/prototype/take/result-is-iterator.js (previously Passed)
test/built-ins/Iterator/prototype/take/underlying-iterator-advanced-in-parallel.js (previously Passed)
test/built-ins/Iterator/prototype/take/limit-greater-than-or-equal-to-total.js (previously Passed)
test/built-ins/Iterator/prototype/take/argument-validation-failure-closes-underlying.js (previously Passed)
test/built-ins/Iterator/prototype/take/argument-effect-order.js (previously Passed)
test/built-ins/Iterator/prototype/take/limit-tonumber-throws.js (previously Passed)
test/built-ins/Iterator/prototype/take/underlying-iterator-closed.js (previously Passed)
test/built-ins/Iterator/prototype/take/underlying-iterator-closed-in-parallel.js (previously Passed)
test/built-ins/Iterator/prototype/take/callable.js (previously Passed)
test/built-ins/Iterator/prototype/take/limit-tonumber.js (previously Passed)
test/built-ins/Iterator/prototype/take/limit-less-than-total.js (previously Passed)
test/built-ins/Iterator/prototype/toArray/iterator-already-exhausted.js (previously Passed)
test/built-ins/Iterator/prototype/toArray/callable.js (previously Passed)
test/built-ins/Iterator/from/iterable-primitives.js (previously Passed)
test/built-ins/Iterator/from/result-proto.js (previously Passed)
test/built-ins/Iterator/concat/throws-typeerror-when-generator-is-running-return.js (previously Passed)
test/built-ins/Object/prototype/toString/symbol-tag-map-builtin.js (previously Passed)
test/built-ins/Object/prototype/toString/symbol-tag-set-builtin.js (previously Passed)
test/built-ins/Object/prototype/toString/symbol-tag-array-builtin.js (previously Passed)
test/built-ins/Object/prototype/toString/symbol-tag-string-builtin.js (previously Passed)
test/staging/sm/Iterator/prototype/lazy-methods-handle-empty-iterators.js (previously Passed)
test/staging/sm/Iterator/prototype/lazy-methods-reentry-not-close-iterator.js (previously Passed)
test/staging/sm/Iterator/prototype/iterator-helper-methods-throw-on-generators.js (previously Passed)
test/staging/sm/Iterator/prototype/generator-methods-throw-on-iterator-helpers.js (previously Passed)
test/staging/sm/Iterator/prototype/iterator-helpers-from-other-global.js (previously Passed)
test/staging/sm/Iterator/prototype/take-drop-throw-eagerly-on-negative.js (previously Passed)
test/staging/sm/Iterator/prototype/lazy-methods-throw-on-reentry.js (previously Passed)
test/staging/sm/Iterator/prototype/some/coerce-result-to-boolean.js (previously Passed)
test/staging/sm/Iterator/prototype/some/return-false-if-none-match.js (previously Passed)
test/staging/sm/Iterator/prototype/some/check-fn-after-getting-iterator.js (previously Passed)
test/staging/sm/Iterator/prototype/some/short-circuit-on-true.js (previously Passed)
test/staging/sm/Iterator/prototype/reduce/iterator-empty-return-initial-value.js (previously Passed)
test/staging/sm/Iterator/prototype/reduce/left-associative.js (previously Passed)
test/staging/sm/Iterator/prototype/reduce/no-initial-value-set-accumulator-to-first-value.js (previously Passed)
test/staging/sm/Iterator/prototype/reduce/check-fn-after-getting-iterator.js (previously Passed)
test/staging/sm/Iterator/prototype/reduce/reduce.js (previously Passed)
test/staging/sm/Iterator/prototype/reduce/accumulator-set-to-initial-value.js (previously Passed)
test/staging/sm/Iterator/prototype/find/short-circuit-on-match.js (previously Passed)
test/staging/sm/Iterator/prototype/find/coerce-result-to-boolean.js (previously Passed)
test/staging/sm/Iterator/prototype/find/check-fn-after-getting-iterator.js (previously Passed)
test/staging/sm/Iterator/prototype/find/return-undefined-if-none-match.js (previously Passed)
test/staging/sm/Iterator/prototype/map/interleaved-map-calls.js (previously Passed)
test/staging/sm/Iterator/prototype/map/call-next-on-iterator-while-iterating.js (previously Passed)
test/staging/sm/Iterator/prototype/map/reenter-map-generator-from-mapper.js (previously Passed)
test/staging/sm/Iterator/prototype/map/clobber-symbol.js (previously Passed)
test/staging/sm/Iterator/prototype/map/mutate-iterator.js (previously Passed)
test/staging/sm/Iterator/prototype/map/mutate-iterator-after-done.js (previously Passed)
test/staging/sm/Iterator/prototype/map/output-at-generator-end.js (previously Passed)
test/staging/sm/Iterator/prototype/flatMap/inner-generator.js (previously Passed)
test/staging/sm/Iterator/prototype/flatMap/inner-empty-iterable.js (previously Passed)
test/staging/sm/Iterator/prototype/flatMap/throw-when-inner-not-iterable.js (previously Passed)
test/staging/sm/Iterator/prototype/flatMap/close-iterator-when-inner-value-throws.js (previously Passed)
test/staging/sm/Iterator/prototype/flatMap/close-iterator-when-inner-complete-throws.js (previously Passed)
test/staging/sm/Iterator/prototype/flatMap/close-iterator-when-inner-next-throws.js (previously Passed)
test/staging/sm/Iterator/prototype/every/short-circuit-on-false.js (previously Passed)
test/staging/sm/Iterator/prototype/every/coerce-result-to-boolean.js (previously Passed)
test/staging/sm/Iterator/prototype/every/check-fn-after-getting-iterator.js (previously Passed)
test/staging/sm/Iterator/prototype/every/return-true-if-all-match.js (previously Passed)
test/staging/sm/Iterator/prototype/forEach/forEach.js (previously Passed)
test/staging/sm/Iterator/prototype/forEach/check-fn-after-getting-iterator.js (previously Passed)
test/staging/sm/Iterator/prototype/filter/coerce-result-to-boolean.js (previously Passed)
test/staging/sm/Iterator/prototype/drop/drop-more-than-available.js (previously Passed)
test/staging/sm/Iterator/prototype/take/take-more-than-available.js (previously Passed)
test/staging/sm/Iterator/prototype/toArray/toArray.js (previously Passed)
test/staging/sm/Iterator/prototype/toArray/iterator-empty.js (previously Passed)
test/staging/sm/Iterator/prototype/toArray/create-in-current-realm.js (previously Passed)
test/staging/sm/Iterator/from/return-iterator-if-iterable.js (previously Passed)
test/staging/sm/Iterator/from/wrap-new-global.js (previously Passed)
test/staging/sm/Array/from_primitive.js (previously Passed)
New panics (2): 
test/built-ins/Iterator/prototype/drop/throws-typeerror-when-generator-is-running.js (previously Passed)
test/built-ins/Iterator/prototype/take/throws-typeerror-when-generator-is-running.js (previously Passed)

Tested main commit: 8fcbfcbb8b9e70fe878d28376a6496eaa57a70d7
Tested PR commit: 4c52207e66e1e1fe37d9c0702f4d3c77f6891303
Compare commits: 8fcbfcb...4c52207

ivankra added 4 commits March 26, 2026 12:03
@ivankra
refactor(runtime): move $262 to boa_runtime, add --test262-object to cli
590138d 
Move `$262` implementation from `tests/tester/src/exec/js262.rs` to
`core/runtime/src/test262.rs` behind a new `test262` feature gate
(pulls `bus` as optional dep). Add `annex-b` feature to boa_runtime
forwarding to `boa_engine/annex-b` for `$262.IsHTMLDDA`.

Both cli and boa_tester now use `boa_runtime/test262`, removing the
tester's direct `bus` and `boa_gc` deps.

CLI registers `$262` when `--test262-object` flag is passed.

This change makes CLI more suitable for running test262 via an external
harness such as eshost + test262-harness. The exact same `$262` object as
used by the internal boa_tester can now be directly accessed in CLI.
Existing `$boa` object doesn't expose some of the methods (like IsHTMLDDA,
detachArrayBuffer and agent helpers) needed for full test262 support,
and would have required users to maintain a custom shim mapping it to `$262`.

Fixes boa-dev#5054
@ivankra
cli: define print() for test262 compatibility
43df77e 
test262 tests use print() to signal async completion and errors.
The tester has its own print() implementation to catch reported errors,
but CLI should just print them, so let's alias it to console.log().
@ivankra
cli: auto-detect .mjs files as modules
09fcf35 
Treat files with .mjs extension as modules without requiring --module flag.

Many other JavaScript shells do the same - d8, jsc, graaljs, xs, escargot.
A small quality-of-life improvement that among other things simplifies
configuration for an external test262 harness. The harness could simply
stage module tests into *.mjs and forget about --module flag.
@ivankra
cli: enable can_block by default, add --no-can-block flag
fb4249c 
Make boa's cli consistent with the behaviour of other major JavaScript
shells (d8, jsc, spidermonkey at least) as well as boa_tester by
enabling [[CanBlock]] by default.

Add --no-can-block flag to opt out, same as V8
(https://github.com/v8/v8/blob/main/src/d8/d8.cc#L6417).

Needed for Atomics tests to pass when using an external test262 harness.
@ivankra
Copy link
Author

ivankra commented Mar 26, 2026

Fixed lint errors in lib.rs / test262.rs and rebased.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jedel1043 jedel1043 jedel1043 left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

C-CLI Issues and PRs related to the Boa command line interface. C-Dependencies Pull requests that update a dependency file C-Documentation update documentation C-Runtime Issues and PRs related to Boa's runtime features C-Tests Issues and PRs related to the tests. Waiting On Review Waiting on reviews from the maintainers

Projects

None yet

Milestone

v1.0.0

Development

Successfully merging this pull request may close these issues.

Expose full $262 object in boa_cli

2 participants

@ivankra @jedel1043