test case: canister sandbox panic

[nomeata]

upgrades drun, disables crashing amounts in basic-cycles.mo and exhibits the panic in canster-sandbox-panic.mo until upstream has fixed that. Reported bug at https://forum.dfinity.org/t/10729?u=nomeata

[nomeata]

This triggers

ingress Completed: Reply: 0x4449444c016c01b3c4b1f204680100010a00000000000000000101
ingress Completed: Reply: 0x4449444c0000
debug.print: topping up
thread '<unnamed>' panicked at 'Failed to serialize message: ErrorImpl { code: Message("The number can't be stored in CBOR"), offset: 0 }', /build/rs/canister_sandbox/common/src/transport.rs:285:55
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'MR Batch Processor' panicked at 'Thread pool worker panicked', /build/drun-vendor.tar.gz/scoped_threadpool/src/lib.rs:236:13
thread 'main' panicked at 'Failed to send batch 24: background worker is dead', messaging/src/message_routing.rs:949:51

[nomeata]

This seems to come from https://rustdoc.etonomy.org/src/serde_cbor/ser.rs.html#267-279 or https://rustdoc.etonomy.org/src/serde_cbor/ser.rs.html#302-308, i.e serialize_u128 or serialize_u64. Looks like there is u128 or i128 used somewhere in the system API, and the sandbox uses CBOR, but serde doesn’t support that (despite the fact that CBOR does, via bignums).

[nomeata]

Related: pyfisch/cbor#77 and pyfisch/cbor#145

[nomeata]

BTW, looks like cbor_serde is no longer maintained by upstream: https://github.com/pyfisch/cbor. It recommends two other libraries. And https://crates.io/crates/ciborium at least get CBOR numbers right (always encodes at smallest size)

[nomeata]

Maybe we can actually merge this, so that we notice other breakage in drun in the meantime.

[github-actions]

Comparing from defd131 to bf8ec4d:
The produced WebAssembly code seems to be completely unchanged.

[ggreif]

Do we need an issue for the commented-out tests?

[nomeata]

Do we need an issue for the commented-out tests?

Not really, we'll here on https://forum.dfinity.org/t/panicked-at-failed-to-serialize-message-errorimpl-code-message-the-number-cant-be-stored-in-cbor-offset-0-build-rs-canister-sandbox-common-src-transport-rs55/10729/4 once it's fixed, I hope

[nomeata]

Hmm, looks like drun on MacOS isn’t using the canister sandbox, so there we get different output. Maybe we can just wait for the bug to be fixed.

[nomeata]

@ulan says that this is fixed. Let’s see…

[nomeata]

I see

       >   The package requires the Cargo feature called `named-profiles`, but that feature is not stabilized in this version of Cargo (1.56.0).
       >   Consider adding `cargo-features = ["named-profiles"]` to the top of Cargo.toml (above the [package] table) to tell Cargo you are opting in to use this unstable feature.
       >   See https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#custom-named-profiles for more information about the status of this feature.

Do we need to update rust and/or cargo maybe? @ggreif didn’t you recently try to bump rust?

[nomeata]

@ggreif, could you help us updating drun maybe?

[nomeata]

With rust-1.57 I am making progress. I still need to apply this patch to ic, because otherwise cargo vendor complains about multiple versions of wabt:

diff --git a/rs/boundary_node/prober/Cargo.toml b/rs/boundary_node/prober/Cargo.toml
index b28edf48..6beb9bd1 100644
--- a/rs/boundary_node/prober/Cargo.toml
+++ b/rs/boundary_node/prober/Cargo.toml
@@ -35,4 +35,4 @@ slog-scope = "4.1.2"
 slog-term = "2.6.0"
 tokio = { version = "1.15.0", features = ["full"] }
 tokio-openssl = "0.6.0"
-wabt = "0.10.0"
+wabt = { git = "https://github.com/dfinity-lab/wabt-rs", tag = "0.10.0-dfinity" }

Maybe someone can upstream this? Once fixed, this should work:

~/dfinity/ic/rs $ cargo vendor vendor-test
error: failed to sync

Caused by:
  found duplicate version of package `wabt v0.10.0` vendored from two sources:

  	source 1: registry `crates-io`
  	source 2: https://github.com/dfinity-lab/wabt-rs?tag=0.10.0-dfinity#7ab9062d

[nomeata]

Next error would be

   Compiling ic-registry-provisional-whitelist v0.8.0 (/build/rs/registry/provisional_whitelist)
The following warnings were emitted during compilation:

warning: ../src/backtrace.c:11:10: fatal error: libunwind.h: No such file or directory
warning:    11 | #include <libunwind.h>
warning:       |          ^~~~~~~~~~~~~
warning: compilation terminated.

error: failed to run custom build command for `ic-canister-sandbox-launcher v0.8.0 (/build/rs/canister_sandbox/sandbox_launcher)`

Caused by:
  process didn't exit successfully: `/build/rs/target/release/build/ic-canister-sandbox-launcher-4d61a222ccdb986f/build-script-build` (exit status: 1)
  --- stdout
  TARGET = Some("x86_64-unknown-linux-gnu")
  OPT_LEVEL = Some("3")
  HOST = Some("x86_64-unknown-linux-gnu")
  CC_x86_64-unknown-linux-gnu = Some("/nix/store/ajisnbqj2qyirds1hnwf3pn919c2c8yc-gcc-wrapper-10.3.0/bin/cc")
  CFLAGS_x86_64-unknown-linux-gnu = None
  CFLAGS_x86_64_unknown_linux_gnu = None
  HOST_CFLAGS = None
  CFLAGS = None
  CRATE_CC_NO_DEFAULTS = None
  CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
  running: "/nix/store/ajisnbqj2qyirds1hnwf3pn919c2c8yc-gcc-wrapper-10.3.0/bin/cc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-Wall" "-Wextra" "-o" "/build/rs/target/x86_64-unknown-linux-gnu/release/build/ic-canister-sandbox-launcher-fee8adc3b2e6dbd4/out/../src/backtrace.o" "-c" "../src/backtrace.c"
  cargo:warning=../src/backtrace.c:11:10: fatal error: libunwind.h: No such file or directory
  cargo:warning=   11 | #include <libunwind.h>
  cargo:warning=      |          ^~~~~~~~~~~~~
  cargo:warning=compilation terminated.
  exit status: 1

  --- stderr


  error occurred: Command "/nix/store/ajisnbqj2qyirds1hnwf3pn919c2c8yc-gcc-wrapper-10.3.0/bin/cc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-Wall" "-Wextra" "-o" "/build/rs/target/x86_64-unknown-linux-gnu/release/build/ic-canister-sandbox-launcher-fee8adc3b2e6dbd4/out/../src/backtrace.o" "-c" "../src/backtrace.c" with args "cc" did not execute successfully (status code exit status: 1).


warning: build failed, waiting for other jobs to finish...

[ulan]

@hcbdfinity ^ any ideas why libunwind.h is failing for @nomeata ?

Maybe someone can upstream this?

Thanks! I'll upstream the fix.

[hcbdfinity]

I think you would simply need to get libunwind into your nix env, AFAICT it is supported already: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/libraries/libunwind/default.nix
there may be a need to pass pkg-config flags, TBOMK libunwind upstream has been pkg-config'ed

[nomeata]

I can investigate the libunwind issue, I didn’t try hard yet; would be nice though to get past the other issue first (cargo vendor failing due to different wabt crates appearing in the Cargo.lock).

[nomeata]

As @hcbdfinity said, making libunwind accessible in the nix env is enough. So once the Cargo files are in better order in the ic repo, we can start testing the latest replica code here again (I hope)

[ulan]

Upstreamed fix: dfinity/ic@bedc417

[nomeata]

Bumping drun separately in #3181, afterwards this PR can add the new test case

[nomeata]

Probably not needed anymore, as there is a fix in the replica already (#3181)