Reduce dotnet runtime

[kmiller68]

Add a parameter to pass in the BenchTask batch size from benchmark.js. This makes it possible to change the workload without rebuilding the .wasm blobs. The new batch size is dramatically smaller (100/50/10 for old/aot/interp respectively). I also reduced the frame size for the RayTracing half of the benchmark.

Overall though the distribution of time within the .NET wasm blobs is roughly the same between the two versions (in jsc anyway). With the exception of the interpreter subtest spending roughly 10% longer in the interpreter loop.

Lastly, I added some commentary to the README.md about what the benchmark is doing.

N.B.: I tried to remove the old BenchSize getter but that caused some .NET Out of Memory exceptions in the MONO runtime. The whole build process seems weirdly flakey. Sometimes the build will just not run. In fact 0b7a21d is one that fails and the subsequent commit e3dc40e (which didn't change anything, just rebuilt) works.

[netlify]

✅ Deploy Preview for webkit-jetstream-preview ready!

Name	Link
🔨 Latest commit	b72a665
🔍 Latest deploy log	https://app.netlify.com/projects/webkit-jetstream-preview/deploys/68af384402967d0008294e62
😎 Deploy Preview	https://deploy-preview-168--webkit-jetstream-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[kmiller68]

@eqrion @danleh please take a look and let me know what you think.

[kmiller68]

@maraf Have you seen issues with different builds of the same sources failing? Also, do you have comments or concerns with the PR?

[maraf]

Sorry for late reply, I was OOF in recent days. I'll take lot at changes.

I'll take a look at removing the BatchSize property as well, but it should simply fail during (normal) .NET build and not result in OOM at runtime. It looks more like mismatch between build individual files or unrelated to the property removal. Was it interp or AOT? Or both?

I didn't encounter any build flakiness, what arch/os is your host?

[kmiller68]

I know the interpreter was the one that had flakey builds for me. Not sure about the removing BatchSize.

Maybe it's the macOS version of dotnet that has issues? I'm on:

% dotnet --info
.NET SDK:
 Version:           9.0.304
 Commit:            f12f5f689e
 Workload version:  9.0.300-manifests.407ced12
 MSBuild version:   17.14.16+5d8159c5f

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  26
 OS Platform: Darwin
 RID:         osx-arm64
 Base Path:   /usr/local/share/dotnet/sdk/9.0.304/

.NET workloads installed:
 [wasm-tools]
   Installation Source: SDK 9.0.300
   Manifest Version:    9.0.8/9.0.100
   Manifest Path:       /usr/local/share/dotnet/sdk-manifests/9.0.100/microsoft.net.workload.mono.toolchain.current/9.0.8/WorkloadManifest.json
   Install Type:        FileBased

[maraf]

I wasn't able to reproduce the build problem from 0b7a21d on win-x64 nor osx-arm64 (both are long-time supported). Do you see similar problems regularly? If so, you can pass -bl to dotnet (like dotnet publish -bl ... and it will generate a .binlog, containing almost everything about the build. If you would be able to collect one for "invalid" build and one for success, it would help me to dig in.

I believe removing the BatchSize property also don't cause anything. Feel free to remove it. Otherwise the changes look good to me 👍

[danleh]

LGTM.

Regarding the flaky build: I could just rebuild successfully with the provided build.sh script and dotnet --version 9.0.302 on x64 Linux without any issue. Removing BatchSize from the BenchTask was also fine, i.e., the program ran as before without OOM on d8 and jsc, example output:

$ jsc cli.js -- dotnet
Starting JetStream3
Running dotnet-interp-wasm:
    Startup: 8.03
    Worst Case: 17.02
    Average: 18.02
    Score: 13.51
    Wall time: 3313.900 ms
Running dotnet-aot-wasm:
    Startup: 15.91
    Worst Case: 24.74
    Average: 26.06
    Score: 21.73
    Wall time: 3239.260 ms


First: 11.30
Worst: 20.52
Average: 21.67

Total Score:  17.13 

dotnet/system information:

$ dotnet --info
.NET SDK:
 Version:           9.0.302
 Commit:            bb2550b9af
 Workload version:  9.0.300-manifests.183aaee6
 MSBuild version:   17.14.13+65391c53b

Runtime Environment:
 OS Name:     debian
 OS Version:  
 OS Platform: Linux
 RID:         linux-x64
 Base Path:   /home/dlehmann/.dotnet/sdk/9.0.302/

.NET workloads installed:
 [wasm-tools]
   Installation Source: SDK 9.0.300
   Manifest Version:    9.0.7/9.0.100
   Manifest Path:       /home/dlehmann/.dotnet/sdk-manifests/9.0.100/microsoft.net.workload.mono.toolchain.current/9.0.7/WorkloadManifest.json
   Install Type:        FileBased

[danleh]

@kmiller68 shall we merge this?

[danleh]

LGTM

[kmiller68]

Shall we merge this?

Sounds good. We can always remove the now dead BatchSize and recompile in a followup. I filed #179

[kmiller68]

I wasn't able to reproduce the build problem from 0b7a21d on win-x64 nor osx-arm64 (both are long-time supported). Do you see similar problems regularly? If so, you can pass -bl to dotnet (like dotnet publish -bl ... and it will generate a .binlog, containing almost everything about the build. If you would be able to collect one for "invalid" build and one for success, it would help me to dig in.

I believe removing the BatchSize property also don't cause anything. Feel free to remove it. Otherwise the changes look good to me 👍

Forgot to get back to you on this. It seems like the issue was with stale build artifacts. Once I added rm -r src/dotnet/bin src/dotnet/obj to the build script the problem went away.

[maraf]

@kmiller68 We have seen some problem with build incrementalism, but not in such a simple case as we have here. The .binlog would be valuable if you ever run into the problem again.

[kmiller68]

@maraf I have a .binlog, if you want. Would you be able to join WebKit's OOS slack webkit.slack.com and ping me there?

[maraf]

Hmm, apparently I'm not able to. I have an account on slack, but when I try to log in at webkit.slack.com it keeps telling me "No account found".

Anyway, can you please send me the binlog by email, marekfisera at microsoft?