Skip to content

chore: add devenv tests#26

Merged
dastarruer merged 13 commits intomasterfrom
chore/add-devenv-tests
Mar 29, 2026
Merged

chore: add devenv tests#26
dastarruer merged 13 commits intomasterfrom
chore/add-devenv-tests

Conversation

@dastarruer
Copy link
Copy Markdown
Owner

@dastarruer dastarruer commented Mar 29, 2026

Closes #21

Summary

  • Add test to check that node is correctly installed
  • Add workflow to run devenv tests automatically (based on this example)

Summary by CodeRabbit

  • Tests

    • Added an automated CI workflow that runs on PRs and pushes across Linux and macOS, installs dependencies, uses caching, and executes repository tests and pre-commit checks.
  • Chores

    • Formatting switched to Prettier for project files.
    • Development environment now performs a startup verification to confirm the expected runtime version for consistent tooling.

@dastarruer
Copy link
Copy Markdown
Owner Author

@CodeRabbit review

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Mar 29, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Mar 29, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • ✅ Review completed - (🔄 Check again to review again)
📝 Walkthrough

Walkthrough

Adds a GitHub Actions “Test” workflow that runs devenv test on Ubuntu and macOS with Nix and Cachix; tightens and changes the npm-format hook in devenv.nix to use npx prettier --write for staged files under app/, and adds an enterTest shell check that verifies the exact Node.js version.

Changes

Cohort / File(s) Summary
CI workflow
\.github/workflows/test.yml
New GitHub Actions workflow triggered on push and pull_request; matrix across ubuntu-latest and macos-latest; installs Nix, configures Cachix cache devenv, adds nixpkgs#devenv profile, conditionally installs nixVersions.latest on macOS, runs devenv shell and npm ci in app/, then runs devenv test.
Devenv hooks
devenv.nix
Replaced the npm-format hook command to use npx prettier --write on staged filenames after stripping the app/ prefix and tightened the files glob to ^app/.* for supported extensions; added an enterTest shell check that runs `node --version

Sequence Diagram(s)

sequenceDiagram
  participant GH as "GitHub Actions"
  participant Runner as "Runner (ubuntu/macos)"
  participant Installer as "Nix installer"
  participant Cachix as "Cachix (devenv)"
  participant Devenv as "devenv"
  participant App as "app/ (npm)"
  participant Hooks as "pre-commit / tests"

  GH->>Runner: start job (matrix)
  Runner->>Installer: install Nix
  Runner->>Cachix: configure cache (devenv)
  Runner->>Installer: nix profile add nixpkgs#devenv
  Runner->>Devenv: enter `devenv shell`
  Devenv->>App: run `npm ci` in `app/`
  Devenv->>Devenv: run `enterTest` (check node version)
  Runner->>Devenv: run `devenv test`
  Devenv->>Hooks: execute configured tests / hooks
  Hooks-->>GH: report status
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Poem

🐰
I hop through CI with Nix on my back,
Prettier tidies the trail along the track,
I sniff the node version — exact and right,
Then tests nibble carrots through the night,
A happy rabbit in a green dev light.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title 'chore: add devenv tests' directly and clearly summarizes the main change: introducing automated tests for the devenv development environment.
Linked Issues check ✅ Passed The PR fulfills all coding requirements from issue #21: introduces automated tests for devenv, validates Node installation, and implements a GitHub Actions workflow for automated testing.
Out of Scope Changes check ✅ Passed All changes are directly scoped to issue #21 objectives: the new GitHub Actions workflow and Node version check in devenv.nix both address the goal of automated devenv testing.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch chore/add-devenv-tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/workflows/test.yml:
- Line 12: Replace the malformed runs-on expression that currently uses nested
template delimiters (the string "{{ '${{ matrix.os }}' }}") with the proper
GitHub Actions expression `${{ matrix.os }}` so the workflow parser receives a
valid runs-on value; locate the runs-on assignment in the workflow (the line
containing runs-on: {{ '${{ matrix.os }}' }}) and update it to use `${{
matrix.os }}`.

In `@devenv.nix`:
- Around line 86-88: The enterTest script uses a substring grep against
"${pkgs.nodejs_latest.version}", which allows partial matches; update the test
to assert the full node --version output exactly (e.g. use exact-match grep with
start/end anchors or grep -x, or compare the complete "$(node --version)" string
to "v${pkgs.nodejs_latest.version}") so the check in enterTest fails on any
non-exact version.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 7745a310-246a-4052-8045-d04885ec9022

📥 Commits

Reviewing files that changed from the base of the PR and between 1abe245 and 3bd81c3.

📒 Files selected for processing (2)
  • .github/workflows/test.yml
  • devenv.nix

Comment thread .github/workflows/test.yml Outdated
Comment thread devenv.nix
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
devenv.nix (1)

86-88: ⚠️ Potential issue | 🟠 Major

Node version assertion currently mismatches actual node --version output.

Line 87 compares against ${pkgs.nodejs_latest.version} (e.g., 22.12.0), but node --version returns v22.12.0, so this test will fail on a correct setup.

Proposed fix
   enterTest = ''
-    node --version | grep -Fx "${pkgs.nodejs_latest.version}"
+    node --version | grep -Fx "v${pkgs.nodejs_latest.version}"
   '';
Does `node --version` include a leading "v" in its output (for example, `v22.12.0`)?
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@devenv.nix` around lines 86 - 88, The test in enterTest compares raw `node
--version` output (which is like "v22.12.0") against
`${pkgs.nodejs_latest.version}` (e.g., "22.12.0"), causing a mismatch; update
the assertion in enterTest so the strings match by either comparing to a
prefixed value ("v${pkgs.nodejs_latest.version}") or normalizing the command
output (e.g., strip a leading "v") before comparison; locate the enterTest block
containing `node --version | grep -Fx ...` and change the pattern to match the
actual `node --version` output.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/workflows/test.yml:
- Around line 15-17: Replace the floating action tags with immutable commit SHAs
in the workflow: change the three usages (actions/checkout@v5,
cachix/install-nix-action@v31, cachix/cachix-action@v16) to pin to the full
commit SHA for the corresponding release commit in each upstream repo, keeping
the human-readable tag as a trailing comment if desired; locate these strings in
the workflow file and update the references to the exact commit SHAs (retrieved
from each action's GitHub repo), e.g., actions/checkout@<full-sha>,
cachix/install-nix-action@<full-sha>, cachix/cachix-action@<full-sha>.

---

Duplicate comments:
In `@devenv.nix`:
- Around line 86-88: The test in enterTest compares raw `node --version` output
(which is like "v22.12.0") against `${pkgs.nodejs_latest.version}` (e.g.,
"22.12.0"), causing a mismatch; update the assertion in enterTest so the strings
match by either comparing to a prefixed value ("v${pkgs.nodejs_latest.version}")
or normalizing the command output (e.g., strip a leading "v") before comparison;
locate the enterTest block containing `node --version | grep -Fx ...` and change
the pattern to match the actual `node --version` output.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: a1db64fc-6b21-4714-b235-3e032ad936b3

📥 Commits

Reviewing files that changed from the base of the PR and between 3bd81c3 and aea3ed1.

📒 Files selected for processing (2)
  • .github/workflows/test.yml
  • devenv.nix

Comment thread .github/workflows/test.yml Outdated
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
devenv.nix (1)

28-39: ⚠️ Potential issue | 🟡 Minor

Hook ignores passed filenames despite pass_filenames = true.

The npm-format hook is configured with pass_filenames = true (line 75), but the script ignores $@ after checking if files were passed. It runs npx prettier --write . unconditionally, formatting the entire app/ directory rather than just the staged files. This is inconsistent with the eslint hook pattern which uses "$@".

Proposed fix to format only staged files
       cd "${config.git.root}/app"
-      npx prettier --write .
+      npx prettier --write "$@"
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@devenv.nix` around lines 28 - 39, The npm-format hook currently ignores
filenames passed via pass_filenames by always running `npx prettier --write .`;
update the script in the npm-format writeShellApplication so it exits when no
files are passed (keep the existing if [ "$#" -eq 0 ] exit 0 check) but replace
the unconditional `npx prettier --write .` with a call that formats only the
staged files, e.g. run `npx prettier --write "$@"` (ensure the script cds into
config.git.root/app before invoking `npx` and preserves quoting to handle
filenames with spaces).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/workflows/test.yml:
- Around line 24-26: Update the GitHub Actions step named "Install npm
dependencies" to use `npm ci` instead of `npm install` so CI uses the lockfile
for reproducible installs; locate the step that currently runs `cd app && npm
install` and change the command to `cd app && npm ci` (ensuring
package-lock.json is committed) to enforce exact versions and fail on lockfile
mismatches.
- Around line 21-22: Pin the devenv package in the workflow instead of using the
floating nixpkgs#devenv reference: replace the nix profile add nixpkgs#devenv
invocation with a pinned reference (e.g., nixpkgs@<commit>#devenv) or switch to
the official devenv installer/flake URL with a locked revision so the job uses a
reproducible devenv version; update the step that currently runs nix profile add
nixpkgs#devenv to reference the chosen pinned revision.
- Around line 14-18: The workflow currently pins cachix/install-nix-action@v31
and cachix/cachix-action@v16 to SHAs that don't match official releases; verify
the correct commit SHA for "cachix/install-nix-action@v31" (replace
4eae8bea4afaa8f8ea8aa638ab9a7fead2f3d21e if incorrect) and for
"cachix/cachix-action@v16" (replace 97e5ff84d02b3b4f9936339f767e17f03c9252ed if
incorrect) by checking the upstream repo tags or release commits, then update
the uses entries to the correct commit IDs (or switch to the canonical tag) so
the workflow pins match the actual tagged releases.

---

Outside diff comments:
In `@devenv.nix`:
- Around line 28-39: The npm-format hook currently ignores filenames passed via
pass_filenames by always running `npx prettier --write .`; update the script in
the npm-format writeShellApplication so it exits when no files are passed (keep
the existing if [ "$#" -eq 0 ] exit 0 check) but replace the unconditional `npx
prettier --write .` with a call that formats only the staged files, e.g. run
`npx prettier --write "$@"` (ensure the script cds into config.git.root/app
before invoking `npx` and preserves quoting to handle filenames with spaces).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 44971915-2bd5-4bdf-8536-b25d8530d296

📥 Commits

Reviewing files that changed from the base of the PR and between aea3ed1 and 5605199.

📒 Files selected for processing (2)
  • .github/workflows/test.yml
  • devenv.nix

Comment thread .github/workflows/test.yml
Comment thread .github/workflows/test.yml
Comment thread .github/workflows/test.yml Outdated
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (3)
.github/workflows/test.yml (3)

25-27: 🧹 Nitpick | 🔵 Trivial

Use npm ci in CI to enforce lockfile reproducibility.

Line 27 should use npm ci (with committed app/package-lock.json) instead of npm install.

Proposed change
-        run: devenv shell -- bash -c 'cd app && npm install'
+        run: devenv shell -- bash -c 'cd app && npm ci'
#!/bin/bash
set -euo pipefail

# Verify lockfile exists for npm ci
test -f app/package-lock.json && echo "app/package-lock.json found" || {
  echo "Missing app/package-lock.json (required for npm ci)"
  exit 1
}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/test.yml around lines 25 - 27, Update the CI step named
"Install npm dependencies" to use npm ci instead of npm install to ensure
lockfile-based reproducible installs; specifically change the command invoked by
the step (the string passed to devenv shell -- bash -c 'cd app && npm install')
to run npm ci and add a pre-check that app/package-lock.json exists before
running npm ci, failing the job with a clear message if the lockfile is missing.

21-22: 🧹 Nitpick | 🔵 Trivial

Pin devenv install source to a locked revision for reproducible CI.

Line 22 currently tracks floating nixpkgs#devenv, which can drift from your locked devenv inputs.

Proposed change
-      - name: Install devenv.sh
-        run: nix profile add nixpkgs#devenv
+      - name: Install devenv.sh
+        run: nix profile add github:cachix/devenv-nixpkgs/906534d75b0e2fe74a719559dfb1ad3563485f43#devenv
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/test.yml around lines 21 - 22, Replace the floating source
in the "Install devenv.sh" step (the run command using `nix profile add
nixpkgs#devenv`) with a pinned reference that matches your repository's locked
nix inputs (e.g. point to the exact locked nixpkgs revision or the local pinned
flake entry) so the CI uses a reproducible `devenv`; update the `nix profile
add` argument to reference that fixed revision/flake entry instead of
`nixpkgs#devenv`.

17-18: ⚠️ Potential issue | 🟠 Major

Verify pinned Cachix SHAs actually correspond to the annotated tags (v31, v16).

This was previously flagged and still needs confirmation; if the SHA/tag mapping is wrong, you may run unintended action code while comments imply otherwise.

#!/bin/bash
set -euo pipefail

# Expected pinned SHAs from workflow
expected_install_nix="4eae8bea4afaa8f8ea8aa638ab9a7fead2f3d21e"
expected_cachix_action="97e5ff84d02b3b4f9936339f767e17f03c9252ed"

echo "Checking cachix/install-nix-action tag v31"
gh api repos/cachix/install-nix-action/git/ref/tags/v31 --jq '.object.sha'
gh api repos/cachix/install-nix-action/git/tags/$(gh api repos/cachix/install-nix-action/git/ref/tags/v31 --jq '.object.sha') --jq '.object.sha' 2>/dev/null || true
echo "Expected in workflow: $expected_install_nix"

echo
echo "Checking cachix/cachix-action tag v16"
gh api repos/cachix/cachix-action/git/ref/tags/v16 --jq '.object.sha'
gh api repos/cachix/cachix-action/git/tags/$(gh api repos/cachix/cachix-action/git/ref/tags/v16 --jq '.object.sha') --jq '.object.sha' 2>/dev/null || true
echo "Expected in workflow: $expected_cachix_action"
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/test.yml around lines 17 - 18, Confirm that the pinned
SHAs for the actions referenced by the workflow entries "uses:
cachix/install-nix-action@4eae8bea4afaa8f8ea8aa638ab9a7fead2f3d21e" and "uses:
cachix/cachix-action@97e5ff84d02b3b4f9936339f767e17f03c9252ed" actually
correspond to their annotated tags v31 and v16; if they do not, either replace
the pinned SHA with the correct commit SHA that the tag points to or switch to
the annotated tag reference (e.g., `@v31/`@v16) consistently, and update the two
workflow lines accordingly so the workflow uses the intended action code.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In @.github/workflows/test.yml:
- Around line 25-27: Update the CI step named "Install npm dependencies" to use
npm ci instead of npm install to ensure lockfile-based reproducible installs;
specifically change the command invoked by the step (the string passed to devenv
shell -- bash -c 'cd app && npm install') to run npm ci and add a pre-check that
app/package-lock.json exists before running npm ci, failing the job with a clear
message if the lockfile is missing.
- Around line 21-22: Replace the floating source in the "Install devenv.sh" step
(the run command using `nix profile add nixpkgs#devenv`) with a pinned reference
that matches your repository's locked nix inputs (e.g. point to the exact locked
nixpkgs revision or the local pinned flake entry) so the CI uses a reproducible
`devenv`; update the `nix profile add` argument to reference that fixed
revision/flake entry instead of `nixpkgs#devenv`.
- Around line 17-18: Confirm that the pinned SHAs for the actions referenced by
the workflow entries "uses:
cachix/install-nix-action@4eae8bea4afaa8f8ea8aa638ab9a7fead2f3d21e" and "uses:
cachix/cachix-action@97e5ff84d02b3b4f9936339f767e17f03c9252ed" actually
correspond to their annotated tags v31 and v16; if they do not, either replace
the pinned SHA with the correct commit SHA that the tag points to or switch to
the annotated tag reference (e.g., `@v31/`@v16) consistently, and update the two
workflow lines accordingly so the workflow uses the intended action code.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: a822c767-d527-4f93-83e3-0d8192e439db

📥 Commits

Reviewing files that changed from the base of the PR and between 5605199 and db83640.

📒 Files selected for processing (1)
  • .github/workflows/test.yml

@dastarruer dastarruer force-pushed the chore/add-devenv-tests branch from 5bafee2 to a3cdb48 Compare March 29, 2026 16:38
@dastarruer dastarruer force-pushed the chore/add-devenv-tests branch from 76a3885 to 93a597a Compare March 29, 2026 16:55
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/workflows/test.yml:
- Around line 24-29: In the "Install nix unstable on macOS" step replace the
echo appending to $GITHUB_ENV with writing the new PATH entry to the GITHUB_PATH
file so subsequent steps get the updated PATH; specifically, after running "nix
profile add nixpkgs#nixVersions.latest" write the path "$HOME/.nix-profile/bin"
(without an "export" prefix) into the GITHUB_PATH file so the runner prepends it
to PATH for later steps.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 7ef6c92c-df2a-405c-9e48-dd9df54ad31d

📥 Commits

Reviewing files that changed from the base of the PR and between db83640 and 93a597a.

📒 Files selected for processing (2)
  • .github/workflows/test.yml
  • devenv.nix

Comment thread .github/workflows/test.yml Outdated
@dastarruer dastarruer force-pushed the chore/add-devenv-tests branch from d45435d to c6b25f0 Compare March 29, 2026 17:13
@dastarruer dastarruer merged commit 2c7af3d into master Mar 29, 2026
9 checks passed
@dastarruer dastarruer deleted the chore/add-devenv-tests branch March 29, 2026 17:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add tests for devenv

1 participant