Skip to content

Commit e9187cc

Browse files
committed
test(e2e): safety hardening e2e suite
Adds four new integration test files exercising the safety primitives end-to-end through the `socket-patch` binary, plus a shared `tests/common/mod.rs` helper module. - **e2e_safety_lock.rs** (non-ignored, 6 tests). Test process takes the same `.socket/apply.lock` via `fs2` that the binary uses, then spawns `socket-patch apply` and asserts exit 1 + JSON `errorCode: lock_held`. Zero production-code hooks — the test races the binary for the literal OS-level flock. - **e2e_safety_yarn_pnp.rs** (non-ignored, 5 tests). Yarn-berry PnP project markers (`.pnp.cjs`, `.pnp.loader.mjs`) trigger `errorCode: yarn_pnp_unsupported`. Negative control: plain npm layout does NOT trigger the refusal. - **e2e_safety_cargo_build.rs** (#[ignore], 4 tests, `--features cargo`). Honest cargo round trip via a vendored directory source. Asserts: 1. baseline `cargo check --offline --frozen` succeeds; 2. mutating a source file WITHOUT the sidecar fixup makes cargo fail with "checksum changed" (negative control — proves cargo actually verifies, which is what makes the positive test meaningful); 3. `socket-patch apply` rewrites `.cargo-checksum.json` AND the source file, and `cargo check` then succeeds; 4. the apply JSON envelope reports `.cargo-checksum.json` in `sidecarsUpdated`. - **e2e_safety_pnpm.rs** (#[ignore], 4 tests). Two projects share a pnpm content store (`--config.package-import-method=hardlink`). `socket-patch get` in project A patches A; project B AND the pnpm store entry stay byte-identical. `pnpm install --frozen-lockfile` in B afterwards does not revert A. Exercises CoW against a real pnpm install rather than a hand-rolled hardlink. CI matrix gains four entries (cargo_build + pnpm on ubuntu + macos); the lock + yarn-pnp suites are non-ignored and run via the standard `cargo test --workspace --all-features` job. Adds a `Setup pnpm` step (`npm install -g pnpm@10`) gated on the pnpm suite. `fs2` added to socket-patch-cli's dev-dependencies for the lock test (same crate the binary uses internally). Total: 19 new e2e tests. Local run: `cargo test --workspace --all-features` (435 lib + 11 fast e2e) + `cargo test --features cargo --test e2e_safety_cargo_build -- --ignored` (4) + `cargo test --test e2e_safety_pnpm -- --ignored` (4) all green. Assisted-by: Claude Code:claude-opus-4-7
1 parent 15639fe commit e9187cc

8 files changed

Lines changed: 1422 additions & 1 deletion

File tree

.github/workflows/ci.yml

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -145,6 +145,19 @@ jobs:
145145
suite: e2e_npm
146146
- os: macos-latest
147147
suite: e2e_pypi
148+
# Safety-hardening e2e suites. Both #[ignore]-gated because
149+
# they shell out to cargo / pnpm respectively. The fast
150+
# non-ignored safety suites (e2e_safety_lock,
151+
# e2e_safety_yarn_pnp) run via the standard `test` job
152+
# above, so no matrix entry is needed for them.
153+
- os: ubuntu-latest
154+
suite: e2e_safety_cargo_build
155+
- os: macos-latest
156+
suite: e2e_safety_cargo_build
157+
- os: ubuntu-latest
158+
suite: e2e_safety_pnpm
159+
- os: macos-latest
160+
suite: e2e_safety_pnpm
148161
runs-on: ${{ matrix.os }}
149162
steps:
150163
- name: Checkout
@@ -168,11 +181,18 @@ jobs:
168181
restore-keys: ${{ matrix.os }}-cargo-e2e-
169182

170183
- name: Setup Node.js
171-
if: matrix.suite == 'e2e_npm'
184+
if: matrix.suite == 'e2e_npm' || matrix.suite == 'e2e_safety_pnpm'
172185
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
173186
with:
174187
node-version: 20
175188

189+
- name: Setup pnpm
190+
if: matrix.suite == 'e2e_safety_pnpm'
191+
# `pnpm install -g pnpm` via the Node setup is the simplest
192+
# cross-platform install path. Pin the major version so the
193+
# store layout the test asserts on stays stable.
194+
run: npm install -g pnpm@10
195+
176196
- name: Setup Python
177197
if: matrix.suite == 'e2e_pypi'
178198
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5

Cargo.lock

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

crates/socket-patch-cli/Cargo.toml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,3 +38,8 @@ nuget = ["socket-patch-core/nuget"]
3838
[dev-dependencies]
3939
sha2 = { workspace = true }
4040
hex = { workspace = true }
41+
# Used by `tests/e2e_safety_lock.rs` to externally hold the same
42+
# `.socket/apply.lock` the binary takes, then spawn the binary and
43+
# assert the lock_held exit-code contract. Same crate the binary
44+
# uses internally (`socket-patch-core::patch::apply_lock`).
45+
fs2 = { workspace = true }
Lines changed: 248 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,248 @@
1+
//! Helpers shared across the e2e-safety test suites.
2+
//!
3+
//! The original e2e files (`e2e_npm.rs`, `e2e_pypi.rs`, `e2e_gem.rs`)
4+
//! each carry their own copy of the same `binary` / `run` /
5+
//! `assert_run_ok` / `git_sha256` helpers. Rather than refactor those
6+
//! files in this PR, this module is an additive landing place for the
7+
//! same surface plus the new helpers the safety suites need
8+
//! (synthetic manifest writers, pnpm runners, cargo runners). Existing
9+
//! suites can migrate in a follow-up.
10+
//!
11+
//! Each test file pulls this in with `#[path = "common/mod.rs"] mod common;`.
12+
//!
13+
//! `#![allow(dead_code)]` because each test file uses a different
14+
//! subset of these helpers; the unused ones would otherwise produce
15+
//! warnings under `-D warnings`.
16+
17+
#![allow(dead_code)]
18+
19+
use std::collections::HashMap;
20+
use std::path::{Path, PathBuf};
21+
use std::process::{Command, Output};
22+
23+
use sha2::{Digest, Sha256};
24+
25+
// ── Binary discovery + invocation ─────────────────────────────────────
26+
27+
/// Absolute path to the built `socket-patch` binary that cargo
28+
/// provides via the `CARGO_BIN_EXE_*` env var. Available because
29+
/// these tests live in the same crate that produces the binary.
30+
pub fn binary() -> PathBuf {
31+
env!("CARGO_BIN_EXE_socket-patch").into()
32+
}
33+
34+
/// Quick check whether `cmd` is on PATH. Used to soft-skip
35+
/// toolchain-dependent tests when the toolchain isn't installed
36+
/// (CI gates the toolchain at the workflow level; this is a
37+
/// belt-and-braces guard for local runs).
38+
pub fn has_command(cmd: &str) -> bool {
39+
Command::new(cmd)
40+
.arg("--version")
41+
.stdout(std::process::Stdio::null())
42+
.stderr(std::process::Stdio::null())
43+
.status()
44+
.is_ok()
45+
}
46+
47+
/// Run the CLI binary with `args`, working dir `cwd`. Returns
48+
/// `(exit_code, stdout, stderr)`. Strips `SOCKET_API_TOKEN` from the
49+
/// environment so apply paths default to the public proxy and tests
50+
/// don't accidentally exercise authed endpoints.
51+
pub fn run(cwd: &Path, args: &[&str]) -> (i32, String, String) {
52+
let out: Output = Command::new(binary())
53+
.args(args)
54+
.current_dir(cwd)
55+
.env_remove("SOCKET_API_TOKEN")
56+
.output()
57+
.expect("failed to execute socket-patch binary");
58+
let code = out.status.code().unwrap_or(-1);
59+
let stdout = String::from_utf8_lossy(&out.stdout).to_string();
60+
let stderr = String::from_utf8_lossy(&out.stderr).to_string();
61+
(code, stdout, stderr)
62+
}
63+
64+
/// `run` + assertion that exit code is 0. Returns `(stdout, stderr)`
65+
/// on success; panics with a context message + both streams on
66+
/// failure (so test logs show exactly what the binary printed).
67+
pub fn assert_run_ok(cwd: &Path, args: &[&str], context: &str) -> (String, String) {
68+
let (code, stdout, stderr) = run(cwd, args);
69+
assert_eq!(
70+
code, 0,
71+
"{context} failed (exit {code}).\nstdout:\n{stdout}\nstderr:\n{stderr}"
72+
);
73+
(stdout, stderr)
74+
}
75+
76+
// ── Hashing ───────────────────────────────────────────────────────────
77+
78+
/// Compute Git-flavored SHA-256: `SHA256("blob <len>\0" ++ content)`.
79+
/// This is the hash socket-patch records in manifests under
80+
/// `before_hash` / `after_hash`.
81+
pub fn git_sha256(content: &[u8]) -> String {
82+
let header = format!("blob {}\0", content.len());
83+
let mut hasher = Sha256::new();
84+
hasher.update(header.as_bytes());
85+
hasher.update(content);
86+
hex::encode(hasher.finalize())
87+
}
88+
89+
/// Git-SHA-256 of the file at `path`. Panics if the file can't be
90+
/// read — tests use this on paths they know exist.
91+
pub fn git_sha256_file(path: &Path) -> String {
92+
let content =
93+
std::fs::read(path).unwrap_or_else(|e| panic!("read {}: {e}", path.display()));
94+
git_sha256(&content)
95+
}
96+
97+
/// Raw lowercase-hex SHA-256 (no Git blob framing). Used by the
98+
/// Cargo sidecar which embeds plain digests in
99+
/// `.cargo-checksum.json`.
100+
pub fn sha256_hex(content: &[u8]) -> String {
101+
let mut hasher = Sha256::new();
102+
hasher.update(content);
103+
format!("{:x}", hasher.finalize())
104+
}
105+
106+
// ── Toolchain runners ─────────────────────────────────────────────────
107+
108+
/// Run `npm` in `cwd`, panic on non-zero exit with full output.
109+
pub fn npm_run(cwd: &Path, args: &[&str]) {
110+
run_toolchain(cwd, "npm", args, &[]);
111+
}
112+
113+
/// Run `pnpm` in `cwd`. Same shape as `npm_run`; `extra_env` lets
114+
/// the caller force store-dir overrides etc.
115+
pub fn pnpm_run(cwd: &Path, args: &[&str], extra_env: &[(&str, &str)]) {
116+
run_toolchain(cwd, "pnpm", args, extra_env);
117+
}
118+
119+
/// Run `cargo` in `cwd`. Returns the raw Output so callers can
120+
/// inspect stdout/stderr/exit on either pass or fail — the cargo
121+
/// e2e test wants both passing and failing cases (negative control).
122+
pub fn cargo_run(cwd: &Path, args: &[&str], extra_env: &[(&str, &str)]) -> Output {
123+
let mut cmd = Command::new("cargo");
124+
cmd.args(args).current_dir(cwd);
125+
for (k, v) in extra_env {
126+
cmd.env(k, v);
127+
}
128+
cmd.output().expect("failed to run cargo")
129+
}
130+
131+
fn run_toolchain(cwd: &Path, exe: &str, args: &[&str], extra_env: &[(&str, &str)]) {
132+
let mut cmd = Command::new(exe);
133+
cmd.args(args).current_dir(cwd);
134+
for (k, v) in extra_env {
135+
cmd.env(k, v);
136+
}
137+
let out = cmd
138+
.output()
139+
.unwrap_or_else(|e| panic!("failed to run {exe}: {e}"));
140+
assert!(
141+
out.status.success(),
142+
"{exe} {args:?} failed (exit {:?}).\nstdout:\n{}\nstderr:\n{}",
143+
out.status.code(),
144+
String::from_utf8_lossy(&out.stdout),
145+
String::from_utf8_lossy(&out.stderr),
146+
);
147+
}
148+
149+
// ── Project scaffolding ───────────────────────────────────────────────
150+
151+
/// Write a minimal package.json. Avoids `npm init -y` which rejects
152+
/// temp dir names that start with `.` or contain invalid chars.
153+
pub fn write_package_json(cwd: &Path) {
154+
std::fs::write(
155+
cwd.join("package.json"),
156+
r#"{"name":"e2e-test","version":"0.0.0","private":true}"#,
157+
)
158+
.expect("write package.json");
159+
}
160+
161+
// ── Synthetic manifest + blob construction ────────────────────────────
162+
163+
/// Describe a single patched-file row in a synthetic manifest.
164+
pub struct PatchEntry<'a> {
165+
/// File path as recorded by the manifest (may include the
166+
/// `package/` prefix used by the API; apply strips it before
167+
/// resolving against pkg_path).
168+
pub file_name: &'a str,
169+
pub before_hash: &'a str,
170+
pub after_hash: &'a str,
171+
}
172+
173+
/// Write a minimal `.socket/manifest.json` at `socket_dir/manifest.json`
174+
/// describing one patch for `purl` with the given `uuid` and `files`.
175+
///
176+
/// Returns the path to the manifest file.
177+
///
178+
/// Does NOT write the `after_hash` blobs — that's `write_blob`'s
179+
/// job, and the test gets to decide which blobs to omit (e.g. to
180+
/// force an offline-apply failure).
181+
pub fn write_minimal_manifest(
182+
socket_dir: &Path,
183+
purl: &str,
184+
uuid: &str,
185+
files: &[PatchEntry<'_>],
186+
) -> PathBuf {
187+
std::fs::create_dir_all(socket_dir).expect("create .socket dir");
188+
let mut files_map = serde_json::Map::new();
189+
for f in files {
190+
files_map.insert(
191+
f.file_name.to_string(),
192+
serde_json::json!({
193+
"beforeHash": f.before_hash,
194+
"afterHash": f.after_hash,
195+
}),
196+
);
197+
}
198+
let manifest = serde_json::json!({
199+
"patches": {
200+
purl: {
201+
"uuid": uuid,
202+
"exportedAt": "2026-01-01T00:00:00Z",
203+
"files": files_map,
204+
"vulnerabilities": {},
205+
"description": "synthetic test patch",
206+
"license": "MIT",
207+
"tier": "free",
208+
}
209+
}
210+
});
211+
let path = socket_dir.join("manifest.json");
212+
std::fs::write(&path, serde_json::to_string_pretty(&manifest).unwrap())
213+
.expect("write manifest.json");
214+
path
215+
}
216+
217+
/// Drop `content` at `<socket_dir>/blobs/<hash>`. Used to stage the
218+
/// `after_hash` blob a synthetic manifest references so apply can
219+
/// run fully offline.
220+
pub fn write_blob(socket_dir: &Path, hash: &str, content: &[u8]) {
221+
let blobs = socket_dir.join("blobs");
222+
std::fs::create_dir_all(&blobs).expect("create .socket/blobs");
223+
std::fs::write(blobs.join(hash), content).expect("write blob");
224+
}
225+
226+
/// Parse `--json` apply output, returning the top-level JSON object
227+
/// or panicking with the raw text on parse failure. Most safety tests
228+
/// want to assert on specific fields (`errorCode`, `status`, etc.).
229+
pub fn parse_json_envelope(stdout: &str) -> serde_json::Value {
230+
serde_json::from_str(stdout)
231+
.unwrap_or_else(|e| panic!("failed to parse JSON envelope: {e}\nstdout:\n{stdout}"))
232+
}
233+
234+
/// Extract a stringified field from a parsed JSON envelope, or None
235+
/// if the field is missing / not a string. Convenience for the
236+
/// `errorCode` / `status` checks the safety tests do repeatedly.
237+
pub fn json_string<'a>(env: &'a serde_json::Value, key: &str) -> Option<&'a str> {
238+
env.get(key).and_then(|v| v.as_str())
239+
}
240+
241+
/// Map a slice of `(env-var-name, env-var-value)` tuples into a
242+
/// HashMap for callers that want a stable container.
243+
pub fn env_map(pairs: &[(&str, &str)]) -> HashMap<String, String> {
244+
pairs
245+
.iter()
246+
.map(|(k, v)| ((*k).to_string(), (*v).to_string()))
247+
.collect()
248+
}

0 commit comments

Comments
 (0)