Add README.md into plugin package by f8thl3ss · Pull Request #10 · caido-community/dev

f8thl3ss · 2026-04-28T14:48:32Z

Bundle README.md in the plugin_package.zip

Copilot

Pull request overview

This PR updates the packaging workflow to include a project README in plugin_package.zip, along with new logic to rewrite/remove README asset URLs during bundling.

Changes:

Include a transformed README.md at the root of the generated plugin package zip.
Add a markdown/HTML URL transformer for README assets (GitHub raw URL rewriting + external URL stripping).
Extend playground fixtures and Vitest coverage to validate README inclusion and URL transformations.

Reviewed changes

Copilot reviewed 12 out of 13 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
src/commands/watch.ts	Import reordering only.
src/bundle/readme-assets.ts	New README markdown/HTML AST transformer + GitHub repo parsing + optional URL validation.
src/bundle/index.ts	Bundling now writes a transformed README into the plugin package directory before zipping.
package.json	Adds unified/remark-related dependencies and mdast types for the transformer.
pnpm-lock.yaml	Lockfile updates for new dependencies.
playgrounds/build-frontend/package.json	Adds `repository` field needed by new bundling behavior.
playgrounds/build-frontend/tests/build-frontend.spec.ts	Adds tests asserting README inclusion and URL rewrite/removal behavior.
playgrounds/build-frontend/README.md	Adds README fixture with local/external/data/HTML URL cases.
playgrounds/build-backend/package.json	Adds `repository` field needed by new bundling behavior.
playgrounds/build-backend/tests/build-backend.spec.ts	Adds tests asserting README inclusion and URL rewrite/removal behavior.
playgrounds/build-backend/README.md	Adds README fixture for backend playground.
assets/test.txt	Adds a fixture asset referenced by README tests.
assets/test.png	Adds a fixture “png” asset referenced by README HTML tests.

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-28T15:34:13Z

+  // Copy README.md (always included, always at root)
+  const readmePath = path.join(cwd, "README.md");
+  try {
+    await fs.access(readmePath);
+  } catch {
+    throw new Error("README.md is required but not found in project root");


bundlePackage now throws if the project root has no README.md. This is a breaking behavior change for existing plugin projects that may not ship a README; consider making README inclusion optional (skip if missing) or gating it behind a config/CLI flag rather than hard-failing the build.

Suggested change

// Copy README.md (always included, always at root)

const readmePath = path.join(cwd, "README.md");

try {

await fs.access(readmePath);

} catch {

throw new Error("README.md is required but not found in project root");

// Copy README.md when present; otherwise use an empty placeholder so bundling

// remains backward compatible for projects that do not ship a README.

let readmePath = path.join(cwd, "README.md");

try {

await fs.access(readmePath);

} catch {

logInfo("README.md not found in project root; skipping README contents");

readmePath = path.join(pluginPackageDir, "README.md");

await fs.writeFile(readmePath, "", "utf-8");

Copilot · 2026-04-28T15:34:14Z

+  // Read repository URL from project's package.json
+  const packageJsonPath = path.join(cwd, "package.json");
+  const packageJsonContent = await fs.readFile(packageJsonPath, "utf-8");
+  const packageJson = JSON.parse(packageJsonContent);
+  const repoUrl =
+    typeof packageJson.repository === "string"
+      ? packageJson.repository
+      : packageJson.repository?.url;
+  if (!repoUrl) {
+    throw new Error("package.json must have a valid 'repository' field");
+  }
+
+  // Parse GitHub repo info (default to main branch)
+  const repoInfo = parseGitHubRepoInfo(repoUrl, "main");
+
+  // Transform README image links to GitHub raw URLs
+  const transformedReadme = await transformReadmeImages(readmePath, repoInfo);
+
+  // Write transformed README to package directory


The bundler now hard-requires package.json.repository and assumes it points to GitHub. This will fail builds for projects without a repository field or hosted outside GitHub; consider making repo info optional (e.g., only rewrite links when a GitHub repo URL is available) and fall back to leaving README URLs unchanged when it isn’t.

Suggested change

// Read repository URL from project's package.json

const packageJsonPath = path.join(cwd, "package.json");

const packageJsonContent = await fs.readFile(packageJsonPath, "utf-8");

const packageJson = JSON.parse(packageJsonContent);

const repoUrl =

typeof packageJson.repository === "string"

? packageJson.repository

: packageJson.repository?.url;

if (!repoUrl) {

throw new Error("package.json must have a valid 'repository' field");

}

// Parse GitHub repo info (default to main branch)

const repoInfo = parseGitHubRepoInfo(repoUrl, "main");

// Transform README image links to GitHub raw URLs

const transformedReadme = await transformReadmeImages(readmePath, repoInfo);

// Write transformed README to package directory

// Read repository URL from project's package.json when available

const packageJsonPath = path.join(cwd, "package.json");

const originalReadme = await fs.readFile(readmePath, "utf-8");

let transformedReadme = originalReadme;

try {

const packageJsonContent = await fs.readFile(packageJsonPath, "utf-8");

const packageJson = JSON.parse(packageJsonContent);

const repoUrl =

typeof packageJson.repository === "string"

? packageJson.repository

: packageJson.repository?.url;

if (repoUrl) {

// Parse GitHub repo info (default to main branch) and rewrite README image links

const repoInfo = parseGitHubRepoInfo(repoUrl, "main");

transformedReadme = await transformReadmeImages(readmePath, repoInfo);

}

} catch {

// If package.json is missing, malformed, or repository is not a GitHub URL,

// leave README URLs unchanged.

}

// Write README to package directory

Copilot · 2026-04-28T15:34:14Z

+export function parseGitHubRepoInfo(
+  repoUrl: string,
+  branch = "main",
+): GitHubRepoInfo {
+  // Clean common URL prefixes/suffixes
+  const cleanedUrl = repoUrl.replace(/^git\+/, "").replace(/\.git$/, "");
+  const match = cleanedUrl.match(/github\.com[:/]([^/]+)\/([^/.]+)/);
+
+  if (!match) {
+    throw new Error(`Invalid GitHub repository URL: ${repoUrl}`);
+  }
+
+  const [, owner, repo] = match;
+
+  // Check if running in GitHub Actions
+  const isGitHubActions = process.env.GITHUB_ACTIONS === "true";
+
+  let commitHash: string;
+  if (isGitHubActions) {
+    // Use GITHUB_SHA environment variable (automatically set in GitHub Actions)
+    const githubSha = process.env.GITHUB_SHA;
+    if (!githubSha) {
+      throw new Error(
+        "GITHUB_SHA environment variable is not set in GitHub Actions",
+      );
+    }
+    commitHash = githubSha;
+    logInfo(`Using commit hash from GITHUB_SHA: ${commitHash}`);
+  } else {
+    // Not in GitHub Actions: use placeholder and log transformation
+    commitHash = "LOCAL_BUILD";
+    logInfo(
+      `Not running in GitHub Actions, skipping commit hash fetch. README image links will use placeholder URL.`,
+    );
+  }
+
+  return { owner, repo, commitHash };


parseGitHubRepoInfo accepts a branch parameter but never uses it, and local builds set commitHash to LOCAL_BUILD, which produces raw.githubusercontent.com URLs that will 404. Either use the provided branch as the ref for non-GitHub-Actions builds (or detect the local git commit), or skip URL rewriting outside CI.

Copilot · 2026-04-28T15:34:14Z

+  // Clean common URL prefixes/suffixes
+  const cleanedUrl = repoUrl.replace(/^git\+/, "").replace(/\.git$/, "");
+  const match = cleanedUrl.match(/github\.com[:/]([^/]+)\/([^/.]+)/);
+
+  if (!match) {
+    throw new Error(`Invalid GitHub repository URL: ${repoUrl}`);
+  }


The repo parsing regex cleanedUrl.match(/github\.com[:/]([^/]+)\/([^/.]+)/) rejects valid GitHub repo names containing dots (e.g. owner/my.repo). Since .git is already stripped, the repo capture group can safely allow dots (stop only at /).

Copilot · 2026-04-28T15:34:15Z

+ * Checks if a URL is external (http or https).
+ * data: URIs and fragment identifiers are not considered external.
+ */
+function isExternalUrl(url: string): boolean {
+  if (url.startsWith("data:") || url.startsWith("#")) {
+    return false;
+  }
+  try {
+    const parsed = new URL(url);
+    return parsed.protocol === "http:" || parsed.protocol === "https:";
+  } catch {
+    return false;
+  }


isExternalUrl only returns true for http/https. Absolute URLs with other schemes (e.g. mailto:, tel:, ftp:) and protocol-relative URLs (//example.com/...) will be treated as “local” and rewritten into a GitHub raw URL, which breaks those links. Consider detecting any URL with a scheme (or //) as external and either preserve it or remove it explicitly.

Suggested change

* Checks if a URL is external (http or https).

* data: URIs and fragment identifiers are not considered external.

*/

function isExternalUrl(url: string): boolean {

if (url.startsWith("data:") || url.startsWith("#")) {

return false;

}

try {

const parsed = new URL(url);

return parsed.protocol === "http:" || parsed.protocol === "https:";

} catch {

return false;

}

* Checks if a URL is external/absolute.

* data: URIs and fragment identifiers are not considered external.

*/

function isExternalUrl(url: string): boolean {

if (url.startsWith("data:") || url.startsWith("#")) {

return false;

}

if (url.startsWith("//")) {

return true;

}

return /^[A-Za-z][A-Za-z\d+.-]*:/.test(url);

Copilot · 2026-04-28T15:34:15Z

+  // Process each node type (async for URL validation via fetch)
+  for (const node of imageNodes) {
+    await transformNodeUrl(node, repoInfo, "image");
+  }
+  for (const node of linkNodes) {
+    await transformNodeUrl(node, repoInfo, "link");
+  }
+  for (const node of definitionNodes) {
+    await transformNodeUrl(node, repoInfo, "definition");
+  }
+  for (const node of htmlNodes) {
+    await transformHtmlNode(node, repoInfo);
+  }


transformReadmeImages validates/transforms URL nodes sequentially with for ... await, which can be noticeably slow for READMEs with many links/images (especially in CI where validation is enabled). Consider batching these transforms (e.g., Promise.all with a small concurrency limit) to reduce total build time.

Copilot · 2026-04-28T15:34:15Z

    "jszip": "3.10.1",
-    "ws": "8.18.0",
-    "zod": "3.24.1",
+    "remark": "^15.0.1",
+    "remark-parse": "^11.0.0",
+    "remark-stringify": "^11.0.0",
+    "tsup": "8.3.5",
+    "unified": "^11.0.5",
+    "unist-util-visit": "^5.1.0",


remark is added as a runtime dependency but isn’t imported/used anywhere in the codebase (only remark-parse and remark-stringify are). Removing the unused dependency will reduce install size and lockfile churn.

Copilot · 2026-04-28T15:34:16Z

+async function validateRawUrl(rawUrl: string): Promise<boolean> {
+  if (process.env.GITHUB_ACTIONS !== "true") {
+    return true;
+  }
+
+  try {
+    const response = await fetch(rawUrl, { method: "HEAD" });
+    if (!response.ok) {
+      logInfo(`Warning: Asset not found at ${rawUrl}, removing reference`);
+      return false;
+    }
+    return true;
+  } catch {
+    logInfo(
+      `Warning: Failed to validate asset URL ${rawUrl}, removing reference`,
+    );
+    return false;
+  }


On any transient network error during HEAD validation (catch block), validateRawUrl returns false, which causes the README link to be removed from the packaged output. This makes builds non-deterministic and can silently strip valid links. Prefer treating validation failures as “unknown” and keeping the rewritten (or original) URL, only removing links when you get a definitive non-2xx response.

Copilot · 2026-04-28T15:34:16Z

+  it("should remove external image URLs (http, https, data)", async () => {
+    const zipPath = path.resolve(__dirname, "../dist/plugin_package.zip");
+    const readmeContent = await getZipFileContent(zipPath, "README.md");
+    expect(readmeContent).toBeDefined();
+    // Verify that external URLs are removed (set to empty string)
+    // The README has: ![External Image](https://example.com/image.png)
+    // After transformation, it should be: ![](...)
+    expect(readmeContent).toMatch(/!\[External Image\]\(\)/);


This test name says it removes external image URLs "(http, https, data)", but the fixture README only contains an https external URL and the implementation intentionally preserves data: URIs. Consider renaming the test to match the actual covered behavior (or add a data: case if you want to enforce it).

… unit tests for image compression function

socket-security · 2026-04-28T18:11:11Z

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	@types/mdast@4.0.4
	unified@11.0.5
	unist-util-visit@5.1.0
	remark-parse@11.0.0
	remark-stringify@11.0.0
	parse5@8.0.1
	sharp@0.34.5
	knip@6.7.0

View full report

Add README.md into plugin package

b6ae9e0

f8thl3ss requested a review from a team as a code owner April 28, 2026 14:48

Fix assets tests

03f4a4d

f8thl3ss requested a review from Corb3nik April 28, 2026 15:25

Corb3nik requested a review from Copilot April 28, 2026 15:26

Copilot started reviewing on behalf of Corb3nik April 28, 2026 15:27 View session

Copilot AI reviewed Apr 28, 2026

View reviewed changes

Add knip, fix knip issues, migrated to inline assets in README, added…

a011430

… unit tests for image compression function

f8thl3ss added 4 commits April 28, 2026 14:15

Removed unused assets

e366c7d

Fix paths

223d1dd

Fix tests

b5f0067

Rework html element parsing

08c73f3

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add README.md into plugin package#10

Add README.md into plugin package#10
f8thl3ss wants to merge 7 commits intomainfrom
cg-bundle-readme

f8thl3ss commented Apr 28, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 28, 2026

Uh oh!

Copilot AI Apr 28, 2026

Uh oh!

Copilot AI Apr 28, 2026

Uh oh!

Copilot AI Apr 28, 2026

Uh oh!

Copilot AI Apr 28, 2026

Uh oh!

Copilot AI Apr 28, 2026

Uh oh!

Copilot AI Apr 28, 2026

Uh oh!

Copilot AI Apr 28, 2026

Uh oh!

Copilot AI Apr 28, 2026

Uh oh!

socket-security Bot commented Apr 28, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

f8thl3ss commented Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

socket-security Bot commented Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

f8thl3ss commented Apr 28, 2026 •

edited

Loading

socket-security Bot commented Apr 28, 2026 •

edited

Loading