Skip to content
4 changes: 4 additions & 0 deletions Changelog.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# FOSSA CLI Changelog

## Unreleased

- Node.js: Yarn, npm, and pnpm workspace packages now appear as individual build targets (e.g. `yarn@./:my-package`, `npm@./:my-package`, `pnpm@./:my-package`), enabling per-package dependency scoping via `.fossa.yml`.

## 3.16.0

- nodejs: Add support for Bun lockfiles (`bun.lock`). Analyzes npm and git dependencies, workspaces, and environment labeling. ([#1648](https://github.com/fossas/fossa-cli/pull/1648))
Expand Down
23 changes: 23 additions & 0 deletions docs/references/strategies/languages/nodejs/npm-lockfile.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,29 @@ Search for files named `package.json` and check for a corresponding
> will be combined to determine which dependencies are direct and which ones are
> development.

### Workspace Build Targets

Each workspace package (including the root) is exposed as an individual build
target. For example, a monorepo named `my-monorepo` with packages `api` and
`web` will produce targets:

```
npm@./:my-monorepo
npm@./:api
npm@./:web
```

When a subset of targets is selected, only those packages' dependencies are
included in the analysis.

When no filtering is applied, all targets are selected and all dependencies
from every workspace package are included in the analysis.

> Note: Target-level dependency scoping is only supported for lockfile version 1
> (npm v5–v6) and version 2 (npm v7–v8). Version 3 lockfiles (npm v9+) will
> show workspace build targets, but filtering to specific targets does not yet
> scope the dependency results.

## Analysis (for lockFile version 3)

We consider `package-lock.json` to be version 3 compatible, if and only if,
Expand Down
7 changes: 6 additions & 1 deletion docs/references/strategies/languages/nodejs/pnpm.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,12 @@ CLI will infer the package name and version using `/${dependencyName}/${dependen
```

* Peer dependencies will be included in the analysis (they are treated like any other dependency).
* Pnpm workspaces are supported.
* Pnpm workspaces are supported. Each workspace package (including the root)
is exposed as an individual build target (e.g. `pnpm@./:my-app`,
`pnpm@./:lib-utils`). When a subset of targets is selected, only those
packages' dependencies are included in the analysis. When no filtering is
applied, all targets are selected and all dependencies from every workspace
package are included in the analysis.
* Development dependencies (`dev: true`) are ignored by default from analysis. To include them in the analysis, execute CLI with `--include-unused` flag e.g. `fossa analyze --include-unused`.
* Optional dependencies are included in the analysis by default. They can be ignored in FOSSA UI.
* `fossa-cli` supports lockFileVersion: 4.x, 5.x, 6.x, 7.x, 8.x, and 9.x.
Expand Down
18 changes: 18 additions & 0 deletions docs/references/strategies/languages/nodejs/yarn.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,24 @@ the `package.json` files used to build the `yarn.lock` file are also checked,
and the knowledge of both is combined to form a (usually) complete picture of
the full graph of dependencies.

### Workspace Build Targets

Each workspace package (including the root) is exposed as an individual build
target. For example, a monorepo named `my-monorepo` with packages `app`,
`lib-utils`, and `lib-core` will produce targets:

```
yarn@./:my-monorepo
yarn@./:app
yarn@./:lib-utils
yarn@./:lib-core
```

When a subset of targets is selected, only those packages' dependencies are
included in the analysis.

When no filtering is applied, all targets are selected and all dependencies
from every workspace package are included in the analysis.

## FAQ

Expand Down
137 changes: 116 additions & 21 deletions src/Strategy/Node.hs
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ module Strategy.Node (
pkgGraph,
NodeProject (..),
getDeps,
findWorkspaceBuildTargets,
extractDepListsForTargets,
resolveImporterPaths,
) where

import Algebra.Graph.AdjacencyMap qualified as AM
Expand Down Expand Up @@ -36,7 +39,8 @@ import Data.Map.Strict qualified as Map
import Data.Maybe (catMaybes, isJust, mapMaybe)
import Data.Set (Set)
import Data.Set qualified as Set
import Data.String.Conversion (decodeUtf8, toString)
import Data.Set.NonEmpty qualified as NonEmptySet
import Data.String.Conversion (decodeUtf8, toString, toText)
import Data.Tagged (applyTag)
import Data.Text (Text)
import Data.Text qualified as Text
Expand Down Expand Up @@ -70,6 +74,7 @@ import Path (
Rel,
mkRelFile,
parent,
stripProperPrefix,
toFilePath,
(</>),
)
Expand All @@ -96,16 +101,19 @@ import Strategy.Node.Pnpm.PnpmLock qualified as PnpmLock
import Strategy.Node.Pnpm.Workspace (PnpmWorkspace (workspaceSpecs))
import Strategy.Node.YarnV1.YarnLock qualified as V1
import Strategy.Node.YarnV2.YarnLock qualified as V2
import System.FilePath.Posix qualified as FP
import Types (
BuildTarget (BuildTarget),
DependencyResults (DependencyResults),
DiscoveredProject (..),
DiscoveredProjectType (BunProjectType, NpmProjectType, PnpmProjectType, YarnProjectType),
FoundTargets (ProjectWithoutTargets),
FoundTargets (FoundTargets, ProjectWithoutTargets),
GraphBreadth (Complete, Partial),
License (License),
LicenseResult (LicenseResult, licensesFound),
LicenseType (LicenseURL, UnknownType),
licenseFile,
unBuildTarget,
)

skipJsFolders :: WalkStep
Expand Down Expand Up @@ -149,50 +157,109 @@ mkProject project = do
NPM g -> (g, NpmProjectType)
Bun _ g -> (g, BunProjectType)
Pnpm _ g -> (g, PnpmProjectType)
-- Only expose build targets for project types whose getDeps actually
-- honors them. Otherwise users see per-package targets in list-targets
-- but filtering has no effect on analysis.
projectBuildTargets' = case project of
Yarn _ _ -> findWorkspaceBuildTargets graph
NPMLock _ _ -> findWorkspaceBuildTargets graph
Pnpm _ _ -> findWorkspaceBuildTargets graph
_ -> ProjectWithoutTargets
Manifest rootManifest <- fromEitherShow $ findWorkspaceRootManifest graph
pure $
DiscoveredProject
{ projectType = typename
, projectPath = parent rootManifest
, projectBuildTargets = ProjectWithoutTargets
, projectBuildTargets = projectBuildTargets'
, projectData = project
}

-- | Build targets from workspace package names (root + members).
-- If the workspace graph has children (i.e., workspace members), each
-- package name (including the root) becomes a 'BuildTarget'. If there
-- are no workspace children (single-package project), returns
-- 'ProjectWithoutTargets'.
findWorkspaceBuildTargets :: PkgJsonGraph -> FoundTargets
findWorkspaceBuildTargets graph@PkgJsonGraph{..} =
let WorkspacePackageNames childNames = findWorkspaceNames graph
in if Set.null childNames
then ProjectWithoutTargets
else
let rootName = findWorkspaceRootManifest graph >>= \m -> maybe (Left "no name") Right (packageName =<< Map.lookup m jsonLookup)
in case rootName of
-- If the root package.json has no name field, fall back to
-- ProjectWithoutTargets so its deps aren't silently dropped.
Left _ -> ProjectWithoutTargets
Right n ->
let allNames = Set.insert n childNames
in maybe ProjectWithoutTargets FoundTargets (NonEmptySet.nonEmpty (Set.map BuildTarget allNames))

instance AnalyzeProject NodeProject where
analyzeProject _ = getDeps
analyzeProjectStaticOnly _ = getDeps
analyzeProject = getDeps
analyzeProjectStaticOnly = getDeps

-- Since we don't natively support workspaces, we don't attempt to preserve them from this point on.
-- In the future, if you're adding generalized workspace support, start here.
getDeps ::
( Has ReadFS sig m
, Has Diagnostics sig m
, Has Logger sig m
) =>
FoundTargets ->
NodeProject ->
m DependencyResults
getDeps (Yarn yarnLockFile graph) = analyzeYarn yarnLockFile graph
getDeps (NPMLock packageLockFile graph) = analyzeNpmLock packageLockFile graph
getDeps (Pnpm pnpmLockFile _) = analyzePnpmLock pnpmLockFile
getDeps (Bun bunLockFile _) = analyzeBunLock bunLockFile
getDeps (NPM graph) = analyzeNpm graph

analyzePnpmLock :: (Has Diagnostics sig m, Has ReadFS sig m, Has Logger sig m) => Manifest -> m DependencyResults
analyzePnpmLock (Manifest pnpmLockFile) = do
result <- PnpmLock.analyze pnpmLockFile
getDeps targets (Yarn yarnLockFile graph) = analyzeYarn targets yarnLockFile graph
getDeps targets (NPMLock packageLockFile graph) = analyzeNpmLock targets packageLockFile graph
getDeps targets (Pnpm pnpmLockFile graph) = analyzePnpmLock targets pnpmLockFile graph
getDeps _ (Bun bunLockFile _) = analyzeBunLock bunLockFile
getDeps _ (NPM graph) = analyzeNpm graph

analyzePnpmLock :: (Has Diagnostics sig m, Has ReadFS sig m, Has Logger sig m) => FoundTargets -> Manifest -> PkgJsonGraph -> m DependencyResults
analyzePnpmLock targets (Manifest pnpmLockFile) graph = do
let selectedImporterPaths = resolveImporterPaths targets graph
result <- PnpmLock.analyze selectedImporterPaths pnpmLockFile
pure $ DependencyResults result Complete [pnpmLockFile]

analyzeBunLock :: (Has Diagnostics sig m, Has ReadFS sig m) => Manifest -> m DependencyResults
analyzeBunLock (Manifest bunLockFile) = do
result <- BunLock.analyze bunLockFile
pure $ DependencyResults result Complete [bunLockFile]

analyzeNpmLock :: (Has Diagnostics sig m, Has ReadFS sig m) => Manifest -> PkgJsonGraph -> m DependencyResults
analyzeNpmLock (Manifest npmLockFile) graph = do
-- | Map selected build targets (package names) to pnpm importer paths
-- (relative paths from the workspace root like ".", "packages/a").
-- When 'ProjectWithoutTargets', returns Nothing (no filtering).
-- When 'FoundTargets', looks up each target's package name in the
-- PkgJsonGraph to find its manifest path, then computes the relative
-- path from the workspace root.
resolveImporterPaths :: FoundTargets -> PkgJsonGraph -> Maybe (Set Text)
resolveImporterPaths ProjectWithoutTargets _ = Nothing
resolveImporterPaths (FoundTargets targets) graph@PkgJsonGraph{..} =
case findWorkspaceRootManifest graph of
Left _ -> Nothing
Right (Manifest rootManifest) ->
Just $ Set.fromList [p | (name, p) <- namePathPairs, name `Set.member` targetNames]
where
rootDir = parent rootManifest
targetNames = Set.map unBuildTarget (NonEmptySet.toSet targets)

namePathPairs :: [(Text, Text)]
namePathPairs =
[ (name, manifestToImporterPath m)
| (Manifest m, pj) <- Map.toList jsonLookup
, Just name <- [packageName pj]
]

manifestToImporterPath :: Path Abs File -> Text
manifestToImporterPath m =
let manifestDir = parent m
in if manifestDir == rootDir
then "."
else maybe "." (toText . FP.dropTrailingPathSeparator . toFilePath) (stripProperPrefix rootDir manifestDir)

analyzeNpmLock :: (Has Diagnostics sig m, Has ReadFS sig m) => FoundTargets -> Manifest -> PkgJsonGraph -> m DependencyResults
analyzeNpmLock targets (Manifest npmLockFile) graph = do
npmLockVersion <- detectNpmLockVersion npmLockFile
result <- case npmLockVersion of
NpmLockV3Compatible -> PackageLockV3.analyze npmLockFile
NpmLockV1Compatible -> PackageLock.analyze npmLockFile (extractDepLists graph) (findWorkspaceNames graph)
NpmLockV1Compatible -> PackageLock.analyze npmLockFile (extractDepListsForTargets targets graph) (findWorkspaceNames graph)
pure $ DependencyResults result Complete [npmLockFile]

analyzeNpm :: (Has Diagnostics sig m) => PkgJsonGraph -> m DependencyResults
Expand All @@ -216,16 +283,17 @@ analyzeYarn ::
( Has Diagnostics sig m
, Has ReadFS sig m
) =>
FoundTargets ->
Manifest ->
PkgJsonGraph ->
m DependencyResults
analyzeYarn (Manifest yarnLockFile) pkgJsonGraph = do
analyzeYarn targets (Manifest yarnLockFile) pkgJsonGraph = do
yarnVersion <- detectYarnVersion yarnLockFile
let analyzeFunc = case yarnVersion of
V1 -> V1.analyze
V2Compatible -> V2.analyze

graph <- analyzeFunc yarnLockFile $ extractDepLists pkgJsonGraph
graph <- analyzeFunc yarnLockFile $ extractDepListsForTargets targets pkgJsonGraph
pure . DependencyResults graph Complete $ yarnLockFile : pkgFileList pkgJsonGraph

detectYarnVersion ::
Expand Down Expand Up @@ -288,6 +356,33 @@ extractDepLists PkgJsonGraph{..} = foldMap extractSingle $ Map.elems jsonLookup
(applyTag @Development $ mapToSet packageDevDeps)
(Map.keysSet jsonLookup)

-- | Like 'extractDepLists', but scoped to the selected workspace targets.
-- When 'ProjectWithoutTargets', includes all deps.
-- When 'FoundTargets', only includes deps from packages whose
-- package name matches a selected target (root or workspace member).
extractDepListsForTargets :: FoundTargets -> PkgJsonGraph -> FlatDeps
extractDepListsForTargets ProjectWithoutTargets graph = extractDepLists graph
extractDepListsForTargets (FoundTargets targets) PkgJsonGraph{..} =
foldMap extractSingle selectedPackageJsons
where
targetNames :: Set Text
targetNames = Set.map unBuildTarget (NonEmptySet.toSet targets)

selectedPackageJsons :: [PackageJson]
selectedPackageJsons =
filter (maybe False (`Set.member` targetNames) . packageName) $
Map.elems jsonLookup

mapToSet :: Map Text Text -> Set NodePackage
mapToSet = Set.fromList . map (uncurry NodePackage) . Map.toList

extractSingle :: PackageJson -> FlatDeps
extractSingle PackageJson{..} =
FlatDeps
(applyTag @Production $ mapToSet (packageDeps `Map.union` packagePeerDeps))
(applyTag @Development $ mapToSet packageDevDeps)
(Map.keysSet jsonLookup)

loadPackage :: (Has Logger sig m, Has ReadFS sig m, Has Diagnostics sig m) => Manifest -> m (Maybe (Manifest, PackageJson))
loadPackage (Manifest file) = do
result <- recover $ readContentsJson @PackageJson file
Expand Down
18 changes: 12 additions & 6 deletions src/Strategy/Node/Pnpm/PnpmLock.hs
Original file line number Diff line number Diff line change
Expand Up @@ -290,20 +290,26 @@ instance FromJSON Resolution where
gitRes :: Object -> Parser Resolution
gitRes obj = GitResolve <$> (GitResolution <$> obj .: "repo" <*> obj .: "commit")

analyze :: (Has ReadFS sig m, Has Logger sig m, Has Diagnostics sig m) => Path Abs File -> m (Graphing Dependency)
analyze file = context "Analyzing Npm Lockfile (v3)" $ do
-- | Analyze a pnpm lockfile. When @selectedImporterPaths@ is 'Nothing',
-- all importers are included. When @Just paths@, only importers whose
-- key is in @paths@ are treated as direct dependency sources.
analyze :: (Has ReadFS sig m, Has Logger sig m, Has Diagnostics sig m) => Maybe (Set.Set Text) -> Path Abs File -> m (Graphing Dependency)
analyze selectedImporterPaths file = context "Analyzing Npm Lockfile (v3)" $ do
pnpmLockFile <- context "Parsing pnpm-lock file" $ readContentsYaml file

case lockFileVersion pnpmLockFile of
PnpmLockLt4 raw -> logWarn . pretty $ "pnpm-lock file is using older lockFileVersion: " <> raw <> " of, which is not officially supported!"
_ -> pure ()

context "Building dependency graph" $ pure $ buildGraph pnpmLockFile
context "Building dependency graph" $ pure $ buildGraph selectedImporterPaths pnpmLockFile

buildGraph :: PnpmLockfile -> Graphing Dependency
buildGraph lockFile = withoutLocalPackages $
buildGraph :: Maybe (Set.Set Text) -> PnpmLockfile -> Graphing Dependency
buildGraph selectedImporterPaths lockFile = withoutLocalPackages $
run . evalGrapher $ do
for_ (toList lockFile.importers) $ \(_, projectImporters) -> do
let filteredImporters = case selectedImporterPaths of
Nothing -> toList lockFile.importers
Just paths -> filter (\(k, _) -> k `Set.member` paths) (toList lockFile.importers)
for_ filteredImporters $ \(_, projectImporters) -> do
let allDirectDependencies =
toList (directDependencies projectImporters)
<> toList (directDevDependencies projectImporters)
Expand Down
Loading
Loading