Better support for dependencies shared between multiple projects

[coderabbitai]

Problem

In general, install-deps-and-generate-settings is was not designed to handle multiple projects. Trying to combine projects in the same CMAKE_SETTINGS_DIR has a lot of friction with repeated work and confusing behaviour.

Ideally, each project calling install-deps-and-generate-settings would be able to share their dependencies and generate correct CMAKE_SETTINGS_FILEs for their usage.

Example:

When multiple projects within a single repository (e.g., spider and clp) call install-deps-and-generate-settings with:

• The same CMAKE_SETTINGS_DIR
• Different CMAKE_SETTINGS_FILE values (e.g., all-core.cmake and all-spider.cmake)

The current implementation handles shared dependencies correctly, but the generated CMake settings files may include unnecessary dependencies from other projects.

Context

This issue was identified during the refactoring of dependency tasks in y-scope/clp#1377. The PR works around the limitation by splitting dependency tasks into separate modular files, but a proper solution in dev-utils would provide better support for this use case.

Requested Enhancement

Add better support for scenarios where multiple projects in the same repository share dependencies and need to call install-deps-and-generate-settings multiple times with different output files.

References

• PR discussion: refactor(taskfile): Split main dependency Taskfile into category-specific modules; Migrate antlr-runtime and utfcpp to use utils:install-remote-cmake-lib. clp#1377 (comment)
• Original PR introducing spider deps: feat(clp-package): Add Spider and its dependencies to the build. clp#1318
• Follow-up mitigation: fix(taskfile): Make deps:core and deps:spider-dep write into different top-level CMake configuration files (fixes #1330). clp#1331
• Requested by: @sitaowang1998

[sitaowang1998]

The major problem is not unnecessary files. This is only a good-to-have feature.
The biggest problem is the install-deps-and-generate-settings removes and recreates the CMAKE_SETTING_DIR. This leads to concurrency issue when more than one tasks execute this task.
Another good-to-have feature is to creates directory of CMAKE_SETTING_FILE in install-deps-and-generate-settings, so we can write the combined cmake setting files into different folders.

[Bill-hbrhbr]

The major problem is not unnecessary files. This is only a good-to-have feature.
The biggest problem is the install-deps-and-generate-settings removes and recreates the CMAKE_SETTING_DIR. This leads to concurrency issue when more than one tasks execute this task.
Another good-to-have feature is to creates directory of CMAKE_SETTING_FILE in install-deps-and-generate-settings, so we can write the combined cmake setting files into different folders.

@coderabbitai
Open an issue in CLP about resolving concurrency issue in yscope-dev-utils:cmake:install-deps-and-generate-settings removes and recreates the CMAKE_SETTING_DIR

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash

# Get details about the current issue to provide context
gh issue view 89 --repo y-scope/yscope-dev-utils --json title,body,url

# Create an issue in the CLP repository about the concurrency issue
gh issue create \
  --repo y-scope/clp \
  --title "Resolve concurrency issue in yscope-dev-utils:cmake:install-deps-and-generate-settings" \
  --body "## Problem

The \`install-deps-and-generate-settings\` task in yscope-dev-utils removes and recreates the \`CMAKE_SETTING_DIR\`. This leads to a concurrency issue when more than one task executes this simultaneously.

## Context

This issue was identified in y-scope/yscope-dev-utils#89.

## Requested by

@Bill-hbrhbr" \
  --assignee Bill-hbrhbr

Length of output: 1362

---

@Bill-hbrhbr I've successfully created issue #1553 in the CLP repository to track the concurrency issue with yscope-dev-utils:cmake:install-deps-and-generate-settings removing and recreating CMAKE_SETTING_DIR.

You can view the issue here: y-scope/clp#1553

The issue has been assigned to you and includes a backlink to this discussion.

---

🧠 Learnings used

Learnt from: Bill-hbrhbr
Repo: y-scope/yscope-dev-utils PR: 91
File: exports/taskfiles/utils/misc.yaml:70-90
Timestamp: 2025-10-13T19:09:44.404Z
Learning: In the yscope-dev-utils repository Taskfiles, prefer minimal guardrails that only protect against benevolent mistakes (e.g., enforcing absolute paths where required) rather than defensive checks like rejecting root paths, empty paths, or validating file vs directory types.

[davidlion]

#91

Resolves a race condition when sharing CMAKE_SETTING_DIR across multiple projects. However, the larger design issue with install-deps-and-generate-settings remains.