-
Notifications
You must be signed in to change notification settings - Fork 0
Description
Summary
loadEnvFromHierarchy() currently merges .env files from the eval/test folder up to repo root, but parent values win because dotenv is called with override: false.
It would be more useful if the nearest .env overrode parent .env values while still merging missing keys.
Why this matters
For evals, it is common to keep shared defaults in the repo root .env, but set per-eval workspace or repo paths in a local .env near the eval YAML.
Right now, a closer .env can add new variables, but it cannot override an existing variable that was already loaded from a parent/root .env.
That makes per-eval configuration awkward and forces repeated shell exports for variables that should be naturally scoped to the eval folder.
Current behavior
If both of these exist:
Repo root .env:
MY_VAR=root
SHARED_ONLY=from_rootSubfolder .env (for example evals/foo/.env):
MY_VAR=local
LOCAL_ONLY=from_subfolderAnd an eval is run from that subtree, both .env files are loaded, but MY_VAR remains root.
Expected behavior
AgentV should still merge .env files from the hierarchy, but the closest .env should take precedence.
Expected effective environment:
MY_VAR=localSHARED_ONLY=from_rootLOCAL_ONLY=from_subfolder
Repro steps
-
Create a repo root
.envwith:MY_VAR=root SHARED_ONLY=from_root
-
Create a nested eval folder
.envwith:MY_VAR=local LOCAL_ONLY=from_subfolder
-
Run an eval from the nested folder with
--verbose. -
Observe that AgentV logs both
.envfiles as loaded, but the parent/root value still wins forMY_VAR.
Implementation note
In agentv 2.18.2, this behavior comes from loadEnvFromHierarchy().
Installed build reference:
dist/chunk-ZCUOH72A.js- around lines
304-329
This line appears to be the key behavior:
loadDotenv({ path: envFile, override: false });Because files are loaded from higher scope to lower scope with override: false, nearer .env files cannot override values already set by parents.
Suggested fix
Either:
- keep the current load order and use
override: truefor later/nearer.envfiles, or - reverse the load order so the nearest
.envis loaded first and then preserve current semantics only for missing keys
The important behavior is:
- merge all discovered
.envfiles - let the nearest
.envoverride parent values - keep parent/root values for keys that are not set locally