Seperate ks env

README.md

@@ -7,125 +7,202 @@ Play around with it and raise Github issues if anything fails
 
 # Setting up
 
-1. Install `conda`
-   - You can use either [Miniconda](https://docs.anaconda.com/miniconda/install/#quick-command-line-install) or [Miniforge](https://github.com/conda-forge/miniforge)
-2. Clone repo
+## 1. Install `bnd`
+
+### Option A — pipx (recommended)
+
+[pipx](https://pipx.pypa.io) installs `bnd` in an isolated environment and makes the CLI available system-wide.
+
+1. Install pipx if you don't have it:
    ```shell
-   git clone git@github.com:BeNeuroLab/bnd.git
-   cd ./bnd
+   # Windows (requires Python ≥ 3.10)
+   pip install pipx
+   pipx ensurepath   # restart your terminal after this
+
+   # Linux
+   sudo apt install pipx
+   pipx ensurepath
    ```
-3. Open either Miniconda prompt or Miniforge promt and run the following command. This 
-   may take some time: 
+
+2. Install `bnd`:
    ```shell
-   conda env create --file=env.yml
+   # Lightweight (upload, download, config only — fast install):
+   pipx install "bnd @ git+https://github.com/BeNeuroLab/bnd.git"
+
+   # Full install with processing dependencies (NWB, kilosort, pyaldata):
+   pipx install "bnd[processing] @ git+https://github.com/BeNeuroLab/bnd.git"
    ```
-   or if you want the processing depedencies:
+   To install a specific branch (e.g. for testing):
    ```shell
-   conda env create --file=processing_env.yml
+   pipx install "bnd[processing] @ git+https://github.com/BeNeuroLab/bnd.git@seperate-ks-env"
    ```
 
-   For kilosorting you will also need:
-   1. Install kilosort and the GUI, run `python -m pip install kilosort[gui]`. If you're on a zsh server, you may need to use `python -m pip install "kilosort[gui]"` 
-   2. You can also just install the minimal version of kilosort with python -m pip install kilosort.
-   3. Next, if the CPU version of pytorch was installed (will happen on Windows), remove it with `pip uninstall torch`
-   4. Then install the GPU version of pytorch `conda install pytorch pytorch-cuda=11.8 -c pytorch -c nvidia`
+3. Verify:
+   ```shell
+   bnd --help
+   ```
 
-   If you installed the base environment and want to update later on:
+To **update** to the latest commits:
+```shell
+pipx install --force "bnd[processing] @ git+https://github.com/BeNeuroLab/bnd.git"
+```
+
+### Option B — conda
+
+1. Install [Miniconda](https://docs.anaconda.com/miniconda/install/#quick-command-line-install) or [Miniforge](https://github.com/conda-forge/miniforge).
+2. Clone the repo and create the environment:
+   ```shell
+   git clone git@github.com:BeNeuroLab/bnd.git
+   cd ./bnd
+   conda env create --file=processing_env.yml   # includes scientific dependencies
+   conda activate bnd
+   pip install -e .
+   ```
+   To update later:
    ```shell
    conda env update --file=processing_env.yml
    ```
-   And then do the kilosort step
-4. Create your configuration file:
+
+## 2. Set up Kilosort (separate conda env)
+
+Kilosort runs in its own conda environment — `bnd` invokes it via `conda run -n kilosort ...`.
+
+1. Create and activate the env:
+   ```shell
+   conda create -n kilosort python=3.10 pip
+   conda activate kilosort
+   ```
+2. Install Kilosort following the [official instructions](https://github.com/MouseLand/Kilosort):
+   ```shell
+   python -m pip install "kilosort[gui]"
+   ```
+   Or minimal (no GUI):
    ```shell
-   bnd init  # Provide the path to local and remote data storage
-   bnd --help # Start reading about the functions!
+   python -m pip install kilosort
+   ```
+3. Install GPU-enabled PyTorch (example for CUDA 11.8):
+   ```shell
+   conda install pytorch pytorch-cuda=11.8 -c pytorch -c nvidia
+   ```
+
+> **Note:** If your env is not named `kilosort`, set the environment variable `BND_KILOSORT_ENV` to
+> the env name before running `bnd`.
+
+## 3. Configure `bnd`
+
+```shell
+bnd init    # Provide the path to local and remote data storage
+bnd --help  # Start reading about the functions!
+```
 
 # Example usage
+
 Complete your experimental session on animal M099. Then:
+
 ```shell
 bnd up M099
 ```
 
 Now, you want to process your data into a pyaldata format. Its a good idea to do this on one of the lab workstations:
+
 ```shell
 bnd dl M099_2025_01_01_10_00 -v  # Downloads everything
 bnd to-pyal M099_2025_01_01_10_00  # Run kilosort, nwb conversion, and pyaldata conversion
 bnd up M099_2025_01_01_10_00  # Uploads new files to server
 ```
-If you want specific things during your pipeline (e.g., dont run kilosort, use a custom channel map) read the API below. 
 
+If you want specific things during your pipeline (e.g., dont run kilosort, use a custom channel map) read the API below.
 
 # API
 
 ## Config
+
 ### `bnd init`
+
 Create a .env file (if there isnt one) to store the paths to the local and remote data storage.
 
 ### `bnd show-config`
+
 Show the contents of the config file.
 
 ## Updating
+
 ### `bnd check-updates`
+
 Check if there are any new commits on the repo's main branch.
 
 ### `bnd self-update`
-Update the bnd tool by pulling the latest commits from the repo's main branch.
 
+Update the bnd tool by pulling the latest commits from the repo's main branch.
 
 ## Data Transfer
+
 ### `bnd up <session_or_animal_name>`
+
 Upload data from session or animal name to the server. If the file exists on the server, it won't be replaced. Every file in the session folder will get uploaded.
 
 Example usage to upload everything of a given session:
+
 ```shell
 bnd up M017_2024_03_12_18_45
 bnd up M017
 ```
+
 ### `bnd dl <session>`
+
 Download experimental data from a given session from the remote server.
 
 Example usage to download everything:
+
 ```shell
 bnd dl M017_2024_03_12_18_45 -v  # will download everything, including videos
 bnd dl M017_2024_03_12_18_45  # will download everything, except videos
 bnd dl M017_2024_03_12_18_45 --max-size=50  # will download files smaller than 50MB
 ```
 
 ## Pipeline
+
 ### `bnd to-pyal <session>`
+
 Convert session data into a pyaldata dataframe and saves it as a .mat
 
 If no .nwb file is present it will automatically generate one and if a nwb file is present it will skip it. If you want to generate a new one run `bnd to-nwb`
 
 If no kilosorted data is available it will not kilosort by default. If you want to kilosort add the flag `-k`
 
 Example usage:
+
 ```shell
 bnd to-pyal M037_2024_01_01_10_00  # Kilosorts data, runs nwb and converts to pyaldata
 bnd to-pyal M037_2024_01_01_10_00 -K  # converts to pyaldata without kilosorting (if no .nwb file is present)
 bnd to-pyal M037_2024_01_01_10_00 -c  # Use custom mapping during nwb conversion if custom_map.json is available (see template in repo). -C uses available default mapping
 ```
 
 ### `bnd to-nwb <session>`
+
 Convert session data into a nwb file and saves it as a .nwb
 
 If no kilosorted data is available it will not kilosort by default. If you want to kilosort add the flag `-k`
 
 Example usage:
+
 ```shell
 bnd to-nwb M037_2024_01_01_10_00  # Kilosorts data and run nwb
 bnd to-nwb M037_2024_01_01_10_00 -K  # converts to nwb without kilosorting (if no .nwb file is present)
 bnd to-nwb M037_2024_01_01_10_00 -c  # Use custom mapping during conversion if custom_map.json is available (see template in repo). Option `-C` uses available default mapping
 ```
 
 ### `bnd ksort <session>`
+
 Kilosorts data from a single session on all available probes and recordings
 
 Example usage:
+
 ```shell
 bnd ksort M037_2024_01_01_10_00
 ```
 
 # TODOs:
+
 - Add `AniposeInterface` in nwb conversion
 - Implement Npx2.0 functionality

bnd/cli.py

@@ -8,7 +8,6 @@
 from rich import print
 
 from .config import (
-    _check_is_git_track,
     _check_root,
     _check_session_directory,
     _get_env_path,
@@ -311,8 +310,6 @@ def init():
 
     else:
         print("\nConfig file doesn't exist. Let's create one.")
-        repo_path = _get_package_path()
-        _check_is_git_track(repo_path)
 
         local_path = Path(
             typer.prompt("Enter the absolute path to the root of the local data storage")
@@ -325,7 +322,6 @@ def init():
         _check_root(remote_path)
 
         with open(env_path, "w") as f:
-            f.write(f"REPO_PATH = {repo_path}\n")
             f.write(f"LOCAL_PATH = {local_path}\n")
             f.write(f"REMOTE_PATH = {remote_path}\n")
 

bnd/config.py

@@ -14,12 +14,32 @@ def _get_package_path() -> Path:
     return Path(__file__).absolute().parent.parent
 
 
+def _get_config_dir() -> Path:
+    """
+    Returns the path to the bnd configuration directory (~/.bnd/).
+    Creates it if it doesn't exist.
+    """
+    config_dir = Path.home() / ".bnd"
+    config_dir.mkdir(parents=True, exist_ok=True)
+    return config_dir
+
+
 def _get_env_path() -> Path:
     """
     Returns the path to the .env file containing the configuration settings.
+    Checks ~/.bnd/.env first, falls back to legacy location (next to package) for migration.
     """
-    package_path = _get_package_path()
-    return package_path / ".env"
+    new_path = _get_config_dir() / ".env"
+    if new_path.exists():
+        return new_path
+
+    # Legacy: config stored next to the package source (editable / conda installs)
+    legacy_path = _get_package_path() / ".env"
+    if legacy_path.exists():
+        return legacy_path
+
+    # Default to the new location for fresh installs
+    return new_path
 
 
 def _check_session_directory(session_path):
@@ -48,7 +68,6 @@ class Config:
     def __init__(self, env_path=_get_env_path()):
         self.REMOTE_PATH = None
         self.LOCAL_PATH = None
-        self.REPO_PATH = None
         # Load the actual environment PATHs
         self.load_env(env_path)
         self.datetime_pattern = "%Y_%m_%d_%H_%M"

bnd/pipeline/__init__.py

@@ -8,9 +8,12 @@ def _check_processing_dependencies():
         from .kilosort import run_kilosort_on_session
         from .nwb import run_nwb_conversion
         from .pyaldata import run_pyaldata_conversion
-    except Exception as e:
+    except ImportError as e:
         raise ImportError(
-            f"Could not import processing dependencies: {e}. Update your environment "
-            "with `conda env update -n bnd --file=processing_env.yml`"
-        )
+            f"Missing processing dependencies: {e}.\n"
+            "Install them with:\n"
+            '  pipx install --force "bnd[processing] @ git+https://github.com/BeNeuroLab/bnd.git"\n'
+            "or:\n"
+            '  pip install "bnd[processing]"'
+        ) from e
     return