Skip to content

Commit 951a195

Browse files
precicelyprecicely
authored
Merge branch 'master' into 460-element-equality
2 parents de5a883 + b24bfa1 commit 951a195

5 files changed

Lines changed: 152 additions & 71 deletions

File tree

‎.github/workflows/pytest.yml‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -58,16 +58,15 @@ jobs:
5858
files: |
5959
./collectedValues
6060
outPath: collectedValues.tar.gz
61-
- name: upload reference values artifact
61+
- name: Upload reference values artifact
6262
id: artifact-upload-step
63-
if: ${{ steps.matlab-refs-cache.outputs.cache-hit != 'true' }}
6463
uses: actions/upload-artifact@v4
6564
with:
6665
name: matlab_reference_test_values
6766
path: collectedValues.tar.gz
6867
# overwrite: true
68+
6969
- name: Output artifact URL
70-
if: ${{ steps.matlab-refs-cache.outputs.cache-hit != 'true' }}
7170
run: echo 'Artifact URL is ${{ steps.artifact-upload-step.outputs.artifact-url }}'
7271
test:
7372
needs: collect_references

‎docs/development/development_environment.rst‎

Lines changed: 142 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -1,108 +1,190 @@
11
Development Environment
22
=======================
33

4-
Currently, this package serves as an interface to the cpp binaries of k-Wave.
5-
For this reason, binaries are required to run simulations with `k-Wave-python`.
6-
The binaries are downloaded by k-Wave-python when the package is run for the first time.
4+
Overview
5+
--------
6+
k-Wave-python is a Python interface to the k-Wave C++ binaries, which are required to run simulations. The binaries are automatically downloaded when the package runs for the first time.
77

8-
To correctly set up your development environment for this repository, clone the repository from github, and install the project dependencies.
8+
Environment Setup with uv
9+
-------------------------
910

11+
1. Install uv
12+
~~~~~~~~~~~~~
13+
.. code-block:: bash
14+
15+
curl -LsSf https://astral.sh/uv/install.sh | sh
16+
17+
2. Clone the Repository
18+
~~~~~~~~~~~~~~~~~~~~~~~
1019
.. code-block:: bash
1120
1221
git clone https://github.com/waltsims/k-wave-python
1322
cd k-wave-python
14-
pip install -e '.[dev,test]'
1523
16-
This installs all the dependencies for development, and testing.
24+
3. Create and Activate Virtual Environment
25+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
26+
.. code-block:: bash
1727
18-
Ensure pre-commit is configured by running the following command:
28+
uv venv
29+
source .venv/bin/activate # On Windows: .venv\Scripts\activate
30+
31+
4. Install Dependencies
32+
~~~~~~~~~~~~~~~~~~~~~~~
33+
Install development and testing dependencies using uv:
1934

35+
.. code-block:: bash
36+
37+
uv pip install -e '.[dev,test]'
38+
39+
5. Configure Pre-commit Hooks
40+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2041
.. code-block:: bash
2142
2243
pre-commit install
2344
24-
Running Tests
25-
=======================
26-
Assuming matlab is installed locally, and `k-wave <https://github.com/ucl-bug/k-wave>` is installed in a parallel directory, testing can be performed using the make file located in the project root.
45+
Testing
46+
-------
2747

28-
.. code-block:: bash
48+
Prerequisites
49+
~~~~~~~~~~~~~
50+
- MATLAB installed locally (see `Alternative to MATLAB`_ if you don't have MATLAB)
51+
- `k-Wave <https://github.com/ucl-bug/k-wave>`_ installed in a parallel directory to k-wave-python
2952

30-
make test
53+
**Directory structure for parallel installation:**
54+
55+
.. code-block:: text
3156
32-
This process will first generate reference files in matlab and run the complete python test suite against them.
57+
<parent_dir>/
58+
├── k-wave/
59+
│ ├── k-Wave/
60+
│ ├── LICENSE.txt
61+
│ └── README.md
62+
└── k-wave-python/
63+
├── kwave/
64+
└── ...
3365
34-
To run the tests manually after reference generation, use the following command:
66+
67+
Running Tests
68+
~~~~~~~~~~~~~
69+
70+
Full Test Suite
71+
^^^^^^^^^^^^^^^
72+
Generate reference files with MATLAB and run the complete Python test suite:
3573

3674
.. code-block:: bash
75+
cd k-wave-python/
76+
make test
77+
78+
Manual Test Execution
79+
^^^^^^^^^^^^^^^^^^^^^
80+
After reference generation:
3781

38-
pytest
82+
.. code-block:: bash
3983
40-
To run the tests with coverage, use the following command:
84+
pytest
4185
86+
Test Coverage
87+
^^^^^^^^^^^^^
4288
.. code-block:: bash
4389
4490
coverage run
4591
46-
To run all examples, to ensure they still run after changes use the following command:
92+
Running Examples
93+
~~~~~~~~~~~~~~~~
4794

95+
Default (GPU-enabled)
96+
^^^^^^^^^^^^^^^^^^^^^
4897
.. code-block:: bash
4998
5099
make run-examples
100+
# or
101+
MPLBACKEND=Agg python run_examples.py
51102
52-
or
53-
103+
Force CPU Execution
104+
^^^^^^^^^^^^^^^^^^
54105
.. code-block:: bash
55106
56-
MPLBACKEND=Agg python run_examples.py
107+
MPLBACKEND=Agg KWAVE_FORCE_CPU=1 python run_examples.py
57108
109+
Test Architecture
110+
-----------------
58111

112+
The test suite compares Python and MATLAB outputs through two methodologies:
59113

60-
If you want to force the examples to run on the cpu:
114+
1. Unit Testing
115+
~~~~~~~~~~~~~~~
116+
- Tests k-Wave-python functions against their MATLAB counterparts
117+
- Reference outputs stored in ``.mat`` files
118+
- Generated by MATLAB scripts in ``tests/matlab_test_data_collectors/matlab_collectors/``
119+
- Master script: ``tests/matlab_test_data_collectors/run_all_collectors.m``
120+
- Output location: ``tests/matlab_test_data_collectors/python_testers/collectedValues/``
61121

62-
.. code-block:: bash
122+
.. _`Alternative to MATLAB`:
63123

64-
MPLBACKEND=Agg KWAVE_FORCE_CPU=1 python run_examples.py
124+
.. note::
125+
**Alternative to MATLAB:** If you don't have a local MATLAB installation, you can download pre-generated reference artifacts from `GitHub CI <https://nightly.link/waltsims/k-wave-python/workflows/pytest/master/matlab_reference_test_values.zip>`_.
65126

66127

67-
Test References
68-
=======================
128+
.. _`GitHub CI artifacts link`: https://nightly.link/waltsims/k-wave-python/workflows/pytest/master/matlab_reference_test_values.zip
129+
130+
2. Integration Testing
131+
~~~~~~~~~~~~~~~~~~~~~~
132+
- Validates ``.h5`` files produced by k-Wave-python against original k-Wave outputs
133+
- Uses hash values from MATLAB examples stored in JSON files
134+
- Hash files location: ``tests/reference_outputs/``
135+
- These files are committed to the repository and only require updates for new k-Wave releases
136+
137+
Generating MATLAB Reference Files
138+
---------------------------------
139+
140+
Process for Creating Reference Files
141+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
142+
143+
1. Open target MATLAB example (e.g., ``example_pr_2D_TR_directional_sensors.m`` from the `k-Wave repository <https://github.com/ucl-bug/k-wave/blob/main/k-Wave/examples/example_pr_2D_TR_directional_sensors.m>`_)
144+
145+
2. Locate ``kSpaceFirstOrder`` function call:
146+
147+
.. code-block:: matlab
148+
149+
input_args = {'PMLInside', false, 'PMLSize', PML_size, 'PlotPML', false, 'Smooth', false};
150+
sensor_data = kspaceFirstOrder2D(kgrid, medium, source, sensor, input_args{:});
151+
152+
3. Add save options to ``input_args``:
153+
154+
.. code-block:: matlab
155+
156+
input_args = {'PMLInside', false, 'PMLSize', PML_size, 'PlotPML', false, 'Smooth', false, 'SaveToDisk', true, 'SaveToDiskExit', true};
157+
158+
4. Run modified example to generate ``.h5`` files in your ``tmp`` folder:
159+
160+
- Single function call: creates ``example_input.h5``
161+
- Multiple calls: creates ``example_input_1.h5``, ``example_input_2.h5``, etc.
162+
163+
5. Convert ``.h5`` files to JSON hashes using ``H5Summary``:
69164

70-
Tests compare the outputs of the python and the matlab interfaces.
71-
These tests are located in the ``tests`` directory. The comparison between ``matlab`` and ``python`` outputs are done in two ways:
165+
- Single file: see `lines 92-95 <https://github.com/waltsims/k-wave-python/blob/1f9df5d987d0b3edb1a8a43fad0885d3d6079029/tests/h5_summary.py#L92-L95>`_
166+
- Multiple files: see `lines 97-106 <https://github.com/waltsims/k-wave-python/blob/1f9df5d987d0b3edb1a8a43fad0885d3d6079029/tests/h5_summary.py#L97-L106>`_
167+
168+
Publishing k-wave-python
169+
-----------------
170+
171+
`Hatch <https://hatch.pypa.io/latest/>`_ is used to publish k-wave-python to `PyPI <https://pypi.org/>`_.
72172

73-
- **Unit testing**: k-Wave-python functions that have a direct counterpart in original k-Wave are tested by comparing the outputs of the two functions.
74-
The output of the original k-Wave functions are stored in ``.mat`` files.
75-
These files can be generated by running the corresponding MATLAB scripts located in the ``tests/matlab_test_data_collectors/matlab_collectors/`` directory by running ``tests/matlab_test_data_collectors/run_all_collectors.m``.
76-
After running the scripts, the reference files can be found in ``tests/matlab_test_data_collectors/python_testes/collectedValues/``.
77-
78173
.. note::
79-
If you do not have MATLAB installed to generate the reference files, you can download recently generated reference file outputs from the GitHub CI and place them in the ``python_testers/collectedValues/`` directory.
80-
The latest reference files can be found in the artifacts of the latest CI run of ``pytest.yml`` (e.g. `here <https://github.com/waltsims/k-wave-python/actions/runs/7770639710/artifacts/1217868112>`_).
81-
82-
- **Integration testing**: k-Wave-python tests output .h5 files that are passed to the k-Wave binaries and ensures that they match the output of the original k-Wave.
83-
This testing compares the output for many of the example scripts from the original k-Wave package.
84-
Hash values of the reference output ``.h5`` file from MATLAB examples are generated and stored in ``.json`` files in ``tests/reference_outputs/``.
85-
These ``.json`` files are stored in the code repository and do not need to be regenerated.
86-
Since these files are generated from the original k-Wave package, they only need to be updated when a new release of k-Wave is made.
87-
88-
**Matlab reference file generation** can be described in the following steps.
89-
90-
#. Open desired example in matlab, e.g. `example_pr_2D_TR_directional_sensors.m <https://github.com/ucl-bug/k-wave/blob/main/k-Wave/examples/example_pr_2D_TR_directional_sensors.m>`_
91-
#. Find the lines where the call to one of the `kSpaceFirstOrder-family` function is made. For example,
92-
93-
.. code-block:: matlab
94-
95-
input_args = {'PMLInside', false, 'PMLSize', PML_size, 'PlotPML', false, 'Smooth', false};
96-
sensor_data = kspaceFirstOrder2D(kgrid, medium, source, sensor, input_args{:});
97-
98-
#. Update the ``input_args`` field by adding two new options - ``{'SaveToDisk', true, 'SaveToDiskExit': true}``. These options will ensure that we a ``.h5`` file will be created and saved in your ``tmp`` folder, while avoiding to run the actual simulation.
99-
#. Run the modified example. You will find created files in your ``tmp`` folder. Usually exact file name depends on how many calls are made to the `kSpaceFirstOrder-family` function in the example:
100-
* If there is only a single call, created file name will be ``example_input.h5``
101-
* If there are two or more calls, created files will have names like ``example_input_1.h5``, ``example_input_2.h5``, ``example_input_3.h5`` and so on
102-
#. Now it is time to turn the ``.h5`` files to the hashed ``.json`` files. This can be done with the ``H5Summary``.
103-
* If you have a single ``.h5`` file, adapt the lines below and run the script:
104-
https://github.com/waltsims/k-wave-python/blob/1f9df5d987d0b3edb1a8a43fad0885d3d6079029/tests/h5_summary.py#L92-L95
105-
* For multiple files, adapt the lines below:
106-
https://github.com/waltsims/k-wave-python/blob/1f9df5d987d0b3edb1a8a43fad0885d3d6079029/tests/h5_summary.py#L97-L106
174+
This is only performed by developers with write access to the k-wave-python package on PiPI.
175+
176+
The package can be built using:
177+
178+
.. code-block:: bash
179+
180+
hatch build
181+
182+
And pushed to the production index with:
183+
184+
.. code-block:: bash
185+
186+
hatch publish -u __token__
187+
188+
107189
108190

‎kwave/kWaveSimulation_helper/expand_grid_matrices.py‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -262,11 +262,11 @@ def print_grid_size(kgrid):
262262
"""
263263
k_Nx, k_Ny, k_Nz = kgrid.Nx, kgrid.Ny, kgrid.Nz
264264
if kgrid.dim == 1:
265-
logging.log(logging.INFO, " computational grid size:", int(k_Nx), "grid points")
265+
logging.log(logging.INFO, f" computational grid size: {k_Nx} grid points")
266266
elif kgrid.dim == 2:
267-
logging.log(logging.INFO, " computational grid size:", int(k_Nx), "by", int(k_Ny), "grid points")
267+
logging.log(logging.INFO, f" computational grid size: {k_Nx} by {k_Ny} grid points")
268268
elif kgrid.dim == 3:
269-
logging.log(logging.INFO, " computational grid size:", int(k_Nx), "by", int(k_Ny), "by", int(k_Nz), "grid points")
269+
logging.log(logging.INFO, f" computational grid size: {k_Nx} by {k_Ny} by {k_Nz} grid points")
270270

271271

272272
def expand_cuboid_corner_list(is_cuboid_list, kgrid, pml_size: Vector):

‎kwave/kWaveSimulation_helper/save_to_disk_func.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ def save_to_disk_func(
1818
kgrid: kWaveGrid, medium: kWaveMedium, source, opt: SimulationOptions, auto_chunk: bool, values: dotdict, flags: dotdict
1919
):
2020
# update command line status
21-
logging.log(logging.INFO, " precomputation completed in ", scale_time(TicToc.toc()))
21+
logging.log(logging.INFO, f" precomputation completed in {scale_time(TicToc.toc())}")
2222
TicToc.tic()
2323
logging.log(logging.INFO, " saving input files to disk...")
2424

@@ -66,7 +66,7 @@ def save_to_disk_func(
6666
save_file(opt.input_filename, integer_variables, float_variables, opt.hdf_compression_level, auto_chunk=auto_chunk)
6767

6868
# update command line status
69-
logging.log(logging.INFO, " completed in ", scale_time(TicToc.toc()))
69+
logging.log(logging.INFO, f" completed in {scale_time(TicToc.toc())}")
7070

7171

7272
def grab_integer_variables(integer_variables, kgrid, flags, medium):

‎pyproject.toml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ dependencies = [
3131
"numpy>=1.22.2,<2.3.0",
3232
"matplotlib==3.10.1",
3333
"beartype==0.20.2",
34-
"jaxtyping==0.2.38",
34+
"jaxtyping==0.3.1",
3535
"deprecated>=1.2.14"
3636
]
3737

@@ -43,7 +43,7 @@ Bug-tracker = "https://github.com/waltsims/k-wave-python/issues"
4343

4444
[project.optional-dependencies]
4545
test = ["pytest",
46-
"coverage==7.7.0",
46+
"coverage==7.8.0",
4747
"phantominator",
4848
"testfixtures==8.3.0",
4949
"requests==2.32.3"]
@@ -53,7 +53,7 @@ docs = [ "sphinx-mdinclude==0.6.2",
5353
"sphinx-tabs==3.4.7",
5454
"sphinx-toolbox==3.8.0",
5555
"furo==2024.8.6"]
56-
dev = ["pre-commit==4.1.0"]
56+
dev = ["pre-commit==4.2.0"]
5757

5858
[tool.hatch.version]
5959
path = "kwave/__init__.py"

0 commit comments

Comments
 (0)