Skip to content

many recent memory-savings and performance tricks, consider a new release#78

Merged
h-walk merged 80 commits intomainfrom
beta
Mar 23, 2026
Merged

many recent memory-savings and performance tricks, consider a new release#78
h-walk merged 80 commits intomainfrom
beta

Conversation

@tpchuckles
Copy link
Copy Markdown
Collaborator

tested in staging, moved to beta, tested in beta

memory savings tricks include:

  • real-space cropping of probe and potential before propagation (arg "min_dk")
  • ADF on-the-fly (summation inside the calculator, eliminating the need to store the full kx,ky exit wave) (arg "ADF")
  • memmapping (arg "use_memmap")
  • chunking instead of processing all probes in parallel (arg "loop_probes")
  • prism: fourier components of probes are computed (some optimizations still left to do though) (arg "prism")
  • skip processing of probes where there are no atoms (arg "ignore_vacuum")

some of these have already made it to main, but were broken in certain scenarios. there is now more rigorous testing via tests/04_haadf_otf.py, which runs through all combinations of these arguments to make sure they run. note that in some cases, the result will differ, as some things are an approximation (prism, min_dk).

this will close the following issues: #75 #68 #67 #35 #33, and make #30 almost irrelevant.

…(memmapped maybe) to PrismProbe.calculateProbesFromS, and use einsum. 256x256 probes is now about 5 minutes
…create a dummy HAADFData object, get the mask from it, do the summation, and load the result into the HAADFData object as we go
…_prism medium FOV 256x256 now runs without memmapping in 1.5GB ram, wowwzers
pulling straight-to-main changes back to beta
@tpchuckles
Copy link
Copy Markdown
Collaborator Author

performance update (in how probes are pixel-shifted when min_dk is used, 5x speed-up when this is used for large systems), and electron counts functionality added (addressing issue #80). tests pass, and changes moved to beta this morning. so these changes can be included in this PR

Critical fixes:
- Remove debug plt.savefig("probecropping.png") in applyShifts
- Fix PrismProbe empty slice when i1==0 (a[0:-0] is empty)
- Fix whichtimestep typo in wf_data.py plot_reciprocal
- Fix haadf_data.py in-place mutation of _wf_array
- Remove debug print("summation took...") in wf_data.py

Ported from main:
- Correct Kirkland Eq C.1 prefactor (47.878 V·Å²) for potential normalization
- Device-aware tensor creation (explicit device= on all zeros/ones calls)
- Remove global torch.set_default_device()
- Infer device from probe in Propagate when not specified

Regenerated test baselines for new normalization.
@h-walk h-walk merged commit 72b11cd into main Mar 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants