fix: make digital dtype11 decode spec-compliant and deterministic#40
Open
jgoedeke wants to merge 14 commits intoRecordEvolution:masterfrom
Open
fix: make digital dtype11 decode spec-compliant and deterministic#40jgoedeke wants to merge 14 commits intoRecordEvolution:masterfrom
jgoedeke wants to merge 14 commits intoRecordEvolution:masterfrom
Conversation
- Add pytest configuration for test management - Implement test suite for CLI functionality and Python module - Update README with testing instructions and badge - Fix Dockerfile - Create .dockerignore to exclude unnecessary files from Docker builds - Add GitHub Actions workflows for testing - Clean up makefile to include test commands
…example and tests
…es for chunked NumPy export
- Modified `component_group` and `channel` constructors to accept raw buffer pointers instead of vectors. - Enhanced `load_all_data` and `init_metadata` methods for better data initialization and loading. - Implemented `read_chunk` method in `channel` to facilitate chunked data reading with support for raw and scaled modes. - Updated `convert_data_to_type` and `convert_chunk_to_double` functions to handle raw data more efficiently. - Removed redundant `imc_result.hpp` file to streamline the codebase. - Adjusted Python bindings in `imctermite.pyx` to manage C++ instance memory correctly.
- Update GitHub Actions workflow to support testing on multiple OS - Refactor memory mapping in imc_buffer.hpp for Windows compatibility - Improve makefile to handle .pyd files for Python builds - Add comprehensive tests for streaming and chunking functionality in test_streaming.py
…ficiency (RecordEvolution#9) Reduces memory usage by 90% for large datasets while maintaining comparable processing speed.
…ples and core functionality
- Migrate from setup.cfg to pyproject.toml with PEP 517/621 compliance - Update to Python build tools (replace setup.py commands with python -m build) - Upgrade all GitHub Actions to latest versions (@v4, ubuntu-latest) - Remove outdated cibuildwheel version pinning - Add numpy as explicit build and runtime dependency - Bump package version to 3.0.0 - Improve test documentation with development install guidance - Add Python version badge to README - Standardize python3 usage across makefiles
* restructure package significantly: - move native cython extension to private `_imctermite` module - introduce pure-python `__init__.py` wrapper for public api * rename main class `imctermite` -> `ImcTermite` to follow PEP 8 class naming standards * add deprecated alias for `imctermite` (with warning) for backwards compatibility * add PEP 484 type hints and PEP 561 `py.typed` marker for proper IDE/mypy support * support PEP 519 file system path protocol (allowing `pathlib.Path` objects) * update usage examples and tests to reflect new API * update build configuration (setup.py, pyproject.toml, MANIFEST.in) to support new structure
- Prevent uninitialized CR/CD metadata from leaking into decode paths by adding safe defaults and explicit presence handling - Apply CR scaling only where valid; keep datatype 11 (2-byte digital) on identity transform in scaled mode - Enforce clear fail-fast behavior when required CD timing metadata is missing for single-component channels - Add regression tests: - datatype 11 scaled output matches raw values - non-digital channels still satisfy scaled = raw * factor + offset
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I found an issue with datatype 11 parsing in scaled mode, where unset CR/CD fields lead to jibberish calculations and random float values. This PR is based on #39