Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 28 additions & 16 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,29 +14,41 @@ jobs:
runs-on: ${{ matrix.os }}
steps:
- name: checkout
uses: actions/checkout@v6
- name: Setup Poetry
uses: abatilo/actions-poetry@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up uv
uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
with:
python-version: ${{ matrix.python-version }}
cache: poetry
cache-dependency-path: poetry.lock
- name: Setup FFMPEG
uses: FedericoCarboni/setup-ffmpeg@v3
if: ${{ ! startsWith(matrix.os, 'macos') }}
enable-cache: true
cache-python: true
cache-dependency-glob: |
pyproject.toml
uv.lock
cache-suffix: ${{ runner.os }}-${{ matrix.python-version }}
- name: Setup FFMPEG on Ubuntu
if: startsWith(matrix.os, 'ubuntu')
run: |
sudo apt-get update
sudo apt-get install -y ffmpeg
- name: Setup FFMPEG on Windows
if: startsWith(matrix.os, 'windows')
run: choco install ffmpeg --yes --no-progress
- name: Setup Dependencies with Homebrew
if: startsWith(matrix.os, 'macos')
run: |
brew install llvm ffmpeg
echo "CC=$(brew --prefix)/opt/llvm/bin/clang" >> $GITHUB_ENV
echo "CXX=$(brew --prefix)/opt/llvm/bin/clang++" >> $GITHUB_ENV
- name: Verify FFMPEG
run: |
ffmpeg -version
ffprobe -version
- name: Setup Project
run: make init-project
- name: Normalize OpenCV package
run: uv sync --all-extras --frozen
- name: Run checks
run: |
poetry run python -m pip uninstall -y opencv-python-headless
poetry run python -m pip install --no-deps --force-reinstall opencv-contrib-python-headless
- name: Run precommit
run: make precommit
uv lock --check
uv run ruff check perception tests
uv run mypy perception
uv run black --check .
uv run pytest tests/
4 changes: 2 additions & 2 deletions .github/workflows/gh-pages.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,11 @@ jobs:
pages: write
id-token: write
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
- id: deployment
uses: sphinx-notes/pages@v3
uses: sphinx-notes/pages@e9c8f79f285fb41a150ba6e08a3c2a29635e7ee1 # 3.5
with:
checkout: false
documentation_path: docs
Expand Down
80 changes: 44 additions & 36 deletions .github/workflows/release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,30 +13,35 @@ jobs:
os: ["ubuntu-latest", "windows-latest", "macos-latest"]
name: Build for ${{ matrix.os }} on Python ${{ matrix.python-version }}
steps:
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
# Full clone for version calculation
fetch-depth: 0
fetch-tags: true
ref: ${{ github.event_name == 'release' && format('refs/tags/{0}', github.event.release.tag_name) || github.ref }}
- name: Set up uv
uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
with:
python-version: ${{ matrix.python-version }}
- name: Setup Poetry
uses: abatilo/actions-poetry@v4
- name: Setup FFMPEG
uses: FedericoCarboni/setup-ffmpeg@v3
if: ${{ ! startsWith(matrix.os, 'macos') }}
- name: Setup Dependencies with Homebrew
enable-cache: false
- name: Setup Build Dependencies with Homebrew
if: startsWith(matrix.os, 'macos')
run: |
brew install llvm ffmpeg
brew install llvm
echo "CC=$(brew --prefix)/opt/llvm/bin/clang" >> $GITHUB_ENV
echo "CXX=$(brew --prefix)/opt/llvm/bin/clang++" >> $GITHUB_ENV
- uses: actions/checkout@v6
with:
# Full clone for version calculation
fetch-depth: 0
fetch-tags: true
ref: ${{ github.event_name == 'release' && format('refs/tags/{0}', github.event.release.tag_name) || github.ref }}
- name: Build Project
run: make build-wheel
- uses: actions/upload-artifact@v7
shell: bash
run: |
rm -rf build Perception.egg-info
echo "uv: $(uv --version)"
echo "Python: $(uv run python --version)"
echo "Git describe: $(git describe --tags --always)"
uv build --wheel --out-dir="dist-tmp" --clear
uv tool run --from repairwheel repairwheel -o dist dist-tmp/*.whl
find dist -name "*.whl" -type f | sed -n "s/\(.*\)\.linux.*\.whl$/& \1.whl/p" | xargs -r -n 2 mv
rm -rf dist-tmp
- uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: package-wheels-${{ matrix.os }}-${{ matrix.python-version }}
path: dist/*
Expand All @@ -45,21 +50,26 @@ jobs:
runs-on: ubuntu-latest
name: Build sdist
steps:
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"
- name: Setup Poetry
uses: abatilo/actions-poetry@v4
- uses: actions/checkout@v6
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
# Full clone for version calculation
fetch-depth: 0
fetch-tags: true
ref: ${{ github.event_name == 'release' && format('refs/tags/{0}', github.event.release.tag_name) || github.ref }}
- name: Set up uv
uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
with:
python-version: "3.13"
enable-cache: false
- name: Build Project
run: make build-sdist
- uses: actions/upload-artifact@v7
shell: bash
run: |
rm -rf build Perception.egg-info
echo "uv: $(uv --version)"
echo "Python: $(uv run python --version)"
echo "Git describe: $(git describe --tags --always)"
uv build --sdist --out-dir="dist"
- uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: package-sdist
path: dist/*
Expand All @@ -69,33 +79,31 @@ jobs:
runs-on: ubuntu-latest
if: ${{ github.repository_owner == 'thorn-oss' && github.event_name == 'release' }}
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
# Full clone for version calculation
fetch-depth: 0
fetch-tags: true
ref: refs/tags/${{ github.event.release.tag_name }}
- uses: actions/setup-python@v6
- name: Set up uv
uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
with:
python-version: "3.13"
- name: Setup Poetry
uses: abatilo/actions-poetry@v4
- name: Setup Dynamic Versioning
run: poetry self add "poetry-dynamic-versioning[plugin]"
enable-cache: false
- name: Download wheels
uses: actions/download-artifact@v8
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
path: dist
pattern: package-*
merge-multiple: true
- name: Load PyPI Token
uses: 1password/load-secrets-action@v4
uses: 1password/load-secrets-action@7ce42673ba9ed69053d678faeba29ea36bd25755 # v4.0.0
with:
# Export loaded secrets as environment variables
export-env: true
env:
OP_SERVICE_ACCOUNT_TOKEN: ${{ secrets.DATA_SCIENCE_OP_SERVICE_ACCOUNT_TOKEN }}
POETRY_PYPI_TOKEN_PYPI: op://data-science-oss/perception-pypi-api-key/secret/value
UV_PUBLISH_TOKEN: op://data-science-oss/perception-pypi-api-key/secret/value
- name: Verify artifacts
run: |
mapfile -t artifacts < <(find dist -type f \( -name "*.whl" -o -name "*.tar.gz" \))
Expand All @@ -109,4 +117,4 @@ jobs:
exit 1
fi
- name: Publish package
run: poetry publish -n
run: uv publish --trusted-publishing never --no-attestations
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ notebooks
# Local environment
.venv
.python-version
.pre-commit-cache/

# Coverage file
.coverage
Expand Down
54 changes: 29 additions & 25 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,54 +1,58 @@
TEST_SCOPE?=tests/

.PHONY: build build-wheel build-sdist verify-version init-project init test lint_check type_check format format_check precommit
.PHONY: build build-wheel build-sdist verify-version init-project init test lint_check type_check format format_check precommit clean-build

init-project:
poetry install --all-extras
uv sync --all-extras

init: init-project
poetry run pre-commit install
uv run pre-commit install

test:
poetry run pytest $(TEST_SCOPE)
uv run pytest $(TEST_SCOPE)

lint_check:
poetry run ruff check perception tests
uv run ruff check perception tests

type_check:
poetry run mypy perception
uv run mypy perception

format:
poetry run black .
uv run black .

format_check:
poetry run black --check . || (echo '\nUnexpected format.' && exit 1)
uv run black --check . || (echo '\nUnexpected format.' && exit 1)

precommit:
poetry check
make lint_check
make type_check
make format_check
make test
uv lock --check
uv run ruff check perception tests
uv run mypy perception
uv run black --check . || (echo '\nUnexpected format.' && exit 1)
uv run pytest $(TEST_SCOPE)

verify-version:
@echo "Poetry: $$(poetry --version)"
@echo "Poetry plugins:"
poetry self show plugins
@echo "uv: $$(uv --version)"
@echo "Python: $$(uv run python --version)"
@echo "Git describe: $$(git describe --tags --always)"
@poetry self show plugins | grep -q "poetry-dynamic-versioning"

clean-build:
@rm -rf build Perception.egg-info

build-wheel:
poetry run pip -q install repairwheel
poetry self add "poetry-dynamic-versioning[plugin]"
$(MAKE) verify-version
poetry build --format="wheel" --output="dist-tmp"
poetry run repairwheel -o dist dist-tmp/*.whl
@rm -rf build Perception.egg-info
@echo "uv: $$(uv --version)"
@echo "Python: $$(uv run python --version)"
@echo "Git describe: $$(git describe --tags --always)"
uv build --wheel --out-dir="dist-tmp" --clear
uv tool run --from repairwheel repairwheel -o dist dist-tmp/*.whl
@find dist -name "*.whl" -type f | sed -n "s/\(.*\)\.linux.*\.whl$$/& \1.whl/p" | xargs -r -n 2 mv # Fix wheel name
@rm -rf dist-tmp

build-sdist:
poetry self add "poetry-dynamic-versioning[plugin]"
$(MAKE) verify-version
poetry build --format="sdist" --output="dist"
@rm -rf build Perception.egg-info
@echo "uv: $$(uv --version)"
@echo "Python: $$(uv run python --version)"
@echo "Git describe: $$(git describe --tags --always)"
uv build --sdist --out-dir="dist"

build: build-wheel build-sdist
37 changes: 33 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,35 @@

`pip install perception`

#### Optional extras

`perception` provides optional extras for additional functionality:

- `benchmarking` – tools for benchmarking perceptual hashes
- `matching` – async matching utilities
- `pdq` – Facebook's PDQ hash support

**Note for `benchmarking` extra users:** The `benchmarking` extra depends on
`albumentations`, which in turn requires `opencv-python-headless`. However,
`perception` already depends on `opencv-contrib-python-headless` (needed for
contrib modules such as `cv2.img_hash` and `cv2.SIFT_create`). Installing both
OpenCV distributions simultaneously causes file-level conflicts.

If you are using [uv](https://docs.astral.sh/uv/), this is handled
automatically:

```bash
uv pip install "perception[benchmarking]"
```

If you are using plain `pip`, install the extra and then force-reinstall the
contrib variant to remove the conflicting headless package:

```bash
pip install "perception[benchmarking]"
pip install --force-reinstall --no-deps opencv-contrib-python-headless
```

### Hashing

Hashing with different functions is simple with `perception`.
Expand Down Expand Up @@ -51,11 +80,12 @@ See below for end-to-end examples for common use cases for perceptual hashes.
To work on the project, start by doing the following.

```bash
# Install local dependencies for
# code completion, etc.
# Install local dependencies for code completion,
# testing, and linting.
make init
```

- To do a (close to) comprehensive check before committing code, you can use `make precommit`.
To do a (close to) comprehensive check before committing code, use `make precommit`.

To implement new features, please first file an issue proposing your change for discussion.

Expand All @@ -70,4 +100,3 @@ examples.
- [imagededup](https://idealo.github.io/imagededup/)
- [ImageHash](https://github.com/JohannesBuchner/imagehash)
- [PhotoHash](https://github.com/bunchesofdonald/photohash)
```
15 changes: 0 additions & 15 deletions build.py

This file was deleted.

Loading
Loading