hir-94: filter prefetcher hits from open-tracking pixel#22
Open
jaredzwick wants to merge 1 commit intopypesdev:mainfrom
Open
hir-94: filter prefetcher hits from open-tracking pixel#22jaredzwick wants to merge 1 commit intopypesdev:mainfrom
jaredzwick wants to merge 1 commit intopypesdev:mainfrom
Conversation
Email-client scanners fetch tracking pixels before the human ever
opens the message: Gmail's image proxy pre-caches inline images,
Apple Mail Privacy Protection (iOS 15+) does the same, and corporate
security gateways (Bitdefender / Mimecast / Proofpoint / etc.) scan
every inbound image. Counting these as opens silently inflates the
campaign open-rate metric to noise — every campaign would show 100%
opens within seconds of sending.
- src/lib/openTrackingFilter.ts: pure classifyPixelRequest() returns
{ isPrefetcher, reason }. Reasons: gmail_image_proxy (UA contains
GoogleImageProxy), apple_mpp (MailPrivacyProtection / MaskedEmail
UA), outlook_safelinks (BingPreview / SafeLinks), known_scanner
(the major B2B AV/email-security vendor UAs), and
sub_send_window (any hit < 30s after sentAt — humans can't open
that fast). Pure, no DB or env access; tests are deterministic.
- /api/email-tracking/pixel/[trackingId]: classify before recording.
Prefetcher hits still write a discrete email_event (so debugging
stays possible) but with metadata.prefetcher = true and never
increment campaign.openCount. Real opens compute "first open" by
excluding prefetcher events so the human's first hit still counts.
- libs/db/src/queries/emailEvent.ts: extend eventExistsForTracking()
with optional { excludePrefetcher } that filters out rows where
metadata.prefetcher === true. Existing two-arg callers unchanged.
14 vitest specs cover every UA branch (case-insensitive), the
time-window heuristic (custom threshold, negative-age skip, missing
sentAt skip), and the precedence rule (specific UA reason wins over
sub_send_window). tsc clean. test:int 109 passed (only pre-existing
PAYLOAD_SECRET failure remains).
Independent of all pending hir-94 PRs — branches from main.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Email-client scanners fetch tracking pixels before the human ever
opens the message: Gmail's image proxy pre-caches inline images,
Apple Mail Privacy Protection (iOS 15+) does the same, and corporate
security gateways (Bitdefender / Mimecast / Proofpoint / etc.) scan
every inbound image. Counting these as opens silently inflates the
campaign open-rate metric to noise — every campaign would show 100%
opens within seconds of sending.
{ isPrefetcher, reason }. Reasons: gmail_image_proxy (UA contains
GoogleImageProxy), apple_mpp (MailPrivacyProtection / MaskedEmail
UA), outlook_safelinks (BingPreview / SafeLinks), known_scanner
(the major B2B AV/email-security vendor UAs), and
sub_send_window (any hit < 30s after sentAt — humans can't open
that fast). Pure, no DB or env access; tests are deterministic.
Prefetcher hits still write a discrete email_event (so debugging
stays possible) but with metadata.prefetcher = true and never
increment campaign.openCount. Real opens compute "first open" by
excluding prefetcher events so the human's first hit still counts.
with optional { excludePrefetcher } that filters out rows where
metadata.prefetcher === true. Existing two-arg callers unchanged.
14 vitest specs cover every UA branch (case-insensitive), the
time-window heuristic (custom threshold, negative-age skip, missing
sentAt skip), and the precedence rule (specific UA reason wins over
sub_send_window). tsc clean. test:int 109 passed (only pre-existing
PAYLOAD_SECRET failure remains).
Independent of all pending hir-94 PRs — branches from main.
Co-Authored-By: Paperclip noreply@paperclip.ing
🤖 Generated with Claude Code