purview-v1.10.8 + PAX-v1.0.20

PAX_Purview_Audit_Log_Processor_v1.10.7.ps1 → PAX_Purview_Audit_Log_Processor_v1.10.8.ps1

@@ -1,5 +1,5 @@
 # Portable Audit eXporter (PAX) - Purview Audit Log Processor
-# Version: v1.10.7
+# Version: v1.10.8
 # Default Activity Type: CopilotInteraction (captures ALL M365 Copilot usage including all M365 apps and Teams meetings)
 # DSPM for AI: Microsoft Purview Data Security Posture Management integration
 #              MIXED FREE/PAYG Activity Types: AIInteraction (currently Microsoft platforms only), ConnectedAIAppInteraction (Microsoft + third-party)
@@ -42,6 +42,8 @@
 		- Partial success: Continues processing with successful partitions if some fail
 		- Query naming: PAX_Query_<DateRange>_PartX/Total visible in Purview UI
 		- Unified concurrency: MaxConcurrency parameter controls both EOM and Graph API modes (default: 10)
+		- Date-range accuracy: Client-side trimming ensures output contains only records within
+		  [StartDate, EndDate), compensating for Purview API date-range bleed
 		- Checkpoint & Resume: All auth modes automatically save progress to checkpoint files,
 		  enabling resumption after Ctrl+C, network failures, or any interruption via -Resume
 
@@ -750,13 +752,13 @@
 	Forms, Stream, Planner, PowerApps, and Office desktop apps.
 
 	ACTIVITY TYPES INCLUDED:
-	  Exchange: MailboxLogin, MailItemsAccessed, Send, SendOnBehalf, SoftDelete, HardDelete,
+	  Exchange: MailItemsAccessed, Send, SendOnBehalf, SoftDelete, HardDelete,
 	            MoveToDeletedItems, CopyToFolder
 	  SharePoint/OneDrive (Files): FileAccessed, FileDownloaded, FileUploaded, FileModified,
 	            FileDeleted, FileMoved, FileCheckedIn, FileCheckedOut, FileRecycled, FileRestored,
 	            FileVersionsAllDeleted
-	  SharePoint/OneDrive (Sharing): SharingSet, SharingInvitationCreated, SharingInvitationAccepted,
-	            SharedLinkCreated, SharingRevoked, AddedToSecureLink, RemovedFromSecureLink, SecureLinkUsed
+	  SharePoint/OneDrive (Sharing): SharingInvitationCreated, SharingInvitationAccepted,
+	            SharedLinkCreated, SharingRevoked, RemovedFromSecureLink
 	  Groups: AddMemberToUnifiedGroup, RemoveMemberFromUnifiedGroup
 	  Teams (Team/Channel): TeamCreated, TeamDeleted, TeamArchived, TeamSettingChanged,
 	            TeamMemberAdded, TeamMemberRemoved, MemberAdded, MemberRemoved, MemberRoleChanged,
@@ -1678,9 +1680,9 @@ $recordTypeWorkloadMap = @{
 
 $serviceOperationMap = @{
 	'AzureActiveDirectory' = @('UserLoggedIn','UserLoginFailed','AdminLoggedIn','ResetUserPassword','AddRegisteredUser','UpdateUser','ChangedUserSetting')
-	'Exchange' = @('MailboxLogin','MailItemsAccessed','Send','SendOnBehalf','SoftDelete','HardDelete','MoveToDeletedItems','CopyToFolder','NewInboxRule','UpdateInboxRules','AddMailboxPermission','RemoveMailboxPermission')
-	'SharePoint' = @('FileAccessed','FileDownloaded','FileUploaded','FileModified','FileDeleted','FileMoved','SharingSet','SharingInvitationCreated','SharingInvitationAccepted','SharedLinkCreated','SharingRevoked','AddMemberToUnifiedGroup','RemoveMemberFromUnifiedGroup')
-	'OneDrive' = @('FileAccessed','FileDownloaded','FileUploaded','FileModified','FileDeleted','FileMoved','SharingSet','SharingInvitationCreated','SharingInvitationAccepted','SharedLinkCreated','SharingRevoked','AddMemberToUnifiedGroup','RemoveMemberFromUnifiedGroup')
+	'Exchange' = @('MailItemsAccessed','Send','SendOnBehalf','SoftDelete','HardDelete','MoveToDeletedItems','CopyToFolder','AddMailboxPermission','RemoveMailboxPermission')
+	'SharePoint' = @('FileAccessed','FileDownloaded','FileUploaded','FileModified','FileDeleted','FileMoved','SharingInvitationCreated','SharingInvitationAccepted','SharedLinkCreated','SharingRevoked','AddMemberToUnifiedGroup','RemoveMemberFromUnifiedGroup')
+	'OneDrive' = @('FileAccessed','FileDownloaded','FileUploaded','FileModified','FileDeleted','FileMoved','SharingInvitationCreated','SharingInvitationAccepted','SharedLinkCreated','SharingRevoked','AddMemberToUnifiedGroup','RemoveMemberFromUnifiedGroup')
 	'Teams' = @('TeamMemberAdded','TeamMemberRemoved','ChannelAdded','ChannelDeleted','ChannelMessageSent','ChannelMessageDeleted','TeamDeleted','TeamArchived','AddMemberToUnifiedGroup','RemoveMemberFromUnifiedGroup')
 	'MicrosoftForms' = @('CreateForm','EditForm','DeleteForm','ViewForm','CreateResponse','SubmitResponse','ViewResponse','DeleteResponse')
 	'MicrosoftStream' = @('StreamModified','StreamViewed','StreamDeleted','StreamDownloaded')
@@ -1694,15 +1696,15 @@ $m365UsageRecordBundle = @('ExchangeAdmin','ExchangeItem','ExchangeMailbox','Sha
 # Curated M365 usage operations spanning Exchange/SharePoint/OneDrive/Teams/Forms/Stream/Planner/PowerApps and Office desktop apps (Word/Excel/PowerPoint/OneNote)
 $m365UsageActivityBundle = @(
 	# === Exchange/Email ===
-	'MailboxLogin','MailItemsAccessed','Send','SendOnBehalf','SoftDelete','HardDelete','MoveToDeletedItems','CopyToFolder',
+	'MailItemsAccessed','Send','SendOnBehalf','SoftDelete','HardDelete','MoveToDeletedItems','CopyToFolder',
 
 	# === SharePoint/OneDrive - Files ===
 	'FileAccessed','FileDownloaded','FileUploaded','FileModified','FileDeleted','FileMoved',
 	'FileCheckedIn','FileCheckedOut','FileRecycled','FileRestored','FileVersionsAllDeleted',
 
 	# === SharePoint/OneDrive - Sharing ===
-	'SharingSet','SharingInvitationCreated','SharingInvitationAccepted','SharedLinkCreated','SharingRevoked',
-	'AddedToSecureLink','RemovedFromSecureLink','SecureLinkUsed',
+	'SharingInvitationCreated','SharingInvitationAccepted','SharedLinkCreated','SharingRevoked',
+	'RemovedFromSecureLink',
 
 	# === Groups/Unified Groups ===
 	'AddMemberToUnifiedGroup','RemoveMemberFromUnifiedGroup',
@@ -1757,7 +1759,7 @@ $m365UsageActivityBundle = @(
 ) | Select-Object -Unique
 
 # Script version constant (must appear after param/help to keep param() valid as first executable block)
-$ScriptVersion = '1.10.7'
+$ScriptVersion = '1.10.8'
 
 # --- Initialize/Clear persistent script variables to prevent cross-run contamination ---
 # Note: Script-scoped variables persist across multiple script invocations in the same PowerShell session
@@ -2439,6 +2441,15 @@ else {
 	}
 }
 
+# Client-side date-range trim boundaries — Purview's partition-based indexing can
+# return records outside the requested date range (observed up to ~10 h past EndDate).
+# These UTC boundaries are used after dedup to trim any out-of-range records.
+# SpecifyKind(Utc) is critical: ParseExact returns Kind=Unspecified, and .ToUniversalTime()
+# on Unspecified assumes LOCAL time, shifting the boundary by the machine's UTC offset.
+$script:TrimStartDateUTC = if ($StartDate -ne '*') { [datetime]::SpecifyKind([datetime]::ParseExact($StartDate, 'yyyy-MM-dd', $null), [System.DateTimeKind]::Utc) } else { $null }
+$script:TrimEndDateUTC   = if ($EndDate   -ne '*') { [datetime]::SpecifyKind([datetime]::ParseExact($EndDate,   'yyyy-MM-dd', $null), [System.DateTimeKind]::Utc) } else { $null }
+$script:DateTrimCount    = 0
+
 if ($BlockHours -le 0) { Write-Host "ERROR: BlockHours must be positive." -ForegroundColor Red; exit 1 }
 
 try { if ($PSVersionTable.PSEdition -eq 'Core' -and ($global:InformationPreference -in @('SilentlyContinue', 'Ignore'))) { $global:InformationPreference = 'Continue' } } catch {}
@@ -4802,6 +4813,16 @@ function Merge-IncrementalSaves-Streaming {
 						}
 						if ($recordId) { [void]$seenIds.Add($recordId) }
 
+						# Client-side date-range trimming for streaming path
+						if ($script:TrimStartDateUTC -or $script:TrimEndDateUTC) {
+							$recDate = script:Parse-DateSafe $record.CreationDate
+							if ($recDate) {
+								$recDateUtc = $recDate.ToUniversalTime()
+								if ($script:TrimStartDateUTC -and $recDateUtc -lt $script:TrimStartDateUTC) { $script:DateTrimCount++; continue }
+								if ($script:TrimEndDateUTC   -and $recDateUtc -ge $script:TrimEndDateUTC)   { $script:DateTrimCount++; continue }
+							}
+						}
+
 						# Parse AuditData for Operation if needed
 						$auditData = $record.AuditData
 						$parsedAudit = if ($record.PSObject.Properties['_ParsedAuditData']) { 
@@ -15127,6 +15148,27 @@ Write-Output "[403-MAX] Partition $idx/$tot - Max transient 403 poll retries exc
 		}
 	}
 
+	# Client-side date-range trimming — remove records returned outside the requested date boundaries.
+	# Purview's partition-based indexing can bleed records up to ~10 hours past EndDate on large tenants.
+	if (($script:TrimStartDateUTC -or $script:TrimEndDateUTC) -and $allLogs.Count -gt 0) {
+		$preTrimCount = $allLogs.Count
+		$trimmedLogs = New-Object System.Collections.ArrayList($preTrimCount)
+		foreach ($log in $allLogs) {
+			$cd = script:Parse-DateSafe $log.CreationDate
+			if (-not $cd) { [void]$trimmedLogs.Add($log); continue }  # Keep records with unparseable dates
+			$cdUtc = $cd.ToUniversalTime()
+			if ($script:TrimStartDateUTC -and $cdUtc -lt $script:TrimStartDateUTC) { continue }
+			if ($script:TrimEndDateUTC   -and $cdUtc -ge $script:TrimEndDateUTC)   { continue }
+			[void]$trimmedLogs.Add($log)
+		}
+		$trimCount = $preTrimCount - $trimmedLogs.Count
+		if ($trimCount -gt 0) {
+			$script:DateTrimCount += $trimCount
+			$allLogs = $trimmedLogs
+			Write-LogHost "Date-range trim: Removed $trimCount record(s) outside requested date boundaries" -ForegroundColor Yellow
+		}
+	}
+
 	# Show accurate record count — in streaming mode allLogs may be empty because records went JSONL→CSV directly
 	if ($script:UseStreamingMergeForExport) {
 		# StreamingMergeRecordCount = memory flush fresh run; mergedFromIncremental = deferred resume merge
@@ -16973,6 +17015,10 @@ function Profile-AuditData { param([object]$AuditData) } # No-op stub for thread
 		if ($script:StreamingMergeDuplicatesSkipped -gt 0) {
 			Write-LogHost "  Deduped:   $($script:StreamingMergeDuplicatesSkipped) duplicate records removed" -ForegroundColor DarkGray
 		}
+		# Show date-range trim count in Pipeline Summary
+		if ($script:DateTrimCount -gt 0) {
+			Write-LogHost "  Trimmed:   $($script:DateTrimCount) record(s) outside requested date range" -ForegroundColor DarkGray
+		}
 		# Show data loss warning in Pipeline Summary if partitions were missing
 		if ($script:StreamingMergeDataLoss) {
 			Write-LogHost "  [DATA-LOSS] WARNING: Output is PARTIAL — missing partitions: $($script:StreamingMergeMissingPartitions -join ', ')" -ForegroundColor Yellow

README.md

@@ -66,7 +66,7 @@ This is an experimental script. On occasion, you may notice small deviations fro
 
 ---
 
-> **🔍 Purview Audit Log Processor:** Download the script → [`PAX_Purview_Audit_Log_Processor_v1.10.7.ps1`](https://github.com/microsoft/PAX/releases/download/purview-v1.10.7/PAX_Purview_Audit_Log_Processor_v1.10.7.ps1)
+> **🔍 Purview Audit Log Processor:** Download the script → [`PAX_Purview_Audit_Log_Processor_v1.10.8.ps1`](https://github.com/microsoft/PAX/releases/download/purview-v1.10.8/PAX_Purview_Audit_Log_Processor_v1.10.8.ps1)
 >
 > **📖 Resources:** [Latest Documentation](https://github.com/microsoft/PAX/blob/release/release_documentation/Purview_Audit_Log_Processor/PAX_Purview_Audit_Log_Processor_Documentation_v1.10.0.md) | [Latest Release Notes](https://github.com/microsoft/PAX/blob/release/release_notes/Purview_Audit_Log_Processor/PAX_Purview_Audit_Log_Processor_Release_Note_v1.10.0.md)
 >

release_documentation/.gitkeep

@@ -1 +1 @@
-# Last updated: 2026-03-05 (PAX v1.0.19, Graph v1.0.1, Purview v1.10.7, CopilotInteractions v1.2.0)
+# Last updated: 2026-03-09 (PAX v1.0.20, Graph v1.0.1, Purview v1.10.8, CopilotInteractions v1.2.0)

release_documentation/Purview_Audit_Log_Processor/PAX_Purview_Audit_Log_Processor_Documentation_v1.10.0.md

@@ -1,14 +1,14 @@
 # Portable Audit eXporter (PAX) - <br/>Purview Audit Log Processor
 
-> **📥 Quick Start:** Download the script → [`PAX_Purview_Audit_Log_Processor_v1.10.7.ps1`](https://github.com/microsoft/PAX/releases/download/purview-v1.10.7/PAX_Purview_Audit_Log_Processor_v1.10.7.ps1)
+> **📥 Quick Start:** Download the script → [`PAX_Purview_Audit_Log_Processor_v1.10.8.ps1`](https://github.com/microsoft/PAX/releases/download/purview-v1.10.8/PAX_Purview_Audit_Log_Processor_v1.10.8.ps1)
 >
 > **📋 Release Notes:** See what's new → [v1.10.x Release Notes](https://github.com/microsoft/PAX/blob/release/release_notes/Purview_Audit_Log_Processor/PAX_Purview_Audit_Log_Processor_Release_Note_v1.10.0.md) | [All Release Notes](https://github.com/microsoft/PAX/tree/release/release_notes/Purview_Audit_Log_Processor)
 >
 > **📜 Previous Script Versions:** [All Purview Releases](https://github.com/microsoft/PAX/releases?q=purview-&expanded=true)
 >
 > **📚 Documentation Archive:** [v1.10.x Documentation](https://github.com/microsoft/PAX/blob/release/release_documentation/Purview_Audit_Log_Processor/PAX_Purview_Audit_Log_Processor_Documentation_v1.10.0.md) | [All Documentation](https://github.com/microsoft/PAX/tree/release/release_documentation/Purview_Audit_Log_Processor)
 
-**Script:** [`PAX_Purview_Audit_Log_Processor_v1.10.7.ps1`](https://github.com/microsoft/PAX/releases/download/purview-v1.10.7/PAX_Purview_Audit_Log_Processor_v1.10.7.ps1)  
+**Script:** [`PAX_Purview_Audit_Log_Processor_v1.10.8.ps1`](https://github.com/microsoft/PAX/releases/download/purview-v1.10.8/PAX_Purview_Audit_Log_Processor_v1.10.8.ps1)  
 **Documentation Version:** 1.10.x  
 **Audience:** IT admins, security/compliance analysts, BI/data teams  
 **Runtime:** PowerShell 5.1 (compatible) / PowerShell 7+ (recommended)  
@@ -409,6 +409,7 @@ powershell -ExecutionPolicy Bypass -File .\PAX_Purview_Audit_Log_Processor.ps1 -
 - **Live mode:** Must specify both or neither (partial specification rejected)
 - **Replay mode:** Both dates optional; act as filters on `CreationDate` column
 - **Time zone:** All dates interpreted as UTC; convert local times before invocation
+- **Date-range accuracy:** The Purview API may return records slightly beyond the requested `EndDate`. PAX automatically trims these so the exported output contains only records within your specified `[StartDate, EndDate)` range. The number of trimmed records is reported in the log file's Pipeline Summary.
 
 ---
 
@@ -924,9 +925,9 @@ Automating scripts, using headless terminals, or SSO scenarios
 
 | Category | Operations |
 |----------|------------|
-| Outlook (Exchange) | MailboxLogin, MailItemsAccessed, Send, SendOnBehalf, SoftDelete, HardDelete, MoveToDeletedItems, CopyToFolder |
+| Outlook (Exchange) | MailItemsAccessed, Send, SendOnBehalf, SoftDelete, HardDelete, MoveToDeletedItems, CopyToFolder |
 | SharePoint/OneDrive (Files) | FileAccessed, FileDownloaded, FileUploaded, FileModified, FileDeleted, FileMoved, FileCheckedIn, FileCheckedOut, FileRecycled, FileRestored, FileVersionsAllDeleted |
-| SharePoint/OneDrive (Sharing) | SharingSet, SharingInvitationCreated, SharingInvitationAccepted, SharedLinkCreated, SharingRevoked, AddedToSecureLink, RemovedFromSecureLink, SecureLinkUsed |
+| SharePoint/OneDrive (Sharing) | SharingInvitationCreated, SharingInvitationAccepted, SharedLinkCreated, SharingRevoked, RemovedFromSecureLink |
 | Groups | AddMemberToUnifiedGroup, RemoveMemberFromUnifiedGroup |
 | Teams (Team/Channel) | TeamCreated, TeamDeleted, TeamArchived, TeamSettingChanged, TeamMemberAdded, TeamMemberRemoved, MemberAdded, MemberRemoved, MemberRoleChanged, ChannelAdded, ChannelDeleted, ChannelSettingChanged, ChannelOwnerResponded, ChannelMessageSent, ChannelMessageDeleted, BotAddedToTeam, BotRemovedFromTeam, TabAdded, TabRemoved, TabUpdated, ConnectorAdded, ConnectorRemoved, ConnectorUpdated |
 | Teams (Chat/Messaging) | TeamsSessionStarted, ChatCreated, ChatRetrieved, ChatUpdated, MessageSent, MessageRead, MessageDeleted, MessageUpdated, MessagesListed, MessageCreation, MessageCreatedHasLink, MessageEditedHasLink, MessageHostedContentRead, MessageHostedContentsListed, SensitiveContentShared |
@@ -4471,7 +4472,6 @@ The bundle includes activities from 10 categories:
 
 | Operation | Description |
 |-----------|-------------|
-| MailboxLogin | User accessed mailbox |
 | MailItemsAccessed | Email items accessed (read/preview) |
 | Send | Email sent |
 | SendOnBehalf | Email sent on behalf of another user |
@@ -4500,14 +4500,11 @@ The bundle includes activities from 10 categories:
 
 | Operation | Description |
 |-----------|-------------|
-| SharingSet | Sharing permissions set |
 | SharingInvitationCreated | Sharing invitation created |
 | SharingInvitationAccepted | Sharing invitation accepted |
 | SharedLinkCreated | Shared link created |
 | SharingRevoked | Sharing permissions revoked |
-| AddedToSecureLink | User added to secure link |
 | RemovedFromSecureLink | User removed from secure link |
-| SecureLinkUsed | Secure link accessed |
 
 #### Groups
 

release_notes/.gitkeep

@@ -1 +1 @@
-# Last updated: 2026-03-05 (PAX v1.0.19, Graph v1.0.1, Purview v1.10.7, CopilotInteractions v1.2.0)
+# Last updated: 2026-03-09 (PAX v1.0.20, Graph v1.0.1, Purview v1.10.8, CopilotInteractions v1.2.0)

release_notes/Purview_Audit_Log_Processor/PAX_Purview_Audit_Log_Processor_Release_Note_v1.10.0.md

@@ -3,7 +3,7 @@
 ## Release Information
 
 - **Version:** 1.10.x
-- **Release Date:** 2026-03-05
+- **Release Date:** 2026-03-09
 - **Released By:** Microsoft Copilot Growth ROI Advisory Team (copilot-roi-advisory-team-gh@microsoft.com)
 
 ---
@@ -12,7 +12,7 @@
 
 Download the script below.  For questions or issues, refer to the documentation.
 
-- **PAX Purview Audit Log Processor Script v1.10.7:** [PAX_Purview_Audit_Log_Processor_v1.10.7.ps1](https://github.com/microsoft/PAX/releases/download/purview-v1.10.7/PAX_Purview_Audit_Log_Processor_v1.10.7.ps1)
+- **PAX Purview Audit Log Processor Script v1.10.8:** [PAX_Purview_Audit_Log_Processor_v1.10.8.ps1](https://github.com/microsoft/PAX/releases/download/purview-v1.10.8/PAX_Purview_Audit_Log_Processor_v1.10.8.ps1)
 - **Documentation v1.10.x (Markdown):** [PAX_Purview_Audit_Log_Processor_Documentation_v1.10.x.md](https://github.com/microsoft/PAX/blob/release/release_documentation/Purview_Audit_Log_Processor/PAX_Purview_Audit_Log_Processor_Documentation_v1.10.0.md)
 
 ---
@@ -511,6 +511,10 @@ If minimum window reached:
 
 - **(v1.10.7) Checkpoint log file rename in split mode:** Fixed `_PARTIAL` suffix remaining on the log file after a fully successful export when CSV split mode (`-SplitByActivityType` or `-SplitByRecordType`) is active. The fallback rename is guarded to preserve `_PARTIAL` on genuinely interrupted runs for Resume mode detection.
 
+- **(v1.10.8) Purview date-range bleed — client-side trimming:** Added client-side date-range trimming to eliminate records that bleed past the requested `EndDate` boundary by up to ~10 hours (a known Purview API behavior affecting ~3–5% of returned records). Output CSV is now guaranteed to contain only records within the user-specified `[StartDate, EndDate)` range. Timezone-safe boundary parsing uses `SpecifyKind(..., Utc)` to prevent local timezone offsets from shifting UTC midnight boundaries. Trimmed record count is reported in the Pipeline Summary for operator visibility.
+
+- **(v1.10.8) M365 Usage Bundle — noisy operations removed:** Removed six high-volume, low-signal operations from the `-IncludeM365Usage` bundle: `MailboxLogin`, `SharingSet`, `AddedToSecureLink`, `SecureLinkUsed`, `NewInboxRule`, and `UpdateInboxRules`. Bundle size reduced from ~121 to ~117 curated operation types. All removed operations remain available for explicit queries via `-ActivityTypes`.
+
 ---
 
 ## Known Considerations

script_archive/.gitkeep

@@ -1 +1 @@
-# Last updated: 2026-03-05 (PAX v1.0.19, Graph v1.0.1, Purview v1.10.7, CopilotInteractions v1.2.0)
+# Last updated: 2026-03-09 (PAX v1.0.20, Graph v1.0.1, Purview v1.10.8, CopilotInteractions v1.2.0)