We did the spike work before writing recovery code.
Recovery is only as trustworthy as the evidence it's built on. Before we shipped a single normalizer, we ran a 22-hour audit-log capture against a live Microsoft Entra test tenant, fired four classes of agent-driven change, and catalogued exactly what the audit log tells us — and what it doesn't. The findings below drive every confidence tag in our pipeline.
WI-05 audit-log completeness spike
Real test tenant. Real Microsoft Graph audit events. No simulation, no synthesised activity logs.
What the audit log gives us — and what it doesn't
Each row is the actual count from the WI-05 capture. The before-state strategy column is what KavachIQ does in normalization today.
| Change class | Events | Before-state | After-state | Strategy | Normalizer |
|---|---|---|---|---|---|
| Group membership | 12 / 12 | 0 / 12 audit oldValue | 12 / 12 audit newValue | Snapshot-reconstructed | Done |
| Conditional Access policy | 1 / 1 | 1 / 1 (full policy JSON) | 1 / 1 (full policy JSON) | Audit-authoritative | Done |
| App role assignment | 1 / 1 | 0 / 1 audit oldValue | 1 / 1 audit newValue | Snapshot-reconstructed | Done |
| Service principal credential | 2 / 2 | 2 / 2 KeyDescription metadata | 2 / 2 KeyDescription metadata | Audit-authoritative metadata · secretText unavailable | Pending |
Authoritative source: docs/SPIKE_REPORT_AUDIT_LOG_COMPLETENESS.md §5 + §7 in the repository.
Findings that shape the recovery model
Audit log is sufficient for two of four classes
Conditional Access policy edits and SP credential changes carry both pre and post state in the audit event itself. We don't need a baseline snapshot to reconstruct what changed.
The other two need a baseline snapshot
Group membership and app role assignment events carry only the post-change state — Microsoft does not include oldValue in those audit events. Our normalization pipeline reconstructs before-state from a baseline snapshot tagged confidence: "reconstructed".
We never fabricate data
Where audit cannot provide an authoritative answer (e.g., the secretText of a credential), we mark it confidence: "unavailable". Recovery decisions are made against what we know, with explicit honesty about what we don't.
activityDisplayName is the discriminator
Microsoft's category tag is unreliable — app role assignments live under UserManagement, not ApplicationManagement. Our normalizer matches on activityDisplayName, which is consistent across change classes.
Honest roadmap to pilot-ready MVP
Phase 0 is the spike work above. Phase 1 is the normalization + correlation + detection pipeline that produces today's incident output. Phases 2-5 are on the build path.
Architecture spikes
Audit-log completeness, schema specification, and connector design validated against a live Microsoft Entra test tenant.
Ingestion backbone
3 of 4 change classes normalized end-to-end (group-membership, Conditional Access, app-role). Correlation, detection, and snapshot-based baseline reconstruction shipped. SP-credential normalization next.
Operator console + blast radius
Cross-system blast-radius computation across SharePoint, Exchange, Teams, Conditional Access, and downstream applications. Operator UI for incident review.
Trusted-state baseline + recovery planning
Baseline approval workflow. Recovery-plan generation: identity-first sequencing, dependency chains, approval gates.
Limited execution + validation
Approved actions executed against Microsoft Graph with idempotency, retry, and post-action validation.
Pilot hardening
Multi-tenant isolation, observability, audit trail, on-call runbooks. Pilot-ready MVP.
Want to dig deeper?
The full WI-05 spike report — every event ID, every encoding anomaly, every anomalous race condition observed during capture — is in the repository. For technical buyer due-diligence we'll walk you through it directly.