Shadow AI Audit answers one question: which AI applications can currently reach your company's data through Microsoft Entra ID, who let them in, and what can they read?
It is free, MIT-licensed, and runs against your own tenant from your own machine. Nothing is uploaded anywhere.
Why it exists
Search for shadow AI discovery tooling and page one is almost entirely vendor blogs ranking their own products. None of them tell you how far you can get with the licences you already own. This does, in about five minutes.
Read-only, on purpose
The script requests only *.Read.All scopes. It never writes, revokes, or modifies anything.
That is a deliberate design constraint rather than a missing feature. An audit you can run against production on a Tuesday afternoon without raising a change ticket is an audit that actually gets run. Everything it finds is remediated by a human, in the portal, with full context.
Requirements
Install-Module Microsoft.Graph -Scope CurrentUserFour read-only consents:
| Scope | Used for |
|---|---|
Application.Read.All | Enterprise apps and service principals |
Directory.Read.All | OAuth2 permission grants |
Policy.Read.All | Tenant user-consent policy |
User.ReadBasic.All | Resolving consenting users to UPNs |
Running it
# Audit against known signatures
.\Invoke-ShadowAIAudit.ps1
Surface AI apps the signature file does not know yet
.\Invoke-ShadowAIAudit.ps1 -Discover
Specific tenant, custom output location
.\Invoke-ShadowAIAudit.ps1 -TenantId contoso.onmicrosoft.com -OutputPath C:\audit
-Discover before concluding you are clean. Signature mode only finds apps it has been taught about.What it reports
- Which AI apps hold consent, and how each one was identified
- Tenant-wide versus per-user consent. An admin granting on behalf of everyone is a different problem from one employee clicking Accept
- Who consented, resolved to UPN
- Risky scopes held, tiered critical and high
offline_accessflagged separately. An app holding a refresh token keeps its access when the user is not present, and a password change does not revoke it- Whether your tenant permits user consent at all, which determines whether the findings are a snapshot or a ceiling
What it does not catch
The script prints this at the end of every run, because a tool that hides its own blind spots is worse than no tool:
- Personal accounts. Someone pasting a customer list into a chatbot on personal webmail leaves no trace in your tenant. Probably the most common form of shadow AI, and invisible here
- Desktop and CLI clients. ChatGPT Desktop, Claude Desktop, Ollama — no Entra grant required
- IDE assistants and MCP servers. These run with the developer's own permissions and never appear as an enterprise app
- AI embedded in already-approved SaaS. The grant predates the AI feature
- Prompt content. This shows what apps *can* read, never what anyone actually sent
- Browser extensions. No tenant consent required
A clean result means your Entra consent surface is clean. It does not mean your organisation is not using shadow AI.
About the signature file
ai-app-signatures.json ships with its appIds arrays deliberately empty.
An application's client ID is the same GUID in every tenant on earth, so a confirmed GUID is a reliable detection everywhere. A wrong GUID, though, produces a silent false negative and tells an administrator they are clean when they are not. Publishing unverified GUIDs would be worse than publishing none.
So the file bootstraps on domain and display-name heuristics and gets accurate through contributions. Run -Discover, confirm the GUIDs in your own tenant, and send a pull request. Matching runs appId, then domain, then name pattern, and every finding reports which rule hit and at what confidence.
Roadmap
- v2 — Intune and Jamf queries for installed desktop AI clients, closing the largest blind spot above
- v2 — MCP server inventory on developer endpoints
- v3 — Google Workspace OAuth equivalent
MIT licensed. Issues and pull requests welcome, particularly confirmed application GUIDs from your own tenant.