Conditional Access in Microsoft 365 is one of those things that can either be your best friend or your worst headache! And while most of us have used the “What-If” tool in the Entra portal to test policies, you’ve probably wondered:
Can I automate this testing? Can I simulate hundreds of sign-in scenarios in one go? Can I monitor policy behavior daily instead of manually testing each time?
Meet the What-If Evaluation API, the same engine that powers the portal tool, but available for automated, large-scale simulations.
The Legacy What‑If Tool
The Conditional Access What-If tool has been part of the Microsoft Entra portal for a long. It’s super handy for interactive, one-off testing. You choose a user, maybe an app, set optional conditions like device or location, click “What If,” and voila, you get a UI report showing which policies would apply and which wouldn’t.
It’s great for quick, one-off investigations. But as environments scale and scripts become necessities, it shows its limits. Here’s why:
- It’s manual and you must interact with the UI.
- It’s one scenario at a time.
To handle scaled environments and streamline evaluations, the What-if Evaluation API brings in the flexibility and automation the UI can’t offer.
About Graph-Based What‑If API
Microsoft has released the What‑If Evaluation API on the Graph /identity/conditionalAccess/evaluate endpoint. This is the backend powering the enhanced What‑If experience in portal too. Here’s what sets it apart:
- Official public API: Rather than an undocumented internal endpoint, this is a fully supported Graph API. Great for automation, custom apps, CI/CD – anything scriptable!
- Same logic as real sign-ins: It uses the exact authentication engine used during actual policy enforcement, so simulations reflect real-world outcomes
- Rich conditional input: Compared to the legacy UI, the API supports insider risk, authentication flow, and demands full context.
- Precision over shortcuts: The old UI sometimes says a policy “applies” even with incomplete data. The new API only confirms when all conditions match. Partial or missing data means it won’t apply and you’ll know why.
Ever since it hit public preview, usage has skyrocketed, up by nearly 220%. And honestly, I can totally see why.
Use What-if API to Automate Testing
To secure access to Contoso’s sensitive Payroll app, I started with a simple approach using the What-If tool in the Microsoft Entra portal. The goal was clear: enforce MFA, require reauthentication, and apply the policy only to risky users or those signing in from untrusted networks, while keeping it in report-only mode during the first week to monitor quietly.
I ran a few manual simulations using the tool: a risky user signing in from outside a trusted location correctly triggered the policy, while a trusted user on a compliant device was allowed through. So far, so good, but these were just a handful of cases, tested manually.
To take it further and scale the checks, I built a script using the Graph-based What-If API to simulate a much wider range of sign-in scenarios:
- A user logging in from an obscure country
- A flagged user trying to access resources outside the corporate network
- A sign-in from a region where Contoso has zero presence
- A legacy app targeting the Payroll app
- A risky user using a non-compliant device
The script takes in key parameters: user ID, app ID, location, device info and calls the API to see exactly which Conditional Access policies would apply. If anything unexpected pops up, it flags it immediately.
Even better? You can schedule this to run daily. So. if a policy change unintentionally opens a gap or causes drift, the team gets alerted before it becomes a problem.
How to use What If – Microsoft Graph Evaluation API
Under the hood, the Graph-based What‑If tool is powered by the new /identity/conditionalAccess/evaluate Microsoft Graph endpoint. This API expects detailed sign-in context to evaluate which Conditional Access policies would apply. You’ll need to include key elements such as:
- User or service principal
- Target apps or resources
- Conditions like location, risk level, device platform, and more
You send a POST request with all relevant data. The API returns exact policy determinations—applied or not, and the underlying reason (policyNotEnabled, locationMismatch, etc.)
📌 Be sure to include all relevant conditions in your payload. Missing values like risk level or device platform can result in a vague notEnoughInformation response.
- In Graph Explorer, hit the beta endpoint with your JSON payload.
- In PowerShell, use the Test-MgBetaIdentityConditionalAccess command.
Want test automation? Use frameworks like Maester that integrate API calls and even let you write tests (e.g. enforce MFA, or block access from certain roles)
A Few Friendly Reminders
- Classic policies aren’t supported, the API only evaluates modern Conditional Access policies. If you’re still using legacy setups, it’s time to migrate.
- The scenario simulation doesn’t include dependencies like Teams relying on Exchange, just the policies themselves.
This isn’t just about testing one user or one scenario. It enables automated, large-scale, and continuous validation of your Conditional Access policies.
Give it a shot: build a daily tester, write a few edge-case scenarios, and see what your policies would do. Then deploy with zero doubt!
