Skip to content
Andy Kemp Tools

Sign-in security · Assess and deploy · v1.4.5

How well is sign-in protected in your tenant?

Access Baseline checks your Microsoft tenant against a best-practice, Zero Trust sign-in baseline for Microsoft Entra, and tells you, in plain language, what's in place, what's missing, why it matters and what fixes it. It starts with passkeys and least privilege. Then, when you're ready, it deploys the core Conditional Access policies for you: report-only, on a pilot group first, after you've reviewed every value.

Read-only first
the assessment changes nothing
Report-only
every policy it deploys, until you turn it on
Six areas
each finding with why and the fix
Your browser
where the report is made

What it checks

Six areas of sign-in security

Everything is read from your tenant with your own sign-in. The assessment changes nothing.

Break-glass access

Whether emergency accounts exist, are left out of every Conditional Access policy, sit in a protected group and have a security key.

Conditional Access

Legacy sign-in, device code phishing, MFA for everyone, phishing-resistant MFA for admins, guests, Azure management and risky sign-ins. Also policies that apply to nothing, or sit in report-only.

Authentication methods

Passkeys first, then Microsoft Authenticator, Temporary Access Pass, and whether text and voice codes are still allowed.

How your people sign in

How many have a passkey, Authenticator, only text codes, or nothing at all.

Admin accounts

Admins without a device-bound passkey or security key (and those who only have Windows Hello for Business), admins who rely on a synced passkey, and how many Global Administrators you have.

Licences

Whether you have Entra ID P1 (needed for Conditional Access) and P2 (risk policies and just-in-time admin roles).

What you get

A score, the reasons and the fixes

  1. 1

    A score

    Weighted by severity, so a missing break-glass exclusion counts for more than a policy left in report-only.

  2. 2

    Every finding, explained

    Grouped by area, each with what was found, why it matters and what fixes it, in plain language.

  3. 3

    CSV and JSON reports

    Made in your browser and saved straight to your computer, to keep or share with your team.

Deploy the core policies

From findings to report-only policies

After the assessment, design and deploy the core Conditional Access policies: legacy authentication and device code phishing blocked, security info registration protected, MFA without text or voice codes, passkeys only for privileged accounts, and guests covered.

Break-glass group required

Choose your emergency access group, or create a new role-assignable one. It's left out of every policy, and nothing deploys without it.

Privileged accounts

Choose your admin accounts, suggested from who holds an admin role now, for an existing group or a new role-assignable one. Members are only ever added.

Passkeys only for admins

CA901 accepts only attested, device-bound passkeys (a FIDO2 security key or Microsoft Authenticator), and a one-time Temporary Access Pass so a new admin can register their first one. Attestation in the passkey profile is the main control, so only certified security keys and Microsoft Authenticator can be registered. Limiting CA901 to listed authenticators is optional, and off by default. Smart cards and Windows Hello for Business aren't accepted, and the page explains why.

A device-bound passkey profile

Privileged accounts get a passkey profile that allows device-bound passkeys only, with attestation required, so each key proves it's genuine when it's registered. Allowing synced passkeys is a separate choice, with a warning that it isn't recommended. Every other profile is kept, and the review warns if a broader profile still allows synced passkeys, with what to do. It takes effect straight away, and the review says so.

A pilot group first

The user policies apply to a pilot group, not everyone. CA901 targets the admin roles and your privileged accounts, and guest policies target guests and external users.

Report-only

Every policy is created report-only, so you can see what it would do before anything is enforced.

Every value reviewed

The review shows each group, both authentication strengths, the passkey policy before and after, and every policy's exact settings, and you type DEPLOY to confirm.

Safe to run again

Existing policies are found by name, or by the reference at the start of a renamed one, so nothing is created twice. The review compares each with the baseline, setting by setting, and you choose which to update. An update changes only the baseline's own settings, never a policy's state, who it targets or your exclusions.

The policies

  • Block legacy authentication
  • Block device code flow and authentication transfer
  • Protect security info registration
  • Privileged accounts need device-bound phishing-resistant MFA (CA901)
  • Azure management needs strong MFA
  • Everyone needs MFA, without text or voice codes
  • Guests need MFA, and can't reach admin portals
  • Risky sign-ins and high-risk users, with Entra ID P2

Coming next

Next, Access Baseline will create break-glass accounts, set up the wider authentication methods, turn policies on in rings and widen them beyond the pilot. People will move to passkeys in stages with Auth-Onboard.

Your data

Your tenant, your browser, least privilege

Your sign-in stays in your browser

Your Microsoft sign-in stays in your browser tab. Andy Kemp Consulting never receives your tokens.

Findings stay with you

The assessment runs in your browser and the report is saved to your computer. Nothing it finds is sent to Andy Kemp Consulting.

Only your tenant's ID and name

That's all that's recorded, to link your tenant to your organisation.

Read-only first

The assessment uses three read-only Microsoft Graph permissions. The deploy permissions are asked for only when you deploy.

Tells you when you hold too much

Signed in as a Global Administrator? It still works, and tells you which smaller role would have done.

Easy to remove

When you've finished, remove the app from your tenant, and every permission goes with it.

Get started

What you need

  • A Microsoft 365 tenant. Conditional Access needs Entra ID P1; without it, the assessment still runs and says so.
  • An account with Global Reader to run the assessment. To deploy, Conditional Access Administrator, plus Groups Administrator for new groups, Privileged Role Administrator for the role-assignable break-glass, privileged accounts and exclusion groups, and Authentication Policy Administrator for the passkey profile.
  • One-off admin consent for the app, AKC Baseline Deployer. The assessment reads on your behalf with three read-only Microsoft Graph permissions:
    • Directory.Read.All: users, groups, roles and licences
    • Policy.Read.All: Conditional Access, security defaults and the authentication methods policy
    • AuditLog.Read.All: the sign-in methods report (who has registered what)
    Deploying asks for its own permissions, only when you deploy: Policy.ReadWrite.ConditionalAccess and Application.Read.All, plus Group.ReadWrite.All to create groups, RoleManagement.ReadWrite.Directory for the role-assignable groups and their members, and Policy.ReadWrite.AuthenticationMethod for the passkey profile.
  • Your organisation approved for Access Baseline: request access on deploy.andykemp.com.

Questions

Frequently asked

How do we get access?

Your organisation requests access to Access Baseline on deploy.andykemp.com, and Andy Kemp Consulting approves it. Approval is per organisation.

Does it change anything in our tenant?

The assessment doesn't: it reads your directory, policies and the sign-in methods report, and changes nothing. Changes happen only if you choose to deploy the core policies, after you've reviewed every value and typed DEPLOY. Even then, every policy is created report-only on a pilot group, and an existing policy is only updated if you choose that on the review screen.

Which role do we need?

For the assessment, Global Reader, the smallest role that reads everything it checks. Security Reader works too, apart from the authentication methods policy. To deploy, Conditional Access Administrator, plus Groups Administrator for new groups, Privileged Role Administrator for the role-assignable break-glass, privileged accounts and exclusion groups, and Authentication Policy Administrator for the passkey profile. Granting consent needs someone who can grant admin consent, such as a Cloud Application Administrator.

What does Andy Kemp Consulting see?

Only your tenant's ID and name, to link it to your organisation. Your tokens stay in your browser tab, and the findings and report never leave your browser.

What happens if a policy already exists?

It's left as it is unless you choose otherwise. Access Baseline finds each existing baseline policy by name, or by the reference at the start of the name if someone renamed it, and the review compares it with the baseline, setting by setting. Tick "Update to the baseline" for a policy and only the baseline's own settings change: its state, who it targets, its name and your exclusions are kept. A policy missing the break-glass exclusion is ticked for you. Existing groups and authentication strengths are never changed, and nothing is removed. The passkey policy is merged: its own profile is added, and every other profile and target is kept.

We don't have Entra ID P1. Is it still worth running?

Yes. Conditional Access needs Entra ID P1, but the assessment still runs without it and says so, alongside everything else it checks.

Why only passkeys and a Temporary Access Pass for admin accounts?

Because of human nature. On a PC they also use for everyday work, people set the same Windows Hello PIN for their admin account as for their everyday one, so Windows Hello for Business would make the admin sign-in only as strong as the everyday one. Smart cards and their PINs get lent, written down or left in readers, and certificates aren't checked as genuine when they're issued. A passkey on a security key or in Microsoft Authenticator is bound to that device, checked as genuine when it's registered, and separate from everyday sign-in. A one-time Temporary Access Pass gets a new admin started, and for an admin account only a Privileged Authentication Administrator can issue one.

Can it deploy the baseline for us?

Yes, the core Conditional Access policies, and device-bound passkeys for your privileged accounts. Your break-glass group is required and left out of every policy, the user policies apply to a pilot group, and every policy is created report-only after you've reviewed every value. Turning policies on, widening them beyond the pilot, break-glass accounts and the wider authentication methods come next.

See it before you connect anything

The demo walks through Access Baseline on a made-up tenant, so you can see the findings, the report and the deploy flow before you connect your own tenant.

Downloads need a free account

The tools are free to use. To download what they produce, sign in with your free andykemp.com account: the same one for the blog, the Tenant Audit and the deploy tools. No newsletter unless you choose one.

Prefer to sign up on the main site? andykemp.com/join