How to report on Microsoft Authenticator password-less phone sign-in & FIDO2 security key usage using Azure AD & Azure Monitor Log Analytics

Implementing password-less authentication methods at your organization also requires you to pull metrics on usage of these methods to gauge user adoption. Anything end-user facing must always be tracked to understand usage patterns and ways to improve the user experience should users be running into any adoption challenges. Password-less authentication is extremely user friendly and adoption skyrockets with users in my experience, but it’s always good to have the real-world tangible metrics in hand to support your anecdotal evidence.

I’ve covered the modern passwordless authentication methods Microsoft supports in previous articles: Windows Hello for Business, Authenticator phone sign-in, and FIDO2 security keys. The latter two are very easy for IT departments to deploy which means rapid user adoption you will want to track metrics on closely. Authenticator phone-sign and FIDO2 hardware security keys are newer access management methods so Azure AD doesn’t display them in some of the available authentication-related UIs yet. I had trouble getting the information out from Microsoft Graph initially with lots of empty Kusto queries but eventually found a way to extract the data which I will show you step by step below.

Pre-reqs

• Azure AD Premium P1 or P2 – you need one of these tiers in order to export sign-in data, this is either $6 or $9 per user per month respectively
• Azure Monitor Log Analytics Service – sign-in logs have to be stored in Azure so there may be a cost to you when using this service:
  • Pay-As-You-Go = you get 5 GB per month for FREE, after that it’s $2.76 per GB presently. Anyone with a personal AAD tenant or small business, you should be using this tier as it’s free for you and you will not hit the 5 GB per month limit since sign-in logs are tiny. This is awesome that Microsoft gives you this benefit. Any logs stored beyond 31 days is charged at $0.12 per GB. This 12 cents per month is pretty negligible granted you may not even hit 1 GB worth of sign-in logs, but you can always trim it to save this cost if you wish.
  • Capacity Reservation = used by large organizations to save Azure storage costs, goes from 100 GB to more than 500 GB per day with up to 25% cost savings over Pay-As-You-Go. Data retention here is the same as the above at $0.12 per GB beyond 31 days. For most enterprises needing to meet one or more regulatory compliance policies, you will want to factor this cost of log retention into your planning.
  • Rates may change in the future you can always get the latest pricing info here: https://azure.microsoft.com/en-us/pricing/details/monitor/

Setting up your Azure Log Analytics reporting

1. Go to to the Azure Portal and navigate to “Log Analytics workspaces” under All services or click it in the left nav bar if you have it favorited, then hit Add:

2. Create a new Resource Group in your subscription and call the Log Analytics workspace instance whatever you like and in the Azure region you would like to store the log data:

3. You can skip the Pricing tier and Tags headings if you wish and go straight to Review + Create to get the workspace created:

4. Go to Azure AD > Diagnostic settings

5. Now click “Add diagnostic setting”

6. Now give it a name, click the AuditLogs and SignInLogs, check Send to Log Analytics, choose your subscription, and choose theLog Analytics workspace you created in the previous step. Finally hit Save at the top:

7. It’s going to take some time to send logs into the workspace depending on your organization. Once some time goes by go back to the Log Analytics workspace and click the workspace you had created:

8. Now click on Logs and a new blade will open up giving you the ability to build a Kusto query against the AuditLogs and SigninLogs you started forwarding to this workspace several steps ago.

9. It took a lot of trial and error as initially the Kusto query wasn’t pulling any data on newer passwordless authentication methods. Then finally one day very recently I saw the data was populating with my queries. The Kusto query I have been successful with is below and here is an explanation of what is displayed:

• This Kusto query runs against SigninLogs
• It will select individual users you are targeting but you can modify this bit to target your entire organization if you like
• It will look for sign-in events from the past 30 days but you can extend this or narrow it to whatever time period you like
• It will ignore PHS (password hash sync) based sign-ins, you can also add other sign-in methods you want to filter out the same way
• It will display the passwordless authentication method used
  • FIDO2 security keys will show as “Fido“
  • Authenticator passwordless phone sign-in will show as “RemoteNGC” which I believe to be the short form of the Remote NGC Credential Provider, one of the modern credential providers available in Windows 10
• It will display information about the user, the app they logged into, and additional details about their authentication. You can choose what details you want to display including IP address, Location details, Conditional Access policies that may have been applied, etc.
• It will sort by Date and Time descending

Here is the Kusto query itself:

And the results of this Kusto query will be displayed in the table below:

You can further filter out some of the blank spaces you may see here. So it only shows actual Authenticator phone sign-in and FIDO2 security key sign-in events. Here is a closeup of both authentication methods shown as “RemoteNGC” and “Fido” respectively. I am sure over time these authentication methods may become more descriptive in the logs but for now, you can use this string parsing method in my Kusto query to extract this information:

If all your users aren’t using password-less authentication yet and use traditional user ID + password + Azure MFA TOTP for example, then it will be logged as “Mobile app verification code”. This is legacy and you should encourage your users to move to Authenticator passwordless phone sign-in if you see this or other older sign-in methods:

10. Now you can use the top toolbar to Export the Kusto query results to a .csv format and use this data however you like:

11. Once you are happy with your Kusto query, you need to Save it. You must give it both a Name and a Category in order to save it:

12. A grid format alone is great raw data, and the CSV will certainly help you create graphs in Excel, but it would be nice to have it right in the Azure Portal on-demand, right? Like a dashboard. This is where Workbooks, Power BI, or Azure Dashboards come into play. They take data sources like the Kusto query you just created and make them look pretty and presentable to your leadership. This is a skill I’m still developing so I’m not going to go into it too deep here until I get things looking exactly the way I want. There is a bit of a learning curve with some of these. Once I get the visualizations the way I want I will write a piece going through some of these options.

I hope this helps you get started tracking your modern password-less authentication usage. Please leave a comment below if you have any questions.