Steve on Security

Deprecating NTLM is Easy and Other Lies We Tell Ourselves

site@syfuhs.net — Wed, 04 Oct 2023 18:46:00 GMT

Edit 3: We have a video!

Edit: Oh hey, we announced our strategy.

We've been hinting at the deprecation and removal of NTLM from Windows for a while now. We're finally talking about how we're doing it.

Bluehat Podcast (not the presentation): The BlueHat Podcast: Deprecating NTLM is Easy and Other Lies We Tell Ourselves with Steve Syfuhs on Apple Podcasts

Webinar Recording (not Bluehat): The Evolution of Windows Authentication - YouTube

What: I'm giving a presentation
When: October 12th
Where: Microsoft Bluehat Conference
Streamed: Good question
Recorded: Yes [links TBD]

More details to follow. BlueHat | Microsoft

Finding and Killing the Process that Opened Your Port

site@syfuhs.net — Fri, 04 Aug 2023 02:57:00 GMT

Saved for when Future Steve goes searching for examples on how to do this again.

I had a project that required acting as a server by opening a socket on a particular port and then listen for incoming connections. Not that particularly interesting. However, the process of building it was kind of a pain because the program would periodically linger in the background holding the socket open and launching a new instance would fail because hey look someone already has that socket open.

In order to optimize my laziness and reduce my overall frustration I opted to have the new instance find and kill the old instance. It's a dev server, what do I care? Anyway.

There are tools that show you which processes own which ports. There are tools that kill processes. There aren't really tools that show you port owners and then let you kill them. Certainly not in any easy to use way. On top of that, all the examples online just show you how to marry the two, usually by grepping your way through a whole bunch of string dumps, and well, nah.

Just query the damn socket table:

Improvements in Windows Kerberos Architecture

site@syfuhs.net — Fri, 14 Apr 2023 21:39:00 GMT

It's Friday afternoon, it's sunny out, and I have no desire to start on any new projects this late in the week.

Let's discuss architectural changes to Kerberos cryptography in Windows!

Why do you care? You don't.
Will this practically affect you? No, it won't.
Is it interesting? Well, I think it is.

Anyway...

Many many moons ago Kerberos in Windows supported only RC4 and DES algorithms for the various encryption and signing requirements.

Because of this there was a useful dichotomy of things. Code was easy enough. Was it doing RC4? Yes, carry on. No? do DES.

This meant you could reason about all the different facilities of Kerberos fairly easily because whatever you were doing only needed to know two states.

This actually worked relatively well for a while because technically you could bolt on 3rd party encryption algorithms, and they'd more or less just work as long as they followed either the DES way or the RC4 way.

This was manifested through the CDLocateCSystem and family of cryptdll.dll library functions.

These are undocumented functions, and they certainly were never intended for 3rd parties to use.

Nevertheless.

This all came about in the late 90's for the release of Windows 2000 and went along without issue until Vista/Server 2008 because: AES.

AES was an important milestone because it was and still is a solid crypto suite.

Bringing AES into the picture wasn't that difficult at a low level. There's that cryptdll.dll thing and it exports all the usual functions you'd expect like "please gimme all the crypto suites you support" and "I've chosen AES, please give me the encryptor for that".

On top of that, AES also met the standard that it more or less behaved like RC4 or DES, so it fell into the do-as-DES-does flow of logic.

Beautiful. Life is good.

Ah, but DES is shit. We also need to kill DES quickly and quietly.

Okay, find and replace "DES" with "AES" and life is good, right?

Nah, we also need to continue supporting DES for those ERP systems that only support DES, but never willingly choose DES normally.

Okay, so do AES or RC4, or DES if DESONLY, and if AES follow the DES logic for etype-info and salts and such.

Oh, also AES can only be used on Server 2008 DFL because reasons.

Uhhhhhhhhhhh. K.

And so, this monstruous pattern was born (paraphrased, not real source code).

That's ehh not great. Doubly so when it's duplicated in subtly different ways a few dozen times throughout the code.

It's ugly, but manageable enough so long as you don't have to futz with the logic, and it has worked just fine for 20 years.

Okay, but what happens if you need to kill off another cipher suite because, say, a researcher finds a flaw in how checksumming is handled. 😶😬

Welp.

We *could* just add in checks everywhere for DES... and RC4. It's clunky, but it's not the end of the world.

We could just remove RC4 from the cryptdll implementation.

Yeah nah, that is the end of the world.

Let's look at the cards dealt to us.

DES -- dead, removed in future version.
RC4 -- dying, removed in future version.
AES-SHA1 -- alive, bit ornery, not going anywhere.

That's actually not a great hand. We need to get better cards at some point. We have options, which in the simplest terms is RFC 8009 -- AES-SHA2.

But how do we add that? More important how do we add that AND also remove RC4?

That's right kids! We rewrite the entire crypto stack in Kerberos!

Hey look at that.

The value here is significant because we can (and did) rewrite all the monstrous logic spread throughout the codebase into logic that never has to be touched ever again.

This means all the logic to disable a cipher outright like DES, or disable certain usages of the cipher, like RC4 session keys, is hidden from the rest of the stack.

On top of that, adding a *new* cipher suite is relatively trivial because only one spot in the code needs to know.

And most importantly I get to delete functions like this.

Kerberos Event ID 27

site@syfuhs.net — Thu, 24 Nov 2022 01:20:00 GMT

Update Dec 2: If you're finding this post as a result of the November 2022 Windows patch, I recommend you review KB5021131: How to manage the Kerberos protocol changes related to CVE-2022-37966 - Microsoft Support if you haven't already. Pay special attention to this registry key:

You can try setting this to 0x1C.

There's been a bunch of questions from folks about Event 27 after the 11B changes. The presence of this event is not an indicator of another bug. It is the result of a fix to one of the vulnerabilities in 11B. Let's decode what this is saying.

The event log text:

TGS-REQ: hopefully obvious by now -- you've authenticated already. You've exchanged your password or whatever for a TGT. TGT is good.

You're requesting a ticket to an SPN class/service.domain.com.

Key ID = 9: poorly labelled -- means the action the key is needed for. It's an internal mapping for us for debugging because from a customer diagnostics perspective it's all the same.

9 is service ticket encryption. That means the KDC could not find a key to encrypt the ticket.

Okay, why can't it find a key? How does it decide which key to pick?

It finds the intersection of 3 sets:

Requested ETypes
Account keys
KDC supported ETypes

Two of those sets are listed in the event.

Requested ETypes is literally the requested ETypes in the Kerberos TGS request body. "As a client, I only support XYZ ETypes".

Account keys are the actual keys stored in the database.

KDC supported ETypes is what's recorded in the registry as... supported.

Let's do some math.

1=DES1
3=DES2
17=AES128
18=AES256
23=RC4

23, 3, 1 ∩ 23, 18, 17 = 23

Okay, only one common key, and that is...RC4. Yeeeugh.

But that should work, shouldn't it? RC4 is terrible, but selected EType is selected EType.

Except we're missing an intersection against KDC supported etypes. That's this policy applied to the DC. Default is 0, which means pick whatever Windows thinks is best. Most people pick AES and/or Future encryption types.

The policy most folks choose is AES/Future. It's good and safe. That means we're working with a written value of 0x7FFFFFF8. Let's decode that. Remember start at the right.

0x7FFFFFF8 => 0111 1111 1111 1111 1111 1111 1111 1000
DES1 -----------------------------------------------^
DES2 ----------------------------------------------^
RC4 ----------------------------------------------^
AES128 ------------------------------------------^
AES256 ----------------------------------------^
AES-SK ---------------------------------------^

Okay, back to math.

23, 3, 1 ∩ 23, 18, 17 = 23
23 ∩ 18, 17 = null

Bang. Oops.

Time to write the event.

Okay, well there's clearly a fault here. Why is the request only asking for RC4 and worse DES?

Well, because that's what the client was configured to send.

Why was it configured to send that? Because machine policy said so.

Addendum

Because of course it's always a little more complicated. After reports of clients also being configured for AES-only I reviewed the code again.

The "Requested ETypes" values in the log may also refer to the service account msDS-SupportedEncryptionTypes attribute.

The logger will choose which one to record based on a couple of conditions, so of course it's not actually guaranteed to be what's in the request, just sometimes. exasperated.gif

And for completeness if that attribute is zero, it pulls the value from the default enctype registry value.

So the above logic is still correct for why it fails. You asked for X, system said it only supports X, but the service in question is configured for Y.

Understanding Windows Authentication

site@syfuhs.net — Mon, 13 Jun 2022 22:14:00 GMT

This is just a list of all the things I've written on Windows authentication. I've sorted it by rabbit hole.

Architectural Things

What Happens When you Type Your Password into Windows?

How Azure AD Windows Sign-in Works

How Windows Single Sign-On Works

How Authentication Works when you use Remote Desktop

Identity Delegation in Active Directory

How Managed Service Accounts in Active Directory Work

A Bit About the Local Security Authority

On Computer Passwords

Why We Built Azure AD Kerberos

Windows Hello Cloud Trust

The Protocols

Kerberos Explained in a Little Too Much Detail

Windows and Domain Trusts

Kerberos FAST Armoring

Hybrid Authentication with FIDO

How Azure AD Kerberos Works

Credential Protection

Protecting Against Credential Theft in Windows

A Strategy for Protecting Privileged Access

How Windows Defender Credential Guard Works

How does Remote Credential Guard Work?

Service Hardening

Lessons in Disabling RC4 in Active Directory

Killing NTLM is Hard

MFA is Hard to do Right

How does Remote Credential Guard Work?

site@syfuhs.net — Sat, 26 Feb 2022 20:51:00 GMT

In the spirit of distracting myself from Doom Scrolling, let's talk about a feature that is super useful that many folks don't really know a lot about: Remote Credential Guard.

I've gone into relatively great detail about all the different stages of Windows auth and I encourage everyone to read through the posts because there's literally centuries of dev hours put into building all of this and it's complex.

The first thing is how does basic sign on work? What Happens When you Type Your Password into Windows? (syfuhs.net)

The second is how does basic RDP authentication work? How Authentication Works when you use Remote Desktop (syfuhs.net)

The third one is how does Credential Guard work? How Windows Defender Credential Guard Works (syfuhs.net)

The fourth one is What is the LSA? A Bit About the Local Security Authority (syfuhs.net)

Now, since I don't expect everyone to go off and read those let's summarize a bit.

In a normal world you type your password into Windows and that kicks off a bunch of machinery that verifies the credential, sets up your logon session, creates your desktop, etc.

RDP does the exact same thing, except it has a precursor step that makes sure you're sending the password to the right computer called NLA. After, the RDP client literally sends the password you typed in across the network to the remote machine.

The remote machine effectively does the equivalent of typing that password into a credential provider for you, kicking off all the logon machinery. This is super important because the password unlocks all sorts of important systems within your logon session.

There are things like DPAPI that are bound to your credential. SSO using Kerberos or NTLM require the password to get a TGT or challenge. It's a fairly important prerequisite. How Windows Single Sign-On Works (syfuhs.net)

It turns out this is really rather dangerous if you don't trust that remote machine, because it has your password now whether you like it or not. Incidentally this is why keeping NLA on all the time is important.

So, what do we do about it? Well, if we go back to how Credential Guard works, we have this concept of a tiny locked down VM running beside your regular session where you can periodically ask it certain specific questions.

1. Can you encrypt/decrypt this for me? (Yes)
2. Can you give me my password to send to this other machine (NO!)
3. Can you store this session key so attackers can't steal it? (Yes)

Yadda yadda yadda.

Coding-wise it was implemented [more or less] as an interface of questions you can ask. We have two implementations: the first is the one LSA uses when CG is off. Things just stay in LSA memory. The second lives in LsaIso, where LSA interacts with it over Hyper-V-provided RPC.

When CG is on, LSA knows to call the RPC version, otherwise it calls the normal-world version. Most important secret-y things travel through this interface.

Only LSA can call the interface implementations, so at worst you're still protected from normal attacks because you must first breach LSA before doing anything interesting.

Which, by the way go turn on LSA PPL please.

The engineering value behind this design is also super high because it makes it clear who is responsible for what security decisions. I'm very proud of the devs that designed and built this because it was a monumental engineering task.

Anyway.

So, you have:
Password => LSA => Cred Guard

Cred Guard <=interface=> LSA <=> Apps

Now we introduce RDP. On the target machine it works exactly the same way, except that password came from a remote machine.

What if we did this?

Client:

Password => LSA => Cred Guard [client machine]

RDP Server:

Cred Guard [client machine] <=RDP interface=> LSA <=> Apps

[thinking.gif]

Remember that interface thingy that LSA calls? What if we built a 3rd interface that instead of doing things in memory, or RPC over Hyper-V, it does it with RPC over the RDP transport whatsits (I can't remember what they're called).

Now your creds stay on the client, and whenever the RDP target needs to use those secrets for whatever reason it asks LSA as usual, LSA asks the interface, and it just happens that the interface implementation lives on the other side of the world in your client machine.

[mindblown.gif]

And so we have Remote Credential Guard.

Now, obviously this is not a fool-proof solution to all security problems. You have full SSO to all things on the remote machine just as if you typed in your password. That means something on that machine can do something malicious as you via SSO.

But they can only do it for as long as your session is active. Once you disconnect, the connection to the client LSA is dead. Requesting new tickets fails. The target machine's ticket cache is also useless because it requires Cred Guard to function. NTLM has no ntowf.

Windows Hello Cloud Trust

site@syfuhs.net — Tue, 22 Feb 2022 21:16:00 GMT

Windows Hello Cloud Trust: What is it? Why do you care?

A thread.

In the early days, Windows Hello for Business came in two deployment flavors: Certificate Trust or Key Trust.

Certificate Trust is basically the answer to the question "what if we made smart cards unlockable with your face?"

This is a surprisingly accurate depiction because under the covers Windows Hello uses certificate logon to get you to the desktop, which in the CT case means spinning up PKI, deploying templates, protecting private keys through hardware binding, etc.

Smart cards require the exact same infrastructure, and the only difference is where the private key is stored. In the Hello case it's stored on disk and protected by your TPM. Smart cards store them in silicon embedded on a plastic card.

This model works about as well as you can expect for a system that is not dependent on the cloud and must coexist with existing PKI within large organizations, and actually: it works well. We cut out teeth on getting the infrastructure right with smart cards over 20 years.

All of this came about in Windows 10, and didn't require changes to Active Directory, which means no Domain Controller deployments. Okay, that actually makes for some reasonable tradeoffs.

But then we had this other mode: Key Trust. This mode doesn't require PKI and gives you some flexibility in deployment, but for all intents and purposes requires The Cloud to do all sorts of orchestration plus enough Server 2016 DCs to handle authentication load. Tradeoffs.

Between the two, for the vast majority of customers Key Trust is a lot easier to deploy as they're already hybrid, or are planning it, don't already have PKI in place, and have no problem standing up a handful of 2016 servers.

But there are some downsides. See, when you log into Windows with Windows Hello the authority with which you authenticate against is determined by your join state. Domain join => Domain Controller. Azure AD Join => Azure AD.

Doing the face thing or the fingerprint thing or the PIN thing unlocks a private key, and that private key is used to authenticate to the machine authority. In DJ land that's a certificate, doing Kerberos PKINIT. In AADJ land that's an OAuth signed assertion.

Certificate trust doesn't need to do anything special, since the PKI is all local to AD and AD fundamentally understands the cert presented to it. The cloud requires something like ADFS to translate the certificate to something AAD understands.

Key trust is the reverse: the cloud natively understands the key and AD needs it translated. Most folks deploy AAD Connect and rely on a backsync where AAD pushes down the Windows Hello certificates to AD.

This backsynced key gets stored in a special location different from where certificate trust or smart cards go For Reasons (good reasons) which necessitates 2016+, but still fundamentally works as a smart card does by authenticating using Kerberos PKINIT.

In principle this is fine since most folks have AAD Connect, but there are problems because this sync can take anywhere from 1 minute to 30 minutes on average, plus however long it takes to replicate from the sync target DC and whichever DC you're authenticating against.

In short: it takes a long time to provision a key trust key on-prem, and you can't use Windows Hello to log into your hybrid machine for... a while. And you can't access on-prem resources on AADJ machines for... a while.

It's not ideal.

Remember FIDO? Remember all that work we did to build Kerberos into the cloud? Remember how when you log into a machine, we immediately give you a TGT from AAD and use it to log you into your machine and get you access to on-prem resources?

So: Cloud Trust.

[Ta-da]

Cloud Trust gives us the best of both worlds. We get instant provisioning and instant hybrid auth, all without having to deploy any additional infrastructure. Neat.

So, how does this thing work? Well, you start with Key Trust. You do all the same onboarding into AAD, where you generate a key, register it with AAD and voila instant Hello. This part is a little more complicated, but honestly, not by much. Make key, have key, give key, use key.

The heavy lifting is now the backend. In the before-times AAD had to push the key back down to AD, which took fooooreeeevvveerrrr. Now when you log in with the key you registered 5 seconds ago, we generate a partial TGT and return it to your machine.

Sometime before we generated the TGT you also ran a one-line PowerShell command that:

1. Created an Azure AD RODC object on-prem
2. Synced that RODC secret to AAD

The presence of that RODC secret in AAD is what triggers this magic. It tells us you trust AAD to log you in to your on-prem environment.

So, you log in, we generate a partial TGT encrypted to the RODC secret and return it to you. Your Windows client sees this, sends the partial TGT to a nearby domain controller and asks for a real on-prem TGT.

The domain controller understands that this is a partial (RODC) TGT, meaning it contains exactly one piece of information in it: your username. The reason this is interesting is because it *doesn't* contain your group membership information.

This is actually a pretty useful mechanism because folks don't like synchronizing ALL of their group memberships to AAD, and if we returned a TGT with a subset of your memberships you'd be hosed. So, your DC looks you up by name, does some sanity checks, and generates a full TGT.

Cloud trust takes your hybrid Windows Hello deployment from a very complicated and slow process down to a PowerShell command and some Group Policy or Intune futzing to turn the feature on.

Not bad.

Why We Built Azure AD Kerberos

site@syfuhs.net — Thu, 02 Dec 2021 23:29:00 GMT

Yesterday we announced the public preview of, amongst other things, Azure AD Kerberos. I discussed a bit about how we did it, and why the features depend on it. But I didn't really explain why we chose the solution we did.

Here's why.

There are... mmmmmm ...a bajillion... Windows devices out in the world today (give or take). Some running the latest and greatest Windows 11, others running various versions of Windows 10, others running Windows 8, Windows 7, XP (seriously folks?), and all the various servers.

There are... mmmmmmm ...half... a bajillion line of business apps that rely on Active Directory-backed authentication. Some Windows-based, some not-Windows-based. The vast majority of them use Kerberos to authenticate in some way or another.

Over the next few years folks are going to want to migrate lots of these things to the cloud, either lift-and-shift, rewrite, replace-with-SaaS, etc. and they'll inevitably want to shift away from this on-prem dependency.

There are *very* good reasons for that. AD has networking constraints, doesn't support certain authentication methods like FIDO, doesn't have easy to use policy controls, is showing its age with the likes of NTLM, etc. I could go on.

Conversely the cloud -- Azure AD -- has moved mountains with its capabilities: Conditional Access policies, Multifactor Auth, FIDO, world-class auditing and security and reliability.

Clearly there's a gap here so it makes sense for apps to want to take advantage of what Azure AD can offer. If you're lucky you can replace functionality with OAuth or SAML, but many apps will still require protocols like Kerberos.

These are the laws of big numbers. If you have a billion things using Kerberos and replace half of them with something else, congrats! You've seriously done the unthinkable. However, you still have half a billion things to do something about.

Okay crazy idea: What if we made Kerberos a first-class modern protocol for Azure AD?

Voila: Azure AD Kerberos.

But Kerberos has its issues. It's insecure, it's hard to use, it's ugly, it's it's it's it's.

It's none of those things. You're holding it wrong. Yes, I just said that. Hear me out.

You're holding it wrong because of us.

Making sweeping changes to improve the security of these things is *hard*. Compatibility is one issue, time to market is another, time to mass adoption is yet another. These all add up to great features like gMSAs and Windows Hello, but it take's time. How do we move faster?

Kerberos is insecure because it relies on passwords and humans are bad at keeping passwords secure. The weaknesses are because the security of the protocol relies on the strength of the password and weak passwords can be easy to crack.

We solved that through some pretty simple rules:

You just can't do password auth.
Service Principal secrets are not passwords. They're randomly generated noise.
There's no fallback to NTLM.

Kerberos is hard to use.

I mean, is it?

Nevertheless, we now do all the heavy lifting for you. You had to make sure you typed the password in right and had to give it the right SPN, but now we do that for you in the majority of the cases.

Kerberos is ugly. I guess? But have you looked at how TLS client certificate auth *really* works?

I'm getting off topic.

Anyway, let's talk about it in action. With on-prem AD how would you lock down a file share to require MFA like Windows Hello? Well, I guess you'd have to look up the Windows Hello SIDs and stick deny ACLs on there when they...aren't...present? How the frig do you do that sanely?

Azure AD just requires a Conditional Access policy.

It's a checkbox.

Maybe you only want devices that comply to a certain company policy. With on-prem AD you...deploy PKI for device certs marked with EKUs that indicate compliance, force machine certificate auth, roll out claims policies issued on presence of EKUs and

I can't remember what else.

Just a checkbox in Azure AD.

Maybe you only want users accessing the file share from certain regions or jurisdictions or countries. With on-prem AD you... if you just... ah well... or I guess ...NAP? Is that a thing still?

Still just a checkbox in Azure AD.

Well, okay. They're radio buttons.

Conditional access for Kerberos is a billion-dollar security control if you ask me, and it's just built in to Azure AD for you for free.

Auditing is another critical aspect. You have 300 Domain Controllers in your environment, and you want to find out if a user that lives in North Carolina just logged in at 3:30am from... Angola?

Yeah, I don't even know where to start.

It's a search field in Azure AD.

So, put all of this together and you can see why you might want to modernize your Kerberos usage.

Why Thread Readers are Blocked

site@syfuhs.net — Thu, 02 Dec 2021 17:09:00 GMT

Every time I write out a long thread on some technical topic I inevitably see someone replying to my first post tagging a bot and telling it to unroll the thread.

I block every single one of those bots. They won't work. I will go out of my way to make sure they don't work.

Why?

Because they make money off my content without lifting a finger. I actually don't care if someone reuses my work. Go to town, have at it, make a buttload of money. But, if it's a wholesale copy of my stuff wrapped in advertisements for god knows what and all you've done is copied and pasted? Well, that's a no from me. Also, maybe, you know, ask? I'll say yes; just tell folks where you got it.

To that end I always dutifully put all these threads into posts on this site under the Twitter Thread category and every thread has a link to that post at the end. I even have RSS/ATOM feeds so if you want all these threads in a single place go use an RSS reader.

Not only are the posts formatted better, they also doesn't have advertisements on them. I make no money off this site. It costs me money to host.

Do everyone a favor and stop using these godforsaken bots. It takes time and energy putting stuff into written form and they're taking advantage of people just wanting to create.

How Azure AD Kerberos Works

site@syfuhs.net — Wed, 01 Dec 2021 23:57:00 GMT

I spent the better part of the last two years building the authentication stack used by FSLogix in Azure Virtual Desktop for AADJ machines.

It turns out building an entirely new form of hybrid authentication is complicated. How did we make a platform protocol like SMB work for Windows machines without requiring Active Directory?

Easy (welllll): we turned Azure AD into a KDC for Kerberos.

What do I mean by that? You turned AAD into a KDC? Huh? You did what?!

Let's break this down a bit. You have Azure Virtual Desktop, and you want to stash user profiles into a centralized place. Voila: FSLogix. Not new. But you don't want to host your own file share.

Ah. Well.

Okay, we have this thing called Azure Files which is SMB over Azure Storage, so stuff the profiles there. That's...actually really cool.

How do you authenticate to it? Easy! NTLM. Wait. No, crap. Bad. Bad. Bad.

So AzFiles introduced Active Directory support.

In principle this is somewhat simple. You create a service principal in your own AD and give it an SPN of cifs/your.file.core.windows.net, and whenever you type \\your.file.core.windows.net your Windows client will dutifully ask AD for a ticket and away you go.

"All" Azure Files needs to know is the service principal password and it can decrypt the ticket, get your identity, get your SIDs, and everyone is happy.

Except how does the AVD host contact AD? It needs to be domain joined or AADJ'ed. And it needs line of sight to a DC.

Okay, well your AVD hosts are in the cloud, and your DCs are in your on-prem network and that means VPN or moving DCs into the cloud AND still needing a VPN for replication yadda yadda yadda. Why can't Azure Files just use AAD to authenticate users?

Let's call it impedance mismatch. Windows understands SMB. SMB understands NTLM or Kerberos. AAD understands OAuth.

Ah.

Well, what if we make AAD understand Kerberos? How do you make the world's largest [web-based] identity provider natively understand the world's most-used [not-web-based] authentication protocol?

And so begins a two-year journey.

First of all, let's set some ground rules. Kerberos is great for many reasons, but it's also janky for others. It has two legs: AS and TGS.

AS exchanges primary creds for tickets.

TGS exchanges tickets for tickets.

The AS leg can be a bit janky.

AS requires both parties have knowledge of the client's password (or asymmetric keys) and AAD doesn't necessarily know either of those, at least not in the forms necessary to make the protocol work. Also, what about support for new authenticators like FIDO? Hmm.

Wait a minute. We've solved this problem already. What if AAD gives us a TGT during logon? That's how we made FIDO work. Hybrid Authentication with FIDO (syfuhs.net)

As it turns out that wasn't quite good enough. The TGT we issue for FIDO targets your on-prem servers, and doesn't contain things like a PAC, and is intended to be exchanged for an on-prem TGT the minute it's received so it's just...weird.

Alright, what if we give you a separate TGT? A Cloud TGT, if you will. I've logged into Windows. It's fired off a request to AAD to get my PRT, and with my PRT comes back this FIDO TGT (optionally) as well as a Cloud TGT.

And so it gets interesting.

We have some curious problems to contend with now. A user belongs to a single realm. When you log in you get a single primary TGT, and that more or less dictates who you contact to get tickets. Well, my realm is on-prem. The cloud TGT can't trample that lest we break the world.

So I now belong to two realms: on-prem and the meta-realm KERBEROS.MICROSOFTONLINE.COM. Everyone belongs to it. To clarify up front: this is not an isolation thing, it's just a global name. There's still tenant isolation.

This meta-realm is conceptually simple: when you want to get a Kerberos ticket to a cloud resource you ask the KERBEROS.MICROSOFTONLINE.COM realm. Easy.

Okay, I've logged in, gotten my PRT, gotten my cloud TGT, and now AVD asks FSLogix to load my profile from \\mystuff.file.core.windows.net. FSLogix doesn't know any better so it asks Windows, Windows asks SMB, SMB says "heeeeeey I need a ticket?" Good old SSO. How Windows Single Sign-On Works (syfuhs.net)

Now the Kerberos stack has a request for cifs/mystuff.file.core.windows.net, how does is it know what realm to use, what TGT to use? It's so confusing. It turns out to be quite simple: during that PRT thing at logon we return a mapping of name suffixes to realms.

What does that mean? An example:

*.windows.net => KERBEROS.MICROSOFTONLINE.COM

If the SPN ends with the value on the left, then that means you should use the TGT from the realm on the right.

Okay, so the Kerberos stack now knows cifs/myfiles.core.windows.net belongs to the cloud KERBEROS.MICROSOFTONLINE.COM realm. How do you get a ticket from this nebulously named thing?

Back during logon we contacted AAD and did the PRT thing. In response we got

The PRT
Cloud TGT
FIDO TGT (optionally)
Realm mapping
AAD tenant details

Aha, here we go. Tenant details.

The bit of Windows that did the AAD authentication knows enough about Kerberos to say "It's dangerous to go alone! Take this!", bundles up the TGT, mapping, and tenant info and hands it off to the Kerberos stack.

The Kerberos stack is enlightened enough to see that if it gets this bundle of stuff that it should do something special with it. What does it do?

Stuff the TGT into the cache
Cache the realm mapping
Add a KDC Proxy map between (2) and the endpoint in the tenant details

Oh hey, that's how we magically transit Kerberos over the internet! KDC Proxy. Or more specifically the [MS-KKDCP] protocol. Neeeeeeat.

Okay, back to the ticket request. Kerberos sees cifs/mystuff.file.core.windows.net, sees *.windows.net maps to KERBEROS.MICROSOFTONLINE.COM realm and sees a KDC proxy mapping of that realm to https://login.microsoftonline.com/tenantid/kerberos.

Hmm.

Incidentally you can find that endpoint in the .well-known endpoints list: https://login.microsoftonline.com/common/.well-known/openid-configuration (replace common with your tenant ID).

So we fire off a TGS request over the KDC Proxy protocol to AAD and now it's real good old-fashioned Kerberos and we're cooking with fire.

It turns out this is the easy part. AAD receives the TGS-REQ, verifies the cloud TGT matches the tenant in the endpoint, looks up the user, looks up the requested SPN, generates a ticket, encrypts it to the service principal key, bundles it all up in a TGS-REP and returns it.

Oh right, the service principal. It's just an AD service principal. It has keys. You configured it when you set it up. Azure Files has its' storage keys, those keys are synced with AAD, and when you generate a ticket, it gets encrypted to those keys.

Anyway, the Kerberos stack receives the TGS-REP, strips out the ticket, generates an AP-REQ, hands it back to SMB, SMB stuffs it into a header, sends the SMB hello, Azure Files receives the hello, decrypts the ticket, and hey look at that, an identity.

SMB didn't know any better. Azure Files mostly didn't know any better. FSLogix definitely didn't know any better, and now here you are, logged into AVD with a centralized and roaming profile secured without Active Directory.

Whoa.

On Computer Passwords

site@syfuhs.net — Thu, 23 Sep 2021 21:59:00 GMT

In the spirit of people being wrong on the internet, I wanted to briefly discuss something people rarely get right: the computer account password.

Your computer has an account in Active Directory and that account has a password. This account is what makes Kerberos work. When you want to communicate with this computer you encrypt tickets to the account password.

When your computer wants to act as itself on the network (instead of, say, you) it uses it's password to authenticate to the domain controller. Things running as SYSTEM or NETWORK SERVICE operate as the computer account on the network.

That's why you see COMPUTERNAME$ in DC event logs. Because it's literally the computer account authenticating to-and-fro.

But the thing people get wrong repeatedly is how this account password is managed.

Well, it turns out it's deceptively simple. You have this service on your computer called netlogon. Within this netlogon service is a timer. Every few days it polls a domain controller to find out when the password expires, and if it's nearing expiration it changes it.

See, netlogon is the service that makes function calls to Active Directory on behalf of your computer. There are a handful of ways you communicate with a DC: Kerberos, NTLM, LDAP, and RPC. Netlogon is the RPC bit, and this has historically been called the Secure Channel.

Incidentally not to be confused with SChannel which is the Windows TLS/SSL provider. They are not related.

So Windows uses netlogon to rotate it's computer account password.

The process is fairly simple.

Timer wakes up, connects to a DC, see's its been 30 days
Netlogon generates a password locally, stashes it, authenticates to the DC with the old password
Calls ChangePassword(newPassword)
DC acks
NL Replaces old pwd with stashed pwd

But what happens if that fails? The DC isn't available or the change call doesn't succeed?

Nothing. Notta. Zilch. Not a thing. A big whopping zero.

See, *computer* password changes are optimistic. Until you get a confirmation you must assume it's the old password.

The reason you must assume it's the old one is because of continuity of service. If the computer can't change it's password because it's been WFH for 6 months what happens when you're back in the office? Well, you've now got 10k very annoyed computers to fix manually.

This is incidentally the source of many fantastically incorrect takes on how Active Directory works.

Take: expired computer passwords cause the trust to be broken.

Fact: nope. Computer passwords have an exception to the lockout policy. They just keep on truckin with the old password.

Take: expired passwords cause computer accounts to expire.

Fact: nope. Just, nope.

Take: computer accounts that haven't logged on in so many months are expired.

Fact: nope.

Take: computer accounts that haven't logged on in so many months are deleted from the directory.

Fact: nope. This isn't a feature in AD. Domain admins like to run scripts that periodically purge such computers and *that* is the source of many problems.

For all intents and purposes computer accounts don't break on their own nor do they break because of any built-in timers in Windows. That said...

Computer accounts do break. This is very clearly seen when you find you can't log into your machine anymore with some obscure workstation trust error.

There's fundamentally only a handful of reasons this happens.

The computer lost the password
The computer account was deleted

(2) is pretty obvious. An admin or some third party process removed it. Happens all the time. DELETE * WHERE LASTLOGGEDON < 3mo

But (1) is more problematic because it's insidious -- Windows as a rule doesn't just lose things.

This can occur when system secrets get corrupted. Machines are not infallible. Drives fail, processes aren't transactional, bugs go boo, etc.

Thankfully the fix is usually super easy: Test-ComputerSecureChannel -Repair

But that generally shouldn't be required. Windows doesn't especially care that the password hasn't changed in forever. It'll happily rotate it the next time it has line of sight, but until then it'll just keep on truckin like nothing is wrong. Because nothing is wrong.

A Bit About the Local Security Authority

site@syfuhs.net — Thu, 05 Aug 2021 22:08:00 GMT

It's been a while since our last thread and I need to kill time while a ginormous time travel trace file finishes copying, so let's talk a bit about LSA, the Windows Local Security Authority.

The LSA is a system component that oversees all the security decisions on your machine. Whenever Windows needs to authenticate a user or verify permissions or what not, the system asks the LSA. It primarily lives in a user mode service: LSASS.

The LSA is a bit of an interesting beast in that it's not one singular thing. It's made up of a handful of components, mostly living in user mode, with one teeny-tiny* component living in the kernel.

*not so tiny

Before we dig too much into the LSA itself, we need to take a brief look at how Windows security works. There are two fundamental security boundaries in Windows we need to look at: the kernel/usermode boundary, and the user/user boundary.

The user/user boundary is fairly straightforward. Two separate standard users can't interfere with one another on the same machine. This is enforced through authorization rules bound to identifying information about each user: namely their SIDs.

A SID is just a unique identifier. No two things will have the same SID. So user A and user B have separate SIDs, and everything belonging to each user is stamped somehow with that SID. The authorization rules in Windows say if you don't have this SID then you can't touch.

This is fundamentally how all security decisions in Windows are made. Does the user have a SID that matches the list of SIDs that are required to do the thing? Yes, then carry on. No, screw off. Of course, there are two extensions to this, the first being groups.

The groups you're a member of are an extension of your identity, so as a result you have your SID plus all the SIDs of the groups you're a member of. The second is Windows privileges.

A Windows privilege is a right to interact with Windows itself. This includes benign things like turning the computer off or critical things like running system services. Privileges are bound to SIDs (ish). Each privilege has a list of SIDs that are allowed to hold the privilege.

So when I want to turn the computer off Windows checks for the SeShutdownPrivilege, which is granted to the user by virtue of having specific SIDs.

Eh, sort of. These privileges are actually stamped on a user token.

These user tokens are managed by the LSA and are created when a user logs on. Every single user has a token, and every operation in Windows is bound to a token because all processes have a token mapped to them: How Windows Single Sign-On Works (syfuhs.net)

These user tokens, often called NT Tokens, are the way Windows represents user identities when logged in. The token is a kernel data structure. What this means is only things in the kernel can touch this data. Which brings us to the kernel mode/usermode boundary.

The kernel/user boundary is a hardware-enforced boundary. Your CPU dictates whether your code runs in kernel or in user mode. To call things in kernel mode you have to ask the CPU nicely and it may or may not acquiesce because it has a known list of functions that can be called.

This list is highly specific and dictates whether your API can ask the CPU to execute a function in the context of the kernel. This is how for instance your program can read files off disk. The program asks the CPU if it can execute a kernel call to read IO.

Anyway, in practice what this means is all the stuff in the kernel like your NT Token can't be touched by stuff in user mode without going through these special functions, and it turns out there isn't a specific function for managing tokens. How's this all work?

Well, let's start at the beginning. Remember how I said all processes have tokens, and all tokens are managed by LSA, and LSA is a user mode process? Well that sounds a bit...impossible? It turns out the system cheats a bit.

When the computer turns on the first (ish) thing that executes is the kernel executive. It is "the" kernel process. It's the thing that holds all the system goodies. It's the "in the beginning" in the biblical sense. And it's a no good cheater.

The executive kicks off user mode, which starts with a single process: wininit.exe. It needs to be running as an identity so the executive cheats and makes the very first SYSTEM token. Wininit knows this and that there's no security system yet, so it starts lsass.exe.

Incidentally the kernel also knows there's no security system in place, so it halts all further security decisions until LSA starts up. If LSA fails to start the system fails to start.

So LSASS. That was a long fricken preamble. LSASS is kind of dumb. It only has one job, which is to act as a service host. That service that gets hosted is The LSA. The LSA is *also* a service host, whose job is to host additional services. Oy.

So, that LSA service. It's called the LSA server and is backed by lsasrv.dll. If you go look at the files you'll see this is an absolutely massive DLL compared to lsass.exe. That's where all the meat is.

The server has two jobs, one of which is to host an RPC server (shocked face), and to manage additional child services. These child services are critical, but less interesting. They are things like netlogon, the key isolation service, or LDAP or KDC on your domain controller.

But the RPC server on the other hand is super interesting. Whenever your application makes a security call, like LogonUser, or something like SSPI's InitializeSecurityContext, your process must call this RPC server.

Your application doesn't get to touch LSA directly. It goes through some stub calls like LogonUser, which are RPC clients, which communicate to LSA, and then LSA does all the magic.

That magic is also kinda interesting, because LSA (server) doesn't strictly know what the magic is. LSA (server) knows how to communicate with clients, but it has to hand the actual work off to plugins called authentication providers. Extensibility is fun.

So to recap you have LSASS, which hosts services, specifically the LSA Server service. The LSA Server service hosts the RPC server, and other complementary services, and the RPC server (kinda sorta) hosts authentication providers. Wheeeee.

These APs are where the magic happens. There are a handful of builtin APs like Kerberos and NTLM and a very special one called Negotiate, which knows how to...negotiate...between Kerb and NTLM and whatever else is registered.

I won't go into the logon process, but here it is: What Happens When you Type Your Password into Windows? (syfuhs.net)

Somewhere along the way the logon process needs to create an NT token for the user, so everything running as that user has an identity in the system.

APs now start to cross the streams with LSA. One other thing LSA knows how to do is communicate with the kernel. Aha! Remember the kernel waiting on the security system to start up? This is why. LSA needs to be able to call into the kernel to ask the kernel to create tokens.

So the AP asks LSA to create a token. LSA calls into the kernel via RPC (ALPC specifically) and the kernel says "okie-doke here you go".

But this is RPC. Can't anyone call RPC? Actually yes. But it turns out the kernel-side RPC server checks the caller's token to make sure it's SYSTEM. Funny how that works. /fin

MFA is Hard to do Right

site@syfuhs.net — Fri, 07 May 2021 21:38:00 GMT

There was an internal discussion going on about folks getting too many MFA prompts when they're using RDP. I've danced around this topic quite a bit in past threads, but maybe let's take a look at it: how MFA works in Windows.

In Windows-land MFA comes in all sorts of flavors. Often what folks see is just a second field on the Windows logon screen to enter a one time password or send a request to their phone to approve before being taken to your desktop.

This is not MFA. I mean, it is MFA in the sense that you're providing two factors, but in fact it's not MFA because the authority you're communicating with only ever sees a password. This is a critical distinction.

See, in this scenario you're unlocking your device using the second factor. That's all well and good, but the majority of the things you're doing on your machine actually require communication with other services...not on your machine.

How were you authenticated to those services? If it's an internal service then that means Kerberos or NTLM. The only way you can authenticate to AD for either of those protocols is through password or certificate.

If it's an external service protected by AAD then you're using OAuth. AAD OAuth understands many credential types. Which one did we use?

Well, the logon screen asked for your password and you hit a button on your phone to approve the logon so chances are quite good that you're authenticating to AD or AAD with...just a password.

But, the thing is, Windows doesn't really care how you authenticated. I mean, it does, but the internal mechanics aren't doing authorization checks, yet.

What it boils down to is the authority that Windows trusts just blessed your logon, so you're now at the desktop doing stuff. Windows *then* limits access to things on your machine through authz rules that are driven by the contents of the ticket returned by the authority.

Put in more technical terms the ticket from AD contains a list of SIDs. Things in Windows are ACL'ed against SIDs, so when checking if you have access to stuff, Windows checks your SID list and gives a yay/nay. Other machines do the exact same thing when you connect to them.

These SIDs that are put into the ticket are derived from your group membership as well as derived from the credential you used to authenticate yourself. Password is the baseline and gives you nothing special, and there's no way to tell AD that you did password + superduperMFA.

Therefore in order to do MFA in Windows *and* have it represented to other services, you need to use a method that your authority understands.

Active Directory understands exactly one(-ish) other factor: certificates.

This means you can authenticate using smart cards or using Windows Hello.

AAD understands more. It understands certificates, FIDO, and the mobile authenticator. This means you can [sorta] use smart cards (sorta), Windows Hello, FIDO key, or push authentication to get the MFA blessing. Windows and AAD together understand just a subset of that though.

So back to the internal thread. Folks are working from home and need to RDP to their work machine from their home machine. The problem is that they're getting a bunch of MFA prompts on the remote machine whenever they do anything. What gives? They're using MFA, right?

The problem was that MFA was being enforced outside of Windows, before RDP kicked off so the machine they were remoting in to only ever saw a password.

But Steve, why weren't they using Windows Hello?? Well, because it was their home machine. You can't get an organization-stamped Windows Hello credential put onto your home machine. It just ain't working that way.

So they connect to a service that prompts for MFA and gives you an RD Gateway that can forward your request to the internal machine. To be clear: this is a secure process. To get to the gateway you need to MFA, there's no two ways about it.

But now that you've passed the MFA check you can securely use a password to RDP. Windows only ever sees the password, so AD never stamps your ticket with the certificate or Windows Hello SID (technically there is no SID for either, I'm glossing over part of this machinery).

It's a hybrid joined device so it reaches out to AAD to get a PRT. This process also sees its just a password so it doesn't stamp the PRT with the MFA claim. Then you try and access a resource that enforces MFA, so AAD checks the PRT for the MFA claim and whoops no claim. Prompt.

Ten minutes later you access another resource that enforces MFA, so AAD checks the PRT for the claim and whoops still no claim. Prompt.

Repeat ad infinitum.

This is why remote access is super difficult to get right, where you're balancing usability with security. Everything in the chain needs evidence the previous thing was authenticated to with MFA, otherwise everything further in the chain will prompt for it.

Obviously the answer then is to make Windows understand all the things. Well, which things? Which protocols? Which standards? Which vendor extensions to protocols?

Well, and then I guess make RDP understand them too. And all the RDP clients on Mac and Linux, and oh we have this HTML-based version too.

It's hard work.

So the goal then is to converge on only a couple standards and well known implementations and just try and get everyone to play nicely together. It's the only rational approach.

All of this adds up to the question: how do you do remote desktop securely then?

You need MFA, full stop. Protecting the gateway with MFA is a must. You need to also enforce MFA (or some derivative policy) on all the services you use.

At that point you need to ask if being inundated with MFA prompts is a great user experience or if its leads to prompt blindness?

You can solve that by transferring the MFA state from the client to the target machine by using a strong credential like smart card or Windows Hello.

Windows Hello is amazing technology that folks love to use and works beautifully on work machines connecting to work machines. But it's not suitable when you're on your home machine. Smart cards work great though.

FIDO is an amazing technology and it solves so many problems, but its not quite ready for RDP yet. We'll get there.

Anyway, remote access is hard. I have no other closing thought. Good luck.

Killing NTLM is Hard

site@syfuhs.net — Wed, 05 May 2021 18:49:00 GMT

EDIT 2: Oh hey, we announced our strategy.

EDIT: Good news. Deprecating NTLM is Easy and Other Lies We Tell Ourselves (syfuhs.net)

So I joked earlier today that the reason we can't kill NTLM is because folks turn off telemetry. That's false.

Mostly.

Here's why.

— Steve Syfuhs (@SteveSyfuhs) May 5, 2021

There's one sure-fire way to kill NTLM: switch to Kerberos or similar modern authentication protocol. Alright, done and done -- what more is there? Well, like all things its never that easy. If it were that easy we'd have solved this 15 years ago.

That's because NTLM is both a blessing and a burden. It has some properties that modern protocols don't. Namely that it doesn't require line of sight to a domain controller and that it doesn't enforce server authentication. Can you guess which bit is great and which is terrible?

Line of sight is maybe relatively straightforward to understand. In Kerberos world the client (you) need to speak to a domain controller to get a ticket before you're allowed to access a resource. This sucks majorly when you're outside the boundaries of your network.

Therefore that means you need to set up a VPN or KDC Proxy or some such thing. But that's a chicken and egg problem: to connect to VPN you need to authenticate, to authenticate you need to connect to the DC, to connect to the DC you need...line of sight. Doh!

NTLM doesn't have that problem. You authenticate to the service in question and that service fires the request to the DC. The service is usually in a fixed location so enforcing line of sight there is usually quite easy.

This is why RDP with NLA works from outside the network. You're not doing Kerberos anymore, instead you're doing NTLM, where the client is firing the NTLM request to the target directly, and the target is forwarding it off to the DC.

Fundamentally NTLM gets the same information as if you're doing Kerberos. Kerberos just bundles everything up ahead of time, where NTLM needs to go off and get it every time.

So what's the big deal?

Property 2: NTLM doesn't do server auth.

Server auth is fundamentally just the idea that you can [somehow] guarantee the server you're communicating with is in fact the server you want to communicate with.

As authentication protocols go, not supporting server auth sucks majorly.

The reason for that is because you just fired a credential-thingy to a server that says its the server you want to talk to, but you can't really verify that at all. What's to stop that server from forwarding it off to the real server and pretending to do evil things as you?

This has made the news recently as a new form of attack. It's not particularly new: NTLM has never supported server auth. The flaw here is really just that things are using NTLM when they shouldn't.

And why are things using NTLM when they shouldn't? Well, for the two properties I just described.

1. If a client doesn't have line of sight to a DC it can't do Kerberos, so it falls back to NTLM.

2. Server auth is forcing a downgrade to NTLM.

Wait, isn't that bad? Why would a server downgrade?

Well, it's not the server doing it. It's the client or DC doing it, because they can't guarantee server auth.

See, the guarantee of server auth is by virtue of a trusted third party saying 'yeeeeep, I vouch that server has the same name as the one you requested'. Kerberos does this with SPNs.

When you ask for a ticket to a service you use the SPN to identify it. The DC looks up the SPN and gives you a ticket to it. If that ticket can be decrypted by the server, then the DC has proven the server is who they say they are. Kerberos Explained in a Little Too Much Detail (syfuhs.net)

TLS does this through certificates. The server presents a certificate containing domain names. If the domain name you're contacting the server by is in the list, then you can reasonably say the CA that issued to certificate guarantees that server is who they say they are.

So what happens if the SPN isn't found, or the domain name isn't in the certificate? Well, server authentication CANNOT succeed.

In the TLS case this is catastrophic. It's like always clicking the "yes I know this is dangerous and stupid" button in your browser when you get a cert warning. It means you have no root of trust, so there are zero guarantees of privacy of information passed in the TLS channel.

NTLM can be similarly catastrophic, but in different ways. NTLM, much like Kerberos, can be used to authenticate clients, or to optionally provide a secure channel for communications similar to TLS.

The way this works is basically that either protocol will produce a session key that both the client and server know, that can be used for encrypting traffic between the two. This is how protocols like RPC or SMB work.

Other protocols like HTTP don't need NTLM or Kerberos to provide a secure channel because HTTP can rely on TLS to do that. So NTLM is only used for authenticating the user.

So therefore in the NTLM via HTTP over TLS case, you have some measure of server authentication through TLS. Not quite the end of the world.

So if Kerberos can't happen for whatever reason, then the client will fall back to NTLM.

This is the crux of the problem. If server auth fails then you must fall back to a protocol that doesn't do server auth. You can replace the protocol with something that does do server auth, but suddenly you're back to the original problem: server auth *already* failed.

This is why killing NTLM is so hard. We can replace the protocol with something better, but that something is starting it's life right out the gate fighting with basic fundamentals.

This is also why the approach to date has been more about remediating scenarios that are falling back to NTLM vs replacing NTLM outright: replacing it outright doesn't necessarily buy us anything.

Alternatively we can just turn it off. That actually just makes things worse.

Obviously security is important, but continuity of business is important-er. Turning off business critical services is a dangerous game.

So the path forward is through information gathering. You need to know what's doing NTLM, and you need to know why it's doing it. There's an audit policy for that: Network security Restrict NTLM Audit incoming NTLM traffic (Windows 10) - Windows security | Microsoft Docs

Which brings us to the original joke: we can't deprecate NTLM because folks don't turn on telemetry.

That's false. We have ways of doing it, but it doesn't necessarily buy us much.

What telemetry does tell us though is *when* we can turn it off.

Lessons in Disabling RC4 in Active Directory

site@syfuhs.net — Tue, 02 Mar 2021 00:33:00 GMT

Was pulled in to a fun customer issue last Friday around disabling RC4 in Active Directory. What happened was, as you can imagine, not good: RC4 was disabled and half their environment promptly started having a Very Bad Day.

— Steve Syfuhs (@SteveSyfuhs) March 1, 2021

RC4 is a stream cipher. A stream cipher is kinda sorta like a one time pad (note: kinda, and sorta). A one-time pad is a cryptographic operation that takes one value and XORs it against a random value. A^B = C. A is your data, B is random noise. C is your encrypted cipher.

They're incredibly useful because the XOR operation is only reversible when you know A or B, and if B is suitably random that means you have to guess for all combinations of B. In other words you have to brute force it. As far as cryptography goes that's nearly perfection.

However, the trick with one-time pads is that you need as many random bits as you have data. If you have 10 data you need 10 random. If you 10k data you need 10k random. You cannot repeat the random, lest you introduce a pattern and code breakers just love patterns.

So a stream cipher could take a one-time pad and cut the key down to a fixed length, manipulating the key every operation. Let's say you have 100 data and 10 random. The first 10 data get XOR'ed to the 10 random, then the 10 random get XOR'ed to something else. Repeat 10 times.

This turns out to be incredibly simple to code and is incredibly fast relative to other crypto algorithms. However, the cost is that you're now doing key scheduling which means if you can predict the schedule you've broken the cipher.

RC4 fits the bill here. It's painfully simple to implement. Here it is in entirety. But it's also irreparably broken.

The thing with RC4 is that if you have enough data transformed by a single key you can eventually predict what the original plaintext is. This became a semi-practical attack in 2013 when some smart folks figured out how to apply this to TLS. https://isg.rhul.ac.uk/tls/

For this attack to happen it requires observing billions of bytes of data. This is done easily enough with TLS, hence why folks jumped at disabling RC4 cipher suites. TLS isn't the only place RC4 is used, and RC4 is still broken, so it's just good form to disable it everywhere.

So now we have Active Directory and RC4 is enabled by default. In 2021?! How dare. Weeeeeelllll, RC4 isn't quite that bad in this case. Like, it's bad, but not super bad because each key will only ever be used for encrypting small amounts of data.

The current attacks require observing *lots* of data encrypted with a single RC4 key. For Kerberos we're talking maybe 8-16kb at most, not the gigabytes required for the attack. So it's bad, but not end of the world bad.

But that's no excuse. Why is it still there? Well, it turns out the RC4 cipher suite has a unique property: it doesn't require a salt when doing key agreement.

Huh?

So Kerberos has two legs between a client and KDC. AS and TGS.

AS is how you authenticate yourself: here's password gimme krbtgt.

TGS is: here's krbtgt gimme service ticket.

Now we don't ever just send the password to the server. What we do is we encrypt a thing using the password as a key and fire that thing over to the server. The server also has your password and can decrypt. If it decrypts then we agree we both know the password.

But Active Directory doesn't store the password itself. It stores a key derived from the password. That is, take the password and hash it, and store that hashed value. You encrypt against this hashed value.

So lets go back in time, circa mid 90's when Active Directory was being built. Back then, in the real world, Windows authentication was NTLM. NTLM kinda sorta worked similarly where you derived a key from a password and used that key to sign some stuff.

That derivation was and is admittedly very lame. It's md4(password). So in NT Server in the directory database was username + md4(password), and that's it.

And taking the cryptographic properties of NTLM out the picture for a moment, it still kinda sucked as an authentication protocol. It didn't offer server authentication and it wasn't easily extendable for future changes. So Active Directory switched to Kerberos.

But we wanted upgrades to be seamless. When you left work Friday afternoon logging in to NT and came back Monday morning we wanted you to be able to log in to AD without any fuss, despite the fact that we changed literally everything out from under you.

And Kerberos at the time was using DES (eww). This posed a problem. You can't transform an MD4 key into a DES key. The protocol certainly didn't let you do weird things like DES(MD4(password)), so Windows created the MD4+RC4 crypto system.

The magic of this is that the NTLM key and Kerberos key are interchangeable so no password changes were required. Over time as users changed their passwords Active Directory would derive these newer, stronger, keys and would inform clients to prioritize these stronger keys.

This turns out to be phenomenally powerful because it transparently migrates users to stronger keys without breaking anyone, at a small cost of delaying weeks or months as password changes occur. Is it perfect? No. Does it keep the masses happy? Yes.

But there's another small problem: salts. Salts add randomness to passwords, making their hashed form incomparable to other hashes. hash('password', salt1) <> hash('password', salt2). We do this for all sorts of reasons, but the primary is so it's harder to guess the password.

But salts complicate the protocol. Both client and KDC need to take the password and salt and derive the same key, so both need to know them. Seems simple enough? Just use a well known value...?

Ah no, they need to be unique, otherwise two identical passwords with the same salt will form the same hash. Oops. Okay, compute a salt from the username. Aha, here we go, this works. It's unique and both parties know it. And it doesn't change! Oh wait, it can.

Users change their usernames all the time because names change all the time. If the salt were just the username that means we'd have to recompute the hash in AD every time the username changed, which means we'd need to know the password and only admins change usernames so 🤔.

Okay, so what if the KDC only changes the salt on password change and just tells us what it is whenever we authenticate? And so this is how Kerberos works. The client "pings" (no not ICMP) the KDC and the KDC says here ya go.

And so long story short (hahahaha, oh.) we come to the customer issue. RC4 doesn't require a salt. AES requires a salt. You can see in the previous screenshot the ARCFOUR instance doesn't provide a salt because there just isn't one.

If you use RC4 you don't need to ping the KDC. You can just derive the key from the password and go. Therefore if you disable RC4 conversely you must always know the salt.

This poses a small challenge. Sometimes you don't want the system to know the password itself, so you provide these things called keytabs. They're files with some structured data that map users to derived keys. You derive the keys upfront once so you only need the password once.

Sometimes these files are generated without having line of sight to a KDC too so the tool generating the file has to guess which salt to use. Sometimes the tool guesses the wrong salt. Oops.

So you generate these files and deploy the thing and the system chugs along just fine until you disable RC4 and the whole things comes crashing down. The system was using RC4, and RC4 worked because there wasn't a salt. When it switched to AES it required the salt and...oops.

Well that's a pretty lame experience for anyone, let me tell you. Why did the tool guess the wrong salt? I've already told you: because the name changed.

Active Directory lets you change your username. You actually have two different usernames. Your sAMAccountName and your UPN. You can use either to log in with Kerberos, but Active Directory will only derive your salt from a single value: the sAMAccountName + Realm.

Well it happens that their UPN is in the form samaaccountname@customrealm.com, but their realm is anotherrealmentirely[.]com. As such their salt was ANOTHERREALMENTIRELY.COMsamaccountname. This happens when you add custom UPN suffixes.

The password never changed so the salt never had a chance to change (not that it would). And the tool to generate the keytab just derived the salt from the user principal name, which in most cases is fine, but in this case wasn't.

And so you might think to yourself that this isn't that big a deal you haven't ever changed the usernames or the suffixes or anything like that. It turns out there's exactly one scenario that every environment hits: when you promote the first DC in a forest.

When you create a forest the domain admin is the machine's local administrator account. I'm not talking implicitly, I'm talking the local admin is copied into the AD database including it's password. The password that was set when you built the machine, before it had a realm.

And the domain admin is part of the protected users group so RC4 is already disabled for them. If the admin is a member of the Protected Users group, that means RC4 is disabled for them, and that then means they MUST use AES, which means they MUST use a salt, which means the salt had to exist when the machine was first built. Oy.

So the realm is the computer name.

Anyway. Here's some useful links as you consider getting rid of RC4.

Tough Questions Answered: Can I disable RC4 Etype for Kerberos on Windows 10 ? (microsoft.com)

Decrypting the Selection of Supported Kerberos Encryption Types - Microsoft Tech Community

Protecting Against Credential Theft in Windows

site@syfuhs.net — Fri, 12 Feb 2021 22:43:00 GMT

Have you ever thought to yourself "boy, I sure wish I had to use passwords more often"? No, of course not. Passwords suck. Good passwords are long and hard to remember and easy to remember passwords aren't good. They're inherently portable and easy to steal.

— Steve Syfuhs (@SteveSyfuhs) February 12, 2021

We've known this a great many years and platforms like Active Directory and Windows have supported better options like smart cards for 20 years because of this. So what is it about passwords that make them so hard to protect?

First of all, we need them to be random. Random means people can't guess them. Suppose I pick a simple 5 letter password from the lowercase alphabet. That's 26 letters and 5 positions, or 26^5, or ~11.8 million combinations.

As a human it'll take you forever to guess it, but it'll take a computer less than a second to generate all 11.8 million combinations. But I'm also human so I'm going to pick something easy for me to remember: "riley". That's super easy to guess if you've looked me up.

So we start to enforce strength requirements: must be 8 characters with uppercase, lowercase, numbers, symbols, one reoccurring character from season 6 of your favorite TV show. This gives you (say) 92 characters and 8 positions or 92^8 or 5132 trillion combinations.

So my password is now aSdX3$CC;^k@. I can barely remember what I had for lunch yesterday, am I expected to remember this? Oh, and I have to change it every 90 days? Ugggggh.

But it turns out this is equally easy for a computer to guess anyway, so. 🤔

Of course, suppose you don't have $30k to drop on a password cracking rig and you don't have benefactors that will donate to the cause, brute-forcing your way through is out of the question. So you attack the next best thing: Windows.

To attack Windows you have to understand how authentication works in Windows. In simple terms you have a high value credential (password) which you exchange for a medium value credential (TGT/PRT), that is exchanged for low value credentials (service tickets/access tokens).

You can think of them as first order, which derives a second order, which then derives a third order credential, all with decreasing capabilities as you go further out. This is sort of the mantra for solutions like Credential Guard: protect the first and second order credentials.

That's all well and good however technologies like Credential Guard serve very specific and well defined purposes and aren't holistic catch-all solutions.

This is important: there's never a one-size-fits-all solution to security.

Securing complex systems like Windows, or your enterprise network, requires many layers. It's like the swiss cheese analogy for stopping the spread of COVID-19. Layer one stops something very specific. Layer two stops another thing, layer three... etc.

So Credential Guard protects your 1st and second order credentials at rest *once* they've entered the system. To understand why this matters it's important to go back to how credentials are processed by Windows.

Passwords enter your computer through credential providers, into LSA, and processed by AD or AAD. I've explained this in great detail in both text and video form. What Happens When you Type Your Password into Windows? (syfuhs.net)

Cred provs do their thing and pass it off to LSA.

LSA does a cached logon.

You're taken to your desktop.

And THEN off you go to Active Directory or AAD.

But this is all very hand-wavy. Along the way there are built-in extensibility points where third parties can add in their own code to make the login work however they need it to, because sometimes that's just how things be.

You can drop in your own credential provider that looks just like the password credprov. You can install a custom LSA authentication package that listens for passwords. Or you can install a subauth module and log the password.

But we have solutions for that. LSA Protected Process mode is one. It prevents anyone from loading third party code into LSA so they can't touch those extensibility points. But you can bypass LSAPPL -- so we have HVCI to enforce it. Lots of solutions here.

But there's more to this. Your password doesn't magically go from your keyboard to the credprov. It gets converted to electrical signals and sent along a wire into your computer to be processed by the CPU in a driver in the kernel and then converted back to text.

[page-break]

So now you have to worry about hardware keyloggers snooping the wire. Or kernel-mode keyloggers listening to the drivers. The latter is easier to handle: limit what kinds of drivers can be installed through something like Windows Defender Application Control. The former though?

Not a lot you can do to protect from hardware keyloggers. To truly protect this whole stack you need dedicated hardware that guarantees nothing is listening to the electrical signals. That's hard even with specialized hardware. It's impossible for general purpose systems.

All of this adds up to a particularly complicated mess that is hard to fit into your head and harder to defend. Surely there are better options here?

This is why we've been pushing passwordless. Sure, it's partly a marketing thing, but it's also a line in the sand. Passwords just genuinely suck. We can do better and we want to do better.

So we have smart cards. These were our first go at passwordless 20 years ago. They are more secure than passwords for a few good reasons...

First, they rely on asymmetric crypto (passwords = symmetric), which means you split the key into two separate pieces: a secret (private) key and a public key. You can share the public key with anyone, but you keep the private key to yourself.

To authenticate the user you ask them to perform an operation that only the holder of the private key can perform, which is then verified against the public portion.

This turns out to be an incredibly useful property. With smart cards you can store the private key in a tiny integrated circuit embedded into a piece of plastic. When you need to authenticate you plug it in to a reader which lets the computer ask the IC to perform operations.

This makes the key portable and also prevents you from copying the key off the IC, since the IC is hardened to protect against that sort of thing (in theory).

There's an obvious problem here: anyone can steal the card and use it. So we introduce PINs that unlock the card. You type a PIN into your computer, which is sent to the card and if it's correct that tells the card that it's safe to perform the requested operation.

So smart cards for everyone!! Except, they didn't really take off. Why? Well, a few reasons.

They require dedicated readers.
They're expensive to manufacture.
They're easy to lose.
They're easy to break.
The standards bodies were...insane.

Over time these issues were overcome. We got better form factors, the manufacturing costs went down, the standards bodies mostly came to their senses.

Alas, the standards still kind of aren't great and making changes to them is hard to do.

But there's also another more insidious problem with them in that we're back to typing secrets (your PIN) into a keyboard that translates into electrical signals across a wire into the kernel and

Always evolving we built Windows Hello. The simplest way to describe Windows Hello is you take a smart card and embed it into your computer. We store you private key on your hard drive, encrypted to a key derived from your TPM and something else.

When you want to use this private key to perform an operation, say to log in, you take that "something else", combine it with the TPM and out comes your private key. That "something else" could be a PIN you type in, or a biometric template from your face or fingerprint.

This solves so many fundamental problems with smart cards.

Nothing to physically lose.
Nothing to break.
Doesn't require a dedicated reader (ish).
Manufacturing costs are low(er).
Infinitely easier to use.

But there are tradeoffs. The credential isn't portable anymore. This is a positive and a negative. On the one hand you can't steal the private key and use it elsewhere. On the other hand you have to register yourself on every machine you use.

The cryptographic operations occur in the OS. Therefore you can snoop on the "something else" path and collect the PIN or parts of the biometric template. In a theoretical sense that's potentially very bad. Way way way waaaaay harder to attack than passwords, but still.

This was solved through some serious collaboration between hardware vendors and Windows folks and culminated in something called Windows Hello Enhanced Sign-in Security. Windows Hello Enhanced Sign-in Security | Microsoft Docs

The gist of this is relatively simple: move all the important security (crypto) operations that occur in the normal world OS into VSM. Easy, right? Well no. This was a hard problem to solve. 😂

But now it's incredibly difficult to snoop any data between the hardware sensors through the kernel to VSM and back into LSA. There are no extensibility points that let you touch the raw creds. Now that first order high value cred is seriously protected.

Combine it with Credential Guard and now all you're left with is low value credentials. Not bad.

But we're back to the problem of portability. You need to register the credentials on each machine you use. To register credentials you need another high value credential AND you need MFA.

Did I mention that smart cards and Windows Hello are MFA? They are. They're something you have (card or PC) and something you know or are (PIN or face/fingerprint). You're not going to get a super-high value credential out of just a password, so you need to go the extra step.

Some folks don't like to recognize Windows Hello as MFA. They're wrong, it is MFA. However, it may not fit their requirements *for* MFA creds because you require separation of the credentials from the machine. Sure, that's fine.

So portability is critical here. Which is why we have this thing called FIDO. It's an industry consortium and set of standards that allow different parties to agree on the form of credentials used as well as their relative security worthiness or credibility.

The net product is this security key, which is kinda sorta like a smart card in that is stores a private key, and you have to unlock it before you can ask it to perform operations against that key.

What makes it special and better than smart cards is the implementation. The protocol is simpler, the metadata is structured for specific scenarios (SCs are generic and require things called middleware to interpret functions), and you can store many identities on a single key.

But most importantly the ability to unlock the key has multiple predefined mechanisms. You can use a PIN typed into your computer, or you can use a key-specific mechanism like using a fingerprint reader built into the key. Getting that to work with smart cards is...non-standard.

Actually, another important thing is that these keys are rated for security levels and those levels are imprinted on the authentication. With a smart card all the party knows is a certificate was used. With FIDO you know a hardware key certified to a certain level was used.

AND ANOTHER THING... smart cards expose certificates, which can be used for any number things (a good thing for sure), whereas FIDO exposes specific authentication protocols. When I log into a website with a smart card I'm using TLS client cert auth. FIDO is higher in the stack.

Plugging into the TLS transport stack is expensive and complicated and potentially leads to uuuuugly vulnerabilities. Moving the authentication to the application layer is architecturally more sound. It's safer and easier to implement.

Dangit, and another thing! FIDO isn't just about external keys. It's about abstracting away authentication methods. With FIDO and webauthn you can log into a website with your security key or with Windows Hello. Windows Hello IS a FIDO and webauthn key.

I guess I went off on a tangent there. Anyway...

So you see why passwords are pretty lousy, and why we don't think they should be the predominant method of authenticating people. /fin

How Managed Service Accounts in Active Directory Work

site@syfuhs.net — Thu, 04 Feb 2021 21:20:00 GMT

Have you ever heard of these things called Managed Service Accounts? They allow you to run programs as an account that doesn't require a password while still having the security of a strong password. They're pretty neat. pic.twitter.com/p7nfDnyUqp

— Steve Syfuhs (@SteveSyfuhs) February 4, 2021

To be more precise, it's not that they don't have passwords, it's that they don't require you the administrator to know the password. The password is managed by Active Directory for you. That means not worrying about weak passwords or having to manually rotate them. Neat, right?

These are also just regular accounts in AD. They're special in that they're managed, but under the covers they're computer accounts, which inherit from user accounts. They walk and talk like principals in AD, so they have usernames, credentials, and service principal names.

This therefore means they can log in to Active Directory through Kerberos and NTLM. That makes them perfect for hosting services and tasks that require authenticating users or operating in authenticated sessions.

Which is great, but how do they work? How does Windows know how to use them and how can this be done securely since no one knows the password?

Well, it turns out it's partly super simple, and partly super complicated. The super simple part is that when you create a managed account you specify what other accounts are allowed to see the password.

The things in PrincipalsAllowedToRetrieveManagedPassword can be computers, users, or groups. These things are put into an ACL on an AD LDAP attribute of the (g)MSA that say if you present a SID in this list you can query this attribute.

Add-KdsRootKey -EffectiveTime ((get-date).addhours(-10))

New-ADServiceAccount gmsa1 -DNSHostName gmsa1.corp.identityintervention.com `
-PrincipalsAllowedToRetrieveManagedPassword djpp$ `
-KerberosEncryptionType RC4, AES128, AES256

$gmsa = Get-ADServiceAccount -Identity 'gmsa1'-Properties 'msDS-ManagedPassword'

($gmsa.'msDS-ManagedPassword'|ForEach-Object ToString X2) -join ' '

So when something needs to log in as that user it can query the LDAP attribute and get the password. Easy! ...ish. Well, not so easy. The value in this attribute is...complicated.

"But Steve" you say "the whole point of a managed service account is that you don't need to know the password. How does Windows know to get this password?"

That's a great question, wish I thought of it myself. The answer is actually a little silly.

Windows doesn't actually know it's dealing with a managed account. The administrator configured [whatever thing] to log on as an account, and left the password blank.

There's no rule that says ALL USERS MUST HAVE A PASSWORD. Windows allows users to not have passwords. Active Directory even lets you not have passwords (PSA: FOR THE LOVE OF ALL THINGS HOLY DON'T ALLOW THIS PLEASE).

So back to the question: how? Well, it turns out Windows just accepts that this might be a (g)MSA so during a logon call it opens a connection to AD and asks for the the password in the msDS-ManagedPassword attribute.

PS C:\Windows\system32> $gmsa = Get-ADServiceAccount -Identity 'gmsa1'-Properties 'msDS-ManagedPassword'
PS C:\Windows\system32>
PS C:\Windows\system32> ($gmsa.'msDS-ManagedPassword'|ForEach-Object ToString X2) -join ' '
01 00 00 00 22 01 00 00 10 00 00 00 12 01 1A 01 34 47 EB 28 EE C4 F4 2E 0D E9 83 3E F3 5C 05 58 EA E5 3E 7D C7 BB 3F F1 28 AA 49 84 51 46 DF 8A CF 33 34 83 10 7A 94 BC FB A2 11 27 8E 31 B7 70 C1 D7 7D CA 8D 33 6F 7D 07 33 7E 69 64 05 24 8F 57 2B A5 AA 74 4E 86 03 30 E1 A7 19 69 1B 76 F7 63 60 43 DA 7C 8D 56 46 99 8A F1 1F A7 F1 ED 2B E9 8B 8F 51 FD 17 16 42 19 5C 10 DA D8 53 4B D1 76 9E 88 0F 1F C4 8F 54 E8 DE 8F E0 A5 F0 82 13 CA 89 33 D9 0A 5E 19 6E FA BE CD 50 33 82 A1 CE 9E FF 09 92 06 96 75 7E A5 B2 E6 FB 8B 7A 9A 75 61 DC 06 C4 80 49 33 9C CA B7 C5 A9 BF 36 72 FF A6 7F 31 C8 E4 74 3D C6 B6 2A BE 8A DD A7 41 42 DB 9E EE C1 BB 0A 9F 0F A0 F3 C2 6A 0E B8 FD 23 A7 7D C5 10 F6 4E 59 B4 CE 35 49 47 17 C0 C9 FD 6F E2 F1 77 DF 0A E6 93 1D 84 F3 3F 45 6A AB 42 54 49 3B DD 9F 9B E8 0D 4E CC 57 80 3E 56 BE C4 00 00 EB 66 BA 80 28 14 00 00 EB 08 EA CD 27 14 00 00

Now you're looking at this kinda funny thinking waaaaaait a minute how does this bullsh*t work? Well remember, computers have machine accounts in AD. The computer logs in to AD, gets a Kerberos ticket to the DC asking for this attribute and does an ACL check.

Does principal djpp$ have a SID in their PAC that matches the ACL on the attribute? Yes? Here you go. It's plain old SSO. How Windows Single Sign-On Works (syfuhs.net)

So now the computer has the password (and a bunch of metadata from that attribute? Odd... 🤔). Windows takes that password and the username and...logs it on. Whoa. What Happens When you Type Your Password into Windows? (syfuhs.net)

P.S. another shameless plug: I did a video on how this works too. OPS108: Windows authentication internals in a hybrid world (syfuhs.net)

At this point the account has logged on and has all the usual accoutrement of a logon session. Nothing special. Any application running as that user has the usual SSO. Incoming requests can be authenticated using Kerberos (assuming someone registered an SPN on the attribute).

This is all well and good until the password expires. Wait what? Expires? Who expires it? When? What? Huh?

So the funny thing about managed account passwords is that they aren't real. They aren't

pwd = unicode(rand(254))

as many people might expect. They're derived secrets from a special root key.

In Active Directory you have this thing called a Root Key. It's the base (root) of all derived secrets in a forest. You take this root, append a bunch of localized metadata, and out pops a derived key.

This is incidentally how we can do things like group encryption with PFX files. Take the root key and append the group SID. Stand up a service in AD that accepts tickets, and if the requisite SID is in the PAC it'll provide the metadata to generate the same key and decrypt. Ish.

That isn't exactly how that works. How it actually works is complicated and makes anyone that isn't a cryptographer cry.

But back to managed accounts. The passwords are derived secrets computed from the root key and metadata. That metadata is basically accountKeyId + createdTime + interval. The interval and start time are critical. This allows the password to rotate regularly. Here's how it works.

Your root is "abc" with an interval of 30 days.

"abc" at 12:42 Feb 4 2021 = "aaa"
"abc" at 12:42 Mar 6 2021 = "bbb"
"abc" at 12:42 Apr 5 2021 = "ccc"

Following? This means at any given time AD knows what the password should be, will be, and was. Since the metadata is included in the blob, Windows also knows the interval. It doesn't have the root key so it can't derive it, but it can refresh it on that interval.

"But Steve" you say again "isn't that a bit overkill? It's just a password."

It is. But stuff breaks when the password is wrong.

There are some constants in life. One of them is that Active Directory is a multi-master replicated database, and that means life finds a way when replication fails.

So if we took out this derived key business, we could maybe make the PDC act as the password changer. It runs a scheduled task that goes through every managed account and rotates it regularly. Sure. Works well enough. Except then the PDC needs to replicate that out to everyone.

Another fact of life is that customers have thooooousands of DCs in their environments. Some are on cargo ships in the middle of the ocean and only check in every few hours. Some are in submarines that check in weekly.

If the PDC has to push out passwords and these DCs can't communicate, then these DCs are using an expired password. Breaks policy. Bad.

So option two is just give each DC the control to change the password. That's all great until you have two DCs that aren't talking to each other change the password. Eventually they'll sync up and latest wins, but until then you've got mismatched passwords and, well, screw that.

So when you add this all up you get a requirement that everyone must know the real password at all times.

Of course I left out one big huge thing about managed accounts! There's two kinds! Managed Service Accounts (MSA) and Group Managed Service Accounts (gMSA). What's the diff?

Originally MSAs were designed to be installed only on one computer at a time. What you'd do is register the MSA. This stamped the AD object such that only that one computer could get the password. If you tried it from another it would fail even if it had permission.

In theory this was great because you could very easily lock down who could see the password, and rotate it whenever the MSA was deployed to another machine. Until you decided to deploy a cluster. Oops.

A service gets deployed to 2 machines with a load balancer splitting traffic between them. The service requires Kerberos, so an SPN is registered on... the computer? No, there's two of them. Okay spin up a service account. Ooooh, managed don't need to know the password, great.

So you spin up an MSA and install it on box one and register the SPN and then go to install it on box two and... womp womp. Can't. Okay, spin up a SECOND account and add...the...SPN.........dammit! And now we're back to regular accounts and passwords.

So here we have *Group* Managed Service Accounts. The difference with gMSAs (I have no idea why we lowercase the G) is that they don't get installed on machines. You grant the machine(s) the right to read the password, and that's it.

Why don't you install it? Because installing it was intended to lock the MSA to the machine. Since it's group-based there's just no need to lock it.

These days you can use managed accounts for all sorts of things: running services, scheduled tasks, IIS app pools, etc. You generally have to be running as SYSTEM since you're granting rights to the computer account, but they're easy once you get the hang of them.

You can even try it out with PsExec:

psexec.exe -i -u corp\gmsa1$ -p ~ cmd.exe

So what are you waiting for? Go forth and deploy gMSAs.

OPS108: Windows authentication internals in a hybrid world

site@syfuhs.net — Tue, 02 Feb 2021 17:15:00 GMT

Have you ever wondered what happens when you type your password into Windows? With the cloud becoming a major part of our world, we find ourselves having to talk to both on-premises and cloud-native resources, which dramatically affects what happens when you do type your password into Windows.

Follow along as Steve Syfuhs gives a guided tour of how Windows handles logons internally and secures your authentication in a hybrid world.

This session includes:
02:11 Logging on to Windows
03:36 Types of logins
06:33 The Logon UI
09:39 Local Security Authority
21:53 Logon UI Part II
23:42 Local Security Authority Part II
25:14 Kerberos in Windows
35:35 Logon Sessions including Azure Active Directory
38:09 Local Security Authority Part III
43:53 Oauth in Windows - Types of credentials
45:55 Windows Hello Logon
53:34 FIDO Logon
56:32 Local Security Authority Part IV
1:01:08 Azure AD Join
1:05:14 Community Q&A - How long do we need to keep on-premises AD around?
1:09:39 How can we enable MFA/FIDO keys for normal AD Login and not only for Apps that support Modern Auth?
1:12:44 When will we get rid of passwords once and for all?

Community chat
Want to chat about this session? Come join us on Discord! https://aka.ms/ops108-chat

Identity Delegation in Active Directory

site@syfuhs.net — Tue, 26 Jan 2021 21:55:00 GMT

Have you ever had an app that authenticated users, and then thought wouldn't it be great if it could act as that user for the services it has to call later?

This has historically been called Impersonation, Delegation, Act As, or On Behalf Of depending on protocols in play.

— Steve Syfuhs (@SteveSyfuhs) January 26, 2021

Fundamentally the idea is simple. The user projects their identity to some remote application (i.e. authenticate). The app can't do anything with that projection other than accept it. Delegation is the act of granting that app the right to project the identity further downstream.

There's a fundamental requirement with this though. The method of projecting (authing) the user has to rely on a trusted third party. In other words using something like federation or protocols like Kerberos. This is in contrast to just passing around creds. You might wonder why.

It's because if the app in the middle has your password it can just use your password for downstream stuff, ad infinitum. Trusted third party auth protocols by their nature like having 1:1 relationships with all parties. Unfettered projection complicates things.

I keep saying projecting when I technically mean authenticating. You might wonder why. Authentication is the act of verifying a credential, where that credential could be a password, or a token, or a ticket. Most people associate it with just the first step: the user password.

With delegation there's usually multiple forms of credentials in play. However, the identity represented by those credentials remains the same. Hence projecting the identity.

There's another fundamental requirement with delegation. The middle app must be explicitly granted the right to delegate to other things. This is, I hope, somewhat obvious to everyone that it's for enforcing security. Let me explain.

Authentication is about building a trusted relationship (for some definition of trust) between two parties. When I ask Active Directory for a ticket to a web server I'm relying on the web server to trust AD, and by extension trust me. I project my identity to that server.

If the web server could take that projection and act as me for other services, that original trust is broken because AD hasn't blessed the relationship between me and that other service.

This right to delegate can come in many forms and is primarily dictated by the protocol in play. There are two basic types though.

First, there's the right to delegate to anyone. Meaning this middle app can project to any app whatsoever and there's no (or little) limit on it.

Second, there's the right to delegate to specific apps. Meaning the middle app is only allowed to project your identity to a known list of apps. This particular method is preferred for, again I hope, obvious security reasons. Most delegation services stick to this form nowadays.

There are other forms, but they all kinda fit the above patterns.

There are different implementations of delegation. Active Directory calls it delegation. Federation protocols like WS-Fed/Trust call it ActAs. OAuth calls it delegation or impersonation, Azure AD calls it on-behalf-of. Let's look at AD and Azure AD.

Many folks know about delegation because of this screen in Active Directory. Many people have come to hate this screen. This screen applies to that middle web server.

User A projects their identity to web server B, and B projects A's identity to web service C. A => B => C. The screen from before applies to B. That screen lets you control what form of delegation you want to allow.

The first option is obvious. B isn't allowed to delegate to anyone, period.

The second option is mode 1: B can delegate to whoever it wants. Could be C, could be D, could be Y, could be Z.

The third option is mode 2: B can only delegate to C if it's in the list below it.

But the third option has a catch: you can only delegate when using Kerberos. So option 3b was added: you can delegate to whoever is in this list without authenticating. Whaaaaaat the eff? We'll come back to this.

So AD relies on Kerberos for delegation. The first option is called unconstrained delegation. This is the easiest mode to wrap your head around because what you're doing is giving your client TGT to the web server, so the server can do whatever it wants with it.

That's literally all it's doing. The client is informed that the server uses unconstrained delegation so the client copies the user TGT, stuffs it into the AP-REQ authenticator, and fires it off to the target server.

Once the server receives the AP-REQ, it decrypts it and creates a logon session and ticket cache. The TGT is stuffed into the cache. When the server needs to make a call as the user it uses the TGT to request a ticket to whatever is next. How Windows Single Sign-On Works (syfuhs.net)

Incidentally this is why Credential Guard blocks unconstrained delegation on the client. Unconstrained delegation is Stealing-your-TGT-as-a-Service. How Windows Defender Credential Guard Works (syfuhs.net)

Next is what's called constrained delegation. You provide a list of services web server B is allowed to project the identity to. You might think okay, it's just like unconstrained where they copy the TGT over, and the web server has to ask what it can project to, but it's not.

We don't live in a world where we can blindly trust web server B. If B is even a little untrustworthy and it managed to get a hold of a TGT for the user, it's game over for that user. We in no uncertain terms want that TGT loose on the network. So what do?

Well, we don't do Kerberos. 😲

More precisely what we do is use a protocol that is an extension to Kerberos called Service-for-User [MS-SFU] or s4u. [MS-SFU]: Kerberos Protocol Extensions: Service for User and Constrained Delegation Protocol | Microsoft Docs

S4u is a way for web server B to safely request a ticket to service C on behalf of the user. It does this by using the service ticket user A handed to server B as an evidence ticket. Huh?

Imagine you're web server B. You have an identity "webserverb$". You have a password, and AD knows who you are. You can receive Kerberos service tickets. AD encrypts them to your password. Nothing particularly fancy about this. It's basic Kerberos. A bit about Kerberos (syfuhs.net)

Now, you receive a ticket from user A and decrypt it. You create the SSO logon session for the user and tell the web app it's user A. Now the web app wants to communicate to service C as that user so it asks for a ticket to service C.

Web server B has a logon session for user A, so web server B checks for a TGT. No TGT because we're constrained. Okay, let's try S4U. To do this, the web server makes a request to the domain controller for a service ticket to C.

The server has an identity on the network, therefore it has a TGT, and it's allowed to ask for service tickets. Again, plain old Kerberos. However, the identity in the service ticket would be server B. So server B adds the service ticket it received from A as an evidence ticket.

Well now this is interesting to the domain controller. It receives a ticket request from B for C and also a service ticket from A FOR B, plus some flags to say do S4U. So the DC checks the delegation list. Is B allowed to delegate to C? Yes, okay give B a ticket to C as A.

I promise you aren't having a stroke. The way to follow this is that B needs to prove it's allowed to get the ticket to C on behalf of A, otherwise B could be evil and impersonate whoever they want.

Once web server B receives the ticket, it bubbles it up to the web app, and the web app stuffs the ticket into the request for web service C. C receives it, decrypts it, and sees it's for user A. And we're back to how SSO works. How Windows Single Sign-On Works (syfuhs.net)

But remember how I said there's that other mode, where it doesn't require authentication? Yeah about that. We call it Protocol Transition. This exists so web server B can authenticate users using a protocol Active Directory doesn't understand, while still impersonating them.

Consider if web server B supports SAML. AD doesn't understand SAML, but the web app wants to impersonate the identity projected through SAML so when they call web service C they're acting as user A. This can work through protocol transition.

It uses the same protocol extension -- S4U. The difference is that the evidence ticket isn't real. Or rather, it's an evidence ticket issued to the server itself. Huh?

There's a sub-protocol in S4U called service-for-user-to-self or s4u-self. This is an incredibly useful protocol, because an application can request a service ticket to *itself* on behalf of any user. It's useless for auth, but still contains the user identity. Give it a try.

It's a TGS-REQ to me, with some bits that says please get on behalf of bob.

So for just a name you get a ticket to yourself. A ticket to yourself can get your constrained delegation. Put the two together and you have s4u-proxy, AKA protocol transition!

Of course, here's the problem: protocol transition is increeeedibly dangerous. You've just granted web server B the rights to impersonate whoever they want. That's worse than unconstrained delegation. Oops. As such you've effectively granted web server B the right to act as a DC.

This is, incidentally why selecting that option tends to make things "just work".

Please, please, pleeeeeease be incredibly careful if you choose to use it. Any server configured for protocol transition should have the same security as a DC. No joke.

Anyway, there's one more thing with AD delegation that isn't on that screen. It's something called Resource-Based Constrained Delegation. Huh?

It's basically regular constrained delegation, but the approval list is flipped. The idea is that instead of saying web server B is allowed to delegate to web service C, it says web service C is allowed to be delegated from web server B.

This actually makes more sense from a security perspective. The thing you're protecting is web service C, so why are the controls on B? The armed money delivery transports don't dictate which vaults they can go in to, the banks dictate which transports are allowed into the vault.

The other useful thing about RBCD is more of a practical one. Regular constrained delegation doesn't work across forests because of how the protocol happens to work. There isn't enough authz details in it to identify the service because SPNs are only unique within forests.

RBCD relies on SIDs. Web service C has a list of SIDs that can delegate to it, and when web server B makes a request to the other forest, it includes web server B's SID.

But enough about that. Let's look at another form of delegation: Azure AD on-behalf-of.

Azure AD-based apps are an evolution of AD-based apps. They still have similar, or higher, security requirements. As web server B I want to project user A's identity to service C.

The difference is in the choice of protocols. Azure AD relies on OAuth2 for authentication and authorization. A client authenticates user A and gets a JWT access token signed by AAD to web server B. Web server B validates the ticket and the app now knows it's user A, so says AAD.

But the web app wants to access another resource, say Microsoft Graph. The web app can use it's own identity, or it can use the identity of the user. You could keep it simple and use the app identity, but then you need to grant that identity access to all users resources.

That's a fairly broad set of permissions, and attacking that app gets you everything in Graph. That's bad.

But if you use the user identity, the app doesn't need anything special, and the app can operate as just that single user. That's good.

Protocol-wise this is an extension to OAuth called on-behalf-of. Microsoft identity platform and OAuth2.0 On-Behalf-Of flow - Microsoft identity platform | Microsoft Docs

Not surprisingly it kinda sorta acts a lot like s4u. User A gets a token to server B from AAD. Server B has it's own identity and authenticates to AAD, providing user A's access token to B as an evidence token (assertion).

AAD checks the list of apps that B is allowed to delegate to, and if C is in the list, it issues a token to B on behalf of A for C.

This is actually pretty cool. First, it's way easier to follow along because it's all HTTP instead of binary encoded ASN.1 goop. Second, it supports multiple token types. Ergo it supports multiple protocols.

That means you can transition between OAuth and (say) SAML without turning your app into a major target, because every exchange and token is still authenticated. Awesome.

Third, you can enforce consent. This shifts authorization a bit so the user is now in control over who can project and forward their identity. That's a nice privacy win.

So obviously there's been a delegation evolution. From AD unconstrained, all the way to OAuth on-behalf-of. Hopefully it's gotten easier and more secure along the way. /fin

Preventing UAC Bypass through Kerberos Loopback

site@syfuhs.net — Thu, 21 Jan 2021 15:32:00 GMT

Have you ever tried to access an admin SMB share on your local machine (\\localname\c$) only to find out you don't have admin perms despite being a local administrator? Why is that? Here's something you maybe didn't know: It's by design -- it's a way of preventing UAC bypass. pic.twitter.com/1SwNrWn4j8

— Steve Syfuhs (@SteveSyfuhs) January 19, 2021

You might be saying "well duh! That's what UAC does!"

But we're talking SMB, so it's over the network, therefore it's Kerberos, and Kerberos can't doesn't do UAC over the network because Kerberos doesn't contain local authz information, and UAC is all about local authz.

Note: SMB loopback actually uses NTLM. But let's not let details get in the way of a good explainer. (NTLM does work similarly though)

What's the difference between \\localmachine\c$ and \\othermachine\c$ then? Actually, nothing.

Some background: when a client needs a kerb ticket to something it asks Windows. Windows creates an AP-REQ, which is the requested ticket plus authenticator.

The ticket is created by the KDC. The client can't see inside it, and can't manipulate it. It's opaque. However, the client can ask the KDC to include extra bits in the ticket. See here for fiddly details: A bit about Kerberos (syfuhs.net)

These extra bits are just a way to carry information from the client to the target service during authentication. As it happens one of the things the client always asks to include is a machine nonce.

See, when the client asks the client Kerberos stack for a ticket, the stack creates a random bit of data and stashes it in LSA and associates it to the currently logged on user. This is the nonce. This nonce is also stuck in the ticket, and then received by the target service.

The target service, i.e. the same Windows Kerberos stack, knows about this nonce and asks LSA if it happens to have this nonce stashed somewhere. If it doesn't, well, then it's another machine and just carry on as usual.

However, if it does have this nonce, LSA will inform the Kerberos stack that it originally came from user so and so, and most importantly that the user was *not* elevated at the time.

What does that mean? Not elevated?

This is UAC -- User Account Control. UAC is the thing that *tries* to prevent you from doing something stupid with admin privileges.

See, when you log onto Windows you're granted a set of local privileges. Some of these give you lots of control over the system, like local administrator. UAC is a way of preventing you from doing silly things as administrator unless you reeeeeeally want to.

The way it works is conceptually simple. During login you're given two NT tokens. The first is your high privilege token. It contains all those special privileges. The second is your low privilege token. Everything you do uses your low privilege token by default.

So when you do need to do something with higher privileges you need to ask Windows "Hey! I need the high privilege token, pleeeeeeeeeeeeease".

Anyway, LSA knows all about these tokens, so when it receives this Kerberos ticket and contains that nonce and finds the user, voila it has a low privilege token.

Now the Kerberos stack makes a copy of the ticket contents for it's own SSO session and whenever you do anything in that session, say try write to c$\windows\system32, the authorization system checks the privileges and in this case says nooooope. How Windows Single Sign-On Works (syfuhs.net)

So as a result you don't get to bypass UAC by going through the network. /fin

Hybrid Authentication with FIDO

site@syfuhs.net — Tue, 19 Jan 2021 01:54:00 GMT

Speaking of #passwordless... did you ever wonder how we made it work with on-prem Active Directory? It's kind of magic. We built a KDC in the cloud. Here's how it works... https://t.co/ww7qtg0r3P

— Steve Syfuhs (@SteveSyfuhs) August 21, 2020

You log into Windows with your FIDO key. Our cloud auth package in Windows (CloudAP) knows how to handle FIDO auth and does the handshake with https://login.microsoftonline.com. That returns an OAuth PRT and a special cloud-minted Kerberos Ticket Granting Ticket.

The TGT is encrypted using the session key agreed to earlier, so only the client machine can process it. Windows knows this key, and knows that this TGT is special, so it triggers a TGS-REQ to a nearby on-prem KDC using this special TGT as the evidence ticket.

Prior to all this you had to register your domain with AAD so it could use FIDO, and in doing so what we did was we created a special Read-Only Domain Controller and RODC krbtgt secret. That secret got synced to AAD and is what we use to encrypt the special TGT.

When your on-prem KDC receives this special TGT it knows that it's special and uses the RODC krbtgt_### secret to decrypt it. The KDC exchanges it for a real TGT and returns it to the client. The client is now in a steady state as far as Kerberos is concerned.

But what about NTLM? Guh. Well, it turns out we can't kill NTLM yet, so when the client had this special TGT in hand and knew to exchange it for a real TGT, it included a special flag on the TGS-REQ to ask the KDC to include some additional SSO creds in response.

Those SSO creds are returned to the client and handed off to the various SSO packages in Windows, one being NTLM, and those packages handle them as they would any other credential. Voila, now they know how to handle NTLM and any other supported SSO protocols. Magic!

Can't forget the pups.

Kerberos FAST Armoring

site@syfuhs.net — Sat, 16 Jan 2021 22:32:00 GMT

It's Monday evening, the weather is great, and we're in the middle of a pandemic. Lets talk Kerberos! Or rather, it's little known nephew FAST and Armoring.

— Steve Syfuhs (@SteveSyfuhs) July 28, 2020

You might have heard about it from such features as Windows Kerberos Armoring. It's in fact three separate things. What's New in Kerberos Authentication | Microsoft Docs

First, it's part of a larger body of work called the pre-authentication framework, a way to provide a generic extensible model of adding authentication methods to Kerberos. It gets more interesting, honest.

Second, it's a method of creating a generic secure tunnel for information to flow between a client and a KDC within the original Kerberos protocol. I swear, it'll get interesting.

Third, it's a method where you bind your authentication to the machine account to prevent password attacks like Kerberoasting -- aha, there's the exciting thing!

Kill all password attacks. Here's how it works.

Kerberos is deceptively simple: you, the client, request a ticket from the KDC. The KDC happily responds and encrypts the response to your password. You decrypt it with your password, and voila you have a ticket. This is dangerous because...

You now have this oracle that'll return encrypted blobs of structured data to any caller, encrypted to a password. This makes cryptographic attacks super easy for weak passwords. A solution was added: the pre-auth timestamp, which is the current time encrypted with your password.

The KDC receives this timestamp, decrypts it because it knows your password (-ish) and verifies the request was within a short period of time. But still, attackers can intercept the messages and perform those same attacks. They just need to convince you to get a ticket first.

The protocol itself is bound by this weak thing -- your password. It needs more randomness. Incidentally, most callers happen to be on a domain joined computer when doing Kerberos, and the computer uses a super random password. Innnnnteresting.

So this framework has this thing called Armoring, which is the idea that you take a key and mix in a whole lotta randomness from another source, and use that instead. So here we take the computer account, and have it request a TGT. The TGT has this thing called a session key.

That session key is protected within the TGT and only the client and the KDC know it. So why not let the KDC use the randomness of that session key to protect the client?

The client gets this new mixed key and can now communicate with the KDC. The randomness of the computer password makes any crypto attack infeasible, and the user can still user a password.

Okay, but what about users that aren't on a domain joined computer? There's a small provision in the spec that lets callers request an anonymous ticket. You lose server authentication, but when you're out alone in the wild, what're you gonna do?

...assuming the KDC supports that provision of course.

Now that's all well and good, but what about this tunneling thing? Well back in the dark ages crypto was export controlled (ugh), which left a lot of protocol designers deciding to leave a bunch of crap in cleartext. FAST said nuh uh, attackers can modify that over the wire.

So you have this super secret shiny key now, and a nice dark hole to shove stuff. So what do we do? We make a copy of the entire request, encrypt it, and shove it in the hole. And leave the outer message in cleartext for appearances sake.

Because why not?

The cool thing about all this, aside from making password attacks more difficult, is it lets developers easily extend the protocol to include new pre-auth methods like OTPs or PAKEs, and does a reasonably good job of hardening the aging crypto properties of the protocol.

So, go turn it on. It's super easy.

Requisite puppies.

Should I Turn off NLA?

site@syfuhs.net — Wed, 30 Dec 2020 19:00:00 GMT

No!

Network Level Authentication is how Windows authenticates remote desktop clients and servers before sending your credentials over to a remote machine.

If the client can't authenticate the remote server then there's no guarantee it's the server you actually want to log in to, and therefore may be a server trying to steal your credentials.

This is why you see a credentials prompt when using RDP and never see the Windows logon screen during the initial connection.

See How Authentication Works when you use Remote Desktop (syfuhs.net) for the gory details.

The whole point of NLA is to make sure the name you typed into the RDP application is in fact the server you've connected to. It fails if it isn't the same server. Without NLA the client has no method to prove the remote server is the same as what you've typed in.

This applies to all forms of credentials, not just passwords. Passwords are ubiquitous so they're the most dangerous to leak, but smart card logons and any number of 3rd party mechanisms can be proxied and snooped if you've turned off NLA. This means an attacker sitting in between your client and server can impersonate users without you ever knowing.

So please think twice about turning off NLA. It's there to protect you.

A Strategy for Protecting Privileged Access

site@syfuhs.net — Tue, 22 Dec 2020 22:46:00 GMT

So, there was a major attack recently. Apropos of that, I wanted to re-up how we (Microsoft) recommend folks lay out their environments for security-mindedness. In short we call it Privileged Administration. It's the foundation of our (ugh) zero trust model.

— Steve Syfuhs (@SteveSyfuhs) December 22, 2020

Fundamentally the system is simple* because it's just** about isolation -- isolation of business impact. The more damage you can do to the org the fewer things you're allowed to touch. https://docs.microsoft.com/en-us/security/compass/privileged-access-strategy

*I'm lying.

**It's never "just"

This is somewhat contradictory because by definition highly privileged users have more power. Our model doesn't really reduce these privileges, it just prevents them from being used in certain places.

This is the isolation piece. Domain admins shouldn't be able to do squat in most parts of your network, for instance. They should have an isolated little hole they can live in where *only* domain admins can go.

It's structured into security levels or more often described as tiers. Think of them as high, medium, and low impact.

High impact is all your critical infrastructure and accounts required to manage that infrastructure. These are your DCs and domain admins. They can destroy your business with a single command. Maaaaaybe don't let them log in to user workstations to install random programs?

Medium impact is for specialized roles. They can cause significant damage, but the risk is lower. These are folks like executives, developers touching production, or helpdesk techs.

And then low impact is for everyone else. They can access some stuff, but usually nothing critical.

The delineation of all of these is business impact: how badly can I break the company just by doing my job (poorly)?

That's all well and good, but these are just words and I'm just some person on the internet. Implementation and follow through are critical here.

How does one implement such privileged administration? We have docs: https://docs.microsoft.com/en-us/security/compass/security-rapid-modernization-plan

But start simple. On the Venn diagram of regular user and privileged user accounts, there should be two circles that don't overlap.

That means your domain admin account isn't your daily driver. You don't check your email with DA. You as the admin have two accounts: a named DA account, and your regular low (or medium) tier account.

Also follow the concept of clean source. That is -- the thing you're accessing a resource from should have the same level of security. In other words the laptop you check your email from shouldn't be the machine you RDP into a domain controller with. More on this later.

While you're at it prevent yourself from shooting yourself in the foot: lock down where privileged users can log in. A compromised workstation shouldn't get you keys to the kingdom because a DA logged in three days ago because the user was complaining about something.

Enforce strong authentication starting from high privilege on down. Domain Admins must always log on with smart cards. Period.

Why TF is your DA logging on with a weak password of i@mgR00t!?

This of course gets more complicated the further down the stack you go. Smart cards can be expensive and error prone. Introduce other technologies like Windows Hello or FIDO. Your users will love you and it reduces the number of password resets.

Audit audit audit audit audit audit audit audit audit audit audit. And for the love of god, review the audit logs. There are tools for this. Some cheap, some not. Some good, some...not. Tune it for critical systems and users first, then work your way out.

Build up isolated interfaces for those things that require transitioning between privilege levels. HR apps, group management, etc. Make it easy for users to modify non-impacting data so they don't need super privileges just to change their middle name or add themselves to a DL.

And (or at least) enforce conditional authz policies for such things to require MFA.

Better yet, build out a just-in-time elevation process for these privileges. Even if you *are* logged on as a domain admin you should still have to indicate what permission you want so you can fiddle with stuff.

JIT plus auditing and you have a complete record of what happens in your network.

Once you've done all these things we can revisit this clean source principle. If I'm the IT person managing my domain, how do I access the necessary tools? I'm absolutely not doing it from the machine I check my email from.

Highly privileged access requires high security. Yes, that sounds dumb, but the point is sound: an attacker that compromises your low privilege account should not be able to leverage that into a high privilege account.

So again, that machine you check your email on shouldn't have domain admin creds on it, ever. Use a dedicated machine for highly privileged operations, period. To make your life easier you can create a VM on that machine to check your email.

The difference is you can go from the host into the VM, but not from the VM to the host. These are called Privileged (or Secure) Access Workstations or PAWs/SAWs.

I use one all the time. My daily driver is my corporate desktop. I can access my email and write code on it. However, if I need access to production systems for troubleshooting or fixing stuff I use my SAW.

But I don't use my regular account. I use my privileged account, which requires a smart card to log on. And then I JIT to get the permissions I need to access resources. If I need to access the internet I have a scratch VM running my low privilege account too.

You'll notice I never once mentioned jump servers. That's because they aren't secure as people think. They are by design doing something that breaches isolation. You're transitioning from a low privilege to a high privilege in a low privilege environment.

And also note that this doesn't give you a pass to ignore endpoint security. Security is a group effort. Lock down your machines, keep them patched, deploy ATP, Cred Guard, HVCI, etc.

So go do some reading if you're inclined.

How Windows Defender Credential Guard Works

site@syfuhs.net — Tue, 01 Dec 2020 23:34:00 GMT

Okay, lets talk Credential Guard. What is it, why it matters, and how it works.

— Steve Syfuhs (@SteveSyfuhs) December 1, 2020

Credential Guard is a Windows service that protects credentials from being lifted from a machine. Since that means nothing to the vast majority of people let's expand on that.

Credential Guard protects the secrets used by Windows for single sign-on from being stolen and used on other machines.

These secrets are numerous. First, there's your password. You type it in to log into Windows. You exchange your password for a TGT. Your TGT has a session key to get access to service tickets. Historically these secrets were ripe for the taking if you could run code as system.

The attacks fundamentally work something like this. User logs on, downloads something accidentally. Thing finds an EOP, runs as admin, converts admin to system. This thing can now read all memory on the system. Your secrets are in memory. Voila.

But that's not all. Sometimes it doesn't need to be a complicated attack. We have undocumented APIs that let you get access to these secrets too. They've been around forever and we can't get rid of them for historical reasons. We can only make them harder to access.

A good example is SSO in Java client applications. If a java application wanted to use the current users logon session to access resources without prompting, the end user had to toggle an undocumented registry key that let java steal the user's TGT session key.

Or there's this thing called Credential Manager. If you received a prompt, say for RDP or accessing a file share outside your domain, and clicked save credentials then those credentials go into credman. All you have to do is ask credman to give you those creds and it will.

These were all necessary at one time or another and as our threat models matured we realized there was clearly a security gap here. The problem of course is that our security boundary is system|everything-else. Once you got on as system you could do whatever you wanted.

We attempted to solve this with LSA Protected Process mode, which is an incredibly clever way of preventing anything from being loaded into the LSA process that wasn't signed by a special CA.

This was an important step because loading stuff into LSA was a major attack vector and it'd be a huge win if we could nip that. And it was and is reasonably effective for certain types of attacks. Here's where people "but but mimikatz" me.

LSAPPL isn't perfect. It has weaknesses. If you can make it into the kernel and modify process stuff you can turn it off with the flip of a bit. But you don't build a house with just a hammer. It's one tool of many. Enter Credential Guard.

For decades there was nothing outside the kernel security boundary. Kernel talked to hardware and had access to everything. Ring 0. Nothing lower than ring 0.

Until someone invented the hypervisor, which is just a fancy way of saying "hey lets run multiple kernels on a single machine and doll out access to hardware with us controlling it instead of the kernels." Ish.

It took a generation or two but our hypervisors pretty much let you run all VMs on bare metal now. The hypervisor is this tiny tiny thing that just acts as a resource mediator now and this mediator allows these VMs to stay isolated from one another.

One VM can't peek into another's memory. Can't look at it's processes, can't communicate with it except through external stacks like ethernet, can't do any of that. Full and complete isolation.

That therefore means anything running in a VM kernel doesn't have a full view of the world, and malicious code is pretty much stuck there.

So someone had the brilliant idea: "hey, what if we moved all the critical security-sensitive stuff into a separate VM?"

Enter VSM: Virtual Secure Mode. Or VBS: Virtualization-based Security.

This might seem confusing because your Windows client doesn't run in a VM. It runs on bare metal. You inserted a DVD^H^H^H USB stick and installed it on the physical hard drive. And things running in kernel on the bare metal can access all things on the bare metal. What gives?

As it happens, that's not strictly true anymore. Windows these days runs in a VM on the hypervisor. 😲

Sort of.

It doesn't exist as a VHDX or anything like that. All the resources are physical, but they're now governed by the hypervisor.

So what that means then is if you're running VSM/VBS with Credential Guard or any other service relying on it, you're actually running two virtual environments on your machine.

These are...hybrid VMs if you will. They exist as different modes of the same Windows installation. You have two versions of Windows running side by side. Except one is more locked down than the other.

These VMs are isolated into things we call Virtual Trust Levels. VTL 0 is normal world. It runs all your usual stuff. Your kernel, LSA, explorer, your browser, etc. Regardless of whether you have VSM running or not your stuff runs in VTL 0.

But with VSM enabled we have another VTL -- VTL 1. This is secure world. It runs a different instance of your kernel, but highly optimized for just a special purpose, as well as a separate user mode also highly optimized for a special purpose.

Think of VTL 1 as a side kick. Not quite the hero. Doesn't do everything the hero does, but is super useful in a pinch.

We'll stop using this analogy now because that's as far as it can be taken.

So lets recap. Have secrets living in user mode, can be accessed by evil things running as the user with a little bit of effort. Have virtual secure mode, cannot be accessed by evil things running as the user.

...

With me so far?

What if we moved those secrets out of user mode and into virtual secure mode so evil things can't access them?

And so Credential Guard was born.

Credential Guard is this thing called LsaIso.exe. It's the isolated version of LSA because it lives in Isolated User Mode, AKA user mode of VTL 1 (as opposed to regular user mode in VTL 0).

Processes that run in VTL 1 IUM are normal processes. They're exe's compiled to x64. They're just special in that they don't get to rely on a lot of stuff normally found in Windows.

Because VSM runs as this hybrid thing, it shares certain resources between VTL 0 and 1, such as the process list. The IUM processes are projected into your process list, which is why you can see it. Sorta. It lets you see some things, but not everything. Also can't kill 'em.

When your VTL 1 starts up it eventually starts LSA. LSA is this thing that manages all the security on your machine, and is where all your secrets normally live in memory.

As it starts up LSA checks for Credential Guard. One of the shared resources between VTL 0 and VTL 1 is a communications channel -- RPC. It's always RPC. Even when it never leaves the box it's RPC or it's special friend LPC, but it's still RPC.

And this communications channel is locked down. Only VTL 0 SYSTEM services can communicate over it, which LSA is, so your run of the mill evil thing can't talk to it. It requires a bit more effort if you're evil.

Anyway, this RPC channel is built in such a way that it acts as an oracle. You ask it a question and it gives you a weird answer. This is important because we have to be careful about what it leaks into normal user mode because evil things.

What kind of questions can you ask it? Well, lets walk through a logon first for context.

Way back in the beginning of time I wrote that thread about password logon. You type your password into the textbox. This textbox lives in VTL 0 normal user mode. What Happens When you Type Your Password into Windows? (syfuhs.net)

The password is passed to LSA, and LSA hands it off to Credential Guard. You're doing Kerberos so you also pass a flag saying "heeeeeey can you give me the stuff I need to make Kerberos work".

And so Cred Guard does. You gave it some data, told it to encrypt it using the password, and out came the encrypted goo and a handle to the secret. You never see the secret again.

This is one of those important things to note: the password is unprotected for a brief period of time, but once it's in Cred Guard it's safe, as opposed to stuck around in memory forever.

Anyway, no LSA has this encrypted goo. Can't reeeeeally do anything with it so it fires it off to KDC. The KDC does it's normal Kerberos thing and returns a TGT encrypted to the user's password.

Aha, but LSA doesn't have the password anymore. What do? LSA passes this encrypted blob to Credential Guard, plus the handle to the password and asks Cred Guard nicely to decrypt the blob. And it does, returning to it *most* of the important stuff.

The one thing it doesn't return to LSA is the TGT session key. Or rather, it returns the session key but encrypted to a key only known to Cred Guard, so if someone could steal the blob out of memory it'd be useless.

Now the machine needs a service ticket to itself, or the user is opening a file share or whatever it is people do.

LSA has this TGT (useless on it's own) plus the session key (super important) that's been encrypted to a key it can't use. LSA fires a request off to the KDC and KDC does it's thing, responding with a service ticket, encrypted to the TGT session key. Can you guess what's next?

Only the session key can decrypt to response, and only Credential Guard can decrypt the session key. So LSA hands it both and asks it nicely to decrypt the thing. It does, returns the decrypted service ticket to LSA, which hands it off to the caller, and then off to the app.

And that's all Credential Guard does. You ask it a question and it gives you an answer. But you have to ask the right question.

This is why these legacy and annoying APIs don't work anymore. They ask the question, but don't supply all the right details and get back a weird answer. It's not exactly wrong, just...not useful.

This is also why certain things are outright broken by design. Legacy protocols like MS-CHAP that make it super easy to reverse the password through cryptoanalysis are blocked because it breaks the guarantee Credential Guard provides.

Not to mention unconstrained delegation. We're trying to protect your TGT from being stolen and used on other machines. Unconstrained delegation is Stealing Your TGT as a Service (PS, that's the name of my new consulting firm).

Now all of this hinges on the fact that you have a user mode process in user world communicating with secure world. As an attacker why don't I just turn that off?

Well, yeah. You can do that. At some point you can tamper with the machine enough that it's not really your machine anymore. We can make it a lot harder to do though.

Remember LSA PPL? It prevents stuff from tampering with LSA. That goes a long way to protecting the LSA, but isn't perfect because you can bypass it through the kernel. Thanks to another VSM feature you *can* protect it: HVCI. HVCI watches kernel mode memory for modifications.

We did a whole thing about this last year: Comprehensive protection for your credentials with Credential Guard and HVCI - Microsoft Tech Community

Anyway, that's Credential Guard. It's a relatively small feature, but an incredibly useful tool for protecting your user secrets.

Registry key	HKEY_LOCAL_MACHINE\System\CurrentControlSet\services\KDC
Value	DefaultDomainSupportedEncTypes
Data type	REG_DWORD
Default value	0x27
Restart required?	No