Dynamic Tracing in Windows 10 19H1

August 2nd, 2018

Windows 10 introduces an exciting new feature with potential security implications – dynamic tracing which finally enables long awaited-for features in the operating system.

At boot, the OS now calls KiInitDynamicTraceSupport, which only if kernel debugging is enabled, will call into the TraceInitSystem export provided by the ext-win-ms-ntos-trace-L-1-1-0 API Set, which is not currently shipping in the public OS schemas. This export receives a callback table with the following 4 functions:

  • KeSetSystemServiceCallback
  • KeSetTracepoint
  • EtwRegisterEventCallback
  • MmDbgCopyMemory

If the function returns successfully, KiDynamicTraceEnabled is now set.

The last routine is not terribly interesting, as it is already used by the debugger when accessing physical memory through commands such as !dd or dd /p. But the other three routines, well, Christmas came early this year.

Kernel Mode ETW Event Callbacks

EtwRegisterEventCallback is a new internal function, accessible only by the dynamic trace system, which allows associating a custom ETW event callback routine, and associated context, with any ETW Logger ID. The function validates that the callback function is valid by calling KeIsValidTraceCallbackTarget, which does two things:

  1. Is Dynamic Tracing Enabled? (i.e.: KiDynamicTraceEnabled == 1)
  2. Is this a valid callback (same requirements as Ps and Ob callbacks, i.e.: was the driver containing the callback linked with /INTEGRITYCHECK)

Once the check succeeds, the matching logger context structure (WMI_LOGGER_CONTEXT) is looked up, and an appropriate ETW_EVENT_CALLBACK_CONTEXT structure is allocated from the pool (tag EtwC), and inserted into the CallbackContext field of the logger context.

At this point, any time an ETW event is thrown by this logger, this kernel-mode callback is also called, introducing, for the first time, support for kernel-mode consumption of ETW events, one of the biggest asks of the security industry in the last decade. This call is done by EtwpInvokeEventCallback, which calls the registered ETW callback with the raw ETW buffer data at the correct offset where this event starts, and the size of the event in the buffer. This new callback is called from:

  • EtwTraceEvent and EtwTraceRaw
  • EtwpLogKernelEvent and EtwpWriteUserEvent
  • EtwpEventWriteFull
  • EtwpTraceMessageVa
  • EtwpLogSystemEventUnsafe

This essentially gives access to the callback to any and all ETW event data, including even WPP and TraceLog debug messages.

System Call Hooks

KeSetSystemServiceCallback, on the other hand, fulfills the second Christmas wish of every Windows security researcher: officially implemented system call hooks. The API allows the dynamic trace system to register a system call hook by name and pass an associated callback function and context. It introduces a new table, called the KiSystemServiceTraceCallbackTable, which copies the contents of the KiServicesTab (a new, more comprehensive system call table) into a Red-Black tree which contains an entry for each system call with its absolute location, number of arguments, and a pre and post callback (and context).

Before continuing, it’s worth talking about the format of the new KiServicesTab structure, as it introduces some valuable information for reverse engineering:

  • The first 32-bit value is the hash of the system call function’s name
  • The second 32-bit value is the argument count of the function
  • The last 64-bit value, is the absolute pointer to the function

The hash function, implemented in a sane language as C looks as follows:

for (nameHash = 0; *CallName != ANSI_NULL; CallName++
    nameHash = (1025 * (nameHash + *CallName) >> 6) ^
                1025 * (nameHash + *CallName);

With some cringy JavaScript, we can write a simple WinDbg imperative script:

"use strict";
function hashName(callName)
    var hash = 0;
       c => hash = 1025*(hash + c.charCodeAt(0)) >>> 6 ^
                   1025*(hash + c.charCodeAt(0))
    return hash >>> 0;

The trick, of course, is that the function name must be passed in without its Nt prefix – countless hours having been wasted trying to debug the hash algorithm by yours truly. Let’s take a look at some debugger output:

lkd> dps nt!KiServicesTab L2
fffff803`0e102e50  00000004`b74a2d8f
fffff803`0e102e58  fffff803`0dfe2ac0 nt!NtOpenKeyTransacted
lkd> dx @$scriptContents.hashName("OpenKeyTransacted")
@$scriptContents.hashName("OpenKeyTransacted") : 0xb74a2d8f

Back to KeSetSystemServiceCallback, if a system call callback is being registered, the KiSystemServiceTraceCallbackCount variable is incremented, and the KiDynamicTraceMask has its lowest bit set (the operations are reversed in the case of a system call callback unregistration). Unregistration is done by looping while KiSystemServiceTraceCallbacksActive is set, acting as a barrier to avoid unregistration in the middle of a call. All of these operations are further done under a lock (KiSystemServiceTraceCallbackLock).

Once the callback is registered (which must also satisfy the checks done by KeIsValidTraceCallbackTarget), it will interact with the system call handler as follows: inside of KiSystemServiceCopyEnd, a check is made with KiDynamicTraceMask to verify if the lowest bit is set, if so, system call execution goes through a path where KiTrackSystemCallEntry is called, passing in all of the register-based arguments in a single stack-based structure.  This uses the KiSystemServiceTraceCallbackTable Red-Black Tree to locate any matching callbacks, and if one is present, and KiDynamicTraceEnabled is set, KiSystemServiceTraceCallbacksActive is incremented, the callback is made, and then the KiSystemServiceTraceCallbacksActive is decremented.

When this function returns, the return value is captured, and the actual system call handler is called. Then, KiTrackSystemCallExit is called, passing in both the capture result from earlier, as well as the return value of the system call handler. It performs the same operations as the entry routine, but calling the exit callback instead. Note that callbacks cannot override input parameters nor the return value, at the moment.

Trace Points

KeSetTracepoint is the last of the new capabilities, and introduces an ability to register dynamic trace points, enable and disable them, and finally unregister them. The idea of a ‘trace point’ should be familiar with anyone that has used Linux-based kprobes before.

A trace point is registered by passing in an address and which is then looked up against any currently loaded kernel modules. As long as the address is not part of the INIT (which is discarded by now, or soon will be) or KVASCODE section (which is the KVA Shadow space used to mitigate Meltdown), a trace point structure is allocated in non-paged pool (with the Ftrp tag). Next, the KiTpHashTable is used to scan for existing trace points on the same address. If one is found, an error is returned, as only a single trace point is supported per function. Note that trace point callbacks are also validated by calling KeIsValidTraceCallbackTarget just like in the case of the previous callbacks.

KiTpSetupCompletion is used to finalize registration of a trace point, which first calls KiTpReadImageData based on the instruction size that was specified. An instruction parser (KiTpParseInstructionPrefix, KiTpFetchInstructionBytes) is used, followed by an emulator (KiTpEmulateInstruction, KiTpEmulateMovzx, and many more) are used to determine the instruction size that is required. Once the information is known, the original instructions are copied. For what it’s worth, KiTpReadImageData is a simple function which attaches to the input process and basically does a memcpy of the address and specified bytes.

Once registered, the KiTpRegisteredCount variable is incremented, and the trace point can now be enabled. The first time this happens, KiTpEnabledCount is incremented, and the KiDynamicTraceMask is modified, this time setting the next lowest bit. Then, KiTpWriteMemory is called, which follows a similar code path as when using the debugger to set breakpoints (attaching to the process, if any, calling MmDbgCopyMemory to probe the address, and then using MmDbgCopyMemory wrapped inside of KdEnterDebugger and KdExitDebugger to make the patch.

Disabling a trace point follows the same pattern, but in the opposite direction. Just like system call callback unregistration, a variable, this time called KiTpActiveTrapsCount, is used to avoid removing a trace point while it is still active, and all operations are done by holding a lock (KiTpStateLock).

So how are trace points actually triggered? Simple (again, no surprise to kprobe users) – an “INT 3” instruction is what ends up getting patched on top of the existing code at the target address, which will result in an eventual exception to be handled by KiDispatchException. If the status code is STATUS_BREAKPOINT, and KiDynamicTraceMask has Bit 1 set, KiTpHandleTrap is called.

This increments KiTpActiveTrapsCount to protect against racing unregistration, and looks up the address in KiTpHashTable. If there’s a match, it then makes sure that an INT 3 is actually present at the address. In case of a match, the handler checks if this is a first chance exception and if dynamic tracing is enabled. If so, the callback is executed, and its return value is used to determine if the exception should be raised or not. If the function returns FALSE, then KiTpEmulateInstruction is called to emulate the original instruction stream and resume execution. Otherwise, if dynamic tracing is not enabled, or if this is a second chance exception, KiTpWriteMemory is used to restore the original code to avoid any further traps on that address.


Well, there you have it – the latest 19H1 release of Windows should introduce some exciting new functionality for tracing and debugging. Realistically, it’s unlikely that this will ever be exposed for 3rd security product party use (or even to internal competing security tools), but the addition of these capabilities may one day lead to that functionality being exposed in some way (especially the ETW tracing capability). Since there is no publicly shipping host module for the Tracing API Set, it may be that this functionality will only ever internally be used by Microsoft for their own testing, but it would be great to one day see it for 3rd party debugging and tracing as well.

Finally, it’s worth noting that the KiDynamicTraceEnabled variable is protected by the PsKernelRangeList, which is PatchGuard’s internal way of monitoring specific variables and tables outside of its regular set of behaviors, so attackers that try to manipulate this behavior illicitly will likely incur its wrath. Still, since this functionality is meant to be used when a kernel debugger is attached (which disables PatchGuard), it’s certainly possible to build a custom hand-crafted driver that enables and uses this functionality for legitimate purposes inside of say, a sandbox product.

Bringing Call Gates Back

August 7th, 2017


A few months ago, as part of looking through the changes in Windows 10 Anniversary Update for the Windows Internals 7th Edition book, I noticed that the kernel began enforcing usage of the CR4[FSGSBASE] feature (introduced in Intel Ivy Bridge processors, see Section 4.5.3 in the AMD Manuals) in order to allow usage of User Mode Scheduling (UMS).

This led me to further analyze how UMS worked before this processor feature was added – something which I knew a little bit about, but not enough to write on.

What I discovered completely changed my understanding of 64-bit Long Mode semantics and challenged many assumptions I was making – pinging a few other experts, it seems they were as equally surprised as I was (even Mateusz”j00ru” Jurczyk wasn’t aware!).

Throughout this blog post, you’ll see how x64 processors, even when operating in 64-bit long mode:

  • Still support the usage of a Local Descriptor Table (LDT)
  • Still support the usage of Call Gates, using a new descriptor format
  • Still support descriptor-table-based (GDT/LDT) segmentation using the fs/gs segment – ignoring the new MSR-based mechanism that was intended to “replace” it

Plus, we’ll see how x64 Windows still allows user-mode applications to create an LDT (with specific limitations).

At the end of the day, we’ll show that j00ru’s and Gynvael Coldwind’s amazing paper on abusing Descriptor Tables is still relevant, even on x64 systems, on systems up to Windows 10 Anniversary Update. As such, reading that paper should be considered a prerequisite to this post.

Please, take into consideration that all these techniques no longer work on Anniversary Update systems or later, nor will they work on Intel Ivy Bridge processors or later, which is why I am presenting them now. Additionally, there is no “vulnerability” or “zero-day” presented here, so there is no cause for alarm. This is simply an interesting combination of CPU, System, and OS Internals, which on older systems, could’ve been used as a way to gain code execution in Ring 0, in the presence of an already existing vulnerability.

A brief primer on User Mode Scheduling

UMS efficiently allows user-mode processes to switch between multiple “user” threads without involving the kernel – an extension and large improvement of the older “fiber” mechanism. A number of videos on Channel 9 explain how this is done, as does the patent.

One of the key issues that arises, when trying to switch between threads without involving the kernel, is the per-thread register that’s used on x86 systems and x64 systems to point to the TEB. On x86 systems, the FS segment is used, leveraging an entry in the GDT (KGDT_R3_TEB), and on x64, the GS segment is used, leveraging the two Model Specific Registers (MSRs) that AMD implemented: MSR_GS_BASE and MSR_KERNEL_GS_SWAP.

Because UMS would now need to allow switching the base address of this per-thread register from user-mode (as involving a kernel transition would defy the whole point), two problems exist:

  1. On x86 systems, this could be implemented through segmentation, allowing a process to have additional FS segments. But doing so in the GDT would limit the number of UMS threads available on the system (plus cause performance degradation if multiple processes use UMS), while doing so in the LDT would clash with the existing usage of the LDT in the system (such as NTVDM).
  2. On x64 systems, modifying the base address of the GS segment requires modifying the aforementioned MSRs — which is a Ring 0 operation.

It is worth bringing up the fact that fibers never solved this problem –instead having all fibers share a single thread (and TEB). But the whole point of UMS is to provide true thread isolation. So, what can Windows do?

Well, it turns out that close reading of the AMD Manuals (Section 4.8.2) indicate the following:

  • “Segmentation is disabled in 64-bit mode”
  • “Data segments referenced by the FS and GS segment registers receive special treatment in 64-bit mode.”
  • “For these segments, the base address field is not ignored, and a non-zero value can be used in virtual-address calculations.

I can’t begin to count how many times I’ve heard, seen, and myself repeated the first bullet. But that FS/GS can still be used with a data segment, even in 64-bit long mode? This literally brought back memories of Unreal Mode.

Clearly, though, Microsoft was paying attention (did they request this?). As you can probably now guess, UMS leverages this particular feature (which is why it is only available on x64 versions of Windows). As a matter of fact, the kernel creates a Local Descriptor Table as soon as one UMS thread is present in the process.

This was my second surprise, as I had no idea LDTs were still something supported when executing native 64-bit code (i.e.: ‘long mode’). But they still are, and so adding in the TABLE_INDICATOR (TI) bit (0x4) in a segment will result in the processor reading the LDTR to recover the LDT base address and dereference the segment indicated by the other bits.

Let’s see how we can get our own LDT for a process.

Local Descriptor Table on x64

Unlike the x86 NtSetLdtEntries API and the ProcessLdtInformation information class, the x64 Windows kernel does not provide a mechanism for arbitrary user-mode applications to create an LDT. In fact, these APIs all return STATUS_NOT_SUPPORTED.

That being said, by calling the user-mode API EnterUmsSchedulingMode, which basically calls NtSetInformationThread with the ThreadUmsInformation class, the kernel will go through the creation of an LDT (KeInitializeProcessLdt).

This, in turn, will populate the following fields in KPROCESS:

  1. LdtFreeSelectorHint which indicates the first free selector index in the LDT
  2. LdtTableLength which stores the total number of LDT entries – this is hardcoded to 8192, revealing the fact that a static 64K LDT is allocated
  3. LdtSystemDescriptor which stores the LDT entry that will be stored in the GDT
  4. LdtBaseAddress which stores a pointer to the LDT of this process
  5. LdtProcessLock which is a FAST_MUTEX used to synchronize changes to the LDT

Finally, a DPC is sent to all processors which loads the LDT into all the processors.

This is done by reading the KPROCESS->LdtSystemDescriptor and writing into the GDT at offset 0x60 on Windows 10, or offset 0x70 on Windows 8.1 (bonus round: we’ll see why there’s a difference a bit later).

Then, the LLDT instruction is used, and the selector is stored in the KPRCB->LdtSelector field. At this point, the process has an LDT. The next step is to fill it out.

The function now reads the address of the TEB. If the TEB happens to fall in the 32-bit portion of the address space (i.e.: than 0xFFFFFF000), it is set as the base address of a new segment in the LDT (using LdtFreeSelectorHint to choose which selector – in this case, 0x00), and the TebMappedLowVa field in KTHREAD replicates the real TEB address.

On the other hand, if the TEB address is above 4GB, Windows 8.1 and earlier will transform the private allocation holding the TEB into a shared mapping (using a prototype PTE) and re-allocate a second copy at the first available top-down address available (which would usually be 0xFFFFE000). Then, TebMappedLowVa will have this re-mapped address below 4GB.

Additionally, the VAD, which remains “private” (and this will not show up as a truly shared allocation) will be marked as NoChange, and further will have the VadFlags.Teb field set to indicate it is a special allocation. This prevents any changes to be made to this address through calls such as VirtualProtect.

Why this 4GB limitation and re-mapping? How does an LDT help here? Well, it turns out that the AMD64 manuals are pretty clear about the fact that the mov gs, XXX and pop gs instructions:

  • Wipe the upper 32-bit address of the GS base address shadow register
  • Load the lower 32-bit address of the GS base address shadow register with the contents of the descriptor table entry at the given selector

Therefore, x86-style segmentation is still fully supported when it comes to FS and GS, even when operating in long mode, and overrides the 64-bit base address stored in MSR_GS_BASE. However, because there is no 64-bit data segment descriptor table entry, only a 32-bit base address can be used, requiring this complex remapping done by the kernel.

On Windows 10, however, this functionality is not present, and instead, the kernel checks for presence of the FSGSBASE CPU feature. If the feature is present, an LDT is not created at all, and instead the fact that user-mode applications can use the WRGSBASE and RDGSBASE instructions is leveraged to avoid having to re-map a < 4GB TEB.  On the other hand, if the CPU feature is not available, as long as the real TEB ends up below 4GB, an LDT will still be used.

A further, and final change, occurs in Anniversary Update, where the LDT functionality is completely removed – even if the TEB is below 4GB, FSGSBASE is enforced for UMS availability.

Lastly, during every context switch, if the KPROCESS of the newly scheduled thread contains an LDT base address that’s different than the one currently loaded in the GDT, the new LDT base address is loaded in the GDT, and the LDT selector is loaded against (hardcoded from 0x60 or 0x70 again).

Note that if the new KPROCESS does not have an LDT, the LDT entry in the GDT is not deleted – therefore the GDT will always have an LDT entry now that at least one UMS thread in a process has been created, as can be seen in this debugger output:

lkd> $$>a< c:\class\dumpgdt.wds 70 70
                                                    P Si Gr Pr Lo
Sel        Base              Limit          Type    l ze an es ng
---- ----------------- ----------------- ---------- - -- -- -- --
0070 ffffe000`2037d000 00000000`0000ffff LDT        0 Nb By P  Nl

You can see how this matches the LDT descriptor of “UMS Test” application:

lkd> dt nt!_KPROCESS ffffe0002143e080 Ldt*
+0x26c LdtFreeSelectorHint : 1
+0x26e LdtTableLength : 0x2000
+0x270 LdtSystemDescriptor : _KGDTENTRY64
+0x280 LdtBaseAddress : 0xffffe000`2037d000 Void
lkd> dx ((nt!_KGDTENTRY64 *)0xffffe0002143e2f0)
[+0x000] LimitLow : 0xffff [Type: unsigned short]
[+0x002] BaseLow : 0xd000 [Type: unsigned short]
[+0x004] Bytes [Type: ]
[+0x004] Bits [Type: ]
[+0x008] BaseUpper : 0xffffe000 [Type: unsigned long]
[+0x00c] MustBeZero : 0x0 [Type: unsigned long]

Call Gates on x64

Call gates are a mechanism which allows 16-bit and 32-bit legacy applications to go from a lower privilege level to a higher privilege level. Although Windows NT never used such call gates internally, a number of poorly written AV software did, a few emulators, as well as exploits, both on 9x and NT systems, because of the easy way they allowed someone with access to physical memory (or with a Write-What-Where vulnerability in virtual memory) to create a backdoor way to elevate privileges.

With the advent of Supervisor Mode Execution Prevention (SMEP), however, this technique seems to have fallen out of fashion. Additionally, on x64 systems, since Call Gates are expected to be inserted into the Global Descriptor Table (GDT), which PatchGuard is known to protect, the technique is even further degraded. On top of that, most people (myself included) assumed that AMD had simply removed this oft-unused feature completely from the x64 architecture.

Yet, interestingly, AMD did go through the trouble of re-defining a new x64 long mode call gate descriptor format, removing the legacy “parameter count”, and extending it to a 16-byte format to make room for a 64-bit offset, as shown below:

That means that if a call gate were to find itself into a descriptor table, the processor would still support the usage of a far call or far jmp in order to reference a call gate descriptor and change CS:RIP to a new location!

Exploit Technique: Finding the LDT

First, although SMEP makes a Ring 3 RIP unusable for the purposes of getting Ring 0 execution, setting the Target Offset of a 64-bit Call Gate to a stack pivot instruction, then RET’ing into a disable-SMEP gadget will allow Ring 0 code execution to continue.

Obviously, HyperGuard now prevents this behavior, but HyperGuard was only added in Anniversary Update, which disables usage of the LDT anyway.

This means that the ability to install a 64-bit Call Gate is still a viable technique for getting controlled execution with Ring 0 privileges.

That being said, if the GDT is protected by PatchGuard, then it means that inserting a call gate is not really viable – there’s a chance that it may be detected as soon as its inserted, and even an attempt to clean-up the call gate after using it might come too late. When trying to implement a stable, persistent, exploit technique, it’s best to avoid things which PatchGuard will detect.

On the other hand, now we know that x64 processors still support using an LDT, and that Windows leverages this when implementing UMS. Additionally, since arbitrary processes can have arbitrary LDTs, PatchGuard does not guard individual process’ LDT entries, unlike the GDT.

That still leaves the question of how do we find the LDT of the current process, once we’ve enabled UMS? Well, given that the LDT is a static, 64KB allocation, from non-paged pool, this does still leave us with an option. As explained a few years ago on my post about the Big Pool, such a large allocation will be easily enumerable from user-mode as long as its tag is known:

lkd> !pool ffffe000`22f3b000
Pool page ffffe00022f3b000 region is Nonpaged pool
*ffffe00022f3b000 : large allocation, tag kLDT, size 0x10000 bytes

While this is a nice information leak even on Windows 10, a mitigation comes into play unfortunately in Windows 8.1: Low IL processes can no longer use the API I described, meaning that the LDT address can only be leaked (without an existing Ring 0 arbitrary read/infoleak vulnerability) at Medium IL or higher.

Given that this is a fairly large size allocation, however, it means that if a controlled 64KB allocation can be made in non-paged pool and its address leaked from Low IL, one can still guess the LDT address. Ways for doing so are left as an exercise to the reader 🙂

Alternatively, if an arbitrary read vulnerability is available to the attacker, the LDT address is easily retrievable from the KPROCESS structure by reading the LdtBaseAddress field or by computing it from the LdtSystemDescriptor field. Getting the KPROCESS is easy through a variety of undocumented APIs, although these are now also blocked on Windows 8.1 from Low IL.

Therefore, another common technique is to use a GDI or User object which has an owner such a tagTHREADINFO, which then points to ETHREAD (which then points to EPROCESS). Alternatively, one could retrieve the GDT base address from the KPCR’s GdtBase field, if a way of leaking the KPCR is available, and then read the segment base address at offset 0x60 or 0x70. The myriad ways of leaking pointers and bypassing KASLR, even from Low IL, is beyond (beneath?) the content of this post.

Exploit Technique: Building a Call Gate

The next step is to now write a call gate in one of the selectors present in the LDT. By default, if this is the initial scheduler thread, we expect to find its TEB. Indeed, on this sample Windows 8.1 VM, we can see the re-mapped TEB at 0xFFFFE000:

lkd> dq 0xffffe000`2037d000
ffffe000`2037d000 fffff3ff`e0001820
lkd> dt nt!_KGDTENTRY64 ffffe000`2037d000 -b
+0x000 LimitLow : 0x1820
+0x002 BaseLow : 0xe000
+0x004 Bytes :
+0x000 BaseMiddle : 0xff ''
+0x001 Flags1 : 0xf3 ''
+0x002 Flags2 : 0xff ''
+0x003 BaseHigh : 0xff ''
+0x004 Bits :
+0x000 BaseMiddle : 0y11111111 (0xff)
+0x000 Type : 0y10011 (0x13)
+0x000 Dpl : 0y11
+0x000 Present : 0y1
+0x000 LimitHigh : 0y1111
+0x000 System : 0y1
+0x000 LongMode : 0y1
+0x000 DefaultBig : 0y1
+0x000 Granularity : 0y1
+0x000 BaseHigh : 0y11111111
+0x008 BaseUpper : 0
+0x00c MustBeZero : 0

Converting this data segment into a call gate can be achieved by merely converting the type from 0x13 (User Data Segment, R/W, Accessed) to 0x0C (System Segment, Call Gate).

However, doing so will now create a call gate with the following CS:[RIP] => E000:00000000FFFF1820

We have thus two problems:

  1. 0xE000 is not a valid segment
  2. 0xFFFF1820 is a user-mode address, which will cause a SMEP violation on most modern systems.

The first problem is not easy to solve – while we could create thousands of UMS threads, causing 0xE000 to become a valid segment (which we’d then convert into a Ring 0 Code Segment), this would be segment 0xE004. And if one can change 0xE000, might as well avoid the trouble, and set it to its correct value – (KGDT64_R0_CODE) 0x10, from the get go.

The second problem can be fixed in a few ways.

  1. An arbitrary write can be used to set BaseUpper, BaseHigh, LimitHigh, Flags2, and LimitLow (which make up the 64-bits of Code Offset) to the desired Ring 0 RIP that contains a stack pivot or some other interesting instruction or gadget.
  2. Or, an arbitrary write to modify the PTE to make it Ring 0, since the PTE base address is not randomized on the Windows versions vulnerable to an LDT-based attack.
  3. Lastly, if one is only interested in SYSTEM->Ring 0 escalation, systems prior to Windows 10 can be attacked through the AWE-based attack I described at Infiltrate 2015, which will allow the creation of an executable Ring 0 page.

It is also worth mentioning that since Windows 7 has all of non-paged pool marked as executable, and the LDT is itself a 64KB non-paged pool allocation, it is made up of entirely executable pages, so an arbitrary write could be used to set the Call Gate offset to somewhere within the LDT allocation itself.

Exploit Technique: Writing the Ring 0 Payload

Writing x64 Ring 0 payload code is a lot harder than x86.

For starters, the GS segment must be immediately set to its correct value, else a triple fault could occur. This is done through the swapgs instruction.

Next, it’s important to realize that a call gate sets the stack segment selector (SS) to 0. While x64 natively operates in this fashion, Windows expects SS to be KGDT64_R0_DATA, or 0x18, and it may be a good idea to respect that.

Additionally, note that the value to which RSP will be set to is equal to the TSS’s Rsp0, normally used for interrupts, while a typical system call would use the KPRCB’s RspBase field. These ought to be in sync, but keep in mind that a call gate does not disable interrupts automatically, unlike an interrupt gate.

A reliable exploit must take note of all these details to avoid crashing the machine.

Further, exiting from a call gate must be done with the ‘far return’ instruction. Once again, another caveat applies: some assemblers may not generate a true 64-bit far return (i.e.: lacking a rex.w prefix), which will incorrectly pop 32-bit data from the stack. Make sure that a ‘retfq’ or ‘retfl’ or ‘rex.w retf’ is generated instead.

Exploit Techniques Bonus: Corrupting the LDT Address, Hidden Segment, Lazy GDT Clear

Note that we’ve gone through some difficulty in obtaining the address of the LDT, and describing the ways in which the UMS TEB entries could be corrupted in a way to convert them to Call Gate entries, it’s useful to mention that perhaps a much easier (depending on the attack parameters and vulnerability) technique is to just overwrite the LdtSystemDescriptor field in EPROCESS (something which j00ru’s x86-based paper also pointed out).

That’s because, at the next context switch, the GDT will automatically be updated a copy of this descriptor, which could be set to a user-mode base address (due to a lack of SMAP in the OS), avoiding the need to either patch the GDT (and locating it — which is hard when Hyper-V’s NPIEP feature is enabled) or modifying the true kernel LDT (and leaking its address).

Indeed, for this to work, a single 32-bit (in fact, even less) arbitrary write is required, which must, at minimum, set the fields:

  • P to 1 (Making the segment present)
  • Type to 2 (Setting the segment as an LDT entry)
  • BaseMid to 1 (Setting the base to 0x10000, as an example, as addresses below this are no longer allowed)

Therefore, a write of 0x00008201, for example, is sufficient to achieve the desired result of setting this process’ LDT to 0x10000.

As soon as a context switch occurs back to the process, the GDT will have this LDT segment descriptor loaded:

lkd> $$>a< c:\class\dumpgdt.wds 70 70
                                                    P Si Gr Pr Lo
Sel        Base              Limit          Type    l ze an es ng
---- ----------------- ----------------- ---------- - -- -- -- --
0070 00000000`00010000 00000000`00000000 LDT        0 Nb By P  Nl

But wait – isn’t setting a limit of 0 creating an empty LDT? Not to worry! In long mode, limits on LDT descriptor entries are completely ignored… unfortunately though, although this is what the AMD64 manual states, I get access violations, at least on Hyper-V x64, if the limit is not large enough to contain the segment. So your mileage my vary.

But that’s all right – we can still limit this to a simple 4-byte overwrite! The trick lies into simply going through the process of creating a real LDT in the first place, then leaking its address (as described). Following that, allocate the user-mode fake LDT at the same lower 32-bit address, keeping the upper 32-bits zeroed. Then, use the 4-byte overwrite to clear the KPROCESS’ LdtSystemDescriptor’s BaseUpper field.

Even if the kernel LDT address cannot be leaked for some reason, one can easily “guess” every possibility (knowing it will be page aligned) and spray the entire 32-bit address space. This sounds like a lot, but is really only about a million allocations.

Finally, an alternate technique is to leverage exception handling: if the wrong LDT is overwritten, the kernel won’t crash when loading the invalid LDT segment (as long as it’s canonical, the PTE isn’t checked for validity). Instead, only when the exploit attempts to use the call gate, will a GPF be generated, and only in the context of the Ring 3 application. As such, one can progressively try each possible lower 32-bit LDT address until a GPF is no longer issued. Voila: we have found the correct lower 32-bits.

As another bonus, why is it that the selector for the LDT is 0x70 on Windows 8.1 and earlier, but 0x60 on Windows 10?

The answer lies in an even lesser known fact: up until the latter, the kernel created a Ring 0 Compatibility Mode Segment at offset 0x60! This means that a sneaky attacker can set CS to 0x60 and enjoy a weird combination of 32-bit legacy code execution with Ring 0 privileges (a number of caveats apply, including what an interrupt would do when returning, and the fact that no kernel API could be used at all).

Finally, note that even once a UMS-leveraging process exists, the GDT entry is not cleared, and points to a freed pool allocation. This means that if a way to allocate 64KB of controlled non-paged pool memory is known (such as some of the ways described in my Big Pool blog post), the GDT entry could be made to point to controlled memory (such as a named pipe buffer) which will re-use the same pointer. Then, some way to make the system continue to trust this address/entry should be achieved (either by causing an LLDT of 0x60/0x70 to be issued or having an EPROCESS’s LdtSystemDescriptor field re-use this address).

This is more of an anti-forensics technique than anything, because it keeps the GDT pointing to a kernel-mode LDT, even though it’s attacker controlled.


While I won’t be releasing sample code leveraging this attack, it could easily be added to the various PowerShell-based “Vulnerable Driver” techniques that @b33f has been creating.

Here’s a sample screenshot of the attack based on a C program, with me using the debugger to perform the 32-bit arbitrary write (vs. sending an IOCTL to a vulnerable driver).

It sits in a loop (after leaking and allocating the data that it needs) and attempts to execute the call gate every second, until the arbitrary write is performed.

Once successful, the Ring 0 payload merely reads SharedUserData->SystemTime (every second).

Conclusion: Windows 10 Anniversary Update Mitigations

In Windows 10 Anniversary Update (“Redstone 1”), a number of changes make these exploit techniques impossible to use:

  • All of the LDT-related fields and code in the kernel is removed. There is now no way of having an LDT through any Windows-supported mechanism.
  • PatchGuard now checks the LDTR register. If it’s non-zero, it crashes the system.

MSRC and the various security teams at Microsoft deserve kudos for thinking about — and plugging — the attack vector that LDTs provided, which is certainly not a coincidence 🙂

Further, the following generic mitigations make classes of such attacks much harder to exploit:

  • Randomization of the PTE base make it harder to bypass SMEP by making Ring 3 memory appear as Ring 0.
  • Technologies such as KCFG make it even harder to exploit control over arbitrary CS:RIP.

Finally, as described earlier, even on Windows 8.1, if the FSGSBASE feature is available in KeFeatureBits, the kernel will not allow the creation of an LDT, nor will it load the LDT during a context switch. You can easily verify this by calling (Ex/Rtl)IsProcessorFeaturePresent(PF_RDWRFSGSBASE_AVAILABLE).

Windows Internals, 7th Edition!

May 11th, 2017

What am I up to?

Long-time readers of this blog are probably aware that updates have been rare in the past few years, although I do try to keep time for some interesting articles from time to time. Most of my public research lately has been done through the Infosec Conference Circuit, so if you were not already aware, you can download slides from all my talks at the following URL:


Additionally, I have a number of presentations I am looking forward to giving this year, including:

I also have a number of interesting design flaws I discovered this year in various Windows components —  as these get patched (they are not Tavis-worthy wormable RCEs, not to worry), I have been mulling over a “Windows Design Flaw Garage Sale” talk similar to the famous one that Stefan Esser (i0nic) gave a few years ago about Apple/iOS — covering some past bugs (fixed and unfixed) and more recent ones.

However, this post is not about such small research updates — but rather about a much bigger piece of work that has taken up my time these last 12 months — the release of Windows Internals, 7th Edition (Part 1)!

Windows Internals, 7th Edition

Windows Internals, 7th Edition

Some history…

After the release of the 6th Edition of the book, which covered Windows 7, it’s fair to say that I was pretty burned out. The book incurred heavy delays due to my juggling of college, internships, and various relationships, while also requiring a massive amount of work due to the ambitious new sections, and coverage of the many, many changes that Windows 7 brought to the table (either fine-tuning many small things from Vista, or completely new kernel modules). Additionally, my co-authors also had new plans: David Solomon went on to retire and sunset his training business (David Solomon Expert Seminars), and Mark Russinovich was fully committed to his new role at Microsoft which eventually took him to Azure, where he is now the Chief Technology Officer (CTO), and kicking some major cloud/fabric butt with his extensive OS experience and security background. All of this to say — there was not much of an appetite to immediately begin writing a new book, with Windows 8 looming on the horizon (at that point still called Windows Blue).

Something else happened at that time: under leadership from Satya Nadella, Microsoft began delivering on its “Windows as a Service (WaaS)” model, furiously releasing a Windows 8.1 Update within a year of Windows 8 having shipped. Given that a single OS update had taken us years to cover, this release cycle was simply too rapid to successfully think about releasing a book in a timely fashion. I stopped thinking that a new edition of the book would ever be released, and I certainly didn’t think I’d be able to do one.

All gaps create opportunities, and two other authors decided that they could take on the 7th Edition and ship a successful update. They re-arranged the book in three parts, instead of two, with the first one focusing on Windows 8 User-Mode Metro (now UWP) Application Development, the second one on the Kernel, and the third one on Driver Development. I was not contacted or involved in these changes, and honestly, was not too happy about them. There are excellent driver programming books, just as there are application development books (even on Metro/UWP). This felt, to me, like an attempt to significantly cut down on the kernel portions of the book, and monetize on the Metro/Driver programming books, which obviously have a  much wider audience.

Additionally, with Windows 8 having shipped, Part 1 was slated for that year, with Part 2 (Windows 8.1 would now be out) the year after, and finally, Part 3, a year after that (Windows 10 would now be in beta). By the time you’d get to the last part, the OS would’ve already moved two releases further — or, each part could cover that OS. Becoming a Windows 8 Metro App Development book, with Windows 8.1 Kernel Internals book, and Windows 10 Driver Development book. These were just my personal thoughts at the time — which I kept to myself, because every author needs a chance to be successful, and others may well have liked this model, and the book may have sold more copies than all previous combined – who was I to judge?

One year passed. Then another… then another. By now, given that my name was still on the cover — regardless of my lack of involvement — many people would come to me and ask me “What’s going on? Why are you taking so long? Do you need help?” on the friendly side… and of course, some not-so-friendly comments, from people that had pre-ordered on Day 1, paying anywhere between $30-90, and receiving nothing 3 years later, with an ever-delayed release date. I strongly considered putting out a statement that I had nothing to do with this book — but chose to simply ask Microsoft Press to remove my name from the cover and all marketing materials. I preferred losing my association with this Bible, rather than be responsible for its contents, and its delays.

A new hope

Around the time that I did that, however, I realized that yet –another– name had been added to the pool! It was that of Pavel Yosifovich, a Microsoft MVP whose blog I had followed a few times, and whom I had heard about doing some Windows Internals training in the past, mostly in Israel. I thought highly of Pavel — and he was an established author of previous books. Additionally, he now had a Microsoft e-mail address — suggesting that once again, the series would have a real “internal” presence, who would communicate with the developer team, read source code comments, and more — while Mark and I had only, and solely, been reverse engineering, we had always had help from David’s connections and insight into the developer team, which the new books would’ve lacked.

So I reached out, and to my pleasure, found out that Pavel had now become the sole co-author, the previous two having completely abandoned the project with no materials to show for it. Pavel was doing a herculean task of updating the entire book to now only cover Windows 8 and 8.1, but of course Windows 10 as well, which had reached its Threshold 2 (1511) Update, with Redstone 1 (1607) currently shipping to the Windows Insider Program (WIP). While having source access helps, this is still a task that I knew a single person would struggle with — and I really wanted the book to succeed for all of those that had placed their faith in it. I had also, over the last few years, had made lots of Windows reverse engineering, as many of you know, covering large parts of new Windows 8 and later components. This meant significantly reduced research time for me — all while having an amazing co-author. It seemed obvious that I should jump into the deep abyss of Windows Internals once again.

Pavel was extremely gracious in accepting an uninvited guest to the party, allowing me to make many changes to chapters that he had already completed (I don’t know if I would’ve done the same!). This started adding delays to the book, and Redstone 1 was about to ship — we decided to update the book to cover Redstone 1 from now on, and to go back to any places we knew there were changes. As we kept writing, I came up with new ideas and changes to the book — moving some things around, adding new kernel components, expanding on experiments, and the scope continued to increase. It was clear that I was once again, going to cause delays, which deeply bothered me.

Yet, Pavel was always there to pick up the slack, go beyond the call of duty, and spend nights on researching components as well as the more mundane parts of a book (screenshots and graphics). I could not have asked for a more humble host inside the world of his book. As we were wrapping up, I realized that Redstone 2 (1703) was nearing its feature complete date (around January of this year). I made yet another potentially delaying decision to go back, once again, and to hurriedly find any places where I knew changes had been made, and to update as much of the book as I could. I saw an opportunity — to release a Windows Internals book within weeks of a Windows release, covering that Windows release. A feat which had not happened in many, many releases.

And so, here we are today, a little over a month since Windows 10 Creators Update — Redstone 2 — 1703 has shipped, with the update slowly rolling out over the month of April to hundreds of millions of users, with Build 2017 right around the corner, and with a Windows Internals book in the midst of it all, covering the very same operating system. While I apologize for the additional six months this has cost your pre-orders, I do believe it was the right call.

What’s new in the book? What’s changed?

One of the first things that Pavel had changed (other than returning the book to its usual two-part focus on the kernel and related system components) is to better organize key Windows concepts into the first part of the book, instead of having them spread out over both parts — this way, people could get what would likely be 80% of the material that is relevant to 90% of people as soon as the first part was released, instead of having to wait for both. This meant making the following changes:

  1. Moving Memory Management and I/O Manager to Part 1
  2. Breaking out Processes, Jobs, and Thread Scheduling in two chapters — Processes & Jobs, and Thread Scheduling
  3. Moving System & Management Mechanisms to Part 2
  4. Removing Networking from Part 1

Once I joined, it made sense, with this new flow, to make a few additional changes:

  1. Processes and Jobs, now being its own chapter, became Processes, Jobs and Silos, which is the internal name for Windows Server Containers as well as Centennial/Desktop Bridge containers.
  2. It made little sense that we were covering the User-Mode Loader (a section I first added in the 5th Edition) as a System Mechanism, instead of an integral part of the Process section (which made constant references to Part 2). I moved this section to be part of the same chapter.

Outside of these broad strokes, a full list of all the changes would obviously be too complex. I would estimate the sheer amount of new pages to be around 150 — with probably 50 other pages that have received heavy modification and/or updating. You can definitely expect coverage of the following new features:

  • Auto Boost [Scheduling]
  • Directed Switch [Scheduling]
  • Memory Partitions [Memory]
  • Priority Donation/Inheritance [Scheduling]
  • Security/Process Mitigations [Security]
  • CPU Sets [Scheduling]
  • Windows Containers [Processes]
  • Store Manager [Memory]
  • API Sets [Processes]
  • AppContainer [Security]
  • Token Attributes & Claims [Security]
  • Protected Process Light [Security / Processes]
  • Windows Subsystem for Linux [Architecture]
  • Memory Compression [Memory]
  • Virtual Trust Levels [Architecture]
  • Device Guard & Credential Guard [Security]
  • Processor Enclaves [Memory]
  • Secure Kernel Mode / Isolated User Mode [Architecture]
  • Pico Processes [Processes]
  • Power Management Framework (PoFx) [I/O Manager]
  • Power Availability Requests [I/O Manager]
  • And a lot more

Thank You!

Finally, I’d like to thank many people, inside and outside of Microsoft, that helped with some of the content, ideas, experiments, etc. Especially Andrea Allievi, who helped with some very hairy parts of the Memory Management section!

I know both Pavel and I hope you’ll enjoy this flow a bit better, and that you’ll have lots of reading to do in this new Edition. Feel free to hit me up at @aionescu as usual.

Owning the Image Object File Format, the Compiler Toolchain, and the Operating System: Solving Intractable Performance Problems Through Vertical Engineering

May 6th, 2016

Closing Down Another Attack Vector

As the Windows kernel continues to pursue in its quest for ever-stronger security features and exploit mitigations, the existence of fixed addresses in memory continues to undermine the advances in this area, as attackers can use data corruption vulnerabilities and combine these with stack and instruction pointer control in order to bypass SMEP, DEP, and countless other architectural defense-in-depth techniques. In some cases, entire mitigations (such as CFG) are undone due to their reliance on a single, well-known static address.

In the latest builds of Windows 10 Redstone 1, aka “Anniversary Update”, the kernel takes a much stronger toward Kernel Address Space Layout Randomization (KASLR), employing an arsenal of tools that can only be available to an operating system developer that also happens to own the world’s most commercially successful compiler, and the world’s most pervasive executable object image format.

The Page Table Entry Array

One of the most unique aspects of the Windows kernel is the reliance on a fixed kernel address to represent the virtual base address of an array of page table entries that describes the entire virtual address space, and the usage of a self-referencing entry which acts as a pivot describing the page directory for the space itself (and, on x64 systems, describing the page directory table itself, and the page map level 4 itself).

This elegant solutions allows instant O(1) translation of any virtual address to its corresponding PTE, and with the correct shifts and base addresses, a conversion into the corresponding PDE (and PPE/PXE on x64 systems). For example, the function MmGetPhysicalAddress only needs to work as follows:

MmGetPhysicalAddress (
    _In_ PVOID Address
    MMPTE TempPte;
    /* Check if the PXE/PPE/PDE is valid */
    if (
#if (_MI_PAGING_LEVELS == 4)
       (MiAddressToPxe(Address)->u.Hard.Valid) &&
#if (_MI_PAGING_LEVELS >= 3)
       (MiAddressToPpe(Address)->u.Hard.Valid) &&
       /* Check if the PTE is valid */
       TempPte = *MiAddressToPte(Address);

Each iteration of the MMU table walk uses simple MiAddressTo macros such as the one below, which in turn rely on hard-code static addresses.

/* Convert an address to a corresponding PTE */
#define MiAddressToPte(x) \
   ((PMMPTE)(((((ULONG)(x)) >> 12) << 2) + PTE_BASE))

As attackers have figured out, however, this “elegance” has notable security implications. For example, if a write-what-where is mitigated by the existence of a read-only page (which, in Linux, would often imply requiring the WP bit to be disabled in CR0), a Windows attacker can simply direct the write-what-where attack toward the pre-computed PTE address in order to disable the WriteProtect bit, and then follow that by the actual write-what-where on the data.

Similarly, if an exploit is countered by SMEP, which causes an access violation when a Ring 0 Code Segment’s Instruction Pointer (CS:RIP) points to a Ring 3 PTE, the exploit can simply use a write-what-where (if one exists), or ROP (if the stack can be controlled), in order to mark the target user-mode PTE containing malicious code, as a Ring 0 page.

Other PTE-based attacks are also possible, such as by using write-what-where vulnerabilities to redirect a PTE to a different physical address which is controlled by the attacker (undocumented APIs available to Administrators will leak the physical address space of the OS, and some physical addresses are also leaked in the registry or CPU registers).

Ultimately, the list goes on and on, and many excellent papers exist on the topic. It’s clear that Microsoft needed to address this limitation of the operating system (or clever optimization, as some would call it). Unfortunately, a number of obstacles exist:

  • Using virtual-mapped tables based on the EPROCESS structure (as Linux and OS X do) causes significant performance impact, as pointer chasing the different tables now causes cache misses and page translations. This becomes even worse when thinking about multi-processor systems, and the cache waste that this causes (where the TLB may end up getting filled with the various global (locked) pages corresponding to the page tables of various processes, instead of only the current process).
  • Changing the address of the PTE array has a number of compatibility concerns, as PTE_BASE is actually documented in ntddk.h, part of the Windows Driver Kit. Additionally, once the new address is discovered, attackers can simply adjust their exploits to use the appropriate static address based on the version of the operating system.
  • Randomizing the address of the PTE array means that Windows memory manager functions can no longer use a static constant or preprocessor definition for the address, but must instead access a global address which contains it. Forcing every processor to dereference a single global address every single time a virtual memory operation (allocation, protection, page walk, fault, etc…) is performed is a significantly negative performance hit, especially on multi-socket, NUMA systems.
  • Dealing with the global variable problem above by creating cache-aligned copies of the address in a per-processor structure causes a waste of precious kernel storage space (for example, assuming a 64-byte cache line and 640 processors, 40KB of physical memory are used to replicate the variable most efficiently). However, on NUMA systems, one would also want the page containing this data to be local to the node, so we might imagine an overhead of 4KB per socket. In practice, this wouldn’t be quite as bad, as Windows already has a per-NUMA-node-allocated, per-processor, cache-aligned list of critical kernel variables: the Kernel Processor Region Control Block (KPRCB).

In a normal world, the final bullet would probably be the most efficient solution: sacrificing what is today a modest amount of physical memory (or re-using such a structure) for dealing with effects of global access. Yet, locating this per-processor data would still not be cheap: most operating systems access such a structure by relying on a segment register such as FS or GS on x86 and x64 systems, or use special CPU registers such as those located on CP15 inside of ARM processors. At the very least, this causes more pointer dereferences and potentially complex microcode accesses. But if we own the compiler and the output format, can’t we think outside the box?

Dynamic Relocation Generation

When the Portable Executable (PE) file format was created, its designers realized an important issue: if compiled code made absolute references to data or functions, these hardcoded pointer values might become invalid if the operating system loaded the executable binary at a different base address than its preferred address. Originally a corner case, the advent of user-mode ASLR made this a common occurrence and new reality.

In order to deal with such rebasing operations, the PE format includes the definition of a special data directory entry called the Relocation Table Directory (IMAGE_DIRECTORY_ENTRY_BASERELOC). In turn, this directory includes a number of tables, each of which is an array of entries. Each entry ultimately describes the offset of a piece of code that is accessing an absolute virtual address, and the required adjustment that is needed to fixup the address. On a modern x64 binary, the only possible fixup is an absolute delta (increment or decrement), but more exotic architectures such as MIPS and ARM had different adjustments based on how absolute addresses were encoded on such processors).

These relocations work great to adjust hardcoded virtual addresses that correspond to code or data within the image itself – but if there is a hard-coded access to 0xC0000000, an address which the compiler has no understanding of, and which is not part of the image, it can’t possibly emit relocations for it – this is a meaningless data dereference. But what if it could?

In such an implementation, all accesses to a particular magic hardcoded address could be described as such to the compiler, which could then work with the linker to generate a similar relocation table – but instead of describing addresses within the image, it would describe addresses outside of the image, which, if known and understood by the PE parser, could be adjusted to the new location of the hard-coded external data address. By doing so, compiled code would continue to access what appears to be a single literal value, and no global variable would ever be needed, cancelling out any disadvantages associated with the randomization of this address.

Indeed, the new build of the Microsoft C Compiler, which is expected to ship with Visual Studio 15 (now in preview), address a special annotation that can be associated with constant values that correspond to external virtual addresses. Upon usage of such a constant, the compiler will ensure that accesses are done in a way that does not “break up” the address, but rather causes its absolute value to be expressed in code (i.e.: “mov rax, 0xC0000000”). Then, the linker collects the RVAs of such locations and builds structures of type IMAGE_DYNAMIC_RELOCATION_ENTRY, as shown below:

   DWORD Version;
   DWORD Size;
// IMAGE_DYNAMIC_RELOCATION DynamicRelocations[0];

When all entries have been written in the image, an IMAGE_DYNAMIC_RELOCATION_TABLE structure is written, with the type below:

   PVOID Symbol;
   DWORD BaseRelocSize;
// IMAGE_BASE_RELOCATION BaseRelocations[0];

The RVA of this table is then written into the IMAGE_LOAD_CONFIG_DIRECTORY, which has been extended with the new field DynamicValueRelocTable and whose size has now been increased:

   ULONGLONG DynamicValueRelocTable;         // VA

Now that we know how the compiler and linker work together to generate the data, the next question is who processes it?

Runtime Dynamic Relocation Processing

In the Windows boot architecture, as the kernel is a standard PE file loaded by the boot loader, it is therefore the boot loader’s responsibility to process the import table of the kernel, and load other required dependencies, to generate the security cookie, and to process the static (standard) relocation table. However, the boot loader does not have all the information required by the memory manager in order to randomize the address space as Windows 10 Redstone 1 now does – this remains the purview of the memory manager. Therefore, as far as the boot loader is concerned, the static PTE_BASE address is still the one to use, and indeed, early phases of boot still use this address (and associate PDE/PPE/PXE base addresses and self-referencing entry).

This clearly implies that it is not considered part of a PE loader’s job to process the dynamic relocation table, but rather the job of the component that creates the dynamic address space map, which has now been enlightened with this knowledge. In the most recent builds, this is done by MiRebaseDynamicRelocationRegions, which eventually calls MiPerformDynamicFixups. This routine locates the PE file’s Load Configuration Directory, gets the RVA (now a VA, thanks to relocations done by the boot loader) of the Dynamic Relocation Table, and begins parsing it. At this moment, it only supports version 1 of the table. Then, it loops through each entry, adjusting the absolute address with the required delta to point to the new PTE_BASE address.

It is important to note that the memory manager only calls MiPerformDynamicFixups on the binaries that it knows require such fixups due to the use of PTE_BASE: the kernel (ntoskrnl.exe) and the HAL (hal.dll). As such, this is not (yet) intended as a generic mechanism for all PE files to allow dynamic relocations of hard-coded addresses toward ASLRed regions of memory – but rather a highly vertically integrated feature specifically designed for dealing with the randomization of the PTE array, and the components that have hardcoded dependencies on it.

As such, even if one were to discover the undocumented annotation which allows the new version of the compiler to generate such tables, no component would currently parse such a table.

Sneaky Side Effects

A few interesting details are of note in the implementation. The first is that the initial version of the implementation, which shipped in build 14316, contained a static address in the loader block, which corresponded to the PTE base address that the loader had selected, and was then overwritten by a new fixed PTE base address (0xFFFFFA00`00000000 on x64).

The WDK, which contains the PTE_BASE address for developers to see (but apparently not use!) also contained this new address, and the debugger was updated to support it. This was presumably done to gauge the impact of changing the address in any way – and indeed we can see release notes referring to certain AV products breaking around the time this build was released. I personally noticed this change by disassembling MmGetPhysicalAddress to see if the PTE base had been changed (a normal part of my build analysis).

The next build, 14332, seemingly contained no changes: reverse engineering of the function showed usage of the same address once again. However, as I was playing around with the !pte extension in the debugger, I noticed that a new address was now being used – and that on a separate machine, this address was different again. Staring in IDA/Hex-Rays, I could not understand how this was possible, as MmGetPhysicalAddress was clearly using the same new base as 14316!

It is only once I unassembled the function in WinDBG that I noticed something strange – the base address had been modified to a different value. This led me to the hunt for the dynamic relocation table mechanism. But this is an important point about this implementation – it offers a small amount of “security through obscurity” as a side-effect: attackers or developers attempting to ‘dynamically discover’ the value of the PTE base by analyzing the kernel file on disk will hit a roadblock – they must look at the kernel file in memory, once relocations have been made. Spooky!


It is often said that all software engineering decisions and features lie somewhere between the four quadrants of security, performance, compatibility and functionality. As such, as an example, the only way to increase security without affecting functionality is to impact compatibility and performance. Although randomizing the PTE_BASE does indeed cause potential compatibility issues, we’ve seen here how control of the compiler (and the underlying linked object file) can allow implementers to “cheat” and violate the security quadrant, in a similar way that silicon vendors can often work with operating system vendors in order to create overhead-free security solutions (one major advantage that Apple has, for example).

Closing “Heaven’s Gate”

December 30th, 2015

Brief Overview of WoW64

“Heaven’s Gate” refers to a technique first popularized by the infamous “Roy G. Biv” of 29a fame, and later re-published in Valhalla #1. Cited and improved in various new forms, and even seen in the wild used by the Vawtrak banking malware, it centers around the fact that on a 64-bit Windows OS, seeing as how all kernel-mode components always execute in 64-bit mode, the address space, core OS structures (EPROCESS, PEB, etc…), and code segments for processes are all initially setup for 64-bit “long mode” execution, regardless of the process actually being hosted by a 32-bit executable binary.

In fact, on 64-bit Windows, the first piece of code to execute in *any* process, is always the 64-bit NTDLL, which takes care of initializing the process in user-mode (as a 64-bit process!). It’s only later that the Windows-on-Windows (WoW64) interface takes over, loads a 32-bit NTDLL, and execution begins in 32-bit mode through a far jump to a compatibility code segment. The 64-bit world is never entered again, except whenever the 32-bit code attempts to issue a system call. The 32-bit NTDLL that was loaded, instead of containing the expected SYSENTER instruction, actually contains a series of instructions to jump back into 64-bit mode, so that the system call can be issued with the SYSCALL instruction, and so that parameters can be sent using the x64 ABI, sign-extending as needed.

This process is accurately described in many sources, including in the Windows Internals books, so if you’re interested in reading more, you can do so, but I’ll spare additional details here.

Enter Heaven’s Gate

Heaven’s Gate, then, refers to subverting the fact that a 64-bit NTDLL exists (and a 64-bit heap, PEB and TEB), and manually jumping into the long-mode code segment without having to issue a system call and being subjected to the code flow that WoW64 will attempt to enforce. In other words, it gives one the ability to create “naked” 64-bit code, which will be able to run covertly, including issuing system calls, without the majority of products able to intercept and/or introspect its execution:

  • Microsoft’s EMET, as well as a myriad of similar tools and sandboxes, only hook/protect the 32-bit NTDLL for WoW64 processes, under the assumption that the 64-bit NTDLL can’t be reached in any other way. The mitigations can therefore be bypassed using Heaven’s Gate. The same technique has been used by the Phenom malware to bypass AV solutions.
  • When debugging a 32-bit application with a 64-bit debugger (such as WinDBG), you will initially see the 64-bit state (heap, stack, NTDLL, TEB, etc…). Since this state is uninteresting, as it only contains the WoW64 system call layer, manual commands and extensions must be used to investigate the 32-bit state instead — and so in order to avoid this, even Microsoft often recommends using the 32-bit WinDBG instead, which will provide a much more seamless debugging experience and show the 32-bit state of the process. Other 3rd party debuggers, which are 32-bit only, will also behave the same way. The problem, therefore, is that by using Heaven’s Gate, there IS now interesting 64-bit state, that these debuggers will miss.
  • Many emulation/detonation engines will, upon seeing a 32-bit executable, emulate it using x86 instructions. They will either ignore or be unable to handle x64 instructions, as they never expect them to run. In fact, this was recently shown by a blog post over at Hexacorn. Heaven’s Gate allows such x64 instructions to run, rendering the x86 code into “dummy” code for misdirection purposes.

Memory Restrictions

These and other “benefits” make Heaven’s Gate a tool of choice for malicious code.  However, there always existed an interesting limitation in 32-bit applications running under WoW64: even when executing in 64-bit long-mode, addresses above the 4 GB could never be allocated (in fact, addresses above 2 GB could normally never be used for compatibility purposes, unless the image was linked with /LARGEADDRESSAWARE — the switch was originally designed to support /3GB x86 server environments, but outgrew its original intent to allow full 4 GB addresses under WoW64, a fact leveraged by many 32-bit games and browsers even today).

Using a kernel debugger and the !vad command, it’s simple to see why, such as on this Windows 7 system, where I’ve typed the command before the process has any chance of executing even a single instruction — not even NTDLL has loaded here, folks. This is an interesting view of what are the “earliest” memory structures you can find in a WoW64 process (at least on Windows 7).


Note that a giant VAD at the end, highlighted in teal, occupies the entire 64-bit portion of the address space. Let’s see what !vad has to say about it:


Seeing as how it’s configured as a “NoChange” and “OneSecured” VAD, it cannot be freed or modified in any way. This is further confirmed by the commit charge of -1.

On Windows 8 and later, however, the output changes, as you can see below. Note that I’ve re-used the same colors as in the Windows 7 output for clarity (and the uncolored VADs correspond to the CFG entries).


The 64-bit NTDLL is actually loaded in 64-bit address space now! And we have not one, but two teal-colored VADs, which surround it, re-creating the “no man’s land” just as on Windows 7 and earlier. This change was briefly mentioned, I believe, by Matt Miller (of skape fame) at one of Microsoft’s BlackHat presentations: it made it a bit harder to guess the location of the 64-bit NTDLL by simply adding a fixed size to the 32-bit NTDLL. In my screenshot, since this is a CFG-enabled process, the VADs don’t exactly envelop NTDLL — rather they surround the native CFG bitmap + NTDLL, but the point remains.

This change in NTDLL load behavior also had the likely intended side effect of making hooks in 64-bit NTDLL extremely hard, or outright impossible. You see, without consuming an enormous amount of space, it’s simply not possible to overwrite an x64 instruction with a call or jmp to an absolute 64-bit address efficiently. Instead, hooking engines will allocate a “trampoline” that is within the 32-bit address range of the hooked function, and use a much smaller 5 byte 32-bit relative jump, which happens to fit nicely in the “hotpatch aware” region that Microsoft binaries have (or anyone linking with /hotpadmin). The trampoline then uses the full 64-bit absolute jump instruction.

As you’ve figured out by now, if the trampoline needs to be within 2GB, but there are two large VADs blocking off all 64-bit addresses around NTDLL, this hooking technique is dead in the water. Other, more complex and error-prone techniques must (and can) be used instead.

Nevertheless, nothing stops Heaven’s Gate on Windows 8. There some minor WoW64 changes which one must adapt to, and accessing or hooking 64-bit NTDLL becomes harder.

Control Flow Guard and WoW64

In Windows 10, a new exploit mitigation is introduced called Control Flow Guard, or CFG. It too, has been rather well described in multiple sources, so I won’t go into details inside of this post. The important piece to remember about CFG is that all relative function calls are now subject to an additional compiler-generated check, which is implemented by NTDLL: only valid function prologues (within 8 bytes of alignment) can be the target of such a call. Valid function prologues, in turn, are marked by a bit being set in a very large bitmap (bit array) structure, which describes the entire user-mode address space (all 128TB of it!). I previously posted on some interesting changes this required in the memory manager, as this bit array obviously becomes quite large (2 TB, in fact).

What’s not been documented too clearly in most research is that on 64-bit systems, there are in fact not one, but two CFG bitmaps: one for 32-bit code, and one for 64-bit code. The addresses of both of these bitmaps is stored in the per-process working set structure (called MMWSL). This structure is pointed to by the MMSUPPORT structure inside of EPROCESS (i..e.: PsGetCurrentProcess()->Vm.VmWorkingSetList), but a unique thing about it, is that it’s stored in a region of memory called “hyperspace”, which is at a fixed address… much like the per-process page table entry array. On recent 64-bit systems, this hard-coded address is 0xFFFFF58010804000, a fact I pointed out in a previous blog post addressing the 64-bit address space of Windows 8.1 and later.

As one can see in the symbols that WinDBG can dump, the MMWSL structure contains a field:

+0x1f8 UserVaInfo       : _MI_USER_VA_INFO

And inside of MI_USER_VA_INFO, we can find an array:

+0x0c8 CfgBitMap : [2] _MI_CFG_BITMAP_INFO

Whose two entries correspond to the following enumeration:

 CfgBitMapNative = 0n0
 CfgBitMapWow64 = 0n1
 CfgBitMapMax = 0n2

Clearly, thus, a 64-bit Windows 10 kernel contains not one, but two CFG bitmaps. And indeed, the 32-bit NTDLL will utilize the address of the WoW64 bitmap, while the 64-bit NTDLL will utilize the Native bitmap. But why use two separate bitmaps? What separates a WoW64 bitmap from a native bitmap? One would imagine that 64-bit code is marked as executable in the native bitmap, and 32-bit code is marked as executable in the WoW64 bitmap… but that’s not quite the full story.

At verification time, indeed, it is the version of NTDLL that is being used, which determines which bitmap will be looked at. But how does the OS populate the bits?

In CFG-aware versions of Windows, the CFG bitmap is touched through two paths: MiCommitVadCfgBits, and MiCfgMarkValidEntries. These, in turn, correspond to either intrinsic CFG modifications (side-effects of allocating, protecting and/or mapping executable memory), or explicit CFG modifications (effect of calling SetValidCallTargets). Both of these paths will eventually call MiSelectCfgBitMap, whose pseudo-code is shown below.


As is quite clear from the code, any private memory allocations below the 64-bit boundary will be marked only in the 32-bit bitmap, while the opposite applies to the 64-bit bitmap. In fact, this is the result of an optimization: instead of having two 2TB bit arrays for each processor execution mode, a single 2TB array is used for 64-bit native code, while a single 32MB array is used for 32-bit native code, greatly reducing address space consumption.

Closing the Gate

Basing the decision of which CFG bitmap to populate on the virtual address of the executable allocation creates an obvious dichotomy: 64-bit code, if running in a 32-bit address range, will instantly trip up CFG, because the NTDLL library that is active in that environment is the 64-bit version, which will check the 64-bit bitmap, which will not have any bits set in the 0-4 GB range. Similarly, any 32-bit code must be running below the 4 GB boundary, else the 32-bit NTDLL’s CFG validation routine will trip up, as the 32-bit bitmap isn’t even large enough to account for addresses above 4 GB.

A naive solution is therefore proposed: simply allocate 64-bit code above the 4GB range, and the problem goes away. There is, of course, a problem with this approach: the NoChange VADs which block the entire > 4 GB region of memory and mark it unusable, leaving only 64-bit NTDLL as the only valid allocation in that address range.

In Windows 10, these two factors combined result in the inability to execute any useful 64-bit code in a 32-application/WoW64 process, because the two restrictions combine, creating an impossible condition. You may be tempted to dismiss the reality by stating that all the 64-bit malicious code has to do is not to have been compiled with CFG. In this case, the compiler should not be emitting calls to the validation routine. However, this misses a critical point: it’s not the process’ own executable code/shellcode which are necessarily performing the 64-bit CFG checks — it’s the 64-bit NTDLL itself, or any other additional 64-bit DLLs you may have injected through the initial 64-bit shellcode, into your own process.

Even worse, even if no other 64-bit DLLs are imported, some core system functionality, implemented by NTDLL, also validates the CFG bitmap: Exceptions, User-Mode Callbacks, and APCs. Any usage of these system mechanisms, because they always initially execute in 64-bit mode, will cause a CFG violation if the target is not in the bitmap — which it cannot possibly be. The same goes for higher level functionality like using the Thread Pool, or any other callback-based mechanism owned by NTDLL in 64-bit mode. For example, because kernel-mode injects user-mode APCs through the 64-bit NTDLL, the user-mode APC routine cannot possibly be a custom, non-DLL function: it would’ve been impossible to allocate it > 4 GB, and the APC dispatcher will validate the CFG bits for any address < 4 GB, and be unable to find it.

Perhaps the best example of these unexpected side-effects is to analyze what Heaven’s Gate-using malware often does to gain some usefulness in the hidden 64-bit context: it will lookup LdrLoadDll inside of NTDLL.DLL and attempt to load additional 64-bit DLLs, such as kernel32.dll. With some coercing (as some of the articles I linked to at the beginning showed), this can be made to work. The problem, in a CFG-aware NTDLL.DLL, is that LdrpCallInitRoutines will perform a CFG bitmap check before calling the DllMain of this DLL. As the DLL will be loaded in 32-bit address space, the WoW64 CFG bitmap will be marked, and not the Native CFG bitmap — causing the 64-bit NTDLL to believe that DllMain is not a valid relative call target, and crash the process.

Suffice it to say, although it still is possible to have a very simple 64-bit piece of code, even possibly performing some system calls, execute in the hidden 64-bit world of a WoW64 process/32-bit application, any attempts to load additional DLLs, use APCs, handle exceptions or user-mode callbacks in 64-bit mode will result in the process crashing, as a CFG violation will be tripped. For most intents and purposes, therefore, CFG has a potentially unintended side-effect: it closes down Heaven’s Gate.

Reopening the gate is left as an exercise to the reader 😉

Final Note

Astute readers may have noticed the following discrepancies, especially if following along on their own systems:

  • Windows 8.1 Update 3 does have support for CFG
  • We saw three, not two VADs, on my Windows 8.1 Update 3 screenshot
  • This post mentions how Windows 10 closes Heaven’s Gate, but not Windows 8.1 Update 3

The key is in  dumping the MI_USER_VA_INFO structure on such a system:

+0x060 CfgBitMap : [3] _MI_CFG_BITMAP_INFO

Three entries? Let’s take a look:

 CfgBitMapNative = 0n0
 CfgBitMapWow64 = 0n1
 CfgBitMapWow64NativeLow = 0n2
 CfgBitMapMax = 0n3

This explains the three, not two VADs in my dump: in the original CFG implementation on Windows 8.1, 64-bit code could live in the 32-bit address range, as the Native bitmap had a “Wow64Low” portion. In Windows 10, this is now gone (saving 32MB of address space) — Native code is only aware of the 64-bit address ranges.

What are Little PatchGuards Made Of?

June 22nd, 2015

A number of excellent PatchGuard articles have been written around what PatchGuard is, how to bypass it, what triggers it uses, its obfuscation techniques, and more.

But for some reason, nobody has published a full list of everything that PatchGuard actually verifies. Microsoft used to have a website that listed the initial first 7 checks, but nothing beyond that.

I asked around at conferences, and the answer I got was that the code was too complex to analyze, and nobody really wanted to take the time to figure out every single check. I had my own private list of checks I knew PatchGuard does (through runtime analysis), but I was surprised to see the real reason nobody’s bothered to analyze this…

… Microsoft’s own public debugger (known as WinDBG) tells you — why bother reversing? 🙂

Lo’ and behold, the 39 different checks in PatchGuard Windows 8.1 Update. There’s a few more in Windows 10, I guess they’re not yet documented.

Arg1: 00000000, Reserved
Arg2: 00000000, Reserved
Arg3: 00000000, Failure type dependent information
Arg4: 00000000, Type of corrupted region, can be
0 : A generic data region
1 : Modification of a function or .pdata
2 : A processor IDT
3 : A processor GDT
4 : Type 1 process list corruption
5 : Type 2 process list corruption
6 : Debug routine modification
7 : Critical MSR modification
8 : Object type
9 : A processor IVT
a : Modification of a system service function
b : A generic session data region
c : Modification of a session function or .pdata
d : Modification of an import table
e : Modification of a session import table
f : Ps Win32 callout modification
10 : Debug switch routine modification
11 : IRP allocator modification
12 : Driver call dispatcher modification
13 : IRP completion dispatcher modification
14 : IRP deallocator modification
15 : A processor control register
16 : Critical floating point control register modification
17 : Local APIC modification
18 : Kernel notification callout modification
19 : Loaded module list modification
1a : Type 3 process list corruption
1b : Type 4 process list corruption
1c : Driver object corruption
1d : Executive callback object modification
1e : Modification of module padding
1f : Modification of a protected process
20 : A generic data region
21 : A page hash mismatch
22 : A session page hash mismatch
23 : Load config directory modification
24 : Inverted function table modification
25 : Session configuration modification
102 : Modification of win32k.sys

I have to admit, there are some things I didn’t realize PatchGuard would actually think about protecting, such as the Local APIC. It’s also interesting to see some more esoteric hooks in the list as well, such as PsEstablishWin32Callout protection. I also did not realize PatchGuard now protects the DRIVER_OBJECT structure — indeed, hooking a major function will now give you code 0x1C. And finally, the protection of protected processes means that technically something such as Mimikatz’s “MimiDrv” may crash some machines in the wild.

I usually try to avoid talking about PatchGuard since I’m glad it’s giving AV companies hell, but I can’t have been the only person that never noticed that the checks were documented in the debugger all along, hidden behind a simple command (it makes sense that Microsoft wouldn’t want their own support engineers to be wondering what on Earth they’re looking at):

!analyze -show 109

I can’t even take credit for discovering this on my own. Reading Microsoft’s famous “NT Debugging” blog made me realize that this had been there all along.


Analyzing MS15-050 With Diaphora

May 14th, 2015

One of the most common ways that I glean information on new and upcoming features on releases of Windows is obviously to use reverse engineering such as IDA Pro and look at changed functions and variables, which usually imply a change in functionality.

Of course, such changes can also reveal security fixes, but those are a lot harder to notice at the granular level of diff-analysis that I perform as part of understanding feature changes.

For those type of fixes, a specialized diffing tool, such as BinDiff is often used by reverse engineers and security experts. Recently, such tools have either become obsoleted, abandoned, or cost prohibitive. A good friend of mine, Joxean Koret (previously of Hex-Rays fame, un-coincidentally), has recently developed a Python plugin for IDA Pro, called “Diaphora“, (diaforá, the Greek word for “difference”).

In this blog post, we’ll analyze the recent MS15-050 patch and do a very quick walk-through of how to use Diaphora.


Installing the plugin is as easy as going over to the GitHub page, cloning the repository into a .zip file, and extracting the contents into the appropriate directory (I chose IDA’s plugin folder, but this can be anything you wish).

As long as your IDA Python is configured correctly (which has been a default in IDA for many releases), clicking on File, Script file…, should let you select a .py file


Generating the initial baseline

The first time you run Diaphora, you’ll be making the initial SQLite library. If you don’t have Hex-Rays, or disable the “Use the decompiler if available” flag, this process only takes a few seconds. Otherwise, with Hex-Rays enabled, you’ll be spending more of the time waiting for the decompiler to run on the entire project. Depending on code complexity, this could take a while.

This SQLite library will essentially contain the assembly and pseudo-code in a format easily parsable by the plugin, as well as all your types, enumerations, and decompiler data (such as your annotations and renamed variables). In this case, I had an existing fairly well-maintained IDB for the latest version of the Service Control Manager for Windows 7 SP1, which had actually not changed since 2012. My pseudo-code had over 3 years to grow into a well-documented, thoroughly structured IDA database.

Diff me once, importing your metadata

On the second run of Diaphora (which at this point, should be on your new, fresh binary), this is where you will direct it to the initial SQLite database from the step above, plus select your diffing options. The default set I use are in the screenshot below.


This second run can take much longer than the first, because not only are you taking the time to generate the a second database, but you are then running all of the diffing algorithms that Diaphora implements (which you can customize), which can take significantly longer. Once the run is complete, Diaphora will show you identical code (“Best Matches”), close matches (“Partial Matches”), and Unidentifiable Matches. This is where comparing a very annotated IDB with a fresh IDB for purposes of security research can have problems.

Since I renamed many of the static global variables, any code using them in their renamed format would appear different from the original “loc_325345” format that IDA uses by default. Any function prototypes which I manually fixed up would also appear different (Hex-Rays is especially bad with variable argument __stdcall on x86), as well any callers of those functions.

So in the initial analysis, I got tons of “Partial Matches” and very few “Best Matches”. Nothing was unmatched, however.

One of the great parts of Diaphora, however, is that you can then confirm that the functions are truly identical. Since we’re talking about files which have symbols, it makes sense to claim that ScmFooBar is identical to ScmFooBar. This will now import all the metadata from your first first IDB to the other, and then give you the option of re-running the analysis stage.

At this point, I have taken all of the 3 years of research I had on one IDB, and instantly (well, almost) merged it to a brand new IDB that covers a more recent version of the binary.

Diff me twice, locating truly changed code

Now that the IDBs have been “synced up”, the second run should identify true code changes — new variables that have been added, structures that changed, and new code paths. In truth, those were identified the first time around, but hidden in the noise of all the IDB annotation changes. Here’s an incredible screenshot of what happened the second time I ran Diaphora.

First, note how almost all the functions are now seen as identical:

And then, on the Partial Matches tab… we see one, and only one function. This is likely what MS15-050 targeted (the description in the Security Bulletin is that this fixed an “Impersonation Level Check” — the function name sounds like it could be related to an access check!).

Now that we have our only candidate for the fix delivered in this update, we can investigate what the change actually was. We do this by right-clicking on the function and selecting “Diff pseudo-code”. The screenshot below is Diaphora’s output:


At this point, the vulnerability is pretty clear. In at least some cases where an access check is being made due to someone calling the Service Control Manager, the impersonation level isn’t verified — meaning that someone with an Anonymous SYSTEM token (for example) could pass off as actually being a SYSTEM caller, and therefore be able to perform actions that only SYSTEM could do. In fact, in this case, we see that the Authentication ID (LUID) of 0x3E7 is checked, which is actually SYSTEM_LUID, making our example reality.

At this point, I won’t yet go into the details on which Service Control Manager calls exactly are vulnerable to this incorrect access check (ScAccessCheck, which is normally used, actually isn’t vulnerable, as it calls NtAccessCheck), or how this vulnerability could be used for local privilege escalation, because I wanted to give kudos to Joxean for this amazing plugin and get more people aware of its existence.

Perhaps we’ll keep the exploitation for a later post? For some  ideas, read up James Forshaw’s excellent Project Zero blog post, in which he details another case of poor impersonation checks in the operating system.


How Control Flow Guard Drastically Caused Windows 8.1 Address Space and Behavior Changes

January 22nd, 2015

Windows 8.1 radically changes the address space layout of the system by finally removing the 44-bit limitation which I described in one of the earliest blog posts on this website (and which Wikipedia even links to!). This is a little-known detail about the operating system, and an odd thing for Microsoft not to emphasize on with more aplomb, especially given that 8.1 is considered a “patch” of Windows 8.

Now, you may think that 16 TB to 256 TB is a meaningless change since no applications currently use even a fraction of that space, but the main benefit of this change are not the ability to allocate additional memory, but rather the increased entropy space available for Address Space Load Randomization (ASLR), especially given that Windows 8 introduced High Entropy ASLR (HEASLR), Top-down Randomization and Anonymous Memory Randomization.

Additionally, another key change was done in Windows 8.1 that is not mentioned anywhere. As Pavel Lebedinsky, one of the lead SDETs on the Memory Manager and an extremely helpful individual indicated on one of the blog posts from Mark Russinovich:

1. Reserved memory does contribute to commit charge, because the memory manager charges commit for pagetable space necessary to map the entire reserved range. On 64 bit this can be a significant number (reserving 1 TB of memory will consume approximately 2 GB of commit).

This means that attempting to reserve the full 8 TB of memory on Windows 7 results in 16 GB of commit, which is beyond’s most people’s commit limit, especially at the time. In Windows 8.1, this would result in 128 GB of commit being used, which only a beefy server would tolerate. While such large memory reservations are unusual, they do have usefulness in certain scenarios related to security and low-level testing. This Windows behavior prevented such reservations from reliably working, but in Windows 8.1, the limitation has been removed!

Indeed, you can easily test this by using the TestLimit tool from the Windows Internals Book, and run it with the -r option (and preferably with a large enough block size). Here’s a screenshot of hitting the 128 TB reservation:


And here’s the resulting view in VMMap, which does not show the expected page table commit charge, but rather a much smaller size (256 MB).


So why did Microsoft change this behavior in Windows 8.1? Well, Windows 10, as well as Windows 8.1 Update 3 (November Update) make this clear. As I previously tweeted, these OS versions enable Control Flow Guard (CFG), a feature that laid dormant in the first versions of Windows 8.1. In order to function, CFG requires the use of optimized bitmaps in order to determine the validity of indirect calls, and on 64-bit Windows, this bitmap requires 2 TB of space. Not only would this cut the Windows 8 address space by 25%, it would’ve also resulted in 4 GB of per-process commit!

Here’s a screenshot of Process Hacker showing how all CFG-enabled processes now use 2 TB of virtual address space:


The final effect of this change from 8 TB to 128 TB is that the kernel address space layout has significantly changed. And sadly, the !address extension in WinDBG is broken and continues to show the Windows 8 address space layout (which I expanded on during my Blackhat 2013 talk), while the Windows Internals book is stuck on Windows 7 and doesn’t even cover Windows 8 or higher.

Therefore, I publish below what I believe to be the only public source of information on the Windows 8.1 x64 memory layout. One of the benefits of this new layout is that it now becomes extremely easy by using the first 5 or 6 nibbles of an address to determine where it’s coming from. For example, 0xFFFFD… is a kernel stack, 0xFFFFC… is paged pool, 0xFFFFF8… is a loaded image (driver or kernel), and 0xFFFFE… is nonpaged pool.

FFFF0000`00000000FFFF07FF`FFFFFFFF8TBMemory Hole
FFFF0800`00000000FFFFAFFF`FFFFFFFF168TBUnused Space
FFFFB000`00000000FFFFBFFF`FFFFFFFF16TBSystem Cache
FFFFE000`00000000FFFFEFFF`FFFFFFFF16TBNonpaged Pool
FFFFF000`00000000FFFFF67F`FFFFFFFF6.5TBUnused Space
FFFFF780`00000000FFFFF780`00000FFF4KShared User Data
FFFFF780`C0000000FFFFF780`FFFFFFFF1GBWS Hash Table
FFFFF781`00000000FFFFF791`3FFFFFFF65GBPaged Pool WS
FFFFF791`40000000FFFFF799`3FFFFFFF32GBWS Hash Table
FFFFF799`40000000FFFFF7A9`7FFFFFFF65GBSystem Cache WS
FFFFF7A9`80000000FFFFF7B1`7FFFFFFF32GBWS Hash Table
FFFFF7B1`80000000FFFFF7FF`FFFFFFFF314GBUnused Space
FFFFF900`00000000FFFFF97F`FFFFFFFF512GBSession Space
FFFFF980`00000000FFFFFA70`FFFFFFFF1TBDynamic VA Space
Table describing the various 64-bit memory ranges in Windows 8.1

Sheep Year Kernel Heap Fengshui: Spraying in the Big Kids’ Pool

December 29th, 2014

The State of Kernel Exploitation

The typical write-what-where kernel-mode exploit technique usually relies on either modifying some key kernel-mode data structure, which is easy to do locally on Windows thanks to poor Kernel Address Space Layout Randomization (KASLR), or on redirecting execution to a controlled user-mode address, which will now run with Ring 0 rights.

Relying on a user-mode address is an easy way not to worry about the kernel address space, and to have full control of the code within a process. Editing the tagWND structure or the HAL Dispatch Table are two very common vectors, as are many others.

However, with Supervisor Mode Execution Prevention (SMEP), also called Intel OS Guard, this technique is no longer reliable — a direct user-mode address cannot be used, and other techniques must be employed instead.

One possibility is to disable SMEP Enforcement in the CR4 register through Return-Oriented Programming, or ROP, if stack control is possible. This has been covered in a few papers and presentations.

Another related possibility is to disable SMEP Enforcement on a per-page basis — taking a user-mode page and marking it as a kernel page by making the required changes in the page level translation mapping entries. This has also been talked in at least one presentation, and, if accepted, a future SyScan 2015 talk from a friend of mine will also cover this technique. Additionally, if accepted, an alternate version of the technique will be presented at INFILTRATE 2015, by yours truly.

Finally, a theoretical possibility is being able to transfer execution (through a pointer, callback table, etc) to an existing function that disables SMEP (and thus bypassing KASLR), but then somehow continues to give the attacker control without ROP — nobody has yet found such a function. This would be a type of Jump-Oriented Programming (JOP) attack.

Nonetheless, all of these techniques continue to leverage a user-mode address as the main payload (nothing wrong with that). However, one must also consider the possibility to use a kernel-mode address for the attack, which means that no ROP and/or PTE hacking is needed to disable SMEP in the first place.

Obviously, this means that the function to perform the malicious payload’s work already exists in the kernel, or we have a way of bringing it into the kernel. In the case of a stack/pool overflow, this payload probably already comes with the attack, and the usual tricks have been employed there in order to get code execution. Such attacks are particularly common in true ‘remote-remote’ attacks.

But what of write-what-where bugs, usually the domain of the local (or remote-local) attacker? If we have user-mode code execution available to us, to execute the write-what-where, we can obviously continue using the write-what-where exploit to repeatedly fill an address of our choice with the payload data. This presents a few problems however:

  • The write-what-where may be unreliable, or corrupt adjacent data. This makes it hard to use it to ‘fill’ memory with code.
  • It may not be obvious where to write the code — having to deal with KASLR as well as Kernel NX. On Windows, this is not terribly hard, but it should be recognized as a barrier nonetheless.

This blog post introduces what I believe to be two new techniques, namely a generic kernel-mode heap spraying technique which results in executable memory, followed by a generic kernel-mode heap address discovery technique, bypassing KASLR.

Big Pool

Experts of the Windows heap manager (called the pool) know that there are two different allocators (three, if you’re being pedantic): the regular pool allocator (which can use lookaside lists that work slightly differently than regular pool allocations), and the big/large page pool allocator.

The regular pool is used for any allocations that fit within a page, so either 4080 bytes on x86 (8 bytes for the pool header, and 8 bytes used for the initial free block), or 4064 bytes on x64 (16 bytes for the pool header, 16 bytes used for the initial free block). The tracking, mapping, and accounting of such allocations is handled as part of the regular slush of kernel-mode memory that the pool manager owns, and the pool headers link everything together.

Big pool allocations, on the other hand, take up one or more pages. They’re used for anything over the sizes above, as well as when the CacheAligned type of pool memory is used, regardless of the requested allocation size — there’s no way to easily guarantee cache alignment without dedicating a whole page to an allocation.

Because there’s no room for a header, these pages are tracked in a separate “Big Pool Tracking Table” (nt!PoolBigPageTable), and the pool tags, which are used to identify the owner of an allocation, are also not present in the header (since there isn’t one!), but rather in the table as well. Each entry in this table is represented by a POOL_TRACKER_BIG_PAGES structure, documented in the public symbols:

    +0x000 Va : Ptr32 Void
    +0x004 Key : Uint4B
    +0x008 PoolType : Uint4B
    +0x00c NumberOfBytes : Uint4B

One thing to be aware of is that the Virtual Address (Va) is OR’ed with a bit to indicate if the allocation is freed or allocated — in other words, you may have duplicate Va’s, some freed, and at most one allocated. The following simple WinDBG script will dump all the big pool allocations for you:

r? @$t0 = (nt!_POOL_TRACKER_BIG_PAGES*)@@(poi(nt!PoolBigPageTable))
r? @$t1 = *(int*)@@(nt!PoolBigPageTableSize) / sizeof(nt!_POOL_TRACKER_BIG_PAGES)
.for (r @$t2 = 0; @$t2 < @$t1; r? @$t2 = @$t2 + 1)
    r? @$t3 = @$t0[@$t2];
    .if (@@(@$t3.Va != 1))
        .printf "VA: 0x%p Size: 0x%lx Tag: %c%c%c%c Freed: %d Paged: %d CacheAligned: %d\n", @@((int)@$t3.Va & ~1), @@(@$t3.NumberOfBytes), @@(@$t3.Key >> 0 & 0xFF), @@(@$t3.Key >> 8 & 0xFF), @@(@$t3.Key >> 16 & 0xFF), @@(@$t3.Key >> 24 & 0xFF), @@((int)@$t3.Va & 1), @@(@$t3.PoolType & 1), @@(@$t3.PoolType & 4) == 4

Why are big pool allocations interesting? Unlike small pool allocations, which can share pages, and are hard to track for debugging purposes (without dumping the entire pool slush), big pool allocations are easy to enumerate. So easy, in fact, that the undocumented KASLR-be-damned API NtQuerySystemInformation has an information class specifically designed for dumping big pool allocations. Including not only their size, their tag, and their type (paged or nonpaged), but also their kernel virtual address!

As previously presented, this API requires no privileges, and only in Windows 8.1 has it been locked down against low integrity callers (Metro/Sandboxed applications).

With the little snippet of code below, you can easily enumerate all big pool allocations:

// Note: This is poor programming (hardcoding 4MB).
// The correct way would be to issue the system call
// twice, and use the resultLength of the first call
// to dynamically size the buffer to the correct size
bigPoolInfo = RtlAllocateHeap(RtlGetProcessHeap(),
                              4 * 1024 * 1024);
if (bigPoolInfo == NULL) goto Cleanup;
res = NtQuerySystemInformation(SystemBigPoolInformation,
                               4 * 1024 * 1024,
if (!NT_SUCCESS(res)) goto Cleanup;
printf("TYPE     ADDRESS\tBYTES\tTAG\n");
for (i = 0; i < bigPoolInfo->Count; i++)
            bigPoolInfo->AllocatedInfo[i].NonPaged == 1 ?
            "Nonpaged " : "Paged    ",
if (bigPoolInfo != NULL)
    RtlFreeHeap(RtlGetProcessHeap(), 0, bigPoolInfo);

Pool Control

Obviously, it’s quite useful to have all these handy kernel-mode addresses. But what can we do to control their data, and not only be able to read their address?

You may be aware of previous techniques where a user-mode attacker allocates a kernel-object (say, an APC Reserve Object), which has a few fields that are user-controlled, and which then has an API to get its kernel-mode address. We’re essentially going to do the same here, but rely on more than just a few fields. Our goal, therefore, is to find a user-mode API that can give us full control over the kernel-mode data of a kernel object, and additionally, to result in a big pool allocation.

This isn’t as hard as it sounds: anytime a kernel-mode component allocates over the limits above, a big pool allocation is done instead. Therefore, the exercise reduces itself to finding a user-mode API that can result in a kernel allocation of over 4KB, whose data is controlled. And since Windows XP SP2 and later enforce kernel-mode non-executable memory, the allocation should be executable as well.

Two easy examples may popup in your head:

  1. Creating a local socket, listening to it, connecting from another thread, accepting the connection, and then issuing a write of > 4KB of socket data, but not reading it. This will result in the Ancillary Function Driver for WinSock (AFD.SYS), also affectionally known as “Another F*cking Driver”, allocating the socket data in kernel-mode memory. Because the Windows network stack functions at DISPATCH_LEVEL (IRQL 2), and paging is not available, AFD will use a nonpaged pool buffer for the allocation. This is great, because until Windows 8, nonpaged pool is executable!
  2. Creating a named pipe, and issuing a write of > 4KB of data, but not reading it. This will result in the Named Pipe File System (NPFS.SYS) allocating the pipe data in a nonpaged pool buffer as well (because NPFS performs buffer management at DISPATCH_LEVEL as well).

Ultimately, #2 is a lot easier, requiring only a few lines of code, and being much less inconspicuous than using sockets. The important thing you have to know is that NPFS will prefix our buffer with its own internal header, which is called a DATA_ENTRY. Each version of NPFS has a slightly different size (XP- vs 2003+ vs Windows 8+).

I’ve found that the cleanest way to handle this, and not to worry about offsets in the final kernel payload, is to internally handle this in the user-mode buffer with the right offsets. And finally, remember that the key here is to have a buffer that’s at least the size of a page, so we can force the big pool allocator.

Here’s a little snippet that keeps all this into account and will have the desired effects:

UCHAR payLoad[PAGE_SIZE - 0x1C + 44];
// Fill the first page with 0x41414141, and the next page
// with INT3's (simulating our payload). On x86 Windows 7
// the size of a DATA_ENTRY is 28 bytes (0x1C).
RtlFillMemory(payLoad,  PAGE_SIZE - 0x1C,     0x41);
RtlFillMemory(payLoad + PAGE_SIZE - 0x1C, 44, 0xCC);
// Write the data into the kernel
res = CreatePipe(&readPipe,
if (res == FALSE) goto Cleanup;
res = WriteFile(writePipe,
if (res == FALSE) goto Cleanup;
// extra code goes here...

Now all we need to know is that NPFS uses the pool tag ‘NpFr’ for the read data buffers (you can find this out by using the !pool and !poolfind commands in WinDBG). We can then change the earlier KASLR-defeating snippet to hard-code the pool tag and expected allocation size, and we can instantly find the kernel-mode address of our buffer, which will fully match our user-mode buffer.

Keep in mind that the “Paged vs. Nonpaged” flag is OR’ed into the virtual address (this is different from the structure in the kernel, which tracks free vs. allocated), so we’ll mask that out, and also make sure you align the size to the pool header alignment (it’s enforced even for big pool allocations). Here’s that snippet, for x86 Windows:

// Based on pooltag.txt, we're looking for the following:
// NpFr - npfs.sys - DATA_ENTRY records (r/w buffers)
for (entry = bigPoolInfo->AllocatedInfo;
     entry < (PSYSTEM_BIGPOOL_ENTRY)bigPoolInfo +
    if ((entry->NonPaged == 1) &&
        (entry->TagUlong == 'rFpN') &&
        (entry->SizeInBytes == ALIGN_UP(PAGE_SIZE + 44,
        printf("Kernel payload @ 0x%p\n",
               (ULONG_PTR)entry->VirtualAddress & ~1 +

And here’s the proof in WinDBG:

Kernel Malloc

Voila! Package this into a simple “kmalloc” helper function, and now you too, can allocate executable, kernel-mode memory, at a known address! How big can these allocations get? I’ve gone up to 128MB without a problem, but this being non-paged pool, make sure you have the RAM to handle it. Here’s a link to some sample code which implements exactly this functionality.

An additional benefit of this technique is that not only can you get the virtual address of your allocation, you can even get the physical address! Indeed, as part of the undocumented Superfetch API that I first discovered and implemented in my meminfo tool, which has now been supplanted by the RAMMap utility from SysInternals, the memory manager will happily return the pool tag, virtual address, and physical address of our allocation.

Here’s a screenshot of RAMMap showing another payload allocation and its corresponding physical address (note that the 0x1000 difference is since the command-line PoC biases the pointer, as you saw in the code).


Next Steps

Now, for full disclosure, there are a few additional caveats that make this technique a bit less sexy in 2015 — and why I chose to talk about it today, and not 8 years ago when I first stumbled upon it:

1) Starting with Windows 8, nonpaged pool allocations are now non-executable. This means that while this trick still lets you spray the pool, your code will require some sort of NX bypass first. So you’ve gone from bypassing SMEP to bypassing kernel-mode NX.

2) In Windows 8.1, the API to get the big pool entries and their addresses is no longer usable by low-integrity callers. This significantly reduces the usefulness in local-remote attacks, since those are usually launched through sandboxed applications (Flash, IE, Chrome, etc) and/or Metro containers.

Of course, there are some ways around this — a sandbox escape is often used in local-remote attacks anyway, so #2 can become moot. As for #1, some astute researchers have already figured out that NX was not fully deployed — for example, Session Pool allocations, are STILL executable on newer versions of Windows, but only on x86 (32-bit). I leave it as an exercise to readers to figure out how this technique can be extended to leverage that (hint: there’s a ‘Big Session Pool’).

But what about a modern, 64-bit version of Windows, say even Windows 10? Well, this technique appears to be mostly dead on such systems — or does it? Is everything truly NX in the kernel, or are there still some sneaky ways to get some executable memory, and to get its address? I’ll be sure to blog about it once Windows 14 is out the door in 2022.

PE Trick #1: A Codeless PE Binary File That Runs

September 29th, 2014


One of the annoying things of my Windows Internals/Security research is when every single component and mechanism I’ve looked at in the last six months has ultimately resulted in me finding very interesting design bugs, which I must now wait on Microsoft to fix before being able to talk further about them. As such, I have to take a smaller break from kernel-specific research (although I hope to lift the veil over at least one issue at the No Such Conference in Paris this year). And so, in the next following few blog posts, probably inspired by having spent too much time talking with my friend Ange Albertini, I’ll be going over some neat PE tricks.


Write a portable executable (PE/EXE) file which can be spawned through a standard CreateProcess call and will result in STATUS_SUCCESS being returned as well as a valid Process Handle, but will not

  • Contain any actual x86/x64 assembly code section (i.e.: the whole PE should be read-only, no +X section)
  • Run a single instruction of what could be construed as x86 assembly code, which is part of the file itself (i.e.: random R/O data should not somehow be forced into being executed as machine code)
  • Crash or make any sort of interactive/visible notice to the user, event log entry, or other error condition.

Interesting, this was actually a real-world situation that I was asked to provide a solution for — not a mere mental exercise. The idea was being able to prove, in the court of law, that no “foreign” machine code had executed as a result of this executable file having been launched (i.e.: obviously the kernel ran some code, and the loader ran too, but all this is pre-existing Microsoft OS code). Yet, the PE file had to not only be valid, but to also return a valid process handle to the caller.


HEADER:00000000 .686p
HEADER:00000000 .mmx
HEADER:00000000 .model flat
HEADER:00000000 ; Segment type: Pure data
HEADER:00000000 HEADER segment page public 'DATA' use32
HEADER:00000000 assume cs:HEADER
HEADER:00000000 __ImageBase dw 5A4Dh ; PE magic number
HEADER:00000002 dw 0 ; Bytes on last page of file
HEADER:00000004 dd 4550h ; Signature
HEADER:00000008 dw 14Ch ; Machine
HEADER:0000000A dw 0 ; Number of sections
HEADER:0000000C dd 0 ; Time stamp
HEADER:00000010 dd 0 ; Pointer to symbol table
HEADER:00000014 dd 0 ; Number of symbols
HEADER:00000018 dw 0 ; Size of optional header
HEADER:0000001A dw 2 ; Characteristics
HEADER:0000001C dw 10Bh ; Magic number
HEADER:0000001E db 0 ; Major linker version
HEADER:0000001F db 0 ; Minor linker version
HEADER:00000020 dd 0 ; Size of code
HEADER:00000024 dd 0 ; Size of initialized data
HEADER:00000028 dd 0 ; Size of uninitialized data
HEADER:0000002C dd 7FBE02F8h ; Address of entry point
HEADER:00000030 dd 0 ; Base of code
HEADER:00000034 dd 0 ; Base of data
HEADER:00000038 dd 400000h ; Image base
HEADER:0000003C dd 4 ; Section alignment
HEADER:00000040 dd 4 ; File alignment
HEADER:00000044 dw 0 ; Major operating system version
HEADER:00000046 dw 0 ; Minor operating system version
HEADER:00000048 dw 0 ; Major image version
HEADER:0000004A dw 0 ; Minor image version
HEADER:0000004C dw 4 ; Major subsystem version
HEADER:0000004E dw 0 ; Minor subsystem version
HEADER:00000050 dd 0 ; Reserved 1
HEADER:00000054 dd 40h ; Size of image
HEADER:00000058 dd 0 ; Size of headers
HEADER:0000005C dd 0 ; Checksum
HEADER:00000060 dw 2 ; Subsystem
HEADER:00000062 dw 0 ; Dll characteristics
HEADER:00000064 dd 0 ; Size of stack reserve
HEADER:00000068 dd 0 ; Size of stack commit
HEADER:0000006C dd 0 ; Size of heap reserve
HEADER:00000070 dd 0 ; Size of heap commit
HEADER:00000074 dd 0 ; Loader flag
HEADER:00000078 dd 0 ; Number of data directories
HEADER:0000007C HEADER ends
HEADER:0000007C end

As per Corkami, in Windows 7 and higher, you’ll want to make sure that the PE is at least 252 bytes on x86, or 268 bytes on x64.

Here’s a 64 byte Base64 representation of a .gz file containing the 64-bit compatible (268 byte) executable:



There is one non-standard machine configuration in which this code will actually still crash (but still return STATUS_SUCCESS in CreateProcess, however). This is left as an exercise to the reader.


The application executes and exits successfully. But as you can see, no code is present in the binary. How does it work? Do you have any other solutions which satisfy the challenge?