Here’s a couple of various useful tips I’ve discovered (as I’m sure others have) which make my life easier on Vista, and saved me a lot of trouble.
Fix that debugger!
I had done everything right to get local kernel debugging to work: I added /DEBUG with bcdedit. I used WinDBG in Administrator mode, I even turned off UAC. I made sure that all the debugging permissions were correct for admin accounts (SeDebugPrivilege). I made sure the driver WinDBG uses was extracted. Nothing worked! For some reason, the system was declaring itself as not having a kernel debugger enabled. I searched Google for answers, and found other people were experiencing the same problem, including a certain “Unable to enable kernel debugger, NTSTATUS 0xC0000354 An attempt to do an operation on a debug port failed because the port is in the process of being deleted” error message.
Here’s what I did to fix it:
- I used bcdedit to remove and re-add the /debug command: bcdedit /debug off then bcdedit /debug on
- I used bcdedit to force the debugger to be active and to listen on the Firewire port: bcdedit /dbgsettings 1394 /start active
- I used bcdedit to enable boot debugging: bcdedit /bootdebug
- I rebooted, and voila! I was now able to use WinDBG in local kernel debugging mode.
Get the debugger state
In Vista, one must boot with /DEBUG in order to do any local kernel debugging work, and as we all know, this disables DVD and high-def support to “ensure a reliable and secure playback environment”. After a couple of days work, I find myself sometimes forgetting if I had booted in debug mode or not, and I don’t want to start WinDBG all the time to check — or maybe you’re in a non-administrative environment and don’t want to use WinDBG. It’s even possible a new class of malware may soon start adding /DEBUG to systems in order to better infiltrate kernel-mode. Here’s two ways to check the state:
- Navigate to HKLM\System\CurrentControlSet\Config and look at the SystemStartOptions. If you see /DEBUG, the system was booted in debugging mode. However, because it may have been booted in “auto-enable” mode, or in “block enable” mode (in which the user can then enable/disable the debugger at his will), this may not tell you the whole story.
- Use kdbgctrl, which ships with the Debugging Tools for Windows. kdbgctrl -c will tell you whether or not the kernel debugger is active right now.
What would be really worthwhile, is to see if DVD playback works with /DEBUG on, but kernel debugging disabled with kdbgctrl, and if it stops playing (but debugging works) if kdbgctrl is used to enable it.
Access those 64-bit system binaries from your 32-bit app
I regularly have to use IDA 64 to look at binaries on my Vista system, since I’m using the 64-bit edition. Unfortunately, IDA 64 itself is still a 32-bit binary, which was great when I had a 32-bit system, but not so great anymore. The reason is WOW64’s file redirection, which changes every access to “System32” to “SysWOW64”, the directory containing 32-bit DLLs. But IDA 64 is perfectly able (and has to!) handle 64-bit binaries, so there wasn’t an easy way to access those DLLs. Unfortunately, it looks like the IDA developers haven’t heard of Wow64DisableFsRedirection, so I resorted to copying the binaries I needed out of the directory manually.
However, after reverse engineering some parts of WOW64 to figure out where the WOW64 context is saved, I came across a strange string — “sysnative”. It turns out that after creating (with a 64-bit app, such as Explorer) a “sysnative” directory in my Windows path, any 32-bit application navigating to it will now get the “real” (native) contents of the System32 directory! Problem solved!
So there you have it! I hope at least one of these was useful (or will be), and feel free to share your own!