Blogroll and Publications

It’s been a pretty slow week until now, so I’m going to take this time to do the last “todo” for my blog. As I mentionned previously, I have been setting up my blogroll and publication page, and now that I’ve finished uploading everything, I wanted to give a more in-depth overview of the materials and links:

Blogs

  • Ken Johnson’s Blog Ken is a well-known NT expert known as Skywing. We met while he was still working on ReactOS, and we frequently discuss NT implementation details and security bugs. We both have a hate for crappy drivers, hooks, and other kinds of dangerous code. If there’s something I don’t post here, Ken probably has posted about it before or will 🙂
  • Ero Carrera’s Blog Ero is a fellow security researcher that I met while at Recon, and he is well known for his work at Sabre, especially in analysis of binaries, and he has published various papers and given talks at conferences, as well as trainings. If you have a chance to meet him, take it, because he knows a lot!
  • TinyKRNL Website TinyKRNL is my own personal project, and team of people working on it. You can find more information on its website. It is a Windows NT Kernel implementation for educational use.
  • Larry Osterman’s Blog Larry is one of my favorite bloggers at Microsoft. He has been working there for over two decades, and his experience (and anecdotes) are always insightful. He has interesing topics such as “What’s wrong with this code” and other optimization/algorithm questions sometimes.
  • ReactOS The other project that I work on, ReactOS attempts to create an open-sourced (GPL) version of the Windows NT (XP/2003) Operating System by implementing its own kernel and using Wine’s Win32 libraries.
  • Jason Geffner’s Blog Jason is a reverse engineer at Microsoft that I’m happy to call my friend. He’s one of the smartest guys I’ve met, and he has a lot of neat ideas that he gets the chance to work on. He’s also responsible for saving the world a couple of times.
  • Raymond Chen’s Blog Raymond chen is the iconic Microsoft blogger. Blogging on everything from languages, music to Windows 1.0 compatibility functions present in Vista, Raymond makes it hard for you to hate those annoying features in Windows, and explains just how stupid some 3rd-party developers really are, and how much trouble he and his team have to go through to keep those apps working, so that you don’t complain.
  • OpenRCE OpenRCE is the premier Reverse Code Engineering portal, with all the big names in the field, interesting articles (such as mine ;), forum posts, tools and diagrams, and a helpful bunch of users from all over the industry. It’s also the brainchild of a very good friend of mine, Pedram.
  • Steve Dispensa’s Blog Steve is another NT genius, especially in NDIS, that took his knowledge to the corporate level and now is a CTO for a very sucessful company. We met through Ken Johnson, who is now employed there, and I’m still hoping to finally be able to meet him in the future. His blog is of particular interest to NT driver developers, much like Ken’s.
  • Pedram’s Blog Pedram is another security researcher and big name, as well as the creator of OpenRCE and many other valuable tools, including PaiMei and tools such as PyDbg. He’s also a good friend of mine, and a very fun guy.

Using spaces in your TARGETLIBS path

I’ve been trying to find a solution on this annoying “sources” problems in the Build utility for ages. Thankfully, a post at NotAKernelGuy pointed a way to the solution. It’s in Russian, but the basic solution is simple:

LINKER_OPTIDATA = \\
"$(VS80COMNTOOLS)\\..\\..\\VC\\PlatformSDK\\lib\\mscoree.lib"

Replace that path by whatever you need, but the end result is the same: the library will be added to the response file, and spaces will be preserved and respected.

Part 2 of User-Mode Debugging Internals Article

I’ve almost finished setting up the remaining parts of my blog. I’ve added an About page and pretty much filled my BlogRoll with the blogs I try to read daily. Thanks to everyone that’s visited/linked here in the last few days.

I’m currently working on the “Publications” page of the blog to have a central repository with all my data. I will also duplicate it on OpenRCE, but that site requires a login, and I wanted to make sure anyone could freely access my stuff. The links aren’t live yet, but they should be within the day.

Also, Part 2 of my article should appear soon, but for those that want to beat the clock, you can download it for now directly here. If you haven’t read Part 1 first, make sure you do here.

Brief overview of what’s discussed:

  • Part 1 – Win32: DebugActiveProcess, DebugBreakProcess, DebugSetProcessKillOnExit, CheckRemoteDebuggerPresent, WaitForDebugEvent, ContinueDebugEvent, DebugActiveProcessStop.
  • Part 1 – Win32: Teb->DbgSsReservedData[0] and DBGSS_THREAD_DATA. SaveProcessHandle, SaveThreadHandle, MarkThreadHandle, MarkProcessHandle, RemoveHandles, CloseAllProcessHandles.
  • Part 2 – Native: DbgUiConnectToDbg, DbgUiDebugActiveProcess, DbgUiStopDebugging, DbgUiIssueRemoteBreakin, DbgUiRemoteBreakin, DbgUiGetThreadDebugObject, DbgUiSetThreadDebugObject, DbgUiContinue, DbgUiWaitStateChange, DbgUiConvertStateChangeStructure.
  • Part 2 – Native: Teb->DbgSsReservedData[1], DBGUI_WAIT_STATE_CHANGE, Teb->Tib.ArbitraryUserPointer, DBG_STATE, DBGKM_EXCEPTION, DBGKM_CREATE_THREAD, DBGKM_CREATE_PROCESS, DBGKM_EXIT_THREAD, DBGKM_LOAD_DLL, DBGKM_UNLOAD_DLL.

Part 3 will cover Kernel Mode and the Nt* APIs when it’s out.

WDK RTM Changes

I consistently did diffs (differential changes) between each new release of the WDK. It was interesting to follow the evolution of certain APIs and structures, as well as APIs which were added by mistake.

The latter happens because kits like the WDK are built from a master header file. Suppose it looks like this:

// begin_ntddk

//
// Process Functions
//
NTKERNELAPI
NTAPI
KeSetProcess(IN PRKPROCESS Process);

//
// Thread Functions
//
NTKERNELAPI
NTAPI
KeStartThread(IN PRKTHREAD Thread);

// end_ntddk

NTKERNELAPI
NTAPI
KeSetThreadDrmProtection(IN PRKTHREAD Thread);

What would happen in the DDK is that the KeSetProcess and KeStartThread would be exported, and by definition, “legit” to be used in drivers. Now suppose the developers add a new API in Vista, and don’t properly take a look at the DDK tags, you could end up with this:

// begin_ntddk

//
// Process Functions
//
NTKERNELAPI
NTAPI
KeSetProcess(IN PRKPROCESS Process);

NTKERNELAPI
NTAPI
KeProtectProcessForDrm(IN PRKPROCESS Process);

//
// Thread Functions
//
NTKERNELAPI
NTAPI
KeStartThread(IN PRKTHREAD Thread);

// end_ntddk

NTKERNELAPI
NTAPI
KeSetThreadDrmProtection(IN PRKTHREAD Thread);

Notice that the tags weren’t properly updated to keep the DRM/internal/undocumented function out from the DDK, so it will appear in the WDK. Of course, at the next release, someone is bound to notice and fixup the tags. So by doing cumulative diffs, I was able to get the prototypes of quite a few new APIs that didn’t make it into the final WDK. Of course, I don’t condone their use in a driver, but they’re useful for ReactOS/TinyKRNL development and to better understand some of the changes done in Vista.

One of the more memorable API sets that were added allow drivers (well, at least, were supposed to!) to modify the size of the kernel stack. Typically MmCreateKernelStack was a way to do this, but these new Ke functions give a much greater degree of control as well as give you a Callout function:

#define MAXIMUM_EXPANSION_SIZE (KERNEL_LARGE_STACK_SIZE – (PAGE_SIZE / 2))

typedef
VOID
(NTAPI *PEXPAND_STACK_CALLOUT) (
__in_opt PVOID Parameter
);

#if (NTDDI_VERSION >= NTDDI_WS03SP1)
NTKERNELAPI
NTSTATUS
KeExpandKernelStackAndCallout (
__in PEXPAND_STACK_CALLOUT Callout,
__in_opt PVOID Parameter,
__in SIZE_T Size
);
#endif

#if (NTDDI_VERSION >= NTDDI_LONGHORN)
NTKERNELAPI
NTSTATUS
KeExpandKernelStackAndCalloutEx (
__in PEXPAND_STACK_CALLOUT Callout,
__in_opt PVOID Parameter,
__in SIZE_T Size,
__in BOOLEAN Wait,
__in_opt PVOID Context
);

NTKERNELAPI
PVOID
KeAllocateCalloutStack (
__in BOOLEAN LargeStack
);

NTKERNELAPI
VOID
KeFreeCalloutStack (
__in PVOID Context
);

#endif

Now here’s the ironic thing: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/DevTest_g/hh/DevTest_g/t06_bugs_B0_77bda7e9-4f41-49e9-86db-04446dc9c7b7.xml.asp

“The driver switched stacks using a method that is not supported by the operating system. The only supported way to extend a kernel mode stack is by using KeExpandKernelStackAndCallout.”

I guess it’s either time for a WDK bug or an MSDN documentation bug to be opened!

However, perhaps the best WDK RTM change was this:

Original:

// This logic is a reasonable hack-o-rama to make BillG happy
// since his machine ran chkdsk after he installed Beta 3. Why?
// ’cause setup cracked a non-exclusive DASD handle near the
// end of setup, wrote some data, closed the handle and we
// set the verify bit … came back around and saw that other
// arbitrary activity had left the volume in a temporarily dirty
// state.
//
// Of course, the real problem is that we don’t have a journal.
//

RTM:

// This logic is a reasonable change. Why?
// ’cause setup cracked a non-exclusive DASD handle near the
// end of setup, wrote some data, closed the handle and we
// set the verify bit … came back around and saw that other
// arbitrary activity had left the volume in a temporarily dirty
// state.
//
// Of course, the real problem is that we don’t have a journal.

“hack-o-rama to make BillG happy” => “reasonable change”.

Got any more similar changes of your own? Feel free the post them!

Microsoft Tools for Power Users/Developers

I haven’t fully finished up setting the blog yet, but I wanted to blog about some useful and not-very-well-known Microsoft projects, tools and technologies.

Unfortunately, Microsoft advertises way too much to regular users, and doesn’t take advantage of the influencial student/developer/power user market segment, which is usually the most vocal Anti-Microsoft. You can throw words like “SuperFetch” at an engineering student that uses Linux all you want, it probably won’t impress him much. Now show him PowerShell connecting to his NFS shares through a Windows Server 2003 R2 machine with Subsystem for Unix Applications and he might raise an eyebrow.

I’ve had to explain NT’s subsystem design twice this week and I always get a “wow? really? NT can run POSIX?” Yup, and NT 5.2 SP1 can even run 64-bit POSIX and debug them in Visual Studio 2005. NT 5.2 is also called Windows Server 2003. It’s basically a kernel that’s 30% faster then XP’s and was used as the core for Vista. If you want a fast OS with an optimized kernel and don’t want to take part of the “Vista Experience”, you should consider it.

Anyways, without further ado (visit the links for information, I could blog entire pages about these). All these are FREE!

News/Community

Downloads:

Source Code:

I’ll add more as I remember them, there’s really a lot of great stuff at Microsoft that’s untapped to!

Also, if you have anything to add, please comment!

First Post

I’ve kept many blogs during recent times, often struggling to combine personal, private, profesional and random information into various places, usually without much success, and only fragmenting the pieces of my online life.

This site is now the definitive resource for anything that’s not of a private nature or not of general public interest, and it will contain news, rants, ramblings and otherwise hopefully informative data. I will also post up some biographical information and resume, and and detailed contact information.

I hope you’ll find this site a valuable resource to keep yourself informed of my activities and interests. I will probably blog on random topics, but you can expect to find:

  • Mirrors of my articles, presentations and blog-entries at OpenRCE.org.
  • NT Kernel/Native Mode or otherwise low-level information about Windows NT.
  • Development news from ReactOS and TinyKRNL.
  • Personal projects and achievements.
  • Reversing and coding hints and tips.
  • Links to valuable resources.

As always, don’t hesitate to contact me if you have any questions, or simply leave a comment.