Month: May 2007

As noted in the introductory article, Windows Vista (and XP) ship with a number of default shims which are not exposed through any control panel or dialog available to end-users. Running the CDD Utility however, one can see all the shims installed in the defalt system database (sysmain.sdb):

Compatibility Database Dumper (CDD) v1.0
Copyright (C) 2007 Alex Ionescu
http://www.alex-ionescu.com

usage: cdd.exe [-s][-e][-l][-f][-p][-d kernel-mode database file][-a user-mode database file]
  -s Show shims
  -e Show executables
  -l Show layers
  -f Show flags
  -p Show patches
  -d Use Blocked Driver Database from this path
  -a Use Application Compatibility Database from this path

NOTE: If no paths are given, the default system database is used.

Dumping Entry: SHIM

NAME=”pGetProcAddrExOverride”
DLLFILE=”ShimengInternal”
DESCRIPTION=”Our internal hook for GetProcAddress used to not check include/exclude list at all which means the GetProcAddress calls from all modules are shimmed. Then we added code to take include/exclude list into consideration and that “broke” apps that used to rely on the previous behavior. To compensate for this, you can specify this shim to get back the old behavior.”
Dumping Entry: SHIM

NAME=”3DFrogFrenzy”
DLLFILE=”AcSpecfc.DLL”

The utility continues to dump several dozen other shims. It’s still in beta for now, so the final output might not match, but it allows us to build a list of several interesting system shims, which I’ll enumerate below.  Caveat: my criteria was a mix between usefulness, interesting security implications, and completely out-of-this-world, bizare or uber-hack shims. The ones in bold are some of my favorite, but you should defintely read through them all. Once the tool is completed, you’ll be able to dump your own.

NAME=”AddProcessParametersFlags”
DLLFILE=”AcGenral.DLL”
DESCRIPTION=”Add flags to Peb-ProcessParameter-Flags. The flags are a ULONG. Specify it as a hex number (so at most 8 digits).”
NAME=”APILogger”
DLLFILE=”LogExts.dll”
DESCRIPTION=”Logs API calls made by the application to an .LGV file in %windir%\AppPatch. You must copy LogExts.dll, LogViewer.exe and the Manifest directory to %windir%\AppPatch in order for this shim to function properly.”

NAME=”ChangeAuthenticationLevel”
DLLFILE=”AcLayers.DLL”
DESCRIPTION=”Changes COM Security Level from RPC_C_AUTHN_LEVEL_NONE to RPC_C_AUTHN_LEVEL_COMMON. This enables temporary elevation of the security context for an application.

NAME=”DelayDllInit”
DLLFILE=”AcGenral.DLL”
DESCRIPTION=”Some applications may use static DLLs, which could potentially issue calls to APIs before the application is ready. This compatibility fix provides a workaround for this behavior by causing a delay in the application’s static DLLs. This compatibility fix takes a command line containing a list of the DLLs affected. They will be loaded in the reverse order of the command line listing. Note that this compatibility fix is similar to InjectDll, which works with dynamically loaded DLLs.”

NAME=”DeRandomizeExeName”
DLLFILE=”AcGenral.DLL”
DESCRIPTION=”Some installation programs will create a randomly named executable when they are launched that is responsible for performing the actual setup. This compatibility fix takes a command line that specifies what random executable name is created, and upon creation, renames it to the new name specified on the command line. The command line is given as the source name followed by the desired name. For example: *.EXE;RANDOMSETUP.EXE.”

NAME=”DisableNX”
DLLFILE=”AcGenral.DLL”
DESCRIPTION=”This compatibility fix disables execution protection (NX) for a process. This is useful for applications that decide to execute from memory region marked with NX attribute (like stack, heap etc).”

NAME=”DisableSeh”
DLLFILE=”AcGenral.DLL”
DESCRIPTION=”Disable safe exception handling.”

NAME=”EmulateEnvironmentBlock”
DLLFILE=”AcLayers.DLL”
DESCRIPTION=”This compatibility fix causes Windows XP to return a significantly reduced environment block from the environment APIs. This reduces the chance of a buffer overrun causing corruption.”

NAME=”EmulateHeap”
DLLFILE=”AcLayers.DLL”
DESCRIPTION=”This compatibility fix emulates the functionality of the Windows 9x heap manager. It is is full implementation of the Windows 9x heap manager ported to Windows XP.”

NAME=”EmulateUSER”
DLLFILE=”AcLayers.DLL”
DESCRIPTION=”Fixes for known API differences between Win9x and NT: SetWindowsHookEx, SetWindowLong, RegisterClass, ChangeDisplaySettings/ChangeDisplaySettingsEx, ToAscii/ToAsciiEx, GetMessage/PeekMessage, ShowWindow. Also persists palette state through mode changes.”

NAME=”EnableRestarts”
DLLFILE=”AcLayers.DLL”
DESCRIPTION=”In Windows 9x applications could restart the computer by calling the ExitWindowsEx API. Windows XP requires the application to run with adequate security privileges to successfully call the ExitWindowsEx API. This compatibility fix enables an application to call the ExitWindowsEx API with correct security privileges. Applies to: Windows 95, Windows 98″

NAME=”FixServiceStartupCircularDependency”
DLLFILE=”AcGenral.DLL”
DESCRIPTION=”A service startup circular dependency occurs when two or more installed services depend upon each other to start. That is, service ‘A’ cannot start until service ‘B’ starts, but service ‘B’ cannot start without service ‘A’ running. This compatibility fix attempts to counter this behavior.”

NAME=”ForceAdminAccess”
DLLFILE=”AcGenral.DLL”
DESCRIPTION=”This compatibility fix addresses issues that may be encountered when an application uses the CheckTokenInformation API call to verify if the current user is part of the Administrators group. The fix intercepts calls to CheckTokenInformation and returns a value of true.”

NAME=”HandleAPIExceptions”
DLLFILE=”AcLayers.DLL”
DESCRIPTION=”This compatibility fix addresses issues with APIs that may not gracefully handle receiving bad parameters. Currently, this works with the BackupSeek, CreateEvent, and GetFileAttributes APIs.”

NAME=”HandleWvsprintfExceptions”
DLLFILE=”AcLayers.DLL”
DESCRIPTION=”This compatibility fix provides a facility to convert the argument list from LPSTR into VA_LIST. Some native Windows 9x applications use LPSTR instead of VA_LIST. Without properly checking the return value, these applications may assume that it is safe to use Wvsprintf, but in Windows XP, this may cause an access violation. This compatibility fix takes one command line: “arglistfix” (case insensitive).”

NAME=”HeapClearAllocation”
DLLFILE=”AcGenral.DLL”
DESCRIPTION=”This compatibility fix will clear out every heap allocation for the application with zeroes, or with a DWORD value that has been supplied in the command line.”

NAME=”HeapDelayLocalFree”
DLLFILE=”AcGenral.DLL”
DESCRIPTION=”This compatibility fix will delay calls to LocalFree. This may help applications that are trying to free heap memory using LocalFree before all activities have been concluded.”

NAME=”IgnoreCRTExit”
DLLFILE=”AcGenral.DLL”
DESCRIPTION=”Prevent CRT shutdown routines from running.”

NAME=”IgnoreLoadLibrary”
DLLFILE=”AcLayers.DLL”
DESCRIPTION=”This compatibility fix will prevent specified DLLs from being loaded by the LoadLibrary API, specified on the command line. If specifying multiple DLLs on the command line, they should be seperated by spaces. This may be useful for applications that have fallback mechanisms for features that are not supported. In addition, it reduces the error mode so library problems won’t cause the system to generate an error dialog. Applies to: Windows 95, Windows 98″

NAME=”IgnoreMessageBox”
DLLFILE=”AcGenral.DLL”
DESCRIPTION=”This compatibility fix intercepts calls to the MessageBox API and, based on the supplied command line, prevents the message box from being displayed. Many applications display a message box with debugging or other extraneous content that can be confusing to users. These are normally the result of differences between Windows 9x and Windows XP.”

NAME=”IgnoreVBOverflow”
DLLFILE=”AcGenral.DLL”
DESCRIPTION=”Some VB apps try to store win32 handles in WORD size variables. On Win9x this works because most handles are 16-bit. However, on NT, the VB type checking code throws a “Runtime Error 6″. The shim intercepts the type checking code and ignores the check.”

NAME=”RedirectEXE”
DLLFILE=”AcGenral.DLL”
DESCRIPTION=”This compatibility fix calls WinExec on the passed command line, and then terminates the caller process. The command line can contain any environment variables that need to be passed to the executable.”

NAME=”Shrinker”
DLLFILE=”AcLayers.DLL”
DESCRIPTION=”This compatibility fix fixes problems with any application that uses the Shrinker resource compression library. This library hacks resource functions in ntdll and kernel32 and redirect calls into their own function routines. But Ntdll code has different opcodes in Windows XP. The program failed to find the opcode signature and decided to cancel WriteProcessMemory call to write their redirection. Because of this, the necessary decompression of program code and resources were not executed and caused access violation. Shrinker compatibility fix resolves this by providing necessary opcode signature so the app could write those redirection into ntdll.”

NAME=”StackSwap”
DLLFILE=”AcGenral.DLL”
DESCRIPTION=”Many APIs use much more stack space on NT than Win9x. This compatibility fix is command line driven and takes a list of APIs that will be hooked, making them use no stack. The format the command line is “MODULENAME!APINAME[:X]; MODULENAME!APINAME[:X] …” where X is 0 : fill old stack with zeroes 1 : fill old stack with pointers 2 : fill old stack with pointers to pointers by default, no stack filling occurs.”

NAME=”TerminateExe”
DLLFILE=”AcGenral.DLL”
DESCRIPTION=”This compatibility fix terminates an executable (.EXE) immediately upon launch.”

NAME=”VirtualRegistry”
DLLFILE=”AcLayers.DLL”
DESCRIPTION=”Hooks all the registry functions to allow virtual keys, redirection and expansion values.”

As you can see, the Shim Engine allows from the simplest of hacks (such as adding PEB flags) to complete ports of 9x APIs (such as the Heap Manager). Many other shims are simply extremly useful features that should be accessible easier. The ability to deal with random setup application names is something I’ve had to code on my own in the past, and the VirtualRegistry shim in XP seems to be almost as powerful as the built-in Vista feature. Yet others, dealing with delay loading DLLs, instantly killing, and redirection can be lifesavers during certain debugging scenarios.

For now, these shims have only been presented. Later series will deal with actually using this shims, but for now, we’ll have to continue exploring the system further inside the next article.

For the last few days, I’ve been intimately becoming aquainted with a piece of technology in Windows XP and Vista that rarely gets the attention it deserves. It has raised my esteem and admiration towards Microsoft ten fold, and I feel it would be wise to share it, publicize it, and then of course, find (positive) ways to exploit it to turn it into a powerful backend for various purposes.

The Shim Engine, which is how I’ll call it (and is one of the official names), is a technology implemented in various DLLs (mostly shimeng.dll and apphelp.dll — which is the Application Compatibility Interface) as well as through some callbacks and hacks in the PE Loader present in ntdll.dll. It also contains various registry entries for its configuration, as well as system database files.

What does this technology do? You’ve probably seen it in action when using Windows XP/Vista’s “Compatibility Wizard”, or the dialog which gives you options such as “Disable visual themes”, “Run application in Windows 2000” mode or “run at 640×480”. The checkboxes are called “shims”, while the actual Windows 2000 or Windows 98 combo box selections are called “layers”. Although this is hidden from you, layers are usually simple large combinations of other shims, each which somehow modify the system to behave in a different way. Unfortunately, this dialog contains only 3 shims, while over 100 are present by default on a Windows installation.

However, it is enough to illustrate how the technology works. Once an application has been “shimmed” manually, registry entries are created to notify the loader. As it loads, the loader will run the Shim Engine, which will perform lookups in the system compatibility database, recovering various information. This database is called sysmain.sdb, and it is located in your AppPatch directory. On top of the default database, individual, custom databases can be created, which are registered and installed through the registry. These specify settings for programs that you’ve manually chosen to be shimmed.

The way that shims are implemented is usually through a helper DLL, which the Shim Engine will load during PE Loading, and intercept the APIs being used, much like Detours. These DLLs are prefixed “Ac” and are also in the AppPatch directory. They contain the redirected code which behaves differently then the normal system API.

The most interesting part however, is not the ability to select these options, but the fact of how much of this is being done behind your back every single time you run an application. Upon analysis, the system database contains over 5000 applications (in Windows Vista) from small Chinese publishers to the largest application vendors, including Microsoft itself.

One of the core “objects” that the database supports is the Matching File construct, which does file pattern matching to identify whether or not this entry actually applies to the program being run. These pattern matches can go from the very simple “starcraft.exe” with a timestamp and checksum entry, to the more complex entries which try to match various .bmps, .wavs and data files present in a game’s engine. Wildcards and simple boolean logic is also supported, making for powerful pattern matching abilities.

Once a matching Executable construct has been found through its children Matching Files, 4 different types of modifications can be done. The first are system shims, which are implemented typically in the acgenral.dll or aclayers.dll library, and that many products might benefit from, such as emulating an older version of an API. The second are specific shims, which are tailored to an application, and located in acspecfic.dll. The third kind is also a shim, but a Flag shim, which specifices undocumented flags which are to be sent to LUA or the Installer about this application. Finally, the fourth type of change is a binary patch, which represents actual in-memory patching on the executable, instead of a system API hook.

Sound interesting and powerful? It is. I’ll spend the next few blog entries talking more about the various parts of the system, as well as offering two applications that I’ve been writing on. The first is a complete dumper of any .SDB database, and the second will be announced at the end. Here’s an overview of the different posts that I’m expecting to make:

1 ) Introduction (You are here).
2 ) System Shims – The Most Interesting Ones.
3 ) The Private Shim Engine Interface With The PE Loader.
4 ) Built-in Shimmed Applications and Specific Shims – A Sample.
5 ) Tool 1 – CDD – Compatibility Database Dumper
6 ) Flag Shims – LUA and Installer Flags.
7 ) The Run-Time In-Memory Patching Behaviour and Analysis
8 ) The System Blocked Driver Database – The Kernel Side of SDB.
9 ) Conclusion and Tool 2.