Unknow - The Undocumented Functions Windows Nt-2000-XP Kernel Mode

Part1^Essentials

CHAPTER 1Windows NT: An Inside Look

CHAPTER 2Writing Windows NT Device Drivers

CHAPTER 3Win32 Implementations: A Comparative Look

CHAPTER 4Memory Management

CHAPTER 5Reverse Engineering Techniques

Chapter 1

Windows NT: An InsideLook

IN THIS CHAPTER

+ Evaluating Windows NT - ^^^^+ Delving into the Windows NT architecture ^^ ^

THIS BOOK IS AN EXPLORATION of the internals of the Windows NT operating system.Before entering the jungle of Windows NT internals, an overview of the topic isnecessary. In this chapter, we explain the overall structure of the Windows NT op-erating system.

Evaluating Windows NTThe qualities of an operating system are the result of the way in which the operat-ing system is designed and implemented. For an operating system to be portable,extensible, and compatible with previous releases, the basic architecture has tobe well designed. In the following sections, we evaluate Windows NT in light ofthese issues.

PortabilityAs you know, Windows NT is available on several platforms, namely, Intel, MIPS,Power PC, and DEC Alpha. Many factors contribute to Windows NT's portability.Probably the most important factor of all is the language used for implementation.Windows NT is mostly coded in C, with some parts coded in C++. Assembly lan-guage, which is platform specific, is used only where necessary. The Windows NTteam also isolated the hardware-dependent sections of the operating system inHAL.DLL. As a result, the hardware-independent portions of Windows NT can becoded in a high-level language, such as C, and easily ported across platforms.

3

Part 1: Essentials

ExtensibilityWindows NT is highly extensible, but because of a lack of documentation, its ex-tensibility features are rarely explored. The list of undocumented features startswith the subsystems. The subsystems provide multiple operating system interfacesin one operating system. You can extend Windows NT to have a new operating sys-tem interface simply by adding a new subsystem program. Windows NT providesWin32, OS/2, POSIX, Winl6, and DOS interfaces using the subsystems concept, butMicrosoft keeps mum when it comes to documenting the procedure to add a newsubsystem.

The Windows NT kernel is highly extensible because of dynamically loadablekernel modules that are loaded as device drivers. In Windows NT, Microsoft pro-vides enough documentation for you to write hardware device drivers-that is,hard disk device drivers, network card device drivers, tape drive device drivers, andso on. In Windows NT, you can write device drivers that do not control any hard-ware device. Even file systems are loaded as device drivers under Windows NT.

Another example of Windows NT's extensibility is its implementation of the sys-tem call interface. Developers commonly modify operating system behavior byhooking or adding system calls. The Windows NT development team designed thesystem call interface to facilitate easy hooking and adding of system calls, butagain Microsoft has not documented these mechanisms.

CompatibilityDownward compatibility has been a long-standing characteristic of Intel's micro-processors and Microsoft's operating systems, and a key to the success of these twogiants. Windows NT had to allow programs for DOS, Win 16, and OS/2 to run unal-tered. Compatibility is another reason the NT development team went for the sub-system concept. Apart from binary compatibility, where the executable has to beallowed to run unaltered, Windows NT also provides source compatibility forPOSIX-compliant applications. In another attempt to increase compatibility,Windows NT supports other file systems, such as the file allocation table (FAT) filesystem from DOS and the High Performance File System (HPFS) from OS/2, inaddition to the native NT file system (NTFS).

MaintainabilityWindows NT is a big piece of code, and maintaining it is a big job. The NT devel-opment team has achieved maintainability through an object-oriented design. Also,the breakup of the operating system functionality into various layers improvesmaintainability. The topmost layer, which is the one that is seen by the users of theoperating system, is the subsystems layer. The subsystems use the system call inter-face to provide the application programming interface (API) to the outside world.Below the system call interface layer lies the NT executive, which in turn rests on

Chapter 1: Windows NT: An Inside Look

the kernel, which ultimately relies on the hardware abstraction layer (HAL) thattalks directly with the hardware.

The NT development team's choice of programming language also contributes toWindows NT's maintainability. As we stated previously, the entire operating systemhas been coded in C and C++, except for a few portions where the use of assemblylanguage was inevitable.

Plus Points over Windows 95/98Microsoft has come up with two 32-bit operating systems: Windows 95/98 andWindows NT. Windows NT is a high-end operating system that offers additional fea-tures separate from those provided by conventional PC or desktop operating systems,such as process management, memory management, and storage management.

SecurityWindows NT is a secure operating system based on the following characteristic: Auser needs to log in to the system before he or she can access it. The resources inthe system are treated as objects, and every object has a security descriptor associ-ated with it. A security descriptor has access control lists attached to it that dictatewhich users can access the object.

All this being said, a secure operating system cannot be complete without a se-cure file system, and the FAT file system from the days of DOS does not have anyprovision for security. DOS, being a single-user operating system, did not careabout security.

In response to this shortcoming, the Windows NT team came up with a new filesystem based on the HPFS, which is the native file system for OS/2. This new nativefile system for Windows NT, known as NTFS, has support for access control. A usercan specify the access rights for a file or directory being created under NTFS, andNTFS allows only the processes with proper access rights to access that file ordirectory.

Keep in mind that no system is 100 percent secure. Windows NT, althoughremarkably secure, is not DoD compliant. (For the latest news on DoD com-pliance, check out h t t p : / / w w w . f c w . c o m / p u b s / f c w / 1 9 9 8 / 0 7 2 7 /f c w - n e w s d o d s e c - 7 - 2 7 - 9 8 . htm.)

MultiprocessingWindows NT supports symmetric multiprocessing, the workstation version ofWindows NT can support two processors, and the server version of Windows NT cansupport up to four processors. The operating system needs special synchronization

Parti: Essentials

constructs for supporting multiprocessing. On a single-processor system, criticalportions of code can be executed without interruption by disabling all the hardwareinterrupts. This is required to maintain the integrity of the kernel data structures. Ina multiprocessor environment, it is not possible to disable the interrupts on allprocessors. Windows NT uses spin locks to protect kernel data structures in a multi-processor environment.

Multiprocessing can be classified as asymmetric and symmetric. In asymmet-ric multiprocessing, a single processor acts as the master processor and theother processors act as slaves. Only the master processor runs the kernelcode, while the slaves can run only the user threads. Whenever a thread run-ning on a slave processor invokes a system service, the master processortakes over the thread and executes the requested kernel service.The sched-uler, being a kernel code, runs only on the master processor.Thus, the masterprocessor acts as the scheduler, dispatching user mode threads to the slaveprocessors. Naturally, the master processor is heavily loaded and the systemis not scalable. Compare this with symmetric multiprocessing, where anyprocessor can run the kernel code as well as the user code.

International Language SupportA significant portion of PC users today use languages other than English. The keyto reaching these users is to have the operating system support their languages.Windows NT achieves this by adopting the Unicode standard for character sets. TheUnicode standard has 16-bit character set, while ASCII uses an 8-bit character set.The first 256 characters in Unicode match the ASCII character set. This leavesenough space for representing characters from non-Latin scripts and languages.The Win32 API allows Unicode as well as ASCII character sets, but the Windows NTkernel uses and understands only Unicode. Although the application programmercan get away without knowing Unicode, device driver developers need to be famil-iar with Unicode because the kernel interface functions accept only Unicode stringsand the driver entry points are supplied with Unicode strings.

MultiprogrammingWindows NT 3.51 and Windows NT 4.0 lack an important feature, namely, the sup-port for remote login or Telnet of a server operating system. Both these versionsof Windows NT can operate as file servers because they support the commonInternet file system (CIFS) protocol. But they cannot act as CPU servers becauselogging into a Windows NT machine over the network is not possible.Consequently, only one user can access a Windows NT machine at a time. Windows

Chapter 1: Windows 1MT: An Inside Look

2000 plans to overcome this deficiency by providing a Telnet server along with theoperating system. This will enable multiple programmers to log in on the machineat the same time, making Windows 2000 a true server operating system.

Third-party Telnet servers are available for Windows NT 3.51 andWindows NT 4.0. However, Microsoft's own Telnet server comesonly with Windows 2000.

Delving into the Windows NTArchitectureWindows NT borrows its core architecture from the MACH operating system, whichwas developed at Carnegie Mellon University. The basic approach of the MACH op-erating system is to reduce the kernel size to the minimum by pushing complex op-erating system functionality outside the kernel onto user-level server processes.This client-server architecture of the operating system serves yet another purpose:It allows multiple APIs for the same operating system. This is achieved by imple-menting the APIs through the server processes.

The MACH operating system kernel provides a very simple set of interface func-tions. A server process implementing a particular API uses these interface functionsto provide a more complex set of interface functions. Windows NT borrows thisidea from the MACH operating system. The server processes in Windows NT arecalled as the subsystems. NT's choice of the client-server architecture shows itscommitment to good software management principles such as modularity andstructured programming. Windows NT had the option to implement the requiredAPIs in the kernel. Also, the NT team could have added different layers on top ofthe Windows NT kernel to implement different APIs. The NT team voted in favor ofthe subsystem approach for purposes of maintainability and extensibility.

The SubsystemsThere are two types of subsystems in Windows NT: integral subsystems and envi-ronment subsystems. The integral subsystems, such as the security manager subsys-tem, perform some essential operating system task. The environment subsystemsenable different types of APIs to be used on a Windows NT machine. Windows NTcomes with subsystems to support the following APIs:

+ Win32 Subsystem. The Win32 subsystem provides the Win32 API. Theapplications conforming to the Win32 API are supposed to run unalteredon all the 32-bit platforms provided by Microsoft - that is, Windows NT,Windows 95, and Win32s. Unfortunately, as you will see later in thisbook, this is not always the case.

Part 1: Essentials

+ WOW Subsystem. The Windows on Windows (WOW) subsystem providesbackward compatibility to 16-bit Windows applications, enabling Win 16applications to run on Windows NT. These applications can run onWindows NT unless they use some of the undocumented API functionsfrom Windows 3.1 that are not defined in Windows NT.

+ NTVDM Subsystem. The NT Virtual DOS Machine (NTVDM) provides atext-based environment where DOS applications can run.

+ OS/2 Subsystem. The OS/2 subsystem enables OS/2 applications to run.WOW, NTVDM, and OS/2 are available only on Intel platforms becausethey provide binary compatibility to applications. One cannot run theexecutable files or binary files created for one type of processor on anothertype of processor because of the differences in machine code format.

+ POSIX Subsystem. The POSIX subsystem provides API compliance to thePOSIX 1003.1 standard.

The applications are unaware of the fact that the API calls invoked are processedby the corresponding subsystem. This is hidden from the applications by the re-spective client-side DLLs for each subsystem. This DLL translates the API call into alocal procedure call (LPC). LPC is similar to the remote procedure call (RFC) facilityavailable on networked Unix machines. Using RFC, a client application can invokea function residing in a server process running on another machine over the net-work. LPC is optimized for the client and the server running on the same machine.

THE WIN32 SUBSYSTEMThe Win32 subsystem is the most important subsystem. Other subsystems such asWOW and OS/2 are provided mainly for backward compatibility, while the POSIXsubsystem is very restrictive in functionality. (For example, POSIX applications donot have access to any network that exists.) The Win32 subsystem is important be-cause it controls access to the graphics device. In addition, the other subsystems areactually Win32 applications that use the Win32 API to provide their own differentAPIs. In essence, all the subsystems are based on the core Win32 subsystem.

The Win32 subsystem in Windows NT 3.51 contains the following components:

+ CSRSS.EXE. This is the user mode server process that serves the USER andGDI calls.

Traditionally, Windows API calls are classified as user/gdi calls and kernelcalls.The majority of user/gdi functions are related to the graphical user in-terface (GUI) and reside in USER.DLL under Windows 3.x. The kernel func-tions are related to non-GUI O/S services — such as file system managementand process management — and reside in KERNEL.EXE under Windows 3.x.

Chapter 1: Windows NT: An Inside Look

+ KERNEL32.DLL. The KERNEL.EXE in Windows 3.1 has changed toKERNEL32.DLL in Windows NT. This is more than a change in name. TheKERNEL.EXE contained all the kernel code for Windows 3.1, whileKERNEL32.DLL contains just the stub functions. These stub functions callthe corresponding NTDLL.DLL functions, which in turn invoke system callcode in the kernel. • <- ' ••

+ USER32.DLL. This is another client-side DLL for the Win32 subsystem.The majority of the functions in USER32.DLL are stub functions thatconvert the function call to an LPC for the server process.

+ GDI32.DLL. The functions calls related to the graphical device interfaceare handled by another client-side DLL for the Win32 subsystem. Thefunctions in GDI32.DLL are similar to those in USER32.DLL in that theyare just stubs invoking LPCs for the server process.

Under Windows NT 4.0 and Windows 2000, the functionality of CSRSS is movedinto a kernel mode driver (WIN32K.SYS) and USER32 and GDI32 use the systemcalls interface to call the services in WIN32K.SYS.

The CoreWe have to resort to new terminology for explaining the kernel component of theWindows NT operating system. Generally, the part of an operating system that runsin privileged mode is called as the kernel. The Windows NT design team strove toachieve a structured design for the operating system. The privileged-mode compo-nent of Windows NT is also designed in a layered fashion. A layer uses only the func-tions provided by the layer below itself. The main layers in the Windows NT core arethe HAL, the kernel, and the NT executive. Because one of the layers running in priv-ileged mode is itself called as the kernel, we had to come up with a new term thatrefers to all these layers together. We'll refer to it as the core of Windows NT.

Most modern microprocessors run in at least two modes: normal and privi-leged. Some machine instructions can be executed only when the processoris in privileged mode. Also, some memory area can be marked as "to be ac-cessed in privileged mode only."The operating systems use this feature ofthe processors to implement a secure operating environment for multitask-ing.The user processes run in normal (nonprivileged) mode, and the operat-ing system kernel runs in privileged mode. Thus, the operating systemensures that user processes cannot harm the operating system.

10 Part 1: Essentials

This division of the Windows NT core into layers is logical. Physically, only theHAL comes as a separate module. The kernel, NT executive, and the system calllayer are all packed in a single NTOSKRNL.EXE (or NTKRNLMP.EXE, for multi-processor systems). Though they are considered part of the NT executive in thischapter, the device drivers (including the file system drivers) are separate drivermodules and are loaded dynamically.

THE HAL ' 'The lowest of the aforementioned layers is the hardware abstraction layer, whichdeals directly with the hardware of the machine. The HAL, as its name suggests,hides hardware idiosyncrasies from the layers above it. As we mentioned previ-ously, Windows NT is a highly portable operating system that runs on DEC Alpha,MIPS, and Power-PC, in addition to Intel machines. Along with the processor, theother aspects of a machine, such as the bus architecture, interrupt handling, andDMA management also change. The HAL.DLL file contains the code that hides theprocessor- and machine-specific details from other parts of the core. The kernelcomponent of the core and the device drivers use the HAL interface functions. Thus,only the HAL code changes from platform to platform; the rest of the core code thatuses the HAL interface is highly portable.

THE KERNELThe kernel of Windows NT offers very primitive but essential services such as mul-tiprocessor synchronization, thread scheduling, interrupt dispatching, and so on.The kernel is the only core component that cannot be preempted or paged out.All the other components of the Windows NT core are preemptive. Hence, underWindows NT, one can find more than one thread running in privilegedmode. Windows NT is one of the few operating systems in which the core is alsomultithreaded.

A very natural question to ask is "Why is the kernel nonpreemptive and non-pageable?" Actually, you can page out the kernel, but a problem arises when youpage in. The kernel is responsible for handling page faults and bringing in the re-quired pages in memory from secondary storage. Hence, the kernel itself cannot bepaged out, or rather, it cannot be paged in if it is paged out. The same problem pre-vents the disk drivers supporting the swap space from being pageable. As the ker-nel and the device drivers use the HAL services, naturally, the HAL is alsononpreemptive.

THE NT EXECUTIVEThe NT executive constitutes the majority of the Windows NT core. It sits on top ofthe kernel and provides a complex interface to the outside world. The executive isdesigned in an object-oriented manner. The NT executive forms the part of theWindows NT core that is fully preemptive. Generally, the core components added bydevelopers form a part of the NT executive or rather the I/O Manager. Hence, driverdevelopers should always keep in mind that their code has to be fully preemptive.

Chapter 1: Windows MT: An Inside Look 11

The NT executive can further be subdivided into separate components that im-plement different operating system functionality. The various components of theexecutive are described in the following sections. *.-',,'

THE OBJECT MANAGER Windows NT is designed in an object-oriented fashion.Windows, devices, drivers, files, mutexes, processes, and threads have one thing incommon: All of them are treated as objects. In simpler terms, an object is the databundled with the set of methods that operate on this data. The Object Managermakes the task of handling objects much easier by implementing the commonfunctionality required to manage any type of object. The main tasks of the ObjectManager are as follows:

+ Memory allocation/deallocation for objects.

+ Object name space maintenance. The Windows NT object name space isstructured as a tree, just like a file system directory structure. An object

- name is composed of the entire directory path, starting from the rootdirectory. The Object Manager is responsible for maintaining this object

'. name space. Unrelated processes can access an object by getting a handleto it using the object's name.

+ Handle maintenance. To use an object, a process opens the object andgets back a handle. The process can use this handle to perform furtheroperations on the object. Each process has a handle table that ismaintained by the Object Manager. A handle table is nothing more thanan array of pointers to objects; a handle is just an index in this array.

- When a process refers to a handle, the Object Manager gets hold of theactual object by indexing the handle in the handle table.

+ Reference count maintenance. The Object Manager maintains a referencecount for objects, and automatically deletes an object when thecorresponding reference count drops to zero. The user mode code accessesobjects via handles, while the kernel mode code uses pointers to directlyaccess objects. The Object Manager increments the object reference countfor every handle pointing to the particular object. The reference count isdecremented whenever a handle to the object is closed. Whenever thekernel mode code references an object, the reference count for that objectis incremented. The reference count is decremented as soon as the kernelmode code is finished accessing the object.

+ Object security. The Object Manager also checks whether a process isallowed to perform a certain operation on an object. When a processcreates an object, it specifies the security descriptor for that object. Whenanother process tries to open the object, the Object Manager verifieswhether the process is allowed to open the object in the specified mode.The Object Manager returns a handle to the object if the open requestsucceeds. As described earlier, a handle is simply an index in a per-process

12 Parti: Essentials

table that has pointers to actual objects. The mode in which the openrequest on an object is granted is stored in the handle table along with theobject pointers. Later, when the process tries to access the object using thehandle, the Object Manager ensures that proper access rights are associatedwith the handle.

THE 1/0 MANAGER The I/O Manager controls everything related to input and out-put. It provides a framework that all the I/0-related modules (device drivers, filesystems, Cache Manager, and network drivers) must adhere to.

+ Device Drivers. Windows NT supports a layered device driver model. TheI/O Manager defines a common interface that all the device drivers needto provide. This ensures that the I/O Manager can treat all the devices inthe same manner. Also, device drivers can be layered, and a device drivercan expect the same interface from the driver sitting below it. A typicalexample of layering is the device driver stack to access a hard disk. Thelowest-level driver can talk in terms of sectors, tracks, and sides. Theremay be a second layer that can deal with hard disk partitions and providean interface for dealing with logical block numbers. The third layer can bea volume manager driver that can club several partitions into volumes.Finally, a file system driver that provides an interface to the outside worldcan sit on top of the volume manager.

+ File Systems. File systems are also coded as loadable device drivers underWindows NT. Consequently, a file system can be stacked on top of a diskdevice driver. Also, multiple file systems can be layered in such a mannerthat each layer adds to the functionality. For example, a replication filesystem can be layered on top of a normal disk file system. The replicationfile system need not implement the code for on-disk structuremodifications.

+ Cache Manager. In her book Inside Windows NT, Helen Custer considersthe Cache Manager part of the I/O Manager, though the Cache Managerdoes not adhere to the device driver interface. The Cache Manager isresponsible for ensuring faster file read/write response. Though hard diskspeeds are increasing, reading/writing to a hard disk is much slower thanreading/writing to RAM. Hence, most operating systems cache the filedata in RAM to satisfy the read requests without needing to read theactual disk block.

Also, a write request can be satisfied without actually writing to the disk.The actual block write happens when system activity is low. Thistechnique is called as delayed write.

Another technique called as read ahead improves response time. In this

Chapter 1: Windows NT: An Inside Look 13

technique, the operating system guesses the disk blocks that will be readin the future, depending on the access patterns. These blocks are readeven before they are requested. The Cache Manager uses the memorymapping features of the Virtual Memory Manager to implement caching.

+ Network Drivers. The network drivers have an interface standard differentfrom regular device drivers. The network card drivers stick to the networkdriver interface specification (NDIS) standard. The drivers providingtransport level interface are layered above the network card drivers andprovide transport driver interface (TDI).

THE SECURITY REFERENCE MONITOR The Security Reference Monitor is respon-sible for validating a process's access permissions against the security descriptor ofan object. The Object Manager uses the services of the Security Reference Monitorwhile validating a process's request to access any object.

THE VIRTUAL MEMORY MANAGER An operating system performs two essentialtasks:

1. It provides a virtual machine, which is easy to program, on top of rawhardware, which is cumbersome to program. For example, an operatingsystem provides services to access and manipulate files. Maintaining datain files is much easier than maintaining data on a raw hard disk.

2. It allows the applications to share the hardware in a transparent way. Forexample, an operating system provides applications with a virtual view ofthe CPU, where the CPU is exclusively allotted to the application. Inreality, the CPU is shared by various applications, and the operatingsystem acts as an arbitrator.

These two tasks are performed by the Virtual Memory Manager component ofthe operating system when it comes to the hardware memory. Modern microproces-sors need an intricate data structure setup (for example, the segment table setup orthe page table setup) for accessing the memory. The Virtual Memory Manager per-forms this task for you, which makes life easier. Furthermore, the Virtual MemoryManager enables the applications to share the physical memory transparently. Itpresents each application with a virtual address space where the entire addressspace is owned by the application.

The virtual memory concept is one of the key concepts in modern operating sys-tems. The idea behind it is as follows. In case the operating system loads the entireprogram in memory while executing it, the size of the program is severely con-strained by the size of physical memory. A very straightforward solution to theproblem is not to load the entire program in memory at one time, but to loadportions of it as and when required. A fact that supports this solution is the local-ity of reference phenomenon.


A process accesses only a small number of adjacent memory locations, if oneconsiders a small time frame.This is even more pronounced because of thepresence of looping constructs. In other words, the access is localized to asmall number of memory pages, which is the reason it is called as locality ofreference.

The operating system needs to keep only the working set of a process in memory.The rest of the address space of the process is supported by the swap space on thesecondary storage. The Virtual Memory Manager is responsible for bringing in thepages from the secondary storage to the main memory in case the process accessesa paged-out memory location. The Virtual Memory Manager is also responsible forproviding a separate address space for every process so that no process can hamperthe behavior of any other process. The Virtual Memory Manager is also responsiblefor providing shared memory support and memory-mapped files. The CacheManager uses the memory-mapping interface of the Virtual Memory Manager.

A working set is the set of memory pages that needs to be in memory for aprocess to execute without incurring too many page faults. A page fault isthe hardware exception received by the operating system when an attemptis made to access a paged-out memory location.

THE PROCESS MANAGER The Process Manager is responsible for creatingprocesses and threads. Windows NT makes a very clear distinction betweenprocesses and threads. A process is composed of the memory space along with var-ious objects (such as files, mutexes, and others) opened by the process and thethreads running in the process. A thread is simply an execution context - that is,the CPU state (especially the register contents). A process has one or more threadsrunning in it.

THE LOCAL PROCEDURE CALL FACILITY The local procedure call (LPC) facility isspecially designed for the subsystem communication. LPC is based on remote pro-cedure call (RFC), which is the de facto Unix standard for communication betweenprocesses running on two different machines. LPC has been optimized for commu-nication between processes running on the same machine. As discussed earlier, theLPC facility is used as the communication mechanism between the subsystems andtheir client processes. A client thread invokes LPC when it needs some service fromthe subsystem. The LPC mechanism passes on the parameters for the service invo-cation to the server thread. The server thread executes the service and passes the re-sults back to the client thread using the LPC facility.

Chapter 1: Windows NT: An Inside Look 15

WIN32K.SYS: A Core Architecture ModificationIn Windows NT 3.51, the KERNEL32.DLL calls are translated to system calls viaNTDLLDLL, while the GDI and user calls are passed on to the Win32 subsystemprocess. Windows NT 4.0 has maintained more or less the same architecture asVersion 3.51. However, there is a major modification in the core architecture (apartfrom the completely revamped GUI).

In Windows NT 4.0, Microsoft moved the entire Win32 subsystem to the kernel spacein an attempt to improve performance. A new device driver, WIN32K.SYS, implementsthe Win32 API, and API calls are translated as system calls instead of IPCs. Thesesystem calls invoke the functions in the new WIN32K.SYS driver. Moving the servicesout of the subsystem process avoids the context switches required to process a servicerequest. In Windows NT 3.51, each call to the Win32 subsystem involves two contextswitches: one from the client thread to the subsystem thread, and the second fromthe subsystem thread back to the client thread. Windows 2000 also continues withthe kernel implementation of the Win32 subsystem.

As you will see in Chapter 8, in Windows NT 3.51 the Win32 subsystem uses quickLPC, which is supposed to be much faster than regular LPC. Still, two context switchesper GDI/user call is quite a bit of overhead. In Windows NT 4.0 and Windows 2000, theGDI/user calls are processed by the kernel mode driver in the context of the callingthread, thus avoiding the context switching overheads.

THE SYSTEM CALL INTERFACE The system call interface is a very thin layerwhose only job is to direct the system call requests from the user mode processes toappropriate functions in the Windows NT core. Though the layer is quite thin, it isa very important because it is the face of the core (kernel mode) component ofWindows NT that the outside user-mode world sees. The system call interface de-fines the services offered by the core.

The key portion of the system call interface is to change the processor modefrom user mode to privileged mode. On Intel platforms, this can be achievedthrough software interrupts. Windows NT uses the software interrupt 2Eh to im-plement the system call interface. The handing routine for interrupt 2Eh passeson the control to the appropriate routine in the core component, depending onthe requested system service ID. NTDLL.DLL is the user mode component of thesystem call interface. The user mode programs call NTDLL.DLL functions (throughKERNEL32.DLL functions). The NTDLL.DLL functions are stub routines that set upappropriate parameters and trigger interrupt 2Eh.. The stub functions in NT-DLL.DLL also pass the system service ID to the interrupt 2Eh handler. The inter-rupt handler indexes the service ID in the system call table to get to the corefunction that fulfills the requested system service. The interrupt handler calls this


core function after copying the required parameters from the user mode stack tothe kernel mode stack.

SummaryIn this chapter, we discussed the overall architecture of Windows NT. Windows NTarchitecture is robust in the areas of portability, extensibility, compatibility, andmaintainability. Features such as security, symmetric multiprocessor support, andinternational language support position the Windows NT operating system on thehigh end of the scale compared to Windows 95.

The subsystems that run in user mode and the Windows NT core that runs inkernel mode make up the operating system environment. The Win32 subsystem isthe most important of the environment subsystems. The Win32 subsystem com-prises the client-side DLLs and the CSRSS process. The Win32 subsystem imple-ments the Win32 API atop the native services provided by the Windows NT core.

The Windows NT core comprises the hardware abstraction layer (HAL), the ker-nel, the Windows NT executive, and the system call interface. The NT executive,which forms a major portion of the NT core, consists of the Object Manager, the I/OManager, the Security Reference Monitor, the Virtual Memory Manager, the ProcessManager, and the local procedure call (LPC) facility.

The chapters that follow cover the main components of the Windows NT operat-ing system in detail.

Chapter 2

Writing Windows NTDevice Drivers

IN THIS CHAPTER

+ Prerequisites to getting started

+ The building procedure

+ The structure of a device driver

Most OF THE SAMPLES IN this book are Windows NT kernel mode device drivers.This chapter contains the information you need to build device drivers and under-stand the samples in this book. This chapter is not a complete guide to writing de-vice drivers. The best sources of information for detailed coverage of the topic areArt Baker's The Windows NT Device Driver Book: A Guide for Programmers and thedocumentation that ships with the Windows NT Device Driver Kit (DDK).

Prerequisites to Writing NTDevice DriversYou must install the following tools to create a working development environmentfor Windows NT kernel mode device drivers:

Windows NT Device Driver Kit (DDK) from Microsoft For the development ofdevice drivers, you need to install the Device Driver Kit on your machine. TheDevice Driver Kit is available with the MSDN Level 2 subscription. The kit consistsof sets of header files, libraries, and tools that enable easy development of devicedrivers.

32-bit compiler You need a 32-bit compiler to compile the device drivers. Westrongly recommend using the Microsoft compiler to build the samples in this book.

Win32 Software Development Kit (SDK) Although it is not necessary forcompiling the samples from this book, we recommend installing the latest versionof the Win32 SDK on your machine. Also, when you build device drivers using 17


the DDK tools, you should set the environment variable MSTOOLS to point to thelocation where the Win32 SDK is installed. You can fake the installation of theWin32 SDK by adding the environment variable MSTOOLS with the System appletin the Control Panel.

Driver Build ProcedureThe Windows NT 4.0 Device Driver Kit installation adds four shortcuts to the Startmenu: Free Build Environment, Checked Build Environment, DDK Help, andGetting Started. The Free Build Environment and Checked Build Environmentshortcuts both refer to a batch file called SETENV.BAT, but have different commandline arguments. Assuming that the DDK is installed in directory E:\DDK40, the FreeBuild Environment shortcut refers to this command line:

%SystemRoot%\System32\cmd.exe /k E:\DDK40\bin\setenv.batE: \DDK40 free

The Checked Build Environment shortcut, on the other hand, refers to this com-mand line:

%Sys temRoo t%\Sys tem32 \cmd.exe /k E : \ D D K 4 0 \ b i n \ s e t e n v . b a t E : \DDK40checked

Both shortcuts spawn CMD.EXE and ask it to execute the SETENV.BAT file withappropriate parameters. After executing the command, CMD.EXE still keeps run-ning because of the presence of the /k switch. The SETENV.BAT file sets the envi-ronment variables, which are added to the CMD.EXE process's environmentvariable list. The DDK tools, which are spawned from CMD.EXE, refer to these en-vironment variables. SETENV.BAT sets the environment variables, includingBUILD_DEFAULT, BUILD_DEFAULT_TARGETS, BULLD_MAKE_PROGRAM, and DD-KBUILDENV.

The drivers are compiled using the utility called BUILD.EXE, which is shippedwith the DDK. This utility takes as input a file named SOURCES. This file containsthe list of source files to be compiled to build the driver. This file also contains thename of the target executable, the type of the target executable (for example, DRI-VER or PROGRAM), and the path of the directory where the target executable is tobe created.

Each sample device driver included with the DDK contains a makefile. However,this is not the actual makefile for the device driver sample. Instead, the makefile foreach sample device driver includes a common makefile, named MAKEFILE.DEF,which is present in the INC directory of the DDK installation directory.

Here is the sample makefile from the DDK sample:

_______________Chapter 2: Writing Windows NT Device Drivers____19

## DO NOT EDIT THIS FILE!!! Edit .\sources. if you want to add anew source# file to this component. This file merely indirects to the realmake file# that is shared by a l l the driver components of the Windows NT DDK#! INCLUDE $ ( N T M A K E E N V ) \ m a k e f i l e . d e f

Some of the driver samples in this book have Assembly language files (.ASMfiles). You cannot refer to the .ASM file directly into the SOURCES file. Instead, youhave to create a directory called 1386 in the directory where the source files for thedrivers are kept. All the .ASM files for the drivers must be kept in the 1386 directory.The BUILD.EXE utility automatically uses ML.EXE to compile these .ASM files.

BUILD.EXE generates the appropriate driver or application based on the settingsspecified in the SOURCES file and using the platform-dependent environment vari-ables. If there are any errors during the BUILD process, the errors are logged to afile called as BUILD.ERR. If there are any warnings, they are logged to theBUILD.WRN file. Also, the BUILD utility generates a file called BUILD.LOG, whichcontains lists of commands invoked by the BUILD utility and the messages given bythese tools.

Structure of a Device DriverJust as every Win32 application has an entry point (main/WinMain), every kernelmode device driver has an entry point called DriverEntry. A special process calledSYSTEM loads the device drivers. Hence, the DriverEntry of each device driver iscalled in the context of the SYSTEM process. Each device driver is represented by adevice name in the system, so each driver has to create a device name for its device.This is done with the loCreateDevice function. If Win32 applications need to openthe handle to a device driver, the driver needs to create a symbolic link for its de-vice in the DosDevices object directory. This is done using a call toloCreateSymbolicLink. Typically, in the DriverEntry routine of a device driver, thedevice object and the symbolic link object are created for a device and some driveror device-specific initialization is performed.

Most of the device driver samples in this book involve pseudo device drivers.These drivers do not control any physical device. Instead, they complete tasks thatcan be performed only from the device driver. (The device driver runs at the mostprivileged mode of the processor - Ring 0 in Intel processors.) In addition, theDriverEntry is supposed to provide sets of entry points for other functions, such asOPEN, CLOSE, DEVICEIOCONTROL, and so on. These entry points are provided byfilling in some fields in the device object, which is passed as a parameter to theDriverEntry function.


Because most of the drivers in this book are pseudo device drivers, theDriverEntry routine is the same for all of them. Only the device driver-specific ini-tialization is different. Instead of repeating the same piece of code in each of thedriver samples, a macro is written. The macro is called MYDRTVERENTRY:

The macro takes the following three parameters:

+ The first parameter is the name of the driver, which will be used forcreating the device name and symbolic link.

+ The second parameter is the device ID, which uniquely identifies thedevice.

+ The third parameter is the name of the function, which contains thedriver-specific initialization.

The macro expands into calling the necessary functions such as loCreateDeviceand loCreateSymbolicLink. If these functions succeed, the driver calls the driver-specific initialization function specified by the third parameter. If the function re-turns failures, the macro returns the error code of the specific initializationfunction. If the function succeeds, the macro fills in various function pointers forother functions supported by the driver in the DriverObject. Once this macro is usedin the DriverEntry function, you need to write the DriverDispatch and DriverUnloadfunctions, as the macro refers to these functions.

The macro definition can be found in UNDOCNT.H on the included CD-ROM.All the requests to device driver are sent in the form of an I/O Request packet

(IRP). The driver expects the system to call the specific driver function for all devicedriver requests based on the function pointers filled in during DriverEntry. We as-sume that all the driver functions are filled in with the address of theDriverDispatch function in the following discussion.

The DriverDispatch function is called with an IRP containing the command codeof IRP_MJ_CREATE whenever an application opens a handle to a device driver us-ing the CreateFile API call. The DriverDispatch function is called with an IRP con-taining the command code of IRP_MJ_CLOSE whenever an application closes itshandle to a device driver using the CloseHandle API function. The DriverDispatchfunction is called with an IRP containing the command code of IRP_MJ_DE-VICE_CONTROL whenever the application uses the DeviceloControl API function tosend or receive data from a device driver. If the driver functionality is being usedby multiple processes, the driver can use the CREATE and CLOSE entry points toperform per-process initialization.

Because all these requests end up calling DriverDispatch, you need to have a wayto identify the actual function requested. You can accomplish this by looking at the

Chapter 2: Writing Windows NT Device Drivers____21^


MajorFunction field in an I/O Request Packet (IRP). The request packet contains thefunction code and any other additional parameters required to complete the re-quest. The DriverUnload routine is called when the device driver is unloaded fromthe system. Just like DriverEntry, the DriverUnload function is called in the contextof the SYSTEM process. Typically, in a DriverUnload routine, the device driverdeletes the symbolic link and the device name created during DriverEntry and per-forms some device-specific uninitialization.

SummaryIn this chapter, we covered the software requirements for building Windows NT de-vice drivers, the procedure for building device drivers, and the structure of a typi-cal device driver. Along the way, we explained a simple macro that you can use togenerate the driver entry code for a typical device drive.

Chapter 3

Win32 Implementations: AComparative Look

IN THIS CHAPTER ^ ^ ^

+ Comparing and contrasting implementation of the Win32 API in WindowsNT and Windows 95

+ Aspects of implementation in both Windows NT and Windows 95

EACH OPERATING SYSTEM provides sets of services - referred to as an applicationprogramming interface (API) - to developers in some form or another. The develop-ers write software applications using this API. For example, DOS provides this in-terface in the form of the famous INT 21h interface. Microsoft's newer 32-bitoperating systems, such as Windows 95 and Windows NT, provide the interface inthe form of the Win32 API.

Presently, there are four Win32 API implementations available from Microsoft:

+ Windows 95/98 ^ ^

+ Windows NT -̂ ^ ^^^^ ^^^ ^^^ ^^ ^^^^ ^^ ^ - ^^^^^^^^^^

+ Win32s ^ ^^ ^ ^ ^^^

+ Windows CE

Of these, Win32s is very limited due to bugs and the restrictions of the underly-ing operating system. Presently, Win32 API implementations on Windows 95/98and Windows NT are very popular among developers. Windows CE is meant forpalmtop computers. The Win32 API was first implemented on the Windows NT op-erating system. Later, the same API was made available in Windows 95. Ideally, anapplication written using the standard Win32 API should work on any operatingsystem that supports the Win32 API implementation. (However, this is not neces-sarily true due to the differences between the implementations.) The Win32 APIshould hide all the details of the underlying implementations and provide a consis-tent view to the outside world.

23


In this chapter, we focus on the differences between the implementations of theWin32 API under Windows NT and Windows 95. As developers, you should beaware of these differences while you develop applications that can run on both ofthese operating systems.

Win32 API Implementation onWindows 95The Win32 API is provided in the form of the famous trio of the KERNEL32,USER32, and GDI32 dynamic link libraries (DLLs). However, in most cases, theseDLLs are just wrappers that use generic thunking to call the 16-bit functions.

Generic thunking is a way of calling 16-bit functions from a 32-bit applica-tion. (More on thunking later in this chapter.)

The major design goal for Windows 95 was backward compatibility. Hence, in-stead of porting all the 16-bit functions to 32-bit, Microsoft decided to reuse theexisting 16-bit code (from the Windows 3.x operating system) by wrapping it in 32-bit code. This 32-bit code would in turn call the 16-bit functions. This was a goodapproach because the tried — and — true 16-bit code was already running on millionsof machines all over the world. In this Win32 API implementation, most of thefunctions from KERNEL32 thunk down to KRNL386, USER32 thunks down toUSER.EXE, and GDI32 thunks down to GDI.EXE.

Win32 API implementation onWindows NTOn Windows NT also, the Win32 API is provided in the form of the famous trio ofthe KERNEL32, USER32, and GDI32 DLLs. However, this implementation is donecompletely from scratch without using any existing 16-bit code, so it is purely a32-bit implementation of Win32 API. Even 16-bit applications end up calling this3 2-bit API. Windows NT's 16-bit subsystem uses universal thunking to achieve this.

Chapter 3: Win32 Implementations: A Comparative Look 25

Universal thunking is a way of calling 32-bit functions from 16-bit applica-tions. (More on thunking later in this chapter.)

KRNL386.EXE, USER.EXE, and GDI.EXE, which are used to support 16-bit appli-cations, thunk up to KERNEL32, USER32, and GDI32 through the WOW (Windowson Windows) layer. Most of the functions provided by KERNEL32.DLL call one ormore native system services to do the actual work. The native system services areavailable through a DLL called NTDLL.DLL.

All these system services are discussed in Chapter 6.

As far as USER32 and GDI32 are concerned, the implementation differs in NTversions 3.51 and later versions. Under Windows NT 3.51, a separate subsystemprocess implements the USER32 and GDI32 calls. The DLLs USER32 and GDI32contain stubs, which pass the function parameters to the Win32 subsystem(CSRSS.EXE) and get the results back. The communication between the client ap-plication and the Win32 subsystem is achieved by using the local procedure call fa-cility provided by the NT executive.

Chapter 8 covers the details of the local procedure call (LPC) mechanism.

Under Windows NT 4.0 and Windows 2000, the USER32 GDI32 calls the systemservices provided by a kernel-mode device driver called WIN32K.SYS. USER32 andGDI32 contain stubs that call these system services using the 2Eh interrupt. Hence,most of the functionality of the Win32 Subsystem process (CSRSS.EXE) is taken overby the kernel-mode driver (WIN32K.SYS). The CSRSS process still exists in NT 4.0and Windows 2000 — however, its role is limited to mainly supporting Console I/O.


It is interesting to note that the Win32 API completely hides NTDLL.DLL fromthe developer. Actually, most of the functions provided by the Win32 API ulti-mately call one or more system services. This system service layer is very powerfuland many times contains functions that do not have equivalent Win32 API func-tions. Most of the Windows NT Resource Kit utilities link to this DLL implicitly.

Win32 Implementation DifferencesNow we will consider a few aspects of the Win32 API implementation on WindowsNT and Windows 95 that might affect the way developers program using this so-called standard Win32 API.

Address SpaceBoth Windows 95 and Windows NT deal with flat, 32-bit linear addresses that give4GB of virtual address space. Of this, the upper 2GB (hereafter referred to as theshared address space) is reserved for operating system use, and the lower 2GB(hereafter referred to as the private address space) is used by the running process.The private address space of each process is different for each process. Although thevirtual addresses in the private address space of all processes is the same, they maypoint to a different physical page. The addresses in the shared address space of allthe processes point to the same physical page.

Under Windows 95/98, the operating system DLLs, such as KERNEL32, USER32,and GDI32, reside in the shared address space, whereas in Windows NT these DLLsare loaded in the process's private address space. Hence, under Windows 95/98, it ispossible for one application to interfere with the working of another application.For example, one application can accidentally overwrite memory areas occupied bythese DLLs and affect the working of all the other processes.

Although the shared address space is protected at the page table level, akernel-mode component (for example, a VXD) is able to write at any locationin 4GB address space.

In addition, under Windows 95/98, it is possible to load a dynamic link libraryin the shared address space. These DLLs will have the same problem described pre-viously if the DLL is used by multiple applications in the system.

Windows NT loads all the system DLLs, such as KERNEL32, USER32, and GDI32,in the private address space. As a result, it is never possible for one application to

i

__________Chapter 3: Win32 Implementations: A Comparative Look____27

interfere with the other applications in the system without intending to do so. Ifone application accidentally overwrites these DLLs, it will affect only that applica-tion. Other applications will continue to run without any problems.

Memory-mapped files are loaded in the shared address space under Windows95/98, whereas they are loaded in the private address space in Windows NT. InWindows 95/98, it is possible for one application to create and map a memory-mapped file, pass its address to another application, and have the other applicationuse this address to share memory. This is not possible under Windows NT. You haveto explicitly create and map a named memory-mapped file in one application andopen and map the memory-mapped file in another application in order to share it.

The address space differences have strong impacts on global API hooking. Thetopic of global API hooking has been covered many times in different articles andbooks. There is still no common API hooking solution for both Windows NT andWindows 95/98. The basic problem with global API hooking is that under Windows95/98, it is possible to load a DLL in shared memory. Also, all the system DLLs re-side in shared memory. Hooking an API call amounts to patching the few instruc-tions at the start of function and routing them to a function in a shared DLL usinga simple JMP instruction. This does not work under Windows NT because if youpatch the bytes at the start of the function, they will be patched only in your ad-dress space as the function resides in the private address space.

To do any kind of global API hooking under Windows NT, you have to makesure that the hooking is performed in each of the running processes. For this, youneed to play with the address space of other processes. In addition, the same hook-ing also needs to be done in newly started processes. Windows NT provides a wayto automatically load a particular DLL in each process through the AppInit_DLLregistry key.

Process StartupThere are several differences in the way the process is started under Windows 95/98and Windows NT. Although the same CreateProcess API call is used in Windows95/98 and Windows NT, the implementation is quite different. In this chapter, weare looking only at an example of a CreateProcess API call. Ideally, both of theCreateProcess implementations should give the same view to the outside world.When somebody says that a particular API call is standard, this means that given a \specific set of parameters to a function, the function should behave exactly thesame on all the implementations of this API call. In addition, the function shouldreturn the same error codes based on the type of error.

Consider a simple problem such as detecting the successful start of an applica-tion. If you try to spawn a program that has some startup problem (for example,implicitly linked DLLs are missing), it should return an appropriate error code. TheWindows 95/98 implementation returns an appropriate error code such as STA-TUS_DLL_NOT_FOUND, whereas Windows NT does not return any error. Windows


NT's implementation will return an error only if the file spawned is not present atthe expected location. This happens mainly because of the way the CreateProcesscall is implemented under Windows NT and Windows 95/98. When you spawn aprocess in Windows 95/98, the complete loading and startup of the process is per-formed as part of the CreateProcess call itself. That is, when the CreateProcess callreturns, the spawned process is already running.

It is interesting to see Windows NT's implementation of the CreateProcess call.Windows NT's CreateProcess calls the native system service (NtCreateProcess) tocreate a process object. As part of this call, NTDLL.DLL is mapped in the process'saddress space. Then, the CreateProcess API calls the native system service to createthe primary thread in the process (NtCreateThread). The implicitly linked DLL load-ing does not happen as part of the CreateProcess API call. Instead, the primarythread of the process starts at a function in NTDLL.DLL. This function in turn loadsthe implicitly loaded DLLs. As a result, there is no way for the caller to knowwhether the process has started properly or not. Of course, for GUI applications,you can use WaitForlnputldle to synchronize with the startup of a process.However, for non-GUI applications, there is no standard way to achieve this.

Toolhelp FunctionsWin32 implementation on Windows 95/98 provides some functions that enableyou to enumerate the processes running in the system, module list, and so on. Thesefunctions are provided by KERNEL32.DLL. The functions are CreateToolHelp32Snapshot, Process32First, Process32Next, and others. These functions are not im-plemented under Windows NT's implementation of KERNEL32. The programs thatuse these functions implicitly will not start at all under Windows NT. The WindowsNT 4.0 SDK comes with a new DLL called PSAPI.DLL, which provides the equivalentfunctionality. The header file for this PSAPI.H is also included with the WindowsNT 4.0 SDK. Windows 2000 has this toolhelp functionality built into KERNEL32.DLL.

A function is implicitly linked if the program calls the function directly byname and includes the appropriate .LIB file in the project.That is, it does notuse GetProcAddress to get the address of the function.

MultitaskingBoth Windows 95 and Windows NT use time slice-based preemptive multitasking.However, because the Windows 95 implementation of the WIN32 API depends largelyon 16-bit code, it has a few inherent drawbacks. The major one is the Winl6Mutex.Because the existing 16-bit code is not well suited for multitasking, the easiest choice

__________Chapter 3: Win32 Implementations: A Comparative Look____ 29

for Microsoft was to ensure that the 16-bit code is not entered from multiple tasks. Toachieve this, Microsoft came up with the Winl6Mutex solution.

Before entering the 16-bit code, the operating system acquires the Winl6Mutex,and it leaves the Winl6Mutex while returning from 16-bit code. The Winl6Mutexis always acquired when a 16-bit application is running, which results in reducedmultitasking. Windows NT does not have this problem because the entire code is32-bit and is well suited for time slice-based preemptive multitasking. Also, the 16-bit code thunks up to 32-bit code in the case of Windows NT.

ThunkingThunking enables 16-bit applications to run in a 3 2-bit environment and viceversa. It is a way of calling a function written in one bitness from the code runningat a different bitness. Bitness is a property of the processor, and you can programthe processor to adjust the bitness. Bitness decides the way instructions are decodedby the processor. There are two different types of thunking available:

+ Universal thunking+ Generic thunking

Universal thunking enables you to call a 32-bit function from 16-bit code,whereas generic thunking enables you to call a 16-bit function from 32-bit code.Windows 95/98 supports both generic and universal thunking, but Windows NTsupports only universal thunking. As you saw earlier in this chapter, generic thunk-ing is used extensively in WIN32 API implementation of Windows 95/98. For ex-ample, a 32-bit USER32.DLL calls functions from a 16-bit USER.EXE, and a 32-bitGDI32.DLL calls functions from a 16-bit GDI.EXE. Various issues are involved inthunking, such as converting 16:16 far pointers in 16-bit code to flat 32-bit addressand manipulating a stack for making a proper call from code running at one bitnessto code running at a different bitness. Microsoft provides tools such as thunk com-pilers to automate most of these tasks.

Many vendors who write code for Windows 95/98 use generic thunking to avoida major redesign of their applications. For example, say a particular vendor has aproduct for Windows 3.1 and would like to port it to Windows 95. Instead ofrewriting the code for Windows 95, an easier solution is to use the majority of theexisting 16-bit code and use generic thunking as a way of calling this code from32-bit applications. However, these applications need to be rewritten for WindowsNT as Windows NT does not support generic thunking.

Device DriversDevice drivers are trusted components of the operating system that have full accessto the entire hardware. There are no restrictions on what device drivers can do.Each operating system provides some way of adding new device drivers to the sys-


tern. The device drivers need to be written according to the semantics imposed bythe operating system. The device drivers are called virtual device drivers (VXD) inWindows 95/98, and they are called as kernel-mode device drivers in Windows NT.Windows 95 uses LE file format for virtual device drivers, whereas Windows NTuses the PE format. As a result, the applications that use VXDs cannot be run onWindows NT. They need to be ported to a Windows NT (kernel-mode) device driver.

Chapter 2 explains how to write device drivers.

Microsoft has come up with a Common Driver Model in Windows 98 andWindows 2000. At this point, however, you need to port all the applications thatuse VXDs to Windows NT by writing an equivalent kernel-mode driver.

SecurityThe major WIN32 API implementation difference between Windows 95/98 andWindows NT is security. Windows 95/98's implementation does not have any supportfor security. In all the Win32 API functions that have SECURITY ATTRIBUTES as oneof the parameters, Windows 95/98's implementation just ignores these parameters.This has some impact on the way a developer programs. Registry APIs such asRegSaveKey and RegRestoreKey work fine under Windows 95/98. However, underWindows NT, you need to do a few things before you can use these functions. InWindows NT, there is a concept of privileges. There are different kinds of privileges,such as Shutdown, Backup, and Restore. Before using a function such as RegSaveKey,you need to acquire the Backup privilege. To use RegRestoreKey, you need to acquirethe Restore privilege, and to use the InitiateSystemShutdown function, you need toacquire the Shutdown privilege.

Under Windows 95/98, anybody can install a VXD. To install a kernel-mode de-vice driver under Windows NT, you need administrator privilege for security pur-poses. As mentioned previously, device drivers are trusted components of theoperating system and have access to the entire hardware. By requiring privileges toinstall a device driver, Windows NT restricts the possibility that a guest accountholder will install a device driver, which could potentially bring the whole systemdown to its knees.

Newly Added API Calls 1With each version of Windows NT, new APIs are being added to the WIN32 API set.Most of these APIs do not have an equivalent API under Windows 95/98. Also,there are a few APIs, such as CreateRemoteThread, that do not have the real imple-

__________Chapter 3: Win32 Implementations: A Comparative Look____ 31

mentation under Windows 95/98. Under Windows 95/98, this function returns ER-ROR_CALL_NOT_IMPLEMENTED. As a result, there will always be a few API callsthat are not available on Windows 95/98 or are not implemented on Windows95/98. At this point, one can only hope that Microsoft will implement the API inWindows 95/98 when they add a new API to Windows NT unless the API is archi-tecture dependent.

SummaryThis chapter covered the WIN32 API implementation on Windows 95/98 andWindows NT. We discussed the differences between these two implementationswith respect to address space, process startup, toolhelp functions, multitasking,thunking, device drivers, security, and newly added API calls.

Chapter 4

Memory ManagementIN THIS CHAPTER

+ Examining memory models in Microsoft operating systems

+ Examining how Windows NT uses features of the 80386 processor'sarchitecture

+ Exploring the function of virtual memory

MEMORY MANAGEMENT HAS ALWAYS been one of the most important and interestingaspects of any operating system for serious developers. It is an aspect that kerneldevelopers ignore. Memory management, in essence, provides a thumbnail impres-sion of any operating system.

Microsoft has introduced major changes in the memory management of eachnew operating system they have produced. Microsoft had to make these changesbecause they developed all of their operating systems for Intel microprocessors, andIntel introduced major changes in memory management support with each new mi-croprocessor they introduced. This chapter is a journey through the various Intelmicroprocessors and the memory management changes each one brought alongwith it in the operating system that used it. •

Memory Models in MicrosoftOperating SystemsEarly PCs based on Intel 8086/8088 microprocessors could access only 640K ofRAM and used the segmented memory model. Consequently, good old DOS allowsonly 640K of RAM and restricts the programmer to the segmented memory model.

In the segmented model, the address space is divided into segments. Proponentsof the segmented model claim that it matches the programmer's view of memory.They claim that a programmer views memory as different segments containingcode, data, stack, and heap. Intel 8086 supports very primitive segmentation. Asegment, in the 8086 memory model, has a predefined base address. The length ofeach segment is also fixed and is equal to 64K. Some programs find a single seg-ment insufficient. Hence, there are a number of memory models under DOS. For ex-ample, the tiny model that supports a single segment for code, data, and stack 33


together, or the small model that allows two segments - one for code and the otherfor data plus stack, and so on. This example shows how the memory managementprovided by an operating system directly affects the programming environment.

The Intel 80286 (which followed the Intel 8086) could support more than 640Kof RAM. Hence, programmers got new interface standards for accessing extendedand expanded memory from DOS. Microsoft's second-generation operating system,Windows 3.1, could run on 80286 in standard mode and used the segmented modelof 80286. The 80286 provided better segmentation than the 8086. In 80286'smodel, segments can have a programmable base address and size limit. Windows3.1 had another mode of operation, the enhanced mode, which required the Intel80386 processor. In the enhanced mode, Windows 3.1 used the paging mechanismsof 80386 to provide additional performance. The virtual 8086 mode was also usedto implement multiple DOS boxes on which DOS programs could run.

Windows 3.1 does not make full use of the 80386's capabilities. Windows 3.1 isa 16-bit operating system, meaning that 16-bit addresses are used to access thememory and the default data size is also 16 bits. To make full use of 80386's capa-bilities, a 32-bit operating system is necessary. Microsoft came up with a 32-bit op-erating system, Windows NT. The rest of this chapter examines the details ofWindows NT memory management. Microsoft also developed Windows 95 afterWindows NT. Since both these operating systems run on 80386 and compatibles,their memory management schemes have a lot in common. However, you can bestappreciate the differences between Windows NT and Windows 95/98 after we re-view Windows NT memory management. Therefore, we defer this discussion until alater section of this chapter.

Windows NT Memory ManagementOverviewWe'll first cover the view Windows NT memory management presents to the outsideworld. In the next section, we explain the special features provided by Intel micro-processors to implement memory management. Finally, we discuss how WindowsNT uses these features to implement the interface provided to the outside world.

Memory Management Interface —Programmer's ViewWindows NT offers programmers a 32-bit flat address space. The memory is notsegmented; rather, it is 4GB of continuous address space. (Windows NT marked theend of segmented architecture — programmers clearly preferred flat models to seg-mented ones.) Possibly, with languages such as COBOL where you need to declaredata and code separately, programmers view memory as segments. However, withnew languages such as C and C++, data variables and code can be freely mixed and

Chapter 4: Memory Management 35

the segmented memory model is no longer attractive. Whatever the reason,Microsoft decided to do away with the segmented memory model with WindowsNT. The programmer need not worry whether the code/data fits in 64K segments.With the segmented memory model becoming extinct, the programmer can breathefreely. At last, there is a single memory model, the 32-bit flat address space.

Windows NT is a protected operating system; that is, the behavior (or misbehav-ior) of one process should not affect another process. This requires that no twoprocesses are able to see each other's address space. Thus, Windows NT should pro-vide each process with a separate address space. Out of this 4GB address spaceavailable to each process, Windows NT reserves the upper 2GB as kernel addressspace and the lower 2GB as user address space, which holds the user-mode codeand data. The entire address space is not separate for each process. The kernel codeand kernel data space (the upper 2GB) is common for all processes; that is, the ker-nel-mode address space is shared by all processes. The kernel-mode address space isprotected from being accessed by user-mode code. The system DLLs (for example,KERNEL32.DLL, USER32.DLL, and so on) and other DLLs are mapped in user-modespace. It is inefficient to have a separate copy of a DLL for each process. Hence, allprocesses using the DLL or executable module share the DLL code and incidentallythe executable module code. Such a shared code region is protected from beingmodified because a process modifying shared code can adversely affect otherprocesses using the code.

Sharing of the kernel address space and the DLL code can be called implicit shar-ing. Sometimes two processes need to share data explicitly. Windows NT enables ex-plicit sharing of address space through memory-mapped files. A developer can mapa named file onto some address space, and further accesses to this memory area aretransparently directed to the underlying file. If two or more processes want to sharesome data, they can map the same file in their respective address spaces. To simplyshare memory between processes, no file needs to be created on the hard disk.

Below the Operating SystemIn her book Inside Windows NT, Helen Custer discusses memory management in thecontext of the MIPS processor. Considering that a large number of the readerswould be interested in a similar discussion that focuses on Intel processors, we dis-cuss the topic in the context of the Intel 80386 processor (whose memory manage-ment architecture is mimicked by the later 80486 and Pentium series). If you arealready conversant with the memory management features of the 80386 processor,you may skip this section entirely.

We now examine the 80386's addressing capabilities and the fit that WindowsNT memory management provides for it. Intel 80386 is a 32-bit processor; this im-plies that the address bus is 32-bit wide, and the default data size is as well. Hence,4GB (232 bytes) of physical RAM can be addressed by the microprocessor. Themicroprocessor supports segmentation as well as paging. To access a memory


location, you need to specify a 16-bit segment selector and a 32-bit offset withinthe segment. The segmentation scheme is more advanced than that in 8086. The8086 segments start at a fixed location and are always 64K in size. With 80386, youcan specify the starting location and the segment size separately for each segment.

Segments may overlap - that is, two segments can share address space. The nec-essary information (the starting offset, size, and so forth) is conveyed to the proces-sor via segment tables. A segment selector is an index into the segment table. Atany time, only two segment tables can be active: a Global Descriptor Table (GOT)and a Local Descriptor Table (GOT). A bit in the selector indicates whether theprocessor should refer to the LOT or the GDT. Two special registers, GDTR andLDTR, point to the GDT and the LOT, respectively. The instructions to load theseregisters are privileged, which means that only the operating system code can exe-cute them.

A segment table is an array of segment descriptors. A segment descriptor speci-fies the starting address and the size of the segment. You can also specify some ac-cess permission bits with a segment descriptor. These bits specify whether aparticular segment is read-only, read-write, executable, and so on. Each segmentdescriptor has 2 bits specifying its privilege level, called as the descriptor privilegelevel (DPL).

The processor compares the DPL with the Requested Privilege Level (RPL) beforegranting access to a segment. The RPL is dictated by 2 bits in the segment selectorwhile specifying the address. The Current Privilege Level (CPL) also plays an impor-tant role here. The CPL is the DPL of the code selector being executed. The proces-sor grants access to a particular segment only if the DPL of the segment is less thanor equal to the RPL as well as the CPL. This serves as a protection mechanism forthe operating system. The CPL of the processor can vary between 0 and 3 (because2 bits are assigned for CPL). The operating system code generally runs at CPL=0,also called as ring 0, while the user processes run at ring 3. In addition, all the seg-ments belonging to the operating system are allotted DPL=0. This arrangement en-sures that the user mode cannot access the operating system memory segments.

It is very damaging to performance to consult the segment tables, which arestored in main memory, for every memory access. Caching the segment descriptorin special CPU registers, namely, CS (Code Selector), DS (Data Selector), SS (StackSelector), and two general-purpose selectors called ES and FS, solves this problem.The first three selector registers in this list- that is, CS, DS, and SS - act as defaultregisters for code access, data access, and stack access, respectively.

To access a memory location, you specify the segment and offset within thatsegment. The first step in address translation is to add the base address of the seg-ment to the offset. This 32-bit address is the physical memory address if paging isnot enabled. Otherwise this address is called as the logical or linear address and isconverted to a physical RAM address using the page address translation mechanism(refer to Figure 4-1).


The memory management scheme is popularly known as paging because thememory is divided into fixed-size regions called pages. On Intel processors (80386and higher), the size of one page is 4 kilobytes. The 32-bit address bus can accessup to 4GB of RAM. Hence, there are one million (4GB/4K) pages.

Page address translation is a logical to physical address mapping. Some bits inthe logical/linear address are used as an index in the page table, which provides alogical to physical mapping for pages. The page translation mechanism on Intelplatforms has two levels, with a structure called page table directory at the secondlevel. As the name suggests, a page table directory is an array of pointers to pagetables. Some bits in the linear address are used as an index in the page table direc-tory to get the appropriate page table to be used for address translation.

The page address translation mechanism in the 80386 requires two importantdata structures to be maintained by the operating system, namely, the page tabledirectory and the page tables. A special register, CR3, points to the current pagetable directory. This register is also called Page Directory Base Register (PDBR). Apage table directory is a 4096-byte page with 1024 entries of 4 bytes each. Each en-try in the page table directory points to a page table. A page table is a 4096-bytepage with 1024 entries of 4 bytes (32 bits) each. Each Page Table Entry (PTE) pointsto a physical page. Since there are 1 million pages to be addressed, out of the 32bits in a PTE, 20 bits act as upper 20 bits of physical address. The remaining 12 bitsare used to maintain attributes of the page.

Some of these attributes are access permissions. For example, you can denote apage as read-write or read-only. A page also has an associated security bit called asthe supervisor bit, which specifies whether a page can be accessed from the user-mode code or only from the kernel-mode code. A page can be accessed only at ring0 if this bit is set. Two other bits, namely, the accessed bit and the dirty bit, indicate


the status of the page. The processor sets the accessed bit whenever the page is ac-cessed. The processor sets the dirty bit whenever the page is written to. Some bitsare available for operating system use. For example, Windows NT uses one such bitfor implementing the copy-on-write protection. You can also mark a page as in-valid and need not specify the physical page address. Accessing such a page gener-ates a page fault exception. An exception is similar to a software interrupt. Theoperating system can install an exception handler and service the page faults.You'll read more about this in the following sections.

32-bit memory addresses break down as follows. The upper 10 bits of the linearaddress are used as the page directory index, and a pointer to the correspondingpage table is obtained. The next 10 bits from the linear address are used as an in-dex in this page table to get the base address of the required physical page. The re-maining 12 bits are used as offset within the page and are added to the page baseaddress to get the physical address.

The Inside LookIn this section, we examine how Windows NT has selectively utilized existing fea-tures of the 80386 processor's architecture to achieve its goals.

Flat Address SpaceFirst, let's see how Windows NT provides 32-bit flat address space to the processes.As we know from the previous section, Intel 80386 offers segmentation as well aspaging. So how does Windows NT provide a flat memory instead of a segmentedone? Turn off segmentation? You cannot turn off segmentation on 80386.However, the 80386 processor enables the operating system to load the segmentregister once and then specify only 32-bit offsets for subsequent instructions. Thisis exactly what Windows NT does. Windows NT initializes all the segment registersto point to memory locations from 0 to 4GB, that is, the base is set as 0 and thelimit is set as 4GB. The CS, SS, DS, and ES are initialized with separate segment de-scriptors all pointing to locations from 0 to 4GB. So now the applications can useonly 32-bit offset, and hence see a 32-bit flat address space. A 32-bit applicationrunning under Windows NT is not supposed to change any of its segment registers.

\Process IsolationThe next question that comes to mind is, "How does Windows NT keep processesfrom seeing each other's address space?" Again, the mechanism for achieving thisdesign goal is simple. Windows NT maintains a separate page table directory foreach process and based on the process in execution, it switches to the correspond-ing page table directory. As the page table directories for different processes point


to different page tables and these page tables point to different physical pages andonly one directory is active at a time, no process can see any other process's mem-ory. When Windows NT switches the execution context, it also sets the CR3 registerto point to the appropriate page table directory. The kernel-mode address space ismapped for all processes, and all page table directories have entries for kernel ad-dress space. However, another feature of 80386 is used to disallow user-mode codefrom accessing kernel address space. All the kernel pages are marked as supervisorpages; therefore, user-mode code cannot access them.

Code Page Sharing in DLLsFor sharing code pages of a DLL, Windows NT maps corresponding page table en-tries for all processes sharing the DLL onto the same set of physical pages. For ex-ample, if process A loads X.DLL at address xxxx and process B loads the sameX.DLL at address yyyy, then the PTE for xxxx in process As page table and the PTEf°r yyyy m process B's page table point to the same physical page. Figure 4-2shows two processes sharing a page via same page table entries. The DLL pages aremarked as read-only so that a process inadvertently attempting to write to this areawill not cause other processes to crash.

This is guaranteed to be the case when xxxx==yyyy. However, if xxxx!=yyyy,the physical page might not be same. We will discuss the reason behind thislater in the chapter.

Figure 4-2: Sharing pages via same page table entries


Kernel address space is shared using a similar technique. Because the entire ker-nel space is common for all processes, Windows NT can share page tables directly.Figure 4-3 shows how processes share physical pages by using same page tables.Consequently, the upper half of the page table directory entries are the same forall processes.

Figure 4-3: Sharing pages via same page directory entries

Listing 4-1 shows the sample program that demonstrates this.

Listing 4-1: SHOWDIR.C

/* Should be compiled in release mode to run properly */# inc lude < w i n d o w s . h >^include <str ing.h>

#include <std io .h>#include "gate.h"

/* Global a r ray to hold the page directory */DWORD PageDi rec tory [1024] ;

This initial portion of the SHOWDIR.C file contains, apart from the header inclu-sion, the global definition for the array to hold the page directory. The inclusion ofthe header file GATE.H is of interest. This header file prototypes the functions forusing the callgate mechanism. Using the callgate mechanism, you can execute yourcode in the kernel mode without writing a new device driver.


We discuss the callgate mechanism in Chapter 10.

For this sample program, we need this mechanism because the page directory isnot accessible to the user-mode code. For now, it's sufficient to know that themechanism allows a function inside a normal executable to be executed in kernelmode. Turning on to the definition of the page directory, we have already describedthat the size of each directory entry is 4 bytes and a page directory contains 1024entries. Hence, the PageDirectory is an array of 1024 DWORDs. Each DWORD in thearray represents the corresponding directory entry.

CfuncGetPageDirectoryO is the function that is executed in the kernel mode us-ing the callgate mechanism. This function simply makes a copy of the page direc-tory in the user-mode memory area so that the other user-mode code parts in theprogram can access it. The page directory is mapped at virtual address OxC0300000in every process's address space. This address is not accessible from the user mode.The CFuncGetPageDirectoryO function copies 1024 DWORDs from the OxC0300000address to the global PageDirectory variable that is accessible to the user-modecode in the program.


The DisplayPageDirectoryQ function operates in user mode and prints thePageDirectory array that is initialized by the CfuncGetPageDirectoryO function. Thefunction checks the Least Significant Bit (LSB) of each of the entries. A page direc-tory entry is valid only if the last bit or the LSB is set. The function skips printinginvalid entries. The function prints three entries on every line or, in other words,prints a newline character for every third entry. Each directory entry is printed asthe logical address and the address of the corresponding page table as obtainedfrom the page directory. As described earlier, the first 10 bits (or the 10 MostSignificant Bits [MSB]) of the logical address are used as an index in the page di-rectory. In other words, a directory entry at index i represents the logical addressesthat have i as the first 10 bits. The function prints the base of the logical addressrange for each directory entry. The base address (that is, the least address in therange) has the last 22 bits (or 22 LSBs) as zeros. The function obtains this base ad-dress by shifting i to the first 10 bits. The address of the page table correspondingto the logical address is stored in the first 20 bits (or 20 MSBs) of the page directoryentry. The 12 LSBs are the flags for the entry. The function calculates the page tableaddress by masking off the flag bits.


The main() function starts by creating a callgate that sets up the GetPageDirectoryOfunction to be executed in the kernel mode. The GetPageDirectoryO function is writtenin Assembly language and is a part of the RINGO.ASM file. The CreateCallGateQ func-tion, used by the program to create the callgate, is provided by CALLGATE.DLL. Thefunction returns with a callgate selector.

The mechanism of calling the desired function through callgate is ex-plained in Chapter 10.

We'll quickly mention a few important points here. The callgate selector returnedby CreateCallGate() is a segment selector for the given function: in this case,GetPageDirectoryO. To invoke the function pointed by the callgate selector, youneed to issue a far call instruction. The far call instruction expects a 16-bit segmentselector and a 32-bit offset within the segment. When you are calling through a

i 44 Part 1: Essentials

callgate, the offset does not matter; the processor always jumps at the start of thefunction pointed to by the callgate. Hence, the program only initializes the thirdmember of the farcall array that corresponds to the segment selector. Issuing a callthrough the callgate transfers the execution control to the GetPageDirectoryO func-tion. This function calls the CfuncGetPageDirectoryO function that copies the pagedirectory in the PageDirectory array. After the callgate call returns, the programprints the page directory copied in the PageDirectory by calling theDisplayPageDirectoryO function. The program frees the callgate before exiting.

Listing 4-2: RINGO.ASM

The function to be called from the callgate needs to be written in assembly lan-guage for a couple of reasons. First, the function needs to execute a prolog and anepilog, both of which are assembly macros, to allow paging in kernel mode.Second, the function needs to issue a far return at the end. The function leaves therest of the job to the CFuncGetPageDirectoryO function written in C.

If you compare the output of the showdir program for two different processes,you find that the upper half of the page table directories for the two processes is ex-actly the same except for two entries. In other words, the corresponding kernel ad-dress space for these two entries is not shared by the two processes.

Listing 4-3: First instance of SHOWDIR

Page directory for the process, pid=6f


Let's analyze, one step at a time, why the two entries are different. The page ta-bles themselves need to be mapped onto some linear address. When Windows NTneeds to access the page tables, it uses this linear address range. To represent 4GBof memory divided into 1MB pages of 4K each, we need IK page tables each hav-ing IK entries. To map these IK page tables, Windows NT reserves 4MB of linearaddress space in each process. As we saw earlier, each process has a different set ofpage tables. Whatever the process, Windows NT maps the page tables on the linearaddress range from OxCOOOOOOO to OxC03FFFFF. Let's call this linear address rangeas the page table address range. In other words, the page table address range mapsto different page tables-that is, to different physical pages-for differentprocesses. As you may have noticed, the page table addresses range falls in the ker-nel address space. Windows NT cannot map this crucial system data structure in theuser address space and allow user-mode processes to play with the memory.Ultimately, the result is that two processes cannot share pages in the page table ad-dress range although the addresses lie in the kernel-mode address range.

Exactly one page table is required to map 4MB address space because each pagetable has IK entries and each entry corresponds to a 4K page. Consequently,Windows NT cannot share the page table corresponding to the page table addressrange. This accounts for one of the two mysterious entries in the page table direc-tory. However, the entry's mystery does not end here-there is one more subtletwist to this story. The physical address specified in this entry matches the physicaladdress of the page table directory. The obvious conclusion is that the page tabledirectory acts also as the page table for the page table address range. This ispossible because the formats of the page table directory entry and PTE are the sameon 80386.

The processor carries out an interesting sequence of actions when the linear ad-dress within the page table address range is translated to a physical address. Let'ssay that the CR3 register points to page X. As the first step in the address transla-tion process, the processor treats the page X as the page table directory and findsout the page table for the given linear address. The page table happens to be pageX again. The processor now treats page X as the required page table and finds out


the physical address from it. A more interesting case occurs when the operatingsystem is accessing the page table directory itself. In this case, the physical addressalso falls in page X!

Let's now turn to the second mysterious entry. The 4MB area covered by thispage directory entry is internally referred to as hyperspace. This area is used formapping the physical pages belonging to other processes into virtual address space.For example, a function such as MmMapPagelnHyperspaceQ uses the virtual ad-dresses in this range. This area is also used during the early stages of process cre-ation. For example, when a parent process such as PROGMAN.EXE spawns a childprocess such as NOTEPAD.EXE, PROGMAN.EXE has to create the address space forNOTEPAD.EXE. This is done as a part of the MmCreateProcessAddressSpaceQ func-tion. For starting any process, an address space must be created for the process.Address space is nothing but page directory. Also, the upper-half entries of page di-rectory are common for all processes except for the two entries that we have al-ready discussed. These entries need to be created for the process being spawned.The MmCreateProcessAddressSpaceQ function allocates three pages of memory: thefirst page for the page directory, the second page for holding the hyperspace pagetable entries, and the third page for holding the working set information for theprocess being spawned.

Once these pages are allocated, the function maps the first physical page in theaddress space using the MmMapPagelnHyperSpacefJ function. Note that theMmMapPagelnHyperSpaceQ function runs in the context of PROGMAN.EXE. Nowthe function copies the page directory entries in the upper half of the page directoryto the mapped hyperspace virtual address. In short, PROGMAN.EXE creates thepage directory for the NOTEPAD.EXE.

Windows NT supports memory-mapped files. When two processes map the samefile, they share the same set of physical pages. Hence, memory-mapped files can beused for sharing memory. In fact, Windows NT itself uses memory-mapped files toload DLLs and executables. If two processes map the same DLL, they automaticallyshare the DLL pages. The memory-mapped files are implemented using the sectionobject under Windows NT. A data structure called PROTOPTE is associated witheach section object. This data structure is a variable-length structure based on thesize of the section. This data structure contains a 4-byte entry for each page in thevirtual address space mapped by the section object. Each 4-byte entry has the samestructure as that of the PTE. When the page is not being used by any of theprocesses, the protopte entry is invalid and contains enough information to get thepage back. In this case, the CPU PTE contains a fixed value that is OxFFFFF480,which indicates that accessing this page will be considered a protopte fault.

Now comes the toughest of all questions: "How can Windows NT give away 4GBof memory to each process when there is far less physical RAM available on theboard?" Windows NT, as well as all other operating systems that allow more ad-dress space than actual physical memory, uses a technique called virtual memory toachieve this. In the next section, we discuss virtual memory management inWindows NT.


Virtual Memory ManagementThe basic idea behind virtual memory is very simple. For each process, the operat-ing system maps few addresses to real physical memory because RAM is expensiveand relatively rare. Remaining memory for each process is really maintained onsecondary storage (usually a hard disk). That's why it is called virtual memory. Theaddresses that are not mapped on physical RAM are marked as such. Whenever aprocess accesses such an address, the operating system brings the data into memoryfrom secondary storage. If the operating system runs out of physical RAM, somedata is thrown out to make space. We can always get back this data because a copyis maintained on secondary storage. The data to be thrown out is decided by the re-placement policy. Windows NT uses First-In-First-Out (FIFO) replacement policy.According to this policy, the oldest data (that is, the data that was brought in theRAM first) is thrown out whenever there is a space crunch.

To implement virtual memory management, Windows NT needs to maintain alot of data. First, it needs to maintain whether each address is mapped to physicalRAM or the data is to be brought in from secondary storage when a request withthe address comes. Maintaining this information for each byte itself takes a lot ofspace (actually, more space than the address space for which the information is tobe maintained). So Windows NT breaks the address space into 4KB pages and main-tains this information in page tables. As we saw earlier, a page table entry (PTE)consists of the address of the physical page (if the page is mapped to physical RAM)and attributes of the page. Since the processor heavily depends on PTEs for addresstranslation, the structure of PTE is processor dependent.

If a page is not mapped onto physical RAM, Windows NT marks the page as in-valid. Any access to this page causes a page fault, and the page fault handler canbring in the page from the secondary storage. To be more specific, when the pagecontains DLL code or executable module code, the page is brought in from the DLLor executable file. When the page contains data, it is brought in from the swap file.When the page represents a memory-mapped file area, it is brought in from the cor-responding file. Windows NT needs to keep track of free physical RAM so that itcan allocate space for a page brought in from secondary storage in case of a pagefault. This information is maintained in a kernel data structure called the PageFrame Database (PFD). The PFD also maintains a FIFO list of in-memory pages sothat it can decide on pages to throw out in case of a space crunch.

Before throwing out a page, Windows NT must ensure that the page is not dirty.Otherwise, it needs to write that page to secondary storage before throwing it out.If the page is not shared, the PFD contains the pointer to PTE so that if the operat-ing system decides to throw out a particular page, it can then go back and mark thePTE as invalid. If the page is shared, the PFD contains a pointer to the correspond-ing PROTOPTE entry. In this case, the PFD also contains a reference count for thepage. A page can be thrown out only if its reference count is 0. In general, the PFDmaintains the status of every physical page.


The PFD is an array of 24-byte entries, one for each physical page. Hence, thesize of this array is equal to the number of physical pages that are stored in a ker-nel variable, namely, MmNumberOfPhysicalPages. The pointer to this array isstored in a kernel variable, namely, MmpfnDatabase. A physical page can be in sev-eral states - for example, it can be in-use, free, free but dirty, and so on. A PFD en-try is linked in a doubly linked list, depending on the state of the physical pagerepresented by it. For example, the PFD entry representing a free page is linked inthe free pages list. Figure 4-4 shows these lists linked through the PFD. The forwardlinks are shown on the left side of the PFD, and the backward links are shown onthe right side.

Figure 4-4: Various lists linked through PFD

There are in all six kinds of lists. The heads of these lists are stored in followingkernel variables:

if

MmStandbyPageListHead

MmModifiedNoWritePageListHead

MmModifiedPageListHead

MmFreePageListHead

MmBadPageListHead

MmZeroedPageListHead

All these list heads are actually structures of 16 bytes each. Here is the structuredefinition:

typedef s t ruct P a g e L i s t H e a d {D W O R D N u m b e r O f P a g e s I n L i s t ,

ftipaat*^


Virtual Address DescriptorsAlong with the free physical pages, Windows NT also needs to keep track of the vir-tual address space allocation for each process. Whenever a process allocates amemory block - for example, to load a DLL - Windows NT checks for a free blockin the virtual address space, allocates virtual address space, and updates the virtualaddress map accordingly. The most obvious place to maintain this information ispage tables. For each process, Windows NT maintains separate page tables. Thereare 1 million pages, and each page table entry is 4 bytes. Hence, full page tables fora single process would take 4MB of RAM! There is a solution to this: Page tablesthemselves can be swapped out. It is inefficient to swap in entire page tables whena process wants to allocate memory. Hence, Windows NT maintains a separate bi-nary search tree containing the information about current virtual space allocationfor each process. A node in this binary search tree is called a Virtual AddressDescriptor (VAD). For each block of memory allocated to a process, Windows NTadds a VAD entry to the binary search tree. Each VAD entry contains the allocatedaddress range - that is, the start address and the end address of the allocated block,pointers to left and right children VADs, and a pointer to the parent VAD. Theprocess environment block (PEB) contains a pointer, namely, VadRoot, to the rootof this tree.

The FirstPage field can be used as an index into the PFD. The PFD entry containsa pointer to the next page. Using this, you can traverse any of the lists. Here is thestructure definition for the PFD entry:

Using this, you can easily write a program to dump the PFD. However, there isone problem: kernel variables, such as list heads, MmPfnDatabase, andMmNumberOfPhysicalPages, are not exported. Therefore, you have to deal with ab-solute addresses, which makes the program dependent on the Windows NT versionand build type.

Listing 4-5: VADDUMP.C

52____Part 1: Essentials_______


The initial portion of the VADDUMP.C file has a few definitions apart from theheader inclusion. In this program, we use the callgate mechanism as we did in theshowdir program - hence the inclusion of the GATE.H header file. After the headerinclusion, the file defines the maximum number of VAD entries that we'll process.There is no limit on the nodes in a VAD tree. We use the callgate mechanism forkernel-mode execution of a function that dumps the VAD tree in an array accessi-ble from the user mode. This array can hold up to MAX_VAD_ENTRIES entries.Each entry in the array is of type VADINFO. The VADINFO structure has two mem-bers: the address of the VAD tree node and the actual VAD tree node. The VAD treenode structure is defined in the UNDOCNT.H file as follows:

The first two members dictate the address range represented by the VAD node.Each VAD tree node maintains a pointer to the parent node and a pointer to the leftchild and the right child. The VAD tree is a binary tree. For every node in the tree,the left subtree consists of nodes representing lower address ranges, and the rightsubtree consists of nodes representing the higher address ranges. The last memberin the VAD node is the flags for the address range.

The VADDUMP.C file has a few other global variables apart from theVadlnfoArray. A couple of global variables are used while locating the root of theVAD tree. The PEB of a process points to the VAD tree root for that process. The off-set of this pointer inside the PEB varies with the Windows NT version. We set theVadRootOffset to the appropriate offset value of the VAD root pointer depending onthe Windows NT version. There is a similar problem of Windows NT version depen-dency while accessing the PEB for the process. We use the Thread EnvironmentBlock (TEB) to get to the PEB. One field in TEB points to the PEB, but the offset ofthis field inside the TEB structure varies with the Windows NT version. We set thePebOffset variable to the appropriate offset value of the PEB pointer inside the TEBstructure depending on the Windows NT version. Another global variable,NtVersion, stores the version of Windows NT running on the machine.

That leaves us with two more global variables, namely, VadlnfoArraylndex andVadTreeRoot. The VadlnfoArraylndex is the number of initialized entries in theVadlnfoArray. The VadlnfoArray entries after VadlnfoArraylndex are free. TheVadTreeRoot variable stores the root of the VAD tree.


The sample has been tested on Windows NT 3.51, 4.0 and Windows 2000 beta2.The sample will run on other versions of Windows 2000, provided the offsets ofVadRoot and PEB remain same. * ., .


V a d I n f o A r r a y [ V a d I n f o A r r a y I n d e x ] . V a d . F l a g s =VadNode->F lags ;Vad ln foAr ray Index++;

V a d T r e e W a l k ( V a d N o d e - > R i g h t L i n k ) ;}

The VadTreeWalkQ function is executed in the kernel mode using the callgatemechanism. The function traverses the VAD tree in the in-order fashion and fills upthe VadlnfoArray. The function simply returns if the node pointer parameter isNULL or the VadlnfoArray is full. Otherwise, the function recursively calls itself forthe left subtree. The recursion is terminated when the left child pointer is NULL. Thefunction then fills up the next free entry in the VadlnfoArray and increments theVadlnfoArraylndex to point to the next free entry. Windows 2000 stores the pagenumbers instead of the actual addresses in VAD. Hence, for Windows 2000, we needto calculate the starting address and the ending address from the page numbersstored in these fields. As the last step in the in-order traversal, the function issues aself-recursive to process the right subtree.

/* C funct ion c a l l e d through assemb ly stub */void _stdcal l CFuncDumpVad(PVAD VadRoot ){V a d T r e e R o o t = V a d R o o t ;V a d l n f o A r r a y l n d e x = 0;V a d T r e e W a l k ( V a d R o o t ) ;

The CfuncDumpVad is the caller of the VadTreeWalkO function. It just initializesthe global variables used by the VadTreeWalkO function and calls theVadTreeWalkO function for the root of the VAD tree.

/* D isp lays the Vad tree */vo id V a d T r e e D i s p l a y ( )(i n t i;

p r i n t f ( " V a d R o o t is l oca ted @ % 0 8 x \ n \ n " ,VadTreeRoot ) ;p r i n t f ( " V a d @ \ t Start ingU EndingU ParentU ""LeftLinkU RightL ink\n" ) ;for ( i=0; i < V a d l n f o A r r a y l n d e x ; i++) {p r i n t f ( "%08x %08x %08x %8x %08x % 0 8 x \ n " ,V a d I n f o A r r a y [ i ] . V a d L o c a t i o n ,V a d l n f o A r r a y [ i ] . V a d . S t a r t i n g A d d r e s s ,


V a d l n f o A r r a y [ n ] . V a d . E n d i n g A d d r e s s ,V a d I n f o A r r a y [ i ] . V a d . P a r e n t L i n k ,V a d l n f o A r r a y E i ] . V a d . L e f t L ink,V a d I n f o A r r a y [ i ] . V a d . R i g h t L i nk ) ;}p r i n t f ( " \ n \ n " ) ;

The VadTreeDisplayO function is a very simple function that is executed in usermode. The function iterates through all the entries initialized by the VadTreeWalkQfunction and prints the entries. Essentially, the function prints the VAD tree in the in-fix order because the VadTreeWalk() function dumps the VAD tree in the infix order.

vo id S e t D a t a S t r u c t u r e O f f s e t s ( )(switch (NtVersion) {case 3:PebOffset = 0x40;VadRootOffset = 0x170;break;

case 4:PebOffset = 0x44;VadRootOffset = 0x170;break;

case 5:PebOffset = 0x44;VadRootOffset = 0x194;break;)}

As we described earlier, the offset of the PEB pointer within TEB and the offsetof the VAD root pointer within the PEB are dependent on the Windows NT version.The SetDataStructureOffsetsO function sets the global variables indicating theseoffsets depending on the Windows NT version.

m a i n ( )

WORD CallGateSelector;int re;short farcall[3];void DumpVad(void);


printf("\n\nDumping the Vad tree again...\n\n");VadTreeDisplay();

Quit:re = FreeCallGate(Cal1GateSelector);if (re != SUCCESS) {printf("FreeCal1 Gate failed, Selector=%x, rc=%x\n"

CallGateSelector, re);}

return 0;}

The main() function starts by getting the Windows NT version and callingSetDataStructureOffsetsO to set the global variables storing the offsets for the PEBand the VAD tree root. It then creates a callgate in the same manner as in theSHOWDIR sample program. Issuing a call through this callgate ultimately results inthe execution of the VadTreeWalkQ function that fills up the VadlnfoArray. Themain() function then calls the VadTreeDisplayQ function to print the VadlnfoArrayentries.

We also show you the change in the VAD tree due to memory allocation in thissample program. After printing the VAD tree once, the program allocates a chunkof memory. Then, the program issues the callgate call again and prints the VAD treeafter returning from the call. You can observe the updates that happened to theVAD tree because of the memory allocation. The program frees up the callgate be-fore exiting.


.386

.model small

.code

publi c _DumpVadextrn _CFuncDumpVad@4:nearextrn _PebOffset:nearextrn _VadRootOffset:near

include ..\include\undocnt.inc

_DumpVad procRi ngOProl og

;Gets the current thread


The function to be called from the callgate needs to be written in the Assemblylanguage for reasons already described. The DumpVadO function gets hold of theVAD root pointer and calls the CFuncDumpVadO function that dumps the VAD treein the VadlnfoArray. The function gets hold of the VAD root from the PEB after get-ting hold of the PEB from the TEB. The TEB of the currently executing thread is al-ways pointed to by FS:128h. As described earlier, the offset of the VAD root pointerinside PEB and the offset of the PEB pointer inside the TEB vary with the WindowsNT version. The DumpVadQ function uses the offset values stored in the globalvariable by the SetDataStructureOffsets() function.

Listing 4-7 presents the output from an invocation of the VADDUMP program.Note that the VAD tree printed after allocating memory at address 0x300000 showsan additional entry for that address range.

Listing 4-7: Program output

Dumping the Vad t ree. . .

VadRoo t is l oca ted @fe21a9c8


A l l oca t i ng memory us ing V i r t u a l A l l o cMemory a l l o c a t e d @300000

Dumping the Vad tree a g a i n . . .

VadRoo t is l oca ted @fe21a9c8


fe21a9c8fe21b3e8fe2176c8fe2152c8fe2326e8fe226348fe2197c8fe2333e8fe23c488fe22b408fe22c4a8fe23f5e8fe25aa88fe21da88fe218288

004000001000000077e2000077e5000077e6000077eaOOOO77eeOOOO77f2000077f800007f2dOOOO7f5fOOOO7ff700007ffbOOOO7ffdeOOO7ffdfOOO

0040cffflOOOdfff77e4bfff77e54fff77e9bfff77ed7fff77fl2fff77f73fff77fcdfff7f5cffff7f7effff7ffaffff7ffd3fff7ffdefff7ffdffff

0fe2333e8fe226348fe2326e8fe2176c8fe21b3e8fe226348fe23c488fe21a9c8fe25aa88fe22b408fe22c4a8fe23c488fe218288fe25aa88

fe216b08000000000000000000000000fe2152c8fe2176c800000000fe21b3e8fe2333e8000000000000000000000000fe22b40800000000fe21da88

fe23c488fe226348fe2326e80000000000000000fe2197c80000000000000000fe25aa88fe22c4a8fe23f5e800000000fe2182880000000000000000

The output of the VADDUMP program does not really look like a tree. You haveto trace through the output to get the tree structure. The entry with a null parentlink is the root of the tree. Once you find the root, you can follow the child point-ers. To follow a child pointer, search the pointer in the first column, named Vad@,in the output. The Vad entry with the same Vad@ is the entry for the child that youare looking for. An all-zero entry for a left/right child pointer indicates that there isno left/right subtree for the node. Figure 4-5 shows a partial tree constructed fromthe output shown previously.


Impact on HookingNow we'll look at the impact of the memory management scheme explained in thelast section in the area of hooking DLL API calls. To hook a function from a DLL,you need to change the first few bytes from the function code. As you saw earlier,the DLL code is shared by all processes and is write protected so that a misbehavingprocess cannot affect other processes. Does this mean that you cannot hook a func-tion in Windows NT? The answer is, "Hooking is possible under Windows NT, butyou need to do a bit more work to comply with stability requirements." WindowsNT provides a system call, VirtualProtect, that you can use to change page attrib-utes. Hence, hooking is now a two-step process: Change the attributes of the pagecontaining DLL code to read-write, and then change the code bytes.

Copy-on-Write"Eureka!" you might say, "I violated Windows NT security. I wrote to a shared pageused by other processes also." No! You did not do that. You changed only your copyof the DLL code. The DLL code page was being shared while you did not write to thepage. The moment you wrote on that page, a separate copy of it was made, and thewrites went to this copy. All other processes are safely using the original copy ofthe page. This is how Windows NT protects processes from each other while con-suming as few resources as possible.

The VirtualProtectO function does not mark the page as read-write - it keeps thepage as read-only Nevertheless, to distinguish this page from normal read-onlypages, it is marked for copy-on-write. Windows NT uses one of the available PTEbits for doing this. When this page is written onto, because it is a read-only page,the processor raises a page fault exception. The page fault handler makes a copy ofthe page and modifies the page table of the faulting process accordingly. The newcopy is marked as read-write so that the process can write to it.

Windows NT itself uses the copy-on-write mechanism for various purposes. TheDLL data pages are shared with the copy-on-write mark. Hence, whenever a processwrites to a data page, it gets a personal copy of it. Other processes keep sharing theoriginal copy, thus maximizing the sharing and improving memory usage.

A DLL may be loaded in memory at different linear address for differentprocesses. The memory references - for example, address for call instruction, ad-dress for a memory to register move instruction, and so on — in the DLL need to beadjusted (patched) depending on the linear address where the DLL gets loaded. Thisprocess is called as relocating the DLL. Obviously, relocation has to be done sepa-rately for each process. While relocating, Windows NT marks the DLL code pages ascopy-on-write temporarily. Thus, only the pages requiring page relocation arecopied per process. Other pages that do not have memory references in them areshared by all processes.

This is the reason Microsoft recommends that a DLL be given a preferred baseaddress and be loaded at that address. The binding of the DLL to a specific base ad-


dress ensures that the DLL need not be relocated if it is loaded at the specified baseaddress. Hence, if all processes load the DLL at the preferred base address, all canshare the same copy of DLL code.

The POSIX subsystem of Windows NT uses the copy-on-write mechanism to im-plement the fork system call. The fork system call creates a new process as a childof a calling process. The child process is a replica of the parent process, and it hasthe same state of code and data pages as the parent. Since these are two differentprocesses, the data pages should not be shared by them. However, generally it iswasteful to make a copy of the parent's data pages because in most cases the childimmediately invokes the exec system call. The exec system call discards the currentmemory image of the process, loads a new executable module, and starts executingthe new executable module. To avoid copying the data pages, the fork system callmarks the data pages as copy-on-write. Hence, a data page is copied only if theparent or the child writes to it.

Copy-on-write is an extremely important concept contributing to the efficiencyof NT memory management.

The following sample program demonstrates how copy-on-write works. By run-ning two instances of the program, you can see how the concepts described in thissection work. The application loads a DLL, which contains two functions and twodata variables. One function does not refer to the outside world, so no relocationsare required for it. The other function accesses one global variable, so it containsrelocatable instructions or instructions that need relocation. One data variable isput in a shared data section so it will be shared across multiple instances of DLL.One variable is put in a default data section. The two functions are put in separatecode sections just to make them page aligned.

When you run the first instance of the application, the application loads andprints the physical addresses of two functions and two data variables. After this,you run the second instance of the same application. In the second instance, the ap-plication arranges to load the DLL at a different base address than that of the firstinstance. Then it prints the physical addresses of two functions and two data vari-ables. Next, the application arranges to load the DLL at the same base address asthat of the first instance. In this case, all physical pages are seen to be shared. Next,the application modifies the shared and nonshared variable and modifies the firstfew bytes of one function, and it prints the physical addresses for two functionsand two variables again. We first discuss the code for this sample program and thendescribe how the output from the sample program demonstrates memory sharingand the effects of the copy-on-write mechanism.

Listing 4-8: SHOWPHYS.C

^include <w indows .h>^include <s td io .h>

//"include " g a t e . h "/ / inc lude "ge tphys .h "


HANDLE hF i leMapp ing ;

/* Imported f u n c t i o n / v a r i a b l e addresses */s ta t i c vo id *NonRe loca tab leFunc t ion = NULL;s ta t i c vo id *Re loca tab leFunc t ion = NULL;s ta t i c vo id * S h a r e d V a r i a b l e = NULL;s ta t i c vo id *NonSharedVanab le = NULL;

H I N S T A N C E hDl1 Ins tance;

The initial portion of the file contains the header inclusion and global variabledefinitions. The program demonstrates the use of various page attributes, especiallyto implement the copy-on-write mechanism. As described earlier, the program usesfour different types of memory sections. The pointers to the four different types ofmemory sections are defined as global variables. The hDHInstance stores the in-stance of the instance handle of the DLL that contains the different kind of memorysections used in this demonstration.


The four different types of memory sections that we use for the demonstrationreside in MYDLL.DLL. The LoadDllAndlnitializeVirtualAddressesO function loadsMYDLL.DLL in the calling process's address space and initializes the global vari-ables to point to different types of memory sections in the DLL. The function usesthe GetProcAddressQ function to get hold of pointers to the exported functions andvariables in MYDLL.DLL. The function stores the instance handle for MYDLL.DLL ina global variable so that the FreeDllfJ function can later use it to unload the DLL.The function also locks the different memory sections so that the pages are loadedin memory and the page table entries are valid. Generally, Windows NT does notload the page table entries unless the virtual address is actually accessed. In otherwords, the memory won't be paged in unless accessed. Also, the system can pageout the memory that is not used for some time, again marking the page table entriesas invalid. We use the VirtualLockQ function to ensure that the pages of interest arealways loaded and the corresponding page table entries remain valid.

/ * Un locks the imported a reas and f rees the M Y D L L . D L L*/

vo id FreeDl 1 ( ){V i r t u a l U n l o c k ( N o n R e l o c a tab!eFunct ion, 1);V i r t u a l U n l o c k ( R e l o c a t a b l e F u n c t i o n , 1);V i r t u a l U n l o c k ( S h a r e d V a r i a b l e , 1);V i r t u a l U n l o c k ( N o n S h a r e d V a r i a b l e , 1);

F r e e L i b r a r y ( h D l l I n s t a n c e ) ;


HD11 Instance = 0;NonRelocatabl eFunction = NULL;RelocatableFunction = NULL;SharedVanable = NULL;NonSharedVariable = NULL;}

The FreeDllQ function uses the VirtualUnlock() function to unlock the memorylocations locked by the LoadDllAndlnitializeVirtualAddressesO function. Thefunction unloads MYDLL.DLL after unlocking the memory locations from the DLL.As the DLL is unloaded, the global pointers to the memory sections in the DLLbecome invalid. The function sets all these pointers to NULL according to goodprogramming practice.

The GetPageAttributesStringO function returns a string with characters showingthe page attributes given the page attribute flags. The LSB in the page attributes in-dicates whether the page is present in memory or the page table entry is invalid.This information is printed as P or NP, which stands for present or not present.Similarly, R or RW means a read-only or read-write page; S or U means a supervi-sor-mode or a user-mode page; and D means a dirty page. The various page attrib-utes are represented by different bits in the PageAttr parameter to this function. Thefunction checks the bits and determines whether the page possesses the particularattributes.

/* D isp lays virtual to physical address mapping*/

int D isp layV i r tua lAnd Physical Add resses ( ){


The DisplayVirtualAndPhysicalAddressesO function is a utility function that dis-plays the virtual address, the physical address, and the page attributes for differentmemory sections. It uses the global pointers to the different sections in MYDLL.DLLinitialized by the LoadDIlAndlnitializeVirtualAddressesQ function. It uses theGetPhysicalAddressAndPageAttributesO function to get hold of the physical pageaddress and the page attributes for the given virtual address. The first parameter tothe GetPhysicalAddressAndPageAttributesO function is the input virtual address.The function fills in the physical address for the input virtual address in the mem-ory location pointed to by the second parameter and the page attributes in the lo-cation pointed to by the third parameter.

int F i r s t l n s t a n c e C )(printf("***This is the first instance of the"" showphys program***\n\n");printfC'Loading DLL MYDLL.DLL\n");

if (LoadDllAndlnitializeVirtualAddresses()!=0) {return -1;


We want to demonstrate the sharing of memory sections by the DLL loaded bytwo different processes. You need to run two instances of the demonstration pro-gram. The Firstlnstance() function is executed when you run the first instance ofthe program. The first instance loads the DLL and prints the physical addresses andpage attributes for the various memory sections in the DLL. Then, the function asksyou to run another instance of the program. Now there are two processes thatloaded MYDLL.DLL. You can compare the outputs from these two instances tocheck how the memory sections are shared. More on this when we explain the out-put from this sample program.


}

DnsplayVirtualAndPhysicalAdd resses();

printfC1....Modifying the code bytes at the start ofNonRelocatableFunction\n");

VirtualProtect(NonRelocatableFunction, 1, PAGE_READWRITE,&01dAttr);

*(unsigned char *)NonRelocatableFunction=OxE9;

pnntf ("... .Modifying the value of SharedVan able\n");*(char *)SharedVanable=OxlO;

pnntf ("... .Modi fying the NonSharedVan able 1 s value\n\n";*(char *)NonSharedVanable=OxlO;

DisplayVirtualAndPhysicalAddresses();

FreeDllO;

return 0;}

The second instance of the program does a lot more work than the first instance.The sharing of the DLL memory sections depends on the way the instance loads theDLL and accesses the memory locations in the DLL. In more concrete terms, thesharing depends on whether the second instance loads the DLL at the same base ad-dress as the first instance. It also depends on whether the instances only read thememory sections or any of the instances write to the memory sections. To demon-strate this, the NonFirstlnstanceO function first loads the DLL at a different base ad-dress than the first instance. The function ensures that the DLL is loaded at adifferent base address by loading JUNK.DLL before loading MYDLL.DLL. JUNK.DLLhas the same preferred base address as that of MYDLL.DLL. The first instance loadsMYDLL.DLL at its preferred base address by default. In the second instance, MY-DLL.DLL cannot be loaded at its preferred base address because the address range isalready occupied by JUNK.DLL. After MYDLL.DLL is loaded at a different base ad-dress, there is no reason for the program to keep JUNK.DLL loaded, and so it freesthe JUNK.DLL instance. Next, the function prints the physical addresses and pageattributes of the memory sections in MYDLL.DLL using the DisplayVirtualAndPhysicalAddresses() function. The information printed here can be comparedwith the output of the first instance of the program to get an idea of how the DLLsloaded at different base addresses share the memory sections.


The NonFirstlnstance also demonstrates the sharing of memory sections by MY-DLL.DLL loaded at the same base address by two processes. It unloads MYDLL.DLLand loads it again. This time MYDLL.DLL is loaded at its preferred base address be-cause now that JUNK.DLL is no more loaded, the virtual address space is not occu-pied by anything. Thus, MYDLL.DLL is loaded at the same base address in both thefirst and the second instance of the program. The physical addresses and the pageattributes printed here demonstrate the memory sharing by MYDLL.DLL whenloaded at the same base address in two processes. Next, the NonFirstlnstanceQfunction writes to some of the memory locations in MYDLL.DLL. As we explainsoon, this action affects the memory sharing between the instances. As describedearlier, the code sections are marked read-only by Windows NT. The function usesthe VirtualProtectO API function to change the attributes of the NonRelocatableFunction() so that it can modify a few bytes at the start of this function. You canmodify the data variables from MYDLL.DLL without any such hassle because thedata variables have the read-write attribute.

[

^ "*

5;

"̂The sample program does not accept any parameter to indicate whether it's the

first instance. It uses a simple trick to decide it: It creates a named file mapping. Thecall to the CreateFileMappingO API function sets the last error to ERROR_AL-READY_EXISTS if a mapping with the same name already exists. This indicates thatan instance that created the file mapping is already running. In other words, if theprogram can successfully create the named file mapping, it's the first instance of


the program. Otherwise, another instance (that As, the fixst instance) of the programis already running and the current instance is the second instance. Depending onwhether it's the first instance, the DecideThelnstanceAndActO function calls theNonFirstlnstanceO function or the FirstlnstanceQ function. A file mapping is auto-matically destroyed by the operating system when the reference count drops tozero. The sample program does not explicitly close the handle to the mapping. Thehandle is closed and the reference count for the memory mapping is decrementedwhen the program exits. The mapping is freed up when the last instance of the pro-gram exits.

The mainQ function starts by a call to the CreateRingOCallGate() function that islocated in the GETPHYS.C file. The sample program uses the callgate mechanism toaccess the page tables. As described earlier, the page tables reside in the kernelmemory and are not accessible to the user-mode code. The CreateRingOCallGatefJfunction sets up a function that reads in the page tables to be executed in kernelmode. The DisplayVirtualAndPhysicalAddressesO function later uses this functionto get hold of the physical address and the page attributes for a given virtual ad-dress. After creating the callgate, the main function passes control to theDecideThelnstanceAndDoTheThingsO function. The callgate is freed up by the pro-gram before exiting.

Listing 4-9: GETPHYS.C

^include < w i n d o w s . h >#inc lude <s td io .h>^include " . . \ cga te \d l l \ga te .h"

static short Cal1GateSelector;


The GETPHYS.C file implements the function to access the page table using thecallgate mechanism. The GATE.H file is included because it contains the prototypesfor functions that deal with the callgate manipulation. The segment selector of thecallgate used by the program is stored in the global variable, CallGateSelector.

/* C funct ion ca l l ed f rom assembly l angauage stub */BOOL _s tdca l lC F u n c G e t P h y s i c a l A d d r e s s A n d P a g e A t t r i b u t e s C ' -unsigned int V i r t u a l A d d r e s s , , •unsigned int *Phys i ca lAdd ress ,uns igned int *PageAt tnbu tes ){unsigned int *PageTableEntry;

*PhysicalAddress = 0;*PageAttributes = 0;

PageTableEntry = (unsigned int *)OxCOOOOOOOU +(VirtualAddress > OxOCU);

if ((*PageTableEntry)&Ox01) {*PhysicalAddress =((*PageTableEntry)&OxFFFFFOOOU) +(VirtualAddress&OxOOOOOFFFU);

*PageAttributes = (*PageTableEntry )&OxOOOOOFFFU;return TRUE;} else {return FALSE;)}

The CfuncGetPhysicalAddressAndPageAttributesO function executes in kernelmode using the callgate mechanism. The function depends on the fact that page ta-bles for a process are always mapped at the virtual address OxCOOOOOOO. It's an ar-ray of 1024 page tables where each page table is an array of 1024 page tableentries. You can access the memory area as if it were a single contiguous array ofpage table entries. The first entry in this big array corresponds to a virtual addressin the range 0 - 4096, the second entry corresponds to virtual address range 4096- 8192, and so on. The function calculates the index in the big PTE array by divid-ing the given virtual address by 4096 - that is, by shifting the virtual address by 12bits. Adding the index in the base address of the PTE array gives us the requiredPTE. Each PTE is 4 bytes (32 bits) long. Out of these 32 bits, the upper 20 bits in thePTE denote the address of the physical page, and the lower 12 bits denote the page


attributes. The physical address and the page attributes are valid only if the LSB isset. The function checks the LSB and if the bit is set, it separates out the physicalpage address and the page attributes by masking off appropriate bits from the PTE.The function adds the offset within the page to the physical page address to get thephysical address for the given virtual address.

The GetPhysicalAddressAndPageAttributesO function runs in user mode and in-vokes the CfuncGetPhysicalAddressAndPageAttributesO function in kernel modeusing the callgate mechanism. It uses the callgate initialized by the call to theCreateRingOCallGateQ function. The parameters to the kernel-mode function arepassed through the processor registers. An intermediate Assembly language func-tion, namely, GetPhysicalAddressAndPageAttributes(), converts the register para-meters to stack parameters.


0,& C a l l G a t e S e l e c t o r ) ;r e t u r n r e ;)

The CreateRingOCallGateQ function is a utility function that uses theCreateCallGateQ function provided by GATE.DLL to create a callgate to execute theGetPhysicalAddressAndPageAttributesO function in kernel mode. It stores the seg-ment selector of the created callgate in the CallGateSelector global variable, whichis used later by the GetPhysicalAddressAndPageAttributesQ function while invok-ing the kernel-mode function. - - •

^ *

The FreeRingOCallGateQ function is another utility function that destroys thecallgate created by the CreateCallGateQ function. It uses the FreeCallGateQ interfacefunction provided by GATE.DLL.



RingOEp i logretf_Get Physical AddressAnd PageAttributes endp

END

The GetPhysicalAddressAndPageAttributes() function gets control through thecallgate. The function executes the RingOProlog macro just after entering the func-tion to enable paging in kernel mode. It converts the register parameters to stackparameters because CfuncGetPhysicalAddressAndPageAttributes() is a C functionthat expects the parameters on stack.

Listing 4-11 presents the output from the previous sample program. Note the dif-ferences between the physical addresses and page attributes printed by the first in-stance and the second instance. See if you can explain the output and match yourfindings with our description that comes after this output.

Here are two instances of the showphys program.

Listing 4-11: showphys program

Loading DLL MYDLL.DLLMYDLL.DLL loaded at base address 20000000

V i r t u a l address to Physical address mapping

Vari able/functi onAddressAddressAttributes

Vi rtualPhysical Page

NonRelocatableFunctionRel ocatabl eFuncti onShared Vari ableNonShared Vari abl e

20001000200020002000cOOO2000bOOO

dSbOOO P R UdSaOOO P R Ue44000 P RW U6b7000 P R U

Now Run another copy of showphys ...

This is another instance of the showphys program:

Load ing DLL M Y D L L . D L L at di f f rent base address than that of thef i rst i ns tanceM Y D L L . D L L loaded at base address = 7 e O O O O

Vi r t ua l address to Phys ica l add ress mapping


it

Note the page attributes from the output of the first instance. The functions aremarked read-only, as expected. The unshared variable is also marked read-only.This is because Windows NT tries to share the data space also. As described earlier,such pages are marked for copy-on-write, and as soon as the process modifies anylocation in the page, the process gets a private copy of the page to write to. Theother page attributes- show that the PTE is valid, the page is a user-mode page, andnobody has modified the page so far.

Now, compare the output from the first instance with the output from the secondinstance when it loaded the MYDLL.DLL at a base address different from that in thefirst instance. As expected, the virtual addresses of all the memory sections are dif-ferent than those for the first instance. The physical addresses are the same exceptfor the physical address of the relocatable function. This demonstrates that the codepages are marked as copy-on-write, and when the loader modifies the code pageswhile performing relocation, the process gets a private writable copy. Our nonrelo-catable function does not need any relocation; hence, the corresponding pages arenot modified. The second instance can share these pages with the first instance andhence has the same physical page address.

To cancel out the effects of relocation, the second instance loads MYDLL.DLL atthe same base address as that in the first instance. Yup! Now, the virtual addressmatches the ones from the first instance. Note that the physical address for the re-locatable function also matches that in the output from the first instance. Theloader need not relocate the function because the DLL is loaded at the preferredbase address. This allows more memory sharing and provides optimal perfor-mance. It's reason enough to allocate proper, nonclashing preferred base addressesfor your DLLs.

This ideal share-all situation ceases to exist as soon as a process modifies somememory location. Other processes cannot be allowed to view these modifications.Hence, the modifying process gets its own copy of the page The second instance ofthe sample program demonstrates this by modifying the data variables and a byteat the start of the nonrelocatable function. The output shows that the physical ad-dress of the nonrelocatable doesn't match with the first instance. The nonrelocat-able function is not modified by the loader, but it had the same effect on sharingwhen we modified the function. The shared variable remains a shared variable. Itsphysical address matches that in the first instance because all the processes access-ing a shared variable are allowed to see the modifications made by other processes.But the nonshared variable has a different physical address now. The second in-stance cannot share the variable with the first instance and gets its own copy. Thecopy was created by the system page fault handler when we tried to write to a read-only page and the page was also marked for copy-on-write. Note that the page isnow marked read-write. Hence, further writes go through without the operatingsystem getting any page faults. Also, note that the modified pages are marked asdirty by the processor.


Switching ContextAs we saw earlier, Windows NT can switch the memory context to another processby setting the appropriate page table directory. The 80386 processor requires thatthe pointer to the current page table directory be maintained in the CR3 register.Therefore, when the Windows NT scheduler wants to perform a context switch toanother process, it simply sets the CR3 register to the page table directory of theconcerned process.

Windows NT needs to change only the memory context for some API calls suchas VirtualAllocEx(). The VirtualAllocEx() API call allocates memory in the memoryspace of a process other than the calling process. Other system calls that requirememory context switch are ReadProcessMemoryO and WriteProcessMemoryQ. TheReadProcessMemoryO and WriteProcessMemoryO system calls read and write, re-spectively, memory blocks from and to a process other than the calling process.These functions are used by debuggers to access the memory of the process beingdebugged. The subsystem server processes also use these functions to access theclient process's memory. The undocumented KeAttchProcessO function from theNTOSKRNL module switches the memory context to specified process. The undocu-mented KeDetachProcessO function switches it back. In addition to switching mem-ory context, it also serves as a notion of current process. For example, if you attachto a particular process and create a mutex, it will be created in the context of thatprocess. The prototypes for KeAttachProcessO and KeDetachProcessO are as follows:

NTSTATUS K e A t t a c h P r o c e s s ( P E B *);NTSTATUS KeDetachProcess ( ) ;

Another place where a call to the KeAttachProcessQ function appears is theNtCreateProcessO system call. This system call is executed in the context of the par-ent process. As a part of this system call, Windows NT needs to map the system DLL(NTDLL.DLL) in the child process's address space. Windows NT achieves this bycalling KeAttachProcessO to switch the memory context to the child process. Aftermapping the DLL, Windows NT switches back to the parent process's memory con-text by calling the KeDetachProcessO function.

The following sample demonstrates how you can use the KeAttachProcessO andKeDetachProcessO functions. The sample prints the page directories for all theprocesses running in the system. The complete source code is not included. Only therelevant portion of the code is given. Because these functions can be called onlyfrom a device driver, we have written a device driver and provided an IOCTL thatdemonstrates the use of this function. We are giving the function that is called inresponse to DeviceloControl from the application. Also, the output of the programis shown in kernel mode debugger's window (such as SoftICE). Getting the infor-mation back to the application is left as an exercise for the reader.

void Disp layPageDirectory(vo id *Peb)


The DisplayPageDirectoryO function accepts the PEB for the process whose pagedirectory is to be printed. The function first calls the KeAttachProcess() functionwith the given PEB as the parameter. This switches the page directory to the desiredone. Still, the function can access the local variables because the kernel addressspace is shared by all the processes. Now the address space is switched, and theOxC030000 address points to the page directory to be printed. The function printsthe 1024 entries from the page directory and then switches back to the original ad-dress space using the KeDetachProcessQ function.


NameOffset=OxlFC;} else {

DbgPnnt("Unsupported NT Version\n");return;

}

P r o c e s s L i s t H e a d = P r o c e s s L i s t P t r = ( P L I S T _ E N T R Y ) ( ( ( c h a r* ) P s I m t i a l S y s t e m P r o c e s s ) + L i s t E n t r y O f f s e t ) ;

w h i l e ( P r o c e s s L i s t P t r > F 1 i n k ! = P r o c e s s L i s t H e a d ) {vo id *Peb;char P r o c e s s N a m e [ 1 6 ] ;

Peb=(vo id * ) ( ( ( c h a r * ) P r o c e s s L i s t P t r ) -L i s t E n t r y O f f s e t ) ;

memse t (P rocessName, 0, s i z e o f ( P r o c e s s N a m e ) ) ;m e m c p y ( P r o c e s s N a m e , ( ( c h a r * )Peb)+NameOf fse t , 16);

DbgPr in t ( " * *%s Peb @%x** ", P r o c e s s N a m e , P e b ) ;Disp layPageDirectory(Peb) ;P r o c e s s L i s t P t r = P r o c e s s L i s t P t r - > F l i n k ;

}

The DisplayPageDirectoryForAllProcessesQ function calls the DisplayPageDirectoryO function for each process in the system. All the processes running in asystem are linked in a list. The function gets hold of the list of the processes fromthe PEB of the initial system process. The PsInitialSystemProcess variable inNTOSKRNL holds the PEB for the initial system process. The process list node is lo-cated at an offset of 0x98 (OxAO for Windows NT 5.0) inside the PEB. The processlist is a circular linked list. Once you get hold of any node in the list, you can tra-verse the entire list. The DisplayPageDirectoryForAllProcessesO function completesa traversal through the processes list by following the Flink member, printing thepage directory for the next PEB in the list every time until it reaches back to thePEB it started with. For every process, the function first prints the process namethat is stored at a version-dependent offset within the PEB and then calls theDisplayPageDirectoryO function to print the page directory.

Here, we list partial output from the sample program. Please note a couple ofthings in the following output. First, every page directory has 50-odd valid entrieswhile the page directory size is 1024. The remaining entries are invalid, meaningthat the corresponding page tables are either not used or are swapped out. In otherwords, the main memory overhead of storing page tables is negligible because thepage tables themselves can be swapped out. Also, note that the page directorieshave the same entries in the later portion of the page directory. This is because thispart represents the kernel portion shared across all processes by using the same setof page tables for the kernel address range.


Differences between Windows NTand Windows 95/98Generally, the memory management features offered by Windows 95/98 are thesame as those in Windows NT. Windows 95/98 also offers 32-bit flat separate ad-dress space for each process. Features such as shared memory are still available.However, there are some differences. These differences are due to the fact thatWindows 95/98 is not as secure as Windows NT. Many times, Windows 95/98trades off security for performance reasons. Windows 95/98 still has the concept ofuser-mode and kernel-mode code. The bottom 3GB is user-mode space, and the top


1GB is kernel-mode space. But the 3GB user-mode space can be further divided intoshared space and private space for Windows 95/98. The 2GB to 3GB region is theshared address space for Windows 95/98 processes. For all processes, the page ta-bles for this shared region point to the same set of physical pages.-'• All the shared DLLs are loaded in the shared region. All the system DLLs — forexample, KERNEL32.DLL and USER32.DLL-are shared DLLs. Also, a DLL'scode/data segment can be declared shared while compiling the DLL, and the DLLwill get loaded in the shared region. The shared memory blocks are also allocatedspace in the shared region. In Windows 95/98, once a process maps a shared sec-tion, the section is visible to all processes. Because this section is mapped in sharedregion, other processes need not map it separately.

There are advantages as well as disadvantages of having such a shared region.Windows 95/98 need not map the system DLLs separately for each process; the cor-responding entries of page table directory can be simply copied for each process.Also, the system DLLs loaded in shared region can maintain global data about allthe processes and separate subsystem processes are not required. Also, most systemcalls turn out to be simple function calls to the system DLLs, and as a result arevery fast. In Windows NT, most system calls either cause a context switch to kernelmode or a context switch to the subsystem process, both of which are costly oper-ations. For developers, loading system DLLs in a shared region means that they cannow put global hooks for functions in system DLLs.

For all these advantages, Windows 95/98 pays with security features. InWindows 95/98, any process can access all the shared data even if it has notmapped it. It can also corrupt the system DLLs and affect all processes.

SummaryIn this chapter, we discussed the memory management of Windows NT from threedifferent perspectives. Memory management offers programmers a 32-bit flat ad-dress space for every process. A process cannot access another process's memory ortamper with it, but two processes can share memory if they need to. Windows NTbuilds its memory management on top of the memory management facilities pro-vided by the microprocessor. The 386 (and above) family of Intel microprocessorsprovides support for segmentation plus paging. The address translation mechanismfirst calculates the virtual address from the segment descriptor and the specifiedoffset within the segment. The virtual address is then converted to a physical ad-dress using the page tables. The operating system can restrict access to certainmemory regions by using the security mechanisms that are provided both at thesegment level and the page level.

Windows NT memory management provides the programmer with flat addressspace, data sharing, and so forth by selectively using the memory management fea-tures of the microprocessor. The virtual memory manager takes care of the pagingand allows 4GB of virtual address space for each process, even when the entire


system has much less physical memory at its disposal. The virtual memory managerkeeps track of all the physical pages in the system through the page frame database(PFD). The system also keeps track of the virtual address space for each process us-ing the virtual address descriptor (VAD) tree. Windows NT uses the copy-on-writemechanism for various purposes, especially for sharing the DLL data pages. Thememory manager has an important part in switching the processor context when aprocess is scheduled for execution. Windows 95/98 memory management is similarto Windows NT memory management with the differences being due to the factthat Windows 95/98 is not as security conscious as Windows NT.

Chapter 5

Reverse EngineeringTechniques

IN THIS CHAPTER

+ Preparing for and executing reverse engineering

+ Understanding code generation patterns

+ Debugging information from Windows NT itself

+ Deciphering the parameters of an undocumented function

THIS CHAPTER DIFFERS greatly from other chapters in the book. It does not containany undocumented Windows NT information. Instead, it provides some general tipsregarding how to reverse engineer on your own to explore the undocumentedWindows NT world.

This chapter teaches you how to reverse engineer Windows NT given the rawAssembly code and the useful symbolic information provided by Microsoft in theform of .DBG files. You can access these .DBG files on the Windows NT distributionCD-ROM. This chapter does not provide a complete guide to reverse engineering forthe simple reason that you cannot clearly define a way of approaching this prob-lem. Reverse engineering is like panning for gold; you have to sift through tons ofAssembly code to find a little information. But this chapter contains some usefultricks we have used to come up with undocumented Windows NT. Reverse engi-neering is an art, and it requires a lot of intuition, patience, and logical deduction.

We divided this chapter into different sections with each section describing astep in reverse engineering. We conclude the chapter by illustrating reverse engi-neering of a sample undocumented function. The best tool for implementing re-verse engineering is NuMega's excellent SoftlCE. This book would not have beenpossible without SoftlCE. This chapter assumes that the reader has used debuggers.We recommend trying out SoftlCE to get the most out of this chapter. Although theconcepts explained here specifically apply to reverse engineering NTOSKRNL (NTExecutive image) using SoftlCE, these concepts can apply to reverse engineeringany piece of operating system code.

85


How to Prepare forReverse EngineeringFirst, install SoftICE on your machine with "Boot time" as the option. Now copythe .DBG files in the SUPPORT directory on the Windows NT CD-ROM. There aremany .DBG files in this directory categorized according to the type of the file (forexample, .DLL, SYS, or .EXE). The .DBG files you will require depend upon theWindows NT component you want to explore.

Seethe NuMegaWebsiteat http: / / w w w . numega . com/ for up-to-dateversion information on SoftICE.

You need the following .DBG files to explore the KERNEL component:

KERNEL32.DBG

NTDLL.DBG

NTOSKRNL.DBG

You need the following .DBG files to explore the USER and GDI components:

USER32.DBG

GDI32.DBG

CSRSS.DBG

CSRSRV.DBG

WIN32K.DBG

Copy these .DBG files onto your hard drive, and then, using the symbol loader,convert .DBG files into .NMS (the native symbol format of SoftICE). Then, add thesefiles to SoftlCE's initialization settings using the SoftICE Initialize Settings/Symbolsoption in the symbol loader. This ensures that the symbols get loaded when SoftICEloads. Now, reboot the machine. SoftICE now contains the symbolic informationrather than the hex addresses, making the Assembly code look more readable. TheWindows 2000 symbolic information comes in .DBG and .PDB files instead of just.DBG files. One needs to have MSPDB60.DLL file from Visual C++ to covert thesefiles into native symbol format of SoftICE (.NMS)

Chapter 5: Reverse Engineering Techniques 87

How to Reverse EngineerBecause most of the Windows NT components are written in C, you must under-stand how the C compiler generates the Assembly code that corresponds to a Cfunction. You must also understand how a compiler generates the code to call aparticular function, how the parameters are passed, how compiler implements lo-cal variables, and so on. Compilers follow different function calling conventions.We will not get into the details of each and every compiler calling convention.Instead, we will cover only the stdcall and fastcall calling conventions becausemost of the functions in Windows NT follow either of these calling conventions.The NTOSKRNL.EXE contains a lot of functions with the fastcall calling conven-tion, the fastest of all the calling conventions.

In stdcall calling conventions, the parameters are pushed by the caller from rightto left, and the parameters pop off the stack by the called function. The advantageof using the stdcall calling convention is that it generates compact code becausethe code for popping the parameters off the stack resides in only one place (in thefunction itself). The disadvantage is that since a fixed number of parameters alwayspop off in the function, this calling convention cannot support a variable numberof arguments. To have a variable number of arguments, you must follow the cdedcalling convention.

The fastcall calling convention resembles stdcall, except its first two parametersare passed in registers instead of on a stack. This results in faster code because theregister access proves much faster than memory access.

Let us take one sample C function following the stdcall calling convention andsee the corresponding Assembly code generated by the compiler. In this example,we will also see how compiler-generated Assembly code accesses parameters passedto the function, and how local variables are implemented. The concepts explainedhere form the basis for reverse engineering discussed later in this chapter.


If you take a look at the Assembly code, the compiler generates the code to setthe EBP register to the start of the stack frame. (The stack frame for the functionstarts from EBP+8 since the compiler pushes the EBP register to maintain the stackframe set up by the caller function.) Hence, the parameters passed to the functionstart at EBP+8. Therefore, the first parameter x is accessed as [EBP+8] by the gen-erated Assembly code. The parameters y and z are accessed as [EBP+C] and[EBP+10] . For implementing local variables, compilers typically generate code,which decrements the ESP register by the total number of bytes required to hold allthe local variables defined in the function. In the previous code, there is only onelocal variable s u m ; therefore, the compiler allocates space for 4 bytes (1 DWORD) onthe stack by generating the instruction S U B E S P , 4. The EBP register accesses allsuch local variables as negative offsets. The variable s u m is accessed as [ E B P 4] inthe code. The LEAVE instruction used in the end restores the contents of EBP regis-ter and cleans up the local variables.

Let us demonstrate the preceding mechanism in tables.When the function sum is called, the stack frame looks like:

30 fl Last parameter

20 fl Second parameter

T Chapter 5: Reverse Engineering Techniques 89

After setting up the standard stack frame of PUSH EBP, MOV EBP, ESP and cre-ating space for local variables, the stack looks like:

Most of the functions in the NTOSKRNL access the parameters and local vari-ables in the same way (by setting up the frame using EBP registers and accessingthe local variables using the negative offsets from the EBP register). But a few func-tions do not set up this standard stack frame; instead, the parameters are accesseddirectly using the ESP register (such as ESP+8). In this case, reverse engineering be-comes very difficult because the same parameter is accessed using different offsetsfrom the ESP register at different places. The advantage is that it results in fasterand more compact code.


Understanding CodeGeneration PatternsBecause compilers are themselves software programs, they follow a certain patternwhen generating the Assembly code.

LEA EDI, [ E B P - 2 4 ] -MOV E C X , 6R E P S Z STOSD

This piece of code initializes the memory of 6 DWORD size (0x18 bytes), whichstarts at location EBP-24. This also suggests that probably some structures of size0x18 bytes is locally defined and initialized in the function.

MOV E A X , [EBP+18]TEST E A X , 00008000JZ B i tNo tSe t

Bi tNotSet:

This piece of code tests the fifteenth bit of the fifth parameter passed to thefunction, assuming standard stack frame is generated for the function and doesthe processing based on the bit test results.

MOV E A X , [FS:124]

This statement fills in the EAX register with a pointer to the current thread object.Note that the FS register points to a Processor Control Region (PCR) in kernel mode. 4

MOV E A X , [FS:124] - " " * -'MOV E A X , [EAX+40]

This piece of code fills in the EAX register with a pointer to the current processobject under Windows NT 3.51. Under Windows NT 4.0 and Windows 2000, this in-struction looks like MOV EAX, [EAX+44], since the offset of pointer to process ob-ject is changed from 3.51 to 4.0.


How Windows NT ProvidesDebugging InformationVarious kernel data variables control the output of debug messages. By turning on afew bits in these variables, you can get more debugging messages from the operat-ing system apart from the messages given by default. Some of these bits are alreadyturned on in checked builds of the operating system, although some of them are not.We strongly feel that Microsoft itself likely turns on bits of these variables wheneverthey get any bug information and they want to figure out the problem. ButMicrosoft probably turns these bits off when they get the release out. By doing this,Microsoft hides a wealth of information from operating system reverse engineering.We expose a part of this wealth here. There could be many other such flags. Piecesof hidden debug messages code inside NTOSKRNL appear like this:

TEST [DebugVar iab le ] , 0x80JZ HideFromReverseEngineeringPUSHPUSHPUSHCALL DbgPrint

HideFromReverseEngineering:

Whenever you come across such a piece of code, just set the required bit fromSoftICE, and you will see all those messages that are hidden.

Here are some of the known variables in NTOSKRNL and the debug messagesshown by the operating system when these variables or bits of these variablesare turned on. Most of the variables appear only in the checked builds of theoperating system.

ExpEchoPool CallsBy setting this variable to 1, you can get the information about each memory allo-cation/deallocation performed using functions such as ExAllocatePoolWithTag andExFreePool. The information shown includes the address where the memory was al-located, size of the region allocated, type of the pool used (paged/nonpaged), andtype of memory (cache, aligned, and so on). The information displays as follows:

"Oxel354668 EXALLOC: from Paged s ize 284 Ca l le rs :0 , 0Oxel354668 E X D E A L L O C : from Paged C a l l e r s : 0 , 0"


ObpShowAllocAndFreeBy setting this variable to 1, you can get information about each executive objectwhen it is created/destroyed. The information includes the memory address wherethe object was created and the type of the object (Key, Semaphore, and so on). Theinformation appears like this:

"OB: A l i c e e!304908 (e!304908) 0012 - KeyOB: Free e!304908 (e!304908) - Type: Key"

LpcpTraceMessagesThis variable proves very useful in reverse engineering the local procedure callmechanism (LPC) used by Windows NT for implementing various subsystems. Bysetting this variable to 1, you can get tons of information about how LPC functions.The information displays as follows:

MmDebugBy setting different bits of this variable, you can see different messages generatedby the memory management module. Following, we list the bits of this variablethat the operating system can set and then generate the corresponding messages.

Bit 2


MM.actual fault c030d500 va C3540000***DumpPTE at c030d500 contains 7f4434 protoaddr ellfd068 subsect ffbBObbOremoving wsle 313 C3540661


crea sect access mask fOOlf maxsize 10000 page prot 4allocation attributes 4000000 file handle 0

return crea sect handle IfO status 0mapview process handle ffffffff section IfO base address 0 zero bits 0

view size 0 offset 0 commitsize 10000 protect 4Inheritdisp 2 Allocation type 0

Bit 30

MM:**access fault va 77ealdl7 process fdf787aO thread fdf77020MM:**access fault - va 77ea31ba process fdf787aO thread fdf77020

ObDebugFlags

NtGlobalFlagOne bit of this variable enables the debug messages. Other bits control the valida-tions performed by the operating system and general operation of the operatingsystem. Take a look at the GFLAGS utility in the resource kit for the description ofindividual bits of NtGlobalFlag. The value of this variable is inherited by a variablein NTDLL.DLL during the process startup. NTDLL.DLL uses the second bit of thisvariable to show the loading of a process. During process startup, NTDLL gets thevalue of this flag and sets its internal variable ShowSnap to 1 if the second bit isset. Once this bit is set, you can watch the behavior of the PE executable/DLLloader. Windows NT will show names of all the imported DLLs, plus it will show areal set of DLLs required to start an application. It will also show you the address ofinitialization functions of each of these DLLs as well as a lot of other information.

Two bits of this variable (the fifth and sixth bits) control the operating systemdebug messages. These bits control the security descriptor-related messages


Look at the following messages displayed by the operating system by just turningon one bit of the NtGlobal flag variable. Here, we started pstatexe and terminatedit immediately:

LDR: PID: 0x47 started 'pstat1

LDR: NEW PROCESSImage Path: C:\MSTOOLS\bin\PSTAT.EXE (PSTAT.EXE)Current Directory: C:\MSTOOLS\binSearch Path: C:\MSTOOLS\bin;.;C:\WINNT40\System32;C:\WINNT40\system;C:\WINN

T40;C:\WINNT40\system32;C:\WINNT40;c:\winnt35;c:\winnt35\system32;c:\msdev\bin;C:\DOSLDR: PSTAT.EXE bound to USER32.dllNTICE: Load32 START=77E10000 SIZE=62000 KPEB=FF925DEO MOD=user32LDR: ntdll.dll used by USER32.dllLDR: Snapping imports for USER32.dll from n t d l l . d l lLDR: KERNEL32.dll used by USER32.dllNTICE: Load32 START=77EDOOOO SIZE=5EOOO KPEB=FF925DEO MOD=kernel32LDR: ntdll.dll used by KERNEL32.dllLDR: Snapping imports for KERNEL32.dll from n t d l l . d l lLDR: Snapping imports for USER32.dll from KERNEL32.dllLDR: LdrLoadDll, loading NTDLL.dll fromLDR: LdrGetProcedureAddress by NAME RtlReAllocateHeapLDR: LdrLoadDll, loading NTDLL.dll fromLDR: LdrGetProcedureAddress by NAME - RtlSizeHeap

LDR: LdrLoadDll, loading NTDLL.dll fromLDR: LdrGetProcedureAddress by NAME RtlUnwindLDR: LdrLoadDll, loading NTDLL.dll fromLDR: LdrGetProcedureAddress by NAME - Rtl Al locateHeapLDR: LdrLoadDll, loading NTDLL.dll fromLDR: LdrGetProcedureAddress by NAME - RtlFreeHeapLDR: Refcount USER32.dll (1)LDR: Refcount KERNEL32.dll (1)LDR: Refcount GDI32.dll (1)LDR: Refcount KERNEL32.dll (2)

LDR: Real INIT LISTC:\WINNT40\system32\KERNEL32.dll init routine 77ed47aOC:\WINNT40\system32\RPCRT4.dll i n i t routine 77dc060dC:\WINNT40\system32\ADVAPI32.dll init routine 77d38650C:\WINNT40\system32\USLR32.dll i n i t routine 77e23890

LDR: KERNEL32.dll loaded. Calling init routine at 77ed47aO


LDR: RPCRT4.dll loaded. - C a l l i n g init routine at 77dc060dLDR: ADVAPI32.dll loaded. - C a l l i n g imt routine at 77d38650LDR: USER32.dll loaded. - C a l l i n g imt routine at 77e23890LDR: PID: 0x47 finished - 'pstat'NTICE: Exit32 PID=47 MOD=PSTAT

SepDumpSD

TokenGlobalFlagBy setting this variable to 1, the operating system dumps the security token-relatedmessages.

SE ( T o k e n ) : Acqu i r i ng Token READ Lock for a c c e s s to tokenOxel l826fOSE ( T o k e n ) : R e l e a s i n g Token Lock for a c c e s s to token Oxel l826fOSE ( T o k e n ) : Acqu i r i ng Token READ Lock for access to tokenOxel l826fOSE ( T o k e n ) : Re leas i ng Token Lock for a c c e s s to token Oxel l826fO

By setting this variable to 1, the operating system dumps the security descriptor inthe security handling-related code.


CmLogLevel and CmLogSelectThese variables control the debugging messages given by the registry handlingcode. Different log levels serve as debug levels. By setting the individual bits inCmLogSelect, you can control the volume of messages generated by the operatingsystem. The maximum value of CmLogLevel is 7. By default, the individual bits inCmLogSelect are set to produce the most verbose output.

How to Decipher theParameters Passed to anUndocumented FunctionThis section describes how you can find out the parameters to be passed to an un-documented function. The first step in deciphering parameters is to set a breakpointon the function using SoftlCE. If you know the application that uses this undocu-mented function (from the import dump), start the application. For example, Dr.Watson (DRWTSN32.EXE) uses an undocumented NTDLL function NtOpenThread().


You can find a complete list of functions (documented as well as undocu-mented) imported by an application using the DUMPBIN utility. For example,DUMPBIN PROGMAN.EXE /IMPORTS will display all the functions imported bythe program manager.

To start DRWTSN32, begin an application that faults (GPF) or write one thatdoes the fault explicitly. If you do not know an application that uses this undocu-mented function, try to find an equivalent Win32 API call. If you find such a call,write an application that will call this function. Assuming you want to decipher theparameters passed to a NtAllocateVirtualMemory system service, you may write anapplication that calls VirtualAllocQ. Once the breakpoint for the function that youwant to decipher is triggered, you can look at the details of the function implemen-tation. You can use some general tricks to decipher the parameters. We discuss afew of them in the sections that follow. s

Examining the Error Handling CodeMany times a function checks for the value of a particular parameter, and if it is notappropriate, returns an error code. By examining the error code, you can get infor-mation about the error in NTSTATUS.H file from DDK. Then, we can find out thetype of parameter used.

Consider the following piece of code in an undocumented NtQueryMutantsystem service:

CMP D W O R D PTR [EBP+C] , 0J Z 8019D397MOV DWORD PTR [EBP-34], C0000003 (STATUS_INVALID_INFO_CLASS)

8019D397:CMP DWORD [EBP+14], 8JZ 8019D3B3MOV DWORD PTR [EBP 34] , C0000004 (STATUS_INFO_LENGTH_MISMATCH)

From this Assembly code, you can easily see that [ E B P + C ] , the second parame-ter, contains the InfoClass, and [ EBP+14] , the fourth parameter, contains the size ofthe buffer that holds the mutant information.

Use in the FunctionSometimes, a particular parameter of an undocumented function is passed as a para-meter to some documented function. In this case, by looking at the documented func-tion, you can easily find out the parameter passed to the undocumented function.


Looking at this code, you can clearly see that the first parameter to theNtQueryMutantQ function is the Mutex object handle because the same parameteris passed a first parameter to documented ObReferenceObjectByHandleO function,and first parameter to ObReferenceObjectByHandleO function is the object handle.Hence, using the knowledge that the name of the function is NtQueryMutant andthe first parameter is passed as is to ObReferenceObjectByHandle as a object handle,we can conclude that the first parameter might be a handle to a mutex object.

Checking the Validation CodeSometimes, a piece of code checks for the value of a parameter and displays a mes-sage if it has a particular value. By looking at the message provided by the operat-ing system, you can find out the parameter. Especially in checked builds, asserts areused extensively. By looking at the messages in these asserts, you can find out theparameters. For example, a function that expects PEB as a parameter contains apiece of code that checks if the type field of the object is a Process object.

Typical Assembly LanguagePatterns and Their A/leaningsThis piece of code gets the Current process object pointer (PEB) in the EAX register:

MOV E A X , FS:[124]MOV E A X , [EAX+40]

Consider the following piece of code in the NtQueryMutantQ function:


While executing in kernel mode, FS: [124] always points to the currently exe-cuting thread (TEB) and [TEB+40] always points to the current process. UnderWindows NT 4.0 and Windows 2000, [TEB+44] points to the current process.

The preceding two pieces of code route to the page table entry and the page di-rectory entry, respectively, for the virtual address present in the ESI register. Thefunctioning registers might change; however, the pattern remains the same. Youmay have seen this code in many memory management-related functions. At firstit looks odd; however, it is highly optimized using the 2's complement method. Asan exercise, try to determine how this works. Hint: Page tables are mapped startingat the virtual address OxCOOOOOOO, and Page directory is mapped starting at thevirtual address OxC0300000.

PUSH 00LEA EAX,[EBP 20]PUSH EAXPUSH ECX . ,.PUSH DWORD PTR [_PsProcessType]PUSH 08PUSH DWORD PTR [EBP+08]CALL JDbReferenceObjectByHandle *MOV [EBP 24],EAXTEST EAX.EAXJL .....MOV EAX,FS:[00000124]MOV ECX,[EBP-20]CMP [EAX+40LECX - _ i-

•* > rJZ ...PUSH ECXCALL _KeAt tachProcess

Here, the code attempts to play with other processes. It wants to perform somework on behalf of another process. This piece of code gets the handle to the Processobject as a parameter. Using this handle, the code reaches to the actual object andthen compares the address of the Process object with the address of the currentProcess object stored at [TEB+40] in Windows NT 3.51 and [TEB+44] in Windows


NT 4.0 and Windows 2000. If the Process object dealt with is not the current Processobject, then the code attaches to the desired Process object using KeAttachProces().The code following this will execute in the context of the attached process. You cansee a similar kind of code in the system services that have the ability to play in otherprocesses. The system service NtAllocateVirtualMemory enables allocation of thememory for a process other than the current one. You will find this kind of code inthe NtAllocateVirtualMemoryO function. Other places where you can find this codeare NtFreeVirtualMemoryO and NtLockVirtualMemoryO.

The Practical Application ofReverse EngineeringNow, let's observe the practical application of the reverse engineering techniquesdiscussed in this chapter. We will show clearly how you can arrive at pseudocodegiven the raw assembler listing.

You can study the example we chose in Chapter 10,"Adding New SoftwareInterrupts."

In Chapter 10, we discuss the callgate implementation on Windows NT (for run-ning ring 0 code from ring 3 application). When we decided to design the callgatemechanism, we were in search of some mechanism to allocate the selectors — thebasic requirement for creating callgates. We knew that the Win32 application didnot have a Local Descriptor Table (LDT). Therefore, we wanted to allocate selectorsfrom a Global Descriptor Table (GOT). First, we looked at the symbols of NTOSKRNLby using SoftlCE's command SYM 'Selector*. We received some entries matchingthe regular expression *Selector*.

One symbol we found interesting was KeI386AllocateGdtSelector. We deducedfrom the name that this function must allocate GOT Selectors. Next, we took the ex-port dump of NTOSKRNL to see whether the function is exported. You can make useof undocumented functions only if the function is exported. If the function is not ex-ported then you have to deal with hard-coded addresses. This makes the programbound to the specific version of Windows NT (for example, NT 3.51/4.0/2000, freebuilds/checked builds/service packs). Luckily, we found that the function was ex-ported. Our next step was to put breakpoint on this function. Unfortunately, we foundthat this breakpoint is never triggered on our configuration, so we decided to reverseengineer the function ourselves. We extracted the Assembly output of the functionusing the SoftICE history buffer. Here is the raw Assembly code for the function:


Looking at the last instruction, RET 8, the function clearly followed the _stdcallcalling convention with two parameters to the function. We next had to decipherwhat those parameters were. Because the compiler generated the standard stackframe (PUSH EBP, MOV EBP, ESP), clearly EBP+8 referred to the first parameter,and EBP+C referred to the second parameter.

The following instruction sequence suggests that the second parameter repre-sents the number of selectors to be allocated:

0008:80125003 PUSH ESI


This code moves the second parameter in the SI register and compares the SIregister with the kernel variable KiNumberFreeSelectors$S 10229. If the value in theSI register is less than KiNumberFreeSelectors$S 10229, then the code jumps to a la-bel and from there fills in the EAX register with an error code of STATUS_ABIOS_SELECTOR_NOT_AVAlLABLE. Clearly, the second parameter to the function was"Number of Selectors to allocate."

Next, we looked at the code, assuming an x number of available selectors. Weassumed that the JB condition evaluated to false.

The next two instructions acquired the GDT lock. Locks are extensively used atvarious places to protect multiple threads from accessing some shared kernel datastructure. Most of the time, you can ignore these pieces of code, because they havenothing to do with the actual logic of the function.

0008:80125012 MOV E C X , _ K i A b i o s G d t L o c k0008:80125017 CALL [__ imp_@KfAcqui reSpinLock]

The next instruction decrements the value of the kernel variable _KiNumberFreeSelectors$S 10229 according to the number of selectors to be allocated.

0008:80125010 SUB [_KiNumberFreeSelectors$S10229],SI

Then, the function loads the EDX register with the value of the kernel variable_KiFreeGdtListHead$S 10230. Looking at the instruction, you can see the selectorsare put in a free list.

0008:80125024 MOV E D X , [ _ K i F r e e G d t L i s t H e a d $ S 1 0 2 3 0 ]

Next, the function checks to see if the number of selectors to be allocated is zero.In that case, the function jumps to a label where some rollback is done, and theEAX register is zeroed out indicating success so the function returns.

0008:80125D2A TEST SI,SI0008:80125020 JZ 80125D47


The code fills the ECX register with the first parameter. Then, it loads the EDIregister with the value of the EDX register (_KiFreeGdtlistHead$S 10230). Next, itsubtracts the value of the kernel variable KiAbiosGdt. The value of the kernel vari-able KiAbiosGdt matched with the base address of the Global Descriptor Table.Hence, the preceding piece of code extracts the selector value in the DI register.Next, the code copies the selector value in the location pointed by the ECX register.The code then adds 2 to the ECX register. From this, we deduced that the first para-meter points to a buffer that contains the selector values allocated with each entryconsisting of 2 bytes. Therefore, the first parameter must be an array of short inte-gers. The code reaches to the next free selector using the instruction:

MOV E D X , [ E D X ]

From this, we can see that the free selectors are maintained in a linked list, andthe descriptors are used for keeping track of the next free selector in the list. The SIregister decrements each time in the loop. Initially, the SI register contains the num-ber of selectors to be allocated. In the end, the SI register reaches 0. At this point, thebuffer pointed by second parameter contains the list of selectors allocated.


SummaryIn this chapter, we described how to use symbolic information supplied withWindows NT using SoftlCE. We also discussed some general techniques used for re-verse engineering, such as how to understand the compiler code generation pat-terns. Next, we showed how Windows NT can assist in reverse engineering byenabling some debugging flags in the kernel. We also discussed various ways of de-ciphering the parameters for undocumented functions. Next, we reviewed sometypical Assembly language patterns found throughout the Windows NT kernelcode. The chapter concluded with an example showing the deciphering of an un-documented function called KeI386AllocateGdtSelectors from NTOSKRNL EXE.

PartUndocumented Windows NT

CHAPTER 6Hooking Windows NT System Services

CHAPTER 7Adding New System Services to the

Windows NT KernelCHAPTER 8

Local Procedure CallCHAPTER 9

Hooking Software InterruptsCHAPTER 10

Adding New Software InterruptsCHAPTER 11

Portable Executable File Format

Chapter 6

Hooking Windows NTSystem Services

IN THIS CHAPTER

+ Looking at system services under various operating systems

+ Examining the need for hooking system services

+ Implementing types of hooks

THIS CHAPTER DISCUSSES hooking Windows NT system services. Before we begin,let's first review what we mean by a system service. A system service refers to a setof functions (primitive or elaborate) provided by the operating system. Applicationprogramming interfaces (APIs) enable developers to call several system services, di-rectly or indirectly. The operating system provides APIs in the form of a dynamiclink library (DLL) or a static compiler library. These APIs are often based on systemservices provided by the operating system. Some of the API calls are directly basedon a corresponding system service, and some jlepend on making multiple systemservice calls. Also, some of the API calls may not make any calls to system services.In short, you do not need a one-to-one mapping between API functions and systemservices. Figure 6-1 demonstrates this in context of Windows NT.

System Services: The Long ViewSystem services and the APIs calling these system services have come a long wayfrom DOS to Windows NT.

System Services under DOSUnder DOS, system services comprise part of the MS-DOS kernel (includingMSDOS.SYS and IO.SYS). These system services are available to users in the form ofInterrupt Service Routines (ISRs). ISRs can be invoked by calling the appropriateinterrupt handlers using the INT instruction. API functions, provided by compilerlibraries, call the interrupt handler for system services (the INT 21h interrupt). Forexample, to open a file, MS-DOS provides a system service for which you have to 109

110 Part 11: Undocumented Windows NT

specify the function number Ox3D in the AH register, attribute mask in the CL reg-ister, filename in the DS:DX register, as well as issue the INT 21h instruction.Compilers typically provide wrappers around this and provide a nice API functionfor this purpose.

System Services under Windows 3.x andWindows 95/98Under Windows 3.x or Windows 95/98, the core system services take the form ofVXDs and DLLs and some real-mode DOS code. The APIs are provided in the formof dynamic link libraries. These dynamic link libraries call the system services toimplement the APIs. For example, to open a file, applications call an API functionfrom KERNEL32.DLL such as OpenFileQ or CreateFilefJ. These APIs, in turn, cal1 asystem service.

System Services under Windows NTUnder Windows NT, the NT executive (part of NTOSKRNL.EXE) provides core sys-tem services. These services are rather generic and primitive. Various APIs such asWin32, OS/2, and POSIX are provided in the form of DLLs. These APIs, in turn, callservices provided by the NT executive. The name of the API function to call differsfor users calling from different subsystems even though the same system service isinvoked. For example, to open a file from the Win32 API, applications callCreateFileO and to open a file from the POSIX API, applications call the open()function. Both of these applications ultimately call the NtCreateFileQ system servicefrom the NT executive.

Chapter 6: Hooking Windows NT System Services 111

Under Windows NT 3.51, the system services are provided by a kernel-modecomponent called NTOSKRNL.EXE. Most of the KERNEL32.DLL calls —such asthose related to memory management and kernel objects management —are handled by these system services.The USER32 and GDI32 calls are han-dled by a separate subsystem process called CSRSS. Starting with WindowsNT 4.0, Microsoft moved most of the functionality of CSRSS into a kernel-mode driver called WIN32K.SYS.The functionality moved into WIN32K.SYS ismade available to the applications in the form of system services.These sys-tem services are not truly part of native system services since they are spe-cific to the user interface and not used by all subsystems.This chapter and thenext chapter focus only on the system services provided by NTOSKRNL.EXE.

Need for Hooking System ServicesHooking represents a very common mechanism oTfntercepting a particular sectionof executing code. Hooking provides a useful way of modifying the behavior of theoperating system. Hooking can help the developer in several ways. Often develop-ers are concerned more with how to hook a system service or an API call ratherthan why to hook. Nevertheless, we examine the various possible situations inwhich the need to hook a system service arises. How hooking can help the devel-oper is explained in the following sections.

Trapping Events at OccurrenceDevelopers trap events such as the creation of a file (CreateFileO), creation of a mu-tex (CreateMutex()), or Registry accesses (RegCreateKeyO) for specific purposes.Hooking a particular event-related API or system service call, synchronously, canhelp trap those events. Applications doing system monitoring will find these kindsof hooking invaluable. These hooks could act as interrupts triggered by the occur-rence of these events. A developer could write a routine to handle the occurrence ofthese events and take appropriate action.

Modifying System Behavior to Suit User NeedsDiverting the normal flow of control by introducing the hooks can modify operat-ing system behavior. This enables the developer to change data structures and con-text at the time of hooking - enough to induce new behavior. For example, you canprotect the opening of a sensitive file by hooking the NtCreateFileQ system service.Although NTFS provides user-level security for files, this security is not availableon FAT partitions. You should ensure that hooking does not have any undesirableside effects on the operating system. Protecting modifications to Registry keys is


something easily doable when you hook the Registry system services. This has sev-eral applications, since little protection is provided for Registry settings created byapplications. •- '

Studying the Behavior of the SystemIn order to get a better idea of the internal workings of the operating system, study-ing the behavior of the system is something most debuggers or system hackers willrelate to. Understanding of undocumented operating system functionality requiresa lot of hacking, which goes hand in hand with hooking.

DebuggingComplex programs could make use of system-service hooking to debug the sticki-est problems. For example, a few days back, we had a problem with the installationof a piece of software. We had difficulty creating folders and shortcuts for this ap-plication. Using a systemwide hook, we quickly figured that the installation pro-gram was looking for a Registry value that indicated where to install the folders(which happened to be the Start menu). We hooked the NtQueryValueKeyQ call,then obtained the value the installation program was looking for. We created thatvalue and solved our problem.

Getting Performance Data for Specific Tasks andGenerating StatisticsThese tasks can prove very useful to those writing benchmarks and applications tocritically measure system performance under specific conditions. Even measuringthe frequency of certain system services becomes very easy with this type of hook-ing. Measuring file system performance by hooking the file system-related systemservices exemplify this procedure.

Life without hooking is unthinkable for most Windows developers in today'sMicrosoft-dominated world of operating systems. Windows NT system services lieat the center of the NT universe, and having the ability to hook these can prove ex-tremely handy.

Types of HooksThe following sections explore two types of hooking.

Kern el-Level HookingYou can achieve kernel-level hooking by writing a VXD or device driver. In thismethod, essential functions provided by the kernel are hooked. The advantage of

______________Chapter 6: Hooking Windows NT System Services____113

this type of hooking is that you get one central place from which you can monitorthe events occurring as a result of a user-mode call or a kernel-mode call. The dis-advantage of this method is that you need to decipher the parameters of the callpassed from kernel mode, since many times these services are undocumented. Also,the data passed to the kernel-mode call might differ from the data passed in a user-mode call. Also, a user-level API call might be implemented using multiple calls tothe kernel. In this case, hooking becomes far more difficult. In general, this type ofhooking is more difficult to achieve, but it can produce more rewarding results.

User-Level HookingYou can perform this type of hooking with some help from a VXD or device driver.In this method, the functions provided by the user-mode DLLs are hooked. The ad-vantage of this method is that these functions are usually well documented.Therefore, you know the parameters to expect. This makes it easy to write the hookfunction. This type of hooking limits your field of vision to user mode only anddoes not extend to kernel mode.

Implementations of HooksThe following sections detail the implementation of hooks under various Microsoftplatforms.

DOSIn the DOS world, system services are implemented as an interrupt handler routine(INT 2In). The compiler library routines typically call this interrupt handler to pro-vide an API function to the programmer. It is trivial to hook this handler using theGetVect (INT 21h, AX=25h) and SetVect (Int 21h, AX=35h) services. Hence, hookingsystem services are fairly straightforward. DOS does not contain separate user andkernel modes.

Windows 3.xIn the Windows 3.x world, system services are implemented in DLLs. The compilerlibrary routines represent stubs that jump to the DLL code (this is called dynamiclinking of DLLs). Also, because the address space is common to all applications,hooking amounts to getting the address of that particular system service andchanging a few bytes at that address. Changing of these bytes sometimes requiresthe simple aliasing of selectors. -


Refer to the MSDN article in Microsoft Systems Journal (Vol. 9, No. 1) entitled,"Hook and Monitor Any 16-bit Windows(tm) Function With Our ProcHookDLL," by James Finnegan.

Windows 95 and 98In the Windows 95/98 world, system services are implemented in a DLL as inWindows 3.1. However, under Windows 95/98, all 32-bit applications run in sepa-rate address spaces. Because of this, you cannot easily hook any unshared DLL. It isfairly easy to hook a shared DLL such as KERNEL32.DLL. You simply modify a fewcode bytes at the start of the system service you want to hook and write your hookfunction in a DLL that is loaded in shared memory. Modifying the code bytes mayinvolve writing a VXD, because KERNEL32.DLL is loaded in the upper 2GB of theaddress space and protected by the operating system.

r

Windows NTIn the Windows NT world, system services are implemented in the kernel compo-nent of NT (NTOSKRNL.EXE). The APIs supported by various subsystems (Win32,OS/2, and POSIX) are implemented by using these system services. There is no doc-umented way of hooking these system services from kernel mode. There are severaldocumented ways for hooking user-level API calls.

Refer to the MSDN articles in Microsoft Systems Journal entitled, "LearnSystem-Level Win32(r) Coding Techniques by Writing and API Spy Program,"by Matt Pietrek (Vol.9, No.12), and "Load Your 32-bit DLL into AnotherProcess's Address Space Using INJLIB,"by Jeffrey Richter (Vol.9, No.5). $

Refer to CyberSensor on http://www.cybermedia.co.in

We will present one way of achieving hooking of NT system services in kernelmode in this chapter. We also provide the code for this on the CD-ROM accompa-nying this book.

Windows NT System ServicesWindows NT has been designed with several design goals in mind. Support formultiple (popular) APIs, extensibility, isolation of various APIs from each other,and security are some of the most important ones. The present design incorporates


several protected subsystems (for example, the Win32 subsystem, the POSIX sub-system, and others) that reside in the user space isolated from each other. The NTexecutive runs in the kernel mode and provides native support to all the subsys-tems. All subsystems use the NT system services provided by the NT executive toimplement most of their core functionality.

Windows programmers, when they link with the KERNEL32, USER32, and GDI32DLLs, are completely unaware of the existence of the NT system services supportingthe various Win32 calls they make. Similarly, POSIX clients using the POSIX APIend up using more or less the same set of NT system services to get what they wantfrom the kernel. Thus, NT system services represent the fundamental interface forany user-mode application or subsystem to the kernel.

For example, when a Win32 application calls CreateProcessQ or when a POSIXapplication calls the fork() call, both ultimately call the NtCreateProcessQ systemservice from the NT executive.

NT system services represent routines, which run entirely in the kernel mode. Forthose familiar with the Unix world, NT system services can be considered theequivalent of system calls in Unix.

Figure 6-2: A caller program invoking an NT system service

Currently, Windows NT system services are not completely documented. Theonly place where you can find some documentation regarding the NT system ser-vices is on Windows NT DDK CD-ROMs from Microsoft. The DDK discusses about25 different system services and covers the parameters passed to them in some de-tail. You'll see from Appendix A that this is only the tip of the iceberg. In Windows


NT 3.51, OxC4 different system services exist, in Windows NT 4.0, OxD3 differentsystem services exist, and in Windows 2000 Beta-2, OxF4 different system servicesexist.

We deciphered the parameters of 90°/o of the system services. Prototypes for allthese system services can be found in UNDOCNT.H on the CD-ROM included withthis book. We also provide detailed documentation of some of the system servicesin Appendix A.

In the following section, you will learn how to hook these system services.

Hooking NT System ServicesLet's first look at how NT System Services are implemented in the Windows NT op-erating system. We also will discuss the exact mechanics of hooking an NT systemservice. In addition, we'll explore the kernel data structures involved and providesample code to aid hooking of system services.

ON THE <D Check out hookdrv.c on the accompanying CD-ROM.

Implementation of a System Service inWindows NTThe user mode interface to the system services of NTOSKRNL is provided in theform of wrapper functions. These wrapper functions are present in a DLL called NT-DLL.DLL. These wrappers use the INT 2E instruction to switch to the kernel modeand execute the requested system service. The Win32 API functions (mainly inKERNEL32.DLL and ADVAPI32.DLL) use these wrappers for calling a system ser-vice. The Win32 API functions performs validations on the parameters passed tothe API functions, and translates everything to Unicode. After this, the Win32 APIfunction calls an appropriate wrapper function in NTDLL corresponding to the re-quired service. Each system service in NTOSKRNL is identified by the Service ID.The wrapper function in NTDLL fills in the service id of the requested system ser-vice in the EAX register, fills in the pointer to stack frame of the parameters in EDXregister, and issues the INT 2E instruction. This instruction changes the processor tothe kernel mode, and the processor starts executing the handler specified for theINT 2E in the Interrupt Descriptor Table (TDT). The Windows NT executive sets upthis handler. The INT 2E handler copies the parameters from user-mode stack tokernel-mode stack. The base of the stack frame is identified by the contents of the


EDX register. The INT 2E handler provided by NT Executive is internally called asKiSystemServiceQ.

During the initialization of NTOSKRNL, it creates a function table, hereafter re-ferred to as the System Service Dispatch Table (SSDT), for different services pro-vided by NTOSKRNL (see Figure 6-3). Each entry in the table contains the addressof the function to be executed for a given service ID. The INT 2Eh handler looks upthis table based on the service ID passed in EAX register and calls the correspond-ing system service. The code for each function resides in the kernel. Similarly, an-other table called the ParamTable (hereafter referred to as System Service ParameterTable [SSPT]) provides the handler with the number of parameter bytes to expectfrom a particular service.

Figure 6-3: System Service Dispatch Table and Parameter Table

Hooking NT System ServicesThe easiest way to put a hook into the system services is to locate the SystemService Dispatch Table used by the operating system and change the functionpointers to point to some other function inserted by the developer. You can do thisonly from a kernel-mode device driver because this table is protected by the oper-ating system at the page table level. The page attribute for these pages is set so thatonly kernel-mode components can read from and write to this table. User-level ap-plications cannot read or write these memory locations.


LOCATING THE SYSTEM SERVICE DISPATCH TABLE IN THENTOSKR1MLThere is one undocumented entry in the export list of NTOSKRNL calledKeServiceDescriptorTable(). This entry is the key to accessing the System ServiceDispatch Table. The structure of this entry looks like this:

typedef struct ServiceDescriptorTable {PVOID S e r v i c e T a b l e B a s e ;P V O I D S e r v i c e C o u n t e r T a b l e ( O ) ; • j •unsigned int N u m b e r O f S e r v i c e s ;PVOID P a r a m T a b l e B a s e ;

}

ServiceTableBase and ParamTableBase contain NumberOfServices entries. Eachentry represents a pointer to a function implementing the corresponding systemservice.

The following program provides an example of hooking system services, underWindows NT. The system service NtCreateFileQ hooks and the name of the file cre-ated prints when the hook gets invoked. We encourage you to insert code for hook-ing any other system service of choice. Note the proper places for inserting newhooks in the following code.

Here are the steps to try out the sample (assuming that the sample binaries arecopied in C:\SAMPLES directory):

1. Run "instdrv hooksys c:\samples\hooksys.sys." This will install thehooksys.sys driver. The driver will hook the NtCreateFile system service.

2. Try to access the files on your hard disk. For each accessed file, thehooksys.sys will trap the call and display the name of the file accessed inthe debugger window. These messages can be seen in SoftICE or using thedebug message-capturing tool.

#include "ntddk.h"#include "s tdarg.h"#include "s td io .h "#include "hooksys .h "

Base address of the System Service Dispatch Table.

Number of services described by ServiceTableBase.

This field is used only in checked builds of the operatingsystem and contains the counter of how many times eachservice in SSDT is called. This counter is updated by INT 2Ehhandler (KiSystemService).Base address of the table containing the number of parame-ter bytes for each of the system services.


SummaryIn this chapter, we explored system services under DOS, Windows 3.x, Windows95/98, and Windows NT. We discussed the need for hooking these system services.We discussed kernel- and user-lever hooks. We discussed the data structures usedduring the system call and the mechanism used for hooking Windows NT systemservices. The chapter concluded with an example that hooked the NtCreateFileQsystem service.

Chapter 7

Adding New SystemServices to the WindowsNT Kernel

IN THIS CHAPTER ')+ Detailed Implementation of a system service

+ Adding new system services

CUSTOMIZING THE KERNEL for specific purposes has been very popular among devel-opers long before Windows NT. Ancient Unix gurus and developers alike practicedthe art. In Unix, for example, kernel developers can modify the kernel in severalways, such as adding new device drivers, kernel extensions, system calls, and ker-nel processes. In Windows NT, DDK provide means to add new device drivers.However, one of most effective ways of modifying the kernel — adding new systemservices to it — is not documented. This method proves more efficient than addingdevice drivers for several reasons discussed later in this chapter. Here, we focus onthe detailed implementation of a system service inside the Windows NT kernel andexplain, with examples, how new system services can add to the Windows NT.

In Inside Windows NT, Helen Custer mentions the design of system services andthe possibility of adding new system services to the kernel:

Using a system service dispatch table provides an opportunity to make native NT sys-tem services extensible. The kernel can support new system services simply by ex-panding the table without requiring changes to the system or to applications. After acode is written for a new system service, a system administrator could simply run autility program that dynamically creates a new dispatch table. The new table will con-tain another entry that points to a new system service.

The capability to add new system services exists in Windows NT but it is notdocumented.

Very little changed between NT 3.51 and later versions of Windows NT in this area.The only thing being changed is that some of the data structures involved in imple-mentation of a system service are located at the different offsets in the later versionsof the operating system. We feel that our method of adding new system services mayhold, possibly with very minor modifications, in future releases of Windows NT.


At the end of this chapter, we try to shed some light on the possible thought thatwent into the design of this portion of the operating system.

Detailed Implementation of aSystem Service in Windows NTIn Chapter 6, we discussed how a system service is invoked by the NTDLL.DLL atthe request of the application. The SSDT (System Service Dispatch Table) and SSPT(System Service Parameter Table) help the kernel in accessing the right system ser-vice ID. The implementation of the SSDT and SSPT occurs similarly in all versionsof Windows NT to date. We present the two implementations separately for clarity,one for Windows NT 3.51 and one for the later versions of the operating systemsuch as Windows NT 4.0 and Windows 2000.

Below is the table containing the service ID mappings for all versions ofWindows NT to date.

TABLE 7-1 SERVICE ID MAPPINGS

Windows NT 3.51

Windows NT 4.0(up to Service Pack 5)

KERNEL32 and ADVAPI32

Mapped to 0x0 through OxC3service IDs inside NTOSKRNL

*

Mapped to 0x0 through OxD2service IDs inside NTOSKRNL

USER32 and GDI32Calls

Processed by the Win32subsystem - a user modeprocess. No systemservices are provided inthe kernel for handlingthese directly. These callsuse the Win32 subsystemusing kernel's LPC systemservices.

Mapped to 0x1000through 0x1 20A serviceIDs in the insideWIN32K.SYS. The kernelmode driver WIN32K.SYStakes over thefunctionality of theWin32 subsystem andsupports these services.

Chapter 7: Adding New System Services to the Windows NT Kernel 125

In Windows NT 3.51, only the KERNEL32 and ADVAPI32 functions of the oper-ating system route through NTDLL.DLL to NTOSKRNL. The USER32 and ODD 2functions of the operating system implement as a part of the Win32 subsystemprocess (CSRSS). The USER32.DLL and GDI32.DLL provide wrappers, which callsthe CSRSS process using the local procedure call (LPC) facility.

The functionality of USER32.DLL and GDI32.DLL is implemented differently inWindows NT 4.0 and Windows 2000. The functionality of the USER32 and GDI32components is moved into the kernel mode driver WIN32K.SYS. The workhorseroutines of NT 3.51's Win32 subsystem have transferred their load on the systemservices added by the addition of the WIN32K.SYS component. This explains whywe see more system services versions later to Windows NT 3.51. This new set ofsystem services corresponds to the USER32 and GDI32 components of the operatingsystem.

Figure 7-1: System service tables

^

Windows NT 2000 (beta-2)

KERNEL32 and ADVAPI32

0x0 through OxF3 service IDsinside NTOSKRNL

USER32 and GDI32Calls

Mapped to 0x1000through 0x1285 serviceIDs in the insideWIN32K.SYS. The kernelmode driver WIN32K.SYStakes over thefunctionality of theWin32 subsystem andsupports these services.


Windows NT System Service ImplementationHere, we discuss the implementation of a system service under Windows NT. AnINT 2Eh instruction implements the system services. The INT 2Eh handler is inter-nally named as KiSystemService and hereafter we refer to it as the handler. Beforeentering the handler, the EAX register is loaded with the service ID and the EDXregister with a pointer to the stack frame required for implementation of a particu-lar service. The handler gets to the current TEB (Thread Environment Block) bylooking at the Processor Control Region (PCR). The current TEB is stored at an off-set of 0x124 in the Processor Control Region. The handler gets the address of theSystem Service Descriptor Table from the TEB. You can locate the address of theService Descriptor Table at 0x124 offset in the TEB. Chapter 6 explains the formatof the Service Descriptor Table.

The handler refers to the first entry in the Service Descriptor Table for service IDsless than 0x1000 and refers to the second entry of the table for service IDs greaterthan or equal to Ox 1000. The handler checks the validity of service IDs. If a serviceID is valid, the handler extracts the addresses of the SSDT and SSPT. The handlercopies the number of bytes (equal to the total number of bytes of the parameter list)described by the SSPT for the service-from user-mode stack to kernel-modestack - and then calls the function pointed to by the SSDT for that service.

Initially, when any thread is started, the TEB contains a pointer to the ServiceDescriptor Table - identified internally as KeServiceDescriptorTable. KeServiceDescriptorTable contains four entries. Only the first entry in this table is used,which describes the service ids for some of the KERNEL32 and ADVAPI32 calls.Another Service Descriptor Table, internally named KeServiceDescriptorTableShadow, identically matches KeServiceDescriptorTable under NT 3.51.However, under later versions of the operating system, the second entry in the tableis not NULL. The second entry points to another SSDT and SSPT. This SSDT andSSPT comprise part of the WIN32K.SYS driver. The WIN32K.SYS driver creates thisentry during its initialization (in its DriverEntry routine) by calling the functioncalled KeAddSystemServiceTable. (We provide more information on this later inthis chapter.) This second entry describes the services exported by WIN32K.SYS forUSER32 and GDI32 modules.

You should note that in all versions of Windows NT, KeServiceDescriptorTablecontain only one entry and that all started threads point their TEBs toKeServiceDescriptorTable. This continues so long as the threads call services be-longing to first entry in KeServiceDescriptorTable. When the threads call servicesabove these limits (unlikely in 3.51, but very likely in later versions of Windows NT,because USER and GDI service IDs start with 0x1000), the KiSystemService jumpsto a label JCiEndUnexpectedRange under NT 3.51 and JCiErrorMode under NT 4.0and KiBBTEndUnexpectedRange in Windows 2000. Let's see what role the code ateach label plays.

____Chapter 7: Adding New System Services to the Windows NT Kernel____127

_KiEndUnexpectedRange (NT 3.51)The following example shows the role of the code at the _KiEndUnexpectedRangelabel:

_KiErrormode (in Windows NT 4.0 andKiBBTEndUnexpectedRange in Windows 2000)The code resembles JCiEndUnexpectedRange, except that the PspW32ProcessCallout variable is always nonzero. Hence, the code in PsConvertToGuiThread pro-ceeds further. It performs several tasks; we now describe the one of immediate in-terest.

PsConvertToGuiThread allocates a block of memory and copies KeServiceDescriptorTableShadow to the allocated block. Note that under NT 4.0 andWindows 2000, KeServiceDescriptorTableShadow contains two entries-one forKERNEL32 calls and one for USER32 and GDI32 calls. After copying this, the codeupdates the TEB of the current thread to point to this copy of KeServiceDescriptorTableShadow and then returns. This happens only the first time a


USER32 or GDI32 service is invoked. After this, all system services, including KER-NEL32 module, route through this new table, since the first entry in this table al-ready points to the SSDT and SSPT for the KERNEL32 functions.

KeServiceDescriptorTableShadow is not exported by the NTOSKRNL and there-fore is a nonaccessible table.

Under Windows NT 3.51, both KeServiceDescriptorTable and the Shadow Tablepoint to the same SSDT and SSPT and contain only one entry. Now, ask yourselfthis logical question: "Why do we have the Shadow Table at all when apparently itdoes not provide much help in NT 3.51?" We attempt to answer this question laterin the chapter.

3 Note that once a process makes a USER32/GDI32 call, it permanently stopsusing the original KeServiceDescriptorTable and switches entirely to a copyof KeServiceDescriptorTableShadow.

Figure 7-2: Adding new system services

Adding New System ServicesAdding new system services involve the following steps:

1. Allocate a block of memory large enough to hold existing SSDT and SSPTand the extensions to each of the table.

2. Copy the existing SSDT and SSPT into this block of memory.


3. Append the new entries to the new copies of the two tables as shown inFigure 7-2.

4. Update KeServiceDescriptorTable and KeServiceDescriptorTableShadow topoint to the newly allocated SSDT and SSPT.

In NT 3.51, because the Shadow Table is never used, you could get away withouthaving to update it. In NT 4.0 and Windows 2000, however, the Shadow Table takesa leading role once a GDI32 or a USER32 call has been made. Therefore, it is im-portant that you update both KeServiceDescriptorTable and KeServiceDescriptorTableShadow. If you fail to update KeServiceDescriptorTableShadow inNT 4.0 or Windows 2000, the newly added services will fail to work once a GDI32or USER32 call is made. We recommend that you update both the tables in all ver-sions of Windows NT so that you can use the same piece of code with all the ver-sions of the operating systems.

One implementation issue in updating the KeServiceDescriptorTableShadow isthat NTOSKRNL does not export this table. However, NTOSKRNL does exportKeServiceDescriptorTable. So, how can you get the address of KeServiceDescriptorTableShadow?

The method we used for this is as follows. There is a function in NTOSKRNLcalled KeAddSystemServiceTable. This function is used by WIN32K.SYS driver foradding the USER32 and GDI 32 related functions. This function does refer toKeServiceDescriptorTableShadow. The first entry in both KeServiceDescriptorTableand KeServiceDescriptorTableShadow is the same. We iterate through each DWORDin the KeAddSystemServiceTable code, and for all valid addresses found in thisfunction, we compare the 16 bytes (size of one entry in descriptor table) at this ad-dress with the first entry in KeServiceDescriptorTable. If we find the match, we con-sider that as the address of the KeServiceDescriptorTableShadow. This methodseems to work in all Windows NT versions.

Example of Adding a New SystemServiceThis example consists of three modules. One device driver contains the code fornew system services and the mechanism of adding new system services to aWindows NT kernel. One DLL represents an interface to new system services (just asNTDLL.DLL provides interface for services called by KERNEL32.DLL). And one ap-plication links to this wrapper DLL and calls the newly added services. The newlyadded services print a debug message saying, "kernel service .... Called" and printthe parameters passed to the services. Each service returns values 0, 1, and 2. Thefunction AddServicesQ isolates the code for the mechanism of adding new systemservices.


Assuming first that the sample binaries are copied in C:\SAMPLES directory,here are the steps to try out the sample:

1. Run "instdrv extndsys c:\samples\extndsys.sys." This will install theextndsys.sys driver. The driver will add three new system services toWindows NT Kernel.

. 2. Run MYAPP.EXE. This will call wrapper functions in MYNTDLL.DLL tocall newly added system services in EXTNDSYS.SYS.

Chapter 7: Adding New System Services to the Windows NT Kernel____133

____Chapter 7: Adding New System Services to the Windows NT Kernel 135


Device Drivers as a Means of Extending the Kernelversus Adding New System ServicesWriting pseudo device drivers and providing the DeviceloControl methods to theapplications can also extend the kernel. However, in this case, each application thatwants to use the DeviceloControl has to open a handle to the device, issue theDeviceloControl, and close the device. Extending the kernel by means of systemservices has its distinct advantages; first and foremost is that applications need notbe aware of the device driver. Applications will just link to a DLL that provides aninterface for the system services (just like NTDLL.DLL provides an interface forKERNEL32.DLL). Further, DeviceloContol proves much slower, especially if theDeviceloControl requires a large amount of data transfers between the applicationand the device driver. By using this technique of adding system services, you mightwrite a set of system services and provide a user-level interface DLL that everybodycan use. This implementation looks cleaner and more standardized than theDeviceloControl method.

Ke Ad d System Servi ceTabl eThe WIN32K.SYS driver calls this function during its DriverEntry under WindowsNT 4.0 and Windows 2000. This function looks somehow odd. The function expectsfive parameters: an index in the Service Descriptor Table where this new entry is tobe added, SSDT, SSPT, the number of services, and one parameter for use only inchecked build versions. This last parameter points to a DWORD Table that holds thevalue of the number of times each service gets called.

NT 3.51 Design versus NT 4.0 and Windows 2000Design: Microsoft's OptionsYou might find it interesting to discover that the code manipulating theKeServiceDescriptorTableShadow resides in all versions of Windows NT - the onlydifference is that the code for allocating and copying the Shadow Table is not trig-gered under NT 3.51 based on the value of a PspW2ProcessCallout variable. This in-formation might convince you that the relocation of USER32 and GDI32component into the NT 4.0 and Windows 2000 kernel (as contrasted with the NT3.51 kernel) is not only performance based-as Microsoft claims now-but some-thing well thought out as an option when NT 3.51 was designed. This leads us tobelieve that Microsoft had two solutions implemented for USER32 and GDI32 mod-ules-the LPC-based solution of using the Win32 subsystem and the INT 2Eh-basedsystem service solution. Microsoft attempted the first solution under NT 3.51 andnow settles for the second solution in later versions of Windows NT. The partialcode for both solutions exists in NT 3.51, but there is no trace of the LPC solutionfor the Win32 subsystem under versions later than NT 3.51. So, we can also


conclude that the future releases of NT, unless drastically different, will continue touse the INT 2Eh-based solution for WIN32K.SYS system services.

SummaryIn this chapter, we discussed in detail the system service implementation ofWindows NT. We explored some code fragments from a system service interrupthandler, using KiSystemServicefJ as an example. Next, we detailed the mechanismfor adding new system services to the Windows NT kernel. We also used an exam-ple that adds three new system services to the Windows NT kernel. We comparedextending the kernel with device drivers with extending the kernel by addingsystem services.


Device Drivers as a Means of Extending the Kernelversus Adding New System ServicesWriting pseudo device drivers and providing the DeviceloControl methods to theapplications can also extend the kernel. However, in this case, each application thatwants to use the DeviceloControl has to open a handle to the device, issue theDeviceloControl, and close the device. Extending the kernel by means of systemservices has its distinct advantages; first and foremost is that applications need notbe aware of the device driver. Applications will just link to a DLL that provides aninterface for the system services (just like NTDLL.DLL provides an interface forKERNEL32.DLL). Further, DeviceloContol proves much slower, especially if theDeviceloControl requires a large amount of data transfers between the applicationand the device driver. By using this technique of adding system services, you mightwrite a set of system services and provide a user-level interface DLL that everybodycan use. This implementation looks cleaner and more standardized than theDeviceloControl method.

Ke Ad d System Servi ceTabl eThe WIN32K.SYS driver calls this function during its DriverEntry under WindowsNT 4.0 and Windows 2000. This function looks somehow odd. The function expectsfive parameters: an index in the Service Descriptor Table where this new entry is tobe added, SSDT, SSPT, the number of services, and one parameter for use only inchecked build versions. This last parameter points to a DWORD Table that holds thevalue of the number of times each service gets called.

NT 3.51 Design versus NT 4.0 and Windows 2000Design: Microsoft's OptionsYou might find it interesting to discover that the code manipulating theKeServiceDescriptorTableShadow resides in all versions of Windows NT - the onlydifference is that the code for allocating and copying the Shadow Table is not trig-gered under NT 3.51 based on the value of a PspW2ProcessCallout variable. This in-formation might convince you that the relocation of USER32 and GDI32component into the NT 4.0 and Windows 2000 kernel (as contrasted with the NT3.51 kernel) is not only performance based-as Microsoft claims now-but some-thing well thought out as an option when NT 3.51 was designed. This leads us tobelieve that Microsoft had two solutions implemented for USER32 and GDI32 mod-ules-the LPC-based solution of using the Win32 subsystem and the INT 2Eh-basedsystem service solution. Microsoft attempted the first solution under NT 3.51 andnow settles for the second solution in later versions of Windows NT. The partialcode for both solutions exists in NT 3.51, but there is no trace of the LPC solutionfor the Win32 subsystem under versions later than NT 3.51. So, we can also


conclude that the future releases of NT, unless drastically different, will continue touse the INT 2Eh-based solution for WIN32K.SYS system services.

SummaryIn this chapter, we discussed in detail the system service implementation ofWindows NT. We explored some code fragments from a system service interrupthandler, using KiSystemServicefJ as an example. Next, we detailed the mechanismfor adding new system services to the Windows NT kernel. We also used an exam-ple that adds three new system services to the Windows NT kernel. We comparedextending the kernel with device drivers with extending the kernel by addingsystem services.

Chapter 8

Local Procedure CallIN THIS CHAPTER - .

+ The origin of subsystems

+ Short message and shared section communication

+ Port-related functions

+ LPC sample programs

+ Quick LPC

MICROSOFT DESIGNED THE local procedure call (LPC) facility to enable efficient com-munication with what Windows NT calls the subsystems. Although you do not needto know about subsystems before understanding the LPC mechanism, it is certainlyinteresting and advisable. In this chapter, we discuss the subsystems and then shedsome light on the undocumented LPC mechanism.

The Origin of the SubsystemsAlthough Microsoft never stated what "NT" stood for, one popular theory suggeststhat it refers to "New Technology." That's not to say everything that goes insideWindows NT is new. Windows NT has borrowed several concepts from earlier oper-ating systems. For example, the NTFS (New Technology File System) borrows a lotfrom the HPFS (High Performance File System) of IBM's OS/2. The Win32 API itselfis an extension of the Windows 16-bit API. The Windows NT 3.51 user interfacecomes from Windows 3.1 and Windows NT 4.0 inherits its interface from Windows95. Windows 2000 (Beta 3) maintains more or less the same user interface asWindows NT 4.0. In this section, we discuss the overall architecture of WindowsNT, which Microsoft borrowed from the MACH operating system, originally devel-oped at Carnegie Mellon University.

DOS and Unix variants dominated the operating systems world in the 1980s.DOS has a monolithic architecture, composed of a single lump of code. Unix fol-lows the layered architecture, where the operating system divides into layers suchthat each layer uses only the interface provided by the lower layers. The MACH op-erating system follows a new client-server approach. The initial versions of MACHwere based on BSD Unix 4.3.

144 Part 11 Undocumented Windows NT

The MACH team focused on two major goals. First, they wanted to have a morestructured code than BSD 4.3. Second, they wanted to support different variants ofthe Unix API. They achieved both these goals by pushing the execution of kernelcode to user-mode processes, which acted as servers. The MACH kernel appearsvery small, providing only the basic system services common to all Unix APIs.Therefore, we call it a micro-kernel. The server processes run in user mode andprovide a sophisticated API interface. The normal application processes are clientsof these server processes. When a client process invokes an API function, the em-ulation library, which links with the client code, transparently passes on the call tothe server process. You can accomplish this using a facility similar to RPC (remoteprocedure call). The server process, after carrying out any necessary processing,returns the results to the client.

To support a new API in the MACH environment, you need to write a serverprocess and emulation library, which support the new API. Not all server processesprovide a different API. Some provide generic functionality such as memory man-agement or TTY management.

The Windows NT design team sought goals similar to that of MACH's developers.They wanted to support Win32, OS/2, and POSIX APIs, while keeping room for fu-ture APIs. Client-server architecture proved a natural choice.

The servers are called as the protected subsystems in Windows NT. Subsystemsare user-mode processes running in a local system security context. We call themprotected subsystems because they are separate processes operating in separate ad-dress spaces and hence are protected from client access/modification. There are twotypes of subsystems:

+ Integral subsystems

+ Environment subsystems•T

Integral Subsystems <An integral subsystem performs some essential operating system task. For WindowsNT, this group includes the Local Security Authority (lsass.exe), the Security AccountsManager, the Session Manager (smss.exe), and the network server. The Local SecurityAuthority (LSA) subsystem manages security access tokens for users. The SecurityAccounts Manager (SAM) subsystem maintains a database of information on user ac-counts, including passwords, any account groups a given user belongs to, the accessrights each user is allowed, and any special privileges a given user has. The SessionManager subsystem starts and keeps track of NT logon sessions and serves as an in-termediary among protected subsystems.

Environment SubsystemsAn environment subsystem is a server that appears to perform operating systemfunctions for its native applications by calling system services. An environmentsubsystem runs in user mode and its interface to end-users emulates another oper-

Chapter 8: Local Procedure Call 145

ating system, such as OS/2 or POSIX- on top of Windows NT. Even the Win32 APIimplements through a subsystem process under Windows NT 3.51.

Not all the API functions in the client-side DLLs need to pass the call to thesubsystem process. For example, most of the KERNEL32.DLL calls can di-rectly map onto the system services provided by the kernel. Such API func-tions invoke the system services via NTDLL.DLL. Most of the USER32.DLLfunctions and GDI32.DLL functions pass on the call to the subsystemprocess. (In Windows NT 4.0, Microsoft moved the Win32 subsystem insidethe kernel for performance reasons.)

The system call interface provided by the Windows NT kernel is called as the na-tive API. The Win32 subsystem uses the native API for implementing the Win32API. Generally, user programs make calls to an API provided by some subsystem,avoiding the use of a cumbersome, native API. We refer to the user programs as theclients of the subsystem that provides the API used by these programs.

The communication between the client processes and the subsystem happensthrough a mechanism called local procedure call (LPC), specially designed byMicrosoft for that purpose. For unknown reasons, Microsoft prefers to keep the LPCinterface undocumented. There is no reason why LPC cannot function as an Inter-Process Communication (IPC) mechanism. Microsoft provides a RFC kit for client-server communication across machines. Windows NT optimizes the RPCs byconverting them to LPCs, in case the client and the server reside on the same ma-chine. However, RFC has its own overheads. LPC proves most efficient in the rawform, and the subsystems also use it in that form only. Apart from that, RPC doesnot provide access to the fastest form of LPC — the Quick LPC. For these reasons, weprovide you with useful information on the LPC interface. " '

Local Procedure CallIn Windows NT, client-subsystem communication happens in a fashion similar tothat in the MACH operating system. Each subsystem contains a client-side DLL thatlinks with the client executable. The DLL contains stub functions for the subsystem'sAPI. Whenever a client process-an application using the subsystem interface —makes an API call, the corresponding stub function in the DLL passes on the call tothe subsystem process. The subsystem process, after the necessary processing, returnsthe results to the client DLL. The stub function in the DLL waits for the subsystem toreturn the results and, in turn, passes the results to the caller. The client process sim-ply resembles calling a normal procedure in its own code. In the case of RPC, the


client actually calls a procedure sitting in some remote server over the network-hence the name remote procedure call. In Windows NT, the server runs on the samemachine; hence the mechanism is called as a local procedure call.

There are three types of LPC. The first type sends small messages up to 304 bytes.The second type sends larger messages. The third type of LPC is called as Quick LPCand used by the Win32 subsystem in Windows NT 3.51. '

The first two types of LPC use port objects for communication. Ports resemble thesockets or named pipes in Unix. A port is a bidirectional communication channel be-tween two processes. However, unlike sockets, the data passed through ports is notstreamed. The ports preserve the message boundaries. Simply put, you can send andreceive messages using ports. The subsystems create ports with well-known names.The client processes that need to invoke services from the subsystems open the cor-responding port using the well-known name. After opening the port, the client cancommunicate, with the server, over the port.

Short Message CommunicationThe client-subsystem communication via a port happens as follows. Theserver/subsystem creates a port using the NtCreatePortQ function. The name of theport is well published and known to the clients (or, rather, to the client-side DLL).The NtCreatePortQ function returns a port handle used by the subsystem to waitand accept requests using the NtListenPortQ function. Any process can send con-nection requests on this port and get a port handle for communication. The subsys-tem receives the request messages, processes them, and sends back the replies overthe port to the client.

The client sends a connection request to a waiting subsystem using theNtConnectPortQ function. When the subsystem receives the connect request, itcomes out of the NtListenPortQ function and accepts the connection using theNtAcceptConnectPortQ function. The NtAcceptConnectPort returns a new porthandle specific to the client requesting the connection. The server can break thecommunication link with the particular client by closing this handle. The sub-system completes the connection protocol using the NtCompleteConnectPortOfunction. Now, the client also returns from the NtConnectPortQ function and getsa handle to the communication port. This handle is private to the client process.The child processes do not inherit the port handles so the children need to openthe subsystem port again.

After completing this connection protocol, the client and the subsystem can startcommunicating over this port. The client sends a request to the subsystem using theNtRequestPortO function. When the NtRequestPort() function sends datagram mes-sages to the subsystem, the client does not receive any acknowledgment for the sentmessages. In case the client expects a reply to its request, the client can use theNtRequestWaitReplyPortO function, which sends the request to the subsystem andwaits for a reply from the subsystem. The subsystem receives request messages usingthe NtReplyWaitReceiveO function and sends reply messages using the NtReplyPortQ

T Chapter 8: Local Procedure Call

Figure 8-1: Steps in communication using the Port object

147

function. The subsystem can optimize by replying to the previous request and wait-ing for the next request using a single call to the NtReplyWaitReceivePortQ function.Figure 8-1 displays this entire process of communication.

A subsystem may receive/reply to messages from more than one client using thesame port. The message contains fields, which identify the client process andthread. The kernel fills in the process ID and the thread ID in the messages.Therefore, the subsystems can rely on this information, and the LPC forms a secureand reliable communication mechanism because the sender of the messages can bereliably identified. - , . • " • • • , • • • -

Shared Section CommunicationYou can send only short messages-up to 304 bytes-via ports. You need to use ashared region of memory for passing larger messages. If clients want to pass messagesvia shared memory, they have to do some extra processing before callingNtConnectPortQ. A client creates a section object of required size, using CreateFileMappingQ - a documented function. The size of the message is restricted only by thesize of the section. The client need not map the section onto the address space; the portconnection procedure takes care of that. But the client has to pass the section handleto the NtConnectPortQ call. The function returns the addresses where the section ismapped in the client's as well as the server's address spaces. Now, whenever the clientwants to invoke the server, it simply copies the parameters to the shared section andsends a message over the port. This message simply acts as an indication of the clientrequest because the actual parameters pass via the shared section.


Generally, as a part of the port message, the client specifies the server spaceaddress of the shared section and the offset of the copied parameters within the sharedsection. If the server uses this information, it should first validate it if the client processproves unreliable. After processing the request, the server also sends back the resultsvia the shared section. Apart from the additional processing, the shared section LPCessentially uses the same set of port APIs as the short message communication. The se-quence of operations also resembles that of the short message communication withone exception — in addition to handling the message port, the client must create theshared section and perform the parameter copying. The sequence of operations shownin Figure 8-1 applies to the shared section LPC as well.

Port-Related FunctionsIn this section, we discuss the port-related functions and parameters passed to themin detail. We prepared sample programs demonstrating short message passing andshared section memory message passing. We discuss these programs next.

NtCreatePortint _stdcal1NtCreatePort(PHANDLE PortHandle, rPOBJECT_ATTRIBUTES ObjectAttributes,DWORD MaxConnectlnfoLength,DWORD MaxDataLength,DWORD Unknown);

This function creates a new port for communication. The name of the port and theparent directory in the object hierarchy pass through the ObjectAttributes parameter.The MaxConnectlnfoLength parameter specifies the maximum size of informationthat can pass on to a connection request. (Later in this section, we discuss the con-nection information.) The MaxDataLength parameter is the maximum size of themessage that can pass through the port. Both these parameters are ignored. The oper-ating system always sets the connection information length to 260 bytes and the datalength to 328 bytes, which are the maximum allowed values for these parameters.Just make sure that you pass values less than the maximum allowed values becausethe function returns an error otherwise. The unknown fifth parameter can pass aszero. A handle to the newly created port returns in PortHandle. The server processuses this port handle to accept connection requests from clients.


The client uses this function to establish LPC communication with the server. Thename of the port to connect to is specified as a Unicode string in the PortName para-meter. The second parameter, unknown at this time, cannot pass as NULL because thefunction fails the validation checks otherwise. The third parameter operates only whenyou use the shared section LPC. It is a pointer to a structure, described as follows:

typedef s t ruct LpcSect ion ln fo {DWORD Length;HANDLE Sec t i onHand le ;DWORD Paraml ;DWORD S e c t i o n S n z e ;DWORD C l l e n t B a s e A d d r e s s ;DWORD S e r v e r B a s e A d d r e s s ;

) L P C S E C T I O N I N F O , * P L P C S E C T I O N I N F O ;

The Length field in this structure specifies the size of the structure; it is alwaysset to 24. The caller of this function — the client — fills the SectionHandle andSectionSize fields, apart from the Length. The CreateFileMappingQ function cancreate a shared section of required size. Upon return from the NtConnectPortO func-tion, the ClientBaseAddress and ServerBaseAddress fields, m the LPCSECTIONINFOstructure, contain the addresses where the section is mapped in the client addressspace and the server address space, respectively.

The next parameter to the NtConnectPortO function - maplnfo - also functionsonly for the shared section LPC. This parameter is a pointer to a structure describedas follows:

typedef struct LpcSec t ionMapIn fo {DWORD Length;DWORD S e c t i o n S n z e ;DWORD S e r v e r B a s e A d d r e s s ;

} LPCSECTIONMAPINFO, *PLPCSECTIONMAPINFO;


This structure duplicates the information in the LPCSECTIONINFO structure. Theclient needs to fill only the Length field, which it always sets to 12 -the size of thestructure. We have not been able to decipher the significance of passing this struc-ture to the NtConnectPortO function. Still, you have to pass a valid structure; if youpass a NULL pointer, the function fails. We have observed that the two members ofthe structure, namely, SectionSize and ServerBaseAddress, zero out on return fromthe function.

We do not know the next parameter sent to the NtConnectPortO function, so setit as NULL.

The client can send some information to the server with the connection re-quest. The server receives this information via the LPC message, which it getsfrom the NtReplyWaitReceivePort() function in case of a connection request. TheConnectlnfo parameter points to this connection information. The size of theconnection information passes through the pConnectlnfoLength parameter that isa pointer to a double word. The server, also, can send back some information tothe client at connection time. This information returns in the same Connectlnfobuffer, and the pConnectlnfoLength is set to indicate the length of the returnedconnection information.

IMtReplyWaitReceivePort

int _s tdca l1N t R e p l y W a i t R e c e i v e P o r t (HANDLE PortHandle,PDWORD Unknown,P L P C M E S S A G E pLpcMessageOut ,PLPCMESSAGE p L p c M e s s a g e l n ) ;

This function is used by the server side of LPC to receive requests from clientsand reply to them. The first parameter is the port handle obtained from theNtCreatePortO function. The second parameter, currently unknown, can be passedas NULL. The third parameter is the message that serves as a reply to the previousclient request. This parameter can be NULL, in which case the function simply ac-cepts a request from the client. The fourth parameter, a pointer to a LpcMessagestructure, fills, on return from the function, with the request information. Both thethird and the fourth parameters are pointers to the LpcMessage structure, which wedisplay here.

typedef struct LpcMessage { 'I* LPC Message Header */WORD Ac tua lMessageLeng th ;WORD T o t a l M e s s a g e L e n g t h ;DWORD MessageType ;DWORD Cl lentProcessId;


DWORD ClientThreadld;DWORD Message ld ;DWORD S h a r e d S e c t i o n S i z e ;BYTE M e s s a g e D a t a [ M A X _ M E S S A G E _ D A T A ] ;} LPCMESSAGE, *PLPCMESSAGE;

The ActualMessageLength field is set to the size of the actual message storedin the MessageData field, whereas the TotalMessageLength is set to the size ofthe entire LpcMessage structure along with the MessageData. The system, not theclient-server, sets the MessageType field. There are several message types. Wedetail the important ones:

I

LPC_REQUEST

LPC_REPLY

LPC_DATAGRAM

LPC_PORT_CLOSED

The server receives this type of message when aclient sends a request using the NtRequestWaitReplyPortQ function. The server should reply tothis message using the NtReplyPortQ functionor the NtReplyWaitReceivePortfJ function. Theserver should not reply to any messages otherthan the LPC_REQUEST messages. The NtRequestWaitReplyPortQ function waits until it gets thereply from the server and then returns the replymessage to the client. Effectively, the clientthread that calls the NtRequestWaitReplyPort()function hangs if the server does not send areply message.The client receives this type of message fromthe NtRequestWaitReplyPortQ function, whenthe server replies to the request.The server receives this type of message when aclient sends a request using the NtRequestPortQfunction. As the name of the message typeimplies, the client does not get a reply fromthe server for this kind of message. If theserver tries to reply to this message usingthe NtReplyPortO function or the NtReplyWaitReceivePort() function, the function fails andreturns an error.The server receives this type of message whena client closes the port handle. If a client dieswithout closing the port handle, the operatingsystem closes the handle on behalf of the client.Thus, the server gets the LPC_PORT_CLOSEDmessage in any case and can use it to free theper-client resources it allocates.


The next fields in the LpcMessage structure are set, by the system, to the client'sprocess ID and thread ID, respectively. The next field is set to the unique message IDgenerated by the system. The server can rely on these fields because the operatingsystem, not the client, sets them. These fields do not make sense in the messages re-ceived by the client and therefore are set to zero in the messages returned by theNtRequestWaitReplyPortO function.

Only the shared section LPC uses the SharedSectionSize field. The system setsthis field to the size of the shared section when it passes a LPC_CONNECTION_RE-QUEST type of message to the server.

The last field is the actual message and is a variable length field. The client-servercan choose to allocate only enough memory space to hold the structure parametersand the actual message. When passing a pointer to this structure for receiving amessage, you must allocate enough memory space to fit the message the process cansend at the other end of the port. If you fail to do it, you will receive an "InvalidAccess" or similar kind of fault. To be on the safer side, you should always allocatefor the maximum-sized message while passing a pointer for receiving a message.

1M t AcceptConn ectPortint _s tdca l1NtAcceptConnectPor t (PHANDLE PortHandle,DWORD Unknownl, .,P L P C M E S S A G E pLpcMessage ,DWORD accept l t ,DWORD Unknowns,P L P C S E C T I O N M A P I N F O map ln fo ) ;

Whenever the server receives a connection request, it follows a connection es-tablishment procedure by first calling the NtAcceptConnectPortO function and thenthe NtCompleteConnectPortO function. This sequence of operations establishes acommunication channel between the client and the server. The client end of thechannel represents the handle that it gets from the NtConnectPort() function. Thefirst parameter to NtAcceptConnectPortO is a port handle pointer set to anotherhandle to the message port on return. This handle is the server-side end of the com-

LPC_CLIENT_DIED

LPC_CONNECTION_REQUEST

The server receives this type of message whena client dies. Refer to the description of theNtRegisterThreadTerminatePortO function formore information.

The corresponding server receives this typeof message when a client tries to connect toa port using the NtConnectPortQ function.


munication channel, although the server can use the handle returned from theNtCreatePortO function to accept requests from all clients. The server can close thehandle, returned by the NtAcceptConnectPortQ function, when it no longer wantsto accept requests using the particular communication channel. Any further re-quests by the client on a closed communication channel will fail.

We have not been able to decipher the second parameter—generally set to zero.The third parameter is the LPC message returned to the client as the connection in-formation from the server. The fourth parameter, named accepth, is passed as 0 if theserver cannot accept the connection request. The server passes accepth as a nonzerovalue if it can accept the connection request. The fifth parameter, not deciphered yet,can be set to zero. The last parameter is a pointer to the LpcSectionMapInfo structure,which fills with appropriate data upon return. We already explained the members ofthis structure. This structure supplies shared-section information for future use by theserver for communicating with the client.

NtCompleteConnectPorti nt __s tdca l 1NtCompleteConnectPor tCHANDLE P o r t H a n d l e ) ;

The server finishes the connection procedure with the NtCompleteConnectPortOfunction. The only parameter to this function is the port handle returned by theprevious call to the NtAcceptConnectPort() function. The client waits in the NtCon-nectPort() function until the server completes the connection procedure by callingthe NtCompleteConnectPortO function.

NtRequestWaitReplyPortint _s tdca l1N tReques tWa i tRep l yPo r t (HANDLE PortHandle,P L P C M E S S A G E pLpcMessageln,PLPCMESSAGE p L p c M e s s a g e O u t ) ;

The client uses this function to send a request and wait for a reply to/from theserver. The first parameter is the port handle obtained via a previous call to theNtConnectPortQ function. The pLpcMessageln parameter is a pointer to a LPC re-quest message sent to the server. The last parameter is a pointer to another LPCmessage structure that fills with the reply message from the server, on return fromthe function.


NtListenPort * ; •1 ~ ^ "Fint _stdca l 1 "* '"*NtL is tenPor t ( ' •HANDLE Por tHand le ,PLPCMESSAGE p L p c M e s s a g e ) ; •

This very small function internally uses the NtReplyWaitReceivePortO function.Here we present the pseudocode of this function:

As you can see from this pseudocode, the NtListenPortO function ignores all mes-sages except connection requests. You cannot use this function if servicing multipleclients. While servicing multiple clients, a server gets a mix of connection requests <and other client requests. The server needs to sort out the connection requests fromthe other requests and perform appropriate processing. If only a single client canconnect at a time, the server can get the connection request using the NtListenPortOfunction and then start a loop to accept and process other client requests.

NtRequestPort .> , !

int _stdca l 1 . .NtRequestPort(HANDLE PortHandle, - - -T- -> ^PLPCMESSAGE pLpcMessage);


This function just sends a message on the port and returns. The server threadwaiting on this port gets the message and does the required processing. The serverthread need not return the results to the caller. In this case, the message type in theheader is LPC_DATAGRAM. A message sent using this function resembles a data-gram in the sense that the sender does not receive an acknowledgment.

NtReplyPortint _s tdca l1NtReplyPor t (HANDLE Por tHand le ,PLPCMESSAGE pLpcMessage ) ;

The server uses this function if it wants to send a reply to the client and does notwant to be blocked for the next request from the client. The first parameter to thisfunction is the port handle, and the second parameter is the reply message sent tothe client.

NtRegisterThreadTerminatePortint _s tdca l1NtReg is te rThreadTermina tePor t (HANDLE P o r t H a n d l e ) ;

If a client calls this function after connecting to a port, then the operating sys-tem sends the LPC_CLIENT_DIED message to the server when the client dies. Evenif the client closes the port handle and keeps running, the system maintains a refer-ence to the port. Therefore, the operating system sends the LPC_PORT_CLOSEDmessage after the LPC_CLIENT_DIED message and not after the client closes theport handle.

NtSetDefaultrlardErrorPort ;_,int ^stdcal 1NtSe tDe fau l tHardEr ro r Port(HANDLE P o r t H a n d l e ) ;

The CSRSS subsystem calls this function during its initialization. TheNtRaiseHardErrorfJ function, called in case of serious system errors, sends a mes-sage to the registered hard error port. Hence, the CSRSS subsystem can pop up themessage when application startup problems appear. The kernel houses only one setof global variables. These variables store the pointer to the hard error port so onlyone process can capture system errors. On Windows NT, this happens to be theWin32 subsystem. Calling this function requires special privilege.

A\W ,i\\V


LPC Sample ProgramsIn this section, we present two sample programs. The first program demonstratesthe short message communication using LPC, and the second program demon-strates the communication using shared memory.

ON THE <D The sample program can be found in the PORT.C file on the accompanyingCD-ROM. The data prototypes and structure definitions for port-relatedfunctions can be found in UNDOCNT.H, which is also on the CD-ROM.

NtlmpersonateClientOfPorti nt _stdcal 1NtImpersonated lentOfPor t (HANDLE Por tHand le .PLPCMESSAGE p L p c M e s s a g e ) ;

A subsystem may need to perform some processing in the security context of tcalling thread. The NtImpersonateClientOfPort() function enables the server threadassume the security context of the client thread. The function uses the pLpcMessaparameter to identify the process ID and thread ID of the client thread.


Short Message LPC SampleThe PORT.C file contains the program that acts as both the client and the server fordemonstrating short message communication. When the program is invoked withoutany parameters, it acts as the server. If invoked with some parameter, it acts as aclient (the parameter is a dummy parameter and gets ignored). You should start theprogram in server mode first. The server-mode program first creates a port and thenloops into a "receive request - process request - reply request" sequence. It uses theNtReplyWaitReceivePortO function to accept requests. The connection requests aretreated differently than other requests. In case of a connection request, the serverthread has to accept the connection and complete the connection sequence. For re-quests, other than the connection request, the server prints the message, inverts allthe bytes in the message, and sends this inverted message back as the reply.

Once the server is ready to accept connections, you can run another instance ofthe program - this time in client mode. The client-mode program connects to theport created by the server-mode instance. It first demonstrates the use of theNtRequestPortQ function to send a datagram. Then, the client sends a request andwaits for a reply in a loop. You can start multiple client sessions; the server portionof the program can handle multiple client requests.

We list and explain the PORT.C file in this section. - • , -

Listing 8-1: PORT.C/***************************************************/

/* Demonstrates the short message LPC provided by the* port object*/

#inc lude < w i n d o w s . h >#include <std io.h>

#iinclude "undocnt .h"#include "pr int.h"

#define PORTNAME L " \ \ W i n d o w s \ \ M y P o r t "

Apart from regular header inclusions, the initial portion of the PORT.C file hasthe definition of the name of the message port used by the sample program. It is acomplete path name starting from the root of the object directory. Note that thewide character set is used instead of the normal ASCII character set because we aredirectly invoking the system services and the system services understand only theUnicode character set.

/* A real server funct ion would do some meaningful* p rocess ing here. As we are wr i t ing just a samp le


This is a dummy processing function on the server side. This function is passedthe LPC request message, received by the server. The function should return the re-ply message in the same memory space. As the comment says, the function simplyinverts all the bytes in the message. Because we only want to demonstrate theworking of the LPC, we do not provide any intricate server functionality. You canmodify this function to implement the functionality provided by your server.


The server part of the program calls this function when it receives a connection re-quest from the client. This function receives the message containing the connectionrequest and returns the port handle specific to the client. The function first prints themessage then calls the ProcessMessageDataQ function. As described earlier, the mes-sage data in a connection request consists of nothing but the Connectlnfo passed tothe NtConnectPortO function by the client.

The ProcessConnectionRequest() function starts the real work by calling theNtAcceptConnectPortO function. The only parameter of significance to this functionis the message containing the connection request. The function returns a handle tothe port in the AcceptPortHandle parameter. This function returns a nonzero value ifit fails. If the function succeeds, the ProcessConnectionRequestQ function calls theNtCompleteConnectPortQ function, which accepts the port handle returned by theNtAcceptConnectPortO function as the parameter. The NtCompleteConnectPortQfunction also returns a zero on success and a value other than zero on failure.

In this function, we accept all the connection requests. You may want to modifythis function to selectively accept connection requests. For example, you mightpermit the connection only for certain users or only if the client provides certainconnection information. If your server can accept only a single client at a time, youneed to reject all further connection requests. As described earlier, you can rejectconnection requests by passing the acceptlt parameter as zero.


In this program, we chose to use two function calls to reply to a message and re-ceive the next message, instead of using a single call to the NtReplyWaitReceiveQfunction. The ProcessLpcRequestQ function, a small utility function, prints the receivedmessage, processes it (inverts the bytes by calling the ProcessMessageDataQ function),and sends back the processed data as the reply using the NtReplyPort message.


As described earlier, the same LPC demonstration program acts as the server andthe client. The mainfj function calls the serverQ function when the program is in-voked without any parameters. The server() function is passed a pointer to the OB-JECT_ATTRIBUTES structure that contains the object name of the communicationport. The function creates a port with this name, upon which it gets back a handle tothe port. As described earlier, the MaxConnectlnfoLength and MaxDataLength para-meters to the NtCreatePortQ function are ignored so we simply pass them as zero.The NtCreatePortQ function returns a zero on success and a nonzero value on failure.

After successful creation of the port, the serverQ function goes into a receive-process-reply loop. The function uses the NtReplyWaitReceivePortQ function to re-ceive requests from clients. Since we use this function only to receive requests, the


pLpcMessageOut parameter passes as NULL. The NtReplyWaitReceivePortQ functionreturns zero on success, and the pLpcMessageln contains the client request. This re-quest can take the form of a LPC_CONNECTION_REQUEST, a LPC_DATAGRAM, aLPC_REQUEST, and so on. The server processes each type of requests differently. Itprocesses the LPC_CONNECTION_REQUEST by performing the connection protocol.It accomplishes this by calling the ProcessConnectionRequestfJ function. With aLPC_REQUEST message, the server needs to do the requested processing and replyto the request. Since we are not implementing any significant functionality in theserver, we just print the message, invert the message bytes, and return a reply. Wedo this in the ProcessLpcRequestO function. For LPC_DATAGRAM messages, a replyis not expected. These messages and all other messages, including LPC_PORT_CLOSED and LPC_CLIENT_DIED, are handled in the default case of the switch state-ment. A real server may need to perform different processing for these messages.For example, a real server might free up per-client resources on receiving aLPC_PORT_CLOSED message.

The server side of the program continuously loops, receiving-processing-reply-ing the client requests. We did not program an exit for the server part. This is gen-erally the case with servers, and that's the reason why they are called daemons inUnix terminology. Generally, servers start up with the system boot and continueprocessing client requests until the system shuts down. With our server, you can killit by pressing Ctrl+C in the command window or by using the Task Manager.


The client() function implements the client-side portion of the LPC sample. Thefunction prints the process ID and the thread ID; you can match it with the processID and thread ID printed from the messages received by the server.

The clientQ function starts its job by connecting to the port created by the serverprocess. It passes six double words as the connectlnfo. You can verify that theserver receives these words as the message data with the LPC_CONNECTION_RE-QUEST. Upon return from the NtConnectPortO function, the client gets a handle tothe port. Also, the connectlnfo buffer fills with the data message passed to theNtAcceptConnectPortO function by the server.

Further, the client calls the NtRegisterThreadTerminatePortO function, with thenewly acquired port handle as the parameter, so that the operating system sends aLPC_CLIENT_DIED message over the port when the client terminates. The clientcalls this function only if the server needs to know about the client death. We callthis function here to demonstrate the mechanism.

The client also demonstrates the datagram communication via the LPC. As de-scribed earlier, the NtRequestPortQ function passes LPC_DATAGRAM type requests.Note that the client fills in only the message length fields and the actual messagedata; the operating system fills in the remaining fields in the LPCMESSAGE struc-ture before the message passes to the server. The clientQ function sends two doublewords as the message data, which the server prints upon reception of the message.


After demonstrating the datagram communication, the client goes in a "send re-quest - wait for reply" loop. Every time, before sending the request, it asks the userwhether to continue or quit. If the user wants to continue with the demonstration,the client sends a sample request over the port using the NtRequestWaitReplyQfunction. The message data consists of two double words inverted by the server andsent back as the reply. The NtRequestWaitReplyO function returns to the client afterit gets the reply message from the server. In this program, we used the same bufferto pass the request message and to receive the reply message. You can use differentbuffers for this purpose.

The main() function simply represents the control function that calls either theserver part or the client part depending on whether the user specifies a parameter.Before passing on the control to one of these functions, the main() function initial-izes a UNICODE_STRING and an OBJECT_ATTRIBUTES structure with the portname. These pass as parameters to the serverfj and client() functions.


Apart from the PORT.C file, the sample program contains a PRINT.C file and aPRINT.H file. The PRINT.C file contains utility routines to print the LPCMESSAGEstructure, and the PRINT.H file contains the prototypes for these functions.

Shared Section LPC SampleThe following program demonstrates the LPC using shared memory. The programresembles the one demonstrating short message LPC, except that it uses a sharedmemory to pass parameters and get results to/from the server. This sample programuses the same PORT.H file, used by the short message LPC sample. The SSLPC.C filein this sample program replaces the PORT.C file from the earlier sample program.The SSLPC.C file contains the server code as well as the client code.

Similar to the short message LPC sample, the same program works as the server aswell as the client depending on whether a parameter is passed while invoking theprogram. You should start the program in the server mode first and when the serveris ready, start the same program in client mode from another command window. Theclient creates a shared section for passing parameters and receiving results. The clientthen establishes communication with the server and asks for a string sent to theserver as the parameter. The client copies the string to the shared section and sends amessage to the server. Upon receiving the message, the server reverses the string inthe shared section and sends a reply. The client prints the reversed string after receiv-ing the reply. The server permits you to start multiple client sessions simultaneously.

This initial portion of the file contains, apart from the required include directives,a couple of important definitions. The client creates the section and therefore deter-mines the size of the shared section. The server is intimated about the size of the sec-tion at the time of connection. The operating system sets the SharedSectionSizefield, in the LPC message, to the size of the shared section when it passes aLPC_CONNECTION_REQUEST message to the server. The server might choose to re-ject the connection request if it disagrees with the section size chosen by the client.For example, the section size might prove too small for the replies from the server.


The section size definition is followed by the definition for the message that theclient sends over the port when the client wants to invoke some service from theserver. As described earlier, the actual parameters pass via the shared section; themessage simply indicates to the server that the client wants to invoke some service.In this sample program, we choose to pass the port message containing the server-side base address of the shared section and the offset of the copied parameterswithin the shared section. The server, in this sample program, does not keep track ofthe shared section information for the connected clients. (Remember that the serveris informed of the details of the shared section when it accepts the connection re-quest via NtAcceptConnectPort().) The server depends solely on the shared-sectioninformation passed by the client with every LPC request. In a nondevelopment envi-ronment, with unreliable clients, the server should either maintain the track of theshared-section information itself or verify the information sent by the client.

/* Extract the message str ing from the shared section* and reverse it*/

vo id P r o c e s s M e s s a g e D a t a ( P L P C M E S S A G E p L p c M e s s a g e )(P S H A R E D L P C M E S S A G E SharedLpcMessage ; • • ' . • ' .char * S e r v e r V i e w ;

Sha redLpcMessage =( P S H A R E D L P C M E S S A G E ) ( p L p c M e s s a g e - > M e s s a g e D a t a ) ; •- ' • -S e r v e r V i e w =( ( c h a r * ) S h a r e d L p c M e s s a g e - > S e r v e r B a s e A d d r e s s ) + :S h a r e d L p c M e s s a g e - > M e s s a g e O f f s e t ; - ' • - . . • - - -s t r r e v ( S e r v e r V i e w ) ; . •

The ProcessMessageDataO function resembles that in the short message commu-nication sample, except that it operates on the shared section instead of the datapassed in the LPC message. As described earlier, the client sends LPC requests, con-taining the server-side base address of the shared section and the offset of thecopied parameters within the shared section. The ProcessMessageDataO function re-trieves this information from the LPC message and calculates the memory address,where the client copied the parameter string. The function reverses this string, andthe client sees the reversed string when the client receives the reply from the server.

BOOLProcessConnecti onRequest( • -•'- ' :' - i:--'-- : :-- : " - • • • ' - -PLPCMESSAGE LpcMessage, .' - > - • > • ' - "" - , - : • ':'• - - - - -PHANDLE pAcceptPortHandle) L ~ < - " :• -- : r -•••-.- . ' - •


The ProcessConnectionRequest() here also resembles the one in the shared sec-tion LPC sample. The only difference between the two functions is in the value theypass for the maplnfo parameter to NtAcceptConnectPort(). If the server passes anon-NULL value for the maplnfo parameter and the client has not sent the sharedsection information with the connection request, the call fails. Therefore, the


ProcessConnectionRequestQ function, in the shared section LPC sample, passesNULL as the maplnfo parameter. Here, the ProcessConnectionRequestQ functionpasses a pointer to the LPCSECITONMAPINFO structure, where it receives the infor-mation about the shared section for use in parameter passing. The sample programdoes not use this information. A real server might keep track of the shared-sectioninformation per client; for example, it can maintain a hash table indexed by theclient thread ID. The server can later retrieve the shared-section information fromthe hash table whenever it receives a LPC request. In this sample program, the clientsends the shared section information, with every LPC request, as a part of the mes-sage sent over the port.


The serverfj function implements the server-side functionality of the sample pro-gram. It starts by creating a port object. After successful creation of the port, thefunction goes in a "receive request - process request - send reply" loop. The servercontinues in the loop until you terminate it by pressing Ctrl+C or with the help ofthe Task Manager.


The server() function receives a new request and replies to the previous requestusing a single call to the NtReplyWaitReceiveQ function. A reply needs to be sentonly if the previous request is of type LPC_REQUEST. Hence, the function calls theNtReplyWaitReceiveQ function with a NULL pLpcMessageOut parameter when it re-ceives the first request or the previous request is not of type LPC_REQUEST.Otherwise, the message received from the client sends as the pLpcMessageOut para-meter. In both cases, upon return from the NtReplyWaitReceiveO function, theLpcMessage structure contains the next request sent by the client.

The server handles only the LPC_REQUEST and LPC_CONNECTION_REQUEST typemessages; other messages are ignored. For LPC_CONNECTION_REQUEST messages,the server establishes a communication channel with the client by calling theProcessConnectionRequestf) function. For LPC_REQUEST messages, the server printsthe message and calls the ProcessMessageDataQ function that reverses the string thatpasses as a parameter in the shared section. The reply to the LPC_REQUEST messageis sent by a call to the NtReplyWaitReceivePortQ function in the next iteration.


The clientQ function, which encompasses the client-side functionality of thesample program, substantially differs from the clientQ function in the short mes-sage LPC sample. This is because a majority of the shared-section handling is per-formed in the client.

The client() function starts creating a shared section by calling the CreateFileMappingQ API function. Note that the section is created with read+write permis-sions. Also note that, the file handle, passed as -1, means an unnamed section notassociated with any file is created. You can create a section by mapping a disk file,but it is not necessary. The function passes the section handle, returned by theCreateFileMappingQ function, to the NtConnectPort() function via the sectionlnfoparameter. The NtConnectPortQ function maps the shared section in the client aswell as the server address space before sending a connection request to the server.The NtConnectPort() function returns after successfully establishing a communica-tion channel with the server. Upon return, the sectionlnfo structure contains theinformation about shared-section mapping. The function also returns the handle tothe LPC port, used by the client for issuing requests.

After a successful connection establishment, the client goes in a "send request -wait for reply" loop. The client asks the user for a string that it sends to the server asthe parameter. (If you enter "quit," the client exits.) The client also inputs the offset,within the shared section. After receiving these inputs, the client copies the givenstring at the specified offset in the shared section. It fills up a LPC message indicat-ing the base address, of the shared section, in the server address space and the offsetof the string within the shared section. The client sends the LPC message to theserver over the port by calling the NtRequestWaitReplyPortQ function. Upon receiv-ing the message, the server reverses the string and sends a reply message. The clientprints the reversed string upon return from the NtRequestWaitReplyPortQ function.

maindnt argc, char **argv){OBJECT_ATTRIBUTES ObjectAttr;UNICODE_STRING uString;int. r c;


Similar to the short message LPC sample, the mainfj function in this sample pro-gram does not have any substantial code. It simply acts as a control function that callseither the serverQ function or the client() function depending on whether the programis invoked with command line parameters. The program also uses the PRINT.H andPRINT.C files for printing the LPC messages.

Quick LPCQuick LPC is the fastest form of LPC. Apart from that, Quick LPC has some pecu-liarities. For one, Quick LPC does not use port objects. Second, Quick LPC serves asthe exclusive medium of communication for the Win32 subsystem. The WindowsNT kernel supports only a single server (per client) using Quick LPC; the Win32subsystem occupies this slot. Therefore, if you want to use Quick LPC, you need tomodify the kernel a bit. (Note that until now, we presented only user-level code inthis chapter.) However, talking about the peculiarities without giving details canmake this concept puzzling. So, here we present details about Quick LPC.

Quick LPC is used only in Windows NT 3.51.

Advantages of Quick LPCLet us first see why Quick LPC is faster than the regular LPC. The LPC communica-tion using the port objects proves slow for a couple of reasons. One, there is a sin-gle server thread waiting on the port object and servicing the requests. This single


server thread is naturally overloaded in anticipation of multiple clients making fre-quent requests. You can overcome this disadvantage by using a fleet of slavethreads. The main server thread gets requests from the port and simply passes themon to one of the slave threads for servicing. The server threads run in parallel withthe main thread and process requests when the main thread receives new requests.

Another problem with the regular LPC is that the context switching between theclient thread and the server thread happens in an "uncontrolled" manner.Typically, a client sends a request on the port and waits for a response from theserver (except while sending datagrams using the NtRequestPortQ function). Whilethe client thread waits on the port for a reply, the thread scheduler searches for themost eligible thread for execution. More often than not, this new thread selectedfor execution differs from the server thread. Essentially, the server thread is notimmediately scheduled when the request comes over the port. Similarly, the clientthread may not be scheduled immediately after the subsystem sends a reply.

Quick LPC overcomes both of the aforementioned disadvantages. The first disad-vantage is overcome by creating a dedicated server thread per client thread. Thesecond disadvantage is overcome by using a kernel object named an event pair,which serves as the backbone of the Quick LPC. As implied by its name, an eventpair consists of a pair of event objects, named high event and low event, respec-tively. The NT kernel provides functions, which allow a thread to wait on one of theevents in the pair and signal the second event in an atomic operation. The eventpair object also guarantees that the thread waiting on the signaled event is the nextthread to be scheduled.

a

Two sets of functions operate on the event pair. One set of functions givesthe regular sleep-wakeup protocol; it does not guarantee immediate threadscheduling:The NtSetHighWaitLowEventPairQ function and the NtSetLowWaitHighEventPairf) function. In this chapter, we discuss the other set offunctions that guarantee the immediate scheduling of the signaled thread. 4|

Quick LPC and Win32 SubsystemTo clarify, let's see how the Win32 subsystem uses the Quick LPC. When a clientthread makes the first GUI call, the Win32 subsystem creates a thread dedicated tothe calling client thread. The new server thread creates an event pair object andcalls the KiSetLowWaitHighThread() function, with the event pair object as a para-meter. The server thread waits for the high event from the pair to get signaled. Now,whenever the client thread makes a GUI call, the KiSetHighWaitLowThread() func-tion is called. This call signals the high event in the pair and en-queues the clientthread in the list of threads waiting for the low event to get signaled. In otherwords, the client thread sleeps while the corresponding server thread, waiting on


the high event, gets woken up. After processing the request, the server thread callsthe KiSetLowWaitHighThreadO function that makes the server thread sleep for thehigh event and the client thread, which was waiting for the low event, takes overthe CPU. This sequence repeats for every GUI call made by the client.

The event pair object takes care of the "controlled" thread switching. It provides nomechanism for passing parameters and return values. The Quick LPC achieves thiswith a dedicated shared section for each client thread. The Win32 subsystem also cre-ates a dedicated section object and maps it in the address space of both the client andthe subsystem processes. The client thread fills in the parameters in the shared areabefore passing the control to the server thread and similarly the server thread copiesthe results in the shared area before returning the control to the client thread.

Naturally, you may think, "Why is the Quick LPC restricted to the Win32 sub-system? Why can't it operate as a general-purpose Inter-Process Communicationmechanism?" The reason is that you cannot call the functions KiSetLowWaitHighThread() and KiSetHighWaitLowThreadO from the user-mode process di-rectly. Windows NT reserves two software interrupts for this purpose. InterruptOx2C calls the function KiSetLowWaitHighThreadO and interrupt Ox2B calls thefunction KiSetHighWaitLowThread().These two interrupt routines operate on a de-fault event pair object. The Thread Environment Block (TEB) maintains a pointer tothis default event pair. You can use the NtSetThreadInformation{) function to setthis pointer. Since only one event pair object can associate with every thread, onlyone server thread can make use of the Quick LPC; that server thread typically be-longs to the Win32 subsystem for most applications. However, non-Win32 applica-tions - or for that matter, non-GUI applications - can still use the Quick LPC forgeneral-purpose communication.

Steps in Quick LPC CommunicationThe server application for a non-GUI program can mimic the Win32 subsystem anduse the Quick LPC for communication. Let's see what the Win32 subsystem doeswhile establishing the Quick LPC.

1. It creates one dedicated thread in the CSRSS process.2. It creates a section object, 64K in size.

3. It maps the view of sections in the client thread and the subsystem.

4. It creates an event pair object.5. It duplicates the event pair object handle in the client process.

6. It duplicates the section object handle in the client process.7. It calls NtSetlnformationThreadO function, with SetEventPairThread as the

information class, for the subsystem thread and for the client thread.


8. It returns information such as duplicated event pair handle, sectionhandle, address in the client process where the shared section is mapped,and so on. - ,

9. After this, the thread data in the client thread reflects that the Quick LPCis established.

10. The dedicated CSRSS thread calls INT 2CH (KiSetiowWaitHighThread).Because of this, the CSRSS thread is blocked until the client sends a request.

11. When the client makes a GUI call, the client fills in the parameters in theshared section and issues INT 2BH (KiSetHighWaitLowThread). Becauseof this, the server thread wakes up, performs the specified task, and fillsin the results in the shared section. Then, the server thread issues theinterrupt Ox2B, which wakes up the client thread. This 2B/2C sequencerepeats until the client thread terminates.

Quick LPC SampleHere, we present a program that mimics the Win32 subsystem and shows how youcan use the Quick LPC for general-purpose communication. The program, only ademonstration, does not implement any service. As described earlier, the event pairobject does not provide any parameter passing mechanism. The user of the event pairobject has to implement parameter passing using shared sections. In this sample pro-gram, we do not demonstrate the use of shared section because it is straightforwardand we already demonstrated it at length in the shared section LPC sample program.In this sample program, we demonstrate only how to implement "controlled" switch-ing between the client thread and the server thread using the event pair object.

Following the usual practice in this chapter, the same sample program acts as theserver or the client depending on whether you pass a command line parameter tothe program. You should first start the program in the client mode. The client printsits own process ID and thread ID. The server needs this information to establish theevent pair object. After you start the program in the server mode, it asks you for theprocess ID and the thread ID of the client. After initializing the thread object, theserver issues INT 2CH then waits for a client request. Meantime, the client waits fora user keystroke. After getting a keystroke from the user, the client issues a INT2BH, which switches the execution thread from the client thread to the serverthread. The server prints a message indicating that it is scheduled and then waitsfor a keystroke. Upon receiving the keystroke, it switches the control back to theclient by triggering INT 2C again. This continues until you kill the server and theclient by pressing Ctrl+C or using the Task Manager.

The implementation, for both client and server, resides in a single file, QLPC.C,which we describe in detail in the next section.


Listing 8-3: QLPC.C

^include < w i n d o w s . h >^include <s td io .h>

#inc lude " . . \ i n c l ude \undocn t . h "

#def ine E V E N T P A I R N A M E L " \ \ M y E v e n t P a i r "

Apart from the usual header inclusions, the initial portion of the QLPC.C file de-fines the name of the event pair used by the sample program to demonstrate "con-trolled" thread switching. We create the event pair at the root of the objectdirectory. If you want to create several objects in the object tree, we suggest youcreate these objects under an application-specific directory.


EventPairHandle,ClientProcessHandle,&C1lent Event PairHandle,0,FALSE,DUPLICATE_SAME_ACCESS):if (re == FALSE) {re = G e t L a s t E r r o r ( ) ;pr in t f ( "Dup l i ca teHand le fa i led, rc=%x\n" , re) ;return 1;

printf("Client EventPair handle = %x\n",CllentEventPairHandle);

re = ZwSetlnformationThread(ClientThreadHandle,8,&EventPairHandle,4);

if (re != 0) {printf("NtSetlnformationThread failed for"the client, rc=%x\n", re);return 1;

w h i l e (1) {DWORD ret_val ;

_asm mt 2Ch_asm mov re t_va l , eax

if (ret_val != 0) (pnntfC'int 2C returned er ror , rc=%x\n" , re t_va l ) ;) else {pr in t f ( " in t 2C re turned\n") ;}g e t c h a r ( ) ;

return 0;


The serverQ function creates a named event pair object. It receives a handle tothe newly created event pair upon successful creation of the object. Next, it estab-lishes an association between the event pair and the server thread-the currentthread. The server uses the ZwSetInformationThread() function to associate theevent pair with the thread. This function is documented in the Windows NT DDK,but you can also call it from a user-mode, nondriver application. The prototype forthis function looks like:

•^•T*

NTSTATUSZ w S e t I n f o r m a t i o n T h r e a d (HANDLE Th readHand le ,THREAD INFOCLASS Thread I n f o r m a t i o n C l a s s ,P V O I D Th read ln fo rma t i on ,ULONG Th read ln fo rma t i onLeng th ) ;

As described earlier, each thread points to the associated event pair object, andthe INT 2BH/INT 2CH issued by a thread operates on the associated event pair ob-ject. The operating system stores the pointer of the associated event pair in theThread Environment Block for the thread, and you can set it using theZwSetlnformationThreadO function. The ThreadlnformationClass for the event pairpointer is 8. The actual information to set is the handle of the event pair object. Wepass 4 as the ThreadlnformationLength parameter because it represents the size of ahandle in Windows NT.

The server needs to associate the event pair with the client thread. But this is notas simple as setting up the association for the current thread. First, the server gets ahold of handles to the client process and the client thread. For this, it needs theclient's process ID and thread ID, which input from the user. The function uses theOpenProcessQ API function to get a handle to the client process.

The server process should have security rights to open the client process.

The function uses an undocumented system call - namely, NtOpenThreadQ - toget a handle to the client thread. The NtOpenThread() system call returns a threadhandle given the process ID and the thread ID. Next, the server duplicates the eventpair handle in the client process's context. It uses the DuplicateHandleQ API func-tion to achieve this. The server needs the process handle to get the duplicate eventpair handle and the thread handle to associate the event pair and the thread. TheZwSetlnformationThreadO function is called again, this time with the client threadhandle, to associate the event pair with the client thread. The function requires theevent pair handle to exist in the context of the process that owns the thread. Thatis the reason we duplicated the handle in the context of the client process.


After setting up the Quick LPC channel, the server can now accept requests fromthe client. It goes into a loop, blocking in the INT 2CH, and indicating it to the userwhenever it gets a request from the client. The server waits for a keystroke and thenissues INT 2CH. This causes the server thread to suspend and the client thread to re-lease for execution. We use inline assembly to issue the software interrupt. Note thatthe interruDt routine, for interruot Ox2C. stores the return value in the EAX register.

The clientQ function proves much simpler in comparison to the server() functionbecause the entire Quick LPC initialization is done by the server. The client just pro-vides the process ID and the thread ID for input to the server. After the initialization iscomplete, the server waits for a client request in INT 2CH. You should indicate the endof initialization to the client by a keystroke. After receiving the keystroke, the client is-sues a INT 2BH, releasing the server thread for execution. Now, the client blocks and isrescheduled only when the server issues INT 2CH. The client waits for a keystroke fromthe user before issuing another INT 2BH.

We use inline assembly to issue the software interrupt. Note that the interruptroutine, for interrupt Ox2B, stores the return value in the EAX register.

main ( in t a rgc , char **argv){

int re; '


if ( a rgc == 1) (re = s e r v e r ( ) ;I e l s e fre = c l i e n t ( ) ;} , -return re;}

n~?The main function in this sample program represents the control center. It calls

the server() function if you invoke the program without any parameters; otherwise,it calls the clientQ function.

Enhancements to the Sample ProgramThe sample program, presented in the previous section, can handle a single client.But the user must supply the client's process ID and thread ID to the server. You canovercome these deficiencies by making use of the port LPC for establishing theQuick LPC. The server can create a port and wait for requests on the port.

Whenever a client starts, it connects to the server over the port and sends a LPCrequest containing its process ID and thread ID. The server, upon receiving the re-quest, initializes an event pair object and creates a new thread to handle the newclient. A shared section also needs to be created and mapped in the server addressspace, as well as the client address space. The server can do it explicitly, or it canuse the shared-section LPC so that the client creates the section and the system it-self takes care of the mapping.

After setting up the communication channel like this, the main server thread sendsa reply message to the client indicating that everything is set up. Now, the mainserver thread can freely accept more connection requests from clients. The newly cre-ated thread waits for the client requests by issuing INT 2CH. After the Quick LPCchannel is established, the client can copy the parameters to the shared area and is-sue INT 2BH whenever it needs to invoke some service from the server.

As a result of the software interrupt, the server thread is scheduled for execution.The server thread reads the parameters from the shared area, processes the request,copies the results to the shared area, and invokes INT 2CH. The software interruptcauses the server thread to sleep, and the client thread is scheduled for execution.This continues until the client thread closes the port handle or dies. Now, the mainserver thread gets a LPC_HANDLE_CLOSED message over the port. Upon receivingthe message, the main thread releases all resources allocated for the client; in otherwords, it destroys the shared-section mapping, kills the thread handling the partic-ular client, destroys the event pair handle, and so on.

The sample program presented in the previous section works for console appli-cations under Windows NT 3.51. The program does not work for GUI applicationsbecause the Win32 subsystem also sets the event pair handle in the ThreadEnvironment Block (TEB), overwriting the event pair handle set by our program.The Win32 subsystem sets the event pair handle in the TEB when the thread makes


the first GUI call. One fact in our favor is that the event pair handle is maintainedper thread. Therefore, you can work around this problem very easily by having aseparate client thread to communicate with the server. The other threads in the ap-plication can consist of GUI threads, accessing the GUI functions offered by theWin32 subsystem and using the Quick LPC to talk to the Win32 subsystem. Youshould take care only that the thread, using the Quick LPC to talk to your ownserver, does not make any GUI calls.

1Our sample program does not work in Windows NT 4.0 because the inter-rupt 0x26 serves a different purpose.

As you know, the Win32 subsystem functionality moves entirely into the ker-nel-mode driver, namely, WIN32K.SYS, in Windows NT 4.0. The Win32 GUIcalls also process as system calls in Windows NT 4.0. Therefore, the Win32subsystem no longer needs the Quick LPC interface, also negating the re-quirement of interrupts Ox2C and Ox2B.

We already saw that the functions KiSetLowWaitHighThreadQ and KiSetHighWaitLowThread() are not directly callable from the user land. Being unable to useinterrupt Ox2B means that a way to access these functions from the user land isblocked. There is another way though. A pair of kernel functions, namely,NtSetLowWaitHighThreadO and NtSetHighWaitLowThreadQ, can perform the samejob. You can get to these functions using a pair of system calls that invoke thesefunctions. These system calls don't accept any parameters since the two functionsoperate on the event pair pointed to by the TEB of the calling thread. Surprisingly,the corresponding functions in the NTDLL.DLL don't invoke these system services.Instead, these functions invoke the interrupts Ox2B and Ox2C.

Surprisingly, the Win32 subsystem, under Windows NT 3.51, does not call theNTDLL.DLL functions. It invokes the interrupts Ox2B and Ox2C directly.Performance seems the most likely reason behind this "bypassing" act. First,the system call interface is bypassed.The overheads of system call setup —that is, indexing the system call ID to find out the number of parameters andthe kernel function to be invoked — might prove unacceptable. Hence, wefind the two functions in question by going out of the way and invoking thespecial interrupts instead of using the normal system call interface interruptOx2E. Of course, this required modifying the kernel to handle the two newsoftware interrupts. We still don't understand why the Win32 subsystem by-passes the NTDLL.DLL functions.

r186 Part 11 Undocumented Windows NT

You cannot use these functions to access the Quick LPC on Windows NT 4.0.Obviously, you need to implement the system call invocation yourself; it's fairlyeasy, though. On Windows NT 4.0, you need to change the INT 2Bh instruction tothe following sequence of instructions that invoke the NtSetHighWaitLowThread()system call:

MOV E A X , AOhLEA E D X , [ESPINT 2Eh

+ 4]*»•»*•

You cannot use INT 2CH, under Windows NT 4.0, even though the interrupt han-dler for it remains there in place. (You would expect both the interrupt handlers tobe extinct if the Win32 subsystem no longer requires them, wouldn't you?) This isbecause the interrupt handler returns a STATUS_NO_EVENT_PAlR error even if theTEB of the calling thread points to a proper event pair. Therefore, you need to use acorresponding system call to achieve the same effect as the KiSetLowWaitHighThreadO function. You can replace the INT 2CH instruction with the followinginstructions that invoke the NtSetLowWaitHighThreadQ system call: _ ,^

MOV E A X , ABhLEA EDX, [ESPINT 2Eh

+ 4]

The system call interface exists and can function even under Windows NT 3.51.You might choose to use the same interface for the two versions of Windows NT sothat the same code works on both versions. OK! It's not so straightforward becausethe service IDs changed from Windows NT 3.51 to Windows NT 4.0. In Windows NT3.51, the service ID for the NtSetLowWaitHighThreadQ system call is OxA3, and theNtSetHighWaitLowThreadQ system call is 0x98.

SummaryA local procedure call (LPC) is the communication mechanism used by Windows NTsubsystems. In this chapter, we gave you a brief introduction to subsystems fol-lowed by a detailed discussion on the undocumented LPC mechanism.

There are three types of LPC. The short message LPC passes small messages up to304 bytes in length. The shared section LPC uses shared memory and passes largermessages. Both the short message LPC and the shared section LPC are based on akernel object called port. The functions to manipulate ports are not documented. Inthis chapter, we documented the parameters and use of these functions withdemonstration programs. , - _ - . „ - - . ,


The Quick LPC, the fastest form of LPC, is used exclusively by the Win32 sub-system. The Quick LPC proves faster because it ensures controlled scheduling of theclient and server thread. In contrast with the other two forms of LPC, the Quick LPCrequires a dedicated server thread per client thread. The Quick LPC mechanism usesanother kernel object-the event pair. The context switches between the clientthread and the corresponding dedicated server thread are optimized using the eventpair object.

Chapter 9

Hooking SoftwareInterrupts

IN THIS CHAPTER

+ How operating systems use software interrupts

+ Why software interrupts need hooking

+ How to hook software interrupts

What Are Interrupts?An interrupt refers to a mechanism that breaks into the normal execution of an ap-plication program and transfers control to operating system code. There are threekinds of interrupts: hardware interrupts, software interrupts, and exceptions.

Hardware interrupts come from the physical devices in the machine. For exam-ple, whenever there is a character waiting on the COM port, a hardware interruptwill be triggered. When an I/O operation completes, a hardware interrupt also willbe triggered.

Software interrupts occur as a result of an explicit INT nn request from the ap-plication. Applications typically use this mechanism to get different services fromthe operating system. Exceptions occur as a result of an application's attempt toperform illegal operations, such as dividing by zero.

The next sections detail how processors handle software interrupts in real, pro-tected, and V86 modes.

Interrupt Processing in Real ModeIn real mode, the lower IK of memory holds a data structure known as the InterruptVector Table (TVT). There are nominally 256 entries in this table. (Since the 80286,the TVT is not required to have 256 entries or start at physical address 0. The baseand address and length of the IVT are determined by looking at the InterruptDescriptor Table Register.) Each entry contains a far pointer to an Interrupt ServiceRoutine. Any type of interrupt routes to the appropriate Interrupt Service Routinethrough this table. The processor indexes the interrupt number in this table; pushescurrent CS, IP, and flags on the stack; and calls the far pointer specified in the FvT. 189


The handler processes the interrupt and then executes an IRET instruction to returncontrol to the place where the processor executed at the time of the interrupt.

Interrupt Processing in Protected ModeIn protected mode, interrupts are handled in a similar way as real mode. TheInterrupt Descriptor Table (IDT) does what the TVT does in real mode. LDT consists ofan array of 8-byte segment descriptors called gates. The Interrupt Descriptor TableRegister (IDTR) holds the base address and the limit of IDT. The IDT must exist inphysical memory and should never swap out to virtual memory. This is because if aninterrupt were to occur while the IDT were swapped out, the processor would gener-ate an exception, requiring the IDT to get the handler for handling this exception,and so on until the system crashed. The gates in the IDT can consist of three types:interrupt gates, trap gates, and task gates. We won't dwell on the details of the trapand task gates. For further information, refer to Intel processor documentation.

Interrupt gates interest us. The important fields of interrupt gates include thecode segment selector and the offset of the code for execution for this interrupt, aswell as the privilege level of the interrupt descriptor. The interrupt processingclosely resembles that in real mode. When the interrupt occurs, the processor in-dexes the interrupt number in IDT, pushes EFLAGS, CS, and EIP onto the stack, andcalls the handler specified in the IDT. When the handler finishes executing, itshould execute the IRET instruction to return control. Depending upon the type ofinterrupt, an error code may be pushed on the stack. The handler must clear this er-ror code from the stack. The DPL field in the interrupt gate controls the software in-terrupts. The current privilege level must be at least as privileged as DPL to callthese software interrupts. If not, then a General Protection Fault is triggered. Thisprotection feature permits the operating system to reserve certain software inter-rupts for its own use. Hardware interrupts and exceptions process without regard tothe current privilege level.

Interrupt Processing in V86 A/lodeIn V86 mode, any INT nn instruction causes a General Protection Fault. WindowsNT uses this to map INT 2In calls made from an MS-DOS application to Win32 APIcalls. This mapping occurs as part of a GPF handler for Windows NT. Other types ofinterrupts are handled similarly to those in protected mode.

How Operating Systems UseSoftware InterruptsMS-DOS uses INT 21 to provide core system services to the applications. Other soft-ware interrupts are also provided, such as multiplex interrupt 2F. Applications fillin the parameters in various registers and execute the INT nn instruction to access

Chapter 9: Hooking Software Interrupts 191

these services from the operating system. Various compiler libraries provide wrap-pers around these interrupt interfaces and provide useful C functions, such as_open, _read, _write, and others.

Not much changes in the way software interrupts are used in Windows 95/98and Windows NT. Windows NT provides user-callable software interrupts. The fol-lowing table lists the important software interrupts provided.

TABLE 9-1 WINDOWS SOFTWARE INTERRUPTS

Why Software Interrupts Need to BeHookedSoftware interrupts need to be hooked for several reasons. One reason is to changethe behavior of the system services exported by the operating system. By hookingthe software interrupts, you can write monitoring applications. Hooking can proveuseful in studying operating system internals. This can also serve as a way to hooksystem services, although the mechanism discussed in Chapter 6 provides a betterway of doing that.

MS-DOS provides system services to hook software interrupts by means of INT21h, and functions 25h and 35h. Compiler libraries provide wrapper functions such

Interrupt Number

2Ah

2Bh, 2Ch

2Dh

2Eh

Functionality

Used to get the current timer tick count.

Used by the CSRSS subsystem to force an immediate thread switch.This occurs as part of a LPC mechanism. We discussed LPC in moredetail in Chapter 8. These interrupts are used only in Windows NT3.51 since in later versions of Windows NT, most of thefunctionality in CSRSS is moved to a kernel-mode driverWIN32K.SYS.

Debugging service. This service, used by driver writers, outputsdebugging messages to the Debugger Window. The DbgPrintQfunction provided in DDK calls this interrupt to output debugmessages.

This interrupt is extensively used for calling system servicesprovided by Windows NT. The system services are provided by twocomponents viz. NTOSKRNL and WIN32K.SYS. The services providedby WIN32K.SYS are present only in Windows NT versions later than3.51. We discuss system services in detail in Chapters 6 and 7.


as _dos_getvect and _dos_setvect to hook software interrupts. Windows 95 pro-vides a mechanism to hook software interrupts by means of Set_PM_Int_Vector andHook_V86_Int_Chain VxD services. However, Windows NT does not officially sup-port any way to hook software interrupts. The DDK does provide functions such asHalGetlnterruptVectorO and IoConnectInterrupt() to hook hardware interrupts. Oncewe understand Intel data structures such as IDT and interrupt gates, we can easilyhook software interrupts in Windows NT. Hooking software interrupts basicallyamounts to changing the code selector and offset fields in the Interrupt GateDescriptor. However, this certainly becomes a platform-dependent situation. It willwork only on an Intel implementation of Windows NT.

You can apply the same technique for hooking software interrupts to hook hard-ware interrupts or exceptions although you should use the documentedloConnectlnterruptO function to hook hardware interrupts. You have to write an inter-rupt handler keeping in mind the type of interrupt it is hooking into because the stackframe might differ in various situations. The new interrupt handler must be written inAssembly language because of the restrictions imposed by 32-bit compilers.

How to Hook Software InterruptsAs we already discussed, the two Intel data structures - IDTR and Interrupt GateDescriptor - play crucial roles in interrupt processing. You can discover the con-tents of IDTR with the sidt Assembly instruction. This instruction places the baseand limit of IDT in a 6-byte location specified by the operand. Once you get thebase address of IDT, you can index the interrupt number you want to hook in thistable and change the code selector and offset specified. Before doing this, you mustsave the old code selector and offset. Also, your new handler should ensure that theinterrupt is chained properly to the old handler, meaning the new handler shouldmaintain the state of registers and stack in such a way that the old handler shouldbe called as if it were directly called by the processor through the IDT.

The sample application that we write in this chapter hooks INT 2Eh (SystemService Interrupt) and maintains the counters of how many times a particular sys-tem service was called. The sample maintains only the counter of system servicesprovided by NTOSKRNL.EXE. The user-level application issues DeviceloControl tothis driver to obtain the statistics about the service usage. As we already saw inChapter 7, there are a total of OxC4 system services in NT 3.51, OxD3 services in NT4.0, and OxF4 services in Windows 2000 provided by NTOSKRNL.EXE. This sampleworks on all versions of Windows NT to date.

H O O K I N T . C# inc lude "n tddk.h"# inc lude " s tda rg .h "^include "s td io .h"/ / include "Hook in t .h "

r198 Part 11: Undocumented Windows NT


RingOEpi logjmp dword ptr cs:[_01dHandler]_ N e w H a n d l e r endpEND

SummaryIn this chapter, we discussed interrupt processing in various modes of Intel proces-sors. Then, we saw how the operating system makes use of interrupts. Next, we dis-cussed the need for hooking software interrupts. We also explored a mechanism forhooking software interrupts. We concluded the chapter with an example that hooksInt 2E (the system service interrupt) in Windows NT.

Chapter 10

Adding New SoftwareInterrupts

IN THIS CHAPTER

+ Understanding what happens when a 32-bit application executes an INTinstruction

+ Adding new software interrupts to the Windows NT kernel

+ Using callgates to execute privileged code

+ How to use callgates

As WE SAW IN THE previous chapter, software interrupts are one of the mechanismsused for calling system services. We have also seen that INT 2E is used for gettingthe system services from the Windows NT kernel. By adding new software inter-rupts, it is possible to add new system services to the Windows NT kernel. We havealready seen one way to add new system services to the Windows NT kernel, andthis is just one more method. In this chapter, we will not be playing with the oper-ating system data structures as we did in Chapter 7. Instead, we will use Intel datastructures to add new system services.

What Happens When a 32-BitApplication Executes an INT nnInstruction?Before we proceed with the technique of adding new software interrupts to theWindows NT kernel, let's first see what happens when a 32-bit application executesan INT nn type of instruction. Application programs run at privilege level 3, andthe kernel code executes at privilege level 0. When a 32-bit application programexecutes an INT nn type of instruction, the processor first looks at the descriptorentry for the interrupt and verifies that the current privilege level is at least as highas the descriptor privilege level. If not, the processor raises a General ProtectionFault. If the privilege level of the descriptor allows the interrupt to continue, the 201


processor switches to the kernel stack. The kernel stack is selected by looking at thefield in the Task State Segment (TSS). After this, the processor pushes the old ring 3stack pointer (SS:ESP) and a standard interrupt frame (EFLAGS and CS:EIP) andjumps to the handler routine specified in the interrupt descriptor table entry. Thehandler performs its job and finally executes the IRETD instruction to return to thecalling application. When IRETD is executed, the processor pops off EFLAGS andCS:EIP, notices the switch from ring 0 to ring 3 and pops off the ring 3 SS:ESP, andthen the execution continues from the instruction following the INT nn instruction.

If you see the descriptor entry for INT 2Eh through a debugger such as SoftlCE,you will notice that its descriptor privilege level is 3. That is why NTDLL.DLL cancall INT 2Eh on behalf of the applications.

Adding New Software Interrupts tothe Windows NT KernelAs you saw in the last chapter, an interrupt gate is installed in the IDT for the soft-ware interrupts. Here is the structure of the interrupt gate:

typedef struct InterruptGate {unsigned short O f f se tLow ;unsigned short Se lec to r ;unsigned char Rese rved ;unsigned char SegmentType:4;unsigned char Sys temSegmentF lag :1 ;unsigned char D p l : 2 ;unsigned char Present :1 ;unsigned short Of fse tH igh;

( InterruptGate_t;

There are a few unused interrupts in Windows NT, including INT 20h and INT22-29h. You can use these interrupts to add new software interrupts. Following arethe steps for adding new software interrupts:

1. Get the base address of the interrupt descriptor table using the assemblyinstruction "sidt." This instruction stores the base address and limit of IDTat the specified memory location.

2. Treat this base address an a pointer to array of "InterruptGate_t"structures.

3. Index the interrupt number to be added into this table.4. Fill in the "InterruptGate_t" entry at the index according to the

requirements of the interrupt gate. That is, set the "SegmentType" field to

Chapter 10: Adding New Software Interrupts 203

OEh meaning interrupt gate; set the "SystemSegmentFlag" to 0 meaningsegment; set the "Selector," "OffsetLow," and "OffsetHigh" fields with theaddress of the interrupt handler. Set the "Present" field to 1.

5. Establish some mechanism for passing parameters to the interrupt serviceroutine. For example, INT 2Eh uses the EDX register to point to the userstack frame and the EAX register for the service ID.

We have already seen mechanisms used by INT 2Eh handler in Chapter 6.

//"include "n tddk .h "/ / • include " s t d a r g . h "//include "s td io.h"// include "addint .h"

/ / include " . . \ i nc lude \ i n te l . h "# inc lude " . . \ i nc1ude \undocn t .h "

/* O ld Idt Entry */IdtEntry_t OldldtEntry;

/* Interrupt Handler */extern vo id _cdecl In ter ruptHandler( ) ;

/* Buf fer to s tore result of sidt instruct ion */char buf fer [6] ;

6. Use the INT nn instructions in your application programs according to theconventions established in the previous step.

The sample application that illustrates this method adds INT 22h to the WindowsNT kernel. The interrupt handler expects that the EDX register points to the buffer,which will be filled by the handler with the "Newly added interrupt called" string.The buffer should be at least 29 bytes long.

Following is the device dnver that adds a new software interrupt to the WindowsNT kernel. The driver adds the interrupt in its DriverEntry routine and removes theinterrupt in its DrvUnload routine. The full source code for the application that is-sues this newly added interrupt is not given. Only the relevant part that issues theinterrupt is given here.

Listing 10-1: ADDINT.C

Part 11: Undocumented Windows NT


Using Callgates to ExecutePrivileged CodeNext, we will discuss one generic method of executing ring 0 instructions from auser-level application running at ring 3 with the help of a device driver. This is anequivalent of RINGO by Matt Pietek, which appeared in the May 1993 edition ofMicrosoft Systems Journal in an article called "Run Privileged Code from YourWindows-based Program Using CallGates." This may be used for performing directport I/O under Windows NT (refer to "Direct Port I/O and Windows NT" by DaleRoberts, Dr. Dobb's Journal of Software Tools, May 1996). The whole trick of run-ning ring 0 instructions at ring 3 is based on the concept of callgates.

Callgates are mechanisms that facilitate controlled and secure communicationfrom a lower privilege level to higher privilege level. Right now we will considerthe control transfer from ring 3 to ring 0 since Windows NT uses only these twoprivilege levels. It is as if you have ring 3 and ring 0 code on two sides of a callgate,


with the callgate acting as an intermediary between the two. The callgate enablesmessages to pass from one ring to the other.

When creating a callgate, you have to specify the address of each side of thefence and the number of parameters to be passed from one side of the fence to theother. The privilege level of the callgate dictates which processes have access to it.When the control is transferred though the callgate, the processor switches to thering 0 stack. This stack is selected by looking at the TSS. The TSS contains the stackfor each privilege level. After this, the processor pushes the ring 3 SS:ESP on thisnew stack. Then the processor copies the number of parameters specified by thecallgate from the ring 3 stack to the ring 0 stack. Parameters are in terms of thenumber of DWORDS for 32-bit callgates and the number of WORDS for a 16-bitcallgate. Further, the processor pushes the ring 3 CS:EIP onto the stack and jumpsto the address specified in the callgate. The function at ring 0 is responsible forcleaning the parameters from the stack once it has finished executing. In the end,the ring 0 code should execute a retf nn instruction to clean up the stack and returncontrol to the ring 3 code.

The sample accompanying this technique is based on the sample programPHYS.EXE demonstrated in Matt Pietrek's Windows 95 Programming Secrets (IDGBooks Worldwide). The sample shows you how you can use the same trick underWindows NT. The sample uses three undocumented functions in NTOSKRNLEXE.These functions enable you to allocate and release selectors from the GlobalDescriptor Table (GOT) and modify the descriptor entries corresponding to the se-lectors. Use of the following undocumented functions prevents the need to directlymanipulate Intel data structures such as the GOT.

NTSTATUSKe I386A l l oca teGd tSe lec to r s (unsigned short *Se lec to rA r ray .int NumberOfSe lec to rs ) ; - " ' -

The function allocates the specified number of selectors from the GDT and fillsin the SelectorArray with the allocated selector values. NTOSKRNL keeps a linkedlist of free selectors in the descriptor itself. Also, NTOSKRNL keeps track of thenumber of free selectors. The function checks whether the specified number of se-lectors is present. If enough selectors are available, the function removes those se-lectors from the free list and gives the list to the caller. Interestingly, these functionsare exported from the NTOSKRNL.EXE file, so any driver can use them. Other func-tions also enable descriptor queries and other tasks, but they are not exported.

NTSTATUSKeI386ReleaseGdtSel ectors (unsigned short *SelectorArray,int NumberOfSelectors);


The function releases the specified number of selectors. The selectors are speci-fied in the array SelectorArray. The function updates the variable that keeps trackof the number of selectors and inserts these selectors in the free list of selectors.

N T S T A T U SKeI386SetGdtSelector(unsigned int Selector, void *);

This function fills in the descriptor corresponding to a particular selector. Thesecond parameter should be a pointer to a descriptor entry.

How to Use the Callgate TechniqueThe following sample shows how you can perform direct-to-port I/O and run priv-ileged instructions from a user-level application with the callgate technique. A de-vice driver is provided that enables the user application to allocate and release thecallgates. The user-level application contains a function that does direct port I/O toget the base memory size and extended memory size from CMOS data. The applica-tion also prints the contents of CPU control registers such as CRO, CR2. The in-structions for accessing these registers are privileged.

The sample comprises three modules:

+ CALLGATE.SYS, which provides the functions for allocating and releasingthe GOT selectors.

+ The user mode DLL called CGATEDLL.DLL, which provides wrappers forcalling the functions in CALLGATE.SYS. This DLL uses DeviceloControl totalk to CALLGATE.SYS.

+ The user mode application CGATEAPP.EXE, which uses wrappers inCGATEDLL.DLL to demonstrate the sample. CGATEAPP.EXE contains thefunction that does direct port I/O and tries to access the processor controlregisters.

The function in CGATEAPP.EXE that runs ring 0 code is written in Assemblylanguage due to the restrictions imposed by the 32-bit compiler. These restrictionsare discussed in Matt Pietrek's Windows 95 Programming Secrets, but we will sum-marize those points again. The function that is called through callgate has to makea far return, whereas a standard 32-bit compiler generates a near return. Also, thefunction gets called as a far call, so the stack frame is not compatible with the onegenerated by a standard 32-bit compiler. The 32-bit compiler generates code insuch a way that it expects the first parameter to be at [EBP+8] once it sets up thestack frame with PUSH EBP, MOV EBP, and ESP. However, because the functiongets called as a far call, the first parameter is present at [EBP+OCh].

f^



Paging IssuesWhile writing the callgate sample, we observed that there are certain issues regard-ing accessing the paged/swapped out data in the interrupt routine and also in thefunction called through callgate. All the existing interrupt handlers such as INT 2Ehwere seen to follow certain entry and exit code before performing any real work.Some of the tasks performed by the entry code were:

1. Creates some space on stack.

2. Prepares a trap frame that will record the state of some of the CPUregisters.

3. Saves away some of the fields in Thread Environment Block such asprocessor mode and one field in TEB, which SoftICE calls as "KSS EBP."We don't know the exact meaning of this, but its seems that eachinterrupt handler should set this field to the trap frame created inprevious step.

4. Saves away the contents of FS register and sets FS register to 0x30.

Out of all these steps, the first step is absolutely necessary and is related to thelogic used by page fault handler of the operating system. The page fault handlerdoes some arithmetic on the current stack pointer and the stack pointer at the timeof ring transition from ring 3 to ring 0 and take some decisions. If at least a specific


amount of stack space is not found between these two stack pointer values, thenthe system crashes with a Blue Screen.

It is essential that you follow this while writing interrupt handlers or functionsexecuted through callgate to successfully access paged out data. The fourth step ofsetting FS register to 0x30 is also necessary since the system expects FS register topoint to Processor Control Region when the thread is executing in ring 0 and theselector 0x30 points to the descriptor with the base address equal to address ofprocessor control region.

Note that you have to follow the same steps while hooking softwareinterrupts. I

The second and third step seems to be only for bookkeeping information.All the samples in this book that use callgates or interrupt handlers use a macro

defined in UNDOCNT.INC file called RingOProlog and RingOEpilog. These macrosimplement the code, which takes care of these paging issues.

SummaryIn this chapter, we detailed how interrupts are executed under Windows NT. Thenwe discussed a mechanism for adding new software interrupts. Along the way, wediscussed some processor data structures used while processing the interrupt andpresented an example that adds a software interrupt (0x22) to Windows NT. Wealso showed an example of an application that calls the newly added interrupt.After that, we discussed callgates, used for running ring 0 code from ring 3. Thiswas followed by an example that demonstrated how to use callgates to read proces-sor control registers such as CRO, CR3 and do direct port I/O from ring 3. The chap-ter concluded with the discussion about the paging issues while executingfunctions through callgates and interrupt handlers.

Chapter 11

Portable ExecutableFile Format

IN THIS CHAPTER

+ Understanding the structure of a PE file

+ Talking in terms of RVAs

+ Detailing the PE format

+ The importance of indices in the data directory

+ How the loader interprets a PE file

MICROSOFT INTRODUCED A NEW executable file format with Windows NT. This for-mat is called the Portable Executable (PE) format because it is supposed to beportable across all 32-bit operating systems by Microsoft. The same PE format exe-cutable can be executed on any version of Windows NT, Windows 95, and Win32s.Also, the same format is used for executables for Windows NT running on proces-sors other than Intel x86, such as MIPS, Alpha, and Power PC. The 32-bit DLLs andWindows NT device drivers also follow the same PE format.

It is helpful to understand the PE file format because PE files are almost identi-cal on disk and in RAM. Learning about the PE format is also helpful for under-standing many operating system concepts. For example, how operating systemloader works to support dynamic linking of DLL functions, the data structures in-volved in dynamic linking such as import table, export table, and so on.

The PE format is not really undocumented. The WINNT.H file has several struc-ture definitions representing the PE format. The Microsoft Developer's Network(MSDN) CD-ROMs contain several descriptions of the PE format. However, thesedescriptions are in bits and pieces, and are by no means complete. In this chapter,we try to give you a comprehensive picture of the PE format.

Microsoft also provides a DLL with the SDK that has utility functions for inter-preting PE files. We also discuss these functions and correlate them with other in-formation about the PE format.

223


Overview of a PE FileIn this section, we discuss the overall structure of a PE file. In the sections that fol-low, we go into detail about the PE format. A PE file comprises various sections.Because Microsoft's 32-bit operating systems follow the flat memory model, an ex-ecutable no longer contains segments. Still, different parts of an executable, such ascode and data, have different characteristics. These different parts of an executableare stored as different sections. Thus, a PE file is a concatenation of data storedin sections.

A few sections are always present in a PE file generated by the Microsoft linker.Other linkers may generate similar sections with different names. A PE file generatedwith the Microsoft linker has a .text section that contains the code bytes concate-nated from all the object files. As for the data, it can be classified into different cat-egories. The .data section contains all the initialized global and static data, while the.bss section contains the uninitialized data. The read-only data, such as string liter-als and constants, is stored in the .rdata section. This section also contains someother read-only structures, such as the debug directory, the Thread Local Storage(TLS) directory, and so on, which we explain later in this chapter. The .edata sectioncontains information about the functions exported from a DLL, while the .idata sec-tion stores information about the functions imported by an executable or a DLL. The.rsrc section contains various resources, such as menus and dialog boxes. The .relocsection stores the information required for relocating the image while loading.

The names of the sections do not have any significance. As mentioned earlier,different linkers may use different names for the sections. Programmers can alsocreate new sections of their own. The #pragma code_seg and #pragma data_segmacros can be used to create new sections while working with Microsoft compiler.The operating system loader locates the required piece of information from the datadirectories present in the file headers. Shortly, we will present an overview of fileheaders and then look at them in more detail.

Structure of a PE FileApart from the sections consisting of the actual data, a PE file contains various head-ers that describe the sections and the important information present in the sections.

If you look at the hex dump of a PE file, the first 2 bytes might look familiar.Aren't they M and Z? Yes, a PE file starts with the DOS executable header. It is fol-lowed by a small program that prints an error message saying that the programcannot be run in DOS mode. It's the same idea that was used in 16-bit Windows ex-ecutables. This program code is executed, if the PE image is run under DOS.

After the DOS header and the DOS executable stub comes the PE header. A fieldin the DOS header points to this new header. The PE header starts with the 4-bytesignature "PE" followed by two nulls. The PE format is based on the Common Object

Chapter 11: Portable Executable Eile Format 225

File Format (COFF) used by Unix. The PE signature is followed by the object fileheader borrowed from COFF. This header is present also for the object files producedby Microsoft's 32-bit compilers. This header contains some general informationabout the file, such as the target machine ID, the number of sections in the file, andso forth. The COFF style header is followed by the optional header. This header is op-tional in the sense that it is not required for the object files. As far as executablesand DLLs are concerned, this header is mandatory. The optional header has twoparts. The first part is inherited from COFF and can be found in all COFF files. Thesecond part is an NT-specific extension of COFF. Apart from other NT-specific infor-mation, such as the subsystem type, this part also contains the data directory. Thedata directory is an array in which each entry points to some important piece of in-formation. One of the entries in the data directory points to the import table of theexecutable or DLL, another entry points to the export table of the DLL, and so on.

We will look at the detailed formats of the different pieces of informationlater in this chapter.

The data directory is followed by the section table. The section table is an array ofsection headers. A section header summarizes the important information about therespective section. Finally, the section table is followed by the sections themselves.

We hope that this gives you an overview of the organization of a PE file. Beforediving into the details of the PE format, let's discuss a concept that is vital in inter-preting a PE file. . . . .

Relative Virtual AddressAll the offsets within a PE file are denoted as Relative Virtual Addresses (RVAs). AnRVA is an offset from the base address at which an executable is loaded in memory.This is not the same as the offset within the file because of the section alignmentrequirements. The PE header specifies the section alignment requirements for anexecutable image. A section has to be loaded at a memory address that is a multi-ple of the section alignment. The section alignment has to be a multiple of the pagesize. This is because different sections have different page attribute requirements;for example, the .data section needs read-write permissions, while the .text sectionneeds read-execute permissions. Hence, a page cannot span section boundaries.

Because the PE format always talks in terms of RVAs, it's difficult to find thelocation of the required information within a file. A common practice whileaccessing a PE file is to map the file in memory using the Win32 memory mappingAPI. It's a bit complicated to calculate the address for the given RVA in this


memory-mapped file. You first need to find out the section in which the given RVAlies. You can accomplish this by iterating through the section table. Each sectionheader stores the starting RVA for the section and the size of the section. A sectionis guaranteed to be contiguously loaded in memory. Hence, the offset from the startof the section for a particular piece of data is bound to be the same whether the fileis memory mapped or loaded by the operating system loader for execution. Hence,to find out the address in a memory-mapped file, you simply need to add this off-set to the base address of the section in the memory-mapped file. Now, this base ad-dress can be calculated from within the file offset of the section, which is alsostored in the respective section header. Quite an easy procedure, isn't it?

ImageRvaToVaQDon't worry, there is an easier way out. Microsoft comes to our rescue here withIMAGEHLP.DLL. This DLL exports a function that computes the address in thememory-mapped file, given an RVA.

L P V O I D I m a g e R v a T o V a (PIMAGE_NT_HEADERS NtHeaders ,LPVOID B a s e ,DWORD R v a ,PIMAGE_SECTION_HEADER * L a s t R v a S e c t i o n); " '

PARAMETERS

NtHeaders

Base

RvaLastRvaSection

>,

* f

Pointer to an IMAGE_NT_HEADERS structure. This structurerepresents the PE header and is defined in the WINNT.h file. Apointer to the PE header within a PE file can be obtainedusing the ImageNtHeaderQ function exported by < •IMAGEHLP.DLL.

Base address where the PE file is mapped into memory usingthe Win32 API for the memory mapping of files.

Given relative virtual address.

Last RVA section. This is an optional parameter, and you canpass NULL. When specified, it points to a variable thatcontains the last section value used for the specified image totranslate an RVA to a VA. This is used for optimizing thesection search, in case the given RVA also falls within thesame section as the one for the previous call to the function.The LastRVASection is checked first, and the regularsequential search for the section is carried out only if thegiven RVA does not fall within the LastRVASection.

Chapter 11: Portable Executable File Format 227

RETURN VALUESIf the function succeeds, the return value is the virtual address in the mapped file;otherwise, it is NULL. The error number can be retrieved using the GetLastErrorQfunction.

ImagelMtHeaderQThe ImageRvaToVaQ function needs a pointer to the PE header. The ImageNtHeaderexported from the IMAGEHLP.DLL can provide you this pointer.

PIMAGE_NT_HEADERS ImageNtHeader(L P V O I D ImageBase) ;

PARAMETERS

ImageBase Base address where the PE file is mapped into memory using theWin32 API for the memory mapping of files.

RETURN VALUESIf the function succeeds, the return value is a pointer to the IMAGE_NT_HEADERSstructure within the mapped file; otherwise, it returns NULL.

MapAndLoadQThe IMAGEHLP.DLL can also take care of memory mapping a PE file for you. TheMapAndLoadQ function maps the requested PE file in memory and fills in theLOADED_IMAGE structure with some useful information about the mapped file.

BOOL MapAndLoad(LPSTR ImageName,LPSTR D l l P a t h ,PLOADED_IMAGE Loaded lmage.BOOL DotDl l .BOOL Readonly);

PARAMETERS

ImageName

DllPath

~

Name of the PE file that is loaded.

Path used to locate the file if the name provided cannot befound. If NULL is passed, then normal rules for searchingusing the PATH environment variable are applied.


The function sets the members in the structure appropriately after loading thePE file.

RETURN VALUESIf the function succeeds, the return value is TRUE; otherwise, it is FALSE.

UnrVlapAndLoadQAfter you are done with the mapped file, you should call the UnMapAndLoadOfunction. This function unmaps the PE file and deallocates the resources allocatedby the MapAndLoadQ function.

Loadedlmage

ModuleName

hFile

MappedAddress

FileHeader

LastRvaSection

NumberOfSections

SectionsCharacteristics

fSystemlmage

fDOSImage

Links

SizeOflmage

The structure LOADEDJMAGE is defined in theIMAGEHLP.H file. The structure has the followingmembers:

Name of the loaded file.

Handle obtained through the call to CreateFile.

Memory address where the file is mapped.

Pointer to the PE header within the mapped file.

The function sets it to the first section (seeImageRvaToVaQ

Number of sections in the loaded PE file.

Pointer to the first section header within the mapped file.

Characteristics of the PE file (this is explained in moredetail later in this chapter).

Flag indicating whether it is a kernel-mode driver/DLL.

Flag indicating whether it is a DOS executable.

List of loaded images.

Size of the image.

DotDll

Readonly

If the file needs to be searched and does not have anextension, then either the .exe or the .dll extension isused. If the DotDll flag is set to TRUE, the .dll extensionis used; otherwise, the .exe extension is used.

If the flag is set to TRUE, the file is mapped as read-only.



We will discuss the other useful functions from this DLL as we continue in thischapter.

Details of the PE FormatThe WINNT.H file has the structure definitions representing the PE format. We referto these structure definitions while describing the PE format. Let's begin at the be-ginning. The DOS header that comes at the beginning of a PE file does not containmuch important information from the PE viewpoint. The fields in this header havevalues pertaining to the DOS executable stub that follows this header. The only im-portant field as far as PE format is considered is e_lfanew, which holds the offset tothe PE header. You can add this offset to the base of the memory-mapped file to getthe address of the PE header. You can also use the ImageNtHeaderO function ex-plained earlier, or simply use the FileHeader field from the LOADEDJMAGE after acall to the MapAndLoadQ function.

The IMAGE_NT_HEADERS structure that represents the PE header is defined asfollows in the WINNT.H file:

typedef s t ruct _IMAGE_NT_HEADERS {DWORD S ignature;IMAGE_FILE_HEADER F i l eHeader ;IMAGE_OPTIONAL_HEADER Op t iona lHeader ;} IMAGE_NT_HEADERS, *PIMAGE_NT_HEADERS;

The signature is PE followed by two nulls, as mentioned earlier. The COFF styleheader is represented by the IMAGE_FILE_HEADER structure and is followed by theoptional header represented by the IMAGE_OPTIONAL_HEADER structure. Thefields in the COFF style header are as follows:

MachineTarget machine ID. Various values are defined in the WINNT.H file — forexample, Oxl4C is used for Intel 80386 (and compatibles) and 0x184 is used forAlpha AXP.

PARAMETERS

Loadedlmage Pointer to a LOADEDJMAGE structure that is returned from acall to the MapAndLoadQ function.


NumberOfSections

TimeDateStamp

PointerToSymbolTable

NumberOfSymbols

SizeOfOptionalHeader

Characteristics

IMAGE_FILE_EXECUTABLE_IMAGE

IMAGE_FILE_SYSTEM

IMAGE_FILE_DLL

IMAGE_FILE_UP_SYSTEM_ONLY

IMAGE_FILE_LINE_NUMS_STRIPPED

IMAGE_FILE_LOCAL_SYMS_STRIPPED

IMAGE_FILE_DEBUG_STRIPPED

IMAGE_FILE_RELOCS_STRIPPED

Number of sections in the file.

Time and date when the file was created.

Offset to the COFF symbol table. This fieldis used only for COFF style object files andPE files with COFF style debuginformation.

Number of symbols present in the symboltable.Size, in bytes, of the optional header thatfollows this header. This data can be usedin locating the string table thatimmediately follows the symbol table. Thisfield is set to 0 for the object files becausethe optional header is absent in them.

Attributes of the file. The flag values aredefined m the WINNT.H file. This fieldcontains an OR of these flags. Theimportant flags are as follows:Set for an executable file.

Indicates that it is a kernel-modedriver/DLL.

The file is a dynamic link library (DLL).

This file should be run only on an UPmachine.

Indicates that the COFF line numbers havebeen removed from the file.

Indicates that the COFF symbol table hasbeen removed from the file.

Indicates that the debugging informationhas been removed from the file.

Indicates that the base relocationinformation is stripped from this file, andthe file can be loaded only at the preferredbase address. If the loader cannot load suchan image at the preferred base address, itfails because it cannot relocate the image.


SThe COFF style header is followed by the optionalheader is absent in the object files. The format of thefined as the IMAGE OPTIONAL HEADER structure in

header. The optionaloptional header is de-theWINNT.H file. The

first few fields in this structure are inherited from COFF. •; -- - " - -

Magic

MajorLinkerVersion,MinorLinkerVersion

SizeOfCode

SizeOflnitializedData

SizeOfUninitializedData

This field is set to Ox lOb for a normalexecutable/DLL.

Version of the linker that produced the file.

Size of the code section. If there are multiple codesections, this field contains the sum of sizes of allthese sections.Size of the initialized data section. If there aremultiple initialized data sections, this field containsthe sum of sizes of all these sections.Same as SizeOflnitializedData, but for theuninitialized data (BSS) section.

IMAGE_FILE_AGGRESTVE_WSJTRIM

IMAGE_FILE_BYTES_REVERSED_LO

IMAGE_FILE_BYTES_REVERSED_HI

IMAGE_FILE_3 2BLT_MACHINE

IMAGE_FILE_REMOVABLE_RUN_FROM_SWAP

IMAGE_FILE_NET_RUN_FROM_SWAP

Aggressively trim working set.

Little endian: the least significant bit (LSB)precedes the most significant bit (MSB) inmemory, but they are stored in reverseorder.

Big endian: the MSB precedes the LSB inmemory, but they are stored in reverseorder.

The target machine is based on 3 2 -bit-word architecture.

If this flag is set and the file is run from aremovable media, such as a floppy, theloader copies the file to the swap area andruns it from there.

Similar to the previous flag. It is run fromswap if the file is run from a networkdrive.


AddressOfEntryPoint

BaseOfCode

BaseOfData

RVA of the entry point.

RVA of the start of the code section.

RVA of the start of the data section.

Microsoft added some NT-specific fields to the optional header. These fields areas follows:

c3 '

ImageBase If the file is loaded at this address in memory, the loader neednot do any base relocations. This is because the linker resolvesall the base relocations at the time of linking, assuming that thefile will be loaded at this address. We discuss this in more detailin the section on the relocation table. For now, it is enough toknow that the loading time is reduced if a file gets loaded at thepreferred base address. A file may not get loaded at the preferredbase address because of the nonavailability of the address. Thishappens when more than one DLL used by an executable use thesame preferred base address. The default preferred base address is0x400000. You may want to have a different preferred baseaddress for your DLL so that it does not clash with that of anyother DLL used by your application. You can change thepreferred base address using a linker switch. You can also changethe base address of a file using the rebase utility that comes withthe Win32 SDK.

ReBaselmageQThe ReBaselmageO function from the IMAGEHLP.DLL also enables you to changethe preferred base address.

I "I

Chapter 11: Portable Executable File Eormat 233


The other fields in the optional header are as follows:

PARAMETERS

CurrentlmageName

SymbolPath

fReBasefRebaseSysfileOk

fGoingDown

ChecklmageSize

OldlmageSize

OldlmageBase

NewImageSize

NewImageBase

TimeStamp

Filename that is rebased.

In case the symbolic debug information is stored as aseparate file, the path to find the corresponding symbolfile. This is required to update the header informationand timestamp of the symbol file.

The file is really rebased only if this value is TRUE.If the file is a system file with the preferred base addressabove 0x80000000, it is rebased only if this flag isTRUE.

If you want the loaded image of the file to lie entirelybelow the given address, set this flag to TRUE. Forexample, if the loaded size of a DLL is 0x2000 and youcall the function with the fGoingDown flag as TRUE andgive the address as 0x600000, the DLL will be rebased at0x508000.

Rebasing might change the loaded image size of the filebecause of the section alignment requirements. If thisparameter is nonzero, the file is rebased only if thechanged size is less than this parameter.

Original image size before the rebase operation isreturned here.

Original image base before the rebase operation isreturned here.

New loaded image size after the rebase operation isreturned here.

New base address. Upon return, it contains the actualaddress where the file is rebased.

New timestamp for the file.


SectionAlignment

FileAlignment

MajorOperatingSystem Version,MinorOperatingSystemVersion

Majorlmage Version,MinorlmageVersion

Maj orSubsystemVersion,MinorSubsystemVersion

Win3 2 VersionValue

SizeOflmage

SizeOfHeaders

Checksum

Subsystem

IMAGE_SUBSYSTEM_NATTVE

IMAGE_SUBSYSTEM_WINDOWS_GUI

IMAGE_SUBSYSTEM_WINDOWS_CUI

IMAGE_SUBSYSTEM_OS2_CUI

A section needs to be loaded at an addressthat is a multiple of the section alignment.Refer to the discussion on RVA for moreinformation.

In the file, a section always starts at anoffset that is a multiple of the filealignment. This value is some multiple ofthe sector size.Minimum operating system version requiredto execute this file.

A developer can use these fields to versionhis or her files. It can be specified with alinker flag.

Minimum subsystem version required toexecute this file.Reserved for future use.

Size of the image after considering the sectionalignment. This amount of virtual memoryneeds to be reserved for loading the file.

Total size of the headers, including the DOSheader, the PE header, and the section table.The sections containing the actual data startat this offset in the file.This is used only for the kernel-modedrivers/DLLs. It can be set to 0 for user-mode executables/DLLs.Subsystem used by the file. The followingvalues are defined in the WINNT.H file:Image doesn't require a subsystem. Thekernel-mode drivers and native applicationssuch as CSRSS.EXE have this value for thefield.

File uses the Win32 GUI interface.

File uses the character-based user interface.

File requires the OS/2 subsystem.


typedef struct _IMAGE_DATA_DIRECTORY {DWORD VirtualAddress;DWORD Size;) IMAGE_DATA_DIRECTORY, *PIMAGE_DATA_DIRECTORY;

ImageDirectoryEntryToDataQThe VirtualAddress field contains the RVA of the respective piece of information,and the Size field contains the size of the data. To get to the actual data, you needto convert the RVA to the actual address in the memory-mapped PE file. This can beaccomplished with the ImageDirectoryEntryToDataO function exported by IM-AGEHLP.DLL.

IMAGE_SUBSYSTEM_POSIX_CUI

DllCharacteristics

SizeOfStackReserve

SizeOfStackCommit

SizeOfHeapReserve

SizeOfHeapCommit

LoaderFlags

NumberOfRvaAndSizes

DataDirectory[TMAGE_NUMBEROF_DIRECTORY.ENTRIES]

File uses the POSIX API.

Obsolete.

Address space to be reserved for the stack.Only the virtual address space is marked -the swap space is not allocated.Actual memory committed for the stack.This much swap space is initially allocated.The committed stack size is increased ondemand until it reaches theSizeOfStackReserve.

Address space to be reserved for the heap.Similar to the SizeOfStackReserve field.

Actual committed heap space. Similar to theSizeOfStackCommit field.Obsolete.

Number of entries in the data directory thatfollows this field. It is always set to 16.

As mentioned earlier, each entry in the datadirectory points to some important piece ofinformation. Each of these entnes is of thetype IMAGE_DATA_DIRECTORY, which isdefined as follows:


PARAMETERS ?t»-

BaseMappedAsImage

DirectoryEntry

Size

Base address where the file is mapped in memory.Set this flag to TRUE if the system loader maps the file.Otherwise, set the flag to FALSE.

Index into the data directory array. -« - V .. " -

Upon return, the size from the data directory is filled here.

RETURN VALUESIf the function succeeds, the return value is the address in the memory-mapped filewhere the required data resides. Otherwise, the function returns NULL.

Indices in the Data DirectoryEach index in the data directory (except a few at the end that are still unused) rep-resents some important piece of information. In the following sections, we discusssome of the important entries in this directory and the format in which the respec-tive information is stored.

Export DirectoryThe data directory entry at the IMAGE_DIRECTORY_ENTRY_EXPORT index pointsto the export directory for the file. The RVA in this directory entry points to the.edata section. The information about the functions exported by the file (generallya DLL) is stored here. The data directory entry points to the export directory that isdefined as the IMAGE_EXPORT_DIRECTORY structure in the WINNT.H file. Thefields in this structure are as follows:

CharacteristicsTimeDateStamp

Reserved field. Always set to 0.Date and time of creation.


MajorVersion,MinorVersion

Name

Base

NumberOfFunctions

NumberOfNames

AddressOfFunctions

^

* -^ *-

Developer can set the version of the export table.

RVA of the zero-terminated name of the DLL.Starting ordinal for the exported functions - that is,the least of the ordinals. Generally, this field is 1.

Total number of functions exported from the DLL.

Number of functions that are exported by name.Some functions may be exported only by ordinal, sothis number may be less than NumberOfFunctions.RVA of an array (let's call it as the export-functionsarray) that has an entry for each function exportedfrom the DLL. Hence, the size of this array is equal tothe NumberOfFunctions field. The entry at index icorresponds to the function exported with ordinal i +Base. Each entry in this array is also an RVA. If theRVA for a particular array entry points within theexport section, then it is a forwarder. Forwardermeans that the function is not present in this DLL,but it is a forwarder reference to some function inanother DLL. In such a case, the RVA points to anASCIIZ string that stores the name of the other DLLand the function name separated by a period. In casethe target DLL exports the function by ordinal, thefunction name is formed as # followed by theordinal printed in decimal. For example, theKERNEL32.DLL for Windows NT forwards theHeapAllocO function to the RtlAllocateHeapQfunction in the NTDLL.DLL. Hence, the correspondingRVA in this case points to a location within theexport section that holds the stringNTDLLRtlAllocateHeap. The Win32 applications canimport the HeapAllocO function from theKERNEL32.DLL without worrying about all thesedetails. When the application runs on Windows 95,the loader resolves the import reference to thefunction in the KERNEL32.DLL. When the sameapplication runs on Windows NT, the loader findsthat the function is forwarded to the NTDLL.DLL.Hence, the loader automatically loads the NTDLL.DLLand resolves the imported function to theRtLAllocateHeapO function.


When an export-functions array entry is not a forwarder - that is, the RVA doesnot lie within the export section - the RVA points to the entry point of the functionor to the location of the exported variable.

The export-functions array may have gaps. This is beacause some ordinals mightbe left unused while exporting functions, and some ordinals might not have anycorresponding export. In such a case, the corresponding array entry is set to 0.

AddressOfNames

AddressOfNameOrdinals

RVA of an array called as the export-names arraythat has an entry for every function that isexported by name. Hence, the size of this array isequal to the NumberOfNames field. Each entry inthis array is an RVA pointing to an ASCIIZ stringcontaining the export name. The array is sortedon the lexical order so as to allow binary search.

RVA of an array of ordinals henceforth called asthe export-ordinals array. This array has the sizesame as that of the AddressOfNames array. Allthree arrays, namely, export-names, export-ordinals, and export-functions, are instrumentalin resolving imports by name. For resolving animport by name, the loader first searches thename in the export-names array. If the namematches an entry with index i, the ith entry in theexport-ordinals array is the ordinal of thefunction. Finally, the address of the function canbe found from the export-functions array.

Import DirectoryThe next index in the data directory, IMAGE_DIRECTORY_ENTRY_IMPORT, is re-served for the import directory of an executable/DLL. The RVA in this data directoryentry points to the import directory, which is nothing but a variable-sized array ofIMAGE_IMPORT_DESCRIPTORs, one for each imported DLL. The first field in thisstructure is a union. If the Characteristics field in this union is 0, it indicates the endof the variable-sized import descriptors array. Otherwise, the union is interpretedusing the other member, OriginalFirstThunk.

OriginalFirstThunk

* * ~i-,(, -

This is an RVA of what Microsoft calls as the ImportLookup Table (ILT). Each entry in the ILT is a 32-bitnumber. If the MSB of this number is set, it is treated asan import by ordinal. The bits 0 through 30 are treatedas the ordinal of the imported function. If the MSB isnot set, the number is treated as an RVA to theIMAGE IMPORT BY NAME structure. The first member


of this structure is a hint for searching for the importedname in the export directory of the imported DLL. Theloader uses this hint as the starting index in the export-names array when it does a binary search whileresolving the import reference. The hint is followed byan ASCIIZ name of the import reference.

The WINNT.H file provides the IMAGE_SNAP_BY_ORDINAL macro to determinewhether it's an import by ordinal. It also provides the IMAGE_ORDINAL macro toget the ordinal from the 32-bit number in the ILT. The ILT is a variable-sized array.The end of the ILT is marked with a 0.

The other members in the IMAGE IMPORT_DESCRLPTOR structure are as follows:

DYNAMIC LINKING WITH PE FILESEvery DLL has an import library that can either be created using an import librar-ian or may be generated by the linker itself while creating the DLL. The import li-brary has stub functions with names the same as those of the functions exportedfrom the DLL. The import library also has a .idata section containing an importtable that has entries for all the functions from the DLL. Each stub function is anindirect jump that refers to the appropriate entry in the LAT in the .idata section.When an executable is linked with the import library, the linker resolves the im-ported function calls to the stub functions in the import library. The linker also con-catanates the .text section from the import library that contains the stub functionswith the .text section of the generated executable. The .idata sections and, inciden-tally, the import directories are also concatenated. The stage is now set for loading.

TimeDateStamp

ForwarderChain

Name

FirstThunk

This field is set to 0, unless the imports are bound. Soon,we discuss what's meant by binding the imports of a PEfile.The field is used only if the imports are bound.

RVA of the ASCIIZ string that stores the name of theimported DLL.

RVA of the Import Address Table (LAT). The IAT is anotherarray parallel to the ILT, unless the image is bound. TheIAT also has ordinals or pointers to the IMAGE_IMPORT_BY_NAME structures. When the loader resolves theimport references, it replaces the entries in the IAT withthe actual addresses of the corresponding functions.Astonishingly, that is all it needs to do to achievedynamic linking — everything else is already set in placeby the linker and import librarian. Let's see how all thesecomponents work together to achieve dynamic linking.


While loading, the entries in the IAT are replaced by the actual function addresses,and that's it. Now when the function is called, the control is transferred to the stubfunction that performs an indirect jump. As the IAT entry contains the address ofthe actual function from the DLL, the control is transferred to the required function.

The situation is a bit different if you use the new _declspec(dllimport) directivewhile prototyping an imported function. In that case, the compiler itself generatesan import table. In addition, it generates an indirect call referring to the appropri-ate location in the generated IAT. This method does away with the overhead of anextra jump.

BINDING IMPORTS FOR A PE FILEA major portion of loading time is spent on resolving the imports. The loader hasto search each imported symbol in the export directory of the imported DLL to findout the virtual address of the symbol. The loading time can be drastically reducedif the IAT contains the actual address of the symbol instead of the name or ordi-nal. Such a PE file is called as a bound image. The imported symbol addresses arecalculated assuming that the imported DLL will be loaded at the preferred base ad-dress at the time of loading. The IMAGE_IMPORT_DESCRIPTORs, in a bound PEfile, are also modified. The TimeDateStamp field stores the timestamp of the im-ported DLL. At the time of loading, if this timestamp does not match with that ofthe DLL, the imports need to be resolved again. Because the IAT is modified anddoes not contain the symbol names or ordinals, the ILT is used, in this case, to re-solve the imports.

The forwarded functions pose another problem with binding. The addresses ofthe forwarded functions cannot be calculated at bind time, and so these functionshave to be resolved at load time. A list of all the forwarded functions for an im-ported DLL is maintained through the ForwarderChain member in the correspond-ing IMAGE_IMPORT_DESCRIPTOR. This member stores the index of a forwardedfunction in the IAT. The IAT entry at this index stores the index of the next for-warded function, and so on, forming a list of forwarded functions. The list is termi-nated by a -1 entry.

BindlrnageQThe bind utility that is shipped with Win32 SDK enables binding of PE files. Also,the Bindlmage and BindlmageExO functions in the IMAGEHLP.DLL provide thisfunctionality.

BOOL Bindlmage(LPSTR ImageName,LPSTR D l lPa th ,LPSTR Symbol Path) . -


PARAMETERS

ImageName

DllPath

SymbolPath

The filename of the file to be bound. This can contain only afilename, a partial path, or a full path.

A root path to search for ImageName if the filename containedin ImageName cannot be opened.

A root path to search for the corresponding symbol file. If thesymbol file is stored separately, the header of the symbol file ischanged to reflect the changes in the PE file. -


BindlmageExQThis function is very similar to Bindlmage function except it provides more cus-tomization such as getting a periodic callback during the progress of bindingprocess.

BOOL B indImageEx(IN DWORD F l a g s ,IN LPSTR ImageName,IN LPSTR Dl lPath,IN LPSTR Symbo lPa th ,IN PIMAGEHLP_STATUS_ROUTINE Sta tusRout ine

) ;

PARAMETERSThis function has the following additional parameters:

Flags

BIND_NO_BOUND_IMPORTS

BIND_NO_UPDATE

BIND_ALL_IMAGES

StatusRoutine

The field controls the behavior of the function.It is set to as an OR of the flag values definedin the IMAGEHLP.H file. The following flagvalues are defined in the IMAGEHLP.H file:

Do not generate a new import address table.

Do not make any changes to the file.

Bind all images that are in the call tree for thisfile.

Pointer to a status routine. The status routine iscalled during the progress of the image bindingprocess.



Calling Bindlmage is equivalent to calling BindlmageEx with Flags as 0 andStatusRoutine as NULL. That is, calling BindlmageflmageName, DllPath, SymbolPath)is equivalent to calling BindImageEx(0, ImageName, DllPath, SymbolPath, NULL).

Resource DirectoryThe next index in the data directory, IMAGE_DIRECTORY_ENTRY_RESOURCE,refers to the resource directory for a PE file. The resource directory and the re-sources themselves are generally stored in a section named .rsrc section. The re-sources are maintained in a tree structure similar to that in a file system. The rootdirectory contains subdirectories. A subdirectory can contain subdirectories or re-source data. The subdirectories can be nested to any level. But Windows NT onlyuses a three-level structure. At each level, the resource directory branches accord-ing to certain characteristics of the resources. At the first level, the type of the re-source - bitmap, menu, and so on - is considered. All the bitmaps are stored underone subtree, all the menus are stored under another subtree, and so on. At the nextlevel, the name of the resource is considered, and the third level classifies the re-source according to the language ID. The third-level resource directory points to aleaf node that stores the actual resource data.

A resource directory consists of summary information about the directory fol-lowed by the directory entries. Each directory entry has a name or ID that is inter-preted as a type ID, a name ID, or a language ID, depending on the level of thedirectory. A directory entry can point either to the resource data or to a subdirec-tory that has a similar format.

The format of the resource directory is defined as the IMAGE_RESOURCE_DI-RECTORY structure in WINNT.H.

Characteristics

TimeDateStamp

MajorVersion, MinorVersion

NumberOfNamedEntries

NumberOfldEntries

Currently unused. Set to 0.

Date and time when the resource was generated bythe resource compiler.

Can be set by the user.

Number of directory entries having string names.These entries immediately follow the directorysummary information and are sorted.

Number of directory entries that use integer IDs asthe names. These entries follow the ones havingstring names.


This summary information is followed by the directory entries. Each directoryhas a format as defined by the IMAGE_RESOURCE_DIRECTORY_ENTRY structurein WINNT.H. This structure is composed of two unions. The first union stores the IDof the entry. If the MSB is set, then the lower 31 bits in this field is an RVA of theUnicode string that stores the name of the entry. The Unicode string consists of thelength of the string followed by the 16-bit Unicode characters. If the MSB is not set,then the union stores the integer ID of the resource. This first union stores the typeID, the name ID, or the language ID, depending on the level of the directory. Thesecond union, in the IMAGE_RESOURCE_DIRECTORY_ENTRY structure, points ei-ther to another resource directory or to the resource data, depending on the MSB. Ifthe bit is set, the lower 31 bits is an RVA of another subdirectory. If the MSB is notset, then it's an RVA of the resource data entry that forms a leaf node of the re-source directory tree structure. The format of the resource data entry is defined asthe IMAGE_RESOURCE_DATA_ENTRY structure in the WINNT.H file and has fol-lowing members:

OffsetToData RVA of the actual resource data.

Size Size of the resource data.

CodePage Code page used to decode code point values within theresource data. Typically, the code page would be the Unicodecode page.

Relocation TableA PE file needs only based relocations. The linker resolves all the relative reloca-tions, assuming that the file will get loaded at the preferred base address. For exam-ple, if a function foo has the RVA as 0x100 and the preferred base address is0x400000, the linker resolves the call to foo as a call to address 0x400100. At runtime, if the file is loaded at the preferred base address of 0x400000, then no reloca-tion needs to be preformed. If, for some reason, the file cannot be loaded at the baseaddress of 0x400000, the loader needs to patch the call. If the loader manages toload the file at a base address of 0x600000, it needs to change the call address to0x600100. In general, it needs to add the difference of 0x200000 to all the to-be-patched locations. This process is called as the based relocation. The list of the to-be-patched locations, also called as fixups, is maintained in the relocation table that isgenerally present in the .reloc section and is pointed to by the data directory entry atthe IMAGE_DIRECTORY_ENTRY_BASERELOC index. The relocation table is nothingbut a series of relocation blocks, each representing the fixups for a 4K page. Each re-location block has a header followed by the relocation entries for the correspondingpage. The relocation block format is defined as the IMAGE_BASE_RELOCATIONstructure in the WINNT.H file, and it has following fields:


VirtualAddress RVA of the page to be patched.

SizeOfBlock Total size of the relocation block, including the header andthe relocation entries.

Each relocation entry is a 16-bit word. The higher 4 bits indicate the type of re-location, and the lower 12 bits are the offset of the fixup location within the 4Kpage. The address-to-patched is calculated by adding the base address for loading,the RVA of the page to be patched, and the 12-bit offset within the page. The relo-cation types are defined in the WINNT.H file - only two of them are used on Intelmachines:

IMAGE_REL_BASED_ABSOLUTE

IMAGE_REL_BASED_fflGHLOW

The relocation is skipped. This type can beused to pad a relocation block so that thenext block starts at a 4-byte boundary.The relocation adds the base-addressdifference to the 3 2 -bit double word at thelocation denoted by the 12-bit offset.

Debug DirectoryThe operating system is not concerned with the debug information present in a PEfile. The debugging tools access the debug information in a PE file. There are vari-ous debugging tools, which expect the debug information in different formats. Thecorresponding compilers/linkers also store the debug information in different for-mats. The PE format allows the debug information to be stored in different formats,such as COFF, Frame Pointer Omission (FPO), CodeView (CV4), and so on. A singlefile may contain debug information in more than one format. The debug directorypointed to by the IMAGE_DTRECTORY_ENTRY_DEBUG entry in the data directoryis an array of debug directory entries, one for each debug information format. TheIMAGE_DEBUG_DIRECTORY structure in the WINNT.H file represents the format ofa debug directory entry.

CharacteristicsTimeDateStamp

MajorVersion, MinorVersion

Type

SizeOfDataAddressOfRawData

PointerToRawData

Currently unused. Set to 0.Date and time when the debug data was created.

Version of the debug data format.

Type of the debug data format.

Size of the debug data.RVA of the debug data.

Within file offset to the debug data.

>


Of the different debug information formats, three are frequently encountered inPE files. The first one is the format used by the popular CodeView debugger. Thisformat is defined in the CV4 specification. The FPO format is used to describe non-standard stack frames. Not all the files in a PE file need have an FPO format debugentry. The functions without one are assumed to have a normal stack frame. Thethird important format is COFF, which is the native debug information format forPE files. The PE header itself points to the COFF symbol table. The COFF debug in-formation consists of symbols and line numbers.

Thread Local StorageThe threads executing in a process share the same global data space. Sometimes, itmay be required that each thread has some storage local to itself. For example, saya variable i needs to be local for each thread.

In such a case, each thread gets a private copy of i. Whenever a particular threadis running, its own private copy of i should be automatically activated. This isachieved in Windows NT using the Thread Local Storage (TLS) mechanism. Let's seehow it works.

Do not confuse the local data of a thread with the local variables that are cre-ated on stack. Each thread has a separate stack and local variables that are cre-ated and destroyed separately for each thread as the stack grows and shrinks. Inthis section, the phrase local data means global variables that have a separatecopy for each thread.

The operating system maintains a structure called as the Thread EnvironmentBlock (TEB) for every thread running in the system. The FS segment register is al-ways set such that the address FS:0 points to the TEB of the thread being executed.The TEB contains a pointer to the TLS array. The TLS array is an array of 4-byteDWORDs. Similar to the TEB, a separate TLS array is present for each thread. Athread can store its local data in the TLS array. Generally, programs store pointersto local data in some slot in the TLS array. The slot allocation for the TLS array iscontrolled by the API functions TlsAllocQ and TlsFreeQ. The Win32 API also pro-vides functions to set and get the value at a particular index in the TLS array.

It is cumbersome to access the thread-specific data using the API functions. Aneasier way is to use the _declspec(thread) specification while declaring global vari-ables that need to have a private copy for each thread. All such variables are gath-ered by the compiler/linker, and a single TLS array index is automatically allottedto this bunch of data. The TLS array entry at this index contains the pointer to a lo-cal data buffer that stores all these variables. These variables are accessed as anyother normal variable in the program. Whenever such a variable is accessed, thecompiler takes care to generate the code to access the TLS array entry and the dataat a proper offset within the local data buffer.

This discussion is bit off the track. However, it is necessary before discussing theIMAGE_DIRECTORY_ENTRY_TLS data directory entry. The TLS directory structureis defined as IMAGE_TLS_DIRECTORY in the WINNT.H. Let's have a look at thisstructure and see how it fits in the TLS mechanism.


StartAddressOfRawData

EndAddressOfRawData

AddressOflndex

AddressOfCallBacks

SizeOfZeroFill

Characteristics

Each time a new thread is created, the operatingsystem allocates a new local data buffer for thethread and initializes the buffer with the data that ispointed to by this field. Note that this address is notan RVA, but it is a proper virtual address that has arelocation entry in the .reloc section.

Virtual address of the end of the initialization data.The rest of the local data buffer is filled with zeros.

Address in the data section where the loader shouldstore the automatically allotted TLS index. The codeaccessing TLS variables accesses the index from thislocation.

Pointer to a null-terminated array of TLS callbackfunctions. Each function in this array is calledwhenever a new thread is created. These functionscan perform additional initialization (for example,calling constructors) for the TLS data. The TLScallback has the same parameters as the DLL entry-point function.

Size of the local data that is to be initialized to zero.The total size of the local data is(EndAddressOfRawData StartAddressOfRawData) +SizeOfZeroFill.

-. r * ,_

Reserved.

Section TableWe've roamed through the PE format without bothering about the section formats.This is possible because of the data directory that directly locates the importantpieces of information within a PE file. You need not know about the sections at allto interpret a PE file. Nevertheless, in case you need to modify a PE file, you maybe required to know about the sections and section headers. For example, you maywant to add, remove, or extend a particular section, and this requires changes tothe section table, among other things.

As mentioned earlier, the PE header is followed by the section table. The sectiontable is an array of section headers. The format of the section header is defined bythe IMAGE_SECTION_HEADER structure in the WINNT.H file. The members of asection header are as follows:


Name

VirtualSize

VirtualAddress

SizeOfRawData

PointerToRawData

PointerToRelocations

PointerToLinenumbers

NumberOfRelocations

NumberOfLinenumbers

Characteristics

IMAGE_SCN_CNT_CODE

IMAGE_SCN_CNT_INITIALIZED_DATA

IMAGE_SCN_CNT_UNINITIALIZED_DATA

IMAGE_SCN_LNK_REMOVE

IMAGE_SCN_MEM_DISCARDABLE

Character array of sizeIMAGE_SIZEOF_SHORT_NAME. Containsthe name of the section.

Size of the section.

RVA of the section data when loaded inmemory. ^ „

Size of the section as stored in the file. Thisis equal to the VirtualSize rounded to thenext file alignment multiple.

Within file offset to the section data. If youmemory map a PE file, this field needs to beused to get to the section data.Used only in the object files.

Within file offset to the COFF style linenumber information.

Used only in the object files.

Number of records in the line numberinformation. . ,

The attributes of the section. It is an OR ofthe section characteristics flags defined inthe WINNT.H file. Some of the importantflags are as follows:

Section contains executable code.Section contains initialized data.

Section contains uninitialized data.

Section will not become part of the loadedimage. The .debug section may have thisflag set.Section can be discarded. The relocationtable and debug information can bediscarded after the loading process is over.Hence, the .debug and .reloc sections havethis flag set.

IMAGE SCN_MEM_NOT CACHED Section cannot be cached.


IMAGE_SCN_MEM_NOT_PAGED

IMAGE_SCN_MEM_SHARED

«

IMAGE_SCN_MEM_EXECUTE

TMAGE_SCN_MEM_READ

IMAGE SCN MEM WRITE

Section is not pageable.

Section can be shared in memory. If a DLLhas the data section with this flag set, allthe instances of the DLL in differentprocesses share the same data.

Section can be executed. For the codesections, both the IMAGE_SCN_CNT_CODEand IMAGE_SCN_MEM_EXECUTE flags areset.Section can be read.

Section can be written to.

Loading ProcedureLet's see how the loader interprets a PE file and prepares a memory image for exe-cution. The loader needs to find the free virtual address space to map the file inmemory. The loader tries to load the image at the preferred base address. After thisis done, the loader maps the sections in memory. The loader goes through the sec-tion table and maps each section at the address calculated by adding the RVA of thesection to the base address. The page attributes are set according to the section'scharacteristic requirements. After mapping the section in memory, the loader per-forms based relocation if the base address is not equal to the preferred base address.Then, the import table is checked and the required DLLs are loaded. The same pro-cedure for loading an executable - mapping sections, based relocation, resolvingimports, and so on - is applied while loading a DLL. After loading each DLL, theIAT is fixed to point to the actual imported function address.

That's it! The image is ready for execution.

SummaryMicrosoft introduced the Portable Executable (PE) file format with Windows NT.The PE format serves as the executable file format for all the 32-bit Microsoft oper-ating systems (that is, the various versions of Windows NT and Windows 95/98)though these operating systems still support the older executable file formats, in-cluding the DOS executable file format.


Various components in a PE file are addressed using the relative virtual address(RVA). The IMAGEHLP.DLL provides us with utility functions to memory map a PEfile to find the address in the memory corresponding to the RVA specified in the PEfile. A PE file is composed of the file headers, the data directory, the section table,and the various sections. The data directory points to the important parts of the PEfile: the export directory, the import directory, the relocation table, the debug direc-tory, and the Thread Local Storage. The export directory lists the symbols exportedfrom the PE file, which is most likely a DLL. The import directory lists all the sym-bols imported by the PE file. When a PE file is loaded in memory for execution, theloader resolves the imported symbols to actual virtual addresses in the DLL that ex-ports the symbols. This process is termed dynamic linking.

The PE headers are followed by the section table that points to all the sections,including the ones pointed to by the various data directory entries. The loader readsthe section table and maps various sections of a PE file in memory. Then it preparesthe image for execution by relocating the image for the mapped address and re-solving various imported symbols after loading the required DLLs.

PartAppendixes

APPENDIX ADetails of System Calls with Parameters

APPENDIX BWhat's on the CD-ROM

Appendix A

Details of System Calls

with Parameters

THE DOCUMENTATION OF MOST of the system services provided by Windows NTfollows. Most of these system services are undocumented. Each entry includes aprototype of the system service, its parameters and descriptions of them, returncodes, and in some cases, comments.

This documentation is useful for many reasons, among them:

+ To put a hook into a system service, you must know the parameters itexpects so you can write a new hook service with the same prototype.

+ Few services have no corresponding Win32 API. These are trulyundocumented services. To use these services, one must know theparameters they expect.

+ Although it seems to be Microsoft's position to keep these system servicesundocumented because they might change in future versions, we haveobserved that most of these system services are core and largelyunchanged in versions of Windows NT to date. New system services arebeing added to this list with each new version of Windows NT.

NtLoadDriverNTSTATUS

NtLoadDn ver(IN PUNICODE_STRING DriverRegistryEntry

):

NtLoadDriver loads the device driver specified by the Registry key underHKLM\System\CurrentControlSet\Device. For example, the device driver namedxxx has a Registry key "xxx" under HKLM\System\CurrentControlSet\Device.

PARAMETERS

DriverRegistryEntry Points to the Unicode string containing the name of theRegistry key for the driver where the configurationinformation for the driver is kept. The parameter is ofthe form HKLM\System\CurrentControlSet\Device\xxx,where xxx stands for device driver named xxx. 253 I

254 Appendix A: Details of System Calls with Parameters

RETURN VALUEReturns STATUS_SUCCESS on success and an appropriate error code on failure.

COMMENTSOnly users who have the privilege to load/unload device drivers can use this API.

EQUIVALENT WIN 3 2 APIService Control Manager APIs such as CreateService, StartService, and so on.

NtllnloadDriverNTSTATUS ;

NtUnloadDriver(IN PUNICODE_STRING D M v e r R e g n s t r y E n t r y

);

NtUnLoadDriver unloads the device driver specified by the Registry key underHKLM\System\CurrentControlSet\Device. For example, the device driver namedxxx has a Registry key "xxx" under HKLM\System\CurrentControlSet\Device.

PARAMETERS

DriverRegistryEntry Points to the Unicode string containing the name of theRegistry key for the driver where the configurationinformation for the driver is kept. The parameter is ofthe form HKLM\System\CurrentControlSet\Device\xxx,where xxx stands for the device driver named xxx.


COMMENTSOnly users who have the privilege to load/unload device drivers can use this API.

EQUIVALENT WIN 3 2 APIService Control Manager APIs such as StopService, DeleteService, and so on.

NtCloseNTSTATUS

NtCloseC- IN HANDLE Hand le

):

NtClose closes the open handle to the executive object. This could be any handle,such as a handle to the mutex, semaphore, and so on.

Appendix A: Details of System Calls with Parameters 255

PARAMETERS

Handle Handle to the object.

RETURN VALUEReturns STATUS_SUCCESS on success and an appropnate error code on failure.

COMMENTSEvery object has a handle count and a reference count associated with it. The han-dle count represents number of open handles to the object. The reference count rep-resents the number of pointer references to the object. The object is removed frommemory only when the object handle count and reference count reaches zero.

EQUIVALENT WIN32 APICloseHandle

hSourceProcessHandle

hSourceHandle

hTargetProcessHandle

TargetHandle

AccessMask

Handle to the process in which the handle to beduplicated resides.

Handle to the object to be duplicated.

Handle to the process in which the handle isduplicated.

Pointer to the variable where the duplicated handle isreceived.

Access requested for the new handle.

NtDuplicateObject creates a new handle to the given object in arbitrary process'scontext.

PARAMETERS


blnheritHandle

dwOptions

Boolean value describing whether the handle isinherited by child processes spawned by the processand represented by TargetProcessHandle.

Flags that affect the behavior of the system service. IfDUPLICATE_SAME_ACCESS is specified, then theAccessMask parameter is ignored. IfDUPLICATE_CLOSE_SOURCE is specified, then thesource handle is closed after duplication.


COMMENTSConsole handles cannot be duplicated using this system service.

EQUIVALENT WIN 32 APIDuplicateHandle

NtCreateDi rectoryObjectNTSTATUS

NtCreateDirectoryObjectCOUT PHANDLE hDi rec to ry ,IN ACCESS_MASK A c c e s s M a s k ,IN POBJECT^ATTRIBUTES ObjectAttr ibutes

);

NtCreateDirectoryObject creates a new directory object.

PARAMETERS

hDirectory

AccessMask

ObjectAttributes

Pointer to the variable that receives a handle to thedirectory object.

Type of access requested to the directory object. This can bea combination of any of the following flags:DIRECTORY_QUERY, DIRECTORY_TRAVERSE,DIRECTORY_CREATE_OBJECT,DIRECTORY_CREATE_SUBDIRECTORY, andDIRECTORY_ALL_ACCESS.

Points to the OBJECT_ATTRIBUTES structure containing theinformation about the directory object to be created, such asname, parent directory, objectflags, and so on.



COMMENTSNone.

EQUIVALENT W1M32 API -~ 'None.

NtCreateSymbolicLinkObjectNTSTATUS

N t C r e a t e S y m b o l i c L i n k O b j e c t COUT PHANDLE hSymbol icL ink ,IN ACCESS_MASK A c c e s s M a s k ,IN POBJECT_ATTRIBUTES ObjectAttr ibutes ,IN PUNICODE_STRING S y m b o l i c L i n k V a l u e

);

NtCreateSymbolicLinkObject creates a new symbolic link.

PARAMETERS

hSymbolicLink

AccessMask -

ObjectAttributes

SymbolicLinkValue

Pointer to the variable that receives handle to theSymbolicLink object.

Type of access requested to the symbolic link object. Thiscan be a combination of any of the following flags:SYMBOLIC_LINK_QUERY orSYMBOLIC_LINK_ALL_ACCESS.

Points to the OBJECT_ATTRIBUTES structure containingthe information about the symbolic link object to becreated, such as name, parent directory, objectflags, andso on.

Points to a Unicode string containing the object namethis symbolic link refers to.


COMMENTSThe user-mode API call DefineDosDevice enables you to create a symbolic link ob-ject only under the object directory named "\??", whereas this system service en-ables you to create a symbolic link object anywhere in the object name space


provided you have permission. There is a symbolic link named "\DosDevices" whichpoints to "\??" object directory.

EQUIVALENT WIN32 APIDefmeDosDevice (limited support)

NtMakeTemporaryObject ^NTSTATUS

NtMakeTemporaryOb jec tCIN HANDLE hObject

);

NtMakeTemporaryObject converts the permanent object into a temporary object.

PARAMETERS

hObject A Handle to the permanent object.


COMMENTSThe objects created with the OBJ_PERMANENT attribute in ObjectAttributes'sAttributes member can be converted into a temporary object using this function.Permanent objects with names are not deleted from the Object Manager name spaceeven when the handle count reaches zero. This function can be used to delete per-manent named objects with handle count zero from the Object Manager name space.

-v & jEQUIVALENT WIN32 APINone.

NtOpenDi rectoryObjectNTSTATUS

NtOpenDi rec to ryOb jec tCOUT PHANDLE hDi rec to ry ,IN ACCESS_MASK AccessMask,IN POBJECT_ATTRIBUTES ObjectAt tnbutes

):

NtOpenDirectoryObject opens an existing directory object in the Object Managername space.


PARAMETERS

hDirectory

AccessMask

ObjectAttributes

Pointer to the variable that receives a handle to thedirectory object.Type of access requested to the directory object. This can bea combination of any of the following flags:DIRECTORY_QUERY, DIRECTORYJTRAVERSE, "*~DIRECTORY_CREATE_OBJECT,DIRECTORY_CREATE_SUBDIRECTORY, andDIRECTORY_ALL_ACCESS.

Points to the OBJECT_ATTRIBUTES structure containing theinformation about the directory object to be opened, such asname, parent directory, objectflags, and so on.


COMMENTSNone.

EQUIVALENT WIN32 APINone.

NtQueryDi rectoryObjectNTSTATUS

NtQueryDi rectoryObjectCIN HANDLE hD i rec to ry ,OUT PVOID DirectoryEntryBuffer,IN ULONG D i rec to ryEn t ryBu f fe rS ize ,IN BOOLEAN bOnlyFirs tEntry,IN BOOLEAN bF i rs tEnt ry ,OUT PULONG BytesReturned,OUT PULONG Entrylndex

);

NtQueryDirectoryObject returns individual object names in the given object di-rectory along with the type of these objects.

PARAMETERS

hDirectory Handle to a directory opened usingNtOpenDirectory.


DirectoryEntryBuffer

DirectoryEntryBufferSize

bAllEntries

bFirstEntry

BytesReturned

Entrylndex

Pointer to the buffer that receives the objectnames and object types in the given objectdirectory.

Size of the buffer pointed to byDirectoryEntryBuffer.

Flag indicating whether you are interested in allthe entries in the given object directory.

Flag indicating that the search should start fromthe first entry in the directory.

Pointer to the variable that receives the numberof bytes copied into the buffer pointed to byDirectoryEntryBuffer.

Pointer to the variable that returns the indexcorresponding to the object entry returned.


COMMENTSTo enumerate all the objects in a given object directory, you need to first call thisfunction with bFirstEntry set to TRUE, and then call this function with bFirstEntry setto FALSE. The function returns STATUS_NO_MORE_ENTRIES when all the entries ina given object directory are over. bAllEntries should be set to TRUE in this case.

Data in DirectoryEntryBuffer is of variable length based on the object names andobject types. The fixed portion of this data is as follows:

typedef struct Di rectoryEntryBuf fer_t {UNICODE_STRING ObjectName,UNICODE_STRING ObjectType

} DIRECTORY_ENTRY_BUFFER

This is followed by ObjectName and ObjectType in wide character format.


NtOpenSymboli cLi nkObjectNTSTATUS

NtOpenSymbolicLinkObjectCOUT PHANDLE hSymbolicLnnk,


IN ACCESS_MASK A c c e s s M a s k ,IN POBJECT_ATTRIBUTES ObjectAt tnbutes

);

NtOpenSymbolicLinkObject opens an existing symbolic link object.

PARAMETERS

hSymbolicLink

AccessMask

ObjectAttributes

Pointer to the variable that receives handle to theSymbolicLink object.

Type of access requested to the symbolic link object. Thiscan be a combination of any of the following flags:SYMBOLIC_LINK_QUERY or - - - -.,... .,SYMBOLIC_LINK_ALL_ACCESS

Points to the OBJECT_ATTRIBUTES structure containinginformation about the symbolic link object to be opened,such as name, parent directory, objectflags, and so on.


COMMENTSNone.

EQUIVALENT WIN 3 2 APINone.

NtQuerySymbolicLlnkObject ,NTSTATUS

NtQuerySymbo l i cL inkOb jec t (IN HANDLE hSymbol icL ink , ..-. ..., , -_._ ,IN OUT PUNICODE_STRING Ob jec tName ,OUT PULONG BytesReturned

):

NtQuerySymbolicLinkObject returns the object referred to by a given symboliclink object. . . -

PARAMETERS

hSymbolicLink Handle to the symbolic link object returned usingNtOpenSymbolicLinkObject or NtCreateSymbolicLinkObject.


ObjectName

BytesReturned

Pointer to the initialized Unicode string. The object namereferred to by the given symbolic link is returned. The bufferfor the Unicode string must already be allocated.Pointer to the variable that returns the number of bytescopied into the buffer pointed to by ObjectName.


COMMENTSThe user-mode API call QueryDosDevice enables the query symbolic link objectonly under the object directory named "\??", whereas this system service enablesyou to query any symbolic link in the object name space provided you havepermission.

EQUIVALENT WIN 3 2 APIQueryDosDevice (limited support)

NtQueryObjectNTSTATUS

N t Q u e r y O b j e c t (IN HANDLE hOb jec t ,IN OBJECT_INFORMATION_CLASS In foC lass ,OUT P V O I D Buf fer ,IN ULONG Bu f fe rS i ze ,OUT PULONG BytesRetumed

);

NtQueryObject returns different kinds of information about the object based onthe InfoClass.

PARAMETERS

hObject

InfoClass

Buffer

BufferSize

BytesReturned

Handle to the object.

Type of information to be retrieved. This can take values fromOto 4.

Pointer to the buffer that receives information about theobject.

Size of the buffer in bytes pointed to by Buffer.

Pointer to the variable that receives the number of bytescopied into the Buffer.



COMMENTSDifferent information is returned based upon the InfoClass parameter. Here is thelayout of Buffer based on the InfoClass:

InfoClass=0

typedef struct Objec tBas ic ln fo_t {char Unknownl [8 ] ;ULONG HandleCount ;ULONG ReferenceCount ;ULONG PagedQuota ;ULONG NonPagedQuo ta ;char Unknown2 [32 ] ;

} OBJECT_BASIC_INFO, *POBJECT_BASIC_INFO:

InfoClass=l Variable length structure based on the actual length of the object name.

typedef struct ObjectNameInfo_t (UNICODE_STRING ObjectName;WCHAR Ob jec tNameBuf fe r [ l ] ;

} OBJECT_NAME_INFO, *POBJECT_NAME_INFO;

InfoClass=2 Variable length structure based on the actual length of the object type.

typedef struct ObjectTypeInfo_t {UNICODE_STRING Ob jec tTypeName;char Unknown[0x58] ;W C H A R Ob jec tTypeNameBuf fe rE l ] ;

1 OBJECT_TYPE_ INFO, *POBJECT_TYPE_INFO;

InfoClass=3 Variable length structure based on the number of object types andactual length of each object type.

typedef struct ObjectAl !TypeInfo_t {ULONG NumberOfObjec tTypes;OBJECT_TYPE_INFO ObjectsTypelnfo[ l ] ;

} OBJECT_ALL_TYPES_INFO, *POBJECT_ALL_TYPES_INFO;

InfoClass=4

typedef struct ObjectProtect ionInfo_t {BOOLEAN blnhent;


BOOLEAN bPro tec tHand le ;} OBJECT_PROTECTION_INFO, *POBJECT_PROTECTION_INFO;

EQUIVALENT WIN32 APIGetHandlelnformation (limited support)

NtSetlnformationObjectNTSTATUS

NtSe t In fo rmat ionOb jec t (IN HANDLE hObject ,IN OBJECT_INFORMATION_CLASS InfoClass,IN P V O I D Buf fer ,IN ULONG Buf ferSize

);

NtSetlnformationObject changes the attributes of the object based on the InfoClass.

PARAMETERS


COMMENTSNone.


NtCreateEventNTSTATUS

NtCreateEvent(OUT PHANDLE hEvent,IN ACCESS_MASK AccessMask,IN POBJECT_ATTRIBUTES ObjectAttnbutes.

hObjectInfoClass

Buffer

BufferS ize

Handle to the object.Type of information to be set. This should be 4.

Pointer to the buffer that contains the information about theobject. Buffer should be in the OBJECT_PROTECTION_INFOstructure format.Size of the buffer in bytes pointed to by Buffer.


IN EVENT_TYPE EventType.IN BOOLEAN b lmt ia lS ta te

);

NtCreateEvent creates a new event object.

PARAMETERS IT?

hEvent

AccessMask

ObjectAttributes

EventType

blnitialState

Pointer to the variable that receives handle to the eventobject.

Type of access requested to the event object. This can be acombination of any of the following flags:EVENT_QUERY_STATE, EVENT_MODIFY_STATE, andEVENT_ALL_ACCESS.

Points to the OBJECT_ATTRIBUTES structure containing theinformation about the event object to be created, such asname, parent directory, objectflags, and so on.

Type of the event. This can be either of NotificationEvent,SynchronizationEvent. EventType is enumerated data typedefined in NTDEF.H file in DDK. _ . , ,,-

Specifies whether the initial state of the event will besignaled (TRUE) or not signaled (FALSE).


COMMENTSWhen the EventType is NotificationEvent and the event is signaled, all the threadswaiting on the event are released. The state of the event remains signaled unlesssomeone explicitly calls NtResetEvent or NtClearEvent.

When the EventType is SynchronizationEvent and the event is signaled, onlyone thread waiting on the event is released. All other threads continue to wait andthe state of the event is again reset back to nonsignaled.

EQUIVALENT WIN32 API - :CreateEvent

NtOpenEventNTSTATUS

NtOpenEvent(


OUT PHANDLE hEvent ,IN ACCESS_MASK D e s i r e d A c c e s s ,IN POBJECT_ATTRIBUTES Objec tAt t r ibu tes

);

NtOpenEvent opens a handle to the existing named event object.

PARAMETERS ~VH

hEvent

DesiredAccess

ObjectAttributes

Pointer to the variable that receives handle to the eventobject.

Type of access requested to the event object. This can be acombination of any of the following flags:EVENT_QUERY_STATE, EVENT_MODIFY_STATE, andEVENT_ALL_ACCESS.

Points to the OBJECT_ATrRIBUTES structure containing theinformation about the event object to be opened, such asname, parent directory, objectflags, and so on.


COMMENTS tNone.

EQUIVALENT WIN32 APIOpenEvent

NtClearEventNTSTATUS

NtClearEvent(IN HANDLE hEvent

); -

NtClearEvent sets the state of the event object to nonsignaled.

PARAMETERS

hEvent Handle to the event object returned using the NtCreateEvent orNtOpenEvent system service.



COMMENTSNone.


NtPulseEventNTSTATUS

N t P u l s e E v e n t (IN HANDLE hEventOUT OPTIONAL PULONG P r e v i o u s S t a t e ) ;

NtPulseEvent sets the state of the event object to signaled, releases one or morethreads waiting on the event, and sets the event back to nonsignaled.

PARAMETERS

hEvent

PreviousState

Handle to the event object returned using the NtCreateEvent iNtOpenEvent system service.

Pointer to the variable that receives the previous state of theevent.


COMMENTSIf the event referred to by hEvent is a NoficationEvent, then all the threads waitingon the event are released. If the event is of the type SynchronizationEvent, thenonly one thread waiting on the event is released.

EQUIVALENT WIN32 APIPulseEvent

NtQueryEventNTSTATUS

NtQueryEvenUIN HANDLE hEvent,IN EVENT_INFO_CLASS InfoClass,OUT PVOID EventlnfoBuffer,


IN ULONG Even t ln foBuf fe rS ize ,OUT PULONG BytesCopied

);

NtQueryEvent gets the information about the event object.

PARAMETERS

hEvent

InfoClass

EventlnfoBuffer

EventlnfoBufferSize

BytesCopied

Handle to the event object returned using theNtCreateEvent or NtOpenEvent system service.Information class requested for the event object. Thisshould be 0.

Pointer to the buffer that receives information about theevent object.

Size of the buffer (in bytes) pointed to byEventlnfoBuffer.

Pointer to the variable that receives the number of bytescopied into EventlnfoBuffer.


COMMENTSThe buffer pointed to by EventlnfoBufferSize must be at least 8 bytes. The informa-tion in the buffer is organized as follows:

typedef st ruct Event lnfo_t {EVENT_TYPE Even tType .LONG EventState,

} E V E N T I N F O , * P E V E N T I N F O ;


NtResetEventNTSTATUS

NtResetEvent(IN HANDLE hEvent,OUT OPTIONAL PULONG PreviousState

);

___________Appendix A: Details of System Calls with Parameters___269

NtResetEvent sets the state of the event object to nonsignaled and returns theprevious state of the event in PreviousState.

PARAMETERS


COMMENTSNone.

EQUIVALENT WIN32 APIResetEvent (does not return the previous state of the event)

NtSetEventNTSTATUS

NtSetEvent (IN HANDLE hEvent,OUT OPT IONAL PULONG Prev iousSta te ,

NtSetEvent sets the state of the event object to signaled and returns the previousstate of the event in PreviousState.

PARAMETERS

hEvent

PreviousState

Handle to the event object returned using the NtCreateEvent iNtOpenEvent system service.

Pointer to the variable that receives the previous state of theevent.


hEvent

PreviousState

Handle to the event object returned using the NtCreateEvent orNtOpenEvent system service.Pointer to the variable that receives the previous state of theevent.


COMMENTSNone.

EQUIVALENT WIN32 APISetEvent (does not return the previous state of the event)

NtCreateEventPair • 'NTSTATUS

NtCrea teEven tPa i r (OUT PHANDLE hEven tPa i r .IN ACCESS_MASK AccessMask ,IN POBJECT_ATTRIBUTES ObjectAttn butes

);

NtCreateEventPair creates a new event-pair object.

PARAMETERS

hEventPair

AccessMask

ObjectAttributes

Pointer to the variable that receives handle to the event-pairobject.

Type of access requested to the event-pair object. This can bea combination of any of the following flags:EVENT_QUERY_STATE, EVENT_MODIFY_STATE, andEVENT_ALL_ACCESS.

Points to the OBJECT_ATTRIBUTES structure containing theinformation about the event-pair object to be created, such asname, parent directory, objectflags, and so on.


COMMENTSThe event-pair object consists of two SynchronizationEvent type event objects. Theevent-pair objects are used by Windows NT 3.51 to implement Quick LPC. The firstevent is called the low event of the event-pair, and the second event is called thehigh event of the event pair.


NtOpenEventPairNTSTATUS


NtOpenEventPai r(OUT PHANDLE hEven tPa i r ,IN ACCESS_MASK DesiredAccess,IN POBJECT_ATTRIBUTES Ob jec tA t tnbu tes

);

NtOpenEventPair opens a handle to an existing named event-pair object.

PARAMETERS

hEventPair

DesiredAccess

ObjectAttributes

Pointer to the variable that receives handle to the event-pairobject.

Type of access requested to the event-pair object. This can bea combination of any of the following flags:EVENT_QUERY_STATE, EVENT_MODIFY_STATE, andEVENT_ALL_ACCESS

Points to the OBJECT_ATTRIBUTES structure containing theinformation about the event-pair object to be opened, suchas name, parent directory, objectflags, and so on.


COMMENTSNone.

c


NtSetHighEventPairNTSTATUS

NtSetHighEventPair(IN HANDLE hEven tPa i r

);

NtSetHighEventPair sets the high event of the event-pair to signaled state.

PARAMETERS

hEventPair Handle to the event-pair.

272____Appendix A: Details of System Calls with Parameters_____________


COMMENTSNone.


NtSetHi ghWai tLowEventPairNTSTATUS

N t S e t H i g h W a i t L o w E v e n t P a i r CIN HANDLE hEven tPa i r

);

NtSetHighWaitLowEventPair sets the high event of the event-pair to signaledstate and waits for the low event of the event pair to get signaled.

PARAMETERS



COMMENTSNone.


NtSetLowEventPairNTSTATUS

NtSetLowEventPa i r(IN HANDLE hEven tPa i r

);

NtSetLowEventPair sets the low event of the event-pair to signaled state.

PARAMETERS


____________Appendix A: Details of System Calls with Parameters____273


COMMENTSNone.

EQUIVALENT WIN32 API -None.

NtSetLowWaitHighEventPairNTSTATUS

N t S e t L o w W a i t H i g h E v e n t P a i r CIN HANDLE hEventPa i r

);

NtSetLowWaitHighEventPair sets the low event of the event-pair to signaled 'state and waits for the high event of the event pair to get signaled.

PARAMETERS



COMMENTSNone.

EQUIVALENT WIN3 2 APINone.

NtCreateMutantNTSTATUS

NtCreateMutant (OUT PHANDLE hMutex,IN ACCESS_MASK A c c e s s M a s k ,IN POBJECT_ATTRIBUTES Ob jec tAt t r ibu tes ,IN BOOLEAN b ln i t ia lState

) ;

NtCreateMutant creates a new mutex object.

-f


PARAMETERS

hMutex

AccessMask

ObjectAttributes

blnitialState

Pointer to the variable that receives handle to the mutexobject.Type of access requested to the mutex object.

Points to the OBJECT_ATTRIBUTES structure containing theinformation about the mutex object to be created, such asname, parent directory, objectflags, and so on.Initial state of the mutex: signaled (TRUE), not signaled(FALSE).


COMMENTSNone.

EQUIVALENT WIN32 APICreateMutex

NtOpenMutantNTSTATUS

NtOpenMutant(OUT PHANDLE hMutex,IN ACCESS_MASK Des i r edAccess ,IN POBJECT_ATTRIBUTES Ob jec tAt tnbutes

);

NtOpenMutant opens handle to an existing named mutex object.

PARAMETERS

hMutex

DesiredAccessObjectAttributes

Pointer to the variable that receives handle to the mutexobject.Type of access requested to the mutex object.Points to the OBJECT_ATTRIBUTES structure containing theinformation about the mutex object to be opened, such asname, parent directory, objectflags, and so on.



COMMENTSNone.

EQUIVALENT WIN32 APIOpenMutex

NtQueryMutantNTSTATUS

N t Q u e r y M u t a n t (IN HANDLE hMutex,IN MUTEX_INFO_CLASS InfoClass,OUT PVOID MutexInfoBuffer,IN ULONG MutexInfoBufferSize,OUT PULONG BytesCopied

NtQueryMutant queries the state information about the mutex object.

PARAMETERS

hMutex

InfoClass

MutexInfoBuffer

MutexInfoBufferSize

BytesCopied

Handle to the mutex object returned using theNtCreateMutant or NtOpenMutant system service.Information class requested for the mutex object. Thisshould be 0.

Pointer to the buffer that receives information aboutthe mutex object.

Size of the buffer (in bytes) pointed to byMutexInfoBuffer.

Pointer to the variable that receives the number ofbytes copied into MutexInfoBuffer.


COMMENTSThe buffer pointed to by MutexInfoBufferSize must be at least 8 bytes. The infor-mation in the buffer is organized as follows: , •* -


typedef struct Mutexlnfo_t (LONG MutexState;BOOLEAN bOwnedByCa l l i ngTh read ;BOOLEAN bAbandoned;USHORT Unused:

] MUTEXINFO, *PMUTEXINFO;

If the calling thread of this system service owns the mutex, thenbOwnedByCallingThread will be TRUE. MutexState is TRUE if the mutex object issignaled; otherwise, it is FALSE. bAbandoned is TRUE if the thread owning the mu-tex died without releasing the mutex.


NtReleaseMutantNTSTATUS

NtRe leaseMu tan t (IN HANDLE hMutex,OUT OPTIONAL PULONG bS igna l l ed , -

):

NtReleaseMutant releases an owned mutex object.

PARAMETERS


COMMENTSA mutex can be owned by only one thread at a time. However, the same thread canacquire the same mutex multiple times. If the thread acquires the same mutex mul-tiple times, then it has to call NtReleaseMutant that many times to set the mutex tosignaled. The bSignalled variable receives whether the mutex was set to signaled.

EQUIVALENT WIN 3 2 APIReleaseMutex

hMutex

bSingalled

Handle to the mutex object returned using the NtCreateMutant orNtOpenMutant system service.

Pointer to the variable that receives whether the mutex was set tosignaled state as a result of system service. The returned value is0 if the mutex is set to signaled state.

1Appendix A: Details of System Calls with Parameters 277

NtCreateSemaphoreNTSTATUS

NtCreateSemaphore(OUT PHANDLE hSemaphore,IN ACCESS_MASK AccessMask,IN POBJECT_ATTRIBUTES ObjectAttnbutes ,IN ULONG ImtialCount,IN ULONG MaxCount

);

NtCreateSemaphore creates a new semaphore object.

PARAMETERS

hSemaphore

AccessMask

ObjectAttributes

InitialCount

MaxCount

Pointer to the variable that receives handle to thesemaphore object.

Type of access requested to the semaphore object.

Points to the OBJECT_ATTRIBUTES structure containing theinformation about the semaphore object to be created, suchas name, parent directory, objectflags, and so on.

Initial count of the semaphore.

Maximum count of the semaphore


COMMENTSNone.

EQUIVALENT WIN 3 2 APICreateSemaphore

NtOpenSemaphoreNTSTATUS

NtOpenSemaphore(OUT PHANDLE hSemaphore ,IN A C C E S S _ M A S K D e s i r e d A c c e s s ,IN POBJECT_ATTRIBUTES ObjectAt tnbutes

);

NtOpenSemaphore opens handle to an existing named semaphore object.


PARAMETERS

hSemaphore

DesiredAccess

ObjectAttributes

Pointer to the variable that receives handle to thesemaphore object.

Type of access requested to the semaphore object.

Points to the OBJECT_ATTRIBUTES structure containing theinformation about the semaphore object to be opened, suchas name, parent directory, objectflags, and so on.


COMMENTSNone.

EQUIVALENT WIN 3 2 APIOpenSemaphore

NtQuerySemaphoreNTSTATUS

NtQuerySemaphoreCIN HANDLE hSemaphore,IN SEMAPHORE_INFO_CLASS In foC lass ,OUT P V O I D Semaphore ln foBuf fe r ,IN ULONG Semapho re ln foBu f f e rSnze ,OUT PULONG BytesCopied

):

NtQuerySemaphore queries the information about the Semaphore object.

PARAMETERS

hSemaphore

InfoClass

SemaphorelnfoBuffer

SemaphorelnfoBufferSize

Handle to the semaphore object returned using theNtCreateSemaphore or NtOpenSemaphore systemservice.Information class requested for the semaphoreobject. This should be 0.Pointer to the buffer that receives informationabout the semaphore object.Size of the buffer (in bytes) pointed to bySemaphorelnfoBuffer.


BytesCopied Pointer to the variable that receives the number ofbytes copied into SemaphorelnfoBuffer.


COMMENTSThe buffer pointed to by SemaphorelnfoBufferSize must be at least 8 bytes. The in-formation in the buffer is organized as follows:

typedef struct Semaphorelnfo_t (ULONG CurrentCount ;ULONG MaxCoun t ;

) SEMAPHORE_INFO, *PSEMAPHORE_INFO;


NtReleaseSemaphoreNTSTATUS

NtRel easeSemaphore(IN HANDLE hSemaphore,IN ULONG ReleaseCount,OUT PULONG PreviousCount , ,):

NtReleaseSemaphore releases semaphore object ReleaseCount times.

PARAMETERS

hSemaphore

ReleaseCount

PreviousCount

Handle to the semaphore object returned using theNtCreateSemaphore or NtOpenSemaphore system service.

Number of times to release semaphore.Pointer to the variable that receives the previous count of thesemaphore.



COMMENTSNone.

EQUIVALENT WIN32 APIReleaseSemaphore

NtSignalAndWaitForSirigleObject(NTSTATUS

N t S i g n a l A n d W a i t F o r S i n g l e O b j e c t (IN HANDLE hS igna lOb jec t ,IN HANDLE h W a i t O b j e c t ,IN BOOLEAN bA le r tab le ,IN PLARGE_INTEGER Timeout

);

NtSignalAndWaitForSingleObject releases the object specified by hSignalObjectand waits on the object specified by hWaitObject.

PARAMETERS

hSignalObject

hWaitObject

bAlertable

Timeout

Handle to the object to be released. This should be a handle toa mutex or event, or a semaphore object.

Handle to the object to be acquired (wait on). This should behandle to a mutex or event, or a semaphore object.

Flag specifying whether the wait is alertable.

Pointer to the variable that contains the timeout for wait.


COMMENTSNone.


NtWai tForMulti pieObjects(NTSTATUS

NtWaitForMultipleObjects(IN ULONG nObjects,IN HANDLE *ObjectHandleArray.IN WAIT_TYPE WaitType,


IN BOOLEAN bAler tab le ,IN PLARGE_INTEGER Timeout

);

NtWaitForMultipleObjects waits on multiple objects to be released (set to signaled).

PARAMETERS

nObjects

ObjectHandleArray

WaitTypebAlertable

Timeout

Number of object handles in the array pointed to byObjectHandleArray.

Pointer to the array that contains handle to the objectsto wait on.

Type of wait. This could be WaitAll or WaitAny.Flag specifying whether the wait is alertable.



COMMENTSWhen WaitAll is specified, the function returns only when all the objects referredby handles in ObjectHandleArray are in signaled state, whereas when WaitAny isspecified, the function returns when any one of the objects referred by handles inObjectHandleArray is in signaled state. The wait is abandoned when Timeout oc-curs, irrespective of WaitType.

EQUIVALENT WIN 3 2 APIWaitForMultipIeObjects

NtWaitForSingleObjectCNTSTATUS

NtWaitForSingleObject(IN HANDLE hObject,IN BOOLEAN bAlertable,IN PLARGE_INTEGER Timeout

):

NtWaitForSingleObject waits on the object to be released (set to signaled).



COMMENTSNone.

EQUIVALENT WIN32 APIWaitForSingleObject

NtCancelTimerCNTSTATUS

N t C a n c e l T i m e r (IN HANDLE hTimer,OUT PBOOLEAN pbState

);

NtCancelTimer cancels the timer and removes it from the system timer queue.

PARAMETERS


COMMENTSThe Equivalent Win32 API function does not return the state of the timer object,whereas the system service does return the state.

EQUIVALENT WIN 3 2 APICancelTimer

PARAMETERS

hObjectbAlertable

Timeout

Handle to the object to wait on.

Flag specifying whether the wait is alertable.


hTimer

pbState

Handle to the timer object.

Pointer to the variable that receives the state of the timer at thetime of cancellation. Receives TRUE if the timer is signaled andFALSE if the timer is not signaled.


TimerlnfoBuffer

TimerlnfoBufferSize

BytesCopied

Pointer to the buffer that receives information about thetimer object.

Size of the buffer (in bytes) pointed to byTimerlnfoBuffer.

Pointer to the variable that receives the number of bytescopied into TimerlnfoBuffer. , •?. ̂ j


COMMENTSThe buffer pointed to by TimerlnfoBuffer must be exactly 16 bytes. The informationin the buffer is organized as follows:

typedef struct Timerlnfo_t {LARGE_INTEGER DueTime;ULONG T imerSta te ;ULONG Unused;

) T I M E R I N F O , *PT IMERINFO;

TimerState member is 0 if the timer is in not signaled state and 1 if the timer isin signaled state. DueTime is the time remaining for the timer to expire.


NtQueryTimerResolutionNTSTATUS

NtQueryT imerReso lu t ion(OUT PULONG MaxResolut ion,OUT PULONG M i n R e s o l u t i o n .OUT PULONG SystemResolu t ion

NtQueryTimerResolution gets the information about the granularity of the clockinterrupt.

PARAMETERS

MaxResolution Pointer to the variable that receives the maximumresolution (in nanoseconds) supported on the processor.


PARAMETERS

hObject Handle to the object to wait on.

bAlertable Flag specifying whether the wait is alertable.

Timeout Pointer to the variable that contains the timeout for wait.


COMMENTSNone.

EQUIVALENT WIN 3 2 APIWaitForSingleObject

NtCancelTimeKNTSTATUS

NtCance lT imer (IN HANDLE hTimer ,OUT PBOOLEAN pbState

NtCancelTimer cancels the timer and removes it from the system timer queue.

PARAMETERS

hTimer Handle to the timer object.

pbState Pointer to the variable that receives the state of the timer at thetime of cancellation. Receives TRUE if the timer is signaled andFALSE if the timer is not signaled.


COMMENTSThe Equivalent Win32 API function does not return the state of the timer object,whereas the system service does return the state.

EQUIVALENT WIN 32 APICancelTimer

hTimer

pb State


Pointer to the variable that receives the state of the timer at thetime of cancellation. Receives TRUE if the timer is signaled andFALSE if the timer is not signaled.


NtCreateTimerNTSTATUS

NtCreateTimer(OUT PHANDLE phTimer,IN ACCESS_MASK A c c e s s M a s k .IN POBJECT_ATTRIBUTES Ob jec tA t t r i bu tes ,IN TIMER_TYPE TimerType

);

NtCreateTimer creates a new timer object.

PARAMETERS

phTimer

AccessMask

ObjectAttributes

TimerType

Pointer to the variable that receives handle to the timerobject.

Type of access requested to the timer object.

Points to the OBJECT_ATTRIBUTES structure containing theinformation about the timer object to be created, such asname, parent directory, objectflags, and so on.

Type of the timer. It can be either NotificationTimer orSynchronizationTimer. -0 , _, , ,

H


COMMENTSWhen a timer of type NotificationTimer is signaled, all threads waiting on the timerobject are released and the timer remains in signaled state. When a timer of typeSynchronizationTimer is signaled, only one thread waiting on the timer object isreleased. All other threads waiting for the same timer object continue to wait, andthe timer object is again set back to not signaled. The Equivalent Win32 API of thissystem service enables you to create timers of SynchronizationTimer type only,whereas the system service enables you to create both types of timers.

EQUIVALENT WIN32 API , ._-CreateWaitableTimer

NtOpenTimerNTSTATUS r,

NtOpenTimer(OUT PHANDLE phTimer,IN ACCESS_MASK AccessMask,


IN POBJECT_ATTRIBUTES Ob jec tA t t r i bu tes);

NtOpenTimer opens handle to an existing named timer object.

PARAMETERS

phTimer

AccessMask

ObjectAttributes

Pointer to the variable that receives handle to the timerobject.

Type of access requested to the timer object.

Points to the OBJECT_ATTRIBUTES structure containing theinformation about the timer object to be opened, such asname, parent directory, objectflags, and so on.


COMMENTSNone.

EQUIVALENT WIN32 APIOpenWaitableTimer

NtQueryTimerNTSTATUS

NtQueryT imer (IN HANDLE hTimer,IN TIMER_INFO_CLASS In foClass .OUT PVOID T imer ln foBuf fe r ,IN ULONG T i m e r l n f o B u f f e r S n z e ,OUT PULONG BytesCop ied

);

NtQueryTimer queries the state information about the timer object.

PARAMETERS

hTimer

InfoClass


Information class requested for the timer object. Thisshould be 0.


MinResolution

SystemResolution

Pointer to the variable that receives the minimumresolution (in nanoseconds) supported on the processor.

Pointer to the variable that receives resolution (innanoseconds) currently used by the system.


COMMENTSNone.


NtSetTimerNTSTATUS

NtSetTimer(IN HANDLE hTimer,IN PLARGE_INTEGER pDueTime,IN PTIMERAPCROUTINE pfnCompletionRoutine OPTIONAL,IN DWORD pfnCompletionRoutineArg OPTIONAL,IN BOOLEAN bResume,IN LONG Period.OUT PBOOLEAN bTimerState

NtSetTimer activates the timer specified by hTimer.PARAMETERShTimer

DueTime

pfnCompletionRoutine


Time at which the timer will be set to signaled.Positive values indicate absolute time. Negative valuesrepresent time relative to the current system time. Thedue time is specified in terms of 100ns units.

Pointer to the function that will be called when thetimer expires. The completion routine should bedefined according to following prototype. Thisparameter is optional and can be NULL.

VOID


(APIENTRY *PTIMERAPCROUTINE)(LPVOID IpArgToCompletionRoutine,DWORD dwTimerLowValue,DWORD dwTimerHighValue

);

pfnCompletionRoutineArg

bResume

Period

bTimerState

Optional argument that will be passed topfnCompletionRoutine. This parameter can beNULL.

Flag specifying whether to set the system insuspended power conservation mode when thetimer expires. This parameter is ignored if theplatform does not support this feature.

Specifies the time in milliseconds by which thetimer will be reactivated once the timer elapses. Ifthis parameter is 0, the timer is signaled onlyonce.

Pointer to the variable that receives the presentstate of the timer (TRUE for signaled and FALSEfor not signaled).


COMMENTSThe Equivalent Win32 API of the call does not return the present state of the timer,whereas the system service does return this information.

EQUIVALENT WIN32 APISetWaitableTimer

NtSetTimerResolutionNTSTATUS

NtSetTimerResolution(IN ULONG NewResolution,IN BOOLEAN bSet,OUT PULONG pResolutionSet

)f

NtSetTimerResolution changes the timer resolution for the clock interrupt.


PARAMETERS

NewResolution

bSet

pResolutionSet

Newly requested resolution for the timer in units of 100ns.The acceptable values for X86 platforms are between 1 and10 milliseconds.

Flag specifying whether to set new resolution (TRUE) or torestore previously set resolution (FALSE). The NewResolutionparameter is ignored if this parameter is FALSE.

Pointer to the variable that receives the timer resolution setby the system.


COMMENTSNone.


NtQueryPerformanceCounterNTSTATUS

NtQueryPerformanceCounter(OUT PLARGE_INTEGER pPer fo rmanceCount .OUT PLARGE_INTEGER pFrequency v -

):

NtQueryPerformanceCounter retrieves the current value and frequency of thehigh-resolution performance counter if it exists.

PARAMETERS

pPerformanceCount

pFrequency

Pointer to the variable that receives the currentperformance counter value. If the hardware does notsupport a high-resolution performance counter, thevalue will be set to 0.

Pointer to the variable that receives the frequency ofthe high-resolution performance counter per second. Ifthe hardware does not support a high-resolutionperformance counter, the value will be set to 0.



COMMENTSNone.

EQUIVALENT WIN32 APIQueryPerformanceCounter, QueryPerformanceFrequency

NtQuerySytemTimeNTSTATUS

NtQuerySystemTirne(OUT PLARGE_INTEGER pSystemTime

);

NtQuerySystemTime retrieves the number of 100 nanosecond intervals elapsedsince January 1, 1601.

PARAMETERS

pSystemTime Pointer to the variable that receives the number of 100nanosecond intervals elapsed since January 1, 1601. The timeis expressed as GMT.


COMMENTSNone.

EQUIVALENT WIN 3 2 APIGetSystemTime

NtSetSytemTimeNTSTATUS

NtSetSystemTime(IN PLARGE_INTEGER pNewSystemTime,OUT PLARGE^INTEGER pOldsystemTime OPTIONAL

);

NtSetSystemTime sets the system time.


PARAMETERS

pNewSystemTime

pOldSystemTime

Pointer to the variable that contains the system timeexpressed in GMT.

Pointer to the variable that receives the present systemtime in GMT. This parameter is optional.


COMMENTSNone.

EQUIVALENT WIN32 APISetSystemTime

NtGetTickCountULONG

NtGetT ickCount ();

NtGetTickCount returns the number of milliseconds that have elapsed sinceWindows started.

PARAMETERSNone.

RETURN VALUEReturns the milliseconds elapsed.

COMMENTSNone.

EQUIVALENT WIN32 APIGetTickCount

NtAddAtomNTSTATUS

NtAddAtom(IN PWCHAR pStnng,IN ULONG pStringLength,OUT PATOM pAtom

):


NtAddAtom adds the character string to the global atom table and returns anatom value.

PARAMETERS

pString

pStringLength

pAtom

Pointer to the wide character string to be added to the globalatom table.

Length of the string pointed to by pString.

Pointer to the variable that receives the atom valuecorresponding to the string.

r-


COMMENTSThe memory for the atom table is allocated by WIN32K.SYS. The system servicegets the pointer to this memory using a callout established by WIN32K.SYS. Thiscallout is established by WIN32K.SYS using the PspEstablishWin32Callouts func-tion from NTOSKRNL. The system service uses this callout to get the pointer to theglobal atom table and then manipulates the atom table using its internal functions,such as RtlAddAtomToAtomTable. These internal functions also reside in user-mode DLL NTDLL.DLL. These NTDLL routines are used to support local atom tables(per process). It seems that these internal functions in NTOSKRNL and NTDLL areshared at the source code level because their binary equivalents are identical. Theuser-mode API call GlobalAddAtom uses this system service. The AddAtom func-tion does not call this system service; instead, it manipulates the local atom tableusing internal functions in NTDLL.

The second parameter (pStringLength) is introduced starting from Windows 2000.Previous versions of Windows NT take only two parameters pString and pAtom.

EQUIVALENT WIN32 APIGlobalAddAtom

NtQuerylnformationAtomNTSTATUS

NtQuerylnformationAtonHIN ATOM Atom,IN ATOM_INFO_CLASS AtomlnfoClass,OUT PVOID AtomlnfoBuffer,IN ULONG AtomlnfoBufferLength,OUT PULONG BytesCopied

);


NtQuerylnformationAtom returns information about specific/all atom objects inthe global atom table.

PARAMETERS

Atom

AtomlnfoClass

AtomlnfoBuffer

AtomlnfoBufferLength

BytesCopied

The atom ID returned by NtAddAtom, NtFindAtom,and so on.

The type of information requested. This value can beOor 1.Pointer to the buffer that receives the informationabout the atoms.

Size of the buffer in bytes pointed to byAtomlnfoBuffer.

Pointer to the variable that receives the number ofbytes copied into AtomlnfoBuffer. However, thisvariable is not set by the system service.


COMMENTSIf the AtomlnfoClass is 1, then the Atom field is ignored and the system service re-turns information about all the objects in the global atom table. If the AtomlnfoClassis 0, then the system service returns the information about the atom specified by theAtom parameter.

For AtomlnfoClass 0, the data in AtomlnfoBuffer is laid out as follows:

typedef struct A tomln foS ing le {W O R D Re fe renceCount ;WORD Unknown;WORD A tomSt r ingLeng th ;W C H A R AtomStnng[ l ] ;) A T O M I N F O S I N G L E , *P ATOMINFOSINGLE;

The size of data returned varies based on the size of the atom string.For AtomlnfoClass 1, the data in AtomlnfoBuffer is laid out as follows:

typedef struct A tom ln foA l1 (DWORD T o t a l N u m b e r O f E n t n e s I n G l o b a l A t o m T a b l e ;ATOM A t o m V a l u e s [ l ] ;} ATOMINFOALL , * P A T O M I N F O A L L ;


The size of data returned varies based on the number of entries in the globalatom table.

The user-mode API call can get the atom string corresponding to the atom ID.Other information is available only through this system service.

EQUIVALENT WIN32 APIGlobalGetAtomName

NtFindAtomNTSTATUS

NtFindAtomCIN PWCHAR pString,IN ULONG pStr ingLength,OUT PATOM pAtom

);

NtFindAtom retrieves the atom corresponding to the specified string in theglobal atom table.

PARAMETERS

pString

pStringLength

pAtom

Pointer to the wide character string to be searched in theglobal atom table.

Length of the string pointed to by pString.Pointer to the variable that receives the atom valuecorresponding to the string.


COMMENTSThe second parameter (pStringLength) is introduced starting from Windows 2000.Previous versions of Windows NT take only two parameters, pString and pAtom.

EQUIVALENT WIN32 APIGlobalFindAtom

NtDeleteAtomNTSTATUS

NtDeleteAtom(IN ATOM Atomld.

);


NtDeleteAtom decrements the reference count for the specified atom. If the ref-erence count reaches zero, it deletes the atom from the global atom table.

PARAMETERS

Atomld Atom to be deleted.


COMMENTSNone.

EQUIVALENT WIN 3 2 APIGlobalDeleteAtom

INtCreateKeyNTSTATUS

NtCrea teKey (OUT PHANDLE phKey,IN ACCESS_MASK D e s i r e d A c c e s s ,IN POBJECT_ATTRIBUTES Ob jec tA t t r ibu tes ,IN ULONG T i t le lndex.IN PUNICODE_STRING C l a s s ,IN ULONG Crea teOp t i ons ,OUT PULONG pD ispos i t i on

):

NtCreateKey creates a new Registry key or opens the Registry key if it is alreadypresent.

PARAMETERS

phKey

AccessMask

Pointer to the variable that receives handle to the Registrykey object.

Type of access requested to the Registry key object. Thiscould be KEY_QUERY_VALUE, KEY_SET_VALUE,KEY_CREATE_SUBKEYS, KEY_ENUMERATE_SUB_KEYS,KEY_NOTIFY, or KEY_CREATE_LINK, or set of standardrights such as KEY_READ, KEY_WRiTE, KEY_EXECUTE, orKEY_ALL_ACCESS.


ObjectAttributes

Titlelndex

Class

CreateOptions

pDisposition

Points to the OBJECTATTRIBUTES structure containing theinformation about the key to be created, such as name,parent directory, objectflags, and so on.This parameter should be set to 0.

Points to the object class of the key. This parameter isignored if the key already exists. ^ ___Specifies the options to be applied while creating the key.This could be REG_OPTIONAL_VOLATILE,REG_OPTION_NON_VOLATILE,REG_OPTION_CREATE_LINK, orREG_OPTION_BACKUP_RESTORE.

Pointer to the variable that receives whether the new key iscreated (REG_CREATED_NEW_KEY) or the existing key isopened (REG_OPENED_EXISTING_KEY).


COMMENTSNone.

EQUIVALENT WIN 3 2 APIRegCreateKey, RegCreateKeyEx

NtOpenKeyNTSTATUS

NtOpenKey(OUT PHANDLE phKey,IN ACCESS_MASK Desi redAccess.IN POBJECT_ATTRIBUTES Objec tAt tnbutes

); j

NtOpenKey opens an existing Registry key.

PARAMETERS

phKey Pointer to the variable that receives handle to the Registrykey object.


AccessMask

ObjectAttributes

Type of access requested to the Registry key object. Thiscould be KEY_QUERY_VALUE, KEY_SET_VALUE,KEY_CREATE_SUBKEYS, KEY_ENUMERATE_SUB_KEYS,KEY_NOTIFY, or KEY_CREATE_LINK, or set of standardrights such as KEY_READ, KEY_WRITE, KEY_EXECUTE, orKEY_ALL_ACCESS.

Points to the OBJECTATTRIBUTES structure containing theinformation about the key to be opened, such as name,parent directory, objectflags, and so on.

T


COMMENTSNone.

EQUIVALENT WIN32 APIRegOpenKey, RegOpenKeyExNtDeleteKeyNTSTATUS

NtDeleteKey(IN HANDLE hKey

);NtDeleteKey deletes the Registry key specified by hKey.

PARAMETERS

hKey Handle to the open Registry key.\


COMMENTSNone.

EQUIVALENT WIN 3 2 APIRegDeleteKey

NtDeleteValueKeyNTSTATUS

____________Appendix A: Details of System Calls with Parameters___297

N t D e l e t e V a l u e K e y (IN HANDLE hKey,IN PUNICODE_STRING pVa lueName

);

NtDeleteValueKey deletes the value specified by pValueName under the Registrykey specified by hKey.

PARAMETERS

hKey Handle to the open Registry key.

PValueName Pointer to the Unicode string containing the name of the valueto be deleted. If this parameter is an empty string, the systemservice deletes the default unnamed value under the Registry key.


COMMENTSNone.

EQUIVALENT WIN32 APIRegDeleteValue

NtEnumerateKeyNTSTATUS

NtEnumerateKey(IN HANDLE hKey,IN ULONG Index,IN KEY_INFORMATION_CLASS KeyIn foClass ,OUT PVOID KeylnfoBuffer,IN ULONG KeylnfoBuf ferLength,OUT PULONG BytesCopied

);

NtEnumerateKey retrieves information about subkeys of an existing open key.

PARAMETERS

hKey

Index

Handle to the open Registry key.

Zero-based index of the subkey for which theinformation is to be retrieved.


KeylnfoClass

KeylnfoBuffer

KeylnfoBufferLength

BytesCopied

The information class requested for the subkey. This canbe KeyBasicInformation, KeyNodelnformation, orKeyFullInformation.

Pointer to the buffer that receives the informationabout the subkey.

Size of the buffer in bytes pointed to by KeylnfoBuffer.

Pointer to the variable that receives the number ofbytes copied into KeylnfoBuffer.

RETURN VALUEReturns STATUS_SUCCESS on success, STATUS_NO_MORE_ENTRIES when all theentries are over, and an appropriate error code on failure.

COMMENTSThe layout of the buffer returned is based on the information class; that is, if theKeylnfoClass is KeyBasicInformation, the information in the KeylnfoBuffer is ac-cording to the structure definition KEY_BASIC_INFORMATION. If the KeylnfoClassis KeyNodelnformation, the KeylnfoBuffer is according to the structure definitionof KEY_NODE_INFORMATION. And if the KeylnfoClass is KeyFullInformation, theKeylnfoBuffer is according to the structure definition of KEY_FULL_INFORMA-TION. If the passed KeylnfoBuffer is not enough to hold the requested information,the system service returns the number of bytes required to hold the information inthe BytesCopied variable.

EQUIVALENT WIN 3 2 APIRegEnumKey, RegEnumKeyEx

NtEnumerateVal ueKeyNTSTATUS

N t E n u m e r a t e V a l u e K e y (IN HANDLE hKey,IN ULONG Index,IN K E Y _ V A L U E _ I N F O R M A T I O N _ C L A S S K e y V a l u e l n f o C l a s s ,OUT P V O I D K e y V a l u e l n f o B u f f e r ,IN ULONG KeyVa lue ln foBu f fe rLeng th ,OUT PULONG BytesCop ied

);

NtEnumerateValueKey retrieves information about the value entries of an exist-ing open key.


PARAMETERS

hKey

Index

KeyValuelnfoClass

KeyValuelnfoBuffer

KeyValuelnfoBufferLength

BytesCopied

Handle to the open Registry key.

Zero-based index of the valuename for which theinformation is to be retrieved.

The information class requested for the valuename.This can be KeyValueBasicInformation,KeyValueFullInformation, orKeyValuePartiallnformation.

Pointer to the buffer that receives the informationabout the valuename.

Size of the buffer in bytes pointed to byKeyValuelnfoBuffer.

Pointer to the variable that receives the number ofbytes copied into KeyValuelnfoBuffer.

RETURN VALUEReturns STATUS_SUCCESS on success, STATUS_NO_MORE_ENTRIES when all theentries are over, and an appropriate error code on failure.

COMMENTS - - - 'The layout of the buffer returned is based on the information class; that is, if theKeyValuelnfoClass is KeyValueBasicInformation, the information in theKeyValuelnfoBuffer is according to the structure definition of KEY_VALUE_BASIC_IN-FORMATION. If the KeylnfoClass is KeyValueFullInformation, the KeyValuelnfoBufferis according to the structure definition of KEY_VALUE_FULL_INFORMATION. And ifthe KeyValuelnfoClass is KeyPartiallnformation, the KeyValuelnfoBuffer is accordingto the structure definition of KEY_VALUE_PARTIALJNFORMATION. If the passedKeyValuelnfoBuffer is not enough to hold the requested information, the system ser-vice returns the number of bytes required to hold the information in the BytesCopiedvariable.

EQUIVALENT WIN32 APIRegEnumValue

NtFlushKeyNTSTATUS

N t F l u s h K e y (IN HANDLE hKey

) ;


NtFlushKey syncs (commits) the data under the specified Registry key to disk. -

PARAMETERS

hKey Handle to the open Registry key.


COMMENTSThis function is very costly because it synchronously flushes the changes made tothe Registry. This system service should be used only when it is absolutely neces-sary to do so. The system automatically flushes the Registry periodically.

EQUIVALENT WIN32 APIRegFlushKey

NtlnitializeRegistryNTSTATUS

NtIn i t ia lnzeRegistry(IN DWORD UnknownParam

);

NtlnitializeRegistry initializes the Registry.

PARAMETERS

UnknownParam This parameter does not seem to be used.


COMMENTSThis function is called by SMSS.EXE during the early boot sequence. The single pa-rameter passed to it does not seem to be used. SMSS passes 0 for this parameter.This function does the work only the first time it is called. NTOSKRNL sets its inter-nal variable CMFirstTime to 0 once the first call to this service is made. NTOSKRNLdoes the registry initialization only if CMFirstTime is set to 1.



NtNotifyChangeKeyNTSTATUS

NtNot i fyChangeKeyCIN HANDLE hKey,IN HANDLE hEvent,IN PIO_APC_ROUTINE ApcRout ine,IN PVOID ApcRoutineContext,IN ULONG Not i fyFi l ter ,IN BOOLEAN bWatchSubt ree .OUT P V O I D RegChangesDa taBu f fe r ,IN ULONG RegChangesDataBu f fe rLeng th ,IN BOOL bAynchronous

);

NtNotifyChangeKey monitors the Registry changes under the specified Registrykey and optionally calls an APC routine when the changes occur.

PARAMETERS

hKey

hEvent

ApcRoutine

ApcRoutineContextNotifyFilter

bWatchSubtree

RegChangesDataBuffer

Handle to the Registry key to monitor.Changes under this Registry key aremonitored.Handle to the event object. This parameter isignored if bAsynchronous is FALSE. IfbAsynchronous is TRUE, this event object issignaled when the changes occur under thespecified Registry key.

Pointer to the function that gets called whenthe Registry changes occur.

Parameter to be passed to the ApcRoutine.

Set of flags specifying what changes tomonitor. This could be all or any combinationof the following:REG_N01TFY_CHANGE_NAME,REG_NOTIFY_CHANGE_ATrRIBUTES,REG_NOTIFY_CHANGE_LAST_SET, andREG_NOTIFY_CHANGE_SECURlTY.

Flag specifying whether to monitor allsubkeys under the specifed hKey also.

Pointer to the buffer that receives the changesmade to the Registry.


RegChangesDataBufferLength

bAsynchronous

Size of the buffer pointed to byRegChangesDataBuffer.

Flag specifying whether the monitoring shouldbe done synchronously or asynchronously.


COMMENTSThis function is similar to NtNotifyChangeDirectoryFile. It monitors the Registrytree instead of the file system tree. Presently, the RegChangesDataBuffer andRegChangesDataBufferLength parameters are ignored. Hence, the notification isgiven only when the Registry changes occur. The function does not return inRegChangeDataBuffer the actual changes made in the Registry. This is differentfrom NtNotifyChangeDirectoryFile, which returns actual changes also. We feel thatthe functionality to provide the actual list of changes to the Registry will be addedin future versions of Windows NT. For now, this system service does not provideany more information than its Equivalent Win32 API RegNotifyChangeKeyValueexcept for the fact that you can specify a callback function that gets called whenthe changes occur in the Registry.

EQUIVALENT WIN 3 2 APIRegNotifyChangeKeyValue

NtQueryKeyNTSTATUS

NtQueryKey(IN HANDLE hKey,IN KEY_INFORMATION_CLASS Key In foC lass .OUT PVOID Key ln foBuf fer ,IN ULONG KeylnfoBuf ferLength,OUT PULONG BytesCopied

):

NtQueryKey retrieves information about the specified key.

PARAMETERS

hKey Handle to the key for which the information isrequested.


KeylnfoClass*

KeylnfoBuffer

KeylnfoBufferLength

BytesCopied

The information class requested for the key. This canbe KeyBasicInformation, KeyNodelnformation, orKeyFullInformation.

Pointer to the buffer that receives the informationabout the key.

Size of the buffer (in bytes) pointed to byKeylnfoBuffer.

Pointer to the variable that receives the number ofbytes copied into KeylnfoBuffer.


COMMENTSLook at the Comments section for the NtEnumerateKey function.

EQUIVALENT WIN32 APIRegQueryKey

NtQueryMulti pieValueKeyNTSTATUS

NtQueryMult ip!eValueKeyCIN HANDLE hKey,IN OUT PKEY_VALUE__ENTRY ValueNameArray,IN DWORD nElementsValueNameArray, tOUT PVOID V a l u e D a t a B u f f e r ,IN OUT PULONG Va lueDataBuf ferS ize .OUT PULONG SizeRequ i red

);

NtQueryMultipleValueKey retrieves the data and the type of information aboutthe specified values.

PARAMETERS

hKey

ValueNameArray

Handle to the key to which the values belong.

Pointer to the array of KEY_VALUE_ENTRYstructures. The first member of this should befilled in with a pointer to the Unicode stringrepresentation of the valuename whoseinformation is to be retrieved. The othermembers of this structure are returned upon thesuccessful execution of the service.


nElementsValueNameArray

ValueDataBuffer

ValueDataBufferSize

*

SizeRequired

The number of KEY_VALUE_ENTRY type entriesin the array pointed to by ValueNameArray.Pointer to the buffer that receives the dataassociated with the values specified inValueNameArray. The DataOffset field inKEY_VALUE_ENTRY points to offsets in thisbuffer upon successful execution of the service.

Pointer to the variable that contains the size ofthe buffer pointed to by ValueDataBuffer. Thisvariable receives the number of bytes actuallycopied to the ValueDataBuffer upon successfulexecution of the service.

Pointer to the variable that receives the size ofthe buffer required to fulfill the query request.This member can be used to allocate appropriatespace if the function fails withSTATUS BUFFER OVERFLOW.


COMMENTSNone.

EQUIVALENT WIN32 APIRegQueryMultipleValues

NtQueryValueKeyNTSTATUS

NtQue ryVa lueKey (IN HANDLE hKey,IN PUNICODE_STRING uVa lueName ,IN KEY_VALUE_INFORMATION_CLASS K e y V a l u e l n f o C l a s s ,OUT PVOID KeyValue ln foBuf fer ,IN ULONG KeyValue ln foBuf ferLength ,OUT PULONG BytesCopied

);

NtQueryValueKey retrieves information about the specified value.


PARAMETERS

hKey

uValueName

KeyValuelnfoClassn

KeyValuelnfoBuffer

KeyValuelnfoBufferLength

BytesCopied

Handle to the key to which the valuenamebelongs.

Pointer to the Unicode string representation of thevaluename.The information class requested for the Registryvalue. This can be KeyValueBasicInformation,KeyValueFullInformation, orKeyValuePartiallnformation.

Pointer to the buffer that receives the informationabout the Registry value.

Size of the buffer (in bytes) pointed to byKeyValuelnfoBuffer.

Pointer to the variable that receives the number ofbytes copied into KeyValuelnfoBuffer.


COMMENTSLook at the Comments section for the NtEnumerateValueKey function.

EQUIVALENT WIN32 API /RegQueryValueEx

NtReplaceKeyNTSTATUS

NtRep laceKey(IN POBJECT_ATTRIBUTES NewHiveFi le,IN HANDLE hKey,IN POBJECT_ATTRIBUTES BackupHiveFi le

):

NtReplaceKey replaces the hive file backing up the existing key with a newhive file.

PARAMETERS

NewHiveFile Pointer to the object attribute structure describing the newhive file.


hKeyBackupHiveFile

Handle to root key of the hive.

Pointer to the object attribute structure describing the backuphive file that receives the copy of the existing Registry hive.


COMMENTSNone.

EQUIVALENT WIN 3 2 APIRegReplaceKey

NtSaveKeyNTSTATUS

NtSaveKey(IN HANDLE hKey,IN HANDLE hF i le

);

NtSaveKey backs up the Registry contents under the specified Registry key intoa file.

PARAMETERS


COMMENTSNone.

EQUIVALENT WIN 3 2 APIRegSaveKey

NtRestoreKeyNTSTATUS

NtRestoreKey(

hKey

hFile

Handle to the key to backup.

Handle to the file in which the contents of the Registry are to besaved.


IN HANDLE hKey,IN HANDLE hFile,IN ULONG F lags

) ;

NtRestoreKey restores the Registry contents under the specified Registry keyfrom the specified file. . . M_

PARAMETERS

hKey Handle to the key to restore.

hFile Handle to the file containing the Registry data.

Flags This parameter can be REG_WHOLE_HTVE_VOLATILE.


COMMENTSNone.

EQUIVALENT WIN32 API . ,RegRestoreKey

NtSetlnformationKeyNTSTATUS

NtSetInformat ionKey(IN HANDLE hKey.IN KEY_SET_INFORMATION_CLASS KeySet In foClass ,IN PKEY_WRITE_TIME_INFORMATION p ln foBuf fer .IN ULONG plnfoBufferLength

) ;

NtSetlnformationKey sets the last write time of the specified key.

PARAMETERS

hKey

KeySetlnfoClassplnfoBuffer

plnfoBufferLength

Handle to the Registry key.This parameter should be KeyWriteTimelnformation.

Pointer to the structure of typeKEY_WRITE_TIME_INFORMATION. The structure containsonly one large integer field called LastWriteTime.

Size of the buffer pointed to by plnfoBuffer.



COMMENTSNone.


NtSetValueKeyNTSTATUS

N t S e t V a l u e K e y (IN HANDLE hKey,IN PUNICODE_STRING uVa lueName,IN ULONG T i t le lndex,IN ULONG V a l u e T y p e ,IN PVOID p V a l u e D a t a ,IN ULONG p V a l u e D a t a L e n g t h

);

NtSetValueKey sets/creates a value entry under the specified Registry key.

PARAMETERS

hKey

uValueName

Titlelndex

ValueType

pValueData

pValueDataLength

Handle to the Registry key to which the value is associatedwith.

Pointer to the Unicode string containing the valuename.This parameter should be 0.

Data type for the value. This could be REG_BINARY,REG_DWORD, and so on. For all types, refer to thedocumentation of the RegSetValueEx call.

Pointer to the buffer containing the data to be associatedwith the valuename.

Size of the data buffer pointed to by pValueData.


COMMENTSNone.


EQUIVALENT WIN32 APIRegSetValueEx

NtLoadKeyNTSTATUS

NtLoadKey(IN POBJECT_ATTRIBUTES KeyNameAttnbutes ,IN POBJECT_ATTRIBUTES HiveFi1eNameAttnbutes

);

NtLoadKey loads the specified Registry hive on top of the existing Registry key.

PARAMETERS

KeyNameAttributes

HiveFileNameAttributes

Pointer to the object attributes structure describingthe key on which the hive is to be loaded.Pointer to the object attributes structure describingthe hive filename.


COMMENTSRefer to the documentation of the NtLoadKey2 system service. NtLoadKey inter-nally calls NtLoadKey2 by specifying the third parameter flags as 0.

EQUIVALENT WIN32 APIRegLoadKey

NtLoadKey2NTSTATUS

N t L o a d K e y 2 (IN POBJECT_ATTRIBUTES KeyNameAt t r ibu tes ,IN POBJECT_ATTRIBUTES H iveF i1eNameAt t r i bu tes ,IN ULONG F lags

) ;

NtLoadKey2 loads the specified Registry hive on top of the existing Registry keyand applies the specified flags for the loaded hive.

4


PARAMETERS

KeyNameAttributes

HiveFileNameAttributes

Flags

Pointer to the object attributes structure describingthe key on which the hive is to be loaded.

Pointer to the object attributes structure describingthe hive filename.

The only flag value allowed is 0x00000004.


COMMENTSThe hive loaded with NtLoadKey is periodically synched by the system workerthread. With NtLoadKey2, you can specify a flag 0x00000004 that turns off this pe-riodic synching. Hence, if this flag is specified, the hive is not synched periodicallyby the system worker thread. Internally, the system maintains a linked list of loadedhives data structures. One member of this data structure contains the informationabout the flags passed while loading the hive. The system worker thread periodi-cally walks the list of loaded hives and, based on this member, decides whether tosynch the hive or not.


NtUnloadKeyNTSTATUS

NtUn loadKeyCIN POBJECT_ATTRIBUTES KeyNameAt t r ibutes

): . -

NtUnloadKey unloads the hive loaded on top of the existing key (usingNtLoadKey2/NtLoadKey).

PARAMETERS


KeyNameAttributes Pointer to the object attributes structure describing thekey from which the hive is to be unloaded.


COMMENTSNone.

EQUIVALENT WIN32 APIRegUnloadKey

NtAlertResumeThread ' «-_-NTSTATUS

N t A l e r t R e s u m e T h r e a d (IN HANDLE hThread,OUT PBOOLEAN pbResumed

);

NtAlertResumeThread resumes the thread or alerts the thread that is in alertablewait state.

PARAMETERS

hThread

pbResumed

Handle to the thread.

Pointer to the variable that receives whether the thread wasactually resumed or it was already running (TRUE if the threadwas in the suspended state at the time of function call, otherwiseFALSE).


COMMENTSIf the thread is in alertable wait state, then this function takes the thread out of thewait state with the status of STATUS_ALERTED. However, the user mode alertablewait calls such as WaitForSingleObject and SleepEx put the thread again back inalertable wait state if they return with the status of STATUS_ALERTED.

EQU1VALEMT WIN32 APINone.

NtAlertThreadNTSTATUS

NtA le r tTh read (IN HANDLE hThread

);

NtAlertThread alerts the thread that is in alertable wait state.


PARAMETERS

hThread Handle to the thread.


COMMENTSSee the Comments section for NtAlertResumeThread.


NtTestAlertNTSTATUS

NtTes tA le r t ();

NtTestAlert checks whether the current thread has any pending alerts.

PARAMETERSNone.


COMMENTSNone.


NtCreateProcessNTSTATUS

NtCreateProcess(OUT PHANDLE phProcess,IN ACCESS_MASK AccessMask,IN POBJECT_ATTRIBUTES ObjectAttributes,IN HANDLE hParentProcess,IN BOOLEAN blnheritHandles ,IN HANDLE hSection,IN HANDLE hDebugPort,IN HANDLE hExceptnonPort

);

NtCreateProcess creates a new process object.


PARAMETERS

phProcess

AccessMask

ObjectAttributes

hParentProcess

blnheritHandles

hSection

hDebugPorthExceptionPort

Pointer to the variable that receives handle to the processobject.Type of access requested to the process object.

Points to the OBJECTATTRIBUTES structure containingthe information about the process object to be created,such as name, parent directory, objectflags, and so on.

Handle to the process object that this process will be achild of.

The flag specifying whether the handles from the parentprocess described by hParentProcess are to be inherited bythis process.

Handle to the section object created for the executable file.

Handle to the debug port for the process.

Handle to the exception port for the process.


COMMENTSThe system service enables you to specify a different parent process for the processto be created, whereas in case of the user-mode CreateProcess call, the caller of thefunction becomes the parent of the process being created. The reason for this is thatCreateProcess, by default, passes OxFFFFFFFF as the parent process handle. We feelthat the provision to specify a different process handle than that of the caller is re-quired in the POSIX subsystem where the fork call is actually implemented by thesubsystem. Also note that unlike all other synchronization objects, such as mutexand semaphore, the CreateProcess call does not enable you to name the process ob-ject, whereas the system service enables you to specify the name of the process ob-ject in the ObjectAttributes structure. This way, you can open the process using itsname instead of the process ID that is required for the OpenProcess call. Further, thesystem service enables you to specify specific port handles for DebugPort andExceptionPort, whereas the CreateProcess call does not allow you to do so. TheCreateProcess call, by default, passes NULL handles to these two parameters. WhenNULL handles are passed, the NtCreateProcess call uses the CSRSS's port objects fordebug port and exception port.

EQUIVALENT WIN 3 2 APICreateProcess

314___Appendix A: Details of System Calls with Parameters

NtCreateThreadNTSTATUS

NtCreateThread(OUT PHANDLE phThread, « ,.IN ACCESS_MASK AccessMask,IN POBJECT_ATTRIBUTES ObjectAttributes,IN HANDLE hProcess,OUT PCLIENT_ID pClientld,IN PCONTEXT pContext,OUT PSTACKINFO pStacklnfo,IN BOOLEAN bSuspended

):NtCreateThread creates a new thread object.

PARAMETERSphThread

AccessMaskObjectAttributes

hProcesspClientld

pContext

PStacklnfo

bSuspended

Pointer to the variable that receives handle to the threadobject.Type of access requested to the thread object.

Points to the OBJECTATTRIBUTES structure containing theinformation about the thread object to be created, such asname, parent directory, objectflags, and so on.

Handle to the process object that this thread will belong to.Pointer to the structure that will receive the thread ID andthe process ID for this thread.Pointer to the CONTEXT structure containing the state ofvarious processor registers when the thread beginsexecuting.

Pointer to the variable that receives the information aboutthe stack of the thread.

Flag indicating whether the thread should be in suspendedmode.

RETURM VALUEReturns STATUS_SUCCESS on success and an appropriate error code on failure.

COMMENTSNote that unlike all other synchronization objects, such as mutex and semaphore,the Equivalent Win32 API CreateThread call does not enable you to name the

'


thread object, whereas the system service enables you to specify the name of thethread object in the ObjectAttributes structure. This way, you can open the threadusing its name instead of the thread ID.

The structure PSTACKINFO is defined as follows:

typedef struct S tack ln fo_ t {LONG Unknownl ;LONG Unknown?;LONG TopOfS tack ;LONG O n e P a g e B e l o w T o p O f S t a c k ;LONG Bo t tomOfStack ;

} S T A C K I N F O , * P S T A C K I N F O ;

EQUIVALENT WIN 3 2 APICreateThread

NtDelayExecutionNTSTATUS

NtDe layExecu t i on (IN BOOLEAN bA le r tab le ,IN PLARGE^INTEGER pDurat ion

) ;

NtDelayExecution puts the calling thread in alertable/nonalertable sleep state forthe specified duration.

PARAMETERS

bAlertable

pDuration

Flag specifying whether the sleep is alertable/nonalertable.

Pointer to the LARGEJNTEGER structure containing the durationfor sleep. Negative values represent relative time, whereaspositive values represent absolute time.


COMMENTSNone.

EQUIVALENT WIN 3 2 APISleepEx


NtGetContextThreadNTSTATUS

NtGetContex tThread(IN HANDLE hThread.IN OUT PCONTEXT pContext

):

NtGetContextThread returns the context structure for the specified thread.

PARAMETERS


COMMENTSNone.

EQUIVALENT WIN32 APIGetThreadContext

NtSetContextThreadNTSTATUS

NtSetContextThread(IN HANDLE hThread,IN PCONTEXT pContext

);

NtSetContextThread sets the context structure for the specified thread.

PARAMETERS

hThread Handle to the thread object.

hThreadpContext

Handle to the thread object.

Pointer to the structure that contains new context structure for thethread. The ContextFlags field in this structure must be filled withany combination of CONTEXT_CONTROL, CONTEXTJNTEGER,CONTEXT_SEGMENTS, CONTEXT_FLOATING_POINT,CONTEXT_DEBUG_REGISTERS, CONTEXT_EXTENDED_REGISTERS, and CONTEXT_FULL based on the type of contextinformation filled before invoking this system service.


pContext Pointer to the structure that receives the context structure for thethread. The ContextFlags field in this structure must be filled withany combination of CONTEXT_CONTROL, CONTEXTJNTEGER,CONTEXT_SEGMENTS, CONTEXT_FLOATING_POINT,CONTEXT_DEBUG_REGISTERS, CONTEXT_EXTENDED_REGISTERS, and CONTEXT_FULL before invoking this systemservice. "- -


COMMENTSNone.

EQUIVALENT WIN32 APISetThreadContext

NtOpenProcessNTSTATUS

NtOpenProcess(OUT PHANDLE phProcess ,IN ACCESS_MASK A c c e s s M a s k ,IN POBJECT_ATTRIBUTES ObjectAttnbutes,IN PCLIENT_ID pClientld

):

NtOpenProcess opens a handle to the existing process object.

PARAMETERS

phProcess

AccessMask

ObjectAttributes

pClientld

Pointer to the variable that receives handle to the processobject.Type of access requested to the process object.Points to the OBJECTATTRIBUTES structure containingthe information about the process object to be opened,such as name, parent directory, objectflags, and so on.

Pointer to the CLIENTJD structure. The process ID memberof this structure must be filled by the caller beforeinvoking this system service.



COMMENTSOnly one of the parameters ObjectAttributes and pClientld may be specified. If theprocess object is named, then the ObjectAttribute structure must be filled with thedetails; otherwise, the process ID field must be filled in the pClientld structure.

EQUIVALENT WIN 3 2 APIOpenProcess

NtOpenThreadNTSTATUS v ?

NtOpenThread(OUT PHANDLE phThread,IN ACCESS_MASK A c c e s s M a s k ,IN POBJECT_ATTRIBUTES ObjectAt t r ibutes ,IN PCLIENT_ID pCl ient ld

): -

NtOpenThread opens handle to the existing thread object.

PARAMETERS

phThread

AccessMaskObjectAttributes

pClientld

Pointer to the variable that receives handle to the threadobject.

Type of access requested to the thread object.Points to the OBJECTATTRIBUTES structure containing theinformation about the thread object to be opened, such asname, parent directory, objectflags, and so on.Pointer to the CLIENTJD structure. The thread ID memberof this structure must be filled by the caller before invokingthis system service.


COMMENTSOnly one of the parameters ObjectAttributes and pClientld may be specified. If thethread object has a name, then the ObjectAttribute structure must be filled with thedetails; otherwise, the thread ID field must be filled in the pClientld structure.



NtQuerylnformationProcessNTSTATUS

NtQuery In fo rma t i onProcess (IN HANDLE h P r o c e s s ,IN P R O C E S S I N F O C L A S S P r o c e s s I n f o C l a s s ,OUT PVOID P rocess In foBu f fe r ,IN ULONG Process In foBu f fe rLeng th ,OPTIONAL OUT PULONG BytesCopied.

):

NtQuerylnformationProcess returns information about the specified process object.

PARAMETERS

hProcessProcessInfoClass

ProcessInfoBuffer

ProcessInfoBufferLength

BytesCopied

Handle to the process object.Type of information requested.

Pointer to the buffer that receives the informationabout the process object.

Size of the buffer (in bytes) pointed to byProcessInfoBuffer.

Pointer to the variable that receives the number ofbytes copied into ProcessInfoBuffer.


COMMENTSDifferent information is returned based upon the ProcessInfoClass parameter. Hereis the layout of ProcessInfoBuffer based on the ProcessInfoClass:

ProcessBasicInformation(O)ProcessQuotaLimits( 1 )

ProcessIoCounters(2)

ProcessVmCounters(3)

ProcessTimes(4)

ProcessDebugPort(7)

ProcessLdtlnformationf 10)

PROCESS_BASIC_INFORMATION.QUOTA_LIMITS.IO_COUNTERS.VIVLCOUNTERS.KERNEL_USER_TIMES.ULONG containing the pointer to debugport object.

PROCESS LOT INFORMATION.


Most of the structures defined previously are documented in the NTDDK.H filefrom the Windows NT DDK. Other structures can be found in UNDOCNT.H on theaccompanying CD-ROM.

Some of this information is available from Equivalent Win32 API calls.However, not all of them are available from Win32 API calls.

EQUIVALENT W1M 3 2 APIGetProcessTimes, GetProcessPriorityBoost, GetProcessAffinityMask,GetProcessShutdownParameters, GetPriorityClass, GetProcessWorkingSetSize,GetProcessVersion

NtQueryInformati onThreadNTSTATUS

NtQuery In format ionThread(IN HANDLE hThread ,IN T H R E A D I N F O C L A S S Th read ln foC lass ,OUT PVOID Thread ln foBuf fe r ,IN ULONG ThreadlnfoBuf ferLength,OPTIONAL OUT PULONG BytesCop ied,

);

NtQuerylnformationThread returns information about the specified thread object.

PARAMETERS


ProcessDefaultHardErrorMode( 1 2)

ProcessPooledUsageAndLimits( 1 4)

Process WorkingSetWatch( 1 5)

Process Wx86Information( 1 9)

ProcessHandleCount(20)

ProcessPriorityBoost{22)

ULONG containing the default error modefor the process. This could be anycombination ofSEM_FAlLCRLTICALERRORS,SEM_NOALIGNMENTFAULTEXCEPT,SEM_NOGPFAULTERRORBOX, andSEM_NOOPENFILEERRORBOX.

POOLED_USAGE_AND_LIMITS.

PROCESS_WS_WATCH_INFORMATION.

ULONG. Always returns 0.

ULONG containing the number of openhandles.

BOOLEAN containing the priority boostcontrol state. TRUE means that dynamicboosting is disabled.


ThreadlnfoClassThreadlnfoBuffer

ThreadlnfoBufferLength

BytesCopied

Type of information requested.

Pointer to the buffer that receives the informationabout the thread object.Size of the buffer (in bytes) pointed to byThreadlnfoBuffer.

Pointer to the variable that receives the number ofbytes copied into ThreadlnfoBuffer.


COMMENTSDifferent information is returned based upon the ThreadlnfoClass parameter. Hereis the layout of ThreadlnfoBuffer based on the ThreadlnfoClass:

ThreadBasicInformation(O)ThreadTimes(l)

ThreadDescriptorTableEntry (6)

ThreadQuerySetWin32StartAddress (9)

ThreadPerformanceCount (11)ThreadAmILastThread( 1 2)

ThreadPriorityBoost (14)

THREAD_BASIC_INFORMATION.

KERNEL_USER_TTMES.

DESCRIPTOR_TABLE_ENTRY.

Pointer to ULONG containing the startaddress of the thread.Pointer to an array of two ULONGs.

Pointer to ULONG that receiveswhether the calling thread is the lastthread of the process (hThreadparameter is ignored in this case).

Pointer to ULONG that receiveswhether dynamic priority boosting isenabled or disabled for the thread.

Some of the structures defined previously are documented in the NTDDK.H filefrom the Windows NT DDK. Other structures can be found in UNDOCNT.H on theaccompanying CD-ROM.

Here are the definitions for the structures that are not documented in NTDDK.H:

typedef struct _THREAD_BASIC_INFORMATION (NTSTATUS ExitStatus;PVOID TebBaseAddress ;ULONG UniqueProcessId;ULONG UniqueThreadld;


KAFF IN ITY A f f i n i t y M a s k ;K P R I O R I T Y BasePnonty ;ULONG D i f fP rocessPnor i t y ;} THREAD_BASIC_ INFORMATION, *PTHREAD_BASIC_INFORMATION;

Some of this information is available from Equivalent Win32 API calls.However, not all of them [[[AU: "not all of them" OK?]]]are available from Win32API calls.

EQUIVALENT WIN 3 2 APIGetThreadPriorityBoost, GetThreadTimes, GetThreadPriority

NtQueueApcThreadNTSTATUS

NtQueueApcThreadCIN HANDLE hThread,IN PKNORMAL_ROUTINE ApcRout ine ,IN P V O I D Norma lCon tex t .IN PVOID SystemArgument l ,IN PVOID SystemArgument2,

);

NtQueueApcThread queues an entry to the thread's APC Queue.

PARAMETERS

hThread

ApcRoutine

NormalContext

SystemArgument 1

SystemArgument2

Handle to the thread object. - ,

The function that gets called when the APC is scheduledfor execution.

Points to the context associated with the APC.

Points to the first argument to be passed to the APCfunction.

Points to the second argument to be passed to the APCfunction.


COMMENTSThe Win32 API call corresponding to this system service is prototyped as


WINBASEAPIDWORDWINAPIQueueUserAPC(

PAPCFUNC pfnAPC,HANDLE hThread,DWORD dwData);

The QueueUserAPC function passes pfnAPC for the NormalContext parameterand passes dwData for the SystemArgumentl parameter. It also passes the addressof the internal KERNEL32 routine called BaseDipatchAPC as the ApcRoutine para-meter. TheBaseDispatchRoutine extracts the pfnAPC and dwData parameters fromthe parameters passed to it and calls pfnAPC.

EQUIVALENT WIN32 APIQueueUserAPC

NtResumeThreadNTSTATUS

NtResumeThread(IN HANDLE hThread,OUT PULONG pSuspendCount

);

NtResumeThread decrements the suspend count for the thread and resumes thethread if the suspend count reaches 0.

PARAMETERS

hThread

pSuspendCountHandle to the thread object.

Pointer to the variable that receives the suspend count of thethread at the time this system service is invoked.


COMMENTSThe Equivalent Win32 API call does not return the suspend count of the thread,whereas the system service does return this information.

EQUIVALENT WIN32 APIResumeThread

324___Appendix A: Details of System Calls with Parameters_____________

NtSetLowWaitHighThreadNTSTATUS

N t S e t L o w W a i t H i g h T h r e a d ();

NtSetLowWaitHighThread sets the low event of the event pair associated withthe calling thread and waits on the high event of the event pair to be signaled.

PARAMETERSNone.


COMMENTSChapter 8 provides more information.

EQUIVALENT WIN 32 APINone.

NtSetHighWaitLowThreadNTSTATUS

N t S e t H i g h W a i t L o w T h r e a d ();

NtSetHighWaitLowThread sets the high event of the event pair associated withthe calling thread and waits on the low event of the event pair to be signaled.

PARAMETERSNone.


COMMENTSChapter 8 provides more information.


NtSuspendThreadNTSTATUS

NtSuspendThread(IN HANDLE hThread,OUT PULONG pSuspendCount

);


NtSuspendThread suspends the thread and increments the suspend count for thethread.

PARAMETERS

hThread

pSuspendCount

Handle to the thread object.

Pointer to the variable that receives the suspend count of thethread at the time this system service is invoked.


COMMENTSThe Equivalent Win32 API call does not return the suspend count of the thread,whereas the system service does return this information.

EQUIVALENT WIN 3 2 APISuspendThread

NtTermi nateProcessNTSTATUS

N t T e r m i n a t e P r o c e s s ( ' - •IN HANDLE hP rocess ,IN ULONG ExitCode

);

NtTerminateProcess terminates the specified process.

PARAMETERS

hProcess Handle to the process object.ExitCode Exit code for the process.


COMMENTSNone.

1

EQUIVALENT WIN32 APIExitProcess, TerminateProcess

326___Appendix A: Details of System Calls with Parameters

NtTerminateThreadNTSTATUS

NtTerminateThread(IN HANDLE hThread,IN ULONG ExitCode

):NtTerminateThread terminates the specified thread.

PARAMETERS


ExitCode Exit code for the thread.


COMMENTSNone.

EQUIVALENT WIN 3 2 APIExitThread, TerminateThread

NtYieldExecutionNTSTATUS

NtY ie ldExecu t ion () ;

NtYieldExecution relinquishes the processor from the calling thread.

PARAMETERSNone.


COMMENTSNone.

EQUIVALENT WIN 3 2 API L

SwitchToThread.

Appendix B

What's on the CD-ROMThe CD-ROM that accompanies this book contains the source code and binaries forall the sample applications that we've discussed in this book. Each sample is kept ina separate directory.

To try out each sample, open the accompanying README.TXT file. Each con-tains step-by-step instructions for installing and trying out the sample.

In each README.TXT file, we have also identified which system (or systems) weused to test the sample - Windows NT 3.51, Windows NT 4, and/or Windows 2000beta 1 and beta 2.

The samples are compiled using MSVC 4.2 compiler and Windows NT 4.0 DDK.

327

Unknow - The Undocumented Functions Windows Nt-2000-XP Kernel Mode

Documents

Transcript of Unknow - The Undocumented Functions Windows Nt-2000-XP Kernel Mode