Grouping and sorting of objects using GSON and Comparators in JAVA

  Grouping And Sorting Of Objects

Grouping and sorting of objects is a very general problem and no matter whichever language or platform you are using, you are going to face it sooner or later. This article is an approach to provide an efficient solution for the same.

Grouping of objects:

Take a scenario where you have class like this:

class Data {

int id;

       String color;

       String type;

       Timestamp dateSubmitted;

       String description;

       String vendor;

      

       public String getDescription() {

              return description;

       }

}

And there is an array of objects of this class like:

Data[] dataObjects;

Now suppose we want to create groups in this array of objects on the basis of the field Description which means the objects with the same description field should be in one group. The typical approach would be to write the business logic involving complex manipulations. The other smarter approach would be to use "MultiMaps". Now let's solve this problem using multimaps. The language we will use is Java and in case you are working on some different language then you can google how to use multimaps specific to your platform. So in java to use multimaps we will use library named "Guava" provided by Google to work in a faster and efficient manner on collections. Just download the jar from this link http://code.google.com/p/guava-libraries/ and include it in your project.

Now before going directly to the final answer it will be better to be aware of the concept of multimaps which will make it much easier to grasp the solution. So multimaps is like a key-value pair concept where each key is mapped to some value. In addition to this key-value pair mapping it can also contain duplicate keys which a simple map cannot. Thus multimap groups values on the basis of these duplicate keys. So to understand this let's take a simple example.

The above example is about key-value pairs with no duplicate keys. This example demonstrates the concept of a simple map with no duplicate keys allowed.

                           

The above example is about key-value pairs with duplicate keys. This example demonstrates the concept of how the duplicate keys are allowed in a multimap.

Now the Guava multimap can thus store a map with duplicate keys and it groups the values having the same key. This concept of grouping of values based on keys is actually is the solution of this problem. Here is the diagram demonstrating the grouping of values based on the duplicate keys.

Note: Please notice in the above diagram how the values with the same key "apple" are grouped together

Now let's use this concept to solve our above problem where we have to group the objects which have the same description field. So to solve it first step would be to create a multimap with the value of description field of object as a key and this corresponding object as a value. As a result objects with the same description will be grouped together in one group. Ok now let us try it practically.

Here is the code to create a multimap:

Multimap<String,Data> testMap = ArrayListMultimap.create();

In the above code line a multimap provided by guava library is initialized whose key is of type string and value is of type Data class. Next step is to fill this multimap from the array of objects of data class. Here is the code for it:

for(int i =0;i<dataObjects.length;i++){

testMap.put(dataObjects[i].getDescription(),dataObjects[i]);

}

As a result a map will be created with key as the value of the description field of an object and the value as this corresponding object for this description field. Thus all the objects with the same description field will be grouped together.

Now in order to access these groups we can simply use the value of the field description as a key to get the corresponding group. Here is the code for it:

List<Data> group = (List<Data>) testMap.get("apple");

Sorting of Objects:

Another general problem is of sorting of objects. Again the typical approach to solve this problem will involve algorithms doing complex manipulations. But to solve this problem we can use "Comparators" and "Collections" to sort objects in an efficient and faster manner. If you are working on a different technology please google how to use comparators and collections to sort objects specific to your technology. Ok so let's look quickly on how we can use comparators and collections to sort objects. Only the basic code is provided here for introduction of sorting of objects using comparators and collections and the concept of comparators and collections is not explained here. The language we will use is Java.

So suppose we have to sort the objects of class Data described previously in an ascending order according to the value of the field Id. For this we have to define a comparator which compares two objects based upon the value of the field Id. Thereafter we will use this comparator to sort this array of objects using sort method of class Collections provided by java. Here is the code for it:

Collections.sort(Arrays.asList(dataObjects), new Comparator<Data>() {

@Override

       public int compare(Data dateFirstObject, Data dateSecondObject) {

              // compare the two objects based upon the value of field Id and

              // return values accordingly to sort objects in an ascending order

              if (dateFirstObject.getId() > dateSecondObject.getId()) {

                     return 1;

              } else if (dateFirstObject.getId() < dateSecondObject.getId()) {

                     return -1;

              } else {

                     return 0;

              }

       }

});

Here only the basic code is provided to introduce the concept of sorting using comparators and collections. Please read from some good resources about comparators and collections in Java for their conceptual understanding. Thereafter you can apply them in your code to sort objects according to the need.

Note :- I have also created some sample codes explaining the above mentioned concepts. Here is the github URL for it : https://github.com/pankajc-optimus/android-lib/tree/master/SortingAndGroupingOfObjects/Code/SortingAndGroupingOfObjects

WDM Device driver tutorial and WDM device driver concepts: how to write a WDM device driver ?

  The article is an attempt to cover all device driver and wdm concepts in a simplified way  .It starts with a brief description of device drivers thereafter discussing its historical development leading to the development of WDM drivers. It describes basic device driver types and device driver concepts. It describes how OS interacts with the device driver. The  main focus is on general discussion on development and implementation of WDM device drivers such as how a WDM driver is loaded when it is plugged, how it’s loaded by OS and how a request travels down through the driver layers as it is processed by each one of them. It describes why WDM device driver development is an aspect of kernel programming. The two basic WDM structures for describing device object and driver object have been discussed in detail thereafter how a WDM driver is made up of various routines and how these routines are called based on certain requests is explained.  The basic routines of WDM drivers have been described with their implementation in c language. An explanation runs parallelly with coding illustrating how it’s being done. The first routine of device driver driverinit is discussed such as its purpose and its implementation is shown in c language with a brief explanation how this code is being written. Similarly another add device routine is also described and implemented in c language. The focus is then shifted towards I/O requests .They are described in detail such as how they are developed , processed by driver and how they move from one driver to another in driver layers. Thereafter another routine which handles this request are being discussed. All the general cases of how a routine processes these request are being described and their implementation is shown in c language. It finishes up with how we can install our device driver and how we can load it using a tool.


What is a device driver?
A device driver is a software or computer program which is used by a an OS to communicate or
interact with a device for e.g. your usb drive driver is used by os to communicate with usb drive
for reading /writing files in it i.e. a driver provides an interface for os to communicate with device. 

History

History of device drivers is directly related to the development of hardware and operating system.

The hardware development started with intel 8086 which operates in real mode (a mode in which you have an access to 1 MB of main memory and you have an direct access to all i/o addresses and there is no paging). At this point of time the drivers were in real mode and MS-DOS incorporated a scheme based on the CONFIG.SYS file where end user could load real mode drivers. Here the plug n play features were not available and user has to manually configure his device using a map of i/o instructions.

After it Intel launched 80286 which can operate in protected mode(the mode in which there is virtual memory concept, paging and memory of 4 GB was available) and it can also switch back to real mode by setting a value in control register but it was having many drawbacks in protected mode and also there was no advancement from operating system side and ms dos still worked with real mode device drivers.

The intel 80386 was the leading change in computing history when it removed all the drawbacks of intel 80286 and protected mode functionality is now ease to use but the ms dos still worked with real mode device drivers.

Microsoft launched windows 3.0 to cope up with hardware development which processed application in protected mode with 32-bit virtual addressing. On the driver part

Vxd(virtual device driver) was the new concept was launched but it still used a real mode device driver and cpu has to be switched to real mode to process it.

On the other hand , windows was working with IBM and developed windows NT which used a new kernel based  technology which neither used real mode nor protected mode.

Windows 2000 and xp was successor of windows NT and windows 95 and 98 was successor of windows 3.0

Ok here was the problem for the device driver developer , he have to develop driver for windows NT and keep worrying for developing real mode drivers because some users do not want to upgrade from windows 3.0.

So microsoft launched a new technology WDM(WINDOWS DRIVER MODEL) and incorporated it with both windows 3.0 and NT .This can be used by a driver developer and need not to worry for real mode driver because the wdm technology was same on all platforms and just one driver can work on all platforms.

Classification of Window Device Drivers:-
 
 

classification of drivers

1. Virtual device drivers - Virtual device drivers are device drivers that are used to emulate a hardware device in virtualization environments, for example when a DOS program is run on a Microsoft Windows it uses virtual device drivers. 

2.Kernel Mode Drivers - Kernel mode drivers are the drivers in which the executing code has complete and unrestricted access to the underlying hardware. It can execute any CPU instruction and reference any memory address because they actually execute in kernel memory.

The Kernel mode drivers are further classified as-

  a) File system drivers - A file system driver handles I/O independent of any underlying physical device. File system drivers include drivers for the NTFS and file allocation table (FAT) file systems. 

  b) Legacy device drivers - Legacy device drivers are kernel-mode drivers that directly control a hardware device without help from other drivers i.e. they doesn't use layered structure . 

  c) PnP Drivers - Plug and Play drivers provide: 

        Automatic and dynamic recognition of installed hardware·

          Hardware resource allocation (and reallocation) 

         Loading of appropriate drivers·

         An interface for drivers to interact with the PnP system·

         Mechanisms for drivers and applications to learn of changes in the hardware environment

To just explain in simple way take an example of your usb drive. As soon as you plug it in , the kernel reads a signature(which gives the information of the drive and is embedded on your drive) . The os finds out a particular driver from registry using this information whereupon it loads it in memory.

This above is a example of pnp driver where there is no need of manually loading a driver. It’s the work of I/O manager while in legacy drivers you have to manually load your driver after plugging it in using add new hardware in windows .

 WDM driver - A WDM driver is a PnP driver that actually is developed using WDM specification. provided by microsoft. The specifications actually defines two aspects for wdm .The first aspect  defines it core model  which describes describes how device drivers are installed and started, and how they should service user requests and interact with hardware. A WDM device driver must fit into the Plug and Play (PnP) system that lets users plug in devices that can be configured in software. The second aspect is series of bus and class drivers for common types of devices provided by microsoft which we can use in our driver to communicate with devices for e.g. if we have to develop a  driver to perform some work on usb we can use usb  class driver provided  by microsoft to access functionality provided by it whereupon reducing work for developer .All versions of Microsoft Windows after Windows 95 have implemented WDM, which has continued to evolve for Windows XP, Windows 2000, and Windows Me. With a few special-case statements in the code, WDM drivers for supported device classes can be source-code compatible across Windows XP/Windows 2000 and Windows 98/Me operating systems. Binary compatibility is possible on processors that are compatible with the Intel Architecture. Achieving a single binary still requires rigorous testing on all versions of all operating systems.  

The different WDM drivers are- 

Class Drivers – are the drivers which are used to define some common functionality and these drivers are used by other drivers to access there functionality.  Microsoft provides its class drivers for e.g.  usb class drivers whose functionality can be accessed by the function drivers.

Miniport drivers: These are function drivers for USB, Audio, SCSI and network adapters. They should usually be source and binary compatible between Windows 98 and Windows 2000 and are hardware specific but control access to the hardware through a specific bus class driver(this can be the one provided by microsoft). These miniport drivers actually are defined over the class drivers to access their functionality. A programmar basically builds this driver and place it over class driver. 

Monolithic Function Drivers – are the drivers that does not use the layered structure i.e. to communicate with hardware  it does not use the bus class driver as compared to miniport drivers instead it directly drives a hardware device.  

Filter drivers - Filter drivers are optional drivers that add value to or modify the behavior of a device driver i.e. it’s used to just filter your I/O request to driver below it (for e.g. to check incorrect parameters to usb class driver we can place a filter driver to check for the same and filter out the request). A filter driver service one or more devices. Upper level filter drivers sit above the primary driver for the device (the function driver), while lower level filter drivers sit below the function driver and above the bus driver.

3) Windows Driver Foundation - WDM has been criticized by driver software developers because of the following reasons:- 

    * WDM has a very steep learning curve. 

   * Interactions with power management events and plug and play are difficult. This leads to a variety of situations where Windows machines cannot go to sleep or wake up correctly due to bugs in driver code. 

    * I/O cancellation is almost impossible to get right. 

    * Thousands of lines of support code are required for every driver. 

    * No support for writing pure user-mode drivers.

Because of these issues, Microsoft has released a new framework to replace WDM, called the Windows Driver Foundation, which includes Kernel-Mode Driver Framework (KMDF) and User-Mode Driver Framework (UMDF). Windows Vista supports both WDM and the newer Windows Driver Foundation. KMDF is also available for download for Windows XP and even Windows 2000, while UMDF is available for Windows XP and above. 

OS / DRIVER INTERACTION:

os/driver interaction

The application requests windows to access your driver for e.g. reading or writing a certain file in device.  Its done by using a api call (here api is of win32 and its being called by application called as win32 api calls). The win 32 subsystem interprets the request and using the system service interface communicates with I/O manager to handle it.  

An IRP is actually an  I/O REQUEST PACKET that is used to represent a request such as whether it’s a read or write request or pnp request using major function code for e.g. IRP with IRP_MJ_PNP as major function code is used for representing a pnp request and the minor function code as IRP_MN_START_DEVICE will further elaborate the request that it’s a PNP request for starting a device.

The I/O manager issues an I/O REQUEST PACKET short form IRP(identifies your i/o request ) pertaining to the request and passes it to the device driver whereupon its processed by device driver . A device driver may eventually need to actually access its hardware to perform an IRP for the same it uses  HAL(HARDWARE ABSTRACTION LAYER) calls  . The result is returned back to application.

Layering of devices and drivers in the Windows ­Driver Model:

layering of drivers and devices

The IRP is passed to the upper filter driver and after processing it is passed to the
function driver and then to the lower filter device driver and then finally to the bus
driver.

Device objects actually represents device at a particular driver layer.

PDO - a physical device object represents device at the lowest layer and also
representing your physical device.

FDO - a function device object at function driver layer and is used by function
drivers to manage device functionality

     FiDO - a filter device object used by WDM filter drivers to store information
about hardware

   The IRP after processed by the bus driver is returned back in the same order.

  How the System Finds and Loads Drivers:

A Plug and Play device has an electronic signature that the system can detect. For
Plug and Play devices, a system bus driver detects the existence of the hardware(when
it's plugged)  and reads this signature to determine what kind of hardware it is.
Thereafter, an automatic process based on reading the registry or locating the INF files
allows the system to find the correct driver for this device and then loading it.

Here are the steps explaining the whole procedure:

   

step 1 : When a bus driver detects the insertion or removal of hardware, it calls IoInvalidateDeviceRelations to notify the PnP Manager that new physical device
has been plugged and create a PDO relevant to it. 

step 2: the pnp manager as a result of step 1,  has to update the list of physical device
object by retrieving its information from bus driver, for this it sends an IRP with major
function code IRP_MJ_PNP and minor function code IRP_MN_QUERY_DEVICE_
RELATIONS to bus driver. (major and minor functions codes are used to identify 
the type of request ,here it’s a IRP_MJ_PNP request as a major function code to
identify it’s a request for plug and play manager and the minor function code IRP_MN_QUERY_DEVICE_RELATIONS further identifies this request as it is 
requesting for information of newly updated physical device represented by PDO ) 

Step 3: In response to the bus relations query, the bus driver returns the information to
the pnp manger.. 

Step 4: the pnp manager using the information send by bus driver searches registry or
locates inf file for installing device driver. 

Step 5:  after loading of driver the pnp manager calls addevice to initialize your device. 

Step 6: thereafter the pnp manager configures your driver and start your driver by
sending an IRP with IRP_MJ_PNP(major function code) to your driver with the
minor function code  IRP_MN_START_DEVICE.

 

Interrupt Request Level:

The OS assigns an interrupt request level to each hardware interrupt and to a
few software events. The processor actually executes at a particular interrupt
request level. When an interrupt occurs, the kernel raises the IRQL on the
interrupting CPU to the level associated with that interrupt . The different
levels with their assigned priority is shoen below.

 

Interrupt request level

The activity of processing an interrupt is only to process an interrupt at a higher
IRQL but never to process an interrupt at the same or a lower IRQL. To make
it more clear suppose a thread is executing at PASSIVE_LEVEL and an another
thread wants to interrupt it so this can be only done if the thread that is generating
an interrupt is at higher level than PASSIVE_ LEVEL like APC_LEVEL or
DISPATCH_LEVEL.

    

Significance of DISPATCH_LEVEL: 

Code executing at or above DISPATCH_LEVEL must not cause page faults. i.e. code or thread executing at DISPATCH_LEVEL OR above it needs to be in non-paged memory.

To summarize, drivers are normally concerned with three interrupt request levels:

• PASSIVE_LEVEL, at which many dispatch routines and a few special routines execute
• DISPATCH_LEVEL, at which StartIo and DPC routines execute
• DIRQL, at which an interrupt service routine executes

 Spin Locks: 

To synchronize access to shared data in multiprocessor environment spin locks are
used. In multiprocessor environment when many cpus try to access the shared data 
can create problems as  more than one cpu may simultaneously access the shared 
data. To avoid this we use spin locks , a processor can have access to shared data 
only if it has acquired the spin lock for it and the other cpu has to wait.

mechanism for spin locks

The image above explains mechanism for spin locks .Cpu A acquires the spin lock
at time t1 using the test and set mechanism . In test and set mechanism the cpu tests
on a variable whether its free or not , if its free it sets the variable and acquires the
spin lock. Now at time t2 cpu B tries to acquire spin lock using test and set
mechanism , it tests for the variable and as it is set by cpu A it waits for it in a loop
i.e it tests it again and again until it founds the variable free i.e. it spins between time
interval of t2 and t3 and that’s why its called spin lock. Cpu A release the spin lock
at t3 whereupon the cpu B  gets access to the shared variable.

Spin lock are acquired at DISPATCH_LEVEL that means code acquiring the spin
lock needs to be in non paged memory i.e. your shared data needs to be in non
paged memory. 

WDM Driver Data Structures 

This section introduces the data structures used in Windows drivers as well as how
they are linked together to create a larger structure, which in turn provides a lot of
functionality for device drivers. It is critical during initial driver routines that these
structures are allocated, linked together, and accessed properly to avoid crashing
the system (null pointers, malformed linked lists, etc). 

Driver and Device Objects 

Driver Objects: The I/O manager uses a DRIVER_OBJECT structure to represent each device driver.

driver object structure

 

The structure is partially opaque i.e. in above image there are two fields, one
with grey background and other with white background. The grey background
field can’t be accessed directly and you can change these fields while the fields
with white background can be accessed directly and can be modified , it’s just
like private(grey background) and public members(white background).

Device object(PDEVICE_OBJECT): points to your device which is to be
handled by this driver .

PDEVICE_OBJECT actually specifies  type name of this field , that it’s a
pointer to a device object similarly you can understand  for other type fields.

Driver extension(PDRIVER_EXTENSION): points to the add device routine
and other initialization of your driver which is needed for creating device objects
for your driver i.e. it is used by plug n play manager to load your device and
creating it’s device object when it’s plugged.

The basic responsibility of AddDevice in a function driver is to create a device object
and link it to the lower device object of this current device object( this is how stack
is created). The steps involved are as follows:

1.      Call IoCreateDevice to create a device object and an instance of your own
device extension object.

2.      Register one or more device interfaces so that applications know about the
existence of your device. Alternatively, give the device object a name and then
create a symbolic link.

3.      Next initialize your device extension and the Flags member of the device object.

4.      Call IoAttachDeviceToDeviceStack to put your new device object into the stack.

HardwareDatabase (PUNICODE_STRING) describes a string that names a hardware
database registry key for the device. This is a name like \Registry\Machine\Hardware\
Description\System and names the registry key within which resource allocation
information resides. WDM drivers have no need to access the information below this
key because the PnP Manager performs resource allocation automatically. The name
is stored in Unicode. (In fact, all kernel-mode string data uses Unicode.)

DriverInit(PDRIVER_INIT): points to your init routine of your driver.

DriverStartIo (PDRIVER_STARTIO) :points to a function in your driver that
processes I/O requests that the I/O Manager has serialized for you.

FastIoDispatch (PFAST_IO_DISPATCH): points to a table of function pointers
that file system and network drivers export..

DriverStartIo (PDRIVER_STARTIO): points to your start io routine of your driver.

DriverUnload(PDRIVER_UNLOAD): points to your unload routine for unloading
your driver.

MajorFunction (array of PDRIVER_DISPATCH): is a table of pointers to
functions in your driver that handle each of the roughly two dozen types of
I/O requests. For e.g. for an IRP with major function code such as
IRP_MJ_READ it will contain a pointer to a dispatch read routine(which will
process it) and similar other IRP’s with other routines.

   

Device object: points to your device and store information about it.

device object

  DriverObject (PDRIVER_OBJECT): points to the driver related to your device. 

NextDevice (PDEVICE_OBJECT): points to the next device object that belongs

to the same driver as this one(for e.g. more then one usb drives will actually need

only one driver to process it so we will use this nextdevice field of one usb drive

to point to the other usb drive which will also be processed by the driver of the current

usb drive). This field is the one that links device objects together starting from the driver

object’s DeviceObject member.

  

CurrentIrp (PIRP): is used by the Microsoft IRP queuing routines StartPacket and  

StartNextPacket to record the IRP most recently sent to your StartIo routine.

Flags (ULONG): contains a collection of flag bits(actually used to convey some type
of information ). Table 2-2 lists the bits that are accessible to driver writers.

(ULONG  : Unsigned 32-bit integer)

Table 2-2. Flags in a DEVICE_OBJECT Data Structure
Flag	Description
DO_BUFFERED_IO	Reads and writes use the buffered method (system copy buffer) for accessing user-mode data.
DO_EXCLUSIVE	Only one thread at a time is allowed to open a ­handle.
DO_DIRECT_IO	Reads and writes use the direct method (memory descriptor list) for accessing user-mode data.
DO_DEVICE_INITIALIZING	Device object isn’t initialized yet.
DO_POWER_PAGABLE	IRP_MJ_PNP must be handled at PASSIVE_LEVEL.
DO_POWER_INRUSH	Device requires large inrush of current during power-on

Characteristics (ULONG): is another collection of flag bits describing various
optional  characteristics of the device. (See Table 2-3.) The I/O Manager
initializes these flags based on an argument
to IoCreateDevice(used to create device object related to driver).
Filter drivers propagate some of them upward in the device stack.

Flag	Description
FILE_REMOVABLE_MEDIA	Media can be removed from device.
FILE_READ_ONLY_DEVICE	Media can only be read, not written.
FILE_FLOPPY_DISKETTE	Device is a floppy disk drive.
FILE_WRITE_ONCE_MEDIA	Media can be written once.
FILE_REMOTE_DEVICE	Device accessible through network ­ connection.
FILE_DEVICE_IS_MOUNTED	Physical media is present in device.
FILE_VIRTUAL_VOLUME	This is a virtual volume.
FILE_AUTOGENERATED_DEVICE_NAME	I/O Manager should automatically ­ generate a name for this device.
FILE_DEVICE_SECURE_OPEN	Force security check during open.

DeviceExtension (PVOID): points to a data structure you define that will hold per
-instance information about the device.(PVOID :Generic pointers)

The I/O Manager allocates space for the structure, but its name and contents
are entirely up to you. A common convention is to declare a structure with the type name DEVICE_EXTENSION.
To access it given a pointer (for example, fdo) to the
device object, use a statement like this one:

PDEVICE_EXTENSION pdx = 

  (PDEVICE_EXTENSION) fdo->DeviceExtension;//  note this statement we will use
//to access
//the  driver extension of current driver. 

DeviceType (DEVICE_TYPE): is an enumeration constant describing what type of
device this is. The I/O Manager initializes this member based on an argument to
IoCreateDevice. 

StackSize (CHAR): counts the number of device objects starting from this one
and descending all the way to the PDO (i.e. counting the device objects in driver
layers .Refer diagram of driver layers and see the FiDO, FDO, PDO and we
have to count all of them down the stack). The purpose of this field is to inform
interested parties regarding how many stack locations should be created for an
IRP that will be sent first to this device’s driver. 

AlignmentRequirement (ULONG):specifies the required alignment for data
buffers used in read or write requests to this device.

   
THE WHOLE SCENARIO OF DRIVER OBJECT AND DEVICE OBJECT:

See how the device object maintains a queue and driver object relates to device object.

DEVICE AND DRIVER OBJECTS RELATED TO EACH OTHER

Working with Spin Locks :

To use a spin lock explicitly, allocate storage for a KSPIN_LOCK object in nonpaged
memory. Then call KeInitializeSpinLock to initialize the object. Later, while running at or
below DISPATCH_LEVEL, acquire the lock, perform the work that needs to be
protected from interference, and then release the lock. For example, suppose your
device extension contains a spin lock named QLock that you use for guarding access
to a special IRP queue you’ve set up. You’ll initialize this lock in your AddDevice function:

typedef struct _DEVICE_EXTENSION {

  KSPIN_LOCK QLock;//definition of spin lock in  device extension

  } DEVICE_EXTENSION, *PDEVICE_EXTENSION;

NTSTATUS AddDevice(...)

  {

  PDEVICE_EXTENSION pdx = ...;

  KeInitializeSpinLock(&pdx->QLock);//this function initializes
//the spin lock for our device.

  }

Having  defining the spin lock we can acquire it for the shared data using KeAcquireSpinLock
andafter working with it we release it using KeReleaseSpinLock.

NTSTATUS DispatchSomething(...)

  {

  KIRQL oldirql;

  PDEVICE_EXTENSION pdx = ...;

  KeAcquireSpinLock(&pdx->QLock, &oldirql);

//the code between these two functions is the shared data for
//which spin lock has to be acquired

  KeReleaseSpinLock(&pdx->QLock, oldirql);

  }

1.When KeAcquireSpinLock acquires the spin lock, it also raises IRQL to

DISPATCH_LEVEL and returns the current (that is, preacquisition) level in the

variable to which the second argument points.

2.When KeReleaseSpinLock releases the spin lock, it also lowers IRQL back to

the value specified in the second argument

Why device driver is an aspect of kernel programming?

It is because it uses kernel routines for e.g. it will use Memory Manager (prefix Mm)

component of windows to manage memory (prefix Mm means the name of the routine

of this component will start with Mm)  and similarly the executive (prefix Ex) component

of windows  supplies heap management and synchronization routines which we can use

in our device drivers and to use any routine of executive and it will be having prefix Ex

for e.g. like ExAllocatePool routine with prefix Ex defines it is a part of executive

component. Here are components of windows that we use:

1.The I/O Manager (prefix Io) contains many service functions that drivers use.

2.The Process Structure module (prefix Ps) creates and manages kernel-mode threads.

An ordinary WDM driver might use an independent thread to repeatedly poll a device

incapable of generating interrupts, and for other purposes.

3.The Memory Manager (prefix Mm) controls the page tables that define the mapping

  of virtual addresses onto physical memory.

4.The executive (prefix Ex) supplies heap management and synchronization services.

5.The Object Manager (prefix Ob) provides centralized control over the many data

  objects with which Windows XP works. WDM drivers rely on the Object Manager

  for keeping a reference count that prevents an object from disappearing while someone

   is still using it and to convert object handles to pointers to the objects the handles represent.

6.The Security Reference Monitor (prefix Se) allows file system drivers to perform security

  checks.

7.The so-called run-time library component (prefix Rtl) contains utility routines, such as list

and string-management routines, that kernel-mode drivers can use instead of regular ANSI

-standard library routines such as RtlInitUnicodeString() routine. For the most part, the

operation of these functions is obvious from their names, and you would pretty much know

how to use them in a program if you just were aware of them.

8.Windows XP implements the native API for kernel-mode callers using routine names that

begin with the prefix Zw. The DDK documents just a few of the  ZwXxx functions, namely

the ones that pertain to registry and file access.

9.The Windows XP kernel (prefix Ke) is where all the low-level synchronization of activities

between threads and processors occurs  

Basic components of WDM driver:-

A driver can be considered as a container for a collection of subroutines that the                                        operating system calls to perform various operations that relate to your hardware. These                       routines can be broadly classified into three categories namely Basic Driver Routines ,                                 I/O Control Routines and Dispatch Routine.

 

How operating system calls subroutines of driver: 

1. The user plugs in your device, so the system loads your driver executable into virtual
memory and calls your DriverEntry routine. DriverEntry does a few things  and returns.

2. The Plug and Play Manager (PnP Manager) calls your AddDevice routine, which does
a few things and returns.

3. The PnP Manager sends you a few IRPs. Your dispatch function processes each IRP in
turn and returns.

4. An application opens a handle to your device, whereupon the system sends you  another
IRP. Your dispatch routine does a little work and returns.

5. The application tries to read some data, whereupon the system sends you an IRP. Your
dispatch routine puts the IRP in a queue and returns.

6. A previous I/O operation finishes by signaling a hardware interrupt to which your driver is
connected. Your interrupt routine does a little bit of work, schedules a DPC , and returns.

7. Your DPC routine runs. Among other things, it removes the IRP you queued at step 5 , send
it to your driver and programs your hardware to read the data. Then the DPC routine returns
to the system.

8. Time passes, during which the system makes many other brief calls into your subroutines.

9. Eventually, the end user unplugs your device. The PnP Manager sends you some IRPs, which
you process and return. The operating system calls your DriverUnload routine, which usually
just does a tiny amount of work and returns. Then the system removes your driver code from
virtual memory. 

Basic Driver Routine- 

Driver Entry Routine - Every WDM driver, regardless of its purpose, has to expose
a routine whose name is DriverEntry. This is the routine called when pnp manager loads
your driver for the first device object.This routine is used for global initialization of your
driver by initializing  various driver data structures and prepares the environment for all the
other driver components.

Syntax of DriverEntry routine:

extern "C" NTSTATUS DriverEntry(IN PDRIVER_OBJECT DriverObject,

  IN PUNICODE_STRING RegistryPath)

  {

  }

IN parameter, you’re supposed to infer that it’s purely input to your function.
The first argument to DriverEntry is a pointer to a barely initialized driver
object that represents your driver.

The second argument to DriverEntry is the name of the service key in the registry
that means this key in your registry stores information about your driver service like
its name , path of files etc.

A WDM driver’s main job in DriverEntry is to fill in the various function pointers in
the driver object. These pointers indicate to the operating system where to find the
subroutines you’ve decided to place in your driver container. They include these pointer
members of the driver object:

• DriverUnload

Set this to point to whatever cleanup routine you create. The I/O Manager will
call this routine just prior to unloading the driver. If there’s nothing to clean up,
you need to have a DriverUnload function for the system to be able to unload
your driver dynamically.

• DriverExtension->AddDevice

Set this to point to your AddDevice function. The PnP Manager will call AddDevice
once for each hardware instance you’re responsible for.

• DriverStartIo

If your driver uses the standard method of queuing I/O requests, you’ll set this
member of the driver object to point to your StartIo routine.  It is discussed
about queuing I/O requests in IRP processing.

• MajorFunction

The I/O Manager initializes various IRP requests based upon their major
function codes to point to dispatch function that will serve this request.

For e.g. this statement in driver entry routine

  DriverObject->MajorFunction[IRP_MJ_PNP] = DispatchPnp;

Will initialize the IRP with major function code IRP_MJ_PNP   to be
processed by a dispatch routine named DispatchPnp.

Here is the driver entry routine defined in init.cpp module:

 Init.cpp

// init.cpp:Driver initialization code

///////////////////////////////////////////////////////////////////////////////////////////////

//DriverEntryInitialization entry point

//Wdm1UnloadUnload driver routine

//////////////////////////////////////////////////////////////////////////////////////////////

// Version history

#include “wdm1.h” //including header file which contains the definition of functions and
//definition of  driver extension which we have included here.

#pragma code_seg(”INIT”) // we have to assign our code and data in paged pool or non paged
//pool. We use pragma code_seg(“INIT”) to define the code “INIT"  in paged pool and the code is
//starting after this definition and ending till another #pragma code_seg() definiton.

/////////////////////////////////////////////////////////////////////////////////////////////

// DriverEntry;

//Description;

// This function initializes the driver, and creates

// any object needed to process I/O requests.

//

//Arguments;

// Pointer to the driver object

// registry path string for driver service key

//

// Return Value;

// This function returns STATUS_XXX  , it is discussed  in add driver routine..

extern “C”

NTSTATUS DriverEntry( IN PRIVER_OBJECT DriverObject,

IN PUNICODE_STRING RegistryPath)

{

NTSTATUS status = STATUS_SUCCESS;

#if DBG//a driver is build using check or free build . DBG is used how we want to build our
//driver (checked or build).

DebugPrintInit(“Wdm1 checked”);

#else

DebugPrintInit(“Wdm1 free”);

#endif

DebugPrint(“RegistryPath is %T”, RegistryPath);

// Export other driver entry points...

DriverObject->DriverExtension->AddDevice=Wdm1AddDevice;//initializing the add device
//routine as Wdm1AddDevice and storing its pointer at
//DriverObject->DriverExtension->AddDevice

DriverObject->DriverUnload=Wdm1Unload;//initializing the unload routine.

DriverObject->MajorFunction[IRP_MJ_CREATE]=Wdm1Create;//this will initialize an IRP
//with major function code[IRP_MJ_CREATE] to be processed by Wdm1Create routine.

DriverObject->MajorFunction[IRP_MJ_CLOSE]=Wdm1Close;

DriverObject->MajorFunction[IRP_MJ_PNP]=Wdm1Pnp;

DriverObject->MajorFunction[IRP_MJ_CREATE]=Wdm1Power;

DriverObject->MajorFunction[IRP_MJ_READ]=Wdm1Read;

DriverObject->MajorFunction[IRP_MJ_WRITE]=Wdm1Write;

DriverObject->MajorFunction[IRP_MJ_DEVICE_CONTROL]=Wdm1DeviceControl;

DriverObject->MajorFunction[IRP_MJ_SYSTEM_CONTROL]=Wdm1SystemControl;

//Initialise spin lock with protects acess to shared memory buffer. Spin locks are used to avoid
//various synchronization problems and other problems caused when many processors tries to
//access same variable.

keInitializeSpinLock(&BufferLock);

DebugPrintMsg(“DriverEntry completed”);//its just like your printf(“”) in c and is used to print

//messages on kernel.

return status;//return status whether the code completed with success or error.

}

#pragma code_seg() //end INIT section

//////////////////////////////////////

//Wdmunload

//description

//unload the driver by removing any remaining objects, etc.

//

//arguments:

//pointer to the driver object//

//return value:

//none

#pragma code_seg(“PAGE”) // start PAGE section

VOID Wdm1Unload(IN PDRIVER-OBJECT DriverObject)

{

//free buffer(do not need to acquire spin lock)

If(Buffer!=NULL)

ExFreePool(Buffer);// ExFreePool(Buffer) is used to free the memory occupied by buffer and

 // similiarly for occupying the memory we use ExAllocatePool() or ExAllocatePoolWithTag().

DebugPrintMsg(“WDM1Unload”);

DebugPrintClose();

}

//////////////////////////////////////////////////////////////////////

#pragma code_seg() // end PAGE section

The Add Device Routine: 
The adddevice routine is basically used to initialize  every device associated with
your device driver. The DriverEntry routine is basically used to globally initialize your
driver and each time a device relating to this driver is plugged the PNP manager calls
your AddDevice routine to initialize your device.
The function has the following skeleton:

NTSTATUS AddDevice(PDRIVER_OBJECT DriverObject, PDEVICE_OBJECT pdo)

    {

    return STATUS_SOMETHING;  // e.g., STATUS_SUCCESS

    }

The DriverObject argument points to the same driver object that you initialized
in your DriverEntry routine. The pdo argument is the address of the physical device
object at the bottom of the device stack.

The basic responsibility of AddDevice in a function driver is to create a device
object and link it into the stack rooted in this PDO.
The steps involved are as follows:

1. Call IoCreateDevice to create a device object and an instance of your own                                     device extension object.
2. Register one or more device interfaces so that applications know about the                               existence of your device. Alternatively, give the device object a name and then                                      create a symbolic link.
3. Next initialize your device extension and the Flags member of the device object.
4. Call IoAttachDeviceToDeviceStack to put your new device object into the stack.

Creating a Device Object

You create a device object by calling IoCreateDevice.

For example:

PDEVICE_OBJECT fdo;

NTSTATUS status = IoCreateDevice(DriverObject, 

sizeof(DEVICE_EXTENSION), NULL, FILE_DEVICE_UNKNOWN,

FILE_DEVICE_SECURE_OPEN, FALSE, &fdo);

The first argument DriverObject  establishes the connection between your driver
and the new device object, thereby allowing the I/O Manager to send you IRPs
intended for the device.

The second argument is the size of your device extension structure.

The third argument, which is NULL in this example, can be the address of a
UNICODE_STRING providing a name for the device object.

The fourth argument (FILE_DEVICE_UNKNOWN) is one of the device
types defined in WDM.H

The fifth argument (FILE_DEVICE_SECURE_OPEN) provides the
Characteristics flag for the device object.

The sixth argument to IoCreateDevice (FALSE in my example) indicates
whether the device is exclusive. The I/O Manager allows only one handle to be
opened by normal means to an exclusive device.

The last argument (&fdo) points to a location where IoCreateDevice will store
the address of the device object it creates.

How to check IoCreateDevice for an error whether it has successfully created
a device or not.

For this we use a NTSTATUS code.

An NTSTATUS code actually stores certain status codes returned by a function
indicating it has completed successfully or some type of error has occurred.

The format for it is like:

NTSTATUS status = IoCreateDevice(...); //general syntax of NTSTATUS code 

//status stores the status code.

Here are certain status codes :

STATUS_SUCCESS    //indicates that function completed successfully.

STATUS_INVALID_PARAMETER  //indicating an error has occurred relating to invalid
//parameter

There are other certain status codes each identifying a particular error as shown below:

STATUS_NO_SUCH_FILE

STATUS_ACCESS_DENIED

STATUS_INVALID_DEVICE_REQUEST

ERROR_BUFFER_TOO_SMALL

STATUS_DATA_ERROR

   

Now we can check for error by using the value of status.

if (!NT_SUCCESS(status))// NT_SUCCESS(status) returns true if it completes  successfully    
  {

  return status;

}

if (<some other error discovered>)

  {

  IoDeleteDevice(fdo);

  return status;

  }

NT_SUCCESS macro is actually used to check whether the function completed without error.

 

The second function of add device routine is to name your
device so that an application can access it. The earlier version used naming your device and then creating a symbolic
link which can be used by application to access it. For e.g. c:\ is a symbolic
name  for your hard disk partition 1(device) and this will be pointing to your

device as Device\HardDiskVolume1(name of your device).
 

The application will actually access your device using symbolic name c:\ but
this scheme has drawbacks. So we use a new technique of naming a device called
interfaces. We actually register an interface with a device which creates a
symbolic name for it. It does it by using GUID. GUID is a 128 –bit value which
is generated by running tools like UUIDGEN or GUIDGEN. Just run GUIDGEN and it
will generate a 128 –bit value for your device which we can use in our
interface. We use this 128- bit generated GUID with the help of DEFINE_GUID
macro , assigns a better mnemonic for this value and place it in our header
file to say “ guids.h”. 

For e.g.

// {CAF53C68-A94C-11d2-BB4A-00C04FA330A6} is generated GUID by GUIDGEN

//we will use it by using DEFINE_GUID macro.
DEFINE_GUID(<<name>>,
0xcaf53c68, 0xa94c, 0x11d2, 0xbb, 0x4a, 0x0, 0xc0, 0x4f,
0xa3, 0x30, 0xa6);
// name can be replaced by GUID_DEVINTERFACE_SIMPLE
//Now we can define our interface using this GUID.

#include "guids.h"  //the header file containing  the DEFINE_GUID or
//GUID_DEVINTERFACE_SIMPLE definition

  NTSTATUS AddDevice(...)

    {

    IoRegisterDeviceInterface(pdo, &GUID_DEVINTERFACE_SIMPLE,

      NULL, &pdx->ifname);   }
IoRegisterDeviceInterface registers your interface . The first parameter is pdo
representing your physical device object , and GUID_DEVINTERFACE_SIMPLE
is for GUID as represented by this mnemonic , the third parameter additional qualified
names that further subdivide your interface. The last argument is the address of a
UNICODE_STRING structure that will receive the name of a symbolic link that
resolves to this device object( the address of this symbolic name is in device extension
structure's field "ifname"so we access it as pdx ->ifname).

The return value from IoRegisterDeviceInterface is a Unicode string that applications
will be able to determine without knowing anything special about how you coded your
driver and will then be able to use in opening a handle to the device. The name is pretty
ugly, by the way; here’s an example . \\?\ROOT#UNKNOWN#0000#{b544b9a2-6995-11d3-81b5-00c04fa330a6}..

The third task is to initialize the device extension structure. To initialize the device
extension structure we should know what is a device extension structure.

It has the following syntax.

typedef struct _DEVICE_EXTENSION {

    PDEVICE_OBJECT DeviceObject;// points to related device

    PDEVICE_OBJECT LowerDeviceObject; //address for lower device object
//attached to this device (recall device and driver layers)

    PDEVICE_OBJECT Pdo;// points to the physical device (lowest in stack)

    UNICODE_STRING ifname;//here we store the symbolic name for our
//driver(as we have done in interface)

    IO_REMOVE_LOCK RemoveLock;// this is used to decide when it is safe to
//remove the device object

    DEVSTATE devstate;       // for remembering

    DEVSTATE prevstate;       // certain 

    DEVICE_POWER_STATE devpower;  // power and device

    SYSTEM_POWER_STATE syspower; //states.

    DEVICE_CAPABILITIES devcaps;//stores certain capabilities of a certain

//device as initialized by pnp manger

    } DEVICE_EXTENSION, *PDEVICE_EXTENSION;

so it's pretty clear what a device extension structure actually contains.

Here is how to initialize the value of these fields:

NTSTATUS AddDevice(...)

  {

  PDEVICE_OBJECT fdo;

  IoCreateDevice(..., sizeof(DEVICE_EXTENSION), ..., &fdo);

  PDEVICE_EXTENSION pdx = (PDEVICE_EXTENSION) fdo->DeviceExtension;
//getting access to device extension using fdo which is passed in parameter.

  

 pdx->DeviceObject = fdo;// initializing the first field with device
//object(fdo)

  pdx->Pdo = pdo;//initializing the pdo

  IoInitializeRemoveLock(&pdx->RemoveLock, ...);//function is used to
//initialize the removelcok field

  pdx->devstate = STOPPED;//setting device state as stopped

  pdx->devpower = PowerDeviceD0;//setting the state of device power

  pdx->syspower = PowerSystemWorking;

  IoRegisterDeviceInterface(..., &pdx->ifname);//storing the symbolic name

  pdx->LowerDeviceObject = IoAttachDeviceToDeviceStack(...);//attaching it to
//the lower device object.

  }

The fourth task of add device routine is of attaching it to the lower device object
and building the stack. For this we use IoAttachDeviceToDeviceStack and pass
the current device object and PDO and in turn it returns the address of lower
device object which is just below this one.

NTSTATUS AddDevice(..., PDEVICE_OBJECT pdo)

  {

  PDEVICE_OBJECT fdo;

  IoCreateDevice(..., &fdo);

  pdx->LowerDeviceObject = IoAttachDeviceToDeviceStack(fdo, pdo);

  }

Now putting all the above pieces together :

Pnp.cpp

#define INITGUID // containing your GUID definition.

#include “wdm1.h”

#pragma code_seg(“PAGE”) // start page section

//Wdm1ADDDevice:

//

//description

//cope with a new pnp device being added here usually just attach to the top of driver stack

//do not talk to device here

// arguments:

//pointers to driver object

//pointer to physical device object

//return value:

NTSTATUS Wdm1ADDDevice(IN PDRIVER-OBJECT DriverObject IN
PDEVICE_OBJECT pdo)

{

DebugPrint(“ADDDevice”);

NTSTATUS status;

PDEVICE_OBJECT fdo;

//create our functional device object in fdo

Status = IoCreateDevice(DriverObject, Sizeof(WDM1_DEVICE-EXTENSION);

NULL ,// no name

FILE_DEVICE_UNKNOWN,

FALSE, // NOT EXCLUSIVE

&fdo);

If(!NT_SUCESS(status))

Return status;

//remember fdo in our device extension

PWDM1->DEVICE_EXTENSION dx= (PWDM1->DEVICE_EXTENSION)
fdo->DeviceExtension;

dx->fdo=fdo;

DebugPrint(“FDO is %x”, fdo);

// register and enable our device interface

Status = IoRegisterDeviceInterface(pdo,&WDM1_GUID,NULL,&dx->ifSymLinkName);

If(!NT_SUCECC(status))

{

IoDeleteDevice(fdo);

Return status;

}

IoSetDeviceInterfaceState(&dx->ifSymLinkName);// starting the interface

DebugPrint(“symbolic link name is %t”, &dx-> ifSysLinkName);

//attach to the driver stack below us

dx->NextStackDevice = IoAttachDeviceToDeviceStack(fdo,pdo);

//Set fdo flags apporiately

fdo->Flags &= DO_DEVICE_INITIALIZING;// initializing the flags

fdo->Flags = DO_BUFFERED_IO;

return STATUS_SUCESS;

}

  

I/O Control Routine-  
The StartIo routine –  The driver StartIo field points to this routine. I/O
Manager calls the StartIo routine to process one IRP at a time and mark it
as busy thereafter if any IRP arrives it is placed in queue i.e. it is used in queuing
IRP. The StartIo routine needs to access hardware registers that are also used
by the interrupt service routine (ISR) and, perhaps by other routines in the driver. 

The OnInterrupt routine - It is an interrupt service routine (ISR) and is used to
process an interrupt. Suppose when device finishes processing IRP it will generate
a interrupt that its now free and send it any more queued IRP .This interrupt will be
serviced by ISR which is registered to this interrupt using OnInterrupt routine. 
The DpcForlsr routine - This ISR also registers a DPC(DEFFERED
PROCEDURE CALL) routine with it using DpcforIsr routine. This DPC routine
actually is responsible for finishing the current IRP that caused interrupt thereafter
dequeueing the next IRP and sending it to the startio routine. 

Dispatch Routine- A Dispatch routine is actually used to process an IRP based
upon driver entry routine. We have registered this routines in our driver entry routine with
certain MAJOR FUNCTION code.Every IRP with a major function code will point to a
certain dispatch routine. The dispatch routine processes the IRP and may pass it to the
lower driver. If not needed it may queue it for later processing. 

I/O REQUEST PACKET:

irp actually contains the user request i.e it represents a request for e.g. it can be

a read/write request which has to be processed by the device driver. It is passed
to the driver which is on the top of stack . The irp is processed by this driver and after processing 

it is passed to the lower driver.

irp processed by driver layers

There are two data structures to handle I/O requests: the I/O request packet itself and the IO_STACK_LOCATION structure.

 STRUCTURE OF I/O REQUEST/PACKET:

structure of i/o request packet

Here in the above image you can see two types of fields : one with grey background
and other with white background. The grey background fields are the one which cannot be

Accessed by us just like private members of the class while the  fields with white
background are the one which can be accessed directly and can be modified by us
just like to public members.

IRP data structure actually is called as IRP HEADER and contains . The header
of each IRP contains data that is used by each driver handling the IRP.

The header of each IRP contains pointers to the following:

• Buffers to read the input and write the output of the IRP.
• A memory area for the driver that currently owns the IRP.
• A routine, supplied by the driver that currently owns the IRP, which the                                             operating system calls if the IRP is cancelled.
• The parameters for the current sub-request.
• In addition to the pointers, the IRP header contains other data that describes                                       the nature and state of the request.

MdlAddress (PMDL) and UserBuffer (PVOID):  fields are basically Buffers
to read the input and write the output of the IRP.

 Flags (ULONG) , IoStatus (IO_STATUS_BLOCK) , RequestorMode ,
PendingReturned (BOOLEAN)  : contains other data that describes the
nature and state of the request.

AssociatedIrp (union): A memory area for the driver that currently owns the IRP.

Cancel (BOOLEAN) , CancelIrql (KIRQL) , CancelRoutine (PDRIVER_CANCEL):
fields are actually used for a routine, supplied by the driver that currently owns the
IRP, which the operating system calls if the IRP is cancelled.

Tail: field is actually the  parameters for the current sub-request and is a union of
ail.overlay and tail.apc and tail.completionkey and here it’s described in image.

structure of union object

Its having three parts

Tail.overlay: items at the same level as you read left to right are alternatives
within a union i.e in tail.overlay notice tail.overlay.devicequeueentry and
tail.overlay.drivercontext are at same level so they are alternatives of each other and
anyone of them can be used for same type of work.Tail.overlay.listentry is actually
used as a  linking field for irp and is used in linking an irp to another as a linked list.

I/O STACK.

With every irp there is always an associated array of IO_STACK_LOCATION
structures.This structure actually consists of  one stack location for each driver as
it will be processed downwards.This stack location actually consists of type
codes(i.e to know which type of request is the IRP for eg as read/write and these
are called major requests or minor requests) and parameter information for the IRP
as well as the address of a completion routine.

Here the image describes I/O stack and its relation with drivers.

relation between driver and i/o stack location

 

I/O stack location data structure.

As IRP data structure comes in IRP header part I/O data structure
comes in IRP parmeters section.

The IO_STACK_LOCATION structure includes the following:

• The major and minor function codes for the IRP.
• Arguments specific to these codes.
• A pointer to the device object for the corresponding driver.
• A pointer to an IoCompletion routine if the driver has set one.
• A pointer to the file object associated with the request.
• Various flags and context areas.

i/o stack location structure

MajorFunction (UCHAR): actually is in I/O stack and is used to identify what type
of request is it for e.g. whether it’s a read request by using IRP_MJ_READ.code and
using this code a paticular dispatch routine is called which has registered itself in driver
object using init routine and MinorFunction (UCHAR) is actually used to further elaborate
your major request i.e. suppose if  IRP_MJ_PNP  is major request then it can be again 
elaborated whether it’s a type of start device request or remove device request using IRP_MN_START_DEVICE, IRP_MN_REMOVE_DEVICE as minor function codes.

Parameters field: Arguments specific to these codes

Device object: pointer to the device object for the corresponding driver.

CompletionRoutine :  pointer to an IoCompletion routine if the driver has set one.

File object: pointer to the file object associated with the request.

The standard model for IRP processing:

irp processing by routines

IRP is first created by I/O manager and after  it a particular dispatch routine is called
to process it depending on the major function code  of the IRP that have been registered
with this routine in driver object. IRP in dispatch routine depending on whether the driver
is busy or not is forwarded to startio routine of driver where it is processed by a particular
ISR(INTERRUPT SERVICE ROUTINE) depending on the type of interrupt occurred
whereupon a particular DPC routine is called for finishing its completion and removing the
another waiting IRP from the queue and forwarding to startio routine until the queue ie empty.

Creating an IRP:

We can actually use these four routines to create a IRP:

·         IoBuildAsynchronousFsdRequest builds an IRP on whose completion you don’t
plan to wait. This function and the next are appropriate for building only certain types of IRP.

·         IoBuildSynchronousFsdRequest builds an IRP on whose completion you do plan to wait.

·         IoBuildDeviceIoControlRequest builds a synchronous IRP_MJ_DE­VICE_CONTROL or IRP_MJ_INTERNAL_DEVICE_CONTROL request.

·         IoAllocateIrp builds an asynchronous IRP of any type

Synchronous IRP: Its actually a IRP which depends on the  thread in which it is created i.e.
if the thread in which it is created terminates all the synchronous IRP will automatically will be cancelled.

Support Function	Types of IRP You Can Create
IoBuildSynchronousFsdRequest	IRP_MJ_READ      IRP_MJ_WRITE      IRP_MJ_FLUSH_BUFFERS      IRP_MJ_SHUTDOWN      IRP_MJ_PNP      IRP_MJ_POWER (but only for                  IRP_MN_POWER_SEQUENCE)
IoBuildDeviceIoControlRequest	IRP_MJ_DEVICE_CONTROL IRP_MJ_INTERNAL_DEVICE_CONTROL

Asynchronous IRP: Its actually an IRP which does not depends on the thread in which it is
created i.e.  the thread  termination does not leads to IRP cancellation and the IRP has its
completion routine to complete it rather than synchronous IRP whose completion depends
on completion routine of thread.

Support Function Types of IRP You Can Create IoBuildAsynchronousFsdRequest IRP_MJ_READ IRP_MJ_WRITE IRP_MJ_FLUSH_BUFFERS IRP_MJ_SHUTDOWN IRP_MJ_PNP IRP_MJ_POWER (but only for IRP_MN_POWER_SEQUENCE) IoAllocateIrp Any (but you must initialize the MajorFunction field of the first stack location)
Handling by dispatch routine: When an IRP is forwarded to an dispatch routine it handles it process it in three general ways. Immediately completing it with a return code.      2.Pass down to the lower device driver in stack    -with completion routine Or   -without completion routine        3.Queue for  later processing.    1.Completing it in a dispatch routine: completing in dispatch routine Here you complete the IRP in dispatch routine and do not pass to the lower driver i.e. when  your dispatch routine is called and sometimes you may in case find an error has occurred or you may need to complete the IRP   in dispatch routine then you can complete the IRP using IoCompleteRequest and also when you complete the IRP you set the status(for reporting whether IRP has been completed with an error or success) and information fields(for storing any information) of the IoStatus field of  the IRP datastructure. And here is the code for  it.   NTSTATUS DispatchSomething(PDEVICE_OBJECT fdo, PIRP Irp)   {   PDEVICE_EXTENSION pdx =     (PDEVICE_EXTENSION) fdo->DeviceExtension;   //<process the IRP>   Irp->IoStatus.Status = STATUS_XXX;;//setting the value of status field of     //IoStatus field of the IRP with a status code(see below for type of status  //codes).      Irp->IoStatus.Information = Information; ;//setting the value of  //information field of  IoStatus field of the IRP (for e.g. the no. of  //bytes transferred)   IoCompleteRequest(Irp, IO_NO_INCREMENT);//for completing the IRP   return status;   } Some Commonly Used NTSTATUS Codes Status Code Description STATUS_SUCCESS Normal completion. STATUS_UNSUCCESSFUL Request failed, but no other status code describes the reason specifically. STATUS_NOT_IMPLEMENTED A function hasn’t been implemented. STATUS_INVALID_HANDLE An invalid handle was supplied for an ­operation. STATUS_INVALID_PARAMETER A parameter is in error. STATUS_INVALID_DEVICE_REQUEST The request is invalid for this device. STATUS_END_OF_FILE End-of-file marker reached. STATUS_DELETE_PENDING The device is in the process of being removed from the system. STATUS_INSUFFICIENT_RESOURCES Not enough system resources (often ­memory) to perform an operation.   2. Pass down to lower driver - without completion routine Pass down to lower driver - without completion routine In this scenario you just pass down the driver to lower driver without setting a completion routine in the lower stack. We actually set a completion routine when we do care for IRP after passing to the lower driver i.e. we actually want to check for an IRP whether it is completed successfully or some error has been occurred during its processing by the lower driver and depending on this checking we determine whether a IRP requires some extra processing after being returned from the lower driver..It is actually done by setting a completion routine in the lower driver stack (note we actually set the completion routine for the current driver not in its stack location but in lower driver stack location).It means of course there will be no completion routine for the lower most driver. How to pass an IRP to the lower driver in stack? The first thing is that we have to initialize the lower stack in I/O_ STACK_LOCATION data structure for the lower driver  such as major function codes  , minor function codes,  parameters so that an IRP can be processed by the lower driver. For the same purpose we use  IoCopyCurrentIrpStackLocationToNext(Irp) . It simply copies the current major function and minor function codes to the next stack location  but does not copies the current completion routine to the next location i.e. the next stack location will have its own completion routine and then we call IoCallDriver routine to pass it to the lower driver . IoCallDriver advances the current stack pointer and now the IRP stack pointer points to the next lower stack location. Here is the code:        IoCopyCurrentIrpStackLocationToNext(Irp);        status = IoCallDriver(pdx->LowerDeviceObject, Irp);//here  //IoCallDriver has two parameters ..first one is the address of the  //lower //driver . note that pdx->LowerDeviceObject points to the lower  //driver which we have attached to our device in add device routine // using   //pdx->LowerDeviceObject = IoAttachDeviceToDeviceStack(fdo, pdo ) and other //points the IRP which is being passed.    But actually in this scenario we actually do not care for the IRP after being passed to the lower  driver that means we actually do not need  a completion routine. So there is no need of setting  a new completion routine  in the lower stack location.We therefore use IoSkipCurrentIrpStackLocation(Irp) which does not copies the  current major and minor functions  code to the lower device stack location instead it retards the stack pointer and which is again advanced by IoCallDriver so lower driver stack location  points to the current stack location  and uses the same current minor and major  function code and same completion routine without copying it.     Here is the complete code for this  scenario: NTSTATUS ForwardAndForget(PDEVICE_EXTENSION pdx, PIRP Irp)   {   PDEVICE_EXTENSION pdx =     (PDEVICE_EXTENSION) fdo->DeviceExtension;   NTSTATUS status = IoAcquireRemoveLock(&pdx->RemoveLock, Irp); //acquiring  //lock   if (!NT_SUCCESS(status))     return CompleteRequest(Irp, status);   IoSkipCurrentIrpStackLocation (Irp);   status = IoCallDriver(pdx->LowerDeviceObject, Irp);   IoReleaseRemoveLock(&pdx->RemoveLock, Irp);//releasing lock   return status;   }   For more details here is the image : difference between IoCopyCurrentIrpStackLocationToNext and IoSkipCurrentIrpStackLocation   -Pass Down with Completion Routine: As we have discussed earlier we need completion routine when we actually care for the IRP after passing down to the lower driver. So here we set up a completion routine as we want to check what happened to the IRP which we have passed below(always remember that we setthe  completion routine in lower driver stack location).IoSetCompletionRoutine is used for the very  purpose as   IoSetCompletionRoutine(Irp, CompletionRoutine, context,   InvokeOnSuccess, InvokeOnError, InvokeOnCancel); Here Irp points the IRP , completion routine points to the routine which we want called and context is is an arbitrary pointer-size value you want passed as an argument to the completion routine. InvokeOnXXX actually means when we want our completion routine to be called for e.g. InvokeOnSuccess means we want our completion routine to be called when an IRP has been completed successfully and similiarly InvokeOnerror means we want our completion routine to be called when an IRP has been completed with an error and InvokeOnCancel means we want completion routine to be called when someone has cancelled the IRP. A completion routine which we have set looks like this: NTSTATUS CompletionRoutine(PDEVICE_OBJECT fdo, PIRP Irp,   PVOID context)   {   return <some status code>;   } There are really just two possible return values from a completion routine: STATUS_MORE_PROCESSING_REQUIRED: when this value is returned  IoCompleteRequest aborts its completion routine and transfers control to the  current driver so that it can do anymore processing required. Anything else, which allows the completion process to continue. Now lets look at the completion process of the IRP down the driver .The completion process is usually initiated by IoCompleteRequest . This routine actually commence its process by using the following flowchart. how iocompleterequest works? It actually checks the for the current completion routine and if it founds one it executes it and if completion routine returns STATUS_MORE_ PROCESSING_REQUIRED  there is more processing required by  the current driver and control transfers to the current driver and  IoCompleteRequest stops its execution. While if  anything else is returned the IoCompleteRequest moves to the next stack location and again  repeat the whole procedure until it moves to the top of the stack. 3.Queue for Later Processing: The third alternative action for a dispatch routine is to queue the IRP for later processing. queuing for later processing   Here when the driver is busy you move the IRP in a queue for later processing.When the driver becomes free we remove the IRP from the queue and pass it to the StartIo routine of the driver where it is processed by a ISR(INTERRUPT SERVICE ROUTINE) depending upon the interrupt and is completed by a DPC(DEFFERRED PROCEDURE CALL)  and after completing it DPC routine removes the another waiting IRP  from the queue and pass it to the StartIo routine which has registered itself in startio field of driver object. Here is the code illustrating it: //First we have to define the queue for maintaining the IRP queue: typedef struct _DEVICE_EXTENSION {   DEVQUEUE dqReadWrite;//DEVQUEUE actually represents a queue // and is not the part of wdm header . it’s a user defined object // , here dqReadWrite is defined as queue for queuing IRP.   } DEVICE_EXTENSION, *PDEVICE_EXTENSION;   NTSTATUS AddDevice(PDRIVER_OBJECT DriverObject,   PDEVICE_OBJECT pdo)   {   InitializeQueue(&pdx->dqReadWrite, StartIo);//note queue is  //initialized in adddevice routine and the StartIo routine is  //initialized with it .   IoInitializeDpcRequest(fdo, (PIO_DPC_ROUTINE) DpcForIsr); //dpc request is //also initialized ,fdo is our device object and second is  //pointer to dpc  //routine.   }   //now dispatch routine for handling request NTSTATUS DispatchReadWrite(PDEVICE_OBJECT fdo, PIRP Irp)   {   PDEVICE_EXTENSION pdx =     (PDEVICE_EXTENSION) fdo->DeviceExtension;//accessing the driver //extension   IoMarkIrpPending(Irp); //whenever we return STATUS_PENDING we call IoMarkIrpPending(Irp) // to avoid  //an internal race condition.   StartPacket(&pdx->dqReadWrite, fdo, Irp, CancelRoutine); //startpacket routine actually pass the IRP to the StartIo routine  //if the device is not busy and if its busy it moves it to the queue. //It also registers a cancel routine.   return STATUS_PENDING;// to tell our caller that we’re not done // with this IRP yet.     } //cancel routine that we have registered. VOID CancelRoutine(PDEVICE_OBJECT fdo, PIRP Irp)   {   PDEVICE_EXTENSION pdx =     (PDEVICE_EXTENSION) fdo->DeviceExtension;   CancelRequest(&pdx->dqReadWrite, Irp);//it automatically handles  //cancelling of your requests.   } //startio routine of driver. VOID StartIo(PDEVICE_OBJECT fdo, PIRP Irp)   {   } //ISR routine to handle interrupt occurred. BOOLEAN OnInterrupt(PKINTERRUPT junk, PDEVICE_EXTENSION pdx) //the first //argument is interrupt on which it will execute , second is device  //extension   {   PIRP Irp = GetCurrentIrp(&pdx->dqReadWrite);//retrieving the current IRP //from the queue.   Irp->IoStatus.Status = STATUS_XXX;//setting the status using  //IoStatus.Status field of IRP whether it is completed with error or //without error.   Irp->IoStatus.Information = YYY;//setting the information using //IoStatus.Information field of IRP.    IoRequestDpc(pdx->DeviceObject, NULL, pdx);//calling the dpc routine for  //further processing.   }     //The DpcForIsr routine requested by your ISR receives control at DISPATCH_LEVEL. //Generally, its job is to finish up the processing of the IRP that caused the most recent //interrupt. Often that job entails calling IoComplete­Request to complete this IRP and //StartNextPacket to remove the next IRP from your device queue for forwarding to startio. VOID DpcForIsr(PKDPC Dpc PDEVICE_OBJECT fdo, PIRP junk,    PDEVICE_EXTENSION pdx)// the first argument is device object , second is //your IRP that has to be processed , third is device extension.   {   StartNextPacket(&pdx->dqSomething, fdo); //StartNextPacket to remove the next IRP from your device queue for  //forwarding to StartIo    IoCompleteRequest(Irp, boost); //IoCompleteRequest completes the IRP you specify as the first argument.  //The second argument specifies a priority boost for the thread that has  //been waiting for this IRP.   } Now we  should see the general implementation of  dispatch routine .Recall that we have registered certain dispatch routines  with major function codes . These dispatch routines are discussed here how they generally process certain IRP with major function codes. Every routine will process IRP in the ways that we have discussed above. DISPATCH.Cpp // dispatch.cpp.other IRP handler #include “wdm1.h”//including general definition of certain functions and device extension.  #include “Ioctl.h”//conatin certain control codes to control IRP. KSPIN_LOCK BufferLock;//defining spin lock . PUCHAR Buffer = NULL;// defining buffer which we will use in read/ write request. ULONG BufferSize = 0;//defining the buffer size. /////////////////////////////////////////////////////////////////////////////////// //Wdm1Create: // //description: //handle IRP_MJ_CREATE requests // //arguments: // pointers to our FDO // pointers to the IRP //IrpStack->Parameters.Create.xxx has create parameters // IrpStack->FileObject->FileName has file name of device //// //return value: //This function returns STATUS_XXX NTSTATUS Wdm1Create(IN PDEVICE_OBJECT, IN PIRP Irp)//recall IN specifies it’s //INPUT  to function { PIO_STACK_LOCATION  IrpStack = IoGetCurrentIrpStackLocation(Irp); DebugPrint(“Create File is %T”,&(IrpStack->FileObject->FileName)); //printing on kernel //the file name of device object. // Complete successfully. } //here  the first scenario of IRP handling is done i.e. it is completed here in dispatch  //routine without passing to the lower driver. Just see  this dispatch routine will be called //whenever an IRP with major function code IRP_MJ_CREATE is passed to the driver. The //driver uses the information in I/O STACK to process the IRP .See how it uses //IrpStack->FileObject->FileName to print the name of device object. This is just a demo //how an IRP is processed , how we use IO STACK information to process it. Here we //have simply use it for printing. //now moving to another MAJOR_REQUEST /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// //Wdm1Read: // //Description: //Handle IRP_MJ_READ request //// Arguments: // Pointers to our FDO // Pointers to the IRP // IrpStack->Parameters.Read.xxx has read parameters //User Buffer at:AssociatedIrp.SystemBuffer(buffered I/O) // MDIAddress(direct I/O) // // Return Value: // This function returns STATUS_XXX NTSTATUS Wdm1Read(IN PDEVICE_OBJECT fdo, IN PIRP Irp) { PIO_STACK_LOCATION IrpStack = IoGetCurrentIrpStackLocation(Irp); NTSTATUS status = STATUS_SUCCESS; LONG BytesTxd = 0; // Get call parameters LONGLONG FilePointer = IrpStack->Parameters.Read.ByteOffset.QuadPart; //setting a //pointer to the file which we have to read. ULONG ReadLen = IrpStack->Parameters.Read.Length; //retrieving the length of the data //that we have to read. DebugPrint(“Read %d bytes from file pointer %d”, (int)ReadLen,(int)FilePointer); //printing //the length and both file pointer // Get access to the shared buffer KIROL irql;// defining the interrupt request level KeAcquireSpinLock(&BufferLock,&irql);// KeAcquireSpinLock acquires the spin lock, it //also raises IRQL to DISPATCH_LEVEL and returns the current (that is, preacquisition) //level in the variable to which the second argument points. // Check File pointer If(FilePointer<0) status=STATUS_INVALID_PARAMETER; if(FilePointer>=(LONGLONG)BufferSize) status=STATUS_END_OF_FILE; if(status=STATUS_SUCCESS) { //Get transfer count If(((ULONG)FilePointer)+ReadLen>BufferSize { Bytes Txd = BufferSize – (ULONG)FilePointer; If (Bytes Txd<0)  Bytes Txd=0; } else BytesTxd = ReadLen; // Read from shared buffer If (Bytes Txd>0 && Buffer!=NULL) RtlCopyMemory(Irp->AssociatedIrp.SystemBuffer , Buffer+FilePointer , BytesTxd); } // Relaease shared buffer KeReleaseSpinLock(&BufferLock.irql); DebugPrint(“Read: %d bytes returned”, (int) BytesTxd); // Complete Irp Return CompleteIrp(Irp,status, BytesTxd); } ////////////////////////////////////////////////////////////////////////////////////////////////// //here in the above routine we have processed an IRP with major function code //IRP_MAJOR_READ. In the previous dispatch routine we have only used //parameters to print an information on kernel while here we are processing //the information passed as parameter. We are checking the size of  data that //has to be read and checking it with the size of the buffer thereafter doing some //compensation if its more than the size of buffer and then finally copying the data //to buffer. The whole thing is that we actually use the parameters as //information and after processing on this information we pass it to the lower driver How to build driver: To build our driver from our source code we need to install DDK and then run build command . We specify the path of source code in a file named SOURCES: SOURCES: TARGETNAME = Wdm1 TARGETTYPE=DRIVER DRIVERTYPE=WDM TARGETPATH=DBJ INCLUDES=$(BASEDIR) \inc SOURCES=init.cpp \ Dispatch.cpp \ Pnp.cpp \ Here we specify the path of source code (init.cpp,dispatch.cpp,pnp.cpp) which is compiled by build utility and a .sys file is build with a name Wdm1 as we have specified in TARGETNAME.The Wdm1.sys file is actually our device driver. Besides this we also include a makefile which isused in defining the driver entry of our driver. This file is also used in building our device driver. Installing device driver: After successfully compiling our device driver we have to install this driver . Installation of device driver can be done manually using registry editing. We  add the details of our driver in our registry. The details include creating a  registry key which will specify the name of our device driver as a service. We create attributes in this registry key specifying more information of our  device driver such as the path of the compiled driver represented as Wdm1.sys which will be loaded by pnp manager. The other information such as how our driver will be loaded whether manually, at windows startup or automatically. We specify other information in subkeys also such as its security features.  So after building the device driver we supply a .inf file with it which contains all the information to be added in registry. Loading/unloading your device driver: Once a driver has been registered as a system-service, it can be loaded (and unloaded) using the Service Control Manager. You can start a driver programmatically using the StartService API call, but it is far easier to goto the command-prompt and type: net start hello // the name of our device driver  The following output will then be displayed: The hello service was started successfully. Nothing else will appear to happen though because drivers don't (and can't) output any data to the console. Stopping (and unloading) a driver is as simple as starting it: net stop hello   it will call our device driver unload routine to unload  our device driver.     Note :The all above work registering , loading , or  unloading can be done  automatically using a tool Driver Loader .