Thinking Kernel Mode

Introducing easy-to-use Web Scraper : WebHarvy

Sreejith S — Mon, 13 Feb 2012 08:47:12 GMT

Check out WebHarvy Web Scraper to see how easy it is to scrape online data to an XML, CSV or TSV file. WebHarvy Web Scraper also allows you to export the extracted data to an SQL database. You may watch the video demo of WebHarvy at http://www.webharvy.com/demo.html .

Web Scraping is the process of extracting data from remote websites to local storage in spreadsheet / table format. WebHarvy is a visual web scraper which helps users to extract data from websites using mouse clicks instead of writing scripts or complex code.

Keywords : Web Scraper, Web Scraping, Screen Scraping, Web Data Extraction, Screen Scraper

Share USB devices between 32 bit and 64 bit systems

Sreejith S — Fri, 18 Feb 2011 08:28:21 GMT

The latest version of USBDeviceShare, the USB over IP software, supports sharing USB devices between 32 bit and 64 bit editions of Windows. A USB device plugged in to a 64 bit PC running 64 bit version of Windows can be shared and remotely accessed from a 32 bit PC and vice versa.

Download USBDeviceShare USB over Network software from http://www.sysnucleus.com/usbshare/usbshare_download.html

Keywords : USB over Ethernet, USB Server, Remote USB Access, Share USB

Installing drivers using DPInst

Sreejith S — Tue, 04 May 2010 10:11:19 GMT

You can use DPInst (Driver Package Installer) to install driver files for your device on Windows Operating Systems. DPInst.exe is part of DIFx (Driver Install Framework tools) which comes along with the Windows Driver Kit (WDK). You can find DPInst redistributable inside the redist\DIFx\DPInst folder of the WDK.

Using DPInst to install driver:
1) http://msdn.microsoft.com/en-us/library/ff553601(VS.85).aspx
2) http://blogs.technet.com/svengruenitz/archive/2008/07/02/driver-installation-and-updating-made-easy-dpinst-exe.aspx

How to ?

1. Copy dpinst.exe to the folder where the driver files (sys, inf, dll) reside.

2. Optionally create a dpinst.xml file which is used to configure dpinst (http://msdn.microsoft.com/en-us/library/ff550803(VS.85).aspx)

If your drivers are not signed you must use the <legacyMode/> tag in dpinst.xml.

3. Run dpinst.exe to install drivers using a 'device driver update' wizard.

Access USB over Network

Sreejith S — Wed, 23 Sep 2009 04:20:08 GMT

Today most devices connect to the PC via USB. What if you need to share a USB device with another person at home or at work. What you normally do is unplug the device and hand it over to the other person. Or you can let him/her use your computer where the device is present. What if the device is not portable ? What if you and your friend are located at different parts of the world and need to share a device ? What if the device needs to be shared on a timely basis among a number of persons, so that each one can access the device when it becomes available ? The device can be a Scanner, a hardware protection dongle, an Oscilloscope, a medical sensor or a development board, it can be any USB device. Enter USBDeviceShare, the hassle-free and easy way to share and access USB devices remotely over network (or internet).

USBDeviceShare has 2 components : a server and a client. Install server where devices are physically present and where they need to be shared. Install client where devices need to be remotely accessed. The remote devices can be virtually plugged in to your PC using the client. Applications which work with these devices can then be run as if the devices are locally connected.

Another important application of USBDeviceShare is in virtualization. USB support, i.e ability to access USB ports of host machines, is not present or is unstable in most virtual machines. The solution is to install USBDeviceShare server in the host OS and run client in Guest OSes (VMs). The clients can connect and access the USB devices connected to the host OS.

To know more please visit http://www.sysnucleus.com/usbshare/.

Keywords : USB over Network, USB over IP, USB over Ethernet, Remote USB, Share USB Device

The new software upgrade mantra

Sreejith S — Mon, 07 Sep 2009 05:42:36 GMT

"Smaller, Faster, Smarter"

Every successful software once reaches a stage where adding more features makes the application 'Bigger, Slower and Lamer". Once that critical mass has been reached, trying to cash in by adding more features leads to a process called 'feature creep'(http://en.wikipedia.org/wiki/Feature_creep).

What Microsoft and Apple are trying to do with their latest OS releases (Windows 7 and Snow Leopard) is to offer an upgrade which claims to free up more disk space (smaller), run faster (more optimized, efficient).

Read http://www.nytimes.com/2009/08/27/technology/personaltech/27pogue.html?pagewanted=1&_r=3&ref=technology .

Preinstalling Driver files

Sreejith S — Tue, 25 Aug 2009 06:10:11 GMT

Often we are required to write applications (installers) which installs the driver files associated with a device even before the device is actually plugged in to the system. The process is called 'preinstalling' driver packages.

In order to preinstall a driver package, use the SetupCopyOEMInf API. Uninstallation can be done using the SetupUninstallOEMInf API. Look here http://msdn.microsoft.com/en-us/library/aa906201.aspx for more details.

Programatically update the driver for a device

Sreejith S — Wed, 12 Nov 2008 09:00:02 GMT

So you need to change the driver loaded for a given device from your application. You have a couple of options to do this.

The first method is to use the UpdateDriverForPlugAndPlayDevices API. This method is simple and straightforward to use. But the downside of using this API is that it will update drivers for all devices with the specified hardware id. So if you need to selectively update one of the devices with a new driver you will have to look for something else.

And that method is by using the InstalledSelectedDriver API.

Sample Code: How to use InstallSelectedDriver to update driver ?

USB Monitor for Windows

Sreejith S — Tue, 29 Apr 2008 03:50:19 GMT

This post is about USBTrace, a software based USB monitor / USB sniffer , which can be used to capture & analyze transactions happening in the USB bus. This tool is helpful for USB firmware/device driver developers while debugging and testing their device implementation. USB requests are captured and displayed in easly readable format to make analyzing easy. USB requests passing though different layers of the USB device stack are captured and displayed. Helps developers to highly reduce their debug session's time. Supports USB 1.x and 2.0 host controllers, hubs and devices.

Download : http://www.sysnucleus.com/usbtrace_download.html

Keywords : USB Monitor , USB Analyzer , USB Protocol Analyzer , USB Sniffer

X64 Assembly Code in Windows Drivers

Sreejith S — Fri, 07 Mar 2008 08:27:23 GMT

Writing asm code for 32 bit drivers is straightforward. You can embed the code in an __asm { } block.

void DemoFunction()
{
    __asm
    {
          mov eax, 0x01
          ; more assembly
    }
}

But writing assembly in 64 bit driver source code requires a bit more work. The 64 bit compiler will not allow inline assembly. The assembly code will have to be moved to a seperate assembly module (an .asm file).

Step 1 : Write necessary assembly routines in a seperate .asm file

Example : Test.asm
------------------------------------------

.data

; all data variables in your asm code goes here
myData1   dq   0   ; 64 bit data

.code

; all assembly routines go here

TestFunction PROC

    ; sample function/routine/procedure

    ; assembly code for the function goes here

    ret

TestFunction ENDP

END ; end of assembly file

Step2 : Integrate assembly function with C

In one of your C header files declare the function:

extern void TestFunction(void);

Step 3 : Adding asm file to sources file

In the sources file of your driver you can add the .asm file along with other C files.

Example:

SOURCES = init. c \
ioctl.c \
pnp.c\
power.c\
Test.asm

You can add the same under AMD64_SOURCES or IA64_SOURCES if you required to include the same only in those specific architectures.

X64 Calling

Sreejith S — Fri, 07 Mar 2008 01:13:28 GMT

Computers with 64 bit processors are becoming popular, at least in enterprise circles. Also the X64 version of Windows Vista is more popular than Windows XP 64 bit edition.

2 Types of 64 bit architecture

Yes, unlike 32 bit (aka X32) there are 2 64 bit architectures:

AMD's X64 ; also known as X32-64 & AMD64
Intel's IA-64; also known as Itanium

The subject of this post is AMD's 64 bit chip, which is commonly referred to as X64

So, whats the difference ?

The X64 architecture is a super set of X32 architecture :

64 bit versions of the the existing 32 bit registers
- So X32's 32 bit registers EAX, EBX, ECX etc becomes 64 bit RAX, RBX, RCX etc in X64
8 new 64 bit general purpose registers (R8, R9...R15)
8 new 128 bit XMM registers

To know more about the architecture goto http://www.amd.com/us-en/Processors/DevelopWithAMD/0,,30_2252_875_7044,00.html

And how is programing different ?

A lot has been written about porting existing 32 bit code to 64 bit. A lot of these deals with change in sizeof pointers (from 32 bit to 64 bit) and change in the sizeof some of the basic data types (this depends on the compiler which you are using)

Links:

Porting device drivers to AMD64
[http://www.amd.com/us-en/assets/content_type/DownloadableAssets/dwamd_Porting_Win_DD_to_AMD64_Sept24.pdf]

64 bit driver guidelines
[http://www.microsoft.com/whdc/driver/64bitguide.mspx]

20 issues of porting C++ code on the 64-bit platform
[http://www.viva64.com/articles/20_issues_of_porting_C++_code_on_the_64-bit_platform.html]

..assembly programming ?

Not many program in assembly languages these days. But if it occurs to you there are a few things to keep in mind.

Source Link : http://www.quequero.org/X64_Assembly

Win32 on X32 provided us with many calling conventions (function calling conventions : fastcall, stdcall etc). In X64 there is no choice. There is only one calling convention:

The first parameter is the rcx register, the second one rdx, the third r8 and the fourth r9. Saying that the parameters registers are part of the stack frame, makes it also clear that any function that calls another child function has to initialize the stack providing space for these four registers, even if the parameters passed to the child function are less than four.
The initialization of the stack pointer is done only in the prologue of a function, it has to be large enough to hold all the arguments passed to child functions and it's always a duty of the caller to clean the stack. Now, the most important thing to understand how the space is provided in the stack frame is that the stack has to be 16-byte aligned.
In fact, the return address has to be aligned to 16 bytes. So, the stack space will always be something like 16n + 8, where n depends on the number of parameters. Here's a small figure of a stack frame:

Stack Parameters (5th param onwards)

Return IP address (RIP)

Local Variables of the function

If you see the disassembly of a 64 bit program, you can see that the stack pointer (RSP) is not messed with throughout the function body. Necessary stack is reserverd ( Sub RSP, 0x[ReqSize] ) in the function prolog.

Another important thing to note is that even though the first 4 parameters are passed via registerd (RCX, RDX, R8 and R9) they must be given scratch storage space in the stack (Register Parameters in the above figure/call stack).

So while porting asm from 32bit to 64bit, if you have a void routine

call MyRoutine

must be changed as:

sub rsp, 20h ; Reserve space for register parameters

call MyRoutine

add rsp, 20h

Another difference found was in X64 the luxury of PUSHA/PUSHD POPA/POPD (Push/Pop all registers and flags) is not available.

Footnotes

MS Visual Studio 2005 lets you build 64 bit applications. The 64 bit compiler modules are not included in the installation by default.
VS 2005 has options to turn on 64bit compile warning (probable errors) for your 32 bit code; so you can check whether your 32bit code is 64bit ready
Windows Server 2003 DDK and above comes with MASM64 for writing asm modules.
32 bit applications can run over 64 bit windows (WOW64 http://en.wikipedia.org/wiki/WOW64)
32 bit drivers cannot be used to 64 bit windows.

Kernel Mode Driver Framework

Sreejith S — Tue, 13 Dec 2005 18:54:00 GMT

New Page 1

The Windows Driver Foundation (WDF) Kernel Mode Driver Framework (KMDF) is out.

Get it from:
http://www.microsoft.com/whdc/driver/wdf/KMDF_pkg.mspx

Windows Driver Foundation is the next-generation driver model, which removes a lot of complexity which exists in the current driver model (WDM). WDF achieves this not by replacing WDM, but by providing a framework which runs over WDM. The framework frees the developer from dealing directly with the operating system and allows him to concentrate more on his device (hardware).

Read more about WDF:

http://www.microsoft.com/whdc/driver/wdf/wdf-arch.mspx (WDF Architecture)

http://www.microsoft.com/whdc/driver/wdf/default.mspx (WHDC WDF page)

In search of stupidity

Sreejith S — Mon, 14 Nov 2005 19:17:00 GMT

New Page 1

"Sony's attempt at protecting its music cds from piracy has turned into a tale of security woes that has quickly gone from bad to worse. New software on several of the company's cds installs a copy protection rootkit on a user's PC once the license agreement on the disc is accepted. Virus writers jumped on the fact that Sony's rootkit hides itself on users' computers, and a few Trojan horses have been released that piggyback on the software, effectively hiding from antivirus software"

http://news.com.com/Wills+Sonys+DRM+nightmare+affect+future+policies/2009-1029_3-5947274.html?tag=fd_carsl

Writing a virtual audio driver

Sreejith S — Wed, 26 Oct 2005 14:41:00 GMT

New Page 1

{ Not invented here }

Unless I scribble this down somewhere, I will forget. So let me dump my memory . .

Kernel Streaming is a fairly complicated topic. And to study this, the available resources are the DDK help documentation and the samples. AFAIK, there are no books written on this topic. And what is present online is also meager.

The Streaming Devices (Video & Audio) subsection of the DDK help, has everything documented.

Now, in order to emulate a virtual audio hardware, we have to write an Audio Miniport Adapter driver. The MSAVD sample source code can be used for this purpose.

WHDC Audio Device Technologies for Windows [http://www.microsoft.com/whdc/device/audio/default.mspx]

Getting Started with WDM Audio Drivers [http://www.microsoft.com/whdc/device/audio/wdmaud-drv.mspx]

This is how the audio device stack will look like.

System Audio Device (SysAudio.sys)
Kernel Mixer (Kmix.sys)
Streaming Class Driver (stream.sys)	Port Class Driver (PortCls.sys)
USB Audio (USBAudio.sys)
USB Device	Adapter Driver
USB Controller	Audio Device

The Adapter Driver is what we write and for a virtual audio device it will be a modified MSAVD driver.

Modifying the MSAVD Driver to make your own virtual audio device.

In Kernel Streaming, every component is a Filter. Filters have input (sink) and output (source) pins. Data enters a filter through its input pin and leaves through the output pin. Every filter has a purpose : to process the data flowing through them.

As an audio device (virtual or real), your hardware will have a range of audio data parameters which can be accepted. Or in other words, only data complying to a specific format and parameter range can be accepted by the input pins. This is called the property of the pin.

Specify those ranges in the PinDataRangesStream static variable of the MSAVD source code.

Now, in the WaveCyclic miniport there are two methods named CopyTo and CopyFrom for copying audio data to and from the audio device's memory (DMA). Add code to those function to implement a virtual write and read and the minimal audio driver code is complete.

Entry Point

Sreejith S — Wed, 19 Oct 2005 18:48:00 GMT

New Page 1

The best way to describe this space is :

scrap paper - loose sheets of paper, often already partly used, for writing notes on.

For many reasons I feel the current timeslot to be a highly active one. A lot of things are happening. Applications are moving away from the PC to the internet. Computing is becoming more and more pervasive. The size of information we used to manage is getting bigger and bigger. There was a period when I used to wonder why the chip makers always try to make the clock run faster and faster.

The arena dominated by a single mammoth is now history. Apple and Google are now getting more personal than Microsoft. The future is for everyone.

I am not a visionary. I still remember the day when Digger2 told me that MS is not going to be a monopoly for all time. And that, something and someone better will come up one day.

Since I spend majority of the time writing device drivers, many of the entries (if this weblog lives) will be related to that. The rest will reflect my views on the current tech world.

I read news.com.com regularly :).

And I am not going to spell check here.

Allocating a synchronization primitive

Sreejith S — Thu, 20 Oct 2005 18:11:00 GMT

New Page 1

Never allocate a dispatcher object from PagedPool. Which means, don't even think about doing a

KSPIN_LOCK SpinLock;

within your PAGED_CODE.

If a dispatcher object is allocated from PagedPool, the driver might work peacefully, but occasionally the system will bugcheck saying that it tried to access paged memory from higher IRQL.

So,

Allocate them explicitly from NonPagedPool .
PKSPIN_LOCK pSpinLock = ExAllocatePool(NonPagedPool, sizeof(struct KSPIN_LOCK));

Or declare them globally. (All global variables in a kernel mode driver are allocated from NonPagedPool).

You can place them in the device extension of the driver created device object. The device extension is always allocated from NonPaged memory.

You can also place them in the controller extension of the driver created controller object.