Cross Process Subclassing

Submitted on: 1/5/2015 8:25:00 PM
By: Venkat Mani (from psc cd)  
Level: Advanced
User Rating: By 3 Users
Compatibility: Microsoft Visual C++, Borland C++
Views: 1780
     This article will explain how to subcalss a window which is not running in the same process as your application.In the demo example, i will subclass the notepad application window,add menus to it and react when those menu's are clicked.

This article has accompanying files

Cross ProcessSubclassing

In this article I will explain how to subclass theNotepad application window using hooks & subclassing techniques. Thistechnique can be used to build custom dll-based engines for anyapplication. In this example I will subclass the notepad applicationcreate custom menus and react when the menu is clicked. You can download ademo application which will do this

Before I start describing about cross processsubclassing ,I presume you already know what the terms hooking andwindow subclassing mean. Also I presume you have some experience workingwith hooks and subcalssing in particular and the windows SDK in general. Iwill just briefly go over these 2 techniques and explain how you couldcombine both these techniques to achieve cross-process subclassing.

Windows Hooking

This is what MSDN has to say about hooks

"In the Microsoft® Windows® operating system, a hook is a mechanism bywhich a function can intercept events (messages, mouse actions,keystrokes) before they reach an application. The function can act onevents and, in some cases, modify or discard them. Functions that receiveevents are called filter functions and are classified according tothe type of event they intercept."

You can install different types of hooks like a keyboard hook, amessage hook or a mouse hook depending upon your requirements .Hooks canbe categorized in 2 ways,

1.Global Hooks 2.Thread Specific Hooks

Global hooks are, as the name suggests Global in nature .A globalhook is installed in each and every thread that is running in thesystem. Global hooks when not properly used tend to be bulky and also slows down the system in many cases.

Thread Specific Hooks are installed in only one process. This hookcan be installed in either the same thread as the calling function (to beexplained later) or in a thread running in a different process.

Hooks are installed using the SetWindowsHookEx Api andun-installed using the UnhookWindowsHookEx Api.

Window Subclassing

Subclassing allows you to change the behavior of an existing window,typically a control, by inserting a message map to intercept the window'smessages. Subclassing is process specific, you cannot subclass awindow which is not running in the same process as your application.

Although the Windows OS does not allow us to subclass a window whichis not in the same process as our application, we can work around this byusing hooks and getting into the process space of the window that we wantto subclass.

Now let's get to building our application.

Our application will consist of 2 modules, the hooking dll whichwould install/un-install the hook and also subclass the notepadapplication and an exe which would load the hooking dll.

The Hooking Dll

The hooking dll will be used to install/uninstall windows hookand also subclass the notepad application window. Before getting into thecode for installing/uninstalling and subcalssing let's look at thefollowing piece of code

#pragma data_seg("Shared")
HWND hTarget=NULL;
int num=0 ;// Number of the subclassed window handle ,for use in the dll
bool done=FALSE;
#pragma data_seg()

The #pragma data_seg compiler directive asks the compiler to createa data segment which can be shared by all instances of the dll.The reasonwe need this is because the dll we will be loading using our exeapplication will be in one process and the dll which would eventually hookthe notepad application will be in notepad application's process.So weneed a common data segment which can be shared by different instances ofthe dll .

The code for installing the windows hook looks something like this.

int WINAPI SetHandle(HWND HandleofTarget ,HWNDHandleofApp)

return 0;
return 1;

}//End this function

We would be installing a CBT hook. A CBT (Computer Based Training)hook is used when we want to be notifed when a window iscreated/destroyed/activated/minimized etc. The hook callback procedurelooks something like this

//The CBT hook Proc(Computer Based Training Hook)

if (nCode==HCBT_ACTIVATE) //Called when the applicationwindow is activated
if((HWND)(wParam)==hTarget) //checkif the window activated is Our Target App

int count;
for(count=0;count {
if (blnsubclassed[count]==FALSE)
OldWndHndl[count]=SetWindowLong(hndll[count],GWL_WNDPROC,(long)WindowProc);//Subclass !!!!

blnsubclassed[count]=TRUE; // Set state as subclassed

if (nCode==HCBT_DESTROYWND) //Called when the applicationwindow is destroyed
SendNotifyMessage(hApp,WM_APP +1024,(WPARAM)wParam,(LPARAM)lParam);// Sendthe message to the vb app
return CallNextHookEx(NULL, nCode, wParam, lParam);
//End of the hook procedure

We subclass the notepad application as soon as it is activated thefirst time. The SetWindowLong Api is used to subclass the notepadapplication.The subclassed winodw procedure looks like this

//Window Procedures of the subclassed windows
HWND hwnd,
UINT uMsg,
WPARAM wParam,
long val;
int count;
for(count=0;count {
val=count; //this gets us the exact position of this window procedure in the array
long result;
if(uMsg==273) //Message Implying Menu Clicks
Send the message to//the hooking exe

//End Procedure

Whenever a user clicks on a menu in notepad our hooking exe isnotified and depending on which menu was clicked our application reacts tothe click.

The Hooking Exe
The hooking exe will create the menus and load the hooking dll. The codefor doing this is as follows

hHookedWindow=FindWindow(NULL,"Untitled -Notepad");
MessageBox(0,"Could Not find arunning instance of Notepad.\r\nPlease Start Notepad and try again","Error",0);
HMENU hAppMenu;
HMENU hAppendMenu;
AppendMenu(hAppMenu,MF_STRING + MF_POPUP,(unsigned int)hAppendMenu,"HTML");
AppendMenu(hAppendMenu,MF_STRING,125,"Make HTML");
AppendMenu(hAppendMenu,MF_STRING,126,"Add Line Break");
HWND hMenuWnd;
hLib = LoadLibrary("grimbo.dll");
hMenuWnd = GetWindow(hHookedWindow, 5);
DWORD hThread;
SetHandle = (sthndl)GetProcAddress(hLib, "SetHandle");
UnSubClass = (unsub)GetProcAddress(hLib, "UnSubclass");
FillHandleArray = (filhndl)GetProcAddress(hLib, "FillHandleArray");

Thus by using hooks we can subclass a window running in any process.Cross Process Subclassing should be used very carefully , unsafehooking/subclassing generally results in crashing the applicationwhich was subcalssed/hooked.In order to test the demo, compile the dll first and then place the dll in ur windows/system32 directory and then compile the exe.

winzip iconDownload article

Note: Due to the size or complexity of this submission, the author has submitted it as a .zip file to shorten your download time. Afterdownloading it, you will need a program like Winzip to decompress it.Virus note:All files are scanned once-a-day by Planet Source Code for viruses, but new viruses come out every day, so no prevention program can catch 100% of them. For your own safety, please:
  1. Re-scan downloaded files using your personal virus checker before using it.
  2. NEVER, EVER run compiled files (.exe's, .ocx's, .dll's etc.)--only run source code.

If you don't have a virus scanner, you can get one at many places on the net

Other 2 submission(s) by this author


Report Bad Submission
Use this form to tell us if this entry should be deleted (i.e contains no code, is a virus, etc.).
This submission should be removed because:

Your Vote

What do you think of this article (in the Advanced category)?
(The article with your highest vote will win this month's coding contest!)
Excellent  Good  Average  Below Average  Poor (See voting log ...)

Other User Comments

 There are no comments on this submission.

Add Your Feedback
Your feedback will be posted below and an email sent to the author. Please remember that the author was kind enough to share this with you, so any criticisms must be stated politely, or they will be deleted. (For feedback not related to this particular article, please click here instead.)

To post feedback, first please login.