Next Windows Internals Training

I’m happy to open registration for the next 5 day Windows Internals training to be conducted in November in the following dates and from 11am to 7pm, Eastern Standard Time (EST) (8am to 4pm PST): 21, 22, 28, 29, 30.

The syllabus can be found here (some modifications possible, but the general outline should remain).

Training cost is 900 USD if paid by an individual, or 1800 USD if paid by a company. Participants in any of my previous training classes get 10% off.

If you’d like to register, please send me an email to zodiacon@live.com with “Windows Internals training” in the title, provide your full name, company (if any), preferred contact email, and your time zone.

The sessions will be recorded, so you can watch any part you may be missing, or that may be somewhat overwhelming in “real time”.

As usual, if you have any questions, feel free to send me an email, or DM on twitter (@zodiacon) or Linkedin (https://www.linkedin.com/in/pavely/).

Introduction to Monikers

The foundations of the Component Object Model (COM) are made of two principles:

  1. Clients program against interfaces, never concrete classes.
  2. Location transparency – clients need not know where the actual object is (in-process, out-of-process, another machine).

Although simple in principle, there are many details involved in COM, as those with COM experience are well aware. In this post, I’d like to introduce one extensibility aspect of COM called Monikers.

The idea of a moniker is to provide some way to identify and locate specific objects based on string names instead of some custom mechanism. Windows provides some implementations of monikers, most of which are related to Object Linking and Embedding (OLE), most notably used in Microsoft Office applications. For example, when an Excel chart is embedded in a Word document as a link, an Item moniker is used to point to that specific chart using a string with a specific format understood by the moniker mechanism and the specific monikers involved. This also suggests that monikers can be combined, which is indeed the case. For example, a cell in some Excel document can be located by going to a specific sheet, then a specific range, then a specific cell – each one could be pointed to by a moniker, that when chained together can locate the required object.

Let’s start with perhaps the simplest example of an existing moniker implementation – the Class moniker. This moniker can be used to replace a creation operation. Here is an example that creates a COM object using the “standard” mechanism of calling CoCreateInstance:

#include <shlobjidl.h>
//...
CComPtr<IShellWindows> spShell;
auto hr = spShell.CoCreateInstance(__uuidof(ShellWindows));

I use the ATL smart pointers (#include <atlcomcli.h> or <atlbase.h>). The interface and class I’m using is just an example – any standard COM class would work. The CoCreateInstance method calls the real CoCreateInstance. To make it clearer, here is the CoCreateInstance call without using the helper provided by the smart pointer:

CComPtr<IShellWindows> spShell;
auto hr = ::CoCreateInstance(__uuidof(ShellWindows), nullptr, 
    CLSCTX_ALL, __uuidof(IShellWindows), 
    reinterpret_cast<void**>(&spShell));

CoCreateInstance itself is a glorified wrapper for calling CoGetClassObject to retrieve a class factory, requesting the standard IClassFactory interface, and then calling CreateInstance on it:

CComPtr<IClassFactory> spCF;
auto hr = ::CoGetClassObject(__uuidof(ShellWindows), 
    CLSCTX_ALL, nullptr, __uuidof(IClassFactory), 
    reinterpret_cast<void**>(&spCF));
if (SUCCEEDED(hr)) {
    CComPtr<IShellWindows> spShell;
    hr = spCF->CreateInstance(nullptr, __uuidof(IShellWindows),
        reinterpret_cast<void**>(&spShell));
    if (SUCCEEDED(hr)) {
        // use spShell
    }
}

Here is where the Class moniker comes in: It’s possible to get a class factory directly using a string like so:

CComPtr<IClassFactory> spCF;
BIND_OPTS opts{ sizeof(opts) };
auto hr = ::CoGetObject(
    L"clsid:9BA05972-F6A8-11CF-A442-00A0C90A8F39", 
    &opts, __uuidof(IClassFactory), 
    reinterpret_cast<void**>(&spCF));

Using CoGetObject is the most convenient way in C++ to locate an object based on a moniker. The moniker name is the string provided to CoGetObject. It starts with a ProgID of sorts followed by a colon. The rest of the string is to be interpreted by the moniker behind the scenes. With the class factory in hand, the code can use IClassFactory::CreateInstance just as with the previous example.

How does it work? As is usual with COM, the Registry is involved. If you open RegEdit or TotalRegistry and navigate to HKYE_CLASSES_ROOT, ProgIDs are all there. One of them is “clsid” – yes, it’s a bit weird perhaps, but the entry point to the moniker system is that ProgID. Each ProgID should have a CLSID subkey pointing to the class ID of the moniker. So here, the key is HKCR\CLSID\CLSID!

Class Moniker Registration

Of course, other monikers have different names (not CLSID). If we follow the CLSID on the right to the normal location for COM CLSID registration (HKCR\CLSID), this is what we find:

Class moniker

And the InProcServer32 subkey points to Combase.dll, the DLL implementing the COM infrastructure:

Class Moniker Implementation

At this point, we know how the class moniker got discovered, but it’s still not clear what is that moniker and where is it anyway?

As mentioned earlier, CoGetObject is the simplest way to get an object from a moniker, as it hides the details of the moniker itself. CoGetObject is a shortcut for calling MkParseDisplayName – the real entry point to the COM moniker namespace. Here is the full way to get a class moniker by going through the moniker:

CComPtr<IMoniker> spClsMoniker;
CComPtr<IBindCtx> spBindCtx;
::CreateBindCtx(0, &spBindCtx);
ULONG eaten;
CComPtr<IClassFactory> spCF;
auto hr = ::MkParseDisplayName(
    spBindCtx,
    L"clsid:9BA05972-F6A8-11CF-A442-00A0C90A8F39",
    &eaten, &spClsMoniker);
if (SUCCEEDED(hr)) {
    spClsMoniker->BindToObject(spBindCtx, nullptr,
        __uuidof(IClassFactory), reinterpret_cast<void**>(&spCF));

MkParseDisplayName takes a “display name” – a string, and attempts to locate the moniker based on the information in the Registry (it actually has some special code for certain OLE stuff which is not interesting in this context). The Bind Context is a helper object that can (in the general case) contain an arbitrary set of properties that can be used by the moniker to customize the way it interprets the display name. The class moniker does not use any property, but it’s still necessary to provide the object even if it has no interesting data in it. If successful, MkParseDisplayName returns the moniker interface pointer, implementing the IMoniker interface that all monikers must implement. IMoniker is somewhat a scary interface, having 20 methods (excluding IUnknown). Fortunately, not all have to be implemented. We’ll get to implementing our own moniker soon.

The primary method in IMoniker is BindToObject, which is tasked of interpreting the display name, if possible, and returning the real object that the client is trying to locate. The client provides the interface it expects the target object to implement – IClassFactory in the case of a class moniker.

You might be wondering what’s the point of the class moniker if you could simply create the required object directly with the normal class factory. One advantage of the moniker is that a string is involved, which allows “late binding” of sorts, and allows other languages, such as scripting languages, to create COM objects indirectly. For example, VBScript provides the GetObject function that calls CoGetObject.

Implementing a Moniker

Some details are still missing, such as how does the moniker object itself gets created? To show that, let’s implement our own moniker. We’ll call it the Process Moniker – its purpose is to locate a COM process object we’ll implement that allows working with a Windows Process object.

Here is an example of something a client would do to find a process object based on its PID, and then display its executable path:

BIND_OPTS opts{ sizeof(opts) };
CComPtr<IWinProcess> spProcess;
auto hr = ::CoGetObject(L"process:3284", 
    &opts, __uuidof(IWinProcess), 
    reinterpret_cast<void**>(&spProcess));
if (SUCCEEDED(hr)) {
    CComBSTR path;
    if (S_OK == spProcess->get_ImagePath(&path)) {
        printf("Image path: %ws\n", path.m_str);
    }
}

The IWinProcess is the interface our process object implements, but there is no need to know its CLSID (in fact, it has none, and is created privately by the moniker). The display name “prcess:3284” identifies the string “process” as the moniker name, meaning there must be a subkey under HKCR named “process” for this to have any chance of working. And under the “process” key there must be the CLSID of the moniker. Here is the final result:

process moniker

The CLSID of the process moniker must be registered normally like all COM classes. The text after the colon is passed to the moniker which should interpret it in a way that makes sense for that moniker (or fail trying). In our case, it’s supposed to be a PID of an existing process.

Let’s see the main steps needed to implement the process moniker. From a technical perspective, I created an ATL DLL project in Visual Studio (could be an EXE as well), and then added an “ATL Simple Object” class template to get the boilerplate code the ATL template provides. We just need to implement IMoniker – no need for some custom interface. Here is the layout of the class:

class ATL_NO_VTABLE CProcessMoniker :
	public CComObjectRootEx<CComMultiThreadModel>,
	public CComCoClass<CProcessMoniker, &CLSID_ProcessMoniker>,
	public IMoniker {
public:
	DECLARE_REGISTRY_RESOURCEID(106)
	DECLARE_CLASSFACTORY_EX(CMonikerClassFactory)

	BEGIN_COM_MAP(CProcessMoniker)
		COM_INTERFACE_ENTRY(IMoniker)
	END_COM_MAP()

	DECLARE_PROTECT_FINAL_CONSTRUCT()
	HRESULT FinalConstruct() {
		return S_OK;
	}
	void FinalRelease() {
	}

public:
	// Inherited via IMoniker
	HRESULT __stdcall GetClassID(CLSID* pClassID) override;
	HRESULT __stdcall IsDirty(void) override;
	HRESULT __stdcall Load(IStream* pStm) override;
	HRESULT __stdcall Save(IStream* pStm, BOOL fClearDirty) override;
	HRESULT __stdcall GetSizeMax(ULARGE_INTEGER* pcbSize) override;
	HRESULT __stdcall BindToObject(IBindCtx* pbc, IMoniker* pmkToLeft, REFIID riidResult, void** ppvResult) override;
    // other IMoniker methods...
	std::wstring m_DisplayName;
};

OBJECT_ENTRY_AUTO(__uuidof(ProcessMoniker), CProcessMoniker)

Those familiar with the typical code the ATL wizard generates might notice one important difference from the standard template: the class factory. It turns out that monikers are not created by an IClassFactory when called by a client invoking MkParseDisplayName (or its CoGetObject wrapper), but instead must implement the interface IParseDisplayName, which we’ll tackle in a moment. This is why DECLARE_CLASSFACTORY_EX(CMonikerClassFactory) is used to instruct ATL to use a custom class factory which we must implement.

MkParseDisplayName operation

Before we get to that, let’s implement the “main” method – BindToObject. We have to assume that the m_DisplayName member already has the process ID – it will be provided by our class factory that creates our moniker. First, we’ll convert the display name to a number:

HRESULT __stdcall CProcessMoniker::BindToObject(IBindCtx* pbc, IMoniker* pmkToLeft, REFIID riidResult, void** ppvResult) {
	auto pid = std::stoul(m_DisplayName);

Next, we’ll attempt to open a handle to the process:

auto hProcess = ::OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, 
    FALSE, pid);
if (!hProcess)
    return HRESULT_FROM_WIN32(::GetLastError());

If we fail, we just return a failed HRESULT and we’re done. If successful, we can create the WinProcess object, pass the handle and return the interface requested by the client (if supported):

	CComObject<CWinProcess>* pProcess;
	auto hr = pProcess->CreateInstance(&pProcess);
	pProcess->SetHandle(hProcess);
	pProcess->AddRef();
	
	hr = pProcess->QueryInterface(riidResult, ppvResult);
	pProcess->Release();
	return hr;
}

The creation of the object is internal via CComObject<>. The WinProcess COM class is not registered, which is just a matter of choice. I decided, a WinProcess object can only be obtained through the Process Moniker.

The calls to AddRef/Release may be puzzling, but there is a good reason for using them. When creating a CComObject<> object, the reference count of the object is zero. Then, the call to AddRef increments it to 1. Next, if the QueryInterface call succeeds, the ref count is incremented to 2. Then, the Release call decrements it to 1, as that is the correct count when the object is returned to the client. If, however, the call to QI fails, the ref count remains at 1, and the Release call will destroy the object! More elegant than calling delete.

SetHandle is a function in CWinProcess (outside the IWinProcess interface) that passes the handle to the object.

The WinProcess COM class is the uninteresting part in all of these, so I created a bare minimum class like so:

class ATL_NO_VTABLE CWinProcess :
	public CComObjectRootEx<CComMultiThreadModel>,
	public IDispatchImpl<IWinProcess> {
public:
	DECLARE_NO_REGISTRY()

	BEGIN_COM_MAP(CWinProcess)
		COM_INTERFACE_ENTRY(IWinProcess)
		COM_INTERFACE_ENTRY(IDispatch)
		COM_INTERFACE_ENTRY_AGGREGATE(IID_IMarshal, m_pUnkMarshaler.p)
	END_COM_MAP()

	DECLARE_PROTECT_FINAL_CONSTRUCT()
	DECLARE_GET_CONTROLLING_UNKNOWN()

	HRESULT FinalConstruct() {
		return CoCreateFreeThreadedMarshaler(
			GetControllingUnknown(), &m_pUnkMarshaler.p);
	}

	void FinalRelease() {
		m_pUnkMarshaler.Release();
		if (m_hProcess)
			::CloseHandle(m_hProcess);
	}

	void SetHandle(HANDLE hProcess);

private:
	HANDLE m_hProcess{ nullptr };
	CComPtr<IUnknown> m_pUnkMarshaler;

	// Inherited via IWinProcess
	HRESULT get_Id(DWORD* pId);
	HRESULT get_ImagePath(BSTR* path);
	HRESULT Terminate(DWORD exitCode);
};

The two properties and one method look like this:

void CWinProcess::SetHandle(HANDLE hProcess) {
	m_hProcess = hProcess;
}

HRESULT CWinProcess::get_Id(DWORD* pId) {
	ATLASSERT(m_hProcess);
	return *pId = ::GetProcessId(m_hProcess), S_OK;
}

HRESULT CWinProcess::get_ImagePath(BSTR* pPath) {
	WCHAR path[MAX_PATH];
	DWORD size = _countof(path);
	if (::QueryFullProcessImageName(m_hProcess, 0, path, &size))
		return CComBSTR(path).CopyTo(pPath);

	return HRESULT_FROM_WIN32(::GetLastError());
}

HRESULT CWinProcess::Terminate(DWORD exitCode) {
	HANDLE hKill;
	if (::DuplicateHandle(::GetCurrentProcess(), m_hProcess, 
		::GetCurrentProcess(), &hKill, PROCESS_TERMINATE, FALSE, 0)) {
		auto success = ::TerminateProcess(hKill, exitCode);
		auto error = ::GetLastError();
		::CloseHandle(hKill);
		return success ? S_OK : HRESULT_FROM_WIN32(error);
	}
	return HRESULT_FROM_WIN32(::GetLastError());
}

The APIs used above are fairly straightforward and of course fully documented.

The last piece of the puzzle is the moniker’s class factory:

class ATL_NO_VTABLE CMonikerClassFactory : 
	public ATL::CComObjectRootEx<ATL::CComMultiThreadModel>,
	public IParseDisplayName {
public:
	BEGIN_COM_MAP(CMonikerClassFactory)
		COM_INTERFACE_ENTRY(IParseDisplayName)
	END_COM_MAP()

	// Inherited via IParseDisplayName
	HRESULT __stdcall ParseDisplayName(IBindCtx* pbc, LPOLESTR pszDisplayName, ULONG* pchEaten, IMoniker** ppmkOut) override;
};

Just one method to implement:

HRESULT __stdcall CMonikerClassFactory::ParseDisplayName(
    IBindCtx* pbc, LPOLESTR pszDisplayName, 
    ULONG* pchEaten, IMoniker** ppmkOut) {
    auto colon = wcschr(pszDisplayName, L':');
    ATLASSERT(colon);
    if (colon == nullptr)
        return E_INVALIDARG;

    //
    // simplistic, assume all display name consumed
    //
    *pchEaten = (ULONG)wcslen(pszDisplayName);

    CComObject<CProcessMoniker>* pMon;
    auto hr = pMon->CreateInstance(&pMon);
    if (FAILED(hr))
        return hr;

    //
    // provide the process ID
    //
    pMon->m_DisplayName = colon + 1;
    pMon->AddRef();
    hr = pMon->QueryInterface(ppmkOut);
    pMon->Release();
    return hr;
}

First, the colon is searched for, as the display name looks like “process:xxxx”. The “xxxx” part is stored in the resulting moniker, created with CComObject<>, similarly to the CWinProcess earlier. The pchEaten value reports back how many characters were consumed – the moniker factory should parse as much as it understands, because moniker composition may be in play. Hopefully, I’ll discuss that in a future post.

Finally, registration must be added for the moniker. Here is ProcessMoniker.rgs, where the lower part was added to connect the “process” ProgId/moniker name to the CLSID of the process moniker:

HKCR
{
	NoRemove CLSID
	{
		ForceRemove {6ea3a80e-2936-43be-8725-2e95896da9a4} = s 'ProcessMoniker class'
		{
			InprocServer32 = s '%MODULE%'
			{
				val ThreadingModel = s 'Both'
			}
			TypeLib = s '{97a86fc5-ffef-4e80-88a0-fa3d1b438075}'
			Version = s '1.0'
		}
	}
	process = s 'Process Moniker Class'
	{
		CLSID = s '{6ea3a80e-2936-43be-8725-2e95896da9a4}'
	}
}

And that is it. Here is an example client that terminates a process given its ID:

void Kill(DWORD pid) {
	std::wstring displayName(L"process:");
	displayName += std::to_wstring(pid);
	BIND_OPTS opts{ sizeof(opts) };
	CComPtr<IWinProcess> spProcess;
	auto hr = ::CoGetObject(displayName.c_str(), &opts, 
		__uuidof(IWinProcess), reinterpret_cast<void**>(&spProcess));
	if (SUCCEEDED(hr)) {
		auto hr = spProcess->Terminate(1);
		if (SUCCEEDED(hr))
			printf("Process %u terminated.\n", pid);
		else
			printf("Error terminating process: hr=0x%X\n", hr);
	}
}

All the code can be found in this Github repo: zodiacon/MonikerFun: Demonstrating a simple moniker. (github.com)

Here is VBScript example (this works because WinProcess implements IDispatch):

set process = GetObject("process:25520")
MsgBox process.ImagePath

How about .NET or PowerShell? Here is Powershell:

PS> $p = [System.Runtime.InteropServices.Marshal]::BindToMoniker("process:25520")
PS> $p | Get-Member                                                                                             

   TypeName: System.__ComObject#{3ab0471f-2635-429d-95e9-f2baede2859e}

Name      MemberType Definition
----      ---------- ----------
Terminate Method     void Terminate (uint)
Id        Property   uint Id () {get}
ImagePath Property   string ImagePath () {get}


PS> $p.ImagePath
C:\Windows\System32\notepad.exe

The DisplayWindows function just displays names of Explorer windows obtained by using IShellWindows:

void DisplayWindows(IShellWindows* pShell) {
	long count = 0;
	pShell->get_Count(&count);
	for (long i = 0; i < count; i++) {
		CComPtr<IDispatch> spDisp;
		pShell->Item(CComVariant(i), &spDisp);
		CComQIPtr<IWebBrowserApp> spWin(spDisp);
		if (spWin) {
			CComBSTR name;
			spWin->get_LocationName(&name);
			printf("Name: %ws\n", name.m_str);
		}
	}
}

Happy Moniker day!

Zombie Processes

The term “Zombie Process” in Windows is not an official one, as far as I know. Regardless, I’ll define zombie process to be a process that has exited (for whatever reason), but at least one reference remains to the kernel process object (EPROCESS), so that the process object cannot be destroyed.

How can we recognize zombie processes? Is this even important? Let’s find out.

All kernel objects are reference counted. The reference count includes the handle count (the number of open handles to the object), and a “pointer count”, the number of kernel clients to the object that have incremented its reference count explicitly so the object is not destroyed prematurely if all handles to it are closed.

Process objects are managed within the kernel by the EPROCESS (undocumented) structure, that contains or points to everything about the process – its handle table, image name, access token, job (if any), threads, address space, etc. When a process is done executing, some aspects of the process get destroyed immediately. For example, all handles in its handle table are closed; its address space is destroyed. General properties of the process remain, however, some of which only have true meaning once a process dies, such as its exit code.

Process enumeration tools such as Task Manager or Process Explorer don’t show zombie processes, simply because the process enumeration APIs (EnumProcesses, Process32First/Process32Next, the native NtQuerySystemInformation, and WTSEnumerateProcesses) don’t return these – they only return processes that can still run code. The kernel debugger, on the other hand, shows all processes, zombie or not when you type something like !process 0 0. Identifying zombie processes is easy – their handle table and handle count is shown as zero. Here is one example:

kd> !process ffffc986a505a080 0
PROCESS ffffc986a505a080
    SessionId: 1  Cid: 1010    Peb: 37648ff000  ParentCid: 0588
    DirBase: 16484cd000  ObjectTable: 00000000  HandleCount:   0.
    Image: smartscreen.exe

Any kernel object referenced by the process object remains alive as well – such as a job (if the process is part of a job), and the process primary token (access token object). We can get more details about the process by passing the detail level “1” in the !process command:

lkd> !process ffffc986a505a080 1
PROCESS ffffc986a505a080
    SessionId: 1  Cid: 1010    Peb: 37648ff000  ParentCid: 0588
    DirBase: 16495cd000  ObjectTable: 00000000  HandleCount:   0.
    Image: smartscreen.exe
    VadRoot 0000000000000000 Vads 0 Clone 0 Private 16. Modified 7. Locked 0.
    DeviceMap ffffa2013f24aea0
    Token                             ffffa20147ded060
    ElapsedTime                       1 Day 15:11:50.174
    UserTime                          00:00:00.000
    KernelTime                        00:00:00.015
    QuotaPoolUsage[PagedPool]         0
    QuotaPoolUsage[NonPagedPool]      0
    Working Set Sizes (now,min,max)  (17, 50, 345) (68KB, 200KB, 1380KB)
    PeakWorkingSetSize                2325
    VirtualSize                       0 Mb
    PeakVirtualSize                   2101341 Mb
    PageFaultCount                    2500
    MemoryPriority                    BACKGROUND
    BasePriority                      8
    CommitCharge                      20
    Job                               ffffc98672eea060

Notice the address space does not exist anymore (VadRoot is zero). The VAD (Virtual Address Descriptors) is a data structure managed as a balanced binary search tree that describes the address space of a process – which parts are committed, which parts are reserved, etc. No address space exists anymore. Other details of the process are still there as they are direct members of the EPROCESS structure, such as the kernel and user time the process has used, its start and exit times (not shown in the debugger’s output above).

We can ask the debugger to show the reference count of any kernel object by using the generic !object command, to be followed by !trueref if there are handles open to the object:

lkd> !object ffffc986a505a080
Object: ffffc986a505a080  Type: (ffffc986478ce380) Process
    ObjectHeader: ffffc986a505a050 (new version)
    HandleCount: 1  PointerCount: 32768
lkd> !trueref ffffc986a505a080
ffffc986a505a080: HandleCount: 1 PointerCount: 32768 RealPointerCount: 1

Clearly, there is a single handle open to the process and that’s the only thing keeping it alive.

One other thing that remains is the unique process ID (shown as Cid in the above output). Process and thread IDs are generated by using a private handle table just for this purpose. This explains why process and thread IDs are always multiples of four, just like handles. In fact, the kernel treats PIDs and TIDs with the HANDLE type, rather with something like ULONG. Since there is a limit to the number of handles in a process (16711680, the reason is not described here), that’s also the limit for the number of process and threads that could exist on a system. This is a rather large number, so probably not an issue from a practical perspective, but zombie processes still keep their PIDs “taken”, so it cannot be reused. This means that in theory, some code can create millions of processes, terminate them all, but not close the handles it receives back, and eventually new processes could not be created anymore because PIDs (and TIDs) run out. I don’t know what would happen then 🙂

Here is a simple loop to do something like that by creating and destroying Notepad processes but keeping handles open:

WCHAR name[] = L"notepad";
STARTUPINFO si{ sizeof(si) };
PROCESS_INFORMATION pi;
int i = 0;
for (; i < 1000000; i++) {	// use 1 million as an example
	auto created = ::CreateProcess(nullptr, name, nullptr, nullptr,
        FALSE, 0, nullptr, nullptr, &si, &pi);
	if (!created)
		break;
	::TerminateProcess(pi.hProcess, 100);
	printf("Index: %6d PID: %u\n", i + 1, pi.dwProcessId);
	::CloseHandle(pi.hThread);
}
printf("Total: %d\n", i);

The code closes the handle to the first thread in the process, as keeping it alive would create “Zombie Threads”, much like zombie processes – threads that can no longer run any code, but still exist because at least one handle is keeping them alive.

How can we get a list of zombie processes on a system given that the “normal” tools for process enumeration don’t show them? One way of doing this is to enumerate all the process handles in the system, and check if the process pointed by that handle is truly alive by calling WaitForSingleObject on the handle (of course the handle must first be duplicated into our process so it’s valid to use) with a timeout of zero – we don’t want to wait really. If the result is WAIT_OBJECT_0, this means the process object is signaled, meaning it exited – it’s no longer capable of running any code. I have incorporated that into my Object Explorer (ObjExp.exe) tool. Here is the basic code to get details for zombie processes (the code for enumerating handles is not shown but is available in the source code):

m_Items.clear();
m_Items.reserve(128);
std::unordered_map<DWORD, size_t> processes;
for (auto const& h : ObjectManager::EnumHandles2(L"Process")) {
	auto hDup = ObjectManager::DupHandle(
        (HANDLE)(ULONG_PTR)h->HandleValue , h->ProcessId, 
        SYNCHRONIZE | PROCESS_QUERY_LIMITED_INFORMATION);
	if (hDup && WAIT_OBJECT_0 == ::WaitForSingleObject(hDup, 0)) {
		//
		// zombie process
		//
		auto pid = ::GetProcessId(hDup);
		if (pid) {
			auto it = processes.find(pid);
			ZombieProcess zp;
			auto& z = it == processes.end() ? zp : m_Items[it->second];
			z.Pid = pid;
			z.Handles.push_back({ h->HandleValue, h->ProcessId });
			WCHAR name[MAX_PATH];
			if (::GetProcessImageFileName(hDup, 
                name, _countof(name))) {
				z.FullPath = 
                    ProcessHelper::GetDosNameFromNtName(name);
				z.Name = wcsrchr(name, L'\\') + 1;
			}
			::GetProcessTimes(hDup, 
                (PFILETIME)&z.CreateTime, (PFILETIME)&z.ExitTime, 
                (PFILETIME)&z.KernelTime, (PFILETIME)&z.UserTime);
			::GetExitCodeProcess(hDup, &z.ExitCode);
			if (it == processes.end()) {
				m_Items.push_back(std::move(z));
				processes.insert({ pid, m_Items.size() - 1 });
			}
		}
	}
	if (hDup)
		::CloseHandle(hDup);
}

The data structure built for each process and stored in the m_Items vector is the following:

struct HandleEntry {
	ULONG Handle;
	DWORD Pid;
};
struct ZombieProcess {
	DWORD Pid;
	DWORD ExitCode{ 0 };
	std::wstring Name, FullPath;
	std::vector<HandleEntry> Handles;
	DWORD64 CreateTime, ExitTime, KernelTime, UserTime;
};

The ObjectManager::DupHandle function is not shown, but it basically calls DuplicateHandle for the process handle identified in some process. if that works, and the returned PID is non-zero, we can go do the work. Getting the process image name is done with GetProcessImageFileName – seems simple enough, but this function gets the NT name format of the executable (something like \Device\harddiskVolume3\Windows\System32\Notepad.exe), which is good enough if only the “short” final image name component is desired. if the full image path is needed in Win32 format (e.g. “c:\Windows\System32\notepad.exe”), it must be converted (ProcessHelper::GetDosNameFromNtName). You might be thinking that it would be far simpler to call QueryFullProcessImageName and get the Win32 name directly – but this does not work, and the function fails. Internally, the NtQueryInformationProcess native API is called with ProcessImageFileNameWin32 in the latter case, which fails if the process is a zombie one.

Running Object Explorer and selecting Zombie Processes from the System menu shows a list of all zombie processes (you should run it elevated for best results):

Object Explorer showing zombie processes

The above screenshot shows that many of the zombie processes are kept alive by GameManagerService.exe. This executable is from Razer running on my system. It definitely has a bug that keeps process handle alive way longer than needed. I’m not sure it would ever close these handles. Terminating this process will resolve the issue as the kernel closes all handles in a process handle table once the process terminates. This will allow all those processes that are held by that single handle to be freed from memory.

I plan to add Zombie Threads to Object Explorer – I wonder how many threads are being kept “alive” without good reason.

Windows 10 Desktops vs. Sysinternals Desktops

One of the new Windows 10 features visible to users is the support for additional “Desktops”. It’s now possible to create additional surfaces on which windows can be used. This idea is not new – it has been around in the Linux world for many years (e.g. KDE, Gnome), where users have 4 virtual desktops they can use. The idea is that to prevent clutter, one desktop can be used for web browsing, for example, and another desktop can be used for all dev work, and yet a third desktop could be used for all social / work apps (outlook, WhatsApp, Facebook, whatever).

To create an additional virtual desktop on Windows 10, click on the Task View button on the task bar, and then click the “New Desktop” button marked with a plus sign.

newvirtualdesktop

Now you can switch between desktops by clicking the appropriate desktop button and then launch apps as usual. It’s even possible (by clicking Task View again) to move windows from desktop to desktop, or to request that a window be visible on all desktops.

The Sysinternals tools had a tool called “Desktops” for many years now. It too allows for creation of up to 4 desktops where applications can be launched. The question is – is this Desktops tool the same as the Windows 10 virtual desktops feature? Not quite.

First, some background information. In the kernel object hierarchy under a session object, there are window stations, desktops and other objects. Here’s a diagram summarizing this tree-like relationship:

Sessions

As can be seen in the diagram, a session contains a set of Window Stations. One window station can be interactive, meaning it can receive user input, and is always called winsta0. If there are other window stations, they are non-interactive.

Each window station contains a set of desktops. Each of these desktops can hold windows. So at any given moment, an interactive user can interact with a single desktop under winsta0. Upon logging in, a desktop called “Default” is created and this is where all the normal windows appear. If you click Ctrl+Alt+Del for example, you’ll be transferred to another desktop, called “Winlogon”, that was created by the winlogon process. That’s why your normal windows “disappear” – you have been switched to another desktop where different windows may exist. This switching is done by a documented function – SwitchDesktop.

And here lies the difference between the Windows 10 virtual desktops and the Sysinternals desktops tool. The desktops tool actually creates desktop objects using the CreateDesktop API. In that desktop, it launches Explorer.exe so that a taskbar is created on that desktop – initially the desktop has nothing on it. How can desktops launch a process that by default creates windows in a different desktop? This is possible to do with the normal CreateProcess function by specifying the desktop name in the STARTUPINFO structure’s lpDesktop member. The format is “windowstation\desktop”. So in the desktops tool case, that’s something like “winsta0\Sysinternals Desktop 1”. How do I know the name of the Sysinternals desktop objects? Desktops can be enumerated with the EnumDesktops API. I’ve written a small tool, that enumerates window stations and desktops in the current session. Here’s a sample output when one additional desktop has been created with “desktops”:

desktops1

In the Windows 10 virtual desktops feature, no new desktops are ever created. Win32k.sys just manipulates the visibility of windows and that’s it. Can you guess why? Why doesn’t Window 10 use the CreateDesktop/SwitchDesktop APIs for its virtual desktop feature?

The reason has to do with some limitations that exist on desktop objects. For one, a window (technically a thread) that is bound to a desktop cannot be switched to another; in other words, there is no way to transfer a windows from one desktop to another. This is intentional, because desktops provide some protection. For example, hooks set with SetWindowsHookEx can only be set on the current desktop, so cannot affect other windows in other desktops. The Winlogon desktop, as another example, has a strict security descriptor that prevents non system-level users from accessing that desktop. Otherwise, that desktop could have been tampered with.

The virtual desktops in Windows 10 is not intended for security purposes, but for flexibility and convenience (security always “contradicts” convenience). That’s why it’s possible to move windows between desktops, because there is no real “moving” going on at all. From the kernel’s perspective, everything is still on the same “Default” desktop.

 

 

 

Next Windows Kernel Programming Remote Class

The next public remote Windows kernel Programming class I will be delivering is scheduled for April 15 to 18. It’s going to be very similar to the first one I did at the end of January (with some slight modifications and additions).

Cost: 1950 USD. Early bird (register before March 30th): 1650 USD

I have not yet finalized the time zone the class will be “targeting”. I will update in a few weeks on that.

If you’re interested in registering, please email zodiacon@live.com with the subject “Windows Kernel Programming class” and specify your name, company (if any) and time zone. I’ll reply by providing more information.

Feel free to contact me for questions using the email or through twitter (@zodiacon).

The complete syllabus is outlined below:

Duration: 4 Days
Target Audience: Experienced windows developers, interested in developing kernel mode drivers
Objectives: ·  Understand the Windows kernel driver programming model

·  Write drivers for monitoring processes, threads, registry and some types of objects

·  Use documented kernel hooking mechanisms

·  Write basic file system mini-filter drivers

Pre Requisites: ·  At least 2 years of experience working with the Windows API

·  Basic understanding of Windows OS concepts such as processes, threads, virtual memory and DLLs

Software requirements: ·  Windows 10 Pro 64 bit (latest stable version)
·  Visual Studio 2017 + latest update
·  Windows 10 SDK (latest)
·  Windows 10 WDK (latest)
·  Virtual Machine for testing and debugging

Instructor: Pavel Yosifovich

Abstract

The cyber security industry has grown considerably in recent years, with more sophisticated attacks and consequently more defenders. To have a fighting chance against these kinds of attacks, kernel mode drivers must be employed, where nothing (at least nothing from user mode) can escape their eyes.
The course provides the foundations for the most common software device drivers that are useful not just in cyber security, but also other scenarios, where monitoring and sometimes prevention of operations is required. Participants will write real device drivers with useful features they can then modify and adapt to their particular needs.

Syllabus

  • Module 1: Windows Internals quick overview
    • Processes
    • Virtual memory
    • Threads
    • System architecture
    • User / kernel transitions
    • Introduction to WinDbg
    • Windows APIs
    • Objects and handles
    • Summary

 

  • Module 2: The I/O System
    • I/O System overview
    • Device Drivers
    • The Windows Driver Model (WDM)
    • The Kernel Mode Driver Framework (KMDF)
    • Other device driver models
    • Driver types
    • Software drivers
    • Driver and device objects
    • I/O Processing and Data Flow
    • Accessing devices
    • Asynchronous I/O
    • Summary

 

  • Module 3: Kernel programming basics
    • Setting up for Kernel Development
    • Basic Kernel types and conventions
    • C++ in a kernel driver
    • Creating a driver project
    • Building and deploying
    • The kernel API
    • Strings
    • Linked Lists
    • The DriverEntry function
    • The Unload routine
    • Installation
    • Testing
    • Debugging
    • Summary
    • Lab: deploy a driver

 

  • Module 4: Building a simple driver
    • Creating a device object
    • Exporting a device name
    • Building a driver client
    • Driver dispatch routines
    • Introduction to I/O Request Packets (IRPs)
    • Completing IRPs
    • Handling DeviceIoControl calls
    • Testing the driver
    • Debugging the driver
    • Using WinDbg with a virtual machine
    • Summary
    • Lab: open a process for any access; zero driver; debug a driver

 

  • Module 5: Kernel mechanisms
    • Interrupt Request Levels (IRQLs)
    • Deferred Procedure Calls (DPCs)
    • Dispatcher objects
    • Low IRQL Synchronization
    • Spin locks
    • Work items
    • Summary

 

  • Module 6: Process and thread monitoring
    • Motivation
    • Process creation/destruction callback
    • Specifying process creation status
    • Thread creation/destruction callback
    • Notifying user mode
    • Writing a user mode client
    • Preventing potentially malicious processes from executing
    • Summary
    • Lab: monitoring process/thread activity; prevent specific processes from running; protecting processes

 

  • Module 7: Object and registry notifications
    • Process/thread object notifications
    • Pre and post callbacks
    • Registry notifications
    • Performance considerations
    • Reporting results to user mode
    • Summary
    • Lab: protect specific process from termination; simple registry monitor

 

  • Module 8: File system mini filters
    • File system model
    • Filters vs. mini filters
    • The Filter Manager
    • Filter registration
    • Pre and Post callbacks
    • File name information
    • Contexts
    • File system operations
    • Filter to user mode communication
    • Debugging mini-filters
    • Summary
    • Labs: protect a directory from file deletion; backup file before deletion

Public Windows Kernel Programming Class

After a short twitter questionaire, I’m excited to announce a Remote Windows Kernel Programming class to be scheduled for the end of January 2019 (28 to 31).

If you want to learn how to write software drivers for Windows (not hardware, plug & play drivers), including file system mini filters – this is the class for you! You should be comfortable with programming on Windows in user mode (although we’ll discuss some of the finer points of working with the Windows API) and have a basic understanding of Windows OS concepts such as processes, threads and virtual memory.

If you’re interested, send an email to zodiacon@live.com with the title “Windows Kernel Programming Training” with your name, company name (if any), and time zone. I will reply with further details.

Here is the syllabus (not final, but should be close enough):

Windows Kernel Programming

Duration: 4 Days (January 28th to 31st, 2019)
Target Audience: Experienced windows developers, interested in developing kernel mode drivers
Objectives: · Understand the Windows kernel driver programming model

· Write drivers for monitoring processes, threads, registry and some types of objects

· Use documented kernel hooking mechanisms

· Write basic file system mini-filter drivers

Pre Requisites: · At least 1 year of experience working with the Windows API

· Basic understanding of Windows OS concepts such as processes, threads, virtual memory and DLLs

Software requirements: · Windows 10 Pro 64 bit (latest official release)

· Virtual machine (preferable Windows 10 64 bit) using any virtualization technology (for testing and debugging)

· Visual Studio 2017 (any SKU) + latest update

· Windows 10 SDK (latest)

· Windows 10 WDK (latest)

Cost: $1950

Syllabus

  • Module 1: Windows Internals quick overview
    • Processes and threads
    • System architecture
    • User / kernel transitions
    • Virtual memory
    • APIs
    • Objects and handles
    • Summary

 

  • Module 2: The I/O System and Device Drivers
    • I/O System overview
    • Device Drivers
    • The Windows Driver Model (WDM)
    • The Kernel Mode Driver Framework (KMDF)
    • Other device driver models
    • Driver types
    • Software drivers
    • Driver and device objects
    • I/O Processing and Data Flow
    • Accessing files and devices
    • Asynchronous I/O
    • Summary

 

  • Module 3: Kernel programming basics
    • Installing the tools: Visual Studio, SDK, WDK
    • C++ in a kernel driver
    • Creating a driver project
    • Building and deploying
    • The kernel API
    • Strings
    • Linked Lists
    • Kernel Memory Pools
    • The DriverEntry function
    • The Unload routine
    • Installation
    • Summary
    • Lab: create a simple driver; deploy a driver

 

  • Module 4: Building a simple driver
    • Creating a device object
    • Exporting a device name
    • Building a driver client
    • Driver dispatch routines
    • Introduction to I/O Request Packets (IRPs)
    • Completing IRPs
    • Dealing with user space buffers
    • Handling DeviceIoControl calls
    • Testing the driver
    • Debugging the driver
    • Using WinDbg with a virtual machine
    • Summary
    • Lab: open a process for any access; zero driver; debug a driver

 

  • Module 5: Kernel mechanisms
    • Interrupt Request Levels (IRQLs)
    • Interrupts
    • Deferred Procedure Calls (DPCs)
    • Dispatcher objects
    • Thread Synchronization
    • Spin locks
    • Work items
    • Summary

 

  • Module 6: Process and thread monitoring
    • Process creation/destruction callback
    • Specifying process creation status
    • Thread creation/destruction callback
    • Notifying user mode
    • Writing a user mode client
    • User/kernel communication
    • Summary
    • Labs: monitoring process/thread activity; prevent specific processes from running; protecting processes

 

  • Module 7: Object and registry notifications
    • Process/thread object notifications
    • Pre and post callbacks
    • Registry notifications
    • Performance considerations
    • Reporting results to user mode
    • Summary
    • Lab: protect specific process from termination; hiding registry keys; simple registry monitor

 

  • Module 8: File system mini filters
    • File system model
    • Filters vs. mini filters
    • The Filter Manager
    • Filter registration
    • Pre and Post callbacks
    • File name information
    • Contexts
    • File system operations
    • Driver to user mode communication
    • Debugging mini-filters
    • Summary
    • Labs: protect a directory from write; hide a file/directory; prevent file/directory deletion; log file operations

 

Public Remote Windows Internals Training

  • Public 5-day remote class
  • Dates: November 5, 7, 8, 14, 15
  • Time: 8 hours / day. Exact hours TBD
  • Price: 2250 USD
  • Register by emailing zodiacon@live.com and specifying “Windows Internals Training” in the title
    • Provide names of participants (discount available for multiple participants from the same company), company name and time zone.
    • You’ll receive instructions for payment and other details
  • Virtual space is limited!

 

Objectives: Understand the Windows system architecture

Explore the internal workings of process, threads, jobs, virtual memory, the I/O system and other mechanisms fundamental to the way Windows works

Write a simple software device driver to access/modify information not available from user mode

Target Audience: Experienced windows programmers in user mode or kernel mode, interested in writing better programs, by getting a deeper understanding of the internal mechanisms of the windows operating system.

Security researchers interested in gaining a deeper understanding of Windows mechanisms (security or otherwise), allowing for more productive research

Pre-Requisites: Basic knowledge of OS concepts and architecture.

Power user level working with Windows

Practical experience developing windows applications is an advantage

C/C++ knowledge is an advantage

 

  • Module 1: System Architecture
    • Brief Windows NT History
    • Windows Versions
    • Windows 10 and Future versions
    • Tools: Windows, Sysinternals, Debugging Tools for Windows
    • Processes and Threads
    • Virtual Memory
    • User mode vs. Kernel mode
    • Objects and Handles
    • Architecture Overview
    • Key Components
    • User/kernel transitions
    • APIs: Win32, Native, .NET, COM, WinRT
    • Introduction to WinDbg
    • Lab: Task manager, Process Explorer, WinDbg

 

  • Module 2: Processes & Jobs
    • Process basics
    • Creating and terminating processes
    • Process Internals & Data Structures
    • The loader
    • DLL explicit and implicit linking
    • Process and thread attributes
    • Protected processes and PPL
    • UWP Processes
    • Minimal and Pico processes
    • Jobs
    • Nested jobs
    • Introduction to Silos
    • Lab: viewing process and job information; creating processes; setting job limits

 

  • Module 3: Threads
    • Thread basics
    • Creating and terminating threads
    • Processor Groups
    • Thread Priorities
    • Thread Scheduling
    • Thread Stacks
    • Thread States
    • CPU Sets
    • Thread Synchronization
    • Lab: creating threads; thread synchronization; viewing thread information; CPU sets

 

  • Module 4: Kernel Mechanisms
    • Trap Dispatching
    • Interrupts & Exceptions
    • System Crash
    • Object Management
    • Objects and Handles
    • Sharing Objects
    • Synchronization
    • Synchronization Primitives
    • Signaled vs. Non-Signaled
    • Windows Global Flags
    • Kernel Event Tracing
    • Wow64
    • Lab: Viewing Handles, Interrupts; creating maximum handles

 

  • Module 5: Memory Management
    • Overview
    • Small, large and huge pages
    • Page states
    • Address Space Layout
    • Address Translation Mechanisms
    • Heaps
    • APIs in User mode and Kernel mode
    • Page Faults
    • Page Files
    • Commit Size and Commit Limit
    • Workings Sets
    • Memory Mapped Files (Sections)
    • Page Frame Database
    • Other memory management features
    • Lab: committing & reserving memory; using shared memory; viewing memory related information

 

  • Module 6: Management Mechanisms
    • The Registry
    • Services
    • Starting and controlling services
    • Windows Management Instrumentation
    • Lab: Viewing and configuring services; Process Monitor

  • Module 7: I/O System
    • I/O System overview
    • Device Drivers
    • The Windows Driver Model (WDM)
    • The Windows Driver Foundation (WDF)
    • WDF: KMDF and UMDF
    • I/O Processing and Data Flow
    • IRPs
    • Plug & Play
    • Power Management
    • Driver Verifier
    • Writing a Software Driver
    • Labs: viewing driver and device information; writing a software driver

 

  • Module 8: Security
    • Security Components
    • Virtualization Based Security
    • Protecting objects
    • SIDs
    • Tokens
    • ACLs
    • Privileges
    • Access checks
    • AppContainers
    • Logon
    • User Access Control (UAC)
    • Process mitigations
    • Lab: viewing security information