Showing posts with label Internals. Show all posts

introduction

Most of the time when there is memory pressure on the system it’s easy to find the offending process by simply checking the Task Manager. However, there is one kind of memory usage that can consume Commit Charge but won’t show up on any Task Manager (including Process Explorer, Process Hacker and even opening the correct process with VMMap), it is Pagefile-Backed Sections.

Pagefile-backed sections

Sections are the underlying objects of Memory-Mapped Files and pagefile-backed sections mean that the backing store behind the memory is the Pagefile, and thus they consume Commit Charge. It makes sense to use those objects when you want to store temporary data that you don’t want constantly in memory (by creating the sections, filling them with data and then unmapping them from the process address space, that’s why VMMap doesn’t help here as well – if the sections are unmapped), an advantage of this method over simple file I/O is that you don’t leave unneeded files in case the process or the system crashes.

An example of extensive pagefile-backed sections usage is Process Monitor, that tool can capture a very large amount (many millions) of system events without pressuring physical memory using pagefile-backed sections as the event store. I once forgot that I left it open and ran out of Commit Charge and got the “your computer is low on memory. close programs to prevent information loss” error. Process Explorer couldn’t help finding the guilty process and that’s how this post was born.

Looking for pagefile-backed sections

In this great post (suggested reading for more details about Windows virtual memory management), Sysinternals’ handle.exe with the –l flag is suggested to display the size of the pagefile-backed sections that all processes has handles to, however currently that flag seems broken (crashes), so I’ll suggest alternative methods for tracking pagefile-backed sections.

Both Process Explorer and Process Hacker are capable of displaying the size of the pagefile-backed sections that a process has handles to, with this method you would have to go over suspect processes one by one.

Process explorer

First we should check Show Unnamed Handles and Mappings:

Now in the handles pane we can see unnamed pagefile-backed sections and by double-clicking them we can see their size (in hex bytes):
 

Process hacker

After choosing and double-clicking on a process, in the handles tab uncheck hide unnamed handles and similarly to Process Explorer you can view the section size:

Note that in both applications in this case you can see that the type of the section is “Reserve”  which means that the memory specified is reserved and not necessarily committed (if nothing was committed there is zero impact on Commit Charge).

Checking the Committed memory

In order to see the committed memory by the sections we can do local kernel debugging with LiveKD.
We can take the address of the Section object from either application and dump its _SECTION_OBJECT structure:

0: kd> dt nt!_SECTION_OBJECT 0xFFFFC00065CB95C0
   +0x000 StartingVa       : 0xffffd000`2213d8e8 Void
   +0x008 EndingVa         : 0xfffff802`4e4a8f75 Void
   +0x010 Parent           : 0x00000000`00000001 Void
   +0x018 LeftChild        : (null) 
   +0x020 RightChild       : (null) 
   +0x028 Segment          : 0xffffc000`5c0f66b0 _SEGMENT_OBJECT

The Segment structure contains the committed memory size:

0: kd> dt nt!_SEGMENT 0xffffc000`5c0f66b0
   +0x000 ControlArea      : 0xffffe001`43f15880 _CONTROL_AREA
   +0x008 TotalNumberOfPtes : 0x17d79
   +0x00c SegmentFlags     : _SEGMENT_FLAGS
   +0x010 NumberOfCommittedPages : 0x9c00
   +0x018 SizeOfSegment    : 0x17d79000
   +0x020 ExtendInfo       : (null) 
   +0x020 BasedAddress     : (null) 
   +0x028 SegmentLock      : _EX_PUSH_LOCK
   +0x030 u1               : <unnamed-tag>
   +0x038 u2               : <unnamed-tag>
   +0x040 PrototypePte     : (null) 

I Used _SEGMENT instead of _SEGMENT_OBJECT (that is contained in _SECTION_OBJECT’s symbols) because _SEGMENT_OBJECT’s output didn’t make sense (for example NonExtendedPtes can’t be the same size as the section in bytes, as a PTE is used for each page, which is 0x1000 bytes of memory): 

0: kd> dt nt!_SEGMENT_OBJECT 0xffffc000`5c0f66b0
   +0x000 BaseAddress      : 0xffffe001`43f15880 Void
   +0x008 TotalNumberOfPtes : 0x17d79
   +0x010 SizeOfSegment    : _LARGE_INTEGER 0x9c00
   +0x018 NonExtendedPtes  : 0x17d79000
   +0x01c ImageCommitment  : 0
   +0x020 ControlArea      : (null) 
   +0x028 Subsection       : (null) 
   +0x030 MmSectionFlags   : 0xffffe001`45e7a8c0 _MMSECTION_FLAGS
   +0x038 MmSubSectionFlags : 0x00000000`3fde0000 _MMSUBSECTION_FLAGS

So the committed memory is NumberOfCommittedPages * 0x1000 bytes, which in this case is 0x9c00000 bytes, or 156MBs.

other methods

Possible alternative methods that would not require checking each process are based on kernel debugger commands:

1. !ca 0 21 or !ca 0 28 – that command displays all system control areas related to pagefile-backed sections, control areas are structures related to file mappings, and they are pointing to Segment structures that we’ve seen above. However the 0 parameter seems currently broken (checked both with LiveKD on Windows 8.1 and live debugging on Windows 10).


2. !handle 0 1 Section – that command displays all handles in all processes pointing to section objects and it can be scripted to dump the size of the committed memory using the structures displayed above.

Introduction

A question that one may ask is "Is the Windows kernel aware of the .Net Framework?".
Assuming "No" here is very sensible because the Windows kernel should expose all of the necessary services for any software framework in the Windows API, and any kernel changes made for a software framework would hurt encapsulation.
Indeed, the entire .NET CLR is made of DLL’s that are using the Windows API, However there are few features in .NET that involve kernel modifications and I will discuss them in this post.

How .NET executables launch

.NET executables (and .NET DLLs as well) are like other exe files on Windows, they use the Portable Executable (PE) file format, and are usually launched with a CreateProcess Windows API call.
When Windows launches a native process the last step of the process startup is passing the execution control to the entry point of the executable (can be viewed with dumpbin /headers command). With .NET executables, since Windows XP, the system passes the execution to _CorExeMain function of the mscoree.dll (ignoring the entry point), which initializes the CLR and passes the control to it (On older versions of Windows the system did pass execution to the executables’s entry point which contained short code to jump to _CorExeMain).

The component that runs at the start of the process, and is responsible for initialization tasks like calling _CorExeMain, is called the NTDLL Loader, you can note that its functions start with ntdll!Ldr*.

How does windows recognize .NET executables? They contain .NET specific metadata, the part that is relevant to this post can be viewed (and modified) with the CorFlags utility.
This is the first demonstration of Windows' knowledge of .NET, but the next features are bigger.

AnyCPU Executables

The interesting feature of .NET executables that run as 64-bit  on 64-bit Windows, and as 32-bit on 32-bit windows.

The newer Prefer 32-bit option isn’t very interesting for this post, as on 64-bit windows it makes the executable behave like a 32-bit only executable when launched, thus we will refer to AnyCPU as an executable built without this flag.

How does it work?

The documentation of mscoree’s _CorValidateImage refers to changing 32-bit (PE32) executable image to 64-bit (PE32+) executable image in memory (an executable in memory is called an Image) on 64-bit Windows, and you might think that it's all to it, However that's a user mode function that is supposed to be called in the newly created process by the NTDLL Loader, the decision about the architecture of a process happens earlier, and in kernel mode (kernel structures like EPROCESS depend on it), so there has to be some kernel mode code knowing about .NET which decides which architecture to use.

Note: I debugged and _CorValidateImage isn't getting called on Windows 8.1 (on Windows 7 it does), its responsibility seems to have moved to the Loader's LdrpCorValidateImage and LdrpCorFixupImage functions.

Entering kernel mode

I attached a kernel debugger to a Windows 10 VM, I wanted to inspect the data structures related to executables in memory.

I started with finding the process (I chose ILSpy, an AnyCPU executable):

0: kd> !process 0 1 ILSpy.exe

PROCESS ffffe0001e7e57c0
    SessionId: 1  Cid: 533c    Peb: 7ff5ffb86000  ParentCid: 1808
FreezeCount 1
    DirBase: 23540f000  ObjectTable: ffffc00158aded80  HandleCount: <Data Not Accessible>
    Image: ILSpy.exe
    VadRoot ffffe0001d03dac0 Vads 12 Clone 0 Private 44. Modified 5. Locked 0.
    DeviceMap 0000000000000000
    Token                             ffffc0015c62c930
    ElapsedTime                       09:02:01.350
    UserTime                          00:00:00.000
    KernelTime                        00:00:00.000
    QuotaPoolUsage[PagedPool]         17352
    QuotaPoolUsage[NonPagedPool]      1408
    Working Set Sizes (now,min,max)  (161, 50, 345) (644KB, 200KB, 1380KB)
    PeakWorkingSetSize                148
    VirtualSize                       3 Mb
    PeakVirtualSize                   3 Mb
    PageFaultCount                    199
    MemoryPriority                    BACKGROUND
    BasePriority                      8
    CommitCharge                      70
    DebugPort                         ffffe0000e433cf0
    Job                               ffffe0001f724430

VAD's are structures that windows uses to track the virtual address space of a process, each process has a VAD root (highlighted in the output above) and it points to other VAD's (and so on) forming a tree structure that represents the virtual address space of a process. The !VAD debugger extenstion command helps analyze VAD's. By passing the VAD Root of the process, the command displays the virtual memory layout of the process:

0: kd> !vad ffffe0001d03dac0
VAD             level      start      end    commit
ffffe000127bd430 ( 3)         b90       c15         1 Mapped  Exe  EXECUTE_WRITECOPY  \path\ILSpy.exe
ffffe00012993010 ( 2)         c20       c3f        32 Private      READWRITE          
ffffe0001e92e320 ( 3)         c40       c4e         0 Mapped       READONLY           Pagefile-backed section
ffffe00018be7dc0 ( 1)         c50       d4f         5 Private      READWRITE          
ffffe00019f154c0 ( 3)         d50       d53         0 Mapped       READONLY           Pagefile-backed section
ffffe0001dc9a090 ( 2)         d60       d60         0 Mapped       READONLY           Pagefile-backed section
ffffe00019057120 ( 3)         d70       d71         2 Private      READWRITE          
ffffe0001d03dac0 ( 0)       7ffe0     7ffef        -1 Private      READONLY           
ffffe0000f121ca0 ( 2)   7ff5ffb60 7ff5ffb82         0 Mapped       READONLY           Pagefile-backed section
ffffe0000db3a5a0 ( 1)   7ff5ffb86 7ff5ffb86         1 Private      READWRITE          
ffffe00019a990f0 ( 3)   7ff5ffb8e 7ff5ffb8f         2 Private      READWRITE          
ffffe00019c043d0 ( 2)   7ffb3c5e0 7ffb3c78b        17 Mapped  Exe  EXECUTE_WRITECOPY  \Windows\System32\ntdll.dll

Total VADs: 12, average level: 3, maximum depth: 3

We can see above the VAD describing our executable image (highlighted), now when we pass it to the !VAD command with the extra parameter 1, we’ll get extra details about that specific image:

0: kd> !vad ffffe000127bd430 1

VAD @ ffffe000127bd430
  Start VPN                  b90  End VPN              c15  Control Area  ffffe000196e5d80
  FirstProtoPte ffffc00146cbfbd0  LastPte ffffc00146cbfff8  Commit Charge                1 (1.)
  Secured.Flink                0  Blink                  0  Banked/Extend                0
  File Offset                  0  
      ImageMap ViewShare EXECUTE_WRITECOPY 

ControlArea  @ ffffe000196e5d80
  Segment      ffffc001584de880  Flink      ffffe000127bd490  Blink        ffffe000126fe150
  Section Ref                 2  Pfn Ref                  83  Mapped Views                2
  User Ref                    4  WaitForDel                0  Flush Count              5ed8
  File Object  ffffe0000f1f3c80  ModWriteCount             0  System Views             ffff
  WritableRefs                0  
  Flags (a0) Image File 

      \path\ILSpy.exe

Segment @ ffffc001584de880
  ControlArea     ffffe000196e5d80  BasedAddress  0000000000400000
  Total Ptes                    86
  Segment Size               86000  Committed                    0
  Image Commit                   0  Image Info    ffffc001584de8c8
  ProtoPtes       ffffc00146cbfbd0
  Flags (1c20000) ProtectionMask 

The output contains a property named Image Info (highlighted), that seems to contain the address of some kind of a structure (addresses starting with ffff are kernel space). Now the goal is to figure out which one, unfortunately the debug symbols of the object containing it - _SEGMENT, give no info about its type:

0: kd> dt nt!_SEGMENT ffffc001584de880
   +0x000 ControlArea      : 0xffffe000`196e5d80 _CONTROL_AREA
   +0x008 TotalNumberOfPtes : 0x86
   +0x00c SegmentFlags     : _SEGMENT_FLAGS
   +0x010 NumberOfCommittedPages : 0
   +0x018 SizeOfSegment    : 0x86000
   +0x020 ExtendInfo       : 0x00000000`00400000 _MMEXTEND_INFO
   +0x020 BasedAddress     : 0x00000000`00400000 Void
   +0x028 SegmentLock      : _EX_PUSH_LOCK
   +0x030 u1               : <unnamed-tag>
   +0x038 u2               : <unnamed-tag>
   +0x040 PrototypePte     : 0xffffc001`46cbfbd0 _MMPTE
0: kd> dps ffffc001584de880
ffffc001`584de880  ffffe000`196e5d80
ffffc001`584de888  01c20000`00000086
ffffc001`584de890  00000000`00000000
ffffc001`584de898  00000000`00086000
ffffc001`584de8a0  00000000`00400000
ffffc001`584de8a8  00000000`00000000
ffffc001`584de8b0  00000000`00000000
ffffc001`584de8b8  ffffc001`584de8c8
ffffc001`584de8c0  ffffc001`46cbfbd0

The above commands show the missing symbol information (first command) and the actual raw memory (second command) containing our pointer at the same offset (0x38).

The next option is guessing the type by searching for existing symbols:

0: kd> dt nt!*image*info*
          ntkrnlmp!_MI_SECTION_IMAGE_INFORMATION
          ntkrnlmp!_SECTION_IMAGE_INFORMATION
          ntkrnlmp!_MI_EXTRA_IMAGE_INFORMATION

Trying dt with _SECTION_IMAGE_INFORMATION, and bingo, it fits (data like stack sizes makes sense):

0: kd> dt nt!_SECTION_IMAGE_INFORMATION ffffc001584de8c8
   +0x000 TransferAddress  : 0x00000000`004734be Void
   +0x008 ZeroBits         : 0
   +0x010 MaximumStackSize : 0x100000
   +0x018 CommittedStackSize : 0x1000
   +0x020 SubSystemType    : 2
   +0x024 SubSystemMinorVersion : 0
   +0x026 SubSystemMajorVersion : 4
   +0x024 SubSystemVersion : 0x40000
   +0x028 GpValue          : 0
   +0x02c ImageCharacteristics : 0x102
   +0x02e DllCharacteristics : 0x8540
   +0x030 Machine          : 0x14c
   +0x032 ImageContainsCode : 0x1 ''
   +0x033 ImageFlags       : 0x3 ''
   +0x033 ComPlusNativeReady : 0y1
   +0x033 ComPlusILOnly    : 0y1
   +0x033 ImageDynamicallyRelocated : 0y0
   +0x033 ImageMappedFlat  : 0y0
   +0x033 BaseBelow4gb     : 0y0
   +0x033 Reserved         : 0y000
   +0x034 LoaderFlags      : 1
   +0x038 ImageFileSize    : 0x83000
   +0x03c CheckSum         : 0x8859c

Immediatly it looks like it contains 2 interesting flags: ComPlusILOnly and ComPlusNativeReady.
ComPlusILOnly seems helpful, and it actually is: it simply tells us if the PE contains only IL instructions (CorFlags ILONLY flag) or it's a mixed mode (or native) executable. Obviously an executable that contains any amount of machine code (mixed or native) can't be AnyCPU, thus this flag must be set for it to work.

I couldn't really make sense of the name of ComPlusNativeReady flag but from testing it with different executables it is turned on (value is 1) when the executable is AnyCPU, the relevant CorFlags flags for that - 32BITREQ (only 32-bit) and 32BITPREF (prefer 32-bit) should be turned off, and the ILONLY flag must be turned on.

By changing these value in the debugger with eb <ImageFlagsAddres> <0,1,2,3> I observed how new processes of the same executable spawn differently (it worked because the PE headers aren't parsed each time a new process is launched, they are cached).

As for the kernel execution flow, the parsing of the executable seem to happen at the nt!MiParseImageSectionHeaders and nt!MiParseComImage functions, and the branching based on the Image Flags at nt!PspAllocateProcess and nt!PspDetectComplusILImage (checked using memory access breakpoints and some tracing).

Repositioning the image in memory

For increased security, since Windows Vista, a feature called ASLR is repositioning images in memory between boots, the reason that it happens only between boots and not for every process launch is because images should be shared between processes for best performance (think of images like ntdll.dll that every process contains and the memory that would have been wasted if they weren’t shared), and because of the fact that they contain pointers to absolute addresses, those addresses should be the same for every process. Thus all images are relocated only once per boot and otherwise every image has its own constant base address for all processes.

However, IL Only images contain no native code and thus no pointers to absolute addresses, so unlike native images there is no reason not to relocate them per process without compromising the sharing, and it actually is implemented in the kernel. Based on the ComPlusILOnly flag, the kernel decides whether to relocate the image every time the image is loaded to a process.
Below is a screen capture of Process Explorer’s DLL’s view of two processes of the same executable, note how the Base value changes. Similar relocations happen with managed IL only referenced DLL’s, unlike the Base of ntdll which stays the same.
 

Conclusion

In this post we have seen how few features in .NET required Windows kernel modifications and inspected their high level implementation with some debugging techniques, This list of features wasn’t meant to be exhaustive but those are the features that I’m currently aware of, I’ll be happy to find more such features in the feature.

Let’s imagine we’d like to inspect the memory of the SCM (services.exe) process for some reason, the best option would be taking a memory dump. However, since Windows 8.1, a hindrance was added in doing this and most other actions that touch that process, and a few other system processes. That hindrance is called Protected processes.

Introduction to protected processes

The goal of protected processes is adding another layer of protection between regular user processes (even those running as administrator) and some system processes. It started for mostly DRM related processes at Windows Vista and at Windows 8.1 it extended to other system processes. Probably the most important one is the security related process lsass.exe because its compromise may lead to the compromise of a domain network. The protection is manifested by the fact that regular (not protected) processes are unable to perform most actions on protected processes, and that includes taking a memory dump.

That system isn’t completely sealed yet: for obvious reasons the protection isn’t enforced in kernel mode and there exist few signed drivers who can help regular processes to bypass some of the protection, since regular processes running as an administrator can launch drivers.

What makes a process protected during its runtime is a field in the EPROCESS kernel structure: it used to be a bit field named ProtectedProcess since Vista, and as a few protection options were added in 8.1 it changed to a byte sized structure field named Protection. You can strip the protection of a protected process or turn a regular process into a protected one by modifying this field, which would also allow it to access a protected process.

For more in depth info on protected processes you can read this series, and this post (updated to Vista) which also displays the method of taking a memory dump of a protected process in Vista. I’ll use a similar method but adjusted to the 8.1 EPROCESS structure changes.

Taking a memory dump of a protected process

We start with local kernel debugging, let’s assume we want to take the memory dump of services.exe with Task Manager (TaskMgr.exe).

We first need to get the offset of the Protection field:

0: kd> dt _EPROCESS -y Protection
nt!_EPROCESS
   +0x67a Protection : _PS_PROTECTION

Next we want to check the protection of services.exe to adjust Task Manager’s protection to it (simply setting its protection field to a non-zero value may not be enough as there is some isolation between different types of protected processes, more on this in the series I linked above), so we’ll find its EPROCESS address:

0: kd> !process 0 0 services.exe 
PROCESS ffffe0001a644900
    SessionId: 0  Cid: 0304    Peb: 7ff65bf4c000  ParentCid: 0298
    DirBase: 1520e1000  ObjectTable: ffffc00088b4ad80  HandleCount: <Data Not Accessible>
    Image: services.exe

Now we can dump its Protection field:

0: kd> dt _PS_PROTECTION ffffe0001a644900+0x67a
nt!_PS_PROTECTION
   +0x000 Level            : 0x61 'a'
   +0x000 Type             : 0y001
   +0x000 Audit            : 0y0
   +0x000 Signer           : 0y0110

Now let’s find Task Manager’s EPROCESS location:

0: kd> !process 0 0 TaskMgr.exe 
PROCESS ffffe0001d489080
    SessionId: 1  Cid: 1b20    Peb: 7ff65480f000  ParentCid: 0418
    DirBase: 17ad18000  ObjectTable: ffffc0009f0a9380  HandleCount: <Data Not Accessible>
    Image: Taskmgr.exe

Before we change its protection let’s check its current value:

0: kd> dt _PS_PROTECTION ffffe0001d489080+0x67a
nt!_PS_PROTECTION
   +0x000 Level            : 0 ''
   +0x000 Type             : 0y000
   +0x000 Audit            : 0y0
   +0x000 Signer           : 0y0000

So there’s no protection, Now we can adjust it to the protection of services.exe:

0: kd> eb ffffe0001d489080+0x67a 0x61

Finally we can use the right click menu to create a dump of services.exe: