Why isn’t the !bpmd in sos / windbg not working?


I recently noticed another blog post refer to one of my post. The issue was, sos wasn’t enabling the break-points on non-jitted functions. The classic example being “Main”.  Thanks to Steve I have been using sosex and not sos for setting break-points.

From my previous post you can understand how CLR is using clrn/CLRNotificationException to notify sos/sosex on JIT. With this information when I looked at the rotor code, I noticed an interesting member variable “g_dacNotificationFlags”. So I decided to check the value of this variable when using !bpmd from sos and !mbm from sosex.


.if (dwo(mscorwks!g_dacNotificationFlags) = 0) {.echo bp not set } .else {.echo bp set}

It was “0″ when using sos and “1″ when using sosex. Now I had to change the value to “1″ and check if the break-point becomes active when using sos’s !bpmd.  FYI I don’t have private symbols and haven’t seen CLR Code. Here is the code to set the value to “1″.


ed mscorwks!g_dacNotificationFlags 00000001

And not to my surprise the !bpmd seems to work for non-jitted function with the above hack. FYI we don’t have to resort to this to get !bpmd to work. If the !bpmd is set after load of mscorjit/clrjit it would work as expected.

Using sosex within windbg to understand IL and Assembly code


Sometimes when debugging managed code within the debugger I would like to see the C# code ,the IL translation for the managed code and the Assembly code for the IL. For example I recently learned that callvirt MSIL instruction must do the null-check before invoking method.

C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication13\Program.cs @ 18:
00bc26d8 8b4dec          mov     ecx,dword ptr [ebp-14h]
00bc26db 3909            cmp     dword ptr [ecx],ecx //NULL Check
00bc26dd ff1508a82900    call    dword ptr ds:[29A808h] (System.String.ToLower(), mdToken: 0600031d)
00bc26e3 8945e8          mov     dword ptr [ebp-18h],eax
00bc26e6 8b45e8          mov     eax,dword ptr [ebp-18h]
00bc26e9 8945ec          mov     dword ptr [ebp-14h],eax

I am not an assembly code expert. The above output is from “!u” sos command. It doesn’t show the c# code except the line number and it is missing IL translation.

The “!mu” from sosex does what I want. It is not yet documented because it is not yet stable as per the output of the command. Here is the output for the same call-stack using sosex’s !mu.

0:000> !mu
THIS COMMAND IS UNDOCUMENTED AND NOT YET STABLE.
test = test.ToLower();
IL_001a: ldloc.0  (test)
IL_001b: callvirt System.String::ToLower
00bc26d8 8b4dec          mov     ecx,dword ptr [ebp-14h]
00bc26db 3909            cmp     dword ptr [ecx],ecx
00bc26dd ff1508a82900    call    dword ptr ds:[29A808h]
00bc26e3 8945e8          mov     dword ptr [ebp-18h],eax
IL_0020: stloc.0  (test)
00bc26e6 8b45e8          mov     eax,dword ptr [ebp-18h]
00bc26e9 8945ec          mov     dword ptr [ebp-14h],eax

The above output has c#,IL and assembly.

Debugging Generic System.Nullable within Windbg


In this post I am going to unravel the mystery of debugging the Nullable<T> within Windbg in .NET 3.5 and also compare it with .NET 4.0. Here is the sample code and it is compiled in .NET 3.5

using System;
namespace ConsoleApplication
{
 class Program
 {
 Int32? test;
 int i = 10;
 static void Main(string[] args)
 {
 Nullable<T>
 Int32? i = 10;
 Object o = 10;
 var p = new Program() { test = 20 };
 Console.Read();
 p.test = (Int32?) o ;
 Console.WriteLine(p.test.HasValue);
 }
 }
}

Attached to the debugger on the Console.Read. FYI I always load sos and sosex extensions to debug managed code. Here is !mdt 0x0253c11c output

0:000> !mdt 0x0253c11c
0253c11c (ConsoleApplication.Program)
test:ERROR (0×80070057).
i:0xa (System.Int32)

Notice that “test” does not have a value and has an error. Next issued !dumpobj

!do 0x0253c11c

0:000> !do 0x0253c11c
Name: ConsoleApplication.Program
MethodTable: 002932f0
EEClass: 00291360
Size: 20(0×14) bytes
(C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication9\bin\Debug\ConsoleApplication.exe)
Fields:
MT    Field   Offset                 Type VT     Attr    Value Name
00000000 4000001        8                       1 instance 0280c124 test
7776ab0c  4000002        4         System.Int32  1 instance       10 i

My fault ,I thought sos should be able to get the MethodTable of Nullable<Int32> for “test” when sosex couldn’t.  To my surprise the MT output was 00000000 . To view the contents of the “test” I would have to use  the !dumpvc which requires methodtable. I know I could use the dd command. And here is the output from the dd 0280c124

0:000> dd 0280c124
0280c124  00000001 00000014 00000000 77767c70
0280c134  00000000 00000000 00000000 00000000
0280c144  00000000 00000000 777684dc 00000000
0280c154  40010000 7776d7ec 00000003 00000008
0280c164  00000100 00000000 77767cc4 00000000
0280c174  00000000 00000000 00000000 00000001
0280c184  0280c158 00000001 00000000 7776841c
0280c194  00000000 00000000 00000000 00000000

The second field 00000014 is the actual value of test and here is the actual output

0:000> ? poi(0280c124+0×4)
Evaluate expression: 20 = 00000014

But this does not solve the real issue of figuring out the methodtable to use it in !dumpvc. I could have used !mx System.Nullable* to get the MethodTable,  because I knew the type is Nullable<Int> ,what if I didn’t know the type information.

To get the mt information I had to disassemble the code. First step is to get the !clrstack

0:000> !CLRStack
OS Thread Id: 0x94c (0)
ESP       EIP
0021f1dc 769d73ea [NDirectMethodFrameStandaloneCleanup: 0021f1dc] System.IO.__ConsoleStream.ReadFile(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte*, Int32, Int32 ByRef, IntPtr)
0021f1f8 77c8ae67 System.IO.__ConsoleStream.ReadFileNative(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte[], Int32, Int32, Int32, Int32 ByRef)
0021f224 77c8ad86 System.IO.__ConsoleStream.Read(Byte[], Int32, Int32)
0021f244 776f9fbb System.IO.StreamReader.ReadBuffer()
0021f258 77c677fc System.IO.StreamReader.Read()
0021f264 77c8dd81 System.IO.TextReader+SyncTextReader.Read()
0021f270 77bd328b System.Console.Read()
0021f278 00320115 ConsoleApplication.Program.Main(System.String[])
0021f4d0 59781b6c [GCFrame: 0021f4d0]

The next is to !u 00320115 and here is the partial ouput

00320116 8b45d8          mov     eax,dword ptr [ebp-28h]
00320119 3a4008          cmp     al,byte ptr [eax+8]
0032011c 8d4008          lea     eax,[eax+8]
0032011f 8945c8          mov     dword ptr [ebp-38h],eax
00320122 ff75dc          push    dword ptr [ebp-24h]
00320125 8b4dc8          mov     ecx,dword ptr [ebp-38h]
00320128 bae8397777      mov     edx,offset mscorlib_ni+0x2739e8 (777739e8) (MT: System.Nullable`1[[System.Int32, mscorlib]])
0032012d e8d6a74c59      call    mscorwks!JIT_Unbox_Nullable (597ea908)

Notice the Method table 777739e8 for System.Nullable`1[[System.Int32, mscorlib]] and here is the output from !dumpmt -md 777739e8

0:000> !dumpmt -md 777739e8
EEClass: 7752e7c8
Module: 77501000
Name: System.Nullable`1[[System.Int32, mscorlib]]
mdToken: 0200026d  (C:\Windows\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)
BaseSize: 0×10
ComponentSize: 0×0
Number of IFaces in IFaceMap: 0
Slots in VTable: 14
————————————–
MethodDesc Table
Entry MethodDesc      JIT Name
77697028   775eace0     NONE System.Nullable`1[[System.Int32, mscorlib]].ToString()
77697020   775eacc0     NONE System.Nullable`1[[System.Int32, mscorlib]].Equals(System.Object)
77697018   775eacd0     NONE System.Nullable`1[[System.Int32, mscorlib]].GetHashCode()
777374c0   775412a4   PreJIT System.Object.Finalize()
77d010a0   775eac98   PreJIT System.Nullable`1[[System.Int32, mscorlib]]..ctor(Int32)
77d01100   775eaca0   PreJIT System.Nullable`1[[System.Int32, mscorlib]].get_HasValue()
77d01120   775eaca8   PreJIT System.Nullable`1[[System.Int32, mscorlib]].get_Value()
77d01030   775eacb0   PreJIT System.Nullable`1[[System.Int32, mscorlib]].GetValueOrDefault()
77d01140   775eacb8   PreJIT System.Nullable`1[[System.Int32, mscorlib]].GetValueOrDefault(Int32)
77d0100c   775eacf0   PreJIT System.Nullable`1[[System.Int32, mscorlib]].op_Implicit(Int32)
77d00fe8   775eacf8   PreJIT System.Nullable`1[[System.Int32, mscorlib]].op_Explicit(System.Nullable`1<Int32>)
77d01040   775eacc8   PreJIT System.Nullable`1[[System.Int32, mscorlib]].Equals(System.Object)
77d01078   775eacd8   PreJIT System.Nullable`1[[System.Int32, mscorlib]].GetHashCode()
77d010c0   775eace8   PreJIT System.Nullable`1[[System.Int32, mscorlib]].ToString()

Now that I have confirmed the mt and here is the output from !dumpvc 777739e8 0280c124

0:000> !dumpvc 777739e8 0280c124
Name: System.Nullable`1[[System.Int32, mscorlib]]
MethodTable 777739e8
EEClass: 7752e7c8
Size: 16(0×10) bytes
(C:\Windows\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)
Fields:
MT    Field   Offset                 Type VT     Attr    Value Name
7776eadc  40009a8        0       System.Boolean  1 instance        1 hasValue
7776ab0c  40009a9        4         System.Int32  1 instance       20 value

I decided to test this in .NET 4.0 and here is the output of !mdt for the Program object

0:000> !mdt 0x0236bc44
0236bc44 (ConsoleApplication.Program)
test:(System.Nullable`1[[System.Int32, mscorlib]]) VALTYPE (MT=6229f60c, ADDR=0236bc50)
i:0xa (System.Int32)

Notice the “test” which is Nullable<Int32> is now recognized by sosex and it also provides the method table.

Debugging .Net Framework 4.0 without source code using windbg


In this post I am going to be discussing about debugging .Net Framework 4.0  using windbg . I am going to demonstrating how to have a break-point within a method, but without the framework source code. This would help in debugging .NET framework when you don’t have VS in a production environment and the same technique can be used to debug other third party assemblies where you don’t have the source code.  This is kind of like .NET Reflector where you can step through third party assemblies, but without any cost. It is not going to be as convenient as the professional version of Reflector.

I am going to be using the same example that I used to debug .NET Framework source 3.5 using windbg.

FYI the .NET framework 4.0 has private symbols available on MS symbol server, but the source code is still not available. To debug .NET framework source code it is important to have correct symbol path and here is my symbol path in the _NT_SYMBOL_PATH environment variable.


SRV*d:\dev\symbols*http://referencesource.microsoft.com/symbols; SRV*d:\dev\symbols*http://msdl.microsoft.com/download/symbols

Here is the sample source code that I am going to be using to demonstrate framework debugging


using System;
 using System.Net;
 namespace Test
 {
 class Program
 {
 static void Main(string[] args)
 {
 Console.WriteLine("Hello World of debugging");
 var wr = WebRequest.Create("http://www.google.com");
 Console.WriteLine("Web request created");
 var req = wr.GetRequestStream();
 Console.WriteLine("Hello World Debugging");
 Console.Read();
 }
 }
 }
 

Launched the exe within the debugger

Then issued the command to notify when the clrjit is loaded,


sxe ld:clrjit

This is because, to load sos and sosex after the framework is loaded.Then issued the following commands to load sos, sosex and to set break-point on WebRequest.Create


.loadby sos mscorwks
.load sosex
!mbm System.Net.WebRequest.Create

And when the break-point hits the first time, let it continue by using the g command. It would break into the debugger for other overloaded method for WebRequest.Create and here is the call-stack

0:000> !mk
Thread 0:
ESP              EIP
00:M 000000000023ece0 000007fef6ea5a44 System.Net.WebRequest.Create(System.Uri, Boolean)(+0×0 IL)(+0×14 Native) [f:\dd\ndp\fx\src\Net\System\Net\WebRequest.cs, @ 93,13]
01:M 000000000023ed60 000007ff00140176 Test.Program.Main(System.String[])(+0xc IL)(+0×56 Native)
02:U 000000000023edc0 000007fef8b210b4 clr!CallDescrWorker+0×84
03:U 000000000023ee10 000007fef8b211c9 clr!CallDescrWorkerWithHandler+0xa9
04:U 000000000023ee90 000007fef8b21245 clr!MethodDesc::CallDescr+0x2a1
05:U 000000000023f0c0 000007fef8c21675 clr!ClassLoader::RunMain+0×228
06:U 000000000023f310 000007fef8c217ac clr!Assembly::ExecuteMainMethod+0xac
07:U 000000000023f5c0 000007fef8c21562 clr!SystemDomain::ExecuteMainMethod+0×452
08:U 000000000023fb70 000007fef8c23dd6 clr!ExecuteEXE+0×43
09:U 000000000023fbd0 000007fef8c23cf3 clr!CorExeMainInternal+0xc4
0a:U 000000000023fc40 000007fef8ca7365 clr!CorExeMain+0×15
0b:U 000000000023fc80 000007fef9493309 mscoreei!CorExeMain+0×41
0c:U 000000000023fcb0 000007fef9525b21 MSCOREE!CorExeMain_Exported+0×57
0d:U 000000000023fce0 00000000776cf56d KERNEL32!BaseThreadInitThunk+0xd
0e:U 000000000023fd10 0000000077903281 ntdll!RtlUserThreadStart+0x1d

And here is the source code for this method using reflector


private static WebRequest Create(Uri requestUri, bool useUriBase)
{
 string absoluteUri;
 if (Logging.On)
 {
 Logging.Enter(Logging.Web, "WebRequest", "Create", requestUri.ToString());
 }
 WebRequestPrefixElement element = null;
 bool flag = false;
 if (!useUriBase)
 {
 absoluteUri = requestUri.AbsoluteUri;
 }
 else
 {
 absoluteUri = requestUri.Scheme + ':';
 }
 int length = absoluteUri.Length;
 ArrayList prefixList = PrefixList;
 for (int i = 0; i < prefixList.Count; i++)
 {
 element = (WebRequestPrefixElement) prefixList[i];
 if ((length >= element.Prefix.Length) && (string.Compare(element.Prefix, 0, absoluteUri, 0, element.Prefix.Length, StringComparison.OrdinalIgnoreCase) == 0))
 {
 flag = true;
 break;
 }
 }
 WebRequest retObject = null;
 if (flag)
 {
 retObject = element.Creator.Create(requestUri);
 if (Logging.On)
 {
 Logging.Exit(Logging.Web, "WebRequest", "Create", retObject);
 }
 return retObject;
 }
 if (Logging.On)
 {
 Logging.Exit(Logging.Web, "WebRequest", "Create", (string) null);
 }
 throw new NotSupportedException(SR.GetString("net_unknown_prefix"));
}

Let’s try and have a break-point on line   “ ArrayList prefixList = PrefixList;” so that we can check the local variables value.  Just because I have the private symbols ,I could have counted the line numbers manually and then set a break-point using !mbp command, but that is no fun. Here is another way of doing this.


.shell -ci "!u 000007fef6ea5a44 "  findstr get_PrefixList

In the above command I am disassembling the ip 000007fef6ea5a44 (which is there in the above callstack )to look for  get_PrefixList Instruction pointer . Here is the outcome

0:000> .shell -ci “!u 000007fef6ea5a44 ”  findstr get_PrefixList

000007fe`f6ea5a7a e8f1000000      call    System_ni+0x275b70 (000007fe`f6ea5b70) (System.Net.WebRequest.get_PrefixList(), mdToken: 00000000060019bc)

.shell: Process exited

I use the .shell command to manually avoid searching for an instruction. Now that I have the instruction pointer ,I am going to set a break-point on that using


bp 000007fe`f6ea5a7a "!mdv"

and here is the result of the break-point

0:000> g
(1758.167c): CLR notification exception – code e0444143 (first chance)
(1758.167c): CLR notification exception – code e0444143 (first chance)
Frame 0×0: (System.Net.WebRequest.Create(System.Uri, Boolean)):
[A0]:requestUri:<?>
[A1]:useUriBase:<?>
[L0]:LookupUri:0x00000000022c1a98 (System.String) STRVAL=http://www.google.com/
[L1]:Current:<?>
[L2]:Found:0×0000000000000000 (System.Boolean)
[L3]:LookupLength:0×0000000000000016 (System.Int32)
[L4]:prefixList:<?>
[L5]:i:<?>
[L6]:webRequest:<?>

System_ni+0x275a7a:
000007fe`f6ea5a7a e8f1000000      call    System_ni+0x275b70 (000007fe`f6ea5b70)

Voila! now I am able to have a break-point within the framework method  and also see locals and parameters like Visual Studio. The same technique can be used to debug third party assemblies where you don’t have source code or symbols.

Exploring SOSEX and Windbg to debug .NET 4.0


With the latest release of sosex comes a new set of functions to debug. It is pretty awesome that one person (Steve) alone could pull of such cool things. In this blog post, I am just going to demonstrate how easy it is to debug managed code using sosex compared to sos.

using System;
using System.Collections.Generic;

namespace MemCheck
{
 internal class Test
 {
 Dictionary<int, string> dict = new Dictionary<int, string>();
 private static void Main(string[] args)
 {
 var p = new Test();
 for (int i = 0; i < 100; i++)
 {
 p.dict.Add(i, i.ToString());
 }
 Console.WriteLine("Done");
 Console.Read();
 }
 }
}

I like to keep the code simple , so it is easy to follow.  The debugging goal for today is to get the Dictionary values. First I am going to demonstrate it using sos and then using sosex.

As usual I start the app and then attach it to windbg.

.loadby sos clr

FYI in  .net 4.0 clr is the dll that has CLR implementation. In prior versions it used to be in mscorwks. The next command would look for the object Test in the memory

!dumpheap -type MemCheck.Test
0:000> !dumpheap -type MemCheck.Test
 Address               MT     Size
0000000002761e20 000007ff00054110       24
 total 0 objects
 Statistics:
 MT    Count    TotalSize Class Name
 000007ff00054110        1           24 MemCheck.Test
 Total 1 objects

The next step is to dump the object

0:000> !do 0000000002761e20
Name:        MemCheck.Test
MethodTable: 000007ff00054110
EEClass:     000007ff00162350
Size:        24(0x18) bytes
File:        C:\Users\naveen\Documents\Visual Studio 2010\Projects\Test\bin\Debug\Test.exe
Fields:
 MT    Field   Offset                 Type VT     Attr            Value Name
000007feec2b7a48  4000001        8 ...tring, mscorlib]]  0 instance 0000000002761e38 dict

Notice the dict object is in the 8th offset . To dump contents  dict object I would use the command !do poi(0000000002761e20+8) , which is pointer deference of Test object on it is 8th offset.  And here is the output

0:000> !do poi(0000000002761e20+8)
Name:        System.Collections.Generic.Dictionary`2[[System.Int32, mscorlib],[System.String, mscorlib]]
MethodTable: 000007feec2b7a48
EEClass:     000007feebe113c0
Size:        88(0x58) bytes
File:        C:\Windows\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll
Fields:
 MT    Field   Offset                 Type VT     Attr            Value Name
000007feec27c7d8  4000bee        8       System.Int32[]  0 instance 0000000002764788 buckets
000007feecbd3dc8  4000bef       10 ...non, mscorlib]][]  0 instance 0000000002764ab8 entries
000007feec27c848  4000bf0       40         System.Int32  1 instance              100 count
000007feec27c848  4000bf1       44         System.Int32  1 instance              100 version
000007feec27c848  4000bf2       48         System.Int32  1 instance               -1 freeList
000007feec27c848  4000bf3       4c         System.Int32  1 instance                0 freeCount
000007feec2a5a48  4000bf4       18 ...Int32, mscorlib]]  0 instance 0000000002761ef0 comparer
000007feecc75f78  4000bf5       20 ...Canon, mscorlib]]  0 instance 0000000000000000 keys
000007feecc72078  4000bf6       28 ...Canon, mscorlib]]  0 instance 0000000000000000 values
000007feec275ab8  4000bf7       30        System.Object  0 instance 0000000000000000 _syncRoot
000007feec29a1b8  4000bf8       38 ...SerializationInfo  0 instance 0000000000000000 m_siInfo

And the dictionary object in turn stores them within an array which is again the 8th offset.  This time because we know it is an array we are going to use the !dumparray command on the memory location. The command to get the details is

!dumparray -details poi(poi(0000000002761e20+8)+8)
MT    Field   Offset                 Type VT     Attr            Value Name
 000007feec27c848  400047b        0             System.Int32      1     instance                   -1     m_value
 [195] 0000000002764aa4
 Name:        System.Int32
 MethodTable: 000007feec27c848
 EEClass:     000007feebe00890
 Size:        24(0x18) bytes
 File:        C:\Windows\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll
 Fields:
 MT    Field   Offset                 Type VT     Attr            Value Name
 000007feec27c848  400047b        0             System.Int32      1     instance                   -1     m_value
 [196] 0000000002764aa8
 Name:        System.Int32
 MethodTable: 000007feec27c848
 EEClass:     000007feebe00890
 Size:        24(0x18) bytes
 File:        C:\Windows\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll
 Fields:
 MT    Field   Offset                 Type VT     Attr            Value Name
 000007feec27c848  400047b        0             System.Int32      1     instance                   -1     m_value

Here is the partial output.

Now lets try and do the same thing using sosex. The one thing that I really like about  the new sosex is that  I can use names rather than pointer deference which is way much easier. Launched the app and then loaded sosex using the command

.load F:\Work\Tools\debuggers\sosex.dll

and then switched the thread from 4th to 0th thread using ~0s.  By default the debugger injects a thread into the process for debugging and that was the 4th thread. The next command I issued was to get stack trace

!mk
0:000> !mk
Thread 0:
 ESP              EIP
00:U 000000000015e408 0000000077bc00da ntdll!ZwRequestWaitReplyPort+0xa
01:U 000000000015e410 0000000077a72b08 KERNEL32!ConsoleClientCallServer+0x54
02:U 000000000015e440 0000000077aa5601 KERNEL32!ReadConsoleInternal+0x1f1
03:U 000000000015e590 0000000077aba922 KERNEL32!ReadConsoleA+0xb2
04:U 000000000015e670 0000000077a89934 KERNEL32!zzz_AsmCodeRange_End+0x8bea
05:U 000000000015e6b0 000007feed0317c7 clr!DoNDirectCall__PatchGetThreadCall+0x7b
06:M 000000000015e760 000007feec1d34a1 DomainNeutralILStubClass.IL_STUB_PInvoke(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte*, Int32, Int32 ByRef, IntPtr)(+0x0 IL)(+0x0 Native)
07:M 000000000015e880 000007feec97f59a System.IO.__ConsoleStream.ReadFileNative(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte[], Int32, Int32, Int32, Int32 ByRef)(+0x53 IL)(+0xba Native)
08:M 000000000015e8f0 000007feec97f402 System.IO.__ConsoleStream.Read(Byte[], Int32, Int32)(+0x5d IL)(+0x62 Native)
09:M 000000000015e950 000007feec18e63c System.IO.StreamReader.ReadBuffer()(+0xa0 IL)(+0x5c Native)
0a:M 000000000015e9a0 000007feec915630 System.IO.StreamReader.Read()(+0x21 IL)(+0x30 Native)
0b:M 000000000015e9e0 000007feec987458 System.IO.TextReader+SyncTextReader.Read()(+0x0 IL)(+0x38 Native)
0c:M 000000000015ea30 000007ff00170213 MemCheck.Test.Main(System.String[])(+0x39 IL)(+0xf3 Native) [C:\Users\naveen\Documents\Visual Studio 2010\Projects\Test\Program.cs, @ 17,13]
0d:U 000000000015eaa0 000007feed0710b4 clr!CallDescrWorker+0x84
0e:U 000000000015eaf0 000007feed0711c9 clr!CallDescrWorkerWithHandler+0xa9
0f:U 000000000015eb70 000007feed071245 clr!MethodDesc::CallDescr+0x2a1
10:U 000000000015eda0 000007feed171675 clr!ClassLoader::RunMain+0x228
11:U 000000000015eff0 000007feed1717ac clr!Assembly::ExecuteMainMethod+0xac
12:U 000000000015f2a0 000007feed171562 clr!SystemDomain::ExecuteMainMethod+0x452
13:U 000000000015f850 000007feed173dd6 clr!ExecuteEXE+0x43
14:U 000000000015f8b0 000007feed173cf3 clr!CorExeMainInternal+0xc4
15:U 000000000015f920 000007feed1f7365 clr!CorExeMain+0x15
16:U 000000000015f960 000007fef8f13309 mscoreei!CorExeMain+0x41
17:U 000000000015f990 000007fef8fa5b21 MSCOREE!CorExeMain_Exported+0x57
18:U 000000000015f9c0 0000000077a6f56d KERNEL32!BaseThreadInitThunk+0xd
19:U 000000000015f9f0 0000000077ba3281 ntdll!RtlUserThreadStart+0x1d

FYI the command !mk has been part of sos from the initial version. I am interested in only looking at the code that I wrote so I would like to move stack frame to 0c which is MemCheck.Test.Main . To do that the command  is !mframe 0c, which moves to that stackframe. The reason to move the particular stack frame is to look for variables in the stack and the command to variables is !mdv , which display managed local variables

0:000> !mdv
Frame 0xc: (MemCheck.Test.Main(System.String[])):
[A0]:args:0x0000000002761dd8 (System.String[])
[L0]:p:0x0000000002761e20 (MemCheck.Test)
[L1]:i:0x0000000000000064 (System.Int32)
[L2]:CS$4$0000:0x0000000000000000 (System.Boolean)

Notice we see the local variable “p” which is of type MemCheck.Test. To display type p we issue the command !mdt p


0:000> !mdt p

0000000002761e20 (MemCheck.Test)

dict:0000000002761e38 (System.Collections.Generic.Dictionary`2[[System.Int32, mscorlib],[System.String, mscorlib]])

I didn’t  have to get memory address , I am using the names which is very intuitive ,especially when we have to debug large application with N levels of nesting.So to get the dict values   from p the command to issue is !mdt -e p.dict

!mdt -e p.dict
[98] (System.Collections.Generic.Dictionary`2+Entry[[System.Int32, mscorlib],[System.String, mscorlib]]) VALTYPE (MT=000007feec2b7b28, ADDR=0000000002765400)
 key:0x62 (System.Int32)
 value:0000000002765e48 (System.String: "98")
[99] (System.Collections.Generic.Dictionary`2+Entry[[System.Int32, mscorlib],[System.String, mscorlib]]) VALTYPE (MT=000007feec2b7b28, ADDR=0000000002765418)
 key:0x63 (System.Int32)
 value:0000000002765e68 (System.String: "99")

Here is the partial output. Notice I never had to use a memory pointer or do a pointer deference .This is very similar to VS.NET debugging where I am used to the variable names compared the memory address.  Thanks to Steve for providing such a cool extension.

Debugging .Net framework source code within Windbg


One the coolest thing Microsoft did was to release the .NET Framework source code. In this post, I am going to demonstrate, how we could have a break-point on the .NET framework source code by line numbers ,using Windbg ,very similar to doing in  VS.NET.

The first step towards doing this is to download .NET Framework Source Code and installing it on the local machine. The next step is to set the symbol path environment variable. My _NT_SYMBOL_PATH is set to


SRV*d:\dev\symbols*http://referencesource.microsoft.com/symbols; SRV*d:\dev\symbols*http://msdl.microsoft.com/download/symbols

Setting the correct symbol path is important to download symbols from MS.

Here is the source code that I would be using to demonstrate this


using System;
using System.Net;
namespace Test {
 class Program {
 static void Main(string[] args) {
 Console.WriteLine("Hello World of debugging");
 var wr = WebRequest.Create("http://www.google.com");
 Console.WriteLine("Web request created");
 var req = wr.GetRequestStream();
 Console.WriteLine("Hello World Debugging");
 Console.Read();
 }
 }
}

I am going to demonstrate the same thing using multiple versions of debugger. The first one that I am going to demonstrate is using  Windows Debugger Version 6.12.0002.633 X86 which is the latest version.

Launched the exe within windbg and opened the source code WebRequest.cs and Program.cs withing Windbg.

The WebRequest.cs is the source code that was downloaded from MS Reference Source code and Program.cs is the above sample code.

Then issued the command, to be notified when mscorlib is loaded

sxe ld:mscorlib

And here is the output from the above command

ModLoad: 53fd0000 54ac8000   C:\Windows\assembly\NativeImages_v2.0.50727_32\mscorlib\8c1770d45c63cf5c462eeb945ef9aa5d\mscorlib.ni.dll
eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=7efdd000 edi=0040eaf4
eip=7723fc02 esp=0040e9c8 ebp=0040ea1c iopl=0         nv up ei pl nz na po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202
ntdll!ZwMapViewOfSection+0×12:
7723fc02 83c404          add     esp,4

then issued the following commands to load by sos, sosex and set break-points

.loadby sos mscorwks
.load sosex
!mbm System.Net.WebRequest.Create
!mbp WebRequest.cs 98

So with the above command I am requesting for a break-point on the method  System.Net.WebRequest.Create using symbol (!mbm). I am issuing the command !mbm ,just so that sosex can hook up with CLR for getting notifications on JIT. Without this I was unable to set break-point on source code using line numbers. The next command !mbp WebRequest.cs 98 means, have a break-point on the line number 98  in the WebRequest.cs file . The line 98 contents are “if (!useUriBase)”  .  I forgot to mention the .NET framework 4.0 source code is not released so I am using 3.5

Also make sure that the correct private pdb symbols are loaded for framework assemblies, and to verify that, issue the command “lme” and the output should contain something like this

0:000> lme
start    end        module name
013b0000 013b8000   ConsoleApplication1 C (private pdb symbols)  C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication1\bin\Debug\ConsoleApplication1.pdb
53fd0000 54ac8000   mscorlib_ni C (private pdb symbols)  d:\dev\symbols\mscorlib.pdb\F85F3DD0C7024D528B4C37F1ACF2123D1\mscorlib.pdb
58930000 590c9000   System_ni C (private pdb symbols)  d:\dev\symbols\System.pdb\97A082CB5BC64B30887253632D3901EE1\System.pdb

And here is the output after letting it run

Breakpoint 1 hit
eax=00000001 ebx=0040f37c ecx=0282c160 edx=00000000 esi=008ba398 edi=0282c160
eip=58dec8b7 esp=0040f32c ebp=0040f334 iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
System_ni+0x4bc8b7:
58dec8b7 85ff

then issued the commands to see the list of break-points (managed and native)


!mbl

bl

and here is the output

0:000> !mbl
0 e : *!SYSTEM.NET.WEBREQUEST.CREATE ILOffset=0: pass=1 oneshot=false thread=ANY
System!System.Net.WebRequest.Create(Uri, bool)+0xfffffffe(IL)
0 e 58a71630
System!System.Net.WebRequest.Create(string)
1 e 58dec8b7
System!System.Net.WebRequest.Create(Uri)+0xfffffffe(IL)
2 e 58a71600
1 eu: WebRequest.cs, line 98: pass=1 oneshot=false thread=ANY
0:000> bl
0 e 58a71630     0001 (0001)  0:**** System_ni+0×141630
1 e 58dec8b7     0001 (0001)  0:**** System_ni+0x4bc8b7
2 e 58a71600     0001 (0001)  0:**** System_ni+0×141600

because Webrequest.Create has two overloads, sosex has set a break-point on both these methods and that’s the reason we are seeing 3 break-points, instead of 2. And after issuing the “g” command for couple of hits for the Webrequest.Create , then comes the Breakpoint 3 hit. Voila ,we have the managed to set break-point on framework source code using line numbers and here is the output of  !mk command

Breakpoint 3 hit
eax=00000000 ebx=0014ecdc ecx=0282dde4 edx=00000000 esi=00000000 edi=00000000
eip=58a71650 esp=0014ec68 ebp=0014ec84 iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
System_ni+0×141650:
58a71650 8bc7            mov     eax,edi
0:000> !mk
ESP      RetAddr
00:M 0014ec68 58a71650 System.Net.WebRequest.Create(System.Uri, Boolean)(+0×23 IL)(+0×0 Native) [f:\dd\ndp\fx\src\Net\System\Net\WebRequest.cs, @ 96,13]
01:M 0014ec8c 58dec8dd System.Net.WebRequest.Create(System.String)(+0×14 IL)(+0×26 Native)
02:M 0014ec9c 005000a9 Test.Program.Main(System.String[])(+0xc IL)(+0×18 Native) [C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication1\Program.cs, @ 7,13]
03:U 0014ecb8 6cfb1b6c mscorwks!CallDescrWorker+0×33
04:U 0014ecc8 6cfc2209 mscorwks!CallDescrWorkerWithHandler+0xa3
05:U 0014ed48 6cfd6511 mscorwks!MethodDesc::CallDescr+0x19c
06:U 0014ee8c 6cfd6544 mscorwks!MethodDesc::CallTargetWorker+0x1f
07:U 0014eea8 6cfd6562 mscorwks!MethodDescCallSite::CallWithValueTypes_RetArgSlot+0x1a
08:U 0014eec0 6d040c45 mscorwks!ClassLoader::RunMain+0×223
09:U 0014f024 6d040b65 mscorwks!Assembly::ExecuteMainMethod+0xa6
0a:U 0014f28c 6d0410b5 mscorwks!SystemDomain::ExecuteMainMethod+0×456
0b:U 0014f75c 6d04129f mscorwks!ExecuteEXE+0×59
0c:U 0014f7ac 6d0411cf mscorwks!_CorExeMain+0x15c
0d:U 0014f7f4 73a461f0 mscoreei!_CorExeMain+0×38
0e:U 0014f800 74337f16 MSCOREE!ShellShim__CorExeMain+0×99
0f:U 0014f810 74334de3 MSCOREE!_CorExeMain_Exported+0×8
10:U 0014f818 74f73677 KERNEL32!BaseThreadInitThunk+0xe
11:U 0014f824 77259d72 ntdll!__RtlUserThreadStart+0×70
12:U 0014f864 77259d45 ntdll!_RtlUserThreadStart+0x1b

Notice on the top frame of the stack (00:M) we see source information. The advantage of this is, for example I can check the values of the local variable with in the function after certain calls ,which wouldn’t  have been possible without jumping hoops. Here is the the output !mdv after the line breakpoint

0:000> !mdv
Frame 0×0: (System.Net.WebRequest.Create(System.Uri, Boolean)):
[A0]:requestUri:0x282d2fc (System.Uri)
[A1]:useUriBase:0×0 (System.Boolean)
[L0]:LookupUri:<?>
[L1]:Current:null (System.Net.WebRequestPrefixElement)
[L2]:Found:0×0 (System.Boolean)
[L3]:LookupLength:<?>
[L4]:prefixList:<?>
[L5]:i:<?>
[L6]:webRequest:<?>

And my next quest was to figure What if I could use the actual break-point on the source code directly instead of using sosex.  So chose program.cs and hit the F9 key on the second Console.WriteLine and the color changed to red

And here is the output o f the bl command

0:000> bl
0 e 58a71630     0001 (0001)  0:**** System_ni+0×141630
1 e 58dec8b7     0001 (0001)  0:**** System_ni+0x4bc8b7
2 e 58a71600     0001 (0001)  0:**** System_ni+0×141600
3 e 58a71650     0001 (0001)  0:**** System_ni+0×141650
4 e 013b0017     0001 (0001)  0:**** ConsoleApplication1!Main+0×17

Notice there is a new break-point 4.  But bad luck, when i let it run it didn’t work and I had to clear the break-point 4 . It failed to create a break-point and here is the output

0:000> g
Unable to insert breakpoint 4 at 013b0017, Win32 error 0n998
“Invalid access to memory location.”
bp4 at 013b0017 failed
WaitForEvent failed
eax=00000000 ebx=0014ecdc ecx=0282dde4 edx=00000000 esi=00000000 edi=00000000
eip=58a71650 esp=0014ec68 ebp=0014ec84 iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000346
System_ni+0×141650:
58a71650 8bc7            mov     eax,edi

The next step was to debug the with the version of debugger which had capabilities to see managed call stacks, Windbg version 6.7.5.0. Did the same thing loaded up the exe and the Program.cs in to the debugger and set a break-point on mscorlib load using  ” sxe ld:mscorlib” and then issued the command “.loadby sos mscorwks” ,”.load sosex” and “!mbm System.Net.WebreRequest.Create”. And the let it run, and to my surprise here is the integrated debugging experience, the source code is highlighted, when break-point is hit for WebRequest.Create

And after hitting F10 ,it moved to the next line

Wow this is way cool. The next step was to set a break-point using the bp command on the Program.cs


bp (@@masm(`Program.cs:8+`))

And  the break-point was hit and I was successfully able to have Integrated debugging environment. I know most of us don’t  have Windbg version 6.7.5.0 , but there are few debugging geeks who still have them. I was happy that I kept the version.

Take Away

We should be able to debug within the framework source code using the latest version of debugger and if you have 6.7.5.0 , can have integrated debugging experience similar to VS.NET .

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: