Debugging .Net Framework 4.0 without source code using windbg

In this post I am going to be discussing about debugging .Net Framework 4.0  using windbg . I am going to demonstrating how to have a break-point within a method, but without the framework source code. This would help in debugging .NET framework when you don’t have VS in a production environment and the same technique can be used to debug other third party assemblies where you don’t have the source code.  This is kind of like .NET Reflector where you can step through third party assemblies, but without any cost. It is not going to be as convenient as the professional version of Reflector.

I am going to be using the same example that I used to debug .NET Framework source 3.5 using windbg.

FYI the .NET framework 4.0 has private symbols available on MS symbol server, but the source code is still not available. To debug .NET framework source code it is important to have correct symbol path and here is my symbol path in the _NT_SYMBOL_PATH environment variable.

SRV*d:\dev\symbols*; SRV*d:\dev\symbols*

Here is the sample source code that I am going to be using to demonstrate framework debugging

using System;
 using System.Net;
 namespace Test
 class Program
 static void Main(string[] args)
 Console.WriteLine("Hello World of debugging");
 var wr = WebRequest.Create("");
 Console.WriteLine("Web request created");
 var req = wr.GetRequestStream();
 Console.WriteLine("Hello World Debugging");

Launched the exe within the debugger

Then issued the command to notify when the clrjit is loaded,

sxe ld:clrjit

This is because, to load sos and sosex after the framework is loaded.Then issued the following commands to load sos, sosex and to set break-point on WebRequest.Create

.loadby sos mscorwks
.load sosex
!mbm System.Net.WebRequest.Create

And when the break-point hits the first time, let it continue by using the g command. It would break into the debugger for other overloaded method for WebRequest.Create and here is the call-stack

0:000> !mk
Thread 0:
ESP              EIP
00:M 000000000023ece0 000007fef6ea5a44 System.Net.WebRequest.Create(System.Uri, Boolean)(+0x0 IL)(+0x14 Native) [f:\dd\ndp\fx\src\Net\System\Net\WebRequest.cs, @ 93,13]
01:M 000000000023ed60 000007ff00140176 Test.Program.Main(System.String[])(+0xc IL)(+0x56 Native)
02:U 000000000023edc0 000007fef8b210b4 clr!CallDescrWorker+0x84
03:U 000000000023ee10 000007fef8b211c9 clr!CallDescrWorkerWithHandler+0xa9
04:U 000000000023ee90 000007fef8b21245 clr!MethodDesc::CallDescr+0x2a1
05:U 000000000023f0c0 000007fef8c21675 clr!ClassLoader::RunMain+0x228
06:U 000000000023f310 000007fef8c217ac clr!Assembly::ExecuteMainMethod+0xac
07:U 000000000023f5c0 000007fef8c21562 clr!SystemDomain::ExecuteMainMethod+0x452
08:U 000000000023fb70 000007fef8c23dd6 clr!ExecuteEXE+0x43
09:U 000000000023fbd0 000007fef8c23cf3 clr!CorExeMainInternal+0xc4
0a:U 000000000023fc40 000007fef8ca7365 clr!CorExeMain+0x15
0b:U 000000000023fc80 000007fef9493309 mscoreei!CorExeMain+0x41
0c:U 000000000023fcb0 000007fef9525b21 MSCOREE!CorExeMain_Exported+0x57
0d:U 000000000023fce0 00000000776cf56d KERNEL32!BaseThreadInitThunk+0xd
0e:U 000000000023fd10 0000000077903281 ntdll!RtlUserThreadStart+0x1d

And here is the source code for this method using reflector

private static WebRequest Create(Uri requestUri, bool useUriBase)
 string absoluteUri;
 if (Logging.On)
 Logging.Enter(Logging.Web, "WebRequest", "Create", requestUri.ToString());
 WebRequestPrefixElement element = null;
 bool flag = false;
 if (!useUriBase)
 absoluteUri = requestUri.AbsoluteUri;
 absoluteUri = requestUri.Scheme + ':';
 int length = absoluteUri.Length;
 ArrayList prefixList = PrefixList;
 for (int i = 0; i < prefixList.Count; i++)
 element = (WebRequestPrefixElement) prefixList[i];
 if ((length >= element.Prefix.Length) && (string.Compare(element.Prefix, 0, absoluteUri, 0, element.Prefix.Length, StringComparison.OrdinalIgnoreCase) == 0))
 flag = true;
 WebRequest retObject = null;
 if (flag)
 retObject = element.Creator.Create(requestUri);
 if (Logging.On)
 Logging.Exit(Logging.Web, "WebRequest", "Create", retObject);
 return retObject;
 if (Logging.On)
 Logging.Exit(Logging.Web, "WebRequest", "Create", (string) null);
 throw new NotSupportedException(SR.GetString("net_unknown_prefix"));

Let’s try and have a break-point on line   “ ArrayList prefixList = PrefixList;” so that we can check the local variables value.  Just because I have the private symbols ,I could have counted the line numbers manually and then set a break-point using !mbp command, but that is no fun. Here is another way of doing this.

.shell -ci "!u 000007fef6ea5a44 "  findstr get_PrefixList

In the above command I am disassembling the ip 000007fef6ea5a44 (which is there in the above callstack )to look for  get_PrefixList Instruction pointer . Here is the outcome

0:000> .shell -ci “!u 000007fef6ea5a44 ”  findstr get_PrefixList

000007fe`f6ea5a7a e8f1000000      call    System_ni+0x275b70 (000007fe`f6ea5b70) (System.Net.WebRequest.get_PrefixList(), mdToken: 00000000060019bc)

.shell: Process exited

I use the .shell command to manually avoid searching for an instruction. Now that I have the instruction pointer ,I am going to set a break-point on that using

bp 000007fe`f6ea5a7a "!mdv"

and here is the result of the break-point

0:000> g
(1758.167c): CLR notification exception – code e0444143 (first chance)
(1758.167c): CLR notification exception – code e0444143 (first chance)
Frame 0x0: (System.Net.WebRequest.Create(System.Uri, Boolean)):
[L0]:LookupUri:0x00000000022c1a98 (System.String) STRVAL=
[L2]:Found:0x0000000000000000 (System.Boolean)
[L3]:LookupLength:0x0000000000000016 (System.Int32)

000007fe`f6ea5a7a e8f1000000      call    System_ni+0x275b70 (000007fe`f6ea5b70)

Voila! now I am able to have a break-point within the framework method  and also see locals and parameters like Visual Studio. The same technique can be used to debug third party assemblies where you don’t have source code or symbols.

Case of NullReferenceException not handled by sos / windbg

In this post I am going to be discussing about the NullReferenceException which is not trapped “sxe clr” command. FYI “sxe  clr” is the command in debugging tools to catch any clr exception.  Here is the kind of  code that I was debugging.

using System;

namespace Test
 class Program
 string test;

 static void Main(string[] args)
     new Program().Testing();
   catch {
    Console.WriteLine("Something went wrong");

 int Testing(){
   if (test.Substring(10,20) == "asd")
     return 10;
   return 1;

The application was reporting “something went wrong”, just another day where I had to debug some code which I don’t appreciate debugging. I was asked to debug the issue. And I use windbg for all my production debugging.

Attached the process and issued the command sxe -c “!clrstack;!pe” clr, which instructs the debugger to trap any exception from clr and then print stack-trace and exception whenever an exception is thrown. And to my surprise the debugger didn’t break on the exception and I never got the call-stack.  And  my debugger was set ignore AV exception so it didn’t report on AV, if not I could have managed to get the call-stack and figure out the exception.  And to my surprise when I issued the command !pe I didn’t get any result.

From my past experience of debugging I know if have bp on KERNELBASE!RaiseException  I should be able to catch any exception. This is one advantage of understanding code close to metal , comes in handy when everything else fails. So issued the command bp KERNELBASE!RaiseException and here is the call-stack from the breakpoint

0:000> !mk

Thread 0:

ESP              EIP

00:U 000000000031e738 000007fefdafaa40 KERNELBASE!RaiseException

01:U 000000000031e740 000007fee4b6dbdc mscorwks!NakedThrowHelper2+0xc

02:U 000000000031e770 000007fee4b6dc2a mscorwks!NakedThrowHelper_RspAligned+0x3d

03:U 000000000031ece8 000007fee4b6dc35 mscorwks!NakedThrowHelper_FixRsp+0x5

04:M 000000000031ecf0 000007ff001a027f Test.Program.Testing()(+0x1 IL)(+0x3f Native) [C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication4\Program.cs, @ 19,13]

05:M 000000000031ed40 000007ff001a0170 Test.Program.Main(System.String[])(+0x7 IL)(+0x50 Native) [C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication4\Program.cs, @ 10,17]

06:U 000000000031ed90 000007fee4b6d502 mscorwks!CallDescrWorker+0x82

07:U 000000000031ede0 000007fee4a29fd3 mscorwks!CallDescrWorkerWithHandler+0xd3

08:U 000000000031ee80 000007fee4a3a3af mscorwks!MethodDesc::CallDescr+0x24f

09:U 000000000031f0d0 000007fee49adc7f mscorwks!ClassLoader::RunMain+0x22b

0a:U 000000000031f330 000007fee4991c74 mscorwks!Assembly::ExecuteMainMethod+0xbc

0b:U 000000000031f620 000007fee49c9955 mscorwks!SystemDomain::ExecuteMainMethod+0x491

0c:U 000000000031fbf0 000007fee4addb07 mscorwks!ExecuteEXE+0x47

0d:U 000000000031fc40 000007fee499855c mscorwks!CorExeMain+0xac

0e:U 000000000031fca0 000007fef9493309 mscoreei!CorExeMain+0x41

0f:U 000000000031fcd0 000007fef9525b21 MSCOREE!CorExeMain_Exported+0x57

10:U 000000000031fd00 00000000776cf56d KERNEL32!BaseThreadInitThunk+0xd

11:U 000000000031fd30 0000000077903281 ntdll!RtlUserThreadStart+0x1d

Now I see which line is causing the exception. I could guess what the exception could be. To confirm my assumption I issued the command !dso after the catch block message and here is the output

0:000> !dso

OS Thread Id: 0x1590 (0)

RSP/REG          Object           Name

000000000031ea58 00000000026a5ae0 Microsoft.Win32.SafeHandles.SafeFileHandle

000000000031ea68 00000000026a5ae0 Microsoft.Win32.SafeHandles.SafeFileHandle

000000000031eb18 00000000026a5ae0 Microsoft.Win32.SafeHandles.SafeFileHandle

000000000031eb58 00000000026a5ae0 Microsoft.Win32.SafeHandles.SafeFileHandle

000000000031eb80 00000000026a5ae0 Microsoft.Win32.SafeHandles.SafeFileHandle

000000000031eba0 00000000026a5ae0 Microsoft.Win32.SafeHandles.SafeFileHandle

000000000031ebe0 00000000026a5b58 System.IO.StreamReader

000000000031ebf0 00000000026a5b58 System.IO.StreamReader

000000000031ec10 00000000026a5b58 System.IO.StreamReader

000000000031ec40 00000000026a5b58 System.IO.StreamReader

000000000031ec50 00000000026a5b58 System.IO.StreamReader

000000000031ec60 00000000026a5b00 System.IO.__ConsoleStream

000000000031ec68 00000000026a5e68 System.Byte[]    (System.Byte[])

000000000031ec98 00000000026a6198 System.IO.TextReader+SyncTextReader

000000000031eca0 00000000026a5b58 System.IO.StreamReader

000000000031ecb0 00000000026a5b58 System.IO.StreamReader

000000000031ecd8 00000000026a6198 System.IO.TextReader+SyncTextReader

000000000031ed40 00000000026a6198 System.IO.TextReader+SyncTextReader

000000000031ed48 00000000026a4070 System.String

000000000031ed68 00000000026a4058 Test.Program

000000000031ed78 00000000026a4090 System.NullReferenceException

000000000031ed90 00000000026a3ff0 System.Object[]    (System.String[])

000000000031ef18 00000000026a3ff0 System.Object[]    (System.String[])

000000000031f100 00000000026a3ff0 System.Object[]    (System.String[])

000000000031f128 00000000026a3ff0 System.Object[]    (System.String[])

I could see a NullReferenceException on the stack, issued !pe on exception object and here is the output

0:000> !pe 00000000026a4090

Exception object: 00000000026a4090

Exception type: System.NullReferenceException

Message: Object reference not set to an instance of an object.

InnerException: <none>

StackTrace (generated):

SP               IP               Function

000000000031ECF0 000007FF001A027F ConsoleApplication4!Test.Program.Testing()+0x3f

000000000031ED40 000007FF001A0170 ConsoleApplication4!Test.Program.Main(System.String[])+0x50

StackTraceString: <none>

HResult: 80004003

And now I know which code to fix.

Visualizing Silverlight / .NET managed memory using Pivot

I got to see this amazing video on Pivot from Gary Flake.  Pivot is about visualizing data. It’s primarily for visualizing patterns and trends. What other way I would use other than analyzing managed memory using pivot. I spend a lot of time in windbg to look at memory for patterns, analyzing memory leaks.  And I also know not many of them want to learn the arcane commands of windbg. So I went ahead took the data from managed memory and ported it in to pivot format , which is cxml. I use the “!Traverseheap –xml” command from sos to generate the xml ,which I port it to cxml format.My initial thought was use dgml to visualize the data, but pivot is way better. Pivot can only be used in Vista+  OS.

FYI I know this is not for everyone, there are geeks who wouldn’t want to look at memory in a visual format, they prefer looking at in cdb.  I took Facebook Silverlight client app and got a snapshot of the memory for this demonstration

The few things that can be done with this are

  • Drill down objects by type with just one click : What I mean by this is , you could probably have 6000 odd types of objects and not instances (instances could be in thousands), but if you wanted to look at all the instance of customer objects  , it is as easy as one click from the filter.

Here is an example of filtering securitypermission objects in memory

  • Drill down members: The objects members can be looked up. So if you are looking at the customer object then you can view  strings, ints and all its members. It WILL NOT have the value for these members. It is not provided by the traverseheap command

  • Filter objects by namespace– All the objects belonging to namespaces can be drilled down.

  • Filter by object size

void Main()
 var startString =@"<?xml version=""1.0"" encoding=""utf-8""?>
<?xml-stylesheet type=""text/xsl"" href=""../pivot.xsl""?>
<Collection xmlns:xsi="""" xmlns:xsd=""""
 Name=""Memory Visualization"" SchemaVersion=""1"" d1p1:Icon=""icon.ico""
 xmlns:d1p1="""" xmlns="""">
 <FacetCategory Name=""Type"" Type=""String"" d1p1:IsFilterVisible=""true"" d1p1:IsMetaDataVisible=""true"" d1p1:IsWordWheelVisible=""true"" />
 <FacetCategory Name=""Address"" Type=""String"" d1p1:IsFilterVisible=""true"" d1p1:IsMetaDataVisible=""true"" d1p1:IsWordWheelVisible=""true"" />
 <FacetCategory Name=""Size""  Type=""Number"" d1p1:IsFilterVisible=""true"" d1p1:IsMetaDataVisible=""true"" d1p1:IsWordWheelVisible=""true"" />
 <FacetCategory Name=""Namespaces"" Type=""Link"" d1p1:IsFilterVisible=""true"" d1p1:IsMetaDataVisible=""true"" d1p1:IsWordWheelVisible=""true"" />
 <FacetCategory Name=""Members"" Type=""Link"" d1p1:IsFilterVisible=""true"" d1p1:IsMetaDataVisible=""true"" d1p1:IsWordWheelVisible=""true"" />
 <Items ImgBase=""test.gif"">";

 var endString = @"</Items>

 var counter = 0;
 var outputFileName = "sl.cxml";
 var dict = new Dictionary<string,string>() { {"String","img5.jpg"},{"Object","img6.jpg"}};
 var getimg = new Func<string,string>((t) => dict.ContainsKey(t) ? dict[t] : "img8.jpg");

 var doc = XDocument.Load(@"c:\Temp\fb.xml");

 var types = (from i in doc.Descendants().Descendants().Descendants("type").AsParallel()
 let ns = i.Attribute("name").Value.SplitEx().ToList()
 select new {Id = int.Parse( i.FirstAttribute.Value),Name = ns.Last(),
 Namespaces = ns.Count()  > 2 ? ns.Take(ns.Count () -2) : ns.Take(ns.Count -1)}).ToList();
 var y = from i in doc.Descendants().Descendants().Descendants("object").AsParallel()
 let type = types.First (t => t.Id== int.Parse( i.Attribute("typeid").Value))
 select new {Address = i.FirstAttribute.Value,
 Type = type.Name,
 Namespaces = type.Namespaces,
 Size = int.Parse( i.LastAttribute.Value),
 children = from k in i.Descendants().AsParallel()
 from o in doc.Descendants().Descendants().Descendants("object").AsParallel()
 where k.FirstAttribute.Value == o.FirstAttribute.Value
 select new {Address = k.FirstAttribute.Value,Type = types.First (t => t.Id == int.Parse( o.Attribute("typeid").Value) ).Name}

 var x = from i in y.Take(2000).AsParallel()
 let increment = new Func<int>(() => Interlocked.Increment(ref counter))
 select new XElement("Item",new XAttribute("Img",getimg(i.Type)),new XAttribute("Id",increment()),new XAttribute("Name",i.Type),
 new XElement("Facets",
 new XElement("Facet",new XAttribute("Name","Type"),
 new XElement("String",new XAttribute("Value",i.Type))),
 new XElement("Facet",new XAttribute("Name","Size"),
 new XElement("Number",new XAttribute("Value",i.Size.ToString()))),
 new XElement("Facet",new XAttribute("Name","Address"),
 new XElement("String",new XAttribute("Value",i.Address))),
 new XElement("Facet",new XAttribute("Name","Namespaces"),
 i.Namespaces.Select (n =>new XElement("Link",new XAttribute("Name",n),new XAttribute("Href",string.Format("{0}#Link=EQ.{1}",outputFileName,n))))),
 i.children.Any() == false ? default(XElement):
 new XElement("Facet",new XAttribute("Name","Members"),
 i.children.Select (n =>new XElement("Link",new XAttribute("Name",n.Type),new XAttribute("Href",string.Format("{0}#Address=EQ.{1}",outputFileName,n.Address)))))
 File.WriteAllText(@"c:\temp\sl.cxml",string.Format("{0}{1}{2}",startString,x.Select (s => s).Aggregate(new StringBuilder(), (ag, n) => ag.Append(n)).ToString(),endString));
 Console.WriteLine ("Done");

static class Foo
 public static IEnumerable<string> SplitEx(this string s)
 if (s.Contains("`") == false)
 return s.Split(new []{'.'});
 var stringList = new List<string>();
 var index =0;
 var counter = 0;
 var splitString = "";
 while (counter < s.Length)
 index = s.IndexOf(".",counter);
 if (index < 0)
 splitString = s.Substring(counter,index -counter );
 if (splitString.Contains("`"))
 counter = index +1;
 return stringList;

The above code converts the xml format from !traverseheap to pivot xml format. This is CPU intensive and I have not profiled and fine tuned it. I use plinq for conversion primarily because I have quad-core box. I use pictures from powerpoint for visualizing  objects and I also wrote SplitEx method to avoid splitting namespaces that are part of the type name within generics “GenericEqualityComparer`1[[System.String, mscorlib]]”

And here is the command to get the xml input for the above code as input

!TraverseHeap -xml c:\temp\mem.xml

Remote Debugging with Windbg and Powershell Remoting

Recently I had to debug an issue which was running on Client OS box. I had to do a live debugging more than post-mortem debugging.  FYI the box didn’t have VS.NET and it had copy of debugger’s folder (windbg) along with Powershell v2.0.  I couldn’t  terminal server onto the box because it would kick the other user out and the application would terminate. So only choice was to either go-down actual box and debug it or instruct someone to start a .server process within cdb/windbg. Both the options had its own set of problems. That’s when I realized that I could use powershell remoting to start remote session and then start a remote cdb session. I like to debug from my Dev box because of the tools and settings that I have on my box.  Here is what I did

  1. Started a Powershell ISE environment
  2. Connected to the remote system. (there were quite a few issues is authentication when trying to connect)
  3. Navigated to debuggers folder on the client box cd ‘.\Program Files\Debugging Tools for Windows (x64)’
  4. Then issued the command .\cdb -pn notepad.exe -c “.server npipe:pipe=RemoteDebugging”
  5. I am using a simple notepad.exe for demonstration
  6. Then from my devbox connected to using the command “cdb -remote npipe:Pipe=RemoteDebugging,Server=NAVEEN-PC”

Powershell remoting feels Bash, so one less reason to use remote desktop.

Exploring UnhandledException in .NET and Watson buckets

I wanted to understand about UnhandledExceptions in .NET because of the few questions that I saw in the CLR Forum ,which had watson buckets in the event viewer.  To get deep understanding  unhandled exception filter there is a article on MSDN from CLR Team.

In this post I will be demonstrating how to get the Watson Bucket from within your code . FYI this is the same information  you can get !WatsonBuckets from sos within Windbg whenever there is a termination of the .NET process. !WatsonBuckets is undocumented.  I am using the CLR hosting interfaces to get the watson bucket. Here is the code that throws an unhandled exception and invokes clr to get watson bucket info.

using System;
using System.Runtime.InteropServices;

namespace WatsonTest
 internal class Test
 private static void Main(string[] args)
 var t = AppDomain.CurrentDomain;
 t.UnhandledException += (s, e) =>
 var c = GetWatsonBuckets();
 throw new NullReferenceException();
 private static WatsonBuckets GetWatsonBuckets()
 var pParams = new WatsonBuckets();
 IClrRuntimeHost host = null;
 host = Activator.CreateInstance(Type.GetTypeFromCLSID(ClrGuids.ClsIdClrRuntimeHost)) as IClrRuntimeHost;
 if (host != null)
 var clrControl = host.GetCLRControl();
 if (clrControl == null)
 return pParams;
 var clrErrorReportingManager =
 clrControl.GetCLRManager(ref ClrGuids.IClrErrorReportingManager) as IClrErrorReportingManager;
 if (clrErrorReportingManager == null)
 return pParams;
 clrErrorReportingManager.GetBucketParametersForCurrentException(out pParams);
 return pParams;
 // BucketParameters Structure to get watson buckets back from CLR
 [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Auto)]
 internal struct WatsonBuckets
 internal int fInited;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string pszEventTypeName;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param0;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param1;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param2;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param3;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param4;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param5;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param6;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param7;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param8;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param9;

 internal static class ClrGuids
 internal static readonly Guid ClsIdClrRuntimeHost = new Guid("90F1A06E-7712-4762-86B5-7A5EBA6BDB02");
 internal static Guid IClrErrorReportingManager = new Guid("980D2F1A-BF79-4c08-812A-BB9778928F78");
 internal static readonly Guid IClrRuntimeHost = new Guid("90F1A06C-7712-4762-86B5-7A5EBA6BDB02");

 [Guid("90F1A06C-7712-4762-86B5-7A5EBA6BDB02"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
 internal interface IClrRuntimeHost
 void Start();
 void Stop();
 void SetHostControl(IntPtr pHostControl);
 IClrControl GetCLRControl();
 void UnloadAppDomain(int dwAppDomainId, bool fWaitUntilDone);
 void ExecuteInAppDomain(int dwAppDomainId, IntPtr pCallback, IntPtr cookie);
 int GetCurrentAppDomainId();

 int ExecuteApplication(string pwzAppFullName, int dwManifestPaths, string[] ppwzManifestPaths,
 int dwActivationData, string[] ppwzActivationData);

 int ExecuteInDefaultAppDomain(string pwzAssemblyPath, string pwzTypeName, string pwzMethodName,
 string pwzArgument);

 [Guid("9065597E-D1A1-4fb2-B6BA-7E1FCE230F61"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
 internal interface IClrControl
 [return: MarshalAs(UnmanagedType.IUnknown)]
 object GetCLRManager([In] ref Guid riid);

 void SetAppDomainManagerType(string pwzAppDomainManagerAssembly, string pwzAppDomainManagerType);
 // IClrErrorReportingManager to get watson bukets back from CLR
 [Guid("980D2F1A-BF79-4c08-812A-BB9778928F78"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
 internal interface IClrErrorReportingManager
 int GetBucketParametersForCurrentException(out WatsonBuckets pParams);

The watson bucket information has ten items

  1. AppName
  2. Version
  3. ?
  4. Assembly and Module Name
  5. Assembly Version
  6. ?
  7. MethodDef
  8. IL Offset
  9. Exception Type
  10. ?

In the above code WatsonBuckets is a structure to get watson bucket information back from CLR.  In the code I use 3 hosting interfaces, IClrRuntimeHost which is the main interface, the next interface is IClrControl which has GetCLRManager method to get ErrorReportingManger. IClrErrorReportingManager which has the method to get the exception buckets which is GetBucketParametersForCurrentException. I know that there isn’t going to be much of use for this code. But it gives me better understanding of  CLR integrates with watson when ever there is termination of an application.

Exploring SOSEX and Windbg to debug .NET 4.0

With the latest release of sosex comes a new set of functions to debug. It is pretty awesome that one person (Steve) alone could pull of such cool things. In this blog post, I am just going to demonstrate how easy it is to debug managed code using sosex compared to sos.

using System;
using System.Collections.Generic;

namespace MemCheck
 internal class Test
 Dictionary<int, string> dict = new Dictionary<int, string>();
 private static void Main(string[] args)
 var p = new Test();
 for (int i = 0; i < 100; i++)
 p.dict.Add(i, i.ToString());

I like to keep the code simple , so it is easy to follow.  The debugging goal for today is to get the Dictionary values. First I am going to demonstrate it using sos and then using sosex.

As usual I start the app and then attach it to windbg.

.loadby sos clr

FYI in  .net 4.0 clr is the dll that has CLR implementation. In prior versions it used to be in mscorwks. The next command would look for the object Test in the memory

!dumpheap -type MemCheck.Test
0:000> !dumpheap -type MemCheck.Test
 Address               MT     Size
0000000002761e20 000007ff00054110       24
 total 0 objects
 MT    Count    TotalSize Class Name
 000007ff00054110        1           24 MemCheck.Test
 Total 1 objects

The next step is to dump the object

0:000> !do 0000000002761e20
Name:        MemCheck.Test
MethodTable: 000007ff00054110
EEClass:     000007ff00162350
Size:        24(0x18) bytes
File:        C:\Users\naveen\Documents\Visual Studio 2010\Projects\Test\bin\Debug\Test.exe
 MT    Field   Offset                 Type VT     Attr            Value Name
000007feec2b7a48  4000001        8 ...tring, mscorlib]]  0 instance 0000000002761e38 dict

Notice the dict object is in the 8th offset . To dump contents  dict object I would use the command !do poi(0000000002761e20+8) , which is pointer deference of Test object on it is 8th offset.  And here is the output

0:000> !do poi(0000000002761e20+8)
Name:        System.Collections.Generic.Dictionary`2[[System.Int32, mscorlib],[System.String, mscorlib]]
MethodTable: 000007feec2b7a48
EEClass:     000007feebe113c0
Size:        88(0x58) bytes
File:        C:\Windows\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll
 MT    Field   Offset                 Type VT     Attr            Value Name
000007feec27c7d8  4000bee        8       System.Int32[]  0 instance 0000000002764788 buckets
000007feecbd3dc8  4000bef       10 ...non, mscorlib]][]  0 instance 0000000002764ab8 entries
000007feec27c848  4000bf0       40         System.Int32  1 instance              100 count
000007feec27c848  4000bf1       44         System.Int32  1 instance              100 version
000007feec27c848  4000bf2       48         System.Int32  1 instance               -1 freeList
000007feec27c848  4000bf3       4c         System.Int32  1 instance                0 freeCount
000007feec2a5a48  4000bf4       18 ...Int32, mscorlib]]  0 instance 0000000002761ef0 comparer
000007feecc75f78  4000bf5       20 ...Canon, mscorlib]]  0 instance 0000000000000000 keys
000007feecc72078  4000bf6       28 ...Canon, mscorlib]]  0 instance 0000000000000000 values
000007feec275ab8  4000bf7       30        System.Object  0 instance 0000000000000000 _syncRoot
000007feec29a1b8  4000bf8       38 ...SerializationInfo  0 instance 0000000000000000 m_siInfo

And the dictionary object in turn stores them within an array which is again the 8th offset.  This time because we know it is an array we are going to use the !dumparray command on the memory location. The command to get the details is

!dumparray -details poi(poi(0000000002761e20+8)+8)
MT    Field   Offset                 Type VT     Attr            Value Name
 000007feec27c848  400047b        0             System.Int32      1     instance                   -1     m_value
 [195] 0000000002764aa4
 Name:        System.Int32
 MethodTable: 000007feec27c848
 EEClass:     000007feebe00890
 Size:        24(0x18) bytes
 File:        C:\Windows\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll
 MT    Field   Offset                 Type VT     Attr            Value Name
 000007feec27c848  400047b        0             System.Int32      1     instance                   -1     m_value
 [196] 0000000002764aa8
 Name:        System.Int32
 MethodTable: 000007feec27c848
 EEClass:     000007feebe00890
 Size:        24(0x18) bytes
 File:        C:\Windows\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll
 MT    Field   Offset                 Type VT     Attr            Value Name
 000007feec27c848  400047b        0             System.Int32      1     instance                   -1     m_value

Here is the partial output.

Now lets try and do the same thing using sosex. The one thing that I really like about  the new sosex is that  I can use names rather than pointer deference which is way much easier. Launched the app and then loaded sosex using the command

.load F:\Work\Tools\debuggers\sosex.dll

and then switched the thread from 4th to 0th thread using ~0s.  By default the debugger injects a thread into the process for debugging and that was the 4th thread. The next command I issued was to get stack trace

0:000> !mk
Thread 0:
 ESP              EIP
00:U 000000000015e408 0000000077bc00da ntdll!ZwRequestWaitReplyPort+0xa
01:U 000000000015e410 0000000077a72b08 KERNEL32!ConsoleClientCallServer+0x54
02:U 000000000015e440 0000000077aa5601 KERNEL32!ReadConsoleInternal+0x1f1
03:U 000000000015e590 0000000077aba922 KERNEL32!ReadConsoleA+0xb2
04:U 000000000015e670 0000000077a89934 KERNEL32!zzz_AsmCodeRange_End+0x8bea
05:U 000000000015e6b0 000007feed0317c7 clr!DoNDirectCall__PatchGetThreadCall+0x7b
06:M 000000000015e760 000007feec1d34a1 DomainNeutralILStubClass.IL_STUB_PInvoke(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte*, Int32, Int32 ByRef, IntPtr)(+0x0 IL)(+0x0 Native)
07:M 000000000015e880 000007feec97f59a System.IO.__ConsoleStream.ReadFileNative(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte[], Int32, Int32, Int32, Int32 ByRef)(+0x53 IL)(+0xba Native)
08:M 000000000015e8f0 000007feec97f402 System.IO.__ConsoleStream.Read(Byte[], Int32, Int32)(+0x5d IL)(+0x62 Native)
09:M 000000000015e950 000007feec18e63c System.IO.StreamReader.ReadBuffer()(+0xa0 IL)(+0x5c Native)
0a:M 000000000015e9a0 000007feec915630 System.IO.StreamReader.Read()(+0x21 IL)(+0x30 Native)
0b:M 000000000015e9e0 000007feec987458 System.IO.TextReader+SyncTextReader.Read()(+0x0 IL)(+0x38 Native)
0c:M 000000000015ea30 000007ff00170213 MemCheck.Test.Main(System.String[])(+0x39 IL)(+0xf3 Native) [C:\Users\naveen\Documents\Visual Studio 2010\Projects\Test\Program.cs, @ 17,13]
0d:U 000000000015eaa0 000007feed0710b4 clr!CallDescrWorker+0x84
0e:U 000000000015eaf0 000007feed0711c9 clr!CallDescrWorkerWithHandler+0xa9
0f:U 000000000015eb70 000007feed071245 clr!MethodDesc::CallDescr+0x2a1
10:U 000000000015eda0 000007feed171675 clr!ClassLoader::RunMain+0x228
11:U 000000000015eff0 000007feed1717ac clr!Assembly::ExecuteMainMethod+0xac
12:U 000000000015f2a0 000007feed171562 clr!SystemDomain::ExecuteMainMethod+0x452
13:U 000000000015f850 000007feed173dd6 clr!ExecuteEXE+0x43
14:U 000000000015f8b0 000007feed173cf3 clr!CorExeMainInternal+0xc4
15:U 000000000015f920 000007feed1f7365 clr!CorExeMain+0x15
16:U 000000000015f960 000007fef8f13309 mscoreei!CorExeMain+0x41
17:U 000000000015f990 000007fef8fa5b21 MSCOREE!CorExeMain_Exported+0x57
18:U 000000000015f9c0 0000000077a6f56d KERNEL32!BaseThreadInitThunk+0xd
19:U 000000000015f9f0 0000000077ba3281 ntdll!RtlUserThreadStart+0x1d

FYI the command !mk has been part of sos from the initial version. I am interested in only looking at the code that I wrote so I would like to move stack frame to 0c which is MemCheck.Test.Main . To do that the command  is !mframe 0c, which moves to that stackframe. The reason to move the particular stack frame is to look for variables in the stack and the command to variables is !mdv , which display managed local variables

0:000> !mdv
Frame 0xc: (MemCheck.Test.Main(System.String[])):
[A0]:args:0x0000000002761dd8 (System.String[])
[L0]:p:0x0000000002761e20 (MemCheck.Test)
[L1]:i:0x0000000000000064 (System.Int32)
[L2]:CS$4$0000:0x0000000000000000 (System.Boolean)

Notice we see the local variable “p” which is of type MemCheck.Test. To display type p we issue the command !mdt p

0:000> !mdt p

0000000002761e20 (MemCheck.Test)

dict:0000000002761e38 (System.Collections.Generic.Dictionary`2[[System.Int32, mscorlib],[System.String, mscorlib]])

I didn’t  have to get memory address , I am using the names which is very intuitive ,especially when we have to debug large application with N levels of nesting.So to get the dict values   from p the command to issue is !mdt -e p.dict

!mdt -e p.dict
[98] (System.Collections.Generic.Dictionary`2+Entry[[System.Int32, mscorlib],[System.String, mscorlib]]) VALTYPE (MT=000007feec2b7b28, ADDR=0000000002765400)
 key:0x62 (System.Int32)
 value:0000000002765e48 (System.String: "98")
[99] (System.Collections.Generic.Dictionary`2+Entry[[System.Int32, mscorlib],[System.String, mscorlib]]) VALTYPE (MT=000007feec2b7b28, ADDR=0000000002765418)
 key:0x63 (System.Int32)
 value:0000000002765e68 (System.String: "99")

Here is the partial output. Notice I never had to use a memory pointer or do a pointer deference .This is very similar to VS.NET debugging where I am used to the variable names compared the memory address.  Thanks to Steve for providing such a cool extension.

Undocumented PSSCOR2 functions

With the latest release of PSSCOR2 release from Microsoft comes with a set Undocumented functions. FYI PSSCOR2 is a superset of sos.dll for .NET framework 2.0/3.5, it also has bunch of functions which are available in the .NET 4.0 sos.dll.

The .NET 4.0 also has few undocumented functions and PSSCOR2 does have them. But these functions are totally new and is not even available in .NET 4.0 sos

  1. !FindInterestingStrings
  2. !SaveSnapshot
  3. !LoadSnapshot
  4. !dumpconfig
  5. !GCUsage
  6. !VerifyDAC

I did manage to figure out the usage of few of these, but I let you explore them before I blog about this.