Debugging .Net Framework 4.0 without source code using windbg


In this post I am going to be discussing about debugging .Net Framework 4.0  using windbg . I am going to demonstrating how to have a break-point within a method, but without the framework source code. This would help in debugging .NET framework when you don’t have VS in a production environment and the same technique can be used to debug other third party assemblies where you don’t have the source code.  This is kind of like .NET Reflector where you can step through third party assemblies, but without any cost. It is not going to be as convenient as the professional version of Reflector.

I am going to be using the same example that I used to debug .NET Framework source 3.5 using windbg.

FYI the .NET framework 4.0 has private symbols available on MS symbol server, but the source code is still not available. To debug .NET framework source code it is important to have correct symbol path and here is my symbol path in the _NT_SYMBOL_PATH environment variable.


SRV*d:\dev\symbols*http://referencesource.microsoft.com/symbols; SRV*d:\dev\symbols*http://msdl.microsoft.com/download/symbols

Here is the sample source code that I am going to be using to demonstrate framework debugging


using System;
 using System.Net;
 namespace Test
 {
 class Program
 {
 static void Main(string[] args)
 {
 Console.WriteLine("Hello World of debugging");
 var wr = WebRequest.Create("http://www.google.com");
 Console.WriteLine("Web request created");
 var req = wr.GetRequestStream();
 Console.WriteLine("Hello World Debugging");
 Console.Read();
 }
 }
 }
 

Launched the exe within the debugger

Then issued the command to notify when the clrjit is loaded,


sxe ld:clrjit

This is because, to load sos and sosex after the framework is loaded.Then issued the following commands to load sos, sosex and to set break-point on WebRequest.Create


.loadby sos mscorwks
.load sosex
!mbm System.Net.WebRequest.Create

And when the break-point hits the first time, let it continue by using the g command. It would break into the debugger for other overloaded method for WebRequest.Create and here is the call-stack

0:000> !mk
Thread 0:
ESP              EIP
00:M 000000000023ece0 000007fef6ea5a44 System.Net.WebRequest.Create(System.Uri, Boolean)(+0x0 IL)(+0x14 Native) [f:\dd\ndp\fx\src\Net\System\Net\WebRequest.cs, @ 93,13]
01:M 000000000023ed60 000007ff00140176 Test.Program.Main(System.String[])(+0xc IL)(+0x56 Native)
02:U 000000000023edc0 000007fef8b210b4 clr!CallDescrWorker+0x84
03:U 000000000023ee10 000007fef8b211c9 clr!CallDescrWorkerWithHandler+0xa9
04:U 000000000023ee90 000007fef8b21245 clr!MethodDesc::CallDescr+0x2a1
05:U 000000000023f0c0 000007fef8c21675 clr!ClassLoader::RunMain+0x228
06:U 000000000023f310 000007fef8c217ac clr!Assembly::ExecuteMainMethod+0xac
07:U 000000000023f5c0 000007fef8c21562 clr!SystemDomain::ExecuteMainMethod+0x452
08:U 000000000023fb70 000007fef8c23dd6 clr!ExecuteEXE+0x43
09:U 000000000023fbd0 000007fef8c23cf3 clr!CorExeMainInternal+0xc4
0a:U 000000000023fc40 000007fef8ca7365 clr!CorExeMain+0x15
0b:U 000000000023fc80 000007fef9493309 mscoreei!CorExeMain+0x41
0c:U 000000000023fcb0 000007fef9525b21 MSCOREE!CorExeMain_Exported+0x57
0d:U 000000000023fce0 00000000776cf56d KERNEL32!BaseThreadInitThunk+0xd
0e:U 000000000023fd10 0000000077903281 ntdll!RtlUserThreadStart+0x1d

And here is the source code for this method using reflector


private static WebRequest Create(Uri requestUri, bool useUriBase)
{
 string absoluteUri;
 if (Logging.On)
 {
 Logging.Enter(Logging.Web, "WebRequest", "Create", requestUri.ToString());
 }
 WebRequestPrefixElement element = null;
 bool flag = false;
 if (!useUriBase)
 {
 absoluteUri = requestUri.AbsoluteUri;
 }
 else
 {
 absoluteUri = requestUri.Scheme + ':';
 }
 int length = absoluteUri.Length;
 ArrayList prefixList = PrefixList;
 for (int i = 0; i < prefixList.Count; i++)
 {
 element = (WebRequestPrefixElement) prefixList[i];
 if ((length >= element.Prefix.Length) && (string.Compare(element.Prefix, 0, absoluteUri, 0, element.Prefix.Length, StringComparison.OrdinalIgnoreCase) == 0))
 {
 flag = true;
 break;
 }
 }
 WebRequest retObject = null;
 if (flag)
 {
 retObject = element.Creator.Create(requestUri);
 if (Logging.On)
 {
 Logging.Exit(Logging.Web, "WebRequest", "Create", retObject);
 }
 return retObject;
 }
 if (Logging.On)
 {
 Logging.Exit(Logging.Web, "WebRequest", "Create", (string) null);
 }
 throw new NotSupportedException(SR.GetString("net_unknown_prefix"));
}

Let’s try and have a break-point on line   “ ArrayList prefixList = PrefixList;” so that we can check the local variables value.  Just because I have the private symbols ,I could have counted the line numbers manually and then set a break-point using !mbp command, but that is no fun. Here is another way of doing this.


.shell -ci "!u 000007fef6ea5a44 "  findstr get_PrefixList

In the above command I am disassembling the ip 000007fef6ea5a44 (which is there in the above callstack )to look for  get_PrefixList Instruction pointer . Here is the outcome

0:000> .shell -ci “!u 000007fef6ea5a44 ”  findstr get_PrefixList

000007fe`f6ea5a7a e8f1000000      call    System_ni+0x275b70 (000007fe`f6ea5b70) (System.Net.WebRequest.get_PrefixList(), mdToken: 00000000060019bc)

.shell: Process exited

I use the .shell command to manually avoid searching for an instruction. Now that I have the instruction pointer ,I am going to set a break-point on that using


bp 000007fe`f6ea5a7a "!mdv"

and here is the result of the break-point

0:000> g
(1758.167c): CLR notification exception – code e0444143 (first chance)
(1758.167c): CLR notification exception – code e0444143 (first chance)
Frame 0x0: (System.Net.WebRequest.Create(System.Uri, Boolean)):
[A0]:requestUri:<?>
[A1]:useUriBase:<?>
[L0]:LookupUri:0x00000000022c1a98 (System.String) STRVAL=http://www.google.com/
[L1]:Current:<?>
[L2]:Found:0x0000000000000000 (System.Boolean)
[L3]:LookupLength:0x0000000000000016 (System.Int32)
[L4]:prefixList:<?>
[L5]:i:<?>
[L6]:webRequest:<?>

System_ni+0x275a7a:
000007fe`f6ea5a7a e8f1000000      call    System_ni+0x275b70 (000007fe`f6ea5b70)

Voila! now I am able to have a break-point within the framework method  and also see locals and parameters like Visual Studio. The same technique can be used to debug third party assemblies where you don’t have source code or symbols.

Case of NullReferenceException not handled by sos / windbg


In this post I am going to be discussing about the NullReferenceException which is not trapped “sxe clr” command. FYI “sxe  clr” is the command in debugging tools to catch any clr exception.  Here is the kind of  code that I was debugging.


using System;

namespace Test
{
 class Program
 {
 string test;

 static void Main(string[] args)
  {
   try{
     new Program().Testing();
     }
   catch {
    Console.WriteLine("Something went wrong");
    }
   Console.Read();
 }

 int Testing(){
   if (test.Substring(10,20) == "asd")
     return 10;
   return 1;
   }
 }
}

The application was reporting “something went wrong”, just another day where I had to debug some code which I don’t appreciate debugging. I was asked to debug the issue. And I use windbg for all my production debugging.

Attached the process and issued the command sxe -c “!clrstack;!pe” clr, which instructs the debugger to trap any exception from clr and then print stack-trace and exception whenever an exception is thrown. And to my surprise the debugger didn’t break on the exception and I never got the call-stack.  And  my debugger was set ignore AV exception so it didn’t report on AV, if not I could have managed to get the call-stack and figure out the exception.  And to my surprise when I issued the command !pe I didn’t get any result.

From my past experience of debugging I know if have bp on KERNELBASE!RaiseException  I should be able to catch any exception. This is one advantage of understanding code close to metal , comes in handy when everything else fails. So issued the command bp KERNELBASE!RaiseException and here is the call-stack from the breakpoint

0:000> !mk

Thread 0:

ESP              EIP

00:U 000000000031e738 000007fefdafaa40 KERNELBASE!RaiseException

01:U 000000000031e740 000007fee4b6dbdc mscorwks!NakedThrowHelper2+0xc

02:U 000000000031e770 000007fee4b6dc2a mscorwks!NakedThrowHelper_RspAligned+0x3d

03:U 000000000031ece8 000007fee4b6dc35 mscorwks!NakedThrowHelper_FixRsp+0x5

04:M 000000000031ecf0 000007ff001a027f Test.Program.Testing()(+0x1 IL)(+0x3f Native) [C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication4\Program.cs, @ 19,13]

05:M 000000000031ed40 000007ff001a0170 Test.Program.Main(System.String[])(+0x7 IL)(+0x50 Native) [C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication4\Program.cs, @ 10,17]

06:U 000000000031ed90 000007fee4b6d502 mscorwks!CallDescrWorker+0x82

07:U 000000000031ede0 000007fee4a29fd3 mscorwks!CallDescrWorkerWithHandler+0xd3

08:U 000000000031ee80 000007fee4a3a3af mscorwks!MethodDesc::CallDescr+0x24f

09:U 000000000031f0d0 000007fee49adc7f mscorwks!ClassLoader::RunMain+0x22b

0a:U 000000000031f330 000007fee4991c74 mscorwks!Assembly::ExecuteMainMethod+0xbc

0b:U 000000000031f620 000007fee49c9955 mscorwks!SystemDomain::ExecuteMainMethod+0x491

0c:U 000000000031fbf0 000007fee4addb07 mscorwks!ExecuteEXE+0x47

0d:U 000000000031fc40 000007fee499855c mscorwks!CorExeMain+0xac

0e:U 000000000031fca0 000007fef9493309 mscoreei!CorExeMain+0x41

0f:U 000000000031fcd0 000007fef9525b21 MSCOREE!CorExeMain_Exported+0x57

10:U 000000000031fd00 00000000776cf56d KERNEL32!BaseThreadInitThunk+0xd

11:U 000000000031fd30 0000000077903281 ntdll!RtlUserThreadStart+0x1d

Now I see which line is causing the exception. I could guess what the exception could be. To confirm my assumption I issued the command !dso after the catch block message and here is the output

0:000> !dso

OS Thread Id: 0x1590 (0)

RSP/REG          Object           Name

000000000031ea58 00000000026a5ae0 Microsoft.Win32.SafeHandles.SafeFileHandle

000000000031ea68 00000000026a5ae0 Microsoft.Win32.SafeHandles.SafeFileHandle

000000000031eb18 00000000026a5ae0 Microsoft.Win32.SafeHandles.SafeFileHandle

000000000031eb58 00000000026a5ae0 Microsoft.Win32.SafeHandles.SafeFileHandle

000000000031eb80 00000000026a5ae0 Microsoft.Win32.SafeHandles.SafeFileHandle

000000000031eba0 00000000026a5ae0 Microsoft.Win32.SafeHandles.SafeFileHandle

000000000031ebe0 00000000026a5b58 System.IO.StreamReader

000000000031ebf0 00000000026a5b58 System.IO.StreamReader

000000000031ec10 00000000026a5b58 System.IO.StreamReader

000000000031ec40 00000000026a5b58 System.IO.StreamReader

000000000031ec50 00000000026a5b58 System.IO.StreamReader

000000000031ec60 00000000026a5b00 System.IO.__ConsoleStream

000000000031ec68 00000000026a5e68 System.Byte[]    (System.Byte[])

000000000031ec98 00000000026a6198 System.IO.TextReader+SyncTextReader

000000000031eca0 00000000026a5b58 System.IO.StreamReader

000000000031ecb0 00000000026a5b58 System.IO.StreamReader

000000000031ecd8 00000000026a6198 System.IO.TextReader+SyncTextReader

000000000031ed40 00000000026a6198 System.IO.TextReader+SyncTextReader

000000000031ed48 00000000026a4070 System.String

000000000031ed68 00000000026a4058 Test.Program

000000000031ed78 00000000026a4090 System.NullReferenceException

000000000031ed90 00000000026a3ff0 System.Object[]    (System.String[])

000000000031ef18 00000000026a3ff0 System.Object[]    (System.String[])

000000000031f100 00000000026a3ff0 System.Object[]    (System.String[])

000000000031f128 00000000026a3ff0 System.Object[]    (System.String[])

I could see a NullReferenceException on the stack, issued !pe on exception object and here is the output

0:000> !pe 00000000026a4090

Exception object: 00000000026a4090

Exception type: System.NullReferenceException

Message: Object reference not set to an instance of an object.

InnerException: <none>

StackTrace (generated):

SP               IP               Function

000000000031ECF0 000007FF001A027F ConsoleApplication4!Test.Program.Testing()+0x3f

000000000031ED40 000007FF001A0170 ConsoleApplication4!Test.Program.Main(System.String[])+0x50

StackTraceString: <none>

HResult: 80004003

And now I know which code to fix.

Visualizing Silverlight / .NET managed memory using Pivot


I got to see this amazing video on Pivot from Gary Flake.  Pivot is about visualizing data. It’s primarily for visualizing patterns and trends. What other way I would use other than analyzing managed memory using pivot. I spend a lot of time in windbg to look at memory for patterns, analyzing memory leaks.  And I also know not many of them want to learn the arcane commands of windbg. So I went ahead took the data from managed memory and ported it in to pivot format , which is cxml. I use the “!Traverseheap –xml” command from sos to generate the xml ,which I port it to cxml format.My initial thought was use dgml to visualize the data, but pivot is way better. Pivot can only be used in Vista+  OS.

FYI I know this is not for everyone, there are geeks who wouldn’t want to look at memory in a visual format, they prefer looking at in cdb.  I took Facebook Silverlight client app and got a snapshot of the memory for this demonstration

The few things that can be done with this are

  • Drill down objects by type with just one click : What I mean by this is , you could probably have 6000 odd types of objects and not instances (instances could be in thousands), but if you wanted to look at all the instance of customer objects  , it is as easy as one click from the filter.

Here is an example of filtering securitypermission objects in memory

  • Drill down members: The objects members can be looked up. So if you are looking at the customer object then you can view  strings, ints and all its members. It WILL NOT have the value for these members. It is not provided by the traverseheap command

  • Filter objects by namespace– All the objects belonging to namespaces can be drilled down.

  • Filter by object size


void Main()
{
 var startString =@"<?xml version=""1.0"" encoding=""utf-8""?>
<?xml-stylesheet type=""text/xsl"" href=""../pivot.xsl""?>
<Collection xmlns:xsi=""http://www.w3.org/2001/XMLSchema-instance"" xmlns:xsd=""http://www.w3.org/2001/XMLSchema""
 Name=""Memory Visualization"" SchemaVersion=""1"" d1p1:Icon=""icon.ico""
 xmlns:d1p1=""http://schemas.microsoft.com/livelabs/pivot/collection/2009"" xmlns=""http://schemas.microsoft.com/collection/metadata/2009"">
 <FacetCategories>
 <FacetCategory Name=""Type"" Type=""String"" d1p1:IsFilterVisible=""true"" d1p1:IsMetaDataVisible=""true"" d1p1:IsWordWheelVisible=""true"" />
 <FacetCategory Name=""Address"" Type=""String"" d1p1:IsFilterVisible=""true"" d1p1:IsMetaDataVisible=""true"" d1p1:IsWordWheelVisible=""true"" />
 <FacetCategory Name=""Size""  Type=""Number"" d1p1:IsFilterVisible=""true"" d1p1:IsMetaDataVisible=""true"" d1p1:IsWordWheelVisible=""true"" />
 <FacetCategory Name=""Namespaces"" Type=""Link"" d1p1:IsFilterVisible=""true"" d1p1:IsMetaDataVisible=""true"" d1p1:IsWordWheelVisible=""true"" />
 <FacetCategory Name=""Members"" Type=""Link"" d1p1:IsFilterVisible=""true"" d1p1:IsMetaDataVisible=""true"" d1p1:IsWordWheelVisible=""true"" />
 </FacetCategories>
 <Items ImgBase=""test.gif"">";

 var endString = @"</Items>
</Collection>";

 var counter = 0;
 var outputFileName = "sl.cxml";
 var dict = new Dictionary<string,string>() { {"String","img5.jpg"},{"Object","img6.jpg"}};
 var getimg = new Func<string,string>((t) => dict.ContainsKey(t) ? dict[t] : "img8.jpg");

 var doc = XDocument.Load(@"c:\Temp\fb.xml");

 var types = (from i in doc.Descendants().Descendants().Descendants("type").AsParallel()
 let ns = i.Attribute("name").Value.SplitEx().ToList()
 select new {Id = int.Parse( i.FirstAttribute.Value),Name = ns.Last(),
 Namespaces = ns.Count()  > 2 ? ns.Take(ns.Count () -2) : ns.Take(ns.Count -1)}).ToList();
 var y = from i in doc.Descendants().Descendants().Descendants("object").AsParallel()
 let type = types.First (t => t.Id== int.Parse( i.Attribute("typeid").Value))
 select new {Address = i.FirstAttribute.Value,
 Type = type.Name,
 Namespaces = type.Namespaces,
 Size = int.Parse( i.LastAttribute.Value),
 children = from k in i.Descendants().AsParallel()
 from o in doc.Descendants().Descendants().Descendants("object").AsParallel()
 where k.FirstAttribute.Value == o.FirstAttribute.Value
 select new {Address = k.FirstAttribute.Value,Type = types.First (t => t.Id == int.Parse( o.Attribute("typeid").Value) ).Name}
 };

 var x = from i in y.Take(2000).AsParallel()
 let increment = new Func<int>(() => Interlocked.Increment(ref counter))
 select new XElement("Item",new XAttribute("Img",getimg(i.Type)),new XAttribute("Id",increment()),new XAttribute("Name",i.Type),
 new XElement("Facets",
 new XElement("Facet",new XAttribute("Name","Type"),
 new XElement("String",new XAttribute("Value",i.Type))),
 new XElement("Facet",new XAttribute("Name","Size"),
 new XElement("Number",new XAttribute("Value",i.Size.ToString()))),
 new XElement("Facet",new XAttribute("Name","Address"),
 new XElement("String",new XAttribute("Value",i.Address))),
 new XElement("Facet",new XAttribute("Name","Namespaces"),
 i.Namespaces.Select (n =>new XElement("Link",new XAttribute("Name",n),new XAttribute("Href",string.Format("{0}#Link=EQ.{1}",outputFileName,n))))),
 i.children.Any() == false ? default(XElement):
 new XElement("Facet",new XAttribute("Name","Members"),
 i.children.Select (n =>new XElement("Link",new XAttribute("Name",n.Type),new XAttribute("Href",string.Format("{0}#Address=EQ.{1}",outputFileName,n.Address)))))
 )).ToString();
 File.WriteAllText(@"c:\temp\sl.cxml",string.Format("{0}{1}{2}",startString,x.Select (s => s).Aggregate(new StringBuilder(), (ag, n) => ag.Append(n)).ToString(),endString));
 Console.WriteLine ("Done");
}

static class Foo
{
 public static IEnumerable<string> SplitEx(this string s)
 {
 if (s.Contains("`") == false)
 return s.Split(new []{'.'});
 else
 {
 var stringList = new List<string>();
 var index =0;
 var counter = 0;
 var splitString = "";
 while (counter < s.Length)
 {
 index = s.IndexOf(".",counter);
 if (index < 0)
 {
 stringList.Add(s.Substring(counter));
 break;
 }
 splitString = s.Substring(counter,index -counter );
 if (splitString.Contains("`"))
 {
 stringList.Add(s.Substring(counter));
 break;
 }
 stringList.Add(splitString);
 counter = index +1;
 }
 return stringList;
 }
 }
}

The above code converts the xml format from !traverseheap to pivot xml format. This is CPU intensive and I have not profiled and fine tuned it. I use plinq for conversion primarily because I have quad-core box. I use pictures from powerpoint for visualizing  objects and I also wrote SplitEx method to avoid splitting namespaces that are part of the type name within generics “GenericEqualityComparer`1[[System.String, mscorlib]]“

And here is the command to get the xml input for the above code as input

!TraverseHeap -xml c:\temp\mem.xml

Remote Debugging with Windbg and Powershell Remoting


Recently I had to debug an issue which was running on Client OS box. I had to do a live debugging more than post-mortem debugging.  FYI the box didn’t have VS.NET and it had copy of debugger’s folder (windbg) along with Powershell v2.0.  I couldn’t  terminal server onto the box because it would kick the other user out and the application would terminate. So only choice was to either go-down actual box and debug it or instruct someone to start a .server process within cdb/windbg. Both the options had its own set of problems. That’s when I realized that I could use powershell remoting to start remote session and then start a remote cdb session. I like to debug from my Dev box because of the tools and settings that I have on my box.  Here is what I did

  1. Started a Powershell ISE environment
  2. Connected to the remote system. (there were quite a few issues is authentication when trying to connect)
  3. Navigated to debuggers folder on the client box cd ‘.\Program Files\Debugging Tools for Windows (x64)’
  4. Then issued the command .\cdb -pn notepad.exe -c “.server npipe:pipe=RemoteDebugging”
  5. I am using a simple notepad.exe for demonstration
  6. Then from my devbox connected to using the command “cdb -remote npipe:Pipe=RemoteDebugging,Server=NAVEEN-PC”

Powershell remoting feels Bash, so one less reason to use remote desktop.

Exploring UnhandledException in .NET and Watson buckets


I wanted to understand about UnhandledExceptions in .NET because of the few questions that I saw in the CLR Forum ,which had watson buckets in the event viewer.  To get deep understanding  unhandled exception filter there is a article on MSDN from CLR Team.

In this post I will be demonstrating how to get the Watson Bucket from within your code . FYI this is the same information  you can get !WatsonBuckets from sos within Windbg whenever there is a termination of the .NET process. !WatsonBuckets is undocumented.  I am using the CLR hosting interfaces to get the watson bucket. Here is the code that throws an unhandled exception and invokes clr to get watson bucket info.

using System;
using System.Runtime.InteropServices;

namespace WatsonTest
{
 internal class Test
 {
 private static void Main(string[] args)
 {
 var t = AppDomain.CurrentDomain;
 t.UnhandledException += (s, e) =>
 {
 var c = GetWatsonBuckets();
 Console.WriteLine(c);
 Console.Read();
 };
 throw new NullReferenceException();
 }
 private static WatsonBuckets GetWatsonBuckets()
 {
 var pParams = new WatsonBuckets();
 IClrRuntimeHost host = null;
 host = Activator.CreateInstance(Type.GetTypeFromCLSID(ClrGuids.ClsIdClrRuntimeHost)) as IClrRuntimeHost;
 if (host != null)
 {
 var clrControl = host.GetCLRControl();
 if (clrControl == null)
 {
 return pParams;
 }
 var clrErrorReportingManager =
 clrControl.GetCLRManager(ref ClrGuids.IClrErrorReportingManager) as IClrErrorReportingManager;
 if (clrErrorReportingManager == null)
 {
 return pParams;
 }
 clrErrorReportingManager.GetBucketParametersForCurrentException(out pParams);
 }
 return pParams;
 }
 }
 // BucketParameters Structure to get watson buckets back from CLR
 //http://msdn.microsoft.com/en-us/library/ms404466(v=VS.100).aspx
 [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Auto)]
 internal struct WatsonBuckets
 {
 internal int fInited;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string pszEventTypeName;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param0;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param1;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param2;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param3;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param4;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param5;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param6;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param7;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param8;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param9;
 }

 internal static class ClrGuids
 {
 internal static readonly Guid ClsIdClrRuntimeHost = new Guid("90F1A06E-7712-4762-86B5-7A5EBA6BDB02");
 internal static Guid IClrErrorReportingManager = new Guid("980D2F1A-BF79-4c08-812A-BB9778928F78");
 internal static readonly Guid IClrRuntimeHost = new Guid("90F1A06C-7712-4762-86B5-7A5EBA6BDB02");
 }

 [Guid("90F1A06C-7712-4762-86B5-7A5EBA6BDB02"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
 internal interface IClrRuntimeHost
 {
 void Start();
 void Stop();
 void SetHostControl(IntPtr pHostControl);
 IClrControl GetCLRControl();
 void UnloadAppDomain(int dwAppDomainId, bool fWaitUntilDone);
 void ExecuteInAppDomain(int dwAppDomainId, IntPtr pCallback, IntPtr cookie);
 int GetCurrentAppDomainId();

 int ExecuteApplication(string pwzAppFullName, int dwManifestPaths, string[] ppwzManifestPaths,
 int dwActivationData, string[] ppwzActivationData);

 int ExecuteInDefaultAppDomain(string pwzAssemblyPath, string pwzTypeName, string pwzMethodName,
 string pwzArgument);
 }

 [Guid("9065597E-D1A1-4fb2-B6BA-7E1FCE230F61"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
 internal interface IClrControl
 {
 [return: MarshalAs(UnmanagedType.IUnknown)]
 object GetCLRManager([In] ref Guid riid);

 void SetAppDomainManagerType(string pwzAppDomainManagerAssembly, string pwzAppDomainManagerType);
 }
 // IClrErrorReportingManager to get watson bukets back from CLR
 //http://msdn.microsoft.com/en-us/library/ms164367(v=VS.100).aspx
 [Guid("980D2F1A-BF79-4c08-812A-BB9778928F78"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
 internal interface IClrErrorReportingManager
 {
 [PreserveSig]
 int GetBucketParametersForCurrentException(out WatsonBuckets pParams);
 }
}

The watson bucket information has ten items

  1. AppName
  2. Version
  3. ?
  4. Assembly and Module Name
  5. Assembly Version
  6. ?
  7. MethodDef
  8. IL Offset
  9. Exception Type
  10. ?

In the above code WatsonBuckets is a structure to get watson bucket information back from CLR.  In the code I use 3 hosting interfaces, IClrRuntimeHost which is the main interface, the next interface is IClrControl which has GetCLRManager method to get ErrorReportingManger. IClrErrorReportingManager which has the method to get the exception buckets which is GetBucketParametersForCurrentException. I know that there isn’t going to be much of use for this code. But it gives me better understanding of  CLR integrates with watson when ever there is termination of an application.

Exploring SOSEX and Windbg to debug .NET 4.0


With the latest release of sosex comes a new set of functions to debug. It is pretty awesome that one person (Steve) alone could pull of such cool things. In this blog post, I am just going to demonstrate how easy it is to debug managed code using sosex compared to sos.

using System;
using System.Collections.Generic;

namespace MemCheck
{
 internal class Test
 {
 Dictionary<int, string> dict = new Dictionary<int, string>();
 private static void Main(string[] args)
 {
 var p = new Test();
 for (int i = 0; i < 100; i++)
 {
 p.dict.Add(i, i.ToString());
 }
 Console.WriteLine("Done");
 Console.Read();
 }
 }
}

I like to keep the code simple , so it is easy to follow.  The debugging goal for today is to get the Dictionary values. First I am going to demonstrate it using sos and then using sosex.

As usual I start the app and then attach it to windbg.

.loadby sos clr

FYI in  .net 4.0 clr is the dll that has CLR implementation. In prior versions it used to be in mscorwks. The next command would look for the object Test in the memory

!dumpheap -type MemCheck.Test
0:000> !dumpheap -type MemCheck.Test
 Address               MT     Size
0000000002761e20 000007ff00054110       24
 total 0 objects
 Statistics:
 MT    Count    TotalSize Class Name
 000007ff00054110        1           24 MemCheck.Test
 Total 1 objects

The next step is to dump the object

0:000> !do 0000000002761e20
Name:        MemCheck.Test
MethodTable: 000007ff00054110
EEClass:     000007ff00162350
Size:        24(0x18) bytes
File:        C:\Users\naveen\Documents\Visual Studio 2010\Projects\Test\bin\Debug\Test.exe
Fields:
 MT    Field   Offset                 Type VT     Attr            Value Name
000007feec2b7a48  4000001        8 ...tring, mscorlib]]  0 instance 0000000002761e38 dict

Notice the dict object is in the 8th offset . To dump contents  dict object I would use the command !do poi(0000000002761e20+8) , which is pointer deference of Test object on it is 8th offset.  And here is the output

0:000> !do poi(0000000002761e20+8)
Name:        System.Collections.Generic.Dictionary`2[[System.Int32, mscorlib],[System.String, mscorlib]]
MethodTable: 000007feec2b7a48
EEClass:     000007feebe113c0
Size:        88(0x58) bytes
File:        C:\Windows\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll
Fields:
 MT    Field   Offset                 Type VT     Attr            Value Name
000007feec27c7d8  4000bee        8       System.Int32[]  0 instance 0000000002764788 buckets
000007feecbd3dc8  4000bef       10 ...non, mscorlib]][]  0 instance 0000000002764ab8 entries
000007feec27c848  4000bf0       40         System.Int32  1 instance              100 count
000007feec27c848  4000bf1       44         System.Int32  1 instance              100 version
000007feec27c848  4000bf2       48         System.Int32  1 instance               -1 freeList
000007feec27c848  4000bf3       4c         System.Int32  1 instance                0 freeCount
000007feec2a5a48  4000bf4       18 ...Int32, mscorlib]]  0 instance 0000000002761ef0 comparer
000007feecc75f78  4000bf5       20 ...Canon, mscorlib]]  0 instance 0000000000000000 keys
000007feecc72078  4000bf6       28 ...Canon, mscorlib]]  0 instance 0000000000000000 values
000007feec275ab8  4000bf7       30        System.Object  0 instance 0000000000000000 _syncRoot
000007feec29a1b8  4000bf8       38 ...SerializationInfo  0 instance 0000000000000000 m_siInfo

And the dictionary object in turn stores them within an array which is again the 8th offset.  This time because we know it is an array we are going to use the !dumparray command on the memory location. The command to get the details is

!dumparray -details poi(poi(0000000002761e20+8)+8)
MT    Field   Offset                 Type VT     Attr            Value Name
 000007feec27c848  400047b        0             System.Int32      1     instance                   -1     m_value
 [195] 0000000002764aa4
 Name:        System.Int32
 MethodTable: 000007feec27c848
 EEClass:     000007feebe00890
 Size:        24(0x18) bytes
 File:        C:\Windows\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll
 Fields:
 MT    Field   Offset                 Type VT     Attr            Value Name
 000007feec27c848  400047b        0             System.Int32      1     instance                   -1     m_value
 [196] 0000000002764aa8
 Name:        System.Int32
 MethodTable: 000007feec27c848
 EEClass:     000007feebe00890
 Size:        24(0x18) bytes
 File:        C:\Windows\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll
 Fields:
 MT    Field   Offset                 Type VT     Attr            Value Name
 000007feec27c848  400047b        0             System.Int32      1     instance                   -1     m_value

Here is the partial output.

Now lets try and do the same thing using sosex. The one thing that I really like about  the new sosex is that  I can use names rather than pointer deference which is way much easier. Launched the app and then loaded sosex using the command

.load F:\Work\Tools\debuggers\sosex.dll

and then switched the thread from 4th to 0th thread using ~0s.  By default the debugger injects a thread into the process for debugging and that was the 4th thread. The next command I issued was to get stack trace

!mk
0:000> !mk
Thread 0:
 ESP              EIP
00:U 000000000015e408 0000000077bc00da ntdll!ZwRequestWaitReplyPort+0xa
01:U 000000000015e410 0000000077a72b08 KERNEL32!ConsoleClientCallServer+0x54
02:U 000000000015e440 0000000077aa5601 KERNEL32!ReadConsoleInternal+0x1f1
03:U 000000000015e590 0000000077aba922 KERNEL32!ReadConsoleA+0xb2
04:U 000000000015e670 0000000077a89934 KERNEL32!zzz_AsmCodeRange_End+0x8bea
05:U 000000000015e6b0 000007feed0317c7 clr!DoNDirectCall__PatchGetThreadCall+0x7b
06:M 000000000015e760 000007feec1d34a1 DomainNeutralILStubClass.IL_STUB_PInvoke(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte*, Int32, Int32 ByRef, IntPtr)(+0x0 IL)(+0x0 Native)
07:M 000000000015e880 000007feec97f59a System.IO.__ConsoleStream.ReadFileNative(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte[], Int32, Int32, Int32, Int32 ByRef)(+0x53 IL)(+0xba Native)
08:M 000000000015e8f0 000007feec97f402 System.IO.__ConsoleStream.Read(Byte[], Int32, Int32)(+0x5d IL)(+0x62 Native)
09:M 000000000015e950 000007feec18e63c System.IO.StreamReader.ReadBuffer()(+0xa0 IL)(+0x5c Native)
0a:M 000000000015e9a0 000007feec915630 System.IO.StreamReader.Read()(+0x21 IL)(+0x30 Native)
0b:M 000000000015e9e0 000007feec987458 System.IO.TextReader+SyncTextReader.Read()(+0x0 IL)(+0x38 Native)
0c:M 000000000015ea30 000007ff00170213 MemCheck.Test.Main(System.String[])(+0x39 IL)(+0xf3 Native) [C:\Users\naveen\Documents\Visual Studio 2010\Projects\Test\Program.cs, @ 17,13]
0d:U 000000000015eaa0 000007feed0710b4 clr!CallDescrWorker+0x84
0e:U 000000000015eaf0 000007feed0711c9 clr!CallDescrWorkerWithHandler+0xa9
0f:U 000000000015eb70 000007feed071245 clr!MethodDesc::CallDescr+0x2a1
10:U 000000000015eda0 000007feed171675 clr!ClassLoader::RunMain+0x228
11:U 000000000015eff0 000007feed1717ac clr!Assembly::ExecuteMainMethod+0xac
12:U 000000000015f2a0 000007feed171562 clr!SystemDomain::ExecuteMainMethod+0x452
13:U 000000000015f850 000007feed173dd6 clr!ExecuteEXE+0x43
14:U 000000000015f8b0 000007feed173cf3 clr!CorExeMainInternal+0xc4
15:U 000000000015f920 000007feed1f7365 clr!CorExeMain+0x15
16:U 000000000015f960 000007fef8f13309 mscoreei!CorExeMain+0x41
17:U 000000000015f990 000007fef8fa5b21 MSCOREE!CorExeMain_Exported+0x57
18:U 000000000015f9c0 0000000077a6f56d KERNEL32!BaseThreadInitThunk+0xd
19:U 000000000015f9f0 0000000077ba3281 ntdll!RtlUserThreadStart+0x1d

FYI the command !mk has been part of sos from the initial version. I am interested in only looking at the code that I wrote so I would like to move stack frame to 0c which is MemCheck.Test.Main . To do that the command  is !mframe 0c, which moves to that stackframe. The reason to move the particular stack frame is to look for variables in the stack and the command to variables is !mdv , which display managed local variables

0:000> !mdv
Frame 0xc: (MemCheck.Test.Main(System.String[])):
[A0]:args:0x0000000002761dd8 (System.String[])
[L0]:p:0x0000000002761e20 (MemCheck.Test)
[L1]:i:0x0000000000000064 (System.Int32)
[L2]:CS$4$0000:0x0000000000000000 (System.Boolean)

Notice we see the local variable “p” which is of type MemCheck.Test. To display type p we issue the command !mdt p


0:000> !mdt p

0000000002761e20 (MemCheck.Test)

dict:0000000002761e38 (System.Collections.Generic.Dictionary`2[[System.Int32, mscorlib],[System.String, mscorlib]])

I didn’t  have to get memory address , I am using the names which is very intuitive ,especially when we have to debug large application with N levels of nesting.So to get the dict values   from p the command to issue is !mdt -e p.dict

!mdt -e p.dict
[98] (System.Collections.Generic.Dictionary`2+Entry[[System.Int32, mscorlib],[System.String, mscorlib]]) VALTYPE (MT=000007feec2b7b28, ADDR=0000000002765400)
 key:0x62 (System.Int32)
 value:0000000002765e48 (System.String: "98")
[99] (System.Collections.Generic.Dictionary`2+Entry[[System.Int32, mscorlib],[System.String, mscorlib]]) VALTYPE (MT=000007feec2b7b28, ADDR=0000000002765418)
 key:0x63 (System.Int32)
 value:0000000002765e68 (System.String: "99")

Here is the partial output. Notice I never had to use a memory pointer or do a pointer deference .This is very similar to VS.NET debugging where I am used to the variable names compared the memory address.  Thanks to Steve for providing such a cool extension.

Undocumented PSSCOR2 functions


With the latest release of PSSCOR2 release from Microsoft comes with a set Undocumented functions. FYI PSSCOR2 is a superset of sos.dll for .NET framework 2.0/3.5, it also has bunch of functions which are available in the .NET 4.0 sos.dll.

The .NET 4.0 also has few undocumented functions and PSSCOR2 does have them. But these functions are totally new and is not even available in .NET 4.0 sos

  1. !FindInterestingStrings
  2. !SaveSnapshot
  3. !LoadSnapshot
  4. !dumpconfig
  5. !GCUsage
  6. !VerifyDAC

I did manage to figure out the usage of few of these, but I let you explore them before I blog about this.

Case Study: Tracking .NET Exceptions with Event Tracing for Windows (ETW)


In the past I have debugged customers code ,where the code throws tons of exceptions. This is a huge performance problem. Tess has amazing post on why throwing ton of exceptions are bad. To debug this I had to resort to using Windbg and  getting call stacks of exceptions. The biggest issue was ,there were thousands of exceptions thrown from different points which made debugging extremely hard. I couldn’t just break-point based on exception type because the same type of exception were thrown from n different points. Windbg would  peg cpu for getting call stacks for each of these exceptions. I so wish I had ETW when I had to figure out those exceptions. FYI all these exception were handled and that made even harder.

Now with CLR 4.0 having ETW, this is so much easy to diagnose the same problem. Here is the sample code that I am going to use.

using System;
using System.Threading;
class Program {
 private void ProcessArgs() {
 for (int i = 0; i < 20; i++) {
 try {
 throw new ArgumentNullException(i.ToString());
 }
 catch (Exception e) {
 Console.WriteLine(e);
 }
 }
 }
 private static void Main(string[] args) {
 var p = new Program();
 ThreadPool.QueueUserWorkItem((x) => p.ProcessArgs());
 ThreadPool.QueueUserWorkItem((x) => p.ThrowNullReference());
 Console.Read();
 }
 private void ThrowNullReference() {
 for (int i = 0; i < 20; i++) {
 try {
 throw new NullReferenceException(i.ToString());
 }
 catch (Exception ex) {
 Console.WriteLine(ex);
 }
 }
 }
}

Here is the code to trace CLR exceptions

xperf -start clr -on e13c0d23-ccbc-4e12-931b-d9cc2eee27e4:0x00008000:5 -f clrevents.etl

After which I started the console application. Then stopped the etw and then dumped the contents to a csv file.

xperf -stop clr
xperf -i clrevents.etl -o clrexceptions.csv

Here is a sample exception trace from ETW

Microsoft-Windows-DotNETRuntime/Exception /Start ,    3856446,        "Unknown" (27776),      30268,   0, , , , , "System.ArgumentNullException", "Value cannot be null.", 0x00480270, 0x80004003, 16, 15

Attached the application to Windbg. With in the etw trace is the Instruction Pointer where the exception was raised . And for above example it was raised at 0x00480270.

Issued a command to disassemble the sourcecode at the specific instruction pointer within Windbg

!u  0x00480270 

And here is the output from the above command

0:007> !u  0x00480270
Normal JIT generated code
Program.ProcessArgs()
Begin 00480200, size ad

C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication1\Program.cs @ 4:
00480200 55              push    ebp
00480201 8bec            mov     ebp,esp
00480203 57              push    edi
00480204 56              push    esi
00480205 53              push    ebx
00480206 83ec30          sub     esp,30h
00480209 8bf1            mov     esi,ecx
0048020b 8d7dd8          lea     edi,[ebp-28h]
0048020e b906000000      mov     ecx,6
00480213 33c0            xor     eax,eax
00480215 f3ab            rep stos dword ptr es:[edi]
00480217 8bce            mov     ecx,esi
00480219 33c0            xor     eax,eax
0048021b 8945e8          mov     dword ptr [ebp-18h],eax
0048021e 894ddc          mov     dword ptr [ebp-24h],ecx
00480221 833d3c31310000  cmp     dword ptr ds:[31313Ch],0
00480228 7405            je      0048022f
0048022a e8464be768      call    clr!JIT_DbgIsJustMyCode (692f4d75)
0048022f 33d2            xor     edx,edx
00480231 8955d0          mov     dword ptr [ebp-30h],edx
00480234 c745d400000000  mov     dword ptr [ebp-2Ch],0
0048023b 90              nop

C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication1\Program.cs @ 5:
0048023c 33d2            xor     edx,edx
0048023e 8955d8          mov     dword ptr [ebp-28h],edx
00480241 90              nop
00480242 eb4d            jmp     00480291
00480244 90              nop

C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication1\Program.cs @ 6:
00480245 90              nop

C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication1\Program.cs @ 7:
00480246 8d4dd8          lea     ecx,[ebp-28h]
00480249 e882330568      call    mscorlib_ni+0x2635d0 (684d35d0) (System.Int32.ToString(), mdToken: 06000cd4)
0048024e 8945cc          mov     dword ptr [ebp-34h],eax
00480251 b98c475968      mov     ecx,offset mscorlib_ni+0x32478c (6859478c) (MT: System.ArgumentNullException)
00480256 e8c51de8ff      call    00302020 (JitHelp: CORINFO_HELP_NEWSFAST)
0048025b 8945c8          mov     dword ptr [ebp-38h],eax
0048025e 8b55cc          mov     edx,dword ptr [ebp-34h]
00480261 8b4dc8          mov     ecx,dword ptr [ebp-38h]
00480264 e8b7bbfc67      call    mscorlib_ni+0x1dbe20 (6844be20) (System.ArgumentNullException..ctor(System.String), mdToken: 06000795)
00480269 8b4dc8          mov     ecx,dword ptr [ebp-38h]
0048026c e835ffd668      call    clr!IL_Throw (691f01a6)

C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication1\Program.cs @ 9:
00480271 8945c4          mov     dword ptr [ebp-3Ch],eax
00480274 8b45c4          mov     eax,dword ptr [ebp-3Ch]
00480277 8945d0          mov     dword ptr [ebp-30h],eax
0048027a 90              nop

C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication1\Program.cs @ 10:
0048027b 8b4dd0          mov     ecx,dword ptr [ebp-30h]
0048027e e841416268      call    mscorlib_ni+0x8343c4 (68aa43c4) (System.Console.WriteLine(System.Object), mdToken: 06000918)
00480283 90              nop

C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication1\Program.cs @ 11:
00480284 90              nop
00480285 e8ae1fbc68      call    clr!JIT_EndCatch (69042238)
0048028a eb00            jmp     0048028c
0048028c 90              nop

C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication1\Program.cs @ 12:
0048028d 90              nop

C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication1\Program.cs @ 5:
0048028e ff45d8          inc     dword ptr [ebp-28h]
00480291 837dd814        cmp     dword ptr [ebp-28h],14h
00480295 0f9cc0          setl    al
00480298 0fb6c0          movzx   eax,al
0048029b 8945d4          mov     dword ptr [ebp-2Ch],eax
0048029e 837dd400        cmp     dword ptr [ebp-2Ch],0
004802a2 75a0            jne     00480244

C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication1\Program.cs @ 13:
004802a4 90              nop
004802a5 8d65f4          lea     esp,[ebp-0Ch]
004802a8 5b              pop     ebx
004802a9 5e              pop     esi
004802aa 5f              pop     edi
004802ab 5d              pop     ebp
004802ac c3              ret

Now we see the call stacks of where the exception was raised, even without hooking for exceptions within the debugger.  The key reason for doing this is, for this specific case study I could have got all the unique Instruction pointers from the trace and just take one memory dump.With this I could have managed to get the call stacks all the exceptions. This would be non-invasive, which would save us lot of time and effort.

With ETW we can get all the exception even the ones that were handled, along with Instruction Pointer which helps us trace the root cause of the issue.

With this in hand ,we could easily write a small automation tool to extract the Instruction Pointer from the trace and then disassemble the source code and get the call stack.

All of this can be done on .NET 4.0 (clr.dll) and Silverlight (coreclr.dll).

In my forth coming posts ,I will share few more cool things that can be done with ETW.

Piracy in .NET /Silverlight Code – Part 1 – Even when the code is obfuscated


This is going to be a series of posts where I am going to demonstrating how someone with little advanced knowledge in .NET can hack in to the code, and circumventing licensing logic. I know there are other ways to prevent this ,which I am not going to be discussing about.

The usual assumption is that, if the code is obfuscated and signed , then it is close to impossible for someone to hack in to the code because the method names and variables are jumbled up, which would prevent someone from figuring what is happening.

In this post I am going to demonstrate a simple application which would prevent the user from updating the DataGrid because the user had downloaded only a trial version of the software. The software was supposed to disable grid and prevent the consumer from adding or updating the existing data, in a trial version. This was customer case, whom I helped in figuring out the vulnerability.

Here is the list of things the software did to prevent users from not accessing features that they were not entitled to

  1. Obfuscated the code, that prevented the code from being disassembled.
  2. The code that validated the consumers role either paid / trial alone was maintained in a separate assembly .So that for trial consumer’s, assembly version always returned false, for key features and for the paid subscribers got the assembly version that would return true. The organization assumption was ,by not even having the assembly ,trial users would never be able to circumvent the licensing logic. The customer could not maintain two versions of entire software, one for the trial another one paid because of the cost involved in maintenance and that’s the driving factor behind having a separate assembly.
  3. The code was signed and this prevented the users from hacking into the code and changing the code.

So here is code for figuring out the user role

public interface ICustomer{
bool HasAccess();}

And the trial assembly version code for figuring the role had

public class TrialCustomer : ICustomer {
public bool HasAccess() {
return false;
}
}

And here is the code that actually use the above code

private void Button1Click(object sender, EventArgs e)
{
if (!this.customer.HasAccess()) {
MessageBox.Show("Available only for paid customer");
}
else {
this.dataGridView1.Enabled = true;
}
}

So the idea behind this was, the trial version user would always get false as response, and would never be able to hack, because consumer does not even have library that would return true.

So I was asked to check for vulnerabilities. The first step was to disassemble the code using reflector and you can imagine the code had all weird names because of obfuscation.

The next step was to run the application and check when I get a dialog box for “Available only for paid customer”. The reason behind this was to get a callstack ,because all the methods names are jumbled and I didn’t know where to start.

When the messagebox popped up I attached the application to windbg and here is callstack

0:000> !clrstack

OS Thread Id: 0x4850 (0)

Child SP IP       Call Site

004ce758 752b438d [InlinedCallFrame: 004ce758]

004ce754 636308ec DomainBoundILStubClass.IL_STUB_PInvoke(System.Runtime.InteropServices.HandleRef, System.String, System.String, Int32)

004ce758 636f4a53 [InlinedCallFrame: 004ce758] System.Windows.Forms.SafeNativeMethods.MessageBox(System.Runtime.InteropServices.HandleRef, System.String, System.String, Int32)

004ce7ac 636f4a53 System.Windows.Forms.MessageBox.ShowCore(System.Windows.Forms.IWin32Window, System.String, System.String, System.Windows.Forms.MessageBoxButtons, System.Windows.Forms.MessageBoxIcon, System.Windows.Forms.MessageBoxDefaultButton, System.Windows.Forms.MessageBoxOptions, Boolean)

004ce7b0 001b0513 [InlinedCallFrame: 004ce7b0]

004ce84c 001b0513 c.a(System.Object, System.EventArgs)

004ce85c 630bfd6c System.Windows.Forms.Control.OnClick(System.EventArgs)

004ce874 630beb1e System.Windows.Forms.Button.OnClick(System.EventArgs)

004ce88c 636574b8 System.Windows.Forms.Button.OnMouseUp(System.Windows.Forms.MouseEventArgs)

004ce8a8 63629639 System.Windows.Forms.Control.WmMouseUp(System.Windows.Forms.Message ByRef, System.Windows.Forms.MouseButtons, Int32)

004ce93c 639d1a87 System.Windows.Forms.Control.WndProc(System.Windows.Forms.Message ByRef)

004ce940 639f141d [InlinedCallFrame: 004ce940]

004ce994 639f141d System.Windows.Forms.ButtonBase.WndProc(System.Windows.Forms.Message ByRef)

004ce9d8 6312f8e0 System.Windows.Forms.Button.WndProc(System.Windows.Forms.Message ByRef)

004ce9e4 630ce493 System.Windows.Forms.Control+ControlNativeWindow.OnMessage(System.Windows.Forms.Message ByRef)

004ce9ec 630ce411 System.Windows.Forms.Control+ControlNativeWindow.WndProc(System.Windows.Forms.Message ByRef)

004cea00 630ce356 System.Windows.Forms.NativeWindow.Callback(IntPtr, Int32, IntPtr, IntPtr)

004ceba4 007a09e5 [InlinedCallFrame: 004ceba4]

004ceba0 631347dc DomainBoundILStubClass.IL_STUB_PInvoke(MSG ByRef)

004ceba4 630de59f [InlinedCallFrame: 004ceba4] System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(MSG ByRef)

004cebe8 630de59f System.Windows.Forms.Application+ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop(IntPtr, Int32, Int32)

004cebec 630de1cc [InlinedCallFrame: 004cebec]

004cec84 630de1cc System.Windows.Forms.Application+ThreadContext.RunMessageLoopInner(Int32, System.Windows.Forms.ApplicationContext)

004cecdc 630de021 System.Windows.Forms.Application+ThreadContext.RunMessageLoop(Int32, System.Windows.Forms.ApplicationContext)

004ced0c 630c5ecd System.Windows.Forms.Application.Run(System.Windows.Forms.Form)

004ced20 001b009a e.a()

004cef58 6d1d213b [GCFrame: 004cef58]

Notice that there is a method c.a(System.Object, System.EventArgs) on the top of stack the before the framework code. This was my starting point, and I looked up for it in the reflector

private void a(object A_0, EventArgs A_1)
{
if (!this.d.a())
{
MessageBox.Show("Available only for paid customer");
}
else
{
this.b.Enabled = true;
}
}

So my next step was to disassemble the call to “this.d.a()” and here is output from reflector

public class d : b
{
// Methods
public bool a()
{
return false;
}
}

And I was hoping there would be a class variable that I could update inside the debugger to activate the feature.  Like I mentioned I before ,I couldn’t update the assembly using ILASM or anything, because the assemblies were signed and there wasn’t any class variable to update. This assembly was for the trial version consumers which were supposed to always return false.

But there was one trick that I had. I could update the register on the function return, So what I mean is, when the function “a” is invoked by the button click to validate the consumer , I would update the return register  from false to true. By doing this my grid would be enabled and I circumvent the logic with the existing constraints.

So here are the steps to do it

  • So I looked for the type “d”  !dumpheap -type d, remember the TrialCustomer class  was the renamed  to d by obfuscator .
  • Go the method table as 00146b9c from the above command , using  the method table  I had to get the entry address for the function public bool a() because the buttonclick  was invoking the function.
  • I used the command  !dumpmt -md 00146b9c   to get the entry address for   d.a()  and here was the entry address 001b0540  .

0:005> !dumpmt -md 00146b9c

EEClass:      0020054c

Module:       00142e9c

Name:         d

mdToken:      02000009

File:         C:\Users\naveen\Documents\Visual Studio 2010\Projects\SecureApplication\bin\Debug\Dotfuscated\SecureApplication.exe

BaseSize:        0xc

ComponentSize:   0x0

Slots in VTable: 6

Number of IFaces in IFaceMap: 1

————————————–

MethodDesc Table

Entry MethodDesc      JIT Name

649f5b34   64795750   PreJIT System.Object.ToString()

649c8be0   64795758   PreJIT System.Object.Equals(System.Object)

649c8af0   64795778   PreJIT System.Object.GetHashCode()

649e8aa0   6479578c   PreJIT System.Object.Finalize()

001b0540   00146b8c      JIT d.a()

0014c085   00146b94     NONE d..ctor()

  • The idea behind getting the address was to set a break-point on 001b0540  . I used the command bp 001b0540 . So when the break-point hits I would create another break-point for the function return, which  is stored in the register @esp  bp poi(@esp). So what bp poi(@esp) essentially means is,   create a break-point on the return of function  , it is almost like having a break-point on the last line inside a function within VS.NET.
  • So when the break-point hits for the function return ,the return value is stored in the @eax register and here is the output

0:000> g

Breakpoint 1 hit

eax=00000000 ebx=0230e6f8 ecx=0230cfcc edx=02328884 esi=0230ce7c edi=02328884

eip=001b04f2 esp=004ce8a0 ebp=004ce8a4 iopl=0         nv up ei pl zr na pe nc

cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246

001b04f2 85c0            test    eax,eax

Now the trick is to update the @eax register from 00000000 to 00000001, by doing this we are changing the value from false to true, using the command  “r eax=00000001” and here is the output after updating the register

0:000> r

eax=00000001 ebx=0230e6f8 ecx=0230cfcc edx=02328884 esi=0230ce7c edi=02328884

eip=001b04f2 esp=004ce8a0 ebp=004ce8a4 iopl=0         nv up ei pl zr na pe nc

cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246

001b04f2 85c0            test    eax,eax

Now we have managed to update return of the function, which in turn has let us update the grid ,which was supposed to be available only to the paid customers.

In the forth coming posts I will continue to post of few more things that I figured out.

Debugging .Net framework source code within Windbg


One the coolest thing Microsoft did was to release the .NET Framework source code. In this post, I am going to demonstrate, how we could have a break-point on the .NET framework source code by line numbers ,using Windbg ,very similar to doing in  VS.NET.

The first step towards doing this is to download .NET Framework Source Code and installing it on the local machine. The next step is to set the symbol path environment variable. My _NT_SYMBOL_PATH is set to


SRV*d:\dev\symbols*http://referencesource.microsoft.com/symbols; SRV*d:\dev\symbols*http://msdl.microsoft.com/download/symbols

Setting the correct symbol path is important to download symbols from MS.

Here is the source code that I would be using to demonstrate this


using System;
using System.Net;
namespace Test {
 class Program {
 static void Main(string[] args) {
 Console.WriteLine("Hello World of debugging");
 var wr = WebRequest.Create("http://www.google.com");
 Console.WriteLine("Web request created");
 var req = wr.GetRequestStream();
 Console.WriteLine("Hello World Debugging");
 Console.Read();
 }
 }
}

I am going to demonstrate the same thing using multiple versions of debugger. The first one that I am going to demonstrate is using  Windows Debugger Version 6.12.0002.633 X86 which is the latest version.

Launched the exe within windbg and opened the source code WebRequest.cs and Program.cs withing Windbg.

The WebRequest.cs is the source code that was downloaded from MS Reference Source code and Program.cs is the above sample code.

Then issued the command, to be notified when mscorlib is loaded

sxe ld:mscorlib

And here is the output from the above command

ModLoad: 53fd0000 54ac8000   C:\Windows\assembly\NativeImages_v2.0.50727_32\mscorlib\8c1770d45c63cf5c462eeb945ef9aa5d\mscorlib.ni.dll
eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=7efdd000 edi=0040eaf4
eip=7723fc02 esp=0040e9c8 ebp=0040ea1c iopl=0         nv up ei pl nz na po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202
ntdll!ZwMapViewOfSection+0x12:
7723fc02 83c404          add     esp,4

then issued the following commands to load by sos, sosex and set break-points

.loadby sos mscorwks
.load sosex
!mbm System.Net.WebRequest.Create
!mbp WebRequest.cs 98

So with the above command I am requesting for a break-point on the method  System.Net.WebRequest.Create using symbol (!mbm). I am issuing the command !mbm ,just so that sosex can hook up with CLR for getting notifications on JIT. Without this I was unable to set break-point on source code using line numbers. The next command !mbp WebRequest.cs 98 means, have a break-point on the line number 98  in the WebRequest.cs file . The line 98 contents are “if (!useUriBase)”  .  I forgot to mention the .NET framework 4.0 source code is not released so I am using 3.5

Also make sure that the correct private pdb symbols are loaded for framework assemblies, and to verify that, issue the command “lme” and the output should contain something like this

0:000> lme
start    end        module name
013b0000 013b8000   ConsoleApplication1 C (private pdb symbols)  C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication1\bin\Debug\ConsoleApplication1.pdb
53fd0000 54ac8000   mscorlib_ni C (private pdb symbols)  d:\dev\symbols\mscorlib.pdb\F85F3DD0C7024D528B4C37F1ACF2123D1\mscorlib.pdb
58930000 590c9000   System_ni C (private pdb symbols)  d:\dev\symbols\System.pdb\97A082CB5BC64B30887253632D3901EE1\System.pdb

And here is the output after letting it run

Breakpoint 1 hit
eax=00000001 ebx=0040f37c ecx=0282c160 edx=00000000 esi=008ba398 edi=0282c160
eip=58dec8b7 esp=0040f32c ebp=0040f334 iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
System_ni+0x4bc8b7:
58dec8b7 85ff

then issued the commands to see the list of break-points (managed and native)


!mbl

bl

and here is the output

0:000> !mbl
0 e : *!SYSTEM.NET.WEBREQUEST.CREATE ILOffset=0: pass=1 oneshot=false thread=ANY
System!System.Net.WebRequest.Create(Uri, bool)+0xfffffffe(IL)
0 e 58a71630
System!System.Net.WebRequest.Create(string)
1 e 58dec8b7
System!System.Net.WebRequest.Create(Uri)+0xfffffffe(IL)
2 e 58a71600
1 eu: WebRequest.cs, line 98: pass=1 oneshot=false thread=ANY
0:000> bl
0 e 58a71630     0001 (0001)  0:**** System_ni+0x141630
1 e 58dec8b7     0001 (0001)  0:**** System_ni+0x4bc8b7
2 e 58a71600     0001 (0001)  0:**** System_ni+0x141600

because Webrequest.Create has two overloads, sosex has set a break-point on both these methods and that’s the reason we are seeing 3 break-points, instead of 2. And after issuing the “g” command for couple of hits for the Webrequest.Create , then comes the Breakpoint 3 hit. Voila ,we have the managed to set break-point on framework source code using line numbers and here is the output of  !mk command

Breakpoint 3 hit
eax=00000000 ebx=0014ecdc ecx=0282dde4 edx=00000000 esi=00000000 edi=00000000
eip=58a71650 esp=0014ec68 ebp=0014ec84 iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
System_ni+0x141650:
58a71650 8bc7            mov     eax,edi
0:000> !mk
ESP      RetAddr
00:M 0014ec68 58a71650 System.Net.WebRequest.Create(System.Uri, Boolean)(+0x23 IL)(+0x0 Native) [f:\dd\ndp\fx\src\Net\System\Net\WebRequest.cs, @ 96,13]
01:M 0014ec8c 58dec8dd System.Net.WebRequest.Create(System.String)(+0x14 IL)(+0x26 Native)
02:M 0014ec9c 005000a9 Test.Program.Main(System.String[])(+0xc IL)(+0x18 Native) [C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication1\Program.cs, @ 7,13]
03:U 0014ecb8 6cfb1b6c mscorwks!CallDescrWorker+0x33
04:U 0014ecc8 6cfc2209 mscorwks!CallDescrWorkerWithHandler+0xa3
05:U 0014ed48 6cfd6511 mscorwks!MethodDesc::CallDescr+0x19c
06:U 0014ee8c 6cfd6544 mscorwks!MethodDesc::CallTargetWorker+0x1f
07:U 0014eea8 6cfd6562 mscorwks!MethodDescCallSite::CallWithValueTypes_RetArgSlot+0x1a
08:U 0014eec0 6d040c45 mscorwks!ClassLoader::RunMain+0x223
09:U 0014f024 6d040b65 mscorwks!Assembly::ExecuteMainMethod+0xa6
0a:U 0014f28c 6d0410b5 mscorwks!SystemDomain::ExecuteMainMethod+0x456
0b:U 0014f75c 6d04129f mscorwks!ExecuteEXE+0x59
0c:U 0014f7ac 6d0411cf mscorwks!_CorExeMain+0x15c
0d:U 0014f7f4 73a461f0 mscoreei!_CorExeMain+0x38
0e:U 0014f800 74337f16 MSCOREE!ShellShim__CorExeMain+0x99
0f:U 0014f810 74334de3 MSCOREE!_CorExeMain_Exported+0x8
10:U 0014f818 74f73677 KERNEL32!BaseThreadInitThunk+0xe
11:U 0014f824 77259d72 ntdll!__RtlUserThreadStart+0x70
12:U 0014f864 77259d45 ntdll!_RtlUserThreadStart+0x1b

Notice on the top frame of the stack (00:M) we see source information. The advantage of this is, for example I can check the values of the local variable with in the function after certain calls ,which wouldn’t  have been possible without jumping hoops. Here is the the output !mdv after the line breakpoint

0:000> !mdv
Frame 0x0: (System.Net.WebRequest.Create(System.Uri, Boolean)):
[A0]:requestUri:0x282d2fc (System.Uri)
[A1]:useUriBase:0x0 (System.Boolean)
[L0]:LookupUri:<?>
[L1]:Current:null (System.Net.WebRequestPrefixElement)
[L2]:Found:0x0 (System.Boolean)
[L3]:LookupLength:<?>
[L4]:prefixList:<?>
[L5]:i:<?>
[L6]:webRequest:<?>

And my next quest was to figure What if I could use the actual break-point on the source code directly instead of using sosex.  So chose program.cs and hit the F9 key on the second Console.WriteLine and the color changed to red

And here is the output o f the bl command

0:000> bl
0 e 58a71630     0001 (0001)  0:**** System_ni+0x141630
1 e 58dec8b7     0001 (0001)  0:**** System_ni+0x4bc8b7
2 e 58a71600     0001 (0001)  0:**** System_ni+0x141600
3 e 58a71650     0001 (0001)  0:**** System_ni+0x141650
4 e 013b0017     0001 (0001)  0:**** ConsoleApplication1!Main+0x17

Notice there is a new break-point 4.  But bad luck, when i let it run it didn’t work and I had to clear the break-point 4 . It failed to create a break-point and here is the output

0:000> g
Unable to insert breakpoint 4 at 013b0017, Win32 error 0n998
“Invalid access to memory location.”
bp4 at 013b0017 failed
WaitForEvent failed
eax=00000000 ebx=0014ecdc ecx=0282dde4 edx=00000000 esi=00000000 edi=00000000
eip=58a71650 esp=0014ec68 ebp=0014ec84 iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000346
System_ni+0x141650:
58a71650 8bc7            mov     eax,edi

The next step was to debug the with the version of debugger which had capabilities to see managed call stacks, Windbg version 6.7.5.0. Did the same thing loaded up the exe and the Program.cs in to the debugger and set a break-point on mscorlib load using  ” sxe ld:mscorlib” and then issued the command “.loadby sos mscorwks” ,”.load sosex” and “!mbm System.Net.WebreRequest.Create”. And the let it run, and to my surprise here is the integrated debugging experience, the source code is highlighted, when break-point is hit for WebRequest.Create

And after hitting F10 ,it moved to the next line

Wow this is way cool. The next step was to set a break-point using the bp command on the Program.cs


bp (@@masm(`Program.cs:8+`))

And  the break-point was hit and I was successfully able to have Integrated debugging environment. I know most of us don’t  have Windbg version 6.7.5.0 , but there are few debugging geeks who still have them. I was happy that I kept the version.

Take Away

We should be able to debug within the framework source code using the latest version of debugger and if you have 6.7.5.0 , can have integrated debugging experience similar to VS.NET .

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: