Naveen's Blog

I was recently showing someone the new ETW features in .NET especially the GC Event notification and I was asked if we can get this using Windbg.

So here is the sample code for the GC Collection

namespace GCStartStop
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
button1.Click += (s, b) => GC.Collect(2);
button1.Click += (s, b) => GC.Collect(1);
}
}
}

The goal is set to a break-point only when the collection count is 2. Here is a bp script for doing this.

bp clr!WKS::GCHeap::SuspendEE ".if (dwo(clr!WKS::GCHeap::GcCondemnedGeneration)==2) {.echo start of gen 2;g} .else {gc}"

The same thing can be done for clr!WKS::GCHeap::RestartEE.

When showing this to someone I was asked what does “EE” acronym in “SuspendedEE” ?  “EE” is  Execution Engine.

I had earlier written a post on getting GC information on Silverlight using ETW. With that we would have to write code to parse the ETW csv file.  In this post I am going to be using Perfmonitor to do this. This tools uses the same ETW under covers, but it does all the plumbing and gives a nice report , which is much easier to read.  Here are the reports

To demonstrate this I used the bing’s world leader search page and here is the url

http://www.bing.com/visualsearch?q=World+leaders&g=world_leaders&FORM=Z9GE74#

Steps to get the GC information are

1. Start a cmd or powershell  as admin , this required to collect ETW tracing
2. Browse the above mentioned webpage using IE
3. Issue the command “PerfMonitor.exe /process:4180 start” where 4180 is the internet explorer’s process id
4. Do the necessary actions
5. Then issue “PerfMonitor.exe stop”
6. The command to get the report “PerfMonitor.exe GCTime”. This will generate a report and open it in the browser

Perfmonitor is like xperf for managed code. This is non-intrusive and can collect some valuable information in production. This is an xcopy tool and does not need an install.

I am sure most of us have debugged applications that are build with debug turned on, which is obviously much easier compared to debugging release build (optimized code). In this post I am going to share one of my experiences of debugging release build code. I will demonstrate this with a simple Console Application.

Here is the code

using System;
namespace ConsoleApplication
{
class Program
{
static void Main(string[] args)
{
var x = 10;
var name = "naveen";
Console.WriteLine(name);
Console.Read();
}
}
}

I compiled it under release mode and launched it within the debugger. The goal is to have a break-point on Console.WriteLine(name). First thing was to set up a sx notification on load for mscorlib.

sxe ld:mscorlib

And when the break-point hits for the above event, then issued the following command to load sosex , sos and set an bp on Console.WriteLine which is nothing fancy

.load sosex;.loadby sos clr;!mbm *System.Console.WriteLine* "!mk";g

I would imagine that the break-point would hit and I would get a call-stack, but to my surprise this was the output

0:000> .load sosex;.loadby sos clr;!mbm *System.Console.WriteLine* “!mk”;g
The breakpoint could not be resolved immediately.
Further attempts will be made as modules are loaded.
(1090.11fc): CLR notification exception – code e0444143 (first chance)
Breakpoint set at System.Console.WriteLine().
Breakpoint set at System.Console.WriteLine(Boolean).
Breakpoint set at System.Console.WriteLine(Char).
Breakpoint set at System.Console.WriteLine(Char[]).
Breakpoint set at System.Console.WriteLine(Char[], Int32, Int32).
Breakpoint set at System.Console.WriteLine(System.Decimal).
Breakpoint set at System.Console.WriteLine(Double).
Breakpoint set at System.Console.WriteLine(Single).
Breakpoint set at System.Console.WriteLine(Int32).
Breakpoint set at System.Console.WriteLine(UInt32).
Breakpoint set at System.Console.WriteLine(Int64).
Breakpoint set at System.Console.WriteLine(UInt64).
Breakpoint set at System.Console.WriteLine(System.Object).
Breakpoint set at System.Console.WriteLine(System.String).
Breakpoint set at System.Console.WriteLine(System.String, System.Object).
Breakpoint set at System.Console.WriteLine(System.String, System.Object, System.Object).
Breakpoint set at System.Console.WriteLine(System.String, System.Object, System.Object, System.Object).
Breakpoint set at System.Console.WriteLine(System.String, System.Object, System.Object, System.Object, System.Object, …).
Breakpoint set at System.Console.WriteLine(System.String, System.Object[]).
(1090.11fc): CLR notification exception – code e0444143 (first chance)

That’s it. And I never got an hit for the break-point. I checked to make sure there was an actual breakpoint set by issuing a “bl” command. I could see there were break-points for Console.WriteLine. The next step was to  disassemble the code. So got the instruction pointer from the !mk call-stack. Here is the output of !mk. FYI this is when the code is blocked on Console.Read

00:U 003def90 75d273ea KERNEL32!ReadConsoleInternal+0×15
01:U 003def98 75d27041 KERNEL32!ReadConsoleA+0×40
02:U 003df020 75caf489 KERNEL32!ReadFileImplementation+0×75
03:M 003df068 65651c8b DomainNeutralILStubClass.IL_STUB_PInvoke(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte*, Int32, Int32 ByRef, IntPtr)(+0×0 IL)(+0×0 Native)
04:M 003df0e8 65cbf7e8 System.IO.__ConsoleStream.ReadFileNative(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte[], Int32, Int32, Int32, Int32 ByRef)(+0×53 IL)(+0x8c Native) [f:\dd\ndp\clr\src\BCL\System\IO\__ConsoleStream.cs, @ 16707566,0]
05:M 003df110 65cbf6d0 System.IO.__ConsoleStream.Read(Byte[], Int32, Int32)(+0x5d IL)(+0x9c Native) [f:\dd\ndp\clr\src\BCL\System\IO\__ConsoleStream.cs, @ 131,13]
06:M 003df138 65608bfb System.IO.StreamReader.ReadBuffer()(+0xa0 IL)(+0x3b Native) [f:\dd\ndp\clr\src\BCL\System\IO\StreamReader.cs, @ 488,21]
07:M 003df154 65bcacc3 System.IO.StreamReader.Read()(+0x1b IL)(+0×23 Native) [f:\dd\ndp\clr\src\BCL\System\IO\StreamReader.cs, @ 302,17]
08:M 003df160 65cc5e9d System.IO.TextReader+SyncTextReader.Read()(+0×0 IL)(+0×19 Native) [f:\dd\ndp\clr\src\BCL\System\IO\TextReader.cs, @ 244,17]
09:M 003df170 0066009a *** WARNING: Unable to verify checksum for ConsoleApplication.exe
ConsoleApplication.Program.Main(System.String[])(+0×0 IL)(+0x2a Native) [c:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication11\Program.cs, @ 10,13]
0a:U 003df17c 661621db clr!CallDescrWorker+0×33

Next disassemble Main Method using the ip which is 0066009a

!u 0066009a

Here is the output

0:000> !u 0066009a
Normal JIT generated code
ConsoleApplication.Program.Main(System.String[])
Begin 00660070, size 2d
c:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication11\Program.cs @ 10:
00660070 55 push ebp
00660071 8bec mov ebp,esp
00660073 56 push esi
00660074 8b3530206b03 mov esi,dword ptr ds:[36B2030h] (“naveen”)
c:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication11\Program.cs @ 11:
0066007a e85170f864 call mscorlib_ni+0x2570d0 (655e70d0) (System.Console.get_Out(), mdToken: 060008cd)
0066007f 8bc8 mov ecx,eax
00660081 8bd6 mov edx,esi
00660083 8b01 mov eax,dword ptr [ecx]
00660085 8b403c mov eax,dword ptr [eax+3Ch]
00660088 ff5010 call dword ptr [eax+10h]
0066008b e8f0a55565 call mscorlib_ni+0x82a680 (65bba680) (System.Console.get_In(), mdToken: 060008cc)
00660090 8bc8 mov ecx,eax
00660092 8b01 mov eax,dword ptr [ecx]
00660094 8b402c mov eax,dword ptr [eax+2Ch]
00660097 ff500c call dword ptr [eax+0Ch]
c:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication11\Program.cs @ 13:
>>> 0066009a 5e pop esi
0066009b 5d pop ebp
0066009c c3 ret

And I see System.Console.get_Out instead of System.Console.WriteLine which I was totally surprised. This was  the reason the break-point never hit. Next I wanted check the IL which was compiled , what we see above is jitted x86 mixed with IL. Here is the command to check the compiled IL. First I had to get the methodesc from the ip using !ip2md

!ip2md 0066009a

0:000> !ip2md 0066009a
MethodDesc: 002237f0
Method Name: ConsoleApplication.Program.Main(System.String[])
Class: 002213f8
MethodTable: 00223804
mdToken: 06000001
Module: 00222e9c
IsJitted: yes
CodeAddr: 00660070
Transparency: Critical
Source file: c:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication11\Program.cs @ 13

Here is from the methoddesc to IL

!dumpil 002237f0

0:000> !dumpil 002237f0
ilAddr = 012a2050
IL_0000: ldstr “naveen”
IL_0005: stloc.0
IL_0006: ldloc.0
IL_0007: call System.Console::WriteLine
IL_000c: call System.Console::Read
IL_0011: pop
IL_0012: ret

Which looks very similar to my C# code. So looks like CLR optimized the code ,converted the Console.WriteLine to Console.get_Out. To validate it restarted the app with this as the command for break-point

.load sosex;.loadby sos clr;!mbm *System.Console.get_Out* "!mk";g

And here is the output

00:M 0032ed58 655e70d1 System.Console.get_Out()(+0×0 IL)(+0×1 Native) [f:\dd\ndp\clr\src\BCL\System\Console.cs, @ 193,17]
01:M 0032ed60 003a007f *** WARNING: Unable to verify checksum for ConsoleApplication.exe
ConsoleApplication.Program.Main(System.String[])(+0×6 IL)(+0xf Native) [c:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication11\Program.cs, @ 11,13]
02:U 0032ed6c 661621db clr!CallDescrWorker+0×33

Now that I have solved this I wanted to check the same on the debug build (optimized -) . To validate if it was Console.get_Out or Console.WriteLine. So when mscorlib loaded here was my command to check this

.load sosex;.loadby sos clr;!mbm *Program.Main* "!u @eip";g

In the above command I am setting a break-point on Main method and when the break-point hits “!u @eip” will disassemble the ip, the @eip register will have the address of the current function. Here is the output from !u @eip

*** WARNING: Unable to verify checksum for ConsoleApplication11.exe

c:\users\naveen\documents\visual studio 2010\Projects\ConsoleApplication11\Program.cs @ 11:
01f10070 55 push ebp
01f10071 8bec mov ebp,esp
01f10073 83ec0c sub esp,0Ch
01f10076 894dfc mov dword ptr [ebp-4],ecx
01f10079 833d3c31360000 cmp dword ptr ds:[36313Ch],0
01f10080 7405 je 01f10087
01f10082 e8c85a5064 call clr!JIT_DbgIsJustMyCode (66415b4f)
01f10087 33d2 xor edx,edx
01f10089 8955f4 mov dword ptr [ebp-0Ch],edx
01f1008c 33d2 xor edx,edx
01f1008e 8955f8 mov dword ptr [ebp-8],edx
>>> 01f10091 90 nop
c:\users\naveen\documents\visual studio 2010\Projects\ConsoleApplication11\Program.cs @ 12:
01f10092 c745f80a000000 mov dword ptr [ebp-8],0Ah
c:\users\naveen\documents\visual studio 2010\Projects\ConsoleApplication11\Program.cs @ 13:
01f10099 8b0530200f03 mov eax,dword ptr ds:[30F2030h] (“naveen”)
01f1009f 8945f4 mov dword ptr [ebp-0Ch],eax
c:\users\naveen\documents\visual studio 2010\Projects\ConsoleApplication11\Program.cs @ 14:
01f100a2 8b4df4 mov ecx,dword ptr [ebp-0Ch]
*** WARNING: Unable to verify checksum for C:\Windows\assembly\NativeImages_v4.0.30319_32\mscorlib\246f1a5abb686b9dcdf22d3505b08cea\mscorlib.ni.dll
01f100a5 e802706d63 call mscorlib_ni+0x2570ac (655e70ac) (System.Console.WriteLine(System.String) , mdToken: 06000919)
01f100aa 90 nop
c:\users\naveen\documents\visual studio 2010\Projects\ConsoleApplication11\Program.cs @ 15:
01f100ab e8b4c1ca63 call mscorlib_ni+0x82c264 (65bbc264) (System.Console.Read(), mdToken: 0600090a)
01f100b0 90 nop
c:\users\naveen\documents\visual studio 2010\Projects\ConsoleApplication11\Program.cs @ 17:
01f100b1 90 nop
01f100b2 8be5 mov esp,ebp
01f100b4 5d pop ebp
01f100b5 c3 ret
eax=003637f0 ebx=00000000 ecx=020fbc7c edx=00000000 esi=004bb2d0 edi=0016f400
eip=01f10091 esp=0016f3c8 ebp=0016f3d4 iopl=0 nv up ei pl zr na pe nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246
01f10091 90 nop

Notice in the above code it is Console.WriteLine and not Console.get_Out.

Here is one of gotchas of debugging optimized code.

This post is all about exploring the CLR loader to load an assembly from GAC and from private bin path. This is not possible because, if an assembly is loaded from GAC and if the code uses Assembly.LoadFrom to load the same assembly, then the loader would return the assembly that has already been loaded from the GAC.

Here is the ClassLibrary2 code

using System;

namespace ClassLibrary2
{
 public class Class1
 {
 public string Bar()
 {
 return "bar";
 }

 }
}

Here is the Winforms application to load the Class Library Code

using System;
using System.Reflection;
using System.Windows.Forms;
using ClassLibrary2;
namespace WindowsFormsApplication4
{
 public partial class Form1 : Form
 {
 public Form1()
 {
 InitializeComponent();
 var c = new Class1();
 Console.WriteLine(c.Bar());
 }

 private void button1_Click(object sender, EventArgs e)
 {
 var  ad = AppDomain.CreateDomain("Test");
 ad.AssemblyResolve += ((a1,e1) => Assembly.LoadFile(@"C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication9\ClassLibrary2\bin\Debug\ClassLibrary2.dll"));
 var a = ad.Load("ClassLibrary");
 var c1 = a.CreateInstance("ClassLibrary2.Class1") as Class1;
 c1.Bar();
 }
 }
}

If you notice the code creates an instance of the “Class1” in the Form1 constructor, this will essentially load the ClassLibrary2 in to memory. FYI the ClassLibrary2 is GACed when the application launches. Here is the fusglovwr output , which shows the assembly being loaded from GAC.

LOG: This bind starts in default load context.

LOG: Using application configuration file: C:\Users\naveen\Documents\Visual Studio 2010\Projects\WindowsFormsApplication4\bin\Debug\WindowsFormsApplication4.exe.Config

LOG: Using host configuration file:

LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework\v4.0.30319\config\machine.config.

LOG: Post-policy reference: ClassLibrary2, Version=1.0.0.0, Culture=neutral, PublicKeyToken=3a0a06f4596f3e16

LOG: Found assembly by looking in the GAC.

LOG: Binding succeeds. Returns assembly from C:\Windows\Microsoft.Net\assembly\GAC_MSIL\ClassLibrary2\v4.0_1.0.0.0__3a0a06f4596f3e16\ClassLibrary2.dll.

LOG: Assembly is loaded in default load context.

After verifying fuslogvwr output , I uninstall the assembly from gac using gacutil /u (without closing the app). After which I click the button1 ,which will call the Assembly.LoadFrom to load the same assembly from private bin path. And here fuslogvwr output of loading the assembly using the load from, notice GAC lookup was unsuccessful.

LOG: This bind starts in default load context.

LOG: Using application configuration file: C:\Users\naveen\Documents\Visual Studio 2010\Projects\WindowsFormsApplication4\bin\Debug\WindowsFormsApplication4.exe.Config

LOG: Using host configuration file:

LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework\v4.0.30319\config\machine.config.

LOG: Post-policy reference: ClassLibrary2, Version=1.0.0.0, Culture=neutral, PublicKeyToken=3a0a06f4596f3e16

LOG: GAC Lookup was unsuccessful.

LOG: Attempting download of new URL file:///C:/Users/naveen/Documents/Visual Studio 2010/Projects/WindowsFormsApplication4/bin/Debug/ClassLibrary2.DLL.

LOG: Assembly download was successful. Attempting setup of file: C:\Users\naveen\Documents\Visual Studio 2010\Projects\WindowsFormsApplication4\bin\Debug\ClassLibrary2.dll

LOG: Entering run-from-source setup phase.

LOG: Assembly Name is: ClassLibrary2, Version=1.0.0.0, Culture=neutral, PublicKeyToken=3a0a06f4596f3e16

LOG: Binding succeeds. Returns assembly from C:\Users\naveen\Documents\Visual Studio 2010\Projects\WindowsFormsApplication4\bin\Debug\ClassLibrary2.dll.

LOG: Assembly is loaded in default load context.

And here is the !dumpdomain output from windbg for this process

Assembly:           003ce1e0 [C:\Windows\Microsoft.Net\assembly\GAC_MSIL\ClassLibrary2\v4.0_1.0.0.0__3a0a06f4596f3e16\ClassLibrary2.dll]
ClassLoader:        003ee5b8
SecurityDescriptor: 003d2838
Module Name
00306b28            C:\Windows\Microsoft.Net\assembly\GAC_MSIL\ClassLibrary2\v4.0_1.0.0.0__3a0a06f4596f3e16\ClassLibrary2.dll

Assembly:           003ceae0 [C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication9\ClassLibrary2\bin\Debug\ClassLibrary2.dll]
ClassLoader:        0044f418
SecurityDescriptor: 00445ea8
Module Name
011ee6b4            C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication9\ClassLibrary2\bin\Debug\ClassLibrary2.dll

Notice the first assembly is loaded from the GAC and the second one is loaded from the bin path.

This is not something that I would ever do. This was merely an exercise to bypass the loader.

FYI when the assembly is uninstalled from GAC, the gacutil.exe creates a temp copy and stores the assembly in the temp directory because it is being used by a process. And the temp copy of the assembly is deleted when the process ends. Here is the screen shot of Procmon creating an temp copy of the GACed assembly which I uninstalled.

This is going to be a multipart blog post where I am going to be demonstrating how I have customized Witty twitter client for my need. I chose Witty because it is the only OSS .NET twitter client I know of.

One of the reasons for customizing is primarily using C# as compiler service to extend it for my needs dynamically. For example I follow this twitter list http://twitter.com/shanselman/programmers , it’s a cool list that Scott maintains, thanks to him. But there is one person in this list who keeps tweeting about weight loss, which I am least interested and I didn’t have control over it but to ignore, until now. And it is always fun to write  software for your daily needs, which for me saves time.  Thanks to Mono for C# as compiler service. FYI I have shown a simple usage of Mono Csharp compiler service in this post.

Here are things that I could get done by just spending few hours of my weekend time

Here is my the default witty

Filter the list by user name  which I am not interested in

new Func<Tweet, bool>(t => !t.User.Name.Contains("CNN"));

Here is the same without CNN

FYI the filter C# code is in the Filter Text box

The next filter criteria is to look for tweets that have hashtag as fsharp and also at least have one link

new Func<Tweet, bool>(t => t.HashTags.DefaultIfEmpty().Contains("#fsharp") && t.Urls.DefaultIfEmpty().Count() > 0);

And here is the filtered list

Be even crazier look for tweets that have hashtag as fsharp and at least one link and then open the link automatically

new Func<Tweet, bool>(t => {
 if (t.HashTags.DefaultIfEmpty().Contains("#fsharp") && t.Urls.DefaultIfEmpty().Count() > 0)
 System.Diagnostics.Process.Start(t.Urls.First().ToString());
 return t.HashTags.DefaultIfEmpty().Contains("#fsharp") && t.Urls.DefaultIfEmpty().Count() > 0; });

The browser with the link opened automatically.

It’s a hack and I shouldn’t be doing the above, but it does the job.

There is so much more possibilities. I could easily provide a save feature for these search scripts and reuse the same.  At the end of the series I will post the entire code in GitHub. But if you want to try it before that here are the simple changes I started doing to the code.

Extension Method for Compilation

static class Extensions
 {
 public static object Compile(this string code)
 {
 return Mono.CSharp.Evaluator.Evaluate(code);
 }
 public static void Run(this string code)
 {
 Mono.CSharp.Evaluator.Run(code);
 }
 }

The filter code

public bool TweetFilter(object item)
 {
 Tweet tweet = item as Tweet;

 // this will prevent the fade animation from starting when the tweet is filtered
 tweet.IsNew = false;
 try
 {
 Func<Tweet, bool> compare = (Func<Tweet, bool>)FilterTextBox.Text.Compile();
 return compare.Invoke(tweet);
 }
 catch(Exception ex)
 {
 Console.WriteLine(ex);

 }
 return true;
 }

The hashtags and Links weren’t part of the tweet class , but they were part of the UI class. So I had to bring them to the tweet class and populate them.

public IEnumerable<Uri> Urls
 {
 get
 {
 return links;
 }
 set
 {
 links.Clear();
 links.AddRange(value);
 }
 }
 public IEnumerable<string> HashTags
 {
 get
 {
 return hashTags;
 }
 set
 {
 hashTags.Clear();
 hashTags.AddRange(value);
 }
 }

And here is the code to populate the above properties

private IEnumerable<Uri> GetLinks(string text)
 {
 string[] words = Regex.Split(text, @"([ \(\)\{\}\[\]])");
 return (from word in words
 let isUrl = new Func<string, bool>(s => UrlShorteningService.IsUrl(s))
 where isUrl(word)
 select new Uri(word) ).ToList();
 }
 private IEnumerable<string> GetHashTags(string text)
 {
 string[] words = Regex.Split(text, @"([ \(\)\{\}\[\]])");
 return (from word in words
 let ishash = new Func<string, bool>(s => s.StartsWith("#"))
 where ishash(word)
 select word).ToList();
 }

I know for the above 2 functions I could have written a High-Order Function

I  recently had to create a dictionary which needed a multipart key like <string,int> . To do this I would have to create a custom class  override equals and gethashcode. That’s when someone told I could use Tuple, but weren’t sure it was possible.  Here was the quick sample to try it in F#

let x = [("naveen",1),1;("naveen",1),2] |> Map.ofList

as expected only one item in the Map

val x : Map<(string * int),int> = map [(("naveen", 1), 2)]

And the same in C#

 Console.WriteLine(Tuple.Create("Naveen",1).Equals( Tuple.Create("Naveen",1)));

The next step was to actually disassemble Tuple in reflector

The code implements IStructuralComparable.CompareTo and does the comparison for each item. This is one of the reasons why generics is cool.

Continuing with my series on Piracy, in this post I am going to be  exploring how someone with little advanced knowledge in CLR / Windows can bypass important function calls like  license validation.

Most of the developers assume just because the code is obfuscated nobody can bypass the licensing logic. I am going to be demonstrating how to bypass certain function call,this is very similar to “Set Next Statement ” in VS. I am not going to be discussing on how to fix this problem.

Here is a sample code.

using System;
namespace Conosole
{   class Program
 {
 static void Main(string[] args)
 {
 Console.WriteLine("Test");
 Console.Read();
 }
 }
 }

The code has only two instructions. First one writes to console and the next to reads from the console. I would want to bypass the call to that WriteLine function.

Loaded the assembly within windbg and then issued the command

sxe ld: clrjit

When the break-point hits ,issued the command to set a break-point on the Main Method

.loadby sos clr;.load sosex;!mbm *Program.Main;g

Then when the break-point hits for the Main method , issued the following command to disassemble the Main method.

!u ($ip)

0:000> !u ($ip)
Normal JIT generated code
Conosole.Program.Main(System.String[])
Begin 00230070, size 2d

C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication6\Program.cs @ 6:
00230070 55              push    ebp
00230071 8bec            mov     ebp,esp
00230073 50              push    eax
00230074 894dfc          mov     dword ptr [ebp-4],ecx
00230077 833d3c31180000  cmp     dword ptr ds:[18313Ch],0
0023007e 7405            je      00230085
00230080 e8ca5a6962      call    clr!JIT_DbgIsJustMyCode (628c5b4f)
>>> 00230085 90              nop

C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication6\Program.cs @ 7:
00230086 8b0d30204a03    mov     ecx,dword ptr ds:[34A2030h] (“Test”)
0023008c e81b707a61      call    mscorlib_ni+0x2570ac (619d70ac) (System.Console.WriteLine(System.String), mdToken: 06000919)
00230091 90              nop

C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication6\Program.cs @ 8:
00230092 e8cdc1d761      call    mscorlib_ni+0x82c264 (61fac264) (System.Console.Read(), mdToken: 0600090a)
00230097 90              nop

C:\Users\naveen\Documents\Visual Studio 2010\Projects\ConsoleApplication6\Program.cs @ 9:
00230098 90              nop
00230099 8be5            mov     esp,ebp
0023009b 5d              pop     ebp
0023009c c3              ret
0:000> bp 0023008c
0:000> g

Because I have private symbols the line information is shown. So the function I want to bypass is “0023008c e81b707a61      call    mscorlib_ni+0x2570ac (619d70ac) (System.Console.WriteLine(System.String), mdToken: 06000919)” and the ip for this is 0023008c , so went ahead and set a break-point on the address

bp 0023008c

When the break-point hits on 0023008c, I move pointer to the next instruction that I am interested in ,which is “00230092 e8cdc1d761      call    mscorlib_ni+0x82c264 (61fac264) (System.Console.Read(), mdToken: 0600090a)” to avoid the function being invoked and here is the command

r eip=00230092

0:000> g
Breakpoint 1 hit
eax=001837f0 ebx=00000000 ecx=024abb50 edx=0041efd0 esi=008196c0 edi=0041ef20
eip=0023008c esp=0041eef0 ebp=0041eef4 iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
0023008c e81b707a61      call    mscorlib_ni+0x2570ac (619d70ac)
0:000> r eip=00230092

Now we have managed to bypass the call to Console.WriteLine and here is my output .

So the key takeaway is to understand the working  of the platform closer to the metal, which can help us write better and secure code.

The WER is a pretty cool technology from Microsoft for collecting memory dumps on process crash/ hang. This can be extended to generate on demand when the application needs to. The usual reason for getting a memory dump could be based on certain conditions, for example, the customer feels the application is slow and would want to send the information to WinQual (WER server). If the application happens to be installed on hundreds / thousands of boxes then its not going to be possible to get from individual customers, the best bet is WER. To do this here is an API. But this is unmanaged API and I didn’t see one for managed code. FYI this would work only on Vista + systems, it will not work on XP.

Here is the basic PInvoke for creating dump and submitting a report. I am also using it along with the watsonbuckets that I had blogged about.

internal enum WER_CONSENT
 {
 WerConsentAlwaysPrompt = 4,
 WerConsentApproved = 2,
 WerConsentDenied = 3,
 WerConsentMax = 5,
 WerConsentNotAsked = 1
 }
 internal enum WER_DUMP_TYPE
 {
 WerDumpTypeHeapDump = 3,
 WerDumpTypeMax = 4,
 WerDumpTypeMicroDump = 1,
 WerDumpTypeMiniDump = 2
 }
 internal enum WER_REPORT_TYPE
 {
 WerReportNonCritical,
 WerReportCritical,
 WerReportApplicationCrash,
 WerReportApplicationHange,
 WerReportKernel,
 WerReportInvalid
 }

 internal static class Unmanaged
 {
 [DllImport("wer.dll", CharSet = CharSet.Unicode, SetLastError = true)]
 internal static extern int WerReportAddDump(IntPtr hReportHandle,
 IntPtr hProcess, IntPtr hThread, WER_DUMP_TYPE dumpType, IntPtr pExceptionParam, IntPtr pDumpCustomOptions, int dwFlags);
 [DllImport("wer.dll", CharSet = CharSet.Unicode, SetLastError = true)]
 internal static extern int WerReportCreate(string pwzEventType,
 WER_REPORT_TYPE repType, IntPtr pReportInformation, ref IntPtr phReportHandle);
 [DllImport("wer.dll", CharSet = CharSet.Unicode, SetLastError = true)]
 internal static extern int WerReportSetParameter(IntPtr hReportHandle, int dwparamID, string pwzName, string pwzValue);
 [DllImport("wer.dll", CharSet = CharSet.Unicode, SetLastError = true)]
 internal static extern int WerReportSubmit(IntPtr hReportHandle, WER_CONSENT consent, int dwFlags, ref IntPtr pSubmitResult);
 }

I have shown only few functions in the wer api, there are few more.  Here is the total implementation

using System;
using System.Windows.Forms;
using System.Runtime.InteropServices;
namespace WER
{
 public partial class Form1 : Form
 {
 static bool Failed(int result)
 {
 return (result < 0);
 }
 public Form1()
 {
 InitializeComponent();
 button1.Click += (s, b) =>
 {
 try
 {
 throw new NullReferenceException("Test");
 }
 catch (Exception ex)
 {

 var bucket = GetWatsonBuckets();
 var zero = IntPtr.Zero;

 if ((!Failed(Unmanaged.WerReportCreate("CrashingApp",
 WER_REPORT_TYPE.WerReportCritical, IntPtr.Zero, ref zero)) &&
 (zero != IntPtr.Zero)) && ((((!Failed(Unmanaged.WerReportSetParameter(zero, 0, "AppName", bucket.param0))
 && !Failed(Unmanaged.WerReportSetParameter(zero, 1, "AppVer", bucket.param1))) &&
 (!Failed(Unmanaged.WerReportSetParameter(zero, 2, "AppStamp", bucket.param2)) &&
 !Failed(Unmanaged.WerReportSetParameter(zero, 3, "AsmAndModName", bucket.param3)))) &&
 ((!Failed(Unmanaged.WerReportSetParameter(zero, 4, "AsmVer", bucket.param4)) &&
 !Failed(Unmanaged.WerReportSetParameter(zero, 5, "ModStamp", bucket.param5))) &&
 (!Failed(Unmanaged.WerReportSetParameter(zero, 6, "MethodDef", bucket.param6)) &&
 !Failed(Unmanaged.WerReportSetParameter(zero, 7, "Offset", bucket.param7))))) &&
 !Failed(Unmanaged.WerReportSetParameter(zero, 8, "ExceptionType", bucket.param8))))
 {

 var currentProcess = System.Diagnostics.Process.GetCurrentProcess().Handle;
 if (!Failed(Unmanaged.WerReportAddDump(zero, currentProcess,
 IntPtr.Zero, WER_DUMP_TYPE.WerDumpTypeHeapDump, IntPtr.Zero, IntPtr.Zero, 0)))
 {
 var pSubmitResult = IntPtr.Zero;
 Unmanaged.WerReportSubmit(zero, WER_CONSENT.WerConsentNotAsked, 4, ref pSubmitResult);
 }
 }
 }
 };
 }
 private static WatsonBuckets GetWatsonBuckets()
 {
 var pParams = new WatsonBuckets();
 IClrRuntimeHost host = null;
 host = Activator.CreateInstance(Type.GetTypeFromCLSID(ClrGuids.ClsIdClrRuntimeHost)) as IClrRuntimeHost;
 if (host != null)
 {
 var clrControl = host.GetCLRControl();
 if (clrControl == null)
 {
 return pParams;
 }
 var clrErrorReportingManager =
 clrControl.GetCLRManager(ref ClrGuids.IClrErrorReportingManager) as IClrErrorReportingManager;
 if (clrErrorReportingManager == null)
 {
 return pParams;
 }
 clrErrorReportingManager.GetBucketParametersForCurrentException(out pParams);
 }
 return pParams;
 }
 }
 // BucketParameters Structure to get watson buckets back from CLR
 //http://msdn.microsoft.com/en-us/library/ms404466(v=VS.100).aspx
 [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Auto)]
 internal struct WatsonBuckets
 {
 internal int fInited;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)]
 internal string pszEventTypeName;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)]
 internal string param0;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)]
 internal string param1;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)]
 internal string param2;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)]
 internal string param3;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)]
 internal string param4;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)]
 internal string param5;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)]
 internal string param6;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)]
 internal string param7;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)]
 internal string param8;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)]
 internal string param9;
 }
 internal static class ClrGuids
 {
 internal static readonly Guid ClsIdClrRuntimeHost = new Guid("90F1A06E-7712-4762-86B5-7A5EBA6BDB02");
 internal static Guid IClrErrorReportingManager = new Guid("980D2F1A-BF79-4c08-812A-BB9778928F78");
 internal static readonly Guid IClrRuntimeHost = new Guid("90F1A06C-7712-4762-86B5-7A5EBA6BDB02");
 }
 [Guid("90F1A06C-7712-4762-86B5-7A5EBA6BDB02"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
 internal interface IClrRuntimeHost
 {
 void Start();
 void Stop();
 void SetHostControl(IntPtr pHostControl);
 IClrControl GetCLRControl();
 void UnloadAppDomain(int dwAppDomainId, bool fWaitUntilDone);
 void ExecuteInAppDomain(int dwAppDomainId, IntPtr pCallback, IntPtr cookie);
 int GetCurrentAppDomainId();

 int ExecuteApplication(string pwzAppFullName, int dwManifestPaths, string[] ppwzManifestPaths,
 int dwActivationData, string[] ppwzActivationData);

 int ExecuteInDefaultAppDomain(string pwzAssemblyPath, string pwzTypeName, string pwzMethodName,
 string pwzArgument);
 }
 [Guid("9065597E-D1A1-4fb2-B6BA-7E1FCE230F61"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
 internal interface IClrControl
 {
 [return: MarshalAs(UnmanagedType.IUnknown)]
 object GetCLRManager([In] ref Guid riid);

 void SetAppDomainManagerType(string pwzAppDomainManagerAssembly, string pwzAppDomainManagerType);
 }
 // IClrErrorReportingManager to get watson bukets back from CLR
 //http://msdn.microsoft.com/en-us/library/ms164367(v=VS.100).aspx
 [Guid("980D2F1A-BF79-4c08-812A-BB9778928F78"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
 internal interface IClrErrorReportingManager
 {
 [PreserveSig]
 int GetBucketParametersForCurrentException(out WatsonBuckets pParams);
 }
}

In the above code ,I am using this within a windows form to create a memory dump when an exception is thrown. The GetWatson bucket and the clr hosting interface is just to get the watson bucket information from clr. This can even be extended to attach a custom log file. How cool is this. Here is the call-stack for the above exception

In this post I am going to be discussing about debugging .Net Framework 4.0  using windbg . I am going to demonstrating how to have a break-point within a method, but without the framework source code. This would help in debugging .NET framework when you don’t have VS in a production environment and the same technique can be used to debug other third party assemblies where you don’t have the source code.  This is kind of like .NET Reflector where you can step through third party assemblies, but without any cost. It is not going to be as convenient as the professional version of Reflector.

I am going to be using the same example that I used to debug .NET Framework source 3.5 using windbg.

FYI the .NET framework 4.0 has private symbols available on MS symbol server, but the source code is still not available. To debug .NET framework source code it is important to have correct symbol path and here is my symbol path in the _NT_SYMBOL_PATH environment variable.

SRV*d:\dev\symbols*http://referencesource.microsoft.com/symbols; SRV*d:\dev\symbols*http://msdl.microsoft.com/download/symbols

Here is the sample source code that I am going to be using to demonstrate framework debugging

using System;
 using System.Net;
 namespace Test
 {
 class Program
 {
 static void Main(string[] args)
 {
 Console.WriteLine("Hello World of debugging");
 var wr = WebRequest.Create("http://www.google.com");
 Console.WriteLine("Web request created");
 var req = wr.GetRequestStream();
 Console.WriteLine("Hello World Debugging");
 Console.Read();
 }
 }
 }
 

Launched the exe within the debugger

Then issued the command to notify when the clrjit is loaded,

sxe ld:clrjit

This is because, to load sos and sosex after the framework is loaded.Then issued the following commands to load sos, sosex and to set break-point on WebRequest.Create

.loadby sos mscorwks
.load sosex
!mbm System.Net.WebRequest.Create

And when the break-point hits the first time, let it continue by using the g command. It would break into the debugger for other overloaded method for WebRequest.Create and here is the call-stack

0:000> !mk
Thread 0:
ESP              EIP
00:M 000000000023ece0 000007fef6ea5a44 System.Net.WebRequest.Create(System.Uri, Boolean)(+0×0 IL)(+0×14 Native) [f:\dd\ndp\fx\src\Net\System\Net\WebRequest.cs, @ 93,13]
01:M 000000000023ed60 000007ff00140176 Test.Program.Main(System.String[])(+0xc IL)(+0×56 Native)
02:U 000000000023edc0 000007fef8b210b4 clr!CallDescrWorker+0×84
03:U 000000000023ee10 000007fef8b211c9 clr!CallDescrWorkerWithHandler+0xa9
04:U 000000000023ee90 000007fef8b21245 clr!MethodDesc::CallDescr+0x2a1
05:U 000000000023f0c0 000007fef8c21675 clr!ClassLoader::RunMain+0×228
06:U 000000000023f310 000007fef8c217ac clr!Assembly::ExecuteMainMethod+0xac
07:U 000000000023f5c0 000007fef8c21562 clr!SystemDomain::ExecuteMainMethod+0×452
08:U 000000000023fb70 000007fef8c23dd6 clr!ExecuteEXE+0×43
09:U 000000000023fbd0 000007fef8c23cf3 clr!CorExeMainInternal+0xc4
0a:U 000000000023fc40 000007fef8ca7365 clr!CorExeMain+0×15
0b:U 000000000023fc80 000007fef9493309 mscoreei!CorExeMain+0×41
0c:U 000000000023fcb0 000007fef9525b21 MSCOREE!CorExeMain_Exported+0×57
0d:U 000000000023fce0 00000000776cf56d KERNEL32!BaseThreadInitThunk+0xd
0e:U 000000000023fd10 0000000077903281 ntdll!RtlUserThreadStart+0x1d

And here is the source code for this method using reflector

private static WebRequest Create(Uri requestUri, bool useUriBase)
{
 string absoluteUri;
 if (Logging.On)
 {
 Logging.Enter(Logging.Web, "WebRequest", "Create", requestUri.ToString());
 }
 WebRequestPrefixElement element = null;
 bool flag = false;
 if (!useUriBase)
 {
 absoluteUri = requestUri.AbsoluteUri;
 }
 else
 {
 absoluteUri = requestUri.Scheme + ':';
 }
 int length = absoluteUri.Length;
 ArrayList prefixList = PrefixList;
 for (int i = 0; i < prefixList.Count; i++)
 {
 element = (WebRequestPrefixElement) prefixList[i];
 if ((length >= element.Prefix.Length) && (string.Compare(element.Prefix, 0, absoluteUri, 0, element.Prefix.Length, StringComparison.OrdinalIgnoreCase) == 0))
 {
 flag = true;
 break;
 }
 }
 WebRequest retObject = null;
 if (flag)
 {
 retObject = element.Creator.Create(requestUri);
 if (Logging.On)
 {
 Logging.Exit(Logging.Web, "WebRequest", "Create", retObject);
 }
 return retObject;
 }
 if (Logging.On)
 {
 Logging.Exit(Logging.Web, "WebRequest", "Create", (string) null);
 }
 throw new NotSupportedException(SR.GetString("net_unknown_prefix"));
}

Let’s try and have a break-point on line   “ ArrayList prefixList = PrefixList;” so that we can check the local variables value.  Just because I have the private symbols ,I could have counted the line numbers manually and then set a break-point using !mbp command, but that is no fun. Here is another way of doing this.

.shell -ci "!u 000007fef6ea5a44 "  findstr get_PrefixList

In the above command I am disassembling the ip 000007fef6ea5a44 (which is there in the above callstack )to look for  get_PrefixList Instruction pointer . Here is the outcome

0:000> .shell -ci “!u 000007fef6ea5a44 “  findstr get_PrefixList

000007fe`f6ea5a7a e8f1000000      call    System_ni+0x275b70 (000007fe`f6ea5b70) (System.Net.WebRequest.get_PrefixList(), mdToken: 00000000060019bc)

.shell: Process exited

I use the .shell command to manually avoid searching for an instruction. Now that I have the instruction pointer ,I am going to set a break-point on that using

bp 000007fe`f6ea5a7a "!mdv"

and here is the result of the break-point

0:000> g
(1758.167c): CLR notification exception – code e0444143 (first chance)
(1758.167c): CLR notification exception – code e0444143 (first chance)
Frame 0×0: (System.Net.WebRequest.Create(System.Uri, Boolean)):
[A0]:requestUri:<?>
[A1]:useUriBase:<?>
[L0]:LookupUri:0x00000000022c1a98 (System.String) STRVAL=http://www.google.com/
[L1]:Current:<?>
[L2]:Found:0×0000000000000000 (System.Boolean)
[L3]:LookupLength:0×0000000000000016 (System.Int32)
[L4]:prefixList:<?>
[L5]:i:<?>
[L6]:webRequest:<?>

System_ni+0x275a7a:
000007fe`f6ea5a7a e8f1000000      call    System_ni+0x275b70 (000007fe`f6ea5b70)

Voila! now I am able to have a break-point within the framework method  and also see locals and parameters like Visual Studio. The same technique can be used to debug third party assemblies where you don’t have source code or symbols.

Continuing with my previous post, one of the biggest security holes I have noticed in certain application is using unsecure Network I/O communication, especially when activating license. I have seen software where they have used the best tool to obfuscate the code, it is extremely hard to disassemble this. But lose out by invoking a web service with plaintext xml for registration and communication. Like I mentioned in my previous post, I am not going to be discussing on how to solve this problem.

I have seen applications which would have a trial period after which the application pops-up a dialog box for activating license. Only if license text matches the format criteria (like xxx-xx-xxxx-xx) the send button enables. To circumvent the disabled button, some of the smart developers could enable the button within the debugger (windbg /cdb) or using a something like HawkEye . This is the first line of defense for the application.

The next thing is launching something like Fiddler and checking out the web service request /response for the activation request. I am sure most of us have dealt with fiddler, and if you are not aware, fiddler gives an option to have a break-point on response from the server. So it is easy to do a Man-in-the-middle attack by injecting your own response or someone could even hack the lmhost file to act as a server.

And just because it is plain text, I have usually seen a bool variable in the response being activated or not. And it is not hard for someone to update the response text and pass it as a response back from the server.

The updated xml might not comply with the response expected by the application, it might throw an exception like this

0:020> !pe

Exception object: 045b4314

Exception type:   System.Xml.XmlException

Message:          Unexpected end of file while parsing Test  has occurred. Line 13, position 81.

InnerException:   <none>

StackTrace (generated):

<none>

StackTraceString: <none>

HResult: 80131940

Because it is xml, usually these applications end up using XmlTextReader to parse it and from the exception someone could figure out what element is expected and build the xml out of this. FYI the XmlTextReader library is usually never obfuscated.

The idea behind this post is to let the dev’s understand the security risks and someone with little advanced knowledge can exploit the holes. This is not just applicable for software piracy, but in general, especially with the RIA growing rapidly we would want to consider the security aspect also.