Use IntelliTrace without Visual Studio .NET


IntelliTrace is one of the best things in Visual Studio 2010. You can do “time travel” through your managed code. John Robbins has cool cool blog post on the working on IntelliTrace. I knew I could use IntelliTrace in field where the customer does not have VS.NET. To use in field ,I had to figure out the dependencies for IntelliTrace. So I fired up Windbg attached to the IntelliTrace process which would dump the all the modules loaded


Symbol search path is: SRV*D:\symbols*http://msdl.microsoft.com/download/symbols
Executable search path is:
ModLoad: 01210000 01218000   D:\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\TraceDebugger Tools\IntelliTrace.exe
ModLoad: 77d20000 77ea0000   C:\Windows\SysWOW64\ntdll.dll
ModLoad: 716b0000 716fa000   C:\Windows\SYSTEM32\MSCOREE.DLL
ModLoad: 76fe0000 770e0000   C:\Windows\syswow64\KERNEL32.dll
ModLoad: 75de0000 75e26000   C:\Windows\syswow64\KERNELBASE.dll
ModLoad: 75b90000 75c30000   C:\Windows\syswow64\ADVAPI32.dll
ModLoad: 77360000 7740c000   C:\Windows\syswow64\msvcrt.dll
ModLoad: 76000000 76019000   C:\Windows\SysWOW64\sechost.dll
ModLoad: 75cf0000 75de0000   C:\Windows\syswow64\RPCRT4.dll
ModLoad: 75890000 758f0000   C:\Windows\syswow64\SspiCli.dll
ModLoad: 75880000 7588c000   C:\Windows\syswow64\CRYPTBASE.dll
ModLoad: 71050000 710b6000   C:\Windows\Microsoft.NET\Framework\v4.0.30319\mscoreei.dll
ModLoad: 77440000 77497000   C:\Windows\syswow64\SHLWAPI.dll
ModLoad: 75910000 759a0000   C:\Windows\syswow64\GDI32.dll
ModLoad: 76cf0000 76df0000   C:\Windows\syswow64\USER32.dll
ModLoad: 774a0000 774aa000   C:\Windows\syswow64\LPK.dll
ModLoad: 759a0000 75a3d000   C:\Windows\syswow64\USP10.dll
ModLoad: 770e0000 77140000   C:\Windows\system32\IMM32.DLL
ModLoad: 77710000 777dc000   C:\Windows\syswow64\MSCTF.dll
ModLoad: 6b590000 6bbff000   C:\Windows\Microsoft.NET\Framework\v4.0.30319\clr.dll
ModLoad: 70c00000 70cbe000   C:\Windows\system32\MSVCR100_CLR0400.dll
ModLoad: 6a7c0000 6b583000   C:\Windows\assembly\NativeImages_v4.0.30319_32\mscorlib\246f1a5abb686b9dcdf22d3505b08cea\mscorlib.ni.dll
ModLoad: 71b50000 71b60000   C:\Windows\Microsoft.NET\Framework\v4.0.30319\nlssorting.dll
ModLoad: 76df0000 76f4c000   C:\Windows\syswow64\ole32.dll
ModLoad: 6d2b0000 6db48000   C:\Windows\assembly\NativeImages_v4.0.30319_32\System\964da027ebca3b263a05cadb8eaa20a3\System.ni.dll
ModLoad: 74180000 74188000   C:\Windows\assembly\NativeImages_v4.0.30319_32\IntelliTrace\947780232db1934c92cdfdaf2433bb59\IntelliTrace.ni.exe
ModLoad: 59660000 59a9d000   C:\Windows\assembly\NativeImages_v4.0.30319_32\Microsoft.VisualStu#\c828bb166e9d0df0e9b44a0a7616624a\Microsoft.VisualStudio.IntelliTrace.ni.dll
ModLoad: 618b0000 6196e000   C:\Windows\assembly\NativeImages_v4.0.30319_32\System.Runtime.Remo#\dc1f0dbf1d3ba856eccec90b62b55d79\System.Runtime.Remoting.ni.dll
ModLoad: 66330000 66883000   C:\Windows\assembly\NativeImages_v4.0.30319_32\System.Xml\e997d0200c25f7db6bd32313d50b729d\System.Xml.ni.dll
ModLoad: 75c60000 75cef000   C:\Windows\syswow64\oleaut32.DLL
ModLoad: 70a10000 70a70000   C:\Windows\Microsoft.NET\Framework\v4.0.30319\clrjit.dll
ModLoad: 58fe0000 59085000   C:\Program Files (x86)\Common Files\Microsoft Shared\VSTS 10.0\Trace Debugger\TraceLogProfiler.dll
ModLoad: 77cf0000 77cf5000   C:\Windows\syswow64\PSAPI.DLL
ModLoad: 76020000 76c69000   C:\Windows\syswow64\SHELL32.dll
ModLoad: 752a0000 752a9000   C:\Windows\system32\VERSION.dll
ModLoad: 753e0000 753f6000   C:\Windows\system32\CRYPTSP.dll
ModLoad: 75360000 7539b000   C:\Windows\system32\rsaenh.dll
ModLoad: 753d0000 753de000   C:\Windows\system32\RpcRtRemote.dll
ModLoad: 650e0000 650e5000   C:\Windows\system32\shfolder.dll
ModLoad: 75300000 7535f000   C:\Windows\system32\sxs.dll
(159c.6bc): Break instruction exception - code 80000003 (first chance)
eax=7ef39000 ebx=00000000 ecx=00000000 edx=77dbf50a esi=00000000 edi=00000000
eip=77d3000c esp=0683fca4 ebp=0683fcd0 iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
ntdll!DbgBreakPoint:
77d3000c cc              int     3

So from the above output I figure out the required files for running IntelliTrace without VS.NET is

  1. IntelliTrace.exe
  2. IntelliTrace.exe.config
  3. Microsoft.VisualStudio.IntelliTrace.dll
  4. TraceLogProfiler.dll

The next thing was to check the command line options for IntelliTrace

  1. start
  2. status
  3. stop
  4. run
  5. launch
  6. help

I knew the command line option should have something interesting. So when the IntelliTrace was running I looked at command line options in Process Explorer

"D:\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\TraceDebugger Tools\IntelliTrace.EXE" run /n:"test.exe_00001330_01cae6f016c66e1e"
/cp:"C:\Users\naveen\AppData\Local\Microsoft\VisualStudio\10.0\TraceDebugger\Settings\aoqsbu4g.fpj" /f:"Test_00001330_100428_123018.iTrace"

When i tried to open the “C:\Users\naveen\AppData\Local\Microsoft\VisualStudio\10.0\TraceDebugger\Settings\aoqsbu4g.fpj”, I got this error

So I went to the directory and  just made a copy of the file and then I was able to open the file. It was a xml file which contains most of the settings. It is collection plan settings file. The things that I updated in this file were <LogFileDirectory useDefault=”false”>C:\temp\</LogFileDirectory> and <DeleteLogOnExit>false</DeleteLogOnExit> for my test run. There are lot of settings in this file, which can be tweaked based on need,do explore this file.

With all this information, the next thing was to get a clean Installation of  Windows without VS.NET, good thing I had my laptop which had that. So I copied the required dependencies mentioned above along with a simple test.exe  which I used for figuring out Watson Buckets and here are the contents of the directory

And here is the command line option to start tracing Test.exe

C:\Users\naveen\intellitrace\intellitrace>IntelliTrace.exe launch /cp:q0mmz2ch.ixp Test.exe

The /cp:q0mmz2ch.ixp is the collection plan file. Here is the output from the above command

C:\Users\Naveen\intellitrace\intellitrace>IntelliTrace.exe launch /cp:q0mmz2ch.i
xp Test.exe
Microsoft (R) Visual Studio Logger. Version 10.0.30319.1
Copyright (C) Microsoft Corporation. All rights reserved.
C:\Users\Naveen\intellitrace\intellitrace\IntelliTrace.exe run /name:c__temp_9ce
6b8f1-19a5-44b7-909d-512a4c74e632.itrace /logfile:C:\temp\9ce6b8f1-19a5-44b7-909
d-512a4c74e632.iTrace /buffersize:65536 /buffercount:512 /watch:-1 /help- /nolog
o+ /collectionplan:q0mmz2ch.ixp /hidden-
Logger name is ‘c__temp_9ce6b8f1-19a5-44b7-909d-512a4c74e632.itrace’
Log file path ‘C:\temp\9ce6b8f1-19a5-44b7-909d-512a4c74e632.iTrace’
Using 512 buffers of 65536 bytes each.
Logger started.
Logger started.
Press Ctrl+C to stop logging or use ‘IntelliTrace stop /name:c__temp_9ce6b8f1-19
a5-44b7-909d-512a4c74e632.itrace’ from another command line.
Starting process ‘C:\Users\Naveen\intellitrace\intellitrace\Test.exe’
Waiting for process to exit
WatsonTest.WatsonBuckets

Unhandled Exception: System.NullReferenceException: Object reference not set to
an instance of an object.
at WatsonTest.Test.Main(String[] args)
Process exited with exit code -532462766.
Process execution time: 5097 ms
Logger name is ‘c__temp_9ce6b8f1-19a5-44b7-909d-512a4c74e632.itrace’
Stopping logger

That’s cool. I was able to trace it without VS.NET on the box and  I brought the trace file back to dev machine I was able load it up in VS.NET and could see stacks. And also from the above output I figured out  there are other undocumented command line options.

VS2010 AddIn for setting Break-points on partial function names


I was recently watching cool debugging video from John Robbins. Its got some amazing tips for debugging. In the video, John had mentioned that  VS2010 does not have a way to set partial function name break-points in managed code. He also wanted someone to implement that. So I though why not pick up the gauntlet.   So went ahead and implemented a VS2010 addin that can use wildcard /regex function names for setting break-points. I reused lot of code from John’s macro for setting break-points. The source code and the compiled version can be download from here. I have tested this only my box. If there are any issues please comment back. I could have used Intellisense for the function names, probably I will do it as next release.

  1. The AddIn function name can have wildcard / regular expression for setting break-points.
  2. By default it uses only function name and not the full qualifier like Namespace.Class.FunctionName
  3. To fully qualify with NameSpace.Class.FunctionName , check the FullName check box
  4. By default function name search is not case sensitive , to have case sensitive search check the case sensitive check box

I have not created an installer. To deploy the addin extract the zip file and copy “RegularExpressionBP.AddIn” to addins directory which happened to “C:\Users\naveen\Documents\Visual Studio 2010\Addins” . The addin locations are loaded from the following locations

Update the RegularExpressionBP.AddIn to point to the location of the RegularExpressionBP.dll

<Assembly>c:\users\naveen\documents\visual studio 2010\Projects\RegularExpressionBP\bin\RegularExpressionBP.dll</Assembly>

The addin will be loaded in the tools menu by default.

Here is the partial code for setting the break-point on partial function names . The full source code can be downloaded from here


private void SetPartialNameBreakpointsMethods(string funcName)
 {
 var count = _dte2.Solution.Projects.Count;
 for (var i = 1; i <= count; i++)
 {
 for (var j = 1; j <= _dte2.Solution.Projects.Item(i).ProjectItems.Count; j++)
 {
 if (_dte2.Solution.Projects.Item(i).ProjectItems.Item(j).FileCodeModel == null)
 {
 continue;
 }
 else
 {
 var srcFile = _dte2.Solution.Projects.Item(i).ProjectItems.Item(j).Name;
 ProcessCodeElements(_dte2.Solution.Projects.Item(i).ProjectItems.Item(j).FileCodeModel.CodeElements, srcFile,funcName);
 }
 }
 }
 }
 private void ProcessCodeElements(CodeElements elems, string srcFile, string funcName)
 {
 foreach (CodeElement currElem in elems.Cast<CodeElement>())
 {
 var ns = default(CodeNamespace);
 var codeType = default(CodeType);
 var codeFunction = default(CodeFunction);

 if (currElem is CodeNamespace)
 {
 ns = (CodeNamespace) currElem;
 ProcessCodeElements(ns.Members, srcFile, funcName);
 }
 else if (currElem is CodeType)
 {
 codeType = (CodeType) currElem;
 ProcessCodeElements(codeType.Members, srcFile, funcName);
 }
 else if (currElem is CodeFunction | currElem is CodeProperty)
 {
 try
 {
 codeFunction = (CodeFunction) currElem;
 }
 catch
 {
 }
 TextPoint txtPt = default(TextPoint);
 try
 {
 txtPt = currElem.StartPoint;
 }
 catch
 {
 txtPt = null;
 }

 if ((txtPt != null && codeFunction != null &&
 new Wildcard(funcName,caseSensitive.Checked == false ? RegexOptions.IgnoreCase: RegexOptions.None).
 Match(fullName.Checked ? codeFunction.FullName: codeFunction.Name).Success))
 {
 SafelySetBreakpoint(srcFile, txtPt.Line);
 }
 }
 }
 }
 private void SafelySetBreakpoint(string srcFile, int lineNum)
 {
 try
 {
 _dte2.Debugger.Breakpoints.Add("", srcFile, lineNum);

 }
 catch (COMException ex)
 {
 }
 }

using System.Text.RegularExpressions;

namespace RegularExpressionBP
{
 /// <summary>
 /// Represents a wildcard running on the
 /// <see cref="System.Text.RegularExpressions"/> engine.
 /// </summary>
 public class Wildcard : Regex
 {
 /// <summary>
 /// Initializes a wildcard with the given search pattern.
 /// </summary>
 /// <param name="pattern">The wildcard pattern to match.</param>
 public Wildcard(string pattern)
 : base(WildcardToRegex(pattern))
 {
 }

 /// <summary>
 /// Initializes a wildcard with the given search pattern and options.
 /// </summary>
 /// <param name="pattern">The wildcard pattern to match.</param>
 /// <param name="options">A combination of one or more
 /// <see cref="RegexOptions"/>.</param>
 public Wildcard(string pattern, RegexOptions options)
 : base(WildcardToRegex(pattern), options)
 {
 }

 /// <summary>
 /// Converts a wildcard to a regex.
 /// </summary>
 /// <param name="pattern">The wildcard pattern to convert.</param>
 /// <returns>A regex equivalent of the given wildcard.</returns>
 public static string WildcardToRegex(string pattern)
 {
 return "^" + Escape(pattern).
 Replace("\\*", ".*").
 Replace("\\?", ".") + "$";
 }
 }
}

I used the VS2010 automation object model chart for discovering object hierarchy, which was extremely useful.

Exploring UnhandledException in .NET and Watson buckets


I wanted to understand about UnhandledExceptions in .NET because of the few questions that I saw in the CLR Forum ,which had watson buckets in the event viewer.  To get deep understanding  unhandled exception filter there is a article on MSDN from CLR Team.

In this post I will be demonstrating how to get the Watson Bucket from within your code . FYI this is the same information  you can get !WatsonBuckets from sos within Windbg whenever there is a termination of the .NET process. !WatsonBuckets is undocumented.  I am using the CLR hosting interfaces to get the watson bucket. Here is the code that throws an unhandled exception and invokes clr to get watson bucket info.

using System;
using System.Runtime.InteropServices;

namespace WatsonTest
{
 internal class Test
 {
 private static void Main(string[] args)
 {
 var t = AppDomain.CurrentDomain;
 t.UnhandledException += (s, e) =>
 {
 var c = GetWatsonBuckets();
 Console.WriteLine(c);
 Console.Read();
 };
 throw new NullReferenceException();
 }
 private static WatsonBuckets GetWatsonBuckets()
 {
 var pParams = new WatsonBuckets();
 IClrRuntimeHost host = null;
 host = Activator.CreateInstance(Type.GetTypeFromCLSID(ClrGuids.ClsIdClrRuntimeHost)) as IClrRuntimeHost;
 if (host != null)
 {
 var clrControl = host.GetCLRControl();
 if (clrControl == null)
 {
 return pParams;
 }
 var clrErrorReportingManager =
 clrControl.GetCLRManager(ref ClrGuids.IClrErrorReportingManager) as IClrErrorReportingManager;
 if (clrErrorReportingManager == null)
 {
 return pParams;
 }
 clrErrorReportingManager.GetBucketParametersForCurrentException(out pParams);
 }
 return pParams;
 }
 }
 // BucketParameters Structure to get watson buckets back from CLR
 //http://msdn.microsoft.com/en-us/library/ms404466(v=VS.100).aspx
 [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Auto)]
 internal struct WatsonBuckets
 {
 internal int fInited;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string pszEventTypeName;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param0;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param1;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param2;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param3;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param4;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param5;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param6;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param7;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param8;
 [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 0xff)] internal string param9;
 }

 internal static class ClrGuids
 {
 internal static readonly Guid ClsIdClrRuntimeHost = new Guid("90F1A06E-7712-4762-86B5-7A5EBA6BDB02");
 internal static Guid IClrErrorReportingManager = new Guid("980D2F1A-BF79-4c08-812A-BB9778928F78");
 internal static readonly Guid IClrRuntimeHost = new Guid("90F1A06C-7712-4762-86B5-7A5EBA6BDB02");
 }

 [Guid("90F1A06C-7712-4762-86B5-7A5EBA6BDB02"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
 internal interface IClrRuntimeHost
 {
 void Start();
 void Stop();
 void SetHostControl(IntPtr pHostControl);
 IClrControl GetCLRControl();
 void UnloadAppDomain(int dwAppDomainId, bool fWaitUntilDone);
 void ExecuteInAppDomain(int dwAppDomainId, IntPtr pCallback, IntPtr cookie);
 int GetCurrentAppDomainId();

 int ExecuteApplication(string pwzAppFullName, int dwManifestPaths, string[] ppwzManifestPaths,
 int dwActivationData, string[] ppwzActivationData);

 int ExecuteInDefaultAppDomain(string pwzAssemblyPath, string pwzTypeName, string pwzMethodName,
 string pwzArgument);
 }

 [Guid("9065597E-D1A1-4fb2-B6BA-7E1FCE230F61"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
 internal interface IClrControl
 {
 [return: MarshalAs(UnmanagedType.IUnknown)]
 object GetCLRManager([In] ref Guid riid);

 void SetAppDomainManagerType(string pwzAppDomainManagerAssembly, string pwzAppDomainManagerType);
 }
 // IClrErrorReportingManager to get watson bukets back from CLR
 //http://msdn.microsoft.com/en-us/library/ms164367(v=VS.100).aspx
 [Guid("980D2F1A-BF79-4c08-812A-BB9778928F78"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
 internal interface IClrErrorReportingManager
 {
 [PreserveSig]
 int GetBucketParametersForCurrentException(out WatsonBuckets pParams);
 }
}

The watson bucket information has ten items

  1. AppName
  2. Version
  3. ?
  4. Assembly and Module Name
  5. Assembly Version
  6. ?
  7. MethodDef
  8. IL Offset
  9. Exception Type
  10. ?

In the above code WatsonBuckets is a structure to get watson bucket information back from CLR.  In the code I use 3 hosting interfaces, IClrRuntimeHost which is the main interface, the next interface is IClrControl which has GetCLRManager method to get ErrorReportingManger. IClrErrorReportingManager which has the method to get the exception buckets which is GetBucketParametersForCurrentException. I know that there isn’t going to be much of use for this code. But it gives me better understanding of  CLR integrates with watson when ever there is termination of an application.

Exploring SOSEX and Windbg to debug .NET 4.0


With the latest release of sosex comes a new set of functions to debug. It is pretty awesome that one person (Steve) alone could pull of such cool things. In this blog post, I am just going to demonstrate how easy it is to debug managed code using sosex compared to sos.

using System;
using System.Collections.Generic;

namespace MemCheck
{
 internal class Test
 {
 Dictionary<int, string> dict = new Dictionary<int, string>();
 private static void Main(string[] args)
 {
 var p = new Test();
 for (int i = 0; i < 100; i++)
 {
 p.dict.Add(i, i.ToString());
 }
 Console.WriteLine("Done");
 Console.Read();
 }
 }
}

I like to keep the code simple , so it is easy to follow.  The debugging goal for today is to get the Dictionary values. First I am going to demonstrate it using sos and then using sosex.

As usual I start the app and then attach it to windbg.

.loadby sos clr

FYI in  .net 4.0 clr is the dll that has CLR implementation. In prior versions it used to be in mscorwks. The next command would look for the object Test in the memory

!dumpheap -type MemCheck.Test
0:000> !dumpheap -type MemCheck.Test
 Address               MT     Size
0000000002761e20 000007ff00054110       24
 total 0 objects
 Statistics:
 MT    Count    TotalSize Class Name
 000007ff00054110        1           24 MemCheck.Test
 Total 1 objects

The next step is to dump the object

0:000> !do 0000000002761e20
Name:        MemCheck.Test
MethodTable: 000007ff00054110
EEClass:     000007ff00162350
Size:        24(0x18) bytes
File:        C:\Users\naveen\Documents\Visual Studio 2010\Projects\Test\bin\Debug\Test.exe
Fields:
 MT    Field   Offset                 Type VT     Attr            Value Name
000007feec2b7a48  4000001        8 ...tring, mscorlib]]  0 instance 0000000002761e38 dict

Notice the dict object is in the 8th offset . To dump contents  dict object I would use the command !do poi(0000000002761e20+8) , which is pointer deference of Test object on it is 8th offset.  And here is the output

0:000> !do poi(0000000002761e20+8)
Name:        System.Collections.Generic.Dictionary`2[[System.Int32, mscorlib],[System.String, mscorlib]]
MethodTable: 000007feec2b7a48
EEClass:     000007feebe113c0
Size:        88(0x58) bytes
File:        C:\Windows\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll
Fields:
 MT    Field   Offset                 Type VT     Attr            Value Name
000007feec27c7d8  4000bee        8       System.Int32[]  0 instance 0000000002764788 buckets
000007feecbd3dc8  4000bef       10 ...non, mscorlib]][]  0 instance 0000000002764ab8 entries
000007feec27c848  4000bf0       40         System.Int32  1 instance              100 count
000007feec27c848  4000bf1       44         System.Int32  1 instance              100 version
000007feec27c848  4000bf2       48         System.Int32  1 instance               -1 freeList
000007feec27c848  4000bf3       4c         System.Int32  1 instance                0 freeCount
000007feec2a5a48  4000bf4       18 ...Int32, mscorlib]]  0 instance 0000000002761ef0 comparer
000007feecc75f78  4000bf5       20 ...Canon, mscorlib]]  0 instance 0000000000000000 keys
000007feecc72078  4000bf6       28 ...Canon, mscorlib]]  0 instance 0000000000000000 values
000007feec275ab8  4000bf7       30        System.Object  0 instance 0000000000000000 _syncRoot
000007feec29a1b8  4000bf8       38 ...SerializationInfo  0 instance 0000000000000000 m_siInfo

And the dictionary object in turn stores them within an array which is again the 8th offset.  This time because we know it is an array we are going to use the !dumparray command on the memory location. The command to get the details is

!dumparray -details poi(poi(0000000002761e20+8)+8)
MT    Field   Offset                 Type VT     Attr            Value Name
 000007feec27c848  400047b        0             System.Int32      1     instance                   -1     m_value
 [195] 0000000002764aa4
 Name:        System.Int32
 MethodTable: 000007feec27c848
 EEClass:     000007feebe00890
 Size:        24(0x18) bytes
 File:        C:\Windows\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll
 Fields:
 MT    Field   Offset                 Type VT     Attr            Value Name
 000007feec27c848  400047b        0             System.Int32      1     instance                   -1     m_value
 [196] 0000000002764aa8
 Name:        System.Int32
 MethodTable: 000007feec27c848
 EEClass:     000007feebe00890
 Size:        24(0x18) bytes
 File:        C:\Windows\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll
 Fields:
 MT    Field   Offset                 Type VT     Attr            Value Name
 000007feec27c848  400047b        0             System.Int32      1     instance                   -1     m_value

Here is the partial output.

Now lets try and do the same thing using sosex. The one thing that I really like about  the new sosex is that  I can use names rather than pointer deference which is way much easier. Launched the app and then loaded sosex using the command

.load F:\Work\Tools\debuggers\sosex.dll

and then switched the thread from 4th to 0th thread using ~0s.  By default the debugger injects a thread into the process for debugging and that was the 4th thread. The next command I issued was to get stack trace

!mk
0:000> !mk
Thread 0:
 ESP              EIP
00:U 000000000015e408 0000000077bc00da ntdll!ZwRequestWaitReplyPort+0xa
01:U 000000000015e410 0000000077a72b08 KERNEL32!ConsoleClientCallServer+0x54
02:U 000000000015e440 0000000077aa5601 KERNEL32!ReadConsoleInternal+0x1f1
03:U 000000000015e590 0000000077aba922 KERNEL32!ReadConsoleA+0xb2
04:U 000000000015e670 0000000077a89934 KERNEL32!zzz_AsmCodeRange_End+0x8bea
05:U 000000000015e6b0 000007feed0317c7 clr!DoNDirectCall__PatchGetThreadCall+0x7b
06:M 000000000015e760 000007feec1d34a1 DomainNeutralILStubClass.IL_STUB_PInvoke(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte*, Int32, Int32 ByRef, IntPtr)(+0x0 IL)(+0x0 Native)
07:M 000000000015e880 000007feec97f59a System.IO.__ConsoleStream.ReadFileNative(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte[], Int32, Int32, Int32, Int32 ByRef)(+0x53 IL)(+0xba Native)
08:M 000000000015e8f0 000007feec97f402 System.IO.__ConsoleStream.Read(Byte[], Int32, Int32)(+0x5d IL)(+0x62 Native)
09:M 000000000015e950 000007feec18e63c System.IO.StreamReader.ReadBuffer()(+0xa0 IL)(+0x5c Native)
0a:M 000000000015e9a0 000007feec915630 System.IO.StreamReader.Read()(+0x21 IL)(+0x30 Native)
0b:M 000000000015e9e0 000007feec987458 System.IO.TextReader+SyncTextReader.Read()(+0x0 IL)(+0x38 Native)
0c:M 000000000015ea30 000007ff00170213 MemCheck.Test.Main(System.String[])(+0x39 IL)(+0xf3 Native) [C:\Users\naveen\Documents\Visual Studio 2010\Projects\Test\Program.cs, @ 17,13]
0d:U 000000000015eaa0 000007feed0710b4 clr!CallDescrWorker+0x84
0e:U 000000000015eaf0 000007feed0711c9 clr!CallDescrWorkerWithHandler+0xa9
0f:U 000000000015eb70 000007feed071245 clr!MethodDesc::CallDescr+0x2a1
10:U 000000000015eda0 000007feed171675 clr!ClassLoader::RunMain+0x228
11:U 000000000015eff0 000007feed1717ac clr!Assembly::ExecuteMainMethod+0xac
12:U 000000000015f2a0 000007feed171562 clr!SystemDomain::ExecuteMainMethod+0x452
13:U 000000000015f850 000007feed173dd6 clr!ExecuteEXE+0x43
14:U 000000000015f8b0 000007feed173cf3 clr!CorExeMainInternal+0xc4
15:U 000000000015f920 000007feed1f7365 clr!CorExeMain+0x15
16:U 000000000015f960 000007fef8f13309 mscoreei!CorExeMain+0x41
17:U 000000000015f990 000007fef8fa5b21 MSCOREE!CorExeMain_Exported+0x57
18:U 000000000015f9c0 0000000077a6f56d KERNEL32!BaseThreadInitThunk+0xd
19:U 000000000015f9f0 0000000077ba3281 ntdll!RtlUserThreadStart+0x1d

FYI the command !mk has been part of sos from the initial version. I am interested in only looking at the code that I wrote so I would like to move stack frame to 0c which is MemCheck.Test.Main . To do that the command  is !mframe 0c, which moves to that stackframe. The reason to move the particular stack frame is to look for variables in the stack and the command to variables is !mdv , which display managed local variables

0:000> !mdv
Frame 0xc: (MemCheck.Test.Main(System.String[])):
[A0]:args:0x0000000002761dd8 (System.String[])
[L0]:p:0x0000000002761e20 (MemCheck.Test)
[L1]:i:0x0000000000000064 (System.Int32)
[L2]:CS$4$0000:0x0000000000000000 (System.Boolean)

Notice we see the local variable “p” which is of type MemCheck.Test. To display type p we issue the command !mdt p


0:000> !mdt p

0000000002761e20 (MemCheck.Test)

dict:0000000002761e38 (System.Collections.Generic.Dictionary`2[[System.Int32, mscorlib],[System.String, mscorlib]])

I didn’t  have to get memory address , I am using the names which is very intuitive ,especially when we have to debug large application with N levels of nesting.So to get the dict values   from p the command to issue is !mdt -e p.dict

!mdt -e p.dict
[98] (System.Collections.Generic.Dictionary`2+Entry[[System.Int32, mscorlib],[System.String, mscorlib]]) VALTYPE (MT=000007feec2b7b28, ADDR=0000000002765400)
 key:0x62 (System.Int32)
 value:0000000002765e48 (System.String: "98")
[99] (System.Collections.Generic.Dictionary`2+Entry[[System.Int32, mscorlib],[System.String, mscorlib]]) VALTYPE (MT=000007feec2b7b28, ADDR=0000000002765418)
 key:0x63 (System.Int32)
 value:0000000002765e68 (System.String: "99")

Here is the partial output. Notice I never had to use a memory pointer or do a pointer deference .This is very similar to VS.NET debugging where I am used to the variable names compared the memory address.  Thanks to Steve for providing such a cool extension.

Exploring internal implementation of C# 4.0 Default and Optional arguments


I wanted to explore how C#  compiler has implemented the optional and the default parameter. So here is the sample code that I wrote to verify.

using System;
namespace OptionalTest {
 internal class Program {
 private static void Main(string[] args) {
 var p = new Program();
 Test();
 }
 private static void Test(int x = 10) {
 Console.WriteLine(x);
 }
 }
}

As you can see the default parameter value for x is 10 and it is an optional parameter. After compiling the source code the next step was to disassemble with Reflector.

private static void Test([Optional, DefaultParameterValue(10)] int x){
Console.WriteLine(x);
}

Now it makes sense, looks like the C# compiler emits a custom attribute for System.Runtime.InteropServices.OptionalAttribute which marks the method to have an optional argument and the DefaultParameterValue stores the default value for the parameter. The next step was to disassemble the caller of this function.

And here is the disassembled Main method using reflector

private static void Main(string[] args)
{
Program p = new Program();
Test(10);
}

Notice we didn’t include 10 as an argument to method Test. So the compiler notices that the calling code does not have value for the argument and copies the value from the DefaultParameterValue and sticks it in.

Using LINQPad , PLINQ to grep for files


I use linqpad as my primary dev tool for iterative code development. Today I had to search the source code tree on my hard disk for certain keywords that were present in database. There is big impedance mismatch between database and the rest of the world. I would have to fetch the data from DB and had to use either powershell or cmd to look for them. This is fine if I have to look for one keyword the issue if I have few of them then it becomes a bigger issue.  I remember looking at a cool example from PFX about using parallel grep. Here is the code I modified it to include optional parameters and return a type


public class GrepResult
 {
 public string File {get;set;}
 public int Line {get;set;}
 public string Text {get;set;}
 }
 public static class Extension
 {
 public static IEnumerable<GrepResult> Grep(string regexString, IEnumerable<string> wildcards,bool ignoreCase = true, bool recursive = false)
 {
 var regex = new ThreadLocal<Regex>(() =>
 new Regex(regexString, RegexOptions.Compiled | (ignoreCase ? RegexOptions.IgnoreCase : RegexOptions.None)));

 var files = from wc in wildcards
 let dirName = Path.GetDirectoryName(wc)
 let fileName = Path.GetFileName(wc)
                from file in Directory.EnumerateFiles(
                String.IsNullOrWhiteSpace(dirName) ? "." : dirName,
                String.IsNullOrWhiteSpace(fileName) ? "*.*" : fileName,
                recursive ? SearchOption.AllDirectories : SearchOption.TopDirectoryOnly)
                select file;

 var matches = from file in files.AsParallel(). AsOrdered().WithMergeOptions(ParallelMergeOptions.NotBuffered)
               from line in File.ReadLines(file).Zip(Enumerable.Range(1, int.MaxValue), (s, i) => new { Num = i, Text = s, File = file })
               where regex.Value.IsMatch(line.Text)
               select line;
 foreach (var line in matches)
 {
      yield return new GrepResult() {File = line.File, Line = line.Num,Text = line.Text} ;
 }
 }
 }

I would compile this into a dll and add reference to linqpad.  And here is the code that actually using it


static void Main()
 {
 Environment.SetEnvironmentVariable("LinqQueries",@"c:\Users\naveen\Documents\LINQPad Queries\*.*");
 var linqPath = Environment.GetEnvironmentVariable("LinqQueries");
 var x = from i in ConfigEntries
 select i.Name;
 x.Select (y => Extension.Grep(y.Name,new [] {linqPath})).Dump();
 }

I usually set environment variables (I am setting the env variable just for demonstration) for my projects so that I don’t have type the path manually. In the above code I am getting the values from database and searching those keywords within my hard disk. I could have use the Findstr method via Util class within LINQPad , but the PLINQ  implementation can use all my quad cores. I search for code very often and this is going to be handy.

Using LINQ and Reactive Extensions to overcome limitations in OData query operator


I was pleased to know that Netflix had OData API to query. The practical reason is obviously was to use the API to query for the movies I want to watch. Like I mentioned in my previous post, I will be using LINQPad 4 for querying purposes, because of its built-in capabilities for OData as well as for Rx.

One thing I discovered after playing around with OData is that not every query operator in LINQ is available in OData. For example the Netflix API has only for 4 operators which are

  1. Filter
  2. Skip
  3. Take
  4. Orderby

And also the query returns only 20 rows as the result for each request. So for example if I have to get 40 rows, on my first request  the server would return 20 rows and in my next request I would have to skip first 20 and take next 20 to get 40 rows. These are some of the limitations.

Here is what I wanted from Netflix, I wanted to movie listings that has an average rating greater than 3.5 ,ordered by release year descending and grouped by listings that are available for instant watch.  So that I can have one queue for movies that I want to watch online and another one that I can request via mail (the ones that is not available in instant watch).  And here is the query to do that


 var movies = from counter in (from e in Enumerable.Range(0,400) where e%20  == 0 select e).ToObservable()
 from movieTitle in Titles.Where (t => t.AverageRating > 3.5).OrderByDescending (t => t.ReleaseYear).Skip(counter).Take(20).ToObservable()
 select movieTitle;

var moviesILikeToWatch = from counter in movies
 group counter by counter.Instant.Available into g
 select g;
moviesILikeToWatch.Dump();

The first “from counter” query is to build the skip part, like I mentioned by default the  result returns only 20 rows I wanted 400 rows to achieve that I used the enumerable range to generate sequence that I can use for skipping in my next query. I could have very well used for loop to build this, but that is not what I want. I want to try and write terse code. These are actual calls to Netflix OData  API

http://odata.netflix.com/Catalog/Titles()?$filter=AverageRating gt 3.5&$orderby=ReleaseYear desc&$skip=0&$top=20
http://odata.netflix.com/Catalog/Titles()?$filter=AverageRating gt 3.5&$orderby=ReleaseYear desc&$skip=20&$top=20

In the below picture linqpad makes 20 calls to Netflix for getting 400 movie listings

The next line in the first  query “from movieTitle” is simple Linq query to get movies based on filter criteria along with skip and take. The reason for the second query is because the OData  API doesn’t provide a groupby operator and if I include it in my first query , Linqpad would try and convert it to OData specific  request which would fail. So essentially I am getting all the data from the server and then grouping it locally.

This wouldn’t have been possible without OData.

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: