Technical Is As Technical Does - Hugh Ang's Blog

How to Make Tracing and Caching Providers Work for EF 4.1 Code First

2011-12-21T21:08:00.005-05:00

The tracing and caching providers (or here) provide an excellent way to trace and cache EF queries. Unlike those that attempt to do the same above the EF API, these providers work under the covers and make tracing and caching non-invasive for the application code. Specifically, for cases where entities are lazy-loaded via EF and those entities are to be cached, this caching provider is truly more advantageous.

However, these providers were written before EF 4.1 code first and they wouldn't work if you just follow the old sample. Luckily, it is not too difficult to make them work. The trick is to create a wrapping EFCachingConnection or EFTracingConnection that wraps a DbConnection (such as SqlConnection) and pass it to the DbContext constructor. I will just demonstrate the caching related code here but you should be able to chain tracing, caching and then the underlying SqlConnection using the same pattern.

First, a minor modification to the EFCachingConnection constructor is needed. As shown in the following code, we need to be able to set the wrappedProviderInvariantName via the ctor:

 
        public EFCachingConnection(DbConnection wrappedConnection, string wrappedProviderInvariantName)

        {

            this.WrappedConnection = wrappedConnection;

            this.CachingPolicy = EFCachingProviderConfiguration.DefaultCachingPolicy;

            this.Cache = EFCachingProviderConfiguration.DefaultCache;

            base.wrappedProviderInvariantName = wrappedProviderInvariantName; // make it possible to set the wrappedProviderInvariantName via this ctor

        }

The I use the following helper function to create an instance of the EFCachingConnection that wraps a SqlConnection:

 
        public static EFCachingConnection CreateEFCachingConnectionWrapper(string nameOrConnectionString)

        {

            string connectionString;

            if (nameOrConnectionString.Contains("="))

                connectionString = nameOrConnectionString;

            else

                connectionString = ConfigurationManager.ConnectionStrings[nameOrConnectionString].ConnectionString;

            DbConnection connection = new SqlConnection(connectionString);

            return new EFCachingConnection(connection, "System.Data.SqlClient"); 

        }

Now you would simply pass the newly constucted EFCachingConnection instance to the ctor of DbContext:

 
        public MyContext(DbConnection connection)

            : base(connection, true)

        {

Note that MyContext is the application's EF code first context, which derives from DbContext and has probably overriden the OnModelCreating() method.

Last and not the least you would need to call EFCachingProviderConfiguration.RegisterProvider() first when you start your application.

Happy holidays and happy EF coding!

Recording VS.NET Web Performance Test for Flex

2011-03-01T14:36:00.011-05:00

We are building a web application that integrates with a third party software by embedding their Flex objects in our ASPX pages. When we started using VS.NET 2010 Web Performance Test, everything worked fine except for the weborb requests made by the Flex objects to the server. Basically, the requests recorded by the web test, when played back and sent to the server, were causing the server to respond with errors. From googling, it looks like that the same issue has been experienced by other folks as well. At this point, it would be easy to simply dismiss VS.NET 2010 Web Performance Test as a viable tool for Flex. What a costly mistake this would have been if we had stopped doing more analysis!

Looking at the initially recorded tests, I found out that all the data posted by the Flex objects was recorded and played back as string data type. Examining the payload in Fiddler, however, showed that the data is obviously binary. So our recorded test was sending the data with lost fidelity and the server just choked.

Thinking that there must be configuration to tweak to make our tests record binary data, I started searching MSDN for documentation. Sure enough, I found this link. I went ahead and added "application/x-amf" as the content type. Then I recorded the test again and ran the scripts.

Voila, everything worked like a charm!

Fixing My Custom Built PC After a Power Outage

2010-05-10T22:09:00.004-04:00

I usually wouldn't blog about hardware but since this experience felt just like debugging a software problem, here it goes:

The NJ area experienced a pretty bad wind storm this past weekend. Downed tree branches knocked out some power lines, which caused scattered power outages in the area. My house was one of the unlucky ones that were hit with the outage. After the power came back in the next morning, one of my desktop PCs that I built several years ago wouldn't start. This is the PC that I have just recently upgraded from Vista to Windows 7 and I also built inside the host PC a VPC with VS.NET 2010 and all the other latest cool stuff (Silverlight, etc.). So I was determined to figure out what went bad and hoped to salvage as much as possible.

The symptom of the problem is that initially it wouldn't even turn on when I pressed the power button. After I switched to a different circuit in the house, it could start with fans and lights on. So it was evident that the PSU was still good. But there was no display on the monitor and no system start beep sound either. What was even worse was I could smell burning plastic. Some board was fried during the outage and now with the power on, sending the current through the board is still burning it.

I turned off the power and began visually inspecting the mobo, hoping to find where the burn was but to no avail. I started disconnecting the IDE cable to the DVD rom, SATA cables to the HDD, unplugging RAM. I would make one change at a time while powering on to see if there is a system beep. Nothing so far.

The last item was the video card with dual DVI interfaces for my two monitors. After I unplugged the video card, bingo! the system started making beeping sounds. And as I flipped the video card over, sure enough I found the burning spot, which I wouldn't have been able to see when it was installed closely to the bottom of the tower in the facing down position. Fortunately I had a spare video card. After plugging that in I had no problem booting into Windows 7. It did require resetting the resolution after automatically installing the driver for the card. Minutes later, I noticed that automatic Windows update was complaining about not able to complete the updates and a closer look revealed that the system clock was set to 1/1/2006. I suspect this was done as I was resetting the BIOS during the diagnosis process - while the power cord was unplugged, I pressed and held the power button for a few moments.

So fortunately it was only a small loss instead of the MOBO/CPU that are commonly getting fried during power outages. One thing I will be doing for sure going forward is to use the battery backup surge protector as opposed to a low end one I was using :-)

Application Migration - Vista/Win7 Compatibility

2009-12-16T01:07:00.027-05:00

It has been a while since the last time I blogged. The theme of my work for the last 4 months or so is really migration, migration, and migration!

First I helped our team with a VB6 to VB.NET migration project that ran into some sticky interop problems.

Secondly I worked as an advisor for a customer who was in the process of migrating their apps from Win2K to Vista. Please don't ask me why they chose to go to Vista while Win7 was coming out - it was a decision already rationalized by the management when I was brought in.

And lastly, just a couple of weeks ago, I helped Microsoft deliver a Win7 training course for a customer who is looking to migrate to Win7 in the next 6 months. Although it is not unusual that I present in a conference room or mentor developers privately, this was actually my first time to do a formal training. The sessions were recorded and are being made available for later reuse by the customer. Speaking of the sessions on tape, I actually saw Oliver Stone and Michael Douglas when they were shooting Wall Street 2 on site at the Borders book store at the corner of Wall Street and Broadway. And that was when I was working on the Vista migration mentioned above.

Enough with the chronology and my celebrity sightings. As the title suggests, I'd like to cover the most common compatibility issues for Vista/Win7from a developer perspective. So here are the two top issues I have seen so far:

Writing to %ProgramFiles% (e.g. C:\Program Files) and %SystemRoot% (e.g. C:\Windows) locations

Writing to registry hive: HKLM\Software

It is quite common that legacy apps write configuration data to the above locations. But with UAC enabled in Vista/Win7, an admin user will be running an app with standard user privileges unless the app has a manifest explicitly requesting admin privilege or the app is started with "Run as administrator" context menu; And since standard user will only have read/execute rights to those locations, the attempt made by those apps to write configuration data to those locations will fail. Using the icacls at the command prompt shows the ACL of %ProgramFiles% as in the following on my machine:


C:\Program Files>icacls .
. NT SERVICE\TrustedInstaller:(F)
  NT SERVICE\TrustedInstaller:(CI)(IO)(F)
  NT AUTHORITY\SYSTEM:(M)
  NT AUTHORITY\SYSTEM:(OI)(CI)(IO)(F)
  BUILTIN\Administrators:(M)
  BUILTIN\Administrators:(OI)(CI)(IO)(F)
  BUILTIN\Users:(RX)
  BUILTIN\Users:(OI)(CI)(IO)(GR,GE)
  CREATOR OWNER:(OI)(CI)(IO)(F)

There is an additional twist here. The app may not be failing but you will not see files or registry keys written to the intended location. This is the work of data redirection(or UAC virtualization). The details can be found here. But basically:

Writing to %ProgramFiles%\MyApp\myapp.ini gets redirected to C:\Users\Username\AppData\Local\VirtualStore\Program Files\MyApp\myapp.ini

Writing to HKLM\Software\MyApp\ gets redirected to HKCU\Software\Classes\VirtualStore\MACHINE\Software\MyApp

When virtualization happens, you can fire up Process Monitor and notice a REPARSE event, followed by writing to those redirected locations.

As the redirection is on per-user basis, this can mess up the intended behavior of the application. For example, the application may be intended to be used by multiple users on one machine and the configuration data should be shared. Microsoft is making no guarantee that the virtualization will be in future Windows. So what to do as a developer? The best practice for writing per machine configuration data is using %ProgramData% (e.g. C:\ProgramData). For per user configuration data, use %LocalAppData% (e.g. C:\Users\User\AppData\Local) for local or %AppData% for roaming user profile.

Running icacls on %ProgramData% shows that a standard user would have additional AD(append data/add subdirectory), WD(write data/add file), WEA(write extended attributes) and WA(write attributes) privileges:


C:\ProgramData>icacls .
. NT AUTHORITY\SYSTEM:(OI)(CI)(F)
  BUILTIN\Administrators:(OI)(CI)(F)
  CREATOR OWNER:(OI)(CI)(IO)(F)
  BUILTIN\Users:(OI)(CI)(RX)
  BUILTIN\Users:(CI)(WD,AD,WEA,WA)

And lastly, don't hard code any of the paths we have talked about. There are well-known APIs to get them. In .NET, follow this sample:

string path = Environment.GetFolderPath(Environment.SpecialFolder.CommonApplicationData);

path = Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData);

path = Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData);

And if you write C++ code, here is a sample:

void ShowFolders(HWND hWnd)

{    

    LPWSTR wszPath = NULL;

    HRESULT hr = SHGetKnownFolderPath(FOLDERID_ProgramData, KF_FLAG_NO_ALIAS | KF_FLAG_DONT_VERIFY, NULL, &wszPath); 

    if ( SUCCEEDED(hr) )

    {

        // work with wszPath

        CoTaskMemFree(wszPath);

    }

    hr = SHGetKnownFolderPath(FOLDERID_LocalAppData, KF_FLAG_NO_ALIAS | KF_FLAG_DONT_VERIFY, NULL, &wszPath); 

    if ( SUCCEEDED(hr) )

    {

        // work with wszPath        

        CoTaskMemFree(wszPath);

    }

    hr = SHGetKnownFolderPath(FOLDERID_RoamingAppData, KF_FLAG_NO_ALIAS | KF_FLAG_DONT_VERIFY, NULL, &wszPath); 

    if ( SUCCEEDED(hr) )

    {

        // work with wszPath    

        CoTaskMemFree(wszPath);

    }

}

Hotfix KB971030 Released

2009-09-02T16:58:00.004-04:00

You can finally download this hotfix from here and the detailed information is available here. According to the information posted there, it is a fix for an access violation problem when making virtual function calls to the IList<T>, IEnumerable<T>, or ICollection<T> interface. I am not sure yet if this is another manifestation of the same dispatch token problem I described in my previous blog or the CLR team simply bundled together several virtual dispatch bug fixes that just include what I found.

I ran the same test harness that I used before and confirmed that the hotfix does indeed resolve the problem.

I was having some trouble installing the hotfix today but eventually got it installed. It turned out that my OS was Vista SP1 and I had to upgrade it to SP2 in order to install the hotfix. If you have another OS and wondering about the same question as Chris Dion had, the hotfix should be applicable to Windows Server 2000, Windows Server 2003, Windows Server 2008 and Windows XP.

CLR Virtual Call Stub Issue Resolved

2009-05-15T13:51:00.002-04:00

Good news - I have just received words from the Microsoft CLR team that the issue I detailed in this blog has been fixed in an upcoming QFE, which will be available for download through the Support/KB site using KB971030 in about a week. This also means that the fix will be automatically included in any future 3.5 service pack release. In addition, this has been fixed in the upcoming 4.0 release. I will try the fix in a week and keep everyone posted.

Making Assumptions in Application Design

2009-02-18T11:46:00.005-05:00

Back during the holidays, I wrote a small .NET program to manipulate my Canon XTi camera, which was connected to the PC via USB. Everything worked great when I was using my laptop that had Vista on it. But on my wife's laptop that had XP, the performance became unbearable. My utility program would wait for several minutes before I could start using it. The program's thread was pretty much idle during the waiting so I was asking myself: could this be the PC hardware? I tested this program on one of my desktops that also had XP. Nope, still the same poor performance. I then updated the camera's firmware and that didn't resolve the issue either. I began to suspect that this was OS related. Sure enough, when I plugged in the camera without my program running, XP took a much longer time than Vista to come up with the camera and scanners wizard. I noticed that while XP seemed to be cluelessly waiting, the CF card indicator light of the camera was flashing madly. I realized that XP was probably trying to download pictures from the camera, which was busy reading from the CF card. At the time, I had about 300 high resolution pictures on the 2GB CF card. After I switched it with an empty CF card, the performance problem just went away.

The morale of the story is that we need to be extra careful when making assumptions for users when designing UI applications. In this case, XP made a not necessarily wise assumption about the intention of plugging a camera into the USB port. Even for users who do want to download pictures this way, there is no visual clue provided, leaving users wonder what is going on.

Reverse P/Invoke - Part 2

2008-10-14T12:29:00.011-04:00

In my last post, I described a loosely-coupled pattern for native code to call into managed code. That approach requires control of source code of the native library, although only minor change would be made. There are scenarios where native code is not available and we just can not make any changes to it. For example, I have a Canon Rebel XTi camera connected to my PC and I need to notify my Winforms application of a new picture just taken so that the application can download and display it. The Canon SDK is in native library and has exported functions for callback function pointers to be registered. So what do we do? Scenarios like this require us to call native API from managed code, passing delegates marshaled as function pointers.

The solution is surprisingly straightforward. To demonstrate the technique, here is the native library code - note the definitions of data structure and callback function prototype:

#define NATIVELIB_API __declspec(dllexport)
 
// data structure for the callback function
struct EventData
{
    int I;
    TCHAR* Message; 
};
 
// callback function prototype
typedef void (*FPCallBack)(EventData data);
 
// exported API
NATIVELIB_API void fnNativeAPI(int i, FPCallBack callBack)
{
    EventData data;
    data.I = i;
    data.Message = L"Hello from native code!";
 
    // invoke the callback function
    callBack(data);
}

And here is my corresponding interop code in C#. Notice that in the P/Invoke definition of the fnNativeAPI call, I added the MarshalAs(UnmanagedType.FunctionPtr) attribute in front of the CallBackDelegate parameter to instruct the runtime to marshal the delegate as a function pointer from managed code to native code.

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
public struct EventData
{
    public int I;
    public string Message;
}
 
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
public delegate void CallBackDelegate(EventData data);
 
public class Native
{
    [DllImport("NativeLib.dll")]
    public static extern void fnNativeAPI(int i, [MarshalAs(UnmanagedType.FunctionPtr)] CallBackDelegate callBack);
}

Here is the C# code of the application:

public partial class Form1 : Form
{
    private CallBackDelegate _cb;
 
    public Form1()
    {
        InitializeComponent();
        _cb = new CallBackDelegate(Foo);
    }
 
    private void Form1_Load(object sender, EventArgs e)
    {
        // call the native API, passing our .NET delegate
        int i = 200;
        Native.fnNativeAPI(i, _cb);
    }
 
    // this is our callback function
    private void Foo(EventData data)
    {
        Debug.WriteLine(data.I);
        Debug.WriteLine(data.Message);
    }
}

Careful readers may have noticed that I have kept the delegate as a class field. This is a best practice to prevent the delegate from being garbage-collected. It would be really bad if the native code holds an invalid function pointer and tries to invoke it.

For VB programmers, here is the interop code in VB.NET:

<StructLayout(LayoutKind.Sequential, CharSet:=CharSet.Unicode)> _
Public Structure EventData
    Public I As Integer
    Public Message As String
End Structure
 
<UnmanagedFunctionPointer(CallingConvention.Cdecl)> _
Public Delegate Sub CallBackDelegate(ByVal data As EventData)
 
Public Class Native
    <DllImport("NativeLib.dll")> _
    Public Shared Sub fnNativeAPI(ByVal i As Integer, _
                                  <MarshalAs(UnmanagedType.FunctionPtr)> ByVal callBack As CallBackDelegate)
    End Sub
End Class

And the application code:

Public Class Form1
    Dim _cb As CallBackDelegate
 
    Public Sub New()
 
        ' This call is required by the Windows Form Designer.
        InitializeComponent()
 
        ' Add any initialization after the InitializeComponent() call.
        _cb = New CallBackDelegate(AddressOf Me.Foo)
    End Sub
 
    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        Dim i As Integer = 200
        'call the native API, passing our .NET delegate
        Native.fnNativeAPI(i, _cb)
    End Sub
 
    ' this is our callback function
    Private Sub Foo(ByVal data As EventData)
        Debug.WriteLine(data.I)
        Debug.WriteLine(data.Message)
    End Sub
End Class

Reverse P/Invoke

2008-09-30T19:24:00.000-04:00

While researching for ongoing/upcoming projects that will need approaches for interop between .NET and native code, specifically, for native code to call into .NET code, Reverse P/Invoke has come to my attention as a viable option. Of course there is the official Microsoft recommendation to expose .NET classes as COM components, which are then callable from native code that talks COM.

The Reverse P/Invoke approach allows native code to call into .NET delegate using a function pointer. So it could work well for my requirement, for which I need a way to fire an event from the native app to the .NET app, for instance, an application context change on the native side must be reflected on the .NET side.

The blog by Junfeng, however, does not give a concrete example of such Reverse P/Invoke approach. So I came up with a POC, where I had a VS.NET solution with three projects: (1) a native console application (C++ project) (2) a managed class library (C# project) and (3) a mixed mode dll library with exported C++ function (C++/CLI project).

So this POC is trying to simulate a native application (#1) that needs to notify managed code (#2) of data changes. I came up with a dll library compiled with /clr switch to handle the interop details. Both the native app and the managed code requires very minimum changes.

On the .NET side, we have a managed class that has a Foo() function and a GetDelegate() function that hands out a delegate to Foo to its caller.

public class ManagedClass
{
    private CallBackDelegate _delegate;
 
    public ManagedClass()
    {
        _delegate = new CallBackDelegate(this.Foo);
    }
 
    public CallBackDelegate GetDelegate()
    {
        return _delegate;
    }
 
    public void Foo(EventData data)
    {
        Debug.WriteLine(data.I);
        Debug.WriteLine(data.Message);
    }
}

The EventData is a data structure that shares the same binary layout as the one that will be created and marshaled from the native code.

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
public struct EventData
{
    public int I;
    public string Message;
}

And here is the delegate definition. Note the attribute UnmanagedFunctionPointer with the calling convention.

[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
public delegate void CallBackDelegate(EventData data);

In the mixed mode dll, here is the definition of the EventData data structure and function pointer:

#pragma once
 
#include <windows.h>
 
// data structure for the callback function
struct EventData
{
    int I;
    TCHAR* Message; 
};
 
// callback function prototype
typedef void (*NativeToManaged)(EventData data);

And the exported function is defined as in the following. Note how the .NET delegate gets invoked through the function pointer.

#define INTEROPBRIDGE_API __declspec(dllexport)
 
INTEROPBRIDGE_API void fnInteropBridge(EventData data)
{
    ManagedLib::ManagedClass^ c = gcnew ManagedLib::ManagedClass();
    IntPtr p = Marshal::GetFunctionPointerForDelegate(c->GetDelegate());
 
    NativeToManaged funcPointer = (NativeToManaged) p.ToPointer();
 
    // invoke the delegate
    funcPointer(data);
}

Now in the native app, I have code that creates a copy of EventData and invokes the .NET code through the exported dll function fnInteropBridge:

// forward definition of the API function
void fnInteropBridge(EventData data);
 
int _tmain(int argc, _TCHAR* argv[])
{
    EventData data;
    data.I = 50;
    data.Message = L"Hello from native code!";
 
    fnInteropBridge(data);
 
    return 0;
}

In summary, I like this approach it that it provides quite an easy and non-invasive way for native code to call into managed code. It should especially work well in my scenario, where application context changes initiated from the native app needs to be propagated to the managed code. Furthermore, besides polishing this up, I think I will add code to raise a .NET event from inside ManagedClass.Foo(). Then all interested .NET citizens on the managed app side can subscribe to it.

Follow-up of PIAB and WCF Article

2008-09-23T23:09:00.000-04:00

Since publishing the MSDN article on an approach to integrate PIAB into WCF, I have received quite a few feedback from folks using this approach in production. And I am glad to say it's working out successfully for those folks.

As I mentioned earlier in the comment, Tom Hollander of p&p group has found an issue with our approach when two PIAB-enabled WCF services are hosted in the same IIS worker process. He graciously shared his code with us and I was able to reproduce the problem when the two WCF services were hosted in two separate AppDomains of the same IIS worker process. Although in production deployment scenarios, different WCF services would likely be hosted in separate processes or on different machines, I have still been wanting to figure out the root cause of this particular issue. Unfortunately with work and other things in my life, I just haven't had time until last week when I was finally able to sit down and focused on this problem.

I have to admit this has been the most tedious debugging I have ever done as I was deep in the guts of WCF and CLR, inspecting dynamic types, JIT compiler, call stubs, etc. without a whole lot of background in this area - I only wish I were working for the CLR team :-) After a few days, I finally found out the cause of the problem, which is pretty close to my initial hunch. A sense of elation at last!

Here it goes.

1. The setup to repro the problem
The setup is fairly straightforward. There are two WCF services, each implementing IService and IAnotherService respectively (the code is pretty much verbatim from Tom):

[ServiceContract]
public interface IService
{
    [OperationContract]
    Foo GetFoo(int id);
 
    [OperationContract]
    void AddFoo(int i, [NotNullValidator] Foo foo);
}

[ServiceContract]
public interface IAnotherService
{
    [OperationContract]
    void LogFoo(Foo foo);
}

Both services would be enabled with PIAB of course:

[PolicyInjectionBehaviors.PolicyInjectionBehavior]
[ValidationCallHandler]
[LogCallHandler]
public class Service : IService
{
    private static Dictionary<int, Foo> store = new Dictionary<int, Foo>();
    IAnotherService anotherService;
 
    public Service()
    {
        BasicHttpBinding binding = new BasicHttpBinding();
        binding.SendTimeout = new TimeSpan(4, 0, 0);
 
        anotherService = new AnotherServiceClient(binding, new EndpointAddress("http://localhost/AnotherTestService/AnotherTestService.svc"));
    }
 
    public void AddFoo(int id, Foo foo)
    {
        store[id] = foo;
        anotherService.LogFoo(foo);
    }
 
    public Foo GetFoo(int id)
    {
        if (store.ContainsKey(id))
        {
            anotherService.LogFoo(store[id]);
            return store[id];
        }
        else
            return null;
    }
}

[PolicyInjectionBehaviors.PolicyInjectionBehavior]
[ValidationCallHandler]
[LogCallHandler]
public class AnotherService : IAnotherService
{
    public void LogFoo(Foo foo)
    {
        Logger.Write("LogFoo() called.");
    }
}

Foo is simply a DataContract object that holds both an int and a string properties.

Now the services would be hosted two AppDomains in the same IIS worker process. This is how it looks like on my Vista machine:

As you can see both services are in the DefaultAppPool. With both services set up, we can run the test harness, which first calls Service.AddFoo() and then Service.GetFoo(). The Service.Foo() is completed fine but Service.GetFoo() call fails with a System.Reflection.TargetException: Object does not match target. The stack trace is as follows:


1017e344 79644832 System.Reflection.RuntimeMethodInfo.CheckConsistency(System.Object)
1017e350 793a4124 System.Reflection.RuntimeMethodInfo.Invoke(System.Object, System.Reflection.BindingFlags, System.Reflection.Binder, System.Object[], System.Globalization.CultureInfo, Boolean)
1017e39c 793a40a2 System.Reflection.RuntimeMethodInfo.Invoke(System.Object, System.Reflection.BindingFlags, System.Reflection.Binder, System.Object[], System.Globalization.CultureInfo)
1017e3bc 0f96e699 Microsoft.Practices.EnterpriseLibrary.PolicyInjection.RemotingInterception.InterceptingRealProxy+<>c__DisplayClass1.b__0(Microsoft.Practices.EnterpriseLibrary.PolicyInjection.IMethodInvocation, Microsoft.Practices.EnterpriseLibrary.PolicyInjection.GetNextHandlerDelegate)
1017e3ec 0f968ed1 Microsoft.Practices.EnterpriseLibrary.PolicyInjection.HandlerPipeline.Invoke(Microsoft.Practices.EnterpriseLibrary.PolicyInjection.IMethodInvocation, Microsoft.Practices.EnterpriseLibrary.PolicyInjection.InvokeHandlerDelegate)
1017e404 0f968a2e Microsoft.Practices.EnterpriseLibrary.PolicyInjection.RemotingInterception.InterceptingRealProxy.Invoke(System.Runtime.Remoting.Messaging.IMessage)
1017e418 79374dc3 System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(System.Runtime.Remoting.Proxies.MessageData ByRef, Int32)
1017e6b4 79f98b43 [TPMethodFrame: 1017e6b4] WcfPiabInstability.Services.IAnotherService.LogFoo(WcfPiabInstability.Services.Foo)
1017e6c4 0efd08dc DynamicClass.SyncInvokeGetFoo(System.Object, System.Object[], System.Object[])
1017e6d4 50b8d90b System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(System.Object, System.Object[], System.Object[] ByRef)
1017e74c 50b6d245 System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(System.ServiceModel.Dispatcher.MessageRpc ByRef)
1017e7a0 509137ad System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(System.ServiceModel.Dispatcher.MessageRpc ByRef)
1017e7e0 509136a6 System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage4(System.ServiceModel.Dispatcher.MessageRpc ByRef)
1017e80c 50913613 System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage3(System.ServiceModel.Dispatcher.MessageRpc ByRef)
1017e81c 50913459 System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage2(System.ServiceModel.Dispatcher.MessageRpc ByRef)
1017e82c 50912257 System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage1(System.ServiceModel.Dispatcher.MessageRpc ByRef)
1017e844 50911f8f System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean)
1017e888 509115ff System.ServiceModel.Dispatcher.ChannelHandler.DispatchAndReleasePump(System.ServiceModel.Channels.RequestContext, Boolean, System.ServiceModel.OperationContext)
1017ea34 5090f8c9 System.ServiceModel.Dispatcher.ChannelHandler.HandleRequest(System.ServiceModel.Channels.RequestContext, System.ServiceModel.OperationContext)
1017ea78 5090f35e System.ServiceModel.Dispatcher.ChannelHandler.AsyncMessagePump(System.IAsyncResult)
1017ea8c 5090f2f1 System.ServiceModel.Dispatcher.ChannelHandler.OnAsyncReceiveComplete(System.IAsyncResult)
1017ea98 50232d68 System.ServiceModel.Diagnostics.Utility+AsyncThunk.UnhandledExceptionFrame(System.IAsyncResult)
1017eac4 50904501 System.ServiceModel.AsyncResult.Complete(Boolean)
1017eb00 50992b36 System.ServiceModel.Channels.InputQueue`1+AsyncQueueReader[[System.__Canon, mscorlib]].Set(Item)
1017eb14 50992215 System.ServiceModel.Channels.InputQueue`1[[System.__Canon, mscorlib]].EnqueueAndDispatch(Item, Boolean)
1017eb7c 50991ffb System.ServiceModel.Channels.InputQueue`1[[System.__Canon, mscorlib]].EnqueueAndDispatch(System.__Canon, System.ServiceModel.Channels.ItemDequeuedCallback, Boolean)
1017eba4 5091d7e5 System.ServiceModel.Channels.SingletonChannelAcceptor`3[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].Enqueue(System.__Canon, System.ServiceModel.Channels.ItemDequeuedCallback, Boolean)
1017ebc8 50977b7e System.ServiceModel.Channels.HttpChannelListener.HttpContextReceived(System.ServiceModel.Channels.HttpRequestContext, System.ServiceModel.Channels.ItemDequeuedCallback)
1017ec0c 5094f396 System.ServiceModel.Activation.HostedHttpTransportManager.HttpContextReceived(System.ServiceModel.Activation.HostedHttpRequestAsyncResult)
1017ec50 5094e4cf System.ServiceModel.Activation.HostedHttpRequestAsyncResult.HandleRequest()
1017ec68 5094defd System.ServiceModel.Activation.HostedHttpRequestAsyncResult.BeginRequest()
1017eca4 5094dea5 System.ServiceModel.Activation.HostedHttpRequestAsyncResult.OnBeginRequest(System.Object)
1017ecd0 50903c3c System.ServiceModel.Channels.IOThreadScheduler+CriticalHelper+WorkItem.Invoke2()
1017ed0c 50903b26 System.ServiceModel.Channels.IOThreadScheduler+CriticalHelper+WorkItem.Invoke()
1017ed20 50903ab5 System.ServiceModel.Channels.IOThreadScheduler+CriticalHelper.ProcessCallbacks()
1017ed54 5090390f System.ServiceModel.Channels.IOThreadScheduler+CriticalHelper.CompletionCallback(System.Object)
1017ed80 5090388b System.ServiceModel.Channels.IOThreadScheduler+CriticalHelper+ScheduledOverlapped.IOCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)
1017ed8c 50232e1f System.ServiceModel.Diagnostics.Utility+IOCompletionThunk.UnhandledExceptionFrame(UInt32, UInt32, System.Threading.NativeOverlapped*)
1017edc0 79405534 System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)
1017ef60 79e7c74b [GCFrame: 1017ef60] 
1017f0b8 79e7c74b [ContextTransitionFrame: 1017f0b8]

2. Culprit - a subtle bug in the CLR?
What is going on here? The exception is being thrown by System.Reflection.RuntimeMethodInfo.CheckConsistency() but the problem happened earlier. Notice the red in those two lines of the stack trace. DynamicClass.SyncInvokeGetFoo is the function generated by WCF using Lightweight Code Gen (LCG) to facilitate the GetFoo() call, which is in IService definition. But the next one on the call stack somehow becomes IAnotherService.LogFoo() - please be reminded that IAnotherService.LogFoo() call on the stack here is not to be confused with the one inside the Service.GetFoo() as the call hasn't reached to the actual object yet when exception happens. Using SOS command !clrstack -p and !dumpobject reveals that the object reference in the call context here is the PIAB proxy to the Service object (tp->rp->real object), which implements IService but not IAnotherService. The mismatch simply didn't manifest into an exception until later in System.Reflection.RuntimeMethodInfo.CheckConsistency(). So where the exception is thrown is not that important. We need to understand why IService.GetFoo() suddenly becomes IAnotherService.LogFoo().

Let's review the high level picture of how the calls are being dispatched from the client site (we will only consider synchronous calls for now):

Client makes a call by sending an XML message to the WCF service

WCF processes the message in the pipeline before it finally dispatches the call through System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke() as shown on the stack trace. System.ServiceModel.Dispatcher.InvokerUtil uses LCG to generate a delegate SyncInvokeXXXX, where XXXX is the target method name and SyncInvoke stands for synchronous call. SyncMethodInvoker.Invoke() simply passes control to SyncInvokeXXXX(). This part is not that difficult to figure out by using Reflector.

CLR takes the buck from here. Starting with 2.0, .NET uses dispatch stub to handle interface calls. The runtime figures out the method disptach token and sends it along with the target object reference to mscorwks!ResolveWorkerAsmStub, which calls mscorwks!VirtualCallStubManager::ResolveWorkerStatic and then the heavy lifting mscorwks!VirtualCallStubManager::ResolveWorker to figure out the stub that contains the assembly code to make the actual call.

PIAB proxy gets called. This is where the injection magic happens and service method finally gets called.

The dispatch token is a 32 bit integer with hi as the type id of the interface and lo as the slot number of the method as shown in the Rotor(SSCLI) source code:

 
static const UINT_PTR MASK_TYPE_ID       = 0x0000FFFF;
static const UINT_PTR MASK_SLOT_NUMBER   = 0x0000FFFF;
 
static const UINT_PTR SHIFT_TYPE_ID      = 0x10;
static const UINT_PTR SHIFT_SLOT_NUMBER  = 0x0;
 
//------------------------------------------------------------------------
// Combines the two values into a single 32-bit number.
static UINT_PTR CreateToken(UINT32 typeID, UINT32 slotNumber)
{
    LEAF_CONTRACT;
    CONSISTENCY_CHECK(((UINT_PTR)typeID & MASK_TYPE_ID) == (UINT_PTR)typeID);
    CONSISTENCY_CHECK(((UINT_PTR)slotNumber & MASK_SLOT_NUMBER) == (UINT_PTR)slotNumber);
    return ((((UINT_PTR)typeID & MASK_TYPE_ID) << SHIFT_TYPE_ID) |
            (((UINT_PTR)slotNumber & MASK_SLOT_NUMBER) << SHIFT_SLOT_NUMBER));
}

Type ids are integer identifiers to represent types in an AppDomain. Slot numbers are integer values representing entries of interface methods in the method table. In the example I am using, IService has a type id of 0x0003 and AddFoo has a slot number of 0x0001, therefore yielding a token of 0x00030001. IService.GetFoo has a dispatch token of 0x00030000. And IAnotherService.LogFoo also has a token of 0x0003000. You see that both IService and IAnotherService, living in two AppDomains, happen to have the same type id: 0x0003. It is a coincidence, but not to be ignored.

The reason why the dispatch token is critical here is because of the following:

all interface disptach stubs for our PIAB enabled services are handled by the VirtualCallStubManager in the shared domain of the process.

the stub manager keeps a hash table to cache the stub code using those two keys: token and object type. In our example, the object type is always a transparent proxy for PIAB-enabled services. So effectively token becomes the only key that matters.

the heavy lifting mscorwks!VirtualCallStubManager::ResolveWorker() is responsible for generating and caching the stub. Obviously it always first checks if there is a cached entry. If one is found using the keys, that entry will be returned.

When our unit test harness calls Service.AddFoo, which internally calls AnotherService.LogFoo, two dispatch stubs are created and cached by the shared stub manager, with tokens 0x00030001 and 0x00030000 as the effective key respectively. Now the unit test makes a different call Service.GetFoo. Note that IService.GetFoo also has the dispatch token as 0x00030000, same as IAnotherSevice.LogFoo, despite the fact they are two types in different domains. The stub manager of the shared domain hands out the previously cached dispatch stub for IAnotherService.LogFoo. This is why we saw the strange call stack above and the call eventually fails.

To further prove this, I changed the definition of IAnotherService to the following to include 5 additional functions as fillers to the method table slots:

[ServiceContract]
public interface IAnotherService
{
    // fillers 
    void Filler1();
    void Filler2();
    void Filler3();
    void Filler4();
    void Filler5();
 
    [OperationContract]
    void LogFoo(Foo foo);
}

The purpose is to alter the slot number of IAnotherService.LogFoo to avoid the token clash. As shown in the following Windbg snippet, I did indeed get a token of 0x00030005 as opposed to the 0x0003000 that I had earlier:


eax=00000003 ebx=0e3f6228 ecx=79e89e87 edx=00000003 esi=00000005 edi=01c45b18
eip=79eb45cf esp=1057dd4c ebp=1057dd80 iopl=0 nv up ei pl nz ac po cy
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000213
mscorwks!VirtualCallStubManager::GetCallStub+0x34:
79eb45cf c1e010 shl eax,10h
0:025> p
eax=00030000 ebx=0e3f6228 ecx=79e89e87 edx=00000003 esi=00000005 edi=01c45b18
eip=79eb45d2 esp=1057dd4c ebp=1057dd80 iopl=0 nv up ei pl nz ac pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000216
mscorwks!VirtualCallStubManager::GetCallStub+0x37:
79eb45d2 0bf0 or esi,eax
0:025> p
eax=00030000 ebx=0e3f6228 ecx=79e89e87 edx=00000003 esi=00030005 edi=01c45b18
eip=79eb45d4 esp=1057dd4c ebp=1057dd80 iopl=0 nv up ei pl nz na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000206

And the CLR happily makes all the calls without the dreadful exception!

3. Summary
It is interesting to see how the CLR team may have tried to safeguard the clashing of the keys by using both token and the object type. In our case, the object type, being the transparent proxy for PIAB-enabled services, takes that key out of the equation. The token, although having HI corresponding to the interface type and LO to the slot number, is scoped within the AppDomain where the type is loaded. The fewer the number of types and the fewer the methods of the operation contracts are defined for the WCF service in each AppDomain, the bigger the odds key clashing like this will happen.

So what is the solution? Could other interface method related values such as MethodDesc, which seems to be unique across app domains, be a better candidate? That is the question for the CLR team.

As for us who want to integrate PIAB into WCF while minimizing development team efforts, we should be fine by hosting WCF services in separate processes. If you really, really want to host everything in one IIS worker process like our contrived example, you can get around this issue by not using the default PIAB interception mechanism. Suppose you can come up with a LCG mechanism to generate one dynamic proxy (instead of System.Runtime.Remoting.Proxies.__TransparentProxy) for each target. Having a different object type as each service's PIAB intercepting proxy will therefore avoid the key clashing.

4. Tools etc.
Debugging is always an enlightening experience. And I can't imagine what life is going to be without Windbg and Reflector. Visual Studio 2008 is cool since you can configure it to debug into .NET framework code. But if the symbols are not available for the module you want to investigate or you need to dig deeper below the FCL layer, Reflector and Windbg are just indispensable.

Also, having SSCLI source code is absolutely wonderful. Without it, debugging through machine code in windbg would be a lot harder.

Most of the SSCLI code for this exercise is located inside virtualcallstub.cpp file.

Interview Questions

2008-09-09T09:46:00.000-04:00

Having a list of questions with standard answers to score candidates during technical screenings certainly has its benefits, especially maintaining consistencies across different interviewers. However if the interviewer simply compares the candidate's answers to the official ones, then she is not doing her job. Interview is an interactive process and should be leveraged as such. Many seemingly easy questions can be extended to discussions at both broader and deeper levels. You will get a better picture of candidate's overall skills and experiences this way. For example, there is usually a basic question on the differences between value and reference types. This question can be extended to boxing/unboxing, and the scenarios where boxing/unboxing can occur and the performance implications, which can be a good starting point to test candidate's knowledge on generics. There are semantic implications of boxing and unboxing as well. A boxed integer, e.g. is a brand new object with a copy of the initial integer value. The following is not allowed by the C# compiler:

int i = 1; // class instance field

lock(i)
{
//...
}

You could hack it with:

lock ((object)i)
{
//...
}

But you will not get the intended lock semantics. I will leave it to you to answer why that is the case. If you know the answer, you will see that you can evaluate the candidate's knowledge of threading and synchronization, besides boxing/unboxing.

And there is more! A related topic is passing by reference vs. passing by value in function calls (I had a previous blog in this area). And you can take the initial question and extend it to a discussion on heap vs. stack and further on GC.

So you see how one easy question can be extended quite a bit and become the vehicle for you to test candidate's overall knowledge. Of course you need to be sensitive to time and not go off to all directions. You usually will get a sense of the candidate's knowledge half way through and can decide whether to go further or not from that point.

I Am Back

2008-06-20T15:46:00.000-04:00

For the last three months, I was working for a client on envisioning the next generation architecture of a highly impactful desktop application. Despite the short 3 months time, it was nevertheless a winding journey. Because of the impact of the application, business and politics drove a lot of the architecture decisions, and a lot of long nights too. Now that the project is on hold until the next phase, I can get back inside Visual Studio and be really technical for a short while.

Debug.Assert in ASP.NET Application

2008-03-14T16:31:00.000-04:00

My most recent project was an ASP.NET 2.0 application developed in Visual Studio 2005. During the first build and deployment into dev integration environment, everything went pretty well except for one page. Request for this page was just hanging. I started to investigate and I captured a hang dump using adplus. In windbg, I found out the longest running thread. Suspecting that is the thread that was hanging, I looked at its CLR stack:


0:001> !clrstack
*********************************************************************
* Symbols can not be loaded because symbol path is not initialized. *
* *
* The Symbol Path can be set by: *
* using the _NT_SYMBOL_PATH environment variable. *
* using the -y  argument when starting the debugger. *
* using .sympath and .sympath+ *
*********************************************************************
PDB symbol for mscorwks.dll not loaded
OS Thread Id: 0x2fac (1)
ESP EIP 
006bf414 77f88a77 [NDirectMethodFrameStandalone: 006bf414] Microsoft.Win32.SafeNativeMethods.MessageBox(System.Runtime.InteropServices.HandleRef, System.String, System.String, Int32)
006bf430 7a4f839a System.Diagnostics.AssertWrapper.ShowMessageBoxAssert(System.String, System.String, System.String)
006bf460 7a4fabb2 System.Diagnostics.DefaultTraceListener.Fail(System.String, System.String)
006bf4a0 7a4faad7 System.Diagnostics.DefaultTraceListener.Fail(System.String)
006bf4a4 7a500a22 System.Diagnostics.TraceInternal.Fail(System.String)
006bf4e0 7a6e2523 System.Diagnostics.TraceInternal.Assert(Boolean, System.String)
006bf4e4 7a4fa6cb System.Diagnostics.Debug.Assert(Boolean, System.String)
006bf4e8 1ae8b7c9 CondosPaymentPageBase.GetResidentData()
006bf528 1ae8b567 CondosPaymentPageBase.OnLoad(System.EventArgs)
006bf534 66143ad0 System.Web.UI.Control.LoadRecursive()
006bf548 66155106 System.Web.UI.Page.ProcessRequestMain(Boolean, Boolean)
006bf700 66154a1b System.Web.UI.Page.ProcessRequest(Boolean, Boolean)
006bf738 66154967 System.Web.UI.Page.ProcessRequest()
006bf770 66154887 System.Web.UI.Page.ProcessRequestWithNoAssert(System.Web.HttpContext)
006bf778 6615481a System.Web.UI.Page.ProcessRequest(System.Web.HttpContext)
006bf78c 1bb8ccae ASP.payment_3_history_aspx.ProcessRequest(System.Web.HttpContext)
006bf798 65ff27d4 System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
006bf7cc 65fc15b5 System.Web.HttpApplication.ExecuteStep(IExecutionStep, Boolean ByRef)
006bf80c 65fd32e0 System.Web.HttpApplication+ApplicationStepManager.ResumeSteps(System.Exception)
006bf85c 65fc0225 System.Web.HttpApplication.System.Web.IHttpAsyncHandler.BeginProcessRequest(System.Web.HttpContext, System.AsyncCallback, System.Object)
006bf878 65fc550b System.Web.HttpRuntime.ProcessRequestInternal(System.Web.HttpWorkerRequest)
006bf8ac 65fc5212 System.Web.HttpRuntime.ProcessRequestNoDemand(System.Web.HttpWorkerRequest)
006bf8b8 65fc3587 System.Web.Hosting.ISAPIRuntime.ProcessRequest(IntPtr, Int32)
006bfa68 79f35ee8 [ContextTransitionFrame: 006bfa68] 
006bfab8 79f35ee8 [GCFrame: 006bfab8] 
006bfc10 79f35ee8 [ComMethodFrame: 006bfc10]

And the native stack looked like this:


0:001> kb
ChildEBP RetAddr Args to Child 
006bf1b0 77e4f9f0 50000018 00000003 00000003 NTDLL!ZwRaiseHardError+0xb
006bf20c 77e34398 00d3b684 00d3c298 00040212 USER32!ServiceMessageBox+0x16b
006bf35c 77e339cb 006bf36c 006bfa68 00000028 USER32!MessageBoxWorker+0x10a
006bf3b4 77e4fa54 00000000 00d3b684 00d3c298 USER32!MessageBoxExW+0x77
*** WARNING: Unable to verify checksum for System.ni.dll
006bf43c 7a4fad1d 00000000 00d1c4fc 00d28944 USER32!MessageBoxW+0x49
006bf454 7a4fabb2 00000000 00d24bf8 04cfda28 System_ni+0xbad1d
006bf494 7a4faad7 00000000 7a500a22 00000000 System_ni+0xbabb2
006bf4d8 7a6e2523 7a4fa6cb 1ae8b7c9 00d113c0 System_ni+0xbaad7
00000000 00000000 00000000 00000000 00000000 System_ni+0x2a2523

At this point, it was clear to me that some data condition in dev int environment had caused Debug.Assert to fail. The assert failure message box was waiting to be closed. But on an IIS box with the ASP worker process running in the context of a service account, this UI interaction is simply not going to work.

As you may know, Debug.Assert will be left out of the IL code by compilers for release build. Our MSBuild script that produced the deployment package did have the "release" switch turned on. So what went wrong? It turned out that the web deployment project we had, "Generate debug information" option was checked. I initially thought this would give me pdb files. But obviously this meant a debug build, irrespective of the "Configuration=Release" setting for the MSBuild script.

PIAB and WCF Article Published on MSDN Magazine

2008-01-13T11:27:00.000-05:00

The article that I have coauthored with my Avanade colleague David San Filippo is now published in the February issue of MSDN magazine. If you have subscribed to MSDN, you should have received it in your mail box by now. The online link is here. Basically the article details how to integrate PIAB into WCF via .NET configuration or attribute so developers would not have to write code to apply the goodness of PIAB to WCF.

It's been an absolutely wonderful experience working with MSDN editors, Howard Dierking, Nancy Michell and Debra Kelly so I'd like to extend a big "thank you" to them from here. My colleague David is one of the smartest developers I have worked with; it only took a few weekends for us to come up with a draft of this article after the brainstorming back in last August.

A Good Book on Windbg

2008-01-09T15:17:00.000-05:00

Since I have never worked for any Microsoft product team, a good way for me to get internal system knowledge is through debugging, using tools like Windbg (Visual Studio as an IDE can be good for debugging too, but is limited in functionalities compared with Windbg).

Over the years, I have gained my Windbg skills by reading blogs (thanks to those who post them) and just practicing it on my own whenever I have a chance. Most of the debugging books don't have enough coverage in this area. So I was really excited after I found out that Addison Wesley has published this Advanced Windows Debugging book. You should read this book if you are seriously thinking about learning Windbg.

Debugging MAPI Problems

2007-12-18T14:36:00.000-05:00

I have been quiet here for the last month or so, one of the reasons being that I have been bogged down by a lot of application issues in production. The most recent one had to do with 4 Windows services on the production database server. Those services, developed in VB long time ago, run as work flow engines, polling/querying the database on a timer, changing the state of work flow items according to business rules and emailing the next person in the queue using MAPI function. Those services were developed at different times but they have very similar functionalities. Most of the source code was just copied from the very first one unfortunately.

About a week ago, those services started to hang - users are usually first to tell because the work flow items don't get routed after a couple of minutes as it is supposed to. I have been doing analysis with the hang dumps created by ADPLUS, arriving at the conclusion that MAPI call was the culprit. However, I haven't figured out exactly what in the MAPI call caused the problem due to lack of MAPI debug symbols. I am going to blog the debugging result anyway - hopefully someone with similar experiences with MAPI can point out the issue.

The following shows the stack of the main thread of the services that are hanging:


0:000> kb
ChildEBP RetAddr  Args to Child              
0012e64c 7c59a072 000005a4 00000000 00000000 NTDLL!ZwWaitForSingleObject+0xb
0012e674 7c57b3e9 000005a4 ffffffff 00000000 KERNEL32!WaitForSingleObjectEx+0x71
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for MSMAPI32.DLL - 
0012e684 35ffbee7 000005a4 ffffffff 0de623a0 KERNEL32!WaitForSingleObject+0xf
WARNING: Stack unwind information not available. Following frames may be wrong.
0012e6fc 35f789e7 00000000 02dad738 02da27d8 MSMAPI32!FPropContainsProp+0x2612
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for MAPI32.DLL - 
0012e744 61dd2d09 00000000 02da33f8 02da2228 MSMAPI32!MAPILogonEx+0xa7
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for CDO.DLL - 
0012e76c 0dd025ea 00000000 02da33f8 02da2228 MAPI32!MAPILogonEx+0x79
00000000 00000000 00000000 00000000 00000000 CDO!DllUnregisterServer+0x10eea

To see what WaitForSingleObject() is waiting on:


0:000> !handle 5a4 f
Handle 000005a4
  Type          Mutant
  Attributes    0
  GrantedAccess 0x120001:
         ReadControl,Synch
         QueryState
  HandleCount   6
  PointerCount  12
  Name          \BaseNamedObjects\4D4150494C6F676F6E0070B872C47FD7101B8BEA00AA0038C699_S-1-5-21-1346851753-1016681682-1202159320-1002
  Object specific information
    Mutex is Owned

If the export symbols are close enough, the MAPILogonEx() is waiting on this mutex, which has a name suffixed with a SID, which belongs to the service account the services are running as. The SID suffix seems to reenforce the conjecture that MAPILogonEx() is being called.

While 3 out of the 4 services have their main threads like that, the forth one is different:


0:000> kb
ChildEBP RetAddr  Args to Child              
0012de34 7c59a072 00000664 00000000 00000000 NTDLL!ZwWaitForSingleObject+0xb
0012de5c 7c57b3e9 00000664 ffffffff 00000000 KERNEL32!WaitForSingleObjectEx+0x71
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for MSMAPI32.DLL - 
0012de6c 35f98779 00000664 ffffffff 00000000 KERNEL32!WaitForSingleObject+0xf
WARNING: Stack unwind information not available. Following frames may be wrong.
0012de94 35f98691 00000201 0012deec 0012decc MSMAPI32!DDLUnwrapObjectEx+0xc65
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for EMSABP32.DLL - 
0012dec4 35ae3c6e 00000004 00000201 00000000 MSMAPI32!DDLUnwrapObjectEx+0xb7d
0012df10 35ae3bbd 023e2144 00000001 023e2170 EMSABP32!ABProviderInit+0x2611
0012e028 35f72512 35f72570 360b6700 35f755d0 EMSABP32!ABProviderInit+0x2560
00000000 00000000 00000000 00000000 00000000 MSMAPI32!DDCS_Enter+0x16

This one is waiting on an event object:


0:000> !handle 664 f
Handle 00000664
  Type          Event
  Attributes    0
  GrantedAccess 0x1f0003:
         Delete,ReadControl,WriteDac,WriteOwner,Synch
         QueryState,ModifyState
  HandleCount   2
  PointerCount  4
  Name          
  No object specific information available

I suspect this process is the owner of the mutex, although I didn't have a chance to create a kernel dump to verify.


0:000> !handle 370 f
Handle 00000370
  Type          Mutant
  Attributes    0
  GrantedAccess 0x1f0001:
         Delete,ReadControl,WriteDac,WriteOwner,Synch
         QueryState
  HandleCount   6
  PointerCount  12
  Name          \BaseNamedObjects\4D4150494C6F676F6E0070B872C47FD7101B8BEA00AA0038C699_S-1-5-21-1346851753-1016681682-1202159320-1002
  Object specific information
    Mutex is Owned

We have moved the 4 services from the database server (Win2K) to a different box (Win2K3) and there hasn't been any hang problems so far, which suggests that the MAPI problem is probably local to that machine, rather than having to do with the exchange server. Here is the detail of MAPI32.dll on the database server:


61dd0000 61df1000   MAPI32     (export symbols)       MAPI32.DLL
    Loaded symbol image file: MAPI32.DLL
    Image path: C:\Program Files\Common Files\System\MSMAPI\1033\MAPI32.DLL
    Image name: MAPI32.DLL
    Timestamp:        Wed Jan 13 11:08:14 1999 (369CC4EE)
    CheckSum:         0002A798
    ImageSize:        00021000
    File version:     1.0.2536.0
    Product version:  1.0.2536.0
    File flags:       2 (Mask 3F) Pre-release
    File OS:          40004 NT Win32
    File type:        2.0 Dll
    File date:        00000000.00000000
    Translations:     0409.04e4
    CompanyName:      Microsoft Corporation
    ProductName:      Microsoft Exchange
    InternalName:     MAPI32
    OriginalFilename: MAPI32.DLL
    ProductVersion:   1.0
    FileVersion:      1.0.2536.0
    FileDescription:  Extended MAPI 1.0 for Windows NT
    LegalCopyright:   Copyright © 1986-1999 Microsoft Corp. All rights reserved.
    LegalTrademarks:  Microsoft® and Windows® are registered trademarks of Microsoft Corporation.
    Comments:         Service Pack 3

Now back to investigating ...

Thread Pool

2007-10-31T17:14:00.000-04:00

I mentioned in my last post that I would cover some interesting topics that all relate to thread. Now here is the second one, on thread pool.

Thread pool is an efficient way to achieve concurrency since it saves the cost of creating new threads all the time by reusing threads in a pool. To get a good understanding of how to leverage ThreadPool class in the .NET framework, you can start by reading Jeffery Richter's good article on MSDN. However that was .NET 1.1. In .NET 2.0, the ThreadPool class supports a total of 4 types of uses as opposed to 3, which can be seen clearly from the enumeration in SSCLI2:

enum ThreadpoolThreadType
{
    WorkerThread,
    CompletionPortThread,
    WaitThread,
    TimerMgrThread
};

With the new ThreadPool class, we can

1) call a method on a worker thread by QueueUserWorkItem

2) call a method on an IO Completion port thread by UnsafeQueueNativeOverlapped

3) call a method on an IO Completion port thread when a kernel object is signaled by RegisterWaitForSingleObject

4) call a method on a worker thread by using System.Threading.Timer - note this is the API seemingly irrelevant to ThreadPool

While IO Completion Port (IOCP) is not new to native Windows, the inclusion of its API in thread pool is the latest feature added to .NET 2.0. IOCP is a mechanism on Windows to allow asynchronous IO. The IOCP threads will be woken up and start code execution if IO operation is complete. The Win32 API for this is GetQueuedCompletionStatus, which essentially doesn't return (thread back to the pool) until IO operation is done. And more interestingly, IOCP isn't just limited to being used for IO - ThreadPool.UnsafeQueueNativeOverlapped can be used just like ThreadPool.QueueUserWorkItem. Here is a code snippet for queueing a delegate to the IOCP threads via Overlapped data structure:

unsafe
{
    Overlapped overlapped = new Overlapped(0, 0, IntPtr.Zero, null);
    NativeOverlapped* pOverlapped = overlapped.Pack(IocpThreadProc, null);
    ThreadPool.UnsafeQueueNativeOverlapped(pOverlapped);
}

Note that unsafe keyword is used as native pointer is created. And here is the delegate:

unsafe static void IocpThreadProc(uint x, uint y, NativeOverlapped* p)
{   
    try
    {
        Sort();
    }
    finally
    {
        Overlapped.Free(p);
    }
}

Note that Free must be called in the finally block to ensure that memory is not leaked.

The philosophy of IOCP is that the optimal number of concurrent running threads should be equal to the number of processors, because anything more would introduce the overhead of context switches between different threads. Peeking into the win32threadpool.cpp file of SSCLI2, one can find that while both the number of worker threads and the number of IOCP threads can grow under the ceiling of upper limits set by ThreadPool.SetMaxThreads, IOCP has constraints from CPU utilization also. In other words, even if you set the max of IOCP threads to a number much higher than the number of processros, there won't be more concurrent threads than the number of processors if there is very high CPU utilization. On the other hand, the number of worker threads, under the situation of high CPU utilitzation, will likely grow under the ceiling with requests queued up. This blog has shown empirical results proving this. The example is a bit of extreme in that a large number of highly computation-intensive requests are being queued to the thread pool. However it shouldn't be viewed that IOCP is always the better one to use. Remember now you can still call ThreadPool.SetMaxThreads to throttle the number of concurrent worker threads to achieve good results too. Measuring and tuning would be required in real world situations. While it's great that we have an additional option to be able to queue any requests to IOCP through UnsafeQueueNativeOverlapped, keep in mind that IOCP was designed to be a great mechanism to handle IO asynchronously. Examples, such as System.IO.FileStream, System.Net.Sockets.Socket, System.ServiceModel.Channels.MsmqQueue, etc. can be found in the FCL. So I'd conjecture that while using a custom IO device, if asynchronous IO is desired, the same pattern can be followed by passing the handle of the IO object to ThreadPool.BindHandle to associate the IO device to the IOCP of the ThreadPool.

As to using System.Threading.Timer, this great MSDN Article discusses it pretty well in comparison with System.Timers.Timer and System.Windows.Forms.Timer. It's important to remember that System.Timers.Timer is a wrapper of System.Threading.Timer. System.Windows.Forms.Timer uses a completely different mechanism, which is SetTimer Win32 API, posting WM_TIMER message to the application queue. This means that your delegate added to the Tick event will be running on the same UI thread that created the main application windows forms. As the UI thread processes a lot of messages, including mouse and keyboard messages, this mechanism will surely not get you a lot of mileage for concurrency if that's what you're looking for.

As server apps like ASP.NET, WCF already have concurrency built in, ThreadPool should not be overly used in there. ThreadPool can be ideal in UI applications, Windows service, and batch applications, etc, if they need a lot of concurrency.

.NET Code Instrumentation That Is More Accurate?

2007-10-31T15:24:00.000-04:00

I recently ran into a few interesting topics that all relate to Thread and I will share them with you here.

Someone asked an interesting question to our company's community on how to instrument for just the time spent only in the thread when it is executing the code. Thread switching, thread idling/sleeping while waiting for IO, etc should not be counted.

This is indeed pretty interesting as on a machine where a lot of threads are running concurrently, using the elapsed time (e.g. the Tracer in EntLib) approach, may give you inaccurate results if you just want to focus on measuring algorithm of your code. It will be further skewed if the thread running your code is given lower priorities than the other ones as thread scheduling on Windows is priority-driven and preemptive.

After digging around a bit, I found a reasonable good approach to this problem, with the help of BCL of .NET framework. The Process type in System.Diagnostics namespace has a Threads property, which is a collection that corresponds to the Win32 threads in the process. The .NET type that represents the underlying Win32 thread is ProcessThread. It turns out that ProcessThread has properties for processor time, which is key to the solution. Specifically, the property I want to use is UserProcessorTime. Here is a simple wrapper class that I wrote, following the Tracer class pattern from EntLib. I have only kept necessary code to show the point – email me (hugh dot ang at gmail dot com) if you want to have my Visual Studio project that has the complete source code.

public class ThreadTimer : IDisposable
{
    private const string LOG_CATEGORY = "General";
 
    [ThreadStatic()]
    private ProcessThread _processThread;
 
    private int _threadId;
    private TimeSpan _start;
 
    [DllImport("Kernel32", EntryPoint = "GetCurrentThreadId", ExactSpelling = true)]
    private static extern Int32 GetCurrentWin32ThreadId();
 
    public ThreadTimer()
    {
        _threadId = ProcessThread.Id;
        _start = ProcessThread.UserProcessorTime;
 
        Logger.Write("ThreadTimer started", LOG_CATEGORY);
    }
 
    public void Dispose()
    {
        double duration = ProcessThread.UserProcessorTime.Subtract(_start).TotalSeconds;
        Logger.Write(
            String.Format("ThreadTimer ended with elapsed time of {0} seconds processor user time.",
            duration),
            LOG_CATEGORY);
    }
 
    private ProcessThread ProcessThread
    {
        get
        {
            if (_processThread == null)
            {
                int id = GetCurrentWin32ThreadId();
                foreach (ProcessThread thread in Process.GetCurrentProcess().Threads)
                {
                    if (thread.Id == id)
                    {
                        _processThread = thread;
                        return _processThread;
                    }
                }
            }
 
            return _processThread;
        }
    }
}

Here is how we can instrument a method called SortOperation:

using (ThreadTimer threadTimer = new ThreadTimer())
{
    // simulate an operation that needs to be instrumented
    SortOperation(i);
}

To see how this works out in comparison to instrumentation of EntLib, I implemented the SortOperation() as in the following:

static void SortOperation(int x)
{
    SortedList<int, int> slist = new SortedList<int, int>();
    for (int i = x * 10000; i >= 0; i--)
    {
        slist.Add(i, i);
    }
}

To make it interesting, I first instrumented this method using both Tracer and my ThreadTimer and then instrumented the same method plus another method that does just IO, e.g. read from a file on disk. The IO method takes about 200 milliseconds. The following table summarizes the results:

Without the IO method, both Tracer and ThreadTimer produce almost identical results. Based on the empirical results, it’s not hard to figure out that the algorithm of looping with SortedList <> is O(N²). When the IO method is included, EntLib reports the extra 200 milliseconds while ThreadTimer reports pretty much unchanged results.

So you see this ThreadTimer can be a reasonable solution for instrumentation in the scenario where you want to just focus on measuring your algorithm. However, as EntLib captures the start to end performance, which is what end user will experience, this solution shouldn’t replace that. Another caveat is that since there is no fixed relationship between a Win32 native thread and a managed thread, there is no guarantee that this solution will work for any CLR host.

Debugging Without Source Code Using the Right Tools

2007-10-26T21:11:00.000-04:00

When it comes to debugging without source code, there are two categories of debugging tools for this purpose:

deassemblers/debuggers such as IDA Pro, windbg, etc. If the running application is written in .NET, using windbg with SOS is really handy. This kind of tools are helpful if you need to track and understand program flow and execution state, such as stack and memory for subtle issues. These tools also tend to be favorites of crackers who want to crack/bypass license schemes.

tracing tools. There are those that trace network traffic such as tcpTrace, ethereal, Fiddler and of course netmon. For monitoring process, file and registry activities on the OS itself, nothing beats the tools from Sysinternals, now Microsoft. The top two I recommend are Process Monitor and Process Explorer. The tools in this category are your friends in many debugging scenarios, especially during the initial stage of understanding the issues at hand.

Here I will go over the debugging experience I just had this week with a legacy financial software, with a file-based database (no it's not Access or any of the Microsoft products). For anonymity reasons I will just not name this software.

Anyway, the database file used by this software is installed on a network share so that the client on users desktops can all use it. You'd probably guess by now that locking has been a common issue but that's not what I plan to cover here.

There is a custom built web application (classic ASP app hosted on IIS 5) in house that has to look up data from this database, which are files on the network share as I have mentioned. I was told that when the web app was being designed, the developer had found that although a VB Winform application could access the database through ODBC just fine, ASP application couldn't. Without investigating and further pursuit, a separate ETL program was developed to transfer the needed data to the SQL database used by the ASP application. The data fortunately doesn't change that much so this ETL program is run once every night. Suffice to say that such a moving piece has had its moments with IT operations although it is not the worst pet peeve.

I came to know this application and the ETL program recently and when I asked why we can't query and cache the data from the file based database in the ASP app, the answer was that the ODBC connection didn't work in ASP although it worked in the ETL program written in VB. I was told that the developer hadn't really debugged the application but rather called the help desk of the software vendor, which was not at all helpful like most of the tier 1 support of those vendors. Intuition was telling me that this problem had to do with the security context, such as impersonation and logon sessions. Question I would ask right away when facing this type of issue: does the context in ASP have permission or other issue accessing this share?

Determined to fix this problem and get rid of the ETL program, I started on the developer machine by configuring the IIS application with high isolation process model. This would cause a server COM+ application to be created - remember I had to deal with IIS 5! :-) Then I set the identity of the COM+ application to a domain service account that would have proper permission to the share, where the database files are located. With IIS directory security configured to do integrated windows authentication, the ASP app worked! But the production server has basic auth to allow users access this app from the Internet. Alas when basic auth was configured, it stopped working even when I typed in the same domain user name and password as the developer logged onto the desktop. As IIS 5 impersonates when either windows auth or basic auth is turned on, there must be a difference between the logged on session associated with the impersonation token in the two authentication types. Using a tool called TokenDump from Keith Brown, that I downloaded long time ago, I was able to confirm that 1) with windows auth, the ASP impersonation token is using the same logon session as the interactive user 2) with basic auth, the impersonation token, however, uses a different logon session than the desktop's although they are both for the same domain user account. This is probably by design as I conjecture that IIS may have done a LogonUser() call using the passed in user name and password, which would result in a different logon session.

But why was a different logon session encountering the problem? I used the Process Monitor for two captures, one for each auth type. When I compared the two, I noticed that the basic auth one didn't even have any trace showing attempts to access the network share. Going further, I found out from the developer that during ODBC and client setup, the legacy financial software used a mapped drive to point to the network share, where the database files reside. This is where the answer to the puzzle lies. As you may know, mapped drive is tied to the logon session and therefore not global. Here is the official documentation on MSDN. We should avoid using it, especially for enterprise server applications.

Finally I would need to change the drive letter to UNC. Alas the legacy software UI explicitly disallows it. Not wanting to give up at this point, I asked the developer to use Process Monitor to find out where (registry or configuration files) the software is keeping the data that contains the drive letter. Sure enough, we found a handful in the registry under the hives created by the software installation. I went ahead and manually changed all relevant registry settings. I then ran the ASP application with basic auth. Voila it worked! So the software must already be using the Win32 IO API that supports UNC, when its UI still disallows it.

This experience shows you how handy the right tools can be in hairy situations such as this. I do caution you though that you need to be extra careful when changing registry settings like this as it may cause unexpected behavior and/or breach license agreement.

EntLib 3.1 Configuration - Part II

2007-10-05T23:13:00.000-04:00

All EntLib versions come with source code, which is a great place to start digging around and understanding the framework. For configuration, since we are planning to write a custom configuration section, we will need to look at the Configuration and Configuration.Design sub namespaces as each block has them. For example, look for Microsoft.Practices.EnterpriseLibrary.Logging.Configuration and Microsoft.Practices.EnterpriseLibrary.Logging.Configuration.Design in the Logging block. Given all the overwhelming number of types defined in those namespaces, it can seem daunting first to tackle this.

It's actually not too difficult. I will illustrate the approach by just doing a custom configuration section with simple attributes, instead of a complicated configuration schema with nested hierarchies.

Consider an example where we need to connect to an external web service, which needs a configuration section with URL, User ID and Password.

1) First of all we will need to create a type that corresponds to the section in the configuration XML file. Note that this type extends the EntLib's SerializableConfigurationSection. All we need to do in this type is defining the section name and attribute names. And of course we need to define read/write properties for those attributes to read/persist from the configuration file.

 
public class FooSettings : SerializableConfigurationSection
{
    // Defines the section name 
    public const string SectionName = "FooConfiguration";
 
    // Defines attributes for the section
    private const string URL_ATTR = "url";
    private const string UID_ATTR = "userID";
    private const string PASSWORD_ATTR = "password";
 
    [ConfigurationProperty(URL_ATTR, IsRequired = true)]
    public string Url
    {
        get { return (string)base[URL_ATTR]; }
        set { base[URL_ATTR] = value; }
    }
 
    [ConfigurationProperty(UID_ATTR, IsRequired = true)]
    public string UID
    {
        get { return (string)base[UID_ATTR]; }
        set { base[UID_ATTR] = value; }
    }
 
    [ConfigurationProperty(PASSWORD_ATTR, IsRequired = true)]
    public string Password
    {
        get { return (string)base[PASSWORD_ATTR]; }
        set { base[PASSWORD_ATTR] = value; }
    }
}

2) Secondly, we will need to create a type that represents the node on the EntLib console. This type extends the EntLib's ConfigurationSectionNode. Note that It's very similar to the FooSettings type defined above, in that there are three read/write properties that correspond to the Url, UID and Password.

 
[Image(typeof(FooNode), "ConfigNode_d.bmp")]
[SelectedImage(typeof(FooNode), "ConfigNode_h.bmp")]
public class FooNode : ConfigurationSectionNode
{
    private string _url;
    private string _uid;
    private string _password;
 
    public FooNode()
        : base("Foo Configuration")
    {}
 
    [ReadOnly(true)]
    public override string Name
    {
        get { return base.Name; }
    }
 
    [Required()]
    [Browsable(true)]
    public string Url
    {
        get { return _url; }
        set { _url = value; }
    }
 
    [Required()]
    [Browsable(true)]
    public string UID
    {
        get { return _uid; }
        set { _uid = value; }
    }
 
    [Required()]
    [Browsable(true)]
    public string Password
    {
        get { return _password; }
        set { _password = value; }
    }
}

What is interesting here is that this class as well as its propeties have some attributes attached to them. The SelectedImage() and Image() attributes simply define the bitmap images shown when the node is in selected or deselected state. The two bitmap images are embedded resources within the same VS.NET project. The Required() and Browsable() attributes at the properties are telling the EntLib UI that those properties are required and browsable(visible) on the UI. If you don't put a value for the property, a UI validation error will occur when you try to save the configuration.

3) So far so good. Now we need to have a way to let EntLib UI become aware of our custom configuration node. Here is where we need to extend EntLib's ConfigurationDesignManager to have our FooConifgurationDesignManager.

 
public class FooConfigurationDesignManager : ConfigurationDesignManager
{
    public override void Register(IServiceProvider serviceProvider)
    {
        (new FooCommandRegistrar(serviceProvider)).Register();
    }
 
    protected override void OpenCore(IServiceProvider serviceProvider, 
        ConfigurationApplicationNode rootNode, 
        ConfigurationSection section)
    {
        if (section != null)
        {
            FooSettings settings = section as FooSettings;
            Debug.Assert(settings != null, 
                "Check config section - not of the FooSettings type");
 
            FooNode node = new FooNode();
            // sync data on UI and in config file
            node.Url = settings.Url;
            node.UID = settings.UID;
            node.Password = settings.Password;
 
            SetProtectionProvider(section, node);
 
            rootNode.AddNode(node);
        }
    }
 
    protected override ConfigurationSectionInfo GetConfigurationSectionInfo(
        IServiceProvider serviceProvider)
    {
        ConfigurationNode rootNode = ServiceHelper.GetCurrentRootNode(serviceProvider);
        FooNode node = null;
        if (rootNode != null)
            node = (FooNode)rootNode.Hierarchy.FindNodeByType(rootNode, typeof(FooNode));
 
        FooSettings settings = null;
        if (node == null)
        {
            settings = null;
        }
        else
        {
            settings = new FooSettings();
 
            // sync data on UI and in config file
            settings.Url = node.Url;
            settings.UID = node.UID;
            settings.Password = node.Password;
        }
        string protectionProviderName = GetProtectionProviderName(node);
 
        return new ConfigurationSectionInfo(node, 
            settings, FooSettings.SectionName, protectionProviderName);
    }
}

The two overridden methods OpenCore() and GetConfigurationSectionInfo() are the hooks for EntLib UI to load and save from/to configuration file. You will notice the code in the two methods to sync up data held by FooNode and FooSettings - just remember that FooNode represents the UI view and FooSettings represents the configuration file. The remaining function, Register(), is the hook for us to provide the command in the EntLib context menu. Here we will use the following CommandRegistrar class to add the command "Foo Configuration":

 
public class FooCommandRegistrar : CommandRegistrar
{
    public FooCommandRegistrar(IServiceProvider serviceProvider)
        : base(serviceProvider)
    {
    }
 
    public override void Register()
    {
        ConfigurationUICommand cmd = ConfigurationUICommand.CreateSingleUICommand(ServiceProvider,
            "Foo Configuration",
            "Foo Configuration",
            new AddChildNodeCommand(ServiceProvider, typeof(FooNode)),
            typeof(FooNode));
        AddUICommand(cmd, typeof(ConfigurationApplicationNode));
    }
}

With all these in place, we need to add the following to the AssemblyInfo.cs file:

[assembly: ConfigurationDesignManager(typeof(FooConfigurationDesignManager))]

EntLib UI, when being loaded, will use reflection API to query the assemblies in its folder ("c:\Program Files\Microsoft Enterprise Library 3.1 - May 2007\Bin\" in my case) and find out all the ConfigurationDesignManager types through this assembly attribute. So after the project is compiled, we need to drop the dll into the same folder as the EntLibConfig.exe file.

Whew, now we can see how this works in the EntLib UI. The following figure shows that the "Foo Configuration" command is in the context menu.

One cool feature of EntLib 3.1 configuration is that it works integrated with VS.NET 2005 IDE. The following figure shows just that.

You see that a Foo Configuration is created besides the standard logging application block. In the properties window on the lower-right corner, the values for Url, UID and Password are set. I have also defined two environments, QA and Prod. For QA, override is chosen for the foo configuration and you can also see that it's a different set of values. Note that you can encrypt this section with the standard providers. If you have gone through the code I listed above, you will realize that these two functions are provided by EntLib without us writing any code!

Here is what the app.config file looks like (I have ommitted the logging section for clarity):

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <configSections>
    <section name="FooConfiguration" type="CustomEntLibConfig.FooSettings, CustomEntLibConfig, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" />
  </configSections>
  <FooConfiguration url="http://foo.com/test.asmx" userID="test"
    password="test123" />
</configuration>

In case you're curious, here is what the QA delta file looks like.

<configuration>
    <configSections>
        <section name="EnvironmentMergeData" type="Microsoft.Practices.EnterpriseLibrary.Configuration.EnvironmentalOverrides.Configuration.EnvironmentMergeSection, Microsoft.Practices.EnterpriseLibrary.Configuration.EnvironmentalOverrides, Version=3.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
    </configSections>
    <EnvironmentMergeData environmentName="QA" environmentDeltaFile="app.qa">
        <mergeElements>
            <override nodePath="/Foo Configuration" overrideProperties="true">
                <overridddenProperties>
                    <add name="Password" value="qa123" />
                    <add name="UID" value="qa" />
                    <add name="Url" value="http://qa.foo.com/test.asmx" />
                </overridddenProperties>
            </override>
        </mergeElements>
    </EnvironmentMergeData>
</configuration>

4) To use the configuration programmatically, here is a code snippet:

 
FooSettings node = ConfigurationManager.GetSection(FooSettings.SectionName) as FooSettings;
 
if (node != null)
    Console.WriteLine("Url: {0}, UID: {1}, Password: {2}", 
        node.Url, node.UID, node.Password);
else
    Console.WriteLine("Custom section not found");
 
Console.ReadLine();

I hope you find this helpful. Email me at hugh dot ang @ gmail dot com if you need the source code for the sample I have used here.

EntLib 3.1 Configuration - Part I

2007-09-23T21:36:00.000-04:00

Recently, when an application that my team had built was deployed to production, an external web service's URL was forgotten to have been set from its test site's to the production site's. So the transactions thought to have completed actually never happened. This caused some interruptions to the business.

We had to rush this deployment one month ahead of schedule because of a business issue. While the rush was partly to blame, I think since the application uses Enterprise Library 3.1 for quite a few of its blocks, one feature could have helped us avoid this hiccup.

Since 3.0, Enterprise Library introduced the environment overrides for configuration. This blog had a good overview of this feature. In a nutshell, the configuration files for the most part are the same between different environments, DEV, Staging, QA and Prod, etc. There are only a few minor things that might be different, such as database connection string, logging threshold, etc. This new Enterprise Library configuration feature saves the differentials in delta config files. And the merge function overrides the original (usually DEV) application config file with the delta to produce the config file for the target environment.

Of course, this feature works out of box for any configuration setting if it can be done through the Enterprise Library configuration console tool, which is also available from Visual Studio .NET 2005 IDE. The list includes the "Application Settings", "Caching", "Logging", etc. "Application Settings" is great for independent name/value pairs but for a set of values and more complicated configuration data schema, a separate configuration section is required. In the next blog, I will show how to create custom EntLib 3.1 configuration sections so that not only can we leverage the environment overrides, but also we can easily use the EntLib config UI to protect sensitive config data with cryptography.

Sync Active Item in VS.NET Solution Explorer

2007-08-21T22:22:00.000-04:00

Opening a big VS.NET solution and having a lot of source files can be very disorienting. I've found it especially so when I am tracking deep in the types that are referred by other types, which are referred by others, and so on. And it's pretty difficult to find out which Visual Studio project the type is defined in. I was so desperate that I almost wrote an addin to help me synchronizing the active source file in the solution explorer window. Alas when I was almost done, I found that Visual Studio 2005 already offers such function. To use it, just follow these steps:

Go to "Tools" menu, click "Options"

Under "Projects and Solutions", click "General"

Check the "Track Active Item in Solution Explorer" checkbox on the right side

That is it! Now no matter which source file you end up burying your head into, the solution explorer window always selects and scrolls to that file for you. I guess my lesson learned is that I should always look hard first because Visual Studio may just have that built in already. And it was also interesting to learn the VS.NET automation APIs that have to do with finding the solution item and manipulating the solution explorer window.

Commoditization of Business Applications

2007-08-09T12:19:00.000-04:00

I have been quiet here for the last couple of months. The main reason is that I stepped into the position of vice president of app dev department and became heavily involved in organization management as well as project management at a client. I am also busy collaborating with a colleague on a WCF article that we hope to publish soon.

Being in this particular management position has given me a different perspective on application development. I have been involved in a couple of projects where "buy vs. build" decision was made. Dollars and benefits (ROI) always took precedence over anything else. And it seems obvious that the industry trend is going towards the direction where building everything from scratch is less favored and we see needs for product knowledge and configuration management skills. You can see that with the slew of products like BizTalk, MOSS, Commerce Server 2007, Dynamics, etc. coming out of Microsoft assembly line in a pace like never before.

The trade-off for the product knowledge will be fundamentals and technical skills. An unfortunate effect of this will be that for certain technical issues, the root problems don't get resolved but rather awkward workarounds are contrived at the product level, which could result in convoluted business processes for end users.

For developers, this may not be good news as the creativity aspect (the fun part) of application development is being gradually taken away and it is a big challenge to keep on top of the ever growing list of products.

EntLib 3.1 Is Coming

2007-05-25T14:08:00.000-04:00

It looks like that p&p is addressing the implementation issue of interception for Policy Injection by making the interception mechanism configurable. Here is the blog that talks about this.

A Simple Design Pattern for Extensible Enumeration Types

2007-05-15T12:50:00.000-04:00

In .NET, custom enum types are of value type so they cannot be extended. This can create an issue if you want to include an enum in your framework API, where you wish that the enum could be extended by the applications that are using the framework. After examining the implementation of System.Enum, I came up with the following implementation. I haven't googled thoroughly yet so please no flames if someone has done something similar :-) Also note the code posted is POC quality and I am looking to improve it.

First I will start off with a base class EnumEx. I am using generics to allow different underlying enum types:

 
[Serializable()]
public abstract class EnumEx<T> 
    where T : struct
{
    // holds the actual value
    protected T _t;
 
    // default constructor
    protected EnumEx()
    {
        _t = default(T);
    }
 
    // constructor that takes the enum value
    protected EnumEx(T t)
    {
        _t = t;
    }
 
    // performs an implicit conversion to the underlying value
    public static implicit operator T(EnumEx<T> obj)
    {
        return obj._t;
    }
 
    // parses a string to the specified type, returns null if no string match is defined
    // this is a case-sensitive version
    public static object Parse(string s, Type type)
    {
        FieldInfo[] fis = type.GetFields(BindingFlags.Static | BindingFlags.Public);
        foreach (FieldInfo fi in fis)
        {
            if (s.Equals(fi.Name))
            {
                object obj = fi.GetValue(null);
                return obj;
            }
        }
 
        return null;
    }
 
    // System.Object overrides
    public override int GetHashCode()
    {
        return _t.GetHashCode();
    }
 
    public override string ToString()
    {
        FieldInfo[] fis = this.GetType().GetFields(BindingFlags.Static | BindingFlags.Public);
        foreach (FieldInfo fi in fis)
        {
            object obj = fi.GetValue(this);
            T t = ((EnumEx<T>)obj)._t;
            if (_t.Equals(t))
                return fi.Name;
        }
 
        return string.Empty;
    }
}

This can be how a framework enum is defined:

 
public class LogType : EnumEx<int>
{
    protected LogType() 
        : base()
    {
    }
 
    protected LogType(int value)
        : base(value)
    {
    }
 
    public static implicit operator LogType(int i)
    {
        return new LogType(i);
    }
 
    public static LogType Info = 1;
    public static LogType Warning = 2;
    public static LogType Error = 3;
}

And if the application wishes to extend the enum:

 
public class LogTypeEx : LogType
{
    public static LogType Verbose = 4;
}

If I write the following test code:

 
LogType logType = LogType.Error;
Console.WriteLine(logType.ToString());
 
LogType logType2 = LogType.Parse("Warning", typeof(LogType)) as LogType;
if (logType2.Equals(LogType.Warning))
    Console.WriteLine("Parsing succeeds");
 
int i = logType2;
if (i == 2)
    Console.WriteLine("Implicit conversion to underlying type succeeds");

I will get the following result in the console:

Error
Parsing succeeds
Implicit conversion to underlying type succeeds

So you see that this extensible enum type is pretty straightforward to use, extending the enum is very easy and it achieves pretty much what the native enum type has to offer. One thing left to be desired though is an issue using switch statement with C# compiler. Only the following works:

 
switch (logType2)
{
    case 2:
        Console.WriteLine("Do logic based on LogType.Info");
        break;
}

while this doesn't compile (it fails at the case statement):

 
switch (logType2)
{
    case LogType.Info:
        Console.WriteLine("Do logic based on LogType.Info");
        break;
}

The C# compiler requires a constant in the case statement. It's also picky about what's in the switch statement. Without the implicit operator, the first switch code snippet wouldn't even have compiled either.

Share Application-Scoped Data in Web Farm or Web Garden

2007-04-24T14:37:00.000-04:00

I was asked recently on how to share application-scoped data in a web farm or web garden. The answer would have been easy if the question had been on session data as ASP.NET offers a couple of different ways for sharing session out-of-proc based on configuration, namely, by setting mode attribute to "StateServer", "SQLServer" or "Custom" in the sessionState element of the web.config file.

For sharing application-scoped data arocess processes, though, there is no such feature provided out of box by ASP.NET. Back when EntLib wasn’t available, I implemented a custom caching framework to share application-scoped data in the web farm. Now with EntLib, you can use the caching block to achieve this quite easily. You just need to choose the database cache storage or a custom storage for the caching block. The custom storage, if you choose to do so, will have to be written and plugged into the EntLib framework of course. As the objects put in the cache will be serialized across processes and may be saved in the database, the objects will need to be serializable.

Policy Injection App Block of EntLib 3.0 - An implementation of AOP

2007-02-25T20:03:00.000-05:00

Details have finally emerged about Policy Injection App Block (PIAB) in the upcoming EntLib 3.0 - see here and here. I have been curious and wanting to see this as this is a form of Aspect-Oriented Programming (AOP). I have worked quite a bit on my company's framework, which has a different variation of AOP. What I like about PIAB from p&p is that it doesn't impose restriction of AOP at physical boundaries, as opposed to, say, binding AOP to the service boundary. The arguments for binding AOP to only the service boundary are basically concerning about the undisciplined use of AOP, thus causing aspect "meatballs" throughout the application. I think the decoupling of AOP from physical application boundaries bring more benefits and goodness to the application architecture. Remember that we could configure attributes, such as transaction, security for components in MTS and later COM+. If a framework makes the aspect feature highly configurable, I wouldn't agree to the "meatballs" argument. Besides, any language or framework in existence today, while making developers lives easier, still can't be savior of all problems without disciplines from developers - This is where good development process comes in. So I think most of the developers would appreciate the freedom that PIAB offers.

The PIAB implementation, though, seems to be somewhat controversial. Instead of a few sound alternatives, they picked remoting. p&p didn't want to mess around the runtime by using CLR profiler APIs or JIT Debugger callbacks to rewrite IL at runtime. However, the reason for not using IL rewriting, should not be they are obscure or not supported by PSS. Instead, The CLR already has ContextBoundObject, which is designed for this very purpose of interception. Don Box's Essential .NET book has a good coverage of this topic in chapter 7. As a matter of fact, I have seen a lot of AOP implementations using ContextBoundObject in the field. So I really don't understand the argument "not recommended for customer code" in Edjez's blog. Maybe the CLR team can shed some light on this. Honestly I don't see why remoting is a better choice and wouldn't be replaced in future releases of EntLib. We know that remoting was not designed to BE an interception mechanism, although with an interception mechanism.

Although not stated by the blogs, I suspect that the lack of "new" interception in the object builder framework may have been a driver to using remoting instead of ContextBoundObject. But based on my experience so far, besides being internally used by the other blocks, object builder hasn't got wide adoption, due to lack of documentation and API complexities. An important success factor for a framework, no matter how good the framework is, is the ease of use of its APIs. I am waiting to get my hands on the February CTP of EntLin 3.0 to see how easy the PIAB can be used in conjunction with object builder. I am hoping that tying PIAB to object builder doesn’t make it more cumbersome to use as opposed to intercepting “new” with ContextBoundObject and making it completely transparent for the client code.

Win32++

2007-02-10T14:07:00.000-05:00

The MSDN feed MSDN Just Published interestingly has posted a link pointing to a code project article, which covers a simple C++ framework as an alternative to MFC. I have downloaded the core and sample source files and played with a few samples. Just as the author claims, this framework doesn't use macros, templates, MI as in MFC and WTL so it's much easier to comprehend. It bears a resemblance to the approach that .NET WinForm has taken when wrapping Win32 underneath. So kudos to the author for an exellent job!

The code project site has some reader feedback regarding the choice of using TLS to store the association of a HWND and its corresponding WndProc. Being always conscious about performance, my only concern, though, is related to the implementation of StaticWindowProc:

 
    LRESULT CALLBACK CWnd::StaticWindowProc(HWND hWnd, UINT uMsg, WPARAM wParam, LPARAM lParam)
    {
        try
        {
            {
            // Allocate an iterator for our HWND map
            std::map<HWND, CWnd*, CompareHWND>::iterator m;
 
            GetApp()->m_MapLock.Lock();
            m = GetApp()->GetHWNDMap().find(hWnd);
            GetApp()->m_MapLock.Release();
            if (m != GetApp()->GetHWNDMap().end())
                return m->second->WndProc(hWnd, uMsg, wParam, lParam);
 
            }
            throw CWinException(TEXT("CWnd::StaticWindowProc .. Failed to route message"));
        }
 
        catch (const CWinException &e)
        {
            e.MessageBox();
        }
 
        catch (...)
        {
            DebugErrMsg(TEXT("Exception in CWnd::StaticWindowProc"));
        }
 
        return ::DefWindowProc(hWnd, uMsg, wParam, lParam);;
 
    } // LRESULT CALLBACK StaticWindowProc(...)

Finding the WndProc corresponding to a HWND in the STL map will go through a critial section via m_MapLock.Lock() and m_MapLock.Release(). Albeit the light nature of crtitical sections, I'd always be wary of using it in this scenario, where all the windows messages (keyboard, mouse, etc.) of all the CWnd subclasses in the application get funneled through this function.

Your thoughts? Please post comments.

Why Isn't Form Created on Stack GC'ed?

2007-02-01T12:10:00.002-05:00

Have you run into situations where something is really simple and intuitive so you take it for granted? But if you think harder it makes little sense. Here is an example. In a WinForm application, a form (Form1) creates and opens another form (Form2) on stack (inside a button click handler) non-modally:

 
        private void btnNewForm_Click(object sender, EventArgs e)
        {
            Form2 frm = new Form2();
            frm.Show();
        }

We'd expect Form2 to stay on the screen after the call finishes. After all, the user of the software should be the one to decide when to end the life of Form2 by closing it. But wait, isn't the reference to Form2 created on the stack gone after the function call finishes? So why are we still seeing Form2? (Note that If the form is opened as modal, this question is irrelevant as the ShowDialog() call doesn't return until the modal form is closed)

GC shouldn't bend the rules just for form objects. There must be hidden references that are not seen at programming level. A few SOS commands in the debugger indeed yield the truth for us. I have used .NET 2.0, WinDbg 6.6.0007.5 for this exercise.

After the btnNewForm_Click() function that creates the form returns, we find the Form2 instance on the managed heap:


0:000> !dumpheap -type Form2
 Address       MT     Size
013bd650 00a26d5c      324     
total 1 objects
Statistics:
      MT    Count    TotalSize Class Name
00a26d5c        1          324 Forms.Form2
Total 1 objects

Now let's find out what are keeping references of it:


0:000> !gcroot 013bd650
Note: Roots found on stacks may be false positives. Run "!help gcroot" for
more info.
Scan Thread 0 OSTHread 1f4
Scan Thread 2 OSTHread c3c

DOMAIN(001545F0):HANDLE(WeakLn):a010d0:Root:013bda70(System.Windows.Forms.NativeMethods+WndProc)->
013bd7a4(System.Windows.Forms.Control+ControlNativeWindow)->
013bd650(Forms.Form2)


DOMAIN(001545F0):HANDLE(Strong):a01160:Root:013bd650(Forms.Form2)

DOMAIN(001545F0):HANDLE(Pinned):a013ec:Root:02384b68(System.Object[])->
0139c948(System.Collections.Generic.Dictionary`2[[System.Object, mscorlib],[System.Collections.Generic.List`1[[Microsoft.Win32.SystemEvents+SystemEventInvokeInfo, System]], mscorlib]])->
0139d100(System.Collections.Generic.Dictionary`2+Entry[[System.Object, mscorlib],[System.Collections.Generic.List`1[[Microsoft.Win32.SystemEvents+SystemEventInvokeInfo, System]], mscorlib]][])->
0139e27c(System.Collections.Generic.List`1[[Microsoft.Win32.SystemEvents+SystemEventInvokeInfo, System]])->
0139e2a4(System.Object[])->
013bdb30(Microsoft.Win32.SystemEvents+SystemEventInvokeInfo)->
013bdb10(Microsoft.Win32.UserPreferenceChangedEventHandler)->
013bd650(Forms.Form2)

DOMAIN(001545F0):HANDLE(Pinned):a013f0:Root:02383030(System.Object[])->
013a10e0(System.Windows.Forms.FormCollection)->
013a10f8(System.Collections.ArrayList)->
013a1110(System.Object[])->
013bd650(Forms.Form2)


DOMAIN(001545F0):HANDLE(WeakSh):a01808:Root:013bd7a4(System.Windows.Forms.Control+ControlNativeWindow)
DOMAIN(001545F0):HANDLE(WeakSh):a0180c:Root:013bd7a4(System.Windows.Forms.Control+ControlNativeWindow)

We can ignore the weak references (WeakSh and WeakLn) and concentrate on the Strong and Pinned(indirectly) ones as they are the reasons that Form2 instance will not be GC'ed. The strong reference is actually a GCHandle (a01160) tied to the Form2 instance. Before the Form2 instance was created I added a breakpoint at call to System.Runtime.InteropServices.GCHandle.Alloc using !SOS.bpmd command. When the breakpoint was hit, I examined the CLR stack:


0:000> !clrstack -p
OS Thread Id: 0x1f4 (0)
ESP       EIP     
0012ef28 79361fc4 System.Runtime.InteropServices.GCHandle.Alloc(System.Object, System.Runtime.InteropServices.GCHandleType)
    PARAMETERS:
        value = 0x013bd650
        type = 0x00000002

0012ef2c 7b0783cd System.Windows.Forms.Control+ControlNativeWindow.LockReference(Boolean)
    PARAMETERS:
        this = 0x013bd7a4
        locked = 

0012ef38 7b06ca41 System.Windows.Forms.Control.UpdateRoot()
    PARAMETERS:
        this = 

0012ef40 7b073366 System.Windows.Forms.Control.SetVisibleCore(Boolean)
    PARAMETERS:
        this = 
        value = 

0012efe8 7b06382b System.Windows.Forms.Form.SetVisibleCore(Boolean)
    PARAMETERS:
        this = 0x013bd650
        value = 0x00000001

0012f028 7b0cf369 System.Windows.Forms.Control.Show()
    PARAMETERS:
        this = 

0012f02c 00f50423 Forms.Form1.btnNewForm_Click(System.Object, System.EventArgs)
    PARAMETERS:
        this = 0x01382ce4
        sender = 0x0139c18c
        e = 0x013bb798

...

I have omitted the bottom frames for brevity. Showing the form eventually calls Control.SetVisibleCore(), which then calls ControlNativeWindow.LockReference(), which simply allocates a GC handle for the form instance as shown in the source code via reflector:

 
        internal void LockReference(bool locked)
        {
            if (locked)
            {
                if (!this.rootRef.IsAllocated)
                {
                    this.rootRef = GCHandle.Alloc(this.GetControl(), GCHandleType.Normal);
                }
            }
            else if (this.rootRef.IsAllocated)
            {
                this.rootRef.Free();
            }
        }

According to the framework documentation, GCHandle is created corresponding to a managed object and can be used to prevent GC of the object when it's used by unmanaged client. In this case though, ControlNativeWindow's private variable rootRef, which holds the handle, does nothing but holding on tight to the form object! That's probably the name LockReference :-) To confirm that this is indeed the handle that SOS reports to us in the debugger, I stepped into the code that creates the GC handle. First I set a breakpoint at GCHandle.Alloc. When it's hit, I disassembled the instruction pointer:


0:000> !u eip
preJIT generated code
System.Runtime.InteropServices.GCHandle.Alloc(System.Object, System.Runtime.InteropServices.GCHandleType)
Begin 79361fc4, size 39
>>> 79361fc4 57              push    edi
79361fc5 56              push    esi
79361fc6 53              push    ebx
79361fc7 50              push    eax
79361fc8 8bd9            mov     ebx,ecx
79361fca 8bfa            mov     edi,edx
79361fcc bae4040000      mov     edx,4E4h
79361fd1 b901000000      mov     ecx,1
79361fd6 e8f51ab100      call    mscorwks!JIT_Writeable_Thunks_Buf+0x1a0 (79e73ad0) (JitHelp: CORINFO_HELP_GETSHARED_NONGCSTATIC_BASE)
79361fdb 8d3424          lea     esi,[esp]
79361fde 8bd7            mov     edx,edi
79361fe0 8bcb            mov     ecx,ebx
79361fe2 e80da4b800      call    mscorwks!MarshalNative::GCHandleInternalAlloc (79eec3f4)
79361fe7 8906            mov     dword ptr [esi],eax
79361fe9 83ff03          cmp     edi,3
79361fec 7507            jne     mscorlib_ni+0x2a1ff5 (79361ff5)
79361fee 8bce            mov     ecx,esi
79361ff0 e80f380100      call    mscorlib_ni+0x2b5804 (79375804) (System.Runtime.InteropServices.GCHandle.SetIsPinned(), mdToken: 06002ff9)
79361ff5 8b0424          mov     eax,dword ptr [esp]
79361ff8 59              pop     ecx
79361ff9 5b              pop     ebx
79361ffa 5e              pop     esi
79361ffb 5f              pop     edi
79361ffc c3              ret

Note that I have highlighted the assembly code following the call to GCHandleInternalAlloc(). So then I stepped to that instruction 79361fe7:


0:000> pa 79361fe7
79361fe2 e80da4b800      call    mscorwks!MarshalNative::GCHandleInternalAlloc (79eec3f4)
eax=00a01160 ebx=013bd650 ecx=79eec4fe edx=013bd650 esi=0012ef18 edi=00000002
eip=79361fe7 esp=0012ef18 ebp=0012efe0 iopl=0         nv up ei pl zr na pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246
mscorlib_ni+0x2a1fe7:
79361fe7 8906            mov     dword ptr [esi],eax  ds:0023:0012ef18=00000000

For brevity I have omitted instructions up to the point where GCHandleInternalAlloc() is called. The instruction is now at 79361fe7. The return value, which is the GC handle, should be in the EAX register:


0:000> ?eax
Evaluate expression: 10490208 = 00a01160

Note that the hex value a01160 matches exactly what SOS says.

The other two references to the form instance are 1) a delegate (UserPreferenceChangedEventHandler) for system wide event notification - the name is obvious and 2) a collection FormCollection kept by System.Windows.Forms.Application.Application, exposed through a static property called OpenForms.

When the form is being closed, the UserPreferenceChangedEventHandler is removed (via Control.OnHandleDestroyed), Form is removed from the Application.OpenFormsInternal (via Form.OnHandleDestroyed). As to the GCHandle, ControlNativeWindow.rootRef, is freed by LockReference again, with locked=false as seen in the following call stack:


0:000> !clrstack -p
OS Thread Id: 0x2bc (0)
ESP       EIP     
0012e43c 7b078390 System.Windows.Forms.Control+ControlNativeWindow.LockReference(Boolean)
    PARAMETERS:
        this = 0x01388a14
        locked = 0x00000000

0012e440 7b06ca41 System.Windows.Forms.Control.UpdateRoot()
    PARAMETERS:
        this = 

0012e448 7b06d6ea System.Windows.Forms.Control.SetHandle(IntPtr)
    PARAMETERS:
        this = 
        value = 

0012e454 7b07a484 System.Windows.Forms.Control+ControlNativeWindow.OnHandleChange(IntPtr)
    PARAMETERS:
        this = 
        newHandle = 

0012e458 7b07a46e System.Windows.Forms.Control+ControlNativeWindow.OnHandleChange()
    PARAMETERS:
        this = 

0012e45c 7b0897df System.Windows.Forms.NativeWindow.ReleaseHandle(Boolean)
    PARAMETERS:
        this = 0x01388a14
        handleValid = 

0012e490 7b07a55a System.Windows.Forms.NativeWindow.Callback(IntPtr, Int32, IntPtr, IntPtr)
    PARAMETERS:
        this = 0x01388a14
        hWnd = 
        msg = 0x00000082
        wparam = 
        lparam = 

0012e664 003420d4 [NDirectMethodFrameStandalone: 0012e664] System.Windows.Forms.UnsafeNativeMethods.IntDestroyWindow(System.Runtime.InteropServices.HandleRef)
0012e67c 7b0892ee System.Windows.Forms.UnsafeNativeMethods.DestroyWindow(System.Runtime.InteropServices.HandleRef)
    PARAMETERS:
        hWnd = 

0012e688 7b0891f8 System.Windows.Forms.NativeWindow.DestroyHandle()
    PARAMETERS:
        this = 0x01388a14

0012e6cc 7b05f010 System.Windows.Forms.Control.DestroyHandle()
    PARAMETERS:
        this = 0x013888c0

0012e6d0 7b08901e [InlinedCallFrame: 0012e6d0] 
0012e7a0 7b05c583 System.Windows.Forms.Form.Dispose(Boolean)
    PARAMETERS:
        this = 0x013888c0
        disposing = 

0012e7b4 00f505ac Forms.Form2.Dispose(Boolean)
    PARAMETERS:
        this = 0x013888c0
        disposing = 0x00000001

0012e7c8 7a4a6633 System.ComponentModel.Component.Dispose()
    PARAMETERS:
        this = 0x013888c0

0012e7d0 7b224d92 System.Windows.Forms.Form.WmClose(System.Windows.Forms.Message ByRef)
    PARAMETERS:
        this = 
        m = 

0012e7fc 7b063e7d System.Windows.Forms.Form.WndProc(System.Windows.Forms.Message ByRef)
    PARAMETERS:
        this = 
        m = 

0012e80c 7b07a72d System.Windows.Forms.Control+ControlNativeWindow.OnMessage(System.Windows.Forms.Message ByRef)
    PARAMETERS:
        this = 
        m = 

0012e810 7b07a706 System.Windows.Forms.Control+ControlNativeWindow.WndProc(System.Windows.Forms.Message ByRef)
    PARAMETERS:
        this = 
        m = 

0012e824 7b07a515 System.Windows.Forms.NativeWindow.Callback(IntPtr, Int32, IntPtr, IntPtr)
    PARAMETERS:
        this = 0x01388a14
        hWnd = 
        msg = 0x00000010
        wparam = 
        lparam = 

0012ec04 003420d4 [InlinedCallFrame: 0012ec04] System.Windows.Forms.UnsafeNativeMethods.CallWindowProc(IntPtr, IntPtr, Int32, IntPtr, IntPtr)
0012ec00 7b07a7d4 System.Windows.Forms.NativeWindow.DefWndProc(System.Windows.Forms.Message ByRef)
    PARAMETERS:
        this = 
        m = 0x0012ed3c

0012ec44 7b05ed83 System.Windows.Forms.Form.DefWndProc(System.Windows.Forms.Message ByRef)
    PARAMETERS:
        this = 
        m = 

0012ec48 7b072abc [InlinedCallFrame: 0012ec48] 
0012ec90 7b07f795 [InlinedCallFrame: 0012ec90]

How To Create Custom Database With Enterprise Library 2.0 Data Access Block - Part 2

2007-01-23T14:20:00.000-05:00

As promised in this previous blog, I have created a custom Database implementation for Firebird in the EntLib 2.0 framework.

First I implemented this interface: Microsoft.Practices.EnterpriseLibrary.Data.Configuration.IDatabaseAssembler

using System;
using System.Configuration;
using Microsoft.Practices.EnterpriseLibrary.Common.Configuration;
using Microsoft.Practices.EnterpriseLibrary.Data.Configuration;
 
namespace FirebirdSql.Data.EntLib
{
    public class FbDatabaseAssembler : IDatabaseAssembler
    {
        public Microsoft.Practices.EnterpriseLibrary.Data.Database Assemble(string name, ConnectionStringSettings connectionStringSettings, IConfigurationSource configurationSource)
        {
            return new FbDatabase(connectionStringSettings.ConnectionString);
        }
    }
}

Then I derived from Microsoft.Practices.EnterpriseLibrary.Data.Database (we should do more deligence here to ensure we take care of FbClient's idiosyncracies by implementing appropriate overrides or provide additional functions, as in SqlDatabase and OracleDatabase):

using System;
using System.Data.Common;
using System.Security.Permissions;
 
using Microsoft.Practices.EnterpriseLibrary.Data;
using Microsoft.Practices.EnterpriseLibrary.Data.Configuration;
 
using FirebirdSql.Data.FirebirdClient;
 
namespace FirebirdSql.Data.EntLib
{
    [FirebirdClientPermission(SecurityAction.Demand)]
    [DatabaseAssembler(typeof(FbDatabaseAssembler))]
    public class FbDatabase : Database
    {
        public FbDatabase(string connectionString)
            : base(connectionString, FirebirdClientFactory.Instance)
        {
        }
 
        protected override void DeriveParameters(System.Data.Common.DbCommand discoveryCommand)
        {
            FbCommandBuilder.DeriveParameters((FbCommand)discoveryCommand);
        }
    }
}

And now I set up a winform test application:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Text;
using System.Windows.Forms;
using System.Data.Common;
 
using Microsoft.Practices.EnterpriseLibrary.Data;
 
namespace FirebirdSql.Data.EntLib.Test
{
    public partial class MainForm : Form
    {
        public MainForm()
        {
            InitializeComponent();
        }
 
        private void btnDataset_Click(object sender, EventArgs e)
        {
            Database db = DatabaseFactory.CreateDatabase();
            DbCommand comm = db.GetSqlStringCommand("SELECT JOB_CODE, JOB_REQUIREMENT FROM JOB");
 
            DataSet ds = db.ExecuteDataSet(comm);
            this.dgv.DataSource = ds.Tables[0];
        }
    }
}

Note that I can now execute SQL and bind to the control in only 4 lines of code. The connection string and the mapping between provider and FbDatabase is taken care of by the EntLib configuration console:

So you see it's really easy to implement a custom Database object for EntLib DAAB if its ADO.NET native providers exist. With this, developers can use consistent API to access the databases in an elegant way. It's especially useful when you need to access multiple databases (from different database types) in the same application.

Memory Leaks in .NET Applications

2007-01-23T10:54:00.000-05:00

When I tell people that their .NET applications are leaking memory, I sometimes get frowned upon as if memory leaking is moot in CLR. After all, garbage collection is there so why would we have to worry about this?

MSDN has published an excellent article on this topic. While it covers many memory leaking scenarios to watch out for, I have seen a couple of quite common ones at various customers due to coding practice issues:

1) Custom-rolled caching that has no expiration and throttling mechanisms. The cache eventually will be filled up with references to objects that are not intended to be kept. A common pattern is a singleton wrapping a hashtable.

2) Event handlers (delegates) added to application level events but forgotten to be removed. I have seen this in both Winform and ASP.NET apps, where a form or an ASP.NET page has hooked its instance function to an application event through a delegate. Forgetting to remove such handlers when the form or page is unloaded will simply keep the instance of the form or page in memory, keeping it from garbage collected.

As you can see, just like you need to "free" after "malloc", "delete" after "new", "CloseHandle" after "CreateFile" etc. in C++/Win32 API, discipline is still needed in .NET coding just as well. So don't let your guard down.

ByRef Snafu

2007-01-22T09:26:00.000-05:00

At a client I worked some time ago, a consultant who had worked there prior to me had told the client developers (mostly from VB 6 background) a "best practice": use ByRef when passing reference type parameters and ByVal for value type ones. Besides the obvious confusion of the concepts, this practice and a particular VB.NET compiler design had caused a subtle bug in the application that was difficult to track down. That application was written for .NET framework 1.1 with VS.NET 2003. Now the VB.NET compiler for VS.NET 2005 still does the same so I am able to contrive a simple example to demo this bug in VS.NET 2005.

To get started, I have created a simple form with a textbox "txtName" and a button "btnCallByRef" as in the following:

And here is the code:

Public Class Main
    Private Sub btnCallByRef_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnCallByRef.Click
        ReadTextBox(txtName)
        Debug.WriteLine("txtBox is passed to a function ByRef")
    End Sub
 
    Private Sub ReadTextBox(ByRef txtbox As TextBox)
        ' nop
    End Sub
 
    Private Sub txtName_Leave(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles txtName.Leave
        Debug.WriteLine("txtName_Leave handler")
    End Sub
 
    Private Sub AnotherHandler(ByVal sender As System.Object, ByVal e As System.EventArgs)
        Debug.WriteLine("another handler")
    End Sub
 
    Private Sub Main_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        AddHandler txtName.Leave, AddressOf AnotherHandler
    End Sub
End Class

The textbox "txtName" has a design time handler "txtName_Leave" defined for its "Leave" event. During form load, another handler "AnotherHandler" is added dynamically to handle the same "Leave" event. The client application had a framework, which uses a generic event handler to monitor form all kinds of events - the "AnotherHandler" is just simulating that handler in the framework. Note that the framework's generic handler needs to be called last in the delegate chain as it needs to look at the final attributes of the controls, since they may be changed by handlers that are put there at design time by form developers.

This is all working pretty well when I tab out the textbox to fire the Leave event. See the following in the output window while debugging:

txtName_Leave handler
another handler

well, pretty well until the "btnCallByRef_Click" function is called:

txtBox is passed to a function ByRef
another handler
txtName_Leave handler

When I tab out the textbox to fire the Leave event, the execution order of the handlers has been reversed. What happened here? It turns out that when a control is declared as in the following:

Friend WithEvents txtName As System.Windows.Forms.TextBox

The VB.NET compiler will create a private field "_txtName" and a public property "txtName" encapsulating the field and what's more interesting is the setter of the property as seen in IL:

.method assembly newslot specialname strict virtual 
instance void set_txtName(class [System.Windows.Forms]System.Windows.Forms.TextBox WithEventsValue) cil managed synchronized
{
.custom instance void [mscorlib]System.Diagnostics.DebuggerNonUserCodeAttribute::.ctor() = ( 01 00 00 00 ) 
// Code size 93 (0x5d)
.maxstack 3
.locals init (bool V_0)
IL_0000: ldarg.0
IL_0001: ldfld class [System.Windows.Forms]System.Windows.Forms.TextBox VBWinForm.Main::_txtName
IL_0006: ldnull
IL_0007: ceq
IL_0009: ldc.i4.0
IL_000a: ceq
IL_000c: stloc.0
IL_000d: ldloc.0
IL_000e: brfalse.s IL_0029
IL_0010: ldarg.0
IL_0011: ldfld class [System.Windows.Forms]System.Windows.Forms.TextBox VBWinForm.Main::_txtName
IL_0016: ldarg.0
IL_0017: dup
IL_0018: ldvirtftn instance void VBWinForm.Main::txtName_Leave(object,
class [mscorlib]System.EventArgs)
IL_001e: newobj instance void [mscorlib]System.EventHandler::.ctor(object,
native int)
IL_0023: callvirt instance void [System.Windows.Forms]System.Windows.Forms.Control::remove_Leave(class [mscorlib]System.EventHandler)
IL_0028: nop
IL_0029: nop
IL_002a: ldarg.0
IL_002b: ldarg.1
IL_002c: stfld class [System.Windows.Forms]System.Windows.Forms.TextBox VBWinForm.Main::_txtName
IL_0031: ldarg.0
IL_0032: ldfld class [System.Windows.Forms]System.Windows.Forms.TextBox VBWinForm.Main::_txtName
IL_0037: ldnull
IL_0038: ceq
IL_003a: ldc.i4.0
IL_003b: ceq
IL_003d: stloc.0
IL_003e: ldloc.0
IL_003f: brfalse.s IL_005a
IL_0041: ldarg.0
IL_0042: ldfld class [System.Windows.Forms]System.Windows.Forms.TextBox VBWinForm.Main::_txtName
IL_0047: ldarg.0
IL_0048: dup
IL_0049: ldvirtftn instance void VBWinForm.Main::txtName_Leave(object,
class [mscorlib]System.EventArgs)
IL_004f: newobj instance void [mscorlib]System.EventHandler::.ctor(object,
native int)
IL_0054: callvirt instance void [System.Windows.Forms]System.Windows.Forms.Control::add_Leave(class [mscorlib]System.EventHandler)
IL_0059: nop
IL_005a: nop
IL_005b: nop
IL_005c: ret
} // end of method Main::set_txtName

So it can be seen that the existing handler is removed, the field _txtName is set and the event handler is added back again. And if we take a look at the IL code of btnCallByRef_Click. Note how the "set_txtName" is called after the "ReadTextBox" call as a result of passing ByRef:


.method private instance void btnCallByRef_Click(object sender,
class [mscorlib]System.EventArgs e) cil managed
{
// Code size 38 (0x26)
.maxstack 2
.locals init ([0] class [System.Windows.Forms]System.Windows.Forms.TextBox VB$t_ref$S0)
IL_0000: nop
IL_0001: ldarg.0
IL_0002: ldarg.0
IL_0003: callvirt instance class [System.Windows.Forms]System.Windows.Forms.TextBox VBWinForm.Main::get_txtName()
IL_0008: stloc.0
IL_0009: ldloca.s VB$t_ref$S0
IL_000b: callvirt instance void VBWinForm.Main::ReadTextBox(class [System.Windows.Forms]System.Windows.Forms.TextBox&)
IL_0010: nop
IL_0011: ldarg.0
IL_0012: ldloc.0
IL_0013: callvirt instance void VBWinForm.Main::set_txtName(class [System.Windows.Forms]System.Windows.Forms.TextBox)
IL_0018: nop
IL_0019: ldstr "txtBox is passed to a function ByRef"
IL_001e: call void [System]System.Diagnostics.Debug::WriteLine(string)
IL_0023: nop
IL_0024: nop
IL_0025: ret
} // end of method Main::btnCallByRef_Click

This VB.NET compiler behavior is clearly designed for the convenience of developers so they don't have to manually add or remove handlers in their code. Unfortunately not knowing this and following an ill-advised practice involving ByRef led to a really unexpected bug.

How To Create Custom Database With Enterprise Library 2.0 Data Access Block

2007-01-21T20:54:00.000-05:00

There seems to be little documentation on how to create custom database object to be used for Enterprise Library 2.0 Data Access block. Tom Hollander in this blog confirms that this is definitely a possibility:

In the new version, we still have the SqlDatabase and OracleDatabase class – but we also have a new GenericDatabase class. The GenericDatabase can be used with any .NET managed provider (including the ODBC and OLE-DB providers that ship with .NET) – but the catch is that it doesn’t support all of the DAAB functionality. Most significantly, the overloads that use parameter discovery do not work (since these methods aren’t supported on .NET’s DbConnection or DbCommand classes). There’s still the option to build your own Database-derived classes to support this advanced functionality and provide improved database transparency – this is why we built SqlDatabase and OracleDatabase – but at least you’ll be able to use most of the block against any type of database.

Since I installed Firebird, I have been itching to write a custom Database for Firebird in the EntLib 2.0 framework. I know there are customers who want to do this as well because they have Sybase, DB2 in their environment. The GenericDatanase would be an option with ODBC and OLEDB. But for the long term, it makes sense to leverage the native ADO.NET providers if they do exist.

So I started digging into the source code and trying to figure out how a custom Database object would be created by EntLib besides the three ones provided out of the box. Finally I came upon the GetProviderMapping() function of DatabaseConfigurationView class:

 
        public DbProviderMapping GetProviderMapping(string name, string dbProviderName)
        {
            DatabaseSettings settings = this.DatabaseSettings;
            if (settings != null)
            {
                DbProviderMapping existingMapping = settings.ProviderMappings.Get(dbProviderName);
                if (existingMapping != null)
                {
                    return existingMapping;
                }
            }
 
            DbProviderMapping defaultMapping = this.GetDefaultMapping(name, dbProviderName);
            if (defaultMapping != null)
            {
                return defaultMapping;
            }
 
            return this.GetGenericMapping();
        }

This function calls the ProviderMappings property of the DatabaseSettings class:

 
    public class DatabaseSettings : SerializableConfigurationSection
    {

        // code ommited for brevity

        /// <summary>
        /// Holds the optional mappings from ADO.NET's database providers to Enterprise Library's database types.
        /// </summary>
        /// <seealso cref="DbProviderMapping"/>
        [ConfigurationProperty(dbProviderMappingsProperty, IsRequired = false)]
        public NamedElementCollection<DbProviderMapping> ProviderMappings
        {
            get
            {
                return (NamedElementCollection<DbProviderMapping>)base[dbProviderMappingsProperty];
            }
        }
    }

This shows that the configuration drives the creation of the custom Database, if any. So I openned up the EntLib configuration console and now the "Custom Mapping Providers" made sense - all we have to do is to provide a mapping between a name and a custom Database type.

I will detail creating the assembler class along with the custom Database object in a future blog. To see how these all work together, Here is the CreateObject() function of the DatabaseCustomFactory class that provides a clear view how the mapping is queried, then assembler is created from the mapping before the Database gets created. :

 
        public object CreateObject(IBuilderContext context, string name, IConfigurationSource configurationSource, ConfigurationReflectionCache reflectionCache)
        {
            DatabaseConfigurationView configurationView = new DatabaseConfigurationView(configurationSource);
            ConnectionStringSettings connectionStringSettings = configurationView.GetConnectionStringSettings(name);
            DbProviderMapping mapping = configurationView.GetProviderMapping(name, connectionStringSettings.ProviderName);
 
            IDatabaseAssembler assembler = GetAssembler(mapping.DatabaseType, name, reflectionCache);
            Database database = assembler.Assemble(name, connectionStringSettings, configurationSource);
 
            return database;
        }

Firebird ADO.NET Provider

2007-01-20T11:58:00.000-05:00

Recently I have installed open source database Firebird and its ADO.NET provider. I was really impressed and thought this would definitely be my choice of database for future personal projects. I did run into a few things though.

1) The sample on the web site mentioned that you can use the ADO.NET factory model as in the following:

 
            DbProviderFactory factory = DbProviderFactories.GetFactory("FirebirdSql.Data.FirebirdClient");
            using (DbConnection conn = factory.CreateConnection())
            {
                FbConnectionStringBuilder sb = new FbConnectionStringBuilder();
                // code omiited for brevity
            }

In order for this to work, you will need to modify the system.data section of the machine.config file:

 
  <system.data>
    <DbProviderFactories>
      <add name="Odbc Data Provider" invariant="System.Data.Odbc" description=".Net Framework Data Provider for Odbc" type="System.Data.Odbc.OdbcFactory, System.Data, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
      <add name="OleDb Data Provider" invariant="System.Data.OleDb" description=".Net Framework Data Provider for OleDb" type="System.Data.OleDb.OleDbFactory, System.Data, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
      <add name="OracleClient Data Provider" invariant="System.Data.OracleClient" description=".Net Framework Data Provider for Oracle" type="System.Data.OracleClient.OracleClientFactory, System.Data.OracleClient, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
      <add name="SqlClient Data Provider" invariant="System.Data.SqlClient" description=".Net Framework Data Provider for SqlServer" type="System.Data.SqlClient.SqlClientFactory, System.Data, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
      <add name="SQL Server CE Data Provider" invariant="Microsoft.SqlServerCe.Client" description=".NET Framework Data Provider for Microsoft SQL Server 2005 Mobile Edition" type="Microsoft.SqlServerCe.Client.SqlCeClientFactory, Microsoft.SqlServerCe.Client, Version=9.0.242.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91" />
      <add name="Firebird Data Provider" invariant="FirebirdSql.Data.FirebirdClient" description="Firebird" type="FirebirdSql.Data.FirebirdClient.FirebirdClientFactory, FirebirdSql.Data.FirebirdClient, Version=2.0.1.0, Culture=neutral, PublicKeyToken=3750abcc3150b00c" />
    </DbProviderFactories>
  </system.data>

2) The Firebird 2.0 Release note says that security feature is enhanced with password hashing. I was interested in seeing how the ADO.NET client sends credentials to the server to get authenticated so I started investigation using three of my favorite tools: Reflector, Windbg and SOS (if I had had an additional machine I could have used network monitor or something similar to sniff the packets of course). With Reflector, it's not difficult to find that the FirebirdSql.Data.Client.Gds.GdsDatabase.Attach() is the last method on the call chain to send the parameters from connection string to the database server:

 
        public void Attach(DatabaseParameterBuffer dpb, string dataSource, int port, string database)
        {
            lock (this)
            {
                try
                {
                    this.connection.Connect(dataSource, port, this.packetSize, this.charset);
                    this.Identify(database);
                    this.Send.Write(0x13);
                    this.Send.Write(0);
                    this.Send.Write(database);
                    this.Send.WriteBuffer(dpb.ToArray());
                    this.Send.Flush();
                    try
                    {
                        GdsGenericResponse response1 = (GdsGenericResponse)this.ReadResponse();
                        this.handle = response1.ObjectHandle;
                    }
                    catch (IscException)
                    {
                        try
                        {
                            this.connection.Disconnect();
                        }
                        catch
                        {
                        }
                        throw;
                    }
                }
                catch (IOException)
                {
                    this.connection.Disconnect();
                    throw new IscException(0x14000197);
                }
                this.serverVersion = this.GetServerVersion();
            }

There doesn't seem to be hashing or encryption code so I started debugging. In Windbg, I set a break point at the Attach() function:

0:003> !bpmd FirebirdSql.Data.FirebirdClient.dll FirebirdSql.Data.Client.Gds.GdsDatabase.Attach
Found 1 methods...
MethodDesc = 00a28c90
Setting breakpoint: bp 00D62FA0 [FirebirdSql.Data.Client.Gds.GdsDatabase.Attach(FirebirdSql.Data.Common.DatabaseParameterBuffer, System.String, Int32, System.String)]

After the break point is hit, I print out the stack trace and the parameters passed:

0:000> !clrstack -p
OS Thread Id: 0xadc (0)
ESP EIP 
0012f3c8 00d62fa0 FirebirdSql.Data.Client.Gds.GdsDatabase.Attach(FirebirdSql.Data.Common.DatabaseParameterBuffer, System.String, Int32, System.String)
PARAMETERS:
this = 0x013aa6c0
dpb = 0x013aa70c
dataSource = 0x013a9260
port = 0x00000bea
database = 0x013a906c

0012f3d8 00d61f8c FirebirdSql.Data.FirebirdClient.FbConnectionInternal.Connect()
PARAMETERS:
this = 0x013aa67c

0012f408 00d61c32 FirebirdSql.Data.FirebirdClient.FbConnection.Open()
PARAMETERS:
this = 0x013a02c0

0012f438 00d601d3 FbTest.Program.GetJobCode()
0012f478 00d600a9 FbTest.Program.Main(System.String[])
PARAMETERS:
args = 0x01381ccc

0012f69c 79e88f63 [GCFrame: 0012f69c]

Let's just focus on the paramaeters passed to the Attach function, starting from "database":

0:000> !do 0x013a906c
Name: System.String
MethodTable: 790fa3e0
EEClass: 790fa340
Size: 156(0x9c) bytes
(C:\WINDOWS\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)
String: C:\Program Files\Firebird\Firebird_2_0\examples\empbuild\EMPLOYEE.FDB
Fields:
MT Field Offset Type VT Attr Value Name
790fed1c 4000096 4 System.Int32 0 instance 70 m_arrayLength
790fed1c 4000097 8 System.Int32 0 instance 69 m_stringLength
790fbefc 4000098 c System.Char 0 instance 43 m_firstChar
790fa3e0 4000099 10 System.String 0 shared static Empty
>> Domain:Value 0014eb80:790d6584 <<>> Domain:Value 0014eb80:013813dc <<

So that's just the database file path. "port" is an integer:

0:000> ?0x00000bea
Evaluate expression: 3050 = 00000bea

which shows that it uses tcp port number 3050 to communicate with the server. Next we move on to "dataSource":

0:000> !do 0x013a9260
Name: System.String
MethodTable: 790fa3e0
EEClass: 790fa340
Size: 36(0x24) bytes
(C:\WINDOWS\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)
String: localhost
Fields:
MT Field Offset Type VT Attr Value Name
790fed1c 4000096 4 System.Int32 0 instance 10 m_arrayLength
790fed1c 4000097 8 System.Int32 0 instance 9 m_stringLength
790fbefc 4000098 c System.Char 0 instance 6c m_firstChar
790fa3e0 4000099 10 System.String 0 shared static Empty
>> Domain:Value 0014eb80:790d6584 <<>> Domain:Value 0014eb80:013813dc <<

which is server name "localhost". What's left now is the "dpb" parameter:

0:000> !do 013aa70c
Name: FirebirdSql.Data.Common.DatabaseParameterBuffer
MethodTable: 00a2835c
EEClass: 01078910
Size: 16(0x10) bytes
(C:\WINDOWS\assembly\GAC_MSIL\FirebirdSql.Data.FirebirdClient\2.0.1.0__3750abcc3150b00c\FirebirdSql.Data.FirebirdClient.dll)
Fields:
MT Field Offset Type VT Attr Value Name
791098d8 40000c1 4 ...m.IO.MemoryStream 0 instance 013aa71c stream
79104f64 40000c2 8 System.Boolean 0 instance 0 isLittleEndian

Keep dumping the "stream" object referenced by "dpb":


0:000> !do 013aa71c
Name: System.IO.MemoryStream
MethodTable: 791098d8
EEClass: 79109858
Size: 52(0x34) bytes
(C:\WINDOWS\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)
Fields:
MT Field Offset Type VT Attr Value Name
790f9c18 4000184 4 System.Object 0 instance 00000000 __identity
79177788 4001b1d 8 ...ream+ReadDelegate 0 instance 00000000 _readDelegate
79177818 4001b1e c ...eam+WriteDelegate 0 instance 00000000 _writeDelegate
7910ccf4 4001b1f 10 ...ng.AutoResetEvent 0 instance 00000000 _asyncActiveEvent
790fed1c 4001b20 14 System.Int32 0 instance 1 _asyncActiveCount
790fe3c8 4001b1c 55c System.IO.Stream 0 shared static Null
>> Domain:Value 0014eb80:0139e07c <<
79124418 4001bc7 18 System.Byte[] 0 instance 013aa75c _buffer
790fed1c 4001bc8 1c System.Int32 0 instance 0 _origin
790fed1c 4001bc9 20 System.Int32 0 instance 44 _position
790fed1c 4001bca 24 System.Int32 0 instance 44 _length
790fed1c 4001bcb 28 System.Int32 0 instance 256 _capacity
79104f64 4001bcc 2c System.Boolean 0 instance 1 _expandable
79104f64 4001bcd 2d System.Boolean 0 instance 1 _writable
79104f64 4001bce 2e System.Boolean 0 instance 1 _exposable
79104f64 4001bcf 2f System.Boolean 0 instance 1 _isOpen

The "_buffer" should hold what's interesting to us:

0:000> db 013aa75c
013aa75c 18 44 12 79 00 01 00 00-01 3a 04 78 0a 00 00 3f .D.y.....:.x...?
013aa76c 04 03 00 00 00 30 04 4e-6f 6e 65 39 04 00 00 00 .....0.None9....
013aa77c 0f 1c 06 53 59 53 44 42-41 1d 09 6d 61 73 74 65 ...SYSDBA..maste
013aa78c 72 6b 65 79 00 00 00 00-00 00 00 00 00 00 00 00 rkey............
013aa79c 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
013aa7ac 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
013aa7bc 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
013aa7cc 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................

Sure enough, we can see that the user id and password are in the buffer. And password is not hashed.

Starting to blog about .NET ...

2007-01-19T18:54:00.000-05:00

I have finally decided to start blogging. This will be an interesting experience for me as I've never been a diary writer myself. I remember struggling with it when I was at young age. But I have been learning so much from reading other people's technical blogs that I thought not sharing mine would be too selfish. I do have a selfish incentive as well :-) When I come across certain problems that I have researched before, I sometimes find myself have to do the research again simply because I didn't document the first time.

For the last 6 years or so, I have been working for a consulting company, which is focused on delivering Microsoft-centric solutions. As the company grows and I progress in the career path, I have found myself a minority in terms of working hard to keep up both breadth and depth in technologies. This would mean getting hands dirty all the time, which some of my peers in the consulting business just don't have a whole lot of time and passion for anymore.

Being an architect, I have worked on numerous projects with my current company, ranging from COM/C++ to .NET gigs. With the exception of a few, I've had fun with most of them, be that a project of designing and building a customized framework, refactoring/salvaging a poorly-implemented and seemingly-hopeless application, or debugging/diagnosing bugs and performance issues in production environment. Nothing beats the gratification of delivering a successful solution and/or solving the problems for the customers and making them happy in the end.

So with that, I hope that I can start leveraging this blog to share my experiences and knowledge with the community. I think I will have fun doing this and I hope you will find it interesting too.