TechEd Europe 2009

I had a great time at TechEd Europe in Berlin in November. Just wanted to say thanks to Stephen Kaufman, Paolo Salvatori, Markus Landler and everyone else on the BizTalk stand for their time and knowledge.

Orchestrations, Serialization and WaitHandles

A common pattern I use in BizTalk development is to create an orchestration with an associated C# library. The orchestration does what it does best (i.e. orchestrate the business process) and the library is where most of the underlying business logic code lives. This also allows the business logic to be called from other non-orchestration based code if required.

In a recent scenario, the library code lent itself well to a multithreaded approach, consisting of a number of small, independent tasks which could be executed in parallel. I implemented this using a ThreadPool and used ManualResetEvent instances to enable the threads to signal when complete. The WaitHandle.WaitAll allowed the main thread to wait until all the tasks were complete before continuing. This worked fine and all my unit tests ran successfully.

Then I tried calling it from an orchestration and got the dreaded serialization error advising that ManualResetEvent is not serializable. This was occurring when I hit a persistence point in the orchestration causing my instance of the library to be serialized, in this case unsuccessfully.

One possible solution to this is the use of an atomic scope to prevent persistence, allowing non-serializable classes to be used. However, this is not recommended and in my case not really practical anyway given the structure of the orchestration. After considering this and other solutions, I finally settled on creating a custom notifier class that would allow the main thread to wait until all threads signalled they are complete. This would perform the same role as ManualResetEvent/WaitHandle but would be serializable.

A simple example of this follows below. The Notifier class has just two methods: SetOne which allows each individual worker thread to signal it is complete, and Wait which enables the main thread to wait until either all worker threads are finished or a timeout period expires. This uses the static Pulse and Wait methods of the System.Threading.Monitor class to manage the signalling and waiting, providing an efficient, but serializable way to handle this pattern without resorting to a poll/sleep loop. The Notifier instance is passed to each worker thread (in much the same way as a ManualResetEvent would) allowing each thread to signal completion.

[Serializable()]
    public class Notifier
    {
        private bool m_blnAllWorkItemsCompleted = false;
        private Int32 m_i32CompletedWorkItems = 0;
        private Int32 m_i32WorkItemCount = 0;
        private object m_locker = new object();

        public Notifier(Int32 workItemCount)
        {
            m_i32WorkItemCount = workItemCount;
        }

        public void SetOne()
        {
            lock (m_locker)
            {
                // Increment the completed work item count
                m_i32CompletedWorkItems++;

                // Check if all work items are complete
                if (m_i32CompletedWorkItems == m_i32WorkItemCount)
                {
                    // All complete so send pulse notification to stop the wait
                    m_blnAllWorkItemsCompleted = true;
                    Monitor.Pulse(m_locker);
                }
            }
        }

        public bool Wait(Int32 timeout)
        {
            lock (m_locker)
            {
                // Wait until the pulse is received or the timeout period expires
                Monitor.Wait(m_locker, timeout);

                // Return an indication of whether the work was completed
                return m_blnAllWorkItemsCompleted;
            }
        }
    }

class Program
{
    const Int32 THREAD_COUNT = 5;

    static void Main(string[] args)
    {
        // Create a new Notifier
        Notifier notifier = new Notifier(THREAD_COUNT);

        for (int i = 0; i < THREAD_COUNT; i++)
        {
            // Create state to pass through to the new thread
            // Includes the notifier so that the thread can signal back
            ThreadState state = new ThreadState(i, notifier);

            // Spawn a new thread to do a single split item
            ThreadPool.QueueUserWorkItem(MyThreadProc, state);
        }

        // Wait for 5 seconds for all threads to complete
        if (notifier.Wait(5000))
        {
            Console.WriteLine("All threads completed");
        }
        else
        {
            Console.WriteLine("Notifier timed out");
        }

        Console.ReadLine();
    }

    private static void MyThreadProc(object stateInfo)
    {
        // Get the state passed in from the main thread
        ThreadState myState = (ThreadState)stateInfo;
        Notifier notifier = myState.Notifier;
        Int32 i32Id = myState.Id;

        // Wait for a random period to simulate work
        Thread.Sleep(RandomNumber(500, 2000));

        // Notify that the work is complete
        Console.WriteLine(String.Format("[{0}] thread complete.", i32Id));
        notifier.SetOne();
    }

    private static int RandomNumber(int min, int max)
    {
        Random random = new Random();
        return random.Next(min, max);
    }
}

[Serializable()]
class ThreadState
{
    public int Id { get; set; }
    public Notifier Notifier { get; set; }

    public ThreadState(int id, Notifier notifier)
    {
        this.Id = id;
        this.Notifier = notifier;
    }
}

I found this article very helpful during my research for this code. Hope you find this useful too.

BizTalk 2009 IDE Issues

In my last post (too long ago but hey I’ve been busy) I stated that the BizTalk 2009 Visual Studio integration is very good. How wrong could I be! I think I may have spoken too soon.

Over the past few months I have experienced numerous problems around project references and orchestration corruption. It’s got to the point where I have to take very regular backups of any orchestration I am working on just in case. This has been the experience of several colleagues too.

Recently I have been testing a patch issued by the BizTalk product team to resolve some of these issues. This has been pretty good and after 3 weeks I had not had any problems, until last week. The same orchestration corruption reappeared though I have been unable to recreate it since.

I’m currently awaiting feedback from the product team regarding this issue. Hopefully soon we’ll all have the IDE experience that we should have had when BizTalk 2009 was released.

Changing variable type in Orchestration Designer

In general the BizTalk 2009 Visual Studio integration is very good, however it doesn't seem very receptive to change. An annoyance I found recently in the Orchestration Designer occurred when I tried to change the type of an existing orchestration variable. This caused the project to no longer compile. The solution was to delete the variable and recreate it. This only took a few seconds but should not be necessary.

Unit Testing With XLANGMessage

As a rule I generally try to pass messages from BizTalk orchestrations into .NET components using the Microsoft.XLANGs.BaseTypes.XLANGMessage object. This is an abstract base class used by the orchestration engine to represent a message and provides a performant way of marshalling message content without having to resort to using an XMLDocument. For more detail on the benefits of XLANGMessage see Jon Flanders' post on the subject.

Anyhow, whilst using XLANGMessage is a real boon for BizTalk based applications, it does raise problems when trying to write unit tests. How do I write a test for a class that uses XLANGMessage in its interface without resorting to creating an orchestration, deploying a BizTalk app, dealing with File adapters etc?

Well the solution I have adopted is to create mock XLANGMessage and XLANGPart classes that use generics to allow a specified object type to be returned when calling the RetrieveAs method. I typically use this in conjunction with a class generated from the underlying message schema using the XSD tool. I can then write simple unit test code such as:

void Test()
{
    // Create and populate a new message class instance.
    // Typically this would be done by deserializing an XML string.
    MyMessageClass msg = new MyMessageClass();

    // Create a new mock XLANGMessage to wrap the message
    MockXLANGMessage<MyMessageClass> xlang 
                = new MockXLANGMessage<MyMessageClass>(msg);

    // Call the method on the class to be tested 
    // passing in the mock XLANGMessage
    MyClassToBeTested.DoSomething(xlang);
}

where the class to be tested looks like:

class MyClassToBeTested
{ 
    static void DoSomething(XLANGMessage msg)
    {
        MyMessageClass reader =
            (MyMessageClass)msg[0].RetrieveAs(typeof(MyMessageClass));
    } 
}

As you can see this is pretty simple code but works very effectively. But the real meat of this is the mock classes. So what do they look like? Here are simple versions which you can enhance to flesh them out if required. The key bit is the use of the generic class which provides the flexibility to wrap any class. This is utilised in the RetrieveAs implementation.

public class MockXLANGPart<T> : XLANGPart
{
    T m_obj;

    public MockXLANGPart(T obj)
    {
        m_obj = obj;
    }

    public override void Dispose()
    {
    }

    public override object GetPartProperty(Type propType)
    {
        throw new NotImplementedException();
    }

    public override Type GetPartType()
    {
        throw new NotImplementedException();
    }

    public override string GetXPathValue(string xpath)
    {
        throw new NotImplementedException();
    }

    public override void LoadFrom(object source)
    {
        throw new NotImplementedException();
    }

    public override string Name
    {
        get { return "MockXLANGPart"; }
    }

    public override void PrefetchXPathValue(string xpath)
    {
        throw new NotImplementedException();
    }

    public override object RetrieveAs(Type t)
    {
        if (t == typeof(T))
        {
            return m_obj;
        }

        return null;
    }

    public override void SetPartProperty(Type propType, object value)
    {
        throw new NotImplementedException();
    }

    public override System.Xml.Schema.XmlSchema XmlSchema
    {
        get { throw new NotImplementedException(); }
    }

    public override System.Xml.Schema.XmlSchemaCollection XmlSchemaCollection
    {
        get { throw new NotImplementedException(); }
    }
}

public class MockXLANGMessage<T> : XLANGMessage
{
    List<MockXLANGPart<T>> m_parts = new List<MockXLANGPart<T>>();

    public MockXLANGMessage(T obj)
    {
        m_parts.Add(new MockXLANGPart<T>(obj));
    }

    public override void AddPart(object part, string partName)
    {
        throw new NotImplementedException();
    }

    public override void AddPart(XLANGPart part, string partName)
    {
        throw new NotImplementedException();
    }

    public override void AddPart(XLANGPart part)
    {
        throw new NotImplementedException();
    }

    public override int Count
    {
        get { return m_parts.Count; }
    }

    public override void Dispose()
    {
    }

    public override System.Collections.IEnumerator GetEnumerator()
    {
        return m_parts.GetEnumerator();
    }

    public override object GetPropertyValue(Type propType)
    {
        throw new NotImplementedException();
    }

    public override string Name
    {
        get { return "MockXLANGMessage"; }
    }

    public override void SetPropertyValue(Type propType, object value)
    {
        throw new NotImplementedException();
    }

    public override XLANGPart this[int partIndex]
    {
        get { return m_parts[partIndex]; }
    }

    public override XLANGPart this[string partName]
    {
        get { return m_parts[0]; }
    }
}

My thanks to Anil Prasad for putting me onto this technique.

Where did I leave my HAT?

Following an upgrade to BizTalk 2009 you may be left wondering where the Health and Activity Tracking (HAT) Tool has disappeared to. The Start Menu group for BizTalk 2009 no longer contains a link to HAT.

The answer lies in the BizTalk Administration Console MMC Snap-in. If you navigate to the Group Hub and select New Query you will see some new entries in the Value column such as Tracked Service Instances. This will retrieve data in the same way that the HAT queries did.

Right clicking on one of the resultset entries will offer the same options as HAT used to e.g. Message Flow, Orchestration Debugger. Navigate into one of these options presents the same interface as HAT always used to.

In fact HAT is still there under the covers, it's just been integrated into the Admin Console rather than running as a separate application. Try looking in the Task Manager list of running processes and you'll see BTSHatApp.exe.

HAT is dead...long live HAT.

How to find the BizTalk Edition installed

I recently had to find the edition of an existing BizTalk installation on a test server. Initially I thought this would be a case of looking at a few Help > About dialogs in the admin tools. However, it turns out that this is stored in the registry under

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\BizTalk Server\3.0\ProductEdition

MSBTS_SendAdapter doesn't work!

Just a quick post as a reminder to me (and a pointer for anyone else who happens upon this blog) to save lots of time scratching my head, looking at code that should work but doesn't.

I wrote some code as part of a BizTalk deployment framework which was intended to create an adapter handler if one does not already exist for the specified adapter/host. The Receive handler version worked fine but the Send version gave some strange results.

First I queried the existence of the send handler via WMI using MSBTS_SendHandler. This always returned an empty set indicating that it doesn't (even when the handler does in fact exist). When trying to create a send handler I got an error message (will update with exact text when I get a chance to look).

Anyway, to cut a long story short it turns out that MSBTS_SendHandler2 works fine so the guidance is to use this instead. My question is though - why does MSBTS_SendHandler still exist at all? It can't be for backwards compatibility because IT DOESN'T WORK! Why not just fix MSBTS_SendHandler instead of adding MSBTS_SendHandler2? Answers on a postcard to...