Scope Timeouts

I had an instance recently where a scope in an orchestration would timeout pretty much instantly despite having a timeout period of 10 seconds. This was a real head-scratcher and I couldn't see where the issue was.

After discussing with a colleague we stumbled upon a difference in the clock time between the application server (where the orchestration runs) and the biztalk message box server. The time difference was almost 10 seconds leading to the "inst ant" timeout. A quick look in the event log indicated a problem with the NTP service and the domain controller (another story!) causing the system time to drift out. Correcting the clock fixed my timeout problem.

Host Startup Failure

In my experience, the failure to start a host instance is most often caused by an error in the BizTalk config file btsntsvc.exe.config. This can be because it is not well-formed XML or the content is incorrect.
For example recently I had to investigate an issue with automated unit test failures which turned out to be caused by an incorrect config file update. A WCF client endpoint had been added outside of the element. The error I saw in the event log was:

A failure occurred when executing a Windows service request.

Service request: Start

BizTalk host name: BizTalkServerApplication
Windows service name: BTSSvc$BizTalkServerApplication

Additional error information:
Error code: 0xc0c0153a
Error source: BizTalk Server 2009
Error description: A BizTalk subservice has failed while executing a service request.

Subservice: Tracking
Service request: Start

Additional error information:
Error code: 0x80131534
Error source: System.Data
Error description: The type initializer for 'System.Data.SqlClient.SqlConnection' threw an exception.

TechEd Europe 2009

I had a great time at TechEd Europe in Berlin in November. Just wanted to say thanks to Stephen Kaufman, Paolo Salvatori, Markus Landler and everyone else on the BizTalk stand for their time and knowledge.

Orchestrations, Serialization and WaitHandles

A common pattern I use in BizTalk development is to create an orchestration with an associated C# library. The orchestration does what it does best (i.e. orchestrate the business process) and the library is where most of the underlying business logic code lives. This also allows the business logic to be called from other non-orchestration based code if required.

In a recent scenario, the library code lent itself well to a multithreaded approach, consisting of a number of small, independent tasks which could be executed in parallel. I implemented this using a ThreadPool and used ManualResetEvent instances to enable the threads to signal when complete. The WaitHandle.WaitAll allowed the main thread to wait until all the tasks were complete before continuing. This worked fine and all my unit tests ran successfully.

Then I tried calling it from an orchestration and got the dreaded serialization error advising that ManualResetEvent is not serializable. This was occurring when I hit a persistence point in the orchestration causing my instance of the library to be serialized, in this case unsuccessfully.

One possible solution to this is the use of an atomic scope to prevent persistence, allowing non-serializable classes to be used. However, this is not recommended and in my case not really practical anyway given the structure of the orchestration. After considering this and other solutions, I finally settled on creating a custom notifier class that would allow the main thread to wait until all threads signalled they are complete. This would perform the same role as ManualResetEvent/WaitHandle but would be serializable.

A simple example of this follows below. The Notifier class has just two methods: SetOne which allows each individual worker thread to signal it is complete, and Wait which enables the main thread to wait until either all worker threads are finished or a timeout period expires. This uses the static Pulse and Wait methods of the System.Threading.Monitor class to manage the signalling and waiting, providing an efficient, but serializable way to handle this pattern without resorting to a poll/sleep loop. The Notifier instance is passed to each worker thread (in much the same way as a ManualResetEvent would) allowing each thread to signal completion.

[Serializable()]
    public class Notifier
    {
        private bool m_blnAllWorkItemsCompleted = false;
        private Int32 m_i32CompletedWorkItems = 0;
        private Int32 m_i32WorkItemCount = 0;
        private object m_locker = new object();

        public Notifier(Int32 workItemCount)
        {
            m_i32WorkItemCount = workItemCount;
        }

        public void SetOne()
        {
            lock (m_locker)
            {
                // Increment the completed work item count
                m_i32CompletedWorkItems++;

                // Check if all work items are complete
                if (m_i32CompletedWorkItems == m_i32WorkItemCount)
                {
                    // All complete so send pulse notification to stop the wait
                    m_blnAllWorkItemsCompleted = true;
                    Monitor.Pulse(m_locker);
                }
            }
        }

        public bool Wait(Int32 timeout)
        {
            lock (m_locker)
            {
                // Wait until the pulse is received or the timeout period expires
                Monitor.Wait(m_locker, timeout);

                // Return an indication of whether the work was completed
                return m_blnAllWorkItemsCompleted;
            }
        }
    }

class Program
{
    const Int32 THREAD_COUNT = 5;

    static void Main(string[] args)
    {
        // Create a new Notifier
        Notifier notifier = new Notifier(THREAD_COUNT);

        for (int i = 0; i < THREAD_COUNT; i++)
        {
            // Create state to pass through to the new thread
            // Includes the notifier so that the thread can signal back
            ThreadState state = new ThreadState(i, notifier);

            // Spawn a new thread to do a single split item
            ThreadPool.QueueUserWorkItem(MyThreadProc, state);
        }

        // Wait for 5 seconds for all threads to complete
        if (notifier.Wait(5000))
        {
            Console.WriteLine("All threads completed");
        }
        else
        {
            Console.WriteLine("Notifier timed out");
        }

        Console.ReadLine();
    }

    private static void MyThreadProc(object stateInfo)
    {
        // Get the state passed in from the main thread
        ThreadState myState = (ThreadState)stateInfo;
        Notifier notifier = myState.Notifier;
        Int32 i32Id = myState.Id;

        // Wait for a random period to simulate work
        Thread.Sleep(RandomNumber(500, 2000));

        // Notify that the work is complete
        Console.WriteLine(String.Format("[{0}] thread complete.", i32Id));
        notifier.SetOne();
    }

    private static int RandomNumber(int min, int max)
    {
        Random random = new Random();
        return random.Next(min, max);
    }
}

[Serializable()]
class ThreadState
{
    public int Id { get; set; }
    public Notifier Notifier { get; set; }

    public ThreadState(int id, Notifier notifier)
    {
        this.Id = id;
        this.Notifier = notifier;
    }
}

I found this article very helpful during my research for this code. Hope you find this useful too.

BizTalk 2009 IDE Issues

In my last post (too long ago but hey I’ve been busy) I stated that the BizTalk 2009 Visual Studio integration is very good. How wrong could I be! I think I may have spoken too soon.

Over the past few months I have experienced numerous problems around project references and orchestration corruption. It’s got to the point where I have to take very regular backups of any orchestration I am working on just in case. This has been the experience of several colleagues too.

Recently I have been testing a patch issued by the BizTalk product team to resolve some of these issues. This has been pretty good and after 3 weeks I had not had any problems, until last week. The same orchestration corruption reappeared though I have been unable to recreate it since.

I’m currently awaiting feedback from the product team regarding this issue. Hopefully soon we’ll all have the IDE experience that we should have had when BizTalk 2009 was released.

Changing variable type in Orchestration Designer

In general the BizTalk 2009 Visual Studio integration is very good, however it doesn't seem very receptive to change. An annoyance I found recently in the Orchestration Designer occurred when I tried to change the type of an existing orchestration variable. This caused the project to no longer compile. The solution was to delete the variable and recreate it. This only took a few seconds but should not be necessary.

Unit Testing With XLANGMessage

As a rule I generally try to pass messages from BizTalk orchestrations into .NET components using the Microsoft.XLANGs.BaseTypes.XLANGMessage object. This is an abstract base class used by the orchestration engine to represent a message and provides a performant way of marshalling message content without having to resort to using an XMLDocument. For more detail on the benefits of XLANGMessage see Jon Flanders' post on the subject.

Anyhow, whilst using XLANGMessage is a real boon for BizTalk based applications, it does raise problems when trying to write unit tests. How do I write a test for a class that uses XLANGMessage in its interface without resorting to creating an orchestration, deploying a BizTalk app, dealing with File adapters etc?

Well the solution I have adopted is to create mock XLANGMessage and XLANGPart classes that use generics to allow a specified object type to be returned when calling the RetrieveAs method. I typically use this in conjunction with a class generated from the underlying message schema using the XSD tool. I can then write simple unit test code such as:

void Test()
{
    // Create and populate a new message class instance.
    // Typically this would be done by deserializing an XML string.
    MyMessageClass msg = new MyMessageClass();

    // Create a new mock XLANGMessage to wrap the message
    MockXLANGMessage<MyMessageClass> xlang 
                = new MockXLANGMessage<MyMessageClass>(msg);

    // Call the method on the class to be tested 
    // passing in the mock XLANGMessage
    MyClassToBeTested.DoSomething(xlang);
}

where the class to be tested looks like:

class MyClassToBeTested
{ 
    static void DoSomething(XLANGMessage msg)
    {
        MyMessageClass reader =
            (MyMessageClass)msg[0].RetrieveAs(typeof(MyMessageClass));
    } 
}

As you can see this is pretty simple code but works very effectively. But the real meat of this is the mock classes. So what do they look like? Here are simple versions which you can enhance to flesh them out if required. The key bit is the use of the generic class which provides the flexibility to wrap any class. This is utilised in the RetrieveAs implementation.

public class MockXLANGPart<T> : XLANGPart
{
    T m_obj;

    public MockXLANGPart(T obj)
    {
        m_obj = obj;
    }

    public override void Dispose()
    {
    }

    public override object GetPartProperty(Type propType)
    {
        throw new NotImplementedException();
    }

    public override Type GetPartType()
    {
        throw new NotImplementedException();
    }

    public override string GetXPathValue(string xpath)
    {
        throw new NotImplementedException();
    }

    public override void LoadFrom(object source)
    {
        throw new NotImplementedException();
    }

    public override string Name
    {
        get { return "MockXLANGPart"; }
    }

    public override void PrefetchXPathValue(string xpath)
    {
        throw new NotImplementedException();
    }

    public override object RetrieveAs(Type t)
    {
        if (t == typeof(T))
        {
            return m_obj;
        }

        return null;
    }

    public override void SetPartProperty(Type propType, object value)
    {
        throw new NotImplementedException();
    }

    public override System.Xml.Schema.XmlSchema XmlSchema
    {
        get { throw new NotImplementedException(); }
    }

    public override System.Xml.Schema.XmlSchemaCollection XmlSchemaCollection
    {
        get { throw new NotImplementedException(); }
    }
}

public class MockXLANGMessage<T> : XLANGMessage
{
    List<MockXLANGPart<T>> m_parts = new List<MockXLANGPart<T>>();

    public MockXLANGMessage(T obj)
    {
        m_parts.Add(new MockXLANGPart<T>(obj));
    }

    public override void AddPart(object part, string partName)
    {
        throw new NotImplementedException();
    }

    public override void AddPart(XLANGPart part, string partName)
    {
        throw new NotImplementedException();
    }

    public override void AddPart(XLANGPart part)
    {
        throw new NotImplementedException();
    }

    public override int Count
    {
        get { return m_parts.Count; }
    }

    public override void Dispose()
    {
    }

    public override System.Collections.IEnumerator GetEnumerator()
    {
        return m_parts.GetEnumerator();
    }

    public override object GetPropertyValue(Type propType)
    {
        throw new NotImplementedException();
    }

    public override string Name
    {
        get { return "MockXLANGMessage"; }
    }

    public override void SetPropertyValue(Type propType, object value)
    {
        throw new NotImplementedException();
    }

    public override XLANGPart this[int partIndex]
    {
        get { return m_parts[partIndex]; }
    }

    public override XLANGPart this[string partName]
    {
        get { return m_parts[0]; }
    }
}

My thanks to Anil Prasad for putting me onto this technique.