Host Startup Failure

In my experience, the failure to start a host instance is most often caused by an error in the BizTalk config file btsntsvc.exe.config. This can be because it is not well-formed XML or the content is incorrect.
For example recently I had to investigate an issue with automated unit test failures which turned out to be caused by an incorrect config file update. A WCF client endpoint had been added outside of the element. The error I saw in the event log was:

A failure occurred when executing a Windows service request.

Service request: Start

BizTalk host name: BizTalkServerApplication
Windows service name: BTSSvc$BizTalkServerApplication

Additional error information:
Error code: 0xc0c0153a
Error source: BizTalk Server 2009
Error description: A BizTalk subservice has failed while executing a service request.

Subservice: Tracking
Service request: Start

Additional error information:
Error code: 0x80131534
Error source: System.Data
Error description: The type initializer for 'System.Data.SqlClient.SqlConnection' threw an exception.

Orchestrations, Serialization and WaitHandles

A common pattern I use in BizTalk development is to create an orchestration with an associated C# library. The orchestration does what it does best (i.e. orchestrate the business process) and the library is where most of the underlying business logic code lives. This also allows the business logic to be called from other non-orchestration based code if required.

In a recent scenario, the library code lent itself well to a multithreaded approach, consisting of a number of small, independent tasks which could be executed in parallel. I implemented this using a ThreadPool and used ManualResetEvent instances to enable the threads to signal when complete. The WaitHandle.WaitAll allowed the main thread to wait until all the tasks were complete before continuing. This worked fine and all my unit tests ran successfully.

Then I tried calling it from an orchestration and got the dreaded serialization error advising that ManualResetEvent is not serializable. This was occurring when I hit a persistence point in the orchestration causing my instance of the library to be serialized, in this case unsuccessfully.

One possible solution to this is the use of an atomic scope to prevent persistence, allowing non-serializable classes to be used. However, this is not recommended and in my case not really practical anyway given the structure of the orchestration. After considering this and other solutions, I finally settled on creating a custom notifier class that would allow the main thread to wait until all threads signalled they are complete. This would perform the same role as ManualResetEvent/WaitHandle but would be serializable.

A simple example of this follows below. The Notifier class has just two methods: SetOne which allows each individual worker thread to signal it is complete, and Wait which enables the main thread to wait until either all worker threads are finished or a timeout period expires. This uses the static Pulse and Wait methods of the System.Threading.Monitor class to manage the signalling and waiting, providing an efficient, but serializable way to handle this pattern without resorting to a poll/sleep loop. The Notifier instance is passed to each worker thread (in much the same way as a ManualResetEvent would) allowing each thread to signal completion.

[Serializable()]
    public class Notifier
    {
        private bool m_blnAllWorkItemsCompleted = false;
        private Int32 m_i32CompletedWorkItems = 0;
        private Int32 m_i32WorkItemCount = 0;
        private object m_locker = new object();

        public Notifier(Int32 workItemCount)
        {
            m_i32WorkItemCount = workItemCount;
        }

        public void SetOne()
        {
            lock (m_locker)
            {
                // Increment the completed work item count
                m_i32CompletedWorkItems++;

                // Check if all work items are complete
                if (m_i32CompletedWorkItems == m_i32WorkItemCount)
                {
                    // All complete so send pulse notification to stop the wait
                    m_blnAllWorkItemsCompleted = true;
                    Monitor.Pulse(m_locker);
                }
            }
        }

        public bool Wait(Int32 timeout)
        {
            lock (m_locker)
            {
                // Wait until the pulse is received or the timeout period expires
                Monitor.Wait(m_locker, timeout);

                // Return an indication of whether the work was completed
                return m_blnAllWorkItemsCompleted;
            }
        }
    }

class Program
{
    const Int32 THREAD_COUNT = 5;

    static void Main(string[] args)
    {
        // Create a new Notifier
        Notifier notifier = new Notifier(THREAD_COUNT);

        for (int i = 0; i < THREAD_COUNT; i++)
        {
            // Create state to pass through to the new thread
            // Includes the notifier so that the thread can signal back
            ThreadState state = new ThreadState(i, notifier);

            // Spawn a new thread to do a single split item
            ThreadPool.QueueUserWorkItem(MyThreadProc, state);
        }

        // Wait for 5 seconds for all threads to complete
        if (notifier.Wait(5000))
        {
            Console.WriteLine("All threads completed");
        }
        else
        {
            Console.WriteLine("Notifier timed out");
        }

        Console.ReadLine();
    }

    private static void MyThreadProc(object stateInfo)
    {
        // Get the state passed in from the main thread
        ThreadState myState = (ThreadState)stateInfo;
        Notifier notifier = myState.Notifier;
        Int32 i32Id = myState.Id;

        // Wait for a random period to simulate work
        Thread.Sleep(RandomNumber(500, 2000));

        // Notify that the work is complete
        Console.WriteLine(String.Format("[{0}] thread complete.", i32Id));
        notifier.SetOne();
    }

    private static int RandomNumber(int min, int max)
    {
        Random random = new Random();
        return random.Next(min, max);
    }
}

[Serializable()]
class ThreadState
{
    public int Id { get; set; }
    public Notifier Notifier { get; set; }

    public ThreadState(int id, Notifier notifier)
    {
        this.Id = id;
        this.Notifier = notifier;
    }
}

I found this article very helpful during my research for this code. Hope you find this useful too.

BizTalk 2009 IDE Issues

In my last post (too long ago but hey I’ve been busy) I stated that the BizTalk 2009 Visual Studio integration is very good. How wrong could I be! I think I may have spoken too soon.

Over the past few months I have experienced numerous problems around project references and orchestration corruption. It’s got to the point where I have to take very regular backups of any orchestration I am working on just in case. This has been the experience of several colleagues too.

Recently I have been testing a patch issued by the BizTalk product team to resolve some of these issues. This has been pretty good and after 3 weeks I had not had any problems, until last week. The same orchestration corruption reappeared though I have been unable to recreate it since.

I’m currently awaiting feedback from the product team regarding this issue. Hopefully soon we’ll all have the IDE experience that we should have had when BizTalk 2009 was released.

How to find the BizTalk Edition installed

I recently had to find the edition of an existing BizTalk installation on a test server. Initially I thought this would be a case of looking at a few Help > About dialogs in the admin tools. However, it turns out that this is stored in the registry under

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\BizTalk Server\3.0\ProductEdition

MSBTS_SendAdapter doesn't work!

Just a quick post as a reminder to me (and a pointer for anyone else who happens upon this blog) to save lots of time scratching my head, looking at code that should work but doesn't.

I wrote some code as part of a BizTalk deployment framework which was intended to create an adapter handler if one does not already exist for the specified adapter/host. The Receive handler version worked fine but the Send version gave some strange results.

First I queried the existence of the send handler via WMI using MSBTS_SendHandler. This always returned an empty set indicating that it doesn't (even when the handler does in fact exist). When trying to create a send handler I got an error message (will update with exact text when I get a chance to look).

Anyway, to cut a long story short it turns out that MSBTS_SendHandler2 works fine so the guidance is to use this instead. My question is though - why does MSBTS_SendHandler still exist at all? It can't be for backwards compatibility because IT DOESN'T WORK! Why not just fix MSBTS_SendHandler instead of adding MSBTS_SendHandler2? Answers on a postcard to...

Resolving Schema Type Name Clashes

A common schema related error that may occur when building BizTalk projects that include schemas is:

BEC2017: Node "<Schema>" - This schema file has a TypeName that collides with the RootNode TypeName of one of its root nodes. Make sure that they are different.

The reason for this is quite simple - when the project is compiled, a .NET class will be generated for each top level element in your schema. If you have two schemas which share the same top level element name a clash will occur.

This is quite a common scenario and it is not always practical (or even possible) to change the element names. The solution is to change the generated type name for the schema. To do so, select the schema in the Solution Explorer and press F4 to display the Properties window. Then change the Type Name property so that it is not the same as any of the top level elements.

How can I use Fiddler to debug BizTalk messages?

When using HTTP, SOAP or WCF send ports it can be incredibly useful to view the content of messages that are sent on the wire. This allows you to inspect the headers (SOAP or HTTP). Fiddler is a simple but fantastic tool to allow you to do this.

By default, Fiddler will not trace any messages sent to endpoints by BizTalk as it does not use WinInet. However, BizTalk send ports can be configured to use a proxy allowing Fiddler to intercept them. On the Send Port tick the Use Proxy checkbox and set the Server to 127.0.0.1 and the port to 8888. For dynamic ports, set the following properties (as applicable to the adapter being used)

// Debug via fiddler
msgSendRequest(SOAP.UseProxy) = true;
msgSendRequest(SOAP.ProxyAddress) = "127.0.0.1";
msgSendRequest(SOAP.ProxyPort) = 8888;

Note that this needs to be removed when Fiddler is not running since traffic directed to the proxy will not be received by anything.

Promoting Properties that are not in a Message

In most instances properties are promoted from message content during the disassemble stage of a pipeline. However, in some cases it is useful to promote a property that does not relate to the content of a message. For example you may want to generate a unique identifier for a transaction and promote that into context for later use. To do so:

• Create a new property schema (or open an existing one)
• Add a new Child Field Element named with the required property name
• In the properties window, set the "Property Schema Base" property to MessageContextPropertyBase. This is the key to making this work. The default value is MessageDataPropertyBase indicating that the value is coming from a message
• In the orchestration or pipeline, set the new property value programmatically

See Stephen W Thomas's blog entry for more on this.

Dynamic SOAP Adapters

Following on from my previous post concerning dynamic HTTP adapters, this time I'll extend the same concept to the SOAP adapter. Many of the techniques previously presented are still applicable but the SOAP adapter has it's own little idiosyncrasies which once explained are easy enough to overcome.

The first issue is that of a proxy. In traditional .NET applications, when using a web service it is typical to add a web reference which, under the covers will generate a proxy to the remote service based on its WSDL definition. This is also what would happen if you add a web reference in your orchestration and use a static SOAP send port. However, for a dynamic send port we don't want to create a web reference - we'll generate the proxy manually using the wsdl.exe tool as follows:

C:\>wsdl /l:CS /o:c:\temp\wsdl http://localhost/CalcWebService/CalcService.asmx?wsdl
Microsoft (R) Web Services Description Language Utility
[Microsoft (R) .NET Framework, Version 2.0.50727.42]
Copyright (C) Microsoft Corporation. All rights reserved.
Writing file 'c:\temp\wsdl\CalcService.cs'.

Here we have generated a proxy class from the WSDL of the web service. Add this to a new Class Library project. Ensure the resulting assembly is strong named and add it to the GAC. Note that the WebMethod to be called should have a signature as below that will accept an XML message and return one too. If an alternative method signature is required it will be necessary to update the proxy class to act as an facade between the required signature and the web service one.

[WebMethod]
public XmlNode MyMethod(XmlNode Message)

As with the HTTP example, we'll be invoking a dynamic adapter from an orchestration, setting adapter properties to change its behaviour at runtime. Again, add a request/response send port to the orchestration port surface and mark its type as Dynamic. The only other configuration required on the port is to specify the pipelines to be used, in this case XmlTransmit and XmlReceive. Connect up the request and response operations to the relevant send and receive shapes in the orchestration. Note that the operation on the send port should have the same name as the WebMethod being called.

Prior to sending the message set the following send port properties:

// Assign message
msgSendRequest = msgReceiveRequest;

// Set base properties
MySendPort(Microsoft.XLANGs.BaseTypes.Address) =
                     "http://localhost/CalcWebService/CalcService.asmx";
MySendPort (Microsoft.XLANGs.BaseTypes.TransportType) = "SOAP";

// Set HTTP adapter specific properties
msgSendRequest(BTS.SOAPAction) = "Add";
msgSendRequest(SOAP.MethodName) = "Add";
msgSendRequest(SOAP.AssemblyName) = "CalcTest.CalcWebServiceProxy.CalcService,
    CalcWebServiceProxy, Version=1.0.0.0, Culture=neutral,
    PublicKeyToken=b713f1108bae3109";
msgSendRequest(SOAP.TypeName) = "CalcTest.CalcWebServiceProxy.CalcService";

Some important points to note here:

1. The TransportType is a means of informing BizTalk which adapter you want to use. If it is omitted the prefix of the Address (in this case http://) will be used to query an alias table to lookup the adapter to use. By default this will use the HTTP adapter, therefore we set the Microsoft.XLANGs.BaseTypes.TransportType property to "SOAP" to ensure that the SOAP adapter is targeted.
2. The BTS.SOAPAction and the SOAP.MethodName are both set to the name of the WebMethod being invoked.
3. The SOAP.AssemblyName must be populated with the details of the class name and assembly of the proxy to be used. This value takes the form Namespace.ClassName, Fully Qualified Assembly Name. Set this to the details of the proxy assembly created and GAC'd earlier.
4. The SOAP.TypeName must also be set to the full name of the proxy class i.e. Namespace.ClassName. Not sure why this is required again but if it is not set an error occurs.

OK, now you should be good to go. Build, deploy and (hopefully) watch those SOAP requests flying to and from your web service. Whilst it's a little more fiddly to setup that the HTTP equivalent, using a dynamic SOAP adapter is a really powerful technique for creating flexible and extensible BizTalk applications.

Jon Fancey has taken this a step further by creating a more generic SOAP router. Check it out.

Dynamic Send Ports

Part of the project I'm currently working on involves sending XML messages to a variety of trading partners. Each partner has several products each of which could require different port settings (URL, credentials, timeout, retry strategy). Another variable is the transport type. Currently all support plain XML over HTTP but in the future SOAP may be a requirement. As we have up to 1000 different partner/product combinations to support dynamic ports seem like a logical choice.

So how do you implement dynamic HTTP and SOAP send ports? Let's start with HTTP first as it's a little more straightforward.

Assuming that the orchestration is in place, add a request/response send port to the port surface and mark its type as Dynamic. The only other configuration required on the port is to specify the pipelines to be used and to connect up the request and response operations to the relevant send and receive shapes in the orchestration.

Prior to the send shape (I have used the message assignment shape that builds the request message) add the following code.

// Assign message
msgSendRequest = msgReceiveRequest;

// Set base properties
MySendPort(Microsoft.XLANGs.BaseTypes.Address) = "http://localhost/testpartner/test.aspx";
MySendPort(Microsoft.XLANGs.BaseTypes.TransportType) = "HTTP";

// Set HTTP adapter specific properties
msgSendRequest(HTTP.AuthenticationScheme) = "Basic";
msgSendRequest(HTTP.Username) = "mark";
msgSendRequest(HTTP.Password) = "pwd";

This code sets up the address URI and the transport type to be used. The TransportType is a means of informing BizTalk which adapter you want to use. If it is omitted the prefix of the Address (in this case http://) will be used to query an alias table to lookup the adapter to use. For some prefixes there are multiple adapters that could handle the request. For http:// it could be the HTTP, WCF-Basic or WCF-WSHttp so the TransportType property can be used to explictly choose the correct adapter. In this instance I could have omitted the TransportType since the HTTP is the default for http:// addresses.

Next I have set some properties that are specific to the HTTP adapter. In this instance I have just configured the authentication details but there are lots of other that can be used. Typically these settings could be looked up from a database or other configuration store and set dynamically based on the request.

And that's it! The request should then be routed to the URI configured in code above using the HTTP adapter. This is a very powerful technique to add flexibility to your application. I'll continue by demonstrating how to use the same technique for the SOAP adapter in a future post.

Installation Gotchas

When installing BizTalk Server 2006, the process is generally pretty smooth. I have installed it a couple of times on different machines and had no problems to speak of. However, there are a few things to look out for if you do encounter problems:

• Check the event log is not full. BizTalk Server is not shy about writing to the event log so make sure that the Application log is not full and supports overwriting of events as necessary.
• Make sure you are not logged on with a Windows account that has no password. The Master Secret Server Key is generated using the password of the logged on user as a seed.
• Do not install onto a non-NTFS formatted disk. BizTalk Server does not support FAT32.
• If using a domain account, ensure that access to the domain controller is available.
• Ensure you have enough available disk space. The System Requirements state "15 GB of available hard disk space for a complete installation including the operating system, all prerequisite software, and language packs. This does not include disk space for data storage."