Tag Archive: BizTalk Pipelines

A couple of months ago I released the BRE Pipeline Framework project onto codeplex, the goal of the project being to make it easier to inspect or manipulate messages in pipelines through the use of the Business Rules Engine in BizTalk.

Today I have released v1.2.0 of the framework and you can download the new installer from the codeplex project page. The installation steps described on the codeplex project documentation page are applicable regardless whether you are a first time installer or not, and they will not require you to revise your existing policies at all.

The most promising update in this release is the introduction of Recoverable Interchange Processing to the pipeline component. When Recoverable Interchange Processing is turned on, if an envelope message is processed by a disassembler type component and results in the interchange being debatched into multiple individual messages and a failure is encountered in the BRE Pipeline Framework pipeline component (in a stage after the disassemble) then only the message(s) that encountered the failure will suspend (each failed message showing up as a separate suspended instance) along with the relevant error text for that specific message, while the ones that do not encounter errors will process successfully. This applies to exceptions encountered in the BRE Policies or to routing failure which are a result of there being no subscriptions for the message.

In contrast, if Recoverable Interchange Processing is turned off then if a single message encounters an error then the entire interchange will fail with a single suspended instance created for the interchange with a generic error message. Note that Recoverable Interchange Processing should never be enabled on a pipeline which will be exercised within an orchestration.

A new boolean parameter (default value = false) called RecoverableInterchangeProcessing has been introduced in the BRE Pipeline Framework pipeline component which allows you to control the Recoverable Interchange Processing feature. After installing the new version of the BRE Pipeline Framework it is possible that the new parameter might not show up in existing pipelines that make use of the component, if this is the case then delete the reference to the BREPipelineFrameworkComponent assembly and re-add it to your project (the updated assembly should be in the C:\Program Files (x86)\Microsoft BizTalk Server 2010\Pipeline Components or equivalent folder).

New Property

Another update is that all the get context property type methods (exposed in the BREPipelineFramework.SampleInstructions.ContextInstructions vocabulary) have now been updated such that they possess improved error handling. Previously if the context property didn’t exist and you instructed that an exception should be thrown (in contrast to a blank or other default value being returned), then the exception seen against the suspended instance would be very generic as below, making it hard to determine which specific condition in which specific rule caused the failure.


The new error message format tells you exactly which context property caused the problem.


Lastly, I have updated the default value of the Enabled property on the pipeline component to true. This was previously set to false and resulted in the pipeline component not executing at all unless the developer remembered to enable the component. Instead now it is up to the developer to disable the component if he doesn’t want to use it.

Please post any feedback or ideas on the codeplex project discussions page.

A colleague of mine was implementing a system which required a generic XML on ramp that accepted messages from a variety of trading partners. He quickly encountered the age-old problem of not so sophisticated trading partners sending XML documents which contained no namespaces. This can quickly become a nightmare scenario in the world of BizTalk Server as messages are identified through a combination of the namespace and the root node of the XML document. If multiple trading partners send us documents with the same root node name then it would become very difficult to identify which schema a message is based on.

My colleague’s plan was to probe the incoming messages with regular expressions in a pipeline component and based on the result to promote a context property to the message, and then have a further pipeline component read in that context property and use it to set the namespace of the document’s root element before the message was processed by the XML Disassembler pipeline component. I advised him that he might be safer using XPath to probe the structure of the document rather than relying on regular expressions based on the below reasons.

  • Using regular expressions would most likely require the entire document to be read into a string to probe rather than making use of stream based probing such as the XPathMutatorStream.
  • Using regular expressions wouldn’t be quite as reliable or easy to setup for the purpose of probing the structure of an XML message as it would be if we were using XPath.

I advised him that BRE Pipeline Framework which I have only recently released would quite easily cater for the probing of the documents as it can run XPath statements against the incoming document to extract element values, names, or namespaces, and the only additional piece of work would be to write a set of MetaInstructions/Instructions to add the namespace to the root node of the document. He told me that he would be interested in exploring this option so here I am 🙂

An example of the problem is that we might receive Invoice documents from our trading partners, where the structures of the documents are different however the root node for both of them are Invoice and they contain no namespace. What should ideally be done is that both of these schemas should be deployed to the BizTalk runtime however each should be given their own namespaces. Example instances of the XML documents as they are sent by the trading partners are as below.

XML Instances

The goal of this blog post is to insert a namespace and prefix into the root node of the document depending on the name of the very first child node under the root node. This should be done in the pipeline prior to the XML Disassembler pipeline component trying to lookup the schema for the document in question.

The first step would be to write an instruction to actually execute the addition of the namespace to the XML Document. I decided that I would write the instruction such that the implementor could also specify the namespace prefix, or allow the namespace prefix to default to ns0. I pretty much lifted the source code from Maxime Labelle’s response on this StackOverflow post, modifying code to also allow for the specification of the namespace prefix.

The instruction class now looks like the below (note the use of the AddXmlNamespaceStream which I ported from the aforementioned StackOverflow post and in which I added an extra constructor to also allow for the setting of the namespace prefix).


Next up I created a MetaInstruction with two methods, one which allows for the setting of the namespace, and one which allows for the setting of the namespace as well as the namespace prefix. As you can see from the below snippet, pretty much all these methods are doing is calling the relevant constructor for the Instruction I mentioned above and then adding the instantiated instruction to the base class’ collection.


After signing the assembly and adding it to the GAC, the next step was to create a vocabulary definition for the two methods in the MetaInstruction class (one thing to remember is that after adding an assembly to the GAC the BRE Composer must be closed and reopened so that it can fetch the new assembly from the GAC). Below is the vocabulary definition for the AddDocumentNamespaceAndPrefix method.

Vocabulary Definition

One of the key features of vocabulary definitions is to give them friendly display text so that rules are easy to create and read. The aforementioned vocabulary was given a friendly display text as below.

Friendly Name

Once the vocabulary is published it’s time to setup our BRE Policies. The first policy we need to create is the InstructionLoaderPolicy which needs to instantiate the MetaInstruction class that we mentioned above as well as the BREPipelineFramework.SampleInstructions.MetaInstructions.HelperMetaInstructions class which allows us to probe the incoming message using XPath statements. I created a rule that will check the ApplicationContext (remember that this is set in the pipeline at design time or overridden for a specific pipeline instance in the BizTalk Administration Console) and if it matches the expected value it will add the aforementioned MetaInstructions to the collection. When adding the MetaInstruction to the collection two parameters must be specified which are the fully qualified class name of the MetaInstruction (the .NET namespace and the class name) as well as the fully qualified assembly name (which can be retrieved by typing in “gacutil -l {assembly name minus .dll}” into a Visual Studio Command Prompt). Instantiating these objects allows for the usage of their contained methods in the Execution Policy.

Next up we will create our execution policy containing two rules, one for each of the document types. Note that the conditions for both rules evaluate the application context, make use of the GetXPathResult vocabulary definition from the BREPipelineFramework.SampleInstructions.MetaInstructions.HelperMetaInstructions vocabulary, and the actions make use of the vocabulary definitions to add the namespace / namespace and prefix.


And to pull it all together we need to actually create our receive pipeline that is to be used on the receive location. It should look like the below (note the properties that have been set for the BRE Pipeline Framework component).

Receive Pipeline

After deploying and putting the solution together, and passing in the input files mentioned at the start of the blog post you should see that the files have now been modified so that they look like the below.


If you are interested in playing with this example, I have uploaded the source code to my google drive. Note that you will first need to install the BRE Pipeline Framework as documented on the Codeplex project documentation page, and then you can import/install the MSI in the solution items folder in the linked source code to install this example. I haven’t done any performance testing on this solution at all, and have only run a few functional tests on it, however I do hope that it helps illustrate how the BRE Pipeline Framework can be extended to enable some pretty neat conditional actions.

%d bloggers like this: