I have just published a new version of the BRE Pipeline Framework to the Codeplex site. v1.3.1 has been a stable version and has been in use in many BizTalk projects that I know of in production systems and has been successfully used to conditionally get and set message context property values.
v1.4.0 is somewhat of a revolution compared to previous updates in terms of the number and utility of the new features it delivers. I will discuss them in more detail below, but if you don’t want to read the wall of text then in summary the most important features are the ability to make use of XML and SQL facts to perform lookups or updates within the message body and to SQL databases, the ability to write out rule tracing data to the file system when trying to debug your policies, the addition of BizTalk 2013 context properties to enumerations, the addition of many more message body evaluation and manipulation features, and that the pipeline component is now signed with a strong named key and will be installed into the GAC upon installation which makes the deployment of new versions much easier as they will not require for BizTalk host instances to be stopped during the install, just restarted after. This skims the surface of new features so read on if you are interested.
On to the new features…
XML facts
You are now able to make use of XML based facts in your Execution Policies, just as you would within a BizTalk orchestration. The one catch is that in order to do so you must assert the type of the XML document in the Instruction Loader Policy so that an appropriate TypedXMLDocument instance, which is the underlying class that BizTalk Orchestrations convert your message into when they call BRE Policies from the Call Rules shape, can be created and asserted into the Execution Policy.
In order to aid you in the decision as to which schema type to apply to the TypedXMLDocument a few new vocabulary definitions have been provided which allow you to get the root node name and namespace in the BREPipelineFramework vocabulary (GetMessageRootNodeName and GetMessageRootNodeNamespace). If available these utilities will extract the value from the BTS.MessageType context property and if not then these utilities will attempt to run XPATH statements (obviously these are more expensive so best to use an XML Disassembler pipeline component first to resolve the message type if possible) against the message body to extract the values. Note that these utilities can only be used within Instruction Loader policies, and can be used in conditions while the typing of the TypedXMLDocument instance can be done as the action which is the result of successful condition evaluation. See below for an example.
This leads to some pretty powerful scenarios as you can now extract values from your XML message body in your execution policy and you can update values as well. It also means that you can reuse vocabularies which were originally created for usage within orchestrations only.
Database facts
You can now make use of database facts in your execution policies, which needless to say makes database lookups within your BRE rules a much simpler task. Much like the TypedXMLDocuments you will need to assert the DataConnection instance in the Instruction Loader policy so that the corresponding database facts can be used in your Execution Policy. This is similar to the usage of database facts in BizTalk Orchestrations. New vocabulary definitions have been provided in the BREPipelineFramework vocabulary which allow you to instantiate and assert a DataConnection instance by passing in a connection string (or getting it from an SSO config store if not using integrated security and wanting to protect credentials, or want to use SSO config stores to hold per environment information so your BRE policies can be consistent across environments), database name, and a table name. These vocabulary definitions are called AddSqlDataConnection and AddSqlDataConnectionFromSSO and an example of their usage is below.
String evaluations in Instruction Loader Policies now available
The below vocabulary definitions are now available within the BREPipelineFramework vocabulary for use within Instruction Loader Policies only. These methods require for the message body to be cast to a string which is a relatively expensive operations, especially if you expect your message bodies to be large so please use this with caution and test appropriately. The casting will only happen one time regardless how many times you make use of the vocabulary definitions within a given Instruction Loader Policy.
CheckIfRegexEvaluatesInMessage – Returns a Boolean value which indicates whether a regular expression finds a match in the BizTalk message body.
CheckIfStringExistsInMessage – Returns a Boolean value which indicates whether a substring exists in the BizTalk message body.
MessageBodyLength – Returns an integer value which indicates the string length of the BizTalk message body.
ReturnFirstRegexMatch – Returns a string value of the first match for a regular expression in the BizTalk message body.
ReturnRegexMatchByIndex – Return a regular expression match in the BizTalk message body at a specific index.
Ability to write Rule Tracing Output to the file system for debugging purposes
There is now a new pipeline component parameter called TrackingFolder which by default has a blank value. If a folder location is passed into this parameter then the BRE Pipeline Framework will enlist the use of the out of the box DebugTrackingInterceptor to write rule trace information to that folder. If the parameter value is left blank (which I would encourage on UAT/Production environments except when troubleshooting). A file will be written for each policy executed, thus Instruction Loader Policy traces and Execution Policy traces will be written to separate files. The file mask is “{0}.{1}.{2}-{3}.txt” where {0} is the name of the policy being executed, {1}.{2} are the version number of the policy being executed (this is especially handy if you’re trying to work out whether the latest version of a BRE Policy has taken effect yet), and {3} is a GUID that is created per execution of the pipeline component thus an Instruction Loader Policy and Execution Policy fired within a single run of the pipeline component will have the same GUID value in their trace files. I did think of using the RuleEngineInstanceID instead of an auto-generated GUID in the last case but decided I would like to have a way to link an Instruction Loader Policy’s trace output to an Execution Policy’s trace output. If anyone has a better idea do let me know.
The rule trace information has very useful information for troubleshooting/debugging your rules such as what values were associated with asserted facts, which conditions in your rules evaluated and what values made up the conditions, as well as which rules actually fired. This information is essentially the same as you would see in the BRE Composer if you chose to test a policy; the output does take some getting used to but once you have your head around it the information can prove very handy.
New BizTalk 2013 context property getters and setters now available in Execution Policies
BizTalk 2013 introduced the new SFTP and SBMessaging adapters and they came with their own set of corresponding context properties. The BREPipelineFramework.SampleInstructions.ContextInstructions vocabulary now contains new vocabulary definitions called GetSBMessagingContextProperty, GetSFTPContextProperty, SetSBMessagingContextProperty, and SetSFTPContextProperty which allow you to get or set context properties associated with these adapters.
New message evaluation/manipulation features now available in Execution Policies
The below vocabulary definitions are now available within the BREPipelineInstructions.SampleInstructions.HelperInstructions vocabulary for use within Execution Policies only. Some of these methods require for the message body to be cast to a string which is a relatively expensive operations, especially if you expect your message bodies to be large so please use these with caution and test appropriately. The casting will only happen one time regardless how many times you make use of the vocabulary definitions within a given Execution Policy.
CheckIfRegexEvaluatesInMessage – Returns a Boolean value which indicates whether a regular expression finds a match in the BizTalk message body.
CheckIfStringExistsInMessage – Returns a Boolean value which indicates whether a substring exists in the BizTalk message body.
MessageBodyLength – Returns an integer value which indicates the string length of the BizTalk message body.
ReturnFirstRegexMatch – Returns a string value of the first match for a regular expression in the BizTalk message body.
ReturnRegexMatchByIndex – Return a regular expression match in the BizTalk message body at a specific index.
FindReplaceRegexInMessage – Replaces any strings that match a given regular expression with another string.
FindReplaceStringInMessage – Find and replace any instances of a substring with another string.
GetMessageBody – Returns the message body as a string so that it can be inspected with any other custom vocabulary definitions. Note that vocabulary definition only lends itself to inspection and can’t be used to update message bodies.
NullifyMessage – Sets the message body part to null so that if sent out on a file send port a file will not be written and the message will be effectively discarded. This is useful in scenarios where you receive a successful response from a web service or other send adapter but don’t really want to do anything with it and prefer to get rid of it.
ConcatenateTwoStrings / ConcatenateThreeStrings / ConcatenateFourStrings / ConcatenateFiveStrings / ConcatenateSixStrings – I’ve retired the old ConcatenateStrings vocabulary definition (if using an old vocabulary version it will still work) and replaced them with the aforementioned definitions instead. No longer will you need to chain together multiple ConcatenateStrings definitions if you want to concatenate multiple strings.
StringFormat – This vocabulary definition is effectively a wrapper for the String.Format() function in .NET and can make life a lot easier when trying to construct a complex string.
Ability to create SSO ticket so that you can fetch adapter credentials from an SSO affiliate application
The BREPipelineFramework.SampleInstructions.ContextInstructions vocabulary now contains a vocabulary definition CreateSSOTicketContextProperty which will create an SSO ticket and set it to the appropriate context property so that you can fetch credentials for a compatible send adapter from an SSO affiliate application. This makes for cleaner deployments since credentials do not need to be stored in binding files. I will blog about this in more detail shortly.
Ability to override the Application Context and/or Execution Policy
There are two new vocabularies in the BREPipelineFramework vocabulary called OverrideExecutionPolicy and OverrideApplicationContext which allow you to override which Execution Policy gets called and/or what Application Context is passed into the Execution Policy. Note that these functions can only be used within an Instruction Loader Policy, and overriding the Application Context will not take effect till after the Instruction Loader Policy completes (so this doesn’t really bring up any policy chaining type use cases).
These functions pave the way for generic onramps to be built whereby the Instruction Loader Policy is used to inspect the message, and choose an Execution Policy with an appropriate Application Context specific to the given message type/context.
Installer changes
The installer now respects the application directory you specify during installation (it used to plain all out ignore it previously, sorry!), will write the new vocabulary versions (BREPipelineFramework v1.1, BREPipelineFramework.SampleInstructions.ContextInstructions v1.2, and BREPipelineFramework.SampleInstructions.HelperInstructions v1.4) to the vocabularies subfolder along with the older versions, and will now also write the pipeline component dll to the bins subfolder along with the other assemblies. The pipeline component will also be installed to the GAC by the installer.
Adventures inside the Message Box 