One of the most compelling features that was made available in v1.5 of the BizTalk BRE Pipeline Framework was the ability to cache context properties or any custom strings and then fetch them back later on. I purposely chose not to blog about this feature till now because I wanted to employ it in a real life scenario to prove it’s value first which I have now.
The caching features are contained within the BREPipelineFramework.SampleInstructions.CachingInstructions vocabulary that was shipped with v1.5 of the framework. If you haven’t already published the vocabulary you can find the exported XML for this vocabulary in the Vocabularies subfolder in the BRE Pipeline Framework program files folder (typically “C:\Program Files (x86)\BRE Pipeline Framework”) and can use the Rules Engine Deployment Wizard to publish it. After reading this blog post you will have an idea of what caching features are available in the BRE Pipeline Framework, how the framework achieves the caching, and you’ll also be exposed to a summary description of each vocabulary definition in the BREPipelineFramework.SampleInstructions.CachingInstructions vocabulary.
Caching Context Properties
The most common use for the caching feature in the BRE Pipeline Framework will be to cache context properties and then to reapply them. Let me paint a picture to you, which will seem all too common to those experienced with BizTalk and who strive to create messaging only applications in situations where no orchestration is warranted. You need to call a WCF service which returns either a valid response which doesn’t contain any of the identifiers from the request, or a SOAP fault (unless SOAP faults were generated in a custom fashion you typically only see an error message with no details that help you to identify which request the fault corresponds to), and you need to write the status of the response to a database along with the identifiers from the request for correlation purposes.
BizTalk Server messaging is stateless by nature so you only have access to the message currently being processed and not historical messages when you are performing your inbound mapping on the request/response send port. Thus it wouldn’t typically be possible to satisfy the requirement of writing the response to the database with the identifiers from the request unless you used an orchestration that persisted the state of the original request and then executed a multi-source map that used the original request and the response message to transform to the destination message. I consider orchestration to be overkill for such a scenario and prefer to go down a messaging only route.
The answer to the above scenario is to ensure that all the identifiers in the request message are promoted to context properties on the message that is directed to the WCF service send port, use the BRE Pipeline Framework to cache the context properties from the request, and to then use the BRE Pipeline Framework to reapply those context properties on the response message. You could then use the Context Accessor pipeline component/functoids to insert the context properties into the target message body via a map, or use property demotion (see this TechNet article for more details) to achieve the same result.
Below is an example rule where the BRE Pipeline Framework is used to cache the RequestID and Requestor context properties on the request message on a send port. In the below case the BRE Pipeline Framework will throw an exception if the RequestID context property doesn’t exist on the request message, but will just ignore and carry on if the Requestor context property doesn’t exist on the request message. The vocabulary definition used in the Condition in the below rule is the ApplicationContext definition from the BREPipelineFramework vocabulary, while the definitions used in the Action is the AddCustomContextPropertyToCache definition from the BREPipelineFramework.SampleInstructions.CachingInstructions vocabulary.
Reapplying these context properties is easy as per the below screenshot in which the RequestID context property will be promoted (potentially for routing) while the Requestor context property is merely written to the context of the message (thus not available for routing). In the below case the BRE Pipeline Framework will throw an exception if the RequestID context property doesn’t exist in the cache, but will just ignore and carry on if the Requestor context property doesn’t exist in the cache. The vocabulary definition used in the Condition in the below rule is the ApplicationContext definition from the BREPipelineFramework vocabulary, while the definitions used in the Action is the ReapplyCachedContextProperty definition from the BREPipelineFramework.SampleInstructions.CachingInstructions vocabulary.
How does this work? Under the hood the BRE Pipeline Framework makes use of the .NET MemoryCache class (specifically used because of it’s cached item auto-expiry features) which ships with .NET 4.0 onwards. Each context property that you choose to cache on a given message gets added to a Dictionary in which the context property namespace#name is used as the key and the context property value is the value. The Dictionary is then added to the Memory Cache with the key based on the BTS.TransmitWorkID context property of the request message. The reason the BTS.TransmitWorkID context property is used as the key is because this context property is automatically generated on request-response send ports and response messages will always have the same value for this context property as the request message, thus this value can be used to write to and read from the MemoryCache. Each item in the MemoryCache by default has an expiry time of 30 minutes (if not explicitly removed before), after which they will automatically be removed from the cache to ensure that RAM isn’t permanently earmarked for the caching. The MemoryCache is isolated to a given AppDomain thus is unique per host instance (thus if you cache a value in one host instance you can’t reapply it in another, and is also not shared across machines even if host instances pertain to the same host) and if a single host instance is used for both receive/send ports and orchestration then the receive/send ports will have one MemoryCache while the orchestration will have another (see this fantastic article by Saravana Kumar if you are interested in reading more about BizTalk and AppDomains).
Caching custom strings
The caching functionality in the BRE Pipeline Framework can also be used to cache any string and to fetch it later. A common use for this would be to cache a value that has been read in from a relatively expensive data source to read from such as SQL Server or the SSO. Doing so means that you don’t have to hit the database each time your rule is executed.
Below is an example in which the value ReadInFromSQL (the hardcoded value could be replaced with a vocabulary definition that fetches the value from the relevant data source) is cached using the key Configuration Info, the cached entry being set to expire in 5 hours (note the time units is an Enum so you can choose whichever applicable unit you want). The vocabulary definition used in the Condition in the below rule is the ApplicationContext definition from the BREPipelineFramework vocabulary, while the definitions used in the Action is the AddCustomStringToCache definition from the BREPipelineFramework.SampleInstructions.CachingInstructions vocabulary.
Fetching the cached value is easy as per below. In the below case the BRE Pipeline Framework will return Null if the key isn’t found in the cache (other options are to throw an exception or return a blank string). The vocabulary definition used in the Condition in the below rule is the ApplicationContext definition from the BREPipelineFramework vocabulary, while the definitions used in the Action is the GetCustomStringFromCache definition from the BREPipelineFramework.SampleInstructions.CachingInstructions vocabulary.
The primary difference between the way that caching custom strings works vs. caching context properties works is that with the custom strings each value is cached separately in the MemoryCache rather than within a single dictionary per message which contains all the context properties for that message, and it is up to you to explicitly state the expiry time and the key for each value you choose to cache.
Walkthrough of vocabulary definitions in the BREPipelineFramework.SampleInstructions.CachingInstructions vocabulary
- AddCustomContextPropertyToCache – Allows you to specify a context property against the current message that should be cached.
- AddCustomStringToCache – Allows you to specify any string that should be cached. You will also need to supply a key and an expiry time for this cached entry.
- CacheAllContextProperties – Allows you to cache all the context properties against the current message. Can help you to future proof your rule since adding more context properties to the message will not require a change to the rule.
- ChangeKeyContextProperty (advanced feature)- By default when you cache context properties they are all collapsed into a single Dictionary which is added to the MemoryCache with a key based on the BTS.TransmitWorkID context property which is great for synchronous request/response type scenarios. If you want to cater for asynchronous scenarios whereby you might be trying to reapply the context property on a message processed on a totally separate port then you have the ability to nominate another context property as the key in the MemoryCache. Note that to reapply the context properties on the target message it must already have the same context property value for the key context property as the original message. Also note that this vocabulary definition would need to be contained within the action for the rule that adds to the cache and the rule that gets from the cache.
- DeleteContextFromCacheIfItStillExists – Allows you to explicitly delete context properties from the cache for the current message. You might want to do this after you have reapplied the context properties from the cache rather than wait for the expiry time to pass to reduce strain on RAM.
- DeleteCustomStringFromCache – Allows you to explicitly delete a custom string from the cache based on the specified key. You might want to do this when you are sure you no longer need the cached string rather than wait for the expiry time to pass to reduce strain on RAM.
- GetCustomContextPropertyFromCache – Allows you to fetch a cached context property. Note that this definition doesn’t reapply the cached context property to the current message but rather is just used to fetch the value, potentially for inspection in conditions or to use as a parameter in another definition within a rule’s action.
- GetCustomStringFromCache – Allows you to fetch a cached string which can be used as a string parameter in other definitions in a rule’s condition or action.
- ReapplyAllCachedContextProperties – Allows you to reapply all the cached context properties against the current message. Be wary of using this in tandem with the CacheAllContextProperties definition, as this might result in an attempt to override out of the box context properties such as BTS.MessageType or BTS.MessageID etc…
- ReapplyCachedContextProperty – Allows you to reapply a specific cached context property against the current message.
- UpdateCacheExpiryTime – Allows you to override the default expiry time for cached context properties, the default being 30 minutes.