Category: Testing

I recently worked on a POC which had me spinning up an EDIFACT/AS2 exchange based on both BizTalk Server and MABS (Microsoft Azure BizTalk Server) and comparing the experiences.  Seeing as this was my first time dealing with AS2 (EDIFACT was already an old friend) the very first thing I worked on was finding a means of sending messages to the AS2 onramps.

I found a few open source programs that would let me submit AS2 messages but I decided I wanted to do this programmatically instead via BizUnit 4.0 so I had easily repeatable scenarios, which I could easily re-test across BizTalk Server and MABS.  Matt Frear has written a good post on how to send messages via AS2 using .NET, and I decided I was going to convert this into a BizUnit test step (with his blessing).

You can find the source code for the AS2SendTestStep here, or if you just want to download the assembly then you can do so here.

The test step support the below features.

  • Supports BizTalk Server and MABS (and theoretically other AS2 server products, but I haven’t tested any others).
  • Supports BizUnit data loaders which means that if you have many tests which require only slight variations to input messages then you don’t need to keep multiple copies of input files. You can just have a single base file and use a relevant DataLoader class to manipulate the stream as it is being read.
  • Supports signing and encrypting outbound messages.
  • Supports optional synchronous MDNs and allows you to validate them.
  • Logs decrypted MDN messages to the test execution report.

Usage tips are below.

  • You will need to reference the BizUnit.AS2TestSteps.dll assembly and the BizUnit.dll assembly provided by the BizUnit 4.0 framework.
  • Select an input file directly by supplying the file name into the InputFileLocation property or supply a BizUnit DataLoader (which allows you to manipulate the file as you read it in) in the InputFileLoader property.
  • Encrypt the outbound message – you will need a copy of the public key certificate and will need to supply the file path to it in the EncryptionCertificateFileLocation property.
  • Sign the outbound message – you will need a copy of the private key certificate and will need to know the password.  You will need to supply the path to the certificate to the SigningCertificateFileLocation property and the password to the SigningCertificatePassword property.
  • Supports setting AS2From and AS2To headers via the As2From and As2To properties.
  • Supports the use of a proxy server by setting appropriate values to the Ps property which allows you to supply the proxy server URL, and if required credentials as well.
  • Allows you to set the subject HTTP header by setting the As2FileName property.
  • Allows you to set the URL to post the request to by setting the Url property.
  • Allows you to override the default timeout of 20 seconds by setting the TimeoutMilliseconds property.
  • Allows you to run BizUnit substeps against the decrypted response message in case you want to validate it by suppying substeps into the SubSteps property.

An example test which sends an EDIFACT message to BizTalk Server (using a proxy server) and runs some regular expressions against the synchronous MDN response is below.  Note that the RegexValidationStep in use here is not part of the BizUnit framework and is proprietary so sorry I can’t share that.

var testCase = new BizUnit.Xaml.TestCase();

var as2TestStep = new AS2SendTestStep();

as2TestStep.As2From = "FABRIKAM";

as2TestStep.As2To = "CONTOSO";

as2TestStep.EncryptionCertificateFileLocation = @"c:\host.cer";

as2TestStep.As2FileName = "EFACT_D95B_CODECO_output.txt";

as2TestStep.InputFileLocation = @"c:\temp\EFACT_D95B_CODECO_output.txt";

as2TestStep.SigningCertificateFileLocation = @"c:\fab.pfx";

as2TestStep.SigningCertificatePassword = "test";

as2TestStep.TimeoutMilliseconds = 20000;

as2TestStep.Url = "http://localhost/AS2Receive/BTSHTTPReceive.dll";

WebTestPlugins.AS2Helpers.ProxySettings ps = new WebTestPlugins.AS2Helpers.ProxySettings();

ps.Name = "";

as2TestStep.Ps = ps;

var regexValidationStep = new RegexValidationStep();

regexValidationStep._RegexDefintion.Add(new RegexDefinition("Content-Type: message/disposition-notification"));

regexValidationStep._RegexDefintion.Add(new RegexDefinition("Disposition: automatic-action/MDN-sent-automatically; processed"));

regexValidationStep._RegexDefintion.Add(new RegexDefinition("Final-Recipient: rfc822; CONTOSO"));



var bizUnitTest = new BizUnit.BizUnit(testCase);


And below is a screenshot of the test execution results.  Note that the MDN text is logged here, and you can see the regular expressions being evaluated against the MDN as well.


You can of course take your own test steps much further by validating that BizTalk/MABS has consumed the file and written the message to its target destination, be it a SQL database or a file system folder etc… using the out of the box BizUnit test steps or your own custom ones.

The test step doesn’t currently support compression.  If anyone can point me towards any good documentation on compressing AS2 messages in C# then I might try to include that.

Any other suggestions are welcome.

While doing some integration testing for BizTalk hosted WCF services I met a few challenges that required some creative thinking if I wanted to do things the right way…  As a quick intro I was making use of BizUnit 4.0 to execute my tests and making use of the WCF test step to call on my WCF service, executing my tests from Visual Studio 2012 against a local BizTalk 2013 environment.  This was a one way WCF service making use of the WSHttp binding with transport security and basic authentication, also making use of a WCF service behaviour to perform authorization ensuring that the user in question belongs to a specified active directory group.  At this stage I was only going to be executing the tests against my local VM however at some point in the future I would want to run these tests on actual test environments.

Challenge number one was that since my WCF service made use of transport security I couldn’t have my test project’s app.config reference an endpoint URL like https://localhost/…. since this wouldn’t match up with the self signed security certificate that I had bound to the HTTPS port in IIS.  I would instead need to put the fully qualified name of my VM which matches up with the certificate bound to the HTTPS port in my app.config.  This isn’t all that acceptable given that it isn’t necessarily going to be my PC on which the integration tests are next executed on, given that it could be any other developer working on them in the future.  I would ideally also like to be able to have my integration tests run against staging or UAT environments in the future, and would like to repoint my tests to these environments without making any changes to any code in source control including my app.config or adding new environment specific tests.  I decided that I wanted for the URL to be overridden by replacing a placeholder value in the URL in my app.config with one read in from an SSO application.

Challenge number two was that since my service made use of basic authentication and active directory group based authorization I had to be able to dynamically choose which credentials I was going to make use of when I called the service given that the credentials required to perform the same test on each environment would be different.  Being a very thorough (some might have ruder words for me but I will keep this blog post PG) tester I wanted to also be able to perform negative testing whereby I had credentials that did not authenticate or credentials that did authenticate but weren’t authorized to call the service and I didn’t want any of these credentials hardcoded in my app.config or code or the resolution of the credentials to result in me having to modify the WCF test step.  Once again I decided that I was going to make use of an SSO application to lookup the credentials that I wanted to use when I called on the service.

In order to do this I decided to implement a WCF endpoint behaviour which will implement the out of the box ClientVia behaviour which allows for the overriding of an endpoint URL and a ClientCredentials behaviour to set the relevant credentials.  Now I could have made use of the ClientVia or ClientCredentials behaviors directly in my app.config however that breaks my principle that I don’t want to update the source controlled app.config.  I could have also made a custom implementation of the BizUnit WCF test step which dynamically looks up the URL and credentials I want to use and which then programmatically implements the ClientVia and ClientCredentials behaviors with the resolved values, however that went against my principle of not updating the WCF test step which performs a very generic function very well.

I created a new WCF endpoint behavior and implemented the below ApplyClientBehavior method to make use of the out of the box behaviors with values resolved from an SSO config store (this could of course be split into two discrete WCF behaviors but for the purpose of this blog post I decided to treat them as one).


Note in the above that the _SSOApplication variable is actually a parameter for the behavior’s constructor so it can be set within the app.config allowing the behaviour to be reused across different projects.  The same applies for the keyPrefix variable which is used to specify which set of credentials are being looked up, allowing for the SSO store to contain multiple sets of credentials.  The SSO application would look like the below.

SSO Application

In order to make use of these behaviors the test projects app.config would have to be adjusted so that it looks like the below.


Note the highlighted sections in the above screenshot and my notes below explaining what they do.

  • The first highlighted section shows the registration of the behaviour extension and linking it with a local friendly name.  Note that the type includes the fully qualified behaviour extension element class name and the fully qualified assembly name (you can get this using the “gacutil -l” command).  This allows for us to use the behavior in our app.config.
  • The second section is where we register a behavior configuration with a friendly name and specify which behaviors it implements and with what configuration.  In the above screenshot I registered two behavior configurations one which specifies a keyPrefix value of Johann and one which specifies AITM, both specifying an SSO application name of Testing.
  • The third section is adjusting the client endpoint registration such that they implement one of our registered behavior configurations.  We also adjust the endpoint name so that it is obvious which behavior configuration they implement.  Also note that the URLs have the word placeholder in them where we want to override the URL with the value fetched from the SSO config store.

Now when we actually create our BizUnit test step we need to ensure that we specify the endpoint name which matches the client endpoint in our app.config that implements the behavior that we want to implement for our test.  In the below screenshot the WCF test step specifies an endpoint name of MathJohannCredentials which means that the Johann username and password from the SSO config store will be used as credentials when calling the service, and that the https://placeholder/MathsService/Maths.svc url will be overridden with  Making use of the AITM credentials would be as simple a task as implementing a similar test that specifies an endpoint name of MathAITMCredentials.


These are of course just examples of how you can use WCF behaviors to enable you to write flexible integration tests.  You could very easily use this concept to lookup values from other config stores such as a database, the business rules engine, or a configuration file, or perhaps to override other WCF behaviors.  If anyone has any other good ideas on how WCF behaviors could be used to make testing more flexible then please do add a comment to this post.

My colleague Ian Hui figured out a problem that has had me scratching my head for the last two months and he has made me a very happy man.

While porting unit tests using BizUnit for the BRE Pipeline Framework from BizTalk 2010 to BizTalk 2013 I encountered a System.BadImageFormatException exception with the following error message – “Could not load file or assembly ‘file:///C:\Program Files (x86)\Microsoft BizTalk Server 2013\Pipeline Components\BREPipelineFrameworkComponent.dll’ or one of its dependencies. This assembly is built by a runtime newer than the currently loaded runtime and cannot be loaded”.

Failed test

When I removed my custom pipeline component and only left behind the out of the box pipeline components like the XML Disassembler I noticed that the problem disappeared.  This, and some further digging caused me to believe that the problem is encountered with all pipeline components built in .Net 4.0 and above.  I tried a whole bunch of workarounds such as rebuilding the BizUnit test steps, the Winterdom pipeline testing framework and even the PipelineObjects.dll assembly using .Net 4.5 thinking that this might help work around the problem but I just kept hitting brick walls.  What made the problem even more mind-boggling was that the pipeline components that caused problems in unit tests ran just find in the BizTalk runtime.

In comes Ian who persevered with the problem long after I had resigned this to being something we would have to wait for Microsoft to fix.  He found that you needed to adjust the vstest.executionengine.exe.config file typically found in the “C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow” folder, setting the useLegacyV2RuntimeActivationPolicy attribute on the startup element to true as in the below  (you can download a copy of my config file from here if you want to copy/paste from it).


You can read more about this setting and what it does here.  The discrepancy in behaviour between the test execution and BizTalk runtime is easily explained by examining the BTSNTSvc64.exe.config file in the BizTalk program files folder as you’ll notice that the aforementioned attribute is set to true in this config file by default, which is why the runtime never had a problem with these pipelines.

BT config

Funnily enough after Ian figured out the answer for this problem he found that Shashidharan Krishnan had encountered the same problem in the past (on BizTalk 2010) and fixed it in the exact same way (he has documented this here), however he encountered completely different error messages and I suspect that he was running his unit tests through a console application rather than Visual Studio unit tests.  Either ways as the error messages he encountered are totally different from the ones we did, chances are that if you have the same error as us you might not find his post which is why we decided we would document this anyways.

Thanks again Ian (and Shashidharan).  You guys have just made unit testing BizTalk 2013 components more robust, thus ensuring the success of integration solutions developed for BizTalk 2013.

Whipping up a BizUnit test step using the BizUnit 4.0 libraries is a piece of cake, but chances are that any test step you write is going to be used multiple times throughout the life of a project and in ways you can’t predict and you will want to spend the extra effort when developing your test step to cater for this.

BizUnit promotes the reuse of test steps by providing interfaces that allow you to write them in really flexible ways, with the ability to make use of plugins rather than directly implementing logic such as the loading of data or validation of data. This allows for late binding of actual data loader / validator implementations as it is not upto the test step implementor to choose which of these facilities to make use of, but is rather the responsibility of the test case implementor (ie. the person making use of a test step to implement a test).

For example, when writing a test step that calls upon a WCF service, rather than having the test step load its data from a file we can leave the choice wide open so that the test case implementor can choose to load his data from a database, or perhaps read it in from an XML file but run some XPath statements against the message first to adjust its content (making use of the out of the box XmlDataLoader data loader shipped with the BizUnit 4.0 framework).

In this blog post I will list some of the steps I undertook to port Michael Stephenson’s WCF test step for BizUnit 2.0 (many thanks to Michael for writing this fantastically useful step and giving me permission to discuss it) into a BizUnit 4.0 test step while leveraging off the aforementioned features.

Both the old and the new BizUnit test step require some field values to be set such as the WCF input message type name, and the method name etc… The old test step did this by reading the values from the test context as below.

Old Properties setting

BizUnit 4.0 allows you to programatically instantiate and setup your test steps in a test case, so rather than reading these field values in from the context (one could still do this if required) we can just expose these as public properties, or write a public constructor for the test step class that allows you set these field values.

New Properties setting

To make our test step more flexible its a good idea to support for as many substeps as are required to be executed against the response message (if we are dealing with a request-response service, obviously this is not applicable for one-way services). In order to do this we need to instantiate the SubSteps property (defined on the TestStepBase base class) in the test step constructor to ensure that we can add substeps to the collection when we implement our test cases.


We will also want to implement a Validate method in our test step to check that it has been instantiated with all the required and valid field values necessary. This method would be run after the entire test case has been instantiated and is being prepared for execution (ie. after all the individual test steps have been setup and put into their relevant stages in the test case).

Test step Validation

On to the meat of test step. Whereas the previous version of the test step had the loading of the file built into the test step itself (loading it from the file system which would be good enough in many cases) we will take it one step further by making use of a generic data loader. Note that the dataLoader is a field which can be set through its corresponding public property or via a constructor (I haven’t allowed for it in my example but there is no reason why you couldn’t setup all your properties in the constructor).

Data loader

The code to call on the WCF service and optionally consume its response remains largely unchanged. What has been added is the ability to run the optional response through any number of substeps, giving the test case implementor the ability to validate the response in any number of given fashions.

Output Validation

To illustrate how the test step can be used I have created a WCF Service application with a method called GetDataUsingDataContract (yes, I was too lazy to create my WCF service from scratch) which takes in an object of type CompositeType and returns an object of type CompositeType. CompositeType is defined as below, and the method will add up the FirstValue and SecondValue elements and set the value in the Result element before returning the object.


After publishing the WCF Service I created a test project and added a service reference to the published service. I then added references to the BizUnit assemblies as well as my custom test step project. The next step was to create a test method, instantiate the TestCase and the WCFTestStep as below.


Note that the EndPointName is from the name attribute in the endpoint node in the test project’s app.config. If there were multiple endpoints defined in the config then we can choose any of them.


The InputMessageTypeName and the InterfaceTypeName must be fully qualified, the namespaces being based on the service reference, and the application name being the test project itself (or whatever project your service reference is contained in).

The next step was to load the request message from a file, but what I wanted was to be able to replace values in the input file using XPath expressions, thus meaning that I didn’t have to maintain a copy of the input file for each test method. I achieved this using the XmlDataLoader step, replacing the FirstValue element with the value 4, and the SecondValue element with the value 7 in the below example. Note that the format of the input message is based on the type definition in the service reference which might differ slightly from the message definitions you might have used if you were creating your service in BizTalk (e.g. the message you want to submit might have to be wrapped in a parent node), you might want to run the xsd.exe tool against the test project dll to find out how you must format your message. Using a tool such as DanSharp XmlViewer can aid you in creating your XPath statements.

DataLoader in action

Next up it’s time to implement validation. In the below example I am making use of the XmlValidationStep to validate the response message against its schema (I generated this using the xsd.exe tool), and also to ensure that the value in the Result element is what I was expecting.


Lastly it’s time to add the WCF test step to the test case, to add the test case to a newly instantiated BizUnit object and to execute the test.


If you run the test and view the test result details you get some rather detailed output.


The beauty of the WCF test step is that it should work for any WCF Service regardless of the bindings as long as a service reference has been added which should automatically create the binding and endpoint details in your config file.

If you are interested in downloading the test step you can do so from my google drive. The solution also contains my example WCF Service (which should automatically ask for permission to deploy into IIS when you open the solution), as well as my test project. Let me know if you can think of any improvements to the test step, and once again kudos to Michael Stephenson as most of the implementation logic is based on his pioneering work.

Last week I ran into a really interesting issue. When I was running Visual Studio unit tests to validate instances of XML formatted EDIFact messages (see my previous blog post on BizTalk testing for more details on schema testing) I found that my tests just seemed to take forever and timed out with no reason provided.


I decided to apply a break point in my unit test and debugged it.  Much to my surprise when I stepped into the line of code that was actually validating the instance file I saw the below.

EDIFact Properties

Well, that’s really painful.  The unit test was timing out because it was waiting for user input on the above screen, however unless the test was being run in debug mode the screen was not being displayed at all.

So far I haven’t found any way to get the out of the box ValidateInstance method to work on EDIFact schemas because I can’t supress the above screen.  Interestingly enough, if the instance file actually fails validation then the test will fail rather than time out.  Another alternative I explored was to use some non standard test methods to test instances of EDIFact messages against my schemas.  In my previous blog post discussing BizTalk unit testing I mentioned that I regularly use non standard methods to extend my schema unit tests anyways since the out of the box ValidateInstance method doesn’t return an error message upon failure, and it also doesn’t work when your schemas import/include from other schemas.  I chose to use only these extension methods (as described here), bypassing the out of the box testing methods altogether and the tests now complete succesfully.  Note that in the below screenshot the TestExtensions.ValidateSchema method contains an extension method as described here.


%d bloggers like this: