Latest Entries »

I have just released a new version v1.5.5 of the BRE Pipeline Framework to the CodePlex project page.  This is a relatively minor update in that it only adds some missing WCF context properties to the enumerations used to get and set WCF context properties.

What makes this minor update important is that most of these context properties are REST related, and will enable you to build flexible REST based integration solutions where you can dynamically set HTTP methods, URLs, or headers.  The list of context properties added to the enumeration is below.

  • HttpHeaders
  • HttpMethodAndUrl
  • InboundHttpHeaders
  • InboundHttpMethod
  • InboundHttpStatusCode
  • InboundHttpStatusDescription
  • IssuerName
  • IssuerSecret
  • OutboundHttpStatusCode
  • OutboundHttpStatusDescription
  • StsUri
  • SuppressMessageBodyForHttpVerbs
  • VariablePropertyMapping

This new version is fully backwards compatible, and installing it will only require uninstalling the previous version from the add/remove programs control panel and then running the new installer.  Please post any issues/feedback/comments to the CodePlex project discussions page.

Azure App Services and Logic Apps have now hit public preview, and they have generated a lot of excitement (including with this writer).  It’s a bit like Microsoft has hit a reset button and now all the Microsoft integration specialists have to start from scratch (at least in the cloud, BizTalk Server isn’t going anywhere for awhile, thankfully since I would miss it), at least when it comes to tooling, concepts don’t really change…

One of the first proof of concepts I tried to work with (actually this was a team effort with a bunch of integration specialists excitedly fighting over the keyboard) was creating a Logic App that would pick up emails from a POP3 mailbox, and write the contents to a folder in my DropBox account, the file being named based on the subject of the email + .txt, and the contents of the file being based on the body of the email.  This is easy enough to do with the POP3 and the DropBox connectors as below.

Capture

We had quite a lot of difficulty trying to figure out how to set the filename in DropBox since it required a combination of hardcoded strings and the email subject which flowed out of the POP3 connector.  We tried a kazillion different combinations, and nothing seemed to work.

I reached out to the BizTalk Advisors Yammer forums hosted by Microsoft and got a helpful response from Stephen Siciliano at Microsoft in a snap (that’s Stephen!).  His suggestion is that if you’re mixing string constants and functions then you should not surround your function with quotes, and you should not use the @ escape character before the function.  Also, if you want to concatenate strings you must use the @concat function.  My final function looked like the below.

@concat(‘jc\’, triggers().outputs.body.Subject, ‘.txt’ )

This resulted in the email body being written to a file in a folder in the root path called jc, with the filename being based on the email subject + .txt.  Interestingly, if you view the code view you’ll notice that the slash after the folder name gets automatically escaped (so you don’t have to escape it, in fact if you do then you will end up with two escaped slashes).

Escaped

I hope this helps others who are also struggling to learn the new syntax.  Just like with any new language, once you understand the rules the learning curve is a lot less steep.  Don’t take it for granted that this syntax will remain the same.  I imagine that by the end of the public preview Azure App Services might be a very different beast.

Happy AAS-ing.

A quick blog post to document a tough lesson learnt by myself so others can hopefully avoid making the same mistake. Changing a BizTalk WCF send port’s send handler (i.e. the host instance it is associated with) will remove all passwords that are currently configured on the send port. This includes credentials to access the target service as well as the proxy server password if there is one.  This appears to be the case with WCF-SQL receive locations as well, however a quick test with the File adapter showed that this behavior might not be consistent with non-WCF adapters so it’s best to try your change on a development environment first.

How can you tell if your password has been wiped? Normally when you access the send port, if you have previously entered a password then you will see the black circles indicating a masked password is present. If the password has been wiped then the field will be totally blank as below.

PasswordBlanked

As a practice I would encourage others to avoid hardcoding passwords on send ports and to use the SSO’s credential mapping facilities instead (a blog post on how to achieve this using the BRE Pipeline Framework soon).  This isn’t applicable for proxy server credentials but those can typically be set on the send handler level so you don’t have to worry about associating then with your send ports.  

Moral of the story, if you’re changing your adapter handler on a port and there are passwords associated with the port then re-enter them.

This week a BizTalk solution I had designed encountered an exception I’ve never seen before, as below.

System.ServiceModel.Security.MessageSecurityException: Security processor was unable to find a security header in the message. This might be because the message is an unsecured fault or because there is a binding mismatch between the communicating parties. This can occur if the service is configured for security and the client is not using security.

The issue was intermittent, and encountered while communicating with a third party web service.  Said web service was based on the Java stack rather than .NET and was being hosted by an ESB unknown to me.  This web service was based on the WS-I Basic Profile 1.1 so I was using a WCF-Custom adapter (mainly because I employed some custom behaviors to work around some funny behavior with this web service) employing the basicHttpBinding.  The security mode was set to transportWithMessageCredential while the clientCredentialType was set to Username, so I was effectively using SSL to encrypt my message with the username and password contained within the SOAP Headers in my request.

Seeing as the target system was a Java system and the error raised by the send port was clearly a .NET type error, I immediately suspected that the WCF stack on the BizTalk server must be raising the exception rather than the web service being called upon.  To prove my theory I decided to enable WCF tracing by adding the below within the system.serviceModel section of my BTSNTSVC64.exe.config file (the send host instance was 64-bit) and restarted my send host instance.

  <diagnostics>
    <messageLogging
         logEntireMessage="true"
         logMalformedMessages="true"
         logMessagesAtServiceLevel="true"
         logMessagesAtTransportLevel="true"
         maxMessagesToLog="300000"
         maxSizeOfMessageToLog="200000"
  />
  </diagnostics>
</system.serviceModel>
<system.diagnostics>
  <sources>
    <source name ="System.ServiceModel" switchValue="Verbose">
      <listeners>
        <add name="xml" />
      </listeners>
    </source>
    <source name ="System.ServiceModel.MessageLogging"
            switchValue="Verbose, ActivityTracing">        
      <listeners>
        <add name="xml" />
      </listeners>
    </source>
    <source name ="System.Runtime.Serialization" switchValue="Verbose">
      <listeners>
        <add name="xml" />
      </listeners>
    </source>
 </sources>
 <sharedListeners>
    <add name="xml" type="System.Diagnostics.XmlWriterTraceListener"             
         traceOutputOptions="LogicalOperationStack"
         initializeData="C:\log\WCFTrace.svclog" />
 </sharedListeners>
 <trace autoflush="true" />
</system.diagnostics>

With tracing enabled I played another few messages through BizTalk until I had a few success and failure scenarios. Studying the trace outputs using the Microsoft Service Trace Viewer tool I found that I was always getting a response from the service in question, but in the successful response messages I had SOAP security headers included, whereas in the failure messages (which were otherwise normal response messages from the service) I didn’t.

Below is a snippet of the SOAP headers for a success message.

<SOAP-ENV:Header>
  <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" SOAP-ENV:mustUnderstand="1">
    <wsu:Timestamp xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" wsu:Id="Timestamp-96477">
      <wsu:Created>2015-02-25T02:22:38.760Z</wsu:Created>
      <wsu:Expires>2015-02-25T02:27:38.760Z</wsu:Expires>
    </wsu:Timestamp>
  </wsse:Security>
</SOAP-ENV:Header>

Whereas I found that the SOAP headers were completely empty in the case of the failure messages. The problem was clearly on the service side, and we told the third party in charge of the service that they needed to resolve this. However I decided that I wanted to dive a bit deeper and find out whether there was any way to get WCF/BizTalk to support the missing SOAP security headers.
The first port of call was to attempt to replicate the problem. I managed to create a stub for the service with the exact same binding using .NET on my development VM using all the same bindings and authentication modes. I struggled to find a way to programmatically remove the headers from my message. I suspected that I could write a Message Inspector WCF behavior to do so, but I was too lazy :)
Fiddler came to my rescue. I enabled the logging of HTTPS traffic, and toggled Fiddler to automatically apply breakpoints on responses.  The beauty of fiddler is that once it applied a breakpoint on the response, I was able to decrypt it (since it was encrypted using HTTPS), tamper with it (to remove the security SOAP headers) and then submit my tampered response back to BizTalk.  Voila, I had now found a way to reproduce my problem.

The next step was to find a way to work around the problem.  I found that there was no way to do so with the basicHttpBinding that I was using (and I’d bet that this wouldn’t be available with the other out of the box bindings either).  I changed my send port to use a customBinding instead, replicating my basicHttpBinding configuration by employing textMessageEncoding (very importantly setting the messageVersion attribute to Soap11 since I had to abide by the WS-I Basic Profile 1.1), httpsTransport, and the security binding elements.  I configured the security element such that the authenticationMode was set to UserNameOverTransport to mirror the UserName clientCredentialType in my original configuration.  Now the magic was to set the enableUnsecuredResponse attribute to true in the security binding element.

Port

Doing this resulted in BizTalk being able to consume the response messages, regardless whether the security headers were included in the response or not.  If for whatever reason the third party in question was unable to fix the problem on their end I now had a fallback plan.

The final piece of the puzzle was trying to see if I could get my .NET based WCF service (not BizTalk) to return a response without the security SOAP headers, so that I wouldn’t have to use Fiddler to replicate this problem anymore.  Armed with the knowledge about the enableUnsecuredResponse property on the security binding element I proceeded to switch my WCF Service from using basicHttpBinding to also employ a custom binding.

My original binding looked like the below.

<basicHttpBinding>
  <binding name="BasicHttpEndpointBinding">
    <security mode="TransportWithMessageCredential">
      <message clientCredentialType="UserName" />
    </security>
  </binding>
</basicHttpBinding>

While my updated binding looked like the below.

<customBinding>
  <binding name="customBasicHttpBinding">
    <security authenticationMode="UserNameOverTransport"  enableUnsecuredResponse="true"/>
    <textMessageEncoding messageVersion="Soap11" />
    <httpsTransport />
  </binding>
</customBinding>

Since I had included the enableUnsecuredResponse attribute in my security binding element, my WCF service was now returning responses without the security SOAP headers. The same would apply for a BizTalk receive location using a custom binding as well. Would you ever want to employ this trick? I’d say the answer is a resounding no since this enables non-standard behavior, unless for whatever reason you are backed into a corner on a project where you’re integrating with a fiddly system that will not accept the security SOAP headers (I can’t imagine this being the case but then again I wouldn’t be overly surprised).

I recently worked on a POC which had me spinning up an EDIFACT/AS2 exchange based on both BizTalk Server and MABS (Microsoft Azure BizTalk Server) and comparing the experiences.  Seeing as this was my first time dealing with AS2 (EDIFACT was already an old friend) the very first thing I worked on was finding a means of sending messages to the AS2 onramps.

I found a few open source programs that would let me submit AS2 messages but I decided I wanted to do this programmatically instead via BizUnit 4.0 so I had easily repeatable scenarios, which I could easily re-test across BizTalk Server and MABS.  Matt Frear has written a good post on how to send messages via AS2 using .NET, and I decided I was going to convert this into a BizUnit test step (with his blessing).

You can find the source code for the AS2SendTestStep here, or if you just want to download the assembly then you can do so here.

The test step support the below features.

  • Supports BizTalk Server and MABS (and theoretically other AS2 server products, but I haven’t tested any others).
  • Supports BizUnit data loaders which means that if you have many tests which require only slight variations to input messages then you don’t need to keep multiple copies of input files. You can just have a single base file and use a relevant DataLoader class to manipulate the stream as it is being read.
  • Supports signing and encrypting outbound messages.
  • Supports optional synchronous MDNs and allows you to validate them.
  • Logs decrypted MDN messages to the test execution report.

Usage tips are below.

  • You will need to reference the BizUnit.AS2TestSteps.dll assembly and the BizUnit.dll assembly provided by the BizUnit 4.0 framework.
  • Select an input file directly by supplying the file name into the InputFileLocation property or supply a BizUnit DataLoader (which allows you to manipulate the file as you read it in) in the InputFileLoader property.
  • Encrypt the outbound message – you will need a copy of the public key certificate and will need to supply the file path to it in the EncryptionCertificateFileLocation property.
  • Sign the outbound message – you will need a copy of the private key certificate and will need to know the password.  You will need to supply the path to the certificate to the SigningCertificateFileLocation property and the password to the SigningCertificatePassword property.
  • Supports setting AS2From and AS2To headers via the As2From and As2To properties.
  • Supports the use of a proxy server by setting appropriate values to the Ps property which allows you to supply the proxy server URL, and if required credentials as well.
  • Allows you to set the subject HTTP header by setting the As2FileName property.
  • Allows you to set the URL to post the request to by setting the Url property.
  • Allows you to override the default timeout of 20 seconds by setting the TimeoutMilliseconds property.
  • Allows you to run BizUnit substeps against the decrypted response message in case you want to validate it by suppying substeps into the SubSteps property.

An example test which sends an EDIFACT message to BizTalk Server (using a proxy server) and runs some regular expressions against the synchronous MDN response is below.  Note that the RegexValidationStep in use here is not part of the BizUnit framework and is proprietary so sorry I can’t share that.


var testCase = new BizUnit.Xaml.TestCase();

var as2TestStep = new AS2SendTestStep();

as2TestStep.As2From = "FABRIKAM";

as2TestStep.As2To = "CONTOSO";

as2TestStep.EncryptionCertificateFileLocation = @"c:\host.cer";

as2TestStep.As2FileName = "EFACT_D95B_CODECO_output.txt";

as2TestStep.InputFileLocation = @"c:\temp\EFACT_D95B_CODECO_output.txt";

as2TestStep.SigningCertificateFileLocation = @"c:\fab.pfx";

as2TestStep.SigningCertificatePassword = "test";

as2TestStep.TimeoutMilliseconds = 20000;

as2TestStep.Url = "http://localhost/AS2Receive/BTSHTTPReceive.dll";

WebTestPlugins.AS2Helpers.ProxySettings ps = new WebTestPlugins.AS2Helpers.ProxySettings();

ps.Name = "http://proxyserver.test.co.nz";

as2TestStep.Ps = ps;

var regexValidationStep = new RegexValidationStep();

regexValidationStep._RegexDefintion.Add(new RegexDefinition("Content-Type: message/disposition-notification"));

regexValidationStep._RegexDefintion.Add(new RegexDefinition("Disposition: automatic-action/MDN-sent-automatically; processed"));

regexValidationStep._RegexDefintion.Add(new RegexDefinition("Final-Recipient: rfc822; CONTOSO"));

as2TestStep.SubSteps.Add(regexValidationStep);

testCase.ExecutionSteps.Add(as2TestStep);

var bizUnitTest = new BizUnit.BizUnit(testCase);

bizUnitTest.RunTest();

And below is a screenshot of the test execution results.  Note that the MDN text is logged here, and you can see the regular expressions being evaluated against the MDN as well.

Capture

You can of course take your own test steps much further by validating that BizTalk/MABS has consumed the file and written the message to its target destination, be it a SQL database or a file system folder etc… using the out of the box BizUnit test steps or your own custom ones.

The test step doesn’t currently support compression.  If anyone can point me towards any good documentation on compressing AS2 messages in C# then I might try to include that.

Any other suggestions are welcome.

Follow

Get every new post delivered to your Inbox.

Join 95 other followers

%d bloggers like this: