Latest Entries »

I have heard many integration specialists lament the fact that they could not get BizTalk Server to successfully connect to SOAP services secured by ADFS. This made me a bit nervous when I was recently tasked with doing exactly this.  The challenge was to call a WCF Service which was secured by ADFS with authentication based on AD username/password credentials.

This endeavour never got past the POC stage since the project I was working on decided to go another way rather than use ADFS, but I did manage to whip together a successful POC and thought I’d share my findings. This post is not a deep dive into ADFS (there are lots of those available if you search around, and I’m really not an expert) but will rather focus on providing a step by step guide to getting BizTalk Server to call on your ADFS secured service.

I was provided with a WSDL for the service in question and proceeded to run the “Consume WCF Service” wizard in Visual Studio. My plan was to use the generated binding file to automatically create the send port. However when I ran the wizard, I was met with the below screen at the last step. I can’t say if all of you will encounter this problem when trying to consume an ADFS secured WCF Service in a BizTalk project, but it certainly caused problems for me.

Error

Since this wasn’t going anywhere for me I decided to fall back to my favourite backup plan, whipping up a Console Application and adding a Service Reference corresponding to the ADFS secured service.  What I saw in the app.config surprised me.  Before jumping into what’s so different about the generated config file, below is what the config file looks like at the top level.  The client endpoint is based on a customBinding (pretty much equivalent to a basicHttpBinding with transport security since it is composed of the textMessageEncoding and httpsTransport binding element extensions).  So far that’s nothing too fancy, but what’s hidden within the collapsed security element within the custom binding, and what’s with all the extra bindings contained within the app.config?

TopLevelConfig

Below is a screenshot of a subset of the security element contained within the app.config file.  What you’ll notice is that within the security element is a sub-element called issuedTokenParameters.  Below this node you’ll notice that there are a whole lot more commented out alternative issuedTokenParameter nodes.  Each one of these issuedTokenParameter nodes (including the default one) corresponds to different STS issuer endpoint addresses, each one supporting various authentication modes (such as username, kerberos, or certificate) and various WCF bindings.

TheMeatOfTheConfig

This is actually the most important point.  The binding configuration used by the STS endpoints is actually independent from the binding configuration used against the client endpoint. When calling this WCF Service there are actually going to be two service calls, the first to the STS service and the second to the target service with the relevant security headers based on the response from the STS service.

The default issedTokenParameters element in my app.config file is based on the wsHttpBinding and is based on certificate authentication (the STS address/binding configuration name should give you a hint regarding this, but inspecting that specific binding configuration will confirm this).
CertificateWSHttp

This isn’t quite what I wanted so I commented out the default issuedTokenParameters element in the config file, and uncommented the alternative issuedTokenParameters node which was based on the ws2007HttpBinding binding (note that I could have achieved the same result by choosing the issuedTokenParameters node based on the wsHttpBinding binding) and username credentials.

ws2007usernamebinding

Now when I call on the service I get a successful response as long as I pass in the relevant client credentials.

ServiceReference

Ok, we’ve got this working in a Console Application, but wasn’t the whole point of this exercise to get this working in BizTalk Server? My belief is that it isn’t possible to create a send port directly in the BizTalk Administration console, as it is not possible to create the secondary binding that will be used by the STS endpoint, which as you’ve seen above has a binding that is independent of the target service’s binding. However, we can use the app.config file that got generated in our Console Application to load the relevant configuration into a WCF-Custom send port.

In order to achieve this we need to create a new WCF-Custom send port, browse to the Import/Export tab, click the import button and choose the aforementioned app.config file.
Import
You’ll be prompted to select a client endpoint from which configuration will be imported. Only one of them should name an address against it, so select that one and click ok.
SelectEndpoint

The result of this is that the endpoint address and binding information has been automatically populated in the send port based on the configuration contained within the app.config file.  If you click on the security/IssuedTokenParameters/Issuer node within the binding tab, you’ll notice that the binding and bindingConfiguration values match those that were specified in the app.config. Also of note is that there is nowhere within the send port UIs whereby you can adjust the binding configuration’s configuration for the STS endpoint.

CustomBinding

If you export the bindings for the send port you could adjust the secondary binding configurations manually, though do be careful because the XML configuration will be escaped. I’d also advise you to use lightweight editors like notepad when editing binding files, because Visual Studio is notorious for corrupting send port filters when you manually edit binding files using the built in XML editor.

That’s pretty much all there Is to it. The only additional step will be to set the SOAP Action Header mappings, and inbound/outbound maps/pipelines, and you will be ready to use your send port.

I would like to shout out to Yossi Dahan whose blog post (I can’t find it anymore though so can’t refer to it) gave me some hints that got me on the right path, as well as Chris Mellard who also gave me some hints that got me halfway to a solution – and after re-reading the information he gave me it looks like he actually provided me with the entire solution, which would’ve saved me a lot of pain if I had only read his email to me more carefully :)

As always, please feel free to share your own experiences, feedback, comments, and criticisms with the aforementioned solution.

This blog details problems that you will definitely encounter if you use the BizTalk WCF-WebHttp Receive Adapter in tandem with Azure Service Bus Relays (at least in BizTalk 2013 I haven’t confirmed if the problem has been fixed since but I doubt it). The problem is that if the service consumer tries to append URL parameters to the Service Bus Relay URL then they will be met with the dreaded AddressFilterMismatch exception. The traditional means of working around AddressFilterMismatch exceptions also leads to more problems which I will describe in this post. I have come up with a generic solution that ensures that you will not encounter such problems and that using the WCF-WebHttp BizTalk adapter in Receive Locations in tandem with Service Bus Relays is viable.

In this blog post I’ve detailed the problem, explained why it occurs, and explained how the solution works as well. If all you’re interested in is using the solution then you can download the installer here (or here if your machine is 32-bit), or the source code here.

The installer will GAC the relevant dll, and add a Service Behavior called BTWebHTTPSBRelayCompatibility to your machine.config files (if you prefer not to have your machine.config file updated then please don’t use the installer, download the source code and use the behavior as you wish). You will need to use this Service Behavior on your BizTalk WCF-WebHttp receive location to fix the problem detailed in this post. Please ensure you test the installer and the behavior before you attempt to use it in a real life environment, and let me know if you have any feedback.

 

Preamble about WCF-WebHttp Adapter and Service Bus Relays

Adding a relay endpoint to your WCF-WebHttp receive location is a piece of cake.  When running through the WCF Service Publishing Wizard, you just need to ensure you tick the checkbox in the below screenshot and you will be presented with additional screens at the final stages of the wizard that allow you to choose a service bus namespace and specify the ACS credentials that BizTalk uses to connect to the relay (I’ve found a way to get this to work with SAS, but more on this in another blog post).

Wizard

The wizard doesn’t actually create the relay for you.  This is done at the time the service is started up based on entries that get placed in the service’s web.config file as a result of the wizard (you could even manually add the web.config sections if you missed the relevant wizard options).  Note in the below screenshot that an additional endpoint that uses the webHttpRelayBinding binding has been added to the web.config file.  The beauty of this is that you can change the relay endpoint URL anytime by just updating the web.config.

Web.config

Now when the service spins up you’ll find that the relay has been enlisted automatically for you.  If you tear the service down, for example by taking down the application pool or stopping IIS, you’ll notice that the relay is no longer listed in the Azure Portal.

Portal

 

The Problem

What I found was that when I tried to send requests (regardless of the HTTP method) to the base URL of the relay (which in the above example is https://jctest.servicebus.windows.net/JCTestService/Service1.svc) there were no problems.  However the second I tried to add URL parameters to the end of the URL I would get an AddressFilterMismatch error.  I also encountered similar problems if I tried to change my relay address URL such that it differs from the URL format of my receive location (for example if I specify the relay URL in my web.config to be a value of https://jctest.servicebus.windows.net/API/V1).

AddressFilterMismatch

 

The obvious workaround and why it isn’t good enough

The aforementioned error message shouldn’t be new to anyone who has used the WCF-WSHttp receive location, as similar errors are encountered in scenarios where the URL that service consumers post to doesn’t match the server name (due to a DNS alias/load balancer etc…). The most common solution I’ve seen touted around to get around this problem is to use a WCF Service Behavior that changes the Dispatcher AddressFilter to a MatchAllMessageFilter which basically means that URL matching will not occur. This would be done in the ApplyDispatchBehavior method of the Service Behavior as below.

MatchAllBehavior

The nice thing about Service Behaviors is that they apply to all endpoints within our service, including our relay endpoint. Adding this behavior to the behaviors tab of our receive location will actually get around the two aforementioned problems which resulted in AddressFilterMismatch errors, however they introduce a new, very BizTalk specific problem.
You’ll now find that regardless of your URL/Method/Operation mapping on your BizTalk Receive location, the BTS.Operation context property will never be promoted on your inbound message. This is very limiting, because the BTS.Operation context property is a fantastic way to route your message to relevant orchestrations/send ports without having to have their filters based on the URL of the message. Any URL parameters that you tokenized in the mapping will also not be written to the context.
HTTPURLMethodMapping

 

Why the Problem Occurs
Getting to the bottom of the problem required a fair amount of usage of RedGate’s fantastic Reflector tool, I don’t think anyone could possibly figure out the problem without a reflection tool or having access to the source code.
I reflected the assemblies that are used within the BizTalk receive location and found that the way Microsoft implement the URL/Method to context property mapping is via a private Endpoint Behavior which is aptly named InternalEndpointBehavior. Microsoft actually adds this behavior to the default endpoint of the receive location programmatically within the adapter code, and the behavior does not get added to the relay endpoint which is why the relay endpoint behaves differently from the default endpoint.
The InternalEndpointBehavior sets the AddressFilter on the default endpoint to an instance of the WebHttpReceiveLocationEndpointAddressFilter class. This AddressFilter is what does all the magic of promoting the BTS.Operation and other context properties based on the inbound URL/Method. Unfortunately this class is marked as internal as well, so we can’t instantiate an instance of it from a custom behavior.
Since our first solution attempt was to use the MatchAllMessageFilter on our endpoint we would not see this context property promotion occur.
 
The Solution

The solution I’ve put in place is to create a custom service behavior that performs the below logic in it’s ApplyDispatchBehavior method.  You can download all the source code here.

First the behavior needs to figure out which ChannelDispatcher is related to the relay (since we’ll have the relay as well as the default ChannelDispatcher to contend with).

IdentifyRelayDispatcher
Next the behavior needs to loop through all endpoints, saving the base endpoint addresses for the relay endpoint as well as the default endpoint to local variables (the reason for this will be apparent soon). It also identifies the default endpoint and copies a reference to the contained InternalEndpointBehavior (which as I’ve mentioned applies the WebHttpReceiveLocationEndpointAddressFilter AddressFilter to the endpoint).

GetInternalBehaviorAndURLs

Next we’re going to apply the InternalEndpointBehavior to the relay endpoint which will apply the WebHttpReceiveLocationEndpointAddressFilter to the relay endpoint when it is executed.
ApplyInternalBehavior

This actually isn’t the end. This solution is good enough if the URL of the Service Bus relay mirrors the on-premise URL (except for the host name of course), however the second we want our relay URL to differ in structure (perhaps because we want to remove the .SVC extension, or provide a more friendly URL format) we will start to get an AddressFilterMismatch exception. The reason for this is that the URL format that the filter is searching for is based on the default endpoint’s format.  And because the filter is an internal class we have no ability to override the expected URL format.
So if your relay endpoint URL is https://jctest.servicebus.windows.net/test/service1.svc and the default endpoint URL is https://servername.onpremises.com/test/service1.svc then you won’t have a problem. But if the relay endpoint URL was https://jctest.servicebus.windows.net/API/V1 then you’d get the dreaded AddressFilterMismatch exception. For the default non-relay endpoint, the typical solution to this would be to use IIS’ URL Rewrite feature as described here, but I have found that this didn’t work for me with the Relay Endpoint.

So to get around this, we need to replace the Service Bus URL with the default endpoint URL in the To WCF Addressing header before the message gets to the AddressFilter.  I haven’t found an appropriate component that can change the To SOAP header value before the AdressFilter stage, so the solution I’ve implemented is to delay the execution of the AddressFilter such that it runs in a custom Message Inspector which will first override the To WCF Addressing header value such that it matches the AddressFilter’s accepted format, execute the AddressFilter, and then change the To WCF Addressing header back to it’s original value.

I’ll show you how I’ve achieved the aforementioned solution.  We now have the WebHttpReceiveLocationEndpointAddressFilter filter applied to our relay endpoint but this won’t work for us because we can’t run the Message Inspector prior to the filter.  So what we’ll do instead is copy a reference to the WebHttpReceiveLocationEndpointAddressFilter filter to a local variable, and change the filter on our relay endpoint to be a MatchAllMessageFilter, which will let pretty much any URL through (don’t worry, we’ll run our WebHttpReceiveLocationEndpointAddressFilter later).

ExtractFilter

The final step in our behavior is to now instantiate our custom Message Inspector, passing in the base address of the default endpoint, the base address of the relay endpoint and the reference to the WebHttpReceiveLocationEndpointAddressFilter filter object.  We then add this Message Inspector to our relay endpoint dispatcher.

AddMessageInspector

The custom Message Inspector is very simple. All it needs to do is override the To WCF Addressing Header from the Relay Endpoint URL format to the default URL format, execute the AddressFilter, and then set the To WCF Addressing Header back to the Relay Endpoint URL format.
MessageInspector

The beauty of this solution is that no change is introduced to the default endpoint at all, so you are only changing the behavior of the relay endpoint such that it matches the behavior of the default endpoint. Also, the code in the Service Behavior class will only execute when your service is first spun up. From then on the only code that will execute as each request is received will be the custom message inspector which is pretty lightweight and does very little additional work that isn’t encountered on the default endpoint anyways.  You can now set your Service Bus Relay endpoint URL to be whatever you want, and messages will get successfully relayed to your Receive Location with the BTS.Operation and the context properties based on tokenized URL parameters promoted/written to context as expected.

Whats possible

Of course, what would be even better would be if this behavior wasn’t necessary at all and that the BizTalk WCF-WebHttp receive location automatically set the appropriate filter on the relay endpoint for you…maybe Microsoft will adress this in BizTalk Server vNext? :)  Until then hopefully this will do the trick for you. If you have any better solutions then please do share them in the comments section.

I have just released a new version v1.5.5 of the BRE Pipeline Framework to the CodePlex project page.  This is a relatively minor update in that it only adds some missing WCF context properties to the enumerations used to get and set WCF context properties.

What makes this minor update important is that most of these context properties are REST related, and will enable you to build flexible REST based integration solutions where you can dynamically set HTTP methods, URLs, or headers.  The list of context properties added to the enumeration is below.

  • HttpHeaders
  • HttpMethodAndUrl
  • InboundHttpHeaders
  • InboundHttpMethod
  • InboundHttpStatusCode
  • InboundHttpStatusDescription
  • IssuerName
  • IssuerSecret
  • OutboundHttpStatusCode
  • OutboundHttpStatusDescription
  • StsUri
  • SuppressMessageBodyForHttpVerbs
  • VariablePropertyMapping

This new version is fully backwards compatible, and installing it will only require uninstalling the previous version from the add/remove programs control panel and then running the new installer.  Please post any issues/feedback/comments to the CodePlex project discussions page.

Azure App Services and Logic Apps have now hit public preview, and they have generated a lot of excitement (including with this writer).  It’s a bit like Microsoft has hit a reset button and now all the Microsoft integration specialists have to start from scratch (at least in the cloud, BizTalk Server isn’t going anywhere for awhile, thankfully since I would miss it), at least when it comes to tooling, concepts don’t really change…

One of the first proof of concepts I tried to work with (actually this was a team effort with a bunch of integration specialists excitedly fighting over the keyboard) was creating a Logic App that would pick up emails from a POP3 mailbox, and write the contents to a folder in my DropBox account, the file being named based on the subject of the email + .txt, and the contents of the file being based on the body of the email.  This is easy enough to do with the POP3 and the DropBox connectors as below.

Capture

We had quite a lot of difficulty trying to figure out how to set the filename in DropBox since it required a combination of hardcoded strings and the email subject which flowed out of the POP3 connector.  We tried a kazillion different combinations, and nothing seemed to work.

I reached out to the BizTalk Advisors Yammer forums hosted by Microsoft and got a helpful response from Stephen Siciliano at Microsoft in a snap (that’s Stephen!).  His suggestion is that if you’re mixing string constants and functions then you should not surround your function with quotes, and you should not use the @ escape character before the function.  Also, if you want to concatenate strings you must use the @concat function.  My final function looked like the below.

@concat(‘jc\’, triggers().outputs.body.Subject, ‘.txt’ )

This resulted in the email body being written to a file in a folder in the root path called jc, with the filename being based on the email subject + .txt.  Interestingly, if you view the code view you’ll notice that the slash after the folder name gets automatically escaped (so you don’t have to escape it, in fact if you do then you will end up with two escaped slashes).

Escaped

I hope this helps others who are also struggling to learn the new syntax.  Just like with any new language, once you understand the rules the learning curve is a lot less steep.  Don’t take it for granted that this syntax will remain the same.  I imagine that by the end of the public preview Azure App Services might be a very different beast.

Happy AAS-ing.

A quick blog post to document a tough lesson learnt by myself so others can hopefully avoid making the same mistake. Changing a BizTalk WCF send port’s send handler (i.e. the host instance it is associated with) will remove all passwords that are currently configured on the send port. This includes credentials to access the target service as well as the proxy server password if there is one.  This appears to be the case with WCF-SQL receive locations as well, however a quick test with the File adapter showed that this behavior might not be consistent with non-WCF adapters so it’s best to try your change on a development environment first.

How can you tell if your password has been wiped? Normally when you access the send port, if you have previously entered a password then you will see the black circles indicating a masked password is present. If the password has been wiped then the field will be totally blank as below.

PasswordBlanked

As a practice I would encourage others to avoid hardcoding passwords on send ports and to use the SSO’s credential mapping facilities instead (a blog post on how to achieve this using the BRE Pipeline Framework soon).  This isn’t applicable for proxy server credentials but those can typically be set on the send handler level so you don’t have to worry about associating then with your send ports.  

Moral of the story, if you’re changing your adapter handler on a port and there are passwords associated with the port then re-enter them.

Follow

Get every new post delivered to your Inbox.

Join 99 other followers

%d bloggers like this: