A recent challenge I had to face was to get the ESB Toolkit 2.1 installed on a multi server environment where BizTalk Server 2010 and the SQL Server database are hosted on seperate servers.  We wanted for BizTalk Administrators to use the ESB Portal to view faults, to edit and resubmit messages to the MessageBox, and to create and manage alerts and subscriptions.  The BizTalk Administrators should be able to access the portal from their PCs rather than just on the BizTalk server, as they would be clicking links in alert emails to quickly view fault details and action faulted messages.  I managed to get this working and below is an account of how I managed to achieve this, I hope this helps others who are facing similar challenges.

I am assuming that you have already installed the ESB Toolkit BizTalk application, ESB Portal, ESB Admin/Exception/Itinerary databases, and supporting web services.  There are a lot of blog posts out there on how to do this, I recommend you don’t use the cmd scripts provided by the installer and instead build the installer in the ESB.Portal solution and use that to deploy the portal instead as detailed here – http://www.google.co.nz/url?sa=t&rct=j&q=esb%20toolkit%202.1%20installation%20guide&source=web&cd=2&ved=0CCUQFjAB&url=http%3A%2F%2Fdownload.microsoft.com%2Fdownload%2F9%2FC%2F5%2F9C5AEC46-717F-49B6-A7BC-36B0930B63FF%2FInstalling%2520BizTalk%25202010%2520ESB%2520Toolkit%25202.1.htm&ei=Z2BYT5LkDIWdiAeJxamsDQ&usg=AFQjCNGA97tfJuXd_RTGX2JrtXh1azt_vg (sorry, this isn’t a very nice link, and you might have to save the htm file to your PC and open it manually if you’re using later versions of web browsers, but it is very informative).

The biggest challenge in this scenario is the fact that the ESB Portal web application refused to forward the impersonated credentials to the SQL Server since it is on another server.  This is known as the double hop issue, and is one faced quite often in multi server setups, be they BizTalk, sharepoint, or any other kind of application that relies on forwarding impersoned credentials across servers.  The double hop issue is quite well explained here – http://support.microsoft.com/default.aspx?scid=kb;en-us;329986.  Luckily I managed to find a pretty good blog post that helped me work with our IT Infrastructure team to work around this problem, see the post at http://blogs.technet.com/b/meamcs/archive/2011/11/15/esb-2-1-portal-minimal-access-configuration-kerberos.aspx.  At this stage you should have a new AD User (lets call this Test-ESBPortalUser for the rest of the blog post) created and should setup the SPNs and delegated trust between IIS and SQL Server for this username as detailed in sections 2.2.1-2.2.3 in the aforementioned blog post (our IT Infrastructure guy found this a very straightforward task using the instructions in the aforementioned blog, so you should be able to turn to someone in an equivalent role at your company in case you are not comfortable with these concepts).

Now here is where you may diverge somewhat from me.  I decided that I wanted this Test_ESBPortalUser account to have the minimum access required to the BizTalk databases so I added had it added to the BizTalk Application Users AD Group.  I later found that this group didn’t have all the required permissions and had to manually grant special permissions to this user (detailed further later).  You might have better luck trying to add the user to the BizTalk Server Administrators group, but I really didn’t want to do that.

The next step is to get impersonation working so that all the ESB web applications are able to access the SQL Server database.  You’ll quickly find that even if you enable impersonation in IIS or in the web.config to use the authenticated user, and you set the web applications app pool to a user that has permission to the required databases, or if you access the ESB Portal using credentials that have access to the required databases, the ESB Portal will still fail to load with 401 unauthorized type errors.  Well, according to http://qa.social.msdn.microsoft.com/Forums/en-US/biztalkesb/thread/2a1519c1-bab3-40fb-a645-a7e54f606b22 I found that one must actually enter the username and password of the impersonated user in the web.config file (or through IIS which will just write it to the web.config file).  This is quite a terrible way to store credentials because they are exposed in plain text.  Luckily Mark Brimble at Datacom NZ (thanks a ton Mark!) managed to give me an idea that I explored further and works quite well.

Microsoft has provided a command line tool called aspnet_setreg.exe that you can use to encrypt your credentials in the registry and have your web application look up and decrypt those credentials securely.  Download the tool from here – http://support.microsoft.com/kb/329290 and follow the below instructions.

  • Open a command prompt (you might have to run as administrator if UAC is enabled) window and browse to the folder where you saved the utility.
  •  Type in the following command, replacing the <username> and <password> with the relevant values for your Test-ESBPortalUser equivalent) – aspnet_setreg.exe -k:SOFTWARE\Microsoft\ESBPortal -u:”xxxxx” -p:”xxxx”
  • Open registry editor.
  • Browse to the following key – HKLM\SOFTWARE\Microsoft\ESBPortal\ASPNET_SETREG (note that if you are doing this on a 64-bit machine the key you’re after will be HKLM\SOFTWARE\Wow6432Node\Microsoft\ESBPortal\ASPNET_SETREG even though the output from the command might say otherwise).
  • Right click on the key and select permissions.
  • Grant read access to the Network Service account on the machine.

What we have done is store the impersonation credentials in the registry, and ensured that only the Network Service account (and administrators of course) are able to access this key.  That’s a whole lot better than plain text in a web.config file.  Now the next trick is to update the relevant web.config files to make use of these encrypted credentials.  What you want to do is to replace the <identity impersonate=”true”/> sections in these web.config files with <identity impersonate=”true” password=”registry:HKLM\SOFTWARE\<Wow6432Node\>Microsoft\ESBPortal\ASPNET_SETREG,password” userName=”registry:HKLM\SOFTWARE\<Wow6432Node\>Microsoft\ESBPortal\ASPNET_SETREG,userName” /> instead (the <Wow6432Node\> section is only required if you are on a 64-bit system).  This will allow these web applications to make use of the encrypted credentials for impersonation.  The applications who’s web.config files you want to update are listed below.

  • ESB.Portal
  • ESB.UDDI.Service
  •  ESB.BizTalkOperationsService
  • ESB.ItineraryServices.WCF
  • ESB.ResolverServices
  • ESB.ResolverServices.WCF

You will also want to setup authorization on the ESB.BizTalkOperationsService and ESB.Portal applications in IIS such that only the BizTalk Server Administrators are allowed to access these applications.  There is an additional section in the ESB.Portal web.config (System.Webserver/Security/Authorization) where you will need to manually add permissions for the BizTalk Server Administrators role.

Getting closer to done now but theres still a bit more to go 🙂 Now it’s time to make sure we have the correct authentication settings configured for all the web applications in IIS.  I’ll group the applications which have common settings together, assume that any authentication providers I don’t mention should be disabled.

  • Windows Authentication Enabled (Right click and choose providers and only Negotiate should be selected, Right click and choose advanced settings and ensure that Kernel mode authentication is enabled and extended protection is off), and ASP.NET authentication enabled
    • ESB.Bam.Service
    • ESB.BizTalkOperationsService
    • ESB.Exception.Service
    • ESB.UDDI.Service
  • Windows Authentication Enabled (Right click and choose providers and only Negotiate should be selected, Right click and choose advanced settings and ensure that Kernel mode authentication is enabled and extended protection is off), and ASP.NET authentication enabled, Anonymous authentication enabled
    • ESB.ExceptionHandlingService
    • ESB.ItineraryService
    • ESB.ItineraryService.Generic.Response.WCF
    • ESB.ItineraryService.Generic.WCF
    • ESB.ItineraryService.Response
    • ESB.ItineraryService.Response.WCF
    • ESB.ItineraryServices.WCF
    • ESB.ResolverServices
    • ESB.ResolverServices.WCF
    • ESB.TransformServices
    • ESB.TransformServices.WCF
  • Windows Authentication Enabled (Right click and choose providers and Negotiate and Negotiate:Kerberos should be selected, Right click and choose advanced settings and ensure that Kernel mode authentication is disabledand extended protection is off), and ASP.NET authentication enabled
    • ESB.Portal

Lets ensure that we have all the web applications running in the correct application pools.  Below are the list of app pools you should have setup with their relevant configurations, and the web applications that should be running under these app pools.

  • CoreEsbAppPool – Running under relevant BizTalk Isolated Host User account and in Integrated mode
    • ESB.ItineraryServices
    • ESB.ItineraryServices.Response
    • ESB.TransformServices
  • CoreEsbNetworkAppPool – Running under relevant BizTalk Isolated Host User account and in Integrated mode
    • ESB.ResolverServices
  • CoreEsbOpsAppPool – Running under relevant BizTalk Isolated Host User account and in Integrated mode
    • ESB.BizTalkOperationsService
  •  CoreEsbWCFAppPool – Running under relevant BizTalk Isolated Host User account and in Integrated mode
    • ESB.ItineraryService.Generic.Response.WCF
    • ESB.ItineraryService.Generic.WCF
    • ESB.ItineraryService.Response.WCF
    • ESB.ItineraryServices.WCF
    • ESB.ResolverServices.WCF
    • ESB.TransformServices.WCF
  • EsbPortalNetworkAppPool – Running under relevant Network Service account and in Classis mode
    • ESB.Bam.Service
    • ESB.Exceptions.Service
    • ESB.Portal
    • ESB.UDDI.Service

Ok, next thing we need to do is give the missing SQL permissions to the Test-ESBPortalUser impersonation account.  Create the login for the user in SQL Server, and create user mappings for the BizTalkMgmtDb database with the BTS_Operators and public roles, and to the ESBException database with the ESBPortalAdmin and public roles.  Edit the ESBPortalAdmin role on the ESBExceptionDB such that it has select, delete, insert access on the alert, alertcondition, and alertsubscription tables.

Lastly, enter the services mmc and set the BizTalk ESB Exception Notification service to run under the Test-ESBPortalUser.

I’m hoping this is enough to get you going, please post a comment if you find that I have missed anything, got anything wrong or made incorrect assumptions, could possibly reduce permissions even further, or if you just need help in general.  I’m hopefully looking at doing a production install of the ESB Toolkit in the next week, and I will update this blog post if I find that I had to do anything different.  Good luck.