Client Certificate + Username authentication = WCF Custom Behavior

Recently I had to connect o a WebSphere service using BizTalk. The service required a client cetificate for transport authentication but also a username/password in the soap envelope. Ok, this might be considered as overdoing the security, both requirements authenticating the client, but it did not sound too strange to me and I thought BizTalk with WCF would probably handle that out of the box without any problems.

Well.. turned out I was wrong and it wasn’t an easy nut to crack for me so I’ll blog it for the masses.

First, this was a service that supported SOAP version 1.1, also known as BasicHTTP for WCF people. This offers you the following security types:

Type Descriptions
None Anonymous, so no no security and no authentication
Transport Transport level authentication provider. Comparable to the IIS settings, eabling you to set None (no authentication), Basic, Digest, Ntlm, Windows or Certificate (encryption) authentication. This option requires that you use HTTPS.
Message Enables Authenticaton in the SOAP header.  Offers using a username / password or certificate (signing), In this case, the certificae will be used as a signature over the the soap message and not for transport level encryption. Because this option is for HTTP transport only, WCF does not permi the usage of the usernam/passowrd option.
TransportWithMessageCredential A combination of Transport options and Message options, but does not offer to set a client certificte for transport. So basically this offers you to use a username/password or a certificate (signing) in the message. This enables Transport level security so it requires usage of the HTTPS protocol
TransportCredentialOnly Similar to Transport, but allows you to send unencrypted passwords over HTTP. Good for testing without certificates, but not meant for production purposes.

Basically, these are the samen options you use when using WCF in a C# project. In theory, you would use Transport with Message Credential, but the BizTalk Interface does not give you the opportunity to use a client certificate..

TransportWithMessageCredential

So what if we would do this in a ‘normal’ c# project? I created the service proxy and edited the app.config like this:

inside the binding:

<security mode="TransportWithMessageCredential">
    <transport clientCredentialType="Certificate" proxyCredentialType="None" realm="" />
    <message clientCredentialType="UserName" algorithmSuite="Default" />
</security>

the client definition references a my credential behavior:

<client>
     <endpoint address="https://mysite.com/myservice"
        behaviorConfiguration="CredentialBehavior" binding="basicHttpBinding"
        bindingConfiguration="myserviceHttpBinding"
        contract="mysite.myservice" name="myserviceHttpPort" /> 
</client>

The behavior selects the right certificate

<behaviors>
    <endpointBehaviors>
        <behaviorname="CredentialBehavior">
            <clientCredentials>
                <clientCertificatefindValue="F3F4B10B819A4C1B3E54BA8B8E1F888D75E4FB23"
 x509FindType="FindByThumbprint"storeLocation="LocalMachine"storeName="My" />
            </clientCredentials>
        </behavior>
    </endpointBehaviors>
<behaviors>

Because in a C# project you cannot specify the user credentials in the config file, you need to specify these in code:

client.ClientCredentials.UserName.UserName = "Logica";
client.ClientCredentials.UserName.Password = "itsme";

Unfortunately this did not work. Even when explicitly specified, WCF does not want to send the certificate. Finally, a search with BING on TransportWithMessageCredential BasicHttpBinding clientcertificate led me to this post:

http://social.msdn.microsoft.com/Forums/en-US/wcf/thread/45df57f0-e010-43ef-92f6-ed06c247d733/

So it’s not just me.. it’s official.
With the solution already being there, I tried it out. One more hiccup was solved quick using another search, and now I both ‘forced’ WCF to send the client certificate while suppressing the timestamp. For this, some custom code is needed:

// code to set RequireClientCertificate = True
// also set IncludeTimestamp = false
// necessary to enable client certificates over BasicHttpBinding.
ServiceProxy.serviceClient client;
client = new ServiceProxy.dw_ serviceClient ();
System.ServiceModel.Channels.Binding bbinding =
      (System.ServiceModel.Channels.Binding)client.Endpoint.Binding;
BindingElementCollection bec = bbinding.CreateBindingElements();
TransportSecurityBindingElement tsp = bec.Find();
tsp.IncludeTimestamp = false;
HttpsTransportBindingElement httpsBinding = bec.Find();
TextMessageEncodingBindingElement encoding = bec.Find();
httpsBinding.RequireClientCertificate = true;
CustomBinding binding = new CustomBinding(tsp, encoding, httpsBinding);
client.Endpoint.Binding = binding;

Now this worked in WCF using a ‘normal’ C# project and some custom code. But this being a BizTalk solution, I needed to be able specify this all in configuration. The regular way to do this, is by using a custom behavior in WCF.
A custom behavior is not that hard to create. You just create a new c# project, with a main class (I named “SendClientCertificateBehavior”) that derives from both System.Attribute and System.ServiceModel.Description.IEndpointBehavior. Use the right mouse button to implement the interface and add the code to the “ApplyClientBehavior” behavior (in this case). Use the endpoint as provided in the method parameters.
Also, you need to define the configuration element by creating another class that derives from System.ServiceModel.Configuration.BehaviorExtensionElement. I added that class in the same C# file. The two methods of this class refer to the first class.:

public override Type BehaviorType
{
  get { return typeof(SendClientCertificateBehavior);
}
}
protected override object CreateBehavior()
{
  return new SendClientCertificateBehavior();
}

To work in BizTalk, this assembly needs to be signed, and put into the GAC. Then, in machine.config file you need to add a line for the new Behavior in the section :

Now, for using plain old c#, you add ‘SendClientCertificate’ to the behavior in the the app.config:

<behaviors>
  <endpointBehaviors>
    <behavior name="CustomBehavior">
    <clientCredentials>
      <clientCertificate findValue="F3F4B10B819A4C1B3E54BA8B8E1F888D75E4FB23"
                         x509FindType="FindByThumbprint"
                         storeLocation="LocalMachine" storeName="My" />     </clientCredentials>
    <sendClientCertificate />
    </behavior>
  </endpointBehaviors>
</behaviors>

In BizTalk, you have to use a WCF-Custom port. These ports come with no settings applied, so you have to select some stuff yourself:
First enter the url of your service. Then specify BasicHTTPBinding. In the binding, specify the securitymode as ‘TransportWithMessageCredential’. After that you have to add the behaviors in the “Behavior” tab. Right click on ‘endpointBehavior’ and add both the clientCredentials behavior and the sendClientCertificate behavior we created ourself:

Under ‘clientcredentials’ we can add the certificate information, in my case a thumbprint of a certificate that is in the CurrentUser section.
In the credentials tab, you can then add username and password.

So.. in the end it IS fully configurable. Just needs a lot of work up front.

Copy local that BizTalk 2009 reference

I’m sure by now everyone has met the evil ‘copy local’ problem in BizTalk.  It’s blogged by Jan here and by Ryan as well. Often, you want to add your schema project as a reference to the map or orchestration project. Things might look allright at first, but then after some changes, you get build errors. Once you know what it is, it’s easy to solve, just put that ‘copy local’ setting on your reference to false, then directly back to true. Magically (?) it all works again.

It’s one of the issues that gets attributed to BizTalk 2009, but I just reproduced this in a C# project trying to reference a BizTalk project. What exactly happens? It did all work in 2005!

In fact, Visual Studio is fooling us all. The csproj and btproj files don’t actually have that copy local at all! For a copy local referenc, The project file should contain something like this:

<ProjectReference Include="..Mysolution.SchemasMysolution.Schemas.btproj">
   <Project>{00000000-0000-0000-0000-000000000000}</Project>
   <Name>Mysolution.Schemas</Name>
   <Private>True</Private>
</ProjectReference>

However the part <Private>True</Private> is missing. So the compiler does not copy the dll locally.

Why this strange behaviour? I found an explanation in the MSDN: http://msdn.microsoft.com/en-us/library/ez524kew.aspx

At run time, components must be either in the output path of the project or in the Global Assembly Cache (GAC). If the project contains a reference to an object that is not in one of these locations, you must copy the reference to the output path of the project when you build the project. The CopyLocal property indicates whether this copy has to be made. If the value is True, the reference is copied to the project directory when you build the project. If the value is False, the reference is not copied.

If you deploy an application that contains a reference to a custom component that is registered in the GAC, the component will not be deployed with the application, regardless of the CopyLocal setting. In earlier versions of Visual Studio, you could set the CopyLocal property on a reference to ensure that the assembly was deployed. Now, you must manually add the assembly to the Bin folder. This puts all custom code under scrutiny, reducing the risk of publishing custom code with which you are not familiar.

By default, the CopyLocal property is set to False if the assembly or component is in the global assembly cache or is a framework component. Otherwise, the value is set to True. Project-to-project references are always set to True.

Since BizTalk assemblies are always deployed to the GAC, te fact if your copy local contains the right private setting depends on if your project was deployed before you added the reference. Even though you still seem to have copy local set right!

An unhappy side effect of this is that the orchestration / mapping is compiled against the schema that is in the GAC and not the one that is in your project. Leaving you to guess why that change you made did not compute at build time.

I also remember having some other problems with assemblies in the GAC referred in Visual Studio. Perhaps this is all related. I hope this issue is gets addressed in the next release!

ALWAYS export your itinerary in strict mode

I lost quite some time last week trying to figure out why my itinerary services were processed out of order. I created a simple solution based on the A-B-C scenario I wrote about last week. Difference was there that I am using a common data model in the middle, so I need to transform my incoming message to a common model, and my outgoing messages as well, just like the return messages.

So I created an itinerary with the following sequence:

  1. On-Ramp (file)
  2. Message Transform service (to common request)
  3. Message Route service
  4. Off-Ramp extender
  5. Message Transform (to outgoing request)
  6. Off-Ramp (using the itinerary forwarder pipeline)
  7. Message Transform (to common response)
  8. Message Route back
  9. Off-Ramp extender
  10. Message Transform (to imcoming response)
  11. Off-Ramp (file)

If you export this using the default setting, each ESB pipeline will process as much as it can, until it finds an off-ramp service. Depending on some other configurations (two-way,  request response, putting the route on send or receive side, changing the pipelines on the ports) this resulted in different orders or processing, none of which were what I wanted. Mostly, the send pipeline in the first off-ramp would try to do both the send and receive transform in the send. Looking at the exported XML indeed I could not see how the pipeline would ‘guess’ the order that I indicated so carefully in my DSL.

Turns out you need to set the “Export Model” option to strict. The default setting was intenede for compatibility with V1 of the guidance. On the ESB forum, this is what Brian Birdoes says about the export model:

The strict mode supports new features such as stage and the broker.  As mentioned in the docs the stage attribute is output and utilized by the ESB Dispatcher in On-Ramps and Off-Ramps.  Without it, the engine basically processes as many steps as it can until it ends or hits an orchestration or off-ramp step.  With the stage attribute the engine can skip a step and not execute it until the correct pipeline direction.  For example, if you want a transform to run on a request-response (service pass-through) scenario on the Off-Ramp’s receive pipeline, the only way to do this is to have the stage attribute.  This could be a common scenario since the original requestor may not expect the message in the format returned by the service.  In default mode the transform would always execute on the Off-Ramp’s send pipeline.

The broker capability utilizes the id and next id to work correctly.  In V1 the steps are processed sequentially.  For more advanced routing like the broker the engine needed a way to jump to steps.  The pattern was basically to use a linked list within the itinerary.  The outcome of these filters really is to determine the correct id to jump to for the next step.

Simply put, Default mode is for backward compatibility with V1 itineraries.  Strict mode is required for many of the new V2 features.  For itineraries that I build I generally use Strict mode.

So hope I can prevent you from making the same mistake. Make ‘strict’ your new default.

Create BizTalk Project … Project Creation Failed

Friday I uploaded a new update in microsoft update. It was a 365 MB Update called the “Security Update for Microsoft Visual Studio 2008 Service Pack 1 (KB971092)”. Turns out this is an update of another update that was released a week earlier (see this blog also)

If you are using BizTalk 2009 you will probably experience the same issue as me. After installing it, you cannot create BizTalk projects anymore. You will get the very discriptive text “Project Creation Failed” in your status bar. In the eventlog you will find that the template is not installed.

The solution? Just load the BizTalk CD or ISO to your PC and choose ‘Repair’. After that you can create the projects again.

Update:

Not really a patch, but an easier way to solve this was published on the biztalk customer response team blog. Apparently you only need to change a little thing in the registry. Get the full text at this link

We could get the project system to work by repair installing Biztalk but that doesn’t seem like a solution.  Further debugging revealed that the following registry setting was modified by a VS update.

 [HKEY_LOCAL_MACHINESOFTWAREMicrosoftVisualStudio9.0Projects{FAE04EC0-301F-11d3-BF4B-00C04F79EFBC}]

“PossibleProjectExtensions”=”csproj”   – Original VS install reg value. “PossibleProjectExtensions”=”csproj;btproj” – Post Biztalk installation reg value

The following is the entry for the 64 bit version of VS:

HKEY_LOCAL_MACHINESOFTWAREWow6432NodeMicrosoftVisualStudio9.0Projects{FAE04EC0-301F-11d3-BF4B-00C04F79EFBC}

xpath to the Max() – still no support

Today I wanted to use a simple XPATH function in my orchestrationto find the maximum value of a sequence number in a set of nodes. So I booted up XMLSpy and loaded the xml schema. Simple problem, simple solution:

max(//sequence_nr)

Wrong.. biztalk does not support xpath 2.0. Even worse, I cannot even program a helper function since .NET 3.5 does not support XPATH 2.0. This was kind of hard to verify since msdn states something different.

Since my records are ordered, luckily I could also pick the last record. Which resulted in the uglier xpath statement:

string(//record[count(//record)]/sequence_nr)

That the actual statement was a lot longer since it had namespaces and some subnodes. And I really don’t see how I could do this in xpath when the records were not sorted.

This post in microsoft connect I found acknowledges that xpath 2.0 is still not supported in .Net framework 3.5

I hope they will manage to fit support in framwork version 4.0

A – B – C itinerary using the ESB Toolkit

Now that I had my SQL Adapter insert covered as show in the last post, I wanted to create an itinerary that gets a message from a one way port, then sends it to the SQL Adapter to do that insert (request / response), and then with the result of that insert, trigger another one way send port. So from A (one way) to B (two way) to C (One way) This is a scenario that you can configure in BizTalk as message routing using pub/sub, but how would you do this in an itinerary?

What I did not find right away is how to forward the response message to another port. In fact, it wasn’t easy to find any sample that had more then one (send port) at all. Would this mean creating two off-ramps? and how to configure the off-ramp ports? And would the itinerary be a one way itinerary or a two way?

Brian Birdoes sent me on the right track: in the multiple web services sample, there is a one way scenario. However this one is only provided as an XML and the itinerary DSL file is not provided and on top of that, it has a bug. The key issue was that when you want to send on a response message from a solicit response port, you need the ItineraryForwarderSendReceive pipeline in the response port. And the sample used the normal DynamicResolutionSolicitResp port instead of the DynamicResolutionSolicitRespForwarder

Using Hernan de lahitte’s Itinerary Designer Model Importer did some work, but left me still with a broken itinerary. Using some trial and error I recreated a model that produced exactly the same XML. After that I put some nice names on it and posted it here so we can all profit from it.

Get the updated sample itinerary

WCF SQL Adapter through an itinerary

I just started using the ESB Toolkit, and I wanted to create an itinerary that sends messages to the SQL Adapter to do an insert action.

I found the sample on configuring the SQL WCF port with some help with of the ESB Toolkit forum, so no problems there. But the documentation for the sample is non-existent. So I thought I should give you the head-start I did not get. It’s really not that difficult.

First things first, I am assuming you already know how to do a two-way two-way itinerary. Without mapping this would be a request response itinerary with four sequential shapes, the fourth connecting back to the first:

  1. On ramp (two way)
  2. Routing Service Messaging Extender (itinerary service, select messaging as extender, Routing as service)
  3. Off ramp extender (itinerary service)
  4. Off ramp (two way)

You add a resolver to the messaging extender to enter the parameters for the dynamic send port.

WCF SQL Adapter through an Itinerary

The ‘magic’ for calling the SQL adapter is all in that resolver. To keep things easy, we use a static resolver with the following properties

  • Resolver implementation: static
  • Transport name: WCF-Custom
  • Transport location: mssql://myserver//mydatabase?
  • Endpoint configuration: BindingType=sqlBinding
  • Action: {TableOp/Insert/dbo/myTable}

Basically you can copy those attributes from the binding file that was generated when you used the generate schema option for the WCF SQL Adapter. I did not test if it is possible to use the BTSActionMapping construct but it would make more sense to decide the operation in a business rule resolver when you are doing ESB Toolkit anyway. In any case don’t forget the curly brackets around the action.

The actional sample, the SqlAdapter-TwoWay-MessageRouting-MessageTransform-MessageTwoWaySendPort.itinerary file contains an additional mapping, but is basically the same.

Don’t forget, before you install this sample you have to install the dependent samples: Dynamic Resolution Sample, Resolver Service Sample and the Itinerary On ramp sample.

This sample you can try out yourself by executing setup.bin in the installscripts folder, then execute Esb.Itinerary.Test.exe from the itinerary sample. Select Two Way Service in web service options and deselect Use WCF Service,  Load the itinerary xml provided in the itineraries folder of the sample, load up the request message from the testdata folder and press request. If all is well, you should get the response back.

Once upon a cloud

Welcome to my new and first BizTalk, ESB, SOA and CLOUD related blog. I’ll be using this blog to share (and keep) things I find worth knowing. Much will be about BizTalk and ESB, but as time tends to go, I expect it might move up to the SOA and Cloud computing level soon enough, anywhere the technology brings us.

The name? Well maybe you could say it relates software elements with fantasy and fairy tale. Maybe those nice 3 letter acronym are we use, seem just that. Something that is too good to be true. Hard to achieve and not without peril. Technologies may be compared to dragons, beautiful in a way but hard to conquer. And isn’t it often that when we think we get there, there are clouds rising on the horizon? A Bug? Or worse: A Flaw! Which will be followed by the hunt for more technology, bright and shining? Perhaps another 3 letter word?

In short, my stories on this blog will follow my adventures in a fantasy way. On good days, there can be amazing stories of wonder. When things are down, there will be dark stories of problems. And, of course, I promise now that the days when nothing is notable that will pass like they never happened.

Pim