måndag 26 augusti 2013

Debugging Maven based, embedded Mule ESB Community Edition projects in Mule Studio

If you are running Mule ESB Community Edition embedded in an app server and you are deploying your Mule projects as .war archives to that server you might have come across issues regarding using Maven as build tool for your projects in Mule Studio to enable stand alone debugging of your Mule apps.

When using this setup you are not following Mulesofts recommended format of Maven projects and will probably struggle trying to import them into Mule studio.
Furthermore having a pom.xml building .war files for deploy will not be compatible with the standalone nature of Mule studios setup.

You could of course do remote debugging on your app server platform but I want to be able to completely work in Mule studio first making sure everything works standalone before deploying anything to the embedded Mule CE runtime. Another way is to simply debug as Mule application without Maven, but then come across issues having to add user libraries manually to Mule Studio each time to avoid runtime classloading issues.

My solution to this challenge was to create different profiles in your pom.xml and invoke them based on if you are developing / debugging in Mule or if you are about to deploy and a small trick to make that setup work with Mule studio.

First import your Mule project into Mule studio (you may need to do it without Maven support enabled to be able to import it if you are not following their project format). Then from within the package explorer enable Maven support.
Now edit your pom.xml and add your build plugin if its not there already.


 <build>   
  <plugins>   
   <plugin>   
    <groupId>org.mule.tools</groupId>   
    <artifactId>maven-mule-plugin</artifactId>   
    <version>1.4</version>   
    <extensions>true</extensions>  
   </plugin>   
  </plugins>   
 </build>  

Now to the solution edit your pom.xml and add two profiles one "war" profile and one "standalone" profile.
Make the war version activated by default and set packaging type on it to "war".
In the "standalone" profile set packaging type to "mule" and have the activation being triggered if an environment variable called "environment" is set to "mule" like this:
  <profiles>  
     <profile>  
       <id>war</id>  
       <activation>  
         <activeByDefault>true</activeByDefault>  
       </activation>  
       <properties>  
         <packaging.type>war</packaging.type>  
       </properties>  
     </profile>  
     <profile>  
       <id>standalone</id>  
       <activation>  
             <property>  
                    <name>environment</name>  
                    <value>mule</value>  
                   </property>  
             </activation>  
       <properties>  
         <packaging.type>mule</packaging.type>  
       </properties>  
     </profile>  
  </profiles>  

Now you should be able to build your project as as .war file for deployment on the embedded server just by running your normal maven commands like "mvn clean package".

To be able to do "Debug as Mule application with Maven" in the package explorer menu of Mule Studio you will however need to trigger the other profile.
To do this go into
Windows -> Properties -> Mule Studio -> Maven Settings
and set MAVEN_OPTS environment variable to "-Denvironment=mule".

Now when you do "Debug As Mule Application with Maven" on all your Mule Studio projects it will try to find profiles that are triggered by the "environment" environment variable with value set to "mule" and run it accordingly making sure packaging is .zip rather than .war and that all dependencies and settings for that profile follows your pom structures.

Note that even though projects will build just fine. If you have libraries not included in the Mule CE runtime referenced in a pom project structure with parent pom's that reside outside of the Mule project, the Mule Studio runtime will not find these libraries if they are set as "provided" (read  provided by the parent pom) in the pom.xml. One solution for this is to actually have the dependencies defined inside corresponding profile with "provided" set on the dependency inside the "war" profile and without in the "standalone" profile. Example:

  <profile>  
       <id>war</id>  
       <activation>  
         <activeByDefault>true</activeByDefault>  
       </activation>  
       <dependencies>  
           <dependency>  
                <groupId>org.json</groupId>  
                <artifactId>json</artifactId>  
                <version>20090211</version>  
                <scope>provided</scope>  
           </dependency>  
       </dependencies>    
       <properties>  
         <packaging.type>war</packaging.type>  
       </properties>  
     </profile>  
     <profile>  
       <id>standalone</id>  
       <activation>  
           <property>  
               <name>environment</name>  
               <value>mule</value>  
           </property>  
       </activation>  
       <dependencies>  
       <dependency>  
            <dependency>  
              <groupId>org.json</groupId>  
              <artifactId>json</artifactId>  
              <version>20090211</version>  
            </dependency>  
       </dependencies>              
       <properties>  
         <packaging.type>mule</packaging.type>  
       </properties>  
     </profile>  


Embedded Mule project Mavenized!

fredag 5 juli 2013

RESTful MongoDB and Mule ESB Community Edition pattern

There are some alternative ways to go when RESTifying your flows in Mule. However they are poorly described in manuals and forums and the components and tools available are not very active projects in the community. Struggling with Mule's REST router and other alternatives I settled for the built in Jersey way of doing it. However all examples from the Mule documentation and forums describe only how to enable REST components to your Http endpoint and return directly. The do not describe how to integrate the RESTful endpoint with your Mule ESB integration flows and hence take advantage of Mule ESB's other powerful integration abilities.

Here is how I do it:




I use a Http endpoint or Https if secure and couple it with a REST element like this:
 <jersey:resources doc:name="REST">  
      <component class="se.redpill.mulecomponents.mongo.RedpillRESTMongo"/>  
 </jersey:resources>  


The actual REST component is just plain Jersey Java, like this:
 @GET  
      @Path("mypath/{singleelement}")  
      @Produces({MediaType.APPLICATION_JSON})  
      public Map<String, Object> getQueryByName(@PathParam("singlelement") String name )   
      {   
           Map<String, Object> query = new HashMap<String, Object>();  
           query.put("MyCoolDocument.Type", "Cool");  
           query.put("MyCoolDocument.Unit", name);  
           query.put("Year", new Integer(2012));  
           return query;  
      }  


All fine. Now we use a Choice element to check on the inbound property if the request is
'http.method' GET, POST, PUT or DELETE and route each choice to its corresponding component. In my case I have a mapping to my own MongoDB components which are in the same format as my last blogpost.

Now to the real HACK to get this working:

If you tried something like this and failed you have probably seen that the object passed from the REST element  to your component is actually
org.mule.module.jersey.MuleResponseWriter$1
which is a pain to handle. But the payload can be casted to a
org.mule.api.transport.OutputHandler since its implementation comes from
org.mule.module.jersey.MuleResponseWrite

To avoid having to serialize / deserialize your own return value (in my case a Map) from your REST service to your next element (in my case my MongoDB query component) with casting techniques describe above you can instead do it in a much sager manner using the following statement:

           ContainerResponse cr = (ContainerResponse) eventContext.getMessage().getInvocationProperty("jersey_response");  
           Map<String, Object> map = (Map<String, Object>)cr.getResponse().getEntity();  

VOILA! We now have our Map back that we sent from our REST component.
My MongoDB Query component and all the others for PUT, POST and DELETE are implemented in this fashion like this:

 package se.redpill.mulecomponents.mongo;  
 import java.util.Map;  
 import org.mule.api.MuleEventContext;  
 import org.mule.api.lifecycle.Callable;  
 import com.mongodb.BasicDBObject;  
 import com.mongodb.DBCursor;  
 import com.mongodb.DBObject;  
 import com.sun.jersey.spi.container.ContainerResponse;  
 public class QueryMongoComponent extends AbstractMongoComponent implements Callable  
 {  
      @Override  
      public Object onCall(MuleEventContext eventContext) throws Exception {  
           ContainerResponse cr = (ContainerResponse) eventContext.getMessage().getInvocationProperty("jersey_response");  
           Map<String, Object> map = (Map<String, Object>)cr.getResponse().getEntity();  
           DBObject queryFields = new BasicDBObject(map);  
        String json = "[";  
        if(queryFields.keySet().size()>0)  
        {  
          DBCursor cursor = db.getCollection("mycollection").find(queryFields);  
             try {  
               while(cursor.hasNext()) {  
                 json += cursor.next() + ",";  
               }  
             } finally {  
               cursor.close();  
               json = json.substring(0,json.length()-1) + "]";  
             }  
        }  
        return json;  
      }  
 }  

And that's it!

REST enabled MongoDB with the power of Mule ESB CE integration flows at hand!

MongoDB with Mule ESB Community Edition integration tips

Integrating MongoDB with Mule ESB Community Edition might sound trivial and in some ways it is with the built in cloud connector and transformer. However documentation is not flawless, the user interfaces not very intuitive and last but not least the functionality is very limited compared to what you get using the MongoDB Java libraries as an API.

To demonstrate this I'm going to use a simple use case.

Lets say you wanted to populate your MongoDB with the data  from a CSV but not by means of CSV row -> BSON document (as with the mongoimport tool) but rather a group of CSV rows -> BSON document to get more document like structures in your MongoDB collections. Remember that performance wise it is better to do operations inside of a MongoDB document than on many MongoDB operations. That said you should not use very big documents either.
After population is done you want to do some aggregations on specific fields in the documents.

Getting the data in

The problem here is that you do not want the data to be inserted row by row but grouped by and but based on the in data. The second issue is that not all fields in the CSV should be treated as Strings but might be numbers and that is very crucial when it comes to aggregating the values.
The mongoimport tool handles the second issue well , but Mule doesn't.

Two ways to go really since we cannot use the fancy Datamapper from Mule ESB Enterprise edition.

1) You could do this by using Mule ESB File endpoints listening for CSV's and then use Mule's flow controls like splitters, choice, different filters, Object to JSON.

2) The other way is to simply implement your own transformer extending the AbstractMessageTransformer and in the transformMessage method use some JSON library like Jackson or org.json and divide the CSV into JSON documents yourself.

To use an non-built in JSON library like org.json simply add it to your Maven pom as

 <dependency>  
  <groupId>org.json</groupId>  
  <artifactId>json</artifactId>  
  <version>20090211</version>  
 </dependency>  

If you use Jackson with the CSV extension you can also build a CSV schema based on a POJO that describes each fields value type and hence parse the CSV based on that. I found Jackson to be a bit overkil though for simple operations. If you use org.json and the built in CDL.toJSONArray method you will end up with the same issue with unknown types getting everyting treated as Strings as in the first Mule solution. 

You do not want to end up with documents containing:

 { "This is really a value of numbers" : "214245","This is a value of String" : "Hi there 123" }  

If you do there is a little hack you could use in both case 1) With Mule ESB's Regex filter or in case 2) With Javas String method replaceAll(). I will show you the Java version for clarity:

 String good_json_data = baad_json_data.replaceAll("(\"([0-9]+)(\\.[0-9]+)?\")+", "$2$3");  

This regular expression search out all fields in your JSON and replaces those that actually are numbers with their numeric representation (without quotes).

Now you can pass your JSON data as payload for the next Mule flow item either directly to a MongoDB connector element with insert or your own MongoDB Java component.

Your flow might look something like this depending on which way you choose to attack the issue.




Using the latest Mongo Java driver

If your plan is (like in our use case) to use aggregate functionality on your BSON documents and you are not very fond of writing tons of map / reduce function code into tiny text fields in Mule's MongoDB connectors user interface you will have to come up with something else than Mule's built in MongoDB support. Also be aware that MongoDB's excellent aggregation framework is not even included in the version of the Java driver that Mule uses.

Again you need to edit your Maven pom.

 <dependency>  
  <groupId>org.mongodb</groupId>  
  <artifactId>mongo-java-driver</artifactId>  
  <version>2.11.2</version>  
 </dependency>  

Also after this you need to remove any MongoDB references you might have in the namespaces declarations in your xml configuration our you will end up with very strange error messages because of conflicting libraries.

Now we have access to the sweet com.mongodb.MongoClient (by the way handles connection pooling for you) object which lets us write our own Java components to connect and query MongoDB.

For example the MongoDB insert component from the flow image above simply takes the JSON payload from the element before and stores it straight into MongoDB, sweet.

 package se.redpill.mulecomponents.mongo;  
 import org.bson.types.ObjectId;  
 import org.mule.api.MuleEventContext;  
 import org.mule.api.lifecycle.Callable;  
 import com.mongodb.DBObject;  
 import com.mongodb.WriteConcern;  
 import com.mongodb.util.JSON;  
 public class InsertMongoComponent extends AbstractMongoComponent implements Callable{  
      @Override  
      public Object onCall(MuleEventContext eventContext) throws Exception {  
           Object payload = eventContext.getMessage().getPayload();  
           DBObject thedata = (DBObject) JSON.parse((String) payload);  
           db.getCollection("mycollection").insert(thedata, WriteConcern.SAFE);  
        ObjectId id = (ObjectId) thedata.get("_id");  
     if (id == null) return null;  
     return id.toStringMongod();  
      }  
 }  

The AbstractMongoComponent extended simply holds the db connection details and are instantiated from the xml configuration like:

  <spring:beans>  
        <spring:bean id="mongoDb" class="com.mongodb.MongoClient" scope="singleton">  
          <spring:constructor-arg index="0" type="java.lang.String" value="localhost"/>       
          <spring:constructor-arg index="1" type="int" value="27017"/>  
        </spring:bean>  
        <spring:bean id="aggregateMongo" class="se.redpill.mulecomponents.mongo.AggregateMongoComponent" scope="singleton" init-method="init">  
             <spring:property name="mongoDb" ref="mongoDb"/>  
             <spring:property name="dbName" value="mydatabase"/>  
        </spring:bean>  
        <spring:bean id="insertMongo" class="se.redpill.mulecomponents.mongo.InsertMongoComponent" scope="singleton" init-method="init">  
             <spring:property name="mongoDb" ref="mongoDb"/>  
             <spring:property name="dbName" value="mydatabase"/>  
        </spring:bean>  
   </spring:beans>  

And when using the instance:

  <foreach doc:name="For Each">  
          <component doc:name="MongoDB insert">  
                  <spring-object bean="insertMongo"/>  
                </component>  
     </foreach>  

Sweet!

Getting aggregated data out

Now how to get data out? Well simply enough you just write a QueryMongoComponent in the same style as the InsertMongoComponent above and reference it from the xml in the same way.

But in our case we wanted to use MongoDB's aggregation framework. Same thing. Simply write a component that uses the latest Java drivers aggregation functionality. Like in this example:
 package se.redpill.mulecomponents.mongo;  
 import java.util.Map;  
 import org.mule.api.MuleEventContext;  
 import org.mule.api.annotations.param.OutboundHeaders;  
 import org.mule.api.annotations.param.Payload;  
 import org.mule.api.lifecycle.Callable;  
 import com.mongodb.AggregationOutput;  
 import com.mongodb.BasicDBObject;  
 import com.mongodb.CommandResult;  
 import com.mongodb.DBObject;  
 public class AggregateMongoComponent extends AbstractMongoComponent implements Callable{  
      public String aggregateObject() {  
        // create our pipeline operations,   
        // build the $projection operation  
        DBObject fields = new BasicDBObject("OurCoolDocument.SpecificValue", 1);  
        fields.put("OurCoolDocument.Unit", 1);  
        fields.put("OurCoolDocument.Type",1);   
        fields.put("_id", 0);  
        DBObject project = new BasicDBObject("$project", fields );  
        // now unwind  
        DBObject unwind = new BasicDBObject("$unwind", "$OurCoolDocument");  
        // Now the $group operation  
        DBObject groupFields = new BasicDBObject( "_id", "$OurCoolDocument.Unit");  
        groupFields.put("average", new BasicDBObject( "$avg", "$OurCoolDocument.SpecificValue"));  
        DBObject group = new BasicDBObject("$group", groupFields);  
        // Now the $match operation  
        DBObject match = new BasicDBObject("$match", new BasicDBObject("average", new BasicDBObject( "$gte", 93)) );  
        // run aggregation  
        AggregationOutput output = db.getCollection("mycollection").aggregate( project,unwind,group, match);  
        return output.toString();  
       }  
      @Override  
      public Object onCall(MuleEventContext eventContext) throws Exception {  
           return aggregateObject();  
      }  
 }  

You can of course  pass all the aggregation parameters on the Mule message to the component from the element before.

The results might look like this:

 {  
 serverUsed: "localhost/127.0.0.1:27017",  
 result: [  
 {  
 _id: "My special",  
 average: 93.55555555555556  
 },  
 {  
 _id: "Another special one",  
 average: 96.66666666666667  
 },  
 {  
 _id: "Whats so special?",  
 average: 93.77777777777777  
 },  
 {  
 _id: "Special for you my friend!",  
 average: 96.88888888888889  
 }  
 ],  
 ok: 1  
 }  

Best of luck!

tisdag 18 juni 2013

Integrating Mule ESB Community Edition with IBM WebSphere DataPower SOA or other SOAP services where Signing of Binary Security token is nessesary

Integrating Mule ESB with IBM WebSphere DataPower WS-Security based SOAP services may become very frustrating. Especially if you are trying to do it without WS-Security policies.

The real hurdle to get pass is that some of these product's SOAP endpoints, requires the Binary security token itself to be X509 signed and referenced within the  <ds:SignedInfo> 
along with the message body as part of the SOAP message - not easily accomplished within Mule.

As stated above this can be accomplished using WS-Security profiles and have the <sp:ProtectTokens/> enabled withing your profile.
The profile can be a part of your service WSDL or added on a local copy that you edit yourself, just for the client but in my example I will demonstrate how to do it when you do not have the possibility to use profiles, WSDL's etc.

Enabling WS-Security is by Mulesoft in fact marked as an Enterprise only feature of Mule ESB, however my example demonstrates how to do this with Community Edition.

By the time of this writing ( Mule ESB 3.4 ) it is not even possible to do it by chaining two
org.apache.cxf.ws.security.wss4j.WSS4JOutInterceptor
after each other first signing the body and getting the Binary security token and then in a second sign the Binary security token itself.
I could not even get that done in Apache CXF through Spring beans configuration in Mule.

I have tried lot's of other ways to get it done as well, but let's instead demonstrate a way ahead that actually works.

Pass the SOAP!


First lets create a Web service that represents the server so that we have something standalone to test against.  I have used Mule studios template as a starting point to setup this example.



 <?xml version="1.0" encoding="UTF-8"?>  
 <mule xmlns:http="http://www.mulesoft.org/schema/mule/http"  
      xmlns:cxf="http://www.mulesoft.org/schema/mule/cxf"  
      xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:spring="http://www.springframework.org/schema/beans" version="CE-3.4.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="  
 http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd   
 http://www.mulesoft.org/schema/mule/cxf http://www.mulesoft.org/schema/mule/cxf/current/mule-cxf.xsd http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd  
 http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd">  
   <flow name="X509SignedAndEncryptedServiceFlow" doc:name="X509SignedAndEncryptedServiceFlow">  
     <http:inbound-endpoint address="http://localhost:63081/services/x509" exchange-pattern="request-response" doc:name="HTTP Inbound Endpoint"/>  
     <cxf:jaxws-service serviceClass="com.mulesoft.mule.example.security.Greeter" doc:name="Secure UsernameToken Signed service">  
       <cxf:ws-security>  
         <cxf:ws-config>  
           <cxf:property key="action" value="UsernameToken Signature Timestamp"/>  
           <!-- <cxf:property key="action" value="UsernameToken Signature Timestamp Encrypt"/> -->  
           <cxf:property key="signaturePropFile" value="wssecurity_server.properties"/>  
           <!-- <cxf:property key="decryptionPropFile" value="wssecurity_server.properties"/> -->  
           <cxf:property key="passwordCallbackClass" value="com.mulesoft.mule.example.security.PasswordCallback"/>  
         </cxf:ws-config>  
       </cxf:ws-security>  
     </cxf:jaxws-service>  
     <component class="com.mulesoft.mule.example.security.GreeterService" doc:name="Greeter Service"/>  
   </flow>  
 </mule>  

Create a Greeter service that simply says Hello.

 package com.mulesoft.mule.example.security;  
 /*  
  * $Id: GreeterService.java 18408 2010-12-09 19:44:58Z travis.carlson $  
  * --------------------------------------------------------------------------------------  
  *  
  * (c) 2003-2010 MuleSoft, Inc. This software is protected under international copyright  
  * law. All use of this software is subject to MuleSoft's Master Subscription Agreement  
  * (or other master license agreement) separately entered into in writing between you and  
  * MuleSoft. If such an agreement is not in place, you may not use the software.  
  */  
 public class GreeterService implements Greeter  
 {  
   public String greet(String name)  
   {  
     return "Hello " + name;  
   }  
 }  

Expose it as a Webservice.

 /*  
  * $Id: Greeter.java 18408 2010-12-09 19:44:58Z travis.carlson $  
  * --------------------------------------------------------------------------------------  
  *  
  * (c) 2003-2010 MuleSoft, Inc. This software is protected under international copyright  
  * law. All use of this software is subject to MuleSoft's Master Subscription Agreement  
  * (or other master license agreement) separately entered into in writing between you and  
  * MuleSoft. If such an agreement is not in place, you may not use the software.  
  */  
 package com.mulesoft.mule.example.security;  
 import javax.jws.WebParam;  
 import javax.jws.WebResult;  
 import javax.jws.WebService;  
 @WebService  
 public interface Greeter  
 {  
   @WebResult(name="name")  
   public String greet(@WebParam(name="name") String name);  
 }  

We also need a Password callback handler for our service.

 package com.mulesoft.mule.example.security;  
 /*  
  * $Id: PasswordCallback.java 18408 2010-12-09 19:44:58Z travis.carlson $  
  * --------------------------------------------------------------------------------------  
  *  
  * (c) 2003-2010 MuleSoft, Inc. This software is protected under international copyright  
  * law. All use of this software is subject to MuleSoft's Master Subscription Agreement  
  * (or other master license agreement) separately entered into in writing between you and  
  * MuleSoft. If such an agreement is not in place, you may not use the software.  
  */  
 import java.io.IOException;  
 import javax.security.auth.callback.Callback;  
 import javax.security.auth.callback.CallbackHandler;  
 import javax.security.auth.callback.UnsupportedCallbackException;  
 import org.apache.ws.security.WSPasswordCallback;  
 public class PasswordCallback implements CallbackHandler  
 {  
   public void handle(Callback[] callbacks) throws IOException, UnsupportedCallbackException  
   {  
     WSPasswordCallback pc = (WSPasswordCallback) callbacks[0];  
     if (pc.getIdentifier().equals("myclientkey"))  
     {  
       pc.setPassword("ckpass");  
     }  
     else if (pc.getIdentifier().equals("myservicekey"))  
     {  
       pc.setPassword("skpass");  
     }  
   }  
 }  

We will create a self signed X509 certificate and private / public keypair to test with.

First using java's keytool setup the keystores and keys:

 keytool -genkey -keyalg RSA -sigalg SHA1withRSA -validity 730 -alias myservicekey -keypass skpass -storepass sspass -keystore serviceKeystore.jks -dname "cn=localhost"  
 keytool -genkey -keyalg RSA -sigalg SHA1withRSA -validity 730 -alias myclientkey -keypass ckpass -storepass cspass -keystore clientKeystore.jks -dname "cn=clientuser"  

Note that these keys are self-signed and should never be used in production.

Setup two-way trust between the SOAP client and the web service. That means load each of the public keys into the keystore of the other.

 keytool -export -rfc -keystore clientKeystore.jks -storepass cspass -alias myclientkey -file MyClient.cer  
 keytool -import -trustcacerts -keystore serviceKeystore.jks -storepass sspass -alias myclientkey -file MyClient.cer -noprompt  
 keytool -export -rfc -keystore serviceKeystore.jks -storepass sspass -alias myservicekey -file MyService.cer  
 keytool -import -trustcacerts -keystore clientKeystore.jks -storepass cspass -alias myservicekey -file MyService.cer -noprompt  

It is now safe to delete the temporary .cer files.

Now lets setup two WS-Security configuration files in your resources folder one for the client called wssecurity_client.properties:

 org.apache.ws.security.crypto.provider=org.apache.ws.security.components.crypto.Merlin  
 org.apache.ws.security.crypto.merlin.keystore.type=jks  
 org.apache.ws.security.crypto.merlin.keystore.password=cspass  
 org.apache.ws.security.crypto.merlin.keystore.alias=myclientkey  
 org.apache.ws.security.crypto.merlin.file=clientKeystore.j  

and one for the server called wssecurity_server.properties

 org.apache.ws.security.crypto.provider=org.apache.ws.security.components.crypto.Merlin  
 org.apache.ws.security.crypto.merlin.keystore.type=jks  
 org.apache.ws.security.crypto.merlin.keystore.password=sspass  
 org.apache.ws.security.crypto.merlin.file=serviceKeystore.jks  
 org.apache.ws.security.crypto.m  

Now create a separate .mflow file for our client.



 <?xml version="1.0" encoding="UTF-8"?>  
 <mule xmlns:http="http://www.mulesoft.org/schema/mule/http"  
      xmlns:cxf="http://www.mulesoft.org/schema/mule/cxf"  
      xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:spring="http://www.springframework.org/schema/beans" version="CE-3.4.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd  
 http://www.mulesoft.org/schema/mule/cxf http://www.mulesoft.org/schema/mule/cxf/current/mule-cxf.xsd  
 http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd  
 http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd">  
 <flow name="SecurityClients" doc:name="SecurityClients">  
     <http:inbound-endpoint exchange-pattern="request-response" host="localhost" port="63080" path="client" doc:name="HTTP Inbound Endpoint"/>  
     <set-payload value="#[message.inboundProperties['http.query.params']['name']]" doc:name="Set payload with 'name' query param"/>  
     <set-variable variableName="clientType" value="#[message.inboundProperties['http.query.params']['clientType']]" doc:name="Set clientType"/>  
     <choice doc:name="Choice">  
       <when expression="#[clientType == 'x']">  
           <flow-ref name="X509Flow" doc:name="Invoke x509 Encrypted sub-flow"/>  
       </when>  
       <otherwise>  
           <set-payload value="Client type is not supported" doc:name="Client type is not supported"/>  
       </otherwise>  
     </choice>  
     <set-property propertyName="Content-Type" value="text/plain" doc:name="Set response Content-Type"/>  
     <catch-exception-strategy doc:name="Catch Exception Strategy">  
          <set-payload value="There has been an Error processing the request" doc:name="Set Payload"/>  
          <set-property propertyName="Content-Type" value="text/plain" doc:name="Set response Content-Type"/>  
     </catch-exception-strategy>  
   </flow>  
   <sub-flow name="unsecure" doc:name="unsecure">  
     <cxf:jaxws-client operation="greet" serviceClass="com.mulesoft.mule.example.security.Greeter" doc:name="Unsecure SOAP client" doc:description="Unsecure SOAP client"/>  
     <http:outbound-endpoint exchange-pattern="request-response" host="localhost" port="63081" path="services/unsecure" doc:name="Invoke unsecure Web Service"/>  
   </sub-flow>  
   <sub-flow name="X509Flow" doc:name="X509Flow">  
 --------------------------------  
The exiting stuff
 --------------------------------  
     <http:outbound-endpoint exchange-pattern="request-response" host="localhost" port="63081" path="services/x509" doc:name="Invoke usernameToken Encrypted Web Service"/>  
   </sub-flow>  
 </mule>  




The exiting stuff. Well that is in this case the stuff related to our jaxws client that is about to call our Greeter service.



We make use of cxf:outInterceptor defining three interceptors in a chain like this.

     <cxf:jaxws-client operation="greet" serviceClass="com.mulesoft.mule.example.security.Greeter" doc:name="X509 client">  
             <cxf:outInterceptors>  
             <spring:bean class="org.apache.cxf.ws.security.wss4j.WSS4JOutInterceptor">  
                <spring:constructor-arg>  
                     <spring:map>  
                        <spring:entry key="action" value="UsernameToken"/>  
                        <spring:entry key="user" value="myclientkey"/>   
                        <spring:entry key="passwordCallbackClass" value="com.mulesoft.mule.example.security.PasswordCallback"/>  
                     </spring:map>  
                 </spring:constructor-arg>  
              </spring:bean>  
              <spring:bean class="org.apache.cxf.binding.soap.saaj.SAAJOutInterceptor"/>   
              <spring:bean class="se.redpill.SigningHack">  
              </spring:bean>   
              <spring:bean class="org.apache.cxf.interceptor.LoggingOutInterceptor" id="LoggingOut">  
                  <spring:property name="prettyLogging" value="true" />  
             </spring:bean>  
         </cxf:outInterceptors>    
     </cxf:jaxws-client>  

The first one is simply a WSS4JOutInterceptor  that has the standard Apache CXF action of UsernameToken specified as expected by our Greeter service. The second is a standard SAAJOutInterceptor which we will need to set up the outgoing chain to build a SAAJ tree instead of writing directly to the output stream. The last interceptor is for logging the SOAP that is to be sent to console and the one that is interesting and is doing the magic is se.redpill.SiningHack


 package se.redpill;  
 import java.io.InputStream;  
 import java.security.KeyStore;  
 import java.security.cert.X509Certificate;  
 import java.util.ArrayList;  
 import java.util.List;  
 import javax.xml.soap.MessageFactory;  
 import javax.xml.soap.SOAPEnvelope;  
 import javax.xml.soap.SOAPMessage;  
 import javax.xml.transform.dom.DOMSource;  
 import org.apache.cxf.binding.soap.SoapMessage;  
 import org.apache.cxf.binding.soap.interceptor.AbstractSoapInterceptor;  
 import org.apache.cxf.binding.soap.saaj.SAAJOutInterceptor;  
 import org.apache.cxf.interceptor.Fault;  
 import org.apache.cxf.phase.Phase;  
 import org.apache.ws.security.WSConstants;  
 import org.apache.ws.security.WSEncryptionPart;  
 import org.apache.ws.security.WSSConfig;  
 import org.apache.ws.security.components.crypto.Crypto;  
 import org.apache.ws.security.components.crypto.CryptoType;  
 import org.apache.ws.security.components.crypto.Merlin;  
 import org.apache.ws.security.message.WSSecHeader;  
 import org.apache.ws.security.message.WSSecSignature;  
 import org.apache.ws.security.message.WSSecTimestamp;  
 import org.apache.ws.security.message.token.X509Security;  
 import org.apache.ws.security.util.Loader;  
 import org.apache.ws.security.util.WSSecurityUtil;  
 import org.w3c.dom.Document;  
 /**  
  *   
  * SigningHack.java  
  * Purpose: X509 signs SOAP Body BST.  
  *  
  * @author Jon Åkerström  
  *  
  */  
 public class SigningHack extends AbstractSoapInterceptor  
 {  
      private SAAJOutInterceptor saajOut = new SAAJOutInterceptor();  
      public SigningHack() {  
           super(Phase.PRE_PROTOCOL_ENDING );    
   }  
      public SOAPMessage signSOAPEnvelope(SOAPEnvelope unsignedEnvelope) throws Exception  
       {  
         // Signs a SOAP envelope according to the  
         // WS Specification (X509 profile) and adds the signature data  
         // to the envelope.  
         Crypto crypto = new Merlin();  
         KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());  
         ClassLoader loader = Loader.getClassLoader(SigningHack.class);  
         InputStream input = Merlin.loadInputStream(loader, "clientKeystore.jks");  
         keyStore.load(input, "cspass".toCharArray());  
         ((Merlin)crypto).setKeyStore(keyStore);  
         Document doc = unsignedEnvelope.getOwnerDocument();  
         WSSecHeader secHeader = new WSSecHeader();  
         secHeader.insertSecurityHeader(doc);  
         // Get a certificate, convert it into a BinarySecurityToken and add it to the security header  
         CryptoType cryptoType = new CryptoType(CryptoType.TYPE.ALIAS);  
         cryptoType.setAlias("myclientkey");  
         X509Certificate[] certs = crypto.getX509Certificates(cryptoType);  
         // Add the signature  
         X509Security bst = new X509Security(doc);  
         String certUri = WSSConfig.getNewInstance().getIdAllocator().createSecureId("X509-", certs[0]);  
         bst.setX509Certificate(certs[0]);  
         bst.setID(certUri);  
         WSSecurityUtil.prependChildElement(secHeader.getSecurityHeader(), bst.getElement());  
         WSSecSignature sign = new WSSecSignature();  
         sign.setUserInfo("myclientkey", "ckpass");  
         sign.setSignatureAlgorithm(WSConstants.RSA_SHA1);  
         sign.setKeyIdentifierType(WSConstants.CUSTOM_SYMM_SIGNING);  
         sign.setX509Certificate(certs[0]);  
         List<WSEncryptionPart> parts = new ArrayList<WSEncryptionPart>();  
         // Add SOAP Body  
         String soapNamespace = WSSecurityUtil.getSOAPNamespace(doc.getDocumentElement());  
         WSEncryptionPart encP =new WSEncryptionPart(WSConstants.ELEM_BODY, soapNamespace, "Content");  
         parts.add(encP);  
         // Add BST  
         encP = new WSEncryptionPart(WSConstants.BINARY_TOKEN_LN, WSConstants.WSSE_NS, "Element");  
         encP.setElement(bst.getElement());  
         parts.add(encP);  
         sign.setParts(parts);  
         WSSecTimestamp timestamp = new WSSecTimestamp();  
         timestamp.setTimeToLive(600);  
         timestamp.build(doc, secHeader);  
         parts.add(new WSEncryptionPart(timestamp.getId()));  
         sign.setCustomTokenId(bst.getID());  
         sign.setCustomTokenValueType(bst.getValueType());  
         sign.prepare(doc, crypto, secHeader);  
         List<javax.xml.crypto.dsig.Reference> referenceList =   
         sign.addReferencesToSign(parts, secHeader);  
         sign.computeSignature(referenceList, false, null);  
         Document signedDoc = doc;  
         DOMSource src = new DOMSource(signedDoc);  
         MessageFactory mf = MessageFactory.newInstance();  
            SOAPMessage soapMsg = mf.createMessage();  
            soapMsg.getSOAPPart().setContent(src);  
         return soapMsg;  
       }  
      @Override  
   public void handleMessage(SoapMessage message) throws Fault {  
     SOAPMessage sm = message.getContent(SOAPMessage.class);  
     try   
     {  
       sm.writeTo(System.out);  
     } catch (Exception e)   
     {  
          e.printStackTrace();  
     }  
           try   
           {  
                SOAPEnvelope se = sm.getSOAPPart().getEnvelope();  
                message.setContent(SOAPMessage.class, signSOAPEnvelope(se));  
           } catch (Exception e1) {  
                // TODO Auto-generated catch block  
                e1.printStackTrace();  
           }  
      }  
 }  

Note the super(Phase.PRE_PROTOCOL_ENDING );  
specifying in which CXF interceptor Phase that this SoapInterceptor should insert itself into.
It is very important that you choose the correct phase.

Next the handleMessage method receives the SOAP and passes it to the signSOAPEnvelope 
method for processing by WSS4J statements. Note that in this example i only add the body and the Binary security token to the computing signature and its references. You could of course add whatever elements you like , the Timestamp, UsernameToken etc.


You can test the whole thing by open up your browser and call something like:
http://localhost:63080/client?clientType=x&name=John%20Doe

VOILA!

The SOAP produced should look something like this. Especially notice the
"ds:Reference URI="#X509-"
data
within the <ds:SignedInfo> field that is the cause for this whole demonstration.

 <?xml version="1.0" encoding="UTF-8"?><soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">  
  <soap:Header>  
   <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">  
    <wsu:Timestamp wsu:Id="TS-2">  
     <wsu:Created>2013-06-17T18:14:06.212Z</wsu:Created>  
     <wsu:Expires>2013-06-17T18:24:06.212Z</wsu:Expires>  
    </wsu:Timestamp>  
    <wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="X509-62121A83C77862F71F13714928462051">MIIBoTCCAQqgAwIBAgIEUb8P/zANBgkqhkiG9w0BAQUFADAVMRMwEQYDVQQDEwpjbGllbnR1c2VyMB4XDTEzMDYxNzEzMzI0N1oXDTE1MDYxNzEzMzI0N1owFTETMBEGA1UEAxMKY2xpZW50dXNlcjCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAgHs1KqFTZGbNM/PoqcXk2GWzw9AAsYRhzCf3GOxNUzMgGvKpRZjUYOlwIozcoJ1optb/g/9Ku6NSKhq5EWOBcsTon+C99dbwkgTIugPrbcappAftrqBSU7/fSEyaLhagLt/NVwyyQ++Ax4HU8BFu84TRkpoqyPgjII7hV3FirKMCAwEAATANBgkqhkiG9w0BAQUFAAOBgQBgCqrNwGAlcPUfjx4FcYF/5bBdgbCEkCC2EEJv3Sm8AIAyHDJN0bxQf6qkT8qVlz2g0EWtNBN7KBUUO1PhDXBpuii90XRNtv5C1dF2V+c1uFhIGloSqsWW5/Ia8k3inv0edFPwqp6oXjNjLWWI03msRr5VaNKrASXsWJ7cBIWg5A==</wsse:BinarySecurityToken>  
    <wsse:UsernameToken wsu:Id="UsernameToken-1">  
     <wsse:Username>myclientkey</wsse:Username>  
     <wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordDigest">eGcv38RgFW+oum+97nHwSkm5UKs=</wsse:Password>  
     <wsse:Nonce EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary">S2AMgJhwZjpQiTnNcW1mxw==</wsse:Nonce>  
     <wsu:Created>2013-06-17T18:14:06.128Z</wsu:Created>  
    </wsse:UsernameToken>  
    <ds:Signature xmlns:ds="http://www.w3.org/2000/09/xmldsig#" Id="SIG-4">  
     <ds:SignedInfo>  
      <ds:CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">  
       <ec:InclusiveNamespaces xmlns:ec="http://www.w3.org/2001/10/xml-exc-c14n#" PrefixList="soap"/>  
      </ds:CanonicalizationMethod>  
      <ds:SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/>  
      <ds:Reference URI="#id-3">  
       <ds:Transforms>  
        <ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">  
         <ec:InclusiveNamespaces xmlns:ec="http://www.w3.org/2001/10/xml-exc-c14n#" PrefixList=""/>  
        </ds:Transform>  
       </ds:Transforms>  
       <ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/>  
       <ds:DigestValue>AWj0MyBAJSSQtq9VQKWsHLleyM0=</ds:DigestValue>  
      </ds:Reference>  
      <ds:Reference URI="#X509-62121A83C77862F71F13714928462051">  
       <ds:Transforms>  
        <ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">  
         <ec:InclusiveNamespaces xmlns:ec="http://www.w3.org/2001/10/xml-exc-c14n#" PrefixList="soap"/>  
        </ds:Transform>  
       </ds:Transforms>  
       <ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/>  
       <ds:DigestValue>VQtibH91UIJiSb6dU0UEUsOu9gU=</ds:DigestValue>  
      </ds:Reference>  
      <ds:Reference URI="#TS-2">  
       <ds:Transforms>  
        <ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">  
         <ec:InclusiveNamespaces xmlns:ec="http://www.w3.org/2001/10/xml-exc-c14n#" PrefixList="wsse soap"/>  
        </ds:Transform>  
       </ds:Transforms>  
       <ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/>  
       <ds:DigestValue>aCvHMOWvJRmA99t0o2SrDznkZmI=</ds:DigestValue>  
      </ds:Reference>  
     </ds:SignedInfo>  
     <ds:SignatureValue>P88VKXSErnMoMAMp+xaPDvlPkr+kb6U7Xh8GW0Ctz7HJlH94A3YCrDo9wq1b20jz2Ckr4neSxDQJ  
 QE4unfhu6UG8jnPIbROSv/Eu9NX2MWD/Qtevf+j+AHsRGyaDE6WpHxcAeOnSDPs2KUOiRzLLWaOP  
 ClDsIcLVe/ca2mahXKM=</ds:SignatureValue>  
     <ds:KeyInfo Id="KI-62121A83C77862F71F13714928462132">  
      <wsse:SecurityTokenReference wsu:Id="STR-62121A83C77862F71F13714928462153">  
       <wsse:Reference URI="#X509-62121A83C77862F71F13714928462051" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>  
      </wsse:SecurityTokenReference>  
     </ds:KeyInfo>  
    </ds:Signature>  
   </wsse:Security>  
  </soap:Header>  
  <soap:Body xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" wsu:Id="id-3">  
   <ns2:greet xmlns:ns2="http://security.example.mule.mulesoft.com/">  
    <name>John</name>  
   </ns2:greet>  
  </soap:Body>  
 </soap:Envelope>  

Note that validation of the signed data on the server side is beyond the scope of this article which is focusing on the client side issue of signing the binary security token.

Enterprise caching with Mule ESB Community Edition

I want to share with you a way to do generic enterprise type caching with Mule ESB community edition.

In Mule ESB you have the possibility to use the Cache Scope with various strategies and options to enable caching of the payload for any element in your integration flow. This feature is however only available in the Enterprise or CloudHub version of Mule.

Inspired by an integration blueprint I found in Mule In Action by David Dossot and John D'Emic, I managed to setup a solution that uses the excellent Open Source edition of  EHCache to achieve very similar behaviors as the one found in Mule's Enterprise only features.

Just follow these easy steps and remember Cache is King!

If you are using Maven you need to setup a dependency for EHCache

 <dependency>  
   <groupId>net.sf.ehcache</groupId>  
   <artifactId>ehcache</artifactId>  
   <version>2.7.0</version>  
   <scope>provided</scope>  
 </dependency>  

Otherwise you will need to attach the EHCache .jar depencies in your project.

To use EHCache in our integration flows we need to set it up as a Spring bean configuration in our XML like this. Note that by doing this we expose every config available on EHCache directly to our Mule configuration. We can even setup several different caches with different settings regarding disk persistence , memory eviction policy's time to live etc.

 <spring:beans>  
     <spring:bean id="MyCache" class="org.springframework.cache.ehcache.EhCacheFactoryBean">  
       <spring:property name="cacheName" value="MyCache"/>  
       <spring:property name="diskPersistent" value="false"/>  
       <spring:property name="overflowToDisk" value="false"/>  
       <spring:property name="maxElementsInMemory" value="100000"/>  
       <spring:property name="eternal" value="false"/>  
       <spring:property name="memoryStoreEvictionPolicy" value="LRU"/>  
       <spring:property name="timeToIdle" value="60"/>  
       <spring:property name="timeToLive" value="60"/>  
     </spring:bean>  
 </spring:beans>  


Now in your Mule configuration flow's XML you will need to add the Custom interceptor component and reference it to a cache configuration (Note that you do not have any graphical component in the component palette of Mule Studio for it.

 <custom-interceptor doc:name="PayloadCache"    class="se.redpill.mulecomponents.cache.PayloadCache">  
          <spring:property name="cache" ref="MyCache"/>  
 </custom-interceptor>  

Since this is an interceptor rather than a scope , you should think of it as something that happens before the flow element you put it in front of (Note that you can write this component as an EnvelopeInterceptor as well and then getting "before" and "after" methods with a surrounding behavior instead).

This particular example just uses the payload from the element in the flow before where you put the component as a key in the cache and it uses the result from the element after as cached value.
So if you send in a payload that does not already exist as key in the cache the execution of the flow execution will go on and a new value will be set in the cache before it is returned, otherwise the cached value for the payload key is immediately returned.

Of course you could modify this behavior as you like and for example provide more <spring:property/> elements to your Custom interceptor and handle them properly for example with behaviors to be able to select parts of the payload as key or parts of the returning payload as value using MEL.

The Custom interceptor is implemented simply by making a class implement the org.mule.api.interceptor.Interceptor interface.

 package se.redpill.mulecomponents.cache;  
 import net.sf.ehcache.Ehcache;  
 import net.sf.ehcache.Element;  
 import org.mule.DefaultMuleEvent;  
 import org.mule.DefaultMuleMessage;  
 import org.mule.api.MuleEvent;  
 import org.mule.api.MuleException;  
 import org.mule.api.MuleMessage;  
 import org.mule.api.interceptor.Interceptor;  
 import org.mule.api.processor.MessageProcessor;  
 /**  
  * A mule interceptor acting as a ehCache component.  
  * Based on the Cache interceptor blueprint from Mule In Action by David Dossot and John D'Emic,  
  *   
  */  
 public class PayloadCache implements Interceptor   
 {       
        private MessageProcessor next;  
        private Ehcache cache;  
        public void setListener(MessageProcessor listener)  
        {  
          next = listener;  
        }  
        public void setCache(final Ehcache cache)  
        {  
          this.cache = cache;  
        }  
        public MuleEvent process(MuleEvent event) throws MuleException  
        {  
          final MuleMessage currentMessage = event.getMessage();  
          final Object key = currentMessage.getPayload();  
          final Element cachedElement = cache.get(key);  
          if (cachedElement != null)  
          {  
            return new DefaultMuleEvent(new DefaultMuleMessage(cachedElement.getObjectValue(),  
              currentMessage, event.getMuleContext()), event);  
          }  
          final MuleEvent result = next.process(event);  
          cache.put(new Element(key, result.getMessage().getPayload()));  
          return result;  
        }  
 }  

That's it! You know have a very flexible and professional Open Source caching solution for your Mule ESB CE.