Flink-ESB Tutorials
Use case: create OSGI service to encrypt/decrypt content of XML message
Short description: Create 2 OSGI services to encrypt and to decrypt content of XML message. Should be possible to define during OSGI service call, what nodes of message should be encrypted/decrypted.
AES symmetric encryption algorithm should be used to encrypt/decrypt data.
Decrypted data must have exactly the same content as XML before encryption.
Example of usage: See tutorial on using Flink-ESB Message Gateway to store data on remote system encrypted, and to decrypt it on the fly while reading from the remote system.
Step 1: creating a new Flink-ESB apllication in Flink-ESB Editor
Open Flink-ESB Editor and select "File -> New -> Project" in the main menu. Double click "Flink-ESB" folder to expand it, select "Create New Flink-ESB Project" and press "Next"
This opens a "New Flink-ESB Project" wizard. Enter data as on picture below: "Project Name" : crypt-utils, unselect "Use default location" checkbox and select a folder where a project should be created:
Click "Next" and type some name in "Provider" field, after that click "Finish". This should create a new Flink-ESB project and open a file called "project.esbbp" in the main editor pane. This is where we will create all the components of the application.
Step 2: adding "Property Placeholder" to the project
Click on "Blueprint Elements" drawer on the components palette to the right of the main editor pain. Click on "Property-Placeholder" component, now move mouse pointer over editor canvas and click with the left mouse to put "Property-Placeholder" component onto canvas. Provide properties of the componebt as on picture below:
Click "Save Changes" to save Property-Placeholder's properties.
Step 3: creating keystore containing symmetric key used to encrypt/decrypt data
In order to encrypt/decrypt data a symmetric key will be used. This key will be stored in the keystore of type JCEKS (standard java keystore of type JKS cannot store symmetric keys).
To create a keystore we will use "keytool" command, which is part of java development kit. Open command prompt and type following command:
keytool -genseckey -alias aes128 -keyalg AES -keysize 128 -storepass changeit -keypass changeit -storetype JCEKS -keystore d:\project\esb\ks2.jck
-genseckey parameter tells "keytool" command to create a secret key.
-alias aes128 tells "keytool" to store a secret key with "aes128" alias. We will need this to pick the right entry from the keystore (it can contain multiple entries).
-keyalg AES -keysize 128 tells "keytool" to create a key for AES-128 encryption algorithm.
-storepass changeit is password needed to read entries in the keystore.
-keypass changeit is password needed to read this secret key entry from the keystore.
-storetype JCEKS tells "keytool" to create a keystore of type JCEKS.
-keystore d:\project\esb\ks2.jck is absolute path to the keystore file (adjust it to fit your environment).
You can create multiple secret keys to be used with different encryption algorithms, and store them all in the same keystore. Just make sure to specify the same filename for the keystore, and unique aliases for key entries.
To check the content of JCEKS keystore type this command:
keytool -list -v -storepass changeit -storetype JCEKS -keystore d:\project\esb\ks2.jck
Step 4: adding "Symmetric Cryptor" to the project
Select "Symmetric Cryptor" component within "Crypt Adapter" palette, and drop it onto editor canvas near "Property-Placeholder" component. Enter properties as on picture below:
Click "Save Changes". Now select "Secret Key" tab on properties window and enter parameters as on picture below:
!!! Make sure you use the same keystore password, key password, keystore filename and alias name as those used with "keytool" command to create a secret key entry !!!
Now click on "Obf" button besides "Keystore-Password" and "Key-Password" properties to obfuscate their values:
Click "Save Changes".
Now if we don't want to have keystore path, passwords and alias hard coded, we should make those parameters configurable.
To do that click on "Property-Placeholder" component within Flink-ESB Editor, select "Project Variables" tab in the properties window. You should see an empty table:
Click with the right mouse button somewhere in the table and select "Add new entry" from the context menu. A new row appears in the table, containing property.name as "Property Name", and property.value as "Default Value".
Click on "property.name" string to make it editable, and change its value to keystore.path. Now click on "property.value" string and change its value to D:\project\ESB\ks2.jck (adjust it accordingly if needed).
Click with the right mouse button somewhere in the table again to create the second project variable. Name it "keystore.pass" and give it a value #!ss303s7yu7ncxb/icWDS8w== (this is obfuscated value of the "Keystore-Password" property of "Secret Key" component).
Create the third project variable. Name it "key.pass" and give it a value #!ss303s7yu7ncxb/icWDS8w== (this is obfuscated value of the "Key-Password" property of "Symmetric Cryptor" component).
Create the fourth project variable. Name it "key.alias" and give it a value aes128. Now project variables should look like on picture below:
Now select "Symmetric Cryptor" component within Flink-ESB Editor and click on "Secret Key" tab in the properties window. Change properties as on picture below:
!!! Make sure strings enclosed between "${" and "}" match exactly property names of the respective project variables !!!
Step 5: creating route to generate "initialization vector" (IV)
This route should be called only if IV file does not exist yet, or if you want to replace content of IV file.
Initialization vector (IV) is used together with the secret key. It adds additional level of protection to encryption/decryption. To decrypt encrypted data same combination of secret key and IV must be used, as during data encryption.
Let us create a route that will generate a random 128 bit value (IV for AES encryption algorithm must be 128 bits long).
Select "Route" component within "ESB General" palette, and drop it onto editor canvas. Type generate-id is "ID" in the properties window:
Click "Save Changes". Now add ""Subflow" route node from "ESB General" palette to the route.
Add "Transformer" route node from "Mediation" palette to the route, then add "File Writer" route node from "File Adapter" palette to the route. Resize the route if needed to make all route nodes good visible:
Connect route nodes as on the pictute below:
Select "Transformer" within the route and enter its "Custom-Expression" property as random(16):
Expression random(16) generates a random byte array with length 16 bytes (16*8=128 bits, as required by AES encryption algorithm).
Click "Save Changes". Now select "File Writer" within the route and enter its properties as on picture below:
Click "Save Changes". As you can see, we define "Dynamic-Filename" property as project variable iv.file to make it configurable. So let's add this project variable to the "Properties-Placeholder" component. Select it, click on "Project Variables" tab in the properties window. Now add a new variable with name "iv.file" and value d:/project/ESB/iv2 (adjust the path if needed):
Save project file in the editor. Now start the project by selecting "Run -> Run Project in Test Mode" from the main menu. If all is good a green frame appears around project canvas in the editor, meaning this project is now started:
Go to Flink-ESB Admin Console in the browser using this url: https://192.168.0.199:8443/console/start (replace IP address with your local IP). Login into the console. Default username/password is admin/admin.
Click on "Application Overview" to expand it, then click on triangle on the left side of instance name. Now you should see an application with name "crypt-utils". Click on the small triangle to expand components of the project. Click on "Routes" to expand it. Now you should see a route with name "generate-iv". Click on it to see its details on the right side of Admin Console:
Click on "Transformer" component within a route in the browser. You should see its properties as they were set in the editor:
Now click on "File Writer" component and check its properties. As you can see, Flink-ESB Admin Console shows all the properties as Flink-ESB Editor does, you just can't edit them here:
Step 6: calling "generate-iv" route to generate "initialization vector" (IV)
Click on the route in the browser to select it. Click on "Call Route" tab. Now press "Call" button. You should get a response in light green colour (this colour means: route execution was successfull):
Now make sure that the file with name d:/project/ESB/iv2 was created on your computer, and that this file has exactly 16 bytes size.
Step 7: creating route to read initialization vector after application start
This route should be triggered one second after application is started. It should read the content of IV file, convert the data read into byte array format and store this data as "iv" variable. Encryption and decryption routes should use this variable as IV, instead of reading it from the file system during every execution.
Add a new route to the project in Flink-ESB Editor. Enter its "ID" property as load-iv. Click "Save Changes".
Now add following components to the route from left to right: 1) "Timer" from "Mediation" palette; 2) "File Reader" from "File Adapter" palette; 3) "Variable Writer" from "ESB General" palette.
Connect components as on picture below:
Select "Timer" within the route and define its properties as on picture below:
Timer with "Trigger-Type" SINGLE_EXECUTION starts its route execution only one time after application was started. When exactly is defined by the second property. In our case this route is started 1 second after application is started.
Click "Save Changes". Select "File Reader" within the route and define its properties as on picture below:
Click "Save Changes". Select "Variable Writer" within the route, click on "Add Variable" button in properties window and enter data as on picture below:
"Variable Writer" in this route stores data in message payload ("Value"=payload) into variable with name iv.
"Scope" with value application means this variable is available to all application components as long as this application is up.
Click "Save Changes". Now select a connection link between "File Reader" and "Variable Writer". Click on "Payload Types" tab in the properties window. Enter data as on picture below:
This is needed to convert message payload to byte array format. Click "Save Changes" and save project file.
"File Reader" sets the message payload in java.io.File format. We need it as byte array, When we define payload type as byte[], it tells Flink-ESB to convert message payload into byte[].
Step 8: creating route encrypting data of XML message
This route should apply XPath expression to the message payload (which is expected to be in XML format). All XML nodes found by XPath should be encrypted using AES-128 CBC encryption algorithm. After that Base64 must be applied to encrypted bytes to be able to present them as string. Encrypted string should replace corresponding unencrypted text in XML node.
There should be a default XPath expression to encrypt all text nodes and all attribute values.
If custom XPath expression is needed, it should be passed in input message as message property with name "xpath".
Add a new route to the project in Flink-ESB Editor. Enter its "ID" property as encrypt-xml. Activate "Export as service" checkbox:
Ativate "Export as service" checkbox if you want to export this route as OSGI service. In this case it can be called like any other OSGI service from any application running on this OSGI container.
Activate "Make service remote" checkbox if you want to export this route as distributed OSGI service. In this case any application from any OSGI container in the cluster will be able to call this route.
Click "Save Changes". Now add following components to the route from left to right: 1) "Subflow" from "ESB General" palette; 2) "Variable Reader" from "ESB General" palette; 3) "Splitter Group" from "Groups" palette.
Now place following components onto "Splitter Group" from left to right: 1) "Symmetric Encrypt" from "Crypt Adapter" palette; 2) "Transformer" from "Mediation" palette.
Connect components as on picture below:
Select "Variable Reader" within the route, click on "Add Variable" button in properties window and enter data as on picture below:
What variable reader does in this route: it tries to read variable with name "iv" and sets the value of this variable as the value of message property with name "iv".
So every time this route is called, initialization vector does not need to be read from the file (relatively slow operation), but is taken from the application context.
In real production application you would probably also add check whether iv variable exists, and whether it is a byte array of length 16 bytes. If it is not the case, you would want to throw an appropriate error.
Click "Save Changes". Now select "Splitter Group" and enter its properties as on picture below:
"Splitter-Group" splits a message in some way (in our case: by applying XPath expression). "Splitter-Group" is also itself a flow container, which means, every message in the splitted collection runs through all the route nodes, that are inside of this "Splitter-Group".
property['xpath']!=null ? property['xpath'] : ${xpath.expression} as "XPath-Expression" means: check whether message property with name "xpath" exists and is not null. If it exists: use it as XPath expression to split the message, otherwise use XPath expression defined by the project variable xpath.expression.
Now let us add a new project variable xpath.expression, used by "Splitter Group". Select "Property-Placeholder" component, click on "Project Variables" tab in the properties window. Now add a new variable with name "xpath.expression" and value '//text()[string-length(normalize-space(.))>0] | //@*':
Select "Symmetric Encrypt" component within "Splitter Group" and enter its properties as on picture below:
Click "Save Changes". Now select "Transformer" component within "Splitter Group" and enter its properties as on picture below:
Click "Save Changes", save project file and restart the application in the Editor.
Step 9: testing XML encryption in Flink-ESB Admin Console
Login to Flink-ESB Admin Console, locate "encrypt-xml" route, select its "Call Route" tab.
Paste some XML message in "Payload" textbox enclosed within single quotes, for example:
'<?xml version="1.0" encoding="utf-8"?> <Root> <Customers> <Customer CustomerID="LETSS"> <CompanyName>Lets Stop N Shop</CompanyName> <ContactName>Jaime Yorres</ContactName> <ContactTitle>Owner</ContactTitle> <Phone>(415) 555-5938</Phone> <FullAddress> <Address>87 Polk St. Suite 5</Address> <City>San Francisco</City> <Region>CA</Region> <PostalCode>94117</PostalCode> <Country>USA</Country> </FullAddress> </Customer> </Customers> </Root>'
Click "Call" button. You should get XML as response, where all text nodes and attribute values are encrypted:
Message (id=14cf3548-c500-4364-a589-6ea2b47c571c) {
Headers {
timestamp: 1504284523732
id: 14cf3548-c500-4364-a589-6ea2b47c571c
messageExchangePattern: inOnly
}
Properties {
name: iv, value: (byte[]) [B@79435ef2
name: station, value: (java.lang.String) b5313b90-1250-4210-8ce9-8ca67b07d5b8
}
Payload {
(java.lang.String) <Root>
<Customers>
<Customer CustomerID="HUSGndEFhfw1tsq9vfnUQFH2ldb3d0xPzV/ZE4Qc2QGHBtvY25PR9DUA/mpvG0QD">
<CompanyName>HFzoPURrFoFP0Z5yG6ydUpBK1nwbJgYJx00LalKK4W0BMoFyllc2usiefhWl7O4zIJotiH+lb1ndadrb2O+hDQ==</CompanyName>
<ContactName>6UMvvSefKs1NfoczQRGWgJ6EOxmbXaSl1ELD23WRy13lZruUGtabnoSylHi3wAWA</ContactName>
<ContactTitle>vuySEPavvRhppoU66hFBpjdy/G6LupOmmC5NWiaOyEaY/BlDuflmc8qXuxclg9wU</ContactTitle>
<Phone>GA7VhTSuYc1gDdN0okgrKKbAtROQw4d+AMKssGqVdRKcyv3XepW0VXu0VzPoUjGg</Phone>
<FullAddress>
<Address>Sj68bSPxS2DVBFtk6STsKXjzRV+COM2G3aATE1ULPzir5d4H6MmsSAe9UT48IPuVopg6OJjTW/vFNfBtDvqfIA==</Address>
<City>we8/+iqACuj5yPKNv1uzSRFDRpCA1Mm+3MVSyWcO4+q9SjgOHMGQm5YQCASNQteo</City>
<Region>WwYh5pZBtl6L6KQODAJFAM1jJtjejuxunpLse0cC79j9oGKRixmobmrbr5lc22ou</Region>
<PostalCode>UDAYYGFjeGj7VJz7yXRXAiMNv7vMG/J7VKKme7oLl9ckn4cxBeMJh21uWLdFps60</PostalCode>
<Country>U6E5gMZPfsFlOVoENr6q5Gl5lk7LRDUQR7nX+Y83Jm7aF/MTYYizQJQeNhVCl+C5</Country>
</FullAddress>
</Customer>
</Customers>
</Root>
}
}
If you want to encrypt only certain nodes of XML, you can provide corresponding XPath expression as "xpath" message property.
Let's encrypt only "ContactName" and "Phone" elements. Paste following into "Properties" textbox:
[ 'xpath' : '//ContactName/text() | //Phone/text()' ]
You can use the same XML message as message payload. Click "Call" button. This time you should get XML message, where only "ContactName" and "Phone" nodes are encrypted:
Message (id=49c747d2-760f-4e7f-afc4-27ef6c7c8f09) {
Headers {
timestamp: 1504285059354
id: 49c747d2-760f-4e7f-afc4-27ef6c7c8f09
messageExchangePattern: inOnly
}
Properties {
name: iv, value: (byte[]) [B@79435ef2
name: station, value: (java.lang.String) b5313b90-1250-4210-8ce9-8ca67b07d5b8
name: xpath, value: (java.lang.String) //ContactName/text() | //Phone/text()
}
Payload {
(java.lang.String) <Root>
<Customers>
<Customer CustomerID="LETSS">
<CompanyName>Lets Stop N Shop</CompanyName>
<ContactName>6UMvvSefKs1NfoczQRGWgJ6EOxmbXaSl1ELD23WRy13lZruUGtabnoSylHi3wAWA</ContactName>
<ContactTitle>Owner</ContactTitle>
<Phone>GA7VhTSuYc1gDdN0okgrKKbAtROQw4d+AMKssGqVdRKcyv3XepW0VXu0VzPoUjGg</Phone>
<FullAddress>
<Address>87 Polk St. Suite 5</Address>
<City>San Francisco</City>
<Region>CA</Region>
<PostalCode>94117</PostalCode>
<Country>USA</Country>
</FullAddress>
</Customer>
</Customers>
</Root>
}
}
Step 10: creating route decrypting data of XML message
This route should apply XPath expression to the message payload (which is expected to be in XML format). Base64 decoding should be applied to all XML nodes found by XPath, after that data should be decrypted using AES-128 CBC algorithm. Decrypted data should be converted to string, and replace corresponding encrypted text in XML node.
There should be a default XPath expression to decrypt all text nodes and all attribute values.
If custom XPath expression is needed, it should be passed in input message as message property with name "xpath".
Add a new route to the project in Flink-ESB Editor. Enter its "ID" property as decrypt-xml. Activate "Export as service" checkbox:
Click "Save Changes". Now add following components to the route from left to right: 1) "Subflow" from "ESB General" palette; 2) "Variable Reader" from "ESB General" palette; 3) "Splitter Group" from "Groups" palette.
Now place following components onto "Splitter Group": 1) three "Transformers" from "Mediation" palette; 2) "Symmetric Decrypt" from "Crypt Adapter" palette; 3) "Catch Exception" from "ESB General" palette.
Connect components as on picture below:
Select "Variable Reader" within the route, click on "Add Variable" button in properties window and enter data as on picture below:
Click "Save Changes". Now select "Splitter Group" and enter its properties as on picture below:
Click "Save Changes". Select "Transformer" to the left of "Symmetric Decrypt" component and enter its properties as on picture below:
Click "Save Changes". Select "Message Properties" tab in the properties window and click "Add Property Modification" button. Enter data as on picture below:
Click "Save Changes". Select "Symmetric Decrypt" and enter its properties as on following picture:
Click "Save Changes". Select "Transformer" to the right of "Symmetric Decrypt" component and enter its properties as on picture below:
Click "Save Changes". Select "Transformer" connected to "Catch Exception" component and enter its properties as on picture below:
Step 11: testing XML decryption in Flink-ESB Admin Console
Login to Flink-ESB Admin Console, locate "decrypt-xml" route, select its "Call Route" tab.
Paste encrypted XML message (the one you received as response in step 9) in "Payload" textbox enclosed within single quotes:
'<Root> <Customers> <Customer CustomerID="HUSGndEFhfw1tsq9vfnUQFH2ldb3d0xPzV/ZE4Qc2QGHBtvY25PR9DUA/mpvG0QD"> <CompanyName>HFzoPURrFoFP0Z5yG6ydUpBK1nwbJgYJx00LalKK4W0BMoFyllc2usiefhWl7O4zIJotiH+lb1ndadrb2O+hDQ==</CompanyName> <ContactName>6UMvvSefKs1NfoczQRGWgJ6EOxmbXaSl1ELD23WRy13lZruUGtabnoSylHi3wAWA</ContactName> <ContactTitle>vuySEPavvRhppoU66hFBpjdy/G6LupOmmC5NWiaOyEaY/BlDuflmc8qXuxclg9wU</ContactTitle> <Phone>GA7VhTSuYc1gDdN0okgrKKbAtROQw4d+AMKssGqVdRKcyv3XepW0VXu0VzPoUjGg</Phone> <FullAddress> <Address>Sj68bSPxS2DVBFtk6STsKXjzRV+COM2G3aATE1ULPzir5d4H6MmsSAe9UT48IPuVopg6OJjTW/vFNfBtDvqfIA==</Address> <City>we8/+iqACuj5yPKNv1uzSRFDRpCA1Mm+3MVSyWcO4+q9SjgOHMGQm5YQCASNQteo</City> <Region>WwYh5pZBtl6L6KQODAJFAM1jJtjejuxunpLse0cC79j9oGKRixmobmrbr5lc22ou</Region> <PostalCode>UDAYYGFjeGj7VJz7yXRXAiMNv7vMG/J7VKKme7oLl9ckn4cxBeMJh21uWLdFps60</PostalCode> <Country>U6E5gMZPfsFlOVoENr6q5Gl5lk7LRDUQR7nX+Y83Jm7aF/MTYYizQJQeNhVCl+C5</Country> </FullAddress> </Customer> </Customers> </Root>'
Click "Call" button. You should get XML as response containing data as it was before encryption:
Message (id=dbdc9a43-c7b4-4e81-a436-346eae1906ed) {
Headers {
timestamp: 1504289434198
id: dbdc9a43-c7b4-4e81-a436-346eae1906ed
messageExchangePattern: inOnly
}
Properties {
name: iv, value: (byte[]) [B@25bfda89
name: station, value: (java.lang.String) 4c4c471d-142e-4bda-9e1d-eb7850707622
}
Payload {
(java.lang.String) <Root>
<Customers>
<Customer CustomerID="LETSS">
<CompanyName>Lets Stop N Shop</CompanyName>
<ContactName>Jaime Yorres</ContactName>
<ContactTitle>Owner</ContactTitle>
<Phone>(415) 555-5938</Phone>
<FullAddress>
<Address>87 Polk St. Suite 5</Address>
<City>San Francisco</City>
<Region>CA</Region>
<PostalCode>94117</PostalCode>
<Country>USA</Country>
</FullAddress>
</Customer>
</Customers>
</Root>
}
}
If you want to decrypt only certain nodes of XML, you can provide corresponding XPath expression as "xpath" message property.
Let's decrypt only "ContactName" and "Phone" elements. Paste following into "Properties" textbox:
[ 'xpath' : '//ContactName/text() | //Phone/text()' ]
You can use the same encrypted XML message in message payload as in previous call. Click "Call" button. This time you should get XML message, where only "ContactName" and "Phone" nodes are decrypted:
Message (id=04bf1af8-905f-4018-bd25-f3664c47c33b) {
Headers {
timestamp: 1504289695577
id: 04bf1af8-905f-4018-bd25-f3664c47c33b
messageExchangePattern: inOnly
}
Properties {
name: iv, value: (byte[]) [B@25bfda89
name: station, value: (java.lang.String) 4c4c471d-142e-4bda-9e1d-eb7850707622
name: xpath, value: (java.lang.String) //ContactName/text() | //Phone/text()
}
Payload {
(java.lang.String) <Root>
<Customers>
<Customer CustomerID="HUSGndEFhfw1tsq9vfnUQFH2ldb3d0xPzV/ZE4Qc2QGHBtvY25PR9DUA/mpvG0QD">
<CompanyName>HFzoPURrFoFP0Z5yG6ydUpBK1nwbJgYJx00LalKK4W0BMoFyllc2usiefhWl7O4zIJotiH+lb1ndadrb2O+hDQ==</CompanyName>
<ContactName>Jaime Yorres</ContactName>
<ContactTitle>vuySEPavvRhppoU66hFBpjdy/G6LupOmmC5NWiaOyEaY/BlDuflmc8qXuxclg9wU</ContactTitle>
<Phone>(415) 555-5938</Phone>
<FullAddress>
<Address>Sj68bSPxS2DVBFtk6STsKXjzRV+COM2G3aATE1ULPzir5d4H6MmsSAe9UT48IPuVopg6OJjTW/vFNfBtDvqfIA==</Address>
<City>we8/+iqACuj5yPKNv1uzSRFDRpCA1Mm+3MVSyWcO4+q9SjgOHMGQm5YQCASNQteo</City>
<Region>WwYh5pZBtl6L6KQODAJFAM1jJtjejuxunpLse0cC79j9oGKRixmobmrbr5lc22ou</Region>
<PostalCode>UDAYYGFjeGj7VJz7yXRXAiMNv7vMG/J7VKKme7oLl9ckn4cxBeMJh21uWLdFps60</PostalCode>
<Country>U6E5gMZPfsFlOVoENr6q5Gl5lk7LRDUQR7nX+Y83Jm7aF/MTYYizQJQeNhVCl+C5</Country>
</FullAddress>
</Customer>
</Customers>
</Root>
}
}