SFTP Algorithm Negotiation Fail with JSch and Pentaho Spoon/Kettle

Pentaho offers a step that handles the process of connecting to an SFTP server and putting and getting files, and (as always) orchestrating the steps leading up to and after the file transfer.  The step relies on the Java implementation of the SSH2 connectors done by the good folk at JCRAFT ( http://www.jcraft.com/jsch/ ) and uses the underlying Java policy files.  However, due to US export restrictions, the default policy files only support a max key length of 128bits.  As encryption strength ramps up in response to increased security concerns, this can lead to the SFTP step trying to connect to a server that refuses to accept a 128bit key. The following error will appear in the logs:

2017/04/05 09:41:33 - 2017/04/05 09:41:33 - get sftp (server) - ERROR (version 6.1.0.9-307, build 1 from 2016-12-14 10.21.12 by 
buildguy) : org.pentaho.di.core.exception.KettleJobException:
2017/04/05 09:41:33 - 2017/04/05 09:41:33 - get sftp (server) - com.jcraft.jsch.JSchException: Algorithm negotiation fail
2017/04/05 09:41:33 - 2017/04/05 09:41:33 - get sftp (server) - Algorithm negotiation fail
2017/04/05 09:41:33 - 2017/04/05 09:41:33 - get sftp (server) -
2017/04/05 09:41:33 - 2017/04/05 09:41:33 - get sftp (server) - at org.pentaho.di.job.entries.sftp.SFTPClient.login(SFTPClient.java:186)
2017/04/05 09:41:33 - 2017/04/05 09:41:33 - get sftp (server) - at org.pentaho.di.job.entries.sftp.JobEntrySFTP.execute(JobEntrySFTP.java:621)

I’ve attached a kettle transformation ( test_aes_key_length.ktr ) which has a little bit of Java code that writes the current max AES key length to log.  Running it on my machine (with a base 6.1 GA release install and Java 1.8.0_121) gives me the following output:

2017/04/10 19:29:10 - Write to log.0 - 
2017/04/10 19:29:10 - Write to log.0 - ------------> Linenr 1------------------------------
2017/04/10 19:29:10 - Write to log.0 - MaxKeyLength for encryption is...
2017/04/10 19:29:10 - Write to log.0 - 
2017/04/10 19:29:10 - Write to log.0 - maxkeylength = 128
2017/04/10 19:29:10 - Write to log.0 - 
2017/04/10 19:29:10 - Write to log.0 - ====================

showing that the maximum allowed key length is (as expected) 128 bits.

To fix this we’ll need to get a hold of the JCE Unlimited Strength Jurisdiction Policy Files which is an Oracle extension to Java to allow unlimited strength cryptography (http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html for Java 8, but that might change. Just search to get the current link).   Instructions to install the Policy Files can be found at this Computer Associates page https://www.ca.com/us/services-support/ca-support/ca-support-online/knowledge-base-articles.tec1698523.html

As Pentaho ships with the jsch-0.1.46 jar by default, we should take the opportunity to update this by downloading the file from here (look for the Download section half way down the page) .  To install this, find your install folder for Spoon and in the ‘lib’ folder, find the existing jsch-0.1.46.jar file and rename it to jsch-0.1.46.jar.old.  Copy the .jar file you just downloaded into the lib file to replace it.  Now restart Spoon and re-run the  test_aes_key_length.ktr transformation and you (should) get the following:

2017/04/10 20:26:35 – Write to log.0 – ————> Linenr 1——————————
2017/04/10 20:26:35 – Write to log.0 – MaxKeyLength for encryption is…
2017/04/10 20:26:35 – Write to log.0 –
2017/04/10 20:26:35 – Write to log.0 – maxkeylength = 2147483647
2017/04/10 20:26:35 – Write to log.0 –
2017/04/10 20:26:35 – Write to log.0 – ====================

The max key length has been extended to handle up to 2 Gb length keys.  Not quite ‘Unlimited’ but near enough for practical purposes.

Download the ktr here:

test_aes_key_length