Java SFTP Integration using Hierynomus SSHJ
Last time, I wrote an article about how to integrate Java application with FTP Server using Apache Commons Net. I wrote that article since in the past few months I’m working to create Java application that need to integrate with FTP Server. But I’m not only working on that, I’m also working to create Java application that need to integration with SFTP Server. So in this occasion, I want to share my experience with you guys.
Before we go to our main topic, I want to give you a brief about the differences between FTP and SFTP. SFTP is abbreviation of SSH File Transfer Protocol, and SSH is stand for Secure Shell. SFTP will use a secure channel to transfer file between your system and the server. File will be encrypted when transferred to the Server, and vice versa.
Since file is encrypted when transferred to or from the Server, person in the middle that somehow able to get the file, will not able to read it, since it’s encrypted. Meanwhile, FTP will not do that, the file that transferred to or from FTP Server will not encrypted at all. That is the main difference between FTP and SFTP.
Application Dependencies
We will use spring application in this project, but don’t worry we only use spring to get all dependencies that used in unit testing. You can generate skeleton of project, using start.spring.io. We will use these dependencies in our application.
- com.hierynomus:sshj → dependency for SSHJ, including SFTP integration support. You can read further about this project from this page.
- com.github.stefanbirkner:fake-sftp-server-lambda → it’s a dependency to help us create an in-memory SFTP Server. We will use it to test our application through unit test. You can read further about his project from this page.
- org.apache.sshd:sshd-sftp → Fake SFTP Server above is using Apache SSHD SFTP to create in-memory SFTP Server, but I found that Fake SFTP Server is breaking when using Apache SSHD v2.9.0 or latest. So we need to include Apache SSHD SFTP manually using version under 2.9.0. You can read further about Apache SSHD from this page.
This is the pom.xml file example (please adjust some part from this snippet based on your needs).
Login to FTP Server
In this section, I will share how to connect to a SFTP Server using SSHClient class from Hierynomus SSHJ. We will need SFTP Server host, port, username, and password.
Snipped code above will create and returning SSHClient object after it’s connected and logged in to SFTP Server. If somehow it failed to connect or login to SFTP Server, it will throw an exception.
As you can see in the snippet code above, we use PromiscuousVerifier for host key verifier. That object is used to by-pass host key verification, since we only use it for testing purpose.
Although that method is returning SSHClient object, we cannot use that object to do SFTP operation. We need to create a SFTPClient object. I will show you how to create and use SFTPClient object later.
Print Directories and Files Tree
To get list of directories and files, we will use SFTPClient object that we will create from SSHClient object that we got when login. We will recursively get list of directories and files, and print several information.
Based on snippet code above, we use ls method to get array of RemoteResourceInfo object. RemoteResourceInfo object will represent file and directory in SFTP Server.
If RemoteResourceInfo object type is directory, we will run printTree method again to print all directories and files inside that directory.
These are summary for all method that we use to print RemoteResourceInfo information.
- getName → we use this to print directory or file name
- getPath → we use this to print file path information
- getParent → we use this to print file parent path information
- isRegularFile → we use this to know if file is file or not
- isDirectory → we use this to know if file is directory or not
We also able to get more information about the file from getAttributes method. getAttributes method will return FileAttributes object that represent file attributes information.
These are summary for all method that we use to print RemoteResourceInfo attributes information.
- getSize → we use this to print directory or file size in bytes
- getUID → we use this to print id of user that owned the file
- getGID → we use this to print id of group that owned the file
- getAtime → we use this to print last file accessed time in epoch seconds
- getMtime → we use this to print last file modified time in epoch seconds
We also able to get more detailed information about file mode from FileAttributes getMode method. FileAttributes getMode method will return FileMode object that represent file attributes mode information.
These are summary for all method that we use to print RemoteResourceInfo attributes mode information.
- getType → we use this to print type of the file, like is it file, directory, symbolic link, etc
You can read more about all methods in RemoteResourceInfo from this page.
Create a Directory
To create a directory, we will use SFTPClient object that we will create from SSHClient object that we got when login. We will create a directory in specified path.
Based on snippet code above, we use mkdirs method to create a directory. That method will not return anything. If it’s failed to create the directory, it will throw an exception.
Upload File
In this section, I will show you how to upload a local file into SFTP Server. To make it happen, we will need path of local file that we want to upload.
Based on snippet code above, we use put method to upload a file. That method will not return anything. If it’s failed to upload file, it will throw an exception. If we want to upload a file inside directory, we need to make sure that the parent directories is already exists.
Rename File
In this section, I will show you how to rename a file in SFTP. We will rename a SFTP file path into a new SFTP file path. So basically, we can use this to move a file too from old path into a new path.
Based on snippet code above, we use rename method to rename a file. That method will not return anything. If it’s failed to rename file, it will throw an exception. Since we can use this method to move file too, if we want to move a file inside directory, we need to make sure that the parent directories is already exists.
Download File
In this section, I will show you how to download a file from SFTP Server and save it into a byte array object. The download method that we will use is using an interface to save the downloaded file. So we will create a new class that implement that interface, so we can use that new class to save the downloaded file.
We will pass object of SftpDownloadFileDTO to the download method that we will use. SFTPClient will write the downloaded file into ByteArrayOutputStream object that we created in the SftpDownloadFileDTO.
Based on snippet code above, we use get method to download a file. That method will not return anything. If it’s failed to rename file, it will throw an exception. File that we downloaded will be stored inside ByteArrayOutputStream object that we created inside SftpDownloadFileDTO, and then we can get the file bytes from that output stream.
Delete File and Directory
In this section, I will show you how to delete a file in SFTP and delete a directory in SFTP.
Based on snippet code above, we use rm method to delete a file, and use rmdir method to delete a directory. Those methods will not return anything. If it’s failed to rename file, it will throw an exception. Please be aware, if we need to delete a directory, we need to make sure the directory is empty. Otherwise, directory cannot be deleted.
Wrap It Up!
Congrats!!! You have arrived in this section. You already knew many methods to manage file and directory in SFTP Server. So in this section I will wrap all codes that we already have into a single unit test method. We will use Fake SFTP Server to test, so we didn’t need to configure any SFTP Server.
In this test I will run all codes that we already wrote.
- Login into SFTP → I’m using localhost and port 2244 for the SFTP Server, and try to login using username “admin” and password “admin”
- From login method, we will get SSHClient object. To do a SFTP operation we need to create SFTPClient object from SSHClient using newSFTPClient method
- Print Tree → after successfully logged in into SFTP server, I will print tree from root “/” directory
- Create Directory → I try to create a directory in root “/” with name “create-directory-test-{timestamp}”, timestamp value will be replaced with current timestamp
- Upload File → I try to upload a file from my local “C:\\test.jpg” (I’m using windows, so you can replace this path with your path) to root “/” with name “test-upload-file-{timestamp}.jpg”. If you want to upload another file type, please be aware of remote file type in file name
- Rename File → I try to rename file that I uploaded in point 4 into “test-rename-file-{timestamp}.jpg”
- Download File → I try to download file that I renamed in point 5, and print the byte array that I got
- Delete File → I try to delete file that I renamed in point 5
- Delete Directory → I try to delete directory that I created in point 3
- Close → I close the connection to SFTP Server by closing SFTPClient and SSHClient object
Before test logic running, I prepare the Fake SFTP Server using withSFTPServer method. That method consume a consumer object which contains logic that need to connect to SFTP Server.
To set port and user, I use addUser and setPort method from FakeSftpServer object that created. If we didn’t add user and didn’t set port, SFTP Server will not have user and will run in a random port. Then I try to upload a dummy file to SFTP Server named test.txt in mock-sftp-testing directory, you can find this file when print logic run.
Conclusion
If we want to develop a Java application that can connect and manage files in SFTP Server, we can use Hierynomus SSHJ. Hierynomus SSHJ already give use many functionality to connect and manage files in SFTP Server, for example login, create directory, upload file, etc.
If we want to test our Java application SFTP Server integration code, we can create a unit test with Fake SFTP Server. We can easily setup and do preparation for SFTP Server through code.
