Java FTP Integration using Apache Commons Net
In the past few months, I’m working on FTP server integration. The use case is client will put file in our FTP server and then we will periodically read the file and process each files that client upload. So in this occasion, I want to share my experience regarding FTP integration using Java.
Application Dependencies
We will use spring application in this project, but don’t worry we only use spring to get all dependencies that used in unit testing. You can generate skeleton of project, using start.spring.io. We will use these dependencies in our application.
- commons-net:commons-net → dependency for Apache Commons Net, including FTP integration support. You can read further about this project from this page.
This is the pom.xml file example (please adjust some part from this snippet based on your needs).
Login to FTP Server
In this section, I will share how to connect to a FTP Server using FTPClient class from Apache Commons Net. We will need FTP Server host, port, username, and password.
Snipped code above will create and returning FTPClient object after it’s connected and logged in to FTP Server. If somehow it failed to connect or login to FTP Server, it will throw an exception.
When we run connect and login method, actually we sent a command to FTP Server. Then after FTP Server execute our command, it will return a reply code. You can read further about FTP Command from this page, and FTP Reply from this page.
To better known about command that executed, we can add command listener to log whenever our code sent a command or receive a reply.
After we add a Protocol Command Listener, we can log whenever our code sent a command or receive a reply from FTP Server.
Print Directories and Files Tree
To get list of directories and files, we will use FTPClient object that we created in the previous section. We will recursively get list of directories and files, and print several information.
Based on snippet code above, we use listFiles method to get array of FTPFile object. FTPFile object will represent file and directory in FTP Server.
If FTPFile object type is directory, we will run printTree method again to print all directories and files inside that directory.
These are summary for all method that we use to print FTPFile information.
- getName → we use this to print directory or file name
- getTimestamp → we use this to print file timestamp information
- getGroup → we use this to print file group information
- getLink → we use this to print file link if file is symbolic link, if file is not symbolic link, it will return null
- getUser → we use this to print user that owned the file
- getType → we use this to print file type. It will return file type code, there are 4 types of file. 0 means file, 1 means directory, 2 means symbolic link file, and 3 means unknown
- isFile → we use this to know if file is file or not
- isDirectory → we use this to know if file is directory or not
- toFormattedStirng → we use this to print all information in a single string
You can read more about all methods in FTPFile from this page.
Create a Directory
To create a directory, we will use FTPClient object that we created. We will create a directory in specified path. If we want to create a sub-directory, we need to make sure that the parent directories is already exists.
Based on snippet code above, we use makeDirectory method to create a directory. That method will return boolean value to indicate whether we success or not to create the directory.
Upload File
In this section, I will show you how to upload a local file into FTP Server. To make it happen, we will need local file path that we wrap into FileInputStream object.
Based on snippet code above, we use storeFile method to upload a file. That method will return boolean value to indicate whether we success or not to upload the file. If we want to upload a file inside directory, we need to make sure that the parent directories is already exists.
Rename File
In this section, I will show you how to rename a FTP file. We will rename a FTP file path into a new FTP file path. So basically, we can use this to move a file too from old path into a new path.
Based on snippet code above, we use rename method to rename a file. That method will return boolean value to indicate whether we success or not to rename the file. Since we can use this method to move file too, if we want to move a file inside directory, we need to make sure that the parent directories is already exists.
Download File
In this section, I will show you how to download a FTP file and save it into a byte array object.
Based on snippet code above, we use retrieveFile method to download a file. That method will return boolean value to indicate whether we success or not to rename the file. File that we downloaded will be stored inside ByteArrayOutputStream object that we created.
Delete File and Directory
In this section, I will show you how to delete a FTP file and delete a FTP directory.
Based on snippet code above, we use deleteFile method to delete a file, and use removeDirectory method to delete a directory. Those methods will return boolean value to indicate whether we success or not to delete file or directory.
Wrap It Up!
Congrats!!! You have arrived in this section. You already knew many methods to manage file and directory in FTP Server. So in this section I will wrap all codes that we already have into a single unit test method.
In this test I will run all codes that we already wrote.
- Login into FTP → I’m using localhost and port 2121 for the FTP Server, and try to login using username “admin” and password “admin”
- Print Tree → after successfully logged in into FTP server, I will print tree from root “/” directory
- Create Directory → I try to create a directory in root “/” with name “create-directory-test-{timestamp}”, timestamp value will be replaced with current timestamp
- Upload File → I try to upload a file from my local “C:\\test.jpg” (I’m using windows, so you can replace this path with your path) to root “/” with name “test-upload-file-{timestamp}.jpg”. If you want to upload another file type, please be aware of remote file type in file name
- Rename File → I try to rename file that I uploaded in point 4 into “test-rename-file-{timestamp}.jpg”
- Download File → I try to download file that I renamed in point 5, and print the byte array that I got
- Delete File → I try to delete file that I renamed in point 5
- Delete Directory → I try to delete directory that I created in point 3
- Disconnect → I disconnect the connection to FTP Server
From this point you can try to run the unit test with your FTP Server. If you don’t have FTP Server yet, I will help to create a test FTP Server in next section.
Local Test using Apache Mina FTP Server
If you doesn’t have any FTP Server to test, we can use Apache Mina FTP Server. Apache Mina FTP Server is portable FTP Server that we can run without many configurations.
- Download Apache Mina FTP Server →https://mina.apache.org/ftpserver-project/download_1_2.html
- Extract Downloaded Zip
- Open Command Prompt
- Move to point 2 extracted directory
- Run bin\ftpd.bat res\conf\ftpd-typical.xml
- FTP Server will running in localhost, port 2121. User that available will be “admin” with password “admin”
In this tutorial, I’m using windows. If you want to read further about Apache Mina FTP Server, you can read in this page.
If port 2121 is already in used, you can change the port configuration inside res\conf\ftpd-typical.xml file
Unit Test using Mock FTP Server
If you didn’t want to run the FTP Server manually, we can use MockFtpServer dependency in our unit test. So we didn’t need to start FTP Server manually before test, and stop FTP Server manually after test.
To use this, we need to add new dependency in our pom.xml.
- org.mockftpserver:MockFtpServer → dependency for MockFtpServer that we will use in our unit test.
So the pom.xml will be like this.
After that, we need to modify our unit test to start the Mock FTP Server before test running, and stop the Mock FTP Server after test is done.
As you can see in the code snippet above, I added 2 new methods.
First method, named setUpAll is used to create and start Mock FTP Server. I set FTP Server port to 2121 using setServerControlPort method. And then I set the User Account. I added “admin” user with password “admin” to User Account that will redirected to directory “c:\\mock-ftp-testing”. Then I created new directory named “c:\\mock-ftp-testing”, and created new file named “test.txt” inside “c:\\mock-ftp-testing”. After all configuration is finished, I start the Mock FTP Server.
Second method, named tearDownAll is used to stop the Mock FTP Server.
First method will be executed before the test, and the second method will be executed after the test. So with this we didn’t need to manually start and stop the FTP Server if we want to test the code.
If you want to read further about Mock FTP Server, you can read in this page.
Conclusion
If we want to develop a Java application that can connect and manage files in FTP Server, we can use Apache Commons Net. Apache Commons Net already give use many functionality to connect and manage files in FTP Server, for example login, create directory, upload file, etc.
If we want to test our Java application FTP Server integration code in local environment, we can use Apache Mina FTP Server to easily start a FTP Server. Then if we want to start a FTP Server automatically to support our unit test, we can use Mock FTP Server that we can easily setup through Java code.
