Creating a Time-Triggered Azure Function — SFTP Server Connection
Example: Using Python Paramiko to connect to SFTP Server
Introduction
Azure cloud-based serverless Functions are a great option if one needs a lightweight, event-triggered, scalable application without having to provision infrastructure. This article is intended to give an example of how to create a small, time-triggered Azure function that connects to an external SFTP server.
Before getting started, make sure to have fulfilled the following list of requirements:
- You have installed Visual Studio Code (although you may develop Azure Functions also in another IDE of your choice)
- You have a Python environment set up within your Visual Studio Code
- You have an active Azure subscription and the permission to create an Azure function within that subscription
- You have the permissions and ability to deploy your function using one of the available service plans: Service plan options.
- You have installed the Azure Function Extension in VS Code.
- Recommendation: If working behind a company proxy, use the WSL2 within VS Code. Skip to the lessons Learned section at the bottom of this page before you get started.
Step 1: Create Azure Function in VS Code
Once you have installed and confirmed the requirements above, you can get started with creating your Azure Function. Click the Azure icon in the Activity bar, then in the Azure: Functions area, select the Create new project… icon
Choose a directory location for your project workspace, and then click Select. It is recommended that you create a new folder or choose an empty folder as the project workspace. Then provide the following information at the prompts:
- Select a language for your function project: Choose Python.
- Select a Python alias to create a virtual environment: Choose the location of your Python interpreter.
- If the location isn’t shown, type in the full path to your Python binary.
- Select a template for your project’s first function: Choose time trigger.
- Provide a function name: Type TimeTriggerSFTPExample.
- Add Cron statement for every 15 seconds -> “schedule”: “*/15 * * * * *” → this is just for testing, you should change it later
- Select how you would like to open your project: Choose to Add to Workspace.
More information on how to create an Azure function from VS Code can be found on the Microsoft documentation page here.
Step 2: Set up Azurite Emulator
Now you are almost ready to develop your function locally. For time-triggered functions, you will need the Azurite Emulator. Please follow
this tutorial or this video. This can be a bit tricky; if you have further difficulties, consult this article. Make sure to add the following local.settings.json entry:
“AzureWebJobsStorage”: “UseDevelopmentStorage=true”Step 3: Test Hello World Deployment
Then, having the Emulator set up and running, you can test your setup by writing a small hello world program:
Usually, you would put the program in a separate file (for example, run.py) and simply import and call it in main(); this is just for testing. Hit F5 to see the output of your program. The output should become visible in your Azurite terminal:
Note that you may run into this error:
You can ignore it since the output will still be shown in your terminal. I never found out how to get rid of the pop-up or what it meant.
Now that it has been confirmed that debugging works, it makes sense to test if the deployment of your “hello world” time scheduled function works as well. Press F1 and follow the deployment to Azure prompts. If all goes well, you will be able to see your application on the function tab.
Click on the function. Here you may hit run/test to ensure the program is running remotely. If you do not see any output logs, please navigate
to the Kudo following the Instructions of SO user @cloud SME.
Step 4: Develop Paramiko SFTP Connection Code
Finally, we can go back to the IDE and develop the code to connect to the external SFTP server. For convenience, we will continue to debug the program with Azurite and not push every small change to Azure. Remember to fire Azurite up by writing the command azurite in the terminal.
Tip: create a separate Emulator folder in which you can start Azurite. This
will help you later because you can add the Emulator folder to the gitignore when pushing your code to Version Control. Likely you will not
want the Emulator files on your repo.
Then create a new folder outside of your function. Let’s say the folder is called server_utils. This folder will become a package that is imported with a time-triggered function. Inside theserver_utils folder, create a___init__.py file and leave it empty. Also, create a Python file and name it (for example)service.py. In service.py add your connection code as a function. Sample code can be seen here or see the snippet below:
import paramikoSSHClient_target = paramiko.SSHClient()
SSHClient_target.set_missing_host_key_policy(paramiko.AutoAddPolicy())SSHClient_target_private_key = paramiko.RSAKey.from_private_key_file(filename=RESOURCES_SERVER_SSH_KEY, password=PASSPHRASE)SSHClient_target.connect(hostname=RESOURCES_SERVER_IP,
username=RESOURCES_SERVER_SSH_USERNAME,
pkey=SSHClient_target_private_key,
#sock = jump_host_channel)SFTPClient_target = SSHClient_target.open_sftp()
Step 5: Adjustlocal.seetings.json and service.py for local debugging
Then you are ready to add the variables like RESOURCES_server_SSH_KEY and PASSPHRASE to the local.seetings.json. Note that the local.seetings.json will not be pushed to the remote. It should also not be added to Version Control. Include local.seetings.json in the gitignore file.
Thelocal.seetings.json metafile will only be used for local debugging and should look something like this:
{
"IsEncrypted": false,
"Values": {
"AzureWebJobsStorage": "UseDevelopmentStorage=true",
"FUNCTIONS_WORKER_RUNTIME": "python",
"AzureWebJobs.schedule_file_pickup.Disabled": "false",
"SFTP_USERNAME": "<username>",
"SFTP_PORT": "<port>",
"SFTP_IP": "<sftp_ip>",
"RESOURCES_SERVER_SSH_KEY": "<path to key or key>",
"PASSPHRASE": "********"
}
}In service.py you can now reference the settings following this example:
import osRESOURCES_SERVER_SSH_USERNAME = os.getenv('SFTP_USERNAME')
Step 6: Add Secrets to Azure Key Vault
Once you have confirmed that you are able to establish the connection you can prepare the deployment of the updated function by adding your key,
username, and passphrase to the key vault. Here is a tutorial on how to use key vault reference. Make sure that you give these environment variables the same names as you did in the local.settings.json file. Otherwise, your function will not know the variables.
Then deploy your function and hit run/test. And voila, you are able to establish a connection from Azure!
Lessons learned:
- I wanted to learn how to debug Azure functions locally, however, I realize now it would have been easier to simply go with an HTTP function for debugging and then simply convert it to a time trigger.
- If you work with Azure services (Azure functions) for the first time, the documentation can be overwhelming and it can be
difficult to find the exact fix for your problem. Do not try to debug Azure. - Coding the application took little time. Setting up the workspace and configuring networking-related issues were challenging and the most time-consuming.
(For me as a new/medium experienced Azure user). - Do not change the default setup of the time trigger function init. Rather package your code and only call once in main.
- Install requirements in requirements.txt
