Accessing a Remote Cloud Virtual Machine for ML Development Using VS Code Remote SSH
Table of contents
- Introduction
- Setting up a Remote Development Workspace
- Connecting to the Remote Host
- Configuring the Remote Development Environment
- Developing Machine Learning Solutions on the Remote Machine
- Conclusion
Introduction
Remote development, as the name suggests, refers to the process of developing software or applications on a remote machine instead of working on a local machine. It allows developers to leverage the power of cloud resources and access a wide range of computing capabilities. Now, why would you want to use remote development for ML development?
Well, imagine being able to train machine learning models on a high-performance server with abundant resources without worrying about your laptop crashing or running out of memory. Sounds pretty awesome, right? Plus, you can easily collaborate with teammates and work on projects together without the hassle of setting up the same environment on each machine.
So, buckle up and let me take you through the exciting journey of creating a remote machine for ML development on the cloud using Remote SSH!
Setting up a Remote Development Workspace
Setting up a remote development workspace for machine learning development on the cloud can seem like a daunting task, but fear not! I’m here to guide you through the process in a simple and straightforward way. So, grab your favorite caffeinated beverage and let’s get started!
The first step in setting up your remote development workspace is choosing the right cloud computing platform. There are a plethora of options out there, each with their own set of pros and cons. Whether you go with AWS, Azure, or Google Cloud, just make sure you do a little research to find the one that best fits your needs.
Once you’ve selected your cloud computing platform, it’s time to choose a remote instance. This is where your code will actually run, so it’s important to pick a configuration that can handle the heavy lifting of machine learning tasks. I mean, you don’t want your models to take longer to train than it takes for your food to be delivered, right?
Now that you have your cloud instance, it’s time to install the tools you’ll need. One of the most powerful and popular tools for remote development is VSCode. It’s like the Swiss Army knife of code editors, with a plethora of extensions that can enhance your coding experience. And the best part?
VSCode has native support for remote development, so you don’t have to rely on third-party tools. How cool is that?
So, get ready to unleash your ML skills on the cloud with a remote development workspace.
Your code will run like the wind, and you’ll be able to focus on what really matters: training those models and solving the world’s problems, one line of code at a time.
Now, grab your keyboard and get ready to make some magic happen!
Connecting to the Remote Host
Ah, the moment you’ve been waiting for!
Now it’s time to connect to your remote host and unleash your coding superpowers.
But first, let’s go through the necessary steps:
- Generating your SSH key pair.
- Adding the public key to the remote instance.
- Establishing a connection with VSCode.
- Once it’s installed, go to the command palette and select “Remote-SSH: Open SSH Configuration file”.
- Click on the one of the SSH targets to connect.
Let’s See this in action and you can follow along with me!
Generating your SSH key pair:
To generate your SSH key pair, you can use the magic command “ssh-keygen”. It’s like a secret handshake for your computer and the remote instance. Make sure you specify the correct file path for your key pair.
Adding the public key to the remote instance:
Next, you need to add the public key to the remote instance. Think of it as giving the bouncer your ID at the club. This step ensures that you have access to the remote host. Remember, it’s all about the trust!
Establishing a connection with VSCode:
Now, let’s establish a connection with VSCode. To get started with VSCode’s remote development, you’ll need to install the “Remote SSH” extension from the marketplace. Just a few clicks, and you’re good to go!
Oh, and don’t forget to grab another sip of your coffee while the extension installs. Trust me, this step is caffeine-approved.
With the Remote SSH extension installed, all you need to do is open the command palette (CMD + SHIFT + P or CTRL + SHIFT + P) and select “Remote-SSH: Open SSH Configuration file” Fill in the required details. Refer to steps shared
In the prompt that appears, Select the SSH configuration file to add details required to connect to your remote server.
You should add the details in below:
Here is a sample entry to the config ( remember to indent it as shown in the above screenshot ):
Host project-name.region.vm-name
HostName 35.223.145.244
User bhaveshwadhwani
IdentityFile C:\Users\bhavesh\keys\key-file
Explanation with some more details and tips:
Host project-name.region.vm-name ( This can custom as per choice, we can keep it our own way to identify the remote machine)
HostName 35.223.145.244 ( External IP address from the machine — You can allocate a static ip for better experience )
User bhaveshwadhwani ( Username should be same as the cloud user on the machine ) — Can throw error if the user name is incorrect
IdentityFile C:\Users\bhavesh\keys\key-file ( Private key file path whcih needs to generated using ssh-keygen command if not already present) ( Public file needs to be added to the remote vm )
Click on the one of the SSH targets to connect. Once your connection is established, a window will be launched with your remote host. It’s like teleporting to a whole new coding dimension!
AND you are here!
Congratulations! You are now connected to your remote host using VSCode. It’s time to unleash your coding skills and take your ML development to the cloud!
Configuring the Remote Development Environment
Configuring the Remote Development Environment Now that we have established a connection with the remote host, it’s time to configure our development environment.
First, we need to install the necessary packages and libraries required for our machine learning project. This includes frameworks like TensorFlow or PyTorch, as well as any other dependencies specific to your project.
Once the installations are complete, we can move on to setting up the development environment. This involves configuring the necessary environment variables, paths, and configurations for our project.
It’s important to ensure that everything is set up correctly to avoid any compatibility or runtime issues. Having a well-configured development environment is crucial for smooth and efficient ML development on the remote machine.
It allows us to easily access and utilize the resources of the cloud instance, while also providing a familiar and comfortable coding environment.
So, let’s roll up our sleeves and get our hands dirty with some configuration magic!
Running Machine Learning Models on the Remote Machine
Now that we have set up our remote development workspace and established a connection with the remote host, it’s time to dive into the exciting part — running our machine learning models on the remote machine.
First things first, we need to upload and pre-process our dataset.
You can use various methods to upload the dataset to the remote machine, such as using SSH or SCP commands or syncing it using cloud storage services.
Once the dataset is on the remote machine, we can start pre-processing it to ensure optimal performance and accuracy for our models.
Next, it’s time to train our machine learning models. Utilize the power of the remote machine’s resources, such as GPU instances, to train complex models faster and efficiently.
With the help of VSCode’s remote development capabilities, you can easily write, debug, and experiment with different models directly on the remote machine.
During the training process, it’s crucial to monitor and visualize the performance of our models. With the help of tools like TensorBoard or custom visualization libraries, we can analyze metrics, track training progress, and spot any potential issues or anomalies.
This allows us to make real-time adjustments and ensure our models are performing optimally. Remember, it’s not just about running the models, but also about debugging and troubleshooting any errors or performance bottlenecks that may arise.
VSCode’s remote development features make it seamless to identify and fix issues in the code, optimize workflows, and fine-tune our models for better results.
So, don’t limit yourself to the constraints of your local machine. Embrace the power of remote development and leverage the resources of the cloud to take your machine learning projects to new heights.
Happy coding and may your models achieve unprecedented accuracy and performance!
Conclusion
Benefits of using remote development for ML:
- Enhanced Performance: With remote development, you can leverage the power of cloud resources to run your machine learning models. Say goodbye to slow local machines that struggle to handle large datasets and complex algorithms.
- Scalability: Need more computing power? No worries! Simply scale up your remote instance and handle larger workloads without sweating over hardware limitations.
- Cost-Effective: By utilizing cloud resources, you can optimize costs by paying only for what you use. No need to invest in expensive hardware or worry about maintenance and upgrades.
Tips for seamless remote development experience:
- Choose the right cloud computing platform that suits your requirements and budget. Consider factors like supported programming languages, data storage options, and ease of integration with your existing workflow.
- Select a remote instance that meets the performance and storage needs of your machine learning projects. Take into account factors like CPU, RAM, and GPU capabilities to ensure smooth execution.
- Install VSCode and the Remote SSH extension to enjoy the native support for writing and debugging code on remote servers. This allows you to seamlessly switch between local and remote development environments.
- Generate an SSH key pair and add the public key to the remote instance for secure authentication. This ensures a secure connection between your local machine and the remote host.
- Install the necessary packages and libraries on the remote development environment to avoid compatibility issues. Make sure your development environment replicates the setup of your local machine for consistent results.
In conclusion, remote development offers numerous benefits for ML enthusiasts, including enhanced performance, scalability, and cost-effectiveness. By following our tips, you can ensure a seamless remote development experience and unlock the full potential of cloud resources for your machine learning projects. Happy coding!
Follow Bhavesh Wadhwani on LinkedIn / Medium for more such amazing content on ML developement, MLOps, cloud solutions.
