HOW TO GET AI/ML DEVELOPMENT, TRAINING & INFERENCE USING PYTHON & JUPYTER KIT ON AWS(AMAZON WEB SERVICES)
Introduction
Welcome to our blog on AI/ML Development, Training, and Inference using Python and Jupyter Kit! Artificial Intelligence (AI) and Machine Learning (ML) have revolutionized various industries, ranging from healthcare to finance to technology. Python and Jupyter have emerged as powerful tools in the AI/ML landscape, providing a flexible and intuitive environment for developing, training, and deploying AI/ML models.
In this blog, we will explore the fundamentals of AI/ML development using Python and Jupyter on AWS(Amazon Web Services). We will delve into the rich ecosystem of Python libraries and frameworks specifically designed for AI/ML, such as TensorFlow, Keras, scikit-learn, and PyTorch. These libraries offer an extensive collection of pre-built algorithms, neural networks, and optimization techniques, empowering developers and data scientists to create sophisticated AI/ML models.
Moreover, Jupyter Notebooks will be our go-to platform for experimenting, visualizing, and documenting our AI/ML projects. Jupyter’s interactive interface allows us to combine code, visualizations, and explanatory text in a single document, making it easier to understand, share, and reproduce our work. We will leverage the features of Jupyter Notebooks to iteratively develop our AI/ML models, analyze results, and communicate insights effectively.
Throughout this blog, we will guide you step by step on how to set up an AI/ML development environment using Python and Jupyter on AWS(Amazon Web Services).
Whether you are a beginner looking to enter the world of AI/ML or an experienced practitioner seeking to enhance your skills, this blog will provide you with the knowledge and tools necessary to kickstart your AI/ML journey using Python and Jupyter. So, let’s dive in and explore the exciting realm of AI/ML development together!
Step by Step Guide to Setup AI/ML DEVELOPMENT, TRAINING & INFERENCE USING PYTHON & JUPYTER KIT On AWS(AMAZON WEB SERVICES)
Step 1
Open AI/ML development, training & inference using Python & Jupyter Kit Listed on AWS Marketplace.
Step 2
Click on the Continue to subscribe Button.
Login with your credentials and follow the instruction. Click on Continue to configuration Button.
Select a Region where you want to launch the VM(such as US East (N.Virginia))
Choose Action: You can launch it through EC2 or from Website.(Let’s choose Launch from website.)
Optionally change the EC2 instance type. (This defaults to t2.large instance type, 2 vCPUs and 8 GB ram.)
Optionally change the network name and subnetwork names. Be sure that whichever network you specify has ports 22 (for ssh), 3389 (for RDP) and 80 (for http) exposed.
Be sure to download the key-pair which is available by default, or you can create the new key-pair and download it.
Click on Launch. Python AI & Machine learning Suit will be start deploying.
Step 3
A summary page displays. To see this instance on EC2 Console click on EC2 Console link.
Step 4
On the EC2 Console page, instance is up and running. To connect to this instance through putty via Windows Machine, copy the IPv4 Public IP Address.
Step 5
Open putty, paste the IP address and browse your private key you downloaded while deploying the VM, by going to SSH->Auth , click on Connect Button.
Step 6
Once you are connected, change the password for ubuntu user using below command -
sudo passwd ubuntu
Step 7
Now the password for ubuntu user is set, you can connect to the VM’s desktop environment from any local windows machine using RDP protocol or linux machine using Remmina.
Step 8
From your local windows machine, go to “start” menu , in the search box type and select “Remote desktop connection”.
Step 9
In the “Remote Desktop connection” wizard, copy the public IP address and click on connect Button.
Step 10
This will connect you to the VM’s desktop environment. Provide the username (e.g “ubuntu”) and the password set in the above “Reset password” step to authenticate. Click on the OK Button.
Step 11
Now you are connected to out of box AI/ML environment via Windows Machine.
Step 12
You can use the remote desktop you connected in above step for using the VM, however, more convenient and better method is to use the Jupyter/Ipython notebook which comes with the VM .
The Notebook is available on the same public IP you used for remote desktop and accessible via any browser. Just open the browser and type the public IP address and you will get below screen for login .
Step 13
The Jupyter Notebook is configured with the ubuntu as an admin user. Login with ubuntu as username and use a strong password & note it down somewhere, since this will be the password for the admin user account from now on.
Note : Make sure you use “http” and not “https” in the URL.
Step 14
This VM comes with the default ubuntu as an admin user. So to access the Web UI and to install additional packages, Login with ubuntu user and the password you set during the first login to the Jypyter Notebook.
Step 15
Open a Terminal in your Jupyter Notebook and enter the below command to install the there package using pip.
sudo -E pip install thereNote: Don’t forget to use sudo in the above command.
Step 16
Conda lets you install new languages (such as new versions of python, node, R, etc) as well as packages in those languages. For lots of scientific software, installing with Conda is often simpler & easier than installing with pip — especially if it links to C / Fortran code.
Install a package using Conda with below command.
sudo -E conda install -c conda-forge seaborn
Step 17
The packages seaborn and there are now available to all users in JupyterHub. If a user already had a python notebook running, they have to restart their notebook’s kernel to make the new libraries available.
Step 18
Alternatively, you can install the packages from jupyter notebook itself. Open a new jupyter notebook by click new dropdown and selecting Python 3 (ipykernel)from top right corner.. Run below pip installation as
!sudo pip install matplotlib
Step 19
Install conda package!
!sudo conda install -c conda-forge gdal
The user environment is a conda environment set up in /opt/tljh/user, with a python3 kernel as the default. It is readable by all users, but writeable only by users who have root access. This makes it possible for JupyterHub admins (who have root access with sudo) to install software in the user environment easily.
Conclusion
In conclusion, this step-by-step guide provides instructions for setting up AI/ML development, training, and inference using Python and Jupyter Notebook on AWS (Amazon Web Services). By following these steps, you can easily launch a virtual machine (VM) and connect to it using SSH or RDP. The guide also explains how to change the password for the “ubuntu” user and access the VM’s desktop environment from both Windows and Linux machines. Additionally, it highlights the convenience of using the Jupyter Notebook that comes pre-configured with the VM, which allows you to access it through any browser using the VM’s public IP address. With this setup, you can start working on AI/ML projects using Python and Jupyter Notebook on AWS.
