Amazon SageMaker Studio Lab 把玩心得
簡介
Amazon SageMaker Studio Lab 為去年底 re:Invent 所推出的全新服務,方便學生及 ML 開發者更方便的在無需前置繁瑣的安裝使用帶有 GPU、CPU 運算資源的 Jupyter notebook 。
此外 Amazon SageMaker Studio Lab 可以用非 AWS 帳號註冊,在使用上又更方便了!
Google 上對應的服務為 Colab,以及另一個 ML 學習平台 Kaggle
如何申請 Amazon Sagemaker Studio Lab ?
只要 2 步驟:
- 到 這邊 填帳號申請單
- 等待 2 ~ 3 天即可得到帳號
- 開始玩 ~
小秘密: Why are you interested in Amazon SageMaker Studio Lab? 選取 Hacksthon 貌似會比較快過審核
如何使用 Amazon Sagemaker Studio Lab ?
先選取運算型態
CPU 有 12 小時的 runtime,GPU 只有 4 小時。
按下 Start runtime
這時候按下 Open project,即可看到熟悉的 Jupyter notebook 畫面
Getting Started with Amazon SageMaker Studio Lab
Installing Python packages
這邊強烈建議使用 conda install 指令來安裝各種 python packages,因為權限問題,若遇安裝的 packeage 需要其他 dependency 時會安裝失敗無法使用。
ref: htop installing issue、 opencv installing issue in Amazon SageMaker Studio Lab
Create env in Amazon SageMaker Studio Lab
在 Amazon SageMaker Studio Lab 使用者也可以開很多不同的 pytohn runtime 來區分不同專案所需要的環境。
點選 File -> New -> Terminal 即可開啟 Terminal 視窗
- 在 Terminal 視窗中輸入以下指令即可創立一個 Python 3.9,名為 my_environment 的環境
$ conda create --name my_environment python=3.9#這樣就可以開啟一個 python 3.9 的環境,但是在 Kernel list 還不會被列出來
- 接著在 Terminal 視窗中輸入以下指令即可切換 Python 執行環境到 my_environment
$ conda activate my_environment
- 最後在 Terminal 視窗中輸入以下,並等待 1 ~ 2 分鐘即可在 Kernel list 選取到 my_environment 環境
$ conda install ipykernel
Amazon SageMaker Studio Lab 香嗎?
這邊會以 GPU runtime 來討論
- 首先是硬體規格,GPU 使用的是 Tesla T4,colab 免費方案使用的為 K80,Kaggle 為 P100
- 再來是實際工作負載(mnist CNN)
from tensorflow.keras.datasets import mnist
from tensorflow.keras.utils import to_categorical
import time start = time.time()(train_images, train_labels), (test_images, test_labels) = mnist.load_data()
train_images = train_images.reshape((60000, 28, 28, 1))
train_images = train_images.astype('float32') / 255
test_images = test_images.reshape((10000, 28, 28, 1))
test_images = test_images.astype('float32') / 255
train_labels = to_categorical(train_labels)
test_labels = to_categorical(test_labels) from tensorflow.keras import layers
from tensorflow.keras import models model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu')) model.add(layers.Flatten())
model.add(layers.Dense(64, activation='relu')) model.add(layers.Dense(10, activation='softmax'))
model.summary() model.compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics=['accuracy'])
model.fit(train_images, train_labels, epochs=5, batch_size=64)
test_loss, test_acc = model.evaluate(test_images, test_labels) print(test_acc) done = time.time()
elapsed = done - start
print(f"time spend {elapsed} s")
- Result of Amazon SageMaker Studio Lab @ T4
Model: "sequential_1" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= conv2d (Conv2D) (None, 26, 26, 32) 320 _________________________________________________________________ max_pooling2d (MaxPooling2D) (None, 13, 13, 32) 0 _________________________________________________________________ conv2d_1 (Conv2D) (None, 11, 11, 64) 18496 _________________________________________________________________ max_pooling2d_1 (MaxPooling2 (None, 5, 5, 64) 0 _________________________________________________________________ conv2d_2 (Conv2D) (None, 3, 3, 64) 36928 _________________________________________________________________ flatten (Flatten) (None, 576) 0 _________________________________________________________________ dense_2 (Dense) (None, 64) 36928 _________________________________________________________________ dense_3 (Dense) (None, 10) 650 ================================================================= Total params: 93,322
Trainable params: 93,322
Non-trainable params: 0 _________________________________________________________________ Epoch 1/5 938/938 [==============================] - 7s 3ms/step - loss: 0.1776 - accuracy: 0.9439
Epoch 2/5 938/938 [==============================] - 2s 2ms/step - loss: 0.0468 - accuracy: 0.9857
Epoch 3/5 938/938 [==============================] - 2s 3ms/step - loss: 0.0323 - accuracy: 0.9899
Epoch 4/5 938/938 [==============================] - 2s 3ms/step - loss: 0.0242 - accuracy: 0.9926
Epoch 5/5 938/938 [==============================] - 2s 2ms/step - loss: 0.0188 - accuracy: 0.9944
313/313 [==============================] - 1s 1ms/step - loss: 0.0317 - accuracy: 0.9909 0.9908999800682068
time spend 17.8512921333313 s
- Result of Colab @ K80
Model: "sequential_2" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= conv2d_6 (Conv2D) (None, 26, 26, 32) 320 max_pooling2d_4 (MaxPooling (None, 13, 13, 32) 0 2D) conv2d_7 (Conv2D) (None, 11, 11, 64) 18496 max_pooling2d_5 (MaxPooling (None, 5, 5, 64) 0 2D) conv2d_8 (Conv2D) (None, 3, 3, 64) 36928 flatten_2 (Flatten) (None, 576) 0 dense_4 (Dense) (None, 64) 36928 dense_5 (Dense) (None, 10) 650 ================================================================= Total params: 93,322
Trainable params: 93,322
Non-trainable params: 0 _________________________________________________________________ Epoch 1/5 938/938 [==============================] - 14s 13ms/step - loss: 0.1738 - accuracy: 0.9450
Epoch 2/5 938/938 [==============================] - 9s 10ms/step - loss: 0.0486 - accuracy: 0.9853
Epoch 3/5 938/938 [==============================] - 9s 9ms/step - loss: 0.0327 - accuracy: 0.9901
Epoch 4/5 938/938 [==============================] - 9s 9ms/step - loss: 0.0253 - accuracy: 0.9922
Epoch 5/5 938/938 [==============================] - 9s 9ms/step - loss: 0.0203 - accuracy: 0.9937
313/313 [==============================] - 1s 4ms/step - loss: 0.0258 - accuracy: 0.9926
0.9926000237464905
time spend 53.64968514442444 s
- Result of Kaggle @ P100
Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= conv2d (Conv2D) (None, 26, 26, 32) 320 _________________________________________________________________ max_pooling2d (MaxPooling2D) (None, 13, 13, 32) 0 _________________________________________________________________ conv2d_1 (Conv2D) (None, 11, 11, 64) 18496 _________________________________________________________________ max_pooling2d_1 (MaxPooling2 (None, 5, 5, 64) 0 _________________________________________________________________ conv2d_2 (Conv2D) (None, 3, 3, 64) 36928 _________________________________________________________________ flatten (Flatten) (None, 576) 0 _________________________________________________________________ dense (Dense) (None, 64) 36928 _________________________________________________________________ dense_1 (Dense) (None, 10) 650 ================================================================= Total params: 93,322
Trainable params: 93,322
Non-trainable params: 0 _________________________________________________________________ Epoch 1/5 938/938 [==============================] - 12s 5ms/step - loss: 0.1769 - accuracy: 0.9445
Epoch 2/5 938/938 [==============================] - 5s 5ms/step - loss: 0.0478 - accuracy: 0.9849
Epoch 3/5 938/938 [==============================] - 5s 5ms/step - loss: 0.0317 - accuracy: 0.9898
Epoch 4/5 938/938 [==============================] - 5s 6ms/step - loss: 0.0248 - accuracy: 0.9924
Epoch 5/5 938/938 [==============================] - 5s 5ms/step - loss: 0.0197 - accuracy: 0.9936
313/313 [==============================] - 1s 3ms/step - loss: 0.0368 - accuracy: 0.9904
0.9904000163078308
time spend 35.18884539604187 s
Amazon SageMaker Studio Lab 使用心得
Amazon SageMaker Studio Lab 在使用上不像 Colab、Kaggle 已經有預建好的環境可供使用,使用者需要自己安裝各種 python package 及建立 python 執行環境,這點相對於其他平台帶來的是更高的自由度,但也會造成某些package 不好安裝,或需要特殊版本安裝的問題。
其他比較項目如下表所示,整體來說 Amazon SageMaker Studio Lab 的出現也讓這樣的運算服務有了新選擇。憑藉著較原生的 Jupyter notebook 環境,對於某些使用者說不定是一種更合適的選擇。