Tutorial on how to create a python function with libraries in a package
By far we have discovered the Qinling architecture and basic functionalities of Qinling. In this tutorial, we would be creating a function with libraries in a package.
Environment: DevStack (Rocky) on Ubuntu 16.04
[[local|localrc]]
RECLONE=False
HOST_IP=10.0.2.15enable_plugin qinling https://github.com/openstack/qinlingLIBS_FROM_GIT+=python-qinlingclient
DATABASE_PASSWORD=password
ADMIN_PASSWORD=password
SERVICE_PASSWORD=password
SERVICE_TOKEN=password
RABBIT_PASSWORD=password
LOGFILE=$DEST/logs/stack.sh.log
VERBOSE=True
LOG_COLOR=False
SCREEN_LOGDIR=$DEST/logs
LOGFILE=$DEST/logs/stack.sh.log
LOGDAYS=1ENABLED_SERVICES=rabbit,mysql,key# Swift
ENABLED_SERVICES+=,swift
SWIFT_HASH=66a3d6b56c1f479c8b4e70ab5c2000f5
SWIFT_REPLICAS=1
- The first step would be to create a runtime environment, Qinling already offers Python 2.7. Let’s go ahead with it! For this tutorial, I would be using CLI commands, but if you are interested in exploring curl or httpie then you can find them on the Qinling documentation.
NOTE: Make sure you are using the openstackqinling/python-runtime:0.0.5 image and not openstackqinling/python-runtime:0.0.4
$ openstack runtime create openstackqinling/python-runtime:0.0.5 --name python2.7+-------------+--------------------------------------+
| Field | Value |
+-------------+--------------------------------------+
| id | 4b764820-1775-41b0-bce7-54a5b7b27071 |
| name | python2.7 |
| image | openstackqinling/python-runtime |
| status | creating |
| description | None |
| project_id | 1ee3a658dcf749cf963d8b620c8cca1d |
| created_at | 2018-07-10 19:54:39.373462 |
| updated_at | None |
+-------------+--------------------------------------+
Always create a runtime environment before starting with function creation. The following command shows the status of the runtime environment.
$ openstack runtime list+-------------+--------------------------------------+
| Id | 4b764820-1775-41b0-bce7-54a5b7b27071 |
| Name | python2.7 |
| Image | openstackqinling/python-runtime |
| Status | available |
| Description | None |
| Project_id | 1ee3a658dcf749cf963d8b620c8cca1d |
| created_at | 2018-07-10 19:54:39 |
| updated_at | 2018-07-10 19:54:45 |
+-------------+--------------------------------------+
Now that our runtime is created let's create python function with libraries in a package.
Th function that we are using here would resize an image and stores the image to swift. It then uploads the resized image to a new container with the same object name. The library which we would be using is Pillow in order to carry out this function.
2. Create a directory, for example, ~/qinling_swift_test
$ mkdir ~/qinling_swift_test$ cd qinling_swift_test
3. Create a custom python code for resizing an image at the root level of the directory
$ vi resize_image.pyimport os
from PIL import Image
import swiftclient
from swiftclient.exceptions import ClientException
def resize_image(image_path, resized_path):
with Image.open(image_path) as image:
image.thumbnail(tuple(x / 4 for x in image.size))
image.save(resized_path)
def main(context, container_name, object_name):
conn = swiftclient.Connection(
session=context['os_session'],
os_options={'region_name': 'RegionOne'},
)
# Download original image
image_path = os.path.abspath('./%s' % object_name)
_, obj_contents = conn.get_object(container_name, object_name)
with open(image_path, 'w') as local:
local.write(obj_contents)
print('Downloaded object %s from container %s' % (object_name, container_name))
thumb_path = os.path.abspath('./%s_resized.png' % object_name)
resize_image(image_path, thumb_path)
print('Resized.')
# Create new container if needed
new_container_name = '%s_resized' % container_name
try:
conn.head_container(new_container_name)
except ClientException:
conn.put_container(new_container_name)
print("New container %s created." % new_container_name)
# Upload resized image
with open(thumb_path, 'r') as new_local:
conn.put_object(
new_container_name,
object_name,
contents=new_local,
content_type='text/plain'
)
os.remove(image_path)
os.remove(thumb_path)
print('Uploaded object %s to container %s' % (object_name, new_container_name))
Now install the python library required for the program execution using the pip command. The library needs to be installed at the root level of the directory.
$ pip install module-name -t path/to/dir4. We need to install the pillow module in the directory
$ pip install pillow -t ~/qinling_test5. Now zip the contents of the directory.
cd ~/qinling_test; zip -r9 ~/qinling_test/resize_image.zip .
6. Create a function
openstack function create --name resize_image \
--runtime 4b764820-1775-41b0-bce7-54a5b7b27071 \
--entry resize_image.main \
--package ~/qinling_test/resize_image.zip+-------------+-------------------------------------------------------------------------+
| Field | Value |
+-------------+-------------------------------------------------------------------------+
| id | ddd75fcc-6b2d-406b-8970-ebc9d65cdf47 |
| name | resize_image |
| description | None |
| count | 0 |
| code | {u'source': u'package', u'md5sum': u'24922e19b6ee4020ebe25c8ab8b43cce'} |
| runtime_id | 4b764820-1775-41b0-bce7-54a5b7b27071 |
| entry | resize_image.main |
| project_id | 1ee3a658dcf749cf963d8b620c8cca1d |
| created_at | 2018-07-10 20:37:16.085264 |
| updated_at | None |
| cpu | 100 |
| memory_size | 33554432 |
+-------------+-------------------------------------------------------------------------+
7. Upload an image to the swift container.
$ curl -SL https://goo.gl/images/ccB8mQ -o ~/origin.jpg$ openstack container create origin_folder+---------------------------------------+---------------+------------------------------------+
| account | container | x-trans-id |
+---------------------------------------+---------------+------------------------------------+
| AUTH_1ee3a658dcf749cf963d8b620c8cca1d | origin_folder | tx569e565c93a34239b8a3d-005b453bf6 |
+---------------------------------------+---------------+------------------------------------+$ openstack object create origin_folder ~/origin.jpg --name image
+--------+---------------+----------------------------------+
| object | container | etag |
+--------+---------------+----------------------------------+
| image | origin_folder | efbbbe3f1f6d5e911c84d17baec28f72 |
+--------+---------------+----------------------------------+$ openstack object show origin_folder image
+----------------+---------------------------------------+
| Field | Value |
+----------------+---------------------------------------+
| account | AUTH_1ee3a658dcf749cf963d8b620c8cca1d |
| container | origin_folder |
| content-length | 35904 |
| content-type | application/octet-stream |
| etag | efbbbe3f1f6d5e911c84d17baec28f72 |
| last-modified | Tue, 10 Jul 2018 23:07:05 GMT |
| object | image |
+----------------+---------------------------------------+
8. Invoke the function by specifying the function_id and the function inputs
$ openstack function execution create ddd75fcc-6b2d-406b-8970-ebc9d65cdf47 — input ‘{“container_name”: “origin_folder”, “object_name”: “image”}’+ — — — — — — — — — + — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — -+
| Field | Value |
+ — — — — — — — — — + — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — -+
| id | 17dd2ad0-bf66–4ceb-ae49–825f026c4a42 |
| function_id | ddd75fcc-6b2d-406b-8970-ebc9d65cdf47 |
| function_version | 0 |
| description | None |
| input | {“object_name”: “image”, “container_name”: “origin_folder”} |
| result | {“duration”: 1.167, “output”: null} |
| status | success |
| sync | True |
| project_id | 1ee3a658dcf749cf963d8b620c8cca1d |
| created_at | 2018–07–10 23:08:29 |
| updated_at | 2018–07–10 23:08:32 |
+ — — — — — — — — — + — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — -+
9. Check the execution log using the following command
$ openstack function execution log show 17dd2ad0-bf66–4ceb-ae49–825f026c4a42Start execution: 17dd2ad0-bf66–4ceb-ae49–825f026c4a42
Downloaded object image from container origin_folder
Resized.
New container origin_folder_resized created.
Uploaded object image to container origin_folder_resized
Finished execution: 17dd2ad0-bf66–4ceb-ae49–825f026c4a42
10. Now a new object of a smaller size is created in a new container called origin_folder_resized. Let's verify it!
$ openstack container list
+-----------------------+
| Name |
+-----------------------+
| origin_folder |
| origin_folder_resized |
+-----------------------+
$ openstack object list origin_folder_resized
+-------+
| Name |
+-------+
| image |
+-------+$ openstack object show origin_folder_resized image
+----------------+---------------------------------------+
| Field | Value |
+----------------+---------------------------------------+
| account | AUTH_1ee3a658dcf749cf963d8b620c8cca1d |
| container | origin_folder_resized |
| content-length | 24146 |
| content-type | text/plain |
| etag | ef039d101571b312a28977ee02eae616 |
| last-modified | Tue, 10 Jul 2018 23:08:32 GMT |
| object | image |
+----------------+---------------------------------------+
The content-length before the resizing the image was 35904 bytes, now the image size is 24146 bytes.
In this tutorial, we have seen how to create a runtime environment, a function with libraries (we use pillow) in a package. We used swift object storage service for OpenStack in order to implement the resize image functionality.
That’s all for today!
Happy Qinling! :)
