Running an HTTP & GRPC Python Server Concurrently on AWS Elastic Beanstalk
I will be using the async python web framework AIOHTTP. In case that you may not have heard or used it before I would advice you to jump quickly to the official documentation here and briefly skim through the introduction.
All the code used in this tutorial is also on github.
HTTP
Lets start off with the HTTP server! I am going to use python@3.7 and pipenv as my package manager and virtual environment but feel free to use any other tool that you might be familiar with. Run the following to start a new python3 environment:
pipenv --python 3.7Before writing any code, we need to first install our dependencies. For the time being, aiohttp will suffice to get the HTTP server up and running. so let's install it:
pipenv install aiohttpNow we can start off with our Application class.
# application.pyfrom aiohttp import web
class Application(web.Application):
def __init__(self):
super().__init__() def run(self):
return web.run_app(self, port=8000)
application = Application()if __name__ == '__main__':
application.run()
On a quick note, you have to call the new application instance ‘application’ since elastic beanstalk expects that variable name by default when booting the app, direct quote of the docstring from on the official doc:
“EB looks for an ‘application’ callable by default.”
At this point you have a fully runnable web server:
python application.py>>> ======== Running on http://0.0.0.0:8000 ========
Nothing really exciting since we don’t have any routes, so let's make a ‘/helloworld’ page.
# application.pyfrom aiohttp import web
class HelloWorldView(web.View):
async def get(self) -> web.Response:
return web.Response(text="Hello World!")
class Application(web.Application):
def __init__(self):
super().__init__()
self.add_routes() def add_routes(self):
self.router.add_view('/helloworld', HelloWorldView) def run(self):
web.run_app(self, port=8000)
application = Application()if __name__ == '__main__':
application.run()
Restart the server and enter the following url: ‘http://localhost:8000/helloworld’ in your browser. Voila, you should now see a “Hello World!” text on the page, sweet!
GRPC
Now that we have a working HTTP server, lets add the GRPC server. For that, we will first need to define our service.proto schema. You can read more about protobuf here.
// service.protosyntax = "proto3";
package service;service HelloWorldService {
rpc Hello(HelloRequest) returns (HelloResponse) {}
}message HelloRequest {
string name = 1;
}message HelloResponse {
string message = 1;
}
Here we defined an RPC service that exposes a method called Hello that takes as argument a request with a name attribute and returns a response with a message attribute. We then need to compile the proto file to a more usable python module by running the following:
python -m grpc_tools.protoc -I .. --python_out=/ --grpc_python_out=/ service.proto>>> Error while finding module specification for 'grpc_tools.protoc' (ModuleNotFoundError: No module named 'grpc_tools'
Oops! we forgot to install the packages to compile the protobuf schema…
pipenv install grpcio
pipenv install grpcio-toolsUpon running the command again, you will see 2 auto-generated python files ‘service_pb2.py’ and ‘service_pb2_grpc.py’. Now we have everything needed for our GRPC server, we can add a new GrpcServer class and a HelloServicer class to handle incoming RPC requests.
# application.pyimport asyncio
import grpcfrom aiohttp import web
from grpc.experimental.aio import init_grpc_aio
from service_pb2_grpc import add_HelloWorldServiceServicer_to_server
from service_pb2_grpc import HelloWorldServiceServicer
class HelloWorldView(web.View):
async def get(self):
return web.Response(text="Hello World!")
class Application(web.Application):
def __init__(self):
super().__init__() self.grpc_task = None
self.grpc_server = GrpcServer() self.add_routes() def add_routes(self):
self.router.add_view('/helloworld', HelloWorldView) def run(self):
return web.run_app(self, port=8000)
class HelloServicer(HelloWorldServiceServicer):
def Hello(self, request, context):
response = HelloResponse()
response.message = "Hello {}!".format(request.name)
return response
class GrpcServer:
def __init__(self):
init_grpc_aio() self.server = grpc.experimental.aio.server()
self.servicer = HelloServicer() add_HelloWorldServiceServicer_to_server(
self.servicer,
self.server) self.server.add_insecure_port("[::]:50051") async def start(self):
await self.server.start()
await self.server.wait_for_termination() async def stop(self):
await self.servicer.stop(0)
application = Application()if __name__ == '__main__':
application.run()
In the HelloServicer we implemented the Hello method to match the RPC definition in the service.proto file. Also, note that we are running the GRPC server on port 50051 to allow the client to connect to the 2 servers individually.
Piecing things together…
# application.pyimport asyncio
import grpcfrom aiohttp import web
from grpc.experimental.aio import init_grpc_aio
from service_pb2 import HelloResponse
from service_pb2_grpc import add_HelloWorldServiceServicer_to_server
from service_pb2_grpc import HelloWorldServiceServicer
class HelloWorldView(web.View):
async def get(self):
return web.Response(text="Hello World!")
class Application(web.Application):
def __init__(self):
super().__init__() self.grpc_task = None
self.grpc_server = GrpcServer() self.add_routes()
self.on_startup.append(self.__on_startup())
self.on_shutdown.append(self.__on_shutdown()) def __on_startup(self):
async def _on_startup(app):
self.grpc_task = \
asyncio.ensure_future(app.grpc_server.start()) return _on_startup def __on_shutdown(self):
async def _on_shutdown(app):
await app.grpc_server.stop()
app.grpc_task.cancel()
await app.grpc_task return _on_shutdown def add_routes(self):
self.router.add_view('/helloworld', HelloWorldView) def run(self):
return web.run_app(self, port=8000)
class HelloServicer(HelloWorldServiceServicer):
def Hello(self, request, context):
response = HelloResponse()
response.message = "Hello {}!".format(request.name)
return response
class GrpcServer:
def __init__(self):
init_grpc_aio() self.server = grpc.experimental.aio.server()
self.servicer = HelloServicer() add_HelloWorldServiceServicer_to_server(
self.servicer,
self.server) self.server.add_insecure_port("[::]:50051") async def start(self):
await self.server.start()
await self.server.wait_for_termination() async def stop(self):
await self.servicer.close()
await self.server.wait_for_termination()
application = Application()if __name__ == '__main__':
application.run()
AIOHTTP provides a on_startup and a on_shutdown hook which triggers some registered method when the Application starts and shutdown respectively. In our case, we also want to start the GrpcServer in python Future which will basically sit and wait in the background until a new RPC request comes in to resume its execution in the main python thread.
We can now write a small script to emulate a GRPC client connecting and invoking the RPC call Hello!
# grpc_test_script.pyimport grpcfrom service_pb2_grpc import HelloWorldServiceStub
from service_pb2 import HelloRequestchannel = grpc.insecure_channel('localhost:50051')
hello_world_stub = HelloWorldServiceStub(channel)while True:
name = input("Enter your name: ")
request = HelloRequest()
request.name = name
response = hello_world_stub.Hello(request) print(response.message)
Now run it and enter some text in your terminal
Enter your name: Test01
>>> Hello Test01!
Enter your name: Mr Lazy
>>> Hello Mr Lazy!
Enter your name:Cool our GRPC service works! now comes the fun part.
Deploying to AWS
I will assume that you already have an account on AWS and ‘eb-cli’ installed locally. But if you don’t already, you can signup for a free account here and follow the installation instructions here to install the ‘eb-cli’.
But prior to deploying, we still have some work left to do. Firstly we need to generate a dependency file (requirement.txt) to allow any other environment to know what dependencies are needed to run our application.
pipenv lock -r > requirements.txtNow we need to create a Procfile with the following configuration:
web: gunicorn application:application --bind :8000 --worker-class aiohttp.worker.GunicornWebWorkerThe Procfile allows us to define our run command with specific configurations. Let us break down the individual components that make up the Procfile:
web:Indicates that we want to run the following commands for web server environment. For example, we would have usedworker:if we were running in a worker environment instead.gunicornA popular python HTTP server compatible with most python web frameworks. Simply put, it relays the incoming requests from the outside world (from AWS ELB) to our AIOHTTP server.application:applicationThe first ‘application’ indicates the main entry file, in our case, it would be ‘application.py’ and the second one is the name of the variable that points to a WSGI (or Application).--bindThe bind flag indicates which port gunicorn will bind incoming requests to. We are running our HTTP server on port 8000, the flag value is:8000.--worker-classThe class of worker that will process the request. The default worker class isthreads, which only works for synchronous request handling frameworks like Flask or Django. AIOHTTP being async framework requires its own worker typeGunicornWebWorker.
Finally
We get to finally deploy our application on AWS and watch it run live!
- Initialize a new elastic beanstalk application: (For most part, the default configuration will work just fine for us.)
eb init>>> Select a default region
1) us-east-1 : US East (N. Virginia)
2) us-west-1 : US West (N. California)
3) us-west-2 : US West (Oregon)
4) eu-west-1 : EU (Ireland)
5) eu-central-1 : EU (Frankfurt)
6) ap-south-1 : Asia Pacific (Mumbai)
7) ap-southeast-1 : Asia Pacific (Singapore)
8) ap-southeast-2 : Asia Pacific (Sydney)
9) ap-northeast-1 : Asia Pacific (Tokyo)
10) ap-northeast-2 : Asia Pacific (Seoul)
11) sa-east-1 : South America (Sao Paulo)
12) cn-north-1 : China (Beijing)
13) cn-northwest-1 : China (Ningxia)
14) us-east-2 : US East (Ohio)
15) ca-central-1 : Canada (Central)
16) eu-west-2 : EU (London)
17) eu-west-3 : EU (Paris)
18) eu-north-1 : EU (Stockholm)
19) eu-south-1 : EU (Milano)
20) ap-east-1 : Asia Pacific (Hong Kong)
21) me-south-1 : Middle East (Bahrain)
22) af-south-1 : Africa (Cape Town)
(default is 3): 3>>> Select an application to use
1) [ Create new Application ]
(default is 1): 1>>> Enter Application Name
helloworld>>> It appears you are using Python. Is this correct?
(Y/n): Y>>> Select a platform branch.
1) Python 3.7 running on 64bit Amazon Linux 2
2) Python 3.6 running on 64bit Amazon Linux
3) Python 3.4 running on 64bit Amazon Linux (Deprecated)
4) Python 2.7 running on 64bit Amazon Linux (Deprecated)
5) Python 2.6 running on 64bit Amazon Linux (Deprecated)
6) Preconfigured Docker - Python 3.4 running on 64bit Debian (Deprecated)
(default is 1): 1>>> Do you wish to continue with CodeCommit? (y/N) (default is n):
n>>> Do you want to set up SSH for your instances?
n
- Create a new environment:
eb create helloword-env- Open the application:
eb open- and DONE!
Don’t forget to add a ‘/helloworld’ at the end of the newly deployed URL, since we did not implement an endpoint for ‘/’.
404: Not FoundYou can now modify the ‘grpc_test_script.py’ and update to the deployed URL and test things out.
# grpc_test_script.py...
channel = grpc.insecure_channel("{}:50051".format(EB_DEPLOYED_URL))
...
Conclusion
This is by far not a production-ready system but merely a small setup that I needed for a side project. There are a lot of reasons as to why you might want to have both an HTTP & GRPC server running on the same instances at any time. And my use case was that I needed a GRPC service primarily with the ability to accept multiple long-lived websocket connections. I also wanted to have some degree of flexibility in terms of being able to add some basic web pages for health check purposes.
I hope that this article finds you well and thank you for reading!
