How to configure AWS EBS to handle large files
What if you want to setup a model on AWS Sagemaker that is able to handle large files and requests? Τhe answer is to use the AWS Sagemaker Async Inference. But, there are some issues that need to be handled first to make this work…
Our situation is the following: Create an Instance Segmentation model using Detectron2 [1] that is uploaded and configured on AWS Sagemaker. The Inference is set using the Async Inference mode, in order to handle requests more than 6MB (hanldes pictures in real time). Then, build a simple Flask UI that interacts with this uploaded model.
The simple workflow is depicted below:
The frontend UI was created and uploaded on the AWS Elasting BeanStalk. While AWS creates and handles the required Nginx configuration, the user is not able to easily make changes to the server. Our troubles started when trying to implement a connection between the EBS application and the Sagemaker Endpoint and trying to upload several images (~ 2 MB).
While trying to develop this, our team came up with the following issues.
413 request entity too large: shown on the AWS Elastic Beanstalk logs
What is the meaning of this message? This is caused because the Nginx server running in AWS Beanstalk is configured to only accept no more that 1 MB of data in a single request.
How to solve:
Our solution is the following:
- Enable SSH on your EC2 machine running your Beanstalk application. To do this go to EC2 → Security Groups. Find the Security group that your EC2 is using. Click on Edit Inbound Rules.
- Click Add Rule using and enter the following configuration:
- Once you enable ssh access on your machine, ssh to your EC2 (For windows, use either Windows Terminal [2] or Bitvise SSH []
- Perform the following commands:
$ cd ../..
$ cd etc/nginx
$ sudo vi nginx.conf- Add the line client_max_body_size 20M; (or whatever limit suits you) so your file looks like this:
server {
...
client_max_body_size 60;
keepalive_timeout 60;
gzip off;
gzip_comp_level 4;
...
}- What does this do? According to Nginx documentation [2]
Sets the maximum allowed size of the client request body. If the size in a request exceeds the configured value, the 413 (Request Entity Too Large) error is returned to the client. Please be aware that browsers cannot correctly display this error. Setting
sizeto 0 disables checking of client request body size.
Idle connection timeout
The load balancer has a configured idle timeout period that applies to its connections. If no data has been sent or received by the time that the idle timeout period elapses, the load balancer closes the connection. While we upload and send images, the idle connection timeout needs to be configured.
To perform this change, follow the given instructions [also provided by AWS]
To configure the idle timeout setting for your load balancer
- Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.
- On the navigation pane, under LOAD BALANCING, choose Load Balancers.
- Select your load balancer.
- On the Description tab, choose Edit idle timeout.
- On the Configure Connection Settings page, type a value for Idle timeout. The range for the idle timeout is from 1 to 4,000 seconds.
- Choose Save.
In order for these changes also to be applied on the Elastic Beanstalk uploaded application the user needs to perform the following actions:
- SSH to your EC2 that is running the EBS application
$ cd ../..
$ cd etc/nginx
$ sudo vi nginx.conf- Type in the following in the server section:
server {
...
client_header_timeout 900;
client_body_timeout 900;
keepalive_timeout 900;
proxy_read_timeout 900;
...
}Quoting the Nginx documentation for each configuration:
client_header_timeout time;
Defines a timeout for reading client request header. If a client does not transmit the entire header within this time, the request is terminated with the 408 (Request Time-out) error.
keepalive_timeout time;
The parameter sets a timeout during which a keep-alive client connection will stay open on the server side. The zero value disables keep-alive client connections.
proxy_read_timeout time;
Defines a timeout for reading a response from the proxied server. The timeout is set only between two successive read operations, not for the transmission of the whole response. If the proxied server does not transmit anything within this time, the connection is closed.
The resulting nginx.conf should look like this:
#Elastic Beanstalk Nginx Configuration Fileuser nginx;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
worker_processes auto;
worker_rlimit_nofile 32804;events {
worker_connections 1024;
}http {
include /etc/nginx/mime.types;
default_type application/octet-stream;log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';include conf.d/*.conf;
client_max_body_size 0;
map $http_upgrade $connection_upgrade {
default "upgrade";
}server {
listen 80 default_server;
access_log /var/log/nginx/access.log main;
client_max_body_size 0;
client_header_timeout 900;
client_body_timeout 900;
keepalive_timeout 900;
proxy_read_timeout 900;gzip off;
gzip_comp_level 4;
gzip_types text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript;# Include the Elastic Beanstalk generated locations
include conf.d/elasticbeanstalk/*.conf;
}
}
Note:
To implement all of the aforementioned settings, the user needs to type the following:
$ sudo nginx -s reload- Reload keeps the server running while re-reading any configuration file updates.
- Reload is safer than restarting: if a syntax error is noticed in a config file, it will not proceed with the reload while the server remains running.
Extra
You can also add the nginx.conf file directly in the zip file you are uploading on EBS!
Do the following:
- Create a directory: ./platform/nginx/
- Place the nginx.conf within the directory from step 1.
More information can be found on AWS documentation, here
Stay tuned for more!
References
[1] Wu, Y., Kirillov, A., Massa, F., Lo, W.-Y., & Girshick, R. (2019). Detectron2. https://github.com/facebookresearch/detectron2. https://github.com/facebookresearch/detectron2
[2] https://www.microsoft.com/en-us/p/windows-terminal/9n0dx20hk701#activetab=pivot:overviewtab
