Boyner NOW — Going Live

Boyner NOW is a project by Boyner Büyük Mağazacılık A.Ş. which is designed to eliminate the gap in the e-commerce market and to maximize customer satisfaction. Boyner Büyük Mağazacılık A.Ş. is one of the leading non-food retail and ecommerce groups in Turkey. The main purpose of Boyner NOW is; to ensure that customers reach the products they need as quickly as possible.

Boyner NOW’s operational process starts with the customer order. Customers can order from the nearest Boyner store to their location and they can either choose the “deliver in 90 minutes” or “deliver at the time period that I choose” options. Then, ordered products will be delivered to the customers in the time period that customers choose and the courier waits till customers try or examine the product. Customers can choose which products they like to purchase and make the payment to the courier. After the purchasing process, the courier will take the unliked products back to the store.

The below diagram also shows how Boyner NOW achieves its main purpose;

Although workflows similar to Boyner NOW with fast delivery logic were carried out by food retail companies, it was not the case in the non-food retail market. Therefore, Boyner NOW is a valuable project because it appears as a project that sets a precedent.

Customer Challenge

Boyner NOW’s needs were to take place on a hybrid infrastructure with tightened security, high availability, efficient performance, lowest response time and maximum uptime. The main infrastructure is located on AWS but there is a hybrid architecture containing Azure and on-premises resources. Therefore, to achieve these goals, the infrastructure was designed with considering best practices and using SaaS where applicable.

The verticals of tightened security were determined as user permissions, hosting sensitive data, communicating separate networks in a hybrid infrastructure containing services across AWS, Azure and on-premises.

Boyner NOW’s DevOps needs were as follows;

1. AWS management console and aws-cli accesses needed to be restricted as much as possible.
2. All the network flow inside the hybrid infrastructure needed to be secured and communicate with each other internally and with high speed on runtime.
3. All the sensitive data needed to be located internally and restricted from public access.
4. System needed to be highly available yet cost effective.
5. Application servers for both .NET Classic and containerized workloads needed to be located on environments that can easily autoscale based on the load.
6. In case of an emergency, services needed to be created as fast and automated as possible with the use of Infrastructure as Code tools.
7. Services needed to be easily scalable both horizontally and vertically on high loads.
8. System uptime was critically important, therefore, alert mechanisms needed to be set across all infrastructure (AWS, Azure and on-premises) with appropriate thresholds.
9. It was necessary to ensure that the system could operate effectively even under heavy load by conducting load tests regularly.
10. The system needed to be protected against DDOS ​​attacks and suspicious user activity.
11. CI/CD processes needed to be centralized and maintainable by developers anytime a version upgrade or a critical rollback needed.

As Bestcloudforme, we created the action map in line with these needs and got to work.

Partner Solution

We used AWS IAM service to restrict user access as an important security level. We created different groups with different service permissions and restrictions and gave access only to DevOps and networking teams. Besides that, we aimed to tightened console access with MFA authentication and tightened the command line interface (aws-cli) access as well with MFA must policies attached to each group. We restricted root account activity with setting alerts to inform us every time when a root account activity is detected. Moreover, we used AWS Cloudtrail and Amazon Log Analytics to track all user activity and stored them on an Amazon S3 bucket. In addition, user access to Azure Console was restricted with Active Directory and on-premises console access has only been given to the related IT teams which are DevOps, SysOps, security and networking.

The Amazon VPCs were segmented considering resource use cases with public and private subnets and majority of the resources were placed on private subnets to restrict access. We used security groups for securing the access to the instances and services.

Since there are also some services and applications used in Boyner NOW flow were hosted on Azure and on-premises networks, it was necessary to provide secure data transfer between these networks and AWS. AWS S2S VPN and Azure S2S VPN Connection services are used to provide internal communications. Since these connections are critical for uptime, we have configured alarm definitions for tunnel status on Amazon Cloudwatch.

.NET Classic applications were hosted internally on AWS Elastic Beanstalk environments and containerized workloads were hosted on AWS EKS. With the use of Elastic Beanstalk and EKS, application servers autoscales automatically with different autoscaling policies based on the application behavior. Applications are served via Application Load Balancers and majority of the ALB’s are internal because the incoming traffic is received by the varnish cluster, which is in front of the application load balancers and provides caching. The varnish cluster’s load balancers are located as internet-facing and they handle all the incoming traffic.

Thanks to AWS’s wide range of instance family and type choices, an instance family and type with the wide network bandwidth has been chosen so that varnish servers do not experience performance problems while handling incoming traffic.

To protect the infrastructure from some complex attacks, AWS WAF is placed in front of the load balancers with managed restrictive rule sets.

To meet the need of hosting the load test environment, AWS EKS cluster has been configured because it was necessary to make consecutive requests from a wide range of IP’s that can be easily allocated and deallocated. The EKS nodes have been used as spot instances.

To increase the efficiency and decrease the margin of the error, we preferred to use AWS and Azure managed services for critical tools. We used Amazon OpenSearch, Amazon ElastiCache, Amazon RDS, Azure Cache for Redis, Azure SQL Servers to store data. With the use of these services, we managed a performance efficient, highly available and easily scalable infrastructure.

We used Infrastructure as Code tools such as Terraform to create and manage AWS SaaS resources like Amazon VPC, AWS EKS, AWS Elastic Beanstalk etc. We used Ansible playbooks to create monitoring and database tools.

Gitlab CI is used as a centralized solution for CI/CD processes. It can integrate with multiple services on multiple providers, therefore it’s easy to maintain. To preserve uptime as much as possible, rollback scenarios are highly critical. Therefore, Amazon S3, Amazon ECR and ACR is used for artifact management.

To increase user experience, efficiency and availability, monitoring and alerting mechanisms was a must for Boyner NOW’s workloads. We used Amazon CloudWatch to monitor all resources hosted on AWS and created alerting mechanisms with the help of both Amazon SNS and AWS Lambda services. Monitoring Azure and on-premises resources was also a must, so, to monitor application resources, accessible endpoints and application performance across all platforms, we used 3-party tools which are Grafana, Prometheus, Zabbix and New Relic.

Result and Benefits

By using AWS IAM and Active Directory, authentications tightened for AWS, Azure and on-premises consoles and command line interface. With security group definitions, internal placement of the resources and security requirements are managed.

By using Amazon CloudWatch, SNS, Lambda services and some 3-party tools, we managed to handle monitoring and alerting mechanisms that would primarily affect availability and efficiency needs.

By using AWS S2S VPN and Azure S2S VPN Connection, we managed the connections between different networks hosted on different providers. Therefore, a hybrid infrastructure has been designed.

By using AWS Elastic Beanstalk and AWS EKS, we ensured that the system is scalable, accessible, and performance efficient. Autoscaling groups and load balancers have been easily managed.

With providing a highly available and scalable load test environment hosted on AWS EKS, availability of the system has been controlled regularly.

With the help of Infrastructure as Code tools like Terraform and Ansible, we automated our operations and ease the management of the infrastructure.

With the CI/CD scenario designed on GitLab CI, we have automated the process to make it easier to deploy the new versions or rollback to the previous version without the human factor. With S3, ECR and ACR, we met the need of artifact management.