Infrastructure Refactoring from AWS Elastic Beantsalk to AWS ECS

At Technofy, we're delighted to share with you the case of WeWeb's refactoring architecture on AWS. This article outlines the challenges they faced and the solutions we proposed as a trusted partner to ensure a smooth and efficient infrastructure optimization.

‍
Contexte

‍

WeWeb is a no-code web development platform enabling users to create the front-end of web applications without having to code. Thanks to its intuitive drag & drop interface, it offers the possibility and flexibility to create responsive and aesthetically pleasing front ends without advanced technical skills. 

As a no-code front-end builder, WeWeb can be combined with any back-end (database and server). It supports real-time data integration and automation, making the creation of professional web applications much faster.

The Weweb solution was based primarily on AWS ElasticBeanstalk, AWS Codebuild, AWS Codepipeline and RDS services. 

ElasticBeanstalk is an AWS managed service for deploying and scaling Web applications and services. It is a practical and effective solution for technical teams with little experience of infrastructure management, thanks to its rapid implementation and simplicity.

Due to the growing popularity of its solution, and anticipating a sharp increase in traffic, WeWeb decided to optimize its infrastructure in order to offer its customers first-rate performance and reliability. 

Key issues 

‍

With the rise in popularity of the platform and the evolution of technical teams' needs towards greater granularity, several needs were identified:

• Ensure infrastructure scalability to handle peak loads.
• Guarantee high availability of services.
• Implement optimal infrastructure security.
• Deploy an effective monitoring and alerting system.
• Optimize cost/performance ratio.
• Simplify infrastructure deployment and management processes for the team.

In addition to these immediate needs, there were longer-term perspectives, such as adopting a multi-region architecture, integrating with AWS serverless tools and implementing event-driven solutions. 

After a thorough analysis, it became clear that WeWeb's infrastructure was not optimized for the cloud and did not follow DevOps best practices, making it difficult to optimize the current infrastructure. This, plus the fact that ElasticBeanstalk no longer met WeWeb's requirements in terms of granularity, led us to an architecture refactoring project, where the aim was to improve and optimize both financially and technically the existing infrastructure. 

Solution overview

‍

Following this analysis of the existing system, we were able to establish a suitable, scalable target architecture that will support WeWeb over the long term. 

The core of this architecture solution is load execution withAWS ECS (Elastic Container Service). This solution was chosen because it is a managed service from AWS that enables containers to be run, and above all requires little maintenance and specific skills compared with an AWS EKS (Kubernetes) cluster, for example. To find out more, visit https://www.technofy.io/blog/container-orchestration-and-execution-on-aws 

The main advantages of ECS are: 

• Ease of management (managed service) 
• Automatic scalability of containers (services) and instances (with AWS Autoscaling)
• Enhanced security with IAM
• Perfect integration with other AWS services (Load balancer, EC2, Container Registry, Cloudwatch, etc.).
  
  

This refactoring also offered an opportunity to implement Infrastructure as Code via Terraform and CI/CD pipelines, always with the aim of simplifying infrastructure deployment and management. 

‍

‍

‍

‍

CI/CD pipeline
‍

We've set up a classic CI/CD deployment pipeline on Github: 

• CI part: We have a workflow that triggers a Github Action when a branch is tagged. This action will build a docker image and store it in an AWS ECR.
• CD part: the deployment process continues by updating the container version images used in ECS services. The update is progressive (rollout update).

‍

‍

Monitoring and alerting

In terms of infrastructure monitoring, we considered two main solutions: AWS and Datadog. To help WeWeb make the most appropriate choice, we opted to test both solutions.

Initially, we set up monitoring via AWS, while Datadog integration is planned after migration. This approach allows us to concentrate on one subject at a time, giving us the time we need to evaluate both solutions.

The monitoring system deployed on AWS is based primarily on AWS CloudWatch. It includes alarms for each service, with a notification system via e-mail and/or Slack. These alarms are also associated with customized dashboards for each service and for a global view. In addition, the centralization of logs has been an important element in facilitating the error-finding and debugging process.
‍

Security 

To ensure optimum security on several levels, the following elements have been implemented:

• Identity management with IAM to control access to resources.
• Single sign-on (SSO) to simplify access to different services.
• Use of AWS Organizations with a dedicated account per environment for effective isolation.
• Data encryption using KMS.
• Configuration of private subnets to limit resource exposure.
• Use of Systems Manager for secure connections in debug mode, enabling traceability of actions carried out.
• Set up a web application firewall with AWS WAF.
• Use of a CDN for static object traffic, guaranteeing fast, secure content delivery.

Multi-region infrastructure

To reduce latency in certain regions, we have designed and developed a terraform module that makes it easy to provision part of the infrastructure in one (or more) new region(s). 

When a new region is added, a partial replication of the infrastructure is automatically created. This module deploys the essential components needed to enhance the end-customer experience while controlling costs. Key elements include : 

‍

• A read replica of a database located in the main region.
• Part of the API and server that uses this database.
• A DNS update to add new entries using latency-based DNS routing. 

‍

We used the AWS Route 53 latency-based routing service for DNS services, which automatically routes requests to the nearest Loadbalancer (see diagram). This means that, regardless of the end customer's location, when new regions are added, their requests are routed to the region that responds fastest. This improves the overall experience of Weweb users worldwide, and also adds resilience (in the event of a regional outage).

‍

‍

Migration and change management

‍

As part of this migration, a multi-stage approach was proposed, as well as support for WeWeb teams to increase their skills. 

The phased approach to migration aims to reduce risks (financial and security), offer flexibility to adjust technical guidelines as needed, and facilitate a gradual transition for teams

The stages of this project were as follows: 

1. Build and test in a development environment. We started by initializing the new environments with Terraform and set up continuous deployment of this infrastructure in their version control system (VCS).

2. Deployment on staging environment and monitoring set-up. 

3. MEP

‍
As the WeWeb teams were unfamiliar with ECS and Terraform services, a great deal of effort was put into transferring skills to ensure the success of the project, WeWeb's familiarity with the infrastructure and their long-term autonomy. 

To this end, we initiated the build on the dev environment, and then the WeWeb teams took charge of deployments on the staging (dress rehearsal) and production environments. Technofy provided technical support during migrations to these different environments. 

To this were added skills transfer sessions during phase 1, and a period of skills reinforcement between phases 1 and 2, including practical exercises on concrete cases from their infrastructure. 

‍

Avantages  
‍

• Save time and simplify infrastructure management: The implementation of Infrastructure as Code (IaC) and CI/CD pipelines significantly reduces the time and complexity of IT infrastructure management by automating provisioning, scaling and deployment processes.
  ‍
• Global availability: Thanks to multi-region infrastructure deployment facilitated by the Terraform module, the WeWeb team can deploy new regions in just a few clicks, guaranteeing high availability and reliable service to customers worldwide.
  
  
• Scalability: ECS and ECS Autoscaling automatically adjust the number of container instances in line with changing workloads, ensuring optimum performance and efficient use of resources. 

• Cost savings: Automated provisioning and scaling ensure efficient use of resources, eliminating waste and reducing operational expenses. What's more, the ability to rapidly replicate environments and streamline processes through automation reduces downtime and accelerates deployments. This approach ensures that the infrastructure remains both economical and efficient to meet business requirements.
  
  
• Effective monitoring: Leveraging tools like AWS CloudWatch, WeWeb continuously monitors system metrics, logs and events, enabling the detection and resolution of performance problems and anomalies. This proactive approach to monitoring not only improves uptime and performance, but also optimizes resource utilization.
  ‍
• Enhanced security: By adhering to security best practices, we guarantee robust protection against threats. This includes integrating security policies into the IaC, automating security checks in CI/CD pipelines, enforcing strict access controls and identity management, and WAF service.  These measures provide a secure and resilient environment, protecting both the infrastructure and the data it manages.

‍

Reviews

‍

Marc - CTO WeWeb 

The WeWeb team found this mission to transform our infrastructure extremely rewarding and effective. Initially, our Terraform skills were limited, thanks to the support and effective training provided by Technofy, we were able to quickly gain competence in this critical area.

Technofy provided clear explanations and managed the transition gradually, enabling our team to familiarize themselves with the new infrastructure without difficulty.
The solutions implemented were adapted to our needs in terms of scalability, security and performance.

What's more, the collaboration was positive on a human level, with Technofy being very pleasant to work with. Overall, we're very satisfied with the progress and results of this project. We now look forward to managing and deploying our infrastructure on AWS with Terraform with renewed confidence.

Jérôme - Solution Architect Technofy

A particularly interesting and stimulating aspect of the project was the implementation of a multi-region infrastructure. We had the opportunity to design, test and validate this infrastructure in a staging environment. 

For the migration project itself, the technologies we implemented at Weweb were quite similar to those used at our previous customers (ECS, Terraform etc), which minimized the major technical challenges. This enabled us to concentrate our efforts on skills transfer and team support.

This project was an enriching experience, both on a technical and human level, and it was a real pleasure to support the WeWeb team.

‍

‍

‍