Enhancing Awareness and Optimizing Cloud Infrastructure Costs through Cloud Auto Management Reporting and Analytics.

Dwiki Kurnia
Life at Telkomsel
Published in
3 min readFeb 6, 2024

--

For small organizations, monitoring, cost awareness, and cost optimization might be easy tasks. However, this is not the case for large organizations with dispersed accounts and applications. Sometimes, we are unaware of what is happening, both in terms of usage and cost. Additionally, we need to identify what can be done for cost optimization. With the advent of Cloud Auto Management Reporting and Analytics, we are excited to introduce a transformative approach to this pain point.

We combine several AWS services to generate this solution, utilizing AWS Lambda with the Serverless Application Model, Amazon RDS, Athena, S3, and CloudWatch Synthetics. The sources we utilize include AWS Cost & Usage Report for cloud cost and CloudWatch Metric for several metrics that need consideration.

High-Level Architecture Cloud Auto Management Reporting and Analytics

Implementing Cloud Auto Management Reporting and Analytics brings some benefits:

1. Knowing trend cost usage and traffic: Gain insights into cost trends and traffic patterns over time, allowing for better forecasting and the right decisions.

2. Improving awareness of Cloud Cost usage to be optimized: Enhance understanding of cloud cost usage and identify areas where optimization can lead to cost savings, ultimately improving cost efficiency.

3. Flexibility in Analytics and Visualization, Possible Advanced Analytics Implementation: Enjoy the flexibility of analyzing and visualizing data in various ways, with the potential to implement advanced analytics techniques for deeper insights and optimization strategies.

4. Connect other Accounts with no Cost: Seamlessly integrate additional accounts at no extra cost, providing a comprehensive view of cloud usage across the organization without incurring additional expenses.

5. Possible to integrate with Cloud Observability: Integrate with cloud observability tools to enhance monitoring capabilities and gain insight.

6. Easy to read on both mobile and desktop devices: Access reports and analytics effortlessly on both mobile and desktop devices, ensuring convenience and accessibility for users across different platforms and locations.

Below is a sample of the data visualization and Telegram broadcast output:

Sample Data Visualization & Telegram Broadcast Output (Blurred edition)

Executive Summary Weekly & Monthly Cost and Traffic ALB & CloudFront:

  • This section provides a concise overview of the weekly and monthly costs and traffic of Application Load Balancers (ALB) and CloudFront. It highlights any notable trends or fluctuations using a red & green background.

Top Cost & Bottom Contributor Weekly & Monthly:

  • Here, you will find insights into the top contributors to costs as well as the bottom performers, both on a weekly and monthly basis. This helps prioritize optimization efforts and identify areas for potential cost savings, as well as being aware of the contributor.

Top Product Cost Weekly & Monthly:

  • This section outlines the products or services that incur the highest costs on a weekly and monthly basis.

The dashboard is captured by using Selenium, and the picture is automatically sent to Telegram. Additionally, an auto-generated text summary is provided based on trends and growth for AWS services and accounts that require attention. This approach streamlines the process of monitoring and managing cloud resources, facilitating informed decision-making and cost-optimization strategies.

OpenAI API Integration with Cloud Cost

Another feature, users can interact with the bot using natural language, just as they would with another human. The bot leverages AI capabilities to generate new insights from provided data. For example, if a user asks about trends in their cloud spending over the past few months, the bot can analyze the data and provide insights such as identifying top cost growth or highlighting areas that need to be considered.

Lastly, this solution relies on the user. If users can use these tools effectively, it can make a big difference. But if users don’t keep an eye on things, it would not help much.

--

--