🎯Introduction
MongoDB is a popular NoSQL database known for its scalability and flexibility. However, it can also be expensive to run, especially if you’re not careful about how you use it. This blog post will discuss some hidden secret tips for MongoDB cost savings and some examples.
🎯Optimize Memory Usage and Avoid Fragmentation
Memory is a crucial resource for MongoDB, and inefficient memory usage can lead to increased costs. One common issue is memory fragmentation, which occurs when allocated memory becomes unusable, even though there is enough free memory available.
Example:
Consider a scenario where we had a MongoDB cluster with 8GB of total memory. Due to a resource-intensive aggregation pipeline execution, memory consumption spiked to 60%, resulting in around 60% of the memory becoming fragmented (blocked and unusable). This left only 40% of usable memory.
The problem arose when the cluster’s auto-scale up service would trigger a tier scale up to a more expensive one (M30 to M40) whenever memory consumption exceeded 15% of the remaining 40%. This was puzzling because the auto-scale up trigger was set at a threshold of 75% memory usage.
Upon investigation, it was discovered that the auto-scale up service was considering both the fragmented memory (60%) and the actual used memory (15%) when calculating the total memory consumption. As a result, the perceived usage was hitting 75%, causing the cluster to scale up unnecessarily.
Cost Calculation:
Assuming the M30 tier costs $400 per month and the M40 tier costs $600 per month, the unnecessary scaling would cost an extra $200 per month.
Solution and Savings:
The solution was to contact the MongoDB team to defragment the memory, which brought the usable memory back to its full capacity. Consequently, the cluster scaled down to the M30 tier, saving a significant amount of money.
By defragmenting the memory, the monthly cost was reduced from $600 to $400, resulting in a saving of $200 per month.
Lesson Learned:
1.Monitor memory usage regularly and identify fragmentation issues early on.
2.Seek support from the MongoDB team to reclaim fragmented memory and optimize usage.
🎯Reclaim Disk Space After Data Purges
When you delete a large amount of data from your MongoDB database, the disk space occupied by that data is not immediately released. This can lead to wasted storage space and increased costs.
Example:
There was a situation where we purged a significant amount of data, around 64GB, from a single collection to move it to an archive or cold storage layer. You would expect the storage stats to reflect the freed-up space, but surprisingly, the stats remained unchanged.
Before data purge, disk space stats (used/total): 64GB / 128GB
After data purge, disk space stats (used/total): 64GB / 128GB
Cost Calculation:
Assuming the storage cost is $0.10 per GB per month, the 64GB of unclaimed space would cost $6.40 per month.
Solution and Savings:
After consulting with the MongoDB team, it was revealed that MongoDB does not automatically release disk space after data purges. Instead, it keeps the space allocated and reuses it for future data insertions.
The solution is to use the db.runCommand({compact: “<collection_name>”}) command to reclaim disk space after data purges. This command will compact the collection and release the unused disk space.
By reclaiming the 64GB of unused space, the monthly cost would be reduced by $6.40.
Lesson Learned:
1.Use the db.runCommand({compact: “<collection_name>”}) command to reclaim disk space after data purges.
2.Schedule regular database repairs using MongoDB’s Cloud Manager.
🎯Utilize MongoDB Compression for Storage and Network
MongoDB offers compression options for both storage and network traffic, which can significantly reduce the size of your data and lower your storage and bandwidth costs.
a. MongoDB Storage Compression
Storage compression can be enabled at the collection level. When enabled, MongoDB will compress data before storing it on disk, reducing the amount of storage space required.
Example:
Assume a collection of documents containing text-based data occupies 128GB of storage space. By enabling storage compression, you can achieve a compression ratio of 50%, reducing the storage requirement to 50GB.
Cost Calculation:
Assuming the storage cost is $0.10 per GB per month, the uncompressed data would cost $10 per month, while the compressed data would cost $5 per month.
Savings:
By enabling storage compression, you can save $5 per month on storage costs for this collection.
b. MongoDB Network Compression
Network compression can be enabled for client-server communication, compressing data before sending it over the network. This can reduce bandwidth usage and improve performance, especially for applications that transfer large amounts of data.
Example:
Consider an application that transfers 1GB of data between the client and the MongoDB server every hour. By enabling network compression with a compression ratio of 50%, you can reduce the data transfer to 500MB per hour.
Cost Calculation:
Assuming the bandwidth cost is $0.05 per GB per month, the uncompressed data transfer would cost $3.60 per month, while the compressed data transfer would cost $1.80 per month.
Savings:
By enabling network compression, you can save $1.80 per month on bandwidth costs for this application.
Lesson Learned:
1.Enable storage compression to reduce storage costs.
2.Enable network compression to lower bandwidth costs and improve performance.
By implementing these hidden secret tips, you can optimize your MongoDB deployment and achieve significant cost savings. Remember to monitor your MongoDB usage regularly and make adjustments as needed to ensure you are utilizing resources efficiently and keeping costs under control.
