When to use Amazon RDS over Amazon Redshift

Amazon RDS and Amazon Redshift. They are both managed solutions that AWS offers for working with data. Where RDS is optimized for OLTP (Online Transaction Processing: transaction-oriented applications, where for example each row might represent a purchase made by a customer), Amazon Redshift is optimized for OLAP (Online Analytical Processing: asking multi-dimensional analytical questions, such as what item was purchased the most last Tuesday?).

Redshift’s model is certainly the right choice when trying to set up a Data Warehouse for teams with extremely large quantities of data that need to run analytics and BI queries on a frequent basis. Redshift leverages a few key features (columnar data storage, compression, massively parallel processing, etc.) to greatly speed up query time. However, when working with clients, I’ve found that in a few cases, leveraging RDS as an analytics tool is preferable to setting up a separate Redshift cluster. How can this be the case, given that Redshift’s analytics query performance far exceeds similar analysis on RDS? I’ll go over a few examples where that is the case.

• If your analytics and BI load is very low. Perhaps you only run a few ad-hoc queries infrequently, or you are able to run relevant queries monthly during off-peak hours. In this case, it might not be worth the business costs (maintenance, expense, adding a new component to your stack) of setting up a separate Data Warehousing solution. While being able to use a managed Data Warehouse service like Amazon Redshift decreases some of the maintenance costs associated with the decision, you will still need to take care of some cluster maintenance and optimization. In addition, the financial cost and time necessary to either hire or show new team members how to use an extra component of your stack is non-negligible.
• If your analytics and BI questions are not actionable and are not directly improving your bottom line. When architecting out solutions, it’s important to remember to only build out services that help out your ultimate business goals. There may be things that you are curious about or that would be interesting to know. However, if these insights wouldn’t lead to certain actions that your team would then take based on this additional data, it is hard to justify the additional business costs of setting up a separate Data Warehousing solution. It’s important to not only have questions that you want to ask about your data, but to also have the processes (a data-driven decision-making system, a data team, etc.) that will enable you to justify the costs of these analytics processes.
• If your analytics and BI load is moderate, but you are satisfied with your analytics query performance. In that case, your analytics needs could, for example, be met by querying a non-production copy of your database. However, if you start to experience a significant lag to your query processes, keep in mind that migrating to a Data Warehouse solution would greatly speed up your queries. It will make sense to migrate to a Data Warehouse solution when the negative effects that you experience from using a standard Relational Database solution outweigh the business costs of setting up a new Data Warehousing solution. This line will be crossed at a different time for every team, depending on the size of the team, the importance of BI and analytics to your bottom line, etc. Make sure that you have a plan to revisit your analytics performance on a regular basis so you know when you have outgrown your existing capacity.

What is the exception to these cases, though?

• When analytics is key to your bottom line, or you know that you will shortly have enough data to require a data warehouse. If you are a new company or team that is building out a product that will need to run analytics processes often in order to meet its business goals, it makes sense to start architecting that out from the beginning. There is no point in using RDS for your analytics needs, only to have to switch over all of your processes to Redshift two months later, once your data has grown enough. It’s important to be honest when making a decision to build out for future capacity, though: are you sure that this is necessary? With a cloud-based Data Warehouse like Redshift, the setup and migration costs are decreased, so it will not be as painful as it would have been a few years back to revisit your decision at a future date.

When architecting out solutions, it’s important to not just use the best tool for the problem at hand, but to keep in mind the best solution for the team that will be working with these tools.

Found this blog post useful? If you want to increase your productivity on AWS, check out our Cloud Architecture Dashboard. Visualize, gain insight, reduce costs, and audit your AWS Architecture using our dashboard. We provide useful visualizations of your AWS architecture to enable your data team to be more productive by understanding, troubleshooting, and monitoring your data infrastructure visually.

Horos Solutions provides Big Data and Solutions Architect consulting. Interested in finding out how we can help you leverage the value in your data? Contact us at contact [at] horossolutions [dot] com.