Most Challenging Database Questions for 2024

In today’s digital era, databases are crucial for managing and utilizing data effectively. They organize data for easy retrieval, enhance performance, ensure security, and support scalability. Databases enable advanced analytics, drive strategic decision-making, and underpin key applications like ERP and CRM systems. Understanding databases allows professionals to leverage data efficiently, foster innovation, and support business success.

1. You are tasked with optimizing a database that has grown to several terabytes in size. Outline the steps you would take to improve its performance.

Optimizing a large database involves several key steps:

1. Assess Current Performance:

• Identify Bottlenecks: Use performance monitoring tools to pinpoint slow queries, high latency, and resource-intensive operations.
• Analyze Query Execution Plans: Examine how queries are executed to identify inefficiencies.

2. Index Optimization:

• Review and Update Indexes: Ensure indexes are being used effectively. Remove redundant indexes and create new ones based on query patterns.
• Partitioned Indexes: For very large tables, consider partitioned indexes to improve performance.

3. Database Schema Optimization:

• Normalization/Denormalization: Adjust the level of normalization based on performance needs. Sometimes denormalizing certain tables can speed up read operations.
• Data Types and Constraints: Ensure appropriate data types and constraints are used to optimize storage and performance.

4. Query Optimization:

• Rewrite Inefficient Queries: Optimize SQL queries to reduce complexity and improve execution time.
• Use Stored Procedures: Where applicable, use stored procedures for frequently executed queries.

5. Data Partitioning:

• Horizontal Partitioning: Divide large tables into smaller, more manageable pieces based on a key (e.g., date ranges).
• Vertical Partitioning: Split a table into smaller tables based on columns, useful when certain columns are frequently accessed together.

6. Implement Caching:

• Query Caching: Cache results of frequently executed queries.
• Application-Level Caching: Use caching layers in the application to reduce the load on the database.

7. Resource Allocation:

• Increase Hardware Resources: Allocate more CPU, memory, and storage if the current resources are insufficient.
• Load Balancing: Distribute database load across multiple servers to improve performance.

8. Regular Maintenance:

• Index Rebuilding/Reorganizing: Regularly rebuild or reorganise indexes to maintain performance.
• Update Statistics: Ensure database statistics are up-to-date to help the query optimizer make better decisions.

Scenario Example: For an e-commerce application experiencing slowdowns during peak shopping seasons, the following steps could be taken:

• Identifying bottlenecks revealed that the most resource-intensive operations were search queries on the product table.
• Index optimization involved creating composite indexes on frequently searched columns like product_name and category.
• Partitioning the product table by category helped reduce the size of each partition, thus speeding up queries.
• Caching frequently viewed products at the application level reduced the database load.

2. Explain the concept of concurrency control in databases. How would you ensure data consistency in a multi-user environment?

Concurrency control in databases ensures that multiple transactions can occur simultaneously without leading to data inconsistency. It aims to maintain the ACID properties (Atomicity, Consistency, Isolation, Durability) of transactions.

Ensuring Data Consistency in a Multi-User Environment:

1. Locking Mechanisms:

• Pessimistic Locking: Locks data when a transaction starts, preventing other transactions from modifying it until the lock is released.
• Optimistic Locking: Assumes conflicts are rare and only checks for conflicts at the end of the transaction.

2. Isolation Levels:

• Read Uncommitted: Allows dirty reads, which can lead to inconsistency.
• Read Committed: Prevents dirty reads by ensuring a transaction reads only committed data.
• Repeatable Read: Ensures a transaction can read the same data multiple times without seeing changes made by other transactions.
• Serializable: The highest isolation level, ensuring full consistency by completely isolating transactions.

3. Multiversion Concurrency Control (MVCC):

• Versioning: Maintains multiple versions of data to allow concurrent reads and writes without locking. Transactions see a consistent snapshot of the database.

4. Deadlock Detection and Resolution:

• Deadlock Detection Algorithms: Regularly check for deadlocks and resolve them by aborting one or more transactions.

Scenario Example: In a banking application, ensuring data consistency is crucial:

• Pessimistic locking could be used for high-value transactions to prevent double-spending.
• Repeatable Read isolation level could be employed to ensure account balances remain consistent during a transaction.

3. Describe the concept of database sharding and how it can be implemented to improve scalability.

Database sharding is the process of dividing a large database into smaller, more manageable pieces called shards. Each shard contains a subset of the data and operates independently, improving both performance and scalability.

Implementation Steps:

1. Identify Sharding Key:

• Sharding Key Selection: Choose a key that distributes data evenly across shards, such as user_id or region_id.

2. Design Shard Schema:

• Horizontal Sharding: Split data across multiple tables based on rows. For example, users with IDs 1–1000 go to Shard 1, 1001–2000 go to Shard 2, etc.
• Vertical Sharding: Split data based on columns. For instance, separate frequently accessed columns from less frequently accessed ones.

3. Implement Shard Routing:

• Application Logic: Incorporate logic in the application to route queries to the appropriate shard based on the sharding key.
• Shard Map: Maintain a map to keep track of which data resides in which shard.

4. Handle Shard Operations:

• Distributed Queries: Ensure queries that span multiple shards are handled efficiently.
• Resharding: Plan for data redistribution when a shard grows too large.

Scenario Example: For a social media application with millions of users, sharding by user_id ensures that user data is evenly distributed:

• Shard 1: Users 1–1,000,000
• Shard 2: Users 1,000,001–2,000,000
• Shard 3: Users 2,000,001–3,000,000

The application uses the user_id to determine which shard to query for user data.

4. Discuss different strategies for data partitioning in a distributed database system. When would you use each strategy?

Data partitioning involves dividing a large dataset into smaller, more manageable pieces to improve performance and manageability. The main strategies include:

1. Range Partitioning:

• Description: Data is partitioned based on a range of values.
• Use Case: Effective when queries often filter by a range of values, such as date ranges in a time-series database.
• Example: Orders table partitioned by order_date.

2. Hash Partitioning:

• Description: Data is distributed based on a hash function applied to the partition key.
• Use Case: Suitable when data needs to be evenly distributed without a natural range key.
• Example: User data partitioned by hashing user_id.

3. List Partitioning:

• Description: Data is divided based on a predefined list of values.
• Use Case: Useful when data can be grouped into distinct categories.
• Example: Customer data partitioned by region (e.g., North, South, East, West).

4. Composite Partitioning:

• Description: Combines two or more partitioning methods.
• Use Case: When a single partitioning strategy isn’t sufficient to meet performance needs.
• Example: Orders table first partitioned by region (list partitioning) and then by order_date (range partitioning).

Scenario Example: For an e-commerce platform:

• Range Partitioning could be used on the orders table by order_date to speed up queries for orders within specific date ranges.
• Hash Partitioning could be used on the users table by user_id to ensure even distribution of user data across partitions.

5. Explain the purpose of database replication. What are the different types of replication and their advantages?

Database replication is the process of copying data from one database to another to ensure data redundancy and improve availability, reliability, and performance.

Types of Replication:

1. Master-Slave Replication:

• Description: One master database handles all write operations, and one or more slave databases replicate the master’s data for read operations.
• Advantages:
• Improved Read Performance: Slaves handle read queries, reducing the load on the master.
• Data Redundancy: Slaves serve as backups of the master.
• Use Case: Web applications with a high read-to-write ratio.

2. Master-Master Replication:

• Description: Multiple masters handle read and write operations, and data is synchronized between them.
• Advantages:
• High Availability: Multiple masters ensure continuous availability.
• Scalability: Write operations can be distributed across multiple masters.
• Use Case: Distributed systems requiring high availability and low latency.

3. Synchronous Replication:

• Description: Data changes are replicated to the standby databases immediately.
• Advantages:
• Consistency: Ensures data consistency across replicas.
• Use Case: Financial systems requiring strict consistency.

4. Asynchronous Replication:

• Description: Data changes are replicated with a delay.
• Advantages:
• Performance: Lower latency for write operations as they don’t wait for replication.
• Use Case: Applications where eventual consistency is acceptable.

Scenario Example: For an online store:

• Master-Slave Replication can be used to handle high read traffic by offloading read operations to slave databases.
• Synchronous Replication can be implemented for financial transactions to ensure all replicas have consistent data immediately.

6. How would you ensure data security in a database? Discuss the role of encryption in securing sensitive data.

Ensuring Data Security:

1. Access Control:

• User Authentication: Use strong authentication mechanisms to ensure only authorized users can access the database.
• Role-Based Access Control (RBAC): Assign roles with specific permissions to users based on their job responsibilities.

2. Data Encryption:

• Encryption at Rest: Encrypt data stored on disk to protect it from unauthorized access if the storage medium is compromised.
• Encryption in Transit: Encrypt data transmitted between the database and clients using protocols like SSL/TLS.
• Column-Level Encryption: Encrypt sensitive data at the column level within the database.

3. Auditing and Monitoring:

• Activity Logging: Keep detailed logs of database activities to detect and respond to suspicious actions.
• Regular Audits: Perform regular security audits to identify and address vulnerabilities.

4. Secure Configuration:

• Database Hardening: Apply best practices to secure database configurations and disable unnecessary services.
• Patch Management: Regularly apply security patches to database software to protect against known vulnerabilities.

5. Backup Security:

• Encrypted Backups: Ensure backups are encrypted to protect data during storage and transfer.
• Access Control: Restrict access to backup files to authorized personnel only.

Scenario Example: For a healthcare database storing patient records:

• Encryption at Rest ensures that sensitive patient data remains secure even if the storage media is compromised.
• RBAC is used to ensure that only authorized medical staff can access patient records.
• Regular audits and activity logging help detect and respond to unauthorized access attempts.

7. Outline a comprehensive backup and recovery strategy for a large-scale database system.

Backup and Recovery Strategy:

1. Backup Types:

• Full Backups: Capture the entire database, typically performed weekly.
• Incremental Backups: Capture only the changes since the last backup, performed daily.
• Differential Backups: Capture changes since the last full backup, performed more frequently (e.g., every few hours).

2. Backup Schedule:

• Daily Incremental: Perform incremental backups daily to minimize storage and backup time.
• Weekly Full: Perform full backups weekly to ensure a comprehensive recovery point.
• Frequent Differential: Perform differential backups every few hours to reduce the recovery window.

3. Storage Solutions:

• On-Site and Off-Site: Store backups both on-site for quick recovery and off-site for disaster recovery.
• Cloud Storage: Utilize cloud storage solutions for scalable and reliable backup storage.

4. Backup Verification:

• Regular Testing: Periodically test backups to ensure they can be restored successfully.
• Automated Verification: Implement automated checks to validate backup integrity.

5. Recovery Procedures:

• Disaster Recovery Plan: Develop and document a detailed plan outlining steps to recover the database in case of a failure.
• Recovery Testing: Conduct regular recovery drills to ensure the team is familiar with the process and can execute it efficiently.

6. Data Retention Policies:

• Retention Period: Define and implement policies for how long backups should be retained based on regulatory and business requirements.
• Archival Solutions: Use archival solutions for long-term storage of historical data.

Scenario Example: For a financial institution with a critical database:

• Full backups are performed every Sunday night.
• Incremental backups are scheduled daily at midnight.
• Differential backups are taken every six hours.
• Backups are stored on-site on a dedicated backup server and off-site in a secure cloud storage solution.
• Quarterly recovery drills are conducted to ensure the team can restore the database in case of a failure.

8. What techniques would you use to identify and resolve performance bottlenecks in a database?

Identifying and Resolving Performance Bottlenecks:

1. Monitoring Tools:

• Database Performance Monitoring: Use tools like SQL Server Profiler, AWS CloudWatch, or Azure Monitor to track performance metrics.
• Query Analysis: Analyze slow queries using execution plans and query performance insights.

2. Index Optimization:

• Missing Indexes: Identify and create missing indexes that could speed up query performance.
• Unused Indexes: Remove indexes that are not being used to reduce overhead.
• Index Maintenance: Regularly rebuild and reorganize indexes to maintain their effectiveness.

3. Query Optimization:

• Rewrite Inefficient Queries: Refactor poorly performing queries to improve efficiency.
• Stored Procedures: Use stored procedures for complex queries to reduce overhead.

4. Resource Allocation:

• Increase Hardware Resources: Add more CPU, memory, or storage if current resources are insufficient.
• Load Balancing: Distribute the load across multiple servers to avoid overloading a single server.

5. Database Configuration:

• Configuration Tuning: Adjust database configuration settings like memory allocation, cache sizes, and connection limits based on workload requirements.
• Connection Pooling: Implement connection pooling to reduce the overhead of establishing new connections.

6. Partitioning:

• Table Partitioning: Split large tables into smaller, more manageable partitions to improve query performance.
• Index Partitioning: Partition indexes to enhance performance for large tables.

Scenario Example: For an online retail application experiencing slow search queries:

• Monitoring tools identified that the most resource-intensive operations were search queries on the product table.
• Query analysis revealed that missing indexes on frequently searched columns were causing the slowdown.
• Index optimization involved creating composite indexes on columns like product_name and category.
• Resource allocation was adjusted by adding more CPU and memory to the database server.

9. How would you handle schema changes in a database that is actively used by multiple applications?

Handling Schema Changes:

1. Planning:

• Impact Analysis: Assess the impact of schema changes on existing applications, queries, and reports.
• Versioning: Implement database versioning to keep track of changes and rollback if necessary.

2. Communication:

• Stakeholder Communication: Inform all stakeholders, including developers, testers, and users, about the planned changes and their impact.
• Documentation: Update documentation to reflect the schema changes.

3. Deployment Strategy:

• Backward Compatibility: Ensure schema changes are backward compatible to prevent breaking existing applications.
• Rolling Updates: Apply changes in a phased manner, updating the schema gradually to minimize downtime.
• Blue-Green Deployment: Use blue-green deployment strategies to switch traffic between old and new versions of the database.

4. Testing:

• Staging Environment: Test schema changes in a staging environment that mirrors production.
• Regression Testing: Conduct thorough regression testing to ensure that existing functionality is not broken.

5. Migration Tools:

• Database Migration Tools: Use tools like Liquibase, Flyway, or Entity Framework Migrations to manage schema changes and automate deployment.

Scenario Example: For a CRM application needing to add new columns to the customers table:

• Impact analysis identified that several reports and queries would be affected.
• Communication was sent to all developers and users informing them of the upcoming changes.
• Backward compatibility ensured the existing applications continued to function by making non-breaking changes.
• Testing was conducted in a staging environment with thorough regression testing.
• Blue-green deployment strategy was used to switch from the old schema to the new schema without downtime.

10. Discuss the importance of data archiving in a database system. How would you implement an archiving strategy?

Importance of Data Archiving:

• Performance Improvement: Archiving old, rarely accessed data can improve query performance on the active dataset.
• Cost Savings: Storing archived data on cheaper storage solutions can reduce costs.
• Regulatory Compliance: Archiving ensures data is retained for the required duration to meet legal and regulatory requirements.
• Data Management: Simplifies data management by reducing the size of the active dataset.

Implementing an Archiving Strategy:

1. Identify Data to Archive:

• Data Classification: Classify data based on access patterns, business requirements, and regulatory needs.
• Archival Criteria: Define criteria for identifying data to archive, such as records older than a certain date.

2. Archiving Process:

• Automated Archiving: Implement automated processes to move data to the archive based on the defined criteria.
• Data Movement: Use ETL (Extract, Transform, Load) tools to move data from the active database to the archive.

3. Storage Solutions:

• On-Premises Storage: Use cost-effective on-premises storage solutions for archived data.
• Cloud Storage: Leverage cloud storage services like AWS Glacier, Azure Archive Storage for scalable and durable archival storage.

4. Access to Archived Data:

• Read-Only Access: Ensure archived data is available in a read-only format to prevent modifications.
• Data Retrieval: Implement efficient data retrieval processes to access archived data when needed.

5. Data Retention and Deletion:

• Retention Policies: Define data retention policies to determine how long archived data should be retained.
• Data Deletion: Implement automated processes to delete archived data after the retention period expires.

Scenario Example: For a financial institution needing to archive transaction records older than seven years:

• Archival criteria identified transaction records older than seven years for archiving.
• Automated processes were implemented to move these records to AWS Glacier.
• Read-only access was provided to ensure compliance with regulatory requirements.
• Retention policies ensured records were retained for the required period before being automatically deleted.

11. Discuss the challenges and approaches to data modeling for multi-tenant applications, where multiple clients share the same database instance.

Challenges:

1. Data Isolation:

• Ensuring each tenant’s data is isolated and cannot be accessed by other tenants.

2. Performance:

• Balancing the performance needs of multiple tenants, especially when they have different workloads.

3. Scalability:

• Ensuring the database can scale to accommodate a growing number of tenants.

4. Security:

• Implementing robust security measures to protect tenant data.

Approaches:

1. Separate Databases:

• Each tenant has its own database.
• Pros: Maximum data isolation, easier to manage backups and restores.
• Cons: Higher resource usage, more complex to manage at scale.

2. Shared Database, Separate Schemas:

• Each tenant has its own schema within a shared database.
• Pros: Good balance of isolation and resource usage.
• Cons: More complex to manage, especially with a large number of schemas.

3. Shared Database, Shared Schema:

• All tenants share the same schema, with tenant data differentiated by a tenant identifier.
• Pros: Efficient resource usage, easier to manage at scale.
• Cons: More complex data isolation, potential for noisy neighbor issues.

4. Hybrid Approach:

• Combine multiple approaches based on tenant size or requirements.
• Pros: Flexibility to optimize for different tenant needs.
• Cons: Increased complexity in management and implementation.

Scenario Example: For a SaaS application serving small businesses:

• Shared database with separate schemas can be used for better data isolation while keeping resource usage efficient.
• Data isolation is ensured by using schema-level permissions.
• Scalability is managed by monitoring schema performance and distributing load across multiple servers if needed.

12. How does data modeling differ when designing databases for real-time analytics compared to traditional batch processing? Provide examples.

Real-Time Analytics:

1. Data Ingestion:

• Stream Processing: Continuous data ingestion from sources like IoT devices, logs, or online transactions.
• Examples: Apache Kafka, AWS Kinesis.

2. Schema Design:

• Event-Based Models: Use event-based models to capture real-time changes and updates.
• Time-Series Data: Optimized for time-series data with high-frequency updates.

3. Performance Requirements:

• Low Latency: Focus on low-latency data access and processing to provide real-time insights.
• Examples: Real-time dashboards, fraud detection systems.

4. Data Storage:

• In-Memory Databases: Use in-memory databases for fast access and processing.
• Examples: Redis, Apache Ignite.

Traditional Batch Processing:

1. Data Ingestion:

• Batch Processing: Data is ingested in bulk at scheduled intervals (e.g., daily, hourly).
• Examples: ETL jobs, data warehousing processes.

2. Schema Design:

• Star/Snowflake Schema: Use star or snowflake schemas for organizing data in data warehouses.
• Denormalization: Often involves denormalizing data to optimize for read performance.

3. Performance Requirements:

• Throughput: Focus on high throughput and processing large volumes of data.
• Examples: Monthly financial reports, customer segmentation analysis.

4. Data Storage:

• Disk-Based Storage: Use traditional disk-based storage solutions optimized for large data volumes.
• Examples: Amazon Redshift, Google BigQuery.

Scenario Examples:

• Real-Time Analytics: A ride-sharing application requires real-time data to match drivers with passengers, using stream processing and in-memory databases for low-latency performance.
• Traditional Batch Processing: A retail company generates weekly sales reports by ingesting sales data in batches, using a data warehouse with a star schema.

13. Describe the challenges and considerations in data modelling for GIS applications. How do spatial databases handle geographic data?

Challenges and Considerations:

1. Complex Data Types:

• Spatial Data Types: Support for complex data types like points, lines, and polygons.
• Examples: Representing geographical features, routes, and boundaries.

2. Performance:

• Spatial Indexing: Efficient indexing mechanisms to handle spatial queries.
• Examples: R-trees, Quad-trees.

3. Accuracy:

• Precision and Scale: Ensuring accuracy in representing geographic coordinates and distances.
• Examples: Handling different coordinate systems and projections.

4. Querying:

• Spatial Queries: Supporting spatial queries like distance calculations, containment, and intersection.
• Examples: Finding nearby points of interest, determining if a location is within a boundary.

5. Storage:

• Large Data Volumes: Handling large volumes of spatial data efficiently.
• Examples: Storing satellite imagery, detailed maps.

Handling Geographic Data:

1. Spatial Data Types:

• Points: Represent specific locations (e.g., latitude and longitude).
• Lines: Represent paths or routes (e.g., roads, rivers).
• Polygons: Represent areas (e.g., cities, regions).

2. Spatial Indexing:

• R-Trees: Hierarchical index structure for spatial objects.
• Quad-Trees: Divide space into quadrants for efficient querying.

3. Spatial Functions:

• Distance Calculations: Calculate the distance between two geographic points.
• Containment Queries: Check if a point is within a polygon.

4. GIS-Specific Databases:

• PostGIS: An extension for PostgreSQL that provides support for geographic objects.
• Oracle Spatial: Oracle’s solution for managing spatial data.

Scenario Example: For a city planning application:

• Spatial data types are used to represent buildings (points), roads (lines), and districts (polygons).
• Spatial indexing with R-trees helps efficiently query distances between buildings and roads.
• PostGIS is used to handle spatial data and perform complex geographic queries.

14. Explain the concept of database locking and its role in ensuring data consistency. Discuss different types of locks and their compatibility.

Database Locking:

Role in Ensuring Data Consistency:

• Prevent Conflicts: Locks prevent multiple transactions from conflicting with each other by controlling access to data. This helps maintain the integrity and consistency of the data.
• Serialization: Locks ensure that transactions are serialized in a way that the final outcome is the same as if the transactions had been executed sequentially.

Types of Locks:

1. Shared Lock (S):

• Description: Allows multiple transactions to read a resource but not modify it.
• Compatibility: Compatible with other shared locks but not with exclusive locks.
• Use Case: When a transaction needs to read data without making changes.

2. Exclusive Lock (X):

• Description: Allows a transaction to read and modify a resource.
• Compatibility: Not compatible with any other locks (shared or exclusive).
• Use Case: When a transaction needs to update or delete data.

3. Intent Locks:

• Intent Shared (IS): Indicates intention to read a resource.
• Intent Exclusive (IX): Indicates intention to modify a resource.
• Purpose: Used to manage locks at different levels of granularity (e.g., table-level vs. row-level).
• Use Case: Used to signal the type of lock a transaction intends to acquire on a lower-level resource.

4. Update Lock (U):

• Description: Intermediate lock that allows a transaction to read a resource and prepares to upgrade to an exclusive lock.
• Compatibility: Compatible with shared locks but not with exclusive locks.
• Use Case: When a transaction intends to update data after reading it.

Lock Compatibility

Scenario Example: In an e-commerce application:

• Shared locks are used for read operations on product details to allow multiple users to view the same product simultaneously.
• Exclusive locks are used during checkout to ensure that only one user can modify the stock quantity at a time, preventing overselling.

15. How can database caching improve performance? Discuss different caching strategies and their implementation.

Database Caching:

Performance Improvement:

• Reduced Latency: Caching frequently accessed data in memory reduces the time required to retrieve it from disk.
• Lower Load: Reduces the load on the database by serving repeated queries from the cache instead of querying the database directly.
• Improved Throughput: Frees up database resources to handle more queries, improving overall throughput.

Caching Strategies:

1. Read-Through Cache:

• Mechanism: The application reads from the cache first; if the data is not present, it reads from the database and populates the cache.
• Implementation: Use a caching layer (e.g., Redis, Memcached) to store frequently accessed data. The application first checks the cache, and only if the data is not found, it queries the database and updates the cache.
• Use Case: Useful for read-heavy applications where data changes infrequently.

2. Write-Through Cache:

• Mechanism: Data is written to the cache and the database simultaneously.
• Implementation: When the application writes data, it updates both the cache and the database to ensure consistency.
• Use Case: Suitable for applications where data consistency between the cache and database is critical.

3. Write-Behind (Write-Back) Cache:

• Mechanism: Data is written to the cache first and then asynchronously written to the database.
• Implementation: The cache updates the database in the background, which can improve write performance but may introduce a risk of data loss if the cache fails.
• Use Case: Useful for write-heavy applications where write performance is a priority, and eventual consistency is acceptable.

4. Cache-Aside (Lazy Loading):

• Mechanism: The application directly queries the cache; if the data is not present, it loads the data from the database and updates the cache.
• Implementation: The application logic handles the caching. When a cache miss occurs, the application fetches data from the database and caches it for future use.
• Use Case: Flexible and commonly used strategy where the application manages cache population.

Scenario Example: For a social media application:

• Read-Through Cache: Used to cache user profiles and posts to reduce read latency and offload the database.
• Write-Through Cache: Ensures that updates to user profiles are immediately reflected in both the cache and database to maintain consistency.
• Cache-Aside: Used for infrequently accessed data, where the application fetches data from the database on a cache miss and populates the cache for future requests.