Embracing the Cloud Locally: Elevating Java Spring Projects through LocalStack for AWS Services Testing
As Java Spring developers, testing interactions with AWS services, especially within the context of AWS Lambda function handlers, can be a challenging aspect of our projects. LocalStack comes to the rescue by providing a powerful solution for testing AWS cloud services locally, eliminating the need for a live AWS environment. In this comprehensive guide, we’ll delve into the intricacies of setting up and effectively using LocalStack for testing AWS Lambda function handlers with SQS events in a Java Spring project.
Understanding LocalStack
What is LocalStack?
https://www.localstack.cloud/
LocalStack stands out as a lightweight, self-contained AWS cloud stack designed specifically for local development and testing. It presents a fully functional local environment that mirrors the behavior of AWS cloud services. This makes LocalStack an ideal choice for developers seeking to validate their applications’ interactions with AWS services without the overhead of deploying to an actual AWS environment.
Prerequisite: Docker Installation for Linux Ubuntu
Before setting up LocalStack, ensure that Docker is installed on your Linux Ubuntu machine. Follow these steps to install Docker:
Update Package Lists: Open a terminal and update the package lists:
sudo apt updateInstall Docker Dependencies: Install packages to allow apt to use a repository over HTTPS:
sudo apt install apt-transport-https ca-certificates curl software-properties-commonAdd Docker’s Official GPG Key: Add Docker’s official GPG key to ensure the integrity of the packages:
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpgSet Up the Stable Docker Repository: Set up the stable Docker repository:
echo "deb [signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/nulInstall Docker Engine: Update the package lists once more, then install the Docker engine:
sudo apt update
sudo apt install docker-ce docker-ce-cli containerd.ioVerify Docker Installation: Verify that Docker is installed correctly by running:
sudo docker --versionYou should see information about the Docker version.
Now that Docker is installed, you’re ready to proceed with LocalStack setup.
How LocalStack Works Under the Hood
LocalStack works by emulating AWS cloud services locally using Docker containers. Here’s an overview of how it operates under the hood:
Docker Containers:
- LocalStack uses Docker containers to encapsulate and emulate various AWS services locally.
- Each AWS service supported by LocalStack is typically represented by a separate Docker container.
Service Endpoints:
- LocalStack exposes service endpoints for each emulated AWS service. These endpoints mimic the behavior of the corresponding AWS services.
- For example, there are endpoints for local DynamoDB, SQS, S3, etc.
Java SDK Compatibility:
- LocalStack is compatible with the AWS SDK for Java, allowing Java applications to interact with emulated AWS services using the standard Java SDK.
Service Initialization:
- When LocalStack starts, it initializes the Docker containers for the emulated AWS services. This involves setting up configurations and data storage.
AWS Service Emulation:
- LocalStack intercepts and handles requests made to its service endpoints, emulating the behavior of various AWS services.
- For instance, when a Java application makes an API call to the local DynamoDB endpoint provided by LocalStack, it receives a response as if it were a real DynamoDB service.
Data Storage:
- LocalStack creates local storage to mimic the behavior of AWS cloud storage for services like DynamoDB and S3.
- This allows developers to perform CRUD operations using the AWS SDK for Java.
Configuration and Customization:
- LocalStack provides configuration options for customizing its behavior, such as specifying which AWS services to emulate and configuring service endpoints.
Integration with Test Frameworks:
- LocalStack seamlessly integrates with testing frameworks like JUnit, enabling developers to incorporate local AWS service emulation into their test suites.
In summary, LocalStack provides a local environment that closely simulates AWS cloud services through Docker containers, allowing developers to test and validate their applications’ interactions with AWS services without the need for a live AWS environment.
Setting up LocalStack in a Java Spring Project
Adding Dependencies
Before diving into the setup, ensure your project includes the necessary dependencies. Add the following dependencies to your project’s build file, such as pom.xml:
<!-- Dependencies for LocalStack and Testcontainers -->
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>localstack</artifactId>
<version>LATEST_VERSION</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>junit-jupiter</artifactId>
<version>LATEST_VERSION</version>
<scope>test</scope>
</dependency>Replace LATEST_VERSION with the latest version of Testcontainers.
LocalStackSetup Class
Let’s start with the LocalStackSetup class. This class initializes a LocalStack container with specific AWS services, such as DynamoDB, SQS, and Secrets Manager. Here's an example:
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import org.testcontainers.containers.localstack.LocalStackContainer;
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.utility.DockerImageName;import static org.testcontainers.containers.localstack.LocalStackContainer.Service.*;public class LocalstackSetup {
@Container
public static LocalStackContainer localStack =
new LocalStackContainer(DockerImageName.parse("localstack/localstack:latest"))
.withServices(DYNAMODB, SQS, SECRETSMANAGER); static AWSStaticCredentialsProvider localStackCredentialsProvider = new AWSStaticCredentialsProvider
(new BasicAWSCredentials(localStack.getAccessKey(), localStack.getSecretKey()));
}
ComponentTestConfiguration Class
The ComponentTestConfiguration class serves as a test configuration class that sets up beans for testing components. It starts the LocalStack container in a static block and configures beans for DynamoDB, SQS, Secrets Manager, and more. Below is an illustrative snippet:
package com.base.config;
import com.amazonaws.client.builder.AwsClientBuilder;
import com.amazonaws.secretsmanager.caching.SecretCacheConfiguration;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDB;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClientBuilder;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapper;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapperConfig;
import com.amazonaws.services.secretsmanager.AWSSecretsManager;
import com.amazonaws.services.secretsmanager.AWSSecretsManagerClientBuilder;
import com.amazonaws.services.sqs.AmazonSQS;
import com.amazonaws.services.sqs.AmazonSQSClientBuilder;
import com.amazonaws.xray.AWSXRay;
import com.base.client.PalmsClient;
import com.base.component.utils.DynamoDBTestUtils;
import com.base.component.utils.SecretMangerUtils;
import com.base.component.utils.SqsTestUtils;
import org.mockito.Mockito;
import org.springframework.boot.test.context.TestConfiguration;
import org.springframework.context.annotation.Bean;import java.time.Duration;import static com.base.config.LocalstackSetup.localStack;
import static com.base.config.LocalstackSetup.localStackCredentialsProvider;
import static org.testcontainers.containers.localstack.LocalStackContainer.Service.*;@TestConfiguration
public class ComponentTestConfiguration { static {
localStack.start();
} @Bean
public DynamoDBTestUtils dynamoDBTestUtils() {
return new DynamoDBTestUtils();
} @Bean
public DynamoDBMapper dynamoDBMapper(
final AmazonDynamoDB amazonDynamoDB,
final DynamoDBMapperConfig dynamoDBMapperConfig
) {
AWSXRay.beginSegment("AmazonDynamoDBv2");
return new DynamoDBMapper(amazonDynamoDB, dynamoDBMapperConfig);
} @Bean
public AmazonDynamoDB amazonDynamoDB() {
// Configures Amazon DynamoDB with LocalStack endpoint and credentials.
// ...
return AmazonDynamoDBClientBuilder.standard()
.withEndpointConfiguration(new AwsClientBuilder
.EndpointConfiguration(localStack.getEndpointOverride(DYNAMODB).toString(),
localStack.getRegion()))
.withCredentials(localStackCredentialsProvider)
.build();
} @Bean
public AmazonSQS amazonSQS() {
// Configures Amazon SQS with LocalStack endpoint and credentials.
// ...
return AmazonSQSClientBuilder.standard()
.withEndpointConfiguration(new AwsClientBuilder
.EndpointConfiguration(localStack.getEndpointOverride(SQS).toString(),
localStack.getRegion()))
.withCredentials(localStackCredentialsProvider)
.build();
} @Bean
public PalmsClient thirdPartyClient() {
return Mockito.mock(ThirdPartyClient.class);
} @Bean
public AWSSecretsManager awsSecretsManager() {
return AWSSecretsManagerClientBuilder.standard()
.withEndpointConfiguration(new AwsClientBuilder
.EndpointConfiguration(localStack.getEndpointOverride(SECRETSMANAGER).toString(),
localStack.getRegion()))
.withCredentials(localStackCredentialsProvider)
.build();
} @Bean
public SecretCacheConfiguration secretCacheConfiguration(AWSSecretsManager awsSecretsManager) {
return new SecretCacheConfiguration()
.withClient(awsSecretsManager())
.withCacheItemTTL(Duration.ofHours(12).toMillis());
} @Bean
public SqsTestUtils sqsTestUtils() {
return new SqsTestUtils();
} @Bean
public SecretMangerUtils secretMangerUtils() {
return new SecretMangerUtils();
}
}
DynamoDBTestUtils Class
For DynamoDB testing, the DynamoDBTestUtils class offers utility methods for creating and deleting DynamoDB tables during tests. Consider the following example:
package com.base.component.utils;import com.amazonaws.services.dynamodbv2.AmazonDynamoDB;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapper;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapperConfig;
import com.amazonaws.services.dynamodbv2.model.CreateTableRequest;
import com.amazonaws.services.dynamodbv2.model.DeleteTableRequest;
import com.amazonaws.services.dynamodbv2.model.GlobalSecondaryIndex;
import com.amazonaws.services.dynamodbv2.model.Projection;
import com.amazonaws.services.dynamodbv2.model.ProjectionType;
import com.amazonaws.services.dynamodbv2.model.ProvisionedThroughput;import java.util.Map;
import java.util.Objects;/**
* The DynamoDBTestUtils class provides utility methods for creating and deleting DynamoDB tables for testing purposes.
* It allows the creation and deletion of tables specified in a map where the table name is mapped to its corresponding
* class representing the data model.
*/
public class DynamoDBTestUtils {
private void createTables(final Map<String, Class<?>> tables,
final DynamoDBMapper dynamoDBMapper,
final AmazonDynamoDB amazonDynamoDB) {
// Iterate through all the tables and create them if they don't exist
for (String tableName : tables.keySet()) {
// Create the table if it doesn't exist
if (!amazonDynamoDB.listTables().getTableNames().contains(tableName)) {
CreateTableRequest tableRequest = dynamoDBMapper.generateCreateTableRequest(tables.get(tableName),
new DynamoDBMapperConfig.TableNameOverride(tableName).config());
tableRequest.setProvisionedThroughput(new ProvisionedThroughput(5L, 5L)); if (!Objects.isNull(tableRequest.getGlobalSecondaryIndexes())) {
// Iterate through and Specify the Global Secondary Index (GSI) provisioned throughput
for (GlobalSecondaryIndex index : tableRequest.getGlobalSecondaryIndexes()) {
index.setProvisionedThroughput(new ProvisionedThroughput(5L, 5L));
index.setProjection(new Projection()
.withProjectionType(ProjectionType.ALL));
}
}
amazonDynamoDB.createTable(tableRequest);
}
}
} private void deleteTables(final Map<String, Class<?>> tables,
final DynamoDBMapper dynamoDBMapper,
final AmazonDynamoDB amazonDynamoDB) {
// Iterate through all the tables and delete them
for (String tableName : tables.keySet()) {
// Delete the table if it exists
if (amazonDynamoDB.listTables().getTableNames().contains(tableName)) {
DeleteTableRequest deleteTableRequest = dynamoDBMapper.generateDeleteTableRequest(tables.get(tableName));
amazonDynamoDB.deleteTable(deleteTableRequest);
}
}
} public void setupTables(final Map<String, Class<?>> tables,
final DynamoDBMapper dynamoDBMapper,
final AmazonDynamoDB amazonDynamoDB) {
// Delete tables
//deleteTables(tables, dynamoDBMapper, amazonDynamoDB);
// Create tables
createTables(tables, dynamoDBMapper, amazonDynamoDB);
}
}
SecretManagerUtils and SQSTestUtils Classes
Similar to DynamoDB, the SecretManagerUtils class provides utility methods for creating and deleting secrets in AWS Secrets Manager during tests. Meanwhile, the SQSTestUtils class assists in managing SQS queues during tests.
package com.base.component.utils;import com.amazonaws.services.secretsmanager.AWSSecretsManager;
import com.amazonaws.services.secretsmanager.model.CreateSecretRequest;
import com.amazonaws.services.secretsmanager.model.DeleteSecretRequest;
import com.amazonaws.services.secretsmanager.model.ListSecretsRequest;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import lombok.extern.slf4j.Slf4j;import java.util.Map;@Slf4j
public class SecretMangerUtils { private final ObjectMapper objectMapper = new ObjectMapper(); private void createSecret(final AWSSecretsManager awsSecretsManager,
final String secretName,
final Map<String, String> secretValues)
throws JsonProcessingException {
CreateSecretRequest createSecretRequest = new CreateSecretRequest();
createSecretRequest.setName(secretName);
createSecretRequest.setSecretString(objectMapper.writeValueAsString
(secretValues)); awsSecretsManager.createSecret(createSecretRequest);
} private void deleteSecret(final AWSSecretsManager awsSecretsManager,
final String secretName) {
ListSecretsRequest listSecretsRequest = new ListSecretsRequest();
listSecretsRequest.setMaxResults(100);
if(awsSecretsManager.listSecrets(listSecretsRequest).getSecretList().stream()
.anyMatch(secret -> secret.getName().equals(secretName))) {
DeleteSecretRequest deleteSecretRequest = new DeleteSecretRequest();
deleteSecretRequest.setSecretId(secretName);
deleteSecretRequest.setForceDeleteWithoutRecovery(true);
awsSecretsManager.deleteSecret(deleteSecretRequest);
}
} public void setUpSecrets(final AWSSecretsManager awsSecretsManager,
final String secretName,
final Map<String, String> secretValues) {
try {
deleteSecret(awsSecretsManager, secretName);
createSecret(awsSecretsManager, secretName, secretValues);
} catch (Exception e) {
log.error("Error while setting up secrets: {}", e.getMessage());
}
}
}
package com.base.component.utils;import com.amazonaws.services.sqs.AmazonSQS;
import com.amazonaws.services.sqs.model.CreateQueueRequest;
import com.amazonaws.services.sqs.model.DeleteQueueRequest;
import com.base.Constants;
import lombok.extern.slf4j.Slf4j;@Slf4j
public class SqsTestUtils { private void createQueue(final String queueName,
final AmazonSQS amazonSQS) {
// Check if queue does not exist and create queue
if (amazonSQS.listQueues(queueName).getQueueUrls().isEmpty()) {
CreateQueueRequest createQueueRequest = new CreateQueueRequest(queueName);
createQueueRequest.setQueueName(queueName);
amazonSQS.createQueue(createQueueRequest);
}
} private void deleteQueue(final String queueName,
final AmazonSQS amazonSQS) {
// Check if queue exists and delete queue
if (!amazonSQS.listQueues(queueName).getQueueUrls().isEmpty()) {
DeleteQueueRequest deleteQueueRequest = new DeleteQueueRequest();
deleteQueueRequest.setQueueUrl(amazonSQS.listQueues(queueName).getQueueUrls().get(0));
amazonSQS.deleteQueue(deleteQueueRequest);
}
} public void setUpQueue(final String queueName,
final AmazonSQS amazonSQS) {
deleteQueue(queueName, amazonSQS);
createQueue(queueName, amazonSQS);
}}
Utilizing LocalStack in Tests
With the infrastructure in place, let’s now see how to integrate LocalStack into your tests. Consider a test class like ComponentTest, which utilizes LocalStack for testing interactions with AWS services. The following example illustrates this:
@SpringBootTest
@ContextConfiguration(classes = {ComponentTestConfiguration.class})
@Testcontainers
public class ComponentTest {
private static StreamLambdaHandler handler;
private static Context lambdaContext;@Autowired
private DynamoDBMapper dynamoDBMapper;
@Autowired
private AmazonDynamoDB amazonDynamoDB;
@Autowired
private TransactionRepository transactionRepository;
@Autowired
private OrderRepository orderRepository;
@Autowired
private ThirdPartyClient thirdPartyClient; // Assuming a generic third-party client
@Autowired
private DynamoDBTestUtils dynamoDBTestUtils;
@Autowired
private SqsTestUtils sqsTestUtils;
@Autowired
private SecretMangerUtils secretManagerUtils;
@Autowired
private AmazonSQS amazonSQS;
@Autowired
private AWSSecretsManager awsSecretsManager;
@Autowired
private SecretCacheConfiguration secretCacheConfiguration;
private final ObjectMapper objectMapper = new ObjectMapper(); private static final Map<String, Class<?>> TABLES = Map.of(
"order", Order.class,
"transaction", Transaction.class,
"customer", Customer.class
); @BeforeAll
public static void setupClass() {
handler = new StreamLambdaHandler();
lambdaContext = new MockLambdaContext();
} @BeforeEach
public void setup() throws JsonProcessingException { // Setup tables
dynamoDBTestUtils.setupTables(TABLES, dynamoDBMapper, amazonDynamoDB);
// Seed data
seedTableData(); // Setup secrets
secretManagerUtils.setUpSecrets(awsSecretsManager, "test/component-test",
Map.of("encryption-key-customer",
"dummy-encryption-key"));
// Setup SQS
sqsTestUtils.setUpQueue("alert-queue", amazonSQS, externalConfigHolder); }
@Test
void yourTest() {
//your test goes in here
}
@SpringBootTest: This annotation is used to indicate that the class should be considered a Spring Boot test. It boots up a Spring ApplicationContext and provides the necessary infrastructure for testing.@ContextConfiguration: Specifies the configuration classes that will be used to initialize the application context. In this case, it referencesComponentTestConfiguration.class.@Testcontainers: This annotation is used in conjunction with the Testcontainers library, which allows you to define and use Docker containers within your tests. In this context, it's likely used to manage LocalStack containers for testing AWS services locally.@BeforeAll: This annotation is used on a method to signal that it should be executed once before all the tests in the class. In this example, it's used to set up theStreamLambdaHandlerand the lambda context.@BeforeEach: This annotation is used on a method to signal that it should be executed before each test method in the class. Here, it's used for setting up tables, seeding data, and configuring secrets and SQS before each test.@Test: This is a standard JUnit annotation indicating that the annotated method is a test method.StreamLambdaHandler: This appears to be a class representing a handler for AWS Lambda functions that process stream events (like SQS events). It's used to handle streaming events, possibly in the context of AWS Lambda functions.
Conclusion
In this in-depth guide, we navigated through the process of setting up and effectively using LocalStack for testing AWS services in a Java Spring project. LocalStack, combined with Testcontainers, offers a robust testing environment that closely simulates AWS cloud services.
As you integrate LocalStack into your project, remember to customize the provided code snippets to align with your project’s structure and specific requirements. By doing so, you’ll be on your way to achieving a comprehensive and reliable testing strategy. Happy coding and testing!
