Spring Cloud AWS를 이용한 S3 연동

S3(Simple Storage Service)는 내구성과 확장성이 매우 뛰어난 객체 스토리지를 제공하는 AWS의 대표적인 서비스입니다.

Spring Boot와 Spring Cloud AWS (http://cloud.spring.io/spring-cloud-aws/)를 이용한 예제를 소개합니다.

  • 특정 버킷에 업로드된 파일 목록 가져오기
  • 특정 버킷에 파일 업로드
  • 특정 버킷에 있는 파일 다운로드

먼저, AWS Console에서 accessKey, secretKey를 발급받아야합니다. (참고)

1) Dependency를 구성합니다.

build.gradle

group 'com.axisj.boot.cloud.aws'
version '1.0-SNAPSHOT'
buildscript {
repositories {
mavenCentral()
maven { url "https://repo.spring.io/snapshot" }
maven { url "https://repo.spring.io/milestone" }
maven { url "https://repo.spring.io/release" }
}
dependencies {
classpath("org.springframework.boot:spring-boot-gradle-plugin:1.3.0.M2")
}
}
apply plugin: 'java'
apply plugin: 'idea'
apply plugin: 'eclipse-wtp'
apply plugin: 'spring-boot'
apply plugin: "io.spring.dependency-management"
sourceCompatibility = 1.8
targetCompatibility = 1.8
project.ext {
springBootVersion = '1.3.0.M2'
springCloudVersion = '1.0.2.RELEASE'
}
repositories {
mavenCentral()
maven { url "https://repo.spring.io/snapshot" }
maven { url "https://repo.spring.io/milestone" }
maven { url "https://repo.spring.io/release" }
}
dependencies {
compile("org.springframework.boot:spring-boot-starter-web")
compile("org.springframework.boot:spring-boot-starter-freemarker")
compile("org.springframework.cloud:spring-cloud-aws-context:${springCloudVersion}")
compile("commons-io:commons-io:2.4")
}
task wrapper(type: Wrapper) {
gradleVersion = '2.4'
}

SpringBoot와 SpringCloudAWS를 넣었습니다.
좀더 엄밀히 이야기하면 이 예제에서는 CloudAWS 기능을 쓴다기보단, CloudAWS가 포함하고 있는 AWS SDK를 사용합니다. 좀더 Pure한 구성을 원하면 'com.amazonaws:aws-java-sdk:1.10.6' 만 포함시키는 편이 더 좋습니다.
2) AWS Credentials & S3 Client Bean 생성
@Configuration
public class AWSConfiguration {
@Value("${cloud.aws.credentials.accessKey}")
private String accessKey;
@Value("${cloud.aws.credentials.secretKey}")
private String secretKey;
@Value("${cloud.aws.region}")
private String region;
@Bean
public BasicAWSCredentials basicAWSCredentials() {
return new BasicAWSCredentials(accessKey, secretKey);
}
@Bean
public AmazonS3Client amazonS3Client(AWSCredentials awsCredentials) {
AmazonS3Client amazonS3Client = new AmazonS3Client(awsCredentials);
amazonS3Client.setRegion(Region.getRegion(Regions.fromName(region)));
return amazonS3Client;
}
}

accessKey와 secretKey를 이용해서 AWSCredential Bean을 생성후에 S3 Client Bean도 생성해줍니다.
3) S3 API 구현
@Service
public class S3Wrapper {
@Autowired
private AmazonS3Client amazonS3Client;
@Value("${cloud.aws.s3.bucket}")
private String bucket;
private PutObjectResult upload(String filePath, String uploadKey) throws FileNotFoundException {
return upload(new FileInputStream(filePath), uploadKey);
}
private PutObjectResult upload(InputStream inputStream, String uploadKey) {
PutObjectRequest putObjectRequest = new PutObjectRequest(bucket, uploadKey, inputStream, new ObjectMetadata());
PutObjectResult putObjectResult = amazonS3Client.putObject(putObjectRequest);
IOUtils.closeQuietly(inputStream);
return putObjectResult;
}
public List<PutObjectResult> upload(MultipartFile[] multipartFiles) {
List<PutObjectResult> putObjectResults = new ArrayList<>();
Arrays.stream(multipartFiles)
.filter(multipartFile -> !StringUtils.isEmpty(multipartFile.getOriginalFilename()))
.forEach(multipartFile -> {
try {
putObjectResults.add(upload(multipartFile.getInputStream(), multipartFile.getOriginalFilename()));
} catch (IOException e) {
e.printStackTrace();
}
});
return putObjectResults;
}
public ResponseEntity<byte[]> download(String key) throws IOException {
GetObjectRequest getObjectRequest = new GetObjectRequest(bucket, key);
S3Object s3Object = amazonS3Client.getObject(getObjectRequest);
S3ObjectInputStream objectInputStream = s3Object.getObjectContent();
byte[] bytes = IOUtils.toByteArray(objectInputStream);
String fileName = URLEncoder.encode(key, "UTF-8").replaceAll("\\+", "%20");
HttpHeaders httpHeaders = new HttpHeaders();
httpHeaders.setContentType(MediaType.APPLICATION_OCTET_STREAM);
httpHeaders.setContentLength(bytes.length);
httpHeaders.setContentDispositionFormData("attachment", fileName);
return new ResponseEntity<>(bytes, httpHeaders, HttpStatus.OK);
}
public List<S3ObjectSummary> list() {
ObjectListing objectListing = amazonS3Client.listObjects(new ListObjectsRequest().withBucketName(bucket));
List<S3ObjectSummary> s3ObjectSummaries = objectListing.getObjectSummaries();
return s3ObjectSummaries;
}
}
4) 컨트롤러 생성
@RestController
@RequestMapping("/api/aws/s3")
public class UploadController {
@Autowired
private S3Wrapper s3Wrapper;
@RequestMapping(value = "/upload", method = RequestMethod.POST)
public List<PutObjectResult> upload(@RequestParam("file") MultipartFile[] multipartFiles) {
return s3Wrapper.upload(multipartFiles);
}
@RequestMapping(value = "/download", method = RequestMethod.GET)
public ResponseEntity<byte[]> download(@RequestParam String key) throws IOException {
return s3Wrapper.download(key);
}
@RequestMapping(value = "/list", method = RequestMethod.GET)
public List<S3ObjectSummary> list() throws IOException {
return s3Wrapper.list();
}
}

간단하게 업로드/다운로드/목록을 요청할 수 있는 컨트롤러를 생성했습니다.
5) 실행
1
Drag & Drop으로 파일을 올려봅니다
2
파일을 선택하면 파일 정보가 표시되고, 다운로드를 할 수 있습니다.
3
실제 S3 Console에서도 업로드가 되어있네요!
Screen Shot 2015-07-28 at 12.39.05 PM
PutObjectRequest 에 별도의 옵션이 없으면 기본적인  Permission이 Master Only로 설정됩니다.
Screen Shot 2015-07-28 at 12.46.16 PM
Public 하게 Read권한을 추가하고 싶다면 PubObjectRequest에 CannedAcl을 설정해줍니다.
PutObjectRequest putObjectRequest = new PutObjectRequest(bucket, uploadKey, inputStream, new ObjectMetadata());
putObjectRequest.setCannedAcl(CannedAccessControlList.PublicRead);
PutObjectResult putObjectResult = amazonS3Client.putObject(putObjectRequest);
IOUtils.closeQuietly(inputStream);
return putObjectResult;
PublicRead를 ACL추가후 다시 업로드해보면 Everyone 퍼미션이 추가되었습니다.
Screen Shot 2015-07-28 at 1.44.25 PM
저는 AWS S3 SDK를 최대한 활용해서 예제를 만들었지만, Spring Cloud AWS 문서에는  Spring의 ResourceLoader와, S3 TransferManager를 이용해서 업로드하는 방식도 소개하고 있습니다.
  • ResourceLoader 예제
public class SimpleResourceLoadingBean {
@Autowired
private ResourceLoader resourceLoader;
public void resourceLoadingMethod() throws IOException {
Resource resource = this.resourceLoader.getResource("s3://myBucket/rootFile.log");
Resource secondResource = this.resourceLoader.getResource("s3://myBucket/rootFolder/subFile");
InputStream inputStream = resource.getInputStream();
//read file
}
}
public class SimpleResourceLoadingBean {
@Autowired
private ResourceLoader resourceLoader;
public void writeResource() throws IOException {
Resource resource = this.resourceLoader.getResource("s3://myBucket/rootFile.log");
WritableResource writableResource = (WritableResource) resource;
try (OutputStream outputStream = writableResource.getOutputStream()) {
outputStream.write("test".getBytes());
}
}
}
  • S3 TransferManager 예제
public class SimpleResourceLoadingBean {
@Autowired
private AmazonS3 amazonS3;
public void withTransferManager() {
TransferManager transferManager = new TransferManager(this.amazonS3);
transferManager.upload("myBucket","filename",new File("someFile"));
}
}
public class SimpleResourceLoadingBean {
@Autowired
private ResourcePatternResolver resourcePatternResolver;
public void resolveAndLoad() throws IOException {
Resource[] allTxtFilesInFolder = this.resourcePatternResolver.getResources("s3://bucket/name/*.txt");
Resource[] allTxtFilesInBucket = this.resourcePatternResolver.getResources("s3://bucket/**/*.txt");
Resource[] allTxtFilesGlobally = this.resourcePatternResolver.getResources("s3://**/*.txt");
}
}
SpringBoot용 AutoConfiguration 모듈인 spring-cloud-aws-autoconfigure가 있지만, EC2, Stack, RDS, ElastiCache 등의 기능들이 모두 들어가있어서 S3만을 연동하는 예제에는 적합하지 않은것 같아 제외했습니다. AWS를 좀더 하드하게 쓴다면 spring-cloud-aws-autoconfigure를 적절하게 잘 활용하면 적은 양의 코드로 다양한 설정을 할 수 있습니다.
소스코드 : https://github.com/brant-hwang/spring-cloud-aws-example