Hey Guys, Lets have a quick look on how to handle a file storage(Digital archiving) of Document Management(Doc-Man) system using Spring boot and Amazon S3.
“Digital archiving” — The long-term archiving of digital media
In this I’m gonna show how to manage file upload, download, delete functions using sample app with Spring boot and Amazon S3.
What Is Amazon S3?
Amazon Simple Storage Service is storage for the Internet. It is designed to make web-scale computing easier for developers.
Amazon S3 has a simple web services interface that you can use to store and retrieve any amount of data, at any time, from anywhere on the web. It gives any developer access to the same highly scalable, reliable, fast, inexpensive data storage infrastructure that Amazon uses to run its own global network of web sites. The service aims to maximize benefits of scale and to pass those benefits on to developers.
This guide explains the core concepts of Amazon S3, such as buckets and objects, and how to work with these resources using the Amazon S3 application programming interface (API). — aws.amazon.com
Amazon Simple Storage Service (Amazon S3)
Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. This means customers of all sizes and industries can use it to store and protect any amount of data for a range of use cases, such as websites, mobile applications, backup and restore, archive, enterprise applications, IoT devices, and big data analytics. Amazon S3 provides easy-to-use management features so you can organize your data and configure finely-tuned access controls to meet your specific business, organizational, and compliance requirements. Amazon S3 is designed for 99.999999999% (11 9’s) of durability, and stores data for millions of applications for companies all around the world. — aws.amazon.com
Cloud Object Storage | Store & Retrieve Data Anywhere | Amazon Simple Storage Service
Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data…
I’m gonna create a sample spring boot application to show how this can be done in a simple way. I have used the “SPRING INITIALIZR” to initialize a sample maven project.
Now we can add the AWS sdk to our application, to this we can either use “aws-java-sdk” from maven
<!-- https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk -->
or we could use spring-cloud-starter-aws.
since I’m using spring cloud I have used the “spring-cloud-starter-aws” for this sample app.
Now we have successfully setup a basic app for our demo. Now we need to add user credentials of “AWS iam” user to create programmatically access to the AWS services.
AWS Identity and Access Management (IAM)
AWS Identity and Access Management (IAM) enables you to manage access to AWS services and resources securely. Using IAM, you can create and manage AWS users and groups, and use permissions to allow and deny their access to AWS resources.
IAM is a feature of your AWS account offered at no additional charge. You will be charged only for use of other AWS services by your users.
To get started using IAM, or if you have already registered with AWS,
AWS Identity & Access Management
AWS Identity and Access Management (IAM) enables you to manage access to AWS services and resources securely. Using…
And we also can add the bucket name in property file too
cloud.aws.region.static=your AWS account region
cloud.aws.credentials.accessKey=AWS IAM user access key
cloud.aws.credentials.secretKey=AWS IAM user secret key
After creating the S3 bucket we can add customized policy to the bucket using bucket policy or IAM settings.
After permissions are set for the bucket we can use user credentials (“accessKey” and “secretKey”) to access S3 bucket. To this we can create Amazon S3 Client using “AmazonS3” in aws-sdk
Creating the Spring Bean
in this @Value will get the property from application.properties file “cloud.aws.region.static” property.
Since i have used “cloud.aws.region.static=your AWS account region” it will show like this in the code.
Now we can use the “amazonS3Client” to access S3 bucket we can upload download delete in S3 bucket
in S3 bucket objects are stored as key value pairs so if we need to create a folder inside the bucket we can add folder-name/fileName in fileName param. and in AWS S3 bucket dashboard we can see folder inside the bucket.
Objects are the fundamental entities stored in Amazon S3. Objects consist of object data and metadata. The data portion is opaque to Amazon S3. The metadata is a set of name-value pairs that describe the object. These include some default metadata, such as the date last modified, and standard HTTP metadata, such as Content-Type. You can also specify custom metadata at the time the object is stored.
An object is uniquely identified within a bucket by a key (name) and a version ID. For more information.
A key is the unique identifier for an object within a bucket. Every object in a bucket has exactly one key. Because the combination of a bucket, key, and version ID uniquely identify each object, Amazon S3 can be thought of as a basic data map between “bucket + key + version” and the object itself. Every object in Amazon S3 can be uniquely addressed through the combination of the web service endpoint, bucket name, key, and optionally, a version. For example, in the URL http://doc.s3.amazonaws.com/2006–03–01/AmazonS3.wsdl, “doc” is the name of the bucket and “2006–03–01/AmazonS3.wsdl” is the key.
For more information about object keys
The full implementation and DB scripts of this can be found in github