AEM Assets Cold Archival and Storage (Part 1/3)
Co-Author : Deepali Rathi
Introduction
Lately, growing volumes of data has presented major challenges for organizations of all sizes across various industries. Machine learning and analytics adds more value to the data. Further, heavily regulated industries such as financial services, healthcare, and public sector organizations require long data retention periods.
A modern data archiving solution that can help significantly streamline management of archiving tasks while minimizing the cost of archiving storage and simplifying regulatory compliance is need of the hour. The solution should ensure data durability, faster retrieval times, better security and compliance, and improved access for assessing analytics and business intelligence.
Brief
This blog series talks about how to implement an effective asset archival strategy with cloud native DAM of Adobe — Adobe Experience Manager Assets. Unarchiving of asset should be possible to restore the asset. The DAM admins need an Archive Asset Management solution built into their single source of truth that can ingest, manage, post process, archive, and distribute assets so that it fits archival requirements not just for the present, but also for decades into the future.
Use Cases
Ø Many creative companies actively work only on small set of their asset repository, which means that a large portion of their assets can be archived to ensure storage cost reduction.
Ø Many industries require their assets to be backed up for legal and compliance reasons. Ex. Outdated Legal, policy documents.
Ø Assets are to be retained perpetually because of their inherent value. Eg assets owned by News agencies, media companies, health agency, public universities etc.
Ø Archival of dynamically generated assets to protect against fraud. For example, dynamic PDF documents containing user data and machine generated data.
Ø Seasonal Campaign assets which can be archived/unarchived and brought back when needed.
Ø Raw creative assets used for generating finished assets can be archived for later churning.
Potential Solution Flow
Steps 1a, 1b : Assets pushed into AEM from CC apps and/or Workfront.
Step 2. AEM Assets are consumed, and usage is monitored by leveraging Adobe analytics which provides assets impressions. These insights are then used to makes suggestions to an AEM DAM admin about the potential assets that can be archived safely. (eg : Based on low/negligible asset usage)
Step 3. An unsupervised AI ML model hosted external to AEM can also be used to make suggestions about what assets can be safely archived based on asset metrics, metadata, file size, usage pattern etc
Step 4. Adobe I/O based app can also take insights from AI engine.
Steps 5, 6 : The suggested AEM assets are fetched by the AIO based app and then pushed into external cloud storage.
Steps 7. If the assets are again in needed in AEM, they can be unarchived/downloaded directly from external cloud storage to AEM.
Benefits/Value:
1. Reduced storage costs — Assets when backed up in appropriate archive storage solution that is prebuilt for data archiving reduces the overall DAM storage costs for the businesses.
2. Better backup and restore performance — Archiving removes data from main storage, reducing their size. The unarchive ability if implemented can quickly move data back from archival to the DAM when needed.
3. Prevention of data loss — Archived assets are protected in a separate storage which protects against unintentional asset modification or deletion.
4. Regulatory compliance — Such a solution can help regulated companies with their compliance efforts, assets can be archived for an appropriate amount of time while indexing them ensures their retrievable when needed.
Next Steps :
Part 2 of this series will show you a custom implementation of assets being archived and restored between AEM Assets and AWS S3. This solution offers several different types of configurations that allows on-demand as well as automated archiving/unarchiving.
We appreciate your thoughts, feedback or queries in the comments and kindly stay tuned for next part ! 😊
