8/28 AWS Big Data Workshop
Aug 29, 2017 · 1 min read
Ingest (收集)=> Store => Process => Visualize
Ingest: Web server logs, Devices (mobile deep linking), Framework Logs(Kinesis)
- Database => Redshift (交易類型頻繁,查詢複雜,分析使用; 不適合each raw insert, 適合batch insert, SQL => S3 => Redshift; New feature:Spectrum預熱的服務,可直接使用)
- ETL => AWS Glue -fully managed ETL
- AWS Kinesis => click 3 times to build-up the service. (Access log to csv)
- DAX => for 讀取較重,讀取效率較快 (第一次讀取時間差不多,第二次開始,原本的大約是5毫秒, DAX大約是0.9毫秒)
- Quick Sight
- Athena / EMR => S3可直接丟進分析