What role does Hive play in your elasticsearch/kibana monitoring and dashboard pipeline?
Evan Smith
1

You typically don’t have room for an infinite amount of elastic search indexes; most places store about 30–90 days worth of log data in elastic search. Since all the data goes to S3, which it isn’t deleted from, you can then use a tool such as Hive to map-reduce over it for longer time range queries.

Like what you read? Give Stefan Krawczyk a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.