When we push data to the Elasticsearch, we define index based on the timestamp in the event. We have defined fix number of shards for each type of the data depending on its volume. We mostly query latest data from Elasticsearch so mostly that index is in the memory. Sometime we do query more than one indexes and Elasticsearch is able to handle that…
Yes, we actually convert all event data to UTC.
Since variation like daylight savings are not very frequent, we don’t do any thing specific for this. Our system is able to handle these variations quite well.
We have dashboards developed in Kibana to visualize the event data. Since we have timestamp in the events, we can visualize the events (with filtering based on specific attribute) over time.