Number 1 from 2 million of Java projects. Elasticsearch Phenomenon.

What makes it the most popular Java project on GitHub?

And, by the way, GitHub search is built using Elasticsearch!

What makes the creators of the success of the famous Java project — Spring and the SpringSource company now co-create the Elastic brand?

First, the approach fits well with almost all the fashionable trends of recent years. It addresses the need for processing large amounts of data(Big Data), allows you to automate the installation and configuration(Dev Ops), has the characteristics of a distributed architecture aimed at services (Microservices), and is not based on the SQL database (noSQL).

First of all, Big Data.

Using Elastic tools, you can handle large amounts of data starting from the collection of data from various sources through the data processing ending with the search and analysis.

Elasticsearch grew organically by focusing on refining the basic functionality of “search engine”, building communities and promoting projects related to ES. In this way, the ecosystem created around Elasticsearch with two major projects Logstash and Kibana, many plugins and open source projects.

From this perspective, it is not monolith, but a set of cooperating components and an excellent example of what the benefits of Microservices architecture are. The components can be used separately or in combination, which greatly increases the number of potential applications. Third-party companies get a chance to show off their talents, which in turn has a positive effect on the further development of the ecosystem.

On the other hand, Elastic can freely control the development of the individual components — add completely new components or replace the old with the new one.

Logstash is a good example. It was created with the aim of collecting, processing and delivering logs to Elasticsearch. Its driving force was the trend Log As Data. Someone has finally noticed that system logs contain enormous amount of valuable, not used, information. While commercial solutions like Splunk are gaining strength, the ELK (Elasticsearch, Logstash, Kibana) trio was, perhaps, the only reliable open source alternative.

Initially, Logstash developed the number of supported data sources, but it turned out that it is better to introduce a new component specializing in data collection. Later Logstash development was directed toward event processing (Event Processing Pipeline) and the new lightweight components specialized in collecting data appear in the Elastic portfolio — Beats. Beats may better handle devices with limited resources than Logstash. Beats are created in the GO (Golang by Google). It combines the ease of writing known for scripting languages and compiled languages performance like C ++. It was a masterstroke. Elastic will again be on a roll, this time IoT.

There is one more important reason of the remarkable increase in Elasticsearch popularity. Instead of locking inside the JVM platform:

Elasticsearch is opened (Rest API) to other platforms and programming languages.

Since the Java Family is good at science, a relationship with the top of the art class is welcome. Kibana was created in AngularJS — number 1 from the 2 million JavaScript projects on GitHub. In the package, there was even a place for Go. Although, new to the class, and not everyone likes it yet, it is getting along well with older guys like C ++ and embedded environments. And sociable Ruby, of course. Everyone likes him. Logstash was created in Ruby, but does not mind running on the JVM (JRuby).

Friendly relations with the environment also apply to the neighbors. Even those with a different story. A good example is the Stackoverflow built on .Net with an Elasticsearch engine.

The lessons to learn is simple, if you ask me. Being open minded and elastic are important things.