As big data landscapes change, companies are calibrating their compasses on unidentified dots on the horizon. But how do you secure the hopes of finding new lands of opportunity to become more than yet another wild goose hunt? The maze of technological choices can be stifling, especially when the success of your business is on the line. On the notion of “stagnation equals decline”, we aim to thrust forward and make sense out of the latest big data development. In this five part series you will jump down the rabbit hole with us and discover the benefits of real-time streaming for your business. www.axual.com
A supercar to manage your data streams
Blog 1 of 5
Every digital action is an event. From bank transfers and information exchanges, to app requests. Companies need a secure and scalable architecture to organize the myriad of data streams. In this blog we help you maneuver to the most efficient solution for managing your data streams.
Apache Kafka® — The power of real-time data
Apache Kafka® is the current industry leading event streaming technology. Implemented by nearly 30% of Fortune 500 companies. Reason for this global success is in large part due to its ability to process data streams in real time. In other words: instant information exchange. Organisations across industries such as banking, telecom, energy and logistics have arrived at the point where the alternative — waiting for information to be processed — is no longer sufficient. End consumers expect in-the-moment information, whether they are sending or receiving requests. Apache Kafka® allows it all to happen, in real time.
Do you have the assets to build a Formula 1 engine?
To process your millions of data events through Apache Kafka you need specialists. And lots of them. Think of setting up your data warehouse as if you were building a Formula 1 engine. You have technicians that each specialize in their own tiny part. They work on the engine in decentralized teams and tinker with it until it is close to perfection and ready for assembly.
Those technicians are your DevOps teams. Each one maintaining their own app or stream in an efficient manner without having to check-in with the other teams. Decentralized, effective, but uncoordinated.
The decentralized approach is an ideal strategy for growth. We won’t argue with that. It allows large organisations to move swift and lean. We recognise the power but have also carefully assessed the risks involved. Because as these teams place their piece of the puzzle in your data warehouse there is often a lack of overview whether it fits within the bigger picture. There needs to be a point of control — after all we’re talking sensitive data in ever increasing volumes.
Decentralized engine vs centralized supercar
At Axual we have encountered so many companies with data setups done in an unstructured way and it leaves gaps in the security and stability of your company. We have a strong found belief that in order to manage data streams in a sustainable, self-serviceable manner — without the need for a large team of specialists — you want to have a centralized platform that provides overview. And then combine it with the flexibility of decentralized autonomy. Is this win-win possible?
We made it a priority to not lose any of the great building blocks of Apache Kafka. In Axual Platform, DevOps teams still maintain their autonomous work space, as do data stream producers and consumers. Within the Self-Service module teams can access (and control accessibility) for their own streams. A secure space that is equally flexible and feeds back to the central monitor.
You could say Axual has built the supercar around the engine. Ready for to get in and drive in a manner of days.
Rabobank & Axual
In 2015 Axual found a solution for Rabobank’s Business Events Bus (BEB) — a marketplace for business events. A business event is a description of anything that happened with a customer, product or service. For example “a customer moved” or “transaction is processed”.
Initially, BEB was a stand-alone project within the Rabobank Alerts but it became apparent there was a larger need for sharing business events within the entire organisation. Kafka appeared insufficient to be used as a company-wide solution and this became the start of Axual Platform.
Additional software was developed for:
- Data Governance
- Data Lineage
- Access control
- DevOps support through self-service and CI/CD integration
- Hybrid cloud setup
- Monitoring, alert, logging
Eventually this results in 4 Axual employees supporting more than 40 Rabobank DevOps teams using 65 different streams that carry a continuously growing volume of over 500 million data events per day. Want to learn more about the partnership between Rabobank and Axual, check out this video
“As of Q1 2019 the Axual Platform is also available in a SaaS solution under the name of Axual Cloud”.
Want to learn more how we can accelerate your data driven business? Go to www.axual.com or call us on 0031 (0)30 800 9211 for direct contact.
