What is Memphis.dev?
Memphis{dev} is an open-source real-time data processing platform that provides end-to-end support for in-app streaming use cases using Memphis distributed message broker.
Memphis’ platform requires zero ops, enables rapid development, extreme cost reduction, eliminates coding barriers, and saves a great amount of dev time for data-oriented developers and data engineers.
The source
A quick baseline — From time to time, I will use the term data-driven/real-time/event-driven applications, which are applications that heavily interact with data like:
- Collecting data
- Change and maintain schema (The shape of the ingested data)
- Analyze and enrich data
- Understand the required next event based on the ingested data
- Integrations with 3rd party applications
- Performing an action based on an event and more
The struggles
As a developer who was tasked with building data-driven apps, I had some really painful struggles:
- Too many data sources to handle (Each has its requirements and API).
- Different and complex schemas (For each data source or service).
- Transform the schema and analyze streamed data per source.
- With multiple services that produce and consume data, it takes time to understand who does what after a certain point.
- The event’s journey is hard to debug and troubleshoot.
- Kafka, RabbitMQ, NATS, and other MQs are HARD to deploy, manage, secure, update, onboard, and tune.
- Turning batch processes into real-time can be complicated and time-consuming.
- Data pipelines can travel between different teams and applications, and it is hard to collaborate over high-velocity use cases.
Memphis.dev
Memphis is the next generation of messaging queues.
A simple, robust, and durable cloud-native message broker wrapped with an entire ecosystem that enables fast and reliable development of next-generation event-driven use cases.
Simple as RabbitMQ, Robust as Apache Kafka, and Perfect for event-driven applications.
Memphis broker focuses on four pillars
- Stability — Queues and brokers play a critical part in the modern application’s structure and should be highly available and stable as possible.
- Performance and Efficiency — Provide good performance while maintaining efficient resource consumption.
- Observability — Reduces troubleshooting time to near zero.
- Developer Experience — Rapid Development, Modularity, inline processing, Schema management.
Key Features (v0.4.2)
- Fully optimized message broker in under 3 minutes
- Easy-to-use UI, CLI, and SDKs
- Data-level observability
- Runs on your Docker or Kubernetes
- Real-time event tracing
- SDKs: Python, Go, Node.js, Typescript, Nest.JS
- Embedded schema management using Avro, and Protobuf
Vision
Our mission is to build the best-in-class dev-first, open-source event-processing platform that will enable millions of developers to better engage with data and build greater data-driven applications and streaming pipelines in friction of time and complexity.
Open-Source
Memphis is an open-source first product and company. Memphis will always be open-source and community-driven as we consider the community as the source of power behind building truly disruptive technology.
https://github.com/memphisdev/memphis-broker
yaniv@memphis.dev
