Memphis.dev
Published in

Memphis.dev

What is Memphis{dev}?

memphis.dev

Memphis{dev} is the only low-code messaging platform that provides a full ecosystem for streaming use cases over a cloud-native message broker with a paradigm of produce-consume that supports modern streaming pipelines and async communication by removing frictions of management, cost, resources, language barriers, and time for data-oriented developers and data engineers, unlike other message brokers and queues that requires a great amount of code, optimizations, adjustments, and mainly time.

The source

Photo by aisvri on Unsplash

A quick baseline — From time to time, I will use the term data-driven/real-time/event-driven applications, which are applications that heavily interact with data like:

  • Collecting data
  • Change and maintain schema (The shape of the ingested data)
  • Analyze and enrich data
  • Understand the required next event based on the ingested data
  • Integrations with 3rd party applications
  • Performing an action based on an event and more

The struggles

As a developer who was tasked with building data-driven apps, I had some really painful struggles like:

  1. Too many data sources to handle (Each has its requirements and API).
  2. Different and complex schemas (For each data source or service).
  3. Transform the schema and analyze streamed data per source.
  4. With multiple services that produce and consume data, after a certain point, it's hard to understand who does what.
  5. The event’s journey is hard to debug and troubleshoot.
  6. Kafka, RabbitMQ, NATS, and other MQs are HARD to deploy, manage, secure, update, onboard, and tune.
  7. Turning batch processes into real-time can be complicated and time-consuming.
  8. Data pipelines can travel between different teams and applications, hard to collaborate in complicated pipelines.
Painful. Photo by The New York Times

Please Welcome Memphis

Memphis delivers the most easy-to-use, deploy, and manage, k8s-native messaging queue. Forked from NATS.IO, with amazing UI, CLI, and with built-in observability that allows you, the developer, to actually understand the full journey of your events.

RabbitMQ has Queues, Kafka has Topics, and Memphis has Stations.

The strength of Memphis reveals on day 2. By providing a framework for developers where they can offload most of the business logic out of the producers and consumers and embed it inside Memphis’ station (queue) like:

  • Inline schema changes and schema registry.
  • Enrich and analyze ingested data on-the-fly.
  • Ready-to-use connectors and components (Similar to “Kafka” sinks) that will pull and push data autonomously.

Helping developers develop data-driven apps Faster, Better, and happier!

Dev-First? What does it mean?

Thanks for reaching out to this point :)

We do our best to create any feature, button, and capability in a way it will make sense for the developer and DevOps on the other side, and really help them with their day-2-day tasks. A few examples —

  1. Installation. Single command via Docker or K8S.
  2. UI and CLI. Management tools to better understand and debug your data-driven app.
  3. Troubleshooting and Debug. The bread and butter of every developer. Enabling developers to follow (via GUI) the full path of a single event.
  4. Monitoring. Out-of-the-box ability to track and understand who are your producers and consumers, what do you do, and how much.
  5. Interact with Memphis in any programming language.

More to come!

Thank you for reading, hope you find it useful.

Please let me know in the comments If something is missing or vague.

In the meantime —

memphis.dev

Discord

GitHub

yaniv@memphis.dev

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store