How to start Kafka with API -Part 1

Here is what I am trying to achieve:

build an API from flask, and make POST request allowed to that endpoint sendsmessage to a Kafka topic.

I am on a Mac. I only have Spark set up on my local. Flask set-up was easy. No special tricks. but kafka is difficult….So I have to follow this guidance https://hevodata.com/learn/install-kafka-on-mac/ to set up the kafka

Here are some changes in this post probably it is a while ago.

my kafka is kafka-3.7.0-src version.

bin/kafka-server-start.sh config/server.properties

i still loaded the zookeeper althought i read kafka already stop supporting zookeeper.

bin/zookeeper-server-start.sh config/zookeeper.properties

Then kafka is able to pip install

the constant issues are

distutilis is a depercated package that is inside setuptools pacakge now

solution is simply this (from https://stackoverflow.com/questions/77233855/why-did-i-got-an-error-modulenotfounderror-no-module-named-distutils)

python3 -m pip install setuptools

When I run the python scrip which looks like this

from flask import Flask, request, jsonify
from kafka import KafkaProducer

app = Flask(__name__)

# Initialize Kafka producer
producer = KafkaProducer(bootstrap_servers=['localhost:9092'])

#define a route
@app.route('/api/send_message', methods=['POST'])
def send_message():
    try:
        # Get request body
        message = request.data

        # Send message to Kafka topic
        producer.send('input', message)
        producer.flush()  # Ensure all messages are sent before proceeding

        # Return success message
        return jsonify({"message": "Message sent successfully"}), 200
    #except KafkaError as e:
    except Exception as e:
        # If any Kafka error occurs, return error message
        error_message = f"Kafka error: {str(e)}"
        return jsonify({"error": error_message}), 500

if __name__ == '__main__':
    # Run the Flask app
    app.run(debug=True)

the result of running the python script proded above

I am still trying to figure out what and which thing I did!

Thank you for reading until here.

Here is the part 2 story :How to start Kafka with API -Part 2