Using confluent-kafka on Apple Silicon

  • Install Homebrew
    Homebrew is a package manager for MacOS. If you don’t have homebrew installed on you mac, copy and paste this command in your terminal to install it.
/bin/bash -c “$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
  • Install librdkafka
    librdkafka is Apache Kafka’s c/c++ library. Use below command to install it. Take a note of the version of the library installed. This will be used in the next step.
brew install librdkafka
  • Setup env variables
    We need to export some environment variables so that while installing confluent-kakfa, the required c/c++ binaries can be found. Ensure that you change the version of the librdkafka ( 1.8.2 in below command ) based on your machine
    Note: Don’t forget to update the librdkafka version in the below export statements.
export C_INCLUDE_PATH=/opt/homebrew/Cellar/librdkafka/1.8.2/include
export LIBRARY_PATH=/opt/homebrew/Cellar/librdkafka/1.8.2/lib
  • Install confluent-kafka
pip install confluent-kafka
  • fatal error: ‘librdkafka/rdkafka.h’ file not found
  • file not found: linker command failed with exit code 1

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store