How to Integrate Paystack and ConvertKit Data with PyAirbyte
Improve your business insights within a few lines of code
Airbyte makes it easy to transfer data between systems. You don’t have to create or look after your own data connectors as they get complicated and costly as your setup grows.
Let’s say you own a clothing store and want to predict your customer's trends this summer. To do so, you need data from your online store, customer records, and inventory. Airbyte simplifies this process by offering pre-made connections for different software.
With a few steps, you can integrate data between these systems and store your data securely on Airbyte’s servers. That way, you can focus on growing your business while Airbyte takes care of all the data stuff.
PyAirbyte vs Airbyte
PyAirbyte is like Airbyte, but it works right on your computer. It’s a Python library that helps you integrate data from multiple systems without needing Airbyte’s servers.
It’s handy when you want to test things or don’t want to create an Airbyte account yet. Easily extract data from your favorite sources and save it locally in your internal cache.
It’s like creating an ELT pipeline within your local device.
In this article, we’ll cover how to use PyAirbyte with practical code examples.
We’ll use Paystack and ConvertKit for data integration. These code examples serve as references to help you create your own ELT pipeline for your business.
Let’s begin!
What You Need?
All you need is a Python notebook. We’ll use Google Colab, but you can use tools like Jupyter or text editors like VSCode.
How to Configure Paystack with PyAirbyte
Paystack is a payment service like Paypal, popular in Nigeria. With PyAirbyte, you can gather data from your company’s Paystack account to make smarter business decisions.
Here’s how you can do it:
Installing PyAirbyte
- Create a new notebook and provide a name for it.
- Install PyAirbyte and create a virtual environment in your notebook.
# Add virtual environment support for running in Google Colab
!apt-get install -qq python3.10-venv
# Install PyAirbyte
%pip install - quiet airbyteNote: Please ignore the following error for now:
ERROR: pip’s dependency resolver does not currently take into account all the packages that are installed. This behavior is the source of the following dependency conflicts.
ipython-sql 0.5.0 requires sqlalchemy>=2.0, but you have sqlalchemy 1.4.51 which is incompatible.
- Run the following command to get a list of available sources in Airbyte.
# Show all available connectors
ab.get_available_connectors()- Import PyAirbyte in your notebook.
import airbyte as ab- Create and install the following Paystack source.
import airbyte as ab
# Create and install the source:
source: ab.Source = ab.get_source("source-paystack")This source helps you configure your Paystack account from the code, letting you access all your business’s Paystack data conveniently.
To move forward, configure your company’s Paystack account and provide details.
PyAirbyte offers various sources, and each source page has a table showing the necessary fields required.
Here’s what we need to configure the Paystack source. The fields marked as *required are necessary.
How to Get Your Paystack Test API Key
You’ll need a secret key and a start date from which PyAirbyte will read data. We will use a test API key in this article.
To get this key:
- Create a free Paystack developer account and navigate to this page.
- Copy and paste the above secret key into the following code.
source.set_config(
config={
"secret_key": "enter_your_secret_key_here",
},
)- Select the date after which you want to extract Paystack data. This date should follow the format: “2017–01–25T00:00:00Z”.
source.set_config(
config={
"secret_key": "sk_test_your_api_key",
"start_date": "2024–03–25T00:00:00Z", # Adjust this to get a larger or smaller dataset
},
)Note: Any data before “2017–01–25T00:00:00Z” cannot be accessed.
- Use the following code to verify and load your business’s Paystack data in the local cache.
# Verify the config and creds by running `check`:
source.check()
# Select all of the source's streams and read data into the internal cache:
source.select_all_streams()
read_result: ab.ReadResult = source.read()Congratulations! You’ve successfully extracted (E) the data and loaded (L) it into your machine using PyAirbyte!
Leverage this data (T) to make informed decisions. Visualize, transform, and analyze it to improve your business strategies.
For example, here we visualize the empty data from our Paystack test account:
How to Setup the ConvertKit Source Connector with PyAirbyte
If you’re using ConvertKit to manage your email list and data such as sales or customer information, PyAirbyte can help you integrate these data points.
This means you can see everything in one place and understand your audience better. This will help you send better emails and grow the business.
Integrating ConvertKit data is similar toPaystack but with some minor differences. Let us see how:
- Navigate to the ConvertKit source connector page in the docs.
Getting Your ConvertKit API Key
The only required field needed is the API secret key. Get this key from your ConvertKit account:
- Login to your ConvertKit dashboard. Click on the profile dropdown and select “Settings.”
- Access the “Advanced” tab and copy this key.
Setup the ConvertKit Source with PyAirbyte
Use this secret key to set up ConvertKit from the PyAirbyte code.
- Install the following ConvertKit source.
import airbyte as ab
# Create and install the source:
source: ab.Source = ab.get_source("source-convertkit")- Paste the above API secret key into the following code.
# Configure the source
source.set_config(
config={
"api_secret": "enter_your_api_secret_key",
},
)- Verify the configuration and load your ConvertKit account’s data stream into your local cache.
# Verify the config and creds by running `check`:
source.check()
# Select all of the source's streams and read data into the internal cache:
source.select_all_streams()
read_result: ab.ReadResult = source.read()You should see the following message and output if your connection was successful.
Congratulations, we’re all set with ConvertKit! Remember, this process can be applied to multiple other sources too.
Here are the notebook examples used in this article:
If you have any questions or need further assistance, feel free to reach out. Happy data handling!