How to work with REST API Post method in Python with Practical Example

In this tutorial, we are going to cover the following points:

· How to create JSON request from CSV file

· How to call REST API with POST method on requests one element at a time

· How to read response generated from API

· How to convert JSON response to CSV file

Based on your project requirement, the request JSON layout can be very simple, moderately complex or super complex. Python is not the only option to generate the JSON request layout. You need to keep some points in mind like how complex is the layout and how much it is taking to generate the layout.

Also, I would suggest using some tool to test API calls on the given request layout first. One example of this tool is SOAP UI (https://www.soapui.org/) or Postman. This will help you to fix your request layout and let you know if any modification is required in the requested layout so that proper response can be achieved. So, once you finalize the output the next step would be to generate the finalized input layout through coding in python.

Let’s go through our examples now…

I have found one public API with POST functionality: https://reqres.in/

First, we need to understand what is expected layout for the POST request and based on your API, it could be simple or complex. In our case, it’s quite simple.

Now, suppose we have one CSV file (Generated through ETL processing or any other means)

Records.csv:

name, job
mike, leader
jason, engineer
sonal, dba
ken, manager

To get it processed through API, we need to send these records to API in the JSON format..

Let’s work on creating JSON request from given CSV file..

JSON layout is similar to Python dictionary as we have to deal with Key-Value pair. So, my first focus was to look for some method in CSV module which can read records into Dictionary format and luckily I found DictReader() function. Let’s check how is it different from normal Read() function..

Read()

import csv
with open(‘Records.csv’,’r’) as f:
    print(f.read())

Output :

name,job
mike,leader
jason,engineer
sonal,dba
ken,manager

DictReader()

import csv
with open(‘Records.csv’,’r’) as f:
    reader = csv.DictReader(f)
for records in reader:
    print records

Output :

{‘job’: ‘leader’, ‘name’: ‘mike’}
{‘job’: ‘engineer’, ‘name’: ‘jason’}
{‘job’: ‘dba’, ‘name’: ‘sonal’}
{‘job’: ‘manager’, ‘name’: ‘ken’}

Now, we have key/value pairs ready and we can just dump these records in required format by using python JSON module.

import csv
import json
output =[]
// open input file
with open(‘Records.csv’,’r’) as f:
    reader = csv.DictReader(f)
//appending each record in list
for records in reader:
    output.append(records)
// Create JSON file
with open(‘RecordsJson.json’,’w’)as outfile:
    json.dump(output,outfile,sort_keys=True, indent=4)

RecordsJson.json

[
{
“job”: “leader”,
“name”: “mike”
},
{
“job”: “engineer”,
“name”: “jason”
},
{
“job”: “dba”,
“name”: “sonal”
},
{
“job”: “manager”,
“name”: “ken”
}
]

So, we have successfully created our JSON file. The next step is to call API on these request records. As we have multiple records in input file, we would be sending requests one by one.

Calling API on multiple JSON requests

My approach is to open request file, read records one by one, call POST method on each request and capture response of each request. To call API, we have requests module in Python.

import requests
import json
import csv
// POST API
url =’https://reqres.in/api/users'
// Reading JSON file
with open(‘RecordsJson.json’,’r’)as infile:
    indata = json.load(infile)
output =[]
//Calling API POST method on requests one by one
for data in indata:
    r= requests.post(url, json=data)
    print(r, r.text)

Output :

(<Response [201]>, u’{“job”:”leader”,”name”:”mike”,”id”:”758",”createdAt”:”2018–11–30T14:43:44.611Z”}’)
(<Response [201]>, u’{“job”:”engineer”,”name”:”jason”,”id”:”900",”createdAt”:”2018–11–30T14:43:45.246Z”}’)
(<Response [201]>, u’{“job”:”dba”,”name”:”sonal”,”id”:”927",”createdAt”:”2018–11–30T14:43:45.918Z”}’)
(<Response [201]>, u’{“job”:”manager”,”name”:”ken”,”id”:”166",”createdAt”:”2018–11–30T14:43:46.651Z”}’)

That’s great!! We called API on 3 requests and got a response for all. Instead of printing this output, I would just save it in a JSON file.

// Calling API POST method on requests one by one
for data in indata:
    r= requests.post(url, json=data)
    output.append(json.loads(r.text))
// Saving response in output
with open(‘Response.json,’w’) as outfile:
    json.dump(output, outfile, indent=4)

Response.json :

[
{
“job”: “leader”,
“name”: “mike”,
“createdAt”: “2018–12–03T14:40:08.018Z”,
“id”: “544”
},
{
“job”: “engineer”,
“name”: “jason”,
“createdAt”: “2018–12–03T14:40:09.096Z”,
“id”: “211”
},
{
“job”: “dba”,
“name”: “sonal”,
“createdAt”: “2018–12–03T14:40:09.983Z”,
“id”: “46”
},
{
“job”: “manager”,
“name”: “ken”,
“createdAt”: “2018–12–03T14:40:11.050Z”,
“id”: “437”
}
]

Before moving ahead, I want to highlight one point. I have used below code while reading a response from API.

output.append(json.loads(r.text))

What if I don’t use json.loads while appending records in list, what would be the impact on output.

Let’s check that too.

Output when using below line in the code :

output.append(r.text)

Response.json :

[
“{\”job\”:\”leader\”,\”name\”:\”mike\”,\”id\”:\”466\”,\”createdAt\”:\”2018–12–03T14:46:25.240Z\”}”,
“{\”job\”:\”engineer\”,\”name\”:\”jason\”,\”id\”:\”302\”,\”createdAt\”:\”2018–12–03T14:46:26.112Z\”}”,
“{\”job\”:\”dba\”,\”name\”:\”sonal\”,\”id\”:\”377\”,\”createdAt\”:\”2018–12–03T14:46:27.109Z\”}”,
“{\”job\”:\”manager\”,\”name\”:\”ken\”,\”id\”:\”130\”,\”createdAt\”:\”2018–12–03T14:46:27.988Z\”}”
]

The last step is to pick some fields from API response and save it in CSV file.

Saving Response in CSV file

My goal is to pick id, job and name form the response and save it in a csv file.

Let’s check how to do this.

with open(‘Response.json’,’r’) as inrespfile :
    inrespdata = json.load(inrespfile)
with open(‘OutResponseCSV.csv’,’w’) as outfile:
for data in inrespdata :
    output = str(data[‘id’])+ “,” + str(data[‘job’])+ “,” +  str(data[‘name’])+”\n”
    outfile.write(output)

As the generated output from API is in the JSON format, we can easily pick any value just by picking the corresponding Key. I have applied same logic to pick id, job and name from the file and appended all these value after converting it into string.

OutResponseCSV.csv

20,leader,mike
971,engineer,jason
92,dba,sonal
482,manager,ken

Note : If you are looking to upskill yourself for Tech interviews on Kafka and Spring Kafka then look no further and grab your copy of “75 Interview Questions on Kafka and Spring Kafka” : Available here !

Please do checkout our Youtube channel to get yourself updated with latest Tech News : Available here !