Exploring Aliexpress with RapidAPI
Today I am going to talk about how to extract some data from Aliexpress by using the Rapid API platform, but first I would like to explain some questions that you maybe have and brief introduction.
Maybe you are thinking, why did you use a third-party service instead of use from Aliexpress API?, well if We check on the internet and look for private service platforms’ API, most of the time, We will get the same conclusion, These platforms need to check You before permitting You to consume their APIs, they need to know you, what is your goal for using it, in some cases, we have to pay, all the process could take few days; if We only want to use it for a small project maybe Rapid API could be your choice.
Note: If you don’t know what is an API, I recommend checking the link below before going to the next part.
Before Start
Let’s talk a bit about Aliexpress I am going to be short to get into the main topic, the Asia eBay, I don’t like to compare but It a good reference if you don’t know it. Aliexpress is a retail service based in China and owned by Alibaba Group, that offers products to international and It is like eBay because the seller is independent and use the platforms to offer products.
What is RAPIDAPI?
RapidAPI is an API hub, It is widely used to find, test, and connect to thousands of APIs. It is widely used to find, test, and connect to thousands of APIs. It allows You to manage multiples APIs with a single ApiKey and dashboard. Our goal is only to explain how to get data but can create an API and make it available. Something that I didn’t mention but it is really important, RapidAPI isn’t completely free it has a free subscription and has differents package to increase the features that have cost.
In the next section, we will through the next points
- Create an account and Subscribe to an API (Magic Aliexpress)
- Explaining some API endpoints
- How to consume the data
- Create a DataFrame with pandas
Create an account and subscribe to Magin Aliexpress
To create an account go to ApidApi, add your information, and email.
After completing the register, We log in to see something like below
In the top part, we can search for some API by the description, and at the bottom are the categories that exist and a list of APIS. For this project, we use the API Magic AliExpress.
When we search and select Magic Aliexpress, we see the next page.
Some points that I like from the documentation are the clear sections, we can see the Endpoints by category, the section to test the API, and the last part that we can see the different samples to consume it by programming language.
Note: we need to subscribe to the API to be able to use it, to do that we select the button Subscribe on the same page on the top section.
Explaining some API endpoints
The products category has the Endpoints showing in the image, each Url has its purpose, and some of them need parameters that are required or optional and when we make the request we always need to add the header parameters.
- API/product/{productId}: get product information by Id product
- API/product/search: get a list of products
- API/product/searchByImage: you can get a list of products by sending a URL image
- …
At this moment I think, you are asked yourself where is the API Key to have access to the data, well RapidAPI generates one for general use and you can check on the section that shows the code to consume the API (right section in the image) or in the test section (left section in the image).
How to consume the data
To get the data from Rapid API we created a Class in python to manage the methods that are going to send the request and get the response.
The code above has the same structure in the methods, all start to configure the request, set the Url and headers, and get the response. the different part of that is in the constructor __init__ that we set up the connection and headers with our API Key and the root URL.
When we have the response maybe you will see some strange in the data format, if You haven’t familiarity with JSON format, this class has a part to converting the response (JSON) into something that We can work in python (array, dictionary, etc..). JSON library allows doing that most easily.
Create a DataFrame with pandas
To create a DataFrame in our project, We use a Jupyter notebook to visualize the data, Importing pandas, matplotlib, and our new Service module libraries, after that, We create a new instance of AliApi class, and finally, We get some data in a DataFrame.
The first method gets a result by sending a description, min price, and max price, showing 20 products, this is because the API that We are using has pagination and You can select the page to get, through the parameter.
The second one gets a result by sending the Id product and showing 10 feedbacks.
if you look at the code above, We select from the dictionary all the [‘docs’], because when We analyze the Json response this is the part that contains all the records that we want.
Well, let’s take a look at what We get it.
and the second.
Conclusion
RapidAPI is a platform that has flexibility and gives you an easy way to connect and test APIs, that works excellent with a small project, but if you are going to use a specific API in a professional environment and you need more control over the API and your needs may be your best option is to take the time to get the developer permission from the private platform that you look for.
I have no doubts that AliExpress is a very interesting place to work with a data science project, the amount of data that We can get is large and has many areas and features that We can exploit.
About consuming the data, We could say that most of the APIs work like RapidAPI, You have to send your API key or your token or whatever your auth be, and then use a method to send or get data (GET, POST, DELETE, PUT) through HTTP protocol and give thanks to having a standardization JSON.
References
- RapidAPI: https://rapidapi.com/
- AliExpress: https://best.aliexpress.com/
- Github: https://github.com/ibsanmorales/RapidApi_Aliexpress
