Trending products
In the last couple of years the speed of changes in many areas have gone up a gear, with both individuals and enterprises having to find better, more data-informed ways to adapt.
The importance of measuring the changes/trends has increased and one of the goals is to use data not only to measure current trends but predict future ones.
There are many different sources of data available that can be utilized for this.
One of the biggest sources are social media platforms, both in terms of availability as well as in relevancy (user generated content).
Most of social media platforms have excellent or in some cases at least decent public APIs available that allows to scale up the acquisition of data for the purpose of researching trends.
We can be interested in trends in many different areas, one of the important fields is E-commerce, where one of the tasks can be exploring the trends of purchasing/interest in a set of products. If
We could be interested in finding out which products have displayed the most positive trends, e.g. measured as growth in some given time interval.
And by categorizing products we can also answer what are the most trending products in given vertical. For product classification check out our article on what is product classification?.
Trending products with Social Media
Once we settle on some products that we want to analyze, the next step is to select a social media platform to get data.
Twitter has an excellent API, you can read more about it here: https://developer.twitter.com/en/docs/twitter-api.
Out of all available sources of data from Twitter, there are actually different possibilities for use.
One approach would be to utilize the Twitter Firehose. There are a few difficulties associated with this, however. First is the sheer amount of data that one needs to process and a corresponding large number of servers needed to be maintained for this purpose.
The other downside is the price for access to Firehose, which can mean high costs for your potential new product or service already at the launch.
A better approach would be to use Twitter Search API:
https://developer.twitter.com/en/docs/twitter-api/v1/tweets/search/api-reference/get-search-tweets
It is less expensive and more targeted approach to get the desired outcome of finding data on particular product.
This approach can work for many products, except those that have high volume of mentions per hour, e.g. a smartphone brand. In this case, fetching all tweets may again require a lot of resources to so it may make sense to use some caps for them.
Next very useful social media API that can be used for finding trending products is the one from Youtube, you can find more information about it here: https://developers.google.com/youtube/v3.
Some other platforms, like Facebook and Instagram, can be a bit more restrictive in terms of what data you can get, but could also be a good option.
Trending products — from search engines, using PyTrends
The popularity of trending products can also be estimated by other means. As the purchasing decision for some item A is often preceded by us searching for this item on the search engine, it makes sense to use the search trends as the proxy for the popularity of trending products.
One can use various data sources from Google, e.g. when interested in actual number of searches, then Google AdWords (or recently Google Ads) is the best option.
If we are however interested primarily in trends, then one can obtain the data from Google Trends.
Here is an example of google trends for keyword data science:
In practice, the easiest way to accomplish this is by a library, one of the best ones for this purpose is https://github.com/GeneralMills/pytrends.
Pytrends is a kind of non-official API for Google Trends, It has many features for downloading the Google Trends data and allows us on to do many statistik analysis.
It is very useful if you want to build your own code script for getting the Trends Data and want to rapidly build an automated pipeline which can accomplish this.
The data fetched by PyTrends is scaled from 0 to 100, with 100 indicating the highest popularity. 0 indicates not enough data points for particular keyword.
PyTrends accepts many different parameters that you can pass:
Interest Over Time
Historical Hourly Interest
Interest by Region
Related Topics
Related Queries
Trending Searches
Realtime Search Trends
Top Charts
Suggestions
Let us look in more detail at some of these parameters.
Interest Over Time: is a historical time series (indexed) which indicates when the keyword was searched most.
Historical Hourly Interest: is a historical, indexed, hourly data for when the keyword was searched most. It is obtained by making several requests to Google, and each one is fetching one week of data on hourly basis.
Interest by Region: similar as the first parameter except on per region basis.
Related Topics: a valuable feature if you are looking for related terms and keywords.
Trending Searches: the latest trending searches shown on Google Trends’ Trending Searches segment.
Top Charts: data for given topic as shown in Google Trends’ Top Charts segment.
Suggestions: another useful group of data, suggestions that can be used for further refinement.
The trending data can be combined with other analysis, e.g. one can classify all trending products according to their class. You can find more about product categorization in our article:
Trend data is also interesting for in-depth analysis of technologies, to see what is trending and what is not.
On our platform Alpha Quantum, we built an AI recommender which predicts which technologies are most interesting for someone using some set of technologies.
E.g., if you are using technology of Owl Carousel, then these technologies would be interesting for you:
Conclusion
In recent years, the pace of changes in society and business is increasing and there is a greater need to measure and understand trends in various fields.
In this article we showed two options of obtaining the data necessary to accomplish this, using social media platforms and search engines for this purpose.
