Marketcheck API — Listing Dates Explained
There are two main date fields in a Marketcheck API listing, here in this post I will try to explain those along with few important ones that you might need while developing an application with the API.
This post will also explain how and when a new listing is inserted from a particular source
Fields
- Price*— Price of the car as listed on VDP (Vehicle Details Page) on dealers / source website
- First seen / Scraped at — These fields contain Datetime when this listing was first crawled on a dealers website
- Last seen / Status date — These fields contain Datetime when this listing was last crawled on a dealer website
New Listing Insert
When a new listing (VIN) is found on the dealer website, obviously a new listing is inserted for that VIN on that source. Its subsequent updation depends on its price.
That same listing will be updated in subsequent crawls as long as its price remain same. So in next crawls mostly last seen / status date of a listing is updated along with changes in all other non critical fields
When Marketcheck’s crawler — Autobot, detects a change in value of price of the car and there is already a existing listing from that source, then a new listing is created with new attributes. But other fields remain same or are calculated from VINs first appearance on that source. Like for example DOM fields
A new listing is also inserted for that VIN if it had gone offline for significant (more than 5) number of days and then came back online some time in future with all the attributes still being same as last time. In such cases we insert new listing for VIN on that source, as it had been offline and there might be changes in all or some of the field values
In Search API usually the most recent listing from a source is shown, as it is the one that is active right now. But in other historic endpoints like History API and Recents Search API, you might see these multiple listings for a VIN on the same source with different listing ID. Usually these listings will have different price.
Listing Update
After new listing has been inserted, in next crawls if there are no changes in values of price then same listing is updated. Any update to other fields is merged into the existing listing only. So, any change in exterior color value, stock number, carfax attributes, options, features etc. is updated in existing listing only
Reference Price & Miles
You might have noticed ref_price and ref_miles keys in API listing, these are sort of debug fields, which can be used in case actual price and miles are not present, if listings price and miles are not critical component and their older values are acceptable to you.
Usually ref_price is last known price of that car from that particular dealer / source, same is the case with ref_miles. These can be really old as well, like if car came back to the same dealer for resale, then these fields might have values that are few years older
Get Active Listings in Date Range
Because in each crawl status_date is updated and so it moves forward and scraped_at date remains the same, So getting all the listings that were active on a particular day or were active within a date range is a bit tricky.
Assume if we want to get all the listings that were active between 2nd June 2019 and 10th June 2019 then we have 2 kinds of listings that are of interest to us —
- Listings that came online on or before 10th June (scraped_at ≤ 10 June 2019)
- Listings that have been online since 2nd June (status_date ≥ 2 June 2019)
Below diagram shows these possibilities on x-axis, and to get active listings within given date range, we are interested in red section of that x-axis
Let me explain why we have to fetch those ranges to get active listings —
- Listing must have came online on or before the upper bound of our date range, as then only we can say that it was active within given range
- Same goes for status_date — Listing must have remained online on or after the lower bound of our date range
So to satisfy these criteria we have apply these two conditions to any Marketcheck historic data set, be it from Data Feed, Recents Search API or History API — to get active listings within a date range
Similar conditions can be used to get active listings for a single day as well, Say to get active listings from 25th June 2019, You would need to filter listings that have their scraped_at date less than or equal to 25th and status_date greater than or equal to 25th
Conclusion
This explanation should help in better understanding Marketcheck API listings and may help you in finding new and exciting use cases for Recents Search API. You can checkout our other posts that go deep into explaining all the caveats of the Recents Search API for better understanding of it
Email to us at api@marketcheck.com for any issues or inquiry
Developer Portal — https://www.marketcheck.com
API Docs — https://apidocs.marketcheck.com/
