Apple’s iMessage impersonates Twitter & Facebook bots when scraping
When you send a link to a friend on iMessage you get this nice big link preview that’s much easier on the eyes than your standard blue, underlined link:
In order for iMessage to display this image and the page title, it makes a request for the URL that you’ve sent in your message. When it makes this request, it does so using the following user agent header:
I stumbled upon this while building Hyperlink, an iPhone app that allows you generate links that send you a push notification when they’re clicked on. I noticed that when you sent a Hyperlink via iMessage, you immediately received a push notification. I was doing some user agent detection to label clicks that came from bots, but noticed that links sent via iMessage were appearing as scraped by Facebook or Twitter bot.
The second interesting thing about this user agent is that while this request is made directly from the sending iPhone, not an Apple server somewhere — it claims to come from a Desktop Mac (Macintosh; Intel Mac OS X 10_11_1) rather than an iPhone (iPhone user-agent strings typically begin with Mozilla/5.0 (iPhone; CPU iPhone OS 13_1 like Mac OS X).
My best guess for why Apple decided to spoof these two popular services and pretend to be a desktop when making its requests is that some sites may use custom logic for rich previews shown on Facebook and Twitter. Microsoft Edge does something similar in sending up Mozilla, Safari, and Chrome in its user agent header, but at least it includes Edge/14 at the end of its string so it’s possible to tell where the request is really coming from.
This may seem not seem so bad on the surface, but spoofing user agents makes it difficult to get an accurate picture of where your web traffic is coming from. It also breaks standards like robots.txt (which iMessage doesn’t seem to respect either) which allow you to control how crawlers crawl your site. It’s bad behavior that you might expect from a scrappy startup or a malicious crawler — not the second largest company in the world.
I hope Apple takes steps to be a better internet citizen and tack some iMessage identifier to the end of their user agent string in a future software update. In the meantime, we’ll be checking for requests that claim to be coming from both Twitterbot and Facebot — and labeling those as iMessage.