Before we get started I have started a new slack group dedicated to hacking. We welcome everyone from beginner to advanced to join. I will be on everyday answer questions, doing CTFs, and talking about cool hacks. If you enjoy hacking and are looking for like minded people join below:
For a while robots.txt has been used by cyber security professionals to help identify endpoints on a website. These files use to contain useful endpoints for hackers to explore but in todays world they are usually useless.
Apple App Site Association
Apple-app-site-association(AASA) is a file that exists on domains which associates the domain with an IOS application. On IOS 9 and later if you click a link the device will determine if it should be handled by safari or the IOS app. To do this we can use a universal link which will relate a web URL to the IOS app. Before you had to use URL schemes which looked something like (com.myApp://login). The problem with this is that if the IOS app isnt installed and you open the link in safari it will render to nothing.
Using a universal link we can simply use the link (example.com/login) and the device will determine if it should be opened in a browser or on the app. Universal links allow us to have a fallback webpage if a user does not have the app installed.
To make use of universal links owners must host an apple app site association file on their domain. This file is hosted at “ /.well-known/apple-app-site-association” and holds a list of paths shown below:
Whats interesting about this file is that it exposes a boat load of endpoints. You can use this file to find hidden endpoints that a crawler or directory brute force would miss.
Robots.txt is a thing of the past you should be looking for “/.well-known/apple-app-site-association” if you want to find hidden paths in the application.