Integration & Security: How to safely integrate your apps to 3rd party data
On numerous occasions, we are faced with questions from users relating to how they can use data from an external API without any issues when using devices but not on web (on live preview or by publishing as a web app).
In this blog post, we will try to explain some of the causes of this behavior. Let’s start from the top!
What happens when you type in an URL into your browser? A request will be sent to the server pointed at by that URL, and the browser will download and run any arbitrary code that happens to be hosted at that source. This is potentially dangerous, as we cannot know the nature of that code we are freely running. This is the main reason why browsers tend to implement higher security measures than other artifacts (such as apps or even servers), the most common being that browser processes run in a `sandbox` or isolated environment from the rest of your computer [1].
Additionally, browser processes are also shielded from the outside by what is normally called the `Same-Origin Policy`. From Mozilla’s developer website you can see that:
The same-origin policy is a critical security mechanism that restricts how a document or script loaded from one origin can interact with a resource from another origin. It helps isolate potentially malicious documents, reducing possible attack vectors. [2]
To simplify, and `Origin` is considered to be the same if it comes from the exact same website, but also has to use the same protocol and port. If you are not familiar with the last two terms, please keep reading on the link above. As an example we can say that these two URLs share the same origin:
but the origin is different to
The only reason being that HTTPS (s for secure) is a different protocol from HTTP (insecure). This would also make, for instance, thunkable.com and youtube.com two different origins.
But in today’s web loading resources from multiple origins occurs very often. For instance, if you have ever embedded a YouTube video on your website, you are now requesting resources from different origins, and this generally works so, how is this possible?
CORS, which stands for “Cross-Origin Resource Sharing” is a mechanism that allows servers to decide which “origins” are allowed to load resources or not. CORS is implemented on servers, and they can be configured in different ways, from allowing everyone on the internet to use the resources to basically allow no one to do so. [3]
So, ok, browsers restrict the way we can load resources, but with CORS, we can somehow work around that, so why is my Thunkable App working fine on iOS and Android but not on web? We’ll talk about that in the next section.
The problem(s)
So at this stage, you have an understanding of why some resources might be blocked by the browser, and we also want to clarify that the `Same Origin Policy` we discussed earlier does not apply to devices or servers, only to web browsers. So that is the reason number one to look into if you are having these issues: is the server I am querying stopping my browser from loading the resources I need? By default, the answer is yes, unless the server owners have configured it to behave differently.
We have some documentation already available here if you want to know a bit more. [4] But wait, you also mentioned earlier that CORS can get around that pesky same-origin imposition, so why is this still not working?
Well, CORS is a mechanism that the owner of the data, the one that owns the server can decide to utilize, or not. CORS can be opened to everything on the internet or limited to a particular internet domain or set of IP addresses. So it is not possible for anyone, as a consumer of data, to make those changes. The owner of the server has that privilege.
This is not the only scenario though, and the same-origin policy is not always at blame for a smaller number of cases. For instance, because we use secure HTTP [4] to host Thunkable, our mobile web apps cannot use `HTTP (not secure) calls to talk to an API. On devices, these calls are allowed. [5]
What can we do then?
What can we, as Thunkable, do about it?
In most cases nothing. If the API you are trying to use is not secure, you can only access it through devices. As we do not own the servers that you are trying to read data from, we cannot change their CORS settings either. But there are some workarounds available.
What can you as a user do about it?
There are a number of tips and tricks found on the web, some of which are not reasonably recommendable because they can be big security holes and we do not recommend anyone to use them. As such, we are not even going to talk about them.
In some cases, and please note this is a much more advanced topic, we can recommend the use of a proxy, or an intermediary between yourself and the data you are trying to load. The gist is that, as you may recall, servers alongside devices do not abide by the same-origin policy, so a proxy is basically an intermediary that will make a call to the data you want to load, and then serve that same data to your app. A proxy is a server process, so you can set up CORS so that access to the data is allowed.
So you would basically be running a little server that makes calls to APIs for you, and it is configured so that you can access it from your own app. Instead of calling the API directly, you call your proxy, and in turn the proxy calls and then serves the received data.
This sounds complicated though, especially if you are not a developer. How do you even run a server on what… and where… and how? But don’t fret, there are simple services for this. Here’s an example of such as proxy service, hosted for free with a service called Vercel.com [6]
Note that we offer this as a template or example of how this could work, and there are certain things that you should modify for a production system.
When it comes time to pull in data for your app, make sure you thoroughly scope out all requirements. Pay attention to the Origin and server security. This will set you up to make the data passing process much smoother. And if you do get stuck, try some of the workarounds provided!
Sources:
[1] https://web.dev/browser-sandbox/
[2] https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy
[3] https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS
[4] https://intercom.help/thunkable/en/articles/4515338-what-is-the-cors-policy-affecting-my-web-app
[5] https://en.wikipedia.org/wiki/HTTPS
[6] https://github.com/josmas/vercel-proxy
