Successful apps are built on data. As developers, we don’t always have access to the data that would help make our app successful. While the internet is a nearly-bottomless source of public data in the form of websites, that data is not always structured or available programmatically through an API. Time spent building an extraction algorithm and server is time not spent building your app.
We’re developing LiveAPI, a developer tool to turn any website’s public data into an API in a few minutes. LiveAPI has two parts: a Chrome Extension to select data to extract and a user-hostable server that extracts data and serves up the user-created API endpoints.
You can see LiveAPI in action in the screencast on our site: liveapi.xyz.
This is the first in a three-part guide that walks through how to get started and use LiveAPI.
Installation Guide
A primary goal of ours when building LiveAPI is to make setup painless. Installing the Chrome Extension only takes two clicks on the Chrome Web Store. You can now start using the LiveAPI extension on any site by clicking the icon in the top-right Chrome Menu and following the Using the Chrome Extension part of this guide.
Setting up a server generally isn’t as simple. Keeping to our goal, we’ve automated the server installation process through a script that installs all of the necessary prerequisites, builds the code and starts the LiveAPI services. This script can be executed in one shell command:
sudo curl -s https://raw.githubusercontent.com/live-API/LAPI-Server/master/bin/pull.sh | bash -sThis command is tested for MacOS and Amazon Linux, but should work on other flavors of RHEL and on Ubuntu. If you have any setup or compatibility issues, let us know on Github. We’re still early in determining our compatibility and would love your feedback.
Port Setup
LiveAPI server runs on port 4000 (http) and 4443 (https) by default and is accessible by adding the port to the domain (e.g. liveapi.yoursite.com:4443). To serve and access LiveAPI over the standard http and https ports, you can update the HTTP_PORT and HTTPS_PORT variables in server.js.
If using AWS to host LiveAPI, EC2 requires the application to have root access to run on ports under 1000. To avoid running a web server like LiveAPI as root, you can use iptables to forward port 443 to LiveAPI’s 4443. See this guide for more information.
What We’re Asking to Sudo
The setup command grabs a shell script from github and runs it as the root. Since we don’t recommend executing unfamiliar code as root, we’ll outline what the scripts do.
The pull.sh script is relatively simple and does the following:
- Installs git
- Pulls the latest LiveAPI server code from github
- Starts the installation script (start.sh)
The installation script, start.sh, contains everything else need to install and start the server:
- Installs NodeJS/npm
- Installs MongoDB
- Installs Node module dependencies
- Builds and bundles the front-end JS
- Generates self-signed SSL certificates
- Starts the server
Starting the server using npm start does the following:
- Starts the Mongo daemon
- Starts LiveAPI
Manual Installation
Alternatively, LiveAPI can be installed and started manually. Make sure to do each of the steps listed in the bullets above.
If you are bringing your own SSL certs, such as from a CA, you can drop the certificate (cert.pem) and private key (key.pem) in the SSL folder in the root of LiveAPI.
The minimum NodeJS version required for LiveAPI is 7, but there may be other requirements we have not found yet. We have tested on NodeJS >8.1.4 and MongoDB >3.4.4, but let us know on Github if you have any problems with these or other versions.
Don’t hesitate to reach out with questions, suggestions or issues you have about LiveAPI. We’re excited to keep expanding it.
If you’re interested in learning more, check out the project’s github or website. We’ve written a bit on how it works under the hood as well.
Current contributors: Penn Wu | Melissa Schwartz | Brett Beekley
