Installing Instamancer: a tool for collecting Instagram data

Nicholas Carah
Nov 28, 2020 · 5 min read

Kelly Lewis, Nicholas Carah and Daniel Angus

Instamancer is a data scraping tool created by The University of Queensland and the Queensland University of Technology’s Digital Media Research Centre.

The tool was authored by Adam Smith, in a pilot project led by Daniel Angus and Nicholas Carah.

Instamancer supports Instagram research by collecting public Instagram posts and associated metadata by using public hashtags or accounts as queries.

The outputs are formatted as JSON or CSV files, with images, and videos downloaded in jpg and mp4 formats. The library can allow for batch scraping over several different queries. Our Insta-explorer data visualisation tool enables simple exploration of downloaded material.

In this post we walk you through how to download and install Instamancer. In our next post we’ll show you how to use it to scrape Instagram images, video and their metadata.

How does Instamancer work?

Using a browser for large-scale scraping jobs is memory intensive. To circumvent this Instamancer uses an innovative scaping technique that we call ‘grafting’. In grafting, the scraper intercepts and saves the URL and headers of each request, and then after a certain number of interactions with the page it will restart the browser and navigate back to the same page.

Another specific feature to scraping Instagram is how it sends limited information through its feed API. To get extra metadata such as the tagged users, and comments, Instamancer can open new tabs for each post that it scrapes, and then read the metadata from memory. In contemporary web applications like Instagram, there is an invisible state held in memory that is not necessarily being reflected in the rendered text at any given moment. By accessing this memory Instamancer can reveal processes and data that are building that internal state by fetching data from the API. Older scraping techniques do not do this, potentially missing important platform data.

Downloading and installing Instamancer

Step 1

• First, go to: http://git-scm.com/downloads and install Git from the download page. There are download and install options for both MacOS and Windows. Select the relevant option and click though the prompts;

• Then, go to: https://nodejs.org/en/download/ there are download and install options for both MacOS and Windows. Select the relevant option and click though the prompts.

After downloading and installing Git and Node.js you shouldn’t need to restart your computer; however, if you do encounter any issues try resolving this by restarting your computer.

Step 2

To open Command Prompt (or PowerShell) on Windows:
• On Windows 10: Open the start menu and go to the shortcuts folder called “Windows System”. Pressing the dropdown menu should reveal a shortcut to open the Command Prompt application. Right click on the shortcut, press “More”, and press “Run as Administrator”.
• For Windows 8: Go to the start screen, press “All Apps”, and scroll right until the “Windows System” folder shows up. You can find Command Prompt there.
• For Windows 7: Open the start menu and click on “All Programs”. Click on “Accessories” and you’ll find the Command Prompt shortcut. Right click on the shortcut and press “Run as Administrator”.

To open Terminal on MacOS:
• Click the Launchpad icon in the Dock, type Terminal in the search field, then click Terminal.
• In the Finder , open the /Applications/Utilities folder, then double-click Terminal.
• This should open up an app with a black background. When you see your username followed by a dollar sign, you’re ready to start using command line.

Step 3

To enter the command to install Instamancer, use the mouse/mouse pad to navigate to the command line window to make sure that’s where your keystrokes will go, then type the following commands, all in lower case, before pressing the Enter key to run it.
• cd\ (then press the Enter key)
• git clone https://github.com/ScriptSmith/instamancer.git (then press the Enter key)
• cd instamancer (then press the Enter key) if this doesn’t work try this first: md instamancer (then press the Enter key)

Then: On Windows, enter the following commands
• npm install (then press the Enter key)
• npm run build (then press the Enter key)
• npm install -g (then press the Enter key)

Then: For Mac, enter the following commands (it will prompt for you to enter your password, at which point do so, noting it won’t show these keystrokes but is accepting what you enter behind the scenes, followed by Enter key)
• sudo npm install (then press the Enter key)
• sudo npm run build (then press the Enter key)
• sudo npm install -g (then press the Enter key)

Now you have successfully installed Instamancer you’re ready to begin scaping Instagram data. Check out our next post for instructions on how to use the tool.

Image Machines

Using machine vision to explore Instagram’s everyday promotional cultures

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store