Analysis the Instagram client-side password encryption by Reverse Enginering

Mehmet Yılmaz
10 min readDec 1, 2023

--

Introduction

Instagram and Facebook social media platforms often use encryption protocols to secure user data and communication. AES256-GCM refers to a combination of encryption algorithms used for securing data transmission.

The password entered into the login form is encrypted on the client side before being transmitted to the server.

The purpose of this paper is to try to reverse engineer the encryption process.

Example Code: https://gist.github.com/ylmazmehmet60/a50f2c7d07ff1e2c49942a7141cbf806

Trace analysis

Let’s send a connection request using the following identifiers:

email: test@test.com
password: dreampassword

In the fields of our POST request, we notice the following enc_password field:

#PWD_INSTAGRAM_BROWSER:10:1701464003:AQJQAIYDctyfXAwCokTlLt3L
moXalThecgQh3NGTYZ/sMpJS13jNA9KEgca3OJ/gzJN4sybMaOu8cXI5utR
abX/r2PhAMj+GcaERezZvRyUJjHr/N+etYizU19zWrwBLI7SsxSWIa8bXKxxw1+hpxB4=

The domain name is readily apparent, leading us to strongly believe that the password is encrypted. At an initial assessment, it appears to resemble the format of the /etc/shadow file used in Linux or UNIX-like systems. It could potentially be using SHA-256 encryption. Let’s delve into a closer examination.

Given that encryption is a necessary step on the client side, our approach involves analyzing the trace of our POST request to identify the specific function responsible for encrypting the password. By scrutinizing the request and examining the client-side code, we aim to pinpoint the encryption mechanism applied to enhance the security of the transmitted data. This process is integral to understanding and replicating the encryption procedure for authentication purposes.

Analyzing the trace of our POST request

I systematically examine the functions one by one, searching for the potential starting point. What I am particularly interested in at this point is what it calls before the promises objects function.

PolarisAPILogin: High-layer encryption function

__d("PolarisAPILogin", ["PolarisEncryptionHelper", "PolarisInstapi", "PolarisPasswordEncryptionLogger", "asyncToGeneratorRuntime", "uuidv4"], (function(a, b, c, d, e, f, g) {
"use strict";
function a(a, b, c, d, e, f) {
return h.apply(this, arguments)
}
function h() {
h = b("asyncToGeneratorRuntime").asyncToGenerator(function*(a, b, e, f, g, h) {
g === void 0 && (g = null);
h === void 0 && (h = null);
var i = {
requestUUID: c("uuidv4")()
};
b = babelHelpers["extends"]({}, yield d("PolarisEncryptionHelper").getEncryptedParam("password", b, i), {
optIntoOneTap: f,
queryParams: e,
stopDeletionNonce: g,
trustedDeviceRecords: h,
username: a
});
d("PolarisPasswordEncryptionLogger").logEncryptedPayloadSend("/api/v1/web/accounts/login/ajax/", b, i);
return d("PolarisInstapi").apiPost("/api/v1/web/accounts/login/ajax/", {
body: b
}).then(function(a) {
return a.data
})
});
return h.apply(this, arguments)
}
g.login = a
}

While not requiring a complete understanding of all the displayed variables, it is generally acknowledged that the PolarisApiLogin function functions as the entry point, invoking the specified function.

From this point on, it is evident that the string “password” undergoes processing by the PolarisEncryptionHelperfunction. Before we proceed blindly, we need to perform debugging at this stage to determine the correspondence of the parameters sent to the function.

Debugging

I have determined that b value is the password and i is the identifier generated by uuidv4.

PolarisEncryptionHelper: Middle-layer encryption function

__d("PolarisEncryptionHelper", ["PolarisEncryptionUtils", "PolarisFBBrowserPasswordFormatter", "PolarisPasswordEncryptionLogger", "asyncToGeneratorRuntime", "uuidv4"], (function(a, b, c, d, e, f, g) {
"use strict";
function h(a, b) {
return i.apply(this, arguments)
}
function i() {
i = b("asyncToGeneratorRuntime").asyncToGenerator(function*(a, b) {
if (a === "")
return void 0;
var e = c("uuidv4")();
b = b.requestUUID;
d("PolarisPasswordEncryptionLogger").logEncryptionAttempt(e, b);
var f;
try {
var g = j();
f = (yield d("PolarisEncryptionUtils").encryptAndFormat(a, g));
d("PolarisPasswordEncryptionLogger").logEncryptionSuccess(e, b)
} catch (a) {
d("PolarisPasswordEncryptionLogger").logEncryptionFailure(a)
}
f == null && (f = k(a),
d("PolarisPasswordEncryptionLogger").logEncryptionFallback(e, b));
return f
});
return i.apply(this, arguments)
}
function j() {
return Math.floor(Date.now() / 1e3).toString()
}
function k(a) {
var b = j();
return d("PolarisFBBrowserPasswordFormatter").formatPassword(a, b, d("PolarisFBBrowserPasswordFormatter").formatType.PLAINTEXT)
}
function a(a, b, c, d) {
return l.apply(this, arguments)
}
function l() {
l = b("asyncToGeneratorRuntime").asyncToGenerator(function*(a, b, c, d) {
d === void 0 && (d = "enc_");
var e = {};
b = (yield h(b, c));
if (b != null) {
c = "" + d + a;
e = (d = {},
d[c] = b,
d)
}
return Object.freeze(babelHelpers["extends"]({}, e))
});
return l.apply(this, arguments)
}
g.encrypt = h;
g.getTimestamp = j;
g.formatPlaintextPassword = k;
g.getEncryptedParam = a
}

When I examine the code, this line is very interesting. a and g values are sent to this method but in the previous function b was equal to passport. I’m calling in the rescue dubugger to figure out what happened.

f = (yield d("PolarisEncryptionUtils").encryptAndFormat(a, g));

PolarisEncryptionUtils: PolarisEncryptionHelper: Middle-layer encryption function

__d("PolarisEncryptionUtils", ["PolarisEncryptionKeysStore", "PolarisFBBrowserPasswordEncryption"], (function(a, b, c, d, e, f, g) {
"use strict";
function a(a, b) {
var c = d("PolarisEncryptionKeysStore").getKeyId()
, e = d("PolarisEncryptionKeysStore").getPublicKey()
, f = d("PolarisEncryptionKeysStore").getVersion();
if (c == null || e == null)
throw new Error("Encryption Failure: failed to retrieve keyId and/or publicKey");
return d("PolarisFBBrowserPasswordEncryption").encryptPassword(+c, e, f, a, b)
}
g.encryptAndFormat = a
}

PolarisEncryptionUtils method contains getKeyId(), getPublicId(), getVersion()getters provided by the PolarisEncryptionKeysStore function. I think these values are set somewhere. But first you need to know what these values look like.

What is this values ?

Some values are similar to the elements in the enc_password string, right? It’s very clear that especially b is timestamp. To locate these values, it is apparent that they are not directly stored in .JS files. Therefore, our investigation should focus on the called endpoint.

PolarisAPIGetCookiesFromServer: Shared Data

__d("PolarisAPIGetCookiesFromServer", ["PolarisConfig", "PolarisInstajaxRequestHeader", "PolarisInstapi"], (function(a, b, c, d, e, f, g) {
"use strict";
function a(a) {
a === void 0 && (a = null);
var b = {};
a = (a = a) != null ? a : d("PolarisConfig").getDeviceId();
a != null && (b[c("PolarisInstajaxRequestHeader").DeviceId] = a);
return d("PolarisInstapi").apiGet("/api/v1/web/data/shared_data/", {}, {
options: {
headers: b
}
}).then(function(a) {
return a.data
})
}
g.getCookiesFromServer = a
}

Endpoint contains exactly all the information we are looking for. https://www.instagram.com/api/v1/web/data/shared_data/

Shared Data

PolarisFBBrowserPasswordEncryption: Middle-layer encryption function

__d("PolarisFBBrowserPasswordEncryption", ["PolarisEnvelopeEncryption", "PolarisFBBrowserPasswordFormatter", "asyncToGeneratorRuntime", "tweetnacl-util"], (function(a, b, c, d, e, f, g) {
"use strict";
function a(a, b, c, d, e) {
return h.apply(this, arguments)
}
function h() {
h = b("asyncToGeneratorRuntime").asyncToGenerator(function*(a, b, e, f, g) {
f = c("tweetnacl-util").decodeUTF8(f);
var h = c("tweetnacl-util").decodeUTF8(g);
a = (yield c("PolarisEnvelopeEncryption").encrypt(a, b, f, h));
return d("PolarisFBBrowserPasswordFormatter").formatPassword(c("tweetnacl-util").encodeBase64(a), g, e)
});
return h.apply(this, arguments)
}
g.encryptPassword = a
}

In the current phase of our analysis, it is becoming increasingly evident that we are in the initial stages of obtaining access to the underlying encryption algorithm

tweetnacl-util refers to a utility library for the TweetNaCl cryptographic library. TweetNaCl is a compact and highly regarded cryptography library designed for simplicity, security, and ease of use. It provides a set of cryptographic functions and is often used in scenarios where a lightweight and efficient cryptographic solution is required.

Lets keep examine before base algorithm.

f = c("tweetnacl-util").decodeUTF8(f);

.decodeUTF8(f): This indicates that you are calling the decodeUTF8 function from the tweetnacl-util library, passing the variable f as an argument. This function is likely designed to decode UTF-8 encoded data.

PolarisFBBrowserPasswordFormatter: Formatter function

__d("PolarisFBBrowserPasswordFormatter", [], (function(a, b, c, d, e, f) {
"use strict";
var g = "#PWD_INSTAGRAM_BROWSER";
b = Object.freeze({
FALLBACK_ENCRYPT: "9",
PLAINTEXT: "0",
ROTATED_ENCRYPT: "6"
});
function a(a, b, c) {
return [g, c, b, a].join(":")
}
f.PWD_ENC_TAG_BROWSER = g;
f.formatType = b;
f.formatPassword = a
}

After generating our key, a straightforward invocation of the function yields the corresponding base64 key. By seamlessly combining all components, we obtain a fully-formed and valid key, poised for transmission to the Instagram server

PolarisEnvelopeEncryption: Low-layer encryption function

__d("PolarisEnvelopeEncryption", ["Promise", "asyncToGeneratorRuntime", "tweetnacl-sealedbox-js"], (function(a, b, c, d, e, f) {
"use strict";
var g, h = 64, i = 1, j = 1, k = 1, l = b("tweetnacl-sealedbox-js").overheadLength, m = 2, n = 32, o = 16, p = j + k + m + n + l + o;
function q(a, c) {
return b("tweetnacl-sealedbox-js").seal(a, c)
}
function r(a) {
var b = [];
for (var c = 0; c < a.length; c += 2)
b.push(parseInt(a.slice(c, c + 2), 16));
return new Uint8Array(b)
}
a = {
encrypt: function() {
var a = b("asyncToGeneratorRuntime").asyncToGenerator(function*(a, c, d, e) {
var f = p + d.length;
if (c.length !== h)
throw new Error("public key is not a valid hex sting");
var s = r(c);
if (!s)
throw new Error("public key is not a valid hex string");
var t = new Uint8Array(f)
, u = 0;
t[u] = i;
u += j;
t[u] = a;
u += k;
c = {
length: n * 8,
name: "AES-GCM"
};
var v = {
additionalData: e,
iv: new Uint8Array(12),
name: "AES-GCM",
tagLen: o
}
, w = window.crypto || window.msCrypto;
return w.subtle.generateKey(c, !0, ["encrypt", "decrypt"]).then(function(a) {
var c = w.subtle.exportKey("raw", a);
a = w.subtle.encrypt(v, a, d.buffer);
return (g || (g = b("Promise"))).all([c, a])
}).then(function(a) {
var b = new Uint8Array(a[0]);
b = q(b, s);
t[u] = b.length & 255;
t[u + 1] = b.length >> 8 & 255;
u += m;
t.set(b, u);
u += n;
u += l;
if (b.length !== n + l)
throw new Error("encrypted key is the wrong length");
b = new Uint8Array(a[1]);
a = b.slice(-o);
b = b.slice(0, -o);
t.set(a, u);
u += o;
t.set(b, u);
return t
})["catch"](function(a) {
throw a
})
});
function c(b, c, d, e) {
return a.apply(this, arguments)
}
return c
}()
};
e.exports = a
}

PolarisEnvelopeEncryptionis the base algorithm for AES256-GCM encryption. Reviewing the final parameters through the debugger would enhance the clarity of the algorithm, making it more enjoyable to comprehend and facilitating a deeper understanding of the code execution process.

Final Scope
  • a : key_id
  • b : public_key
  • c : version
  • d : password
  • e : timestamp

Parameters directly to the method without any processing in between. This is good news for us if we are going to code this method with a programming language.

Functions:

  • q(a, c): This function seems to use the seal method from the tweetnacl-sealedbox-js library to encrypt the data a with the public key c.
  • r(a): This function converts a hexadecimal string a into a Uint8Array.
  • encrypt(a, c, d, e): This function is the main functionality. It encrypts some data (d) using envelope encryption with a public key (c). The encryption involves generating a symmetric key, encrypting the data with this key, and then encrypting the symmetric key with the public key. The function returns a Promise that resolves to the encrypted data.

Code Analysis

    var g, h = 64, i = 1, j = 1, k = 1, l = b("tweetnacl-sealedbox-js").overheadLength, m = 2, n = 32, o = 16, p = j + k + m + n + l + o;

Constants are defined:

  • h, i, j, k, m, n, o are assigned numeric values.
  • l is assigned the overheadLength property of the "tweetnacl-sealedbox-js" module.
  • p is calculated based on the values of other constants.
  function q(a, c) {
return b("tweetnacl-sealedbox-js").seal(a, c)
}

Function q is defined, which uses the seal method from "tweetnacl-sealedbox-js" to encrypt data a with a public key c.

 function r(a) {
var b = [];
for (var c = 0; c < a.length; c += 2)
b.push(parseInt(a.slice(c, c + 2), 16));
return new Uint8Array(b)
}

Function r is defined, which converts a hexadecimal string a into a Uint8Array.

  a = {
encrypt: function() {
var a = b("asyncToGeneratorRuntime").asyncToGenerator(function*(a, c, d, e) {

An object a is created, and it contains a method encrypt defined using an asynchronous generator function.

    var f = p + d.length;
if (c.length !== h)
throw new Error("public key is not a valid hex sting");

A variable f is calculated, and there's a check for the length of the public key c.

  var s = r(c);
if (!s)
throw new Error("public key is not a valid hex string");

The function converts the hex public key c into a Uint8Array using the r function and checks its validity.

  var t = new Uint8Array(f)
, u = 0;
t[u] = i;
u += j;
t[u] = a;
u += k;

A Uint8Array t is initialized, and some values are set at specific indices.

    c = {
length: n * 8,
name: "AES-GCM"
};
var v = {
additionalData: e,
iv: new Uint8Array(12),
name: "AES-GCM",
tagLen: o
}

Objects c and v are defined with specific properties for later use in the Web Crypto API.

   var w = window.crypto || window.msCrypto;
return w.subtle.generateKey(c, !0, ["encrypt", "decrypt"]).then(function(a) {
var c = w.subtle.exportKey("raw", a);
a = w.subtle.encrypt(v, a, d.buffer);
return (g || (g = b("Promise"))).all([c, a])
}).then(function(a) {

The code uses the Web Crypto API to generate a key, export it, and encrypt data. The promises are chained using then.

     var b = new Uint8Array(a[0]);
b = q(b, s);
t[u] = b.length & 255;
t[u + 1] = b.length >> 8 & 255;
u += m;
t.set(b, u);
u += n;
u += l;

The encrypted key is processed and added to the Uint8Array t.

      if (b.length !== n + l)
throw new Error("encrypted key is the wrong length");
b = new Uint8Array(a[1]);
a = b.slice(-o);
b = b.slice(0, -o);
t.set(a, u);
u += o;
t.set(b, u);
return t
})["catch"](function(a) {
throw a
})
});

The final steps include additional processing of the encrypted data and error handling using promises.

--

--