Exposing Auto Incremental Identifiers Are Bad. Protect Them By Using One Of These Instead
In the course of a Software Development Life Cycle, the creation of an auto-increment id for unique identifiers is a norm. However, exposing such incremental id poses security concerns that may lead to potential data exposures.
As Software Engineers, part of our responsibility is to ensure we are adhering to best practices especially if it involves potential security breaches and data leakage.
The Problem
Exposing a database table auto-increment id in your site is never a good practice as it does reveal the number of records and allows an attacker to scroll through them. This may lead an attacker to potentially do the following:
Increased guess-based attacks
Auto increment ids are highly predictable and prone to guess-based attacks since it is just a numbered sequence. It is also prone to brute force or other types of attacks that might lead to the leakage of business data.
Understand the total number of records and growth rate
This is part of the business intelligence risk that is exposed. An attacker can identify the total number of records and even track the numbers over time to capture a growth rate.
Know the order in which records were created
An attacker will have an idea that User B was created after User A which can be useful to an attacker.
Generate a list of URLs to resource on the site
If predictable ids are exposed in the URL, an attacker can create a crawler to collect data for malicious and or competition purposes.
The Solution
Fortunately, there are several strategies that we can adopt and these are freely available.
1. Hashids
Hashids is a compact open-source library that creates short, unique, encoded, and non-sequential ids from positive integers. You can also decode those ids back. One popular example of this implementation can be found on Youtube to identify their videos.
Did you Know: Each video on YouTube has an ID made up of 11 characters. This would allow for the existence of 73,786,976,293,838,206,464 videos, which is equivalent to every human being on the planet uploading video every minute for the next 18,000 years.
Javascript version implementation example
Install via npm with npm install hashids/hashids.
// instantiate hashids with a salt
const hashids = new Hashids("this is my salt");
// encode integer
const encodedString = hashids.encode(141120); // K43UrtW
// decode it back
let decodedId = hashids.decode(encodedString); // 141120PHP version implementation example
Install via composer withcomposer require hashids/hashids.
use Hashids\Hashids;
// instantiate hashids with a salt
$hashids = new Hashids("this is my salt");
// encode integer
$encodedString = $hashids->encode(141120); // K43UrtW
// decode it back
$decodedId = $hashids->decode($encodedString); // 141120Go lang version implementation example
Setup library withgo get github.com/speps/go-hashids/v2.
package main
import "fmt"
import "github.com/speps/go-hashids/v2"
func main() {
hd := hashids.NewData()
hd.Salt = "this is my salt"
hd.MinLength = 30
h, _ := hashids.NewWithData(hd)
e, _ := h.Encode([]int{141120})
fmt.Println(e) // K43UrtW
d, _ := h.DecodeWithError(e)
fmt.Println(d) // 141120
}2. UUID (Universally Unique IDentifier) or GUID
UUID is a 128-bit unsigned integer, usually represented as a hexadecimal string split into five groups with dashes. The most widely-known and used types of UUIDs are defined in RFC 4122. Version 4 is considered to be the best choice for implementing UUID as it does not contain any information about the time it is created or the machine that generated them.
Did you Know: A sample of 3.26*1⁰¹⁶ UUIDs has a 99.99% chance of not having any duplicates. Generating that many UUIDs, at a rate of one per second, would take a billion years.
The probability of duplicating a UUID is virtually zero, making them a great choice for generating unique identifiers in distributed systems. The downside to this is that it is 4 times larger than the traditional 4-byte index value; this may cause performance issues and storage implications if you are not being careful implementing it.
Javascript version implementation example
Install via npm withnpm install uuid.
import { v4 as uuidv4 } from 'uuid';
uuidv4(); // ⇨ '9b1deb4d-3b7d-4bad-9bdd-2b0d7b3dcb6d'PHP version implementation example
Install via composer withcomposer require ramsey/uuid.
use Ramsey\Uuid\Uuid;
$uuid = Uuid::uuid4(); // 1ee9aa1b-6510-4105-92b9-7171bb2f3089
printf(
"UUID: %s\nVersion: %d\n",
$uuid->toString(),
$uuid->getFields()->getVersion()
);Go lang version implementation example
Setup library withgo get github.com/google/uuid.
package main
import (
"fmt"
"github.com/google/uuid"
)
func main() {
myUUID := uuid.New()
fmt.Println(myUUID.String())
}3. ULID (Universally Unique Lexicographically Sortable Identifier)
ULID is a 26-character string (128 bits) comprised of ten timestamp characters that provide millisecond precision and sixteen randomly generated characters. Both these parts are base32 encoded strings with 48 bits and 80 bits, respectively. This structure ensures the string’s uniqueness while also allowing it to be sorted.
Did you Know: ULID was built as an alternative to the UUID, and you can generate 1.21e+24 unique ULIDs per millisecond.
One downside of ULID is that it can leak timing information. You can infer the rate at which some resource is being created. For example, the rate at which a service is creating users or business transactions. This can be valuable competitive information that you may consider not sharing with others.
Javascript version implementation example
Install via npm withnpm install ulid.
import { ulid } from ‘ulid’;
ulid(); // 01FHZXHK8PTP9FVK99Z66GXQTXPHP version implementation example
Install via composer withcomposer require robinvdvleuten/ulid.
use Ulid\Ulid;
$ulid = Ulid::generate();
echo (string) $ulid; // 01B8KYR6G8BC61CE8R6K2T16HY
// Or if you prefer a lowercased output
$ulid = Ulid::generate(true);
echo (string) $ulid; // 01b8kyr6g8bc61ce8r6k2t16hy
// If you need the timestamp from an ULID instance
$ulid = Ulid::generate();
echo $ulid->toTimestamp(); // 1561622862
// You can also generate a ULID for a specific UNIX-time in milliseconds
$ulid = Ulid::fromTimestamp(1593048767015);
// or with a lower cased output: $ulid = Ulid::fromTimestamp(1593048767015, true);
echo (string) $ulid; // 01EBMHP6H7TT1Q4B7CA018K5MQGo lang version implementation example
Setup library go get github.com/imdario/go-ulid.
package main
import (
"fmt"
"github.com/imdario/go-ulid"
)
func main() {
u := ulid.New()
fmt.Println(u) // 01G65Z755AFWAKHE12NY0CQ9FH
}4. Snowflake ID
As per Wikipedia, Snowflake IDs are a form of unique identifier used in distributed computing. The format was created by Twitter and is used for the IDs of tweets. The format has been adopted by other majors social platforms, including Discord and Instagram.
Did you Know: Snowflake ID was popularised by Twitter and soon after many big companies started to implement their own version of Snowflake; Discord, Instagram, Sony, Baidu are some of these companies.
Unlike UUIDs, Snowflake IDs are can be sorted by time as they have a timestamp at the beginning which makes them ideal for indexing. Additionally, they’re half the size of UUIDs, practically cutting down the storage size and processing times by half.
Javascript version implementation example
Install via npm withnpm install snowflake-id.
var SnowflakeId = require('snowflake-id');
// initialize snowflake
var snowflake = new SnowflakeId({
mid : 42,
offset : (2019-1970)*31536000*1000
});
var id1 = snowflake.generate(); // returns "285124269753503744"
var id2 = snowflake.generate(); // returns "285124417543999488"PHP version implementation example
Install via composer withcomposer require godruoyi/php.
$snowflake = new \Godruoyi\Snowflake\Snowflake;
$snowflake->id(); // 1537200202186752Go lang version implementation example
Setup library go get github.com/bwmarrin/snowflake.
package main
import (
"fmt"
"github.com/bwmarrin/snowflake"
)
func main() {
// Create a new Node with a Node number of 1
node, err := snowflake.NewNode(1)
if err != nil {
fmt.Println(err)
return
}
// Generate a snowflake ID.
id := node.Generate()
// Print out the ID in a few different ways.
fmt.Printf("Int64 ID: %d\n", id)
fmt.Printf("String ID: %s\n", id)
fmt.Printf("Base2 ID: %s\n", id.Base2())
fmt.Printf("Base64 ID: %s\n", id.Base64())
// Print out the ID's timestamp
fmt.Printf("ID Time : %d\n", id.Time())
// Print out the ID's node number
fmt.Printf("ID Node : %d\n", id.Node())
// Print out the ID's sequence number
fmt.Printf("ID Step : %d\n", id.Step())
// Generate and print, all in one.
fmt.Printf("ID : %d\n", node.Generate().Int64())
}Final Thoughts
These are some of the approaches that you can do to protect your internal IDs, and with a bit of customization to your business’ needs, you can adopt any one of them easily. At Reflex Media, we use a combination of these, with each serving its own unique purpose and challenges. I hope you have learned something new and see you again at the next one.
