Supercharge Your Web App: Efficient Data Compression for Local Storage, APIs, and Beyond

In the world of web development, every byte counts. Whether you’re optimizing load times, reducing data transfer costs, or simply trying to keep your URLs within length limits, data compression is a crucial tool in any developer’s toolkit. That’s where Slim comes in — a new JavaScript object compression suite that seamlessly integrates into both Node.js and browser environments.

Why Slim?

We’ve all been there: you need to pass a complex data structure through a URL, only to find that the resulting link is too long, causing errors or being rejected by APIs. Or maybe you’re building a highly interactive web app and need to keep the data transfer between client and server as efficient as possible. After struggling with existing solutions, which either lacked performance or compatibility, we decided to create Slim, a lightweight yet powerful compression library specifically designed for JavaScript objects.

Slim is built on top of the PAKO zlib port, ensuring that it delivers high compression ratios while being fast enough for real-time applications.

Installation

Getting started with Slim is easy. It’s available on npm, and you can install it using your package manager of choice:

npm install @rescale/slim
pnpm add @rescale/slim
bun add @rescale/slim

How Slim Works

Slim provides two main functions: deflate and inflate. The deflate function compresses your JavaScript objects into a more compact form, while inflate restores them to their original state.

Example Usage in React

Here’s how you can use Slim in a React application:

import { inflate, deflate } from '@rescale/slim';

const data = { a: 1, b: 2, c: 3 };
const compressed = deflate(data);
const decompressed = inflate(compressed);
export function App() {
  return (
    <div>
      <h1>Compressed data</h1>
      <pre>{compressed}</pre>
      <h1>Decompressed data</h1>
      <pre>{JSON.stringify(decompressed, null, 2)}</pre>
    </div>
  );
}

In this example, Slim compresses the data object into a much smaller string. This compressed string can then be safely stored, transmitted, or included in a URL without running into length limitations.

Next.js Example

Slim is also fully compatible with Next.js. Here’s how you can use it in a Next.js project:

// /app/page.tsx

import { inflate, deflate } from '@rescale/slim';

const data = { a: 1, b: 2, c: 3 };

const compressed = deflate(data);
const _decompressed = inflate(compressed);

export default async function Page() {
  const { decompressed } = await fetch(
    `/api/data?compressed=${compressed}`,
  ).then((res) => res.json());

  return (
    <div>
      <h1>Compressed data</h1>
      <pre>{compressed}</pre>
      <h1>compatibility</h1>
      <pre>
        {String(JSON.stringify(_decompressed) === JSON.stringify(decompressed))}
      </pre>
    </div>
  );
}

// /app/api/data/route.ts
import { type NextRequest, NextResponse } from 'next/server';
import { inflate, deflate } from '@rescale/slim';

export function GET(request: NextRequest) {
  const compressed = request.nextUrl.searchParams.get('compressed');

  return NextResponse.json({
    decompressed: inflate(compressed),
  });
}

In this Next.js example, Slim compresses data on the client side and decompresses it on the server, demonstrating how you can efficiently pass large data structures between the client and server without burdening the network.

How Slim can help you with optimization:

• Client-Side Storage: Slim allows you to compress session data before storing it in localStorage or sessionStorage. This minimizes the storage footprint and reduces the risk of hitting browser storage limits.
• URL Encoding: For temporary or shareable session states, you can encode the session data into a URL, making it easier to restore the state across different sessions or share it with others.
• Request Payloads: Compressing data before sending it in a POST request reduces the amount of data transferred, leading to faster response times and reduced bandwidth usage.
• Response Payloads: Similarly, you can compress large datasets before sending them to the client, reducing load times and improving the user experience.
• Static Data Compression: Compress large datasets during the build process. When the client loads the static page, the data can be decompressed on the fly, ensuring quick and efficient data delivery.
• Efficient WebSocket Communication: Compress data before sending it through WebSocket connections, reducing the amount of data transferred in real-time communication scenarios.
• Optimized Polling: In cases where you’re polling an API for updates, Slim can compress the request and response data, minimizing the load on your server and reducing the bandwidth required.
• Server-Side Data Compression: Compress large datasets before sending them to the client, optimizing the server response size and improving page load times.
• Efficient Data Transfers: Reduce the payload size in server-side rendering scenarios, especially when dealing with personalized or filtered data that needs to be fetched on demand.
• Offline Caching: Compress large API responses or application state before storing them locally, allowing your app to retain more data for offline use.

Real-World Use Cases

1. URL Length Shortening

URLs are often used to pass data between different parts of a web application. However, many systems impose strict limits on URL length, typically around 2,048 characters. Slim helps you encode large data structures into compact strings, making it easier to stay within these limits. This is particularly useful for:

• Analytics parameters: Passing user activity data in URLs for tracking purposes.
• Stateful URLs: Encoding complex application state or filters into a shareable URL.

2. Data Transport Optimization

In modern web applications, data is constantly being exchanged between the client and server. By compressing JSON objects before sending them, you can significantly reduce the amount of data transferred, leading to faster load times and reduced bandwidth usage. This is crucial for:

• Mobile apps: Where bandwidth is often limited or expensive.
• Real-time applications: Like collaborative tools or live data dashboards, where every millisecond counts.

3. API Payload Compression

APIs in Next.js often need to handle large payloads, whether it’s sending data from the client to the server or vice versa. Compressing these payloads can lead to significant performance improvements, especially in bandwidth-constrained environments.

How Slim Helps:

• Request Payloads: Compressing data before sending it in a POST request reduces the amount of data transferred, leading to faster response times and reduced bandwidth usage.
• Response Payloads: Similarly, you can compress large datasets before sending them to the client, reducing load times and improving the user experience.

Example:

// /app/api/submit-data.ts

import { deflate } from '@rescale/slim';

export async function submitData(data) {
  const compressedData = deflate(data);
  const response = await fetch('/api/submit', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ compressedData }),
  });
  return response.json();
}

// /app/api/submit/route.ts

import { inflate } from '@rescale/slim';

export async function POST(request) {
  const { compressedData } = await request.json();
  const data = inflate(compressedData);
  // Process the data as needed
  return NextResponse.json({ status: 'success' });
}

4. Optimizing Local Storage

Local storage is a convenient way to store data persistently in the user’s browser, but it comes with a limit (typically around 5MB per origin). This limit can be quickly reached in apps that need to store a lot of data, such as user preferences, cached API responses, or form inputs.

How Slim Helps:

• Space Efficiency: By compressing the data before storing it in local storage, you can significantly increase the amount of data you can store, effectively circumventing the storage limit.
• Faster Access: Compressed data means fewer bytes to read and write, which can lead to faster access times, particularly in mobile environments where performance can be constrained.

Example:

import { inflate, deflate } from '@rescale/slim';

function saveToLocalStorage(key, data) {
  const compressedData = deflate(data);
  localStorage.setItem(key, compressedData);
}

function loadFromLocalStorage(key) {
  const compressedData = localStorage.getItem(key);
  if (compressedData) {
    return inflate(compressedData);
  }
  return null;
}

// Usage
const userPreferences = { theme: 'dark', language: 'en', notifications: true };
saveToLocalStorage('userPreferences', userPreferences);

const loadedPreferences = loadFromLocalStorage('userPreferences');
console.log(loadedPreferences); // Outputs: { theme: 'dark', language: 'en', notifications: true }

5. Efficient IndexedDB Storage

IndexedDB is a more robust client-side storage option compared to local storage, allowing for the storage of larger and more complex datasets, such as images, files, or large collections of objects. However, storage space is still finite, and performance can degrade with large volumes of data.

How Slim Helps:

• Data Compression: Compress data before writing it to IndexedDB to maximize storage efficiency, allowing your application to store more data without hitting storage limits.
• Reduced Read/Write Operations: Compressed data takes up less space, meaning that reading and writing operations involve fewer bytes, which can improve performance, particularly when working with large datasets.

Example:

import { inflate, deflate } from '@rescale/slim';

async function saveToIndexedDB(dbName, storeName, key, data) {
  const db = await openIndexedDB(dbName);
  const transaction = db.transaction(storeName, 'readwrite');
  const store = transaction.objectStore(storeName);

  const compressedData = deflate(data);
  store.put(compressedData, key);

  return transaction.complete;
}

async function loadFromIndexedDB(dbName, storeName, key) {
  const db = await openIndexedDB(dbName);
  const transaction = db.transaction(storeName, 'readonly');
  const store = transaction.objectStore(storeName);

  const compressedData = await store.get(key);
  return compressedData ? inflate(compressedData) : null;
}

function openIndexedDB(dbName) {
  return new Promise((resolve, reject) => {
    const request = indexedDB.open(dbName);

    request.onsuccess = () => resolve(request.result);
    request.onerror = (event) => reject(event.target.error);
  });
}

// Usage
const userData = { id: 1, name: 'John Doe', email: 'john@example.com' };
await saveToIndexedDB('appDB', 'users', 'user_1', userData);

const loadedUserData = await loadFromIndexedDB('appDB', 'users', 'user_1');
console.log(loadedUserData); // Outputs: { id: 1, name: 'John Doe', email: 'john@example.com' }

Why Choose Slim?

There are many compression libraries out there, but Slim stands out due to its focus on JavaScript object compression and its ability to work seamlessly in both Node.js and browser environments. Whether you’re building a client-side application, a server-side API, or a hybrid Next.js app, Slim provides a reliable and efficient solution for your compression needs.

Get Started Today

Ready to start compressing your JavaScript objects? Install Slim via npm and check out the documentation and demo to see it in action.

If you encounter any issues or have suggestions for improvement, feel free to open an issue on our GitHub Discussions.

With Slim, you can keep your URLs short, your data transfers light, and your applications running smoothly. Happy coding!