System Design: Building a URL Shortener from Scratch

A URL shortener is the "gatekeeper question" in system design interviews. It is small enough to cover in 30 minutes, yet broad enough to lead into database sharding, cache consistency, and global acceleration. One question can accurately reveal whether a candidate has merely memorized stock answers or has genuinely thought through the problem. Unfortunately, most candidates I have interviewed start drawing three boxes and some arrows as soon as the question is read, then say, "I would cache it in Redis," and stop there. What truly determines success is whether they are willing to clarify "what exactly does this thing need to solve?" before starting.

This article follows the pace of an interview, from requirements clarification through sharding and scaling, with code and architecture diagrams designed to production-ready standards. After reading it, you should be able to earn both the basic points and the bonus points for this question.

1. Do Not Rush to Draw a Diagram: Clarify the Requirements

The first step in system design is never drawing an architecture diagram; it is asking questions. I have seen too many candidates explain a solution for five minutes before realizing, "Oh, it also needs custom short codes." Everything they said before that becomes irrelevant. At minimum, clarify these URL-shortener requirements:

Functional requirements

• Two core operations: generate a short code for a long URL (write), and redirect a short code back to the long URL (read / redirect).
• Are custom short codes supported? For example, s.cn/my-launch instead of s.cn/x7K9aQ.
• Do short codes expire? Marketing campaign links usually should; personal permanent links usually should not.
• Is analytics required, including click counts, referrers, and regions?

Non-functional requirements (this is what separates strong answers from weak ones)

• Read/write volume: A typical ratio is 100:1, with far more reads than writes. You must ask for specific numbers because they directly affect the caching and sharding strategies.
• Short-code length: Seven base62 characters support approximately 3.5 trillion values and are the industry default. Some scenarios may require six or fewer characters for aesthetics.
• Availability: If the URL shortener goes down, every advertisement, email, and post containing its links stops working. That is a business-critical incident, so availability should target 99.99%.
• Latency: Redirect P99 should be below 50ms. Any slower and users may think the link is broken.

Capacity estimation (mandatory in an interview; omitting it costs points)

Assume 100 million new short URLs per month and a 100:1 read/write ratio:

• Storage: At 100 million records per month and 500B each for the long URL + short code + metadata, storage is about 600GB per year and 6TB over ten years. One database can handle the initial scale, but sharding should already be considered.
• Write QPS: 100 million / 30 / 86400 ≈ 40. At five times the average for peak load, that is approximately 200 QPS, so writes are not a concern.
• Read QPS: At 100 times the write volume, peak traffic is approximately 20,000 QPS. This is where the design is truly tested.

Walking the interviewer through these numbers already earns points because it proves you can translate an abstract problem into concrete resource requirements.

2. Generating Short Codes: Tradeoffs Among Four Approaches

Short-code generation is the heart of a URL shortener. There are roughly four approaches, presented below in reverse order of recommendation.

Approach A: Truncated Hash (First 6-7 Characters of MD5/SHA)

import { createHash } from 'node:crypto';

function hashShorten(url: string): string {
  return createHash('md5').update(url).digest('hex').slice(0, 7);
}

The advantages are statelessness and the ability to calculate codes on the client. The fatal drawback is that collision rates after truncation are not negligible. MD5 is 128 bits, but this takes only 28 bits (7 hex characters). Under the birthday paradox, collisions begin at roughly 2^14 ≈ 16,000 records. What happens after a collision? Add a random salt, recalculate, and query the database again. That disk read gives back all the benefits of being "stateless." This approach is rarely used in production.

Approach B: Auto-Incrementing ID + Base Conversion (Strongly Recommended)

This is the mainstream industry approach; Bitly and Twitter broadly use variants of it. It has two steps: use a globally unique ID generator to assign a number to each long URL, then encode that decimal number as a higher-base string.

const ALPHABET = '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz';
const BASE = 62; // 0-9 + A-Z + a-z

// Auto-incrementing ID → short code
function encode(id: number): string {
  let n = id;
  if (n === 0) return ALPHABET[0];
  let out = '';
  while (n > 0) {
    out = ALPHABET[n % BASE] + out;
    n = Math.floor(n / BASE);
  }
  return out;
}

// Short code → auto-incrementing ID (for reverse lookup and troubleshooting)
function decode(code: string): number {
  let n = 0;
  for (const ch of code) n = n * BASE + ALPHABET.indexOf(ch);
  return n;
}

encode(10000);          // '2bI'
encode(10_000_000_000); // 'Aukyoa' — even a 10-billion-scale ID needs only 6 characters

Why it works so well: uniqueness by construction because the ID generator guarantees it, predictable length because seven base62 characters provide approximately 3.5 trillion values and remain sufficient for decades, no duplicate checks because unique IDs produce unique codes, and reversibility because an ID can be recovered directly from a short code for troubleshooting.

The ID generator is a separate small system. The simplest implementation is a MySQL counter table using AUTO_INCREMENT, which is sufficient at medium scale. At larger scales, use Redis INCR, a Snowflake-style algorithm, or a dedicated ID-generation service. The important thing is to prevent the ID generator from becoming a single-point bottleneck.

Approach C: Random String + Duplicate Check

Generate a random seven-character string, then query the database for a collision. This is simple and direct, but adds a read to the write path and still requires probabilistic retries. Problems emerge at scale, so it is suitable only for demos.

Approach D: Pre-Generation (Keygenning)

Generate tens of millions of short codes in batches in the background and place them in a queue, then pop one for each write. The write path is extremely fast, collisions are impossible, and load on the ID generator is zero. The cost is maintaining another KGS. It is worthwhile in latency-sensitive, write-heavy scenarios and can be mentioned as a "bonus approach" in an interview.

3. Storage: One Table and Two Indexes Are Enough to Start

Do not reach for NoSQL immediately. The storage model of a URL shortener is extremely simple: it is a KV mapping, and a relational database is entirely sufficient.

CREATE TABLE url_map (
  id          BIGINT PRIMARY KEY AUTO_INCREMENT,
  short_code  VARCHAR(10) NOT NULL UNIQUE,
  long_url    TEXT        NOT NULL,
  user_id     BIGINT,
  created_at  DATETIME    NOT NULL,
  expires_at  DATETIME,
  KEY idx_user (user_id)
);

A unique index on short_code is mandatory. It is the record's "business primary key," and every read request begins with it. Keep id because it serves both as the source for ID generation and as the foundation for later sharding. Consider sharding when storage reaches 6TB, as discussed in Section 6. Before then, a MySQL primary-replica setup with sensible indexes can handle the workload on a single database.

4. Redirects: 301 or 302 Is a Product Decision

The redirect status code is not merely a technical detail; it is a product decision. Many people miss the distinction:

• 301 Moved Permanently: A permanent redirect. Browsers cache it aggressively, so the next click on the same short URL redirects locally without contacting the origin server.
• 302 Found: A temporary redirect. Browsers contact the origin every time, allowing the server to record every click.

Technically, 301 is faster and reduces server load. However, if the product requires click analytics, as advertising and email marketing almost always do, it must use 302. Otherwise, return visits are absent from click statistics and commercial reconciliation breaks down. This is why Bitly and similar URL-shortening platforms use 302 by default.

app.get('/:code', async (req, res) => {
  const longUrl = await lookup(req.params.code);
  if (!longUrl) return res.status(404).end();
  analytics.emit('click', { code: req.params.code, ...req.meta }); // Asynchronous and non-blocking
  res.redirect(302, longUrl);
});

One detail: analytics collection must be asynchronous. Redirect latency is highly noticeable. Synchronously writing to an analytics database can push P99 latency to several hundred milliseconds, making users think the link is broken.

5. Caching: The Natural Fit for Read-Heavy Workloads

A 100:1 read/write ratio means each short URL is read 100 times for every write, making this an ideal caching scenario. Redis is practically standard:

GET /:code
  ├─ 1. Query Redis (cache-aside)
  │     Hit → 302
  │     Miss ↓
  ├─ 2. Query MySQL
  │     Hit → Populate Redis (TTL of several hours to one day) → 302
  │     Miss → 404 (also cache the empty result to prevent penetration)

Practical considerations:

• The TTL does not need to match the short URL's expiration exactly. One to seven days is enough, and brief inconsistency after changing the long URL is tolerable.
• Prevent cache penetration: Cache an empty value for nonexistent short codes with a short TTL. Otherwise, an attacker can flood MySQL with random short codes.
• Prevent cache avalanches: Add random jitter to TTLs (ttl + rand(0, 60s)) so a batch of keys does not expire in the same second.
• Hot keys: A single viral short URL may account for 30% of traffic. Add a local LRU layer before Redis to eliminate most origin lookups.

6. Scaling: From a Single Node to Sharding

At this point, the system has moved beyond the "demo" stage and must handle real scale. The overall architecture looks like this:

                       Users
                         │
              ┌──────────┴──────────┐
              ▼                     ▼
          [CDN / Edge]       [Nginx Load Balancer]
       Return 302 on hit              │
              │ miss         ┌────────┴─────────┐
              ▼              ▼                  ▼
     [Read Service Cluster] [Write Service Cluster] [Analytics Service]
              │              │                  (asynchronous)
              ├──── Redis ───┤  (hot cache)
              ▼              ▼
       [MySQL Shards 0..N]  ← ID Generator (independent service)
       (hash by short_code)

Sharding strategy: Hash short_code across N databases. Do not shard by the auto-incrementing id: read requests contain only the short code, not the id, so sharding by id requires decoding before routing and adds unnecessary work. Always shard by the key available on the read path. This is the most common sharding mistake among less experienced engineers.

High availability: Give every MySQL shard a primary and replicas, run Redis in Sentinel or cluster mode, and keep write services stateless so they can scale horizontally. The ID generator is a single-point risk. Either use a Snowflake-style algorithm that assigns each machine a segment of the ID space, or use multiple MySQL primaries with reserved increment steps.

CDN: URL redirect responses are tiny, containing only a few hundred bytes of headers, which makes them well suited to CDN caching. However, this must be considered together with Section 4: CDN caching causes 302 click analytics to be lost. A common approach is: use CDN + 301 for popular links that do not require analytics, and origin + 302 for links that do.

7. How to "Design While Explaining" in an Interview

Finally, consider the methodology. What the interviewer actually wants to see is not whether you know base62, but whether you can:

1. Ask about requirements before drawing: Spend the first 30 seconds clarifying the read/write ratio, whether analytics are required, and whether custom short codes are supported. Skip this step and the rest of the solution will inevitably drift from the requirements.
2. Present a basic solution before optimizing it: Do not begin with "multi-level caching + sharding + KGS." Start with the minimum viable version, "one MySQL database + base62," then optimize one bottleneck at a time. A clear progression matters more than showing off complexity.
3. Explain capacity estimates clearly: State the QPS, storage, and bandwidth numbers to show that you can translate an abstract problem into concrete resources.
4. Proactively explain tradeoffs: For 301 vs. 302, sharding by id vs. code, and whether to use KGS, explain at each step, "Choose A because of X, at the cost of Y." A design without tradeoffs is a design that has not been thought through.
5. Admit what you do not know: If questions reach an unfamiliar area, such as clock rollback in the ID generator, say honestly, "I have not studied the details deeply; the general approach is XXX." That is better than making something up.

Summary

1. The first step in system design is always clarifying requirements. Ask about the read/write ratio, whether analytics are required, and whether custom short codes are supported before drawing anything.
2. The preferred short-code generation method is "auto-incrementing ID + base62". It is unique by construction, has predictable length, is reversible, and is the mainstream industry solution.
3. 301 vs. 302 is a product decision, not a technical detail. Click analytics require 302, even though every request must reach the origin.
4. Every stage of the read path must minimize latency and defend against cache penetration: Redis cache-aside + empty-value caching + TTL jitter + a local LRU for hot keys.
5. Shard by the key available on the read path, not by the database's internal id. This is one of the most common and damaging sharding mistakes among less experienced engineers.

One final sentence: The difficulty of a URL shortener lies not in any individual component, but in connecting generation, storage, redirection, caching, and scaling into one complete path. The interviewer wants to see whether you can clearly explain every tradeoff along that path.