Node.js Backend Engineering Essentials

Writing a few Express routes and getting basic database CRUD working is enough to claim you “know backend development”? Services that run locally, return 502s under load, and crash in the middle of the night a week after launch are never missing business logic; they are missing production engineering. Because Node.js combines a single-threaded execution model with a runtime that does not restart processes by default, it puts traditional backend concerns that “the framework handled for you” (process crashes, connection pools, and concurrency isolation) squarely in your hands. The following sections explain the production concerns every Node backend must address.

1. Writing Routes ≠ Backend Engineering

Many people's idea of a “backend” stops at a demo such as app.get('/users', ...). A production backend needs considerably more engineering: the process must not crash, logs must be searchable, secrets must not leak, the database must not be overwhelmed, slow tasks must not block requests, a multicore CPU must use more than one core, and production problems must be diagnosable within ten minutes.

These concerns are invisible in small projects, become intermittent failures at moderate traffic, and turn into daily firefighting at high traffic. Every section that follows covers lessons usually written down only after something has gone wrong in production.

2. Error Handling: Do Not Let the Process Crash

Node's most counterintuitive behavior is this: one uncaught synchronous exception or unhandled Promise rejection causes the entire process to exit by default. All 10,000 of your concurrent requests can disconnect because of a single JSON.parse(undefined).

Your safety net needs two layers:

// Process-level safety net: the last line of defense. Reaching this point means a bug escaped
process.on('uncaughtException', (err) => {
  logger.fatal({ err }, 'uncaughtException, preparing to exit');
  // Do not pretend everything is fine and keep running! State may already be inconsistent; let the process manager restart it
  server.close(() => process.exit(1));
  // Fallback: force exit within 0.5s to prevent a connection hang
  setTimeout(() => process.exit(1), 500).unref();
});

process.on('unhandledRejection', (reason) => {
  logger.fatal({ reason }, 'unhandledRejection');
  // Newer Node versions exit because of this by default; explicitly control the shutdown sequence here
  process.exit(1);
});

Why exit inside uncaughtException instead of continuing? When the error occurs, the process may be in an intermediate state while holding locks, connections, or file handles. Continuing means operating in a corrupted state and making the data progressively worse. The correct approach is: log the error → shut down gracefully → let PM2/systemd bring the process back up.

You also need a framework-level safety net (implemented differently in Express, Koa, and Fastify) that converts errors into consistent HTTP responses before they reach the top level:

// Express: place this after all routes
app.use((err, req, res, next) => {
  const status = err.status || 500;
  logger.error({ err, path: req.path }, 'request failed');
  res.status(status).json({
    error: status >= 500 ? 'internal_error' : err.message,
  });
});

One final rule: do not use throw for control flow. In Node, throw new Error('user not found') means unwinding the stack and potentially leaking the error to the process-level handler. Use return values or a Result type for expected business errors, and reserve throw for genuine surprises.

3. Logging and Observability: Stop Using console.log

In production, console.log is effectively no logging at all: it has no levels, timestamps, or trace context, and it cannot be searched effectively in ELK/Loki. Structured logging is the minimum standard:

import pino from 'pino';
const logger = pino({
  level: process.env.LOG_LEVEL || 'info',
  redact: ['req.headers.authorization', '*.password'], // Keep secrets out of logs
});

// Usage: every log entry is one line of machine-readable JSON
logger.info({ userId, durationMs }, 'login_success');

Three things are nonnegotiable:

1. Request ID / trace: Inject a reqId into every request and include it in every log entry, so all logs for one request can be correlated. OpenTelemetry also enables tracing across services.
2. Metrics instrumentation: QPS, p99 latency, error rate, connection-pool usage, and queue backlog cannot depend on “grep in the logs.” Export them to Prometheus and view them in Grafana.
3. Error aggregation: Tools such as Sentry automatically consolidate exception stacks. Do not depend on users sending screenshots to report bugs.

4. Configuration and Secrets: env Is Not a Silver Bullet

Do not hard-code secrets. Code such as const DB_URL = 'postgres://...' should get you fired on your first day in production. But you need to go further:

• Schema validation: If process.env.DB_URL is misspelled, fail during startup instead of waiting for the first request to break. Use zod / envalid for validation:

  const env = z.object({
    DATABASE_URL: z.string().url(),
    PORT: z.coerce.number().default(3000),
    JWT_SECRET: z.string().min(32),
  }).parse(process.env);
  

• Environment isolation: Use different secrets for dev/staging/prod. Store production secrets in Vault / AWS Secrets Manager / Alibaba Cloud KMS instead of putting them in .env and committing them to Git.
• Secret rotation: JWT signing keys, database passwords, and similar secrets must support two keys coexisting during a smooth cutover, rather than making everything fail immediately after a change.

Put .env files in .gitignore. That is the bare minimum.

5. Database Connection Pools: Do Not Create a New Connection Every Time

One of the biggest beginner mistakes is calling mysql.createConnection() inside every request, running a query, and then calling connection.end(). The TCP handshake and authentication overhead will destroy throughput.

The correct approach is to create one global connection pool. Requests borrow connections from the pool and return them after use:

import { Pool } from 'pg';
const pool = new Pool({
  connectionString: env.DATABASE_URL,
  max: 20,           // Number of connections; set according to the database's max_connections
  idleTimeoutMillis: 30_000,
});

// Inside a request
const { rows } = await pool.query('SELECT * FROM users WHERE id = $1', [userId]);

Should you use an ORM (Prisma / TypeORM) or a query builder (kysely / Knex)? The tradeoffs are practical:

• Prisma: Top-tier DX and strong type safety, but its abstractions get in the way of complex queries, and debugging slow queries can become a nightmare.
• TypeORM: Feature-rich and decorator-based, but its type inference and overall quality have drawn criticism for years.
• kysely: A fully type-safe SQL builder that stays close to native SQL, offers strong control, and suits performance-sensitive services.

If business complexity is high, CRUD dominates, and the team wants to move quickly, choose Prisma. If concurrency is high, SQL needs fine-grained control, and the team is willing to write SQL, choose kysely. I default to kysely for new projects.

6. Scheduled Jobs and Background Queues: Separate Them from Request Handling

Sending emails, generating reports, and cleaning up expired data are slow tasks. If they run synchronously inside a request, users wait ten seconds for a response, the experience falls apart, and the gateway may time out.

Two useful tools are:

• node-cron: In-process scheduled jobs, suitable for simple, single-instance, lightweight scenarios:

  import cron from 'node-cron';
  cron.schedule('0 3 * * *', () => cleanupExpiredSessions());
  

  This works fine with one instance. In a multi-instance deployment, the same job runs multiple times, so you need a distributed lock.

• BullMQ: A Redis-backed queue suitable for production:

  import { Queue, Worker } from 'bullmq';
  const emailQueue = new Queue('email', { connection: redis });
  await emailQueue.add('send', { to, subject }, { attempts: 3, backoff: { type: 'exponential', delay: 1000 } });
  
  new Worker('email', async (job) => sendMail(job.data), { connection: redis });
  

  Retries, delays, priorities, concurrency control, and dead-letter queues are all provided by BullMQ.

The principle is: the API process only orchestrates; send heavy work to Workers. This keeps the API fast, and Worker failures do not affect request responses.

7. Process Management: PM2 / cluster and Full Multicore Utilization

By default, Node runs one process with one thread. Running it on a 16-core machine wastes 15 cores. There are two options:

1. cluster module: Built into Node. The primary process forks multiple child processes that share a port:

   import cluster from 'node:cluster';
   if (cluster.isPrimary) {
     for (let i = 0; i < os.cpus().length; i++) cluster.fork();
   } else {
     startServer();
   }
   

2. PM2: Handles clustering, log rotation, automatic restarts, and zero-downtime reloads:

   pm2 start dist/main.js -i max   # Start one process per core
   pm2 reload all                  # Reload the cluster without downtime
   

Do you still need PM2 in the container era? With Kubernetes, the usual model is one process per container, with multiple replicas and restarts managed by K8s. PM2 suits small and medium deployments where you have one machine running Node services.

Memory leak monitoring: Node's default heap limit is about 1.5GB on 64-bit systems. If memory usage in a long-running service keeps growing, common causes include closures, Map caches, and listeners that were never removed. Monitor process.memoryUsage().heapUsed, or set a memory limit in PM2 / Docker and restart automatically when it is exceeded:

pm2 start app.js --max-memory-restart 1G

Graceful shutdown: When SIGTERM arrives during deployment or scale-down, do not exit immediately. First stop accepting new requests → finish in-flight requests → close the database connection pool → then exit. Otherwise, requests will be lost and connections terminated:

process.on('SIGTERM', async () => {
  logger.info('SIGTERM received, shutting down');
  server.close();
  await pool.end();
  await emailQueue.close();
  process.exit(0);
});

8. The Truth About Node's Single Thread

The biggest misconception is: “Node is single-threaded, so it cannot handle concurrency.” More precisely:

• JavaScript code runs on the main thread and can do only one thing at a time.
• I/O (network, disk, and database operations) is asynchronous and nonblocking. Underneath, libuv handles it with a thread pool, so the main thread does not wait and can keep tens of thousands of I/O operations in flight.

This makes Node especially well suited to I/O-bound workloads: API gateways, chat rooms, crawler schedulers, and proxy servers. In scenarios where “90% of the time is spent waiting,” one Node machine can comfortably handle tens of thousands of connections.

Its real weakness is CPU-bound work: parsing large JSON documents, processing images, encryption and decryption, and complex algorithms. Running that work on the main thread blocks every request and makes p99 latency explode. The options are:

• worker_threads: Run CPU-intensive code in workers within the same process and share memory through SharedArrayBuffer.
• Child processes: Isolate the work more thoroughly by running separate scripts and communicating over IPC.
• Move it out: Let a dedicated Go / Rust service handle the work and call it from Node over HTTP / gRPC.

import { Worker } from 'node:worker_threads';
const worker = new Worker('./heavy-calc.js', { workerData: input });
worker.on('message', (result) => res.json(result));

Do not use Node for services that require dense, pure CPU computation, such as machine-learning inference or video transcoding. Choosing the wrong tool is more damaging than writing the wrong code.

Summary

• Error handling is the first required subject for Node backends: use uncaughtException / unhandledRejection as final safeguards, normalize error responses at the framework layer, and do not use throw for control flow.
• The logging and observability trio is structured logging (pino) + request tracing (reqId / OpenTelemetry) + metrics instrumentation (Prometheus / Sentry).
• Use configuration schema validation + secrets in Vault. Do not put a database password in Git on day one.
• Use a database connection pool + queues for slow tasks (BullMQ). The API process only orchestrates; Workers handle the heavy work.
• Use cluster / PM2 for multiple cores + graceful SIGTERM shutdown, and understand the truth about Node's single thread: asynchronous I/O is its strength, while CPU-intensive work belongs in workers.

Writing routes is only the entry point; production engineering is the backend's moat. Production stays healthy not because of luck, but because those “invisible safety nets” are working.