The Storage Landscape

From variables and files to databases — understanding where web applications store data

From Flat Files to Full Databases

Your REST API needs somewhere to store data. Understanding your storage options — from flat files to relational databases to document stores — is fundamental to building real web applications.

This tutorial series covers the full storage landscape: when to use each option, how to connect from Node.js and PHP, how to avoid SQL injection, and how to choose the right tool for your project.

The Spectrum of Storage

Data storage isn't a linear progression — it's a landscape of trade-offs. Each technology optimizes for different things: setup simplicity, query flexibility, concurrency, speed, or durability. There's no "best" option, only the right tool for a given job.

The Storage Landscape:

  NO SERVER REQUIRED                      REQUIRES A SERVER
  (embedded / file-based)                 (separate process)

  ┌──────────────┐ ┌──────────────┐       ┌──────────────┐ ┌──────────────┐
  │  In-Memory   │ │  Flat Files  │       │ PostgreSQL / │ │   MongoDB    │
  │  (variables, │ │  (JSON, CSV, │       │    MySQL     │ │  (document   │
  │   arrays)    │ │   log files) │       │  (full RDBMS)│ │    store)    │
  ├──────────────┤ ├──────────────┤       ├──────────────┤ ├──────────────┤
  │ Fastest      │ │ Simple I/O   │       │ Full SQL     │ │ Flexible     │
  │ Zero setup   │ │ No server    │       │ ACID safe    │ │ schemas      │
  │ Volatile     │ │ No queries   │       │ Multi-user   │ │ Good scaling │
  └──────────────┘ └──────────────┘       └──────────────┘ └──────────────┘

  ┌──────────────┐                        ┌──────────────┐
  │    SQLite     │                        │    Redis     │
  │  (embedded   │                        │  (in-memory  │
  │   file DB)   │                        │   key-value) │
  ├──────────────┤                        ├──────────────┤
  │ Real SQL     │                        │ Blazing fast │
  │ No server    │                        │ TTL/expiry   │
  │ Single-user  │                        │ Volatile     │
  └──────────────┘                        └──────────────┘

  Different tools for different jobs — not a ranking.

Comparing Storage Options

Each storage type excels in different areas. The following table summarizes their setup complexity, query capabilities, concurrency support, and ideal use cases:

Storage Type Setup Complexity Query Power Concurrency Best For
In-Memory None Code-level (loops, filters) Single process only Prototyping, caching, Node.js dev
Flat Files (JSON, CSV) Minimal Read/parse entire file Poor — race conditions Tutorials, config files, small datasets
SQLite Low (no server) Full SQL Single writer at a time Mobile apps, embedded systems, dev/test
PostgreSQL / MySQL Medium (server process) Full SQL + extensions Excellent — ACID transactions Production web apps, anything serious
MongoDB Medium (server process) Document queries, aggregation Good — built for scale Flexible schemas, rapid iteration
Redis Medium (server process) Key-value, basic structures Excellent — single-threaded, atomic Caching, sessions, rate limiting, queues

Persistence vs Volatility

One of the most fundamental distinctions in data storage is whether data survives a restart:

  • Volatile storage (in-memory variables, Redis without persistence) — data disappears when the process stops. Fast, but temporary.
  • Persistent storage (files, SQLite, PostgreSQL, MongoDB) — data survives restarts and crashes. Slower due to disk I/O, but durable.

In Node.js, you might start with an in-memory array during development. This works fine for a single developer testing an API — but the moment the server restarts, all data is gone. PHP doesn't even have this option: its per-request execution model means every variable is created and destroyed with each HTTP request.

This is why even the simplest production applications need some form of persistent storage — whether that's a JSON file on disk or a full database server.

The Storage Decision Tree

When deciding how to store data for a web application, ask these questions in order:

  1. Do I need data to survive a restart? If no, in-memory is fine (prototyping, caching). If yes, continue.
  2. Will multiple users write data at the same time? If no, flat files or SQLite may work. If yes, continue.
  3. Does my data have a consistent, well-defined structure? If yes, use a relational database (PostgreSQL, MySQL). If no, consider a document database (MongoDB).
  4. Do I need blazing-fast reads with simple key lookups? Add Redis as a caching layer alongside your primary database.

For this course, we'll progress through these options: we started with in-memory arrays in Node.js, used flat-file JSON storage in our REST tutorials, and now we'll move to relational databases with SQL — the standard for production web applications.