What Is Redis? The In-Memory Data Store for Caching

Redis (Remote Dictionary Server) is an open-source, in-memory key-value database used extensively as a distributed cache and message broker. Salvatore Sanfilippo created it in 2009 to fix the scalability bottlenecks in a real-time web log analyser, and it has since become a standard component for high-performance data management. By holding its entire dataset in RAM, Redis gives you sub-millisecond latency on reads and writes that disk I/O would otherwise throttle.

You reach for Redis when your architecture demands immediate data access and a versatile set of abstract data structures. It is primarily an in-memory store, but it offers optional durability through several disk persistence mechanisms. That combination lets it act not just as a volatile cache but as a reliable state manager, from real-time analytics engines to high-dimensional vector search for AI workloads.

The system is written in ANSI C and runs on POSIX systems like Linux and BSD. It operates as a single-process server, traditionally following a single-threaded execution model for command processing.

How does Redis actually work?

Redis operates as a Remote Dictionary Server where all data is read from and modified in main memory. That design eliminates disk-seek latency and random I/O bottlenecks. A traditional relational database (RDBMS) relies on complex query optimizers and secondary indexes; Redis instead uses a data model where commands operate directly on abstract data types. Data structures are stored in formats optimised for direct memory retrieval, so operations stay predictable and fast.

The core engine executes commands as a single-threaded process. That matters because it removes the need for complex locking mechanisms and mutexes. In multi-threaded databases, lock contention often degrades performance when several clients touch the same structure. By staying single-threaded for execution, Redis avoids context switching and locking overhead, and can run operations like set intersections (SINTER) or list pushes (LPUSH) at full throughput. Background tasks — rewriting the Append Only File, for instance — move onto separate threads so maintenance never blocks client requests.

For durability, Redis uses the fork system call to create a child process that benefits from copy-on-write (CoW) semantics. When persistence triggers, the child receives a point-in-time snapshot of the dataset's memory space. The child does the heavy lifting of writing that data to the filesystem while the parent keeps serving clients. Disk access stays sequential rather than random, which is most of why the persistence layer performs as well as it does.

Data is only reconstructed from disk into RAM during a restart. While the server is up, Redis holds the entire state in memory and syncs changes to the filesystem on the intervals you define. You get the speed of a volatile cache with the safety of a durable database — provided the persistence policies actually match your recovery point objectives (RPO).

What data types can Redis store?

Redis is a data structure server, because its keys map to specialised values rather than opaque blobs. The foundational type is the String, which holds up to 512MB. Beyond plain text, you can treat Strings as Bitmaps or Bitfields; Bitfields encode multiple counters inside a single string value with atomic increment and overflow policies. Lists are collections of strings sorted by insertion order, used for message queues or task stacks where O(1) head and tail access matters.

Hashes, Sets, and Sorted Sets define the rest of the range. Hashes are field-value pairs that behave like Python dictionaries, which makes them a good fit for structured objects such as user profiles or configuration state. Sets are unordered collections of unique strings supporting fast membership tests and set arithmetic. Sorted Sets (ZSets) associate each member with a numerical score, giving you an ordered collection — the foundation for real-time leaderboards and priority queues.

With Redis 8, advanced types integrate directly into the core package. These include JSON documents for hierarchical data, Geospatial indexes for radius queries, and Time Series for timestamped telemetry. There are probabilistic types too: Bloom filters, Cuckoo filters, HyperLogLog. Each one performs membership checks or cardinality estimation with very high memory efficiency, in exchange for a small and tunable margin of error.

One of the most significant recent additions is the Vector Set, built for the high-dimensional embeddings used in AI and semantic search. Vector Sets support the HNSW (Hierarchical Navigable Small World) algorithm for fast similarity search using cosine similarity. Combine vector similarity with structured filters and you can build recommendation systems and context-retrieval pipelines inside the same low-latency memory space as your cache.

# Setting a Redis Hash for a high-performance user session
HSET session:4592 user_id "881" status "active" locale "en-US"
# Adding members to a Sorted Set for a real-time leaderboard
ZADD game:scores 1500 "player_alpha" 2300 "player_beta"

Does your data survive a Redis restart?

Durability runs through two mechanisms: RDB (Redis Database) and AOF (Append Only File). RDB creates compact, binary, point-in-time snapshots of the entire dataset at intervals you configure — say, if 1,000 keys change within 60 seconds. RDB is efficient for disaster recovery and gives noticeably faster restarts on large datasets, because it never replays individual command logs. The trade-off is inherent to snapshots: anything written between the last snapshot and a crash is gone.

AOF provides higher durability by logging every write operation the server receives. Redis 7 introduced Multi-Part AOF (MP-AOF), which splits the log into a base file representing a point-in-time state and incremental files for ongoing writes. A manifest file tracks them, so the system can reconstruct state without risking corruption during a rewrite. Set appendfsync everysec and you cap potential data loss at one second while keeping throughput close to RDB-only mode.

Redis 8.10.0 added the BACKUP command family — a self-contained, restorable backup that does not interrupt active writes. The workflow produces an MP-AOF compatible artifact set: a BASE snapshot, an INCR file for writes during the backup window, and a standalone manifest. You seal a backup at a specific boundary, so the final artifact reflects state at the moment of finalisation rather than the earlier snapshot. It is a sturdier pipeline than copying .rdb or .aof files by hand.

For production systems that need maximum safety, enable both. You get RDB's fast recovery and compact archiving alongside AOF's granular command log. On restart, Redis loads the AOF, since it is the more complete record.

# Durability settings in redis.conf
appendonly yes
appendfsync everysec
# Snapshotting: save if 1000 keys change in 60s
save 60 1000

What do teams actually use Redis for?

The most common use is session and full-page caching. Moving frequently accessed user data or rendered fragments into RAM cuts load on your primary relational database — which matters most on e-commerce and social platforms, where sub-millisecond response times are a prerequisite for keeping users around.

Real-time leaderboards and counting come next, built on Sorted Sets. In gaming or financial analytics, you update a score and immediately read that user's global rank among millions with O(log(N)) complexity — a query that gets expensive fast in a standard SQL environment. The same structures back rate limiting and deduplication, where you track request counts per IP to prevent API abuse.

Redis also works as a message broker through Pub/Sub and Streams. Pub/Sub is lightweight: publishers send data to channels for immediate consumption. Streams provide an append-only log for event-driven pipelines, and unlike Pub/Sub they support consumer groups and message persistence, so multiple workers can process a feed at their own pace.

In AI stacks, Redis doubles as a vector store. Teams keep high-dimensional embeddings in memory and run semantic search against them, retrieving context for large language models with minimal latency. Paired with ordinary caching, that makes Redis a single layer for both application state and machine learning feature storage.

How do you cache with Redis correctly?

Caching correctly starts with choosing between Lazy Caching (Cache-Aside) and Write-Through. In Lazy Caching, the application populates the cache only after a miss, so the cache holds only data someone is actually reading — at the cost of a database query on every first request. Write-Through updates the cache the moment the database changes, keeping data ready in RAM for the next read. That suits performance-critical records like user profiles, though it churns the cache when data is written often and read rarely.

Eviction policy is the next decision. Redis lets you define a maxmemory limit and a policy for which keys to drop when you hit it. allkeys-lru (least recently used) and volatile-lfu (least frequently used among keys carrying a TTL) are the common picks. Always assign a Time-to-Live so stale data cannot linger. And add a little jitter to your TTL values — otherwise a popular key expires and thousands of processes hit your database at once, the "thundering herd".

For nested data models, there is Russian Doll Caching, a pattern that came out of work by the Ruby on Rails team. Nested records — a user's comments within a story — are cached individually under their own keys, and the top-level resource is a collection of those keys. Invalidation then gets granular: change one comment and you expire one key, not the whole page cache.

Finally, prewarm the cache. A new node starts with empty memory, which can spike database load the moment it joins. Run a script that replays common application requests before you attach the node, so it is useful from its first request rather than its thousandth.

# Eviction policy in redis.conf for a cache-heavy workload
maxmemory 4gb
maxmemory-policy allkeys-lru
 
# Setting a key with jittered TTL (pseudocode logic)
# EXPIRE session:user456 (3600 + rand(120))

How does Redis scale and stay available?

Horizontal scaling runs through Redis Cluster, which shards data across nodes. Cluster does not use consistent hashing; it uses a fixed 16,384 hash slots. Storing a key means computing CRC16 of the key modulo 16,384 to find its slot, and each node owns a subset of slots. Because slots migrate between nodes while the cluster stays online, you can add or remove nodes without downtime.

Every node needs two open TCP connections: the standard data port (6379, typically) and the Cluster Bus Port, which is always the data port plus 10,000 — so 16379. The bus is a node-to-node channel for failure detection and failover authorisation. Running Cluster under Docker, use host networking mode so the bus and node IPs advertise correctly; NATted environments break the internal gossip protocol.

High availability comes from the Master-Replica model. Each master can carry one or more replicas, and if a master goes unreachable for longer than cluster-node-timeout, a replica is promoted automatically. Replication is asynchronous, though, which leaves a window for data loss during a partition: a master can acknowledge a write and crash before propagating it, and that write is gone.

The WAIT command narrows the window by blocking the client until a given number of replicas acknowledge the write. Understand what it does and doesn't buy you — WAIT is better than best effort, but it is not strong consistency and not linearizability. If a replica missing the latest write gets promoted during a messy partition, you still lose data. That trade, synchronous acknowledgment against latency, is yours to make.

Redis or Memcached, which one do you need?

The choice comes down to data complexity and durability. Memcached is a high-performance, multi-threaded memory object caching system built for simplicity, with a straightforward key-value model for strings and small objects. Its multi-threaded architecture scales vertically on high-core-count servers, which makes it effective for simple read/write pools where no data manipulation is needed.

Redis is more versatile, supporting a wide range of data structures. Where Memcached is strictly a key-string pool, Redis performs atomic operations on hashes, lists, and sets server-side — so you stop shipping large payloads to the application just to modify them. Redis also has built-in persistence, while Memcached is volatile by design: restart the process and the data is gone.

Reach for Memcached when you need simple, high-throughput key-value storage, data structures are irrelevant, and multi-threading gives you a vertical scaling advantage. Reach for Redis when you need durability, complex data types, messaging, or horizontal scaling through native Cluster mode.

Memcached remains a solid choice for legacy stacks and plain cache pools. Redis has become the default for most new infrastructure, largely because handling Pub/Sub, Lua scripting, and vector search in one engine removes several moving parts from the architecture.

Is Redis still open source?

Redis licensing has changed more than once. It shipped originally under the permissive BSD-3 licence, then became the centre of a long argument about cloud hyperscalers profiting from managed Redis services without contributing back. In 2024, starting with version 7.4, the project moved to a dual-licensing model: the Redis Source Available License v2 (RSALv2) and the Server Side Public License v1 (SSPLv1). That change stopped cloud providers offering Redis as a competing service without a commercial agreement — and prompted the Linux Foundation to fork the last BSD-licensed version as Valkey.

In May 2025, Redis moved to a tri-licence model for version 8.0, adding the GNU Affero General Public License v3 (AGPLv3). AGPLv3 is OSI-approved, so this returned the core project to open source proper. It lets the community use, modify, and distribute the code, while requiring improvements to be shared back when the software is used over a network.

Salvatore Sanfilippo rejoined the project in late 2024 as a developer evangelist, and that return shaped the move back to an open-source licence. Under his guidance, technologies previously confined to Redis Stack — JSON, Time Series, the Redis Query Engine — were integrated directly into the core Redis 8 package under AGPLv3.

That unified distribution ends the split between Community Edition and Stack. You now get the full suite of advanced data types and search capabilities in a single OSI-compliant package, alongside the source-available licences Redis Ltd. keeps for its commercial position.

When should you reach for Redis?

Reach for Redis when sub-millisecond latency is non-negotiable and your application needs more than simple key-value lookups. It is the right choice for architectures that perform complex atomic operations server-side — real-time leaderboards, event-driven streams, distributed session state with durability. If your workload is write-heavy and concurrent, and you want to avoid the locking penalties of a traditional RDBMS, Redis gives you a predictable, high-throughput alternative.

For AI infrastructure, native Vector Sets and HNSW indexing make Redis a practical store for semantic search and agent memory. Use it when you want one engine that can be a cache, a message broker, and a vector store at once — that is one less moving part in the stack, without giving up the latency those workloads depend on.

References

• What is Redis? In-memory database, cache, and message broker
• Redis data types — Redis Docs
• Redis persistence — Redis Docs
• Scale with Redis Cluster — Redis Docs
• Redis is now available under the AGPLv3 open source license — Redis
• Redis — Wikipedia
• Memcached vs Redis: Choose Your In-Memory Cache — Kinsta
• Caching Best Practices — Amazon Web Services