Intro

I love reading the engine room of a system. The loops, the hooks, the unglamorous chores—they tell you how a project really thinks. Hi, I’m Mahmoud Zalt. Today I’m diving into the beating heart of Redis: src/server.c from the redis/redis repository.

Redis is a blazing-fast in-memory data store and message broker written in C, built around an event-driven Reactor model with careful orchestration of persistence (RDB/AOF), replication, modules, scripting, and operational commands. This file wires it all together—initialization, event loop hooks, cron, command dispatch, shutdown—everything.

In this article, we’ll examine how server.c structures the runtime, why its design works under extreme load, and where we can make it easier to evolve. You’ll walk away with practical insights for maintainability, extensibility, dev‑experience, and performance—grounded in real code and tests.

Roadmap: How It Works → What’s Brilliant → Areas for Improvement → Performance at Scale → Conclusion.

redis/
  src/
    ae.c (event loop)
    networking/ (conn*)
    rdb.c, aof.c (persistence)
    replication.c
    cluster.c
    modules/*
    server.c  <— orchestrator
      - initServer/initListeners
      - beforeSleep/afterSleep
      - serverCron
      - processCommand/call
      - shutdown/signals

How It Works

From the intro we zoom into execution. This section traces the main pipeline: initialization → event loop → command lifecycle → periodic work.

Runtime responsibilities

server.c coordinates:

• Initialization: global state, event loop, listeners, modules, ACL defaults.
• Command registry: populates tables and supports lookup and subcommands.
• Event loop hooks: beforeSleep/afterSleep for pre/post IO work.
• Cron: serverCron does periodic, bounded maintenance.
• Command lifecycle: processCommand preflights; call executes and propagates.
• Persistence/replication orchestration: RDB/AOF scheduling, fork child management, offsets.
• Operational commands: INFO, COMMAND, PING, SHUTDOWN—observability and control.
• Graceful shutdown: prepareForShutdown pauses actions and waits for replicas when needed.

Public API and side effects

• int serverCron(...): periodic scheduler invoked server.hz times/sec. Handles expire sampling, incremental rehash, persistence checks, replication, metrics. Mutates global server, can start/finish children, close clients, evict memory.
• int processCommand(client *c): parses and preflights (arity, ACL, loading state, cluster redirection), then queues or executes via call. May change client state, propagate writes, or postpone.
• void call(client *c, int flags): executes a command, records duration/slowlog, and handles AOF/replication propagation. Updates latency histograms.
• void beforeSleep(...)/void afterSleep(...): pre-/post-event loop hooks for draining writes, flushing AOF, tracking invalidations, acquiring/releasing module GIL, cached time, latency snapshots.
• void initServer(void)/void initListeners(void): core initialization and listener setup across TCP/TLS/UNIX.
• void infoCommand(client *c): builds INFO output from many subsystems and metrics.
• int prepareForShutdown(int flags): coordinates controlled shutdowns, including replica acks and timeouts.

Data flow

Requests flow from network events to connAcceptHandler, into the parser to populate c->argv/argc, then through processCommand preflight checks. If not queued by MULTI, execution enters call() where the command handler (cmd->proc) runs and mutations are propagated. Meanwhile, serverCron and beforeSleep/afterSleep keep the world cohesive: clocks are updated, buffers flushed, incremental work bounded, metrics sampled.

Invariants worth noting

• Global server is the source of truth.
• When execution nesting returns to zero, all pending propagations flush atomically.
• Command time snapshot remains consistent within the execution unit.
• Loading-state gating prevents non-allowed commands when server.loading is set.
• RDB/AOF/module fork children are mutually exclusive to control CoW and safety.

Key entry points in code

int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
    /* Software watchdog */
    if (server.watchdog_period) watchdogScheduleSignal(server.watchdog_period);
    server.hz = server.config_hz;
    if (server.dynamic_hz) { /* scale with clients */ }
    if (server.pause_cron) return 1000/server.hz;
    /* metrics sampling and run_with_period slots */
    server.lruclock = getLRUClock();
    cronUpdateMemoryStats();
    /* Shutdown handling */
    /* Clients cron, databases cron, persistence checks */
    return 1000/server.hz;
}

void call(client *c, int flags) {
    long long dirty;
    uint64_t client_old_flags = c->flags;
    struct redisCommand *real_cmd = c->realcmd;
    client *prev_client = server.executing_client;
    server.executing_client = c;
    /* ... */
    c->cmd->proc(c);
    /* ... propagation and stats ... */
}

int prepareForShutdown(int flags) {
    if (isShutdownInitiated()) return C_ERR;
    if (server.loading || server.sentinel_mode)
        flags = (flags & ~SHUTDOWN_SAVE) | SHUTDOWN_NOSAVE;
    server.shutdown_flags = flags;
    serverLog(LL_NOTICE,"User requested shutdown...");
    if (!(flags & SHUTDOWN_NOW) && server.shutdown_timeout != 0 && !isReadyToShutdown()) {
        server.shutdown_mstime = server.mstime + server.shutdown_timeout * 1000;
        if (!isPausedActions(PAUSE_ACTION_REPLICA)) sendGetackToReplicas();
        pauseActions(PAUSE_DURING_SHUTDOWN, LLONG_MAX, PAUSE_ACTIONS_CLIENT_WRITE_SET);
        return C_ERR;
    }
    return finishShutdown();
}

void pingCommand(client *c) {
    if (c->argc > 2) {
        addReplyErrorArity(c);
        return;
    }
    if (c->flags & CLIENT_PUBSUB && c->resp == 2) {
        addReply(c,shared.mbulkhdr[2]);
        addReplyBulkCBuffer(c,"pong",4);
        if (c->argc == 1) addReplyBulkCBuffer(c,"",0);
        else addReplyBulk(c,c->argv[1]);
    } else {
        if (c->argc == 1) addReply(c,shared.pong);
        else addReplyBulk(c,c->argv[1]);
    }
}

What’s Brilliant

With the foundation in view, let’s highlight design choices that pay off in production.

1) A pragmatic reactor with time-bounded background work

The event loop integrates beforeSleep/afterSleep hooks and a periodic serverCron to amortize all background tasks (expire sampling, incremental rehash/defrag, persistence checks, module events). Work is partitioned into run_with_period slots, keeping tail latencies down even under heavy client counts via dynamic_hz scaling.

2) Command pipeline with explicit preflight and unified execution

processCommand gates every call with arity, ACL, stale/loading checks, and cluster routing before reaching call(). This separation clarifies the hot path and enables well-defined places to add policy.

3) Atomic propagation via execution units

The architecture tracks execution nesting and flushes pending AOF/replication writes when it returns to zero. This provides transactional consistency for complex commands, script batches, and chained work.

4) Efficient memory and CoW awareness

server.c coordinates forked children and tunes CoW via buffer dismissal and resize policies. Incremental defrag and sample-based metrics keep overhead low.

5) Observability built into core paths

Durations are categorized (event loop, commands, AOF, cron), command histograms track latencies, and INFO aggregates everything, including ACL/error counters. The suggested metrics make it actionable to operate:

• eventloop_duration_usec: p99 end-to-end loop time (target p99 < 5ms).
• aof_fsync_latency_ms: surface disk stalls (p99 < 10ms typical target).
• fork_time_us: catch pauses during persistence (alert >= 500ms).
• clients_blocked, replication_offset_lag: backpressure and safety.

Areas for Improvement

Next, the pragmatic tradeoffs. This file is a workhorse; these ideas lower cognitive load and improve testability without losing performance.

Refactor sketch: Extract command preflight

Extracting the preflight logic from processCommand reduces cyclomatic complexity and makes unit-level testing practical for ACL/loading/cluster order.

*** a/src/server.c
--- b/src/server.c
@@
-int processCommand(client *c) {
+int processCommand(client *c) {
+    if (!preflightCommand(c)) return C_OK; /* unified rejections handled inside */
     /* existing routing / MULTI / call path remains */
 }
+
+/* New helper encapsulating arity, ACL, state (loading/paused/deny-stale), and cluster redirection. */
+static int preflightCommand(client *c) {
+    sds err = NULL;
+    if (!commandCheckExistence(c, &err)) { rejectCommandSds(c, err); return 0; }
+    if (!commandCheckArity(c->cmd, c->argc, &err)) { rejectCommandSds(c, err); return 0; }
+    if (!preflightAclAndState(c)) return 0;
+    return 1;
+}

Refactor sketch: Isolate INFO section builders

*** a/src/server.c
--- b/src/server.c
@@
-sds genRedisInfoString(dict *section_dict, int all_sections, int everything) {
-   /* ... very long ... */
-}
+/* Moved to info_sections.c: genRedisInfoString and helpers */

Refactor sketch: unify rejection accounting

*** a/src/server.c
--- b/src/server.c
@@
-void rejectCommand(client *c, robj *reply) {
-    flagTransaction(c);
-    c->duration = 0;
-    if (c->cmd) c->cmd->rejected_calls++;
+static inline void incrRejected(client *c) { if (c->cmd) c->cmd->rejected_calls++; }
+void rejectCommand(client *c, robj *reply) {
+    flagTransaction(c);
+    c->duration = 0;
+    incrRejected(c);
     /* ... */
 }

Performance at Scale

Armed with the structure and improvements, let’s focus on scale, latency, and operations.

Hot paths

• Command execution: processCommand → call → cmd->proc. Framework overhead remains O(1); dict lookups dominate lookup; actual cost depends on command-specific logic.
• beforeSleep: drains handleClientsWithPendingWrites, flushes AOF, pushes invalidations, trims replication backlog.
• clientsCron: output/query buffer resize, timeouts, eviction candidates.

Bounded background work

Periodic tasks are sampled and incremental to avoid eventloop stalls. Rehash/defrag and expiration are time-budgeted. dynamic_hz scales cron frequency with client counts to keep up.

Concurrency model

Redis remains single-threaded for command execution with optional IO threads for offloading reads/writes. Module GIL enforces safety across module threads. Some counters/shutdown flags use atomics.

Latency risks to watch

• Long-running commands (CPU-bound computations).
• fsync stalls (AOF), disk slowness.
• Fork pauses (RDB/AOF rewrite).
• Cluster checks under heavy load.

Operational metrics and SLOs

• eventloop_duration_usec (p99 < 5ms): alert on spikes; correlate with command histograms.
• aof_fsync_latency_ms (p99 < 10ms): increases point to disk contention; consider appendfsync policy and storage tier.
• fork_time_us (< 100ms typical; alert ≥ 500ms): noisy neighbors or huge RSS; consider reducing CoW via buffer policies or tuning save cadence.
• clients_blocked: correlate with backpressure and blocked commands; ensure bounded waiting via timeouts.
• replication_offset_lag: keeps failover safe; required for graceful shutdown waits.

Observability hooks

• Logs: startup banner, listeners, fork timings, child lifecycle, replication transitions, disk errors, shutdown flow.
• Metrics: eventloop cycles/durations (EL_DURATION types), net IO (including replication), AOF status and rewrites/saves, client memory buckets, replication offsets/backlog histlen.
• Traces: per-command duration histogram; latency percentiles.
• Alerts: AOF write/fsync errors, failed RDB saves, replication down/lagging, fork time spikes, OOM/eviction anomalies, eventloop duration spikes.

Test plan highlights

Production-grade confidence comes from tests that exercise policy gates and propagation semantics. Here are practical tests derived from the code’s behavior:

1) ACL denial on unauthorized write

# Setup: connect without authentication (default user requires password)
redis-cli SET a 1
# Expect: -NOAUTH error; rejected_calls incremented; no AOF/replication propagation

Validates preflight ACL enforcement in processCommand and correct rejection accounting.

2) Loading state denial

# Simulate: server.loading=1
# Issue: a non-CMD_LOADING command
redis-cli GET x
# Expect: -LOADING error; no side effects; PING still allowed

Checks state gating during load to prevent inconsistent reads/writes.

3) AOF propagation batching

# Run a command that cascades two writes in one execution unit
# Expect: AOF sequence contains MULTI, the two commands, then EXEC

Confirms the atomic propagation behavior of alsoPropagate and the transaction wrapper.

4) Graceful shutdown waits for replicas

# With one lagging replica
redis-cli SHUTDOWN   # no NOW flag
# Expect: logs show pause + waiting for ACK; exit only after ack or timeout

Exercises prepareForShutdown coordination and ack-driven exit conditions.

Conclusion

We walked through server.c—the orchestrator of Redis. Its careful balance of a single-threaded reactor, bounded background work, and atomic propagation keeps performance tight and correctness high.

• Keep hot paths simple and measured. The preflight/execute split and execution unit flushes are instructive patterns.
• Invest in observability. The eventloop and command histograms make regressions obvious and root causes actionable.
• Pay down complexity. Extracting preflight logic and INFO builders improves testability and long-term maintainability.

If you maintain a high-throughput service, borrow these patterns. And if you work on Redis itself: small, focused refactors here will compound in developer velocity without sacrificing the speed that makes Redis beloved.