Open Source
Valkey (iovalkey)

Published: August 15, 2026 · by Srinu Desetti · valkey-io/iovalkey#62 (opens in a new tab)

The Stale Timer Bug That Destroyed Healthy Connections in iovalkey

iovalkey is Valkey's official Node.js client — Valkey being the Linux Foundation project backed by AWS, Google and Oracle. My contribution fixed a bug where a socket timeout timer could outlive its socket, fire after a reconnect, and destroy the new, healthy connection — triggering endless reconnect loops.

The Setup

iovalkey talks to Valkey over a TCP socket. In Node.js, a TCP socket is a Duplex Stream, which is why the code refers to it as this.stream:

Your Application → iovalkey → this.stream (TCP socket) → Valkey

Two naming conventions for this article:

  • S1 — the first socket connection
  • S2 — the new socket created after S1 closes and iovalkey reconnects

The critical fact: S1 and S2 are different socket instances, but this.stream always points to the current one.

The socketTimeout Safety Mechanism

When a command is sent, iovalkey arms a timer: "if the expected data doesn't arrive within N ms, destroy the connection." In the normal flow, the timer is cleared as soon as data arrives:

iovalkey                         Valkey
   |──── GET name ─────────────────>|
   |     timer armed (500ms)        |
   |<──── response ─────────────────|
   |     "data" event               |
   |     clearTimeout()  ✅         |

The key insight for the bug: the socket and the timer are two separate things.

NETWORK               JAVASCRIPT
S1                    Timer
└── TCP connection    └── setTimeout countdown

Destroying the socket does not automatically run clearTimeout(). The program must cancel the timer explicitly.

The Bug, Step by Step

The original code only cleared the timer when data arrived:

lib/Redis.ts — before
this.stream.once("data", () => {
  clearTimeout(this.socketTimeoutTimer);
});

Now watch what happens when the socket dies before any data arrives (socketTimeout = 500ms):

  0ms   S1 connected
 10ms   GET sent — timer armed, will fire at ~510ms
100ms   S1 dies  ❌  (network blip)
        → no "data" event ever fired
        → timer is STILL RUNNING
200ms   iovalkey reconnects → S2 created
        → this.stream = S2  ✅ healthy
510ms   old timer fires:
        → this.stream.destroy()
        → this.stream is S2 now
        → S2.destroy()  💥

The stale timer from S1's era executes against S2. The healthy new connection is destroyed, which triggers another reconnect, and under sustained load the client can chase its own tail — an endless reconnect loop.

Why does the old timer hit the new socket?

Because the callback reads this.stream when it fires, not when it was scheduled:

stale-reference-demo.js
let stream = "S1";

setTimeout(() => {
  console.log(stream);   // prints "S2", not "S1"
}, 500);

stream = "S2";

Same mechanics in the client: setTimeout(() => this.stream.destroy(), 500) evaluates this.stream at fire time — and by then, reconnection has repointed it to S2.

The Fix

Two parts. First, a dedicated cleanup method:

lib/Redis.ts — my fix (1/2)
private clearSocketTimeout() {
  if (this.socketTimeoutTimer !== undefined) {
    clearTimeout(this.socketTimeoutTimer);
    this.socketTimeoutTimer = undefined;
  }
}

It cancels the timer and drops the reference, so a cleared timer can never be double-handled.

Second — the key change — call it in the socket's close handler, before any reconnect logic runs:

lib/redis/event_handler.ts — my fix (2/2)
export function closeHandler(self) {
  return function () {
    self.clearSocketTimeout();

    // rest of close/reconnect logic...
  };
}

Now the lifecycle is airtight:

S1 closes → "close" event → closeHandler()
    → clearSocketTimeout()   ← old timer cancelled here
    → reconnect → S2
    → S2 can never be killed by S1's leftover timer  ✅

What the Fix Does NOT Change

The timeout still does its legitimate job. If the socket stays open but no data arrives, the timer fires and destroys the unresponsive connection — that's the intended safety mechanism:

ScenarioBehavior
Data arrives in timeTimer cleared on "data" — unchanged
Socket closes before dataTimer cleared in closeHandlerthe fix
Socket open but silent past N msTimer fires, destroys socket, reconnect — unchanged (intended)

The mental model that unlocks this bug: "close" and the timeout are two independent mechanisms. The close event says the connection ended; the timeout says I waited too long for data. The bug was that the first didn't cancel the second.

Impact

  • Eliminated a failure mode where one network blip could cascade into an endless reconnect loop — each reconnect being killed by the previous connection's stale timer.
  • Made socketTimeout safe to use in production: before the fix, enabling it exposed every reconnecting client to this race.
  • The fix is minimal and surgical — one private method, one call site in the close handler, existing timeout semantics untouched.

What I Contributed

  • Diagnosed the race between the socket lifecycle and the JavaScript timer lifecycle.
  • Implemented clearSocketTimeout() and wired it into the close handler.
  • Created regression tests covering the close-before-data path.
  • Collaborated with the maintainers until the PR was merged.

View the pull request → valkey-io/iovalkey#62 (opens in a new tab)


Related

This contribution is part of my broader Valkey ecosystem work — I also build and maintain valkey-errors, valkey-parser, and socket.io-valkey-adapter.

← Previous: Nodemailer filename escaping · Next: Tailwind CSS invalid modifiers →


Frequently Asked Questions

Why didn't the timer die when the socket died?

Because they are independent: the socket is a network connection, the timer is a JavaScript setTimeout countdown. Destroying the socket does not run clearTimeout — the program must cancel the timer explicitly, which is exactly what the fix adds to the close handler.

Why did the old timer destroy the NEW socket?

The timeout callback calls this.stream.destroy(), and this.stream is evaluated when the callback fires — not when it was scheduled. After reconnection, this.stream points to the new socket, so the stale timer destroyed the healthy replacement.

Does the fix weaken the socketTimeout safety mechanism?

No. If a socket stays open but silent past the timeout, the timer still fires and destroys it — the intended behavior. The fix only cancels the timer when the socket has already closed, where the timer no longer has a valid job.

How does this cause an endless reconnect loop?

Each reconnect created a healthy socket that the previous connection's stale timer then destroyed, which triggered another reconnect with another armed timer. Under sustained commands, the client kept killing its own recoveries.