Skip to content

Fix FD_SETSIZE mismatch silently dropping watchers on Windows - #103

Merged
Watson1978 merged 1 commit into
socketry:mainfrom
Watson1978:fix-win32-fd-setsize
Jul 31, 2026
Merged

Fix FD_SETSIZE mismatch silently dropping watchers on Windows#103
Watson1978 merged 1 commit into
socketry:mainfrom
Watson1978:fix-win32-fd-setsize

Conversation

@Watson1978

Copy link
Copy Markdown
Collaborator

Problem

On Windows, fd_set is dimensioned by whatever FD_SETSIZE is in effect when winsock2.h is first pulled in. ev.c includes ruby.h at ev.c:41, long before it defines FD_SETSIZE at ev.c:213 — and on RubyInstaller, ruby.h reaches winsock2.h with -DFD_SETSIZE=2048 already on the compiler command line.

So the unconditional # define FD_SETSIZE 1024 could only move the bound that EV_WIN_FD_SET checks against. It could not resize the array that bound indexes. The build has been saying so for a while:

ev.c:213:10: warning: 'FD_SETSIZE' redefined
<command-line>: note: this is the location of the previous definition

EV_WIN_FD_SET does nothing once fd_count reaches FD_SETSIZE, so every watcher past the 1024th was dropped — no error, no return value, no crash — while fd_set had room for 2048.

Reproduction

A Cool.io::TCPServer echo server with 1100 concurrent connections. Before:

clients connected : 1100
server accepted   : 1100
distinct echoed   : 1023      <- plus the listener = exactly 1024
missing echoes    : 77
first missing idx : 1023

Those 77 connections are accepted and attached, but never become readable. They hang forever, and because the application never sees them, their sockets and watchers are never released.

After:

clients connected : 1100
server accepted   : 1100
distinct echoed   : 1100
missing echoes    : 0

Fix

  • ext/libev/ev.c — guard the define with #ifndef so we take whatever dimension is already in effect instead of lowering it.
  • ext/libev/ev_select.c — add a compile-time assert so the bound and the declared array can no longer disagree.
  • libev_win_select.diff — regenerated to match. It had gone stale and no longer applied to the current sources; it does again now.

Memory use is unchanged. The allocation in select_init is sizeof (fd_set), which was already 16392 bytes (2048 slots) — only the bound moved.

On memory safety

This is not a buffer overflow on this toolchain: the bound (1024) was smaller than the array (2048), so nothing was written out of bounds. The overflow would need a Ruby whose CPPFLAGS leaves FD_SETSIZE at the winsock2.h default of 64. The new assert fails the build in that case rather than letting it through.

Note that the existing assert (fd < FD_SETSIZE) in select_modify cannot catch any of this: ruby/assert.h defines NDEBUG, so every libev assert is compiled out of extensions that include ruby.h.

Verified on

ruby 3.3.12 (2026-07-16 revision 0581089df9) [x64-mingw-ucrt]
gcc 16.1.0 (MSYS2 ucrt64)
RbConfig CPPFLAGS = -DFD_SETSIZE=2048 -D_WIN32_WINNT=0x0600 ...

Measured inside the translation unit with temporary static assertions: FD_SETSIZE was 1024 against a 2048-slot array before, and 2048 against 2048 after. The new assert was also checked in the failing direction by temporarily raising ev.c's define to 4096, which fails the build as intended.

rspec: 58 examples, 1 failure (detach_race_condition_spec.rb:35), identical to the pre-change baseline on this platform. CI does not currently cover Windows (.github/workflows/test.yaml runs ubuntu and macos only), so this was verified locally.

🤖 Generated with Claude Code

On Windows, fd_set is dimensioned by whatever FD_SETSIZE is in effect when
winsock2.h is first pulled in. ev.c includes ruby.h (ev.c:41) long before it
defines FD_SETSIZE (ev.c:213), and on RubyInstaller ruby.h reaches winsock2.h
with -DFD_SETSIZE=2048 already on the command line. The unconditional define
that followed could only move the bound EV_WIN_FD_SET checks against; it could
not resize the array that bound indexes.

The result was a silent cap. EV_WIN_FD_SET does nothing once fd_count reaches
FD_SETSIZE, so watchers past the 1024th were dropped without an error, a return
value, or a crash, while fd_set had room for 2048. Guard the define with
#ifndef so we take whatever dimension is already in effect, and add a
compile-time assert in ev_select.c so the two can no longer disagree.

This is not the heap overflow it was reported as: the bound (1024) was smaller
than the array (2048), so nothing was written out of bounds here. That failure
needs a Ruby whose CPPFLAGS leaves FD_SETSIZE at the winsock2.h default of 64.
The assert fails the build in that case rather than letting it through. Note
that the existing assert (fd < FD_SETSIZE) in select_modify cannot catch any of
this: ruby/assert.h defines NDEBUG, so every libev assert is compiled out of
extensions that include ruby.h.

Verified on:
  ruby 3.3.12 (2026-07-16 revision 0581089df9) [x64-mingw-ucrt]
  RUBY_PLATFORM = x64-mingw-ucrt
  gcc 16.1.0 (MSYS2 ucrt64)
  RbConfig CPPFLAGS = -DFD_SETSIZE=2048 -D_WIN32_WINNT=0x0600 ...

Before:
  ev.c:213:10: warning: 'FD_SETSIZE' redefined
  <command-line>: note: this is the location of the previous definition
  measured inside the translation unit: FD_SETSIZE == 1024,
    sizeof(fd_set) == 16392 (2048 slots)
  1100 concurrent connections: 1023 echoed, 77 silently never watched

After:
  no redefinition warning
  measured inside the translation unit: FD_SETSIZE == 2048, 2048 slots
  1100 concurrent connections: 1100 echoed, 0 missing

The assert was also checked in the failing direction by temporarily raising
ev.c's define to 4096, which fails the build as intended.

rspec: 58 examples, 1 failure (detach_race_condition_spec.rb:35), identical to
the pre-change baseline on this platform.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Watson1978
Watson1978 merged commit c8ff7c4 into socketry:main Jul 31, 2026
17 of 19 checks passed
@Watson1978
Watson1978 deleted the fix-win32-fd-setsize branch July 31, 2026 08:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant