Why build an event loop

Everyone uses event loops — Node.js, Python asyncio, Ruby async, Nginx. But few know what actually happens inside them. The best way to understand is to build one from scratch.

evix is exactly that: an event loop written in C with native Ruby bindings, built to understand every decision that libraries like libuv and libev make under the hood.

The core structure

An event loop is, at its core, a while loop that coordinates three types of work: immediate callbacks, timers, and I/O events. The central structure in evix:

struct evix_loop {
  int running;
  evix_cb_node_t *head;      // immediate callback queue
  evix_timer_t *timers;      // timer list
  evix_io_t *ios;            // I/O watcher list
  evix_backend_t *backend;   // kqueue or epoll
  void *backend_data;        // backend internal state
};

Each field is a linked list. Immediate callbacks form a FIFO queue. Timers store an expiration timestamp. I/O watchers associate a file descriptor with a callback.

The main cycle

The loop follows a precise order each iteration:

int evix_loop_run(evix_loop_t *loop) {
  loop->running = 1;
  while (loop->running && has_work(loop)) {
    // 1. Drain all immediate callbacks
    drain_callbacks(loop);

    // 2. Calculate timeout from nearest timer
    int timeout_ms = /* ... */;

    // 3. Poll the backend (kqueue/epoll) with timeout
    loop->backend->poll(loop, timeout_ms);

    // 4. Fire expired timers
    /* ... */
  }
  return 0;
}

Order matters. Immediate callbacks always execute first, before any timer — even zero-delay timers. The poll timeout is computed from the nearest timer, ensuring the loop doesn’t sleep longer than it should.

has_work checks whether any pending work exists. If there are no callbacks, no timers, and no I/O watchers, the loop exits naturally.

Timers

Timers are straightforward: they store an expiration timestamp (expire_at) and, optionally, a repeat interval (repeat_ms):

struct evix_timer {
  uint64_t expire_at;
  uint64_t repeat_ms;
  evix_callback_fn callback;
  void *data;
  struct evix_timer *next;
};

After poll returns, the loop walks the timer list, compares expire_at against the current clock (CLOCK_MONOTONIC), and fires expired ones. One-shot timers are removed and freed. Recurring timers get their expire_at recalculated:

if (timer->repeat_ms > 0) {
  timer->expire_at = current + timer->repeat_ms;
} else {
  // one-shot: remove from list and free
  *prev = timer->next;
  free(timer);
}

CLOCK_MONOTONIC is essential here. Never use gettimeofday() or CLOCK_REALTIME for timers — they’re affected by NTP adjustments and can cause timers to fire at the wrong time or never fire at all.

I/O multiplexing: kqueue and epoll

The most interesting part is how the loop waits for I/O events without blocking. Instead of doing a blocking read() on each file descriptor, we use an OS syscall that monitors multiple FDs at once.

evix abstracts this with a backend interface:

struct evix_backend {
  int  (*init)(evix_loop_t *loop);
  void (*destroy)(evix_loop_t *loop);
  int  (*poll)(evix_loop_t *loop, int timeout_ms);
  int  (*io_add)(evix_loop_t *loop, int fd, int events);
  int  (*io_del)(evix_loop_t *loop, int fd);
};

On macOS, the backend uses kqueue. On Linux, epoll. Selection is automatic at compile time:

#ifdef __linux__
#define EVIX_DEFAULT_BACKEND evix_epoll_backend
#else
#define EVIX_DEFAULT_BACKEND evix_kqueue_backend
#endif

The kqueue implementation, for example, registers read/write filters and polls with a timeout:

static int kqueue_poll(evix_loop_t *loop, int timeout_ms) {
  struct kevent events[64];
  struct timespec ts = {
    .tv_sec = timeout_ms / 1000,
    .tv_nsec = (timeout_ms % 1000) * 1000000L,
  };

  int n = kevent(state->kq, NULL, 0, events, 64, &ts);

  for (int i = 0; i < n; i++) {
    int fd = (int)events[i].ident;
    // find the matching watcher and fire its callback
  }
  return n;
}

The timeout comes from the timer calculation. If there are no timers, poll blocks indefinitely until an I/O event arrives (timeout_ms = -1). If there’s a nearby timer, poll waits at most until the timer expires. This is how event loops stay efficient: they never busy-wait, but they never miss a timer either.

One-shot vs persistent watchers

I/O watchers can be persistent (fire every time data arrives) or one-shot (fire once and get removed):

evix_io_t *io = calloc(1, sizeof(evix_io_t));
io->fd = fd;
io->events = events;
io->oneshot = (events & EVIX_IO_ONESHOT) ? 1 : 0;
io->callback = callback;

One-shot watchers are fundamental to the Fiber model. When a fiber needs to wait for data, it registers a one-shot watcher and yields. When data arrives, the watcher fires, resumes the fiber, and is automatically removed.

Ruby bindings: protecting from GC

The native C extension for Ruby must solve a subtle problem: garbage collection. When we register a Ruby block as a callback in the C event loop, we need to ensure the GC doesn’t collect the block while the event loop might still need it.

The solution is a Ruby array that holds references to all live objects:

typedef struct {
  evix_loop_t *loop;
  VALUE callbacks; // Ruby Array: prevents GC of live procs
} evix_rb_loop_t;

static void loop_mark(void *ptr) {
  evix_rb_loop_t *rb_loop = ptr;
  rb_gc_mark(rb_loop->callbacks); // mark for GC
}

Every registered callback, timer, or I/O watcher is pushed into this array. loop_mark is called by the GC during the mark phase, ensuring no live object gets collected.

Cooperative concurrency with Fibers

With the event loop and bindings in place, the Ruby layer adds cooperative concurrency using Fibers. The idea: code that looks synchronous but doesn’t block.

module Evix
  class IO
    def read(maxlen = 4096)
      wait_readable
      @ruby_io.read_nonblock(maxlen)
    end

    private

    def wait_readable
      fiber = Fiber.current
      @loop.add_io(@fd, IO_READ | IO_ONESHOT) { fiber.resume }
      Fiber.yield
    end
  end
end

When read is called, the wrapper registers a one-shot read watcher on the file descriptor and calls Fiber.yield. The fiber suspends. When data arrives, the event loop fires the callback that does fiber.resume, and execution continues exactly where it left off.

From the caller’s perspective, the code is linear:

loop.spawn do
  data = wrapped_io.read   # yields internally
  wrapped_io.write(data)   # yields if buffer full
end

But underneath, no thread is blocked. The event loop is free to process other fibers, timers, and callbacks while this fiber waits for I/O.

Signals as events

Handling signals in an event loop is tricky because signal handlers have severe restrictions — they can’t allocate memory or call most functions. The classic solution is the self-pipe trick:

def add_signal(sig, &block)
  rd, wr = ::IO.pipe
  Signal.trap(sig) { wr.write_nonblock("\x00") rescue nil }

  add_io(rd.fileno) do
    rd.read_nonblock(256) rescue nil
    block.call
  end
end

The signal handler does only one thing: writes a byte to a pipe. The event loop monitors the read end of the pipe like any other I/O watcher. When the byte arrives, the callback executes in the safe context of the loop, not inside the handler.

TCP server: one fiber per connection

With all the pieces in place, a concurrent TCP server becomes simple:

server = Evix::TCPServer.new(loop, "127.0.0.1", 3000) do |client|
  while (data = client.read)
    client.write(data)
  end
end

Internally, the server monitors the socket with a persistent read watcher. When a connection arrives, it does accept_nonblock and spawns a fiber for the connection:

def accept_connections
  loop do
    client = @server.accept_nonblock
    @loop.spawn do
      wrapped = @loop.wrap(client)
      @handler.call(wrapped)
    ensure
      wrapped&.close
    end
  rescue ::IO::WaitReadable
    break
  end
end

Each connection runs in its own fiber, with linear and readable code, but without OS threads. The event loop coordinates everything.

A subtle bug: non-blocking connect

TCPClient.connect revealed a classic non-blocking socket trap. When connect() returns EINPROGRESS, the socket eventually becomes “writable.” But writable doesn’t mean connected.

If the server isn’t running, the socket becomes writable because the connect finished — with an error. Without checking SO_ERROR, the next write() causes EPIPE (broken pipe), masking the real error (ECONNREFUSED).

The fix:

err = socket.getsockopt(Socket::SOL_SOCKET, Socket::SO_ERROR).int
raise Errno::ECONNREFUSED, "#{host}:#{port}" unless err.zero?

This is the kind of bug that only shows up when the server is unreachable — exactly the scenario you want to handle gracefully, not with a cryptic error.

Takeaways

Building an event loop from scratch forces you to understand decisions that normally stay hidden:

  • Why immediate callbacks need a separate queue from timers
  • How the poll timeout connects timers and I/O into a single mechanism
  • Why CLOCK_MONOTONIC exists and when CLOCK_REALTIME fails
  • How high-level language GC interacts with C callbacks
  • Why the self-pipe trick is the standard way to handle signals in event loops
  • Why SO_ERROR is mandatory after non-blocking connect

Each of these decisions has a concrete reason, and the only way to truly internalize them is to stumble upon them while implementing.

References