FSEvents, inotify, ReadDirectoryChangesW: Cross-Platform File Watching

Every file-watching daemon on a developer’s machine — gity, Watchman, Jest, Webpack, Bazel — is ultimately a thin layer over one of three OS-native APIs: FSEvents on macOS, inotify on Linux, ReadDirectoryChangesW on Windows. Each has its own quirks. This article walks through each, the problems they solve, and the corners they cut.

macOS: FSEvents

FSEvents was introduced in OS X 10.5 (2007). It’s an HFS+/APFS feature that records every file change in a kernel buffer indexed by directory ID. User-space programs subscribe to a tree (or the whole disk) and get events with millisecond latency.

Strengths:

• Coalesces events at the directory level, so a build that touches 100k files produces a small number of “this directory changed” events instead of 100k individual ones.
• Event log is persistent across reboots, so a watcher can pick up where it left off.
• Very low overhead — the journal is already being written for Time Machine.

Quirks:

• Coalescing means you can’t tell which file changed without a follow-up stat() or readdir(). Watchers compensate by maintaining their own snapshot.
• The latency parameter (latency in FSEventStreamCreate) is a hint, not a guarantee. Setting it to 0 still produces ~50ms batches in practice.
• Sandbox restrictions can prevent watching paths outside your container.

Resources:

• Apple’s File System Events Programming Guide (still mostly accurate).

Linux: inotify

inotify landed in kernel 2.6.13 (2005). It’s a per-process API that lets you register watches on individual files or directories and receive precise events: IN_CREATE, IN_MODIFY, IN_DELETE, IN_MOVED_FROM, IN_MOVED_TO, and friends.

Strengths:

• Precise — every changed file gets its own event with the exact filename.
• Low latency, typically under 1ms from filesystem change to user-space event.
• No kernel module required; works in containers.

Quirks:

• Watches do not recurse. To watch a tree, you have to walk it once and add a watch per directory. For 250,000 directories, that’s 250,000 syscalls at startup.
• The kernel’s per-user watch limit defaults to 8,192 on most distros. Users frequently bump this to 524,288 just to make modern toolchains work. Check /proc/sys/fs/inotify/max_user_watches.
• The event queue is fixed-size per inotify instance. Under heavy I/O (think: npm install in a 50k-file node_modules) it can overflow, and you lose every unread event after the overflow.
• Renames produce a pair of events (IN_MOVED_FROM and IN_MOVED_TO) that share a cookie field; watchers have to join them.

Resources:

• inotify(7) man page is excellent.

There’s also fanotify, a more powerful API (designed for AV scanners) that supports system-wide and recursive watches. It’s gaining adoption but isn’t yet the default for most tooling — partly because it requires CAP_SYS_ADMIN for the recursive flavour.

Windows: ReadDirectoryChangesW

ReadDirectoryChangesW is the Win32 API for directory change notifications. You open a directory handle with FILE_LIST_DIRECTORY access, then call ReadDirectoryChangesW with a buffer; the kernel fills your buffer with FILE_NOTIFY_INFORMATION records when something changes.

Strengths:

• Recursive by default if you pass bWatchSubtree=TRUE. One handle, one syscall, the whole tree.
• Reasonable per-file granularity (similar to inotify).
• Plays well with async I/O via overlapped completion ports.

Quirks:

• Buffer size is a parameter. Too small and you overflow on a single big change. Too large and you waste pageable memory. 64KB is a common compromise; for heavy workloads, 1MB or more.
• When the buffer overflows you get ERROR_NOTIFY_ENUM_DIR and have to recover by re-scanning the directory.
• Renames within the watched tree produce paired old-name / new-name events. Renames into or out of the watched tree look like creates or deletes.
• Performance is sensitive to NTFS USN journal state; a fragmented USN can slow notifications by 10x.

Resources:

• Microsoft’s ReadDirectoryChangesW documentation is dry but complete.

The cross-platform layer

Writing to three APIs directly is masochism. The good libraries:

• Rust’s notify crate — what gity uses. Single trait over all three platforms, plus a “poll” backend for filesystems that don’t support change notification (NFS, FUSE).
• Watchman — has its own production-grade abstraction internally, but doesn’t expose it as a library.
• chokidar (Node.js) — the de facto Node.js standard. Powers Webpack, Vite, nodemon.
• fsnotify (Go) — equivalent for Go projects.

Picking a library doesn’t free you from understanding the underlying APIs — corner cases still leak through — but it saves you from rewriting the same wrapper every time.

The overflow problem

Every one of these APIs can lose events under heavy I/O. The mitigation is the same across all three:

1. Detect the overflow.
2. Increment a generation counter.
3. Tell consumers “your last snapshot is invalid; re-scan from scratch.”

This is the safety valve that makes fsmonitor possible at all: even when the watcher gives up, the user-visible answer is still correct because Git falls back to a full walk. Without this fallback, fsmonitor would be a correctness liability rather than a performance feature.

gity uses this pattern in its fsmonitor v2 helper: an overflow on any platform causes the helper to return trivial-response: 2 (“trust nothing”) on the next call, which forces Git into a full walk just once before the cache rebuilds.

How gity uses these APIs

For each registered repo, gity:

1. Walks the tree once and computes a baseline snapshot.
2. Subscribes to the OS-native API for that tree.
3. Issues fsmonitor tokens based on the OS-specific high-water mark (inotify sequence number, FSEvents HFS event ID, Windows USN).
4. When Git asks “what changed since token T?”, computes the delta from the snapshot and returns it.
5. On overflow, invalidates the snapshot and forces Git into a fallback walk.

All of this happens in Rust, in a single ~12 MB binary, with bounded memory and no fork-exec overhead. The fsmonitor v2 protocol is the contract; the corner cases above are the implementation work.

If you’re building tooling that needs to watch files, especially across platforms, learning these APIs is unavoidable — the abstractions leak at exactly the moments you most need them not to.