Inotify | Techlist.io

gitlabJun 29, 20262 min readCurated summary

What's new in Git 2.55.0?

Git 2.55.0 introduces improvements focused on stacked-branch workflows, large repositories, multi-remote setups, and clearer history visualization. Highlights include `git history fixup`, a built-in Linux filesystem monitor, remote-group pushing, and a configurable width limit for `git log --graph`. The release also continues Git’s Rust adoption and improves performance for partial clones. ## `git history fixup` - Adds `git history fixup <commit-id>`. - Takes staged changes and amends them directly into an existing commit. - Avoids creating a separate fixup commit and running an interactive autosquash rebase. - Automatically updates other local branches containing the amended commit, making it useful for stacked branches. ## Built-in fsmonitor support for Linux - Git’s filesystem monitor speeds up `git status` by tracking changed files instead of scanning the entire worktree. - Git 2.55 extends the built-in `core.fsmonitor=true` daemon from Windows and macOS to GNU/Linux. - Linux support uses `inotify`, avoiding the elevated privileges required by `fanotify`. - The daemon needs a watcher for every repository directory, so large repositories may require increasing `fs.inotify.max_user_watches`. ## Pushing to remote groups - Remote groups were previously supported by `git fetch` but not `git push`. - Configure a group, for example: ```bash git config set remotes.forks "origin upstream" ``` - Push to every remote in the group with: ```bash git push forks main ``` - Each remote is handled independently and follows its own `remote.<name>.push` mappings and mirror settings. ## Limiting `git log --graph` width - `git log --graph` can become difficult to read in repositories with many parallel branches. - Git 2.55 adds a way to limit the graph’s lane width, preventing the ASCII history from expanding indefinitely. - This is particularly useful for large projects such as Git itself, where the graph can become many lanes wide after only a few commits. ## Rust adoption and partial-clone performance - The release continues the gradual evolution of Rust within Git’s codebase. - `git grep` and `git cherry` receive performance improvements when operating in partial clones. Git 2.55 is especially useful for developers working in large monorepos or stacked-branch workflows. Enabling the Linux fsmonitor, using `git history fixup`, and configuring remote groups can provide immediate productivity benefits, while graph-width limits make complex histories easier to inspect.

Read original

Inotify Rust Git Linux+2

datadogNov 18, 20253 min readCurated summary

Scaling real-time file monitoring with eBPF: How we filtered billions of kernel events per minute

File integrity monitoring must provide more than proof that a file changed: security teams need to know how, why, and by whom it changed. Datadog found that filesystem scans, inotify, and auditd could not provide sufficient context, reliability, or scalability. An eBPF-based approach delivered kernel-level visibility into processes and containers, but required extensive filtering and edge processing to handle more than 10 billion events per minute. ## Why Traditional Monitoring Falls Short - Periodic scans can miss changes that are made and reverted between scans. - Scans show that a file changed, but not the process, container, or mechanism responsible. - `inotify` lacks the system-level context needed to correlate file events with processes and containers. - `auditd` offers richer information but can impose significant performance overhead and struggle under heavy load. ## eBPF for Context-Rich File Monitoring - eBPF observes file activity directly in the Linux kernel in real time. - Events can include: - The modified file - The process that triggered the change - The container in which the process ran - Additional security-relevant metadata - This context makes events more useful for investigations than simple “file changed” notifications. ## Scaling at the Agent and Backend - Datadog observed more than 10 billion file-related events per minute across its infrastructure. - Each serialized event was approximately 5 KB, making unrestricted transmission infeasible—potentially several terabytes per second. - Sending every event would also overload Agents through excessive CPU, memory, serialization, and network usage. - Agent-side rules filter events locally, discarding noise before transmission. - This reduced the stream to roughly one million events per minute while preserving detection coverage. ## Filtering Events in the Kernel - A basic architecture loads eBPF programs into the Agent, observes system activity, writes events to a ring buffer, and evaluates them in user space. - Sensitive workloads can generate up to 5,000 relevant syscalls per second. - Initial implementations risked ring-buffer backlogs and dropped events, creating security blind spots. - Datadog moved as much evaluation as possible into eBPF programs to reduce the number of events reaching user space. - The Agent could then perform a deeper second-stage evaluation before forwarding events to the backend. ## Two-Stage Evaluation: Approvers and Discarders - eBPF’s safety constraints limit computation, especially on older Linux kernels. - The system therefore separates evaluation into: - **In-kernel filtering:** Lightweight decisions that quickly approve or discard events. - **User-space evaluation:** More complex analysis using richer context, correlations, and logic unsuitable for the kernel. - This design balances kernel safety and performance with the need for detailed security detection. Datadog’s approach shows that scalable FIM requires combining eBPF’s deep visibility with aggressive filtering at the edge and in the kernel. The practical recommendation is to keep expensive analysis in user space while rejecting irrelevant events as early as possible.

Read original

Inotify Database Design Linux Ebpf+4