Facebook has published code for six components of the Linux kernel, designed to solve some problems with managing large streams of data. These include the Btrfs file system, a system for measuring the state of a memory device and a CPU, as well as a mechanism for structuring workload and resource allocation.
More about tools:
- BPF is a flexible and efficient mechanism for executing code in the Linux kernel. It allows you to provide for different code different behavior of the kernel. Facebook exemplifies the ability to filter network packets, as well as running certain processes in the sandbox.
- Btrfs is a file system that works on the principle of copying when writing. It focuses on advanced feature implementation, fault tolerance, recovery, and simple administration. The system is designed to manage large storage subsystems, supports the creation of snapshots and online defragmentation.
- Netconsd is the “daemon” of the Netconsole module for sending kernel log messages. It presents them in a structured form, which makes it easier for engineers to find errors and problems.
- Cgroup2 is a mechanism for grouping and structuring workloads, as well as for controlling the amount of system resources associated with each group. In addition, it helps isolate workloads, prioritize and tune resource allocation.
- PSI (Pressure Stall Information) - a system for measuring the load on the main resources. It provides data on memory, I / O, and CPU devices. These load metrics, coupled with other tools that work in the kernel or user space, allow you to fix the lack of resources on time.
- Oomd is a user-space tool for forcing the process to terminate in the event of an “Out of memory” error. It takes into account its context and priority among other processes running in the system. Oomd also allows you to customize the automatic response of the system to situations with low memory.
All open source Facebook projects, including the Linux components presented, can be viewed on the relevant site.
In May 2018, the company published the Katran library, a software network load balancer in which the BPF mechanism described above plays a leading role.