0
0
Fork 0
mirror of https://github.com/netdata/netdata.git synced 2025-01-22 16:36:39 +00:00

eBPF Functions (enable/disable threads) ()

This commit is contained in:
thiagoftsm 2023-07-12 16:49:48 +00:00 committed by GitHub
parent 83272d4903
commit 929b19f485
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
69 changed files with 3736 additions and 627 deletions

1
.gitignore vendored
View file

@ -196,6 +196,7 @@ tests/acls/acl.sh
tests/urls/request.sh
tests/alarm_repetition/alarm.sh
tests/template_dimension/template_dim.sh
tests/ebpf/ebpf_thread_function.sh
aclk/legacy/tests/install-fake-charts.d.sh
# tests and temp files

View file

@ -624,6 +624,8 @@ set(EBPF_PROCESS_PLUGIN_FILES
collectors/ebpf.plugin/ebpf_cgroup.h
collectors/ebpf.plugin/ebpf_unittest.c
collectors/ebpf.plugin/ebpf_unittest.h
collectors/ebpf.plugin/ebpf_functions.c
collectors/ebpf.plugin/ebpf_functions.h
)
set(PROC_PLUGIN_FILES

View file

@ -365,6 +365,8 @@ EBPF_PLUGIN_FILES = \
collectors/ebpf.plugin/ebpf_cgroup.h \
collectors/ebpf.plugin/ebpf_unittest.c \
collectors/ebpf.plugin/ebpf_unittest.h \
collectors/ebpf.plugin/ebpf_functions.c \
collectors/ebpf.plugin/ebpf_functions.h \
$(LIBNETDATA_FILES) \
$(NULL)

View file

@ -235,13 +235,12 @@ Linux metrics:
The eBPF collector enables and runs the following eBPF programs by default:
- `cachestat`: Netdata's eBPF data collector creates charts about the memory page cache. When the integration with
[`apps.plugin`](https://github.com/netdata/netdata/blob/master/collectors/apps.plugin/README.md) is enabled, this collector creates charts for the whole host _and_
for each application.
- `fd` : This eBPF program creates charts that show information about calls to open files.
- `mount`: This eBPF program creates charts that show calls to syscalls mount(2) and umount(2).
- `shm`: This eBPF program creates charts that show calls to syscalls shmget(2), shmat(2), shmdt(2) and shmctl(2).
- `sync`: Monitor calls to syscalls sync(2), fsync(2), fdatasync(2), syncfs(2), msync(2), and sync_file_range(2).
- `network viewer`: This eBPF program creates charts with information about `TCP` and `UDP` functions, including the
bandwidth consumed by each.
- `vfs`: This eBPF program creates charts that show information about VFS (Virtual File System) functions.
- `process`: This eBPF program creates charts that show information about process life. When in `return` mode, it also
creates charts showing errors when these operations are executed.
- `hardirq`: This eBPF program creates charts that show information about time spent servicing individual hardware
@ -254,9 +253,6 @@ The eBPF collector enables and runs the following eBPF programs by default:
You can also enable the following eBPF programs:
- `cachestat`: Netdata's eBPF data collector creates charts about the memory page cache. When the integration with
[`apps.plugin`](https://github.com/netdata/netdata/blob/master/collectors/apps.plugin/README.md) is enabled, this collector creates charts for the whole host _and_
for each application.
- `dcstat` : This eBPF program creates charts that show information about file access using directory cache. It appends
`kprobes` for `lookup_fast()` and `d_lookup()` to identify if files are inside directory cache, outside and files are
not found.
@ -264,7 +260,11 @@ You can also enable the following eBPF programs:
- `filesystem` : This eBPF program creates charts that show information about some filesystem latency.
- `swap` : This eBPF program creates charts that show information about swap access.
- `mdflush`: This eBPF program creates charts that show information about
- `sync`: Monitor calls to syscalls sync(2), fsync(2), fdatasync(2), syncfs(2), msync(2), and sync_file_range(2).
- `network viewer`: This eBPF program creates charts with information about `TCP` and `UDP` functions, including the
bandwidth consumed by each.
multi-device software flushes.
- `vfs`: This eBPF program creates charts that show information about VFS (Virtual File System) functions.
### Configuring eBPF threads
@ -989,3 +989,50 @@ shows how the lockdown module impacts `ebpf.plugin` based on the selected option
If you or your distribution compiled the kernel with the last combination, your system cannot load shared libraries
required to run `ebpf.plugin`.
## Function
The eBPF plugin has a [function](https://github.com/netdata/netdata/blob/master/docs/cloud/netdata-functions.md) named
`ebpf_thread` that controls its internal threads and helps to reduce the overhead on host. Using the function you
can run the plugin with all threads disabled and enable them only when you want to take a look in specific areas.
### List threads
To list all threads status you can query directly the endpoint function:
`http://localhost:19999/api/v1/function?function=ebpf_thread`
It is also possible to query a specific thread adding keyword `thread` and thread name:
`http://localhost:19999/api/v1/function?function=ebpf_thread%20thread:mount`
### Enable thread
It is possible to enable a specific thread using the keyword `enable`:
`http://localhost:19999/api/v1/function?function=ebpf_thread%20enable:mount`
this will run thread `mount` during 300 seconds (5 minutes). You can specify a specific period by appending the period
after the thread name:
`http://localhost:19999/api/v1/function?function=ebpf_thread%20enable:mount:600`
in this example thread `mount` will run during 600 seconds (10 minutes).
### Disable thread
It is also possible to stop any thread running using the keyword `disable`. For example, to disable `cachestat` you can
request:
`http://localhost:19999/api/v1/function?function=ebpf_thread%20disable:cachestat`
### Debugging threads
You can verify the impact of threads on the host by running the
[ebpf_thread_function.sh](https://github.com/netdata/netdata/blob/master/tests/ebpf/ebpf_thread_function.sh)
script on your environment.
You can check the results of having threads running on your environment in the Netdata monitoring section on your
dashboard
<img src="https://github.com/netdata/netdata/assets/49162938/91823573-114c-4c16-b634-cc46f7bb1bcf" alt="Threads running." />

File diff suppressed because it is too large Load diff

View file

@ -19,6 +19,8 @@
#
# The `maps per core` defines if hash tables will be per core or not. This option is ignored on kernels older than 4.15.
#
# The `lifetime` defines the time length a thread will run when it is enabled by a function.
#
[global]
ebpf load mode = entry
apps = no
@ -27,6 +29,7 @@
pid table size = 32768
btf path = /sys/kernel/btf/
maps per core = yes
lifetime = 300
#
# eBPF Programs

View file

@ -26,6 +26,8 @@
#
# The `maps per core` defines if hash tables will be per core or not. This option is ignored on kernels older than 4.6.
#
# The `lifetime` defines the time length a thread will run when it is enabled by a function.
#
# Uncomment lines to define specific options for thread.
[global]
# ebpf load mode = entry
@ -37,3 +39,4 @@
ebpf co-re tracing = trampoline
collect pid = real parent
# maps per core = yes
lifetime = 300

View file

@ -24,6 +24,8 @@
#
# The `maps per core` defines if hash tables will be per core or not. This option is ignored on kernels older than 4.6.
#
# The `lifetime` defines the time length a thread will run when it is enabled by a function.
#
# Uncomment lines to define specific options for thread.
[global]
# ebpf load mode = entry
@ -35,3 +37,4 @@
ebpf co-re tracing = trampoline
collect pid = real parent
# maps per core = yes
lifetime = 300

View file

@ -3,7 +3,10 @@
# `return : In the `return` mode, the eBPF collector monitors the same kernel functions as `entry`, but also creates
# new charts for the return of these functions, such as errors.
#
#[global]
# The `lifetime` defines the time length a thread will run when it is enabled by a function.
#
[global]
# ebpf load mode = entry
# update every = 10
lifetime = 300

View file

@ -12,6 +12,8 @@
#
# The `maps per core` defines if hash tables will be per core or not. This option is ignored on kernels older than 4.6.
#
# The `lifetime` defines the time length a thread will run when it is enabled by a function.
#
# Uncomment lines to define specific options for thread.
[global]
# ebpf load mode = entry
@ -22,3 +24,4 @@
ebpf type format = auto
ebpf co-re tracing = trampoline
# maps per core = yes
lifetime = 300

View file

@ -3,13 +3,16 @@
# `return : In the `return` mode, the eBPF collector monitors the same kernel functions as `entry`, but also creates
# new charts for the return of these functions, such as errors.
#
# The `lifetime` defines the time length a thread will run when it is enabled by a function.
#
# The eBPF collector also creates charts for each running application through an integration with the `apps plugin`.
# If you want to disable the integration with `apps.plugin` along with the above charts, change the setting `apps` to
# 'no'.
#
#[global]
[global]
# ebpf load mode = entry
# update every = 10
lifetime = 300
# All filesystems are named as 'NAMEdist' where NAME is the filesystem name while 'dist' is a reference for distribution.
[filesystem]

View file

@ -0,0 +1,3 @@
#[global]
# update every = 5

View file

@ -3,6 +3,9 @@
# `return : In the `return` mode, the eBPF collector monitors the same kernel functions as `entry`, but also creates
# new charts for the return of these functions, such as errors.
#
#[global]
# The `lifetime` defines the time length a thread will run when it is enabled by a function.
#
[global]
# ebpf load mode = entry
# update every = 10
lifetime = 300

View file

@ -2,6 +2,10 @@
# `entry` : The eBPF collector only monitors calls for the functions, and does not show charts related to errors.
# `return : In the `return` mode, the eBPF collector monitors the same kernel functions as `entry`, but also creates
# new charts for the return of these functions, such as errors.
#[global]
#
# The `lifetime` defines the time length a thread will run when it is enabled by a function.
#
[global]
# ebpf load mode = entry
# update every = 1
lifetime = 300

View file

@ -12,8 +12,12 @@
# `trampoline`: This is the default mode used by the eBPF collector, due the small overhead added to host.
# `tracepoint`: When available, the eBPF collector will use kernel tracepoint to monitor syscall.
# `probe` : This is the same as legacy code.
#
# The `lifetime` defines the time length a thread will run when it is enabled by a function.
#
[global]
# ebpf load mode = entry
# update every = 1
ebpf type format = auto
ebpf co-re tracing = trampoline
lifetime = 300

View file

@ -26,6 +26,8 @@
#
# The `maps per core` defines if hash tables will be per core or not. This option is ignored on kernels older than 4.6.
#
# The `lifetime` defines the time length a thread will run when it is enabled by a function.
#
# Uncomment lines to define specific options for thread.
[global]
# ebpf load mode = entry
@ -39,6 +41,7 @@
ebpf type format = auto
ebpf co-re tracing = trampoline
maps per core = no
lifetime = 300
#
# Network Connection

View file

@ -2,6 +2,10 @@
# `entry` : The eBPF collector only monitors calls for the functions, and does not show charts related to errors.
# `return : In the `return` mode, the eBPF collector monitors the same kernel functions as `entry`, but also creates
# new charts for the return of these functions, such as errors.
#[global]
#
# The `lifetime` defines the time length a thread will run when it is enabled by a function.
#
[global]
# ebpf load mode = entry
# update every = 1
lifetime = 300

View file

@ -17,6 +17,8 @@
#
# The `maps per core` defines if hash tables will be per core or not. This option is ignored on kernels older than 4.6.
#
# The `lifetime` defines the time length a thread will run when it is enabled by a function.
#
# Uncomment lines to define specific options for thread.
[global]
# ebpf load mode = entry
@ -26,3 +28,4 @@
# pid table size = 32768
collect pid = real parent
# maps per core = yes
lifetime = 300

View file

@ -20,6 +20,8 @@
#
# The `maps per core` defines if hash tables will be per core or not. This option is ignored on kernels older than 4.6.
#
# The `lifetime` defines the time length a thread will run when it is enabled by a function.
#
# Uncomment lines to define specific options for thread.
[global]
# ebpf load mode = entry
@ -30,6 +32,7 @@
ebpf type format = auto
ebpf co-re tracing = trampoline
# maps per core = yes
lifetime = 300
# List of monitored syscalls
[syscalls]

View file

@ -3,6 +3,9 @@
# `return : In the `return` mode, the eBPF collector monitors the same kernel functions as `entry`, but also creates
# new charts for the return of these functions, such as errors.
#
#[global]
# The `lifetime` defines the time length a thread will run when it is enabled by a function.
#
[global]
# ebpf load mode = entry
# update every = 10
lifetime = 300

View file

@ -19,6 +19,8 @@
#
# The `maps per core` defines if hash tables will be per core or not. This option is ignored on kernels older than 4.6.
#
# The `lifetime` defines the time length a thread will run when it is enabled by a function.
#
# Uncomment lines to define specific options for thread.
[global]
# ebpf load mode = entry
@ -29,3 +31,4 @@
ebpf type format = auto
ebpf co-re tracing = trampoline
# maps per core = yes
lifetime = 300

View file

@ -19,6 +19,8 @@
# `probe` : This is the same as legacy code.
#
# The `maps per core` defines if hash tables will be per core or not. This option is ignored on kernels older than 4.6.
#
# The `lifetime` defines the time length a thread will run when it is enabled by a function.
#
# Uncomment lines to define specific options for thread.
[global]
@ -29,6 +31,7 @@
ebpf type format = auto
ebpf co-re tracing = trampoline
# maps per core = yes
lifetime = 300
# List of monitored syscalls
[syscalls]

View file

@ -20,6 +20,8 @@
#
# The `maps per core` defines if hash tables will be per core or not. This option is ignored on kernels older than 4.6.
#
# The `lifetime` defines the time length a thread will run when it is enabled by a function.
#
# Uncomment lines to define specific options for thread.
[global]
# ebpf load mode = entry
@ -30,3 +32,4 @@
ebpf type format = auto
ebpf co-re tracing = trampoline
# maps per core = yes
lifetime = 300

View file

@ -118,6 +118,7 @@ enum ebpf_main_index {
EBPF_MODULE_OOMKILL_IDX,
EBPF_MODULE_SHM_IDX,
EBPF_MODULE_MDFLUSH_IDX,
EBPF_MODULE_FUNCTION_IDX,
/* THREADS MUST BE INCLUDED BEFORE THIS COMMENT */
EBPF_OPTION_ALL_CHARTS,
EBPF_OPTION_VERSION,
@ -163,6 +164,7 @@ typedef struct ebpf_tracepoint {
// Statistics charts
#define NETDATA_EBPF_THREADS "ebpf_threads"
#define NETDATA_EBPF_LIFE_TIME "ebpf_life_time"
#define NETDATA_EBPF_LOAD_METHOD "ebpf_load_methods"
#define NETDATA_EBPF_KERNEL_MEMORY "ebpf_kernel_memory"
#define NETDATA_EBPF_HASH_TABLES_LOADED "ebpf_hash_tables_count"

View file

@ -1338,8 +1338,10 @@ void cleanup_exited_pids()
p = p->next;
// Clean process structure
ebpf_process_stat_release(global_process_stats[r]);
global_process_stats[r] = NULL;
if (global_process_stats) {
ebpf_process_stat_release(global_process_stats[r]);
global_process_stats[r] = NULL;
}
cleanup_variables_from_other_threads(r);
@ -1471,36 +1473,40 @@ void collect_data_for_all_processes(int tbl_pid_stats_fd, int maps_per_core)
uint32_t key;
pids = ebpf_root_of_pids; // global list of all processes running
// while (bpf_map_get_next_key(tbl_pid_stats_fd, &key, &next_key) == 0) {
size_t length = sizeof(ebpf_process_stat_t);
if (maps_per_core)
length *= ebpf_nprocs;
while (pids) {
key = pids->pid;
ebpf_process_stat_t *w = global_process_stats[key];
if (!w) {
w = ebpf_process_stat_get();
global_process_stats[key] = w;
}
if (tbl_pid_stats_fd != -1) {
size_t length = sizeof(ebpf_process_stat_t);
if (maps_per_core)
length *= ebpf_nprocs;
if (bpf_map_lookup_elem(tbl_pid_stats_fd, &key, process_stat_vector)) {
// Clean Process structures
ebpf_process_stat_release(w);
global_process_stats[key] = NULL;
while (pids) {
key = pids->pid;
cleanup_variables_from_other_threads(key);
ebpf_process_stat_t *w = global_process_stats[key];
if (!w) {
w = ebpf_process_stat_get();
global_process_stats[key] = w;
}
if (bpf_map_lookup_elem(tbl_pid_stats_fd, &key, process_stat_vector)) {
// Clean Process structures
ebpf_process_stat_release(w);
global_process_stats[key] = NULL;
cleanup_variables_from_other_threads(key);
pids = pids->next;
continue;
}
ebpf_process_apps_accumulator(process_stat_vector, maps_per_core);
memcpy(w, process_stat_vector, sizeof(ebpf_process_stat_t));
memset(process_stat_vector, 0, length);
pids = pids->next;
continue;
}
ebpf_process_apps_accumulator(process_stat_vector, maps_per_core);
memcpy(w, process_stat_vector, sizeof(ebpf_process_stat_t));
memset(process_stat_vector, 0, length);
pids = pids->next;
}
link_all_processes_to_their_parents();

View file

@ -21,6 +21,7 @@
#include "ebpf_disk.h"
#include "ebpf_fd.h"
#include "ebpf_filesystem.h"
#include "ebpf_functions.h"
#include "ebpf_hardirq.h"
#include "ebpf_cachestat.h"
#include "ebpf_mdflush.h"

View file

@ -58,6 +58,10 @@ netdata_ebpf_targets_t cachestat_targets[] = { {.name = "add_to_page_cache_lru",
static char *account_page[NETDATA_CACHESTAT_ACCOUNT_DIRTY_END] ={ "account_page_dirtied",
"__set_page_dirty", "__folio_mark_dirty" };
#ifdef NETDATA_DEV_MODE
int cachestat_disable_priority;
#endif
#ifdef LIBBPF_MAJOR_VERSION
/**
* Disable probe
@ -336,6 +340,179 @@ static inline int ebpf_cachestat_load_and_attach(struct cachestat_bpf *obj, ebpf
*
*****************************************************************/
static void ebpf_obsolete_specific_cachestat_charts(char *type, int update_every);
/**
* Obsolete services
*
* Obsolete all service charts created
*
* @param em a pointer to `struct ebpf_module`
*/
static void ebpf_obsolete_services(ebpf_module_t *em)
{
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_CACHESTAT_HIT_RATIO_CHART,
"Hit ratio",
EBPF_COMMON_DIMENSION_PERCENTAGE,
NETDATA_CACHESTAT_SUBMENU,
NETDATA_EBPF_CHART_TYPE_LINE,
NETDATA_SYSTEMD_CACHESTAT_HIT_RATIO_CONTEXT,
21100,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_CACHESTAT_DIRTY_CHART,
"Number of dirty pages",
EBPF_CACHESTAT_DIMENSION_PAGE,
NETDATA_CACHESTAT_SUBMENU,
NETDATA_EBPF_CHART_TYPE_LINE,
NETDATA_SYSTEMD_CACHESTAT_MODIFIED_CACHE_CONTEXT,
21101,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_CACHESTAT_HIT_CHART,
"Number of accessed files",
EBPF_CACHESTAT_DIMENSION_HITS,
NETDATA_CACHESTAT_SUBMENU,
NETDATA_EBPF_CHART_TYPE_LINE,
NETDATA_SYSTEMD_CACHESTAT_HIT_FILE_CONTEXT,
21102,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_CACHESTAT_MISSES_CHART,
"Files out of page cache",
EBPF_CACHESTAT_DIMENSION_MISSES,
NETDATA_CACHESTAT_SUBMENU,
NETDATA_EBPF_CHART_TYPE_LINE,
NETDATA_SYSTEMD_CACHESTAT_MISS_FILES_CONTEXT,
21103,
em->update_every);
}
/**
* Obsolete cgroup chart
*
* Send obsolete for all charts created before to close.
*
* @param em a pointer to `struct ebpf_module`
*/
static inline void ebpf_obsolete_cachestat_cgroup_charts(ebpf_module_t *em) {
pthread_mutex_lock(&mutex_cgroup_shm);
ebpf_obsolete_services(em);
ebpf_cgroup_target_t *ect;
for (ect = ebpf_cgroup_pids; ect ; ect = ect->next) {
if (ect->systemd)
continue;
ebpf_obsolete_specific_cachestat_charts(ect->name, em->update_every);
}
pthread_mutex_unlock(&mutex_cgroup_shm);
}
/**
* Obsolete global
*
* Obsolete global charts created by thread.
*
* @param em a pointer to `struct ebpf_module`
*/
static void ebpf_obsolete_cachestat_global(ebpf_module_t *em)
{
ebpf_write_chart_obsolete(NETDATA_EBPF_MEMORY_GROUP,
NETDATA_CACHESTAT_HIT_RATIO_CHART,
"Hit ratio",
EBPF_COMMON_DIMENSION_PERCENTAGE,
NETDATA_CACHESTAT_SUBMENU,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
21100,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_EBPF_MEMORY_GROUP,
NETDATA_CACHESTAT_DIRTY_CHART,
"Number of dirty pages",
EBPF_CACHESTAT_DIMENSION_PAGE,
NETDATA_CACHESTAT_SUBMENU,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
21101,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_EBPF_MEMORY_GROUP,
NETDATA_CACHESTAT_HIT_CHART,
"Number of accessed files",
EBPF_CACHESTAT_DIMENSION_HITS,
NETDATA_CACHESTAT_SUBMENU,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
21102,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_EBPF_MEMORY_GROUP,
NETDATA_CACHESTAT_MISSES_CHART,
"Files out of page cache",
EBPF_CACHESTAT_DIMENSION_MISSES,
NETDATA_CACHESTAT_SUBMENU,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
21103,
em->update_every);
}
/**
* Obsolette apps charts
*
* Obsolete apps charts.
*
* @param em a pointer to the structure with the default values.
*/
void ebpf_obsolete_cachestat_apps_charts(struct ebpf_module *em)
{
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_CACHESTAT_HIT_RATIO_CHART,
"Hit ratio",
EBPF_COMMON_DIMENSION_PERCENTAGE,
NETDATA_CACHESTAT_SUBMENU,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
20090,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_CACHESTAT_DIRTY_CHART,
"Number of dirty pages",
EBPF_CACHESTAT_DIMENSION_PAGE,
NETDATA_CACHESTAT_SUBMENU,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20091,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY, NETDATA_CACHESTAT_HIT_CHART,
"Number of accessed files",
EBPF_CACHESTAT_DIMENSION_HITS,
NETDATA_CACHESTAT_SUBMENU,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20092,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_CACHESTAT_MISSES_CHART,
"Files out of page cache",
EBPF_CACHESTAT_DIMENSION_MISSES,
NETDATA_CACHESTAT_SUBMENU,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20093,
em->update_every);
}
/**
* Cachestat exit.
*
@ -347,17 +524,47 @@ static void ebpf_cachestat_exit(void *ptr)
{
ebpf_module_t *em = (ebpf_module_t *)ptr;
if (em->enabled == NETDATA_THREAD_EBPF_FUNCTION_RUNNING) {
pthread_mutex_lock(&lock);
if (em->cgroup_charts) {
ebpf_obsolete_cachestat_cgroup_charts(em);
fflush(stdout);
}
if (em->apps_charts & NETDATA_EBPF_APPS_FLAG_CHART_CREATED) {
ebpf_obsolete_cachestat_apps_charts(em);
}
ebpf_obsolete_cachestat_global(em);
#ifdef NETDATA_DEV_MODE
if (ebpf_aral_cachestat_pid)
ebpf_statistic_obsolete_aral_chart(em, cachestat_disable_priority);
#endif
fflush(stdout);
pthread_mutex_unlock(&lock);
}
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_REMOVE);
#ifdef LIBBPF_MAJOR_VERSION
if (cachestat_bpf_obj)
if (cachestat_bpf_obj) {
cachestat_bpf__destroy(cachestat_bpf_obj);
cachestat_bpf_obj = NULL;
}
#endif
if (em->objects) {
ebpf_unload_legacy_code(em->objects, em->probe_links);
em->objects = NULL;
em->probe_links = NULL;
}
pthread_mutex_lock(&ebpf_exit_cleanup);
em->enabled = NETDATA_THREAD_EBPF_STOPPED;
ebpf_update_stats(&plugin_statistics, em);
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
@ -1079,7 +1286,9 @@ static void cachestat_collector(ebpf_module_t *em)
heartbeat_init(&hb);
int counter = update_every - 1;
//This will be cancelled by its parent
while (!ebpf_exit_plugin) {
uint32_t running_time = 0;
uint32_t lifetime = em->lifetime;
while (!ebpf_exit_plugin && running_time < lifetime) {
(void)heartbeat_next(&hb, USEC_PER_SEC);
if (ebpf_exit_plugin || ++counter != update_every)
@ -1112,6 +1321,15 @@ static void cachestat_collector(ebpf_module_t *em)
pthread_mutex_unlock(&lock);
pthread_mutex_unlock(&collect_data_mutex);
pthread_mutex_lock(&ebpf_exit_cleanup);
if (running_time && !em->running_time)
running_time = update_every;
else
running_time += update_every;
em->running_time = running_time;
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
}
@ -1307,11 +1525,11 @@ void *ebpf_cachestat_thread(void *ptr)
pthread_mutex_lock(&lock);
ebpf_update_stats(&plugin_statistics, em);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_ADD);
ebpf_create_memory_charts(em);
#ifdef NETDATA_DEV_MODE
if (ebpf_aral_cachestat_pid)
ebpf_statistic_create_aral_chart(NETDATA_EBPF_CACHESTAT_ARAL_NAME, em);
cachestat_disable_priority = ebpf_statistic_create_aral_chart(NETDATA_EBPF_CACHESTAT_ARAL_NAME, em);
#endif
pthread_mutex_unlock(&lock);

View file

@ -3,8 +3,9 @@
#ifndef NETDATA_EBPF_CACHESTAT_H
#define NETDATA_EBPF_CACHESTAT_H 1
// Module name
// Module name & description
#define NETDATA_EBPF_MODULE_NAME_CACHESTAT "cachestat"
#define NETDATA_EBPF_CACHESTAT_MODULE_DESC "Monitor Linux page cache internal functions. This thread is integrated with apps and cgroup."
// charts
#define NETDATA_CACHESTAT_HIT_RATIO_CHART "cachestat_ratio"

View file

@ -59,6 +59,10 @@ netdata_ebpf_targets_t dc_targets[] = { {.name = "lookup_fast", .mode = EBPF_LOA
{.name = "d_lookup", .mode = EBPF_LOAD_TRAMPOLINE},
{.name = NULL, .mode = EBPF_LOAD_TRAMPOLINE}};
#ifdef NETDATA_DEV_MODE
int dcstat_disable_priority;
#endif
#ifdef LIBBPF_MAJOR_VERSION
/**
* Disable probe
@ -285,6 +289,160 @@ void dcstat_update_publish(netdata_publish_dcstat_t *out, uint64_t cache_access,
*
*****************************************************************/
static void ebpf_obsolete_specific_dc_charts(char *type, int update_every);
/**
* Obsolete services
*
* Obsolete all service charts created
*
* @param em a pointer to `struct ebpf_module`
*/
static void ebpf_obsolete_dc_services(ebpf_module_t *em)
{
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_DC_HIT_CHART,
"Percentage of files inside directory cache",
EBPF_COMMON_DIMENSION_PERCENTAGE,
NETDATA_DIRECTORY_CACHE_SUBMENU,
NETDATA_EBPF_CHART_TYPE_LINE,
NETDATA_SYSTEMD_DC_HIT_RATIO_CONTEXT,
21200,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_DC_REFERENCE_CHART,
"Count file access",
EBPF_COMMON_DIMENSION_FILES,
NETDATA_DIRECTORY_CACHE_SUBMENU,
NETDATA_EBPF_CHART_TYPE_LINE,
NETDATA_SYSTEMD_DC_REFERENCE_CONTEXT,
21201,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_DC_REQUEST_NOT_CACHE_CHART,
"Files not present inside directory cache",
EBPF_COMMON_DIMENSION_FILES,
NETDATA_DIRECTORY_CACHE_SUBMENU,
NETDATA_EBPF_CHART_TYPE_LINE,
NETDATA_SYSTEMD_DC_NOT_CACHE_CONTEXT,
21202,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_DC_REQUEST_NOT_FOUND_CHART,
"Files not found",
EBPF_COMMON_DIMENSION_FILES,
NETDATA_DIRECTORY_CACHE_SUBMENU,
NETDATA_EBPF_CHART_TYPE_LINE,
NETDATA_SYSTEMD_DC_NOT_FOUND_CONTEXT,
21202,
em->update_every);
}
/**
* Obsolete cgroup chart
*
* Send obsolete for all charts created before to close.
*
* @param em a pointer to `struct ebpf_module`
*/
static inline void ebpf_obsolete_dc_cgroup_charts(ebpf_module_t *em) {
pthread_mutex_lock(&mutex_cgroup_shm);
ebpf_obsolete_dc_services(em);
ebpf_cgroup_target_t *ect;
for (ect = ebpf_cgroup_pids; ect ; ect = ect->next) {
if (ect->systemd)
continue;
ebpf_obsolete_specific_dc_charts(ect->name, em->update_every);
}
pthread_mutex_unlock(&mutex_cgroup_shm);
}
/**
* Obsolette apps charts
*
* Obsolete apps charts.
*
* @param em a pointer to the structure with the default values.
*/
void ebpf_obsolete_dc_apps_charts(struct ebpf_module *em)
{
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_DC_HIT_CHART,
"Percentage of files inside directory cache",
EBPF_COMMON_DIMENSION_PERCENTAGE,
NETDATA_DIRECTORY_CACHE_SUBMENU,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
20100,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_DC_REFERENCE_CHART,
"Count file access",
EBPF_COMMON_DIMENSION_FILES,
NETDATA_DIRECTORY_CACHE_SUBMENU,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20101,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_DC_REQUEST_NOT_CACHE_CHART,
"Files not present inside directory cache",
EBPF_COMMON_DIMENSION_FILES,
NETDATA_DIRECTORY_CACHE_SUBMENU,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20102,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_DC_REQUEST_NOT_FOUND_CHART,
"Files not found",
EBPF_COMMON_DIMENSION_FILES,
NETDATA_DIRECTORY_CACHE_SUBMENU,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20103,
em->update_every);
}
/**
* Obsolete global
*
* Obsolete global charts created by thread.
*
* @param em a pointer to `struct ebpf_module`
*/
static void ebpf_obsolete_dc_global(ebpf_module_t *em)
{
ebpf_write_chart_obsolete(NETDATA_FILESYSTEM_FAMILY,
NETDATA_DC_HIT_CHART,
"Percentage of files inside directory cache",
EBPF_COMMON_DIMENSION_PERCENTAGE,
NETDATA_DIRECTORY_CACHE_SUBMENU,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
21200,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_FILESYSTEM_FAMILY,
NETDATA_DC_REFERENCE_CHART,
"Variables used to calculate hit ratio.",
EBPF_COMMON_DIMENSION_FILES,
NETDATA_DIRECTORY_CACHE_SUBMENU,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
21201,
em->update_every);
}
/**
* DCstat exit
*
@ -296,16 +454,46 @@ static void ebpf_dcstat_exit(void *ptr)
{
ebpf_module_t *em = (ebpf_module_t *)ptr;
#ifdef LIBBPF_MAJOR_VERSION
if (dc_bpf_obj)
dc_bpf__destroy(dc_bpf_obj);
if (em->enabled == NETDATA_THREAD_EBPF_FUNCTION_RUNNING) {
pthread_mutex_lock(&lock);
if (em->cgroup_charts) {
ebpf_obsolete_dc_cgroup_charts(em);
fflush(stdout);
}
if (em->apps_charts & NETDATA_EBPF_APPS_FLAG_CHART_CREATED) {
ebpf_obsolete_dc_apps_charts(em);
}
ebpf_obsolete_dc_global(em);
#ifdef NETDATA_DEV_MODE
if (ebpf_aral_dcstat_pid)
ebpf_statistic_obsolete_aral_chart(em, dcstat_disable_priority);
#endif
if (em->objects)
fflush(stdout);
pthread_mutex_unlock(&lock);
}
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_REMOVE);
#ifdef LIBBPF_MAJOR_VERSION
if (dc_bpf_obj) {
dc_bpf__destroy(dc_bpf_obj);
dc_bpf_obj = NULL;
}
#endif
if (em->objects){
ebpf_unload_legacy_code(em->objects, em->probe_links);
em->objects = NULL;
em->probe_links = NULL;
}
pthread_mutex_lock(&ebpf_exit_cleanup);
em->enabled = NETDATA_THREAD_EBPF_STOPPED;
ebpf_update_stats(&plugin_statistics, em);
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
@ -979,7 +1167,9 @@ static void dcstat_collector(ebpf_module_t *em)
heartbeat_init(&hb);
int counter = update_every - 1;
int maps_per_core = em->maps_per_core;
while (!ebpf_exit_plugin) {
uint32_t running_time = 0;
uint32_t lifetime = em->lifetime;
while (!ebpf_exit_plugin && running_time < lifetime) {
(void)heartbeat_next(&hb, USEC_PER_SEC);
if (ebpf_exit_plugin || ++counter != update_every)
@ -1012,6 +1202,15 @@ static void dcstat_collector(ebpf_module_t *em)
pthread_mutex_unlock(&lock);
pthread_mutex_unlock(&collect_data_mutex);
pthread_mutex_lock(&ebpf_exit_cleanup);
if (running_time && !em->running_time)
running_time = update_every;
else
running_time += update_every;
em->running_time = running_time;
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
}
@ -1028,7 +1227,7 @@ static void dcstat_collector(ebpf_module_t *em)
*
* @param update_every value to overwrite the update frequency set by the server.
*/
static void ebpf_create_filesystem_charts(int update_every)
static void ebpf_create_dc_global_charts(int update_every)
{
ebpf_create_chart(NETDATA_FILESYSTEM_FAMILY, NETDATA_DC_HIT_CHART,
"Percentage of files inside directory cache",
@ -1156,12 +1355,12 @@ void *ebpf_dcstat_thread(void *ptr)
algorithms, NETDATA_DCSTAT_IDX_END);
pthread_mutex_lock(&lock);
ebpf_create_filesystem_charts(em->update_every);
ebpf_create_dc_global_charts(em->update_every);
ebpf_update_stats(&plugin_statistics, em);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_ADD);
#ifdef NETDATA_DEV_MODE
if (ebpf_aral_dcstat_pid)
ebpf_statistic_create_aral_chart(NETDATA_EBPF_DCSTAT_ARAL_NAME, em);
dcstat_disable_priority = ebpf_statistic_create_aral_chart(NETDATA_EBPF_DCSTAT_ARAL_NAME, em);
#endif
pthread_mutex_unlock(&lock);

View file

@ -3,8 +3,9 @@
#ifndef NETDATA_EBPF_DCSTAT_H
#define NETDATA_EBPF_DCSTAT_H 1
// Module name
// Module name & description
#define NETDATA_EBPF_MODULE_NAME_DCSTAT "dcstat"
#define NETDATA_EBPF_DC_MODULE_DESC "Monitor file access using directory cache. This thread is integrated with apps and cgroup."
// charts
#define NETDATA_DC_HIT_CHART "dc_hit_ratio"

View file

@ -448,6 +448,7 @@ static void ebpf_cleanup_plot_disks()
move = next;
}
plot_disks = NULL;
}
/**
@ -465,6 +466,36 @@ static void ebpf_cleanup_disk_list()
move = next;
}
disk_list = NULL;
}
/**
* Obsolete global
*
* Obsolete global charts created by thread.
*
* @param em a pointer to `struct ebpf_module`
*/
static void ebpf_obsolete_disk_global(ebpf_module_t *em)
{
ebpf_publish_disk_t *move = plot_disks;
while (move) {
netdata_ebpf_disks_t *ned = move->plot;
uint32_t flags = ned->flags;
if (flags & NETDATA_DISK_CHART_CREATED) {
ebpf_write_chart_obsolete(ned->histogram.name,
ned->family,
"Disk latency",
EBPF_COMMON_DIMENSION_CALL,
ned->family,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
ned->histogram.order,
em->update_every);
}
move = move->next;
}
}
/**
@ -478,15 +509,29 @@ static void ebpf_disk_exit(void *ptr)
{
ebpf_module_t *em = (ebpf_module_t *)ptr;
if (em->objects)
ebpf_unload_legacy_code(em->objects, em->probe_links);
if (em->enabled == NETDATA_THREAD_EBPF_FUNCTION_RUNNING) {
pthread_mutex_lock(&lock);
ebpf_obsolete_disk_global(em);
pthread_mutex_unlock(&lock);
fflush(stdout);
}
ebpf_disk_disable_tracepoints();
ebpf_update_kernel_memory_with_vector(&plugin_statistics, disk_maps, EBPF_ACTION_STAT_REMOVE);
if (em->objects) {
ebpf_unload_legacy_code(em->objects, em->probe_links);
em->objects = NULL;
em->probe_links = NULL;
}
if (dimensions)
ebpf_histogram_dimension_cleanup(dimensions, NETDATA_EBPF_HIST_MAX_BINS);
freez(disk_hash_values);
disk_hash_values = NULL;
pthread_mutex_destroy(&plot_mutex);
ebpf_cleanup_plot_disks();
@ -494,6 +539,7 @@ static void ebpf_disk_exit(void *ptr)
pthread_mutex_lock(&ebpf_exit_cleanup);
em->enabled = NETDATA_THREAD_EBPF_STOPPED;
ebpf_update_stats(&plugin_statistics, em);
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
@ -640,6 +686,8 @@ static void ebpf_create_hd_charts(netdata_ebpf_disks_t *w, int update_every)
order++;
w->flags |= NETDATA_DISK_CHART_CREATED;
fflush(stdout);
}
/**
@ -728,7 +776,9 @@ static void disk_collector(ebpf_module_t *em)
heartbeat_init(&hb);
int counter = update_every - 1;
int maps_per_core = em->maps_per_core;
while (!ebpf_exit_plugin) {
uint32_t running_time = 0;
uint32_t lifetime = em->lifetime;
while (!ebpf_exit_plugin && running_time < lifetime) {
(void)heartbeat_next(&hb, USEC_PER_SEC);
if (ebpf_exit_plugin || ++counter != update_every)
@ -743,6 +793,15 @@ static void disk_collector(ebpf_module_t *em)
pthread_mutex_unlock(&lock);
ebpf_update_disks(em);
pthread_mutex_lock(&ebpf_exit_cleanup);
if (running_time && !em->running_time)
running_time = update_every;
else
running_time += update_every;
em->running_time = running_time;
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
}
@ -866,7 +925,7 @@ void *ebpf_disk_thread(void *ptr)
pthread_mutex_lock(&lock);
ebpf_update_stats(&plugin_statistics, em);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, disk_maps);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, disk_maps, EBPF_ACTION_STAT_ADD);
pthread_mutex_unlock(&lock);
disk_collector(em);

View file

@ -3,8 +3,9 @@
#ifndef NETDATA_EBPF_DISK_H
#define NETDATA_EBPF_DISK_H 1
// Module name
// Module name & description
#define NETDATA_EBPF_MODULE_NAME_DISK "disk"
#define NETDATA_EBPF_DISK_MODULE_DESC "Monitor disk latency independent of filesystem."
#include "libnetdata/avl/avl.h"
#include "libnetdata/ebpf/ebpf.h"

View file

@ -57,6 +57,10 @@ netdata_ebpf_targets_t fd_targets[] = { {.name = "open", .mode = EBPF_LOAD_TRAMP
{.name = "close", .mode = EBPF_LOAD_TRAMPOLINE},
{.name = NULL, .mode = EBPF_LOAD_TRAMPOLINE}};
#ifdef NETDATA_DEV_MODE
int fd_disable_priority;
#endif
#ifdef LIBBPF_MAJOR_VERSION
/**
* Disable probe
@ -369,6 +373,170 @@ static inline int ebpf_fd_load_and_attach(struct fd_bpf *obj, ebpf_module_t *em)
*
*****************************************************************/
static void ebpf_obsolete_specific_fd_charts(char *type, ebpf_module_t *em);
/**
* Obsolete services
*
* Obsolete all service charts created
*
* @param em a pointer to `struct ebpf_module`
*/
static void ebpf_obsolete_fd_services(ebpf_module_t *em)
{
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SYSCALL_APPS_FILE_OPEN,
"Number of open files",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_APPS_FILE_CGROUP_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NETDATA_CGROUP_FD_OPEN_CONTEXT,
20061,
em->update_every);
if (em->mode < MODE_ENTRY) {
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SYSCALL_APPS_FILE_OPEN_ERROR,
"Fails to open files",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_APPS_FILE_CGROUP_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NETDATA_CGROUP_FD_OPEN_ERR_CONTEXT,
20062,
em->update_every);
}
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SYSCALL_APPS_FILE_CLOSED,
"Files closed",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_APPS_FILE_CGROUP_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NETDATA_CGROUP_FD_CLOSE_CONTEXT,
20063,
em->update_every);
if (em->mode < MODE_ENTRY) {
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SYSCALL_APPS_FILE_CLOSE_ERROR,
"Fails to close files",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_APPS_FILE_CGROUP_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NETDATA_CGROUP_FD_CLOSE_ERR_CONTEXT,
20064,
em->update_every);
}
}
/**
* Obsolete cgroup chart
*
* Send obsolete for all charts created before to close.
*
* @param em a pointer to `struct ebpf_module`
*/
static inline void ebpf_obsolete_fd_cgroup_charts(ebpf_module_t *em) {
pthread_mutex_lock(&mutex_cgroup_shm);
ebpf_obsolete_fd_services(em);
ebpf_cgroup_target_t *ect;
for (ect = ebpf_cgroup_pids; ect ; ect = ect->next) {
if (ect->systemd)
continue;
ebpf_obsolete_specific_fd_charts(ect->name, em);
}
pthread_mutex_unlock(&mutex_cgroup_shm);
}
/**
* Obsolette apps charts
*
* Obsolete apps charts.
*
* @param em a pointer to the structure with the default values.
*/
void ebpf_obsolete_fd_apps_charts(struct ebpf_module *em)
{
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SYSCALL_APPS_FILE_OPEN,
"Number of open files",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_APPS_FILE_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20061,
em->update_every);
if (em->mode < MODE_ENTRY) {
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SYSCALL_APPS_FILE_OPEN_ERROR,
"Fails to open files",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_APPS_FILE_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20062,
em->update_every);
}
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SYSCALL_APPS_FILE_CLOSED,
"Files closed",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_APPS_FILE_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20063,
em->update_every);
if (em->mode < MODE_ENTRY) {
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SYSCALL_APPS_FILE_CLOSE_ERROR,
"Fails to close files",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_APPS_FILE_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20064,
em->update_every);
}
}
/**
* Obsolete global
*
* Obsolete global charts created by thread.
*
* @param em a pointer to `struct ebpf_module`
*/
static void ebpf_obsolete_fd_global(ebpf_module_t *em)
{
ebpf_write_chart_obsolete(NETDATA_FILESYSTEM_FAMILY,
NETDATA_FILE_OPEN_CLOSE_COUNT,
"Open and close calls",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_FILE_GROUP,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
NETDATA_CHART_PRIO_EBPF_FD_CHARTS,
em->update_every);
if (em->mode < MODE_ENTRY) {
ebpf_write_chart_obsolete(NETDATA_FILESYSTEM_FAMILY,
NETDATA_FILE_OPEN_ERR_COUNT,
"Open fails",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_FILE_GROUP,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
NETDATA_CHART_PRIO_EBPF_FD_CHARTS + 1,
em->update_every);
}
}
/**
* FD Exit
*
@ -380,15 +548,46 @@ static void ebpf_fd_exit(void *ptr)
{
ebpf_module_t *em = (ebpf_module_t *)ptr;
#ifdef LIBBPF_MAJOR_VERSION
if (fd_bpf_obj)
fd_bpf__destroy(fd_bpf_obj);
if (em->enabled == NETDATA_THREAD_EBPF_FUNCTION_RUNNING) {
pthread_mutex_lock(&lock);
if (em->cgroup_charts) {
ebpf_obsolete_fd_cgroup_charts(em);
fflush(stdout);
}
if (em->apps_charts & NETDATA_EBPF_APPS_FLAG_CHART_CREATED) {
ebpf_obsolete_fd_apps_charts(em);
}
ebpf_obsolete_fd_global(em);
#ifdef NETDATA_DEV_MODE
if (ebpf_aral_fd_pid)
ebpf_statistic_obsolete_aral_chart(em, fd_disable_priority);
#endif
if (em->objects)
fflush(stdout);
pthread_mutex_unlock(&lock);
}
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_REMOVE);
#ifdef LIBBPF_MAJOR_VERSION
if (fd_bpf_obj) {
fd_bpf__destroy(fd_bpf_obj);
fd_bpf_obj = NULL;
}
#endif
if (em->objects) {
ebpf_unload_legacy_code(em->objects, em->probe_links);
em->objects = NULL;
em->probe_links = NULL;
}
pthread_mutex_lock(&ebpf_exit_cleanup);
em->enabled = NETDATA_THREAD_EBPF_STOPPED;
ebpf_update_stats(&plugin_statistics, em);
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
@ -935,7 +1134,9 @@ static void fd_collector(ebpf_module_t *em)
int update_every = em->update_every;
int counter = update_every - 1;
int maps_per_core = em->maps_per_core;
while (!ebpf_exit_plugin) {
uint32_t running_time = 0;
uint32_t lifetime = em->lifetime;
while (!ebpf_exit_plugin && running_time < lifetime) {
(void)heartbeat_next(&hb, USEC_PER_SEC);
if (ebpf_exit_plugin || ++counter != update_every)
@ -968,6 +1169,15 @@ static void fd_collector(ebpf_module_t *em)
pthread_mutex_unlock(&lock);
pthread_mutex_unlock(&collect_data_mutex);
pthread_mutex_lock(&ebpf_exit_cleanup);
if (running_time && !em->running_time)
running_time = update_every;
else
running_time += update_every;
em->running_time = running_time;
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
}
@ -1066,6 +1276,8 @@ static void ebpf_create_fd_global_charts(ebpf_module_t *em)
NETDATA_FD_SYSCALL_END,
em->update_every, NETDATA_EBPF_MODULE_NAME_FD);
}
fflush(stdout);
}
/*****************************************************************
@ -1165,10 +1377,10 @@ void *ebpf_fd_thread(void *ptr)
pthread_mutex_lock(&lock);
ebpf_create_fd_global_charts(em);
ebpf_update_stats(&plugin_statistics, em);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_ADD);
#ifdef NETDATA_DEV_MODE
if (ebpf_aral_fd_pid)
ebpf_statistic_create_aral_chart(NETDATA_EBPF_FD_ARAL_NAME, em);
fd_disable_priority = ebpf_statistic_create_aral_chart(NETDATA_EBPF_FD_ARAL_NAME, em);
#endif
pthread_mutex_unlock(&lock);

View file

@ -3,8 +3,9 @@
#ifndef NETDATA_EBPF_FD_H
#define NETDATA_EBPF_FD_H 1
// Module name
// Module name & File description
#define NETDATA_EBPF_MODULE_NAME_FD "filedescriptor"
#define NETDATA_EBPF_FD_MODULE_DESC "Monitor when files are open and closed. This thread is integrated with apps and cgroup."
// Menu group
#define NETDATA_FILE_GROUP "file_access"

View file

@ -395,13 +395,15 @@ static void ebpf_create_fs_charts(int update_every)
snprintfz(chart_name, 63, "%s_read_latency", efp->filesystem);
efp->hread.name = strdupz(chart_name);
efp->hread.title = strdupz(title);
efp->hread.ctx = NULL;
efp->hread.order = order;
efp->family_name = strdupz(family);
ebpf_create_chart(NETDATA_FILESYSTEM_FAMILY, efp->hread.name,
title,
EBPF_COMMON_DIMENSION_CALL, family,
"filesystem.read_latency", NETDATA_EBPF_CHART_TYPE_STACKED, order, ebpf_create_global_dimension,
efp->hread.title,
EBPF_COMMON_DIMENSION_CALL, efp->family_name,
"filesystem.read_latency", NETDATA_EBPF_CHART_TYPE_STACKED, order,
ebpf_create_global_dimension,
filesystem_publish_aggregated, NETDATA_EBPF_HIST_MAX_BINS,
update_every, NETDATA_EBPF_MODULE_NAME_FILESYSTEM);
order++;
@ -410,11 +412,13 @@ static void ebpf_create_fs_charts(int update_every)
snprintfz(chart_name, 63, "%s_write_latency", efp->filesystem);
efp->hwrite.name = strdupz(chart_name);
efp->hwrite.title = strdupz(title);
efp->hwrite.ctx = NULL;
efp->hwrite.order = order;
ebpf_create_chart(NETDATA_FILESYSTEM_FAMILY, efp->hwrite.name,
title,
EBPF_COMMON_DIMENSION_CALL, family,
"filesystem.write_latency", NETDATA_EBPF_CHART_TYPE_STACKED, order, ebpf_create_global_dimension,
efp->hwrite.title,
EBPF_COMMON_DIMENSION_CALL, efp->family_name,
"filesystem.write_latency", NETDATA_EBPF_CHART_TYPE_STACKED, order,
ebpf_create_global_dimension,
filesystem_publish_aggregated, NETDATA_EBPF_HIST_MAX_BINS,
update_every, NETDATA_EBPF_MODULE_NAME_FILESYSTEM);
order++;
@ -423,11 +427,13 @@ static void ebpf_create_fs_charts(int update_every)
snprintfz(chart_name, 63, "%s_open_latency", efp->filesystem);
efp->hopen.name = strdupz(chart_name);
efp->hopen.title = strdupz(title);
efp->hopen.ctx = NULL;
efp->hopen.order = order;
ebpf_create_chart(NETDATA_FILESYSTEM_FAMILY, efp->hopen.name,
title,
EBPF_COMMON_DIMENSION_CALL, family,
"filesystem.open_latency", NETDATA_EBPF_CHART_TYPE_STACKED, order, ebpf_create_global_dimension,
efp->hopen.title,
EBPF_COMMON_DIMENSION_CALL, efp->family_name,
"filesystem.open_latency", NETDATA_EBPF_CHART_TYPE_STACKED, order,
ebpf_create_global_dimension,
filesystem_publish_aggregated, NETDATA_EBPF_HIST_MAX_BINS,
update_every, NETDATA_EBPF_MODULE_NAME_FILESYSTEM);
order++;
@ -438,9 +444,10 @@ static void ebpf_create_fs_charts(int update_every)
snprintfz(ctx, 63, "filesystem.%s_latency", type);
efp->hadditional.name = strdupz(chart_name);
efp->hadditional.title = strdupz(title);
efp->hadditional.ctx = strdupz(ctx);
efp->hadditional.order = order;
ebpf_create_chart(NETDATA_FILESYSTEM_FAMILY, efp->hadditional.name, title,
EBPF_COMMON_DIMENSION_CALL, family,
ebpf_create_chart(NETDATA_FILESYSTEM_FAMILY, efp->hadditional.name, efp->hadditional.title,
EBPF_COMMON_DIMENSION_CALL, efp->family_name,
ctx, NETDATA_EBPF_CHART_TYPE_STACKED, order, ebpf_create_global_dimension,
filesystem_publish_aggregated, NETDATA_EBPF_HIST_MAX_BINS,
update_every, NETDATA_EBPF_MODULE_NAME_FILESYSTEM);
@ -448,6 +455,8 @@ static void ebpf_create_fs_charts(int update_every)
efp->flags |= NETDATA_FILESYSTEM_FLAG_CHART_CREATED;
}
}
fflush(stdout);
}
/**
@ -459,6 +468,7 @@ static void ebpf_create_fs_charts(int update_every)
*/
int ebpf_filesystem_initialize_ebpf_data(ebpf_module_t *em)
{
pthread_mutex_lock(&lock);
int i;
const char *saved_name = em->thread_name;
uint64_t kernels = em->kernels;
@ -476,6 +486,8 @@ int ebpf_filesystem_initialize_ebpf_data(ebpf_module_t *em)
if (!efp->probe_links) {
em->thread_name = saved_name;
em->kernels = kernels;
em->maps = NULL;
pthread_mutex_unlock(&lock);
return -1;
}
}
@ -494,9 +506,7 @@ int ebpf_filesystem_initialize_ebpf_data(ebpf_module_t *em)
}
#endif
efp->flags |= NETDATA_FILESYSTEM_FLAG_HAS_PARTITION;
pthread_mutex_lock(&lock);
ebpf_update_kernel_memory(&plugin_statistics, efp->fs_maps, EBPF_ACTION_STAT_ADD);
pthread_mutex_unlock(&lock);
// Nedeed for filesystems like btrfs
if ((efp->flags & NETDATA_FILESYSTEM_FILL_ADDRESS_TABLE) && (efp->addresses.function)) {
@ -506,6 +516,7 @@ int ebpf_filesystem_initialize_ebpf_data(ebpf_module_t *em)
efp->flags &= ~NETDATA_FILESYSTEM_LOAD_EBPF_PROGRAM;
}
em->thread_name = saved_name;
pthread_mutex_unlock(&lock);
em->kernels = kernels;
em->maps = NULL;
@ -616,43 +627,88 @@ void ebpf_filesystem_cleanup_ebpf_data()
ebpf_filesystem_partitions_t *efp = &localfs[i];
if (efp->probe_links) {
freez(efp->family_name);
efp->family_name = NULL;
freez(efp->hread.name);
efp->hread.name = NULL;
freez(efp->hread.title);
efp->hread.title = NULL;
freez(efp->hwrite.name);
efp->hwrite.name = NULL;
freez(efp->hwrite.title);
efp->hwrite.title = NULL;
freez(efp->hopen.name);
efp->hopen.name = NULL;
freez(efp->hopen.title);
efp->hopen.title = NULL;
freez(efp->hadditional.name);
efp->hadditional.name = NULL;
freez(efp->hadditional.title);
efp->hadditional.title = NULL;
freez(efp->hadditional.ctx);
efp->hadditional.ctx = NULL;
}
}
}
/**
* Filesystem Free
* Obsolete global
*
* Cleanup variables after child threads to stop
* Obsolete global charts created by thread.
*
* @param ptr thread data.
* @param em a pointer to `struct ebpf_module`
*/
static void ebpf_filesystem_free(ebpf_module_t *em)
static void ebpf_obsolete_filesystem_global(ebpf_module_t *em)
{
pthread_mutex_lock(&ebpf_exit_cleanup);
em->enabled = NETDATA_THREAD_EBPF_STOPPING;
pthread_mutex_unlock(&ebpf_exit_cleanup);
int i;
for (i = 0; localfs[i].filesystem; i++) {
ebpf_filesystem_partitions_t *efp = &localfs[i];
if (!efp->objects)
continue;
ebpf_filesystem_cleanup_ebpf_data();
if (dimensions)
ebpf_histogram_dimension_cleanup(dimensions, NETDATA_EBPF_HIST_MAX_BINS);
freez(filesystem_hash_values);
ebpf_write_chart_obsolete(NETDATA_FILESYSTEM_FAMILY,
efp->hread.name,
efp->hread.title,
EBPF_COMMON_DIMENSION_CALL,
efp->family_name,
NETDATA_EBPF_CHART_TYPE_STACKED,
"filesystem.read_latency",
efp->hread.order,
em->update_every);
pthread_mutex_lock(&ebpf_exit_cleanup);
em->enabled = NETDATA_THREAD_EBPF_STOPPED;
pthread_mutex_unlock(&ebpf_exit_cleanup);
ebpf_write_chart_obsolete(NETDATA_FILESYSTEM_FAMILY,
efp->hwrite.name,
efp->hwrite.title,
EBPF_COMMON_DIMENSION_CALL,
efp->family_name,
NETDATA_EBPF_CHART_TYPE_STACKED,
"filesystem.write_latency",
efp->hwrite.order,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_FILESYSTEM_FAMILY,
efp->hopen.name,
efp->hopen.title,
EBPF_COMMON_DIMENSION_CALL,
efp->family_name,
NETDATA_EBPF_CHART_TYPE_STACKED,
"filesystem.open_latency",
efp->hopen.order,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_FILESYSTEM_FAMILY,
efp->hadditional.name,
efp->hadditional.title,
EBPF_COMMON_DIMENSION_CALL,
efp->family_name,
NETDATA_EBPF_CHART_TYPE_STACKED,
efp->hadditional.ctx,
efp->hadditional.order,
em->update_every);
}
}
/**
@ -665,7 +721,39 @@ static void ebpf_filesystem_free(ebpf_module_t *em)
static void ebpf_filesystem_exit(void *ptr)
{
ebpf_module_t *em = (ebpf_module_t *)ptr;
ebpf_filesystem_free(em);
if (em->enabled == NETDATA_THREAD_EBPF_FUNCTION_RUNNING) {
pthread_mutex_lock(&lock);
ebpf_obsolete_filesystem_global(em);
pthread_mutex_unlock(&lock);
fflush(stdout);
}
ebpf_filesystem_cleanup_ebpf_data();
if (dimensions) {
ebpf_histogram_dimension_cleanup(dimensions, NETDATA_EBPF_HIST_MAX_BINS);
dimensions = NULL;
}
freez(filesystem_hash_values);
int i;
for (i = 0; localfs[i].filesystem; i++) {
ebpf_filesystem_partitions_t *efp = &localfs[i];
if (!efp->probe_links)
continue;
ebpf_unload_legacy_code(efp->objects, efp->probe_links);
efp->objects = NULL;
efp->probe_links = NULL;
efp->flags = NETDATA_FILESYSTEM_FLAG_NO_PARTITION;
}
pthread_mutex_lock(&ebpf_exit_cleanup);
em->enabled = NETDATA_THREAD_EBPF_STOPPED;
ebpf_update_stats(&plugin_statistics, em);
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
/*****************************************************************
@ -819,7 +907,9 @@ static void filesystem_collector(ebpf_module_t *em)
heartbeat_t hb;
heartbeat_init(&hb);
int counter = update_every - 1;
while (!ebpf_exit_plugin) {
uint32_t running_time = 0;
uint32_t lifetime = em->lifetime;
while (!ebpf_exit_plugin && running_time < lifetime) {
(void)heartbeat_next(&hb, USEC_PER_SEC);
if (ebpf_exit_plugin || ++counter != update_every)
@ -833,6 +923,15 @@ static void filesystem_collector(ebpf_module_t *em)
ebpf_histogram_send_data();
pthread_mutex_unlock(&lock);
pthread_mutex_lock(&ebpf_exit_cleanup);
if (running_time && !em->running_time)
running_time = update_every;
else
running_time += update_every;
em->running_time = running_time;
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
}

View file

@ -3,8 +3,9 @@
#ifndef NETDATA_EBPF_FILESYSTEM_H
#define NETDATA_EBPF_FILESYSTEM_H 1
// Module name
// Module name & description
#define NETDATA_EBPF_MODULE_NAME_FILESYSTEM "filesystem"
#define NETDATA_EBPF_FS_MODULE_DESC "Monitor filesystem latency for: btrfs, ext4, nfs, xfs and zfs."
#include "ebpf.h"
#ifdef LIBBPF_MAJOR_VERSION

View file

@ -0,0 +1,419 @@
// SPDX-License-Identifier: GPL-3.0-or-later
#include "ebpf.h"
#include "ebpf_functions.h"
/*****************************************************************
* EBPF SELECT MODULE
*****************************************************************/
/**
* Select Module
*
* @param thread_name name of the thread we are looking for.
*
* @return it returns a pointer for the module that has thread_name on success or NULL otherwise.
*/
ebpf_module_t *ebpf_functions_select_module(const char *thread_name) {
int i;
for (i = 0; i < EBPF_MODULE_FUNCTION_IDX; i++) {
if (strcmp(ebpf_modules[i].thread_name, thread_name) == 0) {
return &ebpf_modules[i];
}
}
return NULL;
}
/*****************************************************************
* EBPF HELP FUNCTIONS
*****************************************************************/
/**
* Thread Help
*
* Shows help with all options accepted by thread function.
*
* @param transaction the transaction id that Netdata sent for this function execution
*/
static void ebpf_function_thread_manipulation_help(const char *transaction) {
pthread_mutex_lock(&lock);
pluginsd_function_result_begin_to_stdout(transaction, HTTP_RESP_OK, "text/plain", now_realtime_sec() + 3600);
fprintf(stdout, "%s",
"ebpf.plugin / thread\n"
"\n"
"Function `thread` allows user to control eBPF threads.\n"
"\n"
"The following filters are supported:\n"
"\n"
" thread:NAME\n"
" Shows information for the thread NAME. Names are listed inside `ebpf.d.conf`.\n"
"\n"
" enable:NAME:PERIOD\n"
" Enable a specific thread named `NAME` to run a specific PERIOD in seconds. When PERIOD is not\n"
" specified plugin will use the default 300 seconds\n"
"\n"
" disable:NAME\n"
" Disable a sp.\n"
"\n"
"Filters can be combined. Each filter can be given only one time.\n"
"Process thread is not controlled by functions until we finish the creation of functions per thread..\n"
);
pluginsd_function_result_end_to_stdout();
fflush(stdout);
pthread_mutex_unlock(&lock);
}
/*****************************************************************
* EBPF ERROR FUNCTIONS
*****************************************************************/
/**
* Function error
*
* Show error when a wrong function is given
*
* @param transaction the transaction id that Netdata sent for this function execution
* @param code the error code to show with the message.
* @param msg the error message
*/
static void ebpf_function_error(const char *transaction, int code, const char *msg) {
char buffer[PLUGINSD_LINE_MAX + 1];
json_escape_string(buffer, msg, PLUGINSD_LINE_MAX);
pluginsd_function_result_begin_to_stdout(transaction, code, "application/json", now_realtime_sec());
fprintf(stdout, "{\"status\":%d,\"error_message\":\"%s\"}", code, buffer);
pluginsd_function_result_end_to_stdout();
}
/*****************************************************************
* EBPF THREAD FUNCTION
*****************************************************************/
/**
* Function enable
*
* Enable a specific thread.
*
* @param transaction the transaction id that Netdata sent for this function execution
* @param function function name and arguments given to thread.
* @param line_buffer buffer used to parse args
* @param line_max Number of arguments given
* @param timeout The function timeout
* @param em The structure with thread information
*/
static void ebpf_function_thread_manipulation(const char *transaction,
char *function __maybe_unused,
char *line_buffer __maybe_unused,
int line_max __maybe_unused,
int timeout __maybe_unused,
ebpf_module_t *em)
{
char *words[PLUGINSD_MAX_WORDS] = { NULL };
char message[512];
uint32_t show_specific_thread = 0;
size_t num_words = quoted_strings_splitter_pluginsd(function, words, PLUGINSD_MAX_WORDS);
for(int i = 1; i < PLUGINSD_MAX_WORDS ;i++) {
const char *keyword = get_word(words, num_words, i);
if (!keyword)
break;
ebpf_module_t *lem;
if(strncmp(keyword, EBPF_THREADS_ENABLE_CATEGORY, sizeof(EBPF_THREADS_ENABLE_CATEGORY) -1) == 0) {
char thread_name[128];
int period = -1;
const char *name = &keyword[sizeof(EBPF_THREADS_ENABLE_CATEGORY) - 1];
char *separator = strchr(name, ':');
if (separator) {
strncpyz(thread_name, name, separator - name);
period = str2i(++separator);
} else {
strncpyz(thread_name, name, strlen(name));
}
lem = ebpf_functions_select_module(thread_name);
if (!lem) {
snprintfz(message, 511, "%s%s", EBPF_PLUGIN_THREAD_FUNCTION_ERROR_THREAD_NOT_FOUND, name);
ebpf_function_error(transaction, HTTP_RESP_NOT_FOUND, message);
return;
}
pthread_mutex_lock(&ebpf_exit_cleanup);
if (lem->enabled > NETDATA_THREAD_EBPF_FUNCTION_RUNNING) {
struct netdata_static_thread *st = lem->thread;
// Load configuration again
ebpf_update_module(lem, default_btf, running_on_kernel, isrh);
// another request for thread that already ran, cleanup and restart
if (st->thread)
freez(st->thread);
if (period <= 0)
period = EBPF_DEFAULT_LIFETIME;
st->thread = mallocz(sizeof(netdata_thread_t));
lem->enabled = NETDATA_THREAD_EBPF_FUNCTION_RUNNING;
lem->lifetime = period;
#ifdef NETDATA_INTERNAL_CHECKS
netdata_log_info("Starting thread %s with lifetime = %d", thread_name, period);
#endif
netdata_thread_create(st->thread, st->name, NETDATA_THREAD_OPTION_DEFAULT,
st->start_routine, lem);
} else {
lem->running_time = 0;
if (period > 0) // user is modifying period to run
lem->lifetime = period;
#ifdef NETDATA_INTERNAL_CHECKS
netdata_log_info("Thread %s had lifetime updated for %d", thread_name, period);
#endif
}
pthread_mutex_unlock(&ebpf_exit_cleanup);
} else if(strncmp(keyword, EBPF_THREADS_DISABLE_CATEGORY, sizeof(EBPF_THREADS_DISABLE_CATEGORY) -1) == 0) {
const char *name = &keyword[sizeof(EBPF_THREADS_DISABLE_CATEGORY) - 1];
lem = ebpf_functions_select_module(name);
if (!lem) {
snprintfz(message, 511, "%s%s", EBPF_PLUGIN_THREAD_FUNCTION_ERROR_THREAD_NOT_FOUND, name);
ebpf_function_error(transaction, HTTP_RESP_NOT_FOUND, message);
return;
}
pthread_mutex_lock(&ebpf_exit_cleanup);
if (lem->enabled < NETDATA_THREAD_EBPF_STOPPING && lem->thread->thread) {
lem->lifetime = 0;
lem->running_time = lem->update_every;
netdata_thread_cancel(*lem->thread->thread);
}
pthread_mutex_unlock(&ebpf_exit_cleanup);
} else if(strncmp(keyword, EBPF_THREADS_SELECT_THREAD, sizeof(EBPF_THREADS_SELECT_THREAD) -1) == 0) {
const char *name = &keyword[sizeof(EBPF_THREADS_SELECT_THREAD) - 1];
lem = ebpf_functions_select_module(name);
if (!lem) {
snprintfz(message, 511, "%s%s", EBPF_PLUGIN_THREAD_FUNCTION_ERROR_THREAD_NOT_FOUND, name);
ebpf_function_error(transaction, HTTP_RESP_NOT_FOUND, message);
return;
}
show_specific_thread |= 1<<lem->thread_id;
} else if(strncmp(keyword, "help", 4) == 0) {
ebpf_function_thread_manipulation_help(transaction);
return;
}
}
time_t expires = now_realtime_sec() + em->update_every;
BUFFER *wb = buffer_create(PLUGINSD_LINE_MAX, NULL);
buffer_json_initialize(wb, "\"", "\"", 0, true, false);
buffer_json_member_add_uint64(wb, "status", HTTP_RESP_OK);
buffer_json_member_add_string(wb, "type", "table");
buffer_json_member_add_time_t(wb, "update_every", em->update_every);
buffer_json_member_add_string(wb, "help", EBPF_PLUGIN_THREAD_FUNCTION_DESCRIPTION);
// Collect data
buffer_json_member_add_array(wb, "data");
int i;
for (i = 0; i < EBPF_MODULE_FUNCTION_IDX; i++) {
if (show_specific_thread && !(show_specific_thread & 1<<i))
continue;
ebpf_module_t *wem = &ebpf_modules[i];
buffer_json_add_array_item_array(wb);
// IMPORTANT!
// THE ORDER SHOULD BE THE SAME WITH THE FIELDS!
// thread name
buffer_json_add_array_item_string(wb, wem->thread_name);
// description
buffer_json_add_array_item_string(wb, wem->thread_description);
// Either it is not running or received a disabled signal and it is stopping.
if (wem->enabled > NETDATA_THREAD_EBPF_FUNCTION_RUNNING ||
(!wem->lifetime && (int)wem->running_time == wem->update_every)) {
// status
buffer_json_add_array_item_string(wb, EBPF_THREAD_STATUS_STOPPED);
// Time remaining
buffer_json_add_array_item_uint64(wb, 0);
// action
buffer_json_add_array_item_string(wb, "NULL");
} else {
// status
buffer_json_add_array_item_string(wb, EBPF_THREAD_STATUS_RUNNING);
// Time remaining
buffer_json_add_array_item_uint64(wb, (wem->lifetime) ? (wem->lifetime - wem->running_time) : 0);
// action
buffer_json_add_array_item_string(wb, "Enabled/Disabled");
}
buffer_json_array_close(wb);
}
buffer_json_array_close(wb); // data
buffer_json_member_add_object(wb, "columns");
{
int fields_id = 0;
// IMPORTANT!
// THE ORDER SHOULD BE THE SAME WITH THE VALUES!
buffer_rrdf_table_add_field(wb, fields_id++, "Thread", "Thread Name", RRDF_FIELD_TYPE_STRING,
RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE, 0, NULL, NAN,
RRDF_FIELD_SORT_ASCENDING, NULL, RRDF_FIELD_SUMMARY_COUNT,
RRDF_FIELD_FILTER_MULTISELECT,
RRDF_FIELD_OPTS_VISIBLE | RRDF_FIELD_OPTS_STICKY, NULL);
buffer_rrdf_table_add_field(wb, fields_id++, "Description", "Thread Desc", RRDF_FIELD_TYPE_STRING,
RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE, 0, NULL, NAN,
RRDF_FIELD_SORT_ASCENDING, NULL, RRDF_FIELD_SUMMARY_COUNT,
RRDF_FIELD_FILTER_MULTISELECT,
RRDF_FIELD_OPTS_VISIBLE | RRDF_FIELD_OPTS_STICKY, NULL);
buffer_rrdf_table_add_field(wb, fields_id++, "Status", "Thread Status", RRDF_FIELD_TYPE_STRING,
RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE, 0, NULL, NAN,
RRDF_FIELD_SORT_ASCENDING, NULL, RRDF_FIELD_SUMMARY_COUNT,
RRDF_FIELD_FILTER_MULTISELECT,
RRDF_FIELD_OPTS_VISIBLE | RRDF_FIELD_OPTS_STICKY, NULL);
buffer_rrdf_table_add_field(wb, fields_id++, "Time", "Time Remaining", RRDF_FIELD_TYPE_INTEGER,
RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER, 0, NULL,
NAN, RRDF_FIELD_SORT_ASCENDING, NULL, RRDF_FIELD_SUMMARY_COUNT,
RRDF_FIELD_FILTER_MULTISELECT,
RRDF_FIELD_OPTS_NONE, NULL);
buffer_rrdf_table_add_field(wb, fields_id++, "Action", "Thread Action", RRDF_FIELD_TYPE_STRING,
RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE, 0, NULL, NAN,
RRDF_FIELD_SORT_ASCENDING, NULL, RRDF_FIELD_SUMMARY_COUNT,
RRDF_FIELD_FILTER_MULTISELECT,
RRDF_FIELD_OPTS_VISIBLE | RRDF_FIELD_OPTS_STICKY, NULL);
}
buffer_json_object_close(wb); // columns
buffer_json_member_add_string(wb, "default_sort_column", "Thread");
buffer_json_member_add_object(wb, "charts");
{
// Threads
buffer_json_member_add_object(wb, "eBPFThreads");
{
buffer_json_member_add_string(wb, "name", "Threads");
buffer_json_member_add_string(wb, "type", "line");
buffer_json_member_add_array(wb, "columns");
{
buffer_json_add_array_item_string(wb, "Threads");
}
buffer_json_array_close(wb);
}
buffer_json_object_close(wb);
// Life Time
buffer_json_member_add_object(wb, "eBPFLifeTime");
{
buffer_json_member_add_string(wb, "name", "LifeTime");
buffer_json_member_add_string(wb, "type", "line");
buffer_json_member_add_array(wb, "columns");
{
buffer_json_add_array_item_string(wb, "Threads");
buffer_json_add_array_item_string(wb, "Time");
}
buffer_json_array_close(wb);
}
buffer_json_object_close(wb);
}
buffer_json_object_close(wb); // charts
// Do we use only on fields that can be groupped?
buffer_json_member_add_object(wb, "group_by");
{
// group by Status
buffer_json_member_add_object(wb, "Status");
{
buffer_json_member_add_string(wb, "name", "Thread status");
buffer_json_member_add_array(wb, "columns");
{
buffer_json_add_array_item_string(wb, "Status");
}
buffer_json_array_close(wb);
}
buffer_json_object_close(wb);
}
buffer_json_object_close(wb); // group_by
buffer_json_member_add_time_t(wb, "expires", expires);
buffer_json_finalize(wb);
// Lock necessary to avoid race condition
pthread_mutex_lock(&lock);
pluginsd_function_result_begin_to_stdout(transaction, HTTP_RESP_OK, "application/json", expires);
fwrite(buffer_tostring(wb), buffer_strlen(wb), 1, stdout);
pluginsd_function_result_end_to_stdout();
fflush(stdout);
pthread_mutex_unlock(&lock);
buffer_free(wb);
}
/*****************************************************************
* EBPF FUNCTION THREAD
*****************************************************************/
/**
* FUNCTION thread.
*
* @param ptr a `ebpf_module_t *`.
*
* @return always NULL.
*/
void *ebpf_function_thread(void *ptr)
{
ebpf_module_t *em = (ebpf_module_t *)ptr;
char buffer[PLUGINSD_LINE_MAX + 1];
char *s = NULL;
while(!ebpf_exit_plugin && (s = fgets(buffer, PLUGINSD_LINE_MAX, stdin))) {
char *words[PLUGINSD_MAX_WORDS] = { NULL };
size_t num_words = quoted_strings_splitter_pluginsd(buffer, words, PLUGINSD_MAX_WORDS);
const char *keyword = get_word(words, num_words, 0);
if(keyword && strcmp(keyword, PLUGINSD_KEYWORD_FUNCTION) == 0) {
char *transaction = get_word(words, num_words, 1);
char *timeout_s = get_word(words, num_words, 2);
char *function = get_word(words, num_words, 3);
if(!transaction || !*transaction || !timeout_s || !*timeout_s || !function || !*function) {
netdata_log_error("Received incomplete %s (transaction = '%s', timeout = '%s', function = '%s'). Ignoring it.",
keyword,
transaction?transaction:"(unset)",
timeout_s?timeout_s:"(unset)",
function?function:"(unset)");
}
else {
int timeout = str2i(timeout_s);
if (!strncmp(function, EBPF_FUNCTION_THREAD, sizeof(EBPF_FUNCTION_THREAD) - 1))
ebpf_function_thread_manipulation(transaction,
function,
buffer,
PLUGINSD_LINE_MAX + 1,
timeout,
em);
else
ebpf_function_error(transaction,
HTTP_RESP_NOT_FOUND,
"No function with this name found in ebpf.plugin.");
}
}
else
netdata_log_error("Received unknown command: %s", keyword ? keyword : "(unset)");
}
return NULL;
}

View file

@ -0,0 +1,29 @@
// SPDX-License-Identifier: GPL-3.0-or-later
#ifndef NETDATA_EBPF_FUNCTIONS_H
#define NETDATA_EBPF_FUNCTIONS_H 1
// configuration file & description
#define NETDATA_DIRECTORY_FUNCTIONS_CONFIG_FILE "functions.conf"
#define NETDATA_EBPF_FUNCTIONS_MODULE_DESC "Show information about current function status."
// function list
#define EBPF_FUNCTION_THREAD "ebpf_thread"
#define EBPF_PLUGIN_THREAD_FUNCTION_DESCRIPTION "Detailed information about eBPF threads."
#define EBPF_PLUGIN_THREAD_FUNCTION_ERROR_THREAD_NOT_FOUND "ebpf.plugin does not have thread named "
#define EBPF_PLUGIN_FUNCTIONS(NAME, DESC) do { \
fprintf(stdout, PLUGINSD_KEYWORD_FUNCTION " \"" NAME "\" 10 \"%s\"\n", DESC); \
} while(0)
#define EBPF_THREADS_SELECT_THREAD "thread:"
#define EBPF_THREADS_ENABLE_CATEGORY "enable:"
#define EBPF_THREADS_DISABLE_CATEGORY "disable:"
#define EBPF_THREAD_STATUS_RUNNING "running"
#define EBPF_THREAD_STATUS_STOPPED "stopped"
void *ebpf_function_thread(void *ptr);
#endif

View file

@ -215,6 +215,27 @@ void ebpf_hardirq_release(hardirq_val_t *stat)
*
*****************************************************************/
/**
* Obsolete global
*
* Obsolete global charts created by thread.
*
* @param em a pointer to `struct ebpf_module`
*/
static void ebpf_obsolete_hardirq_global(ebpf_module_t *em)
{
ebpf_write_chart_obsolete(NETDATA_EBPF_SYSTEM_GROUP,
"hardirq_latency",
"Hardware IRQ latency",
EBPF_COMMON_DIMENSION_MILLISECONDS,
"interrupts",
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
NETDATA_CHART_PRIO_HARDIRQ_LATENCY,
em->update_every
);
}
/**
* Hardirq Exit
*
@ -226,8 +247,22 @@ static void hardirq_exit(void *ptr)
{
ebpf_module_t *em = (ebpf_module_t *)ptr;
if (em->objects)
if (em->enabled == NETDATA_THREAD_EBPF_FUNCTION_RUNNING) {
pthread_mutex_lock(&lock);
ebpf_obsolete_hardirq_global(em);
pthread_mutex_unlock(&lock);
fflush(stdout);
}
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_REMOVE);
if (em->objects) {
ebpf_unload_legacy_code(em->objects, em->probe_links);
em->objects = NULL;
em->probe_links = NULL;
}
for (int i = 0; hardirq_tracepoints[i].class != NULL; i++) {
ebpf_disable_tracepoint(&hardirq_tracepoints[i]);
@ -235,6 +270,7 @@ static void hardirq_exit(void *ptr)
pthread_mutex_lock(&ebpf_exit_cleanup);
em->enabled = NETDATA_THREAD_EBPF_STOPPED;
ebpf_update_stats(&plugin_statistics, em);
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
@ -533,7 +569,7 @@ static void hardirq_collector(ebpf_module_t *em)
hardirq_create_charts(em->update_every);
hardirq_create_static_dims();
ebpf_update_stats(&plugin_statistics, em);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_ADD);
pthread_mutex_unlock(&lock);
// loop and read from published data until ebpf plugin is closed.
@ -542,7 +578,9 @@ static void hardirq_collector(ebpf_module_t *em)
int update_every = em->update_every;
int counter = update_every - 1;
//This will be cancelled by its parent
while (!ebpf_exit_plugin) {
uint32_t running_time = 0;
uint32_t lifetime = em->lifetime;
while (!ebpf_exit_plugin && running_time < lifetime) {
(void)heartbeat_next(&hb, USEC_PER_SEC);
if (ebpf_exit_plugin || ++counter != update_every)
@ -561,6 +599,15 @@ static void hardirq_collector(ebpf_module_t *em)
write_end_chart();
pthread_mutex_unlock(&lock);
pthread_mutex_lock(&ebpf_exit_cleanup);
if (running_time && !em->running_time)
running_time = update_every;
else
running_time += update_every;
em->running_time = running_time;
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
}

View file

@ -3,6 +3,9 @@
#ifndef NETDATA_EBPF_HARDIRQ_H
#define NETDATA_EBPF_HARDIRQ_H 1
// Module description
#define NETDATA_EBPF_HARDIRQ_MODULE_DESC "Show time spent servicing individual hardware interrupt requests (hard IRQs)."
#include <stdint.h>
#include "libnetdata/avl/avl.h"

View file

@ -129,6 +129,26 @@ static inline int ebpf_mdflush_load_and_attach(struct mdflush_bpf *obj, ebpf_mod
#endif
/**
* Obsolete global
*
* Obsolete global charts created by thread.
*
* @param em a pointer to `struct ebpf_module`
*/
static void ebpf_obsolete_mdflush_global(ebpf_module_t *em)
{
ebpf_write_chart_obsolete("mdstat",
"mdstat_flush",
"MD flushes",
"flushes",
"flush (eBPF)",
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
NETDATA_CHART_PRIO_MDSTAT_FLUSH,
em->update_every);
}
/**
* MDflush exit
*
@ -140,11 +160,26 @@ static void mdflush_exit(void *ptr)
{
ebpf_module_t *em = (ebpf_module_t *)ptr;
if (em->objects)
if (em->enabled == NETDATA_THREAD_EBPF_FUNCTION_RUNNING) {
pthread_mutex_lock(&lock);
ebpf_obsolete_mdflush_global(em);
pthread_mutex_unlock(&lock);
fflush(stdout);
}
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_REMOVE);
if (em->objects) {
ebpf_unload_legacy_code(em->objects, em->probe_links);
em->objects = NULL;
em->probe_links = NULL;
}
pthread_mutex_lock(&ebpf_exit_cleanup);
em->enabled = NETDATA_THREAD_EBPF_STOPPED;
ebpf_update_stats(&plugin_statistics, em);
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
@ -300,7 +335,7 @@ static void mdflush_collector(ebpf_module_t *em)
pthread_mutex_lock(&lock);
mdflush_create_charts(update_every);
ebpf_update_stats(&plugin_statistics, em);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_ADD);
pthread_mutex_unlock(&lock);
// loop and read from published data until ebpf plugin is closed.
@ -308,7 +343,9 @@ static void mdflush_collector(ebpf_module_t *em)
heartbeat_init(&hb);
int counter = update_every - 1;
int maps_per_core = em->maps_per_core;
while (!ebpf_exit_plugin) {
uint32_t running_time = 0;
uint32_t lifetime = em->lifetime;
while (!ebpf_exit_plugin && running_time < lifetime) {
(void)heartbeat_next(&hb, USEC_PER_SEC);
if (ebpf_exit_plugin || ++counter != update_every)
@ -323,6 +360,15 @@ static void mdflush_collector(ebpf_module_t *em)
write_end_chart();
pthread_mutex_unlock(&lock);
pthread_mutex_lock(&ebpf_exit_cleanup);
if (running_time && !em->running_time)
running_time = update_every;
else
running_time += update_every;
em->running_time = running_time;
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
}

View file

@ -3,8 +3,9 @@
#ifndef NETDATA_EBPF_MDFLUSH_H
#define NETDATA_EBPF_MDFLUSH_H 1
// Module name
// Module name & description
#define NETDATA_EBPF_MODULE_NAME_MDFLUSH "mdflush"
#define NETDATA_EBPF_MD_MODULE_DESC "Show information about multi-device software flushes."
// charts
#define NETDATA_MDFLUSH_GLOBAL_CHART "mdflush"

View file

@ -222,6 +222,36 @@ static inline int ebpf_mount_load_and_attach(struct mount_bpf *obj, ebpf_module_
*
*****************************************************************/
/**
* Obsolete global
*
* Obsolete global charts created by thread.
*
* @param em a pointer to `struct ebpf_module`
*/
static void ebpf_obsolete_mount_global(ebpf_module_t *em)
{
ebpf_write_chart_obsolete(NETDATA_EBPF_MOUNT_GLOBAL_FAMILY,
NETDATA_EBPF_MOUNT_CALLS,
"Calls to mount and umount syscalls",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_EBPF_MOUNT_FAMILY,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
NETDATA_CHART_PRIO_EBPF_MOUNT_CHARTS,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_EBPF_MOUNT_GLOBAL_FAMILY,
NETDATA_EBPF_MOUNT_ERRORS,
"Errors to mount and umount file systems",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_EBPF_MOUNT_FAMILY,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
NETDATA_CHART_PRIO_EBPF_MOUNT_CHARTS + 1,
em->update_every);
}
/**
* Mount Exit
*
@ -233,15 +263,32 @@ static void ebpf_mount_exit(void *ptr)
{
ebpf_module_t *em = (ebpf_module_t *)ptr;
if (em->enabled == NETDATA_THREAD_EBPF_FUNCTION_RUNNING) {
pthread_mutex_lock(&lock);
ebpf_obsolete_mount_global(em);
fflush(stdout);
pthread_mutex_unlock(&lock);
}
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_REMOVE);
#ifdef LIBBPF_MAJOR_VERSION
if (mount_bpf_obj)
if (mount_bpf_obj) {
mount_bpf__destroy(mount_bpf_obj);
mount_bpf_obj = NULL;
}
#endif
if (em->objects)
if (em->objects) {
ebpf_unload_legacy_code(em->objects, em->probe_links);
em->objects = NULL;
em->probe_links = NULL;
}
pthread_mutex_lock(&ebpf_exit_cleanup);
em->enabled = NETDATA_THREAD_EBPF_STOPPED;
ebpf_update_stats(&plugin_statistics, em);
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
@ -318,7 +365,9 @@ static void mount_collector(ebpf_module_t *em)
int update_every = em->update_every;
int counter = update_every - 1;
int maps_per_core = em->maps_per_core;
while (!ebpf_exit_plugin) {
uint32_t running_time = 0;
uint32_t lifetime = em->lifetime;
while (!ebpf_exit_plugin && running_time < lifetime) {
(void)heartbeat_next(&hb, USEC_PER_SEC);
if (ebpf_exit_plugin || ++counter != update_every)
continue;
@ -330,6 +379,15 @@ static void mount_collector(ebpf_module_t *em)
ebpf_mount_send_data();
pthread_mutex_unlock(&lock);
pthread_mutex_lock(&ebpf_exit_cleanup);
if (running_time && !em->running_time)
running_time = update_every;
else
running_time += update_every;
em->running_time = running_time;
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
}
@ -444,7 +502,7 @@ void *ebpf_mount_thread(void *ptr)
pthread_mutex_lock(&lock);
ebpf_create_mount_charts(em->update_every);
ebpf_update_stats(&plugin_statistics, em);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_ADD);
pthread_mutex_unlock(&lock);
mount_collector(em);

View file

@ -3,8 +3,9 @@
#ifndef NETDATA_EBPF_MOUNT_H
#define NETDATA_EBPF_MOUNT_H 1
// Module name
// Module name & description
#define NETDATA_EBPF_MODULE_NAME_MOUNT "mount"
#define NETDATA_EBPF_MOUNT_MODULE_DESC "Show calls to syscalls mount(2) and umount(2)."
#define NETDATA_EBPF_MOUNT_SYSCALL 2

View file

@ -44,6 +44,71 @@ static netdata_publish_syscall_t oomkill_publish_aggregated = {.name = "oomkill"
.algorithm = "absolute",
.next = NULL};
static void ebpf_create_specific_oomkill_charts(char *type, int update_every);
/**
* Obsolete services
*
* Obsolete all service charts created
*
* @param em a pointer to `struct ebpf_module`
*/
static void ebpf_obsolete_oomkill_services(ebpf_module_t *em)
{
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_OOMKILL_CHART,
"OOM kills. This chart is provided by eBPF plugin.",
EBPF_COMMON_DIMENSION_KILLS,
NETDATA_EBPF_MEMORY_GROUP,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
20191,
em->update_every);
}
/**
* Obsolete cgroup chart
*
* Send obsolete for all charts created before to close.
*
* @param em a pointer to `struct ebpf_module`
*/
static inline void ebpf_obsolete_oomkill_cgroup_charts(ebpf_module_t *em)
{
pthread_mutex_lock(&mutex_cgroup_shm);
ebpf_obsolete_oomkill_services(em);
ebpf_cgroup_target_t *ect;
for (ect = ebpf_cgroup_pids; ect ; ect = ect->next) {
if (ect->systemd)
continue;
ebpf_create_specific_oomkill_charts(ect->name, em->update_every);
}
pthread_mutex_unlock(&mutex_cgroup_shm);
}
/**
* Obsolete global
*
* Obsolete global charts created by thread.
*
* @param em a pointer to `struct ebpf_module`
*/
static void ebpf_obsolete_oomkill_apps(ebpf_module_t *em)
{
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_OOMKILL_CHART,
"OOM kills",
EBPF_COMMON_DIMENSION_KILLS,
"mem",
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20020,
em->update_every);
}
/**
* Clean up the main thread.
*
@ -53,11 +118,30 @@ static void oomkill_cleanup(void *ptr)
{
ebpf_module_t *em = (ebpf_module_t *)ptr;
if (em->objects)
if (em->enabled == NETDATA_THREAD_EBPF_FUNCTION_RUNNING) {
pthread_mutex_lock(&lock);
if (em->cgroup_charts) {
ebpf_obsolete_oomkill_cgroup_charts(em);
}
ebpf_obsolete_oomkill_apps(em);
fflush(stdout);
pthread_mutex_unlock(&lock);
}
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_REMOVE);
if (em->objects) {
ebpf_unload_legacy_code(em->objects, em->probe_links);
em->objects = NULL;
em->probe_links = NULL;
}
pthread_mutex_lock(&ebpf_exit_cleanup);
em->enabled = NETDATA_THREAD_EBPF_STOPPED;
ebpf_update_stats(&plugin_statistics, em);
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
@ -293,6 +377,30 @@ static void ebpf_update_oomkill_cgroup(int32_t *keys, uint32_t total)
}
}
/**
* Update OOMkill period
*
* Update oomkill period according function arguments.
*
* @param running_time current value of running_value.
* @param em the thread main structure.
*
* @return It returns new running_time value.
*/
static int ebpf_update_oomkill_period(int running_time, ebpf_module_t *em)
{
pthread_mutex_lock(&ebpf_exit_cleanup);
if (running_time && !em->running_time)
running_time = em->update_every;
else
running_time += em->update_every;
em->running_time = running_time;
pthread_mutex_unlock(&ebpf_exit_cleanup);
return running_time;
}
/**
* Main loop for this collector.
*
@ -309,7 +417,9 @@ static void oomkill_collector(ebpf_module_t *em)
heartbeat_t hb;
heartbeat_init(&hb);
int counter = update_every - 1;
while (!ebpf_exit_plugin) {
uint32_t running_time = 0;
uint32_t lifetime = em->lifetime;
while (!ebpf_exit_plugin && running_time < lifetime) {
(void)heartbeat_next(&hb, USEC_PER_SEC);
if (ebpf_exit_plugin || ++counter != update_every)
continue;
@ -317,8 +427,10 @@ static void oomkill_collector(ebpf_module_t *em)
counter = 0;
uint32_t count = oomkill_read_data(keys);
if (!count)
if (!count) {
running_time = ebpf_update_oomkill_period(running_time, em);
continue;
}
pthread_mutex_lock(&collect_data_mutex);
pthread_mutex_lock(&lock);
@ -335,6 +447,8 @@ static void oomkill_collector(ebpf_module_t *em)
}
pthread_mutex_unlock(&lock);
pthread_mutex_unlock(&collect_data_mutex);
running_time = ebpf_update_oomkill_period(running_time, em);
}
}
@ -406,7 +520,7 @@ void *ebpf_oomkill_thread(void *ptr)
pthread_mutex_lock(&lock);
ebpf_update_stats(&plugin_statistics, em);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_ADD);
pthread_mutex_unlock(&lock);
oomkill_collector(em);

View file

@ -3,6 +3,9 @@
#ifndef NETDATA_EBPF_OOMKILL_H
#define NETDATA_EBPF_OOMKILL_H 1
// Module description
#define NETDATA_EBPF_OOMKILL_MODULE_DESC "Show OOM kills for all applications recognized via the apps.plugin."
/*****************************************************************
* copied from kernel-collectors repo, with modifications needed
* for inclusion here.

View file

@ -59,20 +59,15 @@ ebpf_process_stat_t *process_stat_vector = NULL;
static netdata_syscall_stat_t process_aggregated_data[NETDATA_KEY_PUBLISH_PROCESS_END];
static netdata_publish_syscall_t process_publish_aggregated[NETDATA_KEY_PUBLISH_PROCESS_END];
int process_enabled = 0;
bool publish_internal_metrics = true;
struct config process_config = { .first_section = NULL,
.last_section = NULL,
.mutex = NETDATA_MUTEX_INITIALIZER,
.index = { .avl_tree = { .root = NULL, .compar = appconfig_section_compare },
.rwlock = AVL_LOCK_INITIALIZER } };
static char *threads_stat[NETDATA_EBPF_THREAD_STAT_END] = {"total", "running"};
static char *load_event_stat[NETDATA_EBPF_LOAD_STAT_END] = {"legacy", "co-re"};
static char *memlock_stat = {"memory_locked"};
static char *hash_table_stat = {"hash_table"};
static char *hash_table_core[NETDATA_EBPF_LOAD_STAT_END] = {"per_core", "unique"};
#ifdef NETDATA_DEV_MODE
int process_disable_priority;
#endif
/*****************************************************************
*
@ -427,182 +422,8 @@ static void ebpf_create_global_charts(ebpf_module_t *em)
&process_publish_aggregated[NETDATA_KEY_PUBLISH_PROCESS_FORK],
2, em->update_every, NETDATA_EBPF_MODULE_NAME_PROCESS);
}
}
/**
* Create chart for Statistic Thread
*
* Write to standard output current values for threads.
*
* @param em a pointer to the structure with the default values.
*/
static inline void ebpf_create_statistic_thread_chart(ebpf_module_t *em)
{
ebpf_write_chart_cmd(NETDATA_MONITORING_FAMILY,
NETDATA_EBPF_THREADS,
"Threads info.",
"threads",
NETDATA_EBPF_FAMILY,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
140000,
em->update_every,
NETDATA_EBPF_MODULE_NAME_PROCESS);
ebpf_write_global_dimension(threads_stat[NETDATA_EBPF_THREAD_STAT_TOTAL],
threads_stat[NETDATA_EBPF_THREAD_STAT_TOTAL],
ebpf_algorithms[NETDATA_EBPF_ABSOLUTE_IDX]);
ebpf_write_global_dimension(threads_stat[NETDATA_EBPF_THREAD_STAT_RUNNING],
threads_stat[NETDATA_EBPF_THREAD_STAT_RUNNING],
ebpf_algorithms[NETDATA_EBPF_ABSOLUTE_IDX]);
}
/**
* Create chart for Load Thread
*
* Write to standard output current values for load mode.
*
* @param em a pointer to the structure with the default values.
*/
static inline void ebpf_create_statistic_load_chart(ebpf_module_t *em)
{
ebpf_write_chart_cmd(NETDATA_MONITORING_FAMILY,
NETDATA_EBPF_LOAD_METHOD,
"Load info.",
"methods",
NETDATA_EBPF_FAMILY,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
140001,
em->update_every,
NETDATA_EBPF_MODULE_NAME_PROCESS);
ebpf_write_global_dimension(load_event_stat[NETDATA_EBPF_LOAD_STAT_LEGACY],
load_event_stat[NETDATA_EBPF_LOAD_STAT_LEGACY],
ebpf_algorithms[NETDATA_EBPF_ABSOLUTE_IDX]);
ebpf_write_global_dimension(load_event_stat[NETDATA_EBPF_LOAD_STAT_CORE],
load_event_stat[NETDATA_EBPF_LOAD_STAT_CORE],
ebpf_algorithms[NETDATA_EBPF_ABSOLUTE_IDX]);
}
/**
* Create chart for Kernel Memory
*
* Write to standard output current values for allocated memory.
*
* @param em a pointer to the structure with the default values.
*/
static inline void ebpf_create_statistic_kernel_memory(ebpf_module_t *em)
{
ebpf_write_chart_cmd(NETDATA_MONITORING_FAMILY,
NETDATA_EBPF_KERNEL_MEMORY,
"Memory allocated for hash tables.",
"bytes",
NETDATA_EBPF_FAMILY,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
140002,
em->update_every,
NETDATA_EBPF_MODULE_NAME_PROCESS);
ebpf_write_global_dimension(memlock_stat,
memlock_stat,
ebpf_algorithms[NETDATA_EBPF_ABSOLUTE_IDX]);
}
/**
* Create chart Hash Table
*
* Write to standard output number of hash tables used with this software.
*
* @param em a pointer to the structure with the default values.
*/
static inline void ebpf_create_statistic_hash_tables(ebpf_module_t *em)
{
ebpf_write_chart_cmd(NETDATA_MONITORING_FAMILY,
NETDATA_EBPF_HASH_TABLES_LOADED,
"Number of hash tables loaded.",
"hash tables",
NETDATA_EBPF_FAMILY,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
140003,
em->update_every,
NETDATA_EBPF_MODULE_NAME_PROCESS);
ebpf_write_global_dimension(hash_table_stat,
hash_table_stat,
ebpf_algorithms[NETDATA_EBPF_ABSOLUTE_IDX]);
}
/**
* Create chart for percpu stats
*
* Write to standard output current values for threads.
*
* @param em a pointer to the structure with the default values.
*/
static inline void ebpf_create_statistic_hash_per_core(ebpf_module_t *em)
{
ebpf_write_chart_cmd(NETDATA_MONITORING_FAMILY,
NETDATA_EBPF_HASH_TABLES_PER_CORE,
"How threads are loading hash/array tables.",
"threads",
NETDATA_EBPF_FAMILY,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
140004,
em->update_every,
NETDATA_EBPF_MODULE_NAME_PROCESS);
ebpf_write_global_dimension(hash_table_core[NETDATA_EBPF_THREAD_PER_CORE],
hash_table_core[NETDATA_EBPF_THREAD_PER_CORE],
ebpf_algorithms[NETDATA_EBPF_ABSOLUTE_IDX]);
ebpf_write_global_dimension(hash_table_core[NETDATA_EBPF_THREAD_UNIQUE],
hash_table_core[NETDATA_EBPF_THREAD_UNIQUE],
ebpf_algorithms[NETDATA_EBPF_ABSOLUTE_IDX]);
}
/**
* Update Internal Metric variable
*
* By default eBPF.plugin sends internal metrics for netdata, but user can
* disable this.
*
* The function updates the variable used to send charts.
*/
static void update_internal_metric_variable()
{
const char *s = getenv("NETDATA_INTERNALS_MONITORING");
if (s && *s && strcmp(s, "NO") == 0)
publish_internal_metrics = false;
}
/**
* Create Statistics Charts
*
* Create charts that will show statistics related to eBPF plugin.
*
* @param em a pointer to the structure with the default values.
*/
static void ebpf_create_statistic_charts(ebpf_module_t *em)
{
update_internal_metric_variable();
if (!publish_internal_metrics)
return;
ebpf_create_statistic_thread_chart(em);
ebpf_create_statistic_load_chart(em);
ebpf_create_statistic_kernel_memory(em);
ebpf_create_statistic_hash_tables(em);
ebpf_create_statistic_hash_per_core(em);
fflush(stdout);
}
/**
@ -673,6 +494,206 @@ void ebpf_process_create_apps_charts(struct ebpf_module *em, void *ptr)
*
*****************************************************************/
static void ebpf_obsolete_specific_process_charts(char *type, ebpf_module_t *em);
/**
* Obsolete services
*
* Obsolete all service charts created
*
* @param em a pointer to `struct ebpf_module`
*/
static void ebpf_obsolete_process_services(ebpf_module_t *em)
{
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SYSCALL_APPS_TASK_PROCESS,
"Process started",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_APPS_PROCESS_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20065,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SYSCALL_APPS_TASK_THREAD,
"Threads started",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_APPS_PROCESS_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20066,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SYSCALL_APPS_TASK_CLOSE,
"Tasks starts exit process.",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_APPS_PROCESS_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20067,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SYSCALL_APPS_TASK_EXIT,
"Tasks closed",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_APPS_PROCESS_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20068,
em->update_every);
if (em->mode < MODE_ENTRY) {
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SYSCALL_APPS_TASK_ERROR,
"Errors to create process or threads.",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_APPS_PROCESS_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20069,
em->update_every);
}
}
/**
* Obsolete cgroup chart
*
* Send obsolete for all charts created before to close.
*
* @param em a pointer to `struct ebpf_module`
*/
static inline void ebpf_obsolete_process_cgroup_charts(ebpf_module_t *em) {
pthread_mutex_lock(&mutex_cgroup_shm);
ebpf_obsolete_process_services(em);
ebpf_cgroup_target_t *ect;
for (ect = ebpf_cgroup_pids; ect ; ect = ect->next) {
if (ect->systemd)
continue;
ebpf_obsolete_specific_process_charts(ect->name, em);
}
pthread_mutex_unlock(&mutex_cgroup_shm);
}
/**
* Obsolette apps charts
*
* Obsolete apps charts.
*
* @param em a pointer to the structure with the default values.
*/
void ebpf_obsolete_process_apps_charts(struct ebpf_module *em)
{
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SYSCALL_APPS_TASK_PROCESS,
"Process started",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_PROCESS_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20065,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SYSCALL_APPS_TASK_THREAD,
"Threads started",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_PROCESS_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20066,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SYSCALL_APPS_TASK_EXIT,
"Tasks starts exit process.",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_PROCESS_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20067,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SYSCALL_APPS_TASK_CLOSE,
"Tasks closed",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_PROCESS_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20068,
em->update_every);
if (em->mode < MODE_ENTRY) {
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SYSCALL_APPS_TASK_ERROR,
"Errors to create process or threads.",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_PROCESS_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20069,
em->update_every);
}
}
/**
* Obsolete global
*
* Obsolete global charts created by thread.
*
* @param em a pointer to `struct ebpf_module`
*/
static void ebpf_obsolete_process_global(ebpf_module_t *em)
{
ebpf_write_chart_obsolete(NETDATA_EBPF_SYSTEM_GROUP,
NETDATA_PROCESS_SYSCALL,
"Start process",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_PROCESS_GROUP,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
21002,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_EBPF_SYSTEM_GROUP,
NETDATA_EXIT_SYSCALL,
"Exit process",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_PROCESS_GROUP,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
21003,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_EBPF_SYSTEM_GROUP,
NETDATA_PROCESS_STATUS_NAME,
"Process not closed",
EBPF_COMMON_DIMENSION_DIFFERENCE,
NETDATA_PROCESS_GROUP,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
21004,
em->update_every);
if (em->mode < MODE_ENTRY) {
ebpf_write_chart_obsolete(NETDATA_EBPF_SYSTEM_GROUP,
NETDATA_PROCESS_ERROR_NAME,
"Fails to create process",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_PROCESS_GROUP,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
21005,
em->update_every);
}
}
/**
* Process disable tracepoints
*
@ -708,6 +729,37 @@ static void ebpf_process_exit(void *ptr)
{
ebpf_module_t *em = (ebpf_module_t *)ptr;
if (em->enabled == NETDATA_THREAD_EBPF_FUNCTION_RUNNING) {
pthread_mutex_lock(&lock);
if (em->cgroup_charts) {
ebpf_obsolete_process_cgroup_charts(em);
fflush(stdout);
}
if (em->apps_charts & NETDATA_EBPF_APPS_FLAG_CHART_CREATED) {
ebpf_obsolete_process_apps_charts(em);
}
ebpf_obsolete_process_global(em);
#ifdef NETDATA_DEV_MODE
if (ebpf_aral_process_stat)
ebpf_statistic_obsolete_aral_chart(em, process_disable_priority);
#endif
fflush(stdout);
pthread_mutex_unlock(&lock);
}
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_REMOVE);
if (em->objects) {
ebpf_unload_legacy_code(em->objects, em->probe_links);
em->objects = NULL;
em->probe_links = NULL;
}
freez(process_hash_values);
freez(process_stat_vector);
@ -716,6 +768,7 @@ static void ebpf_process_exit(void *ptr)
pthread_mutex_lock(&ebpf_exit_cleanup);
process_pid_fd = -1;
em->enabled = NETDATA_THREAD_EBPF_STOPPED;
ebpf_update_stats(&plugin_statistics, em);
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
@ -740,14 +793,14 @@ static void ebpf_process_sum_cgroup_pids(ebpf_process_stat_t *ps, struct pid_on_
memset(&accumulator, 0, sizeof(accumulator));
while (pids) {
ebpf_process_stat_t *ps = &pids->ps;
ebpf_process_stat_t *pps = &pids->ps;
accumulator.exit_call += ps->exit_call;
accumulator.release_call += ps->release_call;
accumulator.create_process += ps->create_process;
accumulator.create_thread += ps->create_thread;
accumulator.exit_call += pps->exit_call;
accumulator.release_call += pps->release_call;
accumulator.create_process += pps->create_process;
accumulator.create_thread += pps->create_thread;
accumulator.task_err += ps->task_err;
accumulator.task_err += pps->task_err;
pids = pids->next;
}
@ -1046,40 +1099,6 @@ void ebpf_process_update_cgroup_algorithm()
}
}
/**
* Send Statistic Data
*
* Send statistic information to netdata.
*/
void ebpf_send_statistic_data()
{
if (!publish_internal_metrics)
return;
write_begin_chart(NETDATA_MONITORING_FAMILY, NETDATA_EBPF_THREADS);
write_chart_dimension(threads_stat[NETDATA_EBPF_THREAD_STAT_TOTAL], (long long)plugin_statistics.threads);
write_chart_dimension(threads_stat[NETDATA_EBPF_THREAD_STAT_RUNNING], (long long)plugin_statistics.running);
write_end_chart();
write_begin_chart(NETDATA_MONITORING_FAMILY, NETDATA_EBPF_LOAD_METHOD);
write_chart_dimension(load_event_stat[NETDATA_EBPF_LOAD_STAT_LEGACY], (long long)plugin_statistics.legacy);
write_chart_dimension(load_event_stat[NETDATA_EBPF_LOAD_STAT_CORE], (long long)plugin_statistics.core);
write_end_chart();
write_begin_chart(NETDATA_MONITORING_FAMILY, NETDATA_EBPF_KERNEL_MEMORY);
write_chart_dimension(memlock_stat, (long long)plugin_statistics.memlock_kern);
write_end_chart();
write_begin_chart(NETDATA_MONITORING_FAMILY, NETDATA_EBPF_HASH_TABLES_LOADED);
write_chart_dimension(hash_table_stat, (long long)plugin_statistics.hash_tables);
write_end_chart();
write_begin_chart(NETDATA_MONITORING_FAMILY, NETDATA_EBPF_HASH_TABLES_PER_CORE);
write_chart_dimension(hash_table_core[NETDATA_EBPF_THREAD_PER_CORE], (long long)plugin_statistics.hash_percpu);
write_chart_dimension(hash_table_core[NETDATA_EBPF_THREAD_UNIQUE], (long long)plugin_statistics.hash_unique);
write_end_chart();
}
/**
* Main loop for this collector.
*
@ -1092,7 +1111,6 @@ static void process_collector(ebpf_module_t *em)
int publish_global = em->global_charts;
int cgroups = em->cgroup_charts;
pthread_mutex_lock(&ebpf_exit_cleanup);
int thread_enabled = em->enabled;
process_pid_fd = process_maps[NETDATA_PROCESS_PID_TABLE].map_fd;
pthread_mutex_unlock(&ebpf_exit_cleanup);
if (cgroups)
@ -1101,7 +1119,9 @@ static void process_collector(ebpf_module_t *em)
int update_every = em->update_every;
int counter = update_every - 1;
int maps_per_core = em->maps_per_core;
while (!ebpf_exit_plugin) {
uint32_t running_time = 0;
uint32_t lifetime = em->lifetime;
while (!ebpf_exit_plugin && running_time < lifetime) {
usec_t dt = heartbeat_next(&hb, USEC_PER_SEC);
(void)dt;
if (ebpf_exit_plugin)
@ -1122,28 +1142,35 @@ static void process_collector(ebpf_module_t *em)
}
pthread_mutex_lock(&lock);
ebpf_send_statistic_data();
if (thread_enabled == NETDATA_THREAD_EBPF_RUNNING) {
if (publish_global) {
ebpf_process_send_data(em);
}
if (publish_global) {
ebpf_process_send_data(em);
}
if (apps_enabled & NETDATA_EBPF_APPS_FLAG_CHART_CREATED) {
ebpf_process_send_apps_data(apps_groups_root_target, em);
}
if (apps_enabled & NETDATA_EBPF_APPS_FLAG_CHART_CREATED) {
ebpf_process_send_apps_data(apps_groups_root_target, em);
}
#ifdef NETDATA_DEV_MODE
if (ebpf_aral_process_stat)
ebpf_send_data_aral_chart(ebpf_aral_process_stat, em);
if (ebpf_aral_process_stat)
ebpf_send_data_aral_chart(ebpf_aral_process_stat, em);
#endif
if (cgroups && shm_ebpf_cgroup.header) {
ebpf_process_send_cgroup_data(em);
}
if (cgroups && shm_ebpf_cgroup.header) {
ebpf_process_send_cgroup_data(em);
}
pthread_mutex_unlock(&lock);
pthread_mutex_unlock(&collect_data_mutex);
pthread_mutex_lock(&ebpf_exit_cleanup);
if (running_time && !em->running_time)
running_time = update_every;
else
running_time += update_every;
em->running_time = running_time;
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
fflush(stdout);
@ -1254,7 +1281,6 @@ void *ebpf_process_thread(void *ptr)
if (ebpf_process_enable_tracepoints()) {
em->enabled = em->global_charts = em->apps_charts = em->cgroup_charts = NETDATA_THREAD_EBPF_STOPPING;
}
process_enabled = em->enabled;
pthread_mutex_unlock(&ebpf_exit_cleanup);
pthread_mutex_lock(&lock);
@ -1276,27 +1302,22 @@ void *ebpf_process_thread(void *ptr)
process_aggregated_data, process_publish_aggregated, process_dimension_names, process_id_names,
algorithms, NETDATA_KEY_PUBLISH_PROCESS_END);
if (process_enabled == NETDATA_THREAD_EBPF_RUNNING) {
ebpf_create_global_charts(em);
}
ebpf_create_global_charts(em);
ebpf_update_stats(&plugin_statistics, em);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_ADD);
#ifdef NETDATA_DEV_MODE
if (ebpf_aral_process_stat)
ebpf_statistic_create_aral_chart(NETDATA_EBPF_PROC_ARAL_NAME, em);
process_disable_priority = ebpf_statistic_create_aral_chart(NETDATA_EBPF_PROC_ARAL_NAME, em);
#endif
ebpf_create_statistic_charts(em);
pthread_mutex_unlock(&lock);
process_collector(em);
pthread_mutex_lock(&ebpf_exit_cleanup);
if (em->enabled == NETDATA_THREAD_EBPF_RUNNING)
ebpf_update_disabled_plugin_stats(em);
ebpf_update_disabled_plugin_stats(em);
pthread_mutex_unlock(&ebpf_exit_cleanup);
netdata_thread_cleanup_pop(1);

View file

@ -3,8 +3,9 @@
#ifndef NETDATA_EBPF_PROCESS_H
#define NETDATA_EBPF_PROCESS_H 1
// Module name
// Module name & description
#define NETDATA_EBPF_MODULE_NAME_PROCESS "process"
#define NETDATA_EBPF_MODULE_PROCESS_DESC "Monitor information about process life. This thread is integrated with apps and cgroup."
// Groups used on Dashboard
#define NETDATA_PROCESS_GROUP "processes"
@ -41,12 +42,13 @@
#define NETDATA_EBPF_CGROUP_UPDATE 30
// Statistical information
enum netdata_ebpf_thread_stats{
NETDATA_EBPF_THREAD_STAT_TOTAL,
NETDATA_EBPF_THREAD_STAT_RUNNING,
NETDATA_EBPF_THREAD_STAT_END
enum netdata_ebpf_stats_order {
NETDATA_EBPF_ORDER_STAT_THREADS = 140000,
NETDATA_EBPF_ORDER_STAT_LIFE_TIME,
NETDATA_EBPF_ORDER_STAT_LOAD_METHOD,
NETDATA_EBPF_ORDER_STAT_KERNEL_MEMORY,
NETDATA_EBPF_ORDER_STAT_HASH_TABLES,
NETDATA_EBPF_ORDER_STAT_HASH_CORE
};
enum netdata_ebpf_load_mode_stats{

View file

@ -50,6 +50,10 @@ netdata_ebpf_targets_t shm_targets[] = { {.name = "shmget", .mode = EBPF_LOAD_TR
{.name = "shmctl", .mode = EBPF_LOAD_TRAMPOLINE},
{.name = NULL, .mode = EBPF_LOAD_TRAMPOLINE}};
#ifdef NETDATA_DEV_MODE
int shm_disable_priority;
#endif
#ifdef LIBBPF_MAJOR_VERSION
/*****************************************************************
*
@ -288,6 +292,150 @@ static inline int ebpf_shm_load_and_attach(struct shm_bpf *obj, ebpf_module_t *e
* FUNCTIONS TO CLOSE THE THREAD
*****************************************************************/
static void ebpf_obsolete_specific_shm_charts(char *type, int update_every);
/**
* Obsolete services
*
* Obsolete all service charts created
*
* @param em a pointer to `struct ebpf_module`
*/
static void ebpf_obsolete_shm_services(ebpf_module_t *em)
{
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SHMGET_CHART,
"Calls to syscall <code>shmget(2)</code>.",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_APPS_IPC_SHM_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20191,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SHMAT_CHART,
"Calls to syscall <code>shmat(2)</code>.",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_APPS_IPC_SHM_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20192,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SHMDT_CHART,
"Calls to syscall <code>shmdt(2)</code>.",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_APPS_IPC_SHM_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20193,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SHMCTL_CHART,
"Calls to syscall <code>shmctl(2)</code>.",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_APPS_IPC_SHM_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20193,
em->update_every);
}
/**
* Obsolete cgroup chart
*
* Send obsolete for all charts created before to close.
*
* @param em a pointer to `struct ebpf_module`
*/
static inline void ebpf_obsolete_shm_cgroup_charts(ebpf_module_t *em) {
pthread_mutex_lock(&mutex_cgroup_shm);
ebpf_obsolete_shm_services(em);
ebpf_cgroup_target_t *ect;
for (ect = ebpf_cgroup_pids; ect ; ect = ect->next) {
if (ect->systemd)
continue;
ebpf_obsolete_specific_shm_charts(ect->name, em->update_every);
}
pthread_mutex_unlock(&mutex_cgroup_shm);
}
/**
* Obsolette apps charts
*
* Obsolete apps charts.
*
* @param em a pointer to the structure with the default values.
*/
void ebpf_obsolete_shm_apps_charts(struct ebpf_module *em)
{
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SHMGET_CHART,
"Calls to syscall <code>shmget(2)</code>.",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_APPS_IPC_SHM_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20191,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SHMAT_CHART,
"Calls to syscall <code>shmat(2)</code>.",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_APPS_IPC_SHM_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20192,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SHMDT_CHART,
"Calls to syscall <code>shmdt(2)</code>.",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_APPS_IPC_SHM_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20193,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SHMCTL_CHART,
"Calls to syscall <code>shmctl(2)</code>.",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_APPS_IPC_SHM_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20194,
em->update_every);
}
/**
* Obsolete global
*
* Obsolete global charts created by thread.
*
* @param em a pointer to `struct ebpf_module`
*/
static void ebpf_obsolete_shm_global(ebpf_module_t *em)
{
ebpf_write_chart_obsolete(NETDATA_EBPF_SYSTEM_GROUP,
NETDATA_SHM_GLOBAL_CHART,
"Calls to shared memory system calls",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_SYSTEM_IPC_SHM_SUBMENU,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
NETDATA_CHART_PRIO_SYSTEM_IPC_SHARED_MEM_CALLS,
em->update_every);
}
/**
* SHM Exit
*
@ -299,16 +447,46 @@ static void ebpf_shm_exit(void *ptr)
{
ebpf_module_t *em = (ebpf_module_t *)ptr;
#ifdef LIBBPF_MAJOR_VERSION
if (shm_bpf_obj)
shm_bpf__destroy(shm_bpf_obj);
if (em->enabled == NETDATA_THREAD_EBPF_FUNCTION_RUNNING) {
pthread_mutex_lock(&lock);
if (em->cgroup_charts) {
ebpf_obsolete_shm_cgroup_charts(em);
fflush(stdout);
}
if (em->apps_charts & NETDATA_EBPF_APPS_FLAG_CHART_CREATED) {
ebpf_obsolete_shm_apps_charts(em);
}
ebpf_obsolete_shm_global(em);
#ifdef NETDATA_DEV_MODE
if (ebpf_aral_shm_pid)
ebpf_statistic_obsolete_aral_chart(em, shm_disable_priority);
#endif
if (em->objects)
fflush(stdout);
pthread_mutex_unlock(&lock);
}
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_REMOVE);
#ifdef LIBBPF_MAJOR_VERSION
if (shm_bpf_obj) {
shm_bpf__destroy(shm_bpf_obj);
shm_bpf_obj = NULL;
}
#endif
if (em->objects) {
ebpf_unload_legacy_code(em->objects, em->probe_links);
em->objects = NULL;
em->probe_links = NULL;
}
pthread_mutex_lock(&ebpf_exit_cleanup);
em->enabled = NETDATA_THREAD_EBPF_STOPPED;
ebpf_update_stats(&plugin_statistics, em);
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
@ -859,7 +1037,9 @@ static void shm_collector(ebpf_module_t *em)
heartbeat_init(&hb);
int counter = update_every - 1;
int maps_per_core = em->maps_per_core;
while (!ebpf_exit_plugin) {
uint32_t running_time = 0;
uint32_t lifetime = em->lifetime;
while (!ebpf_exit_plugin && running_time < lifetime) {
(void)heartbeat_next(&hb, USEC_PER_SEC);
if (ebpf_exit_plugin || ++counter != update_every)
continue;
@ -895,6 +1075,15 @@ static void shm_collector(ebpf_module_t *em)
pthread_mutex_unlock(&lock);
pthread_mutex_unlock(&collect_data_mutex);
pthread_mutex_lock(&ebpf_exit_cleanup);
if (running_time && !em->running_time)
running_time = update_every;
else
running_time += update_every;
em->running_time = running_time;
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
}
@ -1084,10 +1273,10 @@ void *ebpf_shm_thread(void *ptr)
pthread_mutex_lock(&lock);
ebpf_create_shm_charts(em->update_every);
ebpf_update_stats(&plugin_statistics, em);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_ADD);
#ifdef NETDATA_DEV_MODE
if (ebpf_aral_shm_pid)
ebpf_statistic_create_aral_chart(NETDATA_EBPF_SHM_ARAL_NAME, em);
shm_disable_priority = ebpf_statistic_create_aral_chart(NETDATA_EBPF_SHM_ARAL_NAME, em);
#endif
pthread_mutex_unlock(&lock);

View file

@ -3,8 +3,9 @@
#ifndef NETDATA_EBPF_SHM_H
#define NETDATA_EBPF_SHM_H 1
// Module name
// Module name & description
#define NETDATA_EBPF_MODULE_NAME_SHM "shm"
#define NETDATA_EBPF_SHM_MODULE_DESC "Show calls to syscalls shmget(2), shmat(2), shmdt(2) and shmctl(2). This thread is integrated with apps and cgroup."
// charts
#define NETDATA_SHM_GLOBAL_CHART "shared_memory_calls"

View file

@ -130,6 +130,10 @@ struct netdata_static_thread socket_threads = {
.start_routine = NULL
};
#ifdef NETDATA_DEV_MODE
int socket_disable_priority;
#endif
#ifdef LIBBPF_MAJOR_VERSION
/**
* Disable Probe
@ -646,6 +650,8 @@ static void ebpf_socket_free(ebpf_module_t *em )
pthread_mutex_lock(&ebpf_exit_cleanup);
em->enabled = NETDATA_THREAD_EBPF_STOPPED;
ebpf_update_stats(&plugin_statistics, em);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_REMOVE);
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
@ -1217,6 +1223,8 @@ static void ebpf_create_global_charts(ebpf_module_t *em)
&socket_publish_aggregated[NETDATA_IDX_UDP_RECVBUF],
2, em->update_every, NETDATA_EBPF_MODULE_NAME_SOCKET);
}
fflush(stdout);
}
/**
@ -2177,7 +2185,9 @@ void *ebpf_socket_read_hash(void *ptr)
int fd_ipv6 = socket_maps[NETDATA_SOCKET_TABLE_IPV6].map_fd;
int maps_per_core = em->maps_per_core;
// This thread is cancelled from another thread
for (;;) {
uint32_t running_time;
uint32_t lifetime = em->lifetime;
for (running_time = 0;!ebpf_exit_plugin && running_time < lifetime; running_time++) {
(void)heartbeat_next(&hb, USEC_PER_SEC);
if (ebpf_exit_plugin)
break;
@ -2918,7 +2928,9 @@ static void socket_collector(ebpf_module_t *em)
int update_every = em->update_every;
int maps_per_core = em->maps_per_core;
int counter = update_every - 1;
while (!ebpf_exit_plugin) {
uint32_t running_time = 0;
uint32_t lifetime = em->lifetime;
while (!ebpf_exit_plugin && running_time < lifetime) {
(void)heartbeat_next(&hb, USEC_PER_SEC);
if (ebpf_exit_plugin || ++counter != update_every)
continue;
@ -2973,6 +2985,15 @@ static void socket_collector(ebpf_module_t *em)
}
pthread_mutex_unlock(&lock);
pthread_mutex_unlock(&collect_data_mutex);
pthread_mutex_lock(&ebpf_exit_cleanup);
if (running_time && !em->running_time)
running_time = update_every;
else
running_time += update_every;
em->running_time = running_time;
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
}
@ -4015,11 +4036,11 @@ void *ebpf_socket_thread(void *ptr)
ebpf_create_global_charts(em);
ebpf_update_stats(&plugin_statistics, em);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_ADD);
#ifdef NETDATA_DEV_MODE
if (ebpf_aral_socket_pid)
ebpf_statistic_create_aral_chart(NETDATA_EBPF_SOCKET_ARAL_NAME, em);
socket_disable_priority = ebpf_statistic_create_aral_chart(NETDATA_EBPF_SOCKET_ARAL_NAME, em);
#endif
pthread_mutex_unlock(&lock);

View file

@ -4,8 +4,9 @@
#include <stdint.h>
#include "libnetdata/avl/avl.h"
// Module name
// Module name & description
#define NETDATA_EBPF_MODULE_NAME_SOCKET "socket"
#define NETDATA_EBPF_SOCKET_MODULE_DESC "Monitors TCP and UDP bandwidth. This thread is integrated with apps and cgroup."
// Vector indexes
#define NETDATA_UDP_START 3

View file

@ -60,6 +60,26 @@ static softirq_val_t softirq_vals[] = {
// tmp store for soft IRQ values we get from a per-CPU eBPF map.
static softirq_ebpf_val_t *softirq_ebpf_vals = NULL;
/**
* Obsolete global
*
* Obsolete global charts created by thread.
*
* @param em a pointer to `struct ebpf_module`
*/
static void ebpf_obsolete_softirq_global(ebpf_module_t *em)
{
ebpf_write_chart_obsolete(NETDATA_EBPF_SYSTEM_GROUP,
"softirq_latency",
"Software IRQ latency",
EBPF_COMMON_DIMENSION_MILLISECONDS,
"softirqs",
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
NETDATA_CHART_PRIO_SYSTEM_SOFTIRQS+1,
em->update_every);
}
/**
* Cleanup
*
@ -71,16 +91,32 @@ static void softirq_cleanup(void *ptr)
{
ebpf_module_t *em = (ebpf_module_t *)ptr;
if (em->objects)
if (em->enabled == NETDATA_THREAD_EBPF_FUNCTION_RUNNING) {
pthread_mutex_lock(&lock);
ebpf_obsolete_softirq_global(em);
pthread_mutex_unlock(&lock);
fflush(stdout);
}
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_REMOVE);
if (em->objects) {
ebpf_unload_legacy_code(em->objects, em->probe_links);
em->objects = NULL;
em->probe_links = NULL;
}
for (int i = 0; softirq_tracepoints[i].class != NULL; i++) {
ebpf_disable_tracepoint(&softirq_tracepoints[i]);
}
freez(softirq_ebpf_vals);
softirq_ebpf_vals = NULL;
pthread_mutex_lock(&ebpf_exit_cleanup);
em->enabled = NETDATA_THREAD_EBPF_STOPPED;
ebpf_update_stats(&plugin_statistics, em);
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
@ -170,7 +206,7 @@ static void softirq_collector(ebpf_module_t *em)
softirq_create_charts(em->update_every);
softirq_create_dims();
ebpf_update_stats(&plugin_statistics, em);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_ADD);
pthread_mutex_unlock(&lock);
// loop and read from published data until ebpf plugin is closed.
@ -180,7 +216,9 @@ static void softirq_collector(ebpf_module_t *em)
int counter = update_every - 1;
int maps_per_core = em->maps_per_core;
//This will be cancelled by its parent
while (!ebpf_exit_plugin) {
uint32_t running_time = 0;
uint32_t lifetime = em->lifetime;
while (!ebpf_exit_plugin && running_time < lifetime) {
(void)heartbeat_next(&hb, USEC_PER_SEC);
if (ebpf_exit_plugin || ++counter != update_every)
continue;
@ -195,6 +233,15 @@ static void softirq_collector(ebpf_module_t *em)
write_end_chart();
pthread_mutex_unlock(&lock);
pthread_mutex_lock(&ebpf_exit_cleanup);
if (running_time && !em->running_time)
running_time = update_every;
else
running_time += update_every;
em->running_time = running_time;
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
}

View file

@ -3,6 +3,9 @@
#ifndef NETDATA_EBPF_SOFTIRQ_H
#define NETDATA_EBPF_SOFTIRQ_H 1
// Module observation
#define NETDATA_EBPF_SOFTIRQ_MODULE_DESC "Show time spent servicing individual software interrupt requests (soft IRQs)."
/*****************************************************************
* copied from kernel-collectors repo, with modifications needed
* for inclusion here.

View file

@ -229,6 +229,109 @@ static inline int ebpf_swap_load_and_attach(struct swap_bpf *obj, ebpf_module_t
*
*****************************************************************/
static void ebpf_obsolete_specific_swap_charts(char *type, int update_every);
/**
* Obsolete services
*
* Obsolete all service charts created
*
* @param em a pointer to `struct ebpf_module`
*/
static void ebpf_obsolete_swap_services(ebpf_module_t *em)
{
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_MEM_SWAP_READ_CHART,
"Calls to function <code>swap_readpage</code>.",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_SYSTEM_CGROUP_SWAP_SUBMENU,
NETDATA_EBPF_CHART_TYPE_LINE,
NETDATA_CGROUP_SWAP_READ_CONTEXT,
NETDATA_CHART_PRIO_CGROUPS_CONTAINERS + 5100,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_MEM_SWAP_WRITE_CHART,
"Calls to function <code>swap_writepage</code>.",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_SYSTEM_CGROUP_SWAP_SUBMENU,
NETDATA_EBPF_CHART_TYPE_LINE,
NETDATA_CGROUP_SWAP_WRITE_CONTEXT,
NETDATA_CHART_PRIO_CGROUPS_CONTAINERS + 5101,
em->update_every);
}
/**
* Obsolete cgroup chart
*
* Send obsolete for all charts created before to close.
*
* @param em a pointer to `struct ebpf_module`
*/
static inline void ebpf_obsolete_swap_cgroup_charts(ebpf_module_t *em) {
pthread_mutex_lock(&mutex_cgroup_shm);
ebpf_obsolete_swap_services(em);
ebpf_cgroup_target_t *ect;
for (ect = ebpf_cgroup_pids; ect ; ect = ect->next) {
if (ect->systemd)
continue;
ebpf_obsolete_specific_swap_charts(ect->name, em->update_every);
}
pthread_mutex_unlock(&mutex_cgroup_shm);
}
/**
* Obsolette apps charts
*
* Obsolete apps charts.
*
* @param em a pointer to the structure with the default values.
*/
void ebpf_obsolete_swap_apps_charts(struct ebpf_module *em)
{
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_MEM_SWAP_READ_CHART,
"Calls to function <code>swap_readpage</code>.",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_SWAP_SUBMENU,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20191,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_MEM_SWAP_WRITE_CHART,
"Calls to function <code>swap_writepage</code>.",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_SWAP_SUBMENU,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20192,
em->update_every);
}
/**
* Obsolete global
*
* Obsolete global charts created by thread.
*
* @param em a pointer to `struct ebpf_module`
*/
static void ebpf_obsolete_swap_global(ebpf_module_t *em)
{
ebpf_write_chart_obsolete(NETDATA_EBPF_SYSTEM_GROUP,
NETDATA_MEM_SWAP_CHART,
"Calls to access swap memory",
EBPF_COMMON_DIMENSION_CALL, NETDATA_SYSTEM_SWAP_SUBMENU,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
NETDATA_CHART_PRIO_SYSTEM_SWAP_CALLS,
em->update_every);
}
/**
* Swap exit
*
@ -240,15 +343,40 @@ static void ebpf_swap_exit(void *ptr)
{
ebpf_module_t *em = (ebpf_module_t *)ptr;
if (em->enabled == NETDATA_THREAD_EBPF_FUNCTION_RUNNING) {
pthread_mutex_lock(&lock);
if (em->cgroup_charts) {
ebpf_obsolete_swap_cgroup_charts(em);
fflush(stdout);
}
if (em->apps_charts & NETDATA_EBPF_APPS_FLAG_CHART_CREATED) {
ebpf_obsolete_swap_apps_charts(em);
}
ebpf_obsolete_swap_global(em);
fflush(stdout);
pthread_mutex_unlock(&lock);
}
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_REMOVE);
#ifdef LIBBPF_MAJOR_VERSION
if (bpf_obj)
if (bpf_obj) {
swap_bpf__destroy(bpf_obj);
bpf_obj = NULL;
}
#endif
if (em->objects)
if (em->objects) {
ebpf_unload_legacy_code(em->objects, em->probe_links);
em->objects = NULL;
em->probe_links = NULL;
}
pthread_mutex_lock(&ebpf_exit_cleanup);
em->enabled = NETDATA_THREAD_EBPF_STOPPED;
ebpf_update_stats(&plugin_statistics, em);
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
@ -674,7 +802,9 @@ static void swap_collector(ebpf_module_t *em)
heartbeat_init(&hb);
int counter = update_every - 1;
int maps_per_core = em->maps_per_core;
while (!ebpf_exit_plugin) {
uint32_t running_time = 0;
uint32_t lifetime = em->lifetime;
while (!ebpf_exit_plugin && running_time < lifetime) {
(void)heartbeat_next(&hb, USEC_PER_SEC);
if (ebpf_exit_plugin || ++counter != update_every)
continue;
@ -701,6 +831,15 @@ static void swap_collector(ebpf_module_t *em)
pthread_mutex_unlock(&lock);
pthread_mutex_unlock(&collect_data_mutex);
pthread_mutex_lock(&ebpf_exit_cleanup);
if (running_time && !em->running_time)
running_time = update_every;
else
running_time += update_every;
em->running_time = running_time;
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
}
@ -784,6 +923,8 @@ static void ebpf_create_swap_charts(int update_every)
ebpf_create_global_dimension,
swap_publish_aggregated, NETDATA_SWAP_END,
update_every, NETDATA_EBPF_MODULE_NAME_SWAP);
fflush(stdout);
}
/*
@ -857,7 +998,7 @@ void *ebpf_swap_thread(void *ptr)
pthread_mutex_lock(&lock);
ebpf_create_swap_charts(em->update_every);
ebpf_update_stats(&plugin_statistics, em);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_ADD);
pthread_mutex_unlock(&lock);
swap_collector(em);

View file

@ -3,8 +3,9 @@
#ifndef NETDATA_EBPF_SWAP_H
#define NETDATA_EBPF_SWAP_H 1
// Module name
// Module name & description
#define NETDATA_EBPF_MODULE_NAME_SWAP "swap"
#define NETDATA_EBPF_SWAP_MODULE_DESC "Monitor swap space usage. This thread is integrated with apps and cgroup."
#define NETDATA_SWAP_SLEEP_MS 850000ULL

View file

@ -248,7 +248,6 @@ static inline int ebpf_sync_load_and_attach(struct sync_bpf *obj, ebpf_module_t
*
*****************************************************************/
#ifdef LIBBPF_MAJOR_VERSION
/**
* Cleanup Objects
*
@ -259,28 +258,86 @@ void ebpf_sync_cleanup_objects()
int i;
for (i = 0; local_syscalls[i].syscall; i++) {
ebpf_sync_syscalls_t *w = &local_syscalls[i];
if (w->sync_obj)
#ifdef LIBBPF_MAJOR_VERSION
if (w->sync_obj) {
sync_bpf__destroy(w->sync_obj);
w->sync_obj = NULL;
}
#endif
if (w->probe_links) {
ebpf_unload_legacy_code(w->objects, w->probe_links);
w->objects = NULL;
w->probe_links = NULL;
}
}
}
#endif
/*
static void ebpf_create_sync_chart(char *id,
char *title,
int order,
int idx,
int end,
int update_every)
{
ebpf_write_chart_cmd(NETDATA_EBPF_MEMORY_GROUP, id, title, EBPF_COMMON_DIMENSION_CALL,
NETDATA_EBPF_SYNC_SUBMENU, NETDATA_EBPF_CHART_TYPE_LINE, NULL, order,
update_every,
NETDATA_EBPF_MODULE_NAME_SYNC);
*/
/**
* Sync Free
* Obsolete global
*
* Cleanup variables after child threads to stop
* Obsolete global charts created by thread.
*
* @param ptr thread data.
* @param em a pointer to `struct ebpf_module`
*/
static void ebpf_sync_free(ebpf_module_t *em)
static void ebpf_obsolete_sync_global(ebpf_module_t *em)
{
#ifdef LIBBPF_MAJOR_VERSION
ebpf_sync_cleanup_objects();
#endif
if (local_syscalls[NETDATA_SYNC_FSYNC_IDX].enabled && local_syscalls[NETDATA_SYNC_FDATASYNC_IDX].enabled)
ebpf_write_chart_obsolete(NETDATA_EBPF_MEMORY_GROUP,
NETDATA_EBPF_FILE_SYNC_CHART,
"Monitor calls for <code>fsync(2)</code> and <code>fdatasync(2)</code>.",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_EBPF_SYNC_SUBMENU,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
21300,
em->update_every);
pthread_mutex_lock(&ebpf_exit_cleanup);
em->enabled = NETDATA_THREAD_EBPF_STOPPED;
pthread_mutex_unlock(&ebpf_exit_cleanup);
if (local_syscalls[NETDATA_SYNC_MSYNC_IDX].enabled)
ebpf_write_chart_obsolete(NETDATA_EBPF_MEMORY_GROUP,
NETDATA_EBPF_MSYNC_CHART,
"Monitor calls for <code>msync(2)</code>.",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_EBPF_SYNC_SUBMENU,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
21301,
em->update_every);
if (local_syscalls[NETDATA_SYNC_SYNC_IDX].enabled && local_syscalls[NETDATA_SYNC_SYNCFS_IDX].enabled)
ebpf_write_chart_obsolete(NETDATA_EBPF_MEMORY_GROUP,
NETDATA_EBPF_SYNC_CHART,
"Monitor calls for <code>sync(2)</code> and <code>syncfs(2)</code>.",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_EBPF_SYNC_SUBMENU,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
21302,
em->update_every);
if (local_syscalls[NETDATA_SYNC_SYNC_FILE_RANGE_IDX].enabled)
ebpf_write_chart_obsolete(NETDATA_EBPF_MEMORY_GROUP,
NETDATA_EBPF_FILE_SEGMENT_CHART,
"Monitor calls for <code>sync_file_range(2)</code>.",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_EBPF_SYNC_SUBMENU,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
21303,
em->update_every);
}
/**
@ -293,7 +350,19 @@ static void ebpf_sync_free(ebpf_module_t *em)
static void ebpf_sync_exit(void *ptr)
{
ebpf_module_t *em = (ebpf_module_t *)ptr;
ebpf_sync_free(em);
if (em->enabled == NETDATA_THREAD_EBPF_FUNCTION_RUNNING) {
pthread_mutex_lock(&lock);
ebpf_obsolete_sync_global(em);
pthread_mutex_unlock(&lock);
}
ebpf_sync_cleanup_objects();
pthread_mutex_lock(&ebpf_exit_cleanup);
em->enabled = NETDATA_THREAD_EBPF_STOPPED;
ebpf_update_stats(&plugin_statistics, em);
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
/*****************************************************************
@ -489,7 +558,9 @@ static void sync_collector(ebpf_module_t *em)
int update_every = em->update_every;
int counter = update_every - 1;
int maps_per_core = em->maps_per_core;
while (!ebpf_exit_plugin) {
uint32_t running_time = 0;
uint32_t lifetime = em->lifetime;
while (!ebpf_exit_plugin && running_time < lifetime) {
(void)heartbeat_next(&hb, USEC_PER_SEC);
if (ebpf_exit_plugin || ++counter != update_every)
continue;
@ -501,6 +572,15 @@ static void sync_collector(ebpf_module_t *em)
sync_send_data();
pthread_mutex_unlock(&lock);
pthread_mutex_lock(&ebpf_exit_cleanup);
if (running_time && !em->running_time)
running_time = update_every;
else
running_time += update_every;
em->running_time = running_time;
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
}
@ -574,6 +654,8 @@ static void ebpf_create_sync_charts(int update_every)
ebpf_create_sync_chart(NETDATA_EBPF_FILE_SEGMENT_CHART,
"Monitor calls for <code>sync_file_range(2)</code>.", 21303,
NETDATA_SYNC_SYNC_FILE_RANGE_IDX, NETDATA_SYNC_SYNC_FILE_RANGE_IDX, update_every);
fflush(stdout);
}
/**

View file

@ -7,8 +7,9 @@
#include "includes/sync.skel.h"
#endif
// Module name
// Module name & description
#define NETDATA_EBPF_MODULE_NAME_SYNC "sync"
#define NETDATA_EBPF_SYNC_MODULE_DESC "Monitor calls to syscalls sync(2), fsync(2), fdatasync(2), syncfs(2), msync(2), and sync_file_range(2)."
// charts
#define NETDATA_EBPF_SYNC_CHART "sync"

View file

@ -60,6 +60,10 @@ netdata_ebpf_targets_t vfs_targets[] = { {.name = "vfs_write", .mode = EBPF_LOAD
{.name = "release_task", .mode = EBPF_LOAD_TRAMPOLINE},
{.name = NULL, .mode = EBPF_LOAD_TRAMPOLINE}};
#ifdef NETDATA_DEV_MODE
int vfs_disable_priority;
#endif
#ifdef LIBBPF_MAJOR_VERSION
/**
* Disable probe
@ -403,6 +407,447 @@ static inline int ebpf_vfs_load_and_attach(struct vfs_bpf *obj, ebpf_module_t *e
*
*****************************************************************/
static void ebpf_obsolete_specific_vfs_charts(char *type, ebpf_module_t *em);
/**
* Obsolete services
*
* Obsolete all service charts created
*
* @param em a pointer to `struct ebpf_module`
*/
static void ebpf_obsolete_vfs_services(ebpf_module_t *em)
{
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SYSCALL_APPS_FILE_DELETED,
"Files deleted",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_CGROUP_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20065,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SYSCALL_APPS_VFS_WRITE_CALLS,
"Write to disk",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_CGROUP_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20066,
em->update_every);
if (em->mode < MODE_ENTRY) {
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SYSCALL_APPS_VFS_WRITE_CALLS_ERROR,
"Fails to write",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_CGROUP_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20067,
em->update_every);
}
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SYSCALL_APPS_VFS_READ_CALLS,
"Read from disk",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_CGROUP_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20068,
em->update_every);
if (em->mode < MODE_ENTRY) {
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SYSCALL_APPS_VFS_READ_CALLS_ERROR,
"Fails to read",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_CGROUP_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20069,
em->update_every);
}
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SYSCALL_APPS_VFS_WRITE_BYTES,
"Bytes written on disk",
EBPF_COMMON_DIMENSION_BYTES,
NETDATA_VFS_CGROUP_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20070,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SYSCALL_APPS_VFS_READ_BYTES,
"Bytes read from disk",
EBPF_COMMON_DIMENSION_BYTES,
NETDATA_VFS_CGROUP_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20071,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SYSCALL_APPS_VFS_FSYNC,
"Calls to <code>vfs_fsync</code>",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_CGROUP_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20072,
em->update_every);
if (em->mode < MODE_ENTRY) {
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SYSCALL_APPS_VFS_FSYNC_CALLS_ERROR,
"Sync error",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_CGROUP_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20073,
em->update_every);
}
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SYSCALL_APPS_VFS_OPEN,
"Calls to <code>vfs_open</code>",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_CGROUP_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20074,
em->update_every);
if (em->mode < MODE_ENTRY) {
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SYSCALL_APPS_VFS_OPEN_CALLS_ERROR,
"Open error",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_CGROUP_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20075,
em->update_every);
}
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SYSCALL_APPS_VFS_CREATE,
"Calls to <code>vfs_create</code>",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_CGROUP_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20076,
em->update_every);
if (em->mode < MODE_ENTRY) {
ebpf_write_chart_obsolete(NETDATA_SERVICE_FAMILY,
NETDATA_SYSCALL_APPS_VFS_CREATE_CALLS_ERROR,
"Create error",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_CGROUP_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20077,
em->update_every);
}
}
/**
* Obsolete cgroup chart
*
* Send obsolete for all charts created before to close.
*
* @param em a pointer to `struct ebpf_module`
*/
static inline void ebpf_obsolete_vfs_cgroup_charts(ebpf_module_t *em) {
pthread_mutex_lock(&mutex_cgroup_shm);
ebpf_obsolete_vfs_services(em);
ebpf_cgroup_target_t *ect;
for (ect = ebpf_cgroup_pids; ect ; ect = ect->next) {
if (ect->systemd)
continue;
ebpf_obsolete_specific_vfs_charts(ect->name, em);
}
pthread_mutex_unlock(&mutex_cgroup_shm);
}
/**
* Obsolette apps charts
*
* Obsolete apps charts.
*
* @param em a pointer to the structure with the default values.
*/
void ebpf_obsolete_vfs_apps_charts(struct ebpf_module *em)
{
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SYSCALL_APPS_FILE_DELETED,
"Files deleted",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20065,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SYSCALL_APPS_VFS_WRITE_CALLS,
"Write to disk",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20066,
em->update_every);
if (em->mode < MODE_ENTRY) {
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SYSCALL_APPS_VFS_WRITE_CALLS_ERROR,
"Fails to write",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20067,
em->update_every);
}
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SYSCALL_APPS_VFS_READ_CALLS,
"Read from disk",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20068,
em->update_every);
if (em->mode < MODE_ENTRY) {
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SYSCALL_APPS_VFS_READ_CALLS_ERROR,
"Fails to read",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20069,
em->update_every);
}
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SYSCALL_APPS_VFS_WRITE_BYTES,
"Bytes written on disk",
EBPF_COMMON_DIMENSION_BYTES,
NETDATA_VFS_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20070,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SYSCALL_APPS_VFS_READ_BYTES,
"Bytes read from disk",
EBPF_COMMON_DIMENSION_BYTES,
NETDATA_VFS_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20071,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SYSCALL_APPS_VFS_FSYNC,
"Calls for <code>vfs_fsync</code>",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20072,
em->update_every);
if (em->mode < MODE_ENTRY) {
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SYSCALL_APPS_VFS_FSYNC_CALLS_ERROR,
"Sync error",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20073,
em->update_every);
}
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SYSCALL_APPS_VFS_OPEN,
"Calls for <code>vfs_open</code>",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20074,
em->update_every);
if (em->mode < MODE_ENTRY) {
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SYSCALL_APPS_VFS_OPEN_CALLS_ERROR,
"Open error",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20075,
em->update_every);
}
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SYSCALL_APPS_VFS_CREATE,
"Calls for <code>vfs_create</code>",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20076,
em->update_every);
if (em->mode < MODE_ENTRY) {
ebpf_write_chart_obsolete(NETDATA_APPS_FAMILY,
NETDATA_SYSCALL_APPS_VFS_CREATE_CALLS_ERROR,
"Create error",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_GROUP,
NETDATA_EBPF_CHART_TYPE_STACKED,
NULL,
20077,
em->update_every);
}
}
/**
* Obsolete global
*
* Obsolete global charts created by thread.
*
* @param em a pointer to `struct ebpf_module`
*/
static void ebpf_obsolete_vfs_global(ebpf_module_t *em)
{
ebpf_write_chart_obsolete(NETDATA_FILESYSTEM_FAMILY,
NETDATA_VFS_FILE_CLEAN_COUNT,
"Remove files",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_GROUP,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
NETDATA_CHART_PRIO_FILESYSTEM_VFS_CLEAN,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_FILESYSTEM_FAMILY,
NETDATA_VFS_FILE_IO_COUNT,
"Calls to IO",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_GROUP,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
NETDATA_CHART_PRIO_FILESYSTEM_VFS_IO_COUNT,
em->update_every);
ebpf_write_chart_obsolete(NETDATA_FILESYSTEM_FAMILY,
NETDATA_VFS_IO_FILE_BYTES,
"Bytes written and read",
EBPF_COMMON_DIMENSION_BYTES,
NETDATA_VFS_GROUP,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
NETDATA_CHART_PRIO_FILESYSTEM_VFS_IO_BYTES,
em->update_every);
if (em->mode < MODE_ENTRY) {
ebpf_write_chart_obsolete(NETDATA_FILESYSTEM_FAMILY,
NETDATA_VFS_FILE_ERR_COUNT,
"Fails to write or read",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_GROUP,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
NETDATA_CHART_PRIO_FILESYSTEM_VFS_IO_EBYTES,
em->update_every);
}
ebpf_write_chart_obsolete(NETDATA_FILESYSTEM_FAMILY,
NETDATA_VFS_FSYNC,
"Calls for <code>vfs_fsync</code>",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_GROUP,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
NETDATA_CHART_PRIO_FILESYSTEM_VFS_IO_FSYNC,
em->update_every);
if (em->mode < MODE_ENTRY) {
ebpf_write_chart_obsolete(NETDATA_FILESYSTEM_FAMILY,
NETDATA_VFS_FSYNC_ERR,
"Fails to synchronize",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_GROUP,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
NETDATA_CHART_PRIO_FILESYSTEM_VFS_IO_EFSYNC,
em->update_every);
}
ebpf_write_chart_obsolete(NETDATA_FILESYSTEM_FAMILY,
NETDATA_VFS_OPEN,
"Calls for <code>vfs_open</code>",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_GROUP,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
NETDATA_CHART_PRIO_FILESYSTEM_VFS_IO_OPEN,
em->update_every);
if (em->mode < MODE_ENTRY) {
ebpf_write_chart_obsolete(NETDATA_FILESYSTEM_FAMILY,
NETDATA_VFS_OPEN_ERR,
"Fails to open a file",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_GROUP,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
NETDATA_CHART_PRIO_FILESYSTEM_VFS_IO_EOPEN,
em->update_every);
}
ebpf_write_chart_obsolete(NETDATA_FILESYSTEM_FAMILY,
NETDATA_VFS_CREATE,
"Calls for <code>vfs_create</code>",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_GROUP,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
NETDATA_CHART_PRIO_FILESYSTEM_VFS_IO_CREATE,
em->update_every);
if (em->mode < MODE_ENTRY) {
ebpf_write_chart_obsolete(NETDATA_FILESYSTEM_FAMILY,
NETDATA_VFS_CREATE_ERR,
"Fails to create a file.",
EBPF_COMMON_DIMENSION_CALL,
NETDATA_VFS_GROUP,
NETDATA_EBPF_CHART_TYPE_LINE,
NULL,
NETDATA_CHART_PRIO_FILESYSTEM_VFS_IO_ECREATE,
em->update_every);
}
}
/**
* Exit
*
@ -414,15 +859,45 @@ static void ebpf_vfs_exit(void *ptr)
{
ebpf_module_t *em = (ebpf_module_t *)ptr;
#ifdef LIBBPF_MAJOR_VERSION
if (vfs_bpf_obj)
vfs_bpf__destroy(vfs_bpf_obj);
if (em->enabled == NETDATA_THREAD_EBPF_FUNCTION_RUNNING) {
pthread_mutex_lock(&lock);
if (em->cgroup_charts) {
ebpf_obsolete_vfs_cgroup_charts(em);
fflush(stdout);
}
if (em->apps_charts & NETDATA_EBPF_APPS_FLAG_CHART_CREATED) {
ebpf_obsolete_vfs_apps_charts(em);
}
ebpf_obsolete_vfs_global(em);
#ifdef NETDATA_DEV_MODE
if (ebpf_aral_vfs_pid)
ebpf_statistic_obsolete_aral_chart(em, vfs_disable_priority);
#endif
if (em->objects)
fflush(stdout);
pthread_mutex_unlock(&lock);
}
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_REMOVE);
#ifdef LIBBPF_MAJOR_VERSION
if (vfs_bpf_obj) {
vfs_bpf__destroy(vfs_bpf_obj);
vfs_bpf_obj = NULL;
}
#endif
if (em->objects) {
ebpf_unload_legacy_code(em->objects, em->probe_links);
em->objects = NULL;
em->probe_links = NULL;
}
pthread_mutex_lock(&ebpf_exit_cleanup);
em->enabled = NETDATA_THREAD_EBPF_STOPPED;
ebpf_update_stats(&plugin_statistics, em);
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
@ -1486,7 +1961,9 @@ static void vfs_collector(ebpf_module_t *em)
int update_every = em->update_every;
int counter = update_every - 1;
int maps_per_core = em->maps_per_core;
while (!ebpf_exit_plugin) {
uint32_t running_time = 0;
uint32_t lifetime = em->lifetime;
while (!ebpf_exit_plugin && running_time < lifetime) {
(void)heartbeat_next(&hb, USEC_PER_SEC);
if (ebpf_exit_plugin || ++counter != update_every)
continue;
@ -1519,6 +1996,15 @@ static void vfs_collector(ebpf_module_t *em)
pthread_mutex_unlock(&lock);
pthread_mutex_unlock(&collect_data_mutex);
pthread_mutex_lock(&ebpf_exit_cleanup);
if (running_time && !em->running_time)
running_time = update_every;
else
running_time += update_every;
em->running_time = running_time;
pthread_mutex_unlock(&ebpf_exit_cleanup);
}
}
@ -1690,6 +2176,8 @@ static void ebpf_create_global_charts(ebpf_module_t *em)
&vfs_publish_aggregated[NETDATA_KEY_PUBLISH_VFS_CREATE],
1, em->update_every, NETDATA_EBPF_MODULE_NAME_VFS);
}
fflush(stdout);
}
/**
@ -1934,10 +2422,10 @@ void *ebpf_vfs_thread(void *ptr)
pthread_mutex_lock(&lock);
ebpf_create_global_charts(em);
ebpf_update_stats(&plugin_statistics, em);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps);
ebpf_update_kernel_memory_with_vector(&plugin_statistics, em->maps, EBPF_ACTION_STAT_ADD);
#ifdef NETDATA_DEV_MODE
if (ebpf_aral_vfs_pid)
ebpf_statistic_create_aral_chart(NETDATA_EBPF_VFS_ARAL_NAME, em);
vfs_disable_priority = ebpf_statistic_create_aral_chart(NETDATA_EBPF_VFS_ARAL_NAME, em);
#endif
pthread_mutex_unlock(&lock);

View file

@ -3,8 +3,9 @@
#ifndef NETDATA_EBPF_VFS_H
#define NETDATA_EBPF_VFS_H 1
// Module name
// Module name & description
#define NETDATA_EBPF_MODULE_NAME_VFS "vfs"
#define NETDATA_EBPF_VFS_MODULE_DESC "Monitor VFS (Virtual File System) functions. This thread is integrated with apps and cgroup."
#define NETDATA_DIRECTORY_VFS_CONFIG_FILE "vfs.conf"

View file

@ -33,6 +33,7 @@ functions - [plugins.d](https://github.com/netdata/netdata/blob/master/collector
| Function | Description | plugin - module |
| :-- | :-- | :-- |
| processes | Detailed information on the currently running processes on the node. | [apps.plugin](https://github.com/netdata/netdata/blob/master/collectors/apps.plugin/README.md) |
| ebpf_thread | Controller for eBPF threads. | [ebpf.plugin](https://github.com/netdata/netdata/blob/master/collectors/ebpf.plugin/README.md) |
If you have ideas or requests for other functions:
* open a [Feature request](https://github.com/netdata/netdata-cloud/issues/new?assignees=&labels=feature+request%2Cneeds+triage&template=FEAT_REQUEST.yml&title=%5BFeat%5D%3A+) on Netdata Cloud repo

View file

@ -391,9 +391,10 @@ static void ebpf_mount_name(char *out, size_t len, char *path, uint32_t kver, co
* Count the information from targets.
*
* @param report the output structure
* @param targets vector with information about the eBPF plugin.
* @param targets vector with information about the eBPF plugin.
* @param value factor used to update calculation
*/
static void ebpf_stats_targets(ebpf_plugin_stats_t *report, netdata_ebpf_targets_t *targets)
static void ebpf_stats_targets(ebpf_plugin_stats_t *report, netdata_ebpf_targets_t *targets, int value)
{
if (!targets) {
report->probes = report->tracepoints = report->trampolines = 0;
@ -404,19 +405,19 @@ static void ebpf_stats_targets(ebpf_plugin_stats_t *report, netdata_ebpf_targets
while (targets[i].name) {
switch (targets[i].mode) {
case EBPF_LOAD_PROBE: {
report->probes++;
report->probes += value;
break;
}
case EBPF_LOAD_RETPROBE: {
report->retprobes++;
report->retprobes += value;
break;
}
case EBPF_LOAD_TRACEPOINT: {
report->tracepoints++;
report->tracepoints += value;
break;
}
case EBPF_LOAD_TRAMPOLINE: {
report->trampolines++;
report->trampolines += value;
break;
}
}
@ -437,27 +438,30 @@ static void ebpf_stats_targets(ebpf_plugin_stats_t *report, netdata_ebpf_targets
*/
void ebpf_update_stats(ebpf_plugin_stats_t *report, ebpf_module_t *em)
{
report->threads++;
int value;
// It is not necessary to report more information.
if (em->enabled != NETDATA_THREAD_EBPF_RUNNING)
return;
if (em->enabled > NETDATA_THREAD_EBPF_FUNCTION_RUNNING)
value = -1;
else
value = 1;
report->running++;
report->threads += value;
report->running += value;
// In theory the `else if` is useless, because when this function is called, the module should not stay in
// EBPF_LOAD_PLAY_DICE. We have this additional condition to detect errors from developers.
if (em->load & EBPF_LOAD_LEGACY)
report->legacy++;
report->legacy += value;
else if (em->load & EBPF_LOAD_CORE)
report->core++;
report->core += value;
if (em->maps_per_core)
report->hash_percpu++;
report->hash_percpu += value;
else
report->hash_unique++;
report->hash_unique += value;
ebpf_stats_targets(report, em->targets);
ebpf_stats_targets(report, em->targets, value);
}
/**
@ -528,8 +532,11 @@ void ebpf_update_kernel_memory(ebpf_plugin_stats_t *report, ebpf_local_maps_t *m
*
* @param report the output structure
* @param map pointer to a map. Last map must fish with name = NULL
* @param action should plugin add or remove values from amount.
*/
void ebpf_update_kernel_memory_with_vector(ebpf_plugin_stats_t *report, ebpf_local_maps_t *maps)
void ebpf_update_kernel_memory_with_vector(ebpf_plugin_stats_t *report,
ebpf_local_maps_t *maps,
ebpf_stats_action_t action)
{
if (!maps)
return;
@ -541,7 +548,7 @@ void ebpf_update_kernel_memory_with_vector(ebpf_plugin_stats_t *report, ebpf_loc
if (fd == ND_EBPF_MAP_FD_NOT_INITIALIZED)
continue;
ebpf_update_kernel_memory(report, map, EBPF_ACTION_STAT_ADD);
ebpf_update_kernel_memory(report, map, action);
}
}
@ -1238,6 +1245,9 @@ void ebpf_update_module_using_config(ebpf_module_t *modules, netdata_ebpf_load_m
modules->pid_map_size = (uint32_t)appconfig_get_number(modules->cfg, EBPF_GLOBAL_SECTION, EBPF_CFG_PID_SIZE,
modules->pid_map_size);
modules->lifetime = (uint32_t) appconfig_get_number(modules->cfg, EBPF_GLOBAL_SECTION,
EBPF_CFG_LIFETIME, EBPF_DEFAULT_LIFETIME);
char *value = ebpf_convert_load_mode_to_string(modules->load & NETDATA_EBPF_LOAD_METHODS);
char *type_format = appconfig_get(modules->cfg, EBPF_GLOBAL_SECTION, EBPF_CFG_TYPE_FORMAT, value);
netdata_ebpf_load_mode_t load = epbf_convert_string_to_load_mode(type_format);
@ -1258,7 +1268,7 @@ void ebpf_update_module_using_config(ebpf_module_t *modules, netdata_ebpf_load_m
modules->maps_per_core = CONFIG_BOOLEAN_NO;
#ifdef NETDATA_DEV_MODE
netdata_log_info("The thread %s was configured with: mode = %s; update every = %d; apps = %s; cgroup = %s; ebpf type format = %s; ebpf co-re tracing = %s; collect pid = %s; maps per core = %s",
netdata_log_info("The thread %s was configured with: mode = %s; update every = %d; apps = %s; cgroup = %s; ebpf type format = %s; ebpf co-re tracing = %s; collect pid = %s; maps per core = %s, lifetime=%u",
modules->thread_name,
load_mode,
modules->update_every,
@ -1267,7 +1277,8 @@ void ebpf_update_module_using_config(ebpf_module_t *modules, netdata_ebpf_load_m
type_format,
core_attach,
collect_pid,
(modules->maps_per_core)?"enabled":"disabled"
(modules->maps_per_core)?"enabled":"disabled",
modules->lifetime
);
#endif
}

View file

@ -43,6 +43,7 @@
#define EBPF_CFG_MAPS_PER_CORE "maps per core"
#define EBPF_CFG_UPDATE_EVERY "update every"
#define EBPF_CFG_LIFETIME "lifetime"
#define EBPF_CFG_UPDATE_APPS_EVERY_DEFAULT 10
#define EBPF_CFG_PID_SIZE "pid table size"
#define EBPF_CFG_APPLICATION "apps"
@ -270,15 +271,17 @@ typedef enum netdata_apps_integration_flags {
#define NETDATA_EBPF_STAT_DIMENSION_ARAL "aral"
enum ebpf_threads_status {
NETDATA_THREAD_EBPF_RUNNING,
NETDATA_THREAD_EBPF_STOPPING,
NETDATA_THREAD_EBPF_STOPPED,
NETDATA_THREAD_EBPF_NOT_RUNNING
NETDATA_THREAD_EBPF_RUNNING, // started by plugin
NETDATA_THREAD_EBPF_FUNCTION_RUNNING, // started by function
NETDATA_THREAD_EBPF_STOPPING, // stopping thread
NETDATA_THREAD_EBPF_STOPPED, // thread stopped
NETDATA_THREAD_EBPF_NOT_RUNNING // thread was never started
};
typedef struct ebpf_module {
const char *thread_name;
const char *config_name;
const char *thread_description;
enum ebpf_threads_status enabled;
void *(*start_routine)(void *);
int update_every;
@ -306,8 +309,16 @@ typedef struct ebpf_module {
char memory_usage[NETDATA_EBPF_CHART_MEM_LENGTH];
char memory_allocations[NETDATA_EBPF_CHART_MEM_LENGTH];
int maps_per_core;
// period to run
uint32_t running_time; // internal usage, this is used to reset a value when a new request happens.
uint32_t lifetime;
} ebpf_module_t;
#define EBPF_DEFAULT_LIFETIME 300
// This will be present until all functions are merged
#define EBPF_NON_FUNCTION_LIFE_TIME 86400
int ebpf_get_kernel_version();
int get_redhat_release();
int has_condition_to_run(int version);
@ -336,6 +347,7 @@ void ebpf_update_map_size(struct bpf_map *map, ebpf_local_maps_t *lmap, ebpf_mod
typedef struct netdata_ebpf_histogram {
char *name;
char *title;
char *ctx;
int order;
uint64_t histogram[NETDATA_EBPF_HIST_MAX_BINS];
} netdata_ebpf_histogram_t;
@ -425,9 +437,11 @@ void ebpf_update_map_type(struct bpf_map *map, ebpf_local_maps_t *w);
void ebpf_define_map_type(ebpf_local_maps_t *maps, int maps_per_core, int kver);
#endif
void ebpf_update_kernel_memory_with_vector(ebpf_plugin_stats_t *report, ebpf_local_maps_t *maps);
void ebpf_update_kernel_memory_with_vector(ebpf_plugin_stats_t *report, ebpf_local_maps_t *maps,
ebpf_stats_action_t action);
void ebpf_update_kernel_memory(ebpf_plugin_stats_t *report, ebpf_local_maps_t *map, ebpf_stats_action_t action);
void ebpf_statistic_create_aral_chart(char *name, ebpf_module_t *em);
int ebpf_statistic_create_aral_chart(char *name, ebpf_module_t *em);
void ebpf_statistic_obsolete_aral_chart(ebpf_module_t *em, int prio);
void ebpf_send_data_aral_chart(ARAL *memory, ebpf_module_t *em);
#endif /* NETDATA_EBPF_H */

View file

@ -530,6 +530,7 @@ rm -rf "${RPM_BUILD_ROOT}"
%{_libexecdir}/%{name}/plugins.d/system-info.sh
%{_libexecdir}/%{name}/plugins.d/tc-qos-helper.sh
%{_libexecdir}/%{name}/plugins.d/template_dim.sh
%{_libexecdir}/%{name}/plugins.d/ebpf_thread_function.sh
# cgroup-network detects the network interfaces of CGROUPs
# it must be able to use setns() and run cgroup-network-helper.sh as root

View file

@ -9,6 +9,7 @@ CLEANFILES = \
$(srcdir)/urls/request.sh \
$(srcdir)/alarm_repetition/alarm.sh \
$(srcdir)/template_dimension/template_dim.sh \
$(srcdir)/ebpf/ebpf_thread_function.sh \
$(NULL)
include $(top_srcdir)/build/subst.inc
@ -20,6 +21,7 @@ dist_noinst_DATA = \
$(srcdir)/urls/request.sh.in \
$(srcdir)/alarm_repetition/alarm.sh.in \
$(srcdir)/template_dimension/template_dim.sh.in \
$(srcdir)/ebpf/ebpf_thread_function.sh.in \
$(NULL)
dist_plugins_SCRIPTS = \
@ -28,6 +30,7 @@ dist_plugins_SCRIPTS = \
$(srcdir)/urls/request.sh \
$(srcdir)/alarm_repetition/alarm.sh \
$(srcdir)/template_dimension/template_dim.sh \
$(srcdir)/ebpf/ebpf_thread_function.sh \
$(NULL)
dist_noinst_SCRIPTS = \

28
tests/ebpf/ebpf.d.conf Normal file
View file

@ -0,0 +1,28 @@
[global]
ebpf load mode = entry
apps = yes
cgroups = no
update every = 5
pid table size = 32768
btf path = /sys/kernel/btf/
maps per core = yes
life time = 300
[ebpf programs]
cachestat = no
dcstat = no
disk = no
fd = no
filesystem = no
hardirq = no
mdflush = no
mount = no
oomkill = no
process = no
shm = no
socket = no
softirq = no
sync = no
swap = no
vfs = no
network connections = no

View file

@ -0,0 +1,52 @@
#!/bin/bash
netdata_ebpf_test_functions() {
echo "QUERYING: ${1}"
curl -k -o /tmp/ebpf_netdata_test_functions.txt "${1}"
TEST=$?
if [ $TEST -ne 0 ]; then
echo "Cannot request run a for ${1}. See '/tmp/ebpf_netdata_test_functions.txt' for more details."
exit 1
fi
grep "${2}" /tmp/ebpf_netdata_test_functions.txt >/dev/null
TEST=$?
if [ $TEST -ne 0 ]; then
echo "Cannot find ${2} in the output. See '/tmp/ebpf_netdata_test_functions.txt' for more details.."
exit 1
fi
rm /tmp/ebpf_netdata_test_functions.txt
}
MURL="http://127.0.0.1:19999"
INTERVAL=60
if [ -n "$1" ]; then
MURL="$1"
fi
# Check function loaded
netdata_ebpf_test_functions "${MURL}/api/v1/functions" "ebpf_thread"
# Check function help
netdata_ebpf_test_functions "${MURL}/api/v1/function?function=ebpf_thread%20help" "allows user to control eBPF threads"
#Test default request
netdata_ebpf_test_functions "${MURL}/api/v1/function?function=ebpf_thread" "columns"
#Test thread requests . The mdflush is not enabled, because it is not present in all distributions by default.
#Socket is not in the list, because it will have a complete refactory with next PR
for THREAD in "cachestat" "dc" "disk" "fd" "filesystem" "hardirq" "mount" "oomkill" "process" "shm" "softirq" "sync" "swap" "vfs" ;
do
echo "TESTING ${THREAD}"
netdata_ebpf_test_functions "${MURL}/api/v1/function?function=ebpf_thread%20enable:${THREAD}:${INTERVAL}%20thread:${THREAD}"
sleep 17
netdata_ebpf_test_functions "${MURL}/api/v1/function?function=ebpf_thread%20thread:${THREAD}" "running"
sleep 17
netdata_ebpf_test_functions "${MURL}/api/v1/function?function=ebpf_thread%20disable:${THREAD}"
sleep 6
netdata_ebpf_test_functions "${MURL}/api/v1/function?function=ebpf_thread%20thread:${THREAD}" "stopped"
sleep 6
done

View file

@ -0,0 +1,52 @@
#!/bin/bash
netdata_ebpf_test_functions() {
echo "QUERYING: ${1}"
curl -k -o /tmp/ebpf_netdata_test_functions.txt "${1}"
TEST=$?
if [ $TEST -ne 0 ]; then
echo "Cannot request run a for ${1}. See '/tmp/ebpf_netdata_test_functions.txt' for more details."
exit 1
fi
grep "${2}" /tmp/ebpf_netdata_test_functions.txt >/dev/null
TEST=$?
if [ $TEST -ne 0 ]; then
echo "Cannot find ${2} in the output. See '/tmp/ebpf_netdata_test_functions.txt' for more details.."
exit 1
fi
rm /tmp/ebpf_netdata_test_functions.txt
}
MURL="http://127.0.0.1:19999"
INTERVAL=60
if [ -n "$1" ]; then
MURL="$1"
fi
# Check function loaded
netdata_ebpf_test_functions "${MURL}/api/v1/functions" "ebpf_thread"
# Check function help
netdata_ebpf_test_functions "${MURL}/api/v1/function?function=ebpf_thread%20help" "allows user to control eBPF threads"
#Test default request
netdata_ebpf_test_functions "${MURL}/api/v1/function?function=ebpf_thread" "columns"
#Test thread requests . The mdflush is not enabled, because it is not present in all distributions by default.
#Socket is not in the list, because it will have a complete refactory with next PR
for THREAD in "cachestat" "dc" "disk" "fd" "filesystem" "hardirq" "mount" "oomkill" "process" "shm" "softirq" "sync" "swap" "vfs" ;
do
echo "TESTING ${THREAD}"
netdata_ebpf_test_functions "${MURL}/api/v1/function?function=ebpf_thread%20enable:${THREAD}:${INTERVAL}%20thread:${THREAD}"
sleep 17
netdata_ebpf_test_functions "${MURL}/api/v1/function?function=ebpf_thread%20thread:${THREAD}" "running"
sleep 17
netdata_ebpf_test_functions "${MURL}/api/v1/function?function=ebpf_thread%20disable:${THREAD}"
sleep 6
netdata_ebpf_test_functions "${MURL}/api/v1/function?function=ebpf_thread%20thread:${THREAD}" "stopped"
sleep 6
done

View file

@ -4935,7 +4935,11 @@ netdataDashboard.context = {
},
'netdata.ebpf_threads': {
info: 'Show total number of threads and number of active threads. For more details about the threads, see the <a href="https://learn.netdata.cloud/docs/agent/collectors/ebpf.plugin#ebpf-programs-configuration-options" target="_blank">official documentation</a>.'
info: 'Show thread status. Threads running have value 1 an stopped value 0. For more details about the threads, see the <a href="https://learn.netdata.cloud/docs/agent/collectors/ebpf.plugin#ebpf-programs-configuration-options" target="_blank">official documentation</a>.'
},
'netdata.ebpf_life_time': {
info: 'Time remaining for thread shutdown itself.'
},
'netdata.ebpf_load_methods': {