mirror of
https://github.com/netdata/netdata.git
synced 2025-04-23 13:00:23 +00:00
Config docs improvements (#4918)
* WIP to add a new config readme * WIP * WIP * WIP * WIP * WIP * Major rewrite of configuration instructions and minor improvements to the html doc site * Major rewrite of configuration instructions and minor improvements to the html doc site * Major rewrite of configuration instructions and minor improvements to the html doc site * Major rewrite of configuration instructions and minor improvements to the html doc site * Major rewrite of configuration instructions and minor improvements to the html doc site * Major rewrite of configuration instructions and minor improvements to the html doc site * Major rewrite of configuration instructions and minor improvements to the html doc site * Major rewrite of configuration instructions and minor improvements to the html doc site
This commit is contained in:
Chris Akritidis
GitHub
parent
31f14e5855
commit
f1036f74f7
19 changed files with 476 additions and 381 deletions
Makefile.am
collectors/python.d.plugin
daemon
docs
Performance.mdconfiguration-guide.md
generator
netdata-for-IoT.mdnetdata-security.mdregistry
streaming
web
|
|
@ -75,7 +75,9 @@ dist_noinst_DATA= \
|
|||
docs/netdata-security.md \
|
||||
docs/Why-Netdata.md \
|
||||
docs/GettingStarted.md \
|
||||
docs/generator/themes/material/partials/footer.html \
|
||||
docs/Charts.md \
|
||||
docs/configuration-guide.md \
|
||||
docs/generator/custom \
|
||||
installer/README.md \
|
||||
installer/UNINSTALL.md \
|
||||
installer/UPDATE.md \
|
||||
|
|
|
|||
|
|
@ -74,7 +74,7 @@ Writing new python module is simple. You just need to remember to include 5 majo
|
|||
|
||||
If you plan to submit the module in a PR, make sure and go through the [PR checklist for new modules](#pull-request-checklist-for-python-plugins) beforehand to make sure you have updated all the files you need to.
|
||||
|
||||
For a quick start, you can look at the [example plugin](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/example/example.chart.py).
|
||||
For a quick start, you can look at the [example plugin](example/example.chart.py).
|
||||
|
||||
### Global variables `ORDER` and `CHART`
|
||||
|
||||
|
|
|
|||
|
|
@ -289,7 +289,7 @@ If you want to control it entirely via systemd, you can set in `netdata.conf`:
|
|||
Using the above, whatever OOM Score you have set at `netdata.service` will be maintained by netdata.
|
||||
|
||||
|
||||
## netdata process scheduling policy
|
||||
## Netdata process scheduling policy
|
||||
|
||||
By default netdata runs with the `idle` process scheduling policy, so that it uses CPU resources, only when there is idle CPU to spare. On very busy servers (or weak servers), this can lead to gaps on the charts.
|
||||
|
||||
|
|
@ -409,20 +409,17 @@ sudo systemctl daemon-reload
|
|||
sudo systemctl restart netdata
|
||||
```
|
||||
|
||||
## virtual memory
|
||||
## Virtual memory
|
||||
|
||||
You may notice that netdata's virtual memory size, as reported by `ps` or `/proc/pid/status`
|
||||
(or even netdata's applications virtual memory chart) is unrealistically high.
|
||||
You may notice that netdata's virtual memory size, as reported by `ps` or `/proc/pid/status` (or even netdata's applications virtual memory chart) is unrealistically high.
|
||||
|
||||
For example, it may be reported to be 150+MB, even if the resident memory size is just 25MB.
|
||||
Similar values may be reported for netdata plugins too.
|
||||
For example, it may be reported to be 150+MB, even if the resident memory size is just 25MB. Similar values may be reported for netdata plugins too.
|
||||
|
||||
Check this for example: A netdata installation with default settings on Ubuntu 16.04LTS.
|
||||
The top chart is **real memory used**, while the bottom one is **virtual memory**:
|
||||
Check this for example: A netdata installation with default settings on Ubuntu 16.04LTS. The top chart is **real memory used**, while the bottom one is **virtual memory**:
|
||||
|
||||
|
||||
|
||||
#### why this happens?
|
||||
**Why does this happen?**
|
||||
|
||||
The system memory allocator allocates virtual memory arenas, per thread running.
|
||||
On Linux systems this defaults to 16MB per thread on 64 bit machines. So, if you get the
|
||||
|
|
@ -437,21 +434,16 @@ linux (that uses **musl** instead of **glibc**) is this:
|
|||
|
||||
|
||||
|
||||
#### can we do anything to lower it?
|
||||
**Can we do anything to lower it?**
|
||||
|
||||
Since netdata already uses minimal memory allocations while it runs (i.e. it adapts its memory
|
||||
on start, so that while repeatedly collects data it does not do memory allocations), it already
|
||||
instructs the system memory allocator to minimize the memory arenas for each thread. We have also
|
||||
added [2 configuration options](https://github.com/netdata/netdata/blob/5645b1ee35248d94e6931b64a8688f7f0d865ec6/src/main.c#L410-L418)
|
||||
to allow you tweak these settings.
|
||||
Since netdata already uses minimal memory allocations while it runs (i.e. it adapts its memory on start, so that while repeatedly collects data it does not do memory allocations), it already instructs the system memory allocator to minimize the memory arenas for each thread. We have also added [2 configuration options](https://github.com/netdata/netdata/blob/5645b1ee35248d94e6931b64a8688f7f0d865ec6/src/main.c#L410-L418)
|
||||
to allow you tweak these settings: `glibc malloc arena max for plugins` and `glibc malloc arena max for netdata`.
|
||||
|
||||
However, even if we instructed the memory allocator to use just one arena, it seems it allocates
|
||||
an arena per thread.
|
||||
However, even if we instructed the memory allocator to use just one arena, it seems it allocates an arena per thread.
|
||||
|
||||
netdata also supports `jemalloc` and `tcmalloc`, however both behave exactly the same to the
|
||||
glibc memory allocator in this aspect.
|
||||
netdata also supports `jemalloc` and `tcmalloc`, however both behave exactly the same to the glibc memory allocator in this aspect.
|
||||
|
||||
#### Is this a problem?
|
||||
**Is this a problem?**
|
||||
|
||||
No, it is not.
|
||||
|
||||
|
|
|
|||
273
daemon/config/README.md
Executable file → Normal file
273
daemon/config/README.md
Executable file → Normal file
|
|
@ -1,163 +1,25 @@
|
|||
# Configuration guide
|
||||
# Daemon configuration
|
||||
|
||||
Configuration files are placed in `/etc/netdata`.
|
||||
|
||||
## Netdata daemon
|
||||
<details markdown="1"><summary>The daemon configuration file is read from `/etc/netdata/netdata.conf`.</summary>
|
||||
Depending on your installation method, Netdata will have been installed either directly under `/`, or under `/opt/netdata`. The paths mentioned here and in the documentation in general assume that your installation is under `/`. If it is not, you will find the exact same paths under `/opt/netdata` as well. (i.e. `/etc/netdata` will be `/opt/netdata/etc/netdata`).</details>
|
||||
|
||||
The daemon configuration file is read from `/etc/netdata/netdata.conf`.
|
||||
This config file **is not needed by default**. Netdata works fine out of the box without it. But it does allow you to adapt the general behavior of Netdata, in great detail. You can find all these settings, with their default values, by accessing the URL `https://netdata.server.hostname:19999/netdata.conf`. For example check the configuration file of [netdata.firehol.org](http://netdata.firehol.org/netdata.conf). HTTP access to this file is limited by default to private IPs, via the [web server access lists](../../web/server/#access-lists).
|
||||
|
||||
In this file you can configure all aspects of netdata. Netdata provides configuration settings for plugins and charts found when started. You can find all these settings, with their default values, by accessing the URL `https://netdata.server.hostname:19999/netdata.conf`. For example check the configuration file of [netdata.firehol.org](http://netdata.firehol.org/netdata.conf).
|
||||
`netdata.conf` has sections stated with `[section]`. You will see the following sections:
|
||||
|
||||
The configuration file has sections stated with `[section]`. There will be the following sections:
|
||||
|
||||
1. `[global]` for global netdata daemon options
|
||||
2. `[plugins]` for controlling which plugins the netdata will use
|
||||
3. `[plugin:NAME]` one such section for each plugin enabled
|
||||
4. `[CHART_NAME]` once such section for each chart defined
|
||||
1. `[global]` to [configure](#global-section-options) the [netdata daemon](../).
|
||||
2. `[web]` to [configure the web server](../../web/server).
|
||||
3. `[plugins]` to [configure](#plugins-section-options) which [collectors](../../collectors) to use and PATH settings.
|
||||
4. `[health]` to [configure](#health-section-options) general settings for [health monitoring](../../health)
|
||||
5. `[registry]` for the [netdata registry](../../registry).
|
||||
6. `[backend]` to set up [streaming and replication](../../streaming) options.
|
||||
7. `[statsd]` for the general settings of the [stats.d.plugin](../../collectors/statsd.plugin).
|
||||
8. `[plugin:NAME]` sections for each collector plugin, under the comment [Per plugin configuration](#per-plugin-configuration).
|
||||
9. `[CHART_NAME]` sections for each chart defined, under the comment [Per chart configuration](#per-chart-configuration).
|
||||
|
||||
The configuration file is a `name = value` dictionary. Netdata will not complain if you set options unknown to it. When you check the running configuration by accessing the URL `/netdata.conf` on your netdata server, netdata will add a comment on settings it does not currently use.
|
||||
|
||||
### [global] section options
|
||||
|
||||
|
||||
setting | default | info
|
||||
:------:|:-------:|:----
|
||||
hostname|auto-detected|The hostname of the computer running netdata.
|
||||
history|3600|The number of entries the netdata daemon will by default keep in memory for each chart dimension. This setting can also be configured per chart. Check [Memory Requirements](../../database/#database) for more information.
|
||||
config directory|`/etc/netdata`|The directory configuration files are kept.
|
||||
plugins directory|`/usr/libexec/netdata/plugins.d`|The directory plugin programs are kept. This setting supports multiple directories, space separated. If any directory path contains spaces, enclose it in single or double quotes.
|
||||
web files directory|`/usr/share/netdata/web`|The directory the web static files are kept.
|
||||
cache directory|`/var/cache/netdata`|The directory the memory database will be stored if and when netdata exits. Netdata will re-read the database when it will start again, to continue from the same point.
|
||||
log directory|`/var/log/netdata`|The directory in which the [log files](../#log-files) are kept.
|
||||
host access prefix|*empty*|This is used in docker environments where /proc, /sys, etc have to be accessed via another path. You may also have to set SYS_PTRACE capability on the docker for this work. Check [issue 43](https://github.com/netdata/netdata/issues/43).
|
||||
debug flags|0x00000000|Bitmap of debug options to enable. For more information check [Tracing Options](../#debugging).
|
||||
memory deduplication (ksm)|yes|When set to `yes`, netdata will offer its in-memory round robin database to kernel same page merging (KSM) for deduplication. For more information check [[Memory Deduplication - Kernel Same Page Merging - KSM]]
|
||||
debug log|`/var/log/netdata/debug.log`|The filename to save debug information. This file will not be created is debugging is not enabled. You can also set it to `syslog` to send the debug messages to syslog, or `none` to disable this log. For more information check [Tracing Options](../#debugging).
|
||||
error log|`/var/log/netdata/error.log`|The filename to save error messages for netdata daemon and all plugins (`stderr` is sent here for all netdata programs, including the plugins). You can also set it to `syslog` to send the errors to syslog, or `none` to disable this log.
|
||||
access log|`/var/log/netdata/access.log`|The filename to save the log of web clients accessing netdata charts. You can also set it to `syslog` to send the access log to syslog, or `none` to disable this log.
|
||||
memory mode|save|When set to `save` netdata will save its round robin database on exit and load it on startup. When set to `map` the cache files will be updated in real time (check `man mmap` - do not set this on systems with heavy load or slow disks - the disks will continuously sync the in-memory database of netdata). When set to `ram` the round robin database will be temporary and it will be lost when netdata exits.
|
||||
update every|1|The frequency in seconds, for data collection. For more information see [Performance](../../docs/Performance.md#performance).
|
||||
run as user|`netdata`|The user netdata will run as.
|
||||
web files owner|`netdata`|The user that owns the web static files. Netdata will refuse to serve a file that is not owned by this user, even if it has read access to that file. If the user given is not found, netdata will only serve files owned by user given in `run as user`.
|
||||
http port listen backlog|100|The port backlog. Check `man 2 listen`.
|
||||
default port|19999|The default port to listen for web clients.
|
||||
bind to|`*`|The IP address and port to listen to. This is a space separated list of IPv4 or IPv6 address and ports. The default will bind to all IP addresses. Example: `bind to = 127.0.0.1:19999 10.11.12.1:19998 [::1]:19999`.
|
||||
disconnect idle web clients after seconds|60|The time in seconds to disconnect web clients after being totally idle.
|
||||
enable web responses gzip compression|yes|When set to `yes`, netdata web responses will be GZIP compressed, if the web client accepts such responses.
|
||||
|
||||
##### Netdata process priority
|
||||
|
||||
By default, netdata runs with the `idle` process scheduler, which assigns CPU resources to netdata, only when the system has such resources to spare.
|
||||
|
||||
The following `netdata.conf` settings control this:
|
||||
|
||||
```
|
||||
[global]
|
||||
process scheduling policy = idle
|
||||
process scheduling priority = 0
|
||||
process nice level = 19
|
||||
```
|
||||
|
||||
The policies supported by netdata are `idle` (the netdata default), `other` (also as `nice`), `batch`, `rr`, `fifo`. netdata also recognizes `keep` and `none` to keep the current settings without changing them.
|
||||
|
||||
For `other`, `nice` and `batch`, the setting `process nice level = 19` is activated to configure the nice level of netdata. Nice gets values -20 (highest) to 19 (lowest).
|
||||
|
||||
For `rr` and `fifo`, the setting `process scheduling priority = 0` is activated to configure the priority of the relative scheduling policy. Priority gets values 1 (lowest) to 99 (highest).
|
||||
|
||||
For the details of each scheduler, see `man sched_setscheduler` and `man sched`.
|
||||
|
||||
When netdata is running under systemd, it can only lower its priority (the default is `other` with `nice level = 0`). If you want to make netdata to get more CPU than that, you will need to set in `netdata.conf`:
|
||||
|
||||
```
|
||||
[global]
|
||||
process scheduling policy = keep
|
||||
```
|
||||
|
||||
and edit `/etc/systemd/system/netdata.service` and add:
|
||||
|
||||
```
|
||||
CPUSchedulingPolicy=other | batch | idle | fifo | rr
|
||||
CPUSchedulingPriority=99
|
||||
Nice=-10
|
||||
```
|
||||
|
||||
|
||||
### [plugins] section options
|
||||
|
||||
In this section there will be a boolean (`yes`/`no`) option for each plugin. Additionally, there will be the following options:
|
||||
|
||||
setting | default | info
|
||||
:------:|:-------:|:----
|
||||
checks|no|This is a debugging plugin for the internal latency of netdata.
|
||||
enable running new plugins|yes|When set to `yes`, netdata will enable plugins not configured specifically for them. Setting this to `no` will disable all plugins you have not set to `yes` explicitly.
|
||||
check for new plugins every|60|The time in seconds to check for new plugins in the plugins directory. This allows having other applications dynamically creating plugins for netdata.
|
||||
|
||||
## Netdata plugins
|
||||
|
||||
The configuration options for plugins appear in sections following the pattern `[plugin:NAME]`.
|
||||
|
||||
### Internal plugins
|
||||
|
||||
Most internal plugins will provide additional options. Check [Internal Plugins](../../collectors/) for more information.
|
||||
|
||||
|
||||
### External plugins
|
||||
|
||||
External plugins will have only 2 options at `netdata.conf`:
|
||||
|
||||
setting | default | info
|
||||
:------:|:-------:|:----
|
||||
update every|the value of `[global].update every` setting|The frequency in seconds the plugin should collect values. For more information check [Performance](../../docs/Performance.md#performance).
|
||||
command options|*empty*|Additional command line options to pass to the plugin.
|
||||
|
||||
External plugins that need additional configuration may support a dedicated file in `/etc/netdata`. Check their documentation.
|
||||
|
||||
---
|
||||
|
||||
## A note about netdata.conf
|
||||
|
||||
This config file is not needed by default. You can just touch it (to be empty) to get rid of the error message displayed when missing.
|
||||
|
||||
The whole idea came up when I was evaluating the documentation involved in maintaining a complex configuration system. My intention was to give configuration options for everything imaginable. But then, documenting all these options would require a tremendous amount of time, users would have to search through endless pages for the option they need, etc.
|
||||
|
||||
I concluded then that configuring software like that is a waste for time and effort. Of course there must be plenty of configuration options, but the implementation itself should require a lot less effort for both the devs and the users.
|
||||
|
||||
So, I did this:
|
||||
|
||||
1. No configuration is required to run netdata
|
||||
2. There are plenty of options to tweak
|
||||
3. There is minimal documentation (or no at all)
|
||||
|
||||
### Why this works?
|
||||
|
||||
The configuration file is a `name = value` dictionary with `[sections]`. Write whatever you like there as long as it follows this simple format.
|
||||
|
||||
Netdata loads this dictionary and then when the code needs a value from it, it just looks up the `name` in the dictionary at the proper `section`. In all places, in the code, there are both the `names` and their `default values`, so if something is not found in the configuration file, the default is used. The lookup is made using B-Trees and hashes (no string comparisons), so they are super fast. Also the `names` of the settings can be `my super duper setting that once set to yes, will turn the world upside down = no` - so goodbye to most of the documentation involved.
|
||||
|
||||
Next, netdata can generate a valid configuration for the user to edit. No need to remember anything. Just get the configuration from the server (`/netdata.conf` on your netdata server), edit it and save it.
|
||||
|
||||
Last, what about options you believe you have set, but you misspelled? When you get the configuration file from the server, there will be a comment above all `name = value` pairs the server does not use. So you know that whatever you wrote there, is not used.
|
||||
|
||||
### Limiting access to netdata.conf
|
||||
|
||||
netdata v1.9+ limit by default access to `http://your.netdata.ip:19999/netdata.conf` to private IP addresses. This is controlled by this settings:
|
||||
|
||||
```
|
||||
[web]
|
||||
allow netdata.conf from = localhost fd* 10.* 192.168.* 172.16.* 172.17.* 172.18.* 172.19.* 172.20.* 172.21.* 172.22.* 172.23.* 172.24.* 172.25.* 172.26.* 172.27.* 172.28.* 172.29.* 172.30.* 172.31.*
|
||||
```
|
||||
|
||||
The IPs listed are all the private IPv4 addresses, including link local IPv6 addresses.
|
||||
|
||||
> Keep in mind that connections to netdata API ports are filtered by `[web].allow connections from`. So, IPs allowed by `[web].allow netdata.conf from` should also be allowed by `[web].allow connections from`.
|
||||
|
||||
|
||||
## Netdata simple patterns
|
||||
|
||||
Unix prefers regular expressions. But they are just too hard, too cryptic to use, write and understand.
|
||||
|
||||
So, netdata supports [simple patterns](../../libnetdata/simple_pattern/).
|
||||
|
||||
## Applying changes
|
||||
|
||||
After `netdata.conf` has been modified, netdata needs to be restarted for changes to apply:
|
||||
|
|
@ -173,3 +35,110 @@ sudo killall netdata; sleep 10; sudo netdata
|
|||
```
|
||||
|
||||
Please note that your data history will be lost if you have modified `history` parameter in section `[global]`.
|
||||
|
||||
## Sections
|
||||
|
||||
### [global] section options
|
||||
|
||||
setting | default | info
|
||||
:------:|:-------:|:----
|
||||
process scheduling policy | `keep` | See [netdata process scheduling policy](../#netdata-process-scheduling-policy)
|
||||
OOM score | `1000` | See [OOM score](../#oom-score)
|
||||
glibc malloc arena max for plugins | `1` | See [Virtual memory](../#virtual-memory).
|
||||
glibc malloc arena max for netdata | `1` | See [Virtual memory](../#virtual-memory).
|
||||
hostname | auto-detected | The hostname of the computer running netdata.
|
||||
history | `3996` | The number of entries the netdata daemon will by default keep in memory for each chart dimension. This setting can also be configured per chart. Check [Memory Requirements](../../database/#database) for more information.
|
||||
update every | `1` | The frequency in seconds, for data collection. For more information see [Performance](../../docs/Performance.md#performance).
|
||||
config directory | `/etc/netdata` | The directory configuration files are kept.
|
||||
stock config directory | `/usr/lib/netdata/conf.d` |
|
||||
log directory | `/var/log/netdata` | The directory in which the [log files](../#log-files) are kept.
|
||||
web files directory | `/usr/share/netdata/web` | The directory the web static files are kept.
|
||||
cache directory | `/var/cache/netdata` | The directory the memory database will be stored if and when netdata exits. Netdata will re-read the database when it will start again, to continue from the same point.
|
||||
lib directory | `/var/lib/netdata` | Contains the alarm log and the netdata instance guid.
|
||||
home directory | `/var/cache/netdata` | Contains the db files for the collected metrics
|
||||
plugins directory | `"/usr/libexec/netdata/plugins.d" "/etc/netdata/custom-plugins.d"` | The directory plugin programs are kept. This setting supports multiple directories, space separated. If any directory path contains spaces, enclose it in single or double quotes.
|
||||
memory mode | `save` | When set to `save` netdata will save its round robin database on exit and load it on startup. When set to `map` the cache files will be updated in real time (check `man mmap` - do not set this on systems with heavy load or slow disks - the disks will continuously sync the in-memory database of netdata). When set to `ram` the round robin database will be temporary and it will be lost when netdata exits. `none` disables the database at this host. This also disables health monitoring (there cannot be health monitoring without a database). host access prefix | | This is used in docker environments where /proc, /sys, etc have to be accessed via another path. You may also have to set SYS_PTRACE capability on the docker for this work. Check [issue 43](https://github.com/netdata/netdata/issues/43).
|
||||
memory deduplication (ksm) | `yes` | When set to `yes`, netdata will offer its in-memory round robin database to kernel same page merging (KSM) for deduplication. For more information check [Memory Deduplication - Kernel Same Page Merging - KSM](../../database/#ksm)
|
||||
TZ environment variable | `:/etc/localtime` | Where to find the timezone
|
||||
timezone | auto-detected | The timezone retrieved from the environment variable
|
||||
debug flags | `0x0000000000000000` | Bitmap of debug options to enable. For more information check [Tracing Options](../#debugging).
|
||||
debug log | `/var/log/netdata/debug.log` | The filename to save debug information. This file will not be created is debugging is not enabled. You can also set it to `syslog` to send the debug messages to syslog, or `none` to disable this log. For more information check [Tracing Options](../#debugging).
|
||||
error log | `/var/log/netdata/error.log` | The filename to save error messages for netdata daemon and all plugins (`stderr` is sent here for all netdata programs, including the plugins). You can also set it to `syslog` to send the errors to syslog, or `none` to disable this log.
|
||||
access log | `/var/log/netdata/access.log` | The filename to save the log of web clients accessing netdata charts. You can also set it to `syslog` to send the access log to syslog, or `none` to disable this log.
|
||||
errors flood protection period | `1200` | UNUSED - Length of period (in sec) during which the number of errors should not exceed the `errors to trigger flood protection`.
|
||||
errors to trigger flood protection | `200` | UNUSED - Number of errors written to the log in `errors flood protection period` sec before flood protection is activated.
|
||||
run as user | `netdata` | The user netdata will run as.
|
||||
pthread stack size | auto-detected |
|
||||
cleanup obsolete charts after seconds | `3600` | See [monitoring ephemeral containers](../../collectors/cgroups.plugin/#monitoring-ephemeral-containers)
|
||||
gap when lost iterations above | `1` |
|
||||
cleanup orphan hosts after seconds | `3600` | How long to wait until automatically removing from the DB a remote netdata host (slave) that is no longer sending data.
|
||||
delete obsolete charts files | `yes` | See [monitoring ephemeral containers](../../collectors/cgroups.plugin/#monitoring-ephemeral-containers)
|
||||
delete orphan hosts files | `yes` | Set to `no` to disable non-responsive host removal.
|
||||
|
||||
### [web] section options
|
||||
|
||||
Refer to the [web server documentation](../../web/server)
|
||||
|
||||
### [plugins] section options
|
||||
|
||||
In this section you will see be a boolean (`yes`/`no`) option for each plugin (e.g. tc, cgroups, apps, proc etc.). Note that the configuration options in this section for the orchestrator plugins `python.d`, `charts.d` and `node.d` control **all the modules** written for that orchestrator. For instance, setting `python.d = no` means that all Python modules under `collectors/python.d.plugin` will be disabled.
|
||||
|
||||
Additionally, there will be the following options:
|
||||
|
||||
setting | default | info
|
||||
:------:|:-------:|:----
|
||||
PATH environment variable | `auto-detected` |
|
||||
PYTHONPATH environment variable | | Used to set a custom python path
|
||||
enable running new plugins | `yes` | When set to `yes`, netdata will enable detected plugins, even if they are not configured explicitly. Setting this to `no` will only enable plugins explicitly configirued in this file with a `yes`
|
||||
check for new plugins every | 60 | The time in seconds to check for new plugins in the plugins directory. This allows having other applications dynamically creating plugins for netdata.
|
||||
checks | `no` | This is a debugging plugin for the internal latency
|
||||
|
||||
### [health] section options
|
||||
|
||||
This section controls the general behavior of the health monitoring capabilities of Netdata.
|
||||
|
||||
Specific alarms are configured in per-collector config files under the `health.d` directory. For more info, see [health monitoring](../../health/#health-monitoring).
|
||||
|
||||
[Alarm notifications](../../health/notifications/#netdata-alarm-notifications) are configured in `health_alarm_notify.conf`.
|
||||
|
||||
setting | default | info
|
||||
:------:|:-------:|:----
|
||||
enabled | `yes` | Set to `no` to disable all alarms and notifications
|
||||
in memory max health log entries | 1000 | Size of the alarm history held in RAM
|
||||
script to execute on alarm | `/usr/libexec/netdata/plugins.d/alarm-notify.sh` | The script that sends alarm notifications.
|
||||
stock health configuration directory | `/usr/lib/netdata/conf.d/health.d` | Contains the stock alarm configuration files for each collector
|
||||
health configuration directory | `/etc/netdata/health.d` | The directory containing the user alarm configuration files, to override the stock configurations
|
||||
run at least every seconds | `10` | Controls how often all alarm conditions should be evaluated.
|
||||
postpone alarms during hibernation for seconds | `60` | Prevents false alarms. May need to be increased if you get alarms during hibernation.
|
||||
rotate log every lines | 2000 | Controls the number of alarm log entries stored in `<lib directory>/health-log.db`, where <lib directory> is the one configured in the [[global] section](#global-section-options)
|
||||
|
||||
### [registry] section options
|
||||
|
||||
To understand what this section is and how it should be configured, please refer to the [registry documentation](../../registry).
|
||||
|
||||
### [backend]
|
||||
|
||||
Refer to the [streaming and replication](../../streaming) documentation.
|
||||
|
||||
### Per plugin configuration
|
||||
|
||||
The configuration options for plugins appear in sections following the pattern `[plugin:NAME]`.
|
||||
|
||||
#### Internal plugins
|
||||
|
||||
Most internal plugins will provide additional options. Check [Internal Plugins](../../collectors/) for more information.
|
||||
|
||||
#### External plugins
|
||||
|
||||
External plugins will have only 2 options at `netdata.conf`:
|
||||
|
||||
setting | default | info
|
||||
:------:|:-------:|:----
|
||||
update every|the value of `[global].update every` setting|The frequency in seconds the plugin should collect values. For more information check [Performance](../../docs/Performance.md#performance).
|
||||
command options|*empty*|Additional command line options to pass to the plugin.
|
||||
|
||||
External plugins that need additional configuration may support a dedicated file in `/etc/netdata`. Check their documentation.
|
||||
|
||||
### Per chart configuration
|
||||
|
||||
In this section you will a separate subsection for each chart shown on the dashboard. You can control all aspects of a specific chart here. You can understand what each option does by reading [how charts are defined](../../collectors/plugins.d/#chart). If you don't know how to find the name of a chart, you can learn about it [here](../../docs/Charts.md).
|
||||
|
|
|
|||
|
|
@ -23,11 +23,14 @@ For most server systems, with a few hundred charts and a few thousand dimensions
|
|||
|
||||
To prove netdata scalability, check issue [#1323](https://github.com/netdata/netdata/issues/1323#issuecomment-265501668) where netdata collects 95.000 metrics per second, with 12% CPU utilization of a single core!
|
||||
|
||||
In embedded systems, if the netdata daemon is using a lot of CPU without any web clients accessing it, you should lower the data collection frequency. To set the data collection frequency, edit `/etc/netdata/netdata.conf` and set `update_every` to a higher number (this is the frequency in seconds data are collected for all charts: higher number of seconds = lower frequency, the default is 1 for per second data collection). You can also set this frequency per module or chart. Check the **[[Configuration]]** section.
|
||||
In embedded systems, if the netdata daemon is using a lot of CPU without any web clients accessing it, you should lower the data collection frequency. To set the data collection frequency, edit `/etc/netdata/netdata.conf` and set `update_every` to a higher number (this is the frequency in seconds data are collected for all charts: higher number of seconds = lower frequency, the default is 1 for per second data collection). You can also set this frequency per module or chart. Check the [daemon configuration](../daemon/config) for plugins and charts. For specific modules, the configuration needs to be changed in:
|
||||
- `python.d.conf` for [python](../collectors/python.d.plugin/#pythondplugin)
|
||||
- `node.d.conf` for [nodejs](../collectors/node.d.plugin/#nodedplugin)
|
||||
- `charts.d.conf` for [bash](../collectors/charts.d.plugin/#chartsdplugin)
|
||||
|
||||
## Plugins
|
||||
|
||||
If a plugin is using a lot of CPU, you should lower its update frequency, or if you wrote it, re-factor it to be more CPU efficient. Check **[[External Plugins]]** for more details on writing plugins.
|
||||
If a plugin is using a lot of CPU, you should lower its update frequency, or if you wrote it, re-factor it to be more CPU efficient. Check [External Plugins](../collectors/plugins.d/) for more details on writing plugins.
|
||||
|
||||
## CPU consumption when web clients are accessing dashboards
|
||||
|
||||
|
|
@ -46,28 +49,185 @@ Netdata, while running, does not depend on disk I/O (apart its log files and `ac
|
|||
|
||||
Keep in mind that netdata saves its database when it exits and loads it back when restarted. While it is running though, its DB is only stored in RAM and no I/O takes place for it.
|
||||
|
||||
## Netdata process priority
|
||||
|
||||
By default, netdata runs with the `idle` process scheduler, which assigns CPU resources to netdata, only when the system has such resources to spare.
|
||||
|
||||
The following `netdata.conf` settings control this:
|
||||
|
||||
```
|
||||
[global]
|
||||
process scheduling policy = idle
|
||||
process scheduling priority = 0
|
||||
process nice level = 19
|
||||
```
|
||||
|
||||
The policies supported by netdata are `idle` (the netdata default), `other` (also as `nice`), `batch`, `rr`, `fifo`. netdata also recognizes `keep` and `none` to keep the current settings without changing them.
|
||||
|
||||
For `other`, `nice` and `batch`, the setting `process nice level = 19` is activated to configure the nice level of netdata. Nice gets values -20 (highest) to 19 (lowest).
|
||||
|
||||
For `rr` and `fifo`, the setting `process scheduling priority = 0` is activated to configure the priority of the relative scheduling policy. Priority gets values 1 (lowest) to 99 (highest).
|
||||
|
||||
For the details of each scheduler, see `man sched_setscheduler` and `man sched`.
|
||||
|
||||
When netdata is running under systemd, it can only lower its priority (the default is `other` with `nice level = 0`). If you want to make netdata to get more CPU than that, you will need to set in `netdata.conf`:
|
||||
|
||||
```
|
||||
[global]
|
||||
process scheduling policy = keep
|
||||
```
|
||||
|
||||
and edit `/etc/systemd/system/netdata.service` and add:
|
||||
|
||||
```
|
||||
CPUSchedulingPolicy=other | batch | idle | fifo | rr
|
||||
CPUSchedulingPriority=99
|
||||
Nice=-10
|
||||
```
|
||||
|
||||
## Running netdata in embedded devices
|
||||
|
||||
Embedded devices usually have very limited CPU resources available, and in most cases, just a single core.
|
||||
|
||||
> keep in mind that netdata on RPi 2 and 3 does not require any tuning. The default settings will be good. The following tunables apply only when running netdata on RPi 1 or other very weak IoT devices.
|
||||
|
||||
We suggest to do the following:
|
||||
|
||||
#### external plugins
|
||||
### 1. Disable External plugins
|
||||
|
||||
`charts.d.plugin` and `apps.plugin`, each consumes twice the CPU resources of the netdata daemon.
|
||||
External plugins can consume more system resources than the netdata server. Disable the ones you don't need. If you need them, increase their `update every` value (again in `/etc/netdata/netdata.conf`), so that they do not run that frequently.
|
||||
|
||||
If you don't need them, disable them (edit `/etc/netdata/netdata.conf` and search for the plugins section).
|
||||
Edit `/etc/netdata/netdata.conf`, find the `[plugins]` section:
|
||||
|
||||
If you need them, increase their `update every` value (again in `/etc/netdata/netdata.conf`), so that they do not run that frequently.
|
||||
```
|
||||
[plugins]
|
||||
proc = yes
|
||||
|
||||
#### internal plugins
|
||||
tc = no
|
||||
idlejitter = no
|
||||
cgroups = no
|
||||
checks = no
|
||||
apps = no
|
||||
charts.d = no
|
||||
node.d = no
|
||||
|
||||
plugins directory = /usr/libexec/netdata/plugins.d
|
||||
enable running new plugins = no
|
||||
check for new plugins every = 60
|
||||
```
|
||||
|
||||
In detail:
|
||||
|
||||
plugin|description
|
||||
:---:|:---------
|
||||
`proc`|the internal plugin used to monitor the system. Normally, you don't want to disable this. You can disable individual functions of it at the next section.
|
||||
`tc`|monitoring network interfaces QoS (tc classes)
|
||||
`idlejitter`|internal plugin (written in C) that attempts show if the systems starved for CPU. Disabling it will eliminate a thread.
|
||||
`cgroups`|monitoring linux containers. Most probably you are not going to need it. This will also eliminate another thread.
|
||||
`checks`|a debugging plugin, which is disabled by default.
|
||||
`apps`|a plugin that monitors system processes. It is very complex and heavy (consumes twice the CPU resources of the netdata daemon), so if you don't need to monitor the process tree, you can disable it.
|
||||
`charts.d`|BASH plugins (squid, nginx, mysql, etc). This is a heavy plugin, that consumes twice the CPU resources of the netdata daemon.
|
||||
`node.d`|node.js plugin, currently used for SNMP data collection and monitoring named (the name server).
|
||||
|
||||
For most IoT devices, you can disable all plugins except `proc`. For `proc` there is another section that controls which functions of it you need. Check the next section.
|
||||
|
||||
---
|
||||
|
||||
### 2. Disable internal plugins
|
||||
|
||||
In this section you can select which modules of the `proc` plugin you need. All these are run in a single thread, one after another. Still, each one needs some RAM and consumes some CPU cycles.
|
||||
|
||||
```
|
||||
[plugin:proc]
|
||||
# /proc/net/dev = yes # network interfaces
|
||||
# /proc/diskstats = yes # disks
|
||||
# /proc/net/snmp = yes # generic IPv4
|
||||
# /proc/net/snmp6 = yes # generic IPv6
|
||||
# /proc/net/netstat = yes # TCP and UDP
|
||||
# /proc/net/stat/conntrack = yes # firewall
|
||||
# /proc/net/ip_vs/stats = yes # IP load balancer
|
||||
# /proc/net/stat/synproxy = yes # Anti-DDoS
|
||||
# /proc/stat = yes # CPU, context switches
|
||||
# /proc/meminfo = yes # Memory
|
||||
# /proc/vmstat = yes # Memory operations
|
||||
# /proc/net/rpc/nfsd = yes # NFS Server
|
||||
# /proc/sys/kernel/random/entropy_avail = yes # Cryptography
|
||||
# /proc/interrupts = yes # Interrupts
|
||||
# /proc/softirqs = yes # SoftIRQs
|
||||
# /proc/loadavg = yes # Load Average
|
||||
# /sys/kernel/mm/ksm = yes # Memory deduper
|
||||
# netdata server resources = yes # netdata charts
|
||||
```
|
||||
|
||||
### 3. Lower internal plugin update frequency
|
||||
|
||||
If netdata is still using a lot of CPU, lower its update frequency. Going from per second updates, to once every 2 seconds updates, will cut the CPU resources of all netdata programs **in half**, and you will still have very frequent updates.
|
||||
|
||||
If the CPU of the embedded device is too weak, try setting even lower update frequency. Experiment with `update every = 5` or `update every = 10` (higher number = lower frequency), until you get acceptable results.
|
||||
If the CPU of the embedded device is too weak, try setting even lower update frequency. Experiment with `update every = 5` or `update every = 10` (higher number = lower frequency) in `netdata.conf`, until you get acceptable results.
|
||||
|
||||
#### Single threaded web server
|
||||
Keep in mind this will also force dashboard chart refreshes to happen at the same rate. So increasing this number actually lowers data collection frequency but also lowers dashboard chart refreshes frequency.
|
||||
|
||||
This is a dashboard on a device with `[global].update every = 5` (this device is a media player and is now playing a movie):
|
||||
|
||||
|
||||
|
||||
### 4. Disable logs
|
||||
|
||||
Normally, you will not need them. To disable them, set:
|
||||
|
||||
```
|
||||
[global]
|
||||
debug log = none
|
||||
error log = none
|
||||
access log = none
|
||||
```
|
||||
### 5. Set memory mode to RAM
|
||||
|
||||
Setting the memory mode to `ram` will disable loading and saving the round robin database. This will not affect anything while running netdata, but it might be required if you have very limited storage available.
|
||||
|
||||
```
|
||||
[global]
|
||||
memory mode = ram
|
||||
```
|
||||
|
||||
### 6. Use the single threaded web server
|
||||
|
||||
Normally, netdata spawns a thread for each web client. This allows netdata to utilize all the available cores for servicing chart refreshes. You can however disable this feature and serve all charts one after another, using a single thread / core. This will might lower the CPU pressure on the embedded device. To enable the single threaded web server, edit `/etc/netdata/netdata.conf` and set `mode = single-threaded` in the `[web]` section.
|
||||
|
||||
### 7. Lower memory requirements
|
||||
|
||||
You can set the default size of the round robin database for all charts, using:
|
||||
|
||||
```
|
||||
[global]
|
||||
history = 600
|
||||
```
|
||||
|
||||
The units for history is `[global].update every` seconds. So if `[global].update every = 6` and `[global].history = 600`, you will have an hour of data ( 6 x 600 = 3.600 ), which will store 600 points per dimension, one every 6 seconds.
|
||||
|
||||
Check also [[Memory Requirements]] for directions on calculating the size of the round robin database.
|
||||
|
||||
|
||||
### 8. Disable gzip compression of responses
|
||||
|
||||
Gzip compression of the web responses is using more CPU that the rest of netdata. You can lower the compression level or disable gzip compression completely. You can disable it, like this:
|
||||
|
||||
```
|
||||
[web]
|
||||
enable gzip compression = no
|
||||
```
|
||||
|
||||
To lower the compression level, do this:
|
||||
|
||||
```
|
||||
[web]
|
||||
enable gzip compression = yes
|
||||
gzip compression level = 1
|
||||
```
|
||||
|
||||
Finally, if no web server is installed on your device, you can use port tcp/80 for netdata:
|
||||
|
||||
```
|
||||
[web]
|
||||
port = 80
|
||||
```
|
||||
|
|
|
|||
116
docs/configuration-guide.md
Normal file
116
docs/configuration-guide.md
Normal file
|
|
@ -0,0 +1,116 @@
|
|||
# Configuration guide
|
||||
|
||||
No configuration is required to run netdata, but you fill find plenty of options to tweak, so that you can adapt it to your particular needs.
|
||||
|
||||
<details markdown="1"><summary>Configuration files are placed in `/etc/netdata`.</summary>
|
||||
Depending on your installation method, Netdata will have been installed either directly under `/`, or under `/opt/netdata`. The paths mentioned here and in the documentation in general assume that your installation is under `/`. If it is not, you will find the exact same paths under `/opt/netdata` as well. (i.e. `/etc/netdata` will be `/opt/netdata/etc/netdata`).</details>
|
||||
|
||||
Under that directory you will see the following:
|
||||
|
||||
- `netdata.conf` is [the main configuration file](../daemon/config/#daemon-configuration)
|
||||
- `edit-config` is an sh script that you can use to easily and safely edit the configuration. Just run it to see its usage.
|
||||
- Other directories, initially empty, where your custom configurations for alarms and collector plugins/modules will be copied from the stock configuration, if and when you customize them using `edit-config`.
|
||||
- `orig` is a symbolic link to the directory `/usr/lib/netdata/conf.d`, which contains the stock configurations for everything not included in `netdata.conf`:
|
||||
- `health_alarm_notify.conf` is where you configure how and to who Netdata will send [alarm notifications](../health/notifications/#netdata-alarm-notifications).
|
||||
- `health.d` is the directory that contains the alarm triggers for [health monitoring](../health/#health-monitoring). It contains one .conf file per collector.
|
||||
- The [modular plugin orchestrators](../collectors/plugins.d/#external-plugins-overview) have:
|
||||
- One config file each, mainly to turn their modules on and off: `python.d.conf` for [python](../collectors/python.d.plugin/#pythondplugin), `node.d.conf` for [nodejs](../collectors/node.d.plugin/#nodedplugin) and `charts.d.conf` for [bash](../collectors/charts.d.plugin/#chartsdplugin) modules.
|
||||
- One directory each, where the module-specific configuration files can be found.
|
||||
- `stream.conf` is where you configure [streaming and replication](../streaming/#streaming-and-replication)
|
||||
- `stats.d` is a directory under which you can add .conf files to add [synthetic charts](../collectors/statsd.plugin/#synthetic-statsd-charts).
|
||||
- Individual collector plugin config files, such as `fping.conf` for the [fping plugin](../collectors/fping.plugin/) and `apps_groups.conf` for the [apps plugin](../collectors/apps.plugin/)
|
||||
|
||||
So there are many configuration files to control every aspect of Netdata's behavior. It can be overwhelming at first, but you won't have to deal with any of them, unless you have specific things you need to change. The following HOWTO will guide you on how to customize your netdata, based on what you want to do.
|
||||
|
||||
## How to
|
||||
|
||||
### Change what I see
|
||||
|
||||
##### Increase the metrics retention period
|
||||
|
||||
Increase `history` in [netdata.conf [global]](../daemon/config/#global-section-options). Just ensure you understand [how much memory will be required](../database/)
|
||||
|
||||
##### Reduce the data collection frequency
|
||||
|
||||
Increase `update every` in [netdata.conf [global]](../daemon/config/#global-section-options). This is another way to increase your metrics retention period, but at a lower resolution than the default 1s.
|
||||
|
||||
##### Modify how a chart is displayed
|
||||
|
||||
In `netdata.conf` under `# Per chart configuration` you will find several [[CHART_NAME] sections](../daemon/config/#per-chart-configuration), where you can control all aspects of a specific chart.
|
||||
|
||||
##### Disable a collector
|
||||
|
||||
Entire plugins can be turned off from the [netdata.conf [plugins]](../daemon/config/#plugins-section-options) section. To disable specific modules of a plugin orchestrator, you need to edit one of the following:
|
||||
- `python.d.conf` for [python](../collectors/python.d.plugin/#pythondplugin)
|
||||
- `node.d.conf` for [nodejs](../collectors/node.d.plugin/#nodedplugin)
|
||||
- `charts.d.conf` for [bash](../collectors/charts.d.plugin/#chartsdplugin)
|
||||
|
||||
### Modify alarms and notifications
|
||||
|
||||
##### Turn off all alarms and notifications
|
||||
|
||||
Just set `enabled = no` in the [netdata.conf [health]](../daemon/config/#health-section-options) section
|
||||
|
||||
##### Modify or disable a specific alarm
|
||||
|
||||
The `health.d` directory that contains the alarm triggers for [health monitoring](../health/#health-monitoring). It has one .conf file per collector. You can easily find the .conf file you will need to modify, by looking for the "source" line on the table that appears on the right side of an alarm on the netdata gui.
|
||||
|
||||
For example, if you click on Alarms and go to the tab 'All', the default netdata installation will show you at the top the configured alarm for `10 min cpu usage` (it's the name of the badge). Looking at the table on the right side, you will see a row that says: `source 4@/usr/lib/netdata/conf.d/health.d/cpu.conf`. This way, you know that you will need to run `/etc/netdata/edit-config health.d/cpu.conf` and look for alarm at line 4 of the conf file.
|
||||
|
||||
As stated at the top of the .conf file, **you can disable an alarm notification by setting the 'to' line to: silent**.
|
||||
To modify how the alarm gets triggered, we suggest that you go through the guide on [health monitoring](../health/#health-monitoring).
|
||||
|
||||
##### Receive notifications using my preferred method
|
||||
|
||||
You only need to configure `health_alarm_notify.conf`. To learn how to do it, read first [alarm notifications](../health/notifications/#netdata-alarm-notifications) and then open the submenu `Supported Notifications` under `Alarm notifications` in the documentation to find the specific page on your prefered notification method.
|
||||
|
||||
### Make security-related customizations
|
||||
|
||||
##### Change the netdata web server access lists
|
||||
|
||||
You have several options under the [netdata.conf [web]](../web/server/#access-lists) section.
|
||||
|
||||
##### Stop sending info to registry.my-netdata.io
|
||||
|
||||
You will need to configure the [registry] section in netdata.conf. First read the [registry documentation](../registry/). In it, are instructions on how to [run your own registry](../registry/#run-your-own-registry).
|
||||
|
||||
##### Change the IP address/port netdata listens to
|
||||
|
||||
The settings are under netdata.conf [web]. Look at the [web server documentation](../web/server/#binding-netdata-to-multiple-ports) for more info.
|
||||
|
||||
### System resource usage
|
||||
|
||||
##### Reduce the resources netdata uses
|
||||
|
||||
The page on [netdata performance](Performance.md) has an excellent guide on how to reduce the netdata cpu/disk/RAM utilization to levels suitable even for the weakest [IoT devices](netdata-for-IoT.md).
|
||||
|
||||
##### Change when netdata saves metrics to disk
|
||||
|
||||
[netdata.conf [global]](../daemon/config/#global-section-options) : `memory mode`</details>
|
||||
|
||||
##### Prevent netdata from getting immediately killed when my server runs out of memory
|
||||
|
||||
You can change the netdata [OOM score](../daemon/#oom-score) in netdata.conf [global].
|
||||
|
||||
### Other
|
||||
|
||||
##### Move netdata directories
|
||||
|
||||
The various directory paths are in [netdata.conf [global]](../daemon/config/#global-section-options).
|
||||
|
||||
|
||||
## How netdata configuration works
|
||||
|
||||
The configuration files are `name = value` dictionaries with `[sections]`. Write whatever you like there as long as it follows this simple format.
|
||||
|
||||
Netdata loads this dictionary and then when the code needs a value from it, it just looks up the `name` in the dictionary at the proper `section`. In all places, in the code, there are both the `names` and their `default values`, so if something is not found in the configuration file, the default is used. The lookup is made using B-Trees and hashes (no string comparisons), so they are super fast. Also the `names` of the settings can be `my super duper setting that once set to yes, will turn the world upside down = no` - so goodbye to most of the documentation involved.
|
||||
|
||||
Next, netdata can generate a valid configuration for the user to edit. No need to remember anything. Just get the configuration from the server (`/netdata.conf` on your netdata server), edit it and save it.
|
||||
|
||||
Last, what about options you believe you have set, but you misspelled?When you get the configuration file from the server, there will be a comment above all `name = value` pairs the server does not use. So you know that whatever you wrote there, is not used.
|
||||
|
||||
## Netdata simple patterns
|
||||
|
||||
Unix prefers regular expressions. But they are just too hard, too cryptic to use, write and understand.
|
||||
|
||||
So, netdata supports [simple patterns](../libnetdata/simple_pattern/).
|
||||
|
|
@ -19,6 +19,9 @@ echo "Copying files"
|
|||
rm -rf ${GENERATOR_DIR}/src
|
||||
find . -type d \( -path ./${GENERATOR_DIR} -o -path ./node_modules \) -prune -o -name "*.md" -print | cpio -pd ${GENERATOR_DIR}/src
|
||||
|
||||
# Copy netdata html resources
|
||||
cp -a ./${GENERATOR_DIR}/custom ./${GENERATOR_DIR}/src/
|
||||
|
||||
# Modify the first line of the main README.md, to enable proper static html generation
|
||||
echo "Modifying README header"
|
||||
sed -i -e '0,/# netdata /s//# Introduction\n\n/' -e 's/\[!\[analytics.*UA-64295674-3)\]()//g' ${GENERATOR_DIR}/src/README.md
|
||||
|
|
@ -44,9 +47,11 @@ echo "Fixing links"
|
|||
# Fix links (recursively, all types, executing replacements)
|
||||
${GENERATOR_DIR}/checklinks.sh -rax
|
||||
|
||||
echo "Calling mkdocs"
|
||||
if [ "${1}" != "nomkdocs" ] ; then
|
||||
echo "Calling mkdocs"
|
||||
|
||||
# Build html docs
|
||||
mkdocs build --config-file=${GENERATOR_DIR}/mkdocs.yml
|
||||
# Build html docs
|
||||
mkdocs build --config-file=${GENERATOR_DIR}/mkdocs.yml
|
||||
fi
|
||||
|
||||
echo "Finished"
|
||||
|
|
|
|||
|
|
@ -63,11 +63,13 @@ extra:
|
|||
link: "https://www.facebook.com/linuxnetdata/"
|
||||
theme:
|
||||
name: "material"
|
||||
custom_dir: themes/material
|
||||
custom_dir: custom/themes/material
|
||||
favicon: custom/img/favicon.ico
|
||||
extra_css:
|
||||
- "https://cdnjs.cloudflare.com/ajax/libs/cookieconsent2/3.1.0/cookieconsent.min.css"
|
||||
- "custom/css/netdata.css"
|
||||
extra_javascript:
|
||||
- "javascripts/cookie-consent.js"
|
||||
- "custom/javascripts/cookie-consent.js"
|
||||
- "https://cdnjs.cloudflare.com/ajax/libs/cookieconsent2/3.1.0/cookieconsent.min.js"
|
||||
markdown_extensions:
|
||||
- extra
|
||||
|
|
@ -123,23 +125,17 @@ echo -ne " - 'docs/Why-Netdata.md'
|
|||
- REDISTRIBUTED.md
|
||||
- CHANGELOG.md
|
||||
- CONTRIBUTING.md
|
||||
"
|
||||
|
||||
echo -ne "- Installation:
|
||||
- Installation:
|
||||
- 'installer/README.md'
|
||||
- 'packaging/docker/README.md'
|
||||
- 'installer/UPDATE.md'
|
||||
- 'installer/UNINSTALL.md'
|
||||
"
|
||||
|
||||
echo -ne "- 'docs/GettingStarted.md'
|
||||
"
|
||||
|
||||
echo -ne "- Running netdata:
|
||||
"
|
||||
navpart 2 daemon
|
||||
navpart 2 daemon/config
|
||||
echo -ne " - 'docs/Charts.md'
|
||||
- 'docs/GettingStarted.md'
|
||||
- Running netdata:
|
||||
- 'daemon/README.md'
|
||||
- 'docs/configuration-guide.md'
|
||||
- 'daemon/config/README.md'
|
||||
- 'docs/Charts.md'
|
||||
"
|
||||
navpart 2 web/server "" "Web server"
|
||||
navpart 3 web/server "" "" 2 excludefirstlevel
|
||||
|
|
|
|||
|
|
@ -119,7 +119,7 @@ testinternal () {
|
|||
ilnk=${3}
|
||||
header=${ilnk//-/}
|
||||
dbg "-- Searching for \"$header\" in $ifile"
|
||||
tr -d ',_.:? `'< "$ifile" | sed 's/-//g' | grep -i "^\\#*$header\$" >/dev/null
|
||||
tr -d '[],_.:? `'< "$ifile" | sed 's/-//g' | grep -i "^\\#*$header\$" >/dev/null
|
||||
if [ $? -eq 0 ] ; then
|
||||
dbg "-- $ilnk found in $ifile"
|
||||
return 0
|
||||
|
|
|
|||
3
docs/generator/custom/css/netdata.css
Normal file
3
docs/generator/custom/css/netdata.css
Normal file
|
|
@ -0,0 +1,3 @@
|
|||
.md-nav__link {
|
||||
white-space: nowrap;
|
||||
}
|
||||
BIN
docs/generator/custom/img/favicon.ico
Normal file
BIN
docs/generator/custom/img/favicon.ico
Normal file
Binary file not shown.
|
After 
Width: 16px | Height: 16px | Size: 1.1 KiB |
|
|
@ -20,169 +20,9 @@ The netdata web API already provides **reduce** functions allowing it to report
|
|||
|
||||
|
||||
|
||||
Although netdata has been significantly optimized to lower the CPU and RAM resources it consumes, the plethora of data collection plugins may be inappropriate for weak IoT devices.
|
||||
Although netdata has been significantly optimized to lower the CPU and RAM resources it consumes, the plethora of data collection plugins may be inappropriate for weak IoT devices. Please follow the guide on [running netdata in embedded devices](Performance.md)
|
||||
|
||||
> keep in mind that netdata on RPi 2 and 3 does not require any tuning. The default settings will be good. The following tunables apply only when running netdata on RPi 1 or other very weak IoT devices.
|
||||
|
||||
Here are a few tricks to control the resources consumed by netdata:
|
||||
|
||||
## 1. Disable External plugins
|
||||
|
||||
External plugins can consume more system resources than the netdata server. Disable the ones you don't need.
|
||||
|
||||
Edit `/etc/netdata/netdata.conf`, find the `[plugins]` section:
|
||||
|
||||
```
|
||||
[plugins]
|
||||
proc = yes
|
||||
|
||||
tc = no
|
||||
idlejitter = no
|
||||
cgroups = no
|
||||
checks = no
|
||||
apps = no
|
||||
charts.d = no
|
||||
node.d = no
|
||||
|
||||
plugins directory = /usr/libexec/netdata/plugins.d
|
||||
enable running new plugins = no
|
||||
check for new plugins every = 60
|
||||
```
|
||||
|
||||
In detail:
|
||||
|
||||
plugin|description
|
||||
:---:|:---------
|
||||
`proc`|the internal plugin used to monitor the system. Normally, you don't want to disable this. You can disable individual functions of it at the next section.
|
||||
`tc`|monitoring network interfaces QoS (tc classes)
|
||||
`idlejitter`|internal plugin (written in C) that attempts show if the systems starved for CPU. Disabling it will eliminate a thread.
|
||||
`cgroups`|monitoring linux containers. Most probably you are not going to need it. This will also eliminate another thread.
|
||||
`checks`|a debugging plugin, which is disabled by default.
|
||||
`apps`|a plugin that monitors system processes. It is very complex and heavy (heavier than the netdata daemon), so if you don't need to monitor the process tree, you can disable it.
|
||||
`charts.d`|BASH plugins (squid, nginx, mysql, etc). This is again a heavy plugin.
|
||||
`node.d`|node.js plugin, currently used for SNMP data collection and monitoring named (the name server).
|
||||
|
||||
For most IoT devices, you can disable all plugins except `proc`. For `proc` there is another section that controls which functions of it you need. Check the next section.
|
||||
|
||||
---
|
||||
|
||||
## 2. Disable internal plugins
|
||||
|
||||
In this section you can select which modules of the `proc` plugin you need. All these are run in a single thread, one after another. Still, each one needs some RAM and consumes some CPU cycles.
|
||||
|
||||
```
|
||||
[plugin:proc]
|
||||
# /proc/net/dev = yes # network interfaces
|
||||
# /proc/diskstats = yes # disks
|
||||
# /proc/net/snmp = yes # generic IPv4
|
||||
# /proc/net/snmp6 = yes # generic IPv6
|
||||
# /proc/net/netstat = yes # TCP and UDP
|
||||
# /proc/net/stat/conntrack = yes # firewall
|
||||
# /proc/net/ip_vs/stats = yes # IP load balancer
|
||||
# /proc/net/stat/synproxy = yes # Anti-DDoS
|
||||
# /proc/stat = yes # CPU, context switches
|
||||
# /proc/meminfo = yes # Memory
|
||||
# /proc/vmstat = yes # Memory operations
|
||||
# /proc/net/rpc/nfsd = yes # NFS Server
|
||||
# /proc/sys/kernel/random/entropy_avail = yes # Cryptography
|
||||
# /proc/interrupts = yes # Interrupts
|
||||
# /proc/softirqs = yes # SoftIRQs
|
||||
# /proc/loadavg = yes # Load Average
|
||||
# /sys/kernel/mm/ksm = yes # Memory deduper
|
||||
# netdata server resources = yes # netdata charts
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Disable logs
|
||||
|
||||
Normally, you will not need them. To disable them, set:
|
||||
|
||||
```
|
||||
[global]
|
||||
debug log = none
|
||||
error log = none
|
||||
access log = none
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Set memory mode to RAM
|
||||
|
||||
Setting the memory mode to `ram` will disable loading and saving the round robin database. This will not affect anything while running netdata, but it might be required if you have very limited storage available.
|
||||
|
||||
```
|
||||
[global]
|
||||
memory mode = ram
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. CPU utilization
|
||||
|
||||
If after disabling the plugins you don't need, netdata still uses a lot of CPU without any clients accessing the dashboard, try lowering its data collection frequency. Going from "once per second" to "once every two seconds" will not have a significant difference on the user experience, but it will cut the CPU resources required **in half**.
|
||||
|
||||
To set the update frequency, edit `/etc/netdata/netdata.conf` and set:
|
||||
|
||||
```
|
||||
[global]
|
||||
update every = 2
|
||||
```
|
||||
|
||||
You may have to increase this to 5 or 10 if the CPU of the device is weak.
|
||||
|
||||
Keep in mind this will also force dashboard chart refreshes to happen at the same rate. So increasing this number actually lowers data collection frequency but also lowers dashboard chart refreshes frequency.
|
||||
|
||||
This is a dashboard on a device with `[global].update every = 5` (this device is a media player and is now playing a movie):
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 6. Lower memory requirements
|
||||
|
||||
You can set the default size of the round robin database for all charts, using:
|
||||
|
||||
```
|
||||
[global]
|
||||
history = 600
|
||||
```
|
||||
|
||||
The units for history is `[global].update every` seconds. So if `[global].update every = 6` and `[global].history = 600`, you will have an hour of data ( 6 x 600 = 3.600 ), which will store 600 points per dimension, one every 6 seconds.
|
||||
|
||||
Check also [[Memory Requirements]] for directions on calculating the size of the round robin database.
|
||||
|
||||
---
|
||||
|
||||
## 7. Disable gzip compression of responses
|
||||
|
||||
Gzip compression of the web responses is using more CPU that the rest of netdata. You can lower the compression level or disable gzip compression completely. You can disable it, like this:
|
||||
|
||||
```
|
||||
[web]
|
||||
enable gzip compression = no
|
||||
```
|
||||
|
||||
To lower the compression level, do this:
|
||||
|
||||
```
|
||||
[web]
|
||||
enable gzip compression = yes
|
||||
gzip compression level = 1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
Finally, if no web server is installed on your device, you can use port tcp/80 for netdata:
|
||||
|
||||
```
|
||||
[global]
|
||||
port = 80
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Monitoring RPi temperature
|
||||
## Monitoring RPi temperature
|
||||
|
||||
The python version of the sensors plugin uses `lm-sensors`. Unfortunately the temperature reading of RPi are not supported by `lm-sensors`.
|
||||
|
||||
|
|
|
|||
|
|
@ -151,20 +151,11 @@ Of course, there are many more methods you could use to protect Netdata:
|
|||
|
||||
## Registry or how to not send any information to a third party server
|
||||
|
||||
The default configuration uses a public registry under registry.my-netdata.io (more information about the registry here: [mynetdata-menu-item](../registry/) ). Please be aware that if you use that public registry, you submit at least the following information to a third party server, which might violate your security policies:
|
||||
The default configuration uses a public registry under registry.my-netdata.io (more information about the registry here: [mynetdata-menu-item](../registry/) ). Please be aware that if you use that public registry, you submit the following information to a third party server:
|
||||
- The url where you open the web-ui in the browser (via http request referer)
|
||||
- The hostnames of the Netdata servers
|
||||
|
||||
You are able to run your own registry, which is pretty simple to do:
|
||||
- If you have just one Netdata web-ui, turn on registry and set the url of that web-ui as "registry to announce"
|
||||
```
|
||||
[registry]
|
||||
enabled = yes
|
||||
registry to announce = URL_OF_THE_NETDATA_WEB-UI
|
||||
```
|
||||
- If you run multiple Netdata servers with web-ui, you need to define one as registry. On that node activate the registry and setting its url as "registry to announce". On all other nodes do not enable the registry but define the same url.
|
||||
|
||||
restart Netdata and check with developer tools of your browser which registry is called.
|
||||
If sending this information to the central Netdata registry violates your security policies, you can configure Netdat to [run your own registry](../registry/#run-your-own-registry).
|
||||
|
||||
## Netdata directories
|
||||
|
||||
|
|
|
|||
|
|
@ -46,13 +46,13 @@ The registry keeps track of 3 entities:
|
|||
|
||||
## Who talks to the registry?
|
||||
|
||||
Your web browser **only**! Check here if this is against your policies: [how to not send any information to a thirdparty server](../docs/netdata-security.md#security-design)
|
||||
Your web browser **only**! If sending this information is against your policies, you can [run your own registry](#run-your-own-registry)
|
||||
|
||||
Your netdata servers do not talk to the registry. This is a UML diagram of its operation:
|
||||
|
||||
|
||||
|
||||
## What data the registry maintains?
|
||||
## What data does the registry store?
|
||||
|
||||
Its database contains:
|
||||
|
||||
|
|
@ -72,9 +72,9 @@ Yeap! The registry can handle 50.000 - 100.000 requests **per second per core**
|
|||
|
||||
We believe, it can do it...
|
||||
|
||||
## Every netdata can be a registry
|
||||
## Run your own registry
|
||||
|
||||
Yes, you read correct, **every netdata can be a registry**. Just pick one and configure it.
|
||||
**Every netdata can be a registry**. Just pick one and configure it.
|
||||
|
||||
**To turn any netdata into a registry**, edit `/etc/netdata/netdata.conf` and set:
|
||||
|
||||
|
|
|
|||
|
|
@ -82,11 +82,14 @@ monitoring (there cannot be health monitoring without a database).
|
|||
```
|
||||
[web]
|
||||
mode = none | static-threaded | single-threaded | multi-threaded
|
||||
accept a streaming request every seconds = 0
|
||||
```
|
||||
|
||||
`[web].mode = none` disables the API (netdata will not listen to any ports).
|
||||
This also disables the registry (there cannot be a registry without an API).
|
||||
|
||||
`accept a streaming request every seconds` can be used to set a limit on how often a master Netdata server will accept streaming requests from the slaves. 0 sets no limit, 1 means maximum once every second. If this is set, you may see error log entries "... too busy to accept new streaming request. Will be allowed in X secs".
|
||||
|
||||
```
|
||||
[backend]
|
||||
enabled = yes | no
|
||||
|
|
|
|||
|
|
@ -20,7 +20,6 @@ The size of the alarm log is configured in `netdata.conf`. There are 2 settings:
|
|||
|
||||
```
|
||||
[health]
|
||||
health db file = /var/lib/netdata/health/health-log.db
|
||||
in memory max health log entries = 1000
|
||||
rotate log every lines = 2000
|
||||
```
|
||||
|
|
|
|||
|
|
@ -90,7 +90,25 @@ Netdata supports access lists in `netdata.conf`:
|
|||
The setting in `netdata.conf` is checked before the ones in [stream.conf](../../streaming/stream.conf).
|
||||
|
||||
- `allow netdata.conf from` checks the IP to allow `http://netdata.host:19999/netdata.conf`.
|
||||
By default it allows only private lans.
|
||||
The IPs listed are all the private IPv4 addresses, including link local IPv6 addresses. Keep in mind that connections to netdata API ports are filtered by `allow connections from`. So, IPs allowed by `allow netdata.conf from` should also be allowed by `allow connections from`.
|
||||
|
||||
### Other netdata.conf [web] section options
|
||||
setting | default | info
|
||||
:------:|:-------:|:----
|
||||
ses max window | `15` | See [single exponential smoothing](../api/queries/des/)
|
||||
des max window | `15` | See [double exponential smoothing](../api/queries/des/)
|
||||
listen backlog | `4096` | The port backlog. Check `man 2 listen`.
|
||||
web files owner | `netdata` | The user that owns the web static files. Netdata will refuse to serve a file that is not owned by this user, even if it has read access to that file. If the user given is not found, netdata will only serve files owned by user given in `run as user`.
|
||||
web files group | `netdata` | If this is set, Netdata will check if the file is owned by this group and refuse to serve the file if it's not.
|
||||
disconnect idle clients after seconds | `60` | The time in seconds to disconnect web clients after being totally idle.
|
||||
timeout for first request | `60` | How long to wait for a client to send a request before closing the socket. Prevents slow request attacks.
|
||||
accept a streaming request every seconds | `0` | Can be used to set a limit on how often a master Netdata server will accept streaming requests from the slaves in a [streaming and replication setup](../../streaming)
|
||||
respect do not track policy | `no` | If set to `yes`, will respect the client's browser preferences on storing cookies.
|
||||
x-frame-options response header | | [Avoid clickjacking attacks, by ensuring that the content is not embedded into other sites](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Frame-Options).
|
||||
enable gzip compression | `yes` | When set to `yes`, netdata web responses will be GZIP compressed, if the web client accepts such responses.
|
||||
gzip compression strategy | `default` | Valid strategies are `default`, `filtered`, `huffman only`, `rle` and `fixed`
|
||||
gzip compression level | `3` | Valid levels are 1 (fastest) to 9 (best ratio)
|
||||
|
||||
|
||||
## DDoS protection
|
||||
|
||||
|
|
@ -101,3 +119,4 @@ If you publish your netdata to the internet, you may want to apply some protecti
|
|||
3. Don't use all your cpu cores for netdata (lower `[web].web server threads`)
|
||||
4. Run netdata with a low process scheduling priority (the default is the lowest)
|
||||
5. If possible, proxy netdata via a full featured web server (nginx, apache, etc)
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue