mirror of
https://github.com/netdata/netdata.git
synced 2025-04-14 09:38:34 +00:00
Htmldoc (#4607)
* First html documentation debug set * Test 2 * Relative path changed * Updated comments * Cleanup, installation draft added * fixes * test * test * test * First html documentation debug set * Test 2 * Relative path changed * Updated comments * Cleanup, installation draft added * fixes * test * test * test * First set of major cleanup/deduplication * 2nd major cleanup * update getting started structure * Cleanup in using netdata * Final cleanup/deduplication * Added initial CONTRIBUTING.md, updated some info related to contributing on the orchestrators * Removed Why-Netdata (included in new README in master), added link to CONTRIBUTING.md * First html documentation debug set * Updated Makefile.am to ignore the new md and htmldoc generation files * Removing files from rebase * First html documentation debug set * Test 2 * Relative path changed * Updated comments * Cleanup, installation draft added * fixes * test * test * test * First html documentation debug set * Test 2 * Relative path changed * Updated comments * Cleanup, installation draft added * test * test * First set of major cleanup/deduplication * 2nd major cleanup * update getting started structure * Cleanup in using netdata * Final cleanup/deduplication * Added initial CONTRIBUTING.md, updated some info related to contributing on the orchestrators * Removed Why-Netdata (included in new README in master), added link to CONTRIBUTING.md * First html documentation debug set * Updated Makefile.am to ignore the new md and htmldoc generation files * Removing files from rebase * Fixed Makefile.am * Same line header and badges * Fixed broken link * CPU monitoring is in apps plugin * Removed obsolete files * Remove obsolete files * - Make the Health API part of health/README.md new file web/api/health/README.md - Make installer/LAUNCH.md part of deamon/README.md - Move installer/MAINTAINERS.md to packaging/maintainers/README.md - Move installer/DOCKER.md to docker/README.md - Move system/README.md to daemon/config/README.md - Move web/CUSTOM-DASHBOARDS.md to web/gui/custom/README.md - Move web/CONFLUENCE-DASHBOARDS.md to web/gui/confluence/README.md * Resolve codacy issue $(..) syntax instead of `..` * Fix following warnings and add svgs to the data_structures/README.md - CHANGELOG.md - CODE_OF_CONDUCT.md - CONTRIBUTORS.md - REDISTRIBUTED.md - diagrams/data_structures/README.md - docker/README.md WARNING - Documentation file 'README.md' contains a link to 'collectors/plugins.d' which does not exist in the documentation directory. WARNING - Documentation file 'README.md' contains a link to 'collectors/statsd.plugin' which does not exist in the documentation directory. WARNING - Documentation file 'CONTRIBUTING.md' contains a link to 'web/CUSTOM-DASHBOARDS.md' which does not exist in the documentation directory. WARNING - Documentation file 'CONTRIBUTING.md' contains a link to 'web/CONFLUENCE-DASHBOARDS.md' which does not exist in the documentation directory. * Wrong urls in data_structures/README.md svgs * Fix svg URLs number 2 * Modify the first line of the main README.md, to enable proper static html generation. Executed after copying the file to htmldoc/src * Added back Why Netdata * Fixed link to registry in Why-Netdata.md * Added Why-Netdata to buildyaml and to Makefile.am * Replaced http links causing mixed content warnings * Made buildhtml ignore the directory node_modules created by Netlify * Corrected CONTRIBUTING.MD to CONTRIBUTING.md
This commit is contained in:
Chris Akritidis
Costa Tsaousis
parent
a29b433526
commit
3aae8f6c2c
71 changed files with 5448 additions and 319 deletions
.gitignoreCONTRIBUTING.mdMakefile.amREADME.md
backends
collectors
README.md
apps.plugin
checks.plugin
freebsd.plugin
macos.plugin
node.d.plugin
plugins.d
proc.plugin
python.d.plugin
tc.plugin
daemon
diagrams/data_structures
doc
Add-more-charts-to-netdata.mdDemo-Sites.mdDonations-netdata-has-received.mdNetdata-Security-and-Disclosure-Information.mdPerformance.mdRunning-behind-apache.mdRunning-behind-caddy.mdRunning-behind-lighttpd.mdRunning-behind-nginx.mdThird-Party-Plugins.mdWhy-Netdata.mda-github-star-is-important.mdhigh-performance-netdata.mdnetdata-for-IoT.mdnetdata-security.md
docker
health
htmldoc
installer
libnetdata/adaptive_resortable_list
packaging
requirements.txtruntime.txtstreaming
tests
web
5
.gitignore
vendored
5
.gitignore
vendored
|
|
@ -143,3 +143,8 @@ sitespeed-result/
|
|||
|
||||
# tests and temp files
|
||||
python.d/python-modules-installer.sh
|
||||
|
||||
# documentation generated files
|
||||
htmldoc/src
|
||||
htmldoc/build
|
||||
htmldoc/mkdocs.yml
|
||||
|
|
|
|||
92
CONTRIBUTING.md
Normal file
92
CONTRIBUTING.md
Normal file
|
|
@ -0,0 +1,92 @@
|
|||
# Contributing
|
||||
|
||||
Thank you for considering contributing to Netdata.
|
||||
|
||||
We love to receive contributions. Maintaining a platform for monitoring everything imaginable requires a broad understanding of a plethora of technologies, systems and applications. We rely on community contributions and user feedback to continue providing the best monitoring solution out there.
|
||||
|
||||
There are many ways to contribute, with varying requirements of skills:
|
||||
|
||||
## All NetData Users
|
||||
|
||||
### Give Netdata a GitHub star
|
||||
|
||||
This is the minimum open-source users should contribute back to the projects they use. Github stars help the project gain visibility, stand out. So, if you use Netdata, consider pressing that button. **It really matters**.
|
||||
|
||||
### Spread the word
|
||||
|
||||
Community growth allows the project to attract new talent willing to contribute. This talent is then developing new features and improves the project. These new features and improvements attract more users and so on. It is a loop. So, post about netdata, present it to local meetups you attend, let your online social network or twitter, facebook, reddit, etc. know you are using it. **The more people involved, the faster the project evolves**.
|
||||
|
||||
### Provide feedback
|
||||
|
||||
Is there anything that bothers you about netdata? Did you experience an issue while installing it or using it? Would you like to see it evolve to you need? Let us know. [Open a github issue](https://github.com/netdata/netdata/issues) to discuss it. Feedback is very important for open-source projects. We can't commit we will do everything, but your feedback influences our road-map significantly. **We rely on your feedback to make Netdata better**.
|
||||
|
||||
#### Help the developers understand what they have to do
|
||||
|
||||
NetData is all about simplicity and meaningful presentation. It's impossible for a handful of people to know which metrics really matter when monitoring a particular software or hardware component you are interested in. Be specific about what should be collected, how the information should be presented in the dashboard and which alarms make sense in most situations.
|
||||
|
||||
## Experienced Users
|
||||
|
||||
### Help other users
|
||||
|
||||
As the project grows, an increasing share of our time is spent on supporting this community of users in terms of answering questions, of helping users understand how netdata works and find their way with it. Helping other users is crucial. It allows the developers and maintainers of the project to focus on improving it.
|
||||
|
||||
### Improve documentation
|
||||
|
||||
Most of our documentation is in markdown (.md) files inside the netdata GitHub project. What remains in our Wiki will soon be moved in there as well. Don't be afraid to edit any of these documents and submit a GitHub Pull Request with your corrections/additions.
|
||||
|
||||
|
||||
## Developers
|
||||
|
||||
We expect most contributions to be for new data collection plugins. You can read about how external plugins work [here](collectors/plugins.d/). Additional instructions are available for [Node.js plugins](collectors/node.d/) and [Python plugis](collectors/python.d/).
|
||||
|
||||
Of course we appreciate contributions for any other part of the NetData agent, including the [deamon](deamon/), [backends for long term archiving](backends/), innovative ways of using the [REST API](web/api) to create cool [Custom Dashboards](web/gui/custom/) or to include NetData charts in other applications, similarly to what can be done with [Confluence](web/gui/confluence/).
|
||||
|
||||
|
||||
### Contributions Ground Rules
|
||||
|
||||
#### Code of Conduct and CLA
|
||||
|
||||
We expect all contributors to abide by the [Contributor Covenant Code of Conduct](CODE_OF_CONDUCT.md). For a pull request to be accepted, you will also need to accept the [netdata contributors license agreement](CONTRIBUTORS.md), as part of the PR process.
|
||||
|
||||
#### Performance and efficiency
|
||||
|
||||
Everything on Netdata is about efficiency. We need netdata to always be the most lightweight monitoring solution available. We will reject to merge PRs that are not optimal in resource utilization and efficiency.
|
||||
|
||||
Of course there are cases that such technical excellence is either not reasonable or not feasible. In these cases, we may require the feature or code submitted to be by disabled by default.
|
||||
|
||||
#### Meaningful metrics
|
||||
|
||||
Unlike other monitoring solutions, Netdata requires all metrics collected to have some structure attached to them. So, Netdata metrics have a name, units, belong to a chart that has a title, a family, a context, belong to an application, etc.
|
||||
|
||||
This structure is what makes netdata different. Most other monitoring solution collect bulk metrics in terms of name-value pairs and then expect their users to give meaning to these metrics during visualization. This does not work. It is neither practical nor reasonable to give to someone 2000 metrics and let him/her visualize them in a meaningful way.
|
||||
|
||||
So, netdata requires all metrics to have a meaning at the time they are collected. We will reject to merge PRs that loosely collect just a "bunch of metrics", but we are very keen to help you fix this.
|
||||
|
||||
#### Automated Testing
|
||||
|
||||
Netdata is a very large application to have automated testing end-to-end. But automated testing is required for crucial functions of it.
|
||||
|
||||
Generally, all pull requests should be coupled with automated testing scenarios. However since we do not currently have a framework in place for testing everything little bit of it, we currently require automated tests for parts of Netdata that seem too risky to be changed without automated testing.
|
||||
|
||||
Of course, manual testing is always required.
|
||||
|
||||
#### Netdata is a distributed application
|
||||
|
||||
Netdata is a distributed monitoring application. A few basic features can become quite complicated for such applications. We may reject features that alter or influence the nature of netdata, though we usually discuss the requirements with contributors and help them adapt their code to be better suited for Netdata.
|
||||
|
||||
#### Operating systems supported
|
||||
|
||||
Netdata should be running everywhere, on every production system out there.
|
||||
|
||||
Although we focus on **supported operating systems**, we still want Netdata to run even on non-supported systems. This, of course, may require some more manual work from the users (to prepare their environment, or enable certain flags, etc).
|
||||
|
||||
If your contributions limit the number of operating systems supported we will request from you to improve it.
|
||||
|
||||
#### Documentation
|
||||
|
||||
Your contributions should be bundled with related documentation to help users understand how to use the features you introduce.
|
||||
|
||||
#### Maintenance
|
||||
|
||||
When you contribute code to Netdata, you are automatically accepting that you will be responsible for maintaining that code in the future. So, if users need help, or report bugs, we will invite you to the related github issues to help them or fix the issues or bugs of your contributions.
|
||||
|
||||
24
Makefile.am
24
Makefile.am
|
|
@ -44,6 +44,7 @@ EXTRA_DIST = \
|
|||
CODE_OF_CONDUCT.md \
|
||||
LICENSE \
|
||||
REDISTRIBUTED.md \
|
||||
CONTRIBUTING.md \
|
||||
$(NULL)
|
||||
|
||||
SUBDIRS = \
|
||||
|
|
@ -61,6 +62,27 @@ dist_noinst_DATA= \
|
|||
netdata.cppcheck \
|
||||
netdata.spec \
|
||||
package.json \
|
||||
doc/Add-more-charts-to-netdata.md \
|
||||
doc/Demo-Sites.md \
|
||||
doc/Donations-netdata-has-received.md \
|
||||
doc/Netdata-Security-and-Disclosure-Information.md \
|
||||
doc/Performance.md \
|
||||
doc/Running-behind-apache.md \
|
||||
doc/Running-behind-caddy.md \
|
||||
doc/Running-behind-lighttpd.md \
|
||||
doc/Running-behind-nginx.md \
|
||||
doc/Third-Party-Plugins.md \
|
||||
doc/a-github-star-is-important.md \
|
||||
doc/high-performance-netdata.md \
|
||||
doc/netdata-for-IoT.md \
|
||||
doc/netdata-security.md \
|
||||
doc/Why-Netdata.md \
|
||||
htmldoc/themes/material/partials/footer.html \
|
||||
installer/README.md \
|
||||
installer/UNINSTALL.md \
|
||||
installer/UPDATE.md \
|
||||
requirements.txt \
|
||||
runtime.txt \
|
||||
$(NULL)
|
||||
|
||||
# until integrated within build
|
||||
|
|
@ -71,6 +93,8 @@ dist_noinst_SCRIPTS= \
|
|||
kickstart-static64.sh \
|
||||
netdata-installer.sh \
|
||||
installer/functions.sh \
|
||||
htmldoc/buildhtml.sh \
|
||||
htmldoc/buildyaml.sh \
|
||||
$(NULL)
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
|
|
|
|||
13
README.md
13
README.md
|
|
@ -1,4 +1,4 @@
|
|||
# netdata [](https://travis-ci.com/netdata/netdata) [](https://bestpractices.coreinfrastructure.org/projects/2231) [](https://www.gnu.org/licenses/gpl-3.0) []()
|
||||
# netdata [](https://travis-ci.com/netdata/netdata) [](https://bestpractices.coreinfrastructure.org/projects/2231) [](https://www.gnu.org/licenses/gpl-3.0) []()
|
||||
|
||||
[](https://codeclimate.com/github/netdata/netdata) [](https://www.codacy.com/app/netdata/netdata?utm_source=github.com&utm_medium=referral&utm_content=netdata/netdata&utm_campaign=Badge_Grade) [](https://lgtm.com/projects/g/netdata/netdata/context:cpp) [](https://lgtm.com/projects/g/netdata/netdata/context:javascript) [](https://lgtm.com/projects/g/netdata/netdata/context:python)
|
||||
|
||||
|
|
@ -52,7 +52,7 @@ The following animated image, shows the top part of a typical netdata dashboard.
|
|||
|
||||
*A typical netdata dashboard, in 1:1 timing. Charts can be panned by dragging them, zoomed in/out with `SHIFT` + `mouse wheel`, an area can be selected for zoom-in with `SHIFT` + `mouse selection`. Netdata is highly interactive and **real-time**, optimized to get the work done!*
|
||||
|
||||
> *We have a few online demos to see it in action: [http://my-netdata.io](http://my-netdata.io)*
|
||||
> *We have a few online demos to check: [https://my-netdata.io](https://my-netdata.io)*
|
||||
|
||||
## User base
|
||||
|
||||
|
|
@ -73,7 +73,7 @@ When you install multiple netdata, they are integrated into **one distributed ap
|
|||
|
||||
[](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [](https://registry.my-netdata.io/#menu_netdata_submenu_registry)
|
||||
|
||||
*in the last 24 hours:*<br/> [](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [](https://registry.my-netdata.io/#menu_netdata_submenu_registry)
|
||||
*in the last 24 hours:*<br/> [](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [](https://registry.my-netdata.io/#menu_netdata_submenu_registry)
|
||||
|
||||
## Quick Start
|
||||
|
||||
|
|
@ -86,7 +86,7 @@ bash
|
|||
# install netdata, directly from github sources
|
||||
bash <(curl -Ss https://my-netdata.io/kickstart.sh)
|
||||
```
|
||||
 
|
||||
 
|
||||
|
||||
The above command will:
|
||||
|
||||
|
|
@ -130,7 +130,6 @@ Netdata is **open-source**, **free**, super **fast**, very **easy**, completely
|
|||
It has been designed by **SysAdmins**, **DevOps** and **Developers** for troubleshooting performance problems,
|
||||
not just visualize metrics.
|
||||
|
||||
|
||||
## News
|
||||
|
||||
`Nov 6th, 2018` - **[netdata v1.11.0 released!](https://github.com/netdata/netdata/releases)**
|
||||
|
|
@ -256,13 +255,13 @@ To improve visual anomaly detection across charts, the user can highlight a time
|
|||
## What does it monitor
|
||||
|
||||
Netdata data collection is **extensible** - you can monitor anything you can get a metric for.
|
||||
Its [Plugin API](collectors/plugins.d) supports all programing languages (anything can be a netdata plugin, BASH, python, perl, node.js, java, Go, ruby, etc).
|
||||
Its [Plugin API](collectors/plugins.d/) supports all programing languages (anything can be a netdata plugin, BASH, python, perl, node.js, java, Go, ruby, etc).
|
||||
|
||||
- For better performance, most system related plugins (cpu, memory, disks, filesystems, networking, etc) have been written in `C`.
|
||||
- For faster development and easier contributions, most application related plugins (databases, web servers, etc) have been written in `python`.
|
||||
|
||||
#### APM (Application Performance Monitoring)
|
||||
- **statsd** - [netdata is a fully featured statsd server](collectors/statsd.plugin).
|
||||
- **statsd** - [netdata is a fully featured statsd server](collectors/statsd.plugin/).
|
||||
- **go_expvar** - collects metrics exposed by applications written in the Go programming language using the expvar package.
|
||||
- **Spring Boot** - monitors running Java Spring Boot applications that expose their metrics with the use of the Spring Boot Actuator included in Spring Boot library.
|
||||
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@ SUBDIRS = \
|
|||
|
||||
dist_noinst_DATA = \
|
||||
README.md \
|
||||
WALKTHROUGH.md \
|
||||
$(NULL)
|
||||
|
||||
dist_noinst_SCRIPTS = \
|
||||
|
|
|
|||
|
|
@ -1,4 +1,3 @@
|
|||
|
||||
# Metrics Long Term Archiving
|
||||
|
||||
netdata supports backends for archiving the metrics, or providing long term dashboards,
|
||||
|
|
@ -111,7 +110,7 @@ of `netdata.conf` from your netdata):
|
|||
When multiple servers are defined, netdata will try the next one when the first one fails. This allows
|
||||
you to load-balance different servers: give your backend servers in different order on each netdata.
|
||||
|
||||
netdata also ships [`nc-backend.sh`](nc-backend.sh),
|
||||
netdata also ships [`nc-backend.sh`](https://github.com/netdata/netdata/tree/master/backends/nc-backend.sh),
|
||||
a script that can be used as a fallback backend to save the metrics to disk and push them to the
|
||||
time-series database when it becomes available again. It can also be used to monitor / trace / debug
|
||||
the metrics netdata generates.
|
||||
|
|
@ -186,7 +185,7 @@ netdata provides 5 charts:
|
|||
|
||||
## alarms
|
||||
|
||||
The latest version of the alarms configuration for monitoring the backend is [here](../health/health.d/backend.conf)
|
||||
The latest version of the alarms configuration for monitoring the backend is [here](https://github.com/netdata/netdata/tree/master/health/health.d/backend.conf)
|
||||
|
||||
netdata adds 4 alarms:
|
||||
|
||||
|
|
|
|||
294
backends/WALKTHROUGH.md
Normal file
294
backends/WALKTHROUGH.md
Normal file
|
|
@ -0,0 +1,294 @@
|
|||
# Netdata, Prometheus, Grafana stack
|
||||
|
||||
## Intro
|
||||
In this article I will walk you through the basics of getting Netdata,
|
||||
Prometheus and Grafana all working together and monitoring your application
|
||||
servers. This article will be using docker on your local workstation. We will be
|
||||
working with docker in an ad-hoc way, launching containers that run ‘/bin/bash’
|
||||
and attaching a TTY to them. I use docker here in a purely academic fashion and
|
||||
do not condone running Netdata in a container. I pick this method so individuals
|
||||
without cloud accounts or access to VMs can try this out and for it’s speed of
|
||||
deployment.
|
||||
|
||||
## Why Netdata, Prometheus, and Grafana
|
||||
Some time ago I was introduced to Netdata by a coworker. We were attempting to
|
||||
troubleshoot python code which seemed to be bottlenecked. I was instantly
|
||||
impressed by the amount of metrics Netdata exposes to you. I quickly added
|
||||
Netdata to my set of go-to tools when troubleshooting systems performance.
|
||||
|
||||
Some time ago, even later, I was introduced to Prometheus. Prometheus is a
|
||||
monitoring application which flips the normal architecture around and polls
|
||||
rest endpoints for its metrics. This architectural change greatly simplifies
|
||||
and decreases the time necessary to begin monitoring your applications.
|
||||
Compared to current monitoring solutions the time spent on designing the
|
||||
infrastructure is greatly reduced. Running a single Prometheus server per
|
||||
application becomes feasible with the help of Grafana.
|
||||
|
||||
Grafana has been the go to graphing tool for… some time now. It’s awesome,
|
||||
anyone that has used it knows it’s awesome. We can point Grafana at Prometheus
|
||||
and use Prometheus as a data source. This allows a pretty simple overall
|
||||
monitoring architecture: Install Netdata on your application servers, point
|
||||
Prometheus at Netdata, and then point Grafana at Prometheus.
|
||||
|
||||
I’m omitting an important ingredient in this stack in order to keep this tutorial
|
||||
simple and that is service discovery. My personal preference is to use Consul.
|
||||
Prometheus can plug into consul and automatically begin to scrape new hosts that
|
||||
register a Netdata client with Consul.
|
||||
|
||||
At the end of this tutorial you will understand how each technology fits
|
||||
together to create a modern monitoring stack. This stack will offer you
|
||||
visibility into your application and systems performance.
|
||||
|
||||
## Getting Started - Netdata
|
||||
To begin let’s create our container which we will install Netdata on. We need
|
||||
to run a container, forward the necessary port that netdata listens on, and
|
||||
attach a tty so we can interact with the bash shell on the container. But
|
||||
before we do this we want name resolution between the two containers to work.
|
||||
In order to accomplish this we will create a user-defined network and attach
|
||||
both containers to this network. The first command we should run is:
|
||||
|
||||
```
|
||||
docker network create --driver bridge netdata-tutorial
|
||||
```
|
||||
|
||||
With this user-defined network created we can now launch our container we will
|
||||
install Netdata on and point it to this network.
|
||||
|
||||
```
|
||||
docker run -it --name netdata --hostname netdata --network=netdata-tutorial -p 19999:19999 centos:latest '/bin/bash'
|
||||
```
|
||||
|
||||
This command creates an interactive tty session (-it), gives the container both
|
||||
a name in relation to the docker daemon and a hostname (this is so you know what
|
||||
container is which when working in the shells and docker maps hostname
|
||||
resolution to this container), forwards the local port 19999 to the container’s
|
||||
port 19999 (-p 19999:19999), sets the command to run (/bin/bash) and then
|
||||
chooses the base container images (centos:latest). After running this you should
|
||||
be sitting inside the shell of the container.
|
||||
|
||||
After we have entered the shell we can install Netdata. This process could not
|
||||
be easier. If you take a look at this link:
|
||||
https://github.com/netdata/netdata/wiki/Installation the Netdata devs give us
|
||||
several one-liners to install netdata. I have not had any issues with these one
|
||||
liners and their bootstrapping scripts so far (If you guys run into anything do
|
||||
share). Run the following command in your container.
|
||||
|
||||
```
|
||||
bash <(curl -Ss https://my-netdata.io/kickstart.sh) --dont-wait
|
||||
```
|
||||
|
||||
After the install completes you should be able to hit the Netdata dashboard at
|
||||
http://localhost:19999/ (replace localhost if you’re doing this on a VM or have
|
||||
the docker container hosted on a machine not on your local system). If this is
|
||||
your first time using Netdata I suggest you take a look around. The amount of
|
||||
time I’ve spent digging through /proc and calculating my own metrics has been
|
||||
greatly reduced by this tool. Take it all in.
|
||||
|
||||
Next I want to draw your attention to a particular endpoint. Navigate to
|
||||
http://localhost:19999/api/v1/allmetrics?format=prometheus&help=yes In your
|
||||
browser. This is the endpoint which publishes all the metrics in a format which
|
||||
Prometheus understands. Let’s take a look at one of these metrics.
|
||||
`netdata_system_cpu_percentage_average{chart="system.cpu",family="cpu",dimension="system"}
|
||||
0.0831255 1501271696000` This metric is representing several things which I will
|
||||
go in more details in the section on prometheus. For now understand that this
|
||||
metric: `netdata_system_cpu_percentage_average` has several labels: [chart,
|
||||
family, dimension]. This corresponds with the first cpu chart you see on the
|
||||
Netdata dashboard.
|
||||
|
||||
|
||||
|
||||
This CHART is called ‘system.cpu’, The FAMILY is cpu, and the DIMENSION we are
|
||||
observing is “system”. You can begin to draw links between the charts in netdata
|
||||
to the prometheus metrics format in this manner.
|
||||
|
||||
## Prometheus
|
||||
We will be installing prometheus in a container for purpose of demonstration.
|
||||
While prometheus does have an official container I would like to walk through
|
||||
the install process and setup on a fresh container. This will allow anyone
|
||||
reading to migrate this tutorial to a VM or Server of any sort.
|
||||
|
||||
Let’s start another container in the same fashion as we did the Netdata
|
||||
container. `docker run -it --name prometheus --hostname prometheus
|
||||
--network=netdata-tutorial -p 9090:9090 centos:latest '/bin/bash'` This should
|
||||
drop you into a shell once again. Once there quickly install your favorite
|
||||
editor as we will be editing files later in this tutorial. `yum install vim -y`
|
||||
|
||||
Prometheus provides a tarball of their latest stable versions here:
|
||||
https://prometheus.io/download/. Let’s download the latest version and install
|
||||
into your container.
|
||||
|
||||
```
|
||||
curl -L 'https://github.com/prometheus/prometheus/releases/download/v1.7.1/prometheus-1.7.1.linux-amd64.tar.gz' -o /tmp/prometheus.tar.gz
|
||||
|
||||
mkdir /opt/prometheus
|
||||
|
||||
tar -xf /tmp/prometheus.tar.gz -C /opt/prometheus/ --strip-components 1
|
||||
```
|
||||
|
||||
This should get prometheus installed into the container. Let’s test that we can run
|
||||
prometheus and connect to it’s web interface. This will look similar to what
|
||||
follows:
|
||||
|
||||
```
|
||||
[root@prometheus prometheus]# /opt/prometheus/prometheus
|
||||
INFO[0000] Starting prometheus (version=1.7.1, branch=master, revision=3afb3fffa3a29c3de865e1172fb740442e9d0133)
|
||||
source="main.go:88"
|
||||
INFO[0000] Build context (go=go1.8.3, user=root@0aa1b7fc430d, date=20170612-11:44:05) source="main.go:89"
|
||||
INFO[0000] Host details (Linux 4.9.36-moby #1 SMP Wed Jul 12 15:29:07 UTC 2017 x86_64 prometheus (none)) source="main.go:90"
|
||||
INFO[0000] Loading configuration file prometheus.yml source="main.go:252"
|
||||
INFO[0000] Loading series map and head chunks... source="storage.go:428"
|
||||
INFO[0000] 0 series loaded. source="storage.go:439"
|
||||
INFO[0000] Starting target manager... source="targetmanager.go:63"
|
||||
INFO[0000] Listening on :9090 source="web.go:259"
|
||||
```
|
||||
|
||||
Now attempt to go to http://localhost:9090/. You should be presented with the
|
||||
prometheus homepage. This is a good point to talk about Prometheus’s data model
|
||||
which can be viewed here: https://prometheus.io/docs/concepts/data_model/ As
|
||||
explained we have two key elements in Prometheus metrics. We have the ‘metric’
|
||||
and its ‘labels’. Labels allow for granularity between metrics. Let’s use our
|
||||
previous example to further explain.
|
||||
|
||||
```
|
||||
netdata_system_cpu_percentage_average{chart="system.cpu",family="cpu",dimension="system"} 0.0831255 1501271696000
|
||||
```
|
||||
|
||||
Here our metric is
|
||||
‘netdata_system_cpu_percentage_average’ and our labels are ‘chart’, ‘family’,
|
||||
and ‘dimension. The last two values constitute the actual metric value for the
|
||||
metric type (gauge, counter, etc…). We can begin graphing system metrics with
|
||||
this information, but first we need to hook up Prometheus to poll Netdata stats.
|
||||
|
||||
Let’s move our attention to Prometheus’s configuration. Prometheus gets it
|
||||
config from the file located (in our example) at
|
||||
`/opt/prometheus/prometheus.yml`. I won’t spend an extensive amount of time
|
||||
going over the configuration values documented here:
|
||||
https://prometheus.io/docs/operating/configuration/. We will be adding a new
|
||||
“job” under the “scrape_configs”. Let’s make the “scrape_configs” section look
|
||||
like this (we can use the dns name Netdata due to the custom user-defined
|
||||
network we created in docker beforehand).
|
||||
|
||||
```yml
|
||||
scrape_configs:
|
||||
# The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
|
||||
- job_name: 'prometheus'
|
||||
|
||||
# metrics_path defaults to '/metrics'
|
||||
# scheme defaults to 'http'.
|
||||
|
||||
static_configs:
|
||||
- targets: ['localhost:9090']
|
||||
|
||||
- job_name: 'netdata'
|
||||
|
||||
metrics_path: /api/v1/allmetrics
|
||||
params:
|
||||
format: [ prometheus ]
|
||||
|
||||
static_configs:
|
||||
- targets: ['netdata:19999']
|
||||
```
|
||||
|
||||
Let’s start prometheus once again by running `/opt/prometheus/prometheus`. If we
|
||||
now navigate to prometheus at ‘http://localhost:9090/targets’ we should see our
|
||||
target being successfully scraped. If we now go back to the Prometheus’s
|
||||
homepage and begin to type ‘netdata_’ Prometheus should auto complete metrics
|
||||
it is now scraping.
|
||||
|
||||
|
||||
|
||||
Let’s now start exploring how we can graph some metrics. Back in our NetData
|
||||
container lets get the CPU spinning with a pointless busy loop. On the shell do
|
||||
the following:
|
||||
|
||||
```
|
||||
[root@netdata /]# while true; do echo "HOT HOT HOT CPU"; done
|
||||
```
|
||||
|
||||
Our NetData cpu graph should be showing some activity. Let’s represent this in
|
||||
Prometheus. In order to do this let’s keep our metrics page open for reference:
|
||||
http://localhost:19999/api/v1/allmetrics?format=prometheus&help=yes We are
|
||||
setting out to graph the data in the CPU chart so let’s search for “system.cpu”
|
||||
in the metrics page above. We come across a section of metrics with the first
|
||||
comments `# COMMENT homogeneus chart "system.cpu", context "system.cpu", family
|
||||
"cpu", units "percentage"` Followed by the metrics. This is a good start now let
|
||||
us drill down to the specific metric we would like to graph.
|
||||
|
||||
```
|
||||
# COMMENT
|
||||
netdata_system_cpu_percentage_average: dimension "system", value is percentage, gauge, dt 1501275951 to 1501275951 inclusive
|
||||
netdata_system_cpu_percentage_average{chart="system.cpu",family="cpu",dimension="system"} 0.0000000 1501275951000
|
||||
```
|
||||
|
||||
Here we learn that the metric name we care about is
|
||||
‘netdata_system_cpu_percentage_average’ so throw this into Prometheus and see
|
||||
what we get. We should see something similar to this (I shut off my busy loop)
|
||||
|
||||
|
||||
|
||||
This is a good step toward what we want. Also make note that Prometheus will tag
|
||||
on an ‘instance’ label for us which corresponds to our statically defined job in
|
||||
the configuration file. This allows us to tailor our queries to specific
|
||||
instances. Now we need to isolate the dimension we want in our query. To do this
|
||||
let us refine the query slightly. Let’s query the dimension also. Place this
|
||||
into our query text box.
|
||||
`netdata_system_cpu_percentage_average{dimension="system"}` We now wind up with
|
||||
the following graph.
|
||||
|
||||
|
||||
|
||||
Awesome, this is exactly what we wanted. If you haven’t caught on yet we can
|
||||
emulate entire charts from NetData by using the `chart` dimension. If you’d like
|
||||
you can combine the ‘chart’ and ‘instance’ dimension to create per-instance
|
||||
charts. Let’s give this a try:
|
||||
`netdata_system_cpu_percentage_average{chart="system.cpu", instance="netdata:19999"}`
|
||||
|
||||
This is the basics of using Prometheus to query NetData. I’d advise everyone at
|
||||
this point to read this page
|
||||
https://github.com/netdata/netdata/wiki/Using-Netdata-with-Prometheus#netdata-support-for-prometheus.
|
||||
The key point here is that NetData can export metrics from its internal DB or
|
||||
can send metrics “as-collected” by specifying the ‘source=as-collected’ url
|
||||
parameter like so.
|
||||
http://localhost:19999/api/v1/allmetrics?format=prometheus&help=yes&types=yes&source=as-collected
|
||||
If you choose to use this method you will need to use Prometheus's set of
|
||||
functions here: https://prometheus.io/docs/querying/functions/ to obtain useful
|
||||
metrics as you are now dealing with raw counters from the system. For example
|
||||
you will have to use the `irate()` function over a counter to get that metric’s
|
||||
rate per second. If your graphing needs are met by using the metrics returned by
|
||||
NetData’s internal database (not specifying any source= url parameter) then use
|
||||
that. If you find limitations then consider re-writing your queries using the
|
||||
raw data and using Prometheus functions to get the desired chart.
|
||||
|
||||
## Grafana
|
||||
Finally we make it to grafana. This is the easiest part in my opinion. This time
|
||||
we will actually run the official grafana docker container as all configuration
|
||||
we need to do is done via the GUI. Let’s run the following command:
|
||||
|
||||
```
|
||||
docker run -i -p 3000:3000 --network=netdata-tutorial grafana/grafana
|
||||
```
|
||||
|
||||
This will get grafana running at ‘http://localhost:3000/’ Let’s go there and
|
||||
login using the credentials Admin:Admin.
|
||||
|
||||
The first thing we want to do is click ‘Add data source’. Let’s make it look
|
||||
like the following screenshot
|
||||
|
||||
|
||||
|
||||
With this completed let’s graph! Create a new Dashboard by clicking on the top
|
||||
left Grafana Icon and create a new graph in that dashboard. Fill in the query
|
||||
like we did above and save.
|
||||
|
||||
|
||||
|
||||
## Conclusion
|
||||
|
||||
There you have it, a complete systems monitoring stack which is very easy to
|
||||
deploy. From here I would begin to understand how Prometheus and a service
|
||||
discovery mechanism such as Consul can play together nicely. My current prod
|
||||
deployments automatically register Netdata services into Consul and Prometheus
|
||||
automatically begins to scrape them. Once achieved you do not have to think
|
||||
about the monitoring system until Prometheus cannot keep up with your scale.
|
||||
Once this happens there are options presented in the Prometheus documentation
|
||||
for solving this. Hope this was helpful, happy monitoring.
|
||||
|
|
@ -4,6 +4,7 @@
|
|||
|
||||
Prometheus is a distributed monitoring system which offers a very simple setup along with a robust data model. Recently netdata added support for Prometheus. I'm going to quickly show you how to install both netdata and prometheus on the same server. We can then use grafana pointed at Prometheus to obtain long term metrics netdata offers. I'm assuming we are starting at a fresh ubuntu shell (whether you'd like to follow along in a VM or a cloud instance is up to you).
|
||||
|
||||
|
||||
## Installing netdata and prometheus
|
||||
|
||||
### Installing netdata
|
||||
|
|
|
|||
|
|
@ -15,9 +15,9 @@ To minimize the number of processes spawn for data collection, netdata also supp
|
|||
data collection modules with the minimum of code.
|
||||
|
||||
Currently netdata provides plugin orchestrators
|
||||
BASH v4+ [charts.d.plugin](charts.d.plugin),
|
||||
node.js [node.d.plugin](node.d.plugin) and
|
||||
python v2+ (including v3) [python.d.plugin](python.d.plugin).
|
||||
BASH v4+ [charts.d.plugin](charts.d.plugin/),
|
||||
node.js [node.d.plugin](node.d.plugin/) and
|
||||
python v2+ (including v3) [python.d.plugin](python.d.plugin/).
|
||||
|
||||
## Netdata Plugins
|
||||
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@ for every process found running.
|
|||
|
||||
Since netdata needs to present this information in charts and track them through time,
|
||||
instead of presenting a `top` like list, `apps.plugin` uses a pre-defined list of **process groups**
|
||||
to which it assigns all running processes. This list is [customizable](apps_groups.conf) and netdata
|
||||
to which it assigns all running processes. This list is [customizable](https://github.com/netdata/netdata/tree/master/collectors/apps.plugin/apps_groups.conf) and netdata
|
||||
ships with a good default for most cases (to edit it on your system run `/etc/netdata/edit-config apps_groups.conf`).
|
||||
|
||||
So, `apps.plugin` builds a process tree (much like `ps fax` does in Linux), and groups
|
||||
|
|
@ -15,7 +15,7 @@ processes together (evaluating both child and parent processes) so that the resu
|
|||
a predefined set of members (of course, only process groups found running are reported).
|
||||
|
||||
> If you find that `apps.plugin` categorizes standard applications as `other`, we would be
|
||||
> glad to accept pull requests improving the [defaults](apps_groups.conf) shipped with netdata.
|
||||
> glad to accept pull requests improving the [defaults](https://github.com/netdata/netdata/tree/master/collectors/apps.plugin/apps_groups.conf) shipped with netdata.
|
||||
|
||||
Unlike traditional process monitoring tools (like `top`), `apps.plugin` is able to account the resource
|
||||
utilization of exit processes. Their utilization is accounted at their currently running parents.
|
||||
|
|
@ -55,7 +55,7 @@ Each of these sections provides the same number of charts:
|
|||
|
||||
The above are reported:
|
||||
|
||||
- For **Applications** per [target configured](apps_groups.conf).
|
||||
- For **Applications** per [target configured](https://github.com/netdata/netdata/tree/master/collectors/apps.plugin/apps_groups.conf).
|
||||
- For **Users** per username or UID (when the username is not available).
|
||||
- For **User Groups** per groupname or GID (when groupname is not available).
|
||||
|
||||
|
|
@ -85,7 +85,7 @@ its CPU resources will be cut in half, and data collection will be once every 2
|
|||
|
||||
## Configuration
|
||||
|
||||
The configuration file is `/etc/netdata/apps_groups.conf` (the default is [here](apps_groups.conf)).
|
||||
The configuration file is `/etc/netdata/apps_groups.conf` (the default is [here](https://github.com/netdata/netdata/tree/master/collectors/apps.plugin/apps_groups.conf)).
|
||||
To edit it on your system run `/etc/netdata/edit-config apps_groups.conf`.
|
||||
|
||||
The configuration file works accepts multiple lines, each having this format:
|
||||
|
|
@ -188,21 +188,21 @@ Here is an example for the process group `sql` at `https://registry.my-netdata.i
|
|||
Netdata is able give you a lot more badges for your app.
|
||||
Examples below for process group `sql`:
|
||||
|
||||
- CPU usage: 
|
||||
- Disk Physical Reads 
|
||||
- Disk Physical Writes 
|
||||
- Disk Logical Reads 
|
||||
- Disk Logical Writes 
|
||||
- Open Files 
|
||||
- Real Memory 
|
||||
- Virtual Memory 
|
||||
- Swap Memory 
|
||||
- Minor Page Faults 
|
||||
- Processes 
|
||||
- Threads 
|
||||
- Major Faults (swap activity) 
|
||||
- Open Pipes 
|
||||
- Open Sockets 
|
||||
- CPU usage: 
|
||||
- Disk Physical Reads 
|
||||
- Disk Physical Writes 
|
||||
- Disk Logical Reads 
|
||||
- Disk Logical Writes 
|
||||
- Open Files 
|
||||
- Real Memory 
|
||||
- Virtual Memory 
|
||||
- Swap Memory 
|
||||
- Minor Page Faults 
|
||||
- Processes 
|
||||
- Threads 
|
||||
- Major Faults (swap activity) 
|
||||
- Open Pipes 
|
||||
- Open Sockets 
|
||||
|
||||
|
||||
For more information about badges check [Generating Badges](../../web/api/badges)
|
||||
|
|
@ -341,7 +341,7 @@ So, the `ssh` session is using 95% CPU time.
|
|||
Why `ssh`?
|
||||
|
||||
`apps.plugin` groups all processes based on its configuration file
|
||||
[`/etc/netdata/apps_groups.conf`](apps_groups.conf)
|
||||
[`/etc/netdata/apps_groups.conf`](https://github.com/netdata/netdata/tree/master/collectors/apps.plugin/apps_groups.conf)
|
||||
(to edit it on your system run `/etc/netdata/edit-config apps_groups.conf`).
|
||||
The default configuration has nothing for `bash`, but it has for `sshd`, so netdata accumulates
|
||||
all ssh sessions to a dimension on the charts, called `ssh`. This includes all the processes in
|
||||
|
|
|
|||
|
|
@ -2,3 +2,7 @@
|
|||
|
||||
AUTOMAKE_OPTIONS = subdir-objects
|
||||
MAINTAINERCLEANFILES = $(srcdir)/Makefile.in
|
||||
|
||||
dist_noinst_DATA = \
|
||||
README.md \
|
||||
$(NULL)
|
||||
|
|
|
|||
3
collectors/checks.plugin/README.md
Normal file
3
collectors/checks.plugin/README.md
Normal file
|
|
@ -0,0 +1,3 @@
|
|||
# Netdata internal checks
|
||||
|
||||
A debugging plugin (by default it is disabled)
|
||||
|
|
@ -3,3 +3,7 @@
|
|||
AUTOMAKE_OPTIONS = subdir-objects
|
||||
|
||||
MAINTAINERCLEANFILES = $(srcdir)/Makefile.in
|
||||
|
||||
dist_noinst_DATA = \
|
||||
README.md \
|
||||
$(NULL)
|
||||
|
|
|
|||
3
collectors/freebsd.plugin/README.md
Normal file
3
collectors/freebsd.plugin/README.md
Normal file
|
|
@ -0,0 +1,3 @@
|
|||
# freebsd
|
||||
|
||||
Collects resource usage and performance data on FreeBSD systems
|
||||
|
|
@ -2,3 +2,7 @@
|
|||
|
||||
AUTOMAKE_OPTIONS = subdir-objects
|
||||
MAINTAINERCLEANFILES = $(srcdir)/Makefile.in
|
||||
|
||||
dist_noinst_DATA = \
|
||||
README.md \
|
||||
$(NULL)
|
||||
|
|
|
|||
3
collectors/macos.plugin/README.md
Normal file
3
collectors/macos.plugin/README.md
Normal file
|
|
@ -0,0 +1,3 @@
|
|||
# macos
|
||||
|
||||
Collects resource usage and performance data on MacOS systems
|
||||
|
|
@ -9,7 +9,21 @@
|
|||
5. Allows each **module** to have one or more data collection **jobs**
|
||||
6. Each **job** is collecting one or more metrics from a single data source
|
||||
|
||||
# Motivation
|
||||
## Pull Request Checklist for Node.js Plugins
|
||||
|
||||
This is a generic checklist for submitting a new Node.js plugin for Netdata. It is by no means comprehensive.
|
||||
|
||||
At minimum, to be buildable and testable, the PR needs to include:
|
||||
|
||||
* The module itself, following proper naming conventions: `node.d/<module_dir>/<module_name>.node.js`
|
||||
* A README.md file for the plugin.
|
||||
* The configuration file for the module
|
||||
* A basic configuration for the plugin in the appropriate global config file: `conf.d/node.d.conf`, which is also in JSON format. If the module should be enabled by default, add a section for it in the `modules` dictionary.
|
||||
* A line for the plugin in the appropriate `Makefile.am` file: `node.d/Makefile.am` under `dist_node_DATA`.
|
||||
* A line for the plugin configuration file in `conf.d/Makefile.am`: under `dist_nodeconfig_DATA`
|
||||
* Optionally, chart information in `web/dashboard_info.js`. This generally involves specifying a name and icon for the section, and may include descriptions for the section or individual charts.
|
||||
|
||||
## Motivation
|
||||
|
||||
Node.js is perfect for asynchronous operations. It is very fast and quite common (actually the whole web is based on it).
|
||||
Since data collection is not a CPU intensive task, node.js is an ideal solution for it.
|
||||
|
|
@ -31,7 +45,7 @@ For more information check the **[[Installation]]** guide.
|
|||
## configuring `node.d.plugin`
|
||||
|
||||
`node.d.plugin` can work even without any configuration. Its default configuration file is
|
||||
[/etc/netdata/node.d.conf](node.d.conf) (to edit it on your system run `/etc/netdata/edit-config node.d.conf`).
|
||||
[/etc/netdata/node.d.conf](https://github.com/netdata/netdata/tree/master/collectors/node.d.plugin/node.d.conf) (to edit it on your system run `/etc/netdata/edit-config node.d.conf`).
|
||||
|
||||
## configuring `node.d.plugin` modules
|
||||
|
||||
|
|
@ -215,4 +229,4 @@ The `service` object defines a set of functions to allow you send information to
|
|||
---
|
||||
|
||||
*FIXME: document an operational node.d.plugin data collector - the best example is the
|
||||
[snmp collector](snmp/snmp.node.js)*
|
||||
[snmp collector](https://github.com/netdata/netdata/tree/master/collectors/node.d.plugin/snmp/snmp.node.js)*
|
||||
|
|
|
|||
|
|
@ -374,23 +374,23 @@ or do not output the line at all.
|
|||
|
||||
## Modular Plugins
|
||||
|
||||
1. **python**, use `python.d.plugin`, there are many examples in the [python.d directory](../python.d.plugin)
|
||||
1. **python**, use `python.d.plugin`, there are many examples in the [python.d directory](../python.d.plugin/)
|
||||
|
||||
python is ideal for netdata plugins. It is a simple, yet powerful way to collect data, it has a very small memory footprint, although it is not the most CPU efficient way to do it.
|
||||
|
||||
2. **node.js**, use `node.d.plugin`, there are a few examples in the [node.d directory](../node.d.plugin)
|
||||
2. **node.js**, use `node.d.plugin`, there are a few examples in the [node.d directory](../node.d.plugin/)
|
||||
|
||||
node.js is the fastest scripting language for collecting data. If your plugin needs to do a lot of work, compute values, etc, node.js is probably the best choice before moving to compiled code. Keep in mind though that node.js is not memory efficient; it will probably need more RAM compared to python.
|
||||
|
||||
3. **BASH**, use `charts.d.plugin`, there are many examples in the [charts.d directory](../charts.d.plugin)
|
||||
3. **BASH**, use `charts.d.plugin`, there are many examples in the [charts.d directory](../charts.d.plugin/)
|
||||
|
||||
BASH is the simplest scripting language for collecting values. It is the less efficient though in terms of CPU resources. You can use it to collect data quickly, but extensive use of it might use a lot of system resources.
|
||||
|
||||
4. **C**
|
||||
|
||||
Of course, C is the most efficient way of collecting data. This is why netdata itself is written in C.
|
||||
|
||||
---
|
||||
|
||||
## Properly Writing Plugins
|
||||
|
||||
## Writing Plugins Properly
|
||||
|
||||
|
|
@ -470,3 +470,4 @@ There are a few rules for writing plugins properly:
|
|||
3. If you are not sure of memory leaks, exit every one hour. Netdata will re-start your process.
|
||||
|
||||
4. If possible, try to autodetect if your plugin should be enabled, without any configuration.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,4 +1,3 @@
|
|||
|
||||
# proc.plugin
|
||||
|
||||
- `/proc/net/dev` (all network interfaces for all their values)
|
||||
|
|
@ -25,7 +24,7 @@
|
|||
|
||||
---
|
||||
|
||||
# Monitoring Disks
|
||||
## Monitoring Disks
|
||||
|
||||
> Live demo of disk monitoring at: **[http://london.netdata.rocks](https://registry.my-netdata.io/#menu_disk)**
|
||||
|
||||
|
|
@ -33,75 +32,45 @@ Performance monitoring for Linux disks is quite complicated. The main reason is
|
|||
|
||||
Hopefully, the Linux kernel provides many metrics that can provide deep insights of what our disks our doing. The kernel measures all these metrics on all layers of storage: **virtual disks**, **physical disks** and **partitions of disks**.
|
||||
|
||||
Let's see the list of metrics provided by netdata for each of the above:
|
||||
### Monitored disk metrics
|
||||
|
||||
### I/O bandwidth/s (kb/s)
|
||||
- I/O bandwidth/s (kb/s)
|
||||
The amount of data transferred from and to the disk.
|
||||
- I/O operations/s
|
||||
The number of I/O operations completed.
|
||||
- Queued I/O operations
|
||||
The number of currently queued I/O operations. For traditional disks that execute commands one after another, one of them is being run by the disk and the rest are just waiting in a queue.
|
||||
- Backlog size (time in ms)
|
||||
The expected duration of the currently queued I/O operations.
|
||||
- Utilization (time percentage)
|
||||
The percentage of time the disk was busy with something. This is a very interesting metric, since for most disks, that execute commands sequentially, **this is the key indication of congestion**. A sequential disk that is 100% of the available time busy, has no time to do anything more, so even if the bandwidth or the number of operations executed by the disk is low, its capacity has been reached.
|
||||
Of course, for newer disk technologies (like fusion cards) that are capable to execute multiple commands in parallel, this metric is just meaningless.
|
||||
- Average I/O operation time (ms)
|
||||
The average time for I/O requests issued to the device to be served. This includes the time spent by the requests in queue and the time spent servicing them.
|
||||
- Average I/O operation size (kb)
|
||||
The average amount of data of the completed I/O operations.
|
||||
- Average Service Time (ms)
|
||||
The average service time for completed I/O operations. This metric is calculated using the total busy time of the disk and the number of completed operations. If the disk is able to execute multiple parallel operations the reporting average service time will be misleading.
|
||||
- Merged I/O operations/s
|
||||
The Linux kernel is capable of merging I/O operations. So, if two requests to read data from the disk are adjacent, the Linux kernel may merge them to one before giving them to disk. This metric measures the number of operations that have been merged by the Linux kernel.
|
||||
- Total I/O time
|
||||
The sum of the duration of all completed I/O operations. This number can exceed the interval if the disk is able to execute multiple I/O operations in parallel.
|
||||
- Space usage
|
||||
For mounted disks, netdata will provide a chart for their space, with 3 dimensions:
|
||||
1. free
|
||||
2. used
|
||||
3. reserved for root
|
||||
- inode usage
|
||||
For mounted disks, netdata will provide a chart for their inodes (number of file and directories), with 3 dimensions:
|
||||
1. free
|
||||
2. used
|
||||
3. reserved for root
|
||||
|
||||
The amount of data transferred from and to the disk.
|
||||
|
||||
### I/O operations/s
|
||||
|
||||
The number of I/O operations completed.
|
||||
|
||||
### Queued I/O operations
|
||||
|
||||
The number of currently queued I/O operations. For traditional disks that execute commands one after another, one of them is being run by the disk and the rest are just waiting in a queue.
|
||||
|
||||
### Backlog size (time in ms)
|
||||
|
||||
The expected duration of the currently queued I/O operations.
|
||||
|
||||
### Utilization (time percentage)
|
||||
|
||||
The percentage of time the disk was busy with something. This is a very interesting metric, since for most disks, that execute commands sequentially, **this is the key indication of congestion**. A sequential disk that is 100% of the available time busy, has no time to do anything more, so even if the bandwidth or the number of operations executed by the disk is low, its capacity has been reached.
|
||||
|
||||
Of course, for newer disk technologies (like fusion cards) that are capable to execute multiple commands in parallel, this metric is just meaningless.
|
||||
|
||||
### Average I/O operation time (ms)
|
||||
|
||||
The average time for I/O requests issued to the device to be served. This includes the time spent by the requests in queue and the time spent servicing them.
|
||||
|
||||
### Average I/O operation size (kb)
|
||||
|
||||
The average amount of data of the completed I/O operations.
|
||||
|
||||
### Average Service Time (ms)
|
||||
|
||||
The average service time for completed I/O operations. This metric is calculated using the total busy time of the disk and the number of completed operations. If the disk is able to execute multiple parallel operations the reporting average service time will be misleading.
|
||||
|
||||
### Merged I/O operations/s
|
||||
|
||||
The Linux kernel is capable of merging I/O operations. So, if two requests to read data from the disk are adjacent, the Linux kernel may merge them to one before giving them to disk. This metric measures the number of operations that have been merged by the Linux kernel.
|
||||
|
||||
### Total I/O time
|
||||
|
||||
The sum of the duration of all completed I/O operations. This number can exceed the interval if the disk is able to execute multiple I/O operations in parallel.
|
||||
|
||||
### Space usage
|
||||
|
||||
For mounted disks, netdata will provide a chart for their space, with 3 dimensions:
|
||||
|
||||
1. free
|
||||
2. used
|
||||
3. reserved for root
|
||||
|
||||
### inode usage
|
||||
|
||||
For mounted disks, netdata will provide a chart for their inodes (number of file and directories), with 3 dimensions:
|
||||
|
||||
1. free
|
||||
2. used
|
||||
3. reserved for root
|
||||
|
||||
---
|
||||
|
||||
## disk names
|
||||
### disk names
|
||||
|
||||
netdata will automatically set the name of disks on the dashboard, from the mount point they are mounted, of course only when they are mounted. Changes in mount points are not currently detected (you will have to restart netdata to change the name of the disk).
|
||||
|
||||
---
|
||||
|
||||
## performance metrics
|
||||
### performance metrics
|
||||
|
||||
By default netdata will enable monitoring metrics only when they are not zero. If they are constantly zero they are ignored. Metrics that will start having values, after netdata is started, will be detected and charts will be automatically added to the dashboard (a refresh of the dashboard is needed for them to appear though).
|
||||
|
||||
|
|
@ -198,3 +167,35 @@ So, to disable performance metrics for all loop devices you could add `performan
|
|||
performance metrics for disks with major 7 = no
|
||||
```
|
||||
|
||||
## Linux Anti-DDoS
|
||||
|
||||
|
||||
|
||||
---
|
||||
SYNPROXY is a TCP SYN packets proxy. It can be used to protect any TCP server (like a web server) from SYN floods and similar DDos attacks.
|
||||
|
||||
SYNPROXY is a netfilter module, in the Linux kernel (since version 3.12). It is optimized to handle millions of packets per second utilizing all CPUs available without any concurrency locking between the connections.
|
||||
|
||||
The net effect of this, is that the real servers will not notice any change during the attack. The valid TCP connections will pass through and served, while the attack will be stopped at the firewall.
|
||||
|
||||
To use SYNPROXY on your firewall, please follow our setup guides:
|
||||
|
||||
- **[Working with SYNPROXY](https://github.com/firehol/firehol/wiki/Working-with-SYNPROXY)**
|
||||
- **[Working with SYNPROXY and traps](https://github.com/firehol/firehol/wiki/Working-with-SYNPROXY-and-traps)**
|
||||
|
||||
### Real-time monitoring of Linux Anti-DDoS
|
||||
|
||||
netdata is able to monitor in real-time (per second updates) the operation of the Linux Anti-DDoS protection.
|
||||
|
||||
It visualizes 4 charts:
|
||||
|
||||
1. TCP SYN Packets received on ports operated by SYNPROXY
|
||||
2. TCP Cookies (valid, invalid, retransmits)
|
||||
3. Connections Reopened
|
||||
4. Entries used
|
||||
|
||||
Example image:
|
||||
|
||||
|
||||
|
||||
See Linux Anti-DDoS in action at: **[netdata demo site (with SYNPROXY enabled)](http://london.my-netdata.io/#netfilter_synproxy)**
|
||||
|
|
|
|||
|
|
@ -9,6 +9,20 @@
|
|||
5. Allows each **module** to have one or more data collection **jobs**
|
||||
6. Each **job** is collecting one or more metrics from a single data source
|
||||
|
||||
## Pull Request Checklist for Python Plugins
|
||||
|
||||
This is a generic checklist for submitting a new Python plugin for Netdata. It is by no means comprehensive.
|
||||
|
||||
At minimum, to be buildable and testable, the PR needs to include:
|
||||
|
||||
* The module itself, following proper naming conventions: `python.d/<module_dir>/<module_name>.chart.py`
|
||||
* A README.md file for the plugin under `python.d/<module_dir>`.
|
||||
* The configuration file for the module: `conf.d/python.d/<module_name>.conf`. Python config files are in YAML format, and should include comments describing what options are present. The instructions are also needed in the configuration section of the README.md
|
||||
* A basic configuration for the plugin in the appropriate global config file: `conf.d/python.d.conf`, which is also in YAML format. Either add a line that reads `# <module_name>: yes` if the module is to be enabled by default, or one that reads `<module_name>: no` if it is to be disabled by default.
|
||||
* A line for the plugin in `python.d/Makefile.am` under `dist_python_DATA`.
|
||||
* A line for the plugin configuration file in `conf.d/Makefile.am`, under `dist_pythonconfig_DATA`
|
||||
* Optionally, chart information in `web/dashboard_info.js`. This generally involves specifying a name and icon for the section, and may include descriptions for the section or individual charts.
|
||||
|
||||
|
||||
## Disclaimer
|
||||
|
||||
|
|
@ -195,4 +209,4 @@ Sockets are accessed in non-blocking mode with 15 second timeout.
|
|||
|
||||
After every execution of `_get_raw_data` socket is closed, to prevent this module needs to set `_keep_alive` variable to `True` and implement custom `_check_raw_data` method.
|
||||
|
||||
`_check_raw_data` should take raw data and return `True` if all data is received otherwise it should return `False`. Also it should do it in fast and efficient way.
|
||||
`_check_raw_data` should take raw data and return `True` if all data is received otherwise it should return `False`. Also it should do it in fast and efficient way.
|
||||
|
|
|
|||
|
|
@ -106,7 +106,7 @@ the use of `netdata`s ```go_expvar``` module.
|
|||
|
||||
### Using netdata go_expvar module
|
||||
|
||||
The `go_expvar` module is disabled by default. To enable it, edit [`python.d.conf`](../python.d.conf)
|
||||
The `go_expvar` module is disabled by default. To enable it, edit [`python.d.conf`](https://github.com/netdata/netdata/tree/master/collectors/python.d.plugin/python.d.conf)
|
||||
(to edit it on your system run `/etc/netdata/edit-config python.d.conf`), and change the `go_expvar`
|
||||
variable to `yes`:
|
||||
|
||||
|
|
@ -124,7 +124,7 @@ go_expvar: yes
|
|||
...
|
||||
```
|
||||
|
||||
Next, we need to edit the module configuration file (found at [`/etc/netdata/python.d/go_expvar.conf`](go_expvar.conf) by default)
|
||||
Next, we need to edit the module configuration file (found at [`/etc/netdata/python.d/go_expvar.conf`](https://github.com/netdata/netdata/tree/master/collectors/python.d.plugin/go_expvar/go_expvar.conf) by default)
|
||||
(to edit it on your system run `/etc/netdata/edit-config python.d/go_expvar.conf`).
|
||||
The module configuration consists of jobs, where each job can be used to monitor a separate Go application.
|
||||
Let's see a sample job configuration:
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ Charts are created dynamically.
|
|||
|
||||
### configuration
|
||||
|
||||
For detailed configuration information please read [`sensors.conf`](sensors.conf) file.
|
||||
For detailed configuration information please read [`sensors.conf`](https://github.com/netdata/netdata/tree/master/collectors/python.d.plugin/sensors/sensors.conf) file.
|
||||
|
||||
### possible issues
|
||||
|
||||
|
|
|
|||
|
|
@ -1,40 +1,10 @@
|
|||
# springboot
|
||||
|
||||
This module will monitor one or more Java Spring-boot applications depending on configuration.
|
||||
|
||||
It produces following charts:
|
||||
|
||||
1. **Response Codes** in requests/s
|
||||
* 1xx
|
||||
* 2xx
|
||||
* 3xx
|
||||
* 4xx
|
||||
* 5xx
|
||||
* others
|
||||
|
||||
2. **Threads**
|
||||
* daemon
|
||||
* total
|
||||
|
||||
3. **GC Time** in milliseconds and **GC Operations** in operations/s
|
||||
* Copy
|
||||
* MarkSweep
|
||||
* ...
|
||||
|
||||
4. **Heap Mmeory Usage** in KB
|
||||
* used
|
||||
* committed
|
||||
|
||||
### configuration
|
||||
|
||||
Please see the [Monitoring Java Spring Boot Applications](https://github.com/netdata/netdata/wiki/Monitoring-Java-Spring-Boot-Applications) page for detailed info about module configuration.
|
||||
|
||||
---
|
||||
|
||||
# Monitoring Java Spring Boot Applications
|
||||
|
||||
Netdata can be used to monitor running Java [Spring Boot](https://spring.io/) applications that expose their metrics with the use of the **Spring Boot Actuator** included in Spring Boot library.
|
||||
|
||||
## Configuration
|
||||
|
||||
The Spring Boot Actuator exposes these metrics over HTTP and is very easy to use:
|
||||
* add `org.springframework.boot:spring-boot-starter-actuator` to your application dependencies
|
||||
* set `endpoints.metrics.sensitive=false` in your `application.properties`
|
||||
|
|
@ -93,7 +63,30 @@ public class HeapPoolMetrics implements PublicMetrics {
|
|||
|
||||
Please refer [Spring Boot Actuator: Production-ready features](https://docs.spring.io/spring-boot/docs/current/reference/html/production-ready.html) and [81. Actuator - Part IX. ‘How-to’ guides](https://docs.spring.io/spring-boot/docs/current/reference/html/howto-actuator.html) for more information.
|
||||
|
||||
## Using netdata springboot module
|
||||
## Charts
|
||||
|
||||
1. **Response Codes** in requests/s
|
||||
* 1xx
|
||||
* 2xx
|
||||
* 3xx
|
||||
* 4xx
|
||||
* 5xx
|
||||
* others
|
||||
|
||||
2. **Threads**
|
||||
* daemon
|
||||
* total
|
||||
|
||||
3. **GC Time** in milliseconds and **GC Operations** in operations/s
|
||||
* Copy
|
||||
* MarkSweep
|
||||
* ...
|
||||
|
||||
4. **Heap Mmeory Usage** in KB
|
||||
* used
|
||||
* committed
|
||||
|
||||
## Usage
|
||||
|
||||
The springboot module is enabled by default. It looks up `http://localhost:8080/metrics` and `http://127.0.0.1:8080/metrics` to detect Spring Boot application by default. You can change it by editing `/etc/netdata/python.d/springboot.conf` (to edit it on your system run `/etc/netdata/edit-config python.d/springboot.conf`).
|
||||
|
||||
|
|
@ -126,4 +119,4 @@ You can disable the default charts by set `defaults.<chart-id>: false`.
|
|||
|
||||
The dimension name of extras charts should replace `.` to `_`.
|
||||
|
||||
Please check [springboot.conf](springboot.conf) for more examples.
|
||||
Please check [springboot.conf](https://github.com/netdata/netdata/tree/master/collectors/python.d.plugin/springboot/springboot.conf) for more examples.
|
||||
|
|
|
|||
|
|
@ -8,6 +8,6 @@ Charts are created dynamically based on the number of detected sensors.
|
|||
|
||||
### configuration
|
||||
|
||||
For detailed configuration information please read [`w1sensor.conf`](w1sensor.conf) file.
|
||||
For detailed configuration information please read [`w1sensor.conf`](https://github.com/netdata/netdata/tree/master/collectors/python.d.plugin/w1sensor/w1sensor.conf) file.
|
||||
|
||||
---
|
||||
|
|
|
|||
|
|
@ -1,50 +1,34 @@
|
|||
# web_log
|
||||
|
||||
Tails the apache/nginx/lighttpd/gunicorn log files to collect real-time web-server statistics.
|
||||
## Motivation
|
||||
|
||||
It produces following charts:
|
||||
Web server log files exist for more than 20 years. All web servers of all kinds, from all vendors, [since the time NCSA httpd was powering the web](https://en.wikipedia.org/wiki/NCSA_HTTPd), produce log files, saving in real-time all accesses to web sites and APIs.
|
||||
|
||||
1. **Response by type** requests/s
|
||||
* success (1xx, 2xx, 304)
|
||||
* error (5xx)
|
||||
* redirect (3xx except 304)
|
||||
* bad (4xx)
|
||||
* other (all other responses)
|
||||
Yet, after the appearance of google analytics and similar services, and the recent rise of APM (Application Performance Monitoring) with sophisticated time-series databases that collect and analyze metrics at the application level, all these web server log files are mostly just filling our disks, rotated every night without any use whatsoever.
|
||||
|
||||
2. **Response by code family** requests/s
|
||||
* 1xx (informational)
|
||||
* 2xx (successful)
|
||||
* 3xx (redirect)
|
||||
* 4xx (bad)
|
||||
* 5xx (internal server errors)
|
||||
* other (non-standart responses)
|
||||
* unmatched (the lines in the log file that are not matched)
|
||||
netdata turns this "useless" log file, into a powerful performance and health monitoring tool, capable of detecting, **in real-time**, most common web server problems, such as:
|
||||
|
||||
3. **Detailed Response Codes** requests/s (number of responses for each response code family individually)
|
||||
- too many redirects (i.e. **oops!** *this should not redirect clients to itself*)
|
||||
- too many bad requests (i.e. **oops!** *a few files were not uploaded*)
|
||||
- too many internal server errors (i.e. **oops!** *this release crashes too much*)
|
||||
- unreasonably too many requests (i.e. **oops!** *we are under attack*)
|
||||
- unreasonably few requests (i.e. **oops!** *call the network guys*)
|
||||
- unreasonably slow responses (i.e. **oops!** *the database is slow again*)
|
||||
- too few successful responses (i.e. **oops!** *help us God!*)
|
||||
|
||||
4. **Bandwidth** KB/s
|
||||
* received (bandwidth of requests)
|
||||
* send (bandwidth of responses)
|
||||
## Usage
|
||||
|
||||
5. **Timings** ms (request processing time)
|
||||
* min (bandwidth of requests)
|
||||
* max (bandwidth of responses)
|
||||
* average (bandwidth of responses)
|
||||
If netdata is installed on a system running a web server, it will detect it and it will automatically present a series of charts, with information obtained from the web server API, like these (*these do not come from the web server log file*):
|
||||
|
||||
6. **Request per url** requests/s (configured by user)
|
||||
|
||||
*[**netdata**](https://my-netdata.io/) charts based on metrics collected by querying the `nginx` API (i.e. `/stab_status`).*
|
||||
|
||||
7. **Http Methods** requests/s (requests per http method)
|
||||
> [**netdata**](https://my-netdata.io/) supports `apache`, `nginx`, `lighttpd` and `tomcat`. To obtain real-time information from a web server API, the web server needs to expose it. For directions on configuring your web server, check [`/etc/netdata/python.d/`](https://github.com/netdata/netdata/tree/master/conf.d/python.d). There is a file there for each web server.
|
||||
|
||||
8. **Http Versions** requests/s (requests per http version)
|
||||
## Configuration
|
||||
|
||||
9. **IP protocols** requests/s (requests per ip protocol version)
|
||||
[**netdata**](https://my-netdata.io/) has a powerful `web_log` plugin, capable of incrementally parsing any number of web server log files. This plugin is automatically started with [**netdata**](https://my-netdata.io/) and comes, pre-configured, for finding web server log files on popular distributions. Its configuration is at [`/etc/netdata/python.d/web_log.conf`](https://github.com/netdata/netdata/blob/master/conf.d/python.d/web_log.conf), like this:
|
||||
|
||||
10. **Current Poll Unique Client IPs** unique ips/s (unique client IPs per data collection iteration)
|
||||
|
||||
11. **All Time Unique Client IPs** unique ips/s (unique client IPs since the last restart of netdata)
|
||||
|
||||
|
||||
### configuration
|
||||
|
||||
```yaml
|
||||
nginx_log:
|
||||
|
|
@ -59,6 +43,159 @@ apache_log:
|
|||
observium : 'observium'
|
||||
```
|
||||
|
||||
Module has preconfigured jobs for nginx, apache and gunicorn on various distros.
|
||||
Theodule has preconfigured jobs for nginx, apache and gunicorn on various distros.
|
||||
You can add one such section, for each of your web server log files.
|
||||
|
||||
> **Important**<br/>Keep in mind [**netdata**](https://my-netdata.io/) runs as user `netdata`. So, make sure user `netdata` has access to the logs directory and can read the log file.
|
||||
|
||||
## Charts
|
||||
|
||||
Once you have all log files configured and [**netdata**](https://my-netdata.io/) restarted, **for each log file** you will get a section at the [**netdata**](https://my-netdata.io/) dashboard, with the following charts.
|
||||
|
||||
### responses by status
|
||||
|
||||
In this chart we tried to provide a meaningful status for all responses. So:
|
||||
|
||||
- `success` counts all the valid responses (i.e. `1xx` informational, `2xx` successful and `304` not modified).
|
||||
- `error` are `5xx` internal server errors. These are very bad, they mean your web site or API is facing difficulties.
|
||||
- `redirect` are `3xx` responses, except `304`. All `3xx` are redirects, but `304` means "not modified" - it tells the browsers the content they already have is still valid and can be used as-is. So, we decided to account it as a successful response.
|
||||
- `bad` are bad requests that cannot be served.
|
||||
- `other` as all the other, non-standard, types of responses.
|
||||
|
||||
|
||||
|
||||
### Responses by type
|
||||
|
||||
Then, we group all responses by code family, without interpreting their meaning.
|
||||
**Response by type** requests/s
|
||||
* success (1xx, 2xx, 304)
|
||||
* error (5xx)
|
||||
* redirect (3xx except 304)
|
||||
* bad (4xx)
|
||||
* other (all other responses)
|
||||
|
||||
|
||||
|
||||
### Responses by code family
|
||||
|
||||
Here we show all the response codes in detail.
|
||||
|
||||
**Response by code family** requests/s
|
||||
* 1xx (informational)
|
||||
* 2xx (successful)
|
||||
* 3xx (redirect)
|
||||
* 4xx (bad)
|
||||
* 5xx (internal server errors)
|
||||
* other (non-standart responses)
|
||||
* unmatched (the lines in the log file that are not matched)
|
||||
|
||||
|
||||
|
||||
|
||||
>**Important**<br/>If your application is using hundreds of non-standard response codes, your browser may become slow while viewing this chart, so we have added a configuration [option to disable this chart](https://github.com/netdata/netdata/blob/419cd0a237275e5eeef3f92dcded84e735ee6c58/conf.d/python.d/web_log.conf#L63).
|
||||
|
||||
### Detailed Response Codes
|
||||
|
||||
Number of responses for each response code family individually (requests/s)
|
||||
|
||||
### bandwidth
|
||||
|
||||
This is a nice view of the traffic the web server is receiving and is sending.
|
||||
|
||||
What is important to know for this chart, is that the bandwidth used for each request and response is accounted at the time the log is written. Since [**netdata**](https://my-netdata.io/) refreshes this chart every single second, you may have unrealistic spikes is the size of the requests or responses is too big. The reason is simple: a response may have needed 1 minute to be completed, but all the bandwidth used during that minute for the specific response will be accounted at the second the log line is written.
|
||||
|
||||
As the legend on the chart suggests, you can use FireQoS to setup QoS on the web server ports and IPs to accurately measure the bandwidth the web server is using. Actually, [there may be a few more reasons to install QoS on your servers](https://github.com/netdata/netdata/wiki/You-should-install-QoS-on-all-your-servers)...
|
||||
|
||||
**Bandwidth** KB/s
|
||||
* received (bandwidth of requests)
|
||||
* send (bandwidth of responses)
|
||||
|
||||
|
||||
|
||||
> **Important**<br/>Most web servers do not log the request size by default.<br/>So, [unless you have configured your web server to log the size of requests](https://github.com/netdata/netdata/blob/419cd0a237275e5eeef3f92dcded84e735ee6c58/conf.d/python.d/web_log.conf#L76-L89), the `received` dimension will be always zero.
|
||||
|
||||
### timings
|
||||
|
||||
[**netdata**](https://my-netdata.io/) will also render the `minimum`, `average` and `maximum` time the web server needed to respond to requests.
|
||||
|
||||
Keep in mind most web servers timings start at the reception of the full request, until the dispatch of the last byte of the response. So, they include network latencies of responses, but they do not include network latencies of requests.
|
||||
|
||||
**Timings** ms (request processing time)
|
||||
* min (bandwidth of requests)
|
||||
* max (bandwidth of responses)
|
||||
* average (bandwidth of responses)
|
||||
|
||||
|
||||
|
||||
> **Important**<br/>Most web servers do not log timing information by default.<br/>So, [unless you have configured your web server to also log timings](https://github.com/netdata/netdata/blob/419cd0a237275e5eeef3f92dcded84e735ee6c58/conf.d/python.d/web_log.conf#L76-L89), this chart will not exist.
|
||||
|
||||
### URL patterns
|
||||
|
||||
This is a very interesting chart. It is configured entirely by you.
|
||||
|
||||
[**netdata**](https://my-netdata.io/) can map the URLs found in the log file into categories. You can define these categories, by providing names and regular expressions in `web_log.conf`.
|
||||
|
||||
So, this configuration:
|
||||
|
||||
```yaml
|
||||
nginx_netdata: # name the charts
|
||||
path: '/var/log/nginx/access.log' # web server log file
|
||||
categories:
|
||||
badges : '^/api/v1/badge\.svg'
|
||||
charts : '^/api/v1/(data|chart|charts)'
|
||||
registry : '^/api/v1/registry'
|
||||
alarms : '^/api/v1/alarm'
|
||||
allmetrics : '^/api/v1/allmetrics'
|
||||
api_other : '^/api/'
|
||||
netdata_conf: '^/netdata.conf'
|
||||
api_old : '^/(data|datasource|graph|list|all\.json)'
|
||||
```
|
||||
|
||||
Produces the following chart. The `categories` section is matched in the order given. So, pay attention to the order you give your patterns.
|
||||
|
||||
|
||||
|
||||
### HTTP versions
|
||||
|
||||
This chart breaks down requests by HTTP version used.
|
||||
|
||||
|
||||
|
||||
### IP versions
|
||||
|
||||
This one provides requests per IP version used by the clients (`IPv4`, `IPv6`).
|
||||
|
||||
|
||||
|
||||
### Unique clients
|
||||
|
||||
The last charts are about the unique IPs accessing your web server.
|
||||
|
||||
**Current Poll Unique Client IPs** unique ips/s. This one counts the unique IPs for each data collection iteration (i.e. **unique clients per second**).
|
||||
|
||||
|
||||
|
||||
**All Time Unique Client IPs** unique ips/s. Counts the unique IPs, since the last [**netdata**](https://my-netdata.io/) restart.
|
||||
|
||||
|
||||
|
||||
>**Important**<br/>To provide this information `web_log` plugin keeps in memory all the IPs seen by the web server. Although this does not require so much memory, if you have a web server with several million unique client IPs, we suggest to [disable this chart](https://github.com/netdata/netdata/blob/419cd0a237275e5eeef3f92dcded84e735ee6c58/conf.d/python.d/web_log.conf#L64).
|
||||
|
||||
|
||||
## Alarms
|
||||
|
||||
The magic of [**netdata**](https://my-netdata.io/) is that all metrics are collected per second, and all metrics can be used or correlated to provide real-time alarms. Out of the box, [**netdata**](https://my-netdata.io/) automatically attaches the [following alarms](https://github.com/netdata/netdata/blob/master/conf.d/health.d/web_log.conf) to all `web_log` charts (i.e. to all log files configured, individually):
|
||||
|
||||
alarm|description|minimum<br/>requests|warning|critical
|
||||
:-------|-------|:------:|:-----:|:------:
|
||||
`1m_redirects`|The ratio of HTTP redirects (3xx except 304) over all the requests, during the last minute.<br/> <br/>*Detects if the site or the web API is suffering from too many or circular redirects.*<br/> <br/>(i.e. **oops!** *this should not redirect clients to itself*)|120/min|> 20%|> 30%
|
||||
`1m_bad_requests`|The ratio of HTTP bad requests (4xx) over all the requests, during the last minute.<br/> <br/>*Detects if the site or the web API is receiving too many bad requests, including `404`, not found.*<br/> <br/>(i.e. **oops!** *a few files were not uploaded*)|120/min|> 30%|> 50%
|
||||
`1m_internal_errors`|The ratio of HTTP internal server errors (5xx), over all the requests, during the last minute.<br/> <br/>*Detects if the site is facing difficulties to serve requests.*<br/> <br/>(i.e. **oops!** *this release crashes too much*)|120/min|> 2%|> 5%
|
||||
`5m_requests_ratio`|The percentage of successful web requests of the last 5 minutes, compared with the previous 5 minutes.<br/> <br/>*Detects if the site or the web API is suddenly getting too many or too few requests.*<br/> <br/>(i.e. too many = **oops!** *we are under attack*)<br/>(i.e. too few = **oops!** *call the network guys*)|120/5min|> double or < half|> 4x or < 1/4x
|
||||
`web_slow`|The average time to respond to requests, over the last 1 minute, compared to the average of last 10 minutes.<br/> <br/>*Detects if the site or the web API is suddenly a lot slower.*<br/> <br/>(i.e. **oops!** *the database is slow again*)|120/min|> 2x|> 4x
|
||||
`1m_successful`|The ratio of successful HTTP responses (1xx, 2xx, 304) over all the requests, during the last minute.<br/> <br/>*Detects if the site or the web API is performing within limits.*<br/> <br/>(i.e. **oops!** *help us God!*)|120/min|< 85%|< 75%
|
||||
|
||||
The column `minimum requests` state the minimum number of requests required for the alarm to be evaluated. We found that when the site is receiving requests above this rate, these alarms are pretty accurate (i.e. no false-positives).
|
||||
|
||||
[**netdata**](https://my-netdata.io/) alarms are [user configurable](https://github.com/netdata/netdata/tree/master/conf.d/health.d). So, even [`web_log` alarms can be adapted to your needs](https://github.com/netdata/netdata/blob/master/conf.d/health.d/web_log.conf).
|
||||
|
||||
---
|
||||
|
|
|
|||
|
|
@ -9,84 +9,78 @@ Netdata monitors `tc` QoS classes for all interfaces.
|
|||
If you also use [FireQOS](http://firehol.org/tutorial/fireqos-new-user/) it will collect
|
||||
interface and class names.
|
||||
|
||||
There is a [shell helper](tc-qos-helper.sh.in) for this (all parsing is done by the plugin
|
||||
There is a [shell helper](https://github.com/netdata/netdata/tree/master/collectors/tc.plugin/tc-qos-helper.sh.in) for this (all parsing is done by the plugin
|
||||
in `C` code - this shell script is just a configuration for the command to run to get `tc` output).
|
||||
|
||||
The source of the tc plugin is [here](plugin_tc.c). It is somewhat complex, because a state
|
||||
The source of the tc plugin is [here](https://github.com/netdata/netdata/tree/master/collectors/tc.plugin/plugin_tc.c). It is somewhat complex, because a state
|
||||
machine was needed to keep track of all the `tc` classes, including the pseudo classes tc
|
||||
dynamically creates.
|
||||
|
||||
## Motivation
|
||||
|
||||
One category of metrics missing in Linux monitoring, is bandwidth consumption for each open
|
||||
socket (inbound and outbound traffic). So, you cannot tell how much bandwidth your web server,
|
||||
your database server, your backup, your ssh sessions, etc are using.
|
||||
One category of metrics missing in Linux monitoring, is bandwidth consumption for each open socket (inbound and outbound traffic). So, you cannot tell how much bandwidth your web server, your database server, your backup, your ssh sessions, etc are using.
|
||||
|
||||
To solve this problem, the most *adventurous* Linux monitoring tools install kernel modules to
|
||||
capture all traffic, analyze it and provide reports per application. A lot of work, CPU intensive
|
||||
and with a great degree of risk (due to the kernel modules involved which might affect the
|
||||
stability of the whole system). Not to mention that such solutions are probably better suited
|
||||
for a core linux router in your network.
|
||||
To solve this problem, the most *adventurous* Linux monitoring tools install kernel modules to capture all traffic, analyze it and provide reports per application. A lot of work, CPU intensive and with a great degree of risk (due to the kernel modules involved which might affect the stability of the whole system). Not to mention that such solutions are probably better suited for a core linux router in your network.
|
||||
|
||||
Others use NFACCT, the netfilter accounting module which is already part of the Linux firewall.
|
||||
However, this would require configuring a firewall on every system you want to measure bandwidth.
|
||||
Others use NFACCT, the netfilter accounting module which is already part of the Linux firewall. However, this would require configuring a firewall on every system you want to measure bandwidth (just FYI, I do install a firewall on every server - and I strongly advise you to do so too - but configuring accounting on all servers seems overkill when you don't really need it for billing purposes).
|
||||
|
||||
QoS monitoring attempts to solve this in a much cleaner way.
|
||||
**There is however a much simpler approach**.
|
||||
|
||||
## Introduction to QoS
|
||||
## QoS
|
||||
|
||||
One of the features the Linux kernel has, but it is rarely used, is its ability to
|
||||
**apply QoS on traffic**. Even most interesting is that it can apply QoS to **both inbound and
|
||||
outbound traffic**.
|
||||
One of the features the Linux kernel has, but it is rarely used, is its ability to **apply QoS on traffic**. Even most interesting is that it can apply QoS to **both inbound and outbound traffic**.
|
||||
|
||||
QoS is about 2 features:
|
||||
|
||||
1. **Classify traffic**
|
||||
|
||||
Classification is the process of organizing traffic in groups, called **classes**.
|
||||
Classification can evaluate every aspect of network packets, like source and destination ports,
|
||||
source and destination IPs, netfilter marks, etc.
|
||||
Classification is the process of organizing traffic in groups, called **classes**. Classification can evaluate every aspect of network packets, like source and destination ports, source and destination IPs, netfilter marks, etc.
|
||||
|
||||
When you classify traffic, you just assign a label to it. For example **I call `web server`
|
||||
traffic, the traffic from my server's tcp/80, tcp/443 and to my server's tcp/80, tcp/443,
|
||||
while I call `web surfing` all other tcp/80 and tcp/443 traffic**. You can use any combinations
|
||||
you like. There is no limit.
|
||||
When you classify traffic, you just assign a label to it. Of course classes have some properties themselves (like queuing mechanisms), but let's say it is that simple: **a label**. For example **I call `web server` traffic, the traffic from my server's tcp/80, tcp/443 and to my server's tcp/80, tcp/443, while I call `web surfing` all other tcp/80 and tcp/443 traffic**. You can use any combinations you like. There is no limit.
|
||||
|
||||
2. **Apply traffic shaping rules to these classes**
|
||||
|
||||
Traffic shaping is used to control how network interface bandwidth should be shared among the
|
||||
classes. Of course we are not interested for this feature to just monitor the traffic.
|
||||
Classification will be enough for monitoring everything.
|
||||
Traffic shaping is used to control how network interface bandwidth should be shared among the classes. Normally, you need to do this, when there is not enough bandwidth to satisfy all the demand, or when you want to control the supply of bandwidth to certain services. Of course classification is sufficient for monitoring traffic, but traffic shaping is also quite important, as we will explain in the next section.
|
||||
|
||||
The key reasons of applying QoS on all servers (even cloud ones) are:
|
||||
## Why you want QoS
|
||||
|
||||
- **ensure administrative tasks (like ssh, dns, etc) will always have a small but guaranteed
|
||||
bandwidth.** QoS can guarantee that services like ssh, dns, ntp, etc will always have a small
|
||||
supply of bandwidth. So, no matter what happens, you will be able to ssh to your server and
|
||||
DNS will always work.
|
||||
1. **Monitoring the bandwidth used by services**
|
||||
|
||||
- **ensure other administrative tasks will not monopolize all the available bandwidth.**
|
||||
Services like backups, file copies, database dumps, etc can easily monopolize all the
|
||||
available bandwidth. It is common for example a nightly backup, or a huge file transfer
|
||||
to negatively influence the end-user experience. QoS can fix that.
|
||||
netdata provides wonderful real-time charts, like this one (wait to see the orange `rsync` part):
|
||||
|
||||
- **ensure each end-user connection will get a fair cut of the available bandwidth.**
|
||||
Several QoS queuing disciplines in Linux do this automatically, without any configuration from you.
|
||||
The result is that new sockets are favored over older ones, so that users will get a snappier
|
||||
experience, while others are transferring large amounts of traffic.
|
||||
|
||||
- **protect the servers from DDoS attacks.**
|
||||
When your system is under a DDoS attack, it will get a lot more bandwidth compared to the one it
|
||||
can handle and probably your applications will crash. Setting a limit on the inbound traffic using
|
||||
QoS, will protect your servers (throttle the requests) and depending on the size of the attack may
|
||||
allow your legitimate users to access the server, while the attack is taking place.
|
||||
|
||||
|
||||
2. **Ensure sensitive administrative tasks will not starve for bandwidth**
|
||||
|
||||
Once **traffic classification** is applied, netdata can visualize the bandwidth consumption per
|
||||
class in real-time (no configuration is needed for netdata - it will figure it out).
|
||||
Have you tried to ssh to a server when the network is congested? If you have, you already know it does not work very well. QoS can guarantee that services like ssh, dns, ntp, etc will always have a small supply of bandwidth. So, no matter what happens, you will be able to ssh to your server and DNS will always work.
|
||||
|
||||
QoS, is extremely light. You will configure it once, and this is it. It will not bother you again
|
||||
and it will not use any noticeable CPU resources, especially on application and database servers.
|
||||
3. **Ensure administrative tasks will not monopolize all the bandwidth**
|
||||
|
||||
Services like backups, file copies, database dumps, etc can easily monopolize all the available bandwidth. It is common for example a nightly backup, or a huge file transfer to negatively influence the end-user experience. QoS can fix that.
|
||||
|
||||
4. **Ensure each end-user connection will get a fair cut of the available bandwidth.**
|
||||
|
||||
Several QoS queuing disciplines in Linux do this automatically, without any configuration from you. The result is that new sockets are favored over older ones, so that users will get a snappier experience, while others are transferring large amounts of traffic.
|
||||
|
||||
5. **Protect the servers from DDoS attacks.**
|
||||
|
||||
When your system is under a DDoS attack, it will get a lot more bandwidth compared to the one it can handle and probably your applications will crash. Setting a limit on the inbound traffic using QoS, will protect your servers (throttle the requests) and depending on the size of the attack may allow your legitimate users to access the server, while the attack is taking place.
|
||||
|
||||
Using QoS together with a [SYNPROXY](https://github.com/netdata/netdata/wiki/Monitoring-SYNPROXY) will provide a great degree of protection against most DDoS attacks. Actually when I wrote that article, a few folks tried to DDoS the netdata demo site to see in real-time the SYNPROXY operation. They did not do it right, but anyway a great deal of requests reached the netdata server. What saved netdata was QoS. The netdata demo server has QoS installed, so the requests were throttled and the server did not even reach the point of resource starvation. Read about it [here](https://github.com/netdata/netdata/wiki/Monitoring-SYNPROXY#a-note-for-ddos-testers).
|
||||
|
||||
On top of all these, QoS is extremely light. You will configure it once, and this is it. It will not bother you again and it will not use any noticeable CPU resources, especially on application and database servers.
|
||||
|
||||
- ensure administrative tasks (like ssh, dns, etc) will always have a small but guaranteed bandwidth. So, no matter what happens, I will be able to ssh to my server and DNS will work.
|
||||
|
||||
- ensure other administrative tasks will not monopolize all the available bandwidth. So, my nightly backup will not hurt my users, a developer that is copying files over the net will not get all the available bandwidth, etc.
|
||||
|
||||
- ensure each end-user connection will get a fair cut of the available bandwidth.
|
||||
|
||||
Once **traffic classification** is applied, we can use **[netdata](https://github.com/netdata/netdata)** to visualize the bandwidth consumption per class in real-time (no configuration is needed for netdata - it will figure it out).
|
||||
|
||||
QoS, is extremely light. You will configure it once, and this is it. It will not bother you again and it will not use any noticeable CPU resources, especially on application and database servers.
|
||||
|
||||
---
|
||||
|
||||
## QoS in Linux? Have you lost your mind?
|
||||
|
||||
|
|
@ -94,28 +88,35 @@ Yes I know... but no, I have not!
|
|||
|
||||
Of course, `tc` is probably **the most undocumented, complicated and unfriendly** command in Linux.
|
||||
|
||||
For example, for matching a simple port range in `tc`, e.g. all the high ports, from 1025 to 65535
|
||||
inclusive, you have to match these:
|
||||
For example, do you know that for matching a simple port range in `tc`, e.g. all the high ports, from 1025 to 65535 inclusive, you have to match these:
|
||||
|
||||
```
|
||||
1025/0xffff 1026/0xfffe 1028/0xfffc 1032/0xfff8 1040/0xfff0
|
||||
1056/0xffe0 1088/0xffc0 1152/0xff80 1280/0xff00 1536/0xfe00
|
||||
2048/0xf800 4096/0xf000 8192/0xe000 16384/0xc000 32768/0x8000
|
||||
1025/0xffff
|
||||
1026/0xfffe
|
||||
1028/0xfffc
|
||||
1032/0xfff8
|
||||
1040/0xfff0
|
||||
1056/0xffe0
|
||||
1088/0xffc0
|
||||
1152/0xff80
|
||||
1280/0xff00
|
||||
1536/0xfe00
|
||||
2048/0xf800
|
||||
4096/0xf000
|
||||
8192/0xe000
|
||||
16384/0xc000
|
||||
32768/0x8000
|
||||
```
|
||||
|
||||
I know what you are thinking right now! **And I agree!**
|
||||
|
||||
This is why I wrote **[FireQOS](https://firehol.org/tutorial/fireqos-new-user/)**, a tool to
|
||||
simplify QoS management in Linux.
|
||||
This is why I wrote **[FireQOS](https://firehol.org/tutorial/fireqos-new-user/)**, a tool to simplify QoS management in Linux.
|
||||
|
||||
The **[FireHOL](https://firehol.org/)** package already distributes **[FireQOS](https://firehol.org/tutorial/fireqos-new-user/)**.
|
||||
Check the **[FireQOS tutorial](https://firehol.org/tutorial/fireqos-new-user/)**
|
||||
to learn how to write your own QoS configuration.
|
||||
The **[FireHOL](https://firehol.org/)** package already distributes **[FireQOS](https://firehol.org/tutorial/fireqos-new-user/)**. Check the **[FireQOS tutorial](https://firehol.org/tutorial/fireqos-new-user/)** to learn how to write your own QoS configuration.
|
||||
|
||||
With **[FireQOS](https://firehol.org/tutorial/fireqos-new-user/)**, it is **really simple for everyone
|
||||
to use QoS in Linux**. Just install the package `firehol`. It should already be available for your
|
||||
distribution. If not, check the **[FireHOL Installation Guide](https://firehol.org/installing/)**.
|
||||
After that, you will have the `fireqos` command.
|
||||
With **[FireQOS](https://firehol.org/tutorial/fireqos-new-user/)**, it is **really simple for everyone to use QoS in Linux**. Just install the package `firehol`. It should already be available for your distribution. If not, check the **[FireHOL Installation Guide](https://firehol.org/installing/)**. After that, you will have the `fireqos` command which uses a configuration like the following:
|
||||
|
||||
## QoS Configuration
|
||||
|
||||
This is the file `/etc/firehol/fireqos.conf` we use at the netdata demo site:
|
||||
|
||||
|
|
@ -157,14 +158,9 @@ This is the file `/etc/firehol/fireqos.conf` we use at the netdata demo site:
|
|||
match input src 10.2.3.5
|
||||
```
|
||||
|
||||
Nothing more is needed. You just run `fireqos start` to apply this configuration, restart netdata
|
||||
and you have real-time visualization of the bandwidth consumption of your applications. FireQOS is
|
||||
not a daemon. It will just convert the configuration to `tc` commands. It will run them and it will
|
||||
exit.
|
||||
Nothing more is needed. You just run `fireqos start` to apply this configuration, restart netdata and you have real-time visualization of the bandwidth consumption of your applications. FireQOS is not a daemon. It will just convert the configuration to `tc` commands. It will run them and it will exit.
|
||||
|
||||
**IMPORTANT**: If you copy this configuration to apply it to your system, please adapt the
|
||||
speeds - experiment in non-production environments to learn the tool, before applying it on
|
||||
your servers.
|
||||
**IMPORTANT**: If you copy this configuration to apply it to your system, please adapt the speeds - experiment in non-production environments to learn the tool, before applying it on your servers.
|
||||
|
||||
And this is what you are going to get:
|
||||
|
||||
|
|
@ -174,10 +170,11 @@ And this is what you are going to get:
|
|||
|
||||
## More examples:
|
||||
|
||||
This is QoS from a linux router. Check these features:
|
||||
This is QoS from my home linux router. Check these features:
|
||||
|
||||
1. It is real-time (per second updates)
|
||||
2. QoS really works in Linux - check that the `background` traffic is squeezed when `surfing` needs it.
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -5,4 +5,5 @@ MAINTAINERCLEANFILES= $(srcdir)/Makefile.in
|
|||
|
||||
dist_noinst_DATA = \
|
||||
README.md \
|
||||
config/README.md \
|
||||
$(NULL)
|
||||
|
|
|
|||
|
|
@ -1,6 +1,87 @@
|
|||
# Netdata daemon
|
||||
# Running the Netdata Daemon
|
||||
|
||||
## Starting netdata
|
||||
|
||||
- You can start netdata by executing it with `/usr/sbin/netdata` (the installer will also start it).
|
||||
|
||||
- You can stop netdata by killing it with `killall netdata`.
|
||||
You can stop and start netdata at any point. Netdata saves on exit its round robbin
|
||||
database to `/var/cache/netdata` so that it will continue from where it stopped the last time.
|
||||
|
||||
Access to the web site, for all graphs, is by default on port `19999`, so go to:
|
||||
|
||||
```
|
||||
http://127.0.0.1:19999/
|
||||
```
|
||||
|
||||
You can get the running config file at any time, by accessing `http://127.0.0.1:19999/netdata.conf`.
|
||||
|
||||
### Starting netdata at boot
|
||||
|
||||
In the `system` directory you can find scripts and configurations for the various distros.
|
||||
|
||||
#### systemd
|
||||
|
||||
The installer already installs `netdata.service` if it detects a systemd system.
|
||||
|
||||
To install `netdata.service` by hand, run:
|
||||
|
||||
```sh
|
||||
# stop netdata
|
||||
killall netdata
|
||||
|
||||
# copy netdata.service to systemd
|
||||
cp system/netdata.service /etc/systemd/system/
|
||||
|
||||
# let systemd know there is a new service
|
||||
systemctl daemon-reload
|
||||
|
||||
# enable netdata at boot
|
||||
systemctl enable netdata
|
||||
|
||||
# start netdata
|
||||
systemctl start netdata
|
||||
```
|
||||
|
||||
#### init.d
|
||||
|
||||
In the system directory you can find `netdata-lsb`. Copy it to the proper place according to your distribution documentation. For Ubuntu, this can be done via running the following commands as root.
|
||||
|
||||
```sh
|
||||
# copy the netdata startup file to /etc/init.d
|
||||
cp system/netdata-lsb /etc/init.d/netdata
|
||||
|
||||
# make sure it is executable
|
||||
chmod +x /etc/init.d/netdata
|
||||
|
||||
# enable it
|
||||
update-rc.d netdata defaults
|
||||
```
|
||||
|
||||
#### openrc (gentoo)
|
||||
|
||||
In the `system` directory you can find `netdata-openrc`. Copy it to the proper place according to your distribution documentation.
|
||||
|
||||
#### CentOS / Red Hat Enterprise Linux
|
||||
|
||||
For older versions of RHEL/CentOS that don't have systemd, an init script is included in the system directory. This can be installed by running the following commands as root.
|
||||
|
||||
```sh
|
||||
# copy the netdata startup file to /etc/init.d
|
||||
cp system/netdata-init-d /etc/init.d/netdata
|
||||
|
||||
# make sure it is executable
|
||||
chmod +x /etc/init.d/netdata
|
||||
|
||||
# enable it
|
||||
chkconfig --add netdata
|
||||
```
|
||||
|
||||
_There have been some recent work on the init script, see PR https://github.com/netdata/netdata/pull/403_
|
||||
|
||||
#### other systems
|
||||
|
||||
You can start netdata by running it from `/etc/rc.local` or equivalent.
|
||||
|
||||
## Command line options
|
||||
|
||||
|
|
|
|||
195
daemon/config/README.md
Executable file
195
daemon/config/README.md
Executable file
|
|
@ -0,0 +1,195 @@
|
|||
# Configuration Guide
|
||||
|
||||
Configuration files are placed in `/etc/netdata`.
|
||||
|
||||
## Netdata Daemon
|
||||
|
||||
The daemon configuration file is read from `/etc/netdata/netdata.conf`.
|
||||
|
||||
In this file you can configure all aspects of netdata. Netdata provides configuration settings for plugins and charts found when started. You can find all these settings, with their default values, by accessing the URL `https://netdata.server.hostname:19999/netdata.conf`. For example check the configuration file of [netdata.firehol.org](http://netdata.firehol.org/netdata.conf).
|
||||
|
||||
The configuration file has sections stated with `[section]`. There will be the following sections:
|
||||
|
||||
1. `[global]` for global netdata daemon options
|
||||
2. `[plugins]` for controlling which plugins the netdata will use
|
||||
3. `[plugin:NAME]` one such section for each plugin enabled
|
||||
4. `[CHART_NAME]` once such section for each chart defined
|
||||
|
||||
The configuration file is a `name = value` dictionary. Netdata will not complain if you set options unknown to it. When you check the running configuration by accessing the URL `/netdata.conf` on your netdata server, netdata will add a comment on settings it does not currently use.
|
||||
|
||||
### [global] section options
|
||||
|
||||
|
||||
setting | default | info
|
||||
:------:|:-------:|:----
|
||||
hostname|auto-detected|The hostname of the computer running netdata.
|
||||
history|3600|The number of entries the netdata daemon will by default keep in memory for each chart dimension. This setting can also be configured per chart. Check [Memory Requirements](Memory-Requirements) for more information.
|
||||
config directory|`/etc/netdata`|The directory configuration files are kept.
|
||||
plugins directory|`/usr/libexec/netdata/plugins.d`|The directory plugin programs are kept. This setting supports multiple directories, space separated. If any directory path contains spaces, enclose it in single or double quotes.
|
||||
web files directory|`/usr/share/netdata/web`|The directory the web static files are kept.
|
||||
cache directory|`/var/cache/netdata`|The directory the memory database will be stored if and when netdata exits. Netdata will re-read the database when it will start again, to continue from the same point.
|
||||
log directory|`/var/log/netdata`|The directory in which the log files are kept.
|
||||
host access prefix|*empty*|This is used in docker environments where /proc, /sys, etc have to be accessed via another path. You may also have to set SYS_PTRACE capability on the docker for this work. Check [issue 43](https://github.com/netdata/netdata/issues/43).
|
||||
debug flags|0x00000000|Bitmap of debug options to enable. For more information check [Tracing Options](Tracing-Options).
|
||||
memory deduplication (ksm)|yes|When set to `yes`, netdata will offer its in-memory round robin database to kernel same page merging (KSM) for deduplication. For more information check [[Memory Deduplication - Kernel Same Page Merging - KSM]]
|
||||
debug log|`/var/log/netdata/debug.log`|The filename to save debug information. This file will not be created is debugging is not enabled. You can also set it to `syslog` to send the debug messages to syslog, or `none` to disable this log. For more information check [Tracing Options](Tracing-Options).
|
||||
error log|`/var/log/netdata/error.log`|The filename to save error messages for netdata daemon and all plugins (`stderr` is sent here for all netdata programs, including the plugins). You can also set it to `syslog` to send the errors to syslog, or `none` to disable this log.
|
||||
access log|`/var/log/netdata/access.log`|The filename to save the log of web clients accessing netdata charts. You can also set it to `syslog` to send the access log to syslog, or `none` to disable this log.
|
||||
memory mode|save|When set to `save` netdata will save its round robin database on exit and load it on startup. When set to `map` the cache files will be updated in real time (check `man mmap` - do not set this on systems with heavy load or slow disks - the disks will continuously sync the in-memory database of netdata). When set to `ram` the round robin database will be temporary and it will be lost when netdata exits.
|
||||
update every|1|The frequency in seconds, for data collection. For more information see [Performance](Performance).
|
||||
run as user|`netdata`|The user netdata will run as.
|
||||
web files owner|`netdata`|The user that owns the web static files. Netdata will refuse to serve a file that is not owned by this user, even if it has read access to that file. If the user given is not found, netdata will only serve files owned by user given in `run as user`.
|
||||
http port listen backlog|100|The port backlog. Check `man 2 listen`.
|
||||
default port|19999|The default port to listen for web clients.
|
||||
bind to|`*`|The IP address and port to listen to. This is a space separated list of IPv4 or IPv6 address and ports. The default will bind to all IP addresses. Example: `bind to = 127.0.0.1:19999 10.11.12.1:19998 [::1]:19999`.
|
||||
disconnect idle web clients after seconds|60|The time in seconds to disconnect web clients after being totally idle.
|
||||
enable web responses gzip compression|yes|When set to `yes`, netdata web responses will be GZIP compressed, if the web client accepts such responses.
|
||||
|
||||
##### netdata process priority
|
||||
|
||||
By default, netdata runs with the `idle` process scheduler, which assigns CPU resources to netdata, only when the system has such resources to spare.
|
||||
|
||||
The following `netdata.conf` settings control this:
|
||||
|
||||
```
|
||||
[global]
|
||||
process scheduling policy = idle
|
||||
process scheduling priority = 0
|
||||
process nice level = 19
|
||||
```
|
||||
|
||||
The policies supported by netdata are `idle` (the netdata default), `other` (also as `nice`), `batch`, `rr`, `fifo`. netdata also recognizes `keep` and `none` to keep the current settings without changing them.
|
||||
|
||||
For `other`, `nice` and `batch`, the setting `process nice level = 19` is activated to configure the nice level of netdata. Nice gets values -20 (highest) to 19 (lowest).
|
||||
|
||||
For `rr` and `fifo`, the setting `process scheduling priority = 0` is activated to configure the priority of the relative scheduling policy. Priority gets values 1 (lowest) to 99 (highest).
|
||||
|
||||
For the details of each scheduler, see `man sched_setscheduler` and `man sched`.
|
||||
|
||||
When netdata is running under systemd, it can only lower its priority (the default is `other` with `nice level = 0`). If you want to make netdata to get more CPU than that, you will need to set in `netdata.conf`:
|
||||
|
||||
```
|
||||
[global]
|
||||
process scheduling policy = keep
|
||||
```
|
||||
|
||||
and edit `/etc/systemd/system/netdata.service` and add:
|
||||
|
||||
```
|
||||
CPUSchedulingPolicy=other | batch | idle | fifo | rr
|
||||
CPUSchedulingPriority=99
|
||||
Nice=-10
|
||||
```
|
||||
|
||||
|
||||
### [plugins] section options
|
||||
|
||||
In this section there will be a boolean (`yes`/`no`) option for each plugin. Additionally, there will be the following options:
|
||||
|
||||
setting | default | info
|
||||
:------:|:-------:|:----
|
||||
checks|no|This is a debugging plugin for the internal latency of netdata.
|
||||
enable running new plugins|yes|When set to `yes`, netdata will enable plugins not configured specifically for them. Setting this to `no` will disable all plugins you have not set to `yes` explicitly.
|
||||
check for new plugins every|60|The time in seconds to check for new plugins in the plugins directory. This allows having other applications dynamically creating plugins for netdata.
|
||||
|
||||
## Netdata Plugins
|
||||
|
||||
The configuration options for plugins appear in sections following the pattern `[plugin:NAME]`.
|
||||
|
||||
### Internal Plugins
|
||||
|
||||
Most internal plugins will provide additional options. Check [Internal Plugins](Internal-Plugins) for more information.
|
||||
|
||||
|
||||
### External Plugins
|
||||
|
||||
External plugins will have only 2 options at `netdata.conf`:
|
||||
|
||||
setting | default | info
|
||||
:------:|:-------:|:----
|
||||
update every|the value of `[global].update every` setting|The frequency in seconds the plugin should collect values. For more information check [Performance](Performance).
|
||||
command options|*empty*|Additional command line options to pass to the plugin.
|
||||
|
||||
External plugins that need additional configuration may support a dedicated file in `/etc/netdata`. Check their documentation.
|
||||
|
||||
---
|
||||
|
||||
## A note about netdata.conf
|
||||
|
||||
This config file is not needed by default. You can just touch it (to be empty) to get rid of the error message displayed when missing.
|
||||
|
||||
The whole idea came up when I was evaluating the documentation involved in maintaining a complex configuration system. My intention was to give configuration options for everything imaginable. But then, documenting all these options would require a tremendous amount of time, users would have to search through endless pages for the option they need, etc.
|
||||
|
||||
I concluded then that configuring software like that is a waste for time and effort. Of course there must be plenty of configuration options, but the implementation itself should require a lot less effort for both the devs and the users.
|
||||
|
||||
So, I did this:
|
||||
|
||||
1. No configuration is required to run netdata
|
||||
2. There are plenty of options to tweak
|
||||
3. There is minimal documentation (or no at all)
|
||||
|
||||
### Why this works?
|
||||
|
||||
The configuration file is a `name = value` dictionary with `[sections]`. Write whatever you like there as long as it follows this simple format.
|
||||
|
||||
Netdata loads this dictionary and then when the code needs a value from it, it just looks up the `name` in the dictionary at the proper `section`. In all places, in the code, there are both the `names` and their `default values`, so if something is not found in the configuration file, the default is used. The lookup is made using B-Trees and hashes (no string comparisons), so they are super fast. Also the `names` of the settings can be `my super duper setting that once set to yes, will turn the world upside down = no` - so goodbye to most of the documentation involved.
|
||||
|
||||
Next, netdata can generate a valid configuration for the user to edit. No need to remember anything. Just get the configuration from the server (`/netdata.conf` on your netdata server), edit it and save it.
|
||||
|
||||
Last, what about options you believe you have set, but you misspelled? When you get the configuration file from the server, there will be a comment above all `name = value` pairs the server does not use. So you know that whatever you wrote there, is not used.
|
||||
|
||||
### limiting access to netdata.conf
|
||||
|
||||
netdata v1.9+ limit by default access to `http://your.netdata.ip:19999/netdata.conf` to private IP addresses. This is controlled by this settings:
|
||||
|
||||
```
|
||||
[web]
|
||||
allow netdata.conf from = localhost fd* 10.* 192.168.* 172.16.* 172.17.* 172.18.* 172.19.* 172.20.* 172.21.* 172.22.* 172.23.* 172.24.* 172.25.* 172.26.* 172.27.* 172.28.* 172.29.* 172.30.* 172.31.*
|
||||
```
|
||||
|
||||
The IPs listed are all the private IPv4 addresses, including link local IPv6 addresses.
|
||||
|
||||
> Keep in mind that connections to netdata API ports are filtered by `[web].allow connections from`. So, IPs allowed by `[web].allow netdata.conf from` should also be allowed by `[web].allow connection from`.
|
||||
|
||||
|
||||
## netdata simple patterns
|
||||
|
||||
Unix prefers regular expressions. But they are just too hard, too cryptic to use, write and understand.
|
||||
|
||||
So, netdata supports simple patterns. Simple patterns are a space separated list of words, that can have `*` as a wildcard. Each world may use any number of `*`. Simple patterns allow negative matches by prefixing a word with `!`.
|
||||
|
||||
So, `pattern = !*bad* *` will match anything, except all those that contain the word `bad`.
|
||||
|
||||
Simple patterns are quite powerful: `pattern = *foobar* !foo* !*bar *` matches everything containing `foobar` and except from strings that start with `foo` or end with `bar`, everything else.
|
||||
|
||||
You can use the netdata command line to check simple patterns, like this:
|
||||
|
||||
```sh
|
||||
# netdata -W simple-pattern '*foobar* !foo* !*bar *' 'hello world'
|
||||
RESULT: MATCHED - pattern '*foobar* !foo* !*bar *' matches 'hello world'
|
||||
|
||||
# netdata -W simple-pattern '*foobar* !foo* !*bar *' 'hello world bar'
|
||||
RESULT: NOT MATCHED - pattern '*foobar* !foo* !*bar *' does not match 'hello world bar'
|
||||
|
||||
# netdata -W simple-pattern '*foobar* !foo* !*bar *' 'hello world foobar'
|
||||
RESULT: MATCHED - pattern '*foobar* !foo* !*bar *' matches 'hello world foobar'
|
||||
```
|
||||
|
||||
netdata stops processing to the first positive or negative match (left to right). If it is not matched by either positive or negative patterns, it is denied at the end.
|
||||
|
||||
|
||||
## Applying changes
|
||||
|
||||
After `netdata.conf` has been modified, netdata needs to be restarted for changes to apply:
|
||||
|
||||
```bash
|
||||
sudo service netdata restart
|
||||
```
|
||||
|
||||
If the above does not work, try the following:
|
||||
|
||||
```bash
|
||||
sudo killall netdata; sleep 10; sudo netdata
|
||||
```
|
||||
|
||||
Please note that your data history will be lost if you have modified `history` parameter in section `[global]`.
|
||||
|
|
@ -1,3 +1,11 @@
|
|||
# Data structures
|
||||
|
||||
These are the main internal data structures of `netdata`. Created with `draw.io`.
|
||||
These are the main internal data structures of `netdata`. Created with `draw.io`.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
384
doc/Add-more-charts-to-netdata.md
Normal file
384
doc/Add-more-charts-to-netdata.md
Normal file
|
|
@ -0,0 +1,384 @@
|
|||
# Add more charts to netdata
|
||||
|
||||
netdata collects system metrics by itself. It has many [[Internal Plugins]] for collecting most of the metrics presented by default when it starts, collecting data from `/proc`, `/sys` and other Linux kernel sources.
|
||||
|
||||
To collect non-system metrics, netdata supports a plugin architecture. The following are the currently available external plugins:
|
||||
|
||||
- **[Web Servers](#web-servers)**, such as apache, nginx, nginx_plus, tomcat, litespeed
|
||||
- **[Web Logs](#web-log-parsers)**, such as apache, nginx, lighttpd, gunicorn, squid access logs, apache cache.log
|
||||
- **[Load Balancers](#load-balancers)**, like haproxy
|
||||
- **[Message Brokers](#message-brokers)**, like rabbitmq, beanstalkd
|
||||
- **[Database Servers](#database-servers)**, such as mysql, mariadb, postgres, couchdb, mongodb, rethinkdb
|
||||
- **[Social Sharing Servers](#social-sharing-servers)**, like retroshare
|
||||
- **[Proxy Servers](#proxy-servers)**, like squid
|
||||
- **[HTTP accelerators](#http-accelerators)**, like varnish cache
|
||||
- **[Search engines](#search-engines)**, like elasticsearch
|
||||
- **[Name Servers](#name-servers)** (DNS), like bind, nsd, powerdns, dnsdist
|
||||
- **[DHCP Servers](#dhcp-servers)**, like ISC DHCP
|
||||
- **[UPS](#ups)**, such as APC UPS, NUT
|
||||
- **[RAID](#raid)**, such as linux software raid (mdadm), MegaRAID
|
||||
- **[Mail Servers](#mail-servers)**, like postfix, exim, dovecot
|
||||
- **[File Servers](#file-servers)**, like samba, NFS, ftp, sftp, WebDAV
|
||||
- **[System](#system)**, for processes and other system metrics
|
||||
- **[Sensors](#sensors)**, like temperature, fans speed, voltage, humidity, HDD/SSD S.M.A.R.T attributes
|
||||
- **[Network](#network)**, such as SNMP devices, `fping`, access points, dns_query_time
|
||||
- **[Time Servers](#time-servers)**, like chrony
|
||||
- **[Security](#security)**, like FreeRADIUS, OpenVPN, Fail2ban
|
||||
- **[Telephony Servers](#telephony-servers)**, like openSIPS
|
||||
- **[Go applications](#go-applications)**
|
||||
- **[Household appliances](#household-appliances)**, like SMA WebBox (solar power), Fronius Symo solar power, Stiebel Eltron heating
|
||||
- **[Java Processes](#java-processes)**, via JMX or Spring Boot Actuator
|
||||
- **[Provisioning Systems](#provisioning-systems)**, like Puppet
|
||||
- **[Game Servers](#game-servers)**, like SpigotMC
|
||||
- **[Distributed Computing Clients](#distributed-computing-clients)**, like BOINC
|
||||
- **[Skeleton Plugins](#skeleton-plugins)**, for writing your own data collectors
|
||||
|
||||
Check also [[Third Party Plugins]]for a list of plugins distributed by third parties.
|
||||
|
||||
## configuring plugins
|
||||
|
||||
Most plugins come with **auto-detection**, configured to work out-of-the-box on popular operating systems with the default settings.
|
||||
|
||||
However, there are cases that auto-detection fails. Usually the reason is that the applications to be monitored do not allow netdata to connect. In most of the cases, allowing the user `netdata` from `localhost` to connect and collect metrics, will automatically enable data collection for the application in question (it will require a netdata restart).
|
||||
|
||||
You can verify netdata plugins are able to collect metrics, following this procedure:
|
||||
|
||||
```sh
|
||||
# become user netdata
|
||||
sudo su -s /bin/bash netdata
|
||||
|
||||
# execute the plugin in debug mode, for a specific module.
|
||||
# example for the python plugin, mysql module:
|
||||
/usr/libexec/netdata/plugins.d/python.d.plugin 1 debug trace mysql
|
||||
```
|
||||
|
||||
Similarly, you can use `charts.d.plugin` for BASH plugins and `node.d.plugin` for node.js plugins.
|
||||
Other plugins (like `apps.plugin`, `freeipmi.plugin`, `fping.plugin`) use the native netdata plugin API and can be run directly.
|
||||
|
||||
If you need to configure a netdata plugin or module, all configuration is kept at `/etc/netdata`. Each file should provide plenty of examples and documentation about each module and plugin.
|
||||
|
||||
This is a map of the all supported configuration options:
|
||||
|
||||
netdata plugin | language | configuration | modules configuration
|
||||
---:|:---:|:---:|:---
|
||||
apps.plugin|`C`|`/etc/netdata/`<br/>`apps_groups.conf`|command line arguments, specified at `/etc/netdata/netdata.conf`, section `[plugin:apps]`
|
||||
fping.plugin|`C` with a `BASH` wrapper|`/etc/netdata/`<br/>`fping.conf`|N/A
|
||||
freeipmi.plugin|`C`|N/A|command line arguments specified at `/etc/netdata/netdata.conf`, section `[plugin:freeipmi]`
|
||||
charts.d.plugin|`BASH`|`/etc/netdata/`<br/>`charts.d.conf`|a file for each module in `/etc/netdata/charts.d/`
|
||||
python.d.plugin|`python`<br/>v2 or v3|`/etc/netdata/`<br/>`python.d.conf`|a file for each module in `/etc/netdata/python.d/`
|
||||
node.d.plugin|`node.js`|`/etc/netdata/`<br/>`node.d.conf`|a file for each module in `/etc/netdata/node.d/`
|
||||
java.d.plugin|`java 8`|`/etc/netdata/`<br/>`java.d.conf`|a file for each module in `/etc/netdata/java.d.conf/`
|
||||
|
||||
|
||||
## writing netdata plugins
|
||||
|
||||
You can add custom plugins following the [External Plugins](Netdata External Plugins) guide.
|
||||
|
||||
---
|
||||
|
||||
# available netdata plugins
|
||||
|
||||
These are all the data collection plugins currently available.
|
||||
|
||||
## Web Servers
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
apache|python<br/>v2 or v3|Connects to multiple apache servers (local or remote) to collect real-time performance metrics.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [apache.chart.py](https://github.com/netdata/netdata/blob/master/python.d/apache.chart.py)<br/>configuration file: [python.d/apache.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/apache.conf)|
|
||||
apache|BASH<br/>Shell Script|Connects to an apache server (local or remote) to collect real-time performance metrics.<br/><br/>DEPRECATED IN FAVOR OF THE PYTHON ONE. It is still supplied only as an example module to shell scripting plugins.<br/> <br/>netdata plugin: [charts.d.plugin](General-Info-charts\.d)<br/>plugin module: [apache.chart.sh](https://github.com/netdata/netdata/blob/master/charts.d/apache.chart.sh)<br/>configuration file: [charts.d/apache.conf](https://github.com/netdata/netdata/blob/master/conf.d/charts.d/apache.conf)|
|
||||
apache_cache|python<br/>v2 or v3|Monitors one or more apache `cache.log` files to provide real-time cache performance statistics.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [apache_cache.chart.py](https://github.com/netdata/netdata/blob/master/python.d/apache_cache.chart.py)<br/>configuration file: [python.d/apache_cache.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/apache_cache.conf)|
|
||||
ipfs|python<br/>v2 or v3|Connects to multiple ipfs servers (local or remote) to collect real-time performance metrics.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [ipfs.chart.py](https://github.com/netdata/netdata/blob/master/python.d/ipfs.chart.py)<br/>configuration file: [python.d/ipfs.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/ipfs.conf)|
|
||||
litespeed|python<br/>v2 or v3|reads the litespeed `rtreport` files to collect metrics.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [litespeed.chart.py](https://github.com/netdata/netdata/blob/master/python.d/litespeed.chart.py)<br/>configuration file: [python.d/litespeed.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/litespeed.conf)
|
||||
nginx|python<br/>v2 or v3|Connects to multiple nginx servers (local or remote) to collect real-time performance metrics.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [nginx.chart.py](https://github.com/netdata/netdata/blob/master/python.d/nginx.chart.py)<br/>configuration file: [python.d/nginx.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/nginx.conf)|
|
||||
nginx_plus|python<br/>v2 or v3|Connects to multiple nginx_plus servers (local or remote) to collect real-time performance metrics.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [nginx_plus.chart.py](https://github.com/netdata/netdata/blob/master/python.d/nginx_plus.chart.py)<br/>configuration file: [python.d/nginx_plus.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/nginx_plus.conf)|
|
||||
nginx|BASH<br/>Shell Script|Connects to an nginx server (local or remote) to collect real-time performance metrics.<br/><br/>DEPRECATED IN FAVOR OF THE PYTHON ONE. It is still supplied only as an example module to shell scripting plugins.<br/> <br/>netdata plugin: [charts.d.plugin](General-Info-charts\.d)<br/>plugin module: [nginx.chart.sh](https://github.com/netdata/netdata/blob/master/charts.d/nginx.chart.sh)<br/>configuration file: [charts.d/nginx.conf](https://github.com/netdata/netdata/blob/master/conf.d/charts.d/nginx.conf)|
|
||||
nginx_log|python<br/>v2 or v3|Use Monitors one or more nginx log files to collect real-time pageviews per response status. <br/> <br/> DEPRECATED. USE web_log INSTEAD. <br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [nginx_log.chart.py](https://github.com/netdata/netdata/blob/master/python.d/nginx_log.chart.py)<br/>configuration file: [python.d/nginx_log.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/nginx_log.conf)|
|
||||
phpfpm|python<br/>v2 or v3|Connects to multiple phpfpm servers (local or remote) to collect real-time performance metrics.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [phpfpm.chart.py](https://github.com/netdata/netdata/blob/master/python.d/phpfpm.chart.py)<br/>configuration file: [python.d/phpfpm.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/phpfpm.conf)|
|
||||
phpfpm|BASH<br/>Shell Script|Connects to one or more phpfpm servers (local or remote) to collect real-time performance metrics.<br/><br/>DEPRECATED IN FAVOR OF THE PYTHON ONE. It is still supplied only as an example module to shell scripting plugins.<br/> <br/>netdata plugin: [charts.d.plugin](General-Info-charts\.d)<br/>plugin module: [phpfpm.chart.sh](https://github.com/netdata/netdata/blob/master/charts.d/phpfpm.chart.sh)<br/>configuration file: [charts.d/phpfpm.conf](https://github.com/netdata/netdata/blob/master/conf.d/charts.d/phpfpm.conf)|
|
||||
tomcat|python<br/>v2 or v3|Connects to multiple tomcat servers (local or remote) to collect real-time performance metrics.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [tomcat.chart.py](https://github.com/netdata/netdata/blob/master/python.d/tomcat.chart.py)<br/>configuration file: [python.d/tomcat.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/tomcat.conf)|
|
||||
tomcat|BASH<br/>Shell Script|Connects to a tomcat server (local or remote) to collect real-time performance metrics.<br/><br/>DEPRECATED IN FAVOR OF THE PYTHON ONE. It is still supplied only as an example module to shell scripting plugins.<br/> <br/>netdata plugin: [charts.d.plugin](General-Info-charts\.d)<br/>plugin module: [tomcat.chart.sh](https://github.com/netdata/netdata/blob/master/charts.d/tomcat.chart.sh)<br/>configuration file: [charts.d/tomcat.conf](https://github.com/netdata/netdata/blob/master/conf.d/charts.d/tomcat.conf)|
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Web Log Parsers
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
apache_cache|python<br/>v2 or v3|tails the apache cache log file to collect cache hit/miss rate metrics <br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [apache_cache.chart.py](https://github.com/netdata/netdata/blob/master/python.d/apache_cache.chart.py)<br/>configuration file: [python.d/apache_cache.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/apache_cache.conf)|
|
||||
web_log|python<br/>v2 or v3|powerful plugin, capable of incrementally parsing any number of web server log files <br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [web_log.chart.py](https://github.com/netdata/netdata/blob/master/python.d/web_log.chart.py)<br/>configuration file: [python.d/web_log.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/web_log.conf)|
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Database Servers
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
couchdb|python<br/>v2 or v3|Connects to multiple couchdb servers (local or remote) to collect real-time performance metrics.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [couchdb.chart.py](https://github.com/netdata/netdata/blob/master/python.d/couchdb.chart.py)<br/>configuration file: [python.d/couchdb.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/couchdb.conf)|
|
||||
memcached|python<br/>v2 or v3|Connects to multiple memcached servers (local or remote) to collect real-time performance metrics.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [memcached.chart.py](https://github.com/netdata/netdata/blob/master/python.d/memcached.chart.py)<br/>configuration file: [python.d/memcached.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/memcached.conf)|
|
||||
mongodb|python<br/>v2 or v3|Connects to multiple `mongodb` servers (local or remote) to collect real-time performance metrics.<br/> <br/>Requires package `python-pymongo`.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [mongodb.chart.py](https://github.com/netdata/netdata/blob/master/python.d/mongodb.chart.py)<br/>configuration file: [python.d/mongodb.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/mongodb.conf)|
|
||||
mysql<br/>mariadb|python<br/>v2 or v3|Connects to multiple mysql or mariadb servers (local or remote) to collect real-time performance metrics.<br/> <br/>Requires package `python-mysqldb` (faster and preferred), or `python-pymysql`. <br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [mysql.chart.py](https://github.com/netdata/netdata/blob/master/python.d/mysql.chart.py)<br/>configuration file: [python.d/mysql.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/mysql.conf)|
|
||||
mysql<br/>mariadb|BASH<br/>Shell Script|Connects to multiple mysql or mariadb servers (local or remote) to collect real-time performance metrics.<br/><br/>DEPRECATED IN FAVOR OF THE PYTHON ONE. It is still supplied only as an example module to shell scripting plugins.<br/> <br/>netdata plugin: [charts.d.plugin](General-Info-charts\.d)<br/>plugin module: [mysql.chart.sh](https://github.com/netdata/netdata/blob/master/charts.d/mysql.chart.sh)<br/>configuration file: [charts.d/mysql.conf](https://github.com/netdata/netdata/blob/master/conf.d/charts.d/mysql.conf)|
|
||||
postgres|python<br/>v2 or v3|Connects to multiple postgres servers (local or remote) to collect real-time performance metrics.<br/> <br/>Requires package `python-psycopg2`.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [postgres.chart.py](https://github.com/netdata/netdata/blob/master/python.d/postgres.chart.py)<br/>configuration file: [python.d/postgres.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/postgres.conf)|
|
||||
redis|python<br/>v2 or v3|Connects to multiple redis servers (local or remote) to collect real-time performance metrics.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [redis.chart.py](https://github.com/netdata/netdata/blob/master/python.d/redis.chart.py)<br/>configuration file: [python.d/redis.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/redis.conf)|
|
||||
rethinkdb|python<br/>v2 or v3|Connects to multiple rethinkdb servers (local or remote) to collect real-time metrics.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [rethinkdb.chart.py](https://github.com/netdata/netdata/blob/master/python.d/rethinkdbs.chart.py)<br/>configuration file: [python.d/rethinkdb.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/rethinkdbs.conf)|
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Social Sharing Servers
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
retroshare|python<br/>v2 or v3|Connects to multiple retroshare servers (local or remote) to collect real-time performance metrics.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [retroshare.chart.py](https://github.com/netdata/netdata/blob/master/python.d/retroshare.chart.py)<br/>configuration file: [python.d/retroshare.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/retroshare.conf)|
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Proxy Servers
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
squid|python<br/>v2 or v3|Connects to multiple squid servers (local or remote) to collect real-time performance metrics.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [squid.chart.py](https://github.com/netdata/netdata/blob/master/python.d/squid.chart.py)<br/>configuration file: [python.d/squid.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/squid.conf)|
|
||||
squid|BASH<br/>Shell Script|Connects to a squid server (local or remote) to collect real-time performance metrics.<br/><br/>DEPRECATED IN FAVOR OF THE PYTHON ONE. It is still supplied only as an example module to shell scripting plugins.<br/> <br/>netdata plugin: [charts.d.plugin](General-Info-charts\.d)<br/>plugin module: [squid.chart.sh](https://github.com/netdata/netdata/blob/master/charts.d/squid.chart.sh)<br/>configuration file: [charts.d/squid.conf](https://github.com/netdata/netdata/blob/master/conf.d/charts.d/squid.conf)|
|
||||
|
||||
|
||||
---
|
||||
|
||||
## HTTP Accelerators
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
varnish|python<br/>v2 or v3|Uses the varnishstat command to provide varnish cache statistics (client metrics, cache perfomance, thread-related metrics, backend health, memory usage etc.).<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [varnish.chart.py](https://github.com/netdata/netdata/blob/master/python.d/varnish.chart.py)<br/>configuration file: [python.d/varnish.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/varnish.conf)|
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Search Engines
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
elasticsearch|python<br/>v2 or v3|Monitor elasticsearch performance and health metrics.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [elasticsearch.chart.py](https://github.com/netdata/netdata/blob/master/python.d/elasticsearch.chart.py)<br/>configuration file: [python.d/elasticsearch.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/elasticsearch.conf)|
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Name Servers
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
named|node.js|Connects to multiple named (ISC-Bind) servers (local or remote) to collect real-time performance metrics. All versions of bind after 9.9.10 are supported.<br/> <br/>netdata plugin: [node.d.plugin](General-Info-node\.d)<br/>plugin module: [named.node.js](https://github.com/netdata/netdata/blob/master/node.d/named.node.js)<br/>configuration file: [node.d/named.conf](https://github.com/netdata/netdata/blob/master/conf.d/node.d/named.conf.md)|
|
||||
bind_rndc|python<br/>v2 or v3|Parses named.stats dump file to collect real-time performance metrics. All versions of bind after 9.6 are supported.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [bind_rndc.chart.py](https://github.com/netdata/netdata/blob/master/python.d/bind_rndc.chart.py)<br/>configuration file: [python.d/bind_rndc.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/bind_rndc.conf)|
|
||||
nsd|python<br/>v2 or v3|Charts the nsd received queries and zones.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [nsd.chart.py](https://github.com/netdata/netdata/blob/master/python.d/nsd.chart.py)<br/>configuration file: [python.d/nsd.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/nsd.conf)
|
||||
powerdns|python<br/>v2 or v3|Monitors powerdns performance and health metrics <br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [powerdns.chart.py](https://github.com/netdata/netdata/blob/master/python.d/powerdns.chart.py)<br/>configuration file: [python.d/powerdns.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/powerdns.conf)|
|
||||
dnsdist|python<br/>v2 or v3|Monitors dnsdist performance and health metrics <br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [dnsdist.chart.py](https://github.com/netdata/netdata/blob/master/python.d/dnsdist.chart.py)<br/>configuration file: [python.d/dnsdist.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/dnsdist.conf)|
|
||||
unbound|python<br/>v2 or v3|Monitors Unbound performance and resource usage metrics <br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [unbound.chart.py](https://github.com/netdata/netdata/blob/master/python.d/unbound.chart.py)<br/>configuration file: [python.d/unbound.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/unbound.conf)|
|
||||
|
||||
|
||||
---
|
||||
|
||||
## DHCP Servers
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
isc dhcp|python<br/>v2 or v3|Monitor lease database to show all active leases.<br/> <br/>Python v2 requires package `python-ipaddress`.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [isc-dhcpd.chart.py](https://github.com/netdata/netdata/blob/master/python.d/isc_dhcpd.chart.py)<br/>configuration file: [python.d/isc-dhcpd.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/isc_dhcpd.conf)|
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Load Balancers
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
haproxy|python<br/>v2 or v3|Monitor frontend, backend and health metrics.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [haproxy.chart.py](https://github.com/netdata/netdata/blob/master/python.d/haproxy.chart.py)<br/>configuration file: [python.d/haproxy.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/haproxy.conf)|
|
||||
traefik|python<br/>v2 or v3|Connects to multiple traefik instances (local or remote) to collect API metrics (response status code, response time, average response time and server uptime).<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [traefik.chart.py](https://github.com/netdata/netdata/blob/master/python.d/traefik.chart.py)<br/>configuration file: [python.d/traefik.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/traefik.conf)|
|
||||
|
||||
---
|
||||
|
||||
## Message Brokers
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
rabbitmq|python<br/>v2 or v3|Monitor rabbitmq performance and health metrics.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [rabbitmq.chart.py](https://github.com/netdata/netdata/blob/master/python.d/rabbitmq.chart.py)<br/>configuration file: [python.d/rabbitmq.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/rabbitmq.conf)|
|
||||
beanstalkd|python<br/>v2 or v3|Provides server and tube level statistics.<br/> <br/>Requires beanstalkc python package (`pip install beanstalkc` or install package `python-beanstalkc`, which also installs `python-yaml`).<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [beanstalk.chart.py](https://github.com/netdata/netdata/blob/master/python.d/beanstalk.chart.py)<br/>configuration file: [python.d/beanstalk.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/beanstalk.conf)|
|
||||
|
||||
|
||||
---
|
||||
|
||||
## UPS
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
apcupsd|BASH<br/>Shell Script|Connects to an apcupsd server to collect real-time statistics of an APC UPS.<br/> <br/>netdata plugin: [charts.d.plugin](General-Info-charts\.d)<br/>plugin module: [apcupsd.chart.sh](https://github.com/netdata/netdata/blob/master/charts.d/apcupsd.chart.sh)<br/>configuration file: [charts.d/apcupsd.conf](https://github.com/netdata/netdata/blob/master/conf.d/charts.d/apcupsd.conf)|
|
||||
nut|BASH<br/>Shell Script|Connects to a nut server (upsd) to collect real-time UPS statistics.<br/> <br/>netdata plugin: [charts.d.plugin](General-Info-charts\.d)<br/>plugin module: [nut.chart.sh](https://github.com/netdata/netdata/blob/master/charts.d/nut.chart.sh)<br/>configuration file: [charts.d/nut.conf](https://github.com/netdata/netdata/blob/master/conf.d/charts.d/nut.conf)|
|
||||
|
||||
|
||||
---
|
||||
|
||||
## RAID
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
mdstat|python<br/>v2 or v3|Parses `/proc/mdstat` to get mds health metrics.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [mdstat.chart.py](https://github.com/netdata/netdata/blob/master/python.d/mdstat.chart.py)<br/>configuration file: [python.d/mdstat.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/mdstat.conf)|
|
||||
megacli|python<br/>v2 or v3|Collects adapter, physical drives and battery stats..<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [megacli.chart.py](https://github.com/netdata/netdata/blob/master/python.d/megacli.chart.py)<br/>configuration file: [python.d/megacli.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/megacli.conf)|
|
||||
|
||||
---
|
||||
|
||||
## Mail Servers
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
dovecot|python<br/>v2 or v3|Connects to multiple dovecot servers (local or remote) to collect real-time performance metrics.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [dovecot.chart.py](https://github.com/netdata/netdata/blob/master/python.d/dovecot.chart.py)<br/>configuration file: [python.d/dovecot.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/dovecot.conf)|
|
||||
exim|python<br/>v2 or v3|Charts the exim queue size.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [exim.chart.py](https://github.com/netdata/netdata/blob/master/python.d/exim.chart.py)<br/>configuration file: [python.d/exim.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/exim.conf)|
|
||||
exim|BASH<br/>Shell Script|Charts the exim queue size.<br/><br/>DEPRECATED IN FAVOR OF THE PYTHON ONE. It is still supplied only as an example module to shell scripting plugins.<br/> <br/>netdata plugin: [charts.d.plugin](General-Info-charts\.d)<br/>plugin module: [exim.chart.sh](https://github.com/netdata/netdata/blob/master/charts.d/exim.chart.sh)<br/>configuration file: [charts.d/exim.conf](https://github.com/netdata/netdata/blob/master/conf.d/charts.d/exim.conf)|
|
||||
postfix|python<br/>v2 or v3|Charts the postfix queue size (supports multiple queues).<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [postfix.chart.py](https://github.com/netdata/netdata/blob/master/python.d/postfix.chart.py)<br/>configuration file: [python.d/postfix.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/postfix.conf)|
|
||||
postfix|BASH<br/>Shell Script|Charts the postfix queue size.<br/><br/>DEPRECATED IN FAVOR OF THE PYTHON ONE. It is still supplied only as an example module to shell scripting plugins.<br/> <br/>netdata plugin: [charts.d.plugin](General-Info-charts\.d)<br/>plugin module: [postfix.chart.sh](https://github.com/netdata/netdata/blob/master/charts.d/postfix.chart.sh)<br/>configuration file: [charts.d/postfix.conf](https://github.com/netdata/netdata/blob/master/conf.d/charts.d/postfix.conf)|
|
||||
|
||||
|
||||
---
|
||||
|
||||
## File Servers
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
NFS Client|`C`|This is handled entirely by the netdata daemon.<br/> <br/>Configuration: `netdata.conf`, section `[plugin:proc:/proc/net/rpc/nfs]`.
|
||||
NFS Server|`C`|This is handled entirely by the netdata daemon.<br/> <br/>Configuration: `netdata.conf`, section `[plugin:proc:/proc/net/rpc/nfsd]`.
|
||||
samba|python<br/>v2 or v3|Performance metrics of Samba SMB2 file sharing.<br/> <br/>wiki page: [python.d.plugin module samba](https://github.com/netdata/netdata/tree/master/python.d#samba)<br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [samba.chart.py](https://github.com/netdata/netdata/blob/master/python.d/samba.chart.py)<br/>configuration file: [python.d/samba.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/samba.conf)|
|
||||
|
||||
|
||||
---
|
||||
|
||||
## System
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
apps|C|`apps.plugin` collects resource usage statistics for all processes running in the system. It groups the entire process tree and reports dozens of metrics for CPU utilization, memory footprint, disk I/O, swap memory, network connections, open files and sockets, etc. It reports metrics for application groups, users and user groups.<br/> <br/>[Wiki page of `apps.plugin`](https://github.com/netdata/netdata/wiki/Apps-Plugin).<br/> <br/>netdata plugin: [`apps_plugin.c`](https://github.com/netdata/netdata/blob/master/src/apps_plugin.c)<br/>configuration file: [`apps_groups.conf`](https://github.com/netdata/netdata/blob/master/conf.d/apps_groups.conf)|
|
||||
cpu_apps|BASH<br/>Shell Script|Collects the CPU utilization of select apps.<br/><br/>DEPRECATED IN FAVOR OF `apps.plugin`. It is still supplied only as an example module to shell scripting plugins.<br/> <br/>netdata plugin: [charts.d.plugin](General-Info-charts\.d)<br/>plugin module: [cpu_apps.chart.sh](https://github.com/netdata/netdata/blob/master/charts.d/cpu_apps.chart.sh)<br/>configuration file: [charts.d/cpu_apps.conf](https://github.com/netdata/netdata/blob/master/conf.d/charts.d/cpu_apps.conf)|
|
||||
load_average|BASH<br/>Shell Script|Collects the current system load average.<br/><br/>DEPRECATED IN FAVOR OF THE NETDATA INTERNAL ONE. It is still supplied only as an example module to shell scripting plugins.<br/> <br/>netdata plugin: [charts.d.plugin](General-Info-charts\.d)<br/>plugin module: [load_average.chart.sh](https://github.com/netdata/netdata/blob/master/charts.d/load_average.chart.sh)<br/>configuration file: [charts.d/load_average.conf](https://github.com/netdata/netdata/blob/master/conf.d/charts.d/load_average.conf)|
|
||||
mem_apps|BASH<br/>Shell Script|Collects the memory footprint of select applications.<br/><br/>DEPRECATED IN FAVOR OF `apps.plugin`. It is still supplied only as an example module to shell scripting plugins.<br/> <br/>netdata plugin: [charts.d.plugin](General-Info-charts\.d)<br/>plugin module: [mem_apps.chart.sh](https://github.com/netdata/netdata/blob/master/charts.d/mem_apps.chart.sh)<br/>configuration file: [charts.d/mem_apps.conf](https://github.com/netdata/netdata/blob/master/conf.d/charts.d/mem_apps.conf)|
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Sensors
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
cpufreq|python<br/>v2 or v3|Collects the current CPU frequency from `/sys/devices`.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [cpufreq.chart.py](https://github.com/netdata/netdata/blob/master/python.d/cpufreq.chart.py)<br/>configuration file: [python.d/cpufreq.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/cpufreq.conf)|
|
||||
cpufreq|BASH<br/>Shell Script|Collects current CPU frequency from `/sys/devices`.<br/><br/>DEPRECATED IN FAVOR OF THE PYTHON ONE. It is still supplied only as an example module to shell scripting plugins.<br/> <br/>netdata plugin: [charts.d.plugin](General-Info-charts\.d)<br/>plugin module: [cpufreq.chart.sh](https://github.com/netdata/netdata/blob/master/charts.d/cpufreq.chart.sh)<br/>configuration file: [charts.d/cpufreq.conf](https://github.com/netdata/netdata/blob/master/conf.d/charts.d/cpufreq.conf)|
|
||||
IPMI|C|Collects temperatures, voltages, currents, power, fans and `SEL` events from IPMI using `libipmimonitoring`.<br/>Check [Monitoring IPMI](https://github.com/netdata/netdata/wiki/monitoring-IPMI) for more information<br/> <br/>netdata plugin: [freeipmi.plugin](https://github.com/netdata/netdata/blob/master/src/freeipmi_plugin.c)<br/>configuration file: none required - to enable it, compile/install netdata with `--enable-plugin-freeipmi`|
|
||||
hddtemp|python<br/>v2 or v3|Connects to multiple hddtemp servers (local or remote) to collect real-time performance metrics.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [hddtemp.chart.py](https://github.com/netdata/netdata/blob/master/python.d/hddtemp.chart.py)<br/>configuration file: [python.d/hddtemp.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/hddtemp.conf)|
|
||||
hddtemp|BASH<br/>Shell Script|Connects to a hddtemp server (local or remote) to collect real-time performance metrics.<br/><br/>DEPRECATED IN FAVOR OF THE PYTHON ONE. It is still supplied only as an example module to shell scripting plugins.<br/> <br/>netdata plugin: [charts.d.plugin](General-Info-charts\.d)<br/>plugin module: [hddtemp.chart.sh](https://github.com/netdata/netdata/blob/master/charts.d/hddtemp.chart.sh)<br/>configuration file: [charts.d/hddtemp.conf](https://github.com/netdata/netdata/blob/master/conf.d/charts.d/hddtemp.conf)|
|
||||
sensors|BASH<br/>Shell Script|Collects sensors values from files in `/sys`.<br/><br/>DEPRECATED IN FAVOR OF THE PYTHON ONE. It is still supplied only as an example module to shell scripting plugins.<br/> <br/>netdata plugin: [charts.d.plugin](General-Info-charts\.d)<br/>plugin module: [sensors.chart.sh](https://github.com/netdata/netdata/blob/master/charts.d/sensors.chart.sh)<br/>configuration file: [charts.d/sensors.conf](https://github.com/netdata/netdata/blob/master/conf.d/charts.d/sensors.conf)|
|
||||
sensors|python<br/>v2 or v3|Uses `lm-sensors` to collect sensor data.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [sensors.chart.py](https://github.com/netdata/netdata/blob/master/python.d/sensors.chart.py)<br/>configuration file: [python.d/sensors.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/sensors.conf)|
|
||||
smartd_log|python<br/>v2 or v3|Collects the S.M.A.R.T attributes from `smartd` log files.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [smartd_log.chart.py](https://github.com/netdata/netdata/blob/master/python.d/smartd_log.chart.py)<br/>configuration file: [python.d/smartd_log.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/smartd_log.conf)|
|
||||
w1sensor|python<br/>v2 or v3|Collects data from connected 1-Wire sensors.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [w1sensor.chart.py](https://github.com/netdata/netdata/blob/master/python.d/w1sensor.chart.py)<br/>configuration file: [python.d/w1sensor.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/w1sensor.conf)|
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Network
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
ap|BASH<br/>Shell Script|Uses the `iw` command to provide statistics of wireless clients connected to a wireless access point running on this host (works well with `hostapd`).<br/> <br/>netdata plugin: [charts.d.plugin](General-Info-charts\.d)<br/>plugin module: [ap.chart.sh](https://github.com/netdata/netdata/blob/master/charts.d/ap.chart.sh)<br/>configuration file: [charts.d/ap.conf](https://github.com/netdata/netdata/blob/master/conf.d/charts.d/ap.conf)|
|
||||
fping|C|Charts network latency statistics for any number of nodes, using the `fping` command. A recent (probably unreleased) version of fping is required. The plugin supplied can install it in `/usr/local`.<br/> <br/>netdata plugin: [fping.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/fping.plugin) (this is a shell wrapper to start fping - once fping is started, netdata and fping communicate directly - it can also install the right version of fping)<br/>configuration file: [fping.conf](https://github.com/netdata/netdata/blob/master/conf.d/fping.conf)|
|
||||
snmp|node.js|Connects to multiple snmp servers to collect real-time performance metrics.<br/> <br/>netdata plugin: [node.d.plugin](General-Info-node\.d)<br/>plugin module: [snmp.node.js](https://github.com/netdata/netdata/blob/master/node.d/snmp.node.js)<br/>configuration file: [node.d/snmp.conf](https://github.com/netdata/netdata/blob/master/conf.d/node.d/snmp.conf.md)|
|
||||
dns_query_time|python<br/>v2 or v3|Provides DNS query time statistics.<br/> <br/>Requires package `dnspython` (`pip install dnspython` or install package `python-dnspython`).<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [dns_query_time.chart.py](https://github.com/netdata/netdata/blob/master/python.d/dns_query_time.chart.py)<br/>configuration file: [python.d/dns_query_time.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/dns_query_time.conf)|
|
||||
http|python<br />v2 or v3|Monitors a generic web page for status code and returned content in HTML
|
||||
port|ptyhon<br />v2 or v3|Checks if a generic TCP port for its availability and response time
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Time Servers
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
chrony|python<br/>v2 or v3|Uses the chronyc command to provide chrony statistics (Frequency, Last offset, RMS offset, Residual freq, Root delay, Root dispersion, Skew, System time).<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [chrony.chart.py](https://github.com/netdata/netdata/blob/master/python.d/chrony.chart.py)<br/>configuration file: [python.d/chrony.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/chrony.conf)|
|
||||
ntpd|python<br/>v2 or v3|Connects to multiple ntpd servers (local or remote) to provide statistics of system variables and optional also peer variables (if enabled in the configuration).<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [ntpd.chart.py](https://github.com/netdata/netdata/blob/master/python.d/ntpd.chart.py)<br/>configuration file: [python.d/ntpd.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/ntpd.conf)|
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Security
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
freeradius|python<br/>v2 or v3|Uses the radclient command to provide freeradius statistics (authentication, accounting, proxy-authentication, proxy-accounting).<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [freeradius.chart.py](https://github.com/netdata/netdata/blob/master/python.d/freeradius.chart.py)<br/>configuration file: [python.d/freeradius.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/freeradius.conf)|
|
||||
openvpn|python<br/>v2 or v3|All data from openvpn-status.log in your dashboard! <br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [ovpn_status_log.chart.py](https://github.com/netdata/netdata/blob/master/python.d/ovpn_status_log.chart.py)<br/>configuration file: [python.d/ovpn_status_log.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/ovpn_status_log.conf)|
|
||||
fail2ban|python<br/>v2 or v3|Monitor fail2ban log file to show all bans for all active jails <br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [fail2ban.chart.py](https://github.com/netdata/netdata/blob/master/python.d/fail2ban.chart.py)<br/>configuration file: [python.d/fail2ban.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/fail2ban.conf)|
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Telephony Servers
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
opensips|BASH<br/>Shell Script|Connects to an opensips server (local only) to collect real-time performance metrics.<br/> <br/>netdata plugin: [charts.d.plugin](General-Info-charts\.d)<br/>plugin module: [opensips.chart.sh](https://github.com/netdata/netdata/blob/master/charts.d/opensips.chart.sh)<br/>configuration file: [charts.d/opensips.conf](https://github.com/netdata/netdata/blob/master/conf.d/charts.d/opensips.conf)|
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Go applications
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
go_expvar|python<br/>v2 or v3|Parses metrics exposed by applications written in the Go programming language using the [expvar package](https://golang.org/pkg/expvar/).<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [go_expvar.chart.py](https://github.com/netdata/netdata/blob/master/python.d/go_expvar.chart.py)<br/>configuration file: [python.d/go_expvar.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/go_expvar.conf)<br/>wiki page / documentation: [Monitoring Go Applications](https://github.com/netdata/netdata/wiki/Monitoring-Go-Applications)|
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Household Appliances
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
sma_webbox|node.js|Connects to multiple remote SMA webboxes to collect real-time performance metrics of the photovoltaic (solar) power generation.<br/> <br/>netdata plugin: [node.d.plugin](General-Info-node\.d)<br/>plugin module: [sma_webbox.node.js](https://github.com/netdata/netdata/blob/master/node.d/sma_webbox.node.js)<br/>configuration file: [node.d/sma_webbox.conf](https://github.com/netdata/netdata/blob/master/conf.d/node.d/sma_webbox.conf.md)|
|
||||
fronius|node.js|Connects to multiple remote Fronius Symo servers to collect real-time performance metrics of the photovoltaic (solar) power generation.<br/> <br/>netdata plugin: [node.d.plugin](General-Info-node\.d)<br/>plugin module: [fronius.node.js](https://github.com/netdata/netdata/blob/master/node.d/fronius.node.js)<br/>configuration file: [node.d/fronius.conf](https://github.com/netdata/netdata/blob/master/conf.d/node.d/fronius.conf.md)|
|
||||
stiebeleltron|node.js|Collects the temperatures and other metrics from your Stiebel Eltron heating system using their Internet Service Gateway (ISG web).<br/> <br/>netdata plugin: [node.d.plugin](General-Info-node\.d)<br/>plugin module: [stiebeleltron.node.js](https://github.com/netdata/netdata/blob/master/node.d/stiebeleltron.node.js)<br/>configuration file: [node.d/stiebeleltron.conf](https://github.com/netdata/netdata/blob/master/conf.d/node.d/stiebeleltron.conf.md)|
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Java Processes
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
java process|java|Connects to multiple java processes (local or remote) and collects metrics of Java Management Extension (JMX) M(X)Beans.<br/> <br/>netdata plugin: [java.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/java.d.plugin)<br/>plugin module: [jmx](https://github.com/netdata/netdata/blob/master/java.d/src/main/java/org/firehol/netdata/module/jmx/JmxPlugin.java)<br/>configuration file: [java.d/jmx.conf](https://github.com/netdata/netdata/blob/master/conf.d/java.d/jmx.conf)
|
||||
Spring Boot Application|java|Monitors running Java [Spring Boot](https://spring.io/) applications that expose their metrics with the use of the **Spring Boot Actuator** included in Spring Boot library.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [springboot](https://github.com/netdata/netdata/blob/master/python.d/springboot.chart.py)<br/>configuration file: [python.d/springboot.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/stringboot.conf)
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Provisioning Systems
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
puppet|python<br/>v2 or v3|Connects to multiple Puppet Server and Puppet DB instances (local or remote) to collect real-time status metrics.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [puppet.chart.py](https://github.com/netdata/netdata/blob/master/python.d/puppet.chart.py)<br/>configuration file: [python.d/puppet.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/puppet.conf)|
|
||||
|
||||
---
|
||||
|
||||
## Game Servers
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
SpigotMC|Python<br/>v2 or v3|Monitors Spigot Minecraft server ticks per second and number of online players using the Minecraft remote console.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d/plugin)<br/>plugin module: [spigotmc.chart.py](https://github.com/netdata/netdata/blob/master/python.d/spigotmc.chart.py)<br/>configuration file: [python.d/spigotmc.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/spigotmc.conf)|
|
||||
|
||||
---
|
||||
|
||||
## Distributed Computing Clients
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
BOINC|Python<br/>v2 or v3|Monitors task states for local and remote BOINC client software using the remote GUI RPC interface. Also provides alarms for a handful of error conditions. Requires manual configuration<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d/plugin)<br/>plugin module: [boinc.chart.py](https://github.com/netdata/netdata/blob/master/python.d/boinc.chart.py)<br/>configuration file: [python.d/boinc.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/boinc.conf)|
|
||||
|
||||
---
|
||||
|
||||
## Skeleton Plugins
|
||||
|
||||
application|language|notes|
|
||||
:---------:|:------:|:----|
|
||||
example|BASH<br/>Shell Script|Skeleton plugin in BASH.<br/><br/>DEPRECATED IN FAVOR OF THE PYTHON ONE. It is still supplied only as an example module to shell scripting plugins.<br/> <br/>netdata plugin: [charts.d.plugin](General-Info-charts\.d)<br/>plugin module: [example.chart.sh](https://github.com/netdata/netdata/blob/master/charts.d/example.chart.sh)<br/>configuration file: [charts.d/example.conf](https://github.com/netdata/netdata/blob/master/conf.d/charts.d/example.conf)|
|
||||
example|python<br/>v2 or v3|Skeleton plugin in Python.<br/> <br/>netdata plugin: [python.d.plugin](https://github.com/netdata/netdata/blob/master/plugins.d/python.d.plugin)<br/>plugin module: [example.chart.py](https://github.com/netdata/netdata/blob/master/python.d/example.chart.py)<br/>configuration file: [python.d/example.conf](https://github.com/netdata/netdata/blob/master/conf.d/python.d/example.conf)|
|
||||
19
doc/Demo-Sites.md
Normal file
19
doc/Demo-Sites.md
Normal file
|
|
@ -0,0 +1,19 @@
|
|||
# Demo Sites
|
||||
|
||||
Live demo installations of netdata are available at **[https://my-netdata.io](https://my-netdata.io)**:
|
||||
|
||||
Location | netdata demo URL | 60 mins reqs | VM Donated by
|
||||
:-------:|:-----------------:|:----------:|:-------------
|
||||
London (UK)|**[london.my-netdata.io](https://london.my-netdata.io)**<br/>(this is the global netdata **registry** and has **named** and **mysql** charts)|[](https://london.my-netdata.io)|[DigitalOcean.com](https://m.do.co/c/83dc9f941745)
|
||||
Atlanta (USA)|**[cdn77.my-netdata.io](https://cdn77.my-netdata.io)**<br/>(with **named** and **mysql** charts)|[](https://cdn77.my-netdata.io)|[CDN77.com](https://www.cdn77.com/)
|
||||
Israel|**[octopuscs.my-netdata.io](https://octopuscs.my-netdata.io)**|[](https://octopuscs.my-netdata.io)|[OctopusCS.com](https://www.octopuscs.com)
|
||||
Roubaix (France)|**[ventureer.my-netdata.io](https://ventureer.my-netdata.io)**|[](https://ventureer.my-netdata.io)|[Ventureer.com](https://ventureer.com/)
|
||||
Madrid (Spain)|**[stackscale.my-netdata.io](https://stackscale.my-netdata.io)**|[](https://stackscale.my-netdata.io)|[StackScale Spain](https://www.stackscale.es/)
|
||||
Bangalore (India)|**[bangalore.my-netdata.io](https://bangalore.my-netdata.io)**|[](https://bangalore.my-netdata.io)|[DigitalOcean.com](https://m.do.co/c/83dc9f941745)
|
||||
Frankfurt (Germany)|**[frankfurt.my-netdata.io](https://frankfurt.my-netdata.io)**|[](https://frankfurt.my-netdata.io)|[DigitalOcean.com](https://m.do.co/c/83dc9f941745)
|
||||
New York (USA)|**[newyork.my-netdata.io](https://newyork.my-netdata.io)**|[](https://newyork.my-netdata.io)|[DigitalOcean.com](https://m.do.co/c/83dc9f941745)
|
||||
San Francisco (USA)|**[sanfrancisco.my-netdata.io](https://sanfrancisco.my-netdata.io)**|[](https://sanfrancisco.my-netdata.io)|[DigitalOcean.com](https://m.do.co/c/83dc9f941745)
|
||||
Singapore|**[singapore.my-netdata.io](https://singapore.my-netdata.io)**|[](https://singapore.my-netdata.io)|[DigitalOcean.com](https://m.do.co/c/83dc9f941745)
|
||||
Toronto (Canada)|**[toronto.my-netdata.io](https://toronto.my-netdata.io)**|[](https://toronto.my-netdata.io)|[DigitalOcean.com](https://m.do.co/c/83dc9f941745)
|
||||
|
||||
*Netdata dashboards are mobile and touch friendly.*
|
||||
23
doc/Donations-netdata-has-received.md
Normal file
23
doc/Donations-netdata-has-received.md
Normal file
|
|
@ -0,0 +1,23 @@
|
|||
# Donations received
|
||||
|
||||
This is a list of the donations we have received for netdata (sorted alphabetically on their name):
|
||||
|
||||
what donated|related links|who donated|description of the donation
|
||||
----:|:-----:|:---:|:-----------
|
||||
Packages Distribution|-|**[PackageCloud.io](https://packagecloud.io/)**|**PackageCloud.io** donated to a free open-source subscription to their awesome Package Distribution services.
|
||||
Cross Browser Testing|-|**[BrowserStack.com](https://www.browserstack.com/)**|**BrowserStack.com** donated a free subscription to their awesome Browser Testing services (all three of them: Live, Screenshots, Responsive).
|
||||
Cloud VM|[cdn77.my-netdata.io](http://cdn77.my-netdata.io)|**[CDN77.com](https://www.cdn77.com/)**|**CDN77.com** donated a VM with 2 CPU cores, 4GB RAM and 20GB HD, on their excellent CDN network.
|
||||
Localization Management|[netdata localization project](https://crowdin.com/project/netdata) (check issue [#279](https://github.com/netdata/netdata/issues/279))|**[Crowdin.com](https://crowdin.com/)**|**Crowdin.com** donated an open source license to their Localization Management Platform.
|
||||
Cloud VMs|[london.my-netdata.io](https://london.my-netdata.io) (Several VMs)|**[DigitalOcean.com](https://www.digitalocean.com/)**|**DigitalOcean.com** donated 1000 USD to be used in their excellent Cloud Computing services. Many thanks to [Justin Paine](https://github.com/xxdesmus) for making this happen.
|
||||
Development IDE|-|**[JetBrains.com](https://www.jetbrains.com/)**|**JetBrains.com** donated an open source license for 4 developers for 1 year, to their excellent IDEs.
|
||||
Cloud VM|[octopuscs.my-netdata.io](https://octopuscs.my-netdata.io)|**[OctopusCS.com](https://octopuscs.com/)**|**OctopusCS.com** donated a VM with 4 CPU cores, 16GB RAM and 50GB HD in their excellent Cloud Computing services.
|
||||
Cloud VM|[ventureer.my-netdata.io](https://ventureer.my-netdata.io)|**[Ventureer.com](https://ventureer.com/)**|**Ventureer.com** donated a VM with 4 CPU cores, 8GB RAM and 50GB HD in their excellent Cloud Computing services.
|
||||
Cloud VM|[stackscale.my-netdata.io](https://stackscale.my-netdata.io)|**[stackscale.com](https://www.stackscale.com/)**|**StackScale.com** donated a VM with 4 CPU cores, 16GB RAM and 100GB HD in their excellent Cloud Computing services.
|
||||
|
||||
Thank you!
|
||||
|
||||
---
|
||||
|
||||
**Do you want to donate?** We are thirsty for on-line services that can help us make netdata better. We also try to build a network of demo sites (VMs) that can help us show the full potential of netdata.
|
||||
|
||||
Please contact me at costa@tsaousis.gr.
|
||||
37
doc/Netdata-Security-and-Disclosure-Information.md
Normal file
37
doc/Netdata-Security-and-Disclosure-Information.md
Normal file
|
|
@ -0,0 +1,37 @@
|
|||
# Netdata Security and Disclosure Information
|
||||
|
||||
This page describes netdata security and disclosure information.
|
||||
|
||||
## Security Announcements
|
||||
|
||||
Every time a security issue is fixed in netdata, we immediately release a new version of it. So, to get notified of all security incidents, please subscribe to our releases on github.
|
||||
|
||||
## Report a Vulnerability
|
||||
|
||||
We’re extremely grateful for security researchers and users that report vulnerabilities to Netdata Open Source Community. All reports are thoroughly investigated by a set of community volunteers.
|
||||
|
||||
To make a report, please email the private [security@netdata.cloud](mailto:security@netdata.cloud) list with the security details and the details expected for [all netdata bug reports](https://github.com/netdata/netdata/.github/ISSUE_TEMPLATE.md).
|
||||
|
||||
## When Should I Report a Vulnerability?
|
||||
|
||||
- You think you discovered a potential security vulnerability in Netdata
|
||||
- You are unsure how a vulnerability affects Netdata
|
||||
- You think you discovered a vulnerability in another project that Netdata depends on (e.g. python, node, etc)
|
||||
|
||||
### When Should I NOT Report a Vulnerability?
|
||||
|
||||
- You need help tuning Netdata for security
|
||||
- You need help applying security related updates
|
||||
- Your issue is not security related
|
||||
|
||||
## Security Vulnerability Response
|
||||
|
||||
Each report is acknowledged and analyzed by Netdata Team members within 3 working days. This will set off a Security Release Process.
|
||||
|
||||
Any vulnerability information shared with Netdata Team stays within Netdata project and will not be disseminated to other projects unless it is necessary to get the issue fixed.
|
||||
|
||||
As the security issue moves from triage, to identified fix, to release planning we will keep the reporter updated.
|
||||
|
||||
## Public Disclosure Timing
|
||||
|
||||
A public disclosure date is negotiated by the Netdata team and the bug submitter. We prefer to fully disclose the bug as soon as possible once a user mitigation is available. It is reasonable to delay disclosure when the bug or the fix is not yet fully understood, the solution is not well-tested, or for vendor coordination. The timeframe for disclosure is from immediate (especially if it's already publicly known) to a few weeks. As a basic default, we expect report date to disclosure date to be on the order of 7 days. The Netdata team holds the final say when setting a disclosure date.
|
||||
73
doc/Performance.md
Normal file
73
doc/Performance.md
Normal file
|
|
@ -0,0 +1,73 @@
|
|||
# Netdata Performance
|
||||
|
||||
Netdata performance is affected by:
|
||||
|
||||
**Data collection**
|
||||
- the number of charts for which data are collected
|
||||
- the number of plugins running
|
||||
- the technology of the plugins (i.e. BASH plugins are slower than binary plugins)
|
||||
- the frequency of data collection
|
||||
|
||||
You can control all the above.
|
||||
|
||||
**Web clients accessing the data**
|
||||
- the duration of the charts in the dashboard
|
||||
- the number of charts refreshes requested
|
||||
- the compression level of the web responses
|
||||
|
||||
---
|
||||
|
||||
## Netdata Daemon
|
||||
|
||||
For most server systems, with a few hundred charts and a few thousand dimensions, the netdata daemon, without any web clients accessing it, should not use more than 1% of a single core.
|
||||
|
||||
To prove netdata scalability, check issue [#1323](https://github.com/netdata/netdata/issues/1323#issuecomment-265501668) where netdata collects 95.000 metrics per second, with 12% CPU utilization of a single core!
|
||||
|
||||
In embedded systems, if the netdata daemon is using a lot of CPU without any web clients accessing it, you should lower the data collection frequency. To set the data collection frequency, edit `/etc/netdata/netdata.conf` and set `update_every` to a higher number (this is the frequency in seconds data are collected for all charts: higher number of seconds = lower frequency, the default is 1 for per second data collection). You can also set this frequency per module or chart. Check the **[[Configuration]]** section.
|
||||
|
||||
## Plugins
|
||||
|
||||
If a plugin is using a lot of CPU, you should lower its update frequency, or if you wrote it, re-factor it to be more CPU efficient. Check **[[External Plugins]]** for more details on writing plugins.
|
||||
|
||||
## CPU consumption when web clients are accessing dashboards
|
||||
|
||||
Netdata is very efficient when servicing web clients. On most server platforms, netdata should be able to serve **1800 web client requests per second per core** for auto-refreshing charts.
|
||||
|
||||
Normally, each user connected will request less than 10 chart refreshes per second (the page may have hundreds of charts, but only the visible are refreshed). So you can expect 180 users per CPU core accessing dashboards before having any delays.
|
||||
|
||||
Netdata runs with the lowest possible process priority, so even if 1000 users are accessing dashboards, it should not influence your applications. CPU utilization will reach 100%, but your applications should get all the CPU they need.
|
||||
|
||||
To lower the CPU utilization of netdata when clients are accessing the dashboard, set `web compression level = 1`, or disable web compression completely by setting `enable web responses gzip compression = no`. Both settings are in the `[web]` section.
|
||||
|
||||
|
||||
## Monitoring a heavy loaded system
|
||||
|
||||
Netdata, while running, does not depend on disk I/O (apart its log files and `access.log` is written with buffering enabled and can be disabled). Some plugins that need disk may stop and show gaps during heavy system load, but the netdata daemon itself should be able to work and collect values from `/proc` and `/sys` and serve web clients accessing it.
|
||||
|
||||
Keep in mind that netdata saves its database when it exits and loads it back when restarted. While it is running though, its DB is only stored in RAM and no I/O takes place for it.
|
||||
|
||||
|
||||
## Running netdata in embedded devices
|
||||
|
||||
Embedded devices usually have very limited CPU resources available, and in most cases, just a single core.
|
||||
|
||||
We suggest to do the following:
|
||||
|
||||
#### external plugins
|
||||
|
||||
`charts.d.plugin` and `apps.plugin`, each consumes twice the CPU resources of the netdata daemon.
|
||||
|
||||
If you don't need them, disable them (edit `/etc/netdata/netdata.conf` and search for the plugins section).
|
||||
|
||||
If you need them, increase their `update every` value (again in `/etc/netdata/netdata.conf`), so that they do not run that frequently.
|
||||
|
||||
#### internal plugins
|
||||
|
||||
If netdata is still using a lot of CPU, lower its update frequency. Going from per second updates, to once every 2 seconds updates, will cut the CPU resources of all netdata programs **in half**, and you will still have very frequent updates.
|
||||
|
||||
If the CPU of the embedded device is too weak, try setting even lower update frequency. Experiment with `update every = 5` or `update every = 10` (higher number = lower frequency), until you get acceptable results.
|
||||
|
||||
#### Single threaded web server
|
||||
|
||||
Normally, netdata spawns a thread for each web client. This allows netdata to utilize all the available cores for servicing chart refreshes. You can however disable this feature and serve all charts one after another, using a single thread / core. This will might lower the CPU pressure on the embedded device. To enable the single threaded web server, edit `/etc/netdata/netdata.conf` and set `mode = single-threaded` in the `[web]` section.
|
||||
|
||||
268
doc/Running-behind-apache.md
Normal file
268
doc/Running-behind-apache.md
Normal file
|
|
@ -0,0 +1,268 @@
|
|||
# netdata via apache's mod_proxy
|
||||
|
||||
Below you can find instructions for configuring an apache server to:
|
||||
|
||||
1. proxy a single netdata via an HTTP and HTTPS virtual host
|
||||
2. dynamically proxy any number of netdata
|
||||
3. add user authentication
|
||||
4. adjust netdata settings to get optimal results
|
||||
|
||||
|
||||
## Requirements
|
||||
|
||||
Make sure your apache has installed `mod_proxy` and `mod_proxy_http`.
|
||||
|
||||
On debian/ubuntu systems, install them with this:
|
||||
|
||||
```sh
|
||||
sudo apt-get install libapache2-mod-proxy-html
|
||||
```
|
||||
|
||||
Also make sure they are enabled:
|
||||
|
||||
```
|
||||
sudo a2enmod proxy
|
||||
sudo a2enmod proxy_http
|
||||
```
|
||||
|
||||
Ensure your rewrite module is enabled:
|
||||
|
||||
```
|
||||
sudo a2enmod rewrite
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## netdata on an existing virtual host
|
||||
|
||||
On any **existing** and already **working** apache virtual host, you can redirect requests for URL `/netdata/` to one or more netdata servers.
|
||||
|
||||
### proxy one netdata, running on the same server apache runs
|
||||
|
||||
Add the following on top of any existing virtual host. It will allow you to access netdata as `http://virtual.host/netdata/`.
|
||||
|
||||
```
|
||||
<VirtualHost *:80>
|
||||
|
||||
RewriteEngine On
|
||||
ProxyRequests Off
|
||||
ProxyPreserveHost On
|
||||
|
||||
<Proxy *>
|
||||
Require all granted
|
||||
</Proxy>
|
||||
|
||||
# Local netdata server accessed with '/netdata/', at localhost:19999
|
||||
ProxyPass "/netdata/" "http://localhost:19999/" connectiontimeout=5 timeout=30 keepalive=on
|
||||
ProxyPassReverse "/netdata/" "http://localhost:19999/"
|
||||
|
||||
# if the user did not give the trailing /, add it
|
||||
# for HTTP (if the virtualhost is HTTP, use this)
|
||||
RewriteRule ^/netdata$ http://%{HTTP_HOST}/netdata/ [L,R=301]
|
||||
# for HTTPS (if the virtualhost is HTTPS, use this)
|
||||
#RewriteRule ^/netdata$ https://%{HTTP_HOST}/netdata/ [L,R=301]
|
||||
|
||||
# rest of virtual host config here
|
||||
|
||||
</VirtualHost>
|
||||
```
|
||||
|
||||
### proxy multiple netdata running on multiple servers
|
||||
|
||||
Add the following on top of any existing virtual host. It will allow you to access multiple netdata as `http://virtual.host/netdata/HOSTNAME/`, where `HOSTNAME` is the hostname of any other netdata server you have (to access the `localhost` netdata, use `http://virtual.host/netdata/localhost/`).
|
||||
|
||||
```
|
||||
<VirtualHost *:80>
|
||||
|
||||
RewriteEngine On
|
||||
ProxyRequests Off
|
||||
ProxyPreserveHost On
|
||||
|
||||
<Proxy *>
|
||||
Require all granted
|
||||
</Proxy>
|
||||
|
||||
# proxy any host, on port 19999
|
||||
ProxyPassMatch "^/netdata/([A-Za-z0-9\._-]+)/(.*)" "http://$1:19999/$2" connectiontimeout=5 timeout=30 keepalive=on
|
||||
|
||||
# make sure the user did not forget to add a trailing /
|
||||
# for HTTP (if the virtualhost is HTTP, use this)
|
||||
RewriteRule "^/netdata/([A-Za-z0-9\._-]+)$" http://%{HTTP_HOST}/netdata/$1/ [L,R=301]
|
||||
# for HTTPS (if the virtualhost is HTTPS, use this)
|
||||
RewriteRule "^/netdata/([A-Za-z0-9\._-]+)$" https://%{HTTP_HOST}/netdata/$1/ [L,R=301]
|
||||
|
||||
# rest of virtual host config here
|
||||
|
||||
</VirtualHost>
|
||||
```
|
||||
|
||||
> IMPORTANT<br/>
|
||||
> The above config allows your apache users to connect to port 19999 on any server on your network.
|
||||
|
||||
If you want to control the servers your users can connect to, replace the `ProxyPassMatch` line with the following. This allows only `server1`, `server2`, `server3` and `server4`.
|
||||
|
||||
```
|
||||
ProxyPassMatch "^/netdata/(server1|server2|server3|server4)/(.*)" "http://$1:19999/$2" connectiontimeout=5 timeout=30 keepalive=on
|
||||
```
|
||||
|
||||
## netdata on a dedicated virtual host
|
||||
|
||||
You can proxy netdata through apache, using a dedicated apache virtual host.
|
||||
|
||||
Create a new apache site:
|
||||
|
||||
```sh
|
||||
nano /etc/apache2/sites-available/netdata.conf
|
||||
```
|
||||
|
||||
with this content:
|
||||
|
||||
```
|
||||
<VirtualHost *:80>
|
||||
RewriteEngine On
|
||||
ProxyRequests Off
|
||||
ProxyPreserveHost On
|
||||
|
||||
ServerName netdata.domain.tld
|
||||
|
||||
<Proxy *>
|
||||
Require all granted
|
||||
</Proxy>
|
||||
|
||||
ProxyPass "/" "http://localhost:19999/" connectiontimeout=5 timeout=30 keepalive=on
|
||||
ProxyPassReverse "/" "http://localhost:19999/"
|
||||
|
||||
ErrorLog ${APACHE_LOG_DIR}/netdata-error.log
|
||||
CustomLog ${APACHE_LOG_DIR}/netdata-access.log combined
|
||||
</VirtualHost>
|
||||
```
|
||||
|
||||
Enable the VirtualHost:
|
||||
|
||||
```sh
|
||||
sudo a2ensite netdata.conf && service apache2 reload
|
||||
```
|
||||
|
||||
## Netdata proxy in Plesk
|
||||
_Assuming the main goal is to make Netdata running in HTTPS._
|
||||
1. Make a subdomain for Netdata on which you enable and force HTTPS - You can use a free Let's Encrypt certificate
|
||||
2. Go to "Apache & nginx Settings", and in the following section, add:
|
||||
```
|
||||
RewriteEngine on
|
||||
RewriteRule (.*) http://localhost:19999/$1 [P,L]
|
||||
```
|
||||
3. Optional: If your server is remote, then just replace "localhost" with your actual hostname or IP, it just works.
|
||||
|
||||
Repeat the operation for as many servers as you need.
|
||||
|
||||
|
||||
## Enable Basic Auth
|
||||
|
||||
If you wish to add an authentication (user/password) to access your netdata, do these:
|
||||
|
||||
Install the package `apache2-utils`. On debian / ubuntu run `sudo apt-get install apache2-utils`.
|
||||
|
||||
Then, generate password for user `netdata`, using `htpasswd -c /etc/apache2/.htpasswd netdata`
|
||||
|
||||
Modify the virtual host with these:
|
||||
|
||||
```
|
||||
# replace the <Proxy *> section
|
||||
<Proxy *>
|
||||
Order deny,allow
|
||||
Allow from all
|
||||
</Proxy>
|
||||
|
||||
# add a <Location /netdata/> section
|
||||
<Location /netdata/>
|
||||
AuthType Basic
|
||||
AuthName "Protected site"
|
||||
AuthUserFile /etc/apache2/.htpasswd
|
||||
Require valid-user
|
||||
Order deny,allow
|
||||
Allow from all
|
||||
</Location>
|
||||
```
|
||||
|
||||
Specify `Location /` if netdata is running on dedicated virtual host.
|
||||
|
||||
Note: Changes are applied by reloading or restarting Apache.
|
||||
|
||||
# Netdata configuration
|
||||
|
||||
You might edit `/etc/netdata/netdata.conf` to optimize your setup a bit. For applying these changes you need to restart netdata.
|
||||
|
||||
## Response compression
|
||||
|
||||
If you plan to use netdata exclusively via apache, you can gain some performance by preventing double compression of its output (netdata compresses its response, apache re-compresses it) by editing `/etc/netdata/netdata.conf` and setting:
|
||||
|
||||
```
|
||||
[web]
|
||||
enable gzip compression = no
|
||||
```
|
||||
|
||||
Once you disable compression at netdata (and restart it), please verify you receive compressed responses from apache (it is important to receive compressed responses - the charts will be more snappy).
|
||||
|
||||
## Limit direct access to netdata
|
||||
|
||||
You would also need to instruct netdata to listen only on `localhost`, `127.0.0.1` or `::1`.
|
||||
|
||||
```
|
||||
[web]
|
||||
bind to = localhost
|
||||
```
|
||||
or
|
||||
```
|
||||
[web]
|
||||
bind to = 127.0.0.1
|
||||
```
|
||||
or
|
||||
```
|
||||
[web]
|
||||
bind to = ::1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
You can also use a unix domain socket. This will also provide a faster route between apache and netdata:
|
||||
|
||||
```
|
||||
[web]
|
||||
bind to = unix:/tmp/netdata.sock
|
||||
```
|
||||
_note: netdata v1.8+ support unix domain sockets_
|
||||
|
||||
At the apache side, prepend the 2nd argument to `ProxyPass` with `unix:/tmp/netdata.sock|`, like this:
|
||||
|
||||
```
|
||||
ProxyPass "/netdata/" "unix:/tmp/netdata.sock|http://localhost:19999/" connectiontimeout=5 timeout=30 keepalive=on
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
If your apache server is not on localhost, you can set:
|
||||
|
||||
```
|
||||
[web]
|
||||
bind to = *
|
||||
allow connections from = IP_OF_APACHE_SERVER
|
||||
```
|
||||
_note: netdata v1.9+ support `allow connections from`_
|
||||
|
||||
`allow connections from` accepts [netdata simple patterns](https://github.com/netdata/netdata/wiki/Configuration#netdata-simple-patterns) to match against the connection IP address.
|
||||
|
||||
## prevent the double access.log
|
||||
|
||||
apache logs accesses and netdata logs them too. You can prevent netdata from generating its access log, by setting this in `/etc/netdata/netdata.conf`:
|
||||
|
||||
```
|
||||
[global]
|
||||
access log = none
|
||||
```
|
||||
|
||||
## Troubleshooting mod_proxy
|
||||
|
||||
Make sure the requests reach netdata, by examing `/var/log/netdata/access.log`.
|
||||
|
||||
1. if the requests do not reach netdata, your apache does not forward them.
|
||||
2. if the requests reach netdata by the URLs are wrong, you have not re-written them properly.
|
||||
27
doc/Running-behind-caddy.md
Normal file
27
doc/Running-behind-caddy.md
Normal file
|
|
@ -0,0 +1,27 @@
|
|||
# netdata via Caddy
|
||||
|
||||
To run netdata via [Caddy's proxying,](https://caddyserver.com/docs/proxy) set your Caddyfile up like this:
|
||||
|
||||
```
|
||||
netdata.domain.tld {
|
||||
proxy / localhost:19999
|
||||
}
|
||||
```
|
||||
|
||||
Other directives can be added between the curly brackets as needed.
|
||||
|
||||
To run netdata in a subfolder:
|
||||
|
||||
```
|
||||
netdata.domain.tld {
|
||||
proxy /netdata/ localhost:19999 {
|
||||
without /netdata
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## limit direct access to netdata
|
||||
|
||||
You would also need to instruct netdata to listen only to `127.0.0.1` or `::1`.
|
||||
|
||||
To limit access to netdata only from localhost, set `bind socket to IP = 127.0.0.1` or `bind socket to IP = ::1` in `/etc/netdata/netdata.conf`.
|
||||
60
doc/Running-behind-lighttpd.md
Normal file
60
doc/Running-behind-lighttpd.md
Normal file
|
|
@ -0,0 +1,60 @@
|
|||
# lighttpd v1.4.x
|
||||
|
||||
Here is a config for accessing netdata in a suburl via lighttpd 1.4.46 and newer:
|
||||
|
||||
```txt
|
||||
$HTTP["url"] =~ "^/netdata/" {
|
||||
proxy.server = ( "" => ("netdata" => ( "host" => "127.0.0.1", "port" => 19999 )))
|
||||
proxy.header = ( "map-urlpath" => ( "/netdata/" => "/") )
|
||||
}
|
||||
```
|
||||
|
||||
If you have older lighttpd you have to use a chain (such as bellow), as explained [at this stackoverflow answer](http://stackoverflow.com/questions/14536554/lighttpd-configuration-to-proxy-rewrite-from-one-domain-to-another).
|
||||
|
||||
```txt
|
||||
$HTTP["url"] =~ "^/netdata/" {
|
||||
proxy.server = ( "" => ("" => ( "host" => "127.0.0.1", "port" => 19998 )))
|
||||
}
|
||||
|
||||
$SERVER["socket"] == ":19998" {
|
||||
url.rewrite-once = ( "^/netdata(.*)$" => "/$1" )
|
||||
proxy.server = ( "" => ( "" => ( "host" => "127.0.0.1", "port" => 19999 )))
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
If the only thing the server is exposing via the web is netdata (and thus no suburl rewriting required),
|
||||
then you can get away with just
|
||||
```
|
||||
proxy.server = ( "" => ( ( "host" => "127.0.0.1", "port" => 19999 )))
|
||||
```
|
||||
Though if it's public facing you might then want to put some authentication on it. htdigest support
|
||||
looks like:
|
||||
```
|
||||
auth.backend = "htdigest"
|
||||
auth.backend.htdigest.userfile = "/etc/lighttpd/lighttpd.htdigest"
|
||||
auth.require = ( "" => ( "method" => "digest",
|
||||
"realm" => "netdata",
|
||||
"require" => "valid-user"
|
||||
)
|
||||
)
|
||||
```
|
||||
other auth methods, and more info on htdigest, can be found in lighttpd's [mod_auth docs](http://redmine.lighttpd.net/projects/lighttpd/wiki/Docs_ModAuth).
|
||||
|
||||
---
|
||||
|
||||
It seems that lighttpd (or some versions of it), fail to proxy compressed web responses.
|
||||
To solve this issue, disable web response compression in netdata.
|
||||
|
||||
Open /etc/netdata/netdata.conf and set in [global]:
|
||||
|
||||
```
|
||||
enable web responses gzip compression = no
|
||||
```
|
||||
|
||||
## limit direct access to netdata
|
||||
|
||||
You would also need to instruct netdata to listen only to `127.0.0.1` or `::1`.
|
||||
|
||||
To limit access to netdata only from localhost, set `bind socket to IP = 127.0.0.1` or `bind socket to IP = ::1` in `/etc/netdata/netdata.conf`.
|
||||
202
doc/Running-behind-nginx.md
Normal file
202
doc/Running-behind-nginx.md
Normal file
|
|
@ -0,0 +1,202 @@
|
|||
# netdata via nginx
|
||||
|
||||
To pass netdata via a nginx, use this:
|
||||
|
||||
### As a virtual host
|
||||
|
||||
```
|
||||
upstream backend {
|
||||
# the netdata server
|
||||
server 127.0.0.1:19999;
|
||||
keepalive 64;
|
||||
}
|
||||
|
||||
server {
|
||||
# nginx listens to this
|
||||
listen 80;
|
||||
|
||||
# the virtual host name of this
|
||||
server_name netdata.example.com;
|
||||
|
||||
location / {
|
||||
proxy_set_header X-Forwarded-Host $host;
|
||||
proxy_set_header X-Forwarded-Server $host;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_pass http://backend;
|
||||
proxy_http_version 1.1;
|
||||
proxy_pass_request_headers on;
|
||||
proxy_set_header Connection "keep-alive";
|
||||
proxy_store off;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### As a subfolder to an existing virtual host
|
||||
|
||||
```
|
||||
upstream netdata {
|
||||
server 127.0.0.1:19999;
|
||||
keepalive 64;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
|
||||
# the virtual host name of this subfolder should be exposed
|
||||
#server_name netdata.example.com;
|
||||
|
||||
location = /netdata {
|
||||
return 301 /netdata/;
|
||||
}
|
||||
|
||||
location ~ /netdata/(?<ndpath>.*) {
|
||||
proxy_redirect off;
|
||||
proxy_set_header Host $host;
|
||||
|
||||
proxy_set_header X-Forwarded-Host $host;
|
||||
proxy_set_header X-Forwarded-Server $host;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_http_version 1.1;
|
||||
proxy_pass_request_headers on;
|
||||
proxy_set_header Connection "keep-alive";
|
||||
proxy_store off;
|
||||
proxy_pass http://netdata/$ndpath$is_args$args;
|
||||
|
||||
gzip on;
|
||||
gzip_proxied any;
|
||||
gzip_types *;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### As a subfolder for multiple netdata servers, via one nginx
|
||||
|
||||
```
|
||||
upstream backend-server1 {
|
||||
server 10.1.1.103:19999;
|
||||
keepalive 64;
|
||||
}
|
||||
upstream backend-server2 {
|
||||
server 10.1.1.104:19999;
|
||||
keepalive 64;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
|
||||
# the virtual host name of this subfolder should be exposed
|
||||
#server_name netdata.example.com;
|
||||
|
||||
location ~ /netdata/(?<behost>.*)/(?<ndpath>.*) {
|
||||
proxy_set_header X-Forwarded-Host $host;
|
||||
proxy_set_header X-Forwarded-Server $host;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_http_version 1.1;
|
||||
proxy_pass_request_headers on;
|
||||
proxy_set_header Connection "keep-alive";
|
||||
proxy_store off;
|
||||
proxy_pass http://backend-$behost/$ndpath$is_args$args;
|
||||
|
||||
gzip on;
|
||||
gzip_proxied any;
|
||||
gzip_types *;
|
||||
}
|
||||
|
||||
# make sure there is a trailing slash at the browser
|
||||
# or the URLs will be wrong
|
||||
location ~ /netdata/(?<behost>.*) {
|
||||
return 301 /netdata/$behost/;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Of course you can add as many backend servers as you like.
|
||||
|
||||
Using the above, you access netdata on the backend servers, like this:
|
||||
|
||||
- `http://nginx.server/netdata/server1/` to reach `backend-server1`
|
||||
- `http://nginx.server/netdata/server2/` to reach `backend-server2`
|
||||
|
||||
|
||||
### Enable authentication
|
||||
|
||||
Create an authentication file to enable the nginx basic authentication.
|
||||
Do not use authentication without SSL/TLS!
|
||||
If you haven't one you can do the following:
|
||||
|
||||
```
|
||||
printf "yourusername:$(openssl passwd -apr1)" > /etc/nginx/passwords
|
||||
```
|
||||
|
||||
And enable the authentication inside your server directive:
|
||||
|
||||
```
|
||||
server {
|
||||
# ...
|
||||
auth_basic "Protected";
|
||||
auth_basic_user_file passwords;
|
||||
# ...
|
||||
}
|
||||
```
|
||||
|
||||
## limit direct access to netdata
|
||||
|
||||
If your nginx is on `localhost`, you can use this to protect your netdata:
|
||||
|
||||
```
|
||||
[web]
|
||||
bind to = 127.0.0.1 ::1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
You can also use a unix domain socket. This will also provide a faster route between nginx and netdata:
|
||||
|
||||
```
|
||||
[web]
|
||||
bind to = unix:/tmp/netdata.sock
|
||||
```
|
||||
_note: netdata v1.8+ support unix domain sockets_
|
||||
|
||||
At the nginx side, use something like this to use the same unix domain socket:
|
||||
|
||||
```
|
||||
upstream backend {
|
||||
server unix:/tmp/netdata.sock;
|
||||
keepalive 64;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
If your nginx server is not on localhost, you can set:
|
||||
|
||||
```
|
||||
[web]
|
||||
bind to = *
|
||||
allow connections from = IP_OF_NGINX_SERVER
|
||||
```
|
||||
|
||||
_note: netdata v1.9+ support `allow connections from`_
|
||||
|
||||
`allow connections from` accepts [netdata simple patterns](https://github.com/netdata/netdata/wiki/Configuration#netdata-simple-patterns) to match against the connection IP address.
|
||||
|
||||
## prevent the double access.log
|
||||
|
||||
nginx logs accesses and netdata logs them too. You can prevent netdata from generating its access log, by setting this in `/etc/netdata/netdata.conf`:
|
||||
|
||||
```
|
||||
[global]
|
||||
access log = none
|
||||
```
|
||||
|
||||
## SELinux
|
||||
|
||||
If you get an 502 Bad Gateway error you might check your nginx error log:
|
||||
|
||||
```sh
|
||||
# cat /var/log/nginx/error.log:
|
||||
2016/09/09 12:34:05 [crit] 5731#5731: *1 connect() to 127.0.0.1:19999 failed (13: Permission denied) while connecting to upstream, client: 1.2.3.4, server: netdata.example.com, request: "GET / HTTP/2.0", upstream: "http://127.0.0.1:19999/", host: "netdata.example.com"
|
||||
```
|
||||
|
||||
If you see something like the above, chances are high that SELinux prevents nginx from connecting to the backend server. To fix that, just use this policy: `setsebool -P httpd_can_network_connect true`.
|
||||
23
doc/Third-Party-Plugins.md
Normal file
23
doc/Third-Party-Plugins.md
Normal file
|
|
@ -0,0 +1,23 @@
|
|||
# Third-party Plugins
|
||||
|
||||
The following is a list of Netdata plugins distributed by third parties:
|
||||
|
||||
## Nvidia GPUs
|
||||
|
||||
[netdata nv plugin](https://github.com/coraxx/netdata_nv_plugin) monitors nvidia GPUs.
|
||||
|
||||
|
||||
|
||||
## teamspeak 3
|
||||
|
||||
[teamspeak 3 plugin](https://github.com/coraxx/netdata_ts3_plugin) polls active users and bandwidth from TeamSpeak 3 servers.
|
||||
|
||||
## SSH
|
||||
|
||||
[SSH module](https://github.com/Yaser-Amiri/netdata-ssh-module) monitors failed authentication requests of SSH server.
|
||||
|
||||
## interactive users count
|
||||
Collect [number of currently logged-on users](https://github.com/veksh/netdata-numsessions)
|
||||
|
||||
## CyberPower UPS
|
||||
[cyberups plugin](https://github.com/HawtDogFlvrWtr/netdata_cyberpwrups_plugin) polls the USB connected CyberPower UPS for stats.
|
||||
170
doc/Why-Netdata.md
Normal file
170
doc/Why-Netdata.md
Normal file
|
|
@ -0,0 +1,170 @@
|
|||
# Why Netdata
|
||||
|
||||
|
||||
|
||||
## Netdata is unique!
|
||||
|
||||
The following is an animated GIF showing **netdata**'s ability to monitor QoS. The timings of this animation have not been altered, this is the real thing:
|
||||
|
||||
|
||||
|
||||
Check the details on this animation:
|
||||
|
||||
1. At the beginning the charts auto-refresh, in real-time
|
||||
2. Charts can be dragged and zoomed (either mouse or touch)
|
||||
3. You pan or zoom one, the others follow
|
||||
4. Mouse over on one, selects the same timestamp on all
|
||||
5. Dimensions can be enabled or disabled
|
||||
6. All refreshes are instant (an 8 year old core-2 duo computer was used to record this)
|
||||
|
||||
There are a lot of excellent open source tools for collecting and visualizing performance metrics. Check for example [collectd](https://collectd.org/), [OpenTSDB](http://opentsdb.net/), [influxdb](https://influxdata.com/), [Grafana](http://grafana.org/), etc.
|
||||
|
||||
So, why **netdata**?
|
||||
|
||||
Well, **netdata** has a quite different approach.
|
||||
|
||||
## Simplicity
|
||||
|
||||
> Most monitoring solutions require endless configuration of whatever imaginable. Well, this is a linux box. Why do we need to configure every single metric we need to monitor. Of course it has a CPU and RAM and a few disks, and ethernet ports, it might run a firewall, a web server, or a database server and so on. Why do we need to configure all these metrics?
|
||||
|
||||
**Netdata** has been designed to auto-detect everything. Of course you can enable, tweak or disable things. But by default, if **netdata** can retrieve `/server-status` from an web server you run on your linux box, it will automatically collect all performance metrics. This happens for apache, squid, nginx, mysql, opensips, etc. It will also automatically collect all available system values for CPU, memory, disks, network interfaces, QoS (with labels if you also use [FireQOS](http://firehol.org)), etc. Even for applications that do not offer performance metrics, it will automatically group the whole process tree and provide metrics like CPU usage, memory allocated, opened files, sockets, disk activity, swap activity, etc per application group.
|
||||
|
||||
Netdata supports plenty of [configuration](../daemon/config/). However, we have done everything we can to allow netdata to auto-detect as much as possible.
|
||||
|
||||
Even netdata plugins are designed to support configuration-less operation. So, you just install and run netdata. You will need to configure something only if it cannot be auto-detected.
|
||||
|
||||
> Take any performance monitoring solution and try to troubleshoot a performance problem. At the end of the day you will have to ssh to the server to understand what exactly is happening. You will have to use `iostat`, `iotop`, `vmstat`, `top`, `iperf`, `ethtool` and probably a few dozen more console tools to figure it out.
|
||||
|
||||
With **netdata**, this need is eliminated significantly. Of course you will ssh. Just not for monitoring performance.
|
||||
|
||||
If you install **netdata** you will prefer it over the console tools. **Netdata** visualizes the data, while the console tools just show their values. The detail is the same - I have spent pretty much time reading the source code of the console tools, to figure out what needs to do done in netdata, so that the data, the values, will be the same. Actually, **netdata** is more precise than most console tools, it will interpolate all collected values to second boundary, so that even if something took a few microseconds more to be collected, netdata will correctly estimate the per second value.
|
||||
|
||||
**Netdata** visualizes data in ways you cannot even imagine on a console. It allows you to see the present in real-time, much like the console tools, but also the recent past, compare different metrics with each other, zoom in to see the recent past in detail, or zoom out to have a helicopter view of what is happening in longer durations, build custom dashboards with just the charts you need for a specific purpose.
|
||||
|
||||
Most engineers that install netdata, ssh to the server to tweak system or application settings and at the same time they monitor the result of the new settings in **netdata** on their browser.
|
||||
|
||||
## Per second data collection and visualization
|
||||
|
||||
**Per second data collection and visualization** is usually only available in dedicated console tools, like `top`, `vmstat`, `iostat`, etc. Netdata brings per second data collection and visualization to all applications, accessible through the web.
|
||||
|
||||
*You are not convinced per second data collection is important?*
|
||||
**Click** this image for a demo:
|
||||
|
||||
[](http://netdata.firehol.org/demo2.html)
|
||||
|
||||
## Realtime monitoring
|
||||
|
||||
> Any performance monitoring solution that does not go down to per second collection and visualization of the data, is useless. It will make you happy to have it, but it will not help you more than that.
|
||||
|
||||
Visualizing the present in **real-time and in great detail**, is the most important value a performance monitoring solution should provide. The next most important is the last hour, again per second. The next is the last 8 hours and so on, up to a week, or at most a month. In my 20+ years in IT, I needed just once or twice to look a year back. And this was mainly out of curiosity.
|
||||
|
||||
Of course real-time monitoring requires resources. **netdata** is designed to be very efficient:
|
||||
|
||||
1. collecting performance data is a repeating process - you do the same thing again and again. **Netdata** has been designed to learn from each iteration, so that the next one will be faster. It learns the sizes of files (it even keeps them open when it can), the number of lines and words per line they contain, the sizes of the buffers it needs to process them, etc. It adapts, so that everything will be as ready as possible for the next iteration.
|
||||
2. internally, it uses hashes and indexes (b-trees), to speed up lookups of metrics, charts, dimensions, settings.
|
||||
3. it has an in-memory round robin database based on a custom floating point number that allows it to pack values and flags together, in 32 bits, to lower its memory footprint.
|
||||
4. its internal web server is capable of generating JSON responses from live performance data with speeds comparable to static content delivery (it does not use `printf`, it is actually 11 times faster than in generating JSON compared to `printf`).
|
||||
|
||||
**Netdata** will use some CPU and memory, but it **will not produce any disk I/O at all**, apart its logs (which you can disable if you like).
|
||||
|
||||
Most servers should have plenty of CPU resources (I consider a hardware upgrade or application split when a server averages around 40% CPU utilization at the peak hour). Even if a server has limited CPU resources available, you can just lower the data collection frequency of **netdata**. Going from per second to every 2 seconds data collection, will cut the **netdata** CPU requirements in half and you will still get charts that are just 2 seconds behind.
|
||||
|
||||
The same goes for memory. If you just keep an hour of data (which is perfect for performance troubleshooting), you will most probably need 15-20MB. You can also enable the kernel de-duper (Kernel Same-Page Merging) and **netdata** will offer to it all its round robin database. KSM can free 20-60% of the memory used by **netdata** (guess why: there are a lot of metrics that are always zero or just constant).
|
||||
|
||||
When netdata runs on modern computers (even on CELERON processors), most chart queries are replied in less than 3 milliseconds! **Not seconds, MILLISECONDS!** Less than 3 milliseconds for calculating the chart, generating JSON text, compressing it and sending it to your web browser. Timings are logged in netdata's `access.log` for you to examine.
|
||||
|
||||
Netdata is written in plain `C` and the key system plugins are written in `C` too. Its speed can only be compared to the native console system administration tools.
|
||||
|
||||
You can also stress test your netdata installation by running the script `tests/stress.sh` found in the distribution. Most modern server hardware can serve more than 300 chart refreshes per second per core. A raspberry pi 2, can serve 300+ chart refreshes per second utilizing all of its 4 cores.
|
||||
|
||||
|
||||
## No disk I/O at all
|
||||
|
||||
Netdata does not use any disk I/O, apart from its logs and even these can be disabled.
|
||||
|
||||
Netdata will use some memory (you size it, check [[Memory Requirements]] and CPU (below 2% of a single core for the daemon, plugins may require more, check [[Performance]]), but normally your systems should have plenty of these resources available and spare.
|
||||
|
||||
The design goal of **NO DISK I/O AT ALL** effectively means netdata will not disrupt your applications.
|
||||
|
||||
## No root access
|
||||
|
||||
You don't need to run netdata as root. If started as root, netdata will switch to the `netdata` user (or any other user given in its configuration or command line argument).
|
||||
|
||||
There are a few plugins that in order to collect values need root access. These (and only these) are setuid to root.
|
||||
|
||||
## Visualizes QoS
|
||||
|
||||
Netdata visualizes `tc` QoS classes automatically. If you also use FireQOS, it will also collect interface and class names.
|
||||
|
||||
Check this animated GIF (generated with [ScreenToGif](https://github.com/NickeManarin/ScreenToGif)):
|
||||
|
||||
|
||||
|
||||
## Embedded web server
|
||||
|
||||
> Most solutions require dedicated servers to actually use the monitoring console. To my perspective, this is totally unneeded for performance monitoring. All of us have a spectacular tool on our desktops, that allows us to connect in real time to any server in the world: **the web browser**. It shouldn't be so hard to use the same tool to connect in real-time to all our servers.
|
||||
|
||||
With **netdata**, there is no need to centralize anything for performance monitoring. You view everything directly from their source. No need to run something else to access netdata. Of course you can use a firewall, or a reverse proxy, to limit access to it. But for most systems, inside your DMZ, just running it will be enough.
|
||||
|
||||
Still, with **netdata** you can build dashboards with charts from any number of servers. And these charts will be connected to each other much like the ones that come from the same server. You will hover on one and all of them will show the relative value for the selected timestamp. You will zoom or pan one and all of them will follow. **Netdata** achieves that because the logic that connects the charts together is at the browser, not the server, so that all charts presented on the same page are connected, no matter where they come from.
|
||||
|
||||
## Performance monitoring, scaled properly
|
||||
|
||||
"Properly"? What is "properly"?
|
||||
|
||||
We know software solutions can **scale up** (i.e. you replace its resources with bigger ones), or **scale out** (i.e. you add more smaller resources to it). In both cases, to get more of it, you need to supply **more resources**.
|
||||
|
||||
So, what is "scaled properly"?
|
||||
|
||||
Traditionally, monitoring solutions centralize all metric data to provide unified dashboards across all servers. So, you install agents on all your servers to collect system and application metrics which are then sent to a central place for storage and processing. Depending on the solution you use, the central place can either **scale up** or **scale out** (or a mix of the two).
|
||||
|
||||
"Scaled properly" is something completely different. "Scaled properly" minimizes the need for a "central place", so that **there is nothing to be scaled**!
|
||||
|
||||
Wait a moment! You cannot take out the "central place" of a monitoring solution!
|
||||
|
||||
Yes, we can! well... most of it, but before explaining how, let's see what happens today:
|
||||
|
||||
Monitoring solutions are a key component for any online service. These solutions usually consume considerable amount of resources. This is true for both "scale-up" and "scale-out" solutions. These resources require maintenance and administration too. To balance the resources required, these monitoring solutions follow a few simple rules:
|
||||
|
||||
1. The number of metrics collected per server is limited. They collect CPU, RAM, DISK, NETWORK metrics and a few application metrics.
|
||||
|
||||
2. The data collection frequency of each metric is also very low, at best it is once every 10 or 15 seconds, at worst every 5 or 10 mins.
|
||||
|
||||
Due to all the above, most centralized monitoring solutions are usually good for alarms and **statistics of past performance**. The alarms usually trigger every 1 to 5 minutes and you get a few low-resolution charts about the past performance of your servers.
|
||||
|
||||
Well... there is something wrong in this approach! Can you see it?
|
||||
|
||||
Let's see the netdata approach:
|
||||
|
||||
1. Data collection happens **per second**. This allows true real-time performance monitoring.
|
||||
|
||||
2. **Thousands of metrics** per server and application are collected, **every single second**. The number of metrics collected is not a problem.
|
||||
|
||||
3. Data do not leave the server they are collected. Data are not centralized, so the need for a huge central place that will process and store gazillions of data is not needed.
|
||||
|
||||
> Ok, I hear a few of you complaining already - you will find out... patience...
|
||||
|
||||
4. netdata does not use any DISK I/O while running (apart its log files - and even these can be disabled) and netdata runs with the lowest possible process priority, so that **your applications will never be affected by it**.
|
||||
|
||||
5. Each netdata is standalone. Your web browser connects directly to each server to present real-time dashboards. The charts are so snappy, so real-time, so fast that we can call netdata, **a console killer for performance monitoring**.
|
||||
|
||||
The charting libraries **netdata** uses, are the fastest possible ([Dygraphs](http://dygraphs.com/) do make the difference!) and **netdata** respects browser resources. Data are just rendered on a canvas. No processing in javascript at all.
|
||||
|
||||
6. netdata is very efficient: just 2% of a single core is required and some RAM, and you can actually control how much of both you want to allocate to it.
|
||||
|
||||
|
||||
Server side, chart data generation scales pretty well. You can expect 400+ chart refreshes per second per core on modern hardware. For a page with 10 charts visible (the page may have hundreds, but only the visible are refreshed), just a tiny fraction of a single CPU core will be used for servicing them. Even these refreshes stop when you switch tabs on your browser, you focus on another window, scroll to a part of the page without charts, zoom or pan a chart. And of course the **netdata** server runs with the lowest possible process priority, so that your production environment, your applications, will not be slowed down by the netdata server.
|
||||
|
||||
7. netdata dashboards can be multi-server (check: [http://my-netdata.io](http://my-netdata.io)) - your browser connects to each netdata server directly.
|
||||
|
||||
So, using netdata, your monitoring infrastructure is embedded on each server, limiting significantly the need of additional resources. netdata is very resource efficient and utilizes server resources that already exist and are spare (on each server).
|
||||
|
||||
Of course, there are a few issues that need to be addressed with this approach:
|
||||
|
||||
1. We need an index of all netdata installations we have
|
||||
2. We need a place to handle notifications and alarms
|
||||
3. We need a place to save statistics of past performance
|
||||
|
||||
Our approach uses the netdata [registry](../registry/). The registry solves the problem of maintaining a list of all the netdata installations we have. It does this transparently, without any configuration. It tracks the netdata servers your web browser has visited and bookmarks them at the `my-netdata` menu.
|
||||
|
||||
Every netdata can be a registry. You can use the global one we provided for free, or pick one of your netdata servers and turn it to a registry for your network.
|
||||
13
doc/a-github-star-is-important.md
Normal file
13
doc/a-github-star-is-important.md
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
# A GitHub start is important
|
||||
|
||||
**GitHub stars** allow netdata to expand its reach, its community, especially attract people with skills willing to contribute to it.
|
||||
|
||||
Compared to its first release, netdata is now **twice as fast**, has all its bugs settled and a lot more functionality. This happened because a lot of people find it useful, use it daily at home and work, **rely on it** and **contribute to it**.
|
||||
|
||||
**GitHub stars** also **motivate** us. They state that you find our work **useful**. They give us strength to continue, to work **harder** to make it even **better**.
|
||||
|
||||
So, give netdata a **GitHub star**, at the top right of this page.
|
||||
|
||||
Thank you!
|
||||
|
||||
Costa Tsaousis
|
||||
149
doc/high-performance-netdata.md
Normal file
149
doc/high-performance-netdata.md
Normal file
|
|
@ -0,0 +1,149 @@
|
|||
# High performance netdata
|
||||
|
||||
If you plan to run a netdata public on the internet, you will get the most performance out of it by following these rules:
|
||||
|
||||
## 1. run behind nginx
|
||||
|
||||
The internal web server is optimized to provide the best experience with few clients connected to it. Normally a web browser will make 4-6 concurrent connections to a web server, so that it can send requests in parallel. To best serve a single client, netdata spawns a thread for each connection it receives (so 4-6 threads per connected web browser).
|
||||
|
||||
If you plan to have your netdata public on the internet, this strategy wastes resources. It provides a lock-free environment so each thread is autonomous to serve the browser, but it does not scale well. Running netdata behind nginx, idle connections to netdata can be reused, thus improving significantly the performance of netdata.
|
||||
|
||||
In the following nginx configuration we do the following:
|
||||
|
||||
- allow nginx to maintain up to 1024 idle connections to netdata (so netdata will have up to 1024 threads waiting for requests)
|
||||
|
||||
- allow nginx to compress the responses of netdata (later we will disable gzip compression at netdata)
|
||||
|
||||
- we disable wordpress pingback attacks and allow only GET, HEAD and OPTIONS requests.
|
||||
|
||||
```
|
||||
upstream backend {
|
||||
server 127.0.0.1:19999;
|
||||
keepalive 1024;
|
||||
}
|
||||
|
||||
server {
|
||||
listen *:80;
|
||||
server_name my.web.server.name;
|
||||
|
||||
location / {
|
||||
proxy_set_header X-Forwarded-Host $host;
|
||||
proxy_set_header X-Forwarded-Server $host;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_pass http://backend;
|
||||
proxy_http_version 1.1;
|
||||
proxy_pass_request_headers on;
|
||||
proxy_set_header Connection "keep-alive";
|
||||
proxy_store off;
|
||||
gzip on;
|
||||
gzip_proxied any;
|
||||
gzip_types *;
|
||||
|
||||
# Block any HTTP requests other than GET, HEAD, and OPTIONS
|
||||
limit_except GET HEAD OPTIONS {
|
||||
deny all;
|
||||
}
|
||||
}
|
||||
|
||||
# WordPress Pingback Request Denial
|
||||
if ($http_user_agent ~* "WordPress") {
|
||||
return 403;
|
||||
}
|
||||
|
||||
}
|
||||
```
|
||||
|
||||
Then edit `/etc/netdata/netdata.conf` and set these config options:
|
||||
|
||||
```
|
||||
[global]
|
||||
bind socket to IP = 127.0.0.1
|
||||
access log = none
|
||||
disconnect idle web clients after seconds = 3600
|
||||
enable web responses gzip compression = no
|
||||
```
|
||||
|
||||
These options:
|
||||
|
||||
- `[global].bind socket to IP = 127.0.0.1` makes netdata listen only for requests from localhost (nginx).
|
||||
- `[global].access log = none` disables the access.log of netdata. It is not needed since netdata only listens for requests on 127.0.0.1 and thus only nginx can access it. nginx has its own access.log for your record.
|
||||
- `[global].disconnect idle web clients after seconds = 3600` will kill inactive web threads after an hour of inactivity.
|
||||
- `[global].enable web responses gzip compression = no` disables gzip compression at netdata (nginx will compress the responses).
|
||||
|
||||
## 2. increase open files limit (non-systemd)
|
||||
|
||||
By default Linux limits open file descriptors per process to 1024. This means that less than half of this number of client connections can be accepted by both nginx and netdata. To increase them, create 2 new files:
|
||||
|
||||
1. `/etc/security/limits.d/nginx.conf`, with these contents:
|
||||
|
||||
```
|
||||
nginx soft nofile 10000
|
||||
nginx hard nofile 30000
|
||||
```
|
||||
|
||||
2. `/etc/security/limits.d/netdata.conf`, with these contents:
|
||||
|
||||
```
|
||||
netdata soft nofile 10000
|
||||
netdata hard nofile 30000
|
||||
```
|
||||
|
||||
and to activate them, run:
|
||||
|
||||
```sh
|
||||
sysctl -p
|
||||
```
|
||||
|
||||
## 2b. increase open files limit (systemd)
|
||||
|
||||
Thanks to [@leleobhz](https://github.com/netdata/netdata/issues/655#issue-163932584), this is what you need to raise the limits using systemd:
|
||||
|
||||
This is based on https://ma.ttias.be/increase-open-files-limit-in-mariadb-on-centos-7-with-systemd/ and here worked as following:
|
||||
|
||||
1. Create the folders in /etc:
|
||||
|
||||
```
|
||||
mkdir -p /etc/systemd/system/netdata.service.d
|
||||
mkdir -p /etc/systemd/system/nginx.service.d
|
||||
```
|
||||
|
||||
2. Create limits.conf in each folder as following:
|
||||
|
||||
```
|
||||
[Service]
|
||||
LimitNOFILE=30000
|
||||
```
|
||||
|
||||
3. Reload systemd daemon list and restart services:
|
||||
|
||||
```sh
|
||||
systemctl daemon-reload
|
||||
systemctl restart netdata.service
|
||||
systemctl restart nginx.service
|
||||
```
|
||||
|
||||
You can check limits with following commands:
|
||||
|
||||
```sh
|
||||
cat /proc/$(ps aux | grep "nginx: master process" | grep -v grep | awk '{print $2}')/limits | grep "Max open files"
|
||||
cat /proc/$(ps aux | grep "netdata" | head -n1 | grep -v grep | awk '{print $2}')/limits | grep "Max open files"
|
||||
```
|
||||
|
||||
View of the files:
|
||||
|
||||
```sh
|
||||
# tree /etc/systemd/system/*service.d/etc/systemd/system/netdata.service.d
|
||||
/etc/systemd/system/netdata.service.d
|
||||
└── limits.conf
|
||||
/etc/systemd/system/nginx.service.d
|
||||
└── limits.conf
|
||||
|
||||
0 directories, 2 files
|
||||
|
||||
# cat /proc/$(ps aux | grep "nginx: master process" | grep -v grep | awk '{print $2}')/limits | grep "Max open files"
|
||||
Max open files 30000 30000 files
|
||||
|
||||
# cat /proc/$(ps aux | grep "netdata" | head -n1 | grep -v grep | awk '{print $2}')/limits | grep "Max open files"
|
||||
Max open files 30000 30000 files
|
||||
|
||||
```
|
||||
199
doc/netdata-for-IoT.md
Normal file
199
doc/netdata-for-IoT.md
Normal file
|
|
@ -0,0 +1,199 @@
|
|||
# Netdata for IoT
|
||||
|
||||
|
||||
|
||||
> New to netdata? Check its demo: **[https://my-netdata.io/](https://my-netdata.io/)**
|
||||
>
|
||||
> [](https://registry.my-netdata.io/#netdata_registry) [](https://registry.my-netdata.io/#netdata_registry) [](https://registry.my-netdata.io/#netdata_registry)
|
||||
>
|
||||
> [](https://registry.my-netdata.io/#netdata_registry) [](https://registry.my-netdata.io/#netdata_registry) [](https://registry.my-netdata.io/#netdata_registry)
|
||||
|
||||
---
|
||||
|
||||
netdata is a **very efficient** server performance monitoring solution. When running in server hardware, it can collect thousands of system and application metrics **per second** with just 1% CPU utilization of a single core. Its web server responds to most data requests in about **half a millisecond** making its web dashboards spontaneous, amazingly fast!
|
||||
|
||||
netdata can also be a very efficient real-time monitoring solution for **IoT devices** (RPIs, routers, media players, wifi access points, industrial controllers and sensors of all kinds). Netdata will generally run everywhere a Linux kernel runs (and it is glibc and [musl-libc](https://www.musl-libc.org/) friendly).
|
||||
|
||||
You can use it as both a data collection agent (where you pull data using its API), for embedding its charts on other web pages / consoles, but also for accessing it directly with your browser to view its dashboard.
|
||||
|
||||
The netdata web API already provides **reduce** functions allowing it to report **average** and **max** for any timeframe. It can also respond in many formats including JSON, JSONP, CSV, HTML. Its API is also a **google charts** provider so it can directly be used by google sheets, google charts, google widgets.
|
||||
|
||||
|
||||
|
||||
Although netdata has been significantly optimized to lower the CPU and RAM resources it consumes, the plethora of data collection plugins may be inappropriate for weak IoT devices.
|
||||
|
||||
> keep in mind that netdata on RPi 2 and 3 does not require any tuning. The default settings will be good. The following tunables apply only when running netdata on RPi 1 or other very weak IoT devices.
|
||||
|
||||
Here are a few tricks to control the resources consumed by netdata:
|
||||
|
||||
## 1. Disable External plugins
|
||||
|
||||
External plugins can consume more system resources than the netdata server. Disable the ones you don't need.
|
||||
|
||||
Edit `/etc/netdata/netdata.conf`, find the `[plugins]` section:
|
||||
|
||||
```
|
||||
[plugins]
|
||||
proc = yes
|
||||
|
||||
tc = no
|
||||
idlejitter = no
|
||||
cgroups = no
|
||||
checks = no
|
||||
apps = no
|
||||
charts.d = no
|
||||
node.d = no
|
||||
|
||||
plugins directory = /usr/libexec/netdata/plugins.d
|
||||
enable running new plugins = no
|
||||
check for new plugins every = 60
|
||||
```
|
||||
|
||||
In detail:
|
||||
|
||||
plugin|description
|
||||
:---:|:---------
|
||||
`proc`|the internal plugin used to monitor the system. Normally, you don't want to disable this. You can disable individual functions of it at the next section.
|
||||
`tc`|monitoring network interfaces QoS (tc classes)
|
||||
`idlejitter`|internal plugin (written in C) that attempts show if the systems starved for CPU. Disabling it will eliminate a thread.
|
||||
`cgroups`|monitoring linux containers. Most probably you are not going to need it. This will also eliminate another thread.
|
||||
`checks`|a debugging plugin, which is disabled by default.
|
||||
`apps`|a plugin that monitors system processes. It is very complex and heavy (heavier than the netdata daemon), so if you don't need to monitor the process tree, you can disable it.
|
||||
`charts.d`|BASH plugins (squid, nginx, mysql, etc). This is again a heavy plugin.
|
||||
`node.d`|node.js plugin, currently used for SNMP data collection and monitoring named (the name server).
|
||||
|
||||
For most IoT devices, you can disable all plugins except `proc`. For `proc` there is another section that controls which functions of it you need. Check the next section.
|
||||
|
||||
---
|
||||
|
||||
## 2. Disable internal plugins
|
||||
|
||||
In this section you can select which modules of the `proc` plugin you need. All these are run in a single thread, one after another. Still, each one needs some RAM and consumes some CPU cycles.
|
||||
|
||||
```
|
||||
[plugin:proc]
|
||||
# /proc/net/dev = yes # network interfaces
|
||||
# /proc/diskstats = yes # disks
|
||||
# /proc/net/snmp = yes # generic IPv4
|
||||
# /proc/net/snmp6 = yes # generic IPv6
|
||||
# /proc/net/netstat = yes # TCP and UDP
|
||||
# /proc/net/stat/conntrack = yes # firewall
|
||||
# /proc/net/ip_vs/stats = yes # IP load balancer
|
||||
# /proc/net/stat/synproxy = yes # Anti-DDoS
|
||||
# /proc/stat = yes # CPU, context switches
|
||||
# /proc/meminfo = yes # Memory
|
||||
# /proc/vmstat = yes # Memory operations
|
||||
# /proc/net/rpc/nfsd = yes # NFS Server
|
||||
# /proc/sys/kernel/random/entropy_avail = yes # Cryptography
|
||||
# /proc/interrupts = yes # Interrupts
|
||||
# /proc/softirqs = yes # SoftIRQs
|
||||
# /proc/loadavg = yes # Load Average
|
||||
# /sys/kernel/mm/ksm = yes # Memory deduper
|
||||
# netdata server resources = yes # netdata charts
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Disable logs
|
||||
|
||||
Normally, you will not need them. To disable them, set:
|
||||
|
||||
```
|
||||
[global]
|
||||
debug log = none
|
||||
error log = none
|
||||
access log = none
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Set memory mode to RAM
|
||||
|
||||
Setting the memory mode to `ram` will disable loading and saving the round robin database. This will not affect anything while running netdata, but it might be required if you have very limited storage available.
|
||||
|
||||
```
|
||||
[global]
|
||||
memory mode = ram
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. CPU utilization
|
||||
|
||||
If after disabling the plugins you don't need, netdata still uses a lot of CPU without any clients accessing the dashboard, try lowering its data collection frequency. Going from "once per second" to "once every two seconds" will not have a significant difference on the user experience, but it will cut the CPU resources required **in half**.
|
||||
|
||||
To set the update frequency, edit `/etc/netdata/netdata.conf` and set:
|
||||
|
||||
```
|
||||
[global]
|
||||
update every = 2
|
||||
```
|
||||
|
||||
You may have to increase this to 5 or 10 if the CPU of the device is weak.
|
||||
|
||||
Keep in mind this will also force dashboard chart refreshes to happen at the same rate. So increasing this number actually lowers data collection frequency but also lowers dashboard chart refreshes frequency.
|
||||
|
||||
This is a dashboard on a device with `[global].update every = 5` (this device is a media player and is now playing a movie):
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 6. Lower memory requirements
|
||||
|
||||
You can set the default size of the round robin database for all charts, using:
|
||||
|
||||
```
|
||||
[global]
|
||||
history = 600
|
||||
```
|
||||
|
||||
The units for history is `[global].update every` seconds. So if `[global].update every = 6` and `[global].history = 600`, you will have an hour of data ( 6 x 600 = 3.600 ), which will store 600 points per dimension, one every 6 seconds.
|
||||
|
||||
Check also [[Memory Requirements]] for directions on calculating the size of the round robin database.
|
||||
|
||||
---
|
||||
|
||||
## 7. Disable gzip compression of responses
|
||||
|
||||
Gzip compression of the web responses is using more CPU that the rest of netdata. You can lower the compression level or disable gzip compression completely. You can disable it, like this:
|
||||
|
||||
```
|
||||
[web]
|
||||
enable gzip compression = no
|
||||
```
|
||||
|
||||
To lower the compression level, do this:
|
||||
|
||||
```
|
||||
[web]
|
||||
enable gzip compression = yes
|
||||
gzip compression level = 1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
Finally, if no web server is installed on your device, you can use port tcp/80 for netdata:
|
||||
|
||||
```
|
||||
[global]
|
||||
port = 80
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Monitoring RPi temperature
|
||||
|
||||
The python version of the sensors plugin uses `lm-sensors`. Unfortunately the temperature reading of RPi are not supported by `lm-sensors`.
|
||||
|
||||
netdata also has a bash version of the sensors plugin that can read RPi temperatures. It is disabled by default to avoid the conflicts with the python version.
|
||||
|
||||
To enable it, edit `/etc/netdata/charts.d.conf` and uncomment this line:
|
||||
|
||||
```sh
|
||||
sensors=force
|
||||
```
|
||||
|
||||
Then restart netdata. You will get this:
|
||||
|
||||
|
||||
179
doc/netdata-security.md
Normal file
179
doc/netdata-security.md
Normal file
|
|
@ -0,0 +1,179 @@
|
|||
# Netdata Security
|
||||
|
||||
We have given special attention to all aspects of netdata, ensuring that everything throughout its operation is as secure as possible. netdata has been designed with security in mind.
|
||||
|
||||
**Table of Contents**
|
||||
|
||||
1. [your data are safe with netdata](#your-data-are-safe-with-netdata)
|
||||
2. [your systems are safe with netdata](#your-systems-are-safe-with-netdata)
|
||||
3. [netdata is read-only](#netdata-is-read-only)
|
||||
4. [netdata viewers authentication](#netdata-viewers-authentication)
|
||||
- [why netdata should be protected?](#why-netdata-should-be-protected)
|
||||
- [protect netdata from the internet](#protect-netdata-from-the-internet)
|
||||
- [expose netdata only in a private LAN](#expose-netdata-only-in-a-private-lan)
|
||||
- [use an authenticating web server in proxy mode](#use-an-authenticating-web-server-in-proxy-mode)
|
||||
- [other methods](#other-methods)
|
||||
5. [registry or how to not send any information to a thirdparty server](#registry-or-how-to-not-send-any-information-to-a-thirdparty-server)
|
||||
|
||||
## your data are safe with netdata
|
||||
|
||||
netdata collects raw data from many sources. For each source, netdata uses a plugin that connects to the source (or reads the relative files produced by the source), receives raw data and processes them to calculate the metrics shown on netdata dashboards.
|
||||
|
||||
Even if netdata plugins connect to your database server, or read your application log file to collect raw data, the product of this data collection process is always a number of **chart metadata and metric values** (summarized data for dashboard visualization). All netdata plugins (internal to the netdata daemon, and external ones written in any computer language), convert raw data collected into metrics, and only these metrics are stored in netdata databases, sent to upstream netdata servers, or archived to backend time-series databases.
|
||||
|
||||
> The **raw data** collected by netdata, do not leave the host they are collected. **The only data netdata exposes are chart metadata and metric values.**
|
||||
|
||||
This means that netdata can safely be used in environments that require the highest level of data isolation (like PCI Level 1).
|
||||
|
||||
## your systems are safe with netdata
|
||||
|
||||
We are very proud that **the netdata daemon runs as a normal system user, without any special privileges**. This is quite an achievement for a monitoring system that collects all kinds of system and application metrics.
|
||||
|
||||
There are a few cases however that raw source data are only exposed to processes with escalated privileges. To support these cases, netdata attempts to minimize and completely isolate the code that runs with escalated privileges.
|
||||
|
||||
So, netdata **plugins**, even those running with escalated capabilities or privileges, perform a **hard coded data collection job**. They do not accept commands from netdata. The communication is strictly **unidirectional**: from the plugin towards the netdata daemon. The original application data collected by each plugin do not leave the process they are collected, are not saved and are not transferred to the netdata daemon. The communication from the plugins to the netdata daemon includes only chart metadata and processed metric values.
|
||||
|
||||
netdata slaves streaming metrics to upstream netdata servers, use exactly the same protocol local plugins use. The raw data collected by the plugins of slave netdata servers are **never leaving the host they are collected**. The only data appearing on the wire are chart metadata and metric values. This communication is also **unidirectional**: slave netdata servers never accept commands from master netdata servers.
|
||||
|
||||
## netdata is read-only
|
||||
|
||||
netdata **dashboards are read-only**. Dashboard users can view and examine metrics collected by netdata, but cannot instruct netdata to do something other than present the already collected metrics.
|
||||
|
||||
netdata dashboards do not expose sensitive information. Business data of any kind, the kernel version, O/S version, application versions, host IPs, etc are not stored and are not exposed by netdata on its dashboards.
|
||||
|
||||
## netdata viewers authentication
|
||||
|
||||
netdata is a monitoring system. It should be protected, the same way you protect all your admin apps. We assume netdata will be installed privately, for your eyes only.
|
||||
|
||||
### why netdata should be protected?
|
||||
|
||||
Viewers will be able to get some information about the system netdata is running. This information is everything the dashboard provides. The dashboard includes a list of the services each system runs (the legends of the charts under the `Systemd Services` section), the applications running (the legends of the charts under the `Applications` section), the disks of the system and their names, the user accounts of the system that are running processes (the `Users` and `User Groups` section of the dashboard), the network interfaces and their names (not the IPs) and detailed information about the performance of the system and its applications.
|
||||
|
||||
This information is not sensitive (meaning that it is not your business data), but **it is important for possible attackers**. It will give them clues on what to check, what to try and in the case of DDoS against your applications, they will know if they are doing it right or not.
|
||||
|
||||
Also, viewers could use netdata itself to stress your servers. Although the netdata daemon runs unprivileged, with the minimum process priority (scheduling priority `idle` - lower than nice 19) and adjusts its OutOfMemory (OOM) score to 1000 (so that it will be first to be killed by the kernel if the system starves for memory), some pressure can be applied on your systems if someone attempts a DDoS against netdata.
|
||||
|
||||
### protect netdata from the internet
|
||||
|
||||
netdata is a distributed application. Most likely you will have many installations of it. Since it is distributed and you are expected to jump from server to server, there is very little usability to add authentication local on each netdata.
|
||||
|
||||
Until we add a distributed authentication method to netdata, you have the following options:
|
||||
|
||||
#### expose netdata only in a private LAN
|
||||
|
||||
If your organisation has a private administration and management LAN, you can bind netdata on this network interface on all your servers. This is done in `netdata.conf` with these settings:
|
||||
|
||||
```
|
||||
[web]
|
||||
bind to = 10.1.1.1:19999 localhost:19999
|
||||
```
|
||||
|
||||
You can bind netdata to multiple IPs and ports. If you use hostnames, netdata will resolve them and use all the IPs (in the above example `localhost` usually resolves to both `127.0.0.1` and `::1`).
|
||||
|
||||
**This is the best and the suggested way to protect netdata**. Your systems **should** have a private administration and management LAN, so that all management tasks are performed without any possibility of them being exposed on the internet.
|
||||
|
||||
For cloud based installations, if your cloud provider does not provide such a private LAN (or if you use multiple providers), you can create a virtual management and administration LAN with tools like `tincd` or `gvpe`. These tools create a mesh VPN allowing all servers to communicate securely and privately. Your administration stations join this mesh VPN to get access to management and administration tasks on all your cloud servers.
|
||||
|
||||
For `gvpe` we have developed a [simple provisioning tool](https://github.com/netdata/netdata-demo-site/tree/master/gvpe) you may find handy (it includes statically compiled `gvpe` binaries for Linux and FreeBSD, and also a script to compile `gvpe` on your Mac). We use this to create a management and administration LAN for all netdata demo sites (spread all over the internet using multiple hosting providers).
|
||||
|
||||
---
|
||||
|
||||
In netdata v1.9+ there is also access list support, like this:
|
||||
|
||||
```
|
||||
[web]
|
||||
bind to = *
|
||||
allow connections from = localhost 10.* 192.168.*
|
||||
```
|
||||
|
||||
|
||||
#### use an authenticating web server in proxy mode
|
||||
|
||||
Use **one nginx** (or one apache) server to provide authentication in front of **all your netdata servers**. So, you will be accessing all your netdata with URLs like `http://nginx.host/netdata/{NETDATA_HOSTNAME}/` and authentication will be shared among all of them (you will sign-in once for all your servers). Check [this wiki page for more information on configuring nginx for such a setup](https://github.com/netdata/netdata/wiki/Running-behind-nginx#as-a-subfolder-for-multiple-netdata-servers-via-one-nginx).
|
||||
|
||||
To use this method, you should firewall protect all your netdata servers, so that only the nginx IP will allowed to directly access netdata. To do this, run this on each of your servers (or use your firewall manager):
|
||||
|
||||
```sh
|
||||
NGINX_IP="1.2.3.4"
|
||||
iptables -t filter -I INPUT -p tcp --dport 19999 \! -s ${NGINX_IP} -m conntrack --ctstate NEW -j DROP
|
||||
```
|
||||
_commands to allow direct access to netdata from an nginx proxy_
|
||||
|
||||
The above will prevent anyone except your nginx server to access a netdata dashboard running on the host.
|
||||
|
||||
For netdata v1.9+ you can also use `netdata.conf`:
|
||||
|
||||
```
|
||||
[web]
|
||||
allow connections from = localhost 1.2.3.4
|
||||
```
|
||||
|
||||
Of course you can add more IPs.
|
||||
|
||||
For netdata prior to v1.9, if you want to allow multiple IPs, use this:
|
||||
|
||||
```sh
|
||||
# space separated list of IPs to allow access netdata
|
||||
NETDATA_ALLOWED="1.2.3.4 5.6.7.8 9.10.11.12"
|
||||
NETDATA_PORT=19999
|
||||
|
||||
# create a new filtering chain || or empty an existing one named netdata
|
||||
iptables -t filter -N netdata 2>/dev/null || iptables -t filter -F netdata
|
||||
for x in ${NETDATA_ALLOWED}
|
||||
do
|
||||
# allow this IP
|
||||
iptables -t filter -A netdata -s ${x} -j ACCEPT
|
||||
done
|
||||
|
||||
# drop all other IPs
|
||||
iptables -t filter -A netdata -j DROP
|
||||
|
||||
# delete the input chain hook (if it exists)
|
||||
iptables -t filter -D INPUT -p tcp --dport ${NETDATA_PORT} -m conntrack --ctstate NEW -j netdata 2>/dev/null
|
||||
|
||||
# add the input chain hook (again)
|
||||
# to send all new netdata connections to our filtering chain
|
||||
iptables -t filter -I INPUT -p tcp --dport ${NETDATA_PORT} -m conntrack --ctstate NEW -j netdata
|
||||
```
|
||||
_script to allow access to netdata only from a number of hosts_
|
||||
|
||||
You can run the above any number of times. Each time it runs it refreshes the list of allowed hosts.
|
||||
|
||||
#### other methods
|
||||
|
||||
Of course, there are many more methods you could use to protect netdata:
|
||||
|
||||
- bind netdata to localhost and use `ssh -L 19998:127.0.0.1:19999 remote.netdata.ip` to forward connections of local port 19998 to remote port 19999. This way you can ssh to a netdata server and then use `http://127.0.0.1:19998/` on your computer to access the remote netdata dashboard.
|
||||
|
||||
- If you are always under a static IP, you can use the script given above to allow direct access to your netdata servers without authentication, from all your static IPs.
|
||||
|
||||
- install all your netdata in **headless data collector** mode, forwarding all metrics in real-time to a master netdata server, which will be protected with authentication using an nginx server running locally at the master netdata server. This requires more resources (you will need a bigger master netdata server), but does not require any firewall changes, since all the slave netdata servers will not be listening for incoming connections.
|
||||
|
||||
## registry or how to not send any information to a third party server
|
||||
|
||||
The default configuration uses a public registry under registry.my-netdata.io (more information about the registry here: [mynetdata-menu-item](https://github.com/netdata/netdata/wiki/mynetdata-menu-item) ). Please be aware that if you use that public registry, you submit at least the following information to a third party server, which might violate your security policies:
|
||||
- Your public ip where the browser runs
|
||||
- The url where you open the web-ui in the browser (via http request referer)
|
||||
- The hostnames of the netdata servers
|
||||
|
||||
You are able to run your own registry, which is pretty simple to do:
|
||||
- If you have just one netdata web-ui, turn on registry and set the url of that web-ui as "registry to announce"
|
||||
```
|
||||
[registry]
|
||||
enabled = yes
|
||||
registry to announce = URL_OF_THE_NETDATA_WEB-UI
|
||||
```
|
||||
- If you run multiple netdata servers with web-ui, you need to define one as registry. On that node activate the registry and setting its url as "registry to announce". On all other nodes do not enable the registry but define the same url.
|
||||
|
||||
restart netdata and check with developer tools of your browser which registry is called.
|
||||
|
||||
## netdata directories
|
||||
|
||||
path|owner|permissions| netdata |comments|
|
||||
:---|:----|:----------|:--------|:-------|
|
||||
`/etc/netdata`|user `root`<br/>group `netdata`|dirs `0755`<br/>files `0640`|reads|**netdata config files**<br/>may contain sensitive information, so group `netdata` is allowed to read them.
|
||||
`/usr/libexec/netdata`|user `root`<br/>group `root`|executable by anyone<br/>dirs `0755`<br/>files `0644` or `0755`|executes|**netdata plugins**<br/>permissions depend on the file - not all of them should have the executable flag.<br/>there are a few plugins that run with escalated privileges (Linux capabilities or `setuid`) - these plugins should be executable only by group `netdata`.
|
||||
`/usr/share/netdata`|user `root`<br/>group `netdata`|readable by anyone<br/>dirs `0755`<br/>files `0644`|reads and sends over the network|**netdata web static files**<br/>these files are sent over the network to anyone that has access to the netdata web server. netdata checks the ownership of these files (using settings at the `[web]` section of `netdata.conf`) and refuses to serve them if they are not properly owned. Symbolic links are not supported. netdata also refuses to serve URLs with `..` in their name.
|
||||
`/var/cache/netdata`|user `netdata`<br/>group `netdata`|dirs `0750`<br/>files `0660`|reads, writes, creates, deletes|**netdata ephemeral database files**<br/>netdata stores its ephemeral real-time database here.
|
||||
`/var/lib/netdata`|user `netdata`<br/>group `netdata`|dirs `0750`<br/>files `0660`|reads, writes, creates, deletes|**netdata permanent database files**<br/>netdata stores here the registry data, health alarm log db, etc.
|
||||
`/var/log/netdata`|user `netdata`<br/>group `root`|dirs `0755`<br/>files `0644`|writes, creates|**netdata log files**<br/>all the netdata applications, logs their errors or other informational messages to files in this directory. These files should be log rotated.
|
||||
117
docker/README.md
Normal file
117
docker/README.md
Normal file
|
|
@ -0,0 +1,117 @@
|
|||
# Install netdata with Docker
|
||||
|
||||
> :warning: As of Sep 9th, 2018 we ship [new docker builds](https://github.com/netdata/netdata/pull/3995), running netdata in docker with an ENTRYPOINT directive, not a COMMAND directive. Please adapt your execution scripts accordingly.
|
||||
> More information about ENTRYPOINT vs COMMAND is presented by goinbigdata [here](http://goinbigdata.com/docker-run-vs-cmd-vs-entrypoint/) and by docker docs [here](https://docs.docker.com/engine/reference/builder/#understand-how-cmd-and-entrypoint-interact).
|
||||
>
|
||||
> Also, the `latest` is now based on alpine, so **`alpine` is not updated any more** and `armv7hf` is now replaced with `armhf` (to comply with https://github.com/multiarch naming), so **`armv7hf` is not updated** either.
|
||||
|
||||
## Limitations
|
||||
|
||||
Running netdata in a container for monitoring the whole host, can limit its capabilities. Some data is not accessible or not as detailed as when running netdata on the host.
|
||||
|
||||
## Run netdata with docker command
|
||||
|
||||
Quickly start netdata with the docker command line.
|
||||
Netdata is then available at http://host:19999
|
||||
|
||||
This is good for an internal network or to quickly analyse a host.
|
||||
|
||||
For a permanent installation on a public server, you should [[secure the netdata instance|netdata-security]]. See below for an example of how to install netdata with an SSL reverse proxy and basic authentication.
|
||||
|
||||
```bash
|
||||
docker run -d --name=netdata \
|
||||
-p 19999:19999 \
|
||||
-v /proc:/host/proc:ro \
|
||||
-v /sys:/host/sys:ro \
|
||||
-v /var/run/docker.sock:/var/run/docker.sock:ro \
|
||||
--cap-add SYS_PTRACE \
|
||||
--security-opt apparmor=unconfined \
|
||||
firehol/netdata
|
||||
```
|
||||
|
||||
above can be converted to docker-compose file for ease of management:
|
||||
|
||||
```yaml
|
||||
version: '3'
|
||||
services:
|
||||
netdata:
|
||||
image: firehol/netdata
|
||||
hostname: example.com # set to fqdn of host
|
||||
ports:
|
||||
- 19999:19999
|
||||
cap_add:
|
||||
- SYS_PTRACE
|
||||
security_opt:
|
||||
- apparmor:unconfined
|
||||
volumes:
|
||||
- /proc:/host/proc:ro
|
||||
- /sys:/host/sys:ro
|
||||
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||
```
|
||||
|
||||
### Docker container names resolution
|
||||
|
||||
If you want to have your container names resolved by netdata it needs to have access to docker group. To achive that just add environment variable `PGID=999` to netdata container, where `999` is a docker group id from your host. This number can be found by running:
|
||||
```bash
|
||||
grep docker /etc/group | cut -d ':' -f 3
|
||||
```
|
||||
|
||||
## Install Netdata using Docker Compose with SSL/TLS enabled http proxy
|
||||
|
||||
You can use use the following docker-compose.yml and Caddyfile files to run netdata with docker.
|
||||
Replace the Domains and email address for Letsencrypt before starting.
|
||||
|
||||
### Prerequisites
|
||||
* [Docker](https://docs.docker.com/install/#server)
|
||||
* [Docker Compose](https://docs.docker.com/compose/install/)
|
||||
* Domain configured in DNS pointing to host.
|
||||
|
||||
### Caddyfile
|
||||
|
||||
This file needs to be placed in /opt with nams Caddyfile. Here you customize your domain and you need to provide your email address to obtain Letsencrypt certificate.
|
||||
Certificate renewal will happen automatically and will be executed internally by caddy server.
|
||||
|
||||
```
|
||||
netdata.example.org {
|
||||
proxy / netdata:19999
|
||||
tls admin@example.org
|
||||
}
|
||||
```
|
||||
|
||||
### docker-compose.yml
|
||||
|
||||
After setting Caddyfile run this with `docker-compose up -d` to have fully functioning netdata setup behind HTTP reverse proxy.
|
||||
|
||||
```yaml
|
||||
version: '3'
|
||||
volumes:
|
||||
caddy:
|
||||
|
||||
services:
|
||||
caddy:
|
||||
image: abiosoft/caddy
|
||||
ports:
|
||||
- 80:80
|
||||
- 443:443
|
||||
volumes:
|
||||
- /opt/Caddyfile:/etc/Caddyfile
|
||||
- caddy:/root/.caddy
|
||||
environment:
|
||||
ACME_AGREE: 'true'
|
||||
netdata:
|
||||
restart: always
|
||||
hostname: netdata.example.org
|
||||
image: firehol/netdata
|
||||
cap_add:
|
||||
- SYS_PTRACE
|
||||
security_opt:
|
||||
- apparmor:unconfined
|
||||
volumes:
|
||||
- /proc:/host/proc:ro
|
||||
- /sys:/host/sys:ro
|
||||
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||
```
|
||||
|
||||
### Restrict access with basic auth
|
||||
|
||||
You can restrict access by following [official caddy guide](https://caddyserver.com/docs/basicauth) and adding lines to Caddyfile.
|
||||
|
|
@ -486,7 +486,7 @@ The external script will be called for all status changes.
|
|||
## Examples
|
||||
|
||||
|
||||
Check the **[health.d directory](health.d)** for all alarms shipped with netdata.
|
||||
Check the **[health.d directory](health.d/)** for all alarms shipped with netdata.
|
||||
|
||||
Here are a few examples:
|
||||
|
||||
|
|
@ -645,7 +645,5 @@ Important: this will generate a lot of output in debug.log.
|
|||
You can find the context of charts by looking up the chart in either
|
||||
`http://your.netdata:19999/netdata.conf` or `http://your.netdata:19999/api/v1/charts`.
|
||||
|
||||
You can find how netdata interpreted the expressions by examining the alarm at
|
||||
`http://your.netdata:19999/api/v1/alarms?all`. For each expression, netdata will return the
|
||||
expression as given in its config file, and the same expression with additional parentheses
|
||||
added to indicate the evaluation flow of the expression.
|
||||
You can find how netdata interpreted the expressions by examining the alarm at `http://your.netdata:19999/api/v1/alarms?all`. For each expression, netdata will return the expression as given in its config file, and the same expression with additional parentheses added to indicate the evaluation flow of the expression.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
# Netdata alarm notifications
|
||||
|
||||
The `exec` line in health configuration defines an external script that will be called once
|
||||
the alarm is triggered. The default script is **[alarm-notify.sh](alarm-notify.sh.in)**.
|
||||
the alarm is triggered. The default script is **[alarm-notify.sh](https://github.com/netdata/netdata/tree/master/health/notifications/alarm-notify.sh.in)**.
|
||||
|
||||
You can change the default script globally by editing `/etc/netdata/netdata.conf`.
|
||||
|
||||
|
|
@ -15,7 +15,7 @@ It uses **roles**. For example `sysadmin`, `webmaster`, `dba`, etc.
|
|||
|
||||
Each alarm is assigned to one or more roles, using the `to` line of the alarm configuration.
|
||||
Then `alarm-notify.sh` uses its own configuration file `/etc/netdata/health_alarm_notify.conf`
|
||||
the default is [here](health_alarm_notify.conf)
|
||||
the default is [here](https://github.com/netdata/netdata/tree/master/health/notifications/health_alarm_notify.conf)
|
||||
(to edit it on your system run `/etc/netdata/edit-config health_alarm_notify.conf`)
|
||||
to find the destination address of the notification for each method.
|
||||
|
||||
|
|
@ -31,7 +31,7 @@ So, for example the `sysadmin` role may send:
|
|||
|
||||
## Configuration
|
||||
|
||||
Edit [`/etc/netdata/health_alarm_notify.conf`](health_alarm_notify.conf)
|
||||
Edit [`/etc/netdata/health_alarm_notify.conf`](https://github.com/netdata/netdata/tree/master/health/notifications/health_alarm_notify.conf)
|
||||
by running `/etc/netdata/edit-config health_alarm_notify.conf`:
|
||||
|
||||
- settings per notification method:
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ consolidate and de-duplicate alerts from multiple sources for quick
|
|||
‘at-a-glance’ visualisation. With just one system you can monitor
|
||||
alerts from many other monitoring tools on a single screen.
|
||||
|
||||
|
||||
|
||||
|
||||
Netadata alarms can be sent to Alerta so you can see in one place
|
||||
alerts coming from many Netdata hosts or also from a multi-host
|
||||
|
|
|
|||
21
htmldoc/buildhtml.sh
Executable file
21
htmldoc/buildhtml.sh
Executable file
|
|
@ -0,0 +1,21 @@
|
|||
#!/bin/bash
|
||||
|
||||
# buildhtml.sh
|
||||
|
||||
# Builds the html static site, using mkdocs
|
||||
# Assumes that the script is executed from the root netdata folder, by calling htmldoc/buildhtml.sh
|
||||
|
||||
# Copy all netdata .md files to htmldoc/src. Exclude htmldoc itself and also the directory node_modules generated by Netlify
|
||||
rm -rf htmldoc/src
|
||||
find . -type d \( -path ./htmldoc -o -path ./node_modules \) -prune -o -name "*.md" -print | cpio -pd htmldoc/src
|
||||
|
||||
# Modify the first line of the main README.md, to enable proper static html generation
|
||||
sed -i '0,/# netdata /s//# Introducing NetData\n\n/' htmldoc/src/README.md
|
||||
|
||||
# Generate mkdocs.yaml
|
||||
htmldoc/buildyaml.sh > htmldoc/mkdocs.yml
|
||||
|
||||
# Build html docs
|
||||
mkdocs build --config-file=htmldoc/mkdocs.yml
|
||||
|
||||
|
||||
171
htmldoc/buildyaml.sh
Executable file
171
htmldoc/buildyaml.sh
Executable file
|
|
@ -0,0 +1,171 @@
|
|||
#!/bin/bash
|
||||
|
||||
cd htmldoc/src
|
||||
|
||||
# create yaml nav subtree with all the files directly under a specific directory
|
||||
# arguments:
|
||||
# tabs - how deep do we show it in the hierarchy. Level 1 is the top level, max should probably be 3
|
||||
# directory - to get mds from to add them to the yaml
|
||||
# file - can be left empty to include all files
|
||||
# name - what do we call the relevant section on the navbar. Empty if no new section is required
|
||||
# maxdepth - how many levels of subdirectories do I include in the yaml in this section. 1 means just the top level and is the default if left empty
|
||||
# excludefirstlevel - Optional param. If passed, mindepth is set to 2, to exclude the READMEs in the first directory level
|
||||
|
||||
navpart () {
|
||||
tabs=$1
|
||||
dir=$2
|
||||
file=$3
|
||||
section=$4
|
||||
maxdepth=$5
|
||||
excludefirstlevel=$6
|
||||
spc=""
|
||||
|
||||
i=1
|
||||
while [ ${i} -lt ${tabs} ] ; do
|
||||
spc=" $spc"
|
||||
i=$[$i + 1]
|
||||
done
|
||||
|
||||
if [ -z "$file" ] ; then file='*' ; fi
|
||||
if [[ ! -z "$section" ]] ; then echo "$spc- ${section}:" ; fi
|
||||
if [ -z "$maxdepth" ] ; then maxdepth=1; fi
|
||||
if [[ ! -z "$excludefirstlevel" ]] ; then mindepth=2 ; else mindepth=1; fi
|
||||
|
||||
for f in $(find $dir -mindepth $mindepth -maxdepth $maxdepth -name "${file}.md" -printf '%h\0%d\0%p\n' | sort -t '\0' -n | awk -F '\0' '{print $3}'); do
|
||||
# If I'm adding a section, I need the child links to be one level deeper than the requested level in "tabs"
|
||||
if [ -z "$section" ] ; then
|
||||
echo "$spc- '$f'"
|
||||
else
|
||||
echo "$spc - '$f'"
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
|
||||
echo -e 'site_name: NetData Documentation
|
||||
repo_url: https://github.com/netdata/netdata
|
||||
repo_name: GitHub
|
||||
edit_uri: blob/htmldoc
|
||||
site_description: NetData Documentation
|
||||
copyright: NetData, 2018
|
||||
docs_dir: src
|
||||
site_dir: build
|
||||
#use_directory_urls: false
|
||||
theme:
|
||||
name: "material"
|
||||
custom_dir: themes/material
|
||||
markdown_extensions:
|
||||
- extra
|
||||
- abbr
|
||||
- attr_list
|
||||
- def_list
|
||||
- fenced_code
|
||||
- footnotes
|
||||
- tables
|
||||
- admonition
|
||||
- codehilite
|
||||
- meta
|
||||
- nl2br
|
||||
- sane_lists
|
||||
- smarty
|
||||
- toc:
|
||||
permalink: True
|
||||
separator: "-"
|
||||
- wikilinks
|
||||
nav:'
|
||||
|
||||
navpart 1 . README "Getting Started"
|
||||
|
||||
echo -ne " - 'doc/Why-Netdata.md'
|
||||
- 'doc/Demo-Sites.md'
|
||||
- Installation:
|
||||
- 'installer/README.md'
|
||||
- 'docker/README.md'
|
||||
- 'installer/UPDATE.md'
|
||||
- 'installer/UNINSTALL.md'
|
||||
"
|
||||
echo -ne "- Using NetData:
|
||||
"
|
||||
navpart 2 daemon
|
||||
navpart 2 web "README" "Web Dashboards"
|
||||
navpart 3 web/gui "" "" 3
|
||||
navpart 2 web/server "" "Web Server"
|
||||
navpart 3 web/server "" "" 2 excludefirstlevel
|
||||
navpart 2 web/api "" "Web API"
|
||||
navpart 3 web/api "" "" 4 excludefirstlevel
|
||||
navpart 2 daemon/config
|
||||
#navpart 2 system
|
||||
navpart 2 registry
|
||||
navpart 2 streaming "" "" 4
|
||||
navpart 2 backends "" "Backends" 3
|
||||
navpart 2 database
|
||||
|
||||
echo -ne " - 'doc/Performance.md'
|
||||
- 'doc/netdata-for-IoT.md'
|
||||
- 'doc/high-performance-netdata.md'
|
||||
- 'doc/netdata-security.md'
|
||||
- 'doc/Netdata-Security-and-Disclosure-Information.md'
|
||||
"
|
||||
|
||||
navpart 2 health README "Health Monitoring"
|
||||
navpart 3 health/notifications "" "" 1
|
||||
navpart 3 health/notifications "" "Supported Notifications" 2 excludefirstlevel
|
||||
|
||||
echo -ne " - Running-behind-another-web-server:
|
||||
- 'doc/Running-behind-nginx.md'
|
||||
- 'doc/Running-behind-apache.md'
|
||||
- 'doc/Running-behind-lighttpd.md'
|
||||
- 'doc/Running-behind-caddy.md'
|
||||
"
|
||||
|
||||
|
||||
navpart 1 collectors "" "Data Collection" 1
|
||||
echo -ne " - 'doc/Add-more-charts-to-netdata.md'
|
||||
- Internal Plugins:
|
||||
"
|
||||
navpart 3 collectors/proc.plugin
|
||||
navpart 3 collectors/statsd.plugin
|
||||
navpart 3 collectors/cgroups.plugin
|
||||
navpart 3 collectors/idlejitter.plugin
|
||||
navpart 3 collectors/tc.plugin
|
||||
navpart 3 collectors/nfacct.plugin
|
||||
navpart 3 collectors/checks.plugin
|
||||
navpart 3 collectors/diskspace.plugin
|
||||
navpart 3 collectors/freebsd.plugin
|
||||
navpart 3 collectors/macos.plugin
|
||||
|
||||
navpart 2 collectors/plugins.d "" "External Plugins"
|
||||
navpart 3 collectors/python.d.plugin "" "Python Plugins" 3
|
||||
navpart 3 collectors/node.d.plugin "" "Node.js Plugins" 3
|
||||
navpart 3 collectors/charts.d.plugin "" "BASH Plugins" 3
|
||||
navpart 3 collectors/apps.plugin
|
||||
navpart 3 collectors/fping.plugin
|
||||
navpart 3 collectors/freeipmi.plugin
|
||||
|
||||
echo -ne " - Third Party Plugins:
|
||||
- 'doc/Third-Party-Plugins.md'
|
||||
"
|
||||
|
||||
echo -ne "- Hacking netdata:
|
||||
- CONTRIBUTING.md
|
||||
- CODE_OF_CONDUCT.md
|
||||
- CONTRIBUTORS.md
|
||||
"
|
||||
navpart 2 makeself "" "" 4
|
||||
navpart 2 packaging "" "" 4
|
||||
navpart 2 libnetdata "" "libnetdata" 4
|
||||
navpart 2 contrib
|
||||
navpart 2 tests
|
||||
navpart 2 diagrams/data_structures
|
||||
|
||||
echo -ne "- About:
|
||||
- 'doc/Donations-netdata-has-received.md'
|
||||
- 'doc/a-github-star-is-important.md'
|
||||
- CHANGELOG.md
|
||||
- HISTORICAL_CHANGELOG.md
|
||||
- REDISTRIBUTED.md
|
||||
"
|
||||
|
||||
|
||||
|
||||
|
||||
57
htmldoc/themes/material/partials/footer.html
Normal file
57
htmldoc/themes/material/partials/footer.html
Normal file
|
|
@ -0,0 +1,57 @@
|
|||
{% import "partials/language.html" as lang with context %}
|
||||
<footer class="md-footer">
|
||||
{% if page.previous_page or page.next_page %}
|
||||
<div class="md-footer-nav">
|
||||
<nav class="md-footer-nav__inner md-grid">
|
||||
{% if page.previous_page %}
|
||||
<a href="{{ page.previous_page.url | url }}" title="{{ page.previous_page.title }}" class="md-flex md-footer-nav__link md-footer-nav__link--prev" rel="prev">
|
||||
<div class="md-flex__cell md-flex__cell--shrink">
|
||||
<i class="md-icon md-icon--arrow-back md-footer-nav__button"></i>
|
||||
</div>
|
||||
<div class="md-flex__cell md-flex__cell--stretch md-footer-nav__title">
|
||||
<span class="md-flex__ellipsis">
|
||||
<span class="md-footer-nav__direction">
|
||||
{{ lang.t("footer.previous") }}
|
||||
</span>
|
||||
{{ page.previous_page.title }}
|
||||
</span>
|
||||
</div>
|
||||
</a>
|
||||
{% endif %}
|
||||
{% if page.next_page %}
|
||||
<a href="{{ page.next_page.url | url }}" title="{{ page.next_page.title }}" class="md-flex md-footer-nav__link md-footer-nav__link--next" rel="next">
|
||||
<div class="md-flex__cell md-flex__cell--stretch md-footer-nav__title">
|
||||
<span class="md-flex__ellipsis">
|
||||
<span class="md-footer-nav__direction">
|
||||
{{ lang.t("footer.next") }}
|
||||
</span>
|
||||
{{ page.next_page.title }}
|
||||
</span>
|
||||
</div>
|
||||
<div class="md-flex__cell md-flex__cell--shrink">
|
||||
<i class="md-icon md-icon--arrow-forward md-footer-nav__button"></i>
|
||||
</div>
|
||||
</a>
|
||||
{% endif %}
|
||||
</nav>
|
||||
</div>
|
||||
{% endif %}
|
||||
<div class="md-footer-meta md-typeset">
|
||||
<div class="md-footer-meta__inner md-grid">
|
||||
<div class="md-footer-copyright">
|
||||
{% if config.copyright %}
|
||||
<div class="md-footer-copyright__highlight">
|
||||
{{ config.copyright }}
|
||||
</div>
|
||||
{% endif %}
|
||||
<a href="https://twitter.com/linuxnetdata" rel="nofollow"><img src="https://img.shields.io/twitter/follow/linuxnetdata.svg?style=social&label=New%20-%20stay%20in%20touch%20-%20follow%20netdata%20on%20twitter"></a>
|
||||
<img src="https://www.google-analytics.com/collect?v=1&t=pageview&_s=1&ds=github&dr=https%3A%2F%2Fgithub.com%2Ffirehol%2Fnetdata%2Fwiki&dl=https%3A%2F%2Fmy-netdata.io%2Fgithub%2Fwiki&_u=MAC%7E&cid=8c51788e-8721-45e3-ae8c-e7c63ba8236b&tid=UA-64295674-3">
|
||||
|
||||
|
||||
</div>
|
||||
{% block social %}
|
||||
{% include "partials/social.html" %}
|
||||
{% endblock %}
|
||||
</div>
|
||||
</div>
|
||||
</footer>
|
||||
361
installer/README.md
Normal file
361
installer/README.md
Normal file
|
|
@ -0,0 +1,361 @@
|
|||
# Installation
|
||||
|
||||
|
||||
### Linux package managers
|
||||
|
||||
You can install the latest release of netdata, using your package manager in
|
||||
|
||||
- Arch Linux (`sudo pacman -S netdata`)
|
||||
- Alpine Linux (`sudo apk add netdata`)
|
||||
- Debian Linux (`sudo apt install netdata`)
|
||||
- Gentoo Linux (`sudo emerge --ask netdata`)
|
||||
- OpenSUSE (`sudo zypper install netdata`)
|
||||
- Solus Linux (`sudo eopkg install netdata`)
|
||||
- Ubuntu Linux >= 18.04 (`sudo apt install netdata`)
|
||||
|
||||
For security and portability reasons, this is the preferred installation method.
|
||||
|
||||
## Linux one liner
|
||||
|
||||
 
|
||||
|
||||
To install netdata from source to your systems and keep it up to date automatically, run the following:
|
||||
|
||||
:hash:**`bash <(curl -Ss https://my-netdata.io/kickstart.sh)`**
|
||||
|
||||
(do not `sudo` this command, it will do it by itself as needed)
|
||||
|
||||
The command:
|
||||
|
||||
1. detects the distro and **installs the required system packages** for building netdata (will ask for confirmation)
|
||||
2. downloads the latest netdata source tree to `/usr/src/netdata.git`.
|
||||
3. installs netdata by running `./netdata-installer.sh` from the source tree.
|
||||
4. installs `netdata-updater.sh` to `cron.daily`, so your netdata installation will be updated daily (you will get a message from cron only if the update fails).
|
||||
|
||||
The `kickstart.sh` script passes all its parameters to `netdata-installer.sh`, so you can add more parameters to change the installation directory, enable/disable plugins, etc (check below).
|
||||
|
||||
For automated installs, append a space + `--dont-wait` to the command line. You can also append `--dont-start-it` to prevent the installer from starting netdata. Example:
|
||||
|
||||
```sh
|
||||
bash <(curl -Ss https://my-netdata.io/kickstart.sh) all --dont-wait --dont-start-it
|
||||
```
|
||||
## Linux 64bit pre-built static binary
|
||||
|
||||
You can install a pre-compiled static binary of netdata for any Intel/AMD 64bit Linux system (even those that don't have a package manager, like CoreOS, CirrOS, busybox systems, etc). You can also use these packages on systems with broken or unsupported package managers.
|
||||
|
||||
<br/> 
|
||||
|
||||
To install netdata with a binary package on any Linux distro, any kernel version - for **Intel/AMD 64bit** hosts, run the following:
|
||||
|
||||
:hash: **`bash <(curl -Ss https://my-netdata.io/kickstart-static64.sh)`**
|
||||
|
||||
(do not `sudo` this command, it will do it by itself as needed; the target system does not need `bash` installed, check below for instructions to run it without `bash`)
|
||||
|
||||
*Note: The static builds install netdata at `/opt/netdata`*
|
||||
|
||||
For automated installs, append a space + `--dont-wait` to the command line. You can also append `--dont-start-it` to prevent the installer from starting netdata. Example:
|
||||
|
||||
```sh
|
||||
bash <(curl -Ss https://my-netdata.io/kickstart-static64.sh) --dont-wait --dont-start-it
|
||||
```
|
||||
|
||||
If your shell fails to handle the above one liner, do this:
|
||||
|
||||
```sh
|
||||
# download the script with curl
|
||||
curl https://my-netdata.io/kickstart-static64.sh >/tmp/kickstart-static64.sh
|
||||
|
||||
# or, download the script with wget
|
||||
wget -O /tmp/kickstart-static64.sh https://my-netdata.io/kickstart-static64.sh
|
||||
|
||||
# run the downloaded script (any sh is fine, no need for bash)
|
||||
sh /tmp/kickstart-static64.sh
|
||||
```
|
||||
|
||||
The static binary files are kept in repo [binary-packages](https://github.com/netdata/binary-packages). You can download any of the `.run` files, and run it. These files are self-extracting shell scripts built with [makeself](https://github.com/megastep/makeself). The target system does **not** need to have bash installed. The same files can be used for updates too.
|
||||
|
||||
## Other installation methods
|
||||
|
||||
- **Linux manual installation from source**
|
||||
|
||||
Semi-automatic, with more details about the steps involved and actions taken [here](#install-netdata-on-linux-manually)
|
||||
|
||||
- **Non-Linux installation**
|
||||
- [Install from package or source, on FreeBSD](#freebsd)
|
||||
- [Install from package, on pfSense](#pfsense)
|
||||
- [Enable netdata on FreeNAS Corral](#freenas)
|
||||
- [Install from package or source, on macOS (OS X)](#macos)
|
||||
|
||||
See also the list of netdata [package maintainers](https://github.com/netdata/netdata/blob/master/MAINTAINERS.md) for ASUSTOR NAS, OpenWRT, ReadyNAS, etc.
|
||||
|
||||
## Install netdata on Linux manually
|
||||
|
||||
To install the latest git version of netdata, please follow these 2 steps:
|
||||
|
||||
1. [Prepare your system](#prepare-your-system)
|
||||
|
||||
Install the required packages on your system.
|
||||
|
||||
2. [Install netdata](#install-netdata)
|
||||
|
||||
Download and install netdata. You can also update it the same way.
|
||||
|
||||
---
|
||||
|
||||
### Prepare your system
|
||||
|
||||
Try our experimental automatic requirements installer (no need to be root). This will try to find the packages that should be installed on your system to build and run netdata. It supports most major Linux distributions released after 2010:
|
||||
|
||||
- **Alpine** Linux and its derivatives (you have to install `bash` yourself, before using the installer)
|
||||
- **Arch** Linux and its derivatives
|
||||
- **Gentoo** Linux and its derivatives
|
||||
- **Debian** Linux and its derivatives (including **Ubuntu**, **Mint**)
|
||||
- **Fedora** and its derivatives (including **Red Hat Enterprise Linux**, **CentOS**, **Amazon Machine Image**)
|
||||
- **SuSe** Linux and its derivatives (including **openSuSe**)
|
||||
- **SLE12** Must have your system registered with Suse Customer Center or have the DVD. See [#1162](https://github.com/netdata/netdata/issues/1162)
|
||||
|
||||
Install the packages for having a **basic netdata installation** (system monitoring and many applications, without `mysql` / `mariadb`, `postgres`, `named`, hardware sensors and `SNMP`):
|
||||
|
||||
```sh
|
||||
curl -Ss 'https://raw.githubusercontent.com/netdata/netdata-demo-site/master/install-required-packages.sh' >/tmp/kickstart.sh && bash /tmp/kickstart.sh -i netdata
|
||||
```
|
||||
|
||||
Install all the required packages for **monitoring everything netdata can monitor**:
|
||||
|
||||
```sh
|
||||
curl -Ss 'https://raw.githubusercontent.com/netdata/netdata-demo-site/master/install-required-packages.sh' >/tmp/kickstart.sh && bash /tmp/kickstart.sh -i netdata-all
|
||||
```
|
||||
|
||||
If the above do not work for you, please [open a github issue](https://github.com/netdata/netdata/issues/new?title=packages%20installer%20failed&labels=installation%20help&body=The%20experimental%20packages%20installer%20failed.%0A%0AThis%20is%20what%20it%20says:%0A%0A%60%60%60txt%0A%0Aplease%20paste%20your%20screen%20here%0A%0A%60%60%60) with a copy of the message you get on screen. We are trying to make it work everywhere (this is also why the script [reports back](https://github.com/netdata/netdata/issues/2054) success or failure for all its runs).
|
||||
|
||||
---
|
||||
|
||||
This is how to do it by hand:
|
||||
|
||||
```sh
|
||||
# Debian / Ubuntu
|
||||
apt-get install zlib1g-dev uuid-dev libmnl-dev gcc make git autoconf autoconf-archive autogen automake pkg-config curl
|
||||
|
||||
# Fedora
|
||||
dnf install zlib-devel libuuid-devel libmnl-devel gcc make git autoconf autoconf-archive autogen automake pkgconfig curl findutils
|
||||
|
||||
# CentOS / Red Hat Enterprise Linux
|
||||
yum install autoconf automake curl gcc git libmnl-devel libuuid-devel lm_sensors make MySQL-python nc pkgconfig python python-psycopg2 PyYAML zlib-devel
|
||||
|
||||
```
|
||||
|
||||
Please note that for RHEL/CentOS you might need [EPEL](http://www.tecmint.com/how-to-enable-epel-repository-for-rhel-centos-6-5/).
|
||||
|
||||
Once netdata is compiled, to run it the following packages are required (already installed using the above commands):
|
||||
|
||||
package|description
|
||||
:-----:|-----------
|
||||
`libuuid`|part of `util-linux` for GUIDs management
|
||||
`zlib`|gzip compression for the internal netdata web server
|
||||
|
||||
*netdata will fail to start without the above.*
|
||||
|
||||
netdata plugins and various aspects of netdata can be enabled or benefit when these are installed (they are optional):
|
||||
|
||||
package|description
|
||||
:-----:|-----------
|
||||
`bash`|for shell plugins and **alarm notifications**
|
||||
`curl`|for shell plugins and **alarm notifications**
|
||||
`iproute` or `iproute2`|for monitoring **Linux traffic QoS**<br/>use `iproute2` if `iproute` reports as not available or obsolete
|
||||
`python`|for most of the external plugins
|
||||
`python-yaml`|used for monitoring **beanstalkd**
|
||||
`python-beanstalkc`|used for monitoring **beanstalkd**
|
||||
`python-dnspython`|used for monitoring DNS query time
|
||||
`python-ipaddress`|used for monitoring **DHCPd**<br/>this package is required only if the system has python v2. python v3 has this functionality embedded
|
||||
`python-mysqldb`<br/>or<br/>`python-pymysql`|used for monitoring **mysql** or **mariadb** databases<br/>`python-mysqldb` is a lot faster and thus preferred
|
||||
`python-psycopg2`|used for monitoring **postgresql** databases
|
||||
`python-pymongo`|used for monitoring **mongodb** databases
|
||||
`nodejs`|used for `node.js` plugins for monitoring **named** and **SNMP** devices
|
||||
`lm-sensors`|for monitoring **hardware sensors**
|
||||
`libmnl`|for collecting netfilter metrics
|
||||
`netcat`|for shell plugins to collect metrics from remote systems
|
||||
|
||||
*netdata will greatly benefit if you have the above packages installed, but it will still work without them.*
|
||||
|
||||
---
|
||||
|
||||
### Install netdata
|
||||
|
||||
Do this to install and run netdata:
|
||||
|
||||
```sh
|
||||
|
||||
# download it - the directory 'netdata' will be created
|
||||
git clone https://github.com/netdata/netdata.git --depth=1
|
||||
cd netdata
|
||||
|
||||
# run script with root privileges to build, install, start netdata
|
||||
./netdata-installer.sh
|
||||
|
||||
```
|
||||
|
||||
* If you don't want to run it straight-away, add `--dont-start-it` option.
|
||||
|
||||
* If you don't want to install it on the default directories, you can run the installer like this: `./netdata-installer.sh --install /opt`. This one will install netdata in `/opt/netdata`.
|
||||
|
||||
Once the installer completes, the file `/etc/netdata/netdata.conf` will be created (if you changed the installation directory, the configuration will appear in that directory too).
|
||||
|
||||
You can edit this file to set options. One common option to tweak is `history`, which controls the size of the memory database netdata will use. By default is `3600` seconds (an hour of data at the charts) which makes netdata use about 10-15MB of RAM (depending on the number of charts detected on your system). Check **[[Memory Requirements]]**.
|
||||
|
||||
To apply the changes you made, you have to restart netdata.
|
||||
|
||||
---
|
||||
|
||||
## Other Systems
|
||||
|
||||
|
||||
|
||||
##### FreeBSD
|
||||
|
||||
You can install netdata from ports or packages collection.
|
||||
|
||||
This is how to install the latest netdata version from sources on FreeBSD:
|
||||
|
||||
```sh
|
||||
# install required packages
|
||||
pkg install bash e2fsprogs-libuuid git curl autoconf automake pkgconf pidof
|
||||
|
||||
# download netdata
|
||||
git clone https://github.com/netdata/netdata.git --depth=1
|
||||
|
||||
# install netdata in /opt/netdata
|
||||
cd netdata
|
||||
./netdata-installer.sh --install /opt
|
||||
```
|
||||
|
||||
##### pfSense
|
||||
To install netdata on pfSense run the following commands (within a shell or under Diagnostics/Command Prompt within the pfSense web interface).
|
||||
|
||||
Change platform (i386/amd64, etc) and FreeBSD versions (10/11, etc) according to your environment and change netdata version (1.10.0 in example) according to latest version present within the FreeSBD repository:-
|
||||
|
||||
Note first three packages are downloaded from the pfSense repository for maintaining compatibility with pfSense, netdata is downloaded from the FreeBSD repository.
|
||||
```
|
||||
pkg install pkgconf
|
||||
pkg install bash
|
||||
pkg install e2fsprogs-libuuid
|
||||
pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/netdata-1.10.0.txz
|
||||
```
|
||||
To start netdata manually run `service netdata onestart`
|
||||
|
||||
To start netdata automatically at each boot add `service netdata start` as a Shellcmd within the pfSense web interface (under **Services/Shellcmd**, which you need to install beforehand under **System/Package Manager/Available Packages**).
|
||||
Shellcmd Type should be set to `Shellcmd`.
|
||||
|
||||
Alternatively more information can be found in https://doc.pfsense.org/index.php/Installing_FreeBSD_Packages, for achieving the same via the command line and scripts.
|
||||
|
||||
If you experience an issue with `/usr/bin/install` absense on pfSense 2.3 or earlier, update pfSense or use workaround from [https://redmine.pfsense.org/issues/6643](https://redmine.pfsense.org/issues/6643)
|
||||
|
||||
##### FreeNAS
|
||||
On FreeNAS-Corral-RELEASE (>=10.0.3), netdata is pre-installed.
|
||||
|
||||
To use netdata, the service will need to be enabled and started from the FreeNAS **[CLI](https://github.com/freenas/cli)**.
|
||||
|
||||
To enable the netdata service:
|
||||
```
|
||||
service netdata config set enable=true
|
||||
```
|
||||
|
||||
To start the netdata service:
|
||||
```
|
||||
service netdata start
|
||||
```
|
||||
|
||||
##### macOS
|
||||
|
||||
netdata on macOS still has limited charts, but external plugins do work.
|
||||
|
||||
You can either install netdata with [Homebrew](https://brew.sh/)
|
||||
|
||||
```sh
|
||||
brew install netdata
|
||||
```
|
||||
|
||||
or from source:
|
||||
|
||||
```sh
|
||||
# install Xcode Command Line Tools
|
||||
xcode-select --install
|
||||
```
|
||||
click `Install` in the software update popup window, then
|
||||
```sh
|
||||
# install HomeBrew package manager
|
||||
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
|
||||
|
||||
# install required packages
|
||||
brew install ossp-uuid autoconf automake pkg-config
|
||||
|
||||
# download netdata
|
||||
git clone https://github.com/netdata/netdata.git --depth=1
|
||||
|
||||
# install netdata in /usr/local/netdata
|
||||
cd netdata
|
||||
sudo ./netdata-installer.sh --install /usr/local
|
||||
```
|
||||
|
||||
The installer will also install a startup plist to start netdata when your Mac boots.
|
||||
|
||||
##### Alpine 3.x
|
||||
|
||||
Execute these commands to install netdata in Alpine Linux 3.x:
|
||||
|
||||
```
|
||||
# install required packages
|
||||
apk add alpine-sdk bash curl zlib-dev util-linux-dev libmnl-dev gcc make git autoconf automake pkgconfig python logrotate
|
||||
|
||||
# if you plan to run node.js netdata plugins
|
||||
apk add nodejs
|
||||
|
||||
# download netdata - the directory 'netdata' will be created
|
||||
git clone https://github.com/netdata/netdata.git --depth=1
|
||||
cd netdata
|
||||
|
||||
|
||||
# build it, install it, start it
|
||||
./netdata-installer.sh
|
||||
|
||||
|
||||
# make netdata start at boot
|
||||
echo -e "#!/usr/bin/env bash\n/usr/sbin/netdata" >/etc/local.d/netdata.start
|
||||
chmod 755 /etc/local.d/netdata.start
|
||||
|
||||
# make netdata stop at shutdown
|
||||
echo -e "#!/usr/bin/env bash\nkillall netdata" >/etc/local.d/netdata.stop
|
||||
chmod 755 /etc/local.d/netdata.stop
|
||||
|
||||
# enable the local service to start automatically
|
||||
rc-update add local
|
||||
```
|
||||
|
||||
##### Synology
|
||||
|
||||
The documentation previously recommended installing the Debian Chroot package from the Synology community package sources and then running netdata from within the chroot. This does not work, as the chroot environment does not have access to `/proc`, and therefore exposes very few metrics to netdata. Additionally, [this issue](https://github.com/SynoCommunity/spksrc/issues/2758), still open as of 2018/06/24, indicates that the Debian Chroot package is not suitable for DSM versions greater than version 5 and may corrupt system libraries and render the NAS unable to boot.
|
||||
|
||||
The good news is that the 64-bit static installer works fine if your NAS is one that uses the amd64 architecture. It will install the content into `/opt/netdata`, making future removal safe and simple.
|
||||
|
||||
###### Additional Work
|
||||
|
||||
When netdata is first installed, it will run as _root_. This may or may not be acceptable for you, and since other installations run it as the _netdata_ user, you might wish to do the same. This requires some extra work:
|
||||
|
||||
1. Creat a group `netdata` via the Synology group interface. Give it no access to anything.
|
||||
2. Create a user `netdata` via the Synology user interface. Give it no access to anything and a random password. Assign the user to the `netdata` group. Netdata will chuid to this user when running.
|
||||
3. Change ownership of the following directories, as defined in [Netdata Security](https://github.com/netdata/netdata/wiki/netdata-security):
|
||||
|
||||
```
|
||||
$ chown -R root:netdata /opt/netdata/usr/share/netdata
|
||||
$ chown -R netdata:netdata /opt/netdata/var/lib/netdata /opt/netdata/var/cache/netdata
|
||||
$ chown -R netdata:root /opt/netdata/var/log/netdata
|
||||
```
|
||||
|
||||
Additionally, as of 2018/06/24, the netdata installer doesn't recognize DSM as an operating system, so no init script is installed. You'll have to do this manually:
|
||||
|
||||
1. Add [this file](https://gist.github.com/oskapt/055d474d7bfef32c49469c1b53e8225f) as `/etc/rc.netdata`. Make it executable with `chmod 0755 /etc/rc.netdata`.
|
||||
2. Edit `/etc/rc.local` and add a line calling `/etc/rc.netdata` to have it start on boot:
|
||||
|
||||
```
|
||||
# Netdata startup
|
||||
[ -x /etc/rc.netdata ] && /etc/rc.netdata start
|
||||
```
|
||||
36
installer/UNINSTALL.md
Normal file
36
installer/UNINSTALL.md
Normal file
|
|
@ -0,0 +1,36 @@
|
|||
# Uninstalling netdata
|
||||
|
||||
## netdata was installed from source (or `kickstart.sh`)
|
||||
|
||||
The script `netdata-installer.sh` generates another script called `netdata-uninstaller.sh`.
|
||||
|
||||
To uninstall netdata, run:
|
||||
|
||||
```
|
||||
cd /path/to/netdata.git
|
||||
./netdata-uninstaller.sh --force
|
||||
```
|
||||
|
||||
The uninstaller will ask you to confirm all deletions.
|
||||
|
||||
## netdata was installed with `kickstart-static64.sh` package
|
||||
|
||||
Stop netdata with one of the following:
|
||||
|
||||
- `service netdata stop` (non-systemd systems)
|
||||
- `systemctl stop netdata` (systemd systems)
|
||||
|
||||
Disable running netdata at startup, with one of the following (based on your distro):
|
||||
|
||||
- `rc-update del netdata`
|
||||
- `update-rc.d netdata disable`
|
||||
- `chkconfig netdata off`
|
||||
- `systemctl disable netdata`
|
||||
|
||||
Delete the netdata files:
|
||||
|
||||
1. `rm -rf /opt/netdata`
|
||||
2. `groupdel netdata`
|
||||
3. `userdel netdata`
|
||||
4. `rm /etc/logrotate.d/netdata`
|
||||
5. `rm /etc/systemd/system/netdata.service` or `rm /etc/init.d/netdata`, depending on the distro.
|
||||
71
installer/UPDATE.md
Normal file
71
installer/UPDATE.md
Normal file
|
|
@ -0,0 +1,71 @@
|
|||
# Updating netdata after its installation
|
||||
|
||||
|
||||
|
||||
|
||||
We suggest to keep your netdata updated. We are actively developing it and you should always update to the latest version.
|
||||
|
||||
The update procedure depends on how you installed it:
|
||||
|
||||
## You downloaded it from github using git
|
||||
|
||||
### Manual update
|
||||
|
||||
The installer `netdata-installer.sh` generates a `netdata-updater.sh` script in the directory you downloaded netdata.
|
||||
You can use this script to update your netdata installation with the same options you used to install it in the first place.
|
||||
Just run it and it will download and install the latest version of netdata. The same script can be put in a cronjob to update your netdata at regular intervals.
|
||||
|
||||
```sh
|
||||
# go to the git downloaded directory
|
||||
cd /path/to/git/downloaded/netdata
|
||||
|
||||
# run the updater
|
||||
./netdata-updater.sh
|
||||
```
|
||||
|
||||
_Netdata will be restarted with the new version._
|
||||
|
||||
If you don't have this script (e.g. you deleted the directory where you downloaded netdata), just follow the **[[Installation]]** instructions again. The installer preserves your configuration. You can also update netdata to the latest version by hand, using this:
|
||||
|
||||
```sh
|
||||
# go to the git downloaded directory
|
||||
cd /path/to/git/downloaded/netdata
|
||||
|
||||
# download the latest version
|
||||
git pull
|
||||
|
||||
# rebuild it, install it, run it
|
||||
./netdata-installer.sh
|
||||
```
|
||||
|
||||
_Netdata will be restarted with the new version._
|
||||
|
||||
Keep in mind, netdata may now have new features, or certain old features may now behave differently. So pay some attention to it after updating.
|
||||
|
||||
### Auto-update
|
||||
|
||||
_Please, consider the risks of running an auto-update. Something can always go wrong. Keep an eye on your installation, and run a manual update if something ever fails._
|
||||
|
||||
You can call `netdata-updater.sh` from a cron-job. A successful update will not trigger an email from cron.
|
||||
|
||||
```sh
|
||||
# Edit your cron-jobs
|
||||
crontab -e
|
||||
|
||||
# add a cron-job at the bottom. This one will update netdata every day at 6:00AM:
|
||||
# update netdata
|
||||
0 6 * * * /path/to/git/downloaded/netdata/netdata-updater.sh
|
||||
```
|
||||
|
||||
## You downloaded a binary package
|
||||
|
||||
If you installed it from a binary package, the best way is to **obtain a newer copy** from the source you got it in the first place.
|
||||
|
||||
If a newer version of netdata is not available from the source you got it, we suggest to uninstall the version you have and follow the **[[Installation]]** instructions for installing a fresh version of netdata.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -85,7 +85,7 @@ Compared to unoptimized code (test No 1: 4.6sec):
|
|||
- before ARL netdata was using test No **7** with hashing and a custom `str2ull()` to achieve 602ms.
|
||||
- the current ARL implementation is test No **9** that needs only 157ms (29 times faster vs unoptimized code, about 4 times faster vs optimized code).
|
||||
|
||||
[Check the source code of this test](../../tests/profile/benchmark-value-pairs.c).
|
||||
[Check the source code of this test](https://github.com/netdata/netdata/tree/master/tests/profile/benchmark-value-pairs.c).
|
||||
|
||||
## Limitations
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,6 @@
|
|||
Packaging Tools
|
||||
===============
|
||||
# Packaging Tools
|
||||
|
||||
The programs in this folder are used when packaging from within git
|
||||
The programs in folder `packaging` are used when packaging from within git
|
||||
and are not included in source or binary packages.
|
||||
|
||||
For the most part they are used from the git commit hooks (copy
|
||||
|
|
|
|||
73
packaging/maintainers/README.md
Normal file
73
packaging/maintainers/README.md
Normal file
|
|
@ -0,0 +1,73 @@
|
|||
# Package Maintainers
|
||||
|
||||
This page tracks the package maintainers for netdata, for various operating systems and versions.
|
||||
|
||||
> Feel free to update it, so that it reflects the current status.
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Official Linux Distributions
|
||||
|
||||
| Linux Distribution | Netdata Version | Maintainer | Related URL |
|
||||
| :-: | :-: | :-: | :-- |
|
||||
| Arch Linux | Release | @svenstaro | [netdata @ Arch Linux](https://www.archlinux.org/packages/community/x86_64/netdata/) |
|
||||
| Arch Linux AUR | Git | @sanskritfritz | [netdata @ AUR](https://aur.archlinux.org/packages/netdata-git/) |
|
||||
| Gentoo Linux | Release + Git | @candrews | [netdata @ gentoo](https://github.com/gentoo/gentoo/tree/master/net-analyzer/netdata) |
|
||||
| Debian | Release | @lhw @FedericoCeratto | [netdata @ debian](http://salsa.debian.org/debian/netdata) |
|
||||
| Slackware | Release | @willysr | [netdata @ slackbuilds](https://slackbuilds.org/repository/14.2/system/netdata/)
|
||||
| Ubuntu | | | |
|
||||
| Red Hat / Fedora / Centos | | | |
|
||||
| SuSe / openSuSe | | | |
|
||||
|
||||
---
|
||||
## FreeBSD
|
||||
|
||||
| System | Initial PR | Core Developer | Package Maintainer
|
||||
|:-:|:-:|:-:|:-:|
|
||||
FreeBSD|#1321|@vlvkobal|@mmokhi
|
||||
|
||||
---
|
||||
## MacOS
|
||||
|
||||
| System | URL | Core Developer | Package Maintainer
|
||||
|:-:|:-:|:-:|:-:|
|
||||
MacOS Homebrew Formula|[link](https://github.com/Homebrew/homebrew-core/blob/master/Formula/netdata.rb)|@vlvkobal|@rickard-von-essen
|
||||
|
||||
---
|
||||
## Unofficial Linux Packages
|
||||
|
||||
| Linux Distribution | Netdata Version | Maintainer | Related URL |
|
||||
| :-: | :-: | :-: | :-- |
|
||||
| Ubuntu | Release | @gslin | [netdata @ gslin ppa](https://launchpad.net/~gslin/+archive/ubuntu/netdata) https://github.com/netdata/netdata/issues/69#issuecomment-217458543 |
|
||||
|
||||
---
|
||||
## Embedded Linux
|
||||
|
||||
| Embedded Linux | Netdata Version | Maintainer | Related URL |
|
||||
| :-: | :-: | :-: | :-- |
|
||||
| ASUSTOR NAS | ? | William Lin | https://www.asustor.com/apps/app_detail?id=532 |
|
||||
| OpenWRT | Release | @nitroshift | [openwrt package](https://github.com/openwrt/packages/tree/master/admin/netdata) |
|
||||
| ReadyNAS | Release | @NAStools | https://github.com/nastools/netdata |
|
||||
| QNAP | Release | QNAP_Stephane | https://forum.qnap.com/viewtopic.php?t=121518 |
|
||||
| DietPi | Release | @Fourdee | https://github.com/Fourdee/DietPi |
|
||||
|
||||
---
|
||||
## Linux Containers
|
||||
|
||||
| Containers | Netdata Version | Maintainer | Related URL |
|
||||
| :-: | :-: | :-: | :-- |
|
||||
| Docker | Git | @titpetric | https://github.com/titpetric/netdata |
|
||||
|
||||
---
|
||||
## Automation Systems
|
||||
|
||||
| Automation Systems | Netdata Version | Maintainer | Related URL |
|
||||
| :-: | :-: | :-: | :-- |
|
||||
| Ansible | git | @jffz | https://galaxy.ansible.com/jffz/netdata/ |
|
||||
| Chef | ? | @sergiopena | https://github.com/sergiopena/netdata-cookbook |
|
||||
|
||||
---
|
||||
## Packages summary from repology.org
|
||||
|
||||
[](https://repology.org/metapackage/netdata/versions)
|
||||
3
requirements.txt
Normal file
3
requirements.txt
Normal file
|
|
@ -0,0 +1,3 @@
|
|||
mkdocs>=1.0.1
|
||||
mkdocs-material
|
||||
|
||||
1
runtime.txt
Normal file
1
runtime.txt
Normal file
|
|
@ -0,0 +1 @@
|
|||
3.6
|
||||
|
|
@ -11,9 +11,9 @@ a netdata performs:
|
|||
- run health checks that trigger alarms and send alarm notifications
|
||||
- archive metrics to a backend time-series database
|
||||
|
||||
The following configurations are supported:
|
||||
## Supported configurations
|
||||
|
||||
#### Netdata without a database or web API (headless collector)
|
||||
### netdata without a database or web API (headless collector)
|
||||
|
||||
Local netdata (`slave`), **without any database or alarms**, collects metrics and sends them to
|
||||
another netdata (`master`).
|
||||
|
|
@ -28,7 +28,7 @@ of maintaining a local database and accepting dashboard requests, it streams all
|
|||
|
||||
The same `master` can collect data for any number of `slaves`.
|
||||
|
||||
#### Database replication
|
||||
### database replication
|
||||
|
||||
Local netdata (`slave`), **with a local database (and possibly alarms)**, collects metrics and
|
||||
sends them to another netdata (`master`).
|
||||
|
|
@ -41,7 +41,7 @@ The `slave` and the `master` may have different data retention policies for the
|
|||
Alarms for the `slave` are triggered by **both** the `slave` and the `master` (and actually
|
||||
each can have different alarms configurations or have alarms disabled).
|
||||
|
||||
#### netdata proxies
|
||||
### netdata proxies
|
||||
|
||||
Local netdata (`slave`), with or without a database, collects metrics and sends them to another
|
||||
netdata (`proxy`), which may or may not maintain a database, which forwards them to another
|
||||
|
|
@ -52,7 +52,7 @@ Alarms for the slave can be triggered by any of the involved hosts that maintain
|
|||
Any number of daisy chaining netdata servers are supported, each with or without a database and
|
||||
with or without alarms for the `slave` metrics.
|
||||
|
||||
#### mix and match with backends
|
||||
### mix and match with backends
|
||||
|
||||
All nodes that maintain a database can also send their data to a backend database.
|
||||
This allows quite complex setups.
|
||||
|
|
@ -67,7 +67,7 @@ Example:
|
|||
6. alarms are triggered by `H` for all hosts
|
||||
7. users can use all the netdata that maintain a database to view metrics (i.e. at `H` all hosts can be viewed).
|
||||
|
||||
#### netdata.conf configuration
|
||||
## Configuration
|
||||
|
||||
These are options that affect the operation of netdata in this area:
|
||||
|
||||
|
|
@ -98,9 +98,9 @@ This also disables the registry (there cannot be a registry without an API).
|
|||
`[backend]` configures data archiving to a backend (it archives all databases maintained on
|
||||
this host).
|
||||
|
||||
#### streaming configuration
|
||||
### streaming configuration
|
||||
|
||||
A new file is introduced: [stream.conf](stream.conf) (to edit it on your system run
|
||||
A new file is introduced: [stream.conf](https://github.com/netdata/netdata/tree/master/streaming/stream.conf) (to edit it on your system run
|
||||
`/etc/netdata/edit-config stream.conf`). This file holds streaming configuration for both the
|
||||
sending and the receiving netdata.
|
||||
|
||||
|
|
@ -167,7 +167,7 @@ the unique id the netdata generating the metrics (i.e. the netdata that original
|
|||
them `/var/lib/netdata/registry/netdata.unique.id`). So, metrics for netdata `A` that pass through
|
||||
any number of other netdata, will have the same `MACHINE_GUID`.
|
||||
|
||||
####### allow from
|
||||
##### allow from
|
||||
|
||||
`allow from` settings are [netdata simple patterns](../libnetdata/simple_pattern): string matches
|
||||
that use `*` as wildcard (any number of times) and a `!` prefix for a negative match.
|
||||
|
|
@ -176,7 +176,7 @@ important: left to right, the first positive or negative match is used.
|
|||
|
||||
`allow from` is available in netdata v1.9+
|
||||
|
||||
#### tracing
|
||||
##### tracing
|
||||
|
||||
When a `slave` is trying to push metrics to a `master` or `proxy`, it logs entries like these:
|
||||
|
||||
|
|
@ -203,7 +203,7 @@ The receiving end (`proxy` or `master`) logs entries like these:
|
|||
For netdata v1.9+, streaming can also be monitored via `access.log`.
|
||||
|
||||
|
||||
#### Viewing remote host dashboards, using mirrored databases
|
||||
## Viewing remote host dashboards, using mirrored databases
|
||||
|
||||
On any receiving netdata, that maintains remote databases and has its web server enabled,
|
||||
`my-netdata` menu will include a list of the mirrored databases.
|
||||
|
|
@ -240,7 +240,7 @@ Following the netdata way of monitoring, we wanted:
|
|||
3. **zero configuration**, all ephemeral servers should have exactly the same configuration, and nothing should be configured at any system for each of the ephemeral nodes. We shouldn't care if 10 or 100 servers are spawned to handle the load.
|
||||
4. **self-cleanup**, so that nothing needs to be done for cleaning up the monitoring infrastructure from the hundreds of nodes that may have been monitored through time.
|
||||
|
||||
#### How it works
|
||||
### How it works
|
||||
|
||||
All monitoring solutions, including netdata, work like this:
|
||||
|
||||
|
|
@ -306,10 +306,10 @@ On each of the slaves, edit `/etc/netdata/stream.conf` (to edit it on your syste
|
|||
[stream]
|
||||
# stream metrics to another netdata
|
||||
enabled = yes
|
||||
|
||||
|
||||
# the IP and PORT of the master
|
||||
destination = 10.11.12.13:19999
|
||||
|
||||
|
||||
# the API key to use
|
||||
api key = 11111111-2222-3333-4444-555555555555
|
||||
```
|
||||
|
|
@ -340,6 +340,7 @@ The file `/var/lib/netdata/registry/netdata.public.unique.id` contains a random
|
|||
|
||||
Both the sender and the receiver of metrics log information at `/var/log/netdata/error.log`.
|
||||
|
||||
|
||||
On both master and slave do this:
|
||||
|
||||
```
|
||||
|
|
@ -393,6 +394,7 @@ This means a setup like the following is also possible:
|
|||
<img src="https://cloud.githubusercontent.com/assets/2662304/23629551/bb1fd9c2-02c0-11e7-90f5-cab5a3ed4c53.png"/>
|
||||
</p>
|
||||
|
||||
|
||||
## proxies
|
||||
|
||||
A proxy is a netdata that is receiving metrics from a netdata, and streams them to another netdata.
|
||||
|
|
@ -402,7 +404,7 @@ When they maintain a database, they can also run health checks (alarms and notif
|
|||
for the remote host that is streaming the metrics.
|
||||
|
||||
To configure a proxy, configure it as a receiving and a sending netdata at the same time,
|
||||
using [stream.conf](stream.conf).
|
||||
using [stream.conf](https://github.com/netdata/netdata/tree/master/streaming/stream.conf).
|
||||
|
||||
The sending side of a netdata proxy, connects and disconnects to the final destination of the
|
||||
metrics, following the same pattern of the receiving side.
|
||||
|
|
|
|||
|
|
@ -1,8 +1,9 @@
|
|||
# Testing
|
||||
This readme is a manual on how to get started with unit testing on javascript and nodejs
|
||||
|
||||
Original author: BrainDoctor (github), July 2017
|
||||
|
||||
# Installation
|
||||
## Installation
|
||||
|
||||
Tested on Linux Mint 18.2 Sara (Ubuntu/debian derivative)
|
||||
|
||||
|
|
@ -19,7 +20,7 @@ That should install the necessary node modules.
|
|||
|
||||
Other browsers work too (Chrome, Firefox). However, only the Chromium Browser 59 has been tested for headless unit testing.
|
||||
|
||||
## Versions
|
||||
### Versions
|
||||
|
||||
The commands above leave me with the following versions (July 2017):
|
||||
|
||||
|
|
@ -28,21 +29,21 @@ The commands above leave me with the following versions (July 2017):
|
|||
- chromium-browser: 59.0.3071.109
|
||||
- WebStorm (optional): 2017.1.4
|
||||
|
||||
# Configuration
|
||||
## Configuration
|
||||
|
||||
## NPM
|
||||
### NPM
|
||||
|
||||
The dependencies are installed in `netdata/package.json`. If you install a new NPM module, it gets added here. Future developers just need to execute `npm install` and every dep gets added automatically.
|
||||
|
||||
## Karma
|
||||
### Karma
|
||||
|
||||
Karma configuration is in `tests/web/karma.conf.js`. Documentation is provided via comments.
|
||||
|
||||
## WebStorm
|
||||
### WebStorm
|
||||
|
||||
If you use the JetBrains WebStorm IDE, you can integrate the karma runtime.
|
||||
|
||||
### for Karma (Client side testing)
|
||||
#### for Karma (Client side testing)
|
||||
|
||||
Headless Chromium:
|
||||
1. Run > Edit Configurations
|
||||
|
|
@ -66,7 +67,7 @@ You may add other browsers too (comma separated). With the "Browsers to start" f
|
|||
|
||||
Also it is recommended to install WebStorm IDE Extension/Addon to Chrome/Chromium for awesome debugging.
|
||||
|
||||
### for node.d plugins (nodejs)
|
||||
#### for node.d plugins (nodejs)
|
||||
|
||||
1. Run > Edit Configurations
|
||||
2. "+" > Node.js
|
||||
|
|
@ -75,23 +76,23 @@ Also it is recommended to install WebStorm IDE Extension/Addon to Chrome/Chromiu
|
|||
- JavaScript file: node_modules/jasmine-node/bin/jasmine-node
|
||||
- Application parameters: --captureExceptions tests/node.d
|
||||
|
||||
# Running
|
||||
## Running
|
||||
|
||||
## In WebStorm
|
||||
### In WebStorm
|
||||
|
||||
### Karma
|
||||
#### Karma
|
||||
Just run the configured run configurations and they produce nice test trees:
|
||||
|
||||
|
||||
|
||||
### node.js
|
||||
#### node.js
|
||||
|
||||
Debugging is awesome too!
|
||||
|
||||
|
||||
## From CLI
|
||||
### From CLI
|
||||
|
||||
### Karma
|
||||
#### Karma
|
||||
|
||||
```sh
|
||||
cd /path/to/your/netdata
|
||||
|
|
@ -103,7 +104,7 @@ will start the karma server, start chromium in headless mode and exit.
|
|||
If a test fails, it produces even a stack trace:
|
||||
|
||||
|
||||
### Node.d plugins
|
||||
#### Node.d plugins
|
||||
|
||||
```sh
|
||||
cd /path/to/your/netdata
|
||||
|
|
@ -114,9 +115,9 @@ nodejs node_modules/jasmine-node/bin/jasmine-node --captureExceptions tests/node
|
|||
will run the tests in `tests/node.d` and produce a stacktrace too on error:
|
||||
|
||||
|
||||
## Coverage
|
||||
### Coverage
|
||||
|
||||
### Karma
|
||||
#### Karma
|
||||
|
||||
A nice HTML is produced from Karma which shows which code paths were executed. It is located somewhere in `/path/to/your/netdata/coverage/`
|
||||
|
||||
|
|
@ -124,11 +125,11 @@ A nice HTML is produced from Karma which shows which code paths were executed. I
|
|||
and
|
||||
|
||||
|
||||
### Node.d
|
||||
#### Node.d
|
||||
|
||||
Apparently, jasmine-node can produce a junit report with the `--junitreport` flag. But that output was not very useful. Maybe it's configurable?
|
||||
|
||||
## CI
|
||||
### CI
|
||||
|
||||
The karma and node.d runners can be integrated in Travis (AFAIK), but that is outside my ability.
|
||||
|
||||
|
|
|
|||
|
|
@ -11,4 +11,6 @@ SUBDIRS = \
|
|||
|
||||
dist_noinst_DATA = \
|
||||
README.md \
|
||||
gui/confluence/README.md \
|
||||
gui/custom/README.md \
|
||||
$(NULL)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,26 @@
|
|||
# Web Dashboards Overview
|
||||
|
||||
The default port is 19999; for example, to access the dashboard on localhost, use: http://localhost:19999
|
||||
|
||||
To view netdata collected data you access its **[REST API v1](api/)**.
|
||||
|
||||
For our convenience, netdata provides 2 more layers:
|
||||
|
||||
1. The `dashboard.js` javascript library that allows us to design custom dashboards using plain HTML. For information on creating custom dashboards, see **[Custom Dashboards](gui/custom/)** and **[Atlassian Confluence Dashboards](gui/confluence/)**
|
||||
|
||||
2. Ready to be used web dashboards that render all the charts a netdata server maintains.
|
||||
|
||||
## customizing the standard dashboards
|
||||
|
||||
Charts information is stored at /usr/share/netdata/web/[dashboard_info.js](https://github.com/netdata/netdata/blob/master/web/dashboard_info.js). This file includes information that is rendered on the dashboard, controls chart colors, section and subsection heading, titles, etc.
|
||||
|
||||
If you change that file, your changes will be overwritten when netdata is updated. You can preserve your settings by creating a new such file (there is /usr/share/netdata/web/[dashboard_info_custom.example.js](https://github.com/netdata/netdata/blob/master/web/dashboard_info_custom_example.js) you can use to start with).
|
||||
|
||||
You have to copy the example file under a new name, so that it will not be overwritten with netdata updates.
|
||||
|
||||
To configure your info file set in netdata.conf:
|
||||
|
||||
```
|
||||
[web]
|
||||
custom dashboard_info.js = your_file_name.js
|
||||
```
|
||||
|
|
@ -56,14 +56,14 @@ Here is what you can put for `options` (these are standard netdata API options):
|
|||
|
||||
```html
|
||||
<a href="#">
|
||||
<img src="http://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu"></img>
|
||||
<img src="https://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu"></img>
|
||||
</a>
|
||||
```
|
||||
|
||||
Which produces this:
|
||||
|
||||
<a href="#">
|
||||
<img src="http://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu"></img>
|
||||
<img src="https://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu"></img>
|
||||
</a>
|
||||
|
||||
- `alarm=NAME`
|
||||
|
|
@ -84,14 +84,14 @@ Here is what you can put for `options` (these are standard netdata API options):
|
|||
|
||||
```html
|
||||
<a href="#">
|
||||
<img src="http://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&dimensions=system%7Cnice"></img>
|
||||
<img src="https://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&dimensions=system%7Cnice"></img>
|
||||
</a>
|
||||
```
|
||||
|
||||
Which produces this:
|
||||
|
||||
<a href="#">
|
||||
<img src="http://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&dimensions=system%7Cnice"></img>
|
||||
<img src="https://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&dimensions=system%7Cnice"></img>
|
||||
</a>
|
||||
|
||||
- `before=SECONDS` and `after=SECONDS`
|
||||
|
|
@ -106,28 +106,28 @@ Here is what you can put for `options` (these are standard netdata API options):
|
|||
|
||||
```html
|
||||
<a href="#">
|
||||
<img src="http://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&after=-60"></img>
|
||||
<img src="https://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&after=-60"></img>
|
||||
</a>
|
||||
```
|
||||
|
||||
Which produces the average of last complete minute (XX:XX:00 - XX:XX:59):
|
||||
|
||||
<a href="#">
|
||||
<img src="http://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&after=-60"></img>
|
||||
<img src="https://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&after=-60"></img>
|
||||
</a>
|
||||
|
||||
While this is the previous minute (one minute before the last one, again aligned XX:XX:00 - XX:XX:59):
|
||||
|
||||
```html
|
||||
<a href="#">
|
||||
<img src="http://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&before=-60&after=-60"></img>
|
||||
<img src="https://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&before=-60&after=-60"></img>
|
||||
</a>
|
||||
```
|
||||
|
||||
It produces this:
|
||||
|
||||
<a href="#">
|
||||
<img src="http://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&before=-60&after=-60"></img>
|
||||
<img src="https://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&before=-60&after=-60"></img>
|
||||
</a>
|
||||
|
||||
- `group=min` or `group=max` or `group=average` (the default) or `group=sum` or `group=incremental-sum`
|
||||
|
|
@ -208,11 +208,11 @@ These are options dedicated to badges:
|
|||
|
||||
This option scales the svg image. It accepts values above or equal to 100 (100% is the default scale). For example, lets get a few different sizes:
|
||||
|
||||
<img src="http://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&after=-60&scale=100"></img> original<br/>
|
||||
<img src="http://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&after=-60&scale=125"></img> `scale=125`<br/>
|
||||
<img src="http://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&after=-60&scale=150"></img> `scale=150`<br/>
|
||||
<img src="http://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&after=-60&scale=175"></img> `scale=175`<br/>
|
||||
<img src="http://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&after=-60&scale=200"></img> `scale=200`
|
||||
<img src="https://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&after=-60&scale=100"></img> original<br/>
|
||||
<img src="https://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&after=-60&scale=125"></img> `scale=125`<br/>
|
||||
<img src="https://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&after=-60&scale=150"></img> `scale=150`<br/>
|
||||
<img src="https://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&after=-60&scale=175"></img> `scale=175`<br/>
|
||||
<img src="https://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&after=-60&scale=200"></img> `scale=200`
|
||||
|
||||
|
||||
- `refresh=auto` or `refresh=SECONDS`
|
||||
|
|
@ -321,4 +321,4 @@ You can refresh them from your browser console though. Press F12 to open the web
|
|||
|
||||
```js
|
||||
var len = document.images.length; while(len--) { document.images[len].src = document.images[len].src.replace(/\?cacheBuster=\d*/, "") + "?cacheBuster=" + new Date().getTime().toString(); };
|
||||
```
|
||||
```
|
||||
|
|
|
|||
|
|
@ -0,0 +1,3 @@
|
|||
# Exporters
|
||||
|
||||
TBD
|
||||
43
web/api/health/README.md
Normal file
43
web/api/health/README.md
Normal file
|
|
@ -0,0 +1,43 @@
|
|||
# Health API Calls
|
||||
|
||||
## Enabled Alarms
|
||||
|
||||
NetData enables alarms on demand, i.e. when the chart they should be linked to starts collecting data. So, although many more alarms are configured, only the useful ones are enabled.
|
||||
|
||||
To get the list of all enabled alarms:
|
||||
|
||||
`http://your.netdata.ip:19999/api/v1/alarms?all`
|
||||
|
||||
## Raised Alarms
|
||||
|
||||
This API call will return the alarms currently in WARNING or CRITICAL state.
|
||||
|
||||
`http://your.netdata.ip:19999/api/v1/alarms`
|
||||
|
||||
## Event Log
|
||||
|
||||
The size of the alarm log is configured in `netdata.conf`. There are 2 settings: the rotation of the alarm log file and the in memory size of the alarm log.
|
||||
|
||||
```
|
||||
[health]
|
||||
health db file = /var/lib/netdata/health/health-log.db
|
||||
in memory max health log entries = 1000
|
||||
rotate log every lines = 2000
|
||||
```
|
||||
|
||||
The API call retrieves all entries of the alarm log:
|
||||
|
||||
`http://your.netdata.ip:19999/api/v1/alarm_log`
|
||||
|
||||
## Alarm Log Incremental Updates
|
||||
|
||||
`http://your.netdata.ip:19999/api/v1/alarm_log?after=UNIQUEID`
|
||||
|
||||
The above returns all the events in the alarm log that occurred after UNIQUEID (you poll it once without `after=`, remember the last UNIQUEID of the returned set, which you give back to get incrementally the next events).
|
||||
|
||||
## Alarm badges
|
||||
|
||||
The following will return an SVG badge of the alarm named `NAME`, attached to the chart named `CHART`.
|
||||
|
||||
`http://your.netdata.ip:19999/api/v1/badge.svg?alarm=NAME&chart=CHART`
|
||||
|
||||
|
|
@ -10,3 +10,19 @@ make
|
|||
```
|
||||
|
||||
After every change in the `src` directory, the `dashboard.js` file should be regenerated and commited to the repository.
|
||||
|
||||
## Custom Dashboards
|
||||
|
||||
For information on creating custom dashboards, see **[Custom Dashboards](custom/)** and **[Atlassian Confluence Dashboards](confluence/)**
|
||||
|
||||
## Supported chart libraries
|
||||
|
||||
- Dygraph
|
||||
- jQuery Sparkline
|
||||
- Peity
|
||||
- Google Charts
|
||||
- Morris
|
||||
- EasyPieChart
|
||||
- Gauge.js
|
||||
- D3
|
||||
- C3
|
||||
|
|
|
|||
1012
web/gui/confluence/README.md
Normal file
1012
web/gui/confluence/README.md
Normal file
File diff suppressed because it is too large
Load diff
412
web/gui/custom/README.md
Normal file
412
web/gui/custom/README.md
Normal file
|
|
@ -0,0 +1,412 @@
|
|||
# Custom Dashboards
|
||||
|
||||
You can:
|
||||
|
||||
- create your own dashboards using simple HTML (no javascript is required for basic dashboards)
|
||||
- utilizing any or all of the available chart libraries, on the same dashboard
|
||||
- using data from one or more netdata servers, on the same dashboard
|
||||
- host your dashboard HTML page on any web server, anywhere
|
||||
|
||||
netdata charts can also be added to existing web pages.
|
||||
|
||||
Check this **[very simple working example of a custom dashboard](http://netdata.firehol.org/demo.html)**, and its **[html source](https://github.com/netdata/netdata/blob/master/web/demo.html)**.
|
||||
|
||||
If you plan to put it on TV, check **[tv.html](https://github.com/netdata/netdata/blob/master/web/tv.html)**. This is a screenshot of it, monitoring 2 servers on the same page:
|
||||
|
||||
|
||||
--
|
||||
|
||||
## Web directory
|
||||
|
||||
The default web root directory is `/usr/share/netdata/web` where you will find examples such as tv.html, and demo.html as well as the main dashboard contained in index.html.
|
||||
Note: index.html have a different syntax. Don't use it as a template for simple custom dashboards.
|
||||
|
||||
## Example empty dashboard
|
||||
|
||||
If you need to create a new dashboard on an empty page, we suggest the following header:
|
||||
|
||||
```html
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<title>Your dashboard</title>
|
||||
|
||||
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
|
||||
<meta charset="utf-8">
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<meta name="apple-mobile-web-app-capable" content="yes">
|
||||
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent">
|
||||
|
||||
<!-- here we will add dashboard.js -->
|
||||
|
||||
</head>
|
||||
<body>
|
||||
|
||||
<!-- here we will add charts -->
|
||||
|
||||
</body>
|
||||
</html>
|
||||
```
|
||||
|
||||
|
||||
## dashboard.js
|
||||
|
||||
To add netdata charts to any web page (dedicated to netdata or not), you need to include the `/dashboard.js` file of a netdata server.
|
||||
|
||||
For example, if your netdata server listens at `http://box:19999/`, you will need to add the following to the `head` section of your web page:
|
||||
|
||||
```html
|
||||
<script type="text/javascript" src="http://box:19999/dashboard.js"></script>
|
||||
```
|
||||
|
||||
### what dashboard.js does?
|
||||
|
||||
`dashboard.js` will automatically load the following:
|
||||
|
||||
1. `dashboard.css`, required for the netdata charts
|
||||
|
||||
2. `jquery.min.js`, (only if jquery is not already loaded for this web page)
|
||||
|
||||
3. `bootstrap.min.js` (only if bootstrap is not already loaded) and `bootstrap.min.css`.
|
||||
|
||||
You can disable this by adding the following before loading `dashboard.js`:
|
||||
|
||||
```html
|
||||
<script>var netdataNoBootstrap = true;</script>
|
||||
```
|
||||
|
||||
4. `jquery.nanoscroller.min.js`, required for the scrollbar of the chart legends.
|
||||
|
||||
5. `bootstrap-toggle.min.js` and `bootstrap-toggle.min.css`, required for the settings toggle buttons.
|
||||
|
||||
6. `font-awesome.min.css`, for icons.
|
||||
|
||||
When `dashboard.js` loads will scan the page for elements that define charts (see below) and immediately start refreshing them. Keep in mind more javascript modules may be loaded (every chart library is a different javascript file, that is loaded on first use).
|
||||
|
||||
### Prevent dashboard.js from starting chart refreshes
|
||||
|
||||
If your web page is not static and you plan to add charts using javascript, you can tell `dashboard.js` not to start processing charts immediately after loaded, by adding this fragment before loading it:
|
||||
|
||||
```html
|
||||
<script>var netdataDontStart = true;</script>
|
||||
```
|
||||
|
||||
The above, will inform the `dashboard.js` to load everything, but not process the web page until you tell it to.
|
||||
You can tell it to start processing the page, by running this javascript code:
|
||||
|
||||
```js
|
||||
NETDATA.start();
|
||||
```
|
||||
|
||||
Be careful not to call the `NETDATA.start()` multiple times. Each call to this function will spawn a new thread that will start refreshing the charts.
|
||||
|
||||
If, after calling `NETDATA.start()` you need to update the page (or even get your javascript code synchronized with `dashboard.js`), you can call (after you loaded `dashboard.js`):
|
||||
|
||||
```js
|
||||
NETDATA.pause(function() {
|
||||
// ok, it is paused
|
||||
|
||||
// update the DOM as you wish
|
||||
|
||||
// and then call this to let the charts refresh:
|
||||
NETDATA.unpause();
|
||||
});
|
||||
```
|
||||
|
||||
### The default netdata server
|
||||
|
||||
`dashboard.js` will attempt to auto-detect the URL of the netdata server it is loaded from, and set this server as the default netdata server for all charts.
|
||||
|
||||
If you need to set any other URL as the default netdata server for all charts that do not specify a netdata server, add this before loading `dashboard.js`:
|
||||
|
||||
```html
|
||||
<script type="text/javascript">var netdataServer = "http://your.netdata.server:19999";</script>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# Adding charts
|
||||
|
||||
To add charts, you need to add a `div` for each of them. Each of these `div` elements accept a few `data-` attributes:
|
||||
|
||||
### The chart unique ID
|
||||
|
||||
The unique ID of a chart is shown at the title of the chart of the default netdata dashboard. You can also find all the charts available at your netdata server with this URL: `http://your.netdata.server:19999/api/v1/charts` ([example](http://netdata.firehol.org/api/v1/charts)).
|
||||
|
||||
To specify the unique id, use this:
|
||||
|
||||
```html
|
||||
<div data-netdata="unique.id"></div>
|
||||
```
|
||||
|
||||
The above is enough for adding a chart. It most probably have the wrong visual settings though. Keep reading...
|
||||
|
||||
### The duration of the chart
|
||||
|
||||
You can specify the duration of the chart (how much time of data it will show) using:
|
||||
|
||||
```html
|
||||
<div data-netdata="unique.id"
|
||||
data-after="AFTER_SECONDS"
|
||||
data-before="BEFORE_SECONDS"
|
||||
></div>
|
||||
```
|
||||
|
||||
`AFTER_SECONDS` and `BEFORE_SECONDS` are numbers representing a time-frame in seconds.
|
||||
|
||||
The can be either:
|
||||
|
||||
- **absolute** unix timestamps (in javascript terms, they are `new Date().getTime() / 1000`. Using absolute timestamps you can have a chart showing always the same time-frame.
|
||||
|
||||
- **relative** number of seconds to now. To show the last 10 minutes of data, `AFTER_SECONDS` must be `-600` (relative to now) and `BEFORE_SECONDS` must be `0` (meaning: now). If you want the chart to auto-refresh the current values, you need to specify **relative** values.
|
||||
|
||||
### Chart sizes
|
||||
|
||||
You can set the size of the chart using this:
|
||||
|
||||
```html
|
||||
<div data-netdata="unique.id"
|
||||
data-width="WIDTH"
|
||||
data-height="HEIGHT"
|
||||
></div>
|
||||
```
|
||||
|
||||
`WIDTH` and `HEIGHT` can be anything CSS accepts for width and height (e.g. percentages, pixels, etc).
|
||||
Keep in mind that for certain chart libraries, `dashboard.js` may apply an aspect ratio to these.
|
||||
|
||||
If you want `dashboard.js` to remember permanently (browser local storage) the dimensions of the chart (the user may resize it), you can add: `data-id="SETTINGS_ID"`, where `SETTINGS_ID` is anything that will be common for this chart across user sessions.
|
||||
|
||||
### Netdata server
|
||||
|
||||
Each chart can get data from a different netdata server. You can give per chart the netdata server using:
|
||||
|
||||
```html
|
||||
<div data-netdata="unique.id"
|
||||
data-host="http://another.netdata.server:19999/"
|
||||
></div>
|
||||
```
|
||||
|
||||
If you have ephemeral monitoring setup ([More info here](https://github.com/netdata/netdata/wiki/Monitoring-ephemeral-nodes)) and have no direct access to the nodes dashboards, you can use the following:
|
||||
|
||||
```html
|
||||
<div data-netdata="unique.id"
|
||||
data-host="http://yournetdata.server:19999/host/reported-hostname"
|
||||
></div>
|
||||
```
|
||||
### Chart library
|
||||
|
||||
The default chart library is `dygraph`. You set a different chart library per chart using this:
|
||||
|
||||
```html
|
||||
<div data-netdata="unique.id"
|
||||
data-chart-library="gauge"
|
||||
></div>
|
||||
```
|
||||
|
||||
Each chart library may support more chart-library specific settings. Please refer to the documentation of the chart library you are interested, in this wiki or the source code:
|
||||
|
||||
- options `data-dygraph-XXX` [here](https://github.com/netdata/netdata/blob/643cfe20a8d8beba0ed31ec6afaade80853fd310/web/dashboard.js#L6251-L6361)
|
||||
- options `data-easypiechart-XXX` [here](https://github.com/netdata/netdata/blob/643cfe20a8d8beba0ed31ec6afaade80853fd310/web/dashboard.js#L7954-L7966)
|
||||
- options `data-gauge-XXX` [here](https://github.com/netdata/netdata/blob/643cfe20a8d8beba0ed31ec6afaade80853fd310/web/dashboard.js#L8182-L8189)
|
||||
- options `data-d3pie-XXX` [here](https://github.com/netdata/netdata/blob/643cfe20a8d8beba0ed31ec6afaade80853fd310/web/dashboard.js#L7394-L7561)
|
||||
- options `data-sparkline-XXX` [here](https://github.com/netdata/netdata/blob/643cfe20a8d8beba0ed31ec6afaade80853fd310/web/dashboard.js#L5940-L5985)
|
||||
- options `data-peity-XXX` [here](https://github.com/netdata/netdata/blob/643cfe20a8d8beba0ed31ec6afaade80853fd310/web/dashboard.js#L5892)
|
||||
|
||||
|
||||
### Data points
|
||||
|
||||
For the time-frame requested, `dashboard.js` will use the chart dimensions and the settings of the chart library to find out how many data points it can show.
|
||||
|
||||
For example, most line chart libraries are using 3 pixels per data point. If the chart shows 10 minutes of data (600 seconds), its update frequency is 1 second, and the chart width is 1800 pixels, then `dashboard.js` will request from the netdata server: 10 minutes of data, represented in 600 points, and the chart will be refreshed per second. If the user resizes the window so that the chart becomes 600 pixels wide, then `dashboard.js` will request the same 10 minutes of data, represented in 200 points and the chart will be refreshed once every 3 seconds.
|
||||
|
||||
If you need to have a fixed number of points in the data source retrieved from the netdata server, you can set:
|
||||
|
||||
```html
|
||||
<div data-netdata="unique.id"
|
||||
data-points="DATA_POINTS"
|
||||
></div>
|
||||
```
|
||||
|
||||
Where `DATA_POINTS` is the number of points you need.
|
||||
|
||||
You can also overwrite the pixels-per-point per chart using this:
|
||||
|
||||
```html
|
||||
<div data-netdata="unique.id"
|
||||
data-pixels-per-point="PIXELS_PER_POINT"
|
||||
></div>
|
||||
```
|
||||
|
||||
Where `PIXELS_PER_POINT` is the number of pixels each data point should occupy.
|
||||
|
||||
### Data grouping method
|
||||
|
||||
Netdata supports **average** (the default), **sum** and **max** grouping methods. The grouping method is used when the netdata server is requested to return fewer points for a time-frame, compared to the number of points available.
|
||||
|
||||
You can give it per chart, using:
|
||||
|
||||
```html
|
||||
<div data-netdata="unique.id"
|
||||
data-method="max"
|
||||
></div>
|
||||
```
|
||||
|
||||
### Changing rates
|
||||
|
||||
Netdata can change the rate of charts on the fly. So a charts that shows values **per second** can be turned to **per minute** (or any other, e.g. **per 10 seconds**), with this:
|
||||
|
||||
```html
|
||||
<div data-netdata="unique.id"
|
||||
data-method="average"
|
||||
data-gtime="60"
|
||||
data-units="per minute"
|
||||
></div>
|
||||
```
|
||||
|
||||
The above will provide the average rate per minute (60 seconds).
|
||||
Use 60 for `/minute`, 3600 for `/hour`, 86400 for `/day` (provided you have that many data).
|
||||
|
||||
- The `data-gtime` setting does not change the units of the chart. You have to change them yourself with `data-units`.
|
||||
- This works only for `data-method="average"`.
|
||||
- netdata may aggregate multiple points to satisfy the `data-points` setting. For example, you request `per minute` but the requested number of points to be returned are not enough to report every single minute. In this case netdata will sum the `per second` raw data of the database to find the `per minute` for every single minute and then **average** them to find the **average per minute rate of every X minutes**. So, it works as if the data collection frequency was per minute.
|
||||
|
||||
### Selecting dimensions
|
||||
|
||||
By default, `dashboard.js` will show all the dimensions of the chart.
|
||||
You can select specific dimensions using this:
|
||||
|
||||
```html
|
||||
<div data-netdata="unique.id"
|
||||
data-dimensions="dimension1,dimension2,dimension3,..."
|
||||
></div>
|
||||
```
|
||||
|
||||
netdata supports coma (` , `) or pipe (` | `) separated [simple patterns](https://github.com/netdata/netdata/wiki/Configuration#netdata-simple-patterns) for dimensions. By default it searches for both dimension IDs and dimension NAMEs. You can control the target of the match with: `data-append-options="match-ids"` or `data-append-options="match-names"`. Spaces in `data-dimensions=""` are matched in the dimension names and IDs.
|
||||
|
||||
### Chart title
|
||||
|
||||
You can overwrite the title of the chart using this:
|
||||
|
||||
```html
|
||||
<div data-netdata="unique.id"
|
||||
data-title="my super chart"
|
||||
></div>
|
||||
```
|
||||
|
||||
### Chart units
|
||||
|
||||
You can overwrite the units of measurement of the dimensions of the chart, using this:
|
||||
|
||||
```html
|
||||
<div data-netdata="unique.id"
|
||||
data-units="words/second"
|
||||
></div>
|
||||
```
|
||||
|
||||
### Chart colors
|
||||
|
||||
`dashboard.js` has an internal palette of colors for the dimensions of the charts.
|
||||
You can prepend colors to it (so that your will be used first) using this:
|
||||
|
||||
```html
|
||||
<div data-netdata="unique.id"
|
||||
data-colors="#AABBCC #DDEEFF ..."
|
||||
></div>
|
||||
```
|
||||
|
||||
### Extracting dimension values
|
||||
|
||||
`dashboard.js` can update the selected values of the chart at elements you specify. For example, let's assume we have a chart that measures the bandwidth of eth0, with 2 dimensions `in` and `out`. You can use this:
|
||||
|
||||
```html
|
||||
<div data-netdata="net.eth0"
|
||||
data-show-value-of-in-at="eth0_in_value"
|
||||
data-show-value-of-out-at="eth0_out_value"
|
||||
></div>
|
||||
|
||||
My eth0 interface, is receiving <span id="eth0_in_value"></span>
|
||||
and transmitting <span id="eth0_out_value"></span>.
|
||||
```
|
||||
|
||||
### Hiding the legend of a chart
|
||||
|
||||
On charts that by default have a legend managed by `dashboard.js` you can remove it, using this:
|
||||
|
||||
```html
|
||||
<div data-netdata="unique.id"
|
||||
data-legend="no"
|
||||
></div>
|
||||
```
|
||||
|
||||
### API options
|
||||
|
||||
You can append netdata **[[REST API v1]]** data options, using this:
|
||||
|
||||
```html
|
||||
<div data-netdata="unique.id"
|
||||
data-append-options="absolute,percentage"
|
||||
></div>
|
||||
```
|
||||
|
||||
A few useful options are:
|
||||
|
||||
- `absolute` to show all values are absolute (i.e. turn negative dimensions to positive)
|
||||
- `percentage` to express the values as a percentage of the chart total (so, the values of the dimensions are added, and the sum of them if expressed as a percentage of the sum of all dimensions)
|
||||
- `unaligned` to prevent netdata from aligning the charts (e.g. when requesting 60 seconds aggregation per point, netdata returns chart data aligned to XX:XX:00 to XX:XX:59 - similarly for hours, days, etc - the `unaligned` option disables this feature)
|
||||
- `match-ids` or `match-names` is used to control what `data-dimensions=` will match.
|
||||
|
||||
### Chart library performance
|
||||
|
||||
`dashboard.js` measures the performance of the chart library when it renders the charts. You can specify an element ID you want this information to be visualized, using this:
|
||||
|
||||
```html
|
||||
<div data-netdata="unique.id"
|
||||
data-dt-element-name="measurement1"
|
||||
></div>
|
||||
|
||||
refreshed in <span id="measurement1"></span> milliseconds!
|
||||
```
|
||||
|
||||
### Syncing charts y-range
|
||||
|
||||
If you give the same `data-common-max="NAME"` to 2+ charts, then all of them will share the same max value of their y-range. If one spikes, all of them will be aligned to have the same scale. This is done for the cpu interrupts and and cpu softnet charts at the dashboard and also for the `gauge` and `easypiecharts` of the netdata home page.
|
||||
|
||||
```html
|
||||
<div data-netdata="chart1"
|
||||
data-common-max="chart-group-1"
|
||||
></div>
|
||||
|
||||
<div data-netdata="chart2"
|
||||
data-common-max="chart-group-1"
|
||||
></div>
|
||||
```
|
||||
|
||||
The same functionality exists for `data-common-min`.
|
||||
|
||||
### Syncing chart units
|
||||
|
||||
netdata dashboards support auto-scaling of units. So, `MB` can become `KB`, `GB`, etc dynamically, based on the value to be shown.
|
||||
|
||||
Giving the same `NAME` with `data-common-units="NAME"`, 2+ charts can be forced to always have the same units.
|
||||
|
||||
```html
|
||||
<div data-netdata="chart1"
|
||||
data-common-units="chart-group-1"
|
||||
></div>
|
||||
|
||||
<div data-netdata="chart2"
|
||||
data-common-units="chart-group-1"
|
||||
></div>
|
||||
```
|
||||
|
||||
### Setting desired units
|
||||
|
||||
Charts can be scaled to specific units with `data-desired-units="UNITS"`. If the dashboard can convert the units to the desired one, it will do.
|
||||
|
||||
```html
|
||||
<div data-netdata="chart1"
|
||||
data-desired-units="GB"
|
||||
></div>
|
||||
```
|
||||
|
||||
|
|
@ -86,8 +86,8 @@ Netdata supports access lists in `netdata.conf`:
|
|||
- `allow badges from` checks if the API request is for a badge. Badges are not matched by `allow dashboard from`.
|
||||
|
||||
- `allow streaming from` checks if the slave willing to stream metrics to this netdata is allowed.
|
||||
This can be controlled per API KEY and MACHINE GUID in [stream.conf](../../streaming/stream.conf).
|
||||
The setting in `netdata.conf` is checked before the ones in [stream.conf](../../streaming/stream.conf).
|
||||
This can be controlled per API KEY and MACHINE GUID in [stream.conf](https://github.com/netdata/netdata/tree/master/streaming/stream.conf).
|
||||
The setting in `netdata.conf` is checked before the ones in [stream.conf](https://github.com/netdata/netdata/tree/master/streaming/stream.conf).
|
||||
|
||||
- `allow netdata.conf from` checks the IP to allow `http://netdata.host:19999/netdata.conf`.
|
||||
By default it allows only private lans.
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue