![]() * rrdset - in progress * rrdset optimal constructor; rrdset conflict * rrdset final touches * re-organization of rrdset object members * prevent use-after-free * dictionary dfe supports also counting of iterations * rrddim managed by dictionary * rrd.h cleanup * DICTIONARY_ITEM now is referencing actual dictionary items in the code * removed rrdset linked list * Revert "removed rrdset linked list" This reverts commit 690d6a588b4b99619c2c5e10f84e8f868ae6def5. * removed rrdset linked list * added comments * Switch chart uuid to static allocation in rrdset Remove unused functions * rrdset_archive() and friends... * always create rrdfamily * enable ml_free_dimension * rrddim_foreach done with dfe * most custom rrddim loops replaced with rrddim_foreach * removed accesses to rrddim->dimensions * removed locks that are no longer needed * rrdsetvar is now managed by the dictionary * set rrdset is rrdsetvar, fixes https://github.com/netdata/netdata/pull/13646#issuecomment-1242574853 * conflict callback of rrdsetvar now properly checks if it has to reset the variable * dictionary registered callbacks accept as first parameter the DICTIONARY_ITEM * dictionary dfe now uses internal counter to report; avoided excess variables defined with dfe * dictionary walkthrough callbacks get dictionary acquired items * dictionary reference counters that can be dupped from zero * added advanced functions for get and del * rrdvar managed by dictionaries * thread safety for rrdsetvar * faster rrdvar initialization * rrdvar string lengths should match in all add, del, get functions * rrdvar internals hidden from the rest of the world * rrdvar is now acquired throughout netdata * hide the internal structures of rrdsetvar * rrdsetvar is now acquired through out netdata * rrddimvar managed by dictionary; rrddimvar linked list removed; rrddimvar structures hidden from the rest of netdata * better error handling * dont create variables if not initialized for health * dont create variables if not initialized for health again * rrdfamily is now managed by dictionaries; references of it are acquired dictionary items * type checking on acquired objects * rrdcalc renaming of functions * type checking for rrdfamily_acquired * rrdcalc managed by dictionaries * rrdcalc double free fix * host rrdvars is always needed * attempt to fix deadlock 1 * attempt to fix deadlock 2 * Remove unused variable * attempt to fix deadlock 3 * snprintfz * rrdcalc index in rrdset fix * Stop storing active charts and computing chart hashes * Remove store active chart function * Remove compute chart hash function * Remove sql_store_chart_hash function * Remove store_active_dimension function * dictionary delayed destruction * formatting and cleanup * zero dictionary base on rrdsetvar * added internal error to log delayed destructions of dictionaries * typo in rrddimvar * added debugging info to dictionary * debug info * fix for rrdcalc keys being empty * remove forgotten unlock * remove deadlock * Switch to metadata version 5 and drop chart_hash chart_hash_map chart_active dimension_active v_chart_hash * SQL cosmetic changes * do not busy wait while destroying a referenced dictionary * remove deadlock * code cleanup; re-organization; * fast cleanup and flushing of dictionaries * number formatting fixes * do not delete configured alerts when archiving a chart * rrddim obsolete linked list management outside dictionaries * removed duplicate contexts call * fix crash when rrdfamily is not initialized * dont keep rrddimvar referenced * properly cleanup rrdvar * removed some locks * Do not attempt to cleanup chart_hash / chart_hash_map * rrdcalctemplate managed by dictionary * register callbacks on the right dictionary * removed some more locks * rrdcalc secondary index replaced with linked-list; rrdcalc labels updates are now executed by health thread * when looking up for an alarm look using both chart id and chart name * host initialization a bit more modular * init rrdlabels on host update * preparation for dictionary views * improved comment * unused variables without internal checks * service threads isolation and worker info * more worker info in service thread * thread cancelability debugging with internal checks * strings data races addressed; fixes https://github.com/netdata/netdata/issues/13647 * dictionary modularization * Remove unused SQL statement definition * unit-tested thread safety of dictionaries; removed data race conditions on dictionaries and strings; dictionaries now can detect if the caller is holds a write lock and automatically all the calls become their unsafe versions; all direct calls to unsafe version is eliminated * remove worker_is_idle() from the exit of service functions, because we lose the lock time between loops * rewritten dictionary to have 2 separate locks, one for indexing and another for traversal * Update collectors/cgroups.plugin/sys_fs_cgroup.c Co-authored-by: Vladimir Kobal <vlad@prokk.net> * Update collectors/cgroups.plugin/sys_fs_cgroup.c Co-authored-by: Vladimir Kobal <vlad@prokk.net> * Update collectors/proc.plugin/proc_net_dev.c Co-authored-by: Vladimir Kobal <vlad@prokk.net> * fix memory leak in rrdset cache_dir * minor dictionary changes * dont use index locks in single threaded * obsolete dict option * rrddim options and flags separation; rrdset_done() optimization to keep array of reference pointers to rrddim; * fix jump on uninitialized value in dictionary; remove double free of cache_dir * addressed codacy findings * removed debugging code * use the private refcount on dictionaries * make dictionary item desctructors work on dictionary destruction; strictier control on dictionary API; proper cleanup sequence on rrddim; * more dictionary statistics * global statistics about dictionary operations, memory, items, callbacks * dictionary support for views - missing the public API * removed warning about unused parameter * chart and context name for cloud * chart and context name for cloud, again * dictionary statistics fixed; first implementation of dictionary views - not currently used * only the master can globally delete an item * context needs netdata prefix * fix context and chart it of spins * fix for host variables when health is not enabled * run garbage collector on item insert too * Fix info message; remove extra "using" * update dict unittest for new placement of garbage collector * we need RRDHOST->rrdvars for maintaining custom host variables * Health initialization needs the host->host_uuid * split STRING to its own files; no code changes other than that * initialize health unconditionally * unit tests do not pollute the global scope with their variables * Skip initialization when creating archived hosts on startup. When a child connects it will initialize properly Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com> Co-authored-by: Vladimir Kobal <vlad@prokk.net> |
||
---|---|---|
.. | ||
Makefile.am | ||
README.md | ||
web_buffer_svg.c | ||
web_buffer_svg.h |
Netdata badges
Badges are cool!
Netdata can generate badges for any chart and any dimension at any time-frame. Badges come in SVG
and can be added to any web page using an <IMG>
HTML tag.
Netdata badges are powerful!
Given that Netdata collects from 1.000 to 5.000 metrics per server (depending on the number of network interfaces, disks, cpu cores, applications running, users logged in, containers running, etc) and that Netdata already has data reduction/aggregation functions embedded, the badges can be quite powerful.
For each metric/dimension and for arbitrary time-frames badges can show min, max or average value, but also sum or incremental-sum to have their volume.
For example, there is a chart in Netdata that shows the current requests/s of nginx. Using this chart alone we can show the following badges (we could add more time-frames, like today, yesterday, etc):
Similarly, there is a chart that shows outbound bandwidth per class, using QoS data. So it shows kilobits/s
per class. Using this chart we can show:
The right one is a volume calculation. Netdata calculated the total of the last 86.400 seconds (a day) which gives kilobits
, then divided it by 8 to make it KB, then by 1024 to make it MB and then by 1024 to make it GB. Calculations like this are quite accurate, since for every value collected, every second, Netdata interpolates it to second boundary using microsecond calculations.
Let's see a few more badge examples (they come from the Netdata registry):
-
cpu usage of user
root
(you can pick any user; 100% = 1 core). This will begreen <10%
,yellow <20%
,orange <50%
,blue <100%
(1 core),red
otherwise (you define thresholds and colors on the URL). -
mysql queries per second
niche ones: mysql SELECT statements with JOIN, which did full table scans:
So, every single line on the charts of a Netdata dashboard, can become a badge and this badge can calculate average, min, max, or volume for any time-frame! And you can also vary the badge color using conditions on the calculated value.
How to create badges
The basic URL is http://your.netdata:19999/api/v1/badge.svg?option1&option2&option3&...
.
Here is what you can put for options
(these are standard Netdata API options):
-
chart=CHART.NAME
The chart to get the values from.
This is the only parameter required and with just this parameter, Netdata will return the sum of the latest values of all chart dimensions.
Example:
<a href="#">
<img src="https://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu"></img>
</a>
Which produces this:
-
alarm=NAME
Render the current value and status of an alarm linked to the chart. This option can be ignored if the badge to be generated is not related to an alarm.
The current value of the alarm will be rendered. The color of the badge will indicate the status of the alarm.
For alarm badges, both
chart
andalarm
parameters are required. -
dimensions=DIMENSION1|DIMENSION2|...
The dimensions of the chart to use. If you don't set any dimension, all will be used. When multiple dimensions are used, Netdata will sum their values. You can append
options=absolute
if you want this sum to convert all values to positive before adding them.Pipes in HTML have to escaped with
%7C
.Example:
<a href="#">
<img src="https://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&dimensions=system%7Cnice"></img>
</a>
Which produces this:
-
before=SECONDS
andafter=SECONDS
The timeframe. These can be absolute unix timestamps, or relative to now, number of seconds. By default
before=0
andafter=-1
(1 second in the past).To get the last minute set
after=-60
. This will give the average of the last complete minute (XX:XX:00 - XX:XX:59).To get the max of the last hour set
after=-3600&group=max
. This will give the maximum value of the last complete hour (XX:00:00 - XX:59:59)Example:
<a href="#">
<img src="https://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&after=-60"></img>
</a>
Which produces the average of last complete minute (XX:XX:00 - XX:XX:59):
While this is the previous minute (one minute before the last one, again aligned XX:XX:00 - XX:XX:59):
<a href="#">
<img src="https://registry.my-netdata.io/api/v1/badge.svg?chart=system.cpu&before=-60&after=-60"></img>
</a>
It produces this:
-
group=min
orgroup=max
orgroup=average
(the default) orgroup=sum
orgroup=incremental-sum
If Netdata will have to reduce (aggregate) the data to calculate the value, which aggregation method to use.
-
max
will find the max value for the timeframe. This works on both positive and negative dimensions. It will find the most extreme value. -
min
will find the min value for the timeframe. This works on both positive and negative dimensions. It will find the number closest to zero. -
average
will calculate the average value for the timeframe. -
sum
will sum all the values for the timeframe. This is nice for finding the volume of dimensions for a timeframe. So if you have a dimension that reportsX per second
, you can find the volume of the dimension in a timeframe, by adding its values in that timeframe. -
incremental-sum
will sum the difference of each value to its next. Let's assume you have a dimension that does not measure the rate of something, but the absolute value of it. So it has values like this "1, 5, 3, 7, 4".incremental-sum
will calculate the difference of adjacent values. In this example, they will be(5 - 1) + (3 - 5) + (7 - 3) + (4 - 7) = 3
(which is equal to the last value minus the first = 4 - 1).
-
-
options=opt1|opt2|opt3|...
These fine tune various options of the API. Here is what you can use for badges (the API has more option, but only these are useful for badges):
-
percentage
, instead of returning the value, calculate the percentage of the sum of the selected dimensions, versus the sum of all the dimensions of the chart. This also sets the units to%
. -
absolute
orabs
, turn all values positive and then sum them. -
display_absolute
ordisplay-absolute
, to use the signed value during color calculation, but display the absolute value on the badge. -
min2max
, when multiple dimensions are given, do not sum them, but take theirmax - min
. -
unaligned
, when data are reduced / aggregated (e.g. the request is about the average of the last minute, or hour), Netdata by default aligns them so that the charts will have a constant shape (so average per minute returns always XX:XX:00 - XX:XX:59). Setting theunaligned
option, Netdata will aggregate data without any alignment, so if the request is for 60 seconds, it will aggregate the latest 60 seconds of collected data.
-
These are options dedicated to badges:
-
label=TEXT
The label of the badge.
-
units=TEXT
The units of the badge. If you want to put a
/
, please put a\
. This is because Netdata allows badges parameters to be given as path in URL, instead of query string. You can also usenull
orempty
to show it without any units.The units
seconds
,minutes
andhours
trigger special formatting. The value has to be in this unit, and Netdata will automatically change it to show a more pretty duration. -
multiply=NUMBER
Multiply the value with this number. The default is
1
. -
divide=NUMBER
Divide the value with this number. The default is
1
. -
Color customization parameters
The following parameters specify colors of each individual part of the badge. Each parameter is documented in detail below.
Area of badge Background color parameter Text color parameter Label (left) part label_color
text_color_lbl
Value (right) part value_color
text_color_val
-
label_color=COLOR
The color of the label (the left part). You can use any HTML color in
RGB
orRRGGBB
hex notation (without the#
character at the beginning). Additionally, you can use one of the following predefined colors (and you can use them by their name):green
brightgreen
yellow
yellowgreen
orange
red
blue
grey
gray
lightgrey
lightgray
These colors are taken from https://github.com/badges/shields, which makes them compatible with standard badges.
-
value_color=COLOR:null|COLOR<VALUE|COLOR>VALUE|COLOR>=VALUE|COLOR<=VALUE|...
You can add a pipe delimited list of conditions to pick the value color. The first matching (left to right) will be used.
Example:
value_color=grey:null|green<10|yellow<100|orange<1000|blue<10000|red
The above will set
grey
if no value exists (not collected within thegap when lost iterations above
innetdata.conf
for the chart),green
if the value is less than 10,yellow
if the value is less than 100, and so on. Netdata will usered
if no other conditions match. Only integers are supported as values.The supported operators are
<
,>
,<=
,>=
,=
(or:
), and!=
(or<>
).You can also use the same syntax as the
label_color
parameter to define each of these colors. You can reference a predefined color by name orRGB
/RRGGBB
hex notation. -
text_color_lbl=RGB
ortext_color_lbl=RRGGBB
ortext_color_lbl=color_by_name
This value specifies the font color for the font of left/label side of the badge. The syntax is the same as the
label_color
parameter. If not given, or given with an empty value, Netdata will use the default color. -
text_color_val=RGB
ortext_color_val=RRGGBB
ortext_color_lbl=color_by_name
This value specifies the font color for the font of right/value side of the badge. The syntax is the same as the
label_color
parameter. If not given, or given with an empty value, Netdata will use the default color.
-
-
precision=NUMBER
The number of decimal digits of the value. By default Netdata will add:
- no decimal digits for values > 1000
- 1 decimal digit for values > 100
- 2 decimal digits for values > 1
- 3 decimal digits for values > 0.1
- 4 decimal digits for values <= 0.1
Using the
precision=NUMBER
you can set your preference per badge. -
scale=XXX
This option scales the svg image. It accepts values above or equal to 100 (100% is the default scale). For example, lets get a few different sizes:
original
scale=125
scale=150
scale=175
scale=200
-
fixed_width_lbl=NUMBER
andfixed_width_val=NUMBER
This parameter overrides auto-sizing of badges and displays them at fixed widths.
fixed_width_lbl
determines the size of the label's left side (label/name).fixed_width_val
determines the size of the the label's right side (value). You must set both parameters together, or they will be ignored.You should set the label/value widths wide enough to provide space for all the possible values/contents of the badge you're requesting. In case the text cannot fit the space given it will be clipped.
The
scale
parameter still applies on the values you give tofixed_width_lbl
andfixed_width_val
. -
refresh=auto
orrefresh=SECONDS
This option enables auto-refreshing of images. Netdata will send the HTTP header
Refresh: SECONDS
to the web browser, thus requesting automatic refresh of the images at regular intervals.auto
will calculate the properSECONDS
to avoid unnecessary refreshes. IfSECONDS
is zero, this feature is disabled (it is also disabled by default).Auto-refreshing like this, works only if you access the badge directly. So, you may have to put it an
embed
oriframe
for it to be auto-refreshed. Use something like this:
<embed src="BADGE_URL" type="image/svg+xml" height="20" />
Another way is to use javascript to auto-refresh them. You can auto-refresh all the Netdata badges on a page using javascript. You have to add a class to all the Netdata badges, like this <img class="netdata-badge" src="..."/>
. Then add this javascript code to your page (it requires jquery):
<script>
var NETDATA_BADGES_AUTOREFRESH_SECONDS = 5;
function refreshNetdataBadges() {
var now = new Date().getTime().toString();
$('.netdata-badge').each(function() {
this.src = this.src.replace(/\&_=\d*/, '') + '&_=' + now;
});
setTimeout(refreshNetdataBadges, NETDATA_BADGES_AUTOREFRESH_SECONDS * 1000);
}
setTimeout(refreshNetdataBadges, NETDATA_BADGES_AUTOREFRESH_SECONDS * 1000);
</script>
A more advanced badges refresh method is to include http://your.netdata.ip:19999/refresh-badges.js
in your page.
Escaping URLs
Keep in mind that if you add badge URLs to your HTML pages you have to escape the special characters:
character | name | escape sequence |
---|---|---|
`` | space (in labels and units) | %20 |
# |
hash (for colors) | %23 |
% |
percent (in units) | %25 |
< |
less than | %3C |
> |
greater than | %3E |
\ |
backslash (when you need a / ) |
%5C |
| |
pipe (delimiting parameters) | %7C |
FAQ
Is it fast?
On modern hardware, Netdata can generate about 2.000 badges per second per core, before noticing any delays. It generates a badge in about half a millisecond!
Of course these timing are for badges that use recent data. If you need badges that do calculations over long durations (a day, or more), timing will differ. Netdata logs its timings at its access.log
, so take a look there before adding a heavy badge on a busy web site. Of course, you can cache such badges or have a cron job get them from Netdata and save them at your web server at regular intervals.
Embedding badges in GitHub
You have 2 options:
- SVG images with markdown
- SVG images with HTML (directly in .md files)
For example, this is the cpu badge shown above:
- Markdown example:
[](https://registry.my-netdata.io/#apps_cpu)
- HTML example:
<a href="https://registry.my-netdata.io/#apps_cpu">
<img src="https://registry.my-netdata.io/api/v1/badge.svg?chart=users.cpu&dimensions=root&value_color=grey:null%7Cgreen%3C10%7Cyellow%3C20%7Corange%3C50%7Cblue%3C100%7Cred&label=root%20user%20cpu%20now&units=%25"></img>
</a>
Both produce this:
Auto-refreshing badges in GitHub
Unfortunately it cannot be done. GitHub fetches all the images using a proxy and rewrites all the URLs to be served by the proxy.
You can refresh them from your browser console though. Press F12 to open the web browser console (switch to the console too), paste the following and press enter. They will refresh:
var len = document.images.length; while(len--) { document.images[len].src = document.images[len].src.replace(/\?cacheBuster=\d*/, "") + "?cacheBuster=" + new Date().getTime().toString(); };