0
0
Fork 0
mirror of https://github.com/netdata/netdata.git synced 2025-05-01 07:59:52 +00:00
netdata_netdata/docs/guides
Stelios Fragkakis 49234f23de
Multi-Tier database backend for long term metrics storage ()
* Tier part 1

* Tier part 2

* Tier part 3

* Tier part 4

* Tier part 5

* Fix some ML compilation errors

* fix more conflicts

* pass proper tier

* move metric_uuid from state to RRDDIM

* move aclk_live_status from state to RRDDIM

* move ml_dimension from state to RRDDIM

* abstracted the data collection interface

* support flushing for mem db too

* abstracted the query api

* abstracted latest/oldest time per metric

* cleanup

* store_metric for tier1

* fix for store_metric

* allow multiple tiers, more than 2

* state to tier

* Change storage type in db. Query param to request min, max, sum or average

* Store tier data correctly

* Fix skipping tier page type

* Add tier grouping in the tier

* Fix to handle archived charts (part 1)

* Temp fix for query granularity when requesting tier1 data

* Fix parameters in the correct order and calculate the anomaly based on the anomaly count

* Proper tiering grouping

* Anomaly calculation based on anomaly count

* force type checking on storage handles

* update cmocka tests

* fully dynamic number of storage tiers

* fix static allocation

* configure grouping for all tiers; disable tiers for unittest; disable statsd configuration for private charts mode

* use default page dt using the tiering info

* automatic selection of tier

* fix for automatic selection of tier

* working prototype of dynamic tier selection

* automatic selection of tier done right (I hope)

* ask for the proper tier value, based on the grouping function

* fixes for unittests and load_metric_next()

* fixes for lgtm findings

* minor renames

* add dbengine to page cache size setting

* add dbengine to page cache with malloc

* query engine optimized to loop as little are required based on the view_update_every

* query engine grouping methods now do not assume a constant number of points per group and they allocate memory with OWA

* report db points per tier in jsonwrap

* query planer that switches database tiers on the fly to satisfy the query for the entire timeframe

* dbegnine statistics and documentation (in progress)

* calculate average point duration in db

* handle single point pages the best we can

* handle single point pages even better

* Keep page type in the rrdeng_page_descr

* updated doc

* handle future backwards compatibility - improved statistics

* support &tier=X in queries

* enfore increasing iterations on tiers

* tier 1 is always 1 iteration

* backfilling higher tiers on first data collection

* reversed anomaly bit

* set up to 5 tiers

* natural points should only be offered on tier 0, except a specific tier is selected

* do not allow more than 65535 points of tier0 to be aggregated on any tier

* Work only on actually activated tiers

* fix query interpolation

* fix query interpolation again

* fix lgtm finding

* Activate one tier for now

* backfilling of higher tiers using raw metrics from lower tiers

* fix for crash on start when storage tiers is increased from the default

* more statistics on exit

* fix bug that prevented higher tiers to get any values; added backfilling options

* fixed the statistics log line

* removed limit of 255 iterations per tier; moved the code of freezing rd->tiers[x]->db_metric_handle

* fixed division by zero on zero points_wanted

* removed dead code

* Decide on the descr->type for the type of metric

* dont store metrics on unknown page types

* free db_metric_handle on sql based context queries

* Disable STORAGE_POINT value check in the exporting engine unit tests

* fix for db modes other than dbengine

* fix for aclk archived chart queries destroying db_metric_handles of valid rrddims

* fix left-over freez() instead of OWA freez on median queries

Co-authored-by: Costa Tsaousis <costa@netdata.cloud>
Co-authored-by: Vladimir Kobal <vlad@prokk.net>
2022-07-06 14:01:53 +03:00
..
configure feat: move dirs, logs, and env vars config options to separate sections () 2022-05-17 17:31:19 +03:00
deploy Docs: Removed Google Analytics tags () 2022-02-17 10:37:46 +00:00
export Docs: Removed Google Analytics tags () 2022-02-17 10:37:46 +00:00
monitor Improve anomaly detection guide () 2022-07-01 17:37:39 +03:00
step-by-step Remove node.d.plugin and relevant files () 2022-05-03 09:16:21 +03:00
troubleshoot Docs: Removed Google Analytics tags () 2022-02-17 10:37:46 +00:00
collect-apache-nginx-web-logs.md chore(python.d): remove python.d/* announced in v1.34.0 deprecation notice () 2022-05-02 18:15:33 +03:00
collect-unbound-metrics.md Docs: Removed Google Analytics tags () 2022-02-17 10:37:46 +00:00
longer-metrics-storage.md Multi-Tier database backend for long term metrics storage () 2022-07-06 14:01:53 +03:00
monitor-cockroachdb.md Docs: Removed Google Analytics tags () 2022-02-17 10:37:46 +00:00
monitor-hadoop-cluster.md Docs: Removed Google Analytics tags () 2022-02-17 10:37:46 +00:00
python-collector.md Docs: Removed Google Analytics tags () 2022-02-17 10:37:46 +00:00
using-host-labels.md Remove backends subsystem () 2022-03-15 11:50:24 +01:00