mirror of https://github.com/netdata/netdata.git synced 2025-04-22 20:42:33 +00:00

History

Costa Tsaousis 5e1b95cf92 Deduplicate all netdata strings (#13570 ) * rrdfamily * rrddim * rrdset plugin and module names * rrdset units * rrdset type * rrdset family * rrdset title * rrdset title more * rrdset context * rrdcalctemplate context and removal of context hash from rrdset * strings statistics * rrdset name * rearranged members of rrdset * eliminate rrdset name hash; rrdcalc chart converted to STRING * rrdset id, eliminated rrdset hash * rrdcalc, alarm_entry, alert_config and some of rrdcalctemplate * rrdcalctemplate * rrdvar * eval_variable * rrddimvar and rrdsetvar * rrdhost hostname, os and tags * fix master commits * added thread cache; implemented string_dup without locks * faster thread cache * rrdset and rrddim now use dictionaries for indexing * rrdhost now uses dictionary * rrdfamily now uses DICTIONARY * rrdvar using dictionary instead of AVL * allocate the right size to rrdvar flag members * rrdhost remaining char * members to STRING * * better error handling on indexing * strings now use a read/write lock to allow parallel searches to the index * removed AVL support from dictionaries; implemented STRING with native Judy calls * string releases should be negative * only 31 bits are allowed for enum flags * proper locking on strings * string threading unittest and fixes * fix lgtm finding * fixed naming * stream chart/dimension definitions at the beginning of a streaming session * thread stack variable is undefined on thread cancel * rrdcontext garbage collect per host on startup * worker control in garbage collection * relaxed deletion of rrdmetrics * type checking on dictfe * netdata chart to monitor rrdcontext triggers * Group chart label updates * rrdcontext better handling of collected rrdsets * rrdpush incremental transmition of definitions should use as much buffer as possible * require 1MB per chart * empty the sender buffer before enabling metrics streaming * fill up to 50% of buffer * reset signaling metrics sending * use the shared variable for status * use separate host flag for enabling streaming of metrics * make sure the flag is clear * add logging for streaming * add logging for streaming on buffer overflow * circular_buffer proper sizing * removed obsolete logs * do not execute worker jobs if not necessary * better messages about compression disabling * proper use of flags and updating rrdset last access time every time the obsoletion flag is flipped * monitor stream sender used buffer ratio * Update exporting unit tests * no need to compare label value with strcmp * streaming send workers now monitor bandwidth * workers now use strings * streaming receiver monitors incoming bandwidth * parser shift of worker ids * minor fixes * Group chart label updates * Populate context with dimensions that have data * Fix chart id * better shift of parser worker ids * fix for streaming compression * properly count received bytes * ensure LZ4 compression ring buffer does not wrap prematurely * do not stream empty charts; do not process empty instances in rrdcontext * need_to_send_chart_definition() does not need an rrdset lock any more * rrdcontext objects are collected, after data have been written to the db * better logging of RRDCONTEXT transitions * always set all variables needed by the worker utilization charts * implemented double linked list for most objects; eliminated alarm indexes from rrdhost; and many more fixes * lockless strings design - string_dup() and string_freez() are totally lockless when they dont need to touch Judy - only Judy is protected with a read/write lock * STRING code re-organization for clarity * thread_cache improvements; double numbers precision on worker threads * STRING_ENTRY now shadown STRING, so no duplicate definition is required; string_length() renamed to string_strlen() to follow the paradigm of all other functions, STRING internal statistics are now only compiled with NETDATA_INTERNAL_CHECKS * rrdhost index by hostname now cleans up; aclk queries of archieved hosts do not index hosts * Add index to speed up database context searches * Removed last_updated optimization (was also buggy after latest merge with master) Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com> Co-authored-by: Vladimir Kobal <vlad@prokk.net>		2022-09-05 19:31:06 +03:00
..
json.c	Deduplicate all netdata strings (#13570 )	2022-09-05 19:31:06 +03:00
json.h	Add data query support for archived charts (#10771 )	2021-03-22 09:47:22 +02:00
Makefile.am	Makefile.am files indentation (#7252 )	2019-11-11 01:30:00 +02:00
README.md	Docs: Removed Google Analytics tags (#12145 )	2022-02-17 10:37:46 +00:00

README.md

JSON formatter

The CSV formatter presents results of database queries in the following formats:

format	content type	description
`json`	application/json	return the query result as a json object
`jsonp`	application/json	return the query result as a JSONP javascript callback
`datatable`	application/json	return the query result as a Google `datatable`
`datasource`	application/json	return the query result as a Google Visualization Provider `datasource` javascript callback

The CSV formatter respects the following API &options=:

option	supported	description
`google_json`	yes	enable the Google flavor of JSON (using double quotes for strings and `Date()` function for dates
`objectrows`	yes	return each row as an object, instead of an array
`nonzero`	yes	to return only the dimensions that have at least a non-zero value
`flip`	yes	to return the rows older to newer (the default is newer to older)
`seconds`	yes	to return the date and time in unix timestamp
`ms`	yes	to return the date and time in unit timestamp as milliseconds
`percent`	yes	to replace all values with their percentage over the row total
`abs`	yes	to turn all values positive
`null2zero`	yes	to replace gaps with zeros (the default prints the string `null`

Examples

To show the differences between each format, in the following examples we query the same chart (having just one dimension called active), changing only the query format and its options.

Using format=json and options=

# curl -Ss 'https://registry.my-netdata.io/api/v1/data?chart=nginx_local.connections&after=-3600&points=6&group=average&format=json&options='
{
 "labels": ["time", "active"],
    "data":
 [
      [ 1540644600, 224.2516667],
      [ 1540644000, 229.29],
      [ 1540643400, 222.41],
      [ 1540642800, 226.6816667],
      [ 1540642200, 246.4083333],
      [ 1540641600, 241.0966667]
  ]
}

Using format=json and options=objectrows

# curl -Ss 'https://registry.my-netdata.io/api/v1/data?chart=nginx_local.connections&after=-3600&points=6&group=average&format=json&options=objectrows'
{
 "labels": ["time", "active"],
    "data":
 [
      { "time": 1540644600, "active": 224.2516667},
      { "time": 1540644000, "active": 229.29},
      { "time": 1540643400, "active": 222.41},
      { "time": 1540642800, "active": 226.6816667},
      { "time": 1540642200, "active": 246.4083333},
      { "time": 1540641600, "active": 241.0966667}
  ]
}

Using format=json and options=objectrows,google_json

# curl -Ss 'https://registry.my-netdata.io/api/v1/data?chart=nginx_local.connections&after=-3600&points=6&group=average&formatjson&options=objectrows,google_json'
{
 "labels": ["time", "active"],
    "data":
 [
      { "time": new Date(2018,9,27,12,50,0), "active": 224.2516667},
      { "time": new Date(2018,9,27,12,40,0), "active": 229.29},
      { "time": new Date(2018,9,27,12,30,0), "active": 222.41},
      { "time": new Date(2018,9,27,12,20,0), "active": 226.6816667},
      { "time": new Date(2018,9,27,12,10,0), "active": 246.4083333},
      { "time": new Date(2018,9,27,12,0,0), "active": 241.0966667}
  ]
}

Using format=jsonp and options=

curl -Ss 'https://registry.my-netdata.io/api/v1/data?chart=nginx_local.connections&after=-3600&points=6&group=average&formjsonp&options='
callback({
 "labels": ["time", "active"],
    "data":
 [
      [ 1540645200, 235.885],
      [ 1540644600, 224.2516667],
      [ 1540644000, 229.29],
      [ 1540643400, 222.41],
      [ 1540642800, 226.6816667],
      [ 1540642200, 246.4083333]
  ]
});

Using format=datatable and options=

curl -Ss 'https://registry.my-netdata.io/api/v1/data?chart=nginx_local.connections&after=-3600&points=6&group=average&formdatatable&options='
{
 "cols":
 [
        {"id":"","label":"time","pattern":"","type":"datetime"},
        {"id":"","label":"","pattern":"","type":"string","p":{"role":"annotation"}},
        {"id":"","label":"","pattern":"","type":"string","p":{"role":"annotationText"}},
     {"id":"","label":"active","pattern":"","type":"number"}
  ],
    "rows":
 [
        {"c":[{"v":"Date(2018,9,27,13,0,0)"},{"v":null},{"v":null},{"v":235.885}]},
        {"c":[{"v":"Date(2018,9,27,12,50,0)"},{"v":null},{"v":null},{"v":224.2516667}]},
        {"c":[{"v":"Date(2018,9,27,12,40,0)"},{"v":null},{"v":null},{"v":229.29}]},
        {"c":[{"v":"Date(2018,9,27,12,30,0)"},{"v":null},{"v":null},{"v":222.41}]},
        {"c":[{"v":"Date(2018,9,27,12,20,0)"},{"v":null},{"v":null},{"v":226.6816667}]},
        {"c":[{"v":"Date(2018,9,27,12,10,0)"},{"v":null},{"v":null},{"v":246.4083333}]}
  ]
}

Using format=datasource and options=

curl -Ss 'https://registry.my-netdata.io/api/v1/data?chart=nginx_local.connections&after=-3600&points=6&group=average&format=datasource&options='
google.visualization.Query.setResponse({version:'0.6',reqId:'0',status:'ok',sig:'1540645368',table:{
 "cols":
 [
        {"id":"","label":"time","pattern":"","type":"datetime"},
        {"id":"","label":"","pattern":"","type":"string","p":{"role":"annotation"}},
        {"id":"","label":"","pattern":"","type":"string","p":{"role":"annotationText"}},
     {"id":"","label":"active","pattern":"","type":"number"}
  ],
    "rows":
 [
        {"c":[{"v":"Date(2018,9,27,13,0,0)"},{"v":null},{"v":null},{"v":235.885}]},
        {"c":[{"v":"Date(2018,9,27,12,50,0)"},{"v":null},{"v":null},{"v":224.2516667}]},
        {"c":[{"v":"Date(2018,9,27,12,40,0)"},{"v":null},{"v":null},{"v":229.29}]},
        {"c":[{"v":"Date(2018,9,27,12,30,0)"},{"v":null},{"v":null},{"v":222.41}]},
        {"c":[{"v":"Date(2018,9,27,12,20,0)"},{"v":null},{"v":null},{"v":226.6816667}]},
        {"c":[{"v":"Date(2018,9,27,12,10,0)"},{"v":null},{"v":null},{"v":246.4083333}]}
  ]
}});