0
0
Fork 0
mirror of https://github.com/netdata/netdata.git synced 2025-04-09 15:47:53 +00:00
Commit graph

11 commits

Author SHA1 Message Date
Costa Tsaousis
cb7af25c09
RRD structures managed by dictionaries ()
* rrdset - in progress

* rrdset optimal constructor; rrdset conflict

* rrdset final touches

* re-organization of rrdset object members

* prevent use-after-free

* dictionary dfe supports also counting of iterations

* rrddim managed by dictionary

* rrd.h cleanup

* DICTIONARY_ITEM now is referencing actual dictionary items in the code

* removed rrdset linked list

* Revert "removed rrdset linked list"

This reverts commit 690d6a588b4b99619c2c5e10f84e8f868ae6def5.

* removed rrdset linked list

* added comments

* Switch chart uuid to static allocation in rrdset
Remove unused functions

* rrdset_archive() and friends...

* always create rrdfamily

* enable ml_free_dimension

* rrddim_foreach done with dfe

* most custom rrddim loops replaced with rrddim_foreach

* removed accesses to rrddim->dimensions

* removed locks that are no longer needed

* rrdsetvar is now managed by the dictionary

* set rrdset is rrdsetvar, fixes https://github.com/netdata/netdata/pull/13646#issuecomment-1242574853

* conflict callback of rrdsetvar now properly checks if it has to reset the variable

* dictionary registered callbacks accept as first parameter the DICTIONARY_ITEM

* dictionary dfe now uses internal counter to report; avoided excess variables defined with dfe

* dictionary walkthrough callbacks get dictionary acquired items

* dictionary reference counters that can be dupped from zero

* added advanced functions for get and del

* rrdvar managed by dictionaries

* thread safety for rrdsetvar

* faster rrdvar initialization

* rrdvar string lengths should match in all add, del, get functions

* rrdvar internals hidden from the rest of the world

* rrdvar is now acquired throughout netdata

* hide the internal structures of rrdsetvar

* rrdsetvar is now acquired through out netdata

* rrddimvar managed by dictionary; rrddimvar linked list removed; rrddimvar structures hidden from the rest of netdata

* better error handling

* dont create variables if not initialized for health

* dont create variables if not initialized for health again

* rrdfamily is now managed by dictionaries; references of it are acquired dictionary items

* type checking on acquired objects

* rrdcalc renaming of functions

* type checking for rrdfamily_acquired

* rrdcalc managed by dictionaries

* rrdcalc double free fix

* host rrdvars is always needed

* attempt to fix deadlock 1

* attempt to fix deadlock 2

* Remove unused variable

* attempt to fix deadlock 3

* snprintfz

* rrdcalc index in rrdset fix

* Stop storing active charts and computing chart hashes

* Remove store active chart function

* Remove compute chart hash function

* Remove sql_store_chart_hash function

* Remove store_active_dimension function

* dictionary delayed destruction

* formatting and cleanup

* zero dictionary base on rrdsetvar

* added internal error to log delayed destructions of dictionaries

* typo in rrddimvar

* added debugging info to dictionary

* debug info

* fix for rrdcalc keys being empty

* remove forgotten unlock

* remove deadlock

* Switch to metadata version 5 and drop
  chart_hash
  chart_hash_map
  chart_active
  dimension_active
  v_chart_hash

* SQL cosmetic changes

* do not busy wait while destroying a referenced dictionary

* remove deadlock

* code cleanup; re-organization;

* fast cleanup and flushing of dictionaries

* number formatting fixes

* do not delete configured alerts when archiving a chart

* rrddim obsolete linked list management outside dictionaries

* removed duplicate contexts call

* fix crash when rrdfamily is not initialized

* dont keep rrddimvar referenced

* properly cleanup rrdvar

* removed some locks

* Do not attempt to cleanup chart_hash / chart_hash_map

* rrdcalctemplate managed by dictionary

* register callbacks on the right dictionary

* removed some more locks

* rrdcalc secondary index replaced with linked-list; rrdcalc labels updates are now executed by health thread

* when looking up for an alarm look using both chart id and chart name

* host initialization a bit more modular

* init rrdlabels on host update

* preparation for dictionary views

* improved comment

* unused variables without internal checks

* service threads isolation and worker info

* more worker info in service thread

* thread cancelability debugging with internal checks

* strings data races addressed; fixes https://github.com/netdata/netdata/issues/13647

* dictionary modularization

* Remove unused SQL statement definition

* unit-tested thread safety of dictionaries; removed data race conditions on dictionaries and strings; dictionaries now can detect if the caller is holds a write lock and automatically all the calls become their unsafe versions; all direct calls to unsafe version is eliminated

* remove worker_is_idle() from the exit of service functions, because we lose the lock time between loops

* rewritten dictionary to have 2 separate locks, one for indexing and another for traversal

* Update collectors/cgroups.plugin/sys_fs_cgroup.c

Co-authored-by: Vladimir Kobal <vlad@prokk.net>

* Update collectors/cgroups.plugin/sys_fs_cgroup.c

Co-authored-by: Vladimir Kobal <vlad@prokk.net>

* Update collectors/proc.plugin/proc_net_dev.c

Co-authored-by: Vladimir Kobal <vlad@prokk.net>

* fix memory leak in rrdset cache_dir

* minor dictionary changes

* dont use index locks in single threaded

* obsolete dict option

* rrddim options and flags separation; rrdset_done() optimization to keep array of reference pointers to rrddim;

* fix jump on uninitialized value in dictionary; remove double free of cache_dir

* addressed codacy findings

* removed debugging code

* use the private refcount on dictionaries

* make dictionary item desctructors work on dictionary destruction; strictier control on dictionary API; proper cleanup sequence on rrddim;

* more dictionary statistics

* global statistics about dictionary operations, memory, items, callbacks

* dictionary support for views - missing the public API

* removed warning about unused parameter

* chart and context name for cloud

* chart and context name for cloud, again

* dictionary statistics fixed; first implementation of dictionary views - not currently used

* only the master can globally delete an item

* context needs netdata prefix

* fix context and chart it of spins

* fix for host variables when health is not enabled

* run garbage collector on item insert too

* Fix info message; remove extra "using"

* update dict unittest for new placement of garbage collector

* we need RRDHOST->rrdvars for maintaining custom host variables

* Health initialization needs the host->host_uuid

* split STRING to its own files; no code changes other than that

* initialize health unconditionally

* unit tests do not pollute the global scope with their variables

* Skip initialization when creating archived hosts on startup. When a child connects it will initialize properly

Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com>
Co-authored-by: Vladimir Kobal <vlad@prokk.net>
2022-09-19 23:46:13 +03:00
Costa Tsaousis
7784a16cc7
Dictionary with JudyHS and double linked list ()
* dictionary internals isolation

* more dictionary cleanups

* added unit test

* we should use DICT internally

* disable cups in cmake

* implement DICTIONARY with Judy arrays

* operational JUDY implementation

* JUDY cleanup

* JUDY summary added

* JudyHS implementation with double linked list

* test negative searches too

* optimize destruction

* optimize set to insert first without lookup

* updated stats

* code cleanup; better organization; updated info

* more code cleanup and commenting

* more cleanup, renames and comments

* fix rename

* more cleanups

* use Judy.h from system paths

* added foreach traversal; added flag to add item in front; isolated locks to their own functions; destruction returns the number of bytes freed

* more comments; flags are now 16-bit

* completed unittesting

* addressed comments and added reference counters maintainance

* added unittest in main; tested removal of items in front, back and middle

* added read/write walkthrough and foreach; allowed walkthrough and foreach in write mode to delete the current element (used by cups.plugin); referenced counters removed from the API

* DICTFE.name should be const too

* added API calls for exposing all statistics

* dictionary flags as enum and reference counters as atomic operations

* more comments; improved error handling at unit tests

* added functions to allow unsafe access while traversing the dictionary with locks in place

* check for libcups in cmake

* added delete callback; implemented statsd with this dictionary

* added missing dfe_done()

* added alternative implementation with AVL

* added documentation

* added comments and warning about AVL

* dictionary walktrhough on new code

* simplified foreach; updated docs

* updated docs

* AVL is much faster without hashes

* AVL should follow DBENGINE
2022-06-01 20:01:52 +03:00
Ilya Mashchenko
0fa55c7dce
feat: move dirs, logs, and env vars config options to separate sections () 2022-05-17 17:31:19 +03:00
vkalintiris
b8cd2bdc50
Remove unecessary relative paths when including headers. ()
Currently, we add the repository's top-level dir in the compiler's
header search path. This means that code in every top-level directory
within the repo can include headers sibling top-level directories.

This patch makes header inclusion consistent when it comes to files
that are included from sibling top-level directories within the repo.
2021-05-24 17:44:50 +03:00
thiagoftsm
51b57dc0a5
Add new cookie to fix 8094 ()
Add missing cookies to Netdata.
2021-03-02 20:00:38 +00:00
Andrew Moss
aa3ec552c8
Enable support for Netdata Cloud.
This PR merges the feature-branch to make the cloud live. It contains the following work:
Co-authored-by: Andrew Moss <1043609+amoss@users.noreply.github.com(opens in new tab)>
Co-authored-by: Jacek Kolasa <jacek.kolasa@gmail.com(opens in new tab)>
Co-authored-by: Austin S. Hemmelgarn <austin@netdata.cloud(opens in new tab)>
Co-authored-by: James Mills <prologic@shortcircuit.net.au(opens in new tab)>
Co-authored-by: Markos Fountoulakis <44345837+mfundul@users.noreply.github.com(opens in new tab)>
Co-authored-by: Timotej S <6674623+underhood@users.noreply.github.com(opens in new tab)>
Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com(opens in new tab)>
* dashboard with new navbars, v1.0-alpha.9: PR 
* dashboard v1.0.11: 
Co-authored-by: Jacek Kolasa <jacek.kolasa@gmail.com(opens in new tab)>
* Added installer code to bundle JSON-c if it's not present. PR 
Co-authored-by: James Mills <prologic@shortcircuit.net.au(opens in new tab)>
* Fix claiming config PR 
* Adds JSON-c as hard dep. for ACLK PR 
* Fix SSL renegotiation errors in old versions of openssl. PR . Also - we have a transient problem with opensuse CI so this PR disables them with a commit from @prologic.
Co-authored-by: James Mills <prologic@shortcircuit.net.au(opens in new tab)>
* Fix claiming error handling PR 
* Added CI to verify JSON-C bundling code in installer PR 
* Make cloud-enabled flag in web/api/v1/info be independent of ACLK build success PR 
* Reduce ACLK_STABLE_TIMEOUT from 10 to 3 seconds PR 
* remove old-cloud related UI from old dashboard (accessible now via /old suffix) PR 
* dashboard v1.0.13 PR 
* dashboard v1.0.14 PR 
* Provide feedback on proxy setting changes PR 
* Change the name of the connect message to update during an ongoing session PR 
* Fetch active alarms from alarm_log PR 
2020-05-11 16:37:27 +10:00
Andrew Moss
fe722cb2a4
Improve the behavior of claiming ()
The default cloud url has been updated to app.netdata.cloud ready for the release. The claiming process now checks the current user executing claiming and refuses to perform the claim for the wrong user. If the current UID is 0 then claiming proceeds but the file ownership is adjusted to be the correct netdata user. The default expected user is `netdata` unless the script can identify the user from the current configuration. After the claiming script is executed the CLI is used to reload the claiming state.
2020-03-31 13:07:24 +02:00
ladakis
1f51a65328 alarm-notify.sh should respect the cloud base url setting ()
Fixes 

This patch exports through netdata daemon as an environment variable the NETDATA_REGISTRY_CLOUD_BASE_URL in order for the alarm-notify.sh.in to be able to use it. Therefore we can skip the hardcoded reference to cloud URL inside the script.
2019-07-04 15:27:22 +02:00
George Moschovitis
b6ce817545
Cloud Sign-In ()
* Manually merged changes from old hub-support branch, tracking 

Call claim url 
Claim ui improvements 
Cleanup
Implement Sign Out
Introduced sign-in modal 
Added sign-in button
More work on the iframe trick
More work
More work on the logic, removed old obsolete stuff
Close modal
Implement account menu
Minor rename
Renamed my-netdata to My Agents
Show migrate button
Collect known agents
Work on migrateRegistryDidClick
Minor
Actually show agents from netdata cloud in the menu
Some cleanup
Keep all the alternate_urls for each agent
Fix for tooltips over SignIn/AccountMenu

* Actually use NETDATA.registry.cloudBaseURL

Tricky!

* Hide switch identity when signed-in 

* Manually merged changes from old hub-support branch, tracking 

Call claim url 
Claim ui improvements 
Cleanup
Implement Sign Out
Introduced sign-in modal 
Added sign-in button
More work on the iframe trick
More work
More work on the logic, removed old obsolete stuff
Close modal
Implement account menu
Minor rename
Renamed my-netdata to My Agents
Show migrate button
Collect known agents
Work on migrateRegistryDidClick
Minor
Actually show agents from netdata cloud in the menu
Some cleanup
Keep all the alternate_urls for each agent
Fix for tooltips over SignIn/AccountMenu

* Actually use NETDATA.registry.cloudBaseURL

Tricky!

* Hide switch identity when signed-in 

* Cleanup

* Refresh menu on sign-in

* Disable cloud functionality if cloud base url is not set.

This wll allow the merging of the branch into master, so we can avoid nasty rebases.

* Updated to use the latest API endpoints

* Fixed a couple of LGTM warnings

* Improved migration algorithm, some cleanup.

* Update My-Netdata menu on sign-out

* Minor

* Replaced modal with window

* Update the My-Agents menu after migration, cleanup

* Make the agent work after switching cloudBaseURL, cleanup

* Introduced event tracing for analytics

* Minor

* Removed GA

* Fixed error reported by LGTM

* Only send the diff when syncing agents to ameliorate the load on the backend, cleanup

* Reverted My-Netdata name, added some logging

* Add Netdata Cloud menu item

* Minor

* Use the merge: false option and a fix

* Added loading message in my-netdata menu

* Show error if we cannot connect to netdata.cloud

* Minor

* Implemented deleteCloudKnownAgentURL api call, use it in my-netdata menu.

* Removed menu entry

* Disable my-netdata menu if user is not signed-in and using the global registry

* Stop accessing the registry if it's not used.

* Mask the agent url if the registry is in 'disabled' mode

* Filter masked urls

* Improved filtering of masked urls

* Try to eagerly initialize the account ui to improve perceived performance

* Minor

* Don't search for other people's urls in cloud-enabled mode.

* Added basic my-netdata filtering

* Filter streamed host, aesthetic fixes

* Minor

* Some improvements of the filter ui

* Removed What is this

* Added placeholder to input, other fixes 

* Show message if no databases match filter criteria

* Fixed bug where agent lists where not merged

* Minor

* Hide modal if it redirects to self.

* autocomplete off for filter input

* Enable delete for custom registries, don't show error if delete fails

* Filter agents without urls

* Fix LGTM warning

* Minor

* Concatenate at client side, used the faster merge: false path

* Added a clear button to the filter for extra usability

* Minor

* Minor

* Improvements for small screens (more needed)

* Combined  my-netdata menu and hostname

* Re-enabled registry masking

* Show agent-filter only when signed-in

* Improved syncAgents

* Don't mask if using custom registry

* Reject agents with empty urls

* Filter valid agents

* Fixed a couple of bugs

* Applied Chris' fixes

* Fix in registry.c

* Cleanup

* Only sync once

* Implemented forceSync

* Added what is this

* sso, wip

* Working SSO sign-in/sign-out, cleanup

* Added Chris' patch

* Added a modal that explains what synchronize is doing

* Use sso-agent

* Use origin as query param in sign-in

* iframe -> origin

* Pass machine_guid to sso

* Make sure that the current netdata agent is synchronized hub#262

* Normalize originURL

* Reenable tryFastInitCloud()

* Updated to the latest endpoints

* Support synchronizing to multiple cloud accounts

* Set default cloud base url to netdata.cloud

* Fix filter issues with Firefox

* Fix for double tooltip on sign-in

* Show known servers in console for debugging purposes

* Don't block on errors to delete from registry when signed in

* Disable tryFastInitCloud

* Improved styling for filter input

* Improved styling in my-netdata menu

* Display the registry url in the sync-registry modal

* agents -> nodes in texts

* Support for sso-precheck

* Do not implicitly synchronize custom registries.

* Improvement to syncAgents (more coming)

* More fixes

* Don't sign in users with private registries if they don't consent on the sync

* Set netdataRegistryAfterMs = 0

* Don't pass url to sso-agent

* Added Chris' patch to alarm-notify

* Refactored syncAgent/mergeAgents, make sure current Agent is synced on sign-in.

* Fix for LGTM warning

* Minor

* Fix for a XSS warning

* Extra check for dataLayer
2019-01-28 12:27:42 +02:00
George Moschovitis
6b05a21302
Pass cloud_base_url from daemon to web/gui through hello endpoint ()
* Pass cloud_base_url from daemon to web/gui through hello endpoint 

* Make sure the [cloud] section goes into the general part of netdata.conf 
2018-12-13 15:30:21 +02:00
Costa Tsaousis
8fbf817ef8
modularized all source code ()
* modularized all external plugins

* added README.md in plugins

* fixed title

* fixed typo

* relative link to external plugins

* external plugins configuration README

* added plugins link

* remove plugins link

* plugin names are links

* added links to external plugins

* removed unecessary spacing

* list to table

* added language

* fixed typo

* list to table on internal plugins

* added more documentation to internal plugins

* moved python, node, and bash code and configs into the external plugins

* added statsd README

* fix bug with corrupting config.h every 2nd compilation

* moved all config files together with their code

* more documentation

* diskspace info

* fixed broken links in apps.plugin

* added backends docs

* updated plugins readme

* move nc-backend.sh to backends

* created daemon directory

* moved all code outside src/

* fixed readme identation

* renamed plugins.d.plugin to plugins.d

* updated readme

* removed linux- from linux plugins

* updated readme

* updated readme

* updated readme

* updated readme

* updated readme

* updated readme

* fixed README.md links

* fixed netdata tree links

* updated codacy, codeclimate and lgtm excluded paths

* update CMakeLists.txt

* updated automake options at top directory

* libnetdata slit into directories

* updated READMEs

* updated READMEs

* updated ARL docs

* updated ARL docs

* moved /plugins to /collectors

* moved all external plugins outside plugins.d

* updated codacy, codeclimate, lgtm

* updated README

* updated url

* updated readme

* updated readme

* updated readme

* updated readme

* moved api and web into webserver

* web/api web/gui web/server

* modularized webserver

* removed web/gui/version.txt
2018-10-15 23:16:42 +03:00
Renamed from src/registry/registry_init.c (Browse further)