Without this, the way our build works causes it to get updated and
rebuilt (and thus trigger relinking of multiple parts of the agent) when
`cmake --install` is called, which in turn makes installs take much
longer.
* Disable generation of debuginfo packages for DEB distros.
actually need to the debuginfo to work correctly, so split debug symbols don’t
Our automatic stacktrace logging and crash reporting functionality
really make sense for our use case.
* Only disable split debuginfo for core, not plugins.
We don’t do crash reporting or stack trace logging for plugins, so they
can still use the split debug info packages.
* Work around bugs in Debian 11 and Ubuntu 20.04.
* initial implementation of libbacktrace
* in buildinfo show the parameters of libbacktrace
* do not disable libbacktrace if threading is not supported
* Don’t install libbacktrace, only build it.
* Disable libbacktrace for 32-bit ARM builds.
* Make libunwind and libbacktrace mutually exclusive at configure time.
Instead of relying on it being mutually exclusive at build time. This
ensures we don’t waste time on libunwind when using libbacktrace.
* Only use libbacktrace on Linux and Windows
* Work around broken logic in openSUSE rpmbuild.
* Fix handling of libbacktrace for 32-bit ARM static builds.
---------
Co-authored-by: Austin S. Hemmelgarn <austin@netdata.cloud>
* Fix Go version requirement detection to not have external deps.
Instead of relying on a UNIX-like environment with the `grep` and `cut`
commands, perform the required data extraction using CMake code. This
makes it more portable, but also should speed things up a tiny bit
because it doesn’t require invoking external commands.
* Preliminary integration of the new otel-collector into the build.
Adding `-DENABLE_PLUGIN_OTEL=On` to CMake options will enable building
the plugin and installing it.
Currently disabled by default, and does not include packaging
integration yet.
The plugin itself can be built independently of the primary build system
but using the same build infrastructure that is used to build it as
part of the regular build by using CMake in the src/go/otel-collector
directory.
* Minor cleanup.
* Fix build prefix.
* Restructure to not use a sub-project.
* Add LTO support in CMake build system.
Internally, CMake calls LTO ‘Interprocedural Optimization’, and it
provides functionality for checking for support for this as well as
enabling it by default for targets. This leverages that support to
auto-detect LTO and enable it if it’s supported.
* Default to disabling LTO for Debug builds.
* Add handling for LTO to netdata installer code.
* Switch back to DISABLE_LTO as option name.
Using `ENABLE_LTO` leads to a possibility for confusion among users, as it
does behave in the most intuitive manner. Instead of ensuring that LTO
is used (and thus behaving like every other `ENABLE_*` option we have),
it allows the usage LTO if it’s supported. Thus a build with
`-DENABLE_LTO=True` may not actually be built with LTO.
By instead using `DISABLE_LTO`, the behavior matches up directly with
how most people are likely to interpret the meaning, because a build
with `-DDISABLE_LTO=True` will _never_ have LTO flags added to the
compiler/linker flags.
* Fix condition for determining default for DISABLE_LTO.
* Turn off LTO auto-detection in RPM package builds.
On pretty much all RPM platforms, the RPM build process itself will
correctly add the required compiler flags when building, so we don’t
actually need to auto-detect LTO support in CMake here.
Additionally, on at least some RPM platforms, CMake’s auto-detection
for LTO support actually breaks the build when used.
* Disable function and data sections when using LTO.
On at least some systems, `-fdata-sections` combined with LTO reliably
causes failures at link time with our code.
The final binary size on systems where the combination _works_ differs by
no more than a few KiB on average (tested on 64-bit x86 on Ubuntu 22.04,
Debian 12, Fedora 39, and Rocky Linux 9), so we’re not actually getting
almost any benefit out of using both with things as they are now, but
LTO gives us a meaasurable performance improvement that per-function and
per-data sections do not.
* Restructure in-line with current repo state.
* Disable LTO on Windows builds since it doesn’t work there.
* Fix compiler flag handling order.
* Switch LTO option name to USE_LTO for consistency with USE_MOLD.
* Fix up handling of libunwind in CMake.
- Fix the questionable default handling of CMAKE_SYSTEM_PROCESSOR so
that it reliably reflects the target architecture.
- Add a case for handling 32-bit x86 builds with libunwind.
- Tweak the match cases for the various architectures to be more
reliable.
* Fix libbpf usage of CMAKE_SYSTEM_PROCESSOR.
* add libsensors to debugfs.plugin
* add string representations of libsensors types
* read sensors3.conf during initialization
* more work
* progress
* working set
* working set 2
* working sensors
* Vendor lm-sensors as a Git submodule
* vendored libsensors
* search for sensors headers in the vendored library
* add flex and bison to required packages
* include sensors.h from the vendored directory
* remove HAVE_LIBSENSORS variable
* do not load sensor subfeatures that are not needed
* added device, driver and subsystem labels
* add message id to log
* copy the default sensors3.conf file to netdata stock configs directory
* move sensors to a separate thread; automatically adapt data collection frequency to match actual data collection latency
* make debugfs plugin wait while libsensors is running
* fix for update every
* update chart ctx and id, remove non-important labels
* dont set label to feat name if none
* just alarm
* add sybsystem and driver labels
---------
Co-authored-by: ilyam8 <ilya@netdata.cloud>
* updated copyright notices everywhere (I hope)
* Update makeself.lsm
* Update coverity-scan.sh
* make all newlines be linux, not windows
* remove copyright from all files (the take it from the repo), unless it is printed to users
* Make Python collectors optional at build time.
Given the general shift of most data collectors to the Go plugin, it’s
no-longer the case that users ‘always’ need to have the Python plugin
included.
This also makes them consistent with the other external collectors.
* Make charts.d collectors optional at build time.
Same reasoning as for the Python collectors.
Also, this fixes up loopsleepms.sh to only be handled if a collector
that needs it is enabled.
* Properly tie config directories to their plugin components.
The Go and Python plugin config directories were being installed
unconditionally as part of the main Netdata component. This is wrong,
they should be installed as part of the plugin packages/components
instead, as they are only needed if those plugins are installed.
* split claiming into multiple files; WIP claiming with api
* pidfile is now dynamically allocated
* netdata_exe_path is now dynamically allocated
* remove ENABLE_CLOUD and ENABLE_ACLK
* fix compilation
* remove ENABLE_HTTPS and ENABLE_OPENSSL
* remove the ability to disable cloud
* remove netdata_cloud_enabled variable; split rooms into a json array
* global libcurl initialization
* detect common claiming errors
* more common claiming errors
* finished claiming via API
* same as before
* same as before
* remove the old claiming logic that runs the claim script
* working claim.conf
* cleanup
* fix log message; default proxy is env
* fix log message
* remove netdata-claim.sh from run.sh
* remove netdata-claim.sh from everywhere, except kickstart scripts
* create cloud.d if it does not exist.
* better error handling and logging
* handle proxy disable
* merged master
* fix cmakelists for new files
* left-overs removal
* Include libcurl in required dependencies.
* Fix typo in dependency script.
* Use pkg-config for finding cURL.
This properly handles transitive dependencies, unlike the FindCURL
module.
* netdata installer writes claiming info to /etc/netdata/claim.conf
* remove claim from netdata
* add libcurl to windows packages
* add libcurl to windows packages
* compile-on-windows.sh installs too
* add NODE_ID streaming back to child and INDIRECT cloud status
* log child kill on windows
* fixes for spawn server on windows to ensure we have a valid pid and the process is properly terminated
* better handling to windows processes exit code
* pass the cloud url from parents to children
* add retries and timeout to claiming curl request
* remove FILE * from plugins.d
* spawn-tester to unittest spawning processes communication
* spawn-tester now tests FILE pointer I/O
* external plugins run in posix mode
* set blocking I/O on all pipes
* working spawn server on windows
* latest changes in spawn_popen applied to linux tools
* push environment
* repeated tests of fds
* export variable CYGWIN_BASE_PATH
* renamed to NETDATA_CYGWIN_BASE_PATH
* added cmd and help to adapt the command and the information to be presented to users during claiming
* split spawn server versions into files
* restored spawn server libuv based
* working libuv based spawn server
* fixes in libuv for windows
* working spawn server based on posix_spawn()
* fix fd leads on all spawn servers
* fixed windows spawn server
* fix signal handling to ensure proper cooperation with libuv
* switched windows to posix_spawn() based spawn server
* improvement on libuv version
* callocz() event loop
* simplification of libuv spawn server
* minor fixes in libuv and spawn tester
* api split into parts and separated by version; introduced /api/v3; no changes to old /api/v1 and /api/v2
* completed APIs splitting
* function renames
* remove dead code
* split basic functions into a directory
* execute external plugins in nofork spawn server with posix_spawn() for improved performance
* reset signals when using posix_spawn()
* fix spawn server logs and log cmdline in posix server
* bearer_get_token() implemented as function
* agent cloud status now exposes parent claim_id in indirect mode
* fixes for node id streaming from parent to children
* extract claimed id to separate file
* claim_id is no longer in host structure; there is a global claim_id for this agent and there are parent and origin claim ids in host structure
* fix issue on older compilers
* implement /api/v3 using calls from v1 and v2
* prevent asan leaks on local-sockets callback
* codacy fixes
* moved claim web api to web/api/v2
* when the agent is offline, prefer indirect connection when available; log a warning when a node changes node id
* improve inheritance of claim id from parent
* claim_id for bearer token show match any of the claim ids known
* aclk_connected replaced with functions
* aclk api can now be limited to node information, implementing [cloud].scope = license manager
* comment out most options in stream.conf so that internal defaults will be applied
* respect negative matches for send charts matching
* hidden functions are not accessible via the API; bearer_get_token function checks the request is coming from Netdata Cloud
* /api/v3/settings API
* added error logs to settings api
* saving and loading of bearer tokens
* Fix parameter when calling send_to_plugin
* Prevent overflow
* expose struct parser and typedef PARSER to enforce strict type checking on send_to_plugin()
* ensure the parser will not go away randomly from the receiver - it is now cleared when the receiver lock is acquired; also ensure the output sockets are set in the parser as long as the parser runs
* Add newline
* Send parent claim id downstream
* do not send anything when nodeid is zero
* code re-organization and cleanup
* add aclk capabilities, nodes summary and api version and protection to /api/v2,3/info
* added /api/v3/me which returns information about the current user
* make /api/v3/info accessible always
* Partially revert "remove netdata-claim.sh from everywhere, except kickstart scripts"
Due to how we handle files in our static builds and local builds, we
actually need to continue installing `netdata-claim.sh` to enable a
seamless transition to the new claiming mechanims without breaking
compatibility with existing installs or existing automation tooling that
is directly invoking the claiming script.
The script itself will be rewritten in a subsequent commit to simply
wrap the new claiming methodology, together with some additional changes
to ensure that a warning is issued if the script is invoked by anything
other than the kickstart script.
* Rewrite claiming script to use new claiming method.
* Revert "netdata installer writes claiming info to /etc/netdata/claim.conf"
Same reasoning as for 2e27bedb3fbf9df523bff407f2e8c8428e350e38.
We need to keep the old claiming support code in the kickstart script
for the forseeable future so that existing installs can still be
claimed, since the kickstart script is _NOT_ versioned with the agent.
A later commit will add native support for the new claiming method and
use that in preference to the claiming script if it appears to be
available.
* Add support for new claiming method to kickstart.sh.
This adds native support to the kickstart script to use the new claiming
method without depending on the claiming script, as well as adding a few
extra tweaks to the claiming script to enable it to better handle the
transition.
Expected behavior is for the kickstart script to use the new claiming
code path if the claiming script is either not installed, or does not
contain the specific string `%%NEW_CLAIMING_METHOD%%`. This way we will
skip the claiming script on systems which have the updated copy that
uses the new claiming approach, which should keep kickstart behavior
consistent with what Netdata itself supports.
* Depend on JSON-C 0.14 as a minimum supported version.
Needed for uint64 functions.
* Fix claiming option validation in kickstart script.
* do not cache auth in web client
* reuse bearer tokens when the request to create one matches an existing
* dictionaries dfe loops now allow using return statement
* bearer token files are now fixed for specific agents by having the machine guid of the agent in them
* systemd journal now respects facets and disables the default facets when not given
* fixed commands.c
* restored log for not openning config file
* Fix Netdata group templating for claiming script.
* Warn on failed templating in claiming script.
* Make `--require-cloud` a slient no-op.
We don’t need to warn users that it does nothing, we should just have ti
do nothing.
* added debugging info to claiming
* log also the response
* do not send double / at the url
* properly remove keyword from parameters
* disable debug during claimming
* fix log messages
* Update packaging/installer/kickstart.sh
* Update packaging/installer/kickstart.sh
* implemented POST request payload parsing for systemd-journal
* added missing reset of facets in json parsing
* JSON payload does not need hashes any more. I can accept the raw values
---------
Co-authored-by: Ilya Mashchenko <ilya@netdata.cloud>
Co-authored-by: Austin S. Hemmelgarn <austin@netdata.cloud>
Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com>
Co-authored-by: Austin S. Hemmelgarn <ahferroin7@gmail.com>
* Add code signing for Windows executables.
* Fix typos and add failure notification.
* Use full version for trusted signing action.
Because MS isn’t publishing it with proper semver tags.
* Avoid reinstalling dependencies that are already installed.
* Fix CMake 3.30 compatibility.
* Don’t let BUILD_DIR propagate to cmake.
* Fix JSON-C build warning.
* Fix handling of externally specified build directories.
While regular Windows paths do actually work under MSYS2, they seem to
confuse CMake, so we need to convert to a standard MSYS2 path if
`BUILD_DIR` is set to a Windows path.
* Fix typo.
* Fix build directory handling.
* Fix up CMake feature handling for Windows.
* Better handle detection of Go on Windows.
* Provide Windows copy of Go for Windows build.
* Explicitly set GOROOT in environment.
* Explicitly disable Prometheus remote write exporter.
* Add note about DEFAULT_FEATURE_STATE_OPTION.
* Properly support CMake 3.30.
As of CMake 3.30, calling `FetchContent_Populate` is officially
deprecated, and you get a warning about eventual removal.
We end up calling this function to compensate for the fact that CMake
prior to 3.28 provides no other way to make an external project managed
through FetchContent available without adding it to the `all` target and
thus installing the files from it, which we need to avoid doing for our
vendored libraries.
This changes things to check for CMake 3.28 or newer, and use the
preferred method on those systems. Unfortunately, this is handled in a
different place than the old workaround needed it to be handled in, so
we need checks in multiple places to make this work.
* Bump supported CMake versions to 3.16-3.30.
The last system we supported that shipped 3.13 was Debian 10, which we
no longer support, and 3.30 is the latest version.
Outside of a few cases involving eBPF, we don’t actually need to have an
exact match between individual package component versions.
Removing this constraint significantly simplifies our dependency graph,
and should both speed up updates, and also make them much more reliable.
This will also simplify consolidation of dependency handling across
package types, because our package names are identical between DEB and
RPM packages.
* Use bundled protobuf for openSUSE packages.
Since their system copy seems to have major issues.
* Force vendored Abseil to be a static build.
Without this, we end up with linking issues if there’s an existing copy
of Abseil on the system.
Semver does not have the concept of a tweak
field. To address this, we just drop the major
field which has not changed in ages. We can
simply ignore/drop old sentry releases if/when
we perform any major releases.
If not explicitly set using exported environment variables, the choice
of compilers and compiler targets used by CMake does not get propagated
to sub-porjects created with ExternalProject_Add or
FetchContent_Declare.
In most cases this does not matter, because very few people are building
using non-default compilers for their environment, but it can cause
issues in two specific cases:
- If building for the same system that the build is happening on, but
using a non-default compiler specified using CMake arguments (instead
of via exported environment variables), sub-projects will still use
the default compiler for the system, which may result in linking
errors (or runtime failures even if the link succeeds).
- If cross-compiling and not using the preferred approaches of either a
toolchain file or exported environment variables, sub-projects may not
even build for the correct CPU architecture, causing the build to
fail.
This adds logic to ensure that the compiler and compiler targets get
propagated correctly even if they are just specified on the command
line, thus avoiding the above two possibilities.
* Define a single variable to identify OS.
* Remove comments.
* Move version-detection logic to separate module.
* s/VERSION/NETDATA_VERSION/g
* Hard-code NETDATA_VERSION wherever possible.
Now `program_version` is only used for getting or
setting the host's field.
* Update version variables
* Add license, fix variable name and provide better message.
* Clean up DEB package dependencies.
- dpkg is an implicit dependency for all DEB packages by virtue of them
being DEB packages, and we do not support any platforms that have an
old enough version to be an issue, so we don’t need a versioned
dependency on it either, so simply drop the explicit dependency on
dpkg.
- We also don’t support any platforms that ship an old enough version of
libcap2-bin to cause us problems, so remove the version constraint on
that as well.
- The lsb-base predepend is only needed if the user is using a
non-systemd init system. We don’t fully support this setup with our
native packages, and it’s a non-default configuration for Debian and
completely unsupported on Ubuntu, so it’s safe to assume that the
user knows what they are doing if they need it and not explicitly
require this.
- The apps plugin was missed in #17584, so fix it’s conflicts line here.
- The chartsd plugin _requires_ bash to function correctly, so actually
depend on it instead of recommending it (this will be a no-op on a
vast majority of systems, but is still useful insurance against users
doing stupid things).
- The CUPS and FreeIPMI plugins both pull in all the things they
actually need via automatic dependency generation, so there is no
reason for us to pull in full installs of CUPS or FreeIPMI as
dependencies.
- Everything in the Go plugin that was using `sudo` has been converted
to use `ndsudo` instead, so we don’t need to ask for `sudo` at all for
it anymore.
- The Python plguin _requires_ Python to function correctly, so actually
depend on it instead of recommending it.
* Fix typo.
* Fall back to querying libc.so.6 if ldd can’t detect libc implementation.
Needed for correct configuration on systems that hide the libc
implementation info in `ldd` output, like Gentoo.
* Fix typo.
`target_compile_definitions` can only be used for actual command
line _macro definitions_. But the `*_CFLAGS_OTHER` may contain flags
that are _not_ macro definitions, and when that happens, any use of
`target_compile_definitions` with the relevant `*_CFLAGS_OTHER` variable
will break the build.
Instead, we should be using `target_compile_options` for these
variables.
* abstraction layer for O/S
* updates
* updates
* updates
* temp fix for protobuf
* emulated waitid()
* fix
* fix
* compatibility layer
* fix for idtype
* fix for missing includes
* fix for missing includes
* added missing includes
* added missing includes
* added missing includes
* added missing includes
* added missing includes
* added missing includes
* UUID renamed to ND_UUID to avoid conflict with windows.h
* include libnetdata.h always - no conflicts
* simplify abstraction headers
* fix missing functions
* fix missing functions
* fix missing functions
* fix missing functions
* rename MSYS to WINDOWS
* moved byteorder.h
* structure for an internal windows plugin
* 1st windows plugin
* working plugin
* fix printf
* Special case windows for protobuf
* remove cygwin, compile both as windows
* log windows libraries used
* fix cmake
* fix protobuf
* compilation
* updated compilation script
* added system.ram
* windows uptime
* perflib
* working perflibdump
* minify dump
* updates to windows plugins, enable ML
* minor compatibility fixes for cygwin and msys
* perflib-dump to its own file
* perflib now indexes names
* improvements to the library; disks module WIP
* API for selectively traversing the metrics
* first working perflib chart: disk.space
* working chart on logical and physical disks
* added windows protocols
* fix datatypes for loops
* tinysleep for native smallest sleep support
* remove libuuid dependency on windows
* fix uuid functions for macos compilation
* fix uuid comparison function
* do not overwrite uuid library functions, define them as aliases to our own
* fixed uuid_unparse functions
* fixed typo
* added perflib processor charts
* updates for compiling without posix emulation
* gather common contexts together
* fix includes on linux
* perflib-memory
* windows mem.available
* Update variable names for protobuf
* network traffic
* add network adapters that have traffic as virtual interfaces
* add -pipe to windows compilation
* reset or overflow flag is now per dimension
* dpc is now counted separately
* verified all perflib fields are processed and no text fields are present in the data
* more common contexts
* fix crash
* do not add system.net multiple times
* install deps update and shortcut
* all threads are now joinable behind the scenes
* fix threads cleanup
* prepare for abstracting threads API
* netdata threads full abstraction from pthreads
* more threads abstraction and cleanup
* more compatibility changes
* fix compiler warnings
* add base-devel to packages
* removed duplicate base-devel
* check for strndup
* check headers in quotes
* fix linux compilation
* fix attribute gnu_printf on macos
* fix for threads on macos
* mingw64 compatibility
* enable compilation on windows clion
* added instructions
* enable cloud
* compatibility fixes
* compatibility fixes
* compatibility fixes
* clion works on windows
* support both MSYSTEM=MSYS and MSYSTEM=MINGW64 for configure
* cleanup and docs
* rename uuid_t to nd_uuid_t to avoid conflict with windows uuid_t
* leftovers uuid_t
* do not include uuid.h on macos
* threads signaled cancellations
* do not install v0 dashboard on windows
* script to install openssh server on windows
* update openssh installation script
* update openssh installation script
* update openssh installation script
* update openssh installation script
* update openssh installation script
* update openssh installation script
* update openssh installation script
* update openssh installation script
* update openssh installation script
* use cleanup variable instead of pthreads push and pop
* replace all calls to netdata_thread_cleanup_push() and netdata_thread_cleanup_pop() with __attribute__((cleanup(...)))
* remove left-over freez
* make sure there are no locks acquired at thread exit
* add missing parameter
* stream receivers and senders are now voluntarily cancelled
* plugins.d now voluntarily exits its threads
* uuid_t may not be aligned to word boundaries - fix the uuid_t functions to work on unaligned objects too.
* collectors evloop is now using the new threading cancellation; ml is now not using pthread_cancel; more fixes
* eliminate threads cancellability from the code base
* fix exit timings and logs; fix uv_threads tags
* use SSL_has_pending() only when it is available
* do not use SSL_has_pending()
* dyncfg files on windows escape collon and pipe characters
* fix compilation on older systems
* fix compilation on older systems
* Create windows installer.
The installer will install everything
under C:\netdata by default.
It will:
- Install msys2 at C:\netdata
- Install netdata dependencies with pacman
- Install the agent itself under C:\netdata\opt
You can start the agent by running an MSYS
shell with C:\netdata\msys2_shell.cmd and
then start the agent normally with:
/opt/netdata/usr/sbin/netdata -D
There are a more couple things to work on:
- Verify publisher.
- Install all deps not just libuv & protobuf.
- Figure out how we want to auto-start the agent as a service.
- Check how to uninstall things.
* fixed typo
* code cleanup
* Create uninstaller
---------
Co-authored-by: vkalintiris <vasilis@netdata.cloud>
* Fix logic error in CMake code for kernel version detection.
* Force legacy libbpf for CentOS 7 and AL 2 package builds.
* Check target kernel version instead of host kernel version.
This should help when building in containers.
* Fix up handling for older versions of CMake.
* Use CMake C compiler for libbpf builds.
* Fix selection of legacy eBPF code in static builds.
* Explicitly pull in kernel headers on CentOS.
* Fix typo in FORCE_LEGACY_LIBBPF option.
* Only enable eBPF by default on x86.
* Fix detection of static builds.
* Fix libc detection logic.
* Fix handling of static builds.
* Fix musl libc detection.
* Fix check messages for libc detection.
