* Deployment Guides Improved
* Windows installer + ML (all) improved
* fix formatting and some minor changes
---------
Co-authored-by: Ilya Mashchenko <ilya@netdata.cloud>
* allow users to install any version; stop netdata before installing the update
* fix compatibility with wix 4
* fix-msi: Remove conflict between options (https://docs.firegiant.com/wix3/xsd/wix/majorupgrade/)
* fix-msi: Comment not allowed code. We cannot have more than one primary key listed (FileInUse)
* fix-msi: Add Power Shell script to be called after installation
* fix-msi: Copy power shell script to installer.
* fix-msi: Add option to do download and move PS file
* fix-msi: First try run at the end
* iis_ad: Fix wrong execution order and call to powershell
* fix-msi: Add messages and test to Power Shell script
---------
Co-authored-by: thiagoftsm <thiagoftsm@gmail.com>
* Update CI to generate MSI installer for Windows using WiX.
* Fix missing option for WiX install.
* Copy icon file alongside WiX sources.
* Run WiX in the build directory instead of CWD.
* Import draft Wix template for creating MSI packages.
* msi-wix: Add missing XML entry
* msi-wix: Format lines using wixtoolset and Heatwave, also remove a wrong close
* msi-wix: Reorganize code allowing it to be compiled with command 'wix build netdata.wxs'
* Install event manifest and netdata service.
* msi-wix: Add basis to claim agent
* msi-wix: Add Icon to Control panel
* msi-wix: Add Script to convert txt to rtf to be used with licenses
* msi-wix: Add UI and Cloud license
* msi-wix: Add file to install wix dependencies
* msi-wix: Add message to last screen and a checkbox to launch terminal
* Fix event manifest registration.
* msi-wix: Restore removed options
* msi-wix: Add basis for new windows
* msi-wix: Add new dialog
* msi-wix: Comment UI that was posponed and users/group not found
* msi-wix: Remove UI and keep only inline installer
* msi-wix: Add necessary parameter to WIX
* msi-wix: Adjust code and installer
* msi-wix: Fix MSI installer
* msi-wix: Adjust claiming to work with different installers
* msi-wix: Remove no compression and embed files to run final tests
* msi-wix: Address part of permissions issues
* msi-wix: Address remaining issue
* msi-wix: Cleanup
* msi-wix: Add current version to MSI
* msi-wix: Remove unused file
* msi-wix: Fix missing variable kept only in Windows host
* msi-wix: Set correct permissions
* msi-wix: Set correct permissions
* msi-wix: Call wevtutil
* msi-wix: Fix multiple entries in Control Panel
* msi-wix: Rename installation directory
* msi-wix: Rename name in Control Panel
* msi-wix: Address comment (cleanup)
* msi-wix: Address comments (Part II)
* msi-wix: Address comments (Part III)
* msi-wix: Fix rtf convertion
* msi-wix: Remove left-over from previous installer that could be affecting current
* msi-wix: Add missing directories
---------
Co-authored-by: thiagoftsm <thiagoftsm@gmail.com>
* split netdata logger into multiple files - no acctual code changes
* move around some more code
* base for implementing windows events logging
* fix for the last commit
* working logging to windows events, but not pretty yet
* fix compilation on linux
* added scripts for compiling the resource file and importing the manifest
* added validation that the provider is available
* working manifest for ETW (Event Tracing for Windows)
* compile the messages dll with msys tools
* handle wevents configuration
* when starting under clion, do not start as service
* unify conversion to utf16
* fix bug in windows-events.plugin that was incorrectly not processing right the publishers that do not have a UUID
* enable wevents as default logging for all methods, under windows
* log to windows using EventCreate.exe for the messages
* do not log all the fields
* added log-forwarder to spawn-server-windows
* fix last character being cut-off when converting from utf-16
* updated info
* updated any_to_utf16() to be always consistent
* added utf16_to_utf8()
* external plugins inherit windows events
* fix wrong log source
* fix spawn server logs
* log to multiple event log sources
* generate custom messages dll for event viewer - working
* removed debugging code
* cleanup log forwarder entries from the thread, to avoid bad file descriptor in poll()
* .mc and its manifest are automatically generated
* sanitizers should not remove trailing underscores
* use the resources dll for the netdata directory; set the default maxSize to windows events
* do not set customer flag on event ids; use the same naming for channels and providers
* work to unify manifest and resources
* netdata now logs using ETW
* implemented etw and wel logging in netdata
* minor changes
* updated windows installer to install the manifest
* do not install etw if the manifest is not there
* allow loggings to WEL and ETW at the same time
* fix the installer conditions
* fix nsi
* detect ci paths for sys utils
* enable ETW is CI
* better integration of spawn server with logger
* use script to find SDK path
* use auto-discovery of sdk and visual studio
* fix overlapping link.exe with msys; do not escape percentage when it is not followed by a number; added more documentation about windows
* debug info for path
* fixes compilation scripts
* ETW and WEL are always required on Windows
* in progress for supporting full text search queries
* find mvc versions
* improve find-sdk-path.sh
* fix the script once again
* fetch event data for full text search
* fix script again
* fix script, yes again
* fts using event data
* code renames and cleanup for clarity
* update documentation
* full text search switches plugin to load everything synchronously
* full text search using the individual event data fields, without using XML
* close all idle provider handles after 5 mins
* added EventsAPI field
* supported exposing all system fields; started documentation about windows events plugin
* avoid crash because of unitialized memory
* remove debugging
* do not add qualifiers and version when they are zero
* updated docs
* copy the manifest too
* rework on installing manifest and dll
* completed documentation
* work on windows-events sources list
* fix windows installer logic
* removed unecessary include
* added image to documentation
* deploy meta, WINDOWS-INSTALLER markdwon changes and deploy template changes
* multiline command
* de-hardcode nighlty version
* note for beta and nightly availability
* note for paid win installer
* frontend does not support blockquotes and bold, make it one sentence
* reorder
* remove dup
* reword
This ensures forward-portability, because I can all but guarantee that
we _will_ eventually want to publish 64-bit ARM binaries for Windows,
and we will then need to differentiate between the installer files.
* address_collection: Fix os version
* address_collection: Cleanup code
* address_collection: Add install type
* address_collection: Address kernel version
* address_collection: Adjust variables for Microsoft
* wins_perm: Modify original netdata_claim manifest adding external version
* wins_perm: Add resources to netdatacli
* wins_perm: Add resources to netdata
* win_rename: Restore lines removed. They will be removed in another PR.
* win_perm: Update manifest
* split claiming into multiple files; WIP claiming with api
* pidfile is now dynamically allocated
* netdata_exe_path is now dynamically allocated
* remove ENABLE_CLOUD and ENABLE_ACLK
* fix compilation
* remove ENABLE_HTTPS and ENABLE_OPENSSL
* remove the ability to disable cloud
* remove netdata_cloud_enabled variable; split rooms into a json array
* global libcurl initialization
* detect common claiming errors
* more common claiming errors
* finished claiming via API
* same as before
* same as before
* remove the old claiming logic that runs the claim script
* working claim.conf
* cleanup
* fix log message; default proxy is env
* fix log message
* remove netdata-claim.sh from run.sh
* remove netdata-claim.sh from everywhere, except kickstart scripts
* create cloud.d if it does not exist.
* better error handling and logging
* handle proxy disable
* merged master
* fix cmakelists for new files
* left-overs removal
* Include libcurl in required dependencies.
* Fix typo in dependency script.
* Use pkg-config for finding cURL.
This properly handles transitive dependencies, unlike the FindCURL
module.
* netdata installer writes claiming info to /etc/netdata/claim.conf
* remove claim from netdata
* add libcurl to windows packages
* add libcurl to windows packages
* compile-on-windows.sh installs too
* add NODE_ID streaming back to child and INDIRECT cloud status
* log child kill on windows
* fixes for spawn server on windows to ensure we have a valid pid and the process is properly terminated
* better handling to windows processes exit code
* pass the cloud url from parents to children
* add retries and timeout to claiming curl request
* remove FILE * from plugins.d
* spawn-tester to unittest spawning processes communication
* spawn-tester now tests FILE pointer I/O
* external plugins run in posix mode
* set blocking I/O on all pipes
* working spawn server on windows
* latest changes in spawn_popen applied to linux tools
* push environment
* repeated tests of fds
* export variable CYGWIN_BASE_PATH
* renamed to NETDATA_CYGWIN_BASE_PATH
* added cmd and help to adapt the command and the information to be presented to users during claiming
* split spawn server versions into files
* restored spawn server libuv based
* working libuv based spawn server
* fixes in libuv for windows
* working spawn server based on posix_spawn()
* fix fd leads on all spawn servers
* fixed windows spawn server
* fix signal handling to ensure proper cooperation with libuv
* switched windows to posix_spawn() based spawn server
* improvement on libuv version
* callocz() event loop
* simplification of libuv spawn server
* minor fixes in libuv and spawn tester
* api split into parts and separated by version; introduced /api/v3; no changes to old /api/v1 and /api/v2
* completed APIs splitting
* function renames
* remove dead code
* split basic functions into a directory
* execute external plugins in nofork spawn server with posix_spawn() for improved performance
* reset signals when using posix_spawn()
* fix spawn server logs and log cmdline in posix server
* bearer_get_token() implemented as function
* agent cloud status now exposes parent claim_id in indirect mode
* fixes for node id streaming from parent to children
* extract claimed id to separate file
* claim_id is no longer in host structure; there is a global claim_id for this agent and there are parent and origin claim ids in host structure
* fix issue on older compilers
* implement /api/v3 using calls from v1 and v2
* prevent asan leaks on local-sockets callback
* codacy fixes
* moved claim web api to web/api/v2
* when the agent is offline, prefer indirect connection when available; log a warning when a node changes node id
* improve inheritance of claim id from parent
* claim_id for bearer token show match any of the claim ids known
* aclk_connected replaced with functions
* aclk api can now be limited to node information, implementing [cloud].scope = license manager
* comment out most options in stream.conf so that internal defaults will be applied
* respect negative matches for send charts matching
* hidden functions are not accessible via the API; bearer_get_token function checks the request is coming from Netdata Cloud
* /api/v3/settings API
* added error logs to settings api
* saving and loading of bearer tokens
* Fix parameter when calling send_to_plugin
* Prevent overflow
* expose struct parser and typedef PARSER to enforce strict type checking on send_to_plugin()
* ensure the parser will not go away randomly from the receiver - it is now cleared when the receiver lock is acquired; also ensure the output sockets are set in the parser as long as the parser runs
* Add newline
* Send parent claim id downstream
* do not send anything when nodeid is zero
* code re-organization and cleanup
* add aclk capabilities, nodes summary and api version and protection to /api/v2,3/info
* added /api/v3/me which returns information about the current user
* make /api/v3/info accessible always
* Partially revert "remove netdata-claim.sh from everywhere, except kickstart scripts"
Due to how we handle files in our static builds and local builds, we
actually need to continue installing `netdata-claim.sh` to enable a
seamless transition to the new claiming mechanims without breaking
compatibility with existing installs or existing automation tooling that
is directly invoking the claiming script.
The script itself will be rewritten in a subsequent commit to simply
wrap the new claiming methodology, together with some additional changes
to ensure that a warning is issued if the script is invoked by anything
other than the kickstart script.
* Rewrite claiming script to use new claiming method.
* Revert "netdata installer writes claiming info to /etc/netdata/claim.conf"
Same reasoning as for 2e27bedb3fbf9df523bff407f2e8c8428e350e38.
We need to keep the old claiming support code in the kickstart script
for the forseeable future so that existing installs can still be
claimed, since the kickstart script is _NOT_ versioned with the agent.
A later commit will add native support for the new claiming method and
use that in preference to the claiming script if it appears to be
available.
* Add support for new claiming method to kickstart.sh.
This adds native support to the kickstart script to use the new claiming
method without depending on the claiming script, as well as adding a few
extra tweaks to the claiming script to enable it to better handle the
transition.
Expected behavior is for the kickstart script to use the new claiming
code path if the claiming script is either not installed, or does not
contain the specific string `%%NEW_CLAIMING_METHOD%%`. This way we will
skip the claiming script on systems which have the updated copy that
uses the new claiming approach, which should keep kickstart behavior
consistent with what Netdata itself supports.
* Depend on JSON-C 0.14 as a minimum supported version.
Needed for uint64 functions.
* Fix claiming option validation in kickstart script.
* do not cache auth in web client
* reuse bearer tokens when the request to create one matches an existing
* dictionaries dfe loops now allow using return statement
* bearer token files are now fixed for specific agents by having the machine guid of the agent in them
* systemd journal now respects facets and disables the default facets when not given
* fixed commands.c
* restored log for not openning config file
* Fix Netdata group templating for claiming script.
* Warn on failed templating in claiming script.
* Make `--require-cloud` a slient no-op.
We don’t need to warn users that it does nothing, we should just have ti
do nothing.
* added debugging info to claiming
* log also the response
* do not send double / at the url
* properly remove keyword from parameters
* disable debug during claimming
* fix log messages
* Update packaging/installer/kickstart.sh
* Update packaging/installer/kickstart.sh
* implemented POST request payload parsing for systemd-journal
* added missing reset of facets in json parsing
* JSON payload does not need hashes any more. I can accept the raw values
---------
Co-authored-by: Ilya Mashchenko <ilya@netdata.cloud>
Co-authored-by: Austin S. Hemmelgarn <austin@netdata.cloud>
Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com>
Co-authored-by: Austin S. Hemmelgarn <ahferroin7@gmail.com>
* Add code signing for Windows executables.
* Fix typos and add failure notification.
* Use full version for trusted signing action.
Because MS isn’t publishing it with proper semver tags.
* Avoid reinstalling dependencies that are already installed.
* Fix CMake 3.30 compatibility.
* Don’t let BUILD_DIR propagate to cmake.
* Fix JSON-C build warning.
* Fix handling of externally specified build directories.
While regular Windows paths do actually work under MSYS2, they seem to
confuse CMake, so we need to convert to a standard MSYS2 path if
`BUILD_DIR` is set to a Windows path.
* Fix typo.
* Fix build directory handling.
* uninstall_netdata_service: Move content to function
* uninstall_netdata_service: Add error check and show info for users
* uninstall_netdata_service: Modify helper to manage service
* uninstall_netdata_service: Stop agent before start installation
* Fix up CMake feature handling for Windows.
* Better handle detection of Go on Windows.
* Provide Windows copy of Go for Windows build.
* Explicitly set GOROOT in environment.
* Explicitly disable Prometheus remote write exporter.
* Add note about DEFAULT_FEATURE_STATE_OPTION.
* Initial Windows CI support.
This also adds a proper set of production-focused packaging scripts
for working with building Netdata on Windows.
* Pivot to using existing tooling and fixing that up as required.
* General script cleanup.
* Auto-detect location of repository based on script location.
This makes the scripts properly independent of the current working
directory, allowing them to be more resilient to possible changes in the
build environment.
* Make dependency handling entirely separate from build.
* Remove development-only packages from dependency list.
They only waste time in CI, and it’s reasonable to assume that anybody
who needs them can install them themselves, just like we do with all
other platforms.
* Use absolute path for makensis.
This way it actually works without needing manual modification of the
environment.
* Add PowerShell scripts to invoke the build and package scripts.
* Clean up build directory handling.
* Fix fetching of msys2 installer.
* Further cleanup.
* Tidy up build output
* Provide proper output grouping in GitHub Actions.
* Use correct command name for makensis.
* Fix installer script path.
* Do not try to install MSYS2 using Chocolatey.
* Fix CPU architecture check.
* Add more detailed messages for MSYS2 install.
* Fix release fetching logic.
* Further release check logic fixes.
* Complete Windows CI support.
* Move second update pass into dependency handling script.
* Fix handling of icon for Windows installer.
* Default to using a working configuration for release builds.
And provide a way to pass extra options to CMake without needing to
modify the script.
* Fix expansion of optionally defined variable.
