mirror of
https://github.com/netdata/netdata.git
synced 2025-05-18 15:11:22 +00:00

* cleanup of logging - wip
* first working iteration
* add errno annotator
* replace old logging functions with netdata_logger()
* cleanup
* update error_limit
* fix remanining error_limit references
* work on fatal()
* started working on structured logs
* full cleanup
* default logging to files; fix all plugins initialization
* fix formatting of numbers
* cleanup and reorg
* fix coverity issues
* cleanup obsolete code
* fix formatting of numbers
* fix log rotation
* fix for older systems
* add detection of systemd journal via stderr
* finished on access.log
* remove left-over transport
* do not add empty fields to the logs
* journal get compact uuids; X-Transaction-ID header is added in web responses
* allow compiling on systems without memfd sealing
* added libnetdata/uuid directory
* move datetime formatters to libnetdata
* add missing files
* link the makefiles in libnetdata
* added uuid_parse_flexi() to parse UUIDs with and without hyphens; the web server now read X-Transaction-ID and uses it for functions and web responses
* added stream receiver, sender, proc plugin and pluginsd log stack
* iso8601 advanced usage; line_splitter module in libnetdata; code cleanup
* add message ids to streaming inbound and outbound connections
* cleanup line_splitter between lines to avoid logging garbage; when killing children, kill them with SIGABRT if internal checks is enabled
* send SIGABRT to external plugins only if we are not shutting down
* fix cross cleanup in pluginsd parser
* fatal when there is a stack error in logs
* compile netdata with -fexceptions
* do not kill external plugins with SIGABRT
* metasync info logs to debug level
* added severity to logs
* added json output; added options per log output; added documentation; fixed issues mentioned
* allow memfd only on linux
* moved journal low level functions to journal.c/h
* move health logs to daemon.log with proper priorities
* fixed a couple of bugs; health log in journal
* updated docs
* systemd-cat-native command to push structured logs to journal from the command line
* fix makefiles
* restored NETDATA_LOG_SEVERITY_LEVEL
* fix makefiles
* systemd-cat-native can also work as the logger of Netdata scripts
* do not require a socket to systemd-journal to log-as-netdata
* alarm notify logs in native format
* properly compare log ids
* fatals log alerts; alarm-notify.sh working
* fix overflow warning
* alarm-notify.sh now logs the request (command line)
* anotate external plugins logs with the function cmd they run
* added context, component and type to alarm-notify.sh; shell sanitization removes control character and characters that may be expanded by bash
* reformatted alarm-notify logs
* unify cgroup-network-helper.sh
* added quotes around params
* charts.d.plugin switched logging to journal native
* quotes for logfmt
* unify the status codes of streaming receivers and senders
* alarm-notify: dont log anything, if there is nothing to do
* all external plugins log to stderr when running outside netdata; alarm-notify now shows an error when notifications menthod are needed but are not available
* migrate cgroup-name.sh to new logging
* systemd-cat-native now supports messages with newlines
* socket.c logs use priority
* cleanup log field types
* inherit the systemd set INVOCATION_ID if found
* allow systemd-cat-native to send messages to a systemd-journal-remote URL
* log2journal command that can convert structured logs to journal export format
* various fixes and documentation of log2journal
* updated log2journal docs
* updated log2journal docs
* updated documentation of fields
* allow compiling without libcurl
* do not use socket as format string
* added version information to newly added tools
* updated documentation and help messages
* fix the namespace socket path
* print errno with error
* do not timeout
* updated docs
* updated docs
* updated docs
* log2journal updated docs and params
* when talking to a remote journal, systemd-cat-native batches the messages
* enable lz4 compression for systemd-cat-native when sending messages to a systemd-journal-remote
* Revert "enable lz4 compression for systemd-cat-native when sending messages to a systemd-journal-remote"
This reverts commit b079d53c11
.
* note about uncompressed traffic
* log2journal: code reorg and cleanup to make modular
* finished rewriting log2journal
* more comments
* rewriting rules support
* increased limits
* updated docs
* updated docs
* fix old log call
* use journal only when stderr is connected to journal
* update netdata.spec for libcurl, libpcre2 and log2journal
* pcre2-devel
* do not require pcre2 in centos < 8, amazonlinux < 2023, open suse
* log2journal only on systems pcre2 is available
* ignore log2journal in .gitignore
* avoid log2journal on centos 7, amazonlinux 2 and opensuse
* add pcre2-8 to static build
* undo last commit
* Bundle to static
Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>
* Add build deps for deb packages
Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>
* Add dependencies; build from source
Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>
* Test build for amazon linux and centos expect to fail for suse
Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>
* fix minor oversight
Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>
* Reorg code
* Add the install from source (deps) as a TODO
* Not enable the build on suse ecosystem
Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>
---------
Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>
Co-authored-by: Tasos Katsoulas <tasos@netdata.cloud>
147 lines
5.2 KiB
C
147 lines
5.2 KiB
C
// SPDX-License-Identifier: GPL-3.0-or-later
|
|
|
|
#define WEB_SERVER_INTERNALS 1
|
|
#include "web_server.h"
|
|
|
|
WEB_SERVER_MODE web_server_mode = WEB_SERVER_MODE_STATIC_THREADED;
|
|
|
|
// --------------------------------------------------------------------------------------
|
|
|
|
WEB_SERVER_MODE web_server_mode_id(const char *mode) {
|
|
if(!strcmp(mode, "none"))
|
|
return WEB_SERVER_MODE_NONE;
|
|
else
|
|
return WEB_SERVER_MODE_STATIC_THREADED;
|
|
|
|
}
|
|
|
|
const char *web_server_mode_name(WEB_SERVER_MODE id) {
|
|
switch(id) {
|
|
case WEB_SERVER_MODE_NONE:
|
|
return "none";
|
|
default:
|
|
case WEB_SERVER_MODE_STATIC_THREADED:
|
|
return "static-threaded";
|
|
}
|
|
}
|
|
|
|
// --------------------------------------------------------------------------------------
|
|
// API sockets
|
|
|
|
LISTEN_SOCKETS api_sockets = {
|
|
.config = &netdata_config,
|
|
.config_section = CONFIG_SECTION_WEB,
|
|
.default_bind_to = "*",
|
|
.default_port = API_LISTEN_PORT,
|
|
.backlog = API_LISTEN_BACKLOG
|
|
};
|
|
|
|
void debug_sockets() {
|
|
BUFFER *wb = buffer_create(256 * sizeof(char), NULL);
|
|
int i;
|
|
|
|
for(i = 0 ; i < (int)api_sockets.opened ; i++) {
|
|
buffer_strcat(wb, (api_sockets.fds_acl_flags[i] & WEB_CLIENT_ACL_NOCHECK)?"NONE ":"");
|
|
buffer_strcat(wb, (api_sockets.fds_acl_flags[i] & WEB_CLIENT_ACL_DASHBOARD)?"dashboard ":"");
|
|
buffer_strcat(wb, (api_sockets.fds_acl_flags[i] & WEB_CLIENT_ACL_REGISTRY)?"registry ":"");
|
|
buffer_strcat(wb, (api_sockets.fds_acl_flags[i] & WEB_CLIENT_ACL_BADGE)?"badges ":"");
|
|
buffer_strcat(wb, (api_sockets.fds_acl_flags[i] & WEB_CLIENT_ACL_MGMT)?"management ":"");
|
|
buffer_strcat(wb, (api_sockets.fds_acl_flags[i] & WEB_CLIENT_ACL_STREAMING)?"streaming ":"");
|
|
buffer_strcat(wb, (api_sockets.fds_acl_flags[i] & WEB_CLIENT_ACL_NETDATACONF)?"netdata.conf ":"");
|
|
netdata_log_debug(D_WEB_CLIENT, "Socket fd %d name '%s' acl_flags: %s",
|
|
i,
|
|
api_sockets.fds_names[i],
|
|
buffer_tostring(wb));
|
|
buffer_reset(wb);
|
|
}
|
|
buffer_free(wb);
|
|
}
|
|
|
|
void api_listen_sockets_setup(void) {
|
|
int socks = listen_sockets_setup(&api_sockets);
|
|
|
|
if(!socks)
|
|
fatal("LISTENER: Cannot listen on any API socket. Exiting...");
|
|
|
|
if(unlikely(debug_flags & D_WEB_CLIENT))
|
|
debug_sockets();
|
|
|
|
return;
|
|
}
|
|
|
|
|
|
// --------------------------------------------------------------------------------------
|
|
// access lists
|
|
|
|
SIMPLE_PATTERN *web_allow_connections_from = NULL;
|
|
int web_allow_connections_dns;
|
|
|
|
// WEB_CLIENT_ACL
|
|
SIMPLE_PATTERN *web_allow_dashboard_from = NULL;
|
|
int web_allow_dashboard_dns;
|
|
SIMPLE_PATTERN *web_allow_registry_from = NULL;
|
|
int web_allow_registry_dns;
|
|
SIMPLE_PATTERN *web_allow_badges_from = NULL;
|
|
int web_allow_badges_dns;
|
|
SIMPLE_PATTERN *web_allow_mgmt_from = NULL;
|
|
int web_allow_mgmt_dns;
|
|
SIMPLE_PATTERN *web_allow_streaming_from = NULL;
|
|
int web_allow_streaming_dns;
|
|
SIMPLE_PATTERN *web_allow_netdataconf_from = NULL;
|
|
int web_allow_netdataconf_dns;
|
|
|
|
void web_client_update_acl_matches(struct web_client *w) {
|
|
w->acl = WEB_CLIENT_ACL_NONE;
|
|
|
|
if (!web_allow_dashboard_from ||
|
|
connection_allowed(w->ifd, w->client_ip, w->client_host, sizeof(w->client_host),
|
|
web_allow_dashboard_from, "dashboard", web_allow_dashboard_dns))
|
|
w->acl |= WEB_CLIENT_ACL_DASHBOARD;
|
|
|
|
if (!web_allow_registry_from ||
|
|
connection_allowed(w->ifd, w->client_ip, w->client_host, sizeof(w->client_host),
|
|
web_allow_registry_from, "registry", web_allow_registry_dns))
|
|
w->acl |= WEB_CLIENT_ACL_REGISTRY;
|
|
|
|
if (!web_allow_badges_from ||
|
|
connection_allowed(w->ifd, w->client_ip, w->client_host, sizeof(w->client_host),
|
|
web_allow_badges_from, "badges", web_allow_badges_dns))
|
|
w->acl |= WEB_CLIENT_ACL_BADGE;
|
|
|
|
if (!web_allow_mgmt_from ||
|
|
connection_allowed(w->ifd, w->client_ip, w->client_host, sizeof(w->client_host),
|
|
web_allow_mgmt_from, "management", web_allow_mgmt_dns))
|
|
w->acl |= WEB_CLIENT_ACL_MGMT;
|
|
|
|
if (!web_allow_streaming_from ||
|
|
connection_allowed(w->ifd, w->client_ip, w->client_host, sizeof(w->client_host),
|
|
web_allow_streaming_from, "streaming", web_allow_streaming_dns))
|
|
w->acl |= WEB_CLIENT_ACL_STREAMING;
|
|
|
|
if (!web_allow_netdataconf_from ||
|
|
connection_allowed(w->ifd, w->client_ip, w->client_host, sizeof(w->client_host),
|
|
web_allow_netdataconf_from, "netdata.conf", web_allow_netdataconf_dns))
|
|
w->acl |= WEB_CLIENT_ACL_NETDATACONF;
|
|
|
|
w->acl &= w->port_acl;
|
|
}
|
|
|
|
|
|
// --------------------------------------------------------------------------------------
|
|
|
|
void web_server_log_connection(struct web_client *w, const char *msg) {
|
|
ND_LOG_STACK lgs[] = {
|
|
ND_LOG_FIELD_U64(NDF_CONNECTION_ID, w->id),
|
|
#ifdef ENABLE_HTTPS
|
|
ND_LOG_FIELD_TXT(NDF_SRC_TRANSPORT, SSL_connection(&w->ssl) ? "https" : "http"),
|
|
#else
|
|
ND_LOG_FIELD_TXT(NDF_SRC_TRANSPORT, "http"),
|
|
#endif
|
|
ND_LOG_FIELD_TXT(NDF_SRC_IP, w->client_ip),
|
|
ND_LOG_FIELD_TXT(NDF_SRC_PORT, w->client_port),
|
|
ND_LOG_FIELD_END(),
|
|
};
|
|
ND_LOG_STACK_PUSH(lgs);
|
|
|
|
nd_log(NDLS_ACCESS, NDLP_DEBUG, "[%s]:%s %s", w->client_ip, w->client_port, msg);
|
|
}
|