62 Commits

Author SHA1 Message Date
VilhoRaatikka
e57944ef44 Bug #399, http://bugs.skysql.com/show_bug.cgi?id=399
Corrected the length variable including length of timestampt string which doesn't include terminating null.
2014-02-11 14:55:16 +02:00
VilhoRaatikka
75af5ee03c Merge https://github.com/skysql/MaxScale 2014-02-07 11:11:47 +02:00
VilhoRaatikka
e8ce2268e0 Test system for MaxScale.
Every test/makefile have the following targets:
 cleantests 	- clean local and subdirectories' tests
 buildtests	- build all local and subdirectories' tests
 runtests	- run all local tests
 testall	- clean, build and run local and subdirectories' tests

Tests for directory random_dir are always in its subdirectory, in this case in random_dir/test . If random_dir has subdirectories with tests, random_dir/child_dir, for example, tests of child_dir can be started from random_dir/test/makefile where make -C child_dir/test <test target> is called.

See MAXSCALE_HOME/test/README for further information.
2014-02-07 11:06:37 +02:00
Mark Riddoch
3edea0afc4 Updated gitignore files 2014-02-05 14:49:05 +01:00
vraatikka
d700754baa log_manager.cc
Bug #372, http://bugs.skysql.com/show_bug.cgi?id=372
	Do not exceed the buffer capacity in log writing. Now longer strings are cut to fit to the buffer.
2013-12-13 15:21:48 +02:00
vraatikka
272ca036e4 log_manager.h
Added macros which allow for testing locally if log is enabled or not.

server/Makefile
	Added directory creation command to Makefile. This fixes problem where errmsg.sys was copied to MaxScale/mysql instead of MaxScale/mysql/errmsg.sys

dcb.b
	Added external variable which carries the information of enabled logs from log_manager.cc

gateway.c
	Tuned logging.
2013-12-12 07:05:38 +02:00
vraatikka
f50de7a084 build_gateway.inc
Removed MARIADB_SRC_PATH
	Added MYSQL_ROOT - root directory where MariaDB headers are installed
	Added MYSQL_HEADERS - -I notation for three header directories
	Added EMBEDDED_LIB - for embedded library directory
	Added ERRMSG - for errmsg.sys file

log_manager.cc
	Little fixes

log_manager/makefile
	Updated header include directive

makefile.inc
	Removed DEBUGGER and BACKGR as unnecessary
	Added LIB for embedded library file name

query_classifier/makefile
	Updated header include directives and embedded library directory

query_classifier/query_classifier.cc
	clean up

server/core/Makefile
	Updated header include directives and embedded library directory
	Added libaio and install of errmsg.sys

server/core/dcb.c
	clean up

server/core/gateway.c
	Added --language and --skip-innodb to mysql_library_init command-line arguments list
	clean up

server/core/load_utils.c
	clean up

server/modules/monitor/Makefile
server/modules/routing/readwritesplit/Makefile
	Updated header include directives and embedded library directory

utils/skygw_debug.h
	http://bugs.skysql.com/show_bug.cgi?id=369

utils/skygw_types.h
utils/skygw_utils.cc
	clean up
2013-12-06 00:12:41 +02:00
vraatikka
258bcb27da log_manager.cc
Added global variable lm_enabled_logfiles_bitmask which holds a bit for each enbaled logfile. It is updated when situation changes. 

gateway.c
	Changed gateway.c:main so that command-line argument -f specifies either the name of config file in $MAXSCALE_HOME/etc or exact location and name of it. Updated README correspondingly.
	Added eternal variable lm_enabled_logfiles_bitmask to be used for quick check before calling logging library functions.
2013-12-04 09:56:48 +02:00
vraatikka
41298a09df Bugzilla entry # 363:
skygw_thread_t and simple_mutex_t make own copy of name argrument. Changed init calls to both accordingly.
2013-11-27 10:51:45 +02:00
vraatikka
4057ae0b0f log_manager.cc
renamed file_exists to file_exists_and_is_writable, added missing file open flags.

gateway.c	
	Removed command-line parameter '-m' because setting LD_LIBRARY_PATH after the program has started has no effect.
	Corrected some comments.
2013-11-26 16:52:48 +02:00
vraatikka
bc97b7f5ec log_manager.cc
external optind variable must be reset before using it because getopt may have been called earlier in the same process and without resetting argument parsing fails.

dcb.c
	check dcb state before attempting to write to dcb in dcb_write

gateway.c
	daemon_mode moved to global so that it can be taken into account when deciding where to print messages.
	added get_config_filename by using home directory name given as a parameter. Returns absolute paths to config file and to home directory regardless of home directory argument was relative of absolute.
	In main, arguments are parsed by using getopt.
	Changed argument '-c' to specify home directory because MaxScale.cnf is always in <home>/etc/ directory.
	Added argument '-m' to specify modules directory. Both arguments override any other settings.

skygw_types.h
	Added PATH_MAX
2013-11-26 12:53:46 +02:00
vraatikka
3b0d6c23ad log_manager.cc
Commented why spread-down logging is disabled. 
	In MaxScale start, log paths are printed in their complete form to make it easier to copy-paste the names.

gateway.c
	Replaced print_signal_set_error with more general-purpose print_log_n_stderr which prints non-formatted messages to error log and to stderr to the point MaxScale switches to run in daemon process. After that only error log is printed.

skyge_utils.cc
	polish
2013-11-25 10:49:45 +02:00
vraatikka
1c96824cb3 log_manager.cc
Declared program_invocation_name, and program_invocation_short_name. They are used as identifier strings in syslog logging in cases where log_manager is not initialized explicitly and/or the caller hasn't specified program name as one of the arguments in arguments array.

	Fixed memory corruption. Length of sequence number of file name was calculated to as one too short and one byte was written to unallocated memory.

	Freed also the linkpath in cases where at least one of the log files is written to shm and a symlink is added to log directory.

mysql_client.c
	Added debug check for protocol pointer because of memory issues in mysql_client.c
2013-11-13 22:08:10 +02:00
vraatikka
517b6a5ee3 Added argument '-m' to skygw_logmanager_init which takes the identity string for syslog as additional argument. If syslog is enabled in configuration, but identity is not specified, host process name is used. 'maxscale', for example. 2013-11-11 11:02:05 +02:00
vraatikka
b996631fc9 log_manager.cc
Added syslog support. There's a new argument '-l' which takes log file ids as additional arguments. Writes to those logs will be written to syslog as well. syslog write causes additional overhead. Thus, writing frequently may become a bottleneck.

gateway.c
	Set LOGFILE_ERROR, and LOGFILE_MESSAGE to be log files whose writes will be copied to syslog too.
2013-11-10 20:45:48 +02:00
vraatikka
9ba7a0d955 log_manager.cc
Added new argument '-s' which takes additional argument composed of list of logfile identifiers. Logfiles listed with '-s' will be written on main memory instead of disk. In practice, the log file in question will be written in /dev/shm but corresponding symlink is added to log directory. In the case of name conflicts with log files and links, a differentiating sequence number is included in hte name of the file. This, however, is done only when existing file is not writable or is of different type (symlink <> file). 
	Added new logfile LOGFILE_DEBUG whose contents will be largerly what was included up to date in trace log. 

	Disabled feature which spreads writes to log files to others because of bug (#338) in the way block buffers are managed.

	Changed log manager parameters to match with the current implementation. List of arguments:
                "-h - help\n"
                "-a <debug prefix>   ............(\"skygw_debug\")\n"
                "-b <debug suffix>   ............(\".log\")\n"
                "-c <trace prefix>   ............(\"skygw_trace\")\n"
                "-d <trace suffix>   ............(\".log\")\n"
                "-e <message prefix> ............(\"skygw_msg\")\n"
                "-f <message suffix> ............(\".log\")\n"
                "-g <error prefix>   ............(\"skygw_err\")\n"
                "-i <error suffix>   ............(\".log\")\n"
                "-j <log path>       ............(\"/tmp\")\n"
                "-s <shmem log file ids> ........(no default)\n";

dcb.c
	dcb_add_to_zombieslist, add dcb to the front of zombies list instead of inserting to the end of it.

gateway.c
	Renamed shutdown_gateway to shutdown_server (Bug #131)
	Call skygw_logmanager_init so that trace and debug logs are written to shared memory.

poll.c 
dcb.h
	Removed some dead code and references to unneeded mutexes.

debugcmd.c
	Added enable/disable log command for debug log.

skygw_utils.cc
	skygw_file_init now takes optional symlink name as a second argument. Symlink is created to point to the file being created.
2013-11-08 12:56:39 +02:00
vraatikka
cd507f1461 log_manager.cc
blockbuf_get_writepos, when all existing buffers in blockbuf list are full, a new block buffer is created and added to the list. Adding to the list is done with mutex on hold. Mutex shouldn't be freed before the next iteration in while loop, which expects that bblist mutex is on hold. At the end of the function, removed an unnecessary debug assertion.

testlog.c
	Enabled more intensive write test. Replaced TRUE and FALSE with true, false, respectively.

skygw_utils.cc
	simple_mutex_unlock, added debug assertion to ensure that pthread mutex's user counter is always at least 0 (it goes negative in double free).
2013-10-10 16:46:51 +03:00
vraatikka
80b67d1083 log_manager.cc:
State update for filewriter was missing and that caused Maxscale to fail if opening of any log file failed.
dcb.c:
	Added EAGAIN and EWOULDBLOCK handling to dcb_read. If dcb_close is called for freshly created dcb, dcb is only freed.
gateway.c:
	Added file_write_footer and write_footer of which the latter is called at exit time. It simply draws a line to screen.
gw_utils.c:
	Some macros for helping comparison between gw_read_gwbuff and dcb_read, which overlap.
poll.c:
	Some macros to help enable/disable mutexing in poll_waitevents
service.c:
	Check return value of listen and session_alloc and behave accordingly.

mysql_client.c:
	If ioctl returned successfully with b==0 it earlier caused closing the client and backend dcbs. Since that doesn't reliably indicate that client has closed socket on its side, Maxscale doesn't close its sockets either.
mysql_common.c:
	In gw_receive_backend_auth, if dcb_read returns n==0, it is not considered as an error anymore. The implemented behavior is not yet complete and correct. Result should be successful but the protocol state shouldn't change to MYSQL_IDLE before backend return is received.
	In gw_send_authentication_to_backend protocol state was always set to MYSQL_AUTH_RECV even if gw_rwite had failed. Now, return value is read and state is set in caller's context basen on the return value.
skygw_utils.cc:
	Removed ss_dassert from skyge_file_init because it prevented from returning meaningful error meassage to the client.:
2013-10-06 22:31:32 +03:00
vraatikka
849a366e95 log_manager.cc :
tuned error printing and log writing output format
dcb.c : 
	dcb_connect, check return value of poll_add_dcb and behave accordingly.
	dcb_write, in case of SIFPIPE, only write to trace log.
	dcb_close, dassert with incorrect dcb states.
gateway.c :
	added file_write_header to print header similar than in logs to stderr.
	main, add signal handler for SIGPIPE
poll.c : 
	poll_remove_dcb, don't fail if dcb is in NOPOLLING or in ZOMBIE states.
	poll_waitevents, write EPOLLHUPs to trace log, don't even attempt to write to closed socket.
readconnection.h : 
	shortened comment.
readwritesplit.h : 
	replaced generic names with more specific ones. 
httpd.c : 
	Check listen return value and behave accordingly.
mysql_backend.c : 
	 Tiny clean up.
mysql_client.c : 
	gw_MySQLListener, Check listen return value and behave accordingly. 
mysql_common.c : 
	Shortened a header.
telnetd.c : 
	telnetd_listen, check listen return value and behave accordingly.
readconnroute.c : 
	Tuned log writing format.
readwritesplit.c : 
	Added function search_backend_servers, which chooses suitable backend and master server among those known by Maxscale. Fixed clean-up routines. Not ready yet but works somehow.
testroute.c : 
	Cleanup.
skygw_utils.cc : 
	Log writing clean up.
2013-10-04 12:06:44 +03:00
vraatikka
15adf134c4 Defined LOGFILE_DEBUG for future use. If there will be separate logfile for debug output then macro will be removed. Currently equals with LOGFILE_TRACE. 2013-09-12 22:01:05 +03:00
vraatikka
b86d3f3dfd Holding error log's block buffer registration until all logs were written caused deadlock if block buffers were full. Now buffer registrations don't overlap. 2013-09-02 10:30:50 +03:00
vraatikka
34b26ad85f Changed enum-type boolean to whta is defined in stdbool.h, and in practise replaced TRUEs and FALSEs with true, and false, respectively. 2013-08-28 22:47:14 +03:00
vraatikka
76de8f3758 log manager now spreads log writings 'down' so that LOGFILE_ERROR logs are duplicated in LOGFILE_MESSAGE, and in LOGFILE_TRACE, and LOGFILE_MESSAGE logs are duplicated in LOGFILE_TRACE. 2013-08-20 10:43:54 +03:00
vraatikka
a5dd900acb Added test cases for speading log messages 2013-08-19 12:58:36 +03:00
vraatikka
4e63df8df8 Disabled spreading of log writes temporarily. 2013-08-19 11:31:53 +03:00
vraatikka
c2743e7543 Logfiles are ordered by their id as follows:
LOGFILE_ERROR   = 1
LOGFILE_MESSAGE = 2
LOGFILE_TRACE   = 4

What is written to file n, will be written to all files m, m>n. Applies to user log writes. Some internal messages are such that they apply to one specific file, such as enable/disable notifications.
2013-08-19 10:28:28 +03:00
vraatikka
4360ae16b8 Moved timestamp string creation to skygw_utils where it can be provide service for others too. 2013-08-12 13:09:29 +03:00
vraatikka
43fb6b87f1 Enabled trace log in DEBUG=Y builds. Modified testlog-c accordingly. 2013-08-09 10:01:34 +03:00
vraatikka
0ad25ba3ab Added functions skygw_log_enable(logfile_id_t) and skygw_log_disable(logfile_id_t) to Log manager API. By calling them log writing to any of predefined log files can be switched on or off.
Added simple test to testlog.c
2013-08-08 23:38:00 +03:00
vraatikka
234b8c40a3 Cleaned up some debug printings (to stdout). 2013-08-05 18:15:41 +03:00
vraatikka
b852079631 Enabled debug assertions for DEBUG=Y builds. Fixed one debug assertion in log manager. 2013-08-05 17:47:21 +03:00
vraatikka
ee68633c22 Removed unnecessary line 2013-08-05 10:25:45 +03:00
vraatikka
cae4d38a7e Removed unnecessary argument from log manager commands because it is not used and it is always NULL. 2013-08-04 23:30:47 +03:00
vraatikka
9fb072ea74 Changed log manager APChanged log manager APII 2013-08-04 23:14:48 +03:00
vraatikka
d2efbcf867 Changed log manager API. Removed unnecessary memory context pointer (always NULL):
bool skygw_logmanager_init(void** buf, int argc, char* argv[]) >>
bool skygw_logmanager_init(int argc, char* argv[])

void skygw_logmanager_done(void** buf) >>
void skygw_logmanager_done(void)

void skygw_log_done(void* writebuf) >>
void skygw_log_done(void)

int  skygw_log_write(void* writebuf, logfile_id_t id, char* format, ...) >>
int  skygw_log_write(logfile_id_t id, char* format, ...)

int  skygw_log_write_flush(void* writebuf, logfile_id_t id, char* format, ...) >>
int  skygw_log_write_flush(logfile_id_t id, char* format, ...)

--- static functions :
static bool logmanager_init_nomutex(void** p_ctx, int argc, char* argv[]) >>
static bool logmanager_init_nomutex(int argc, char* argv[])

static void logmanager_done_nomutex(void** ctx) >>
static void logmanager_done_nomutex(void)
2013-08-04 23:07:54 +03:00
vraatikka
25d26eb3b8 Corrected a typo in line feed handling. 2013-07-27 12:23:47 +03:00
vraatikka
7a6129861e Added timestamp to log file headers and to the beginning of each line. Removed additional line feeds.
Example:

SkySQL MaxScale Sat Jul 27 12:13:10 2013
------------------------------------------
2013 07/27 12:13:10  Loaded module testroute: V1.0.0
2013 07/27 12:13:10  Initialise debug CLI router module V1.0.1.
2013 07/27 12:13:10  Loaded module debugcli: V1.0.1
2013-07-27 12:14:30 +03:00
vraatikka
b566c41067 Log manager sometimes failed to flush after skygw_log_write_flush because log client signaled filw writer thread too early. Fixed so that write is done first and file writer is registered after that. Except in cases where skygw_log_flush was called. Then only flush is triggered.
Added a few trivial cases to test.
2013-07-26 23:06:12 +03:00
vraatikka
3989615197 Code polishing. 2013-07-26 21:46:23 +03:00
vraatikka
ff8b432943 Added run-time path to dynamically loadable libraries; '-Wl,rpath,$(DEST)/lib' to makefile. When compiling, use
make DEST=<path to library directory>
2013-07-26 19:20:06 +03:00
vraatikka
2f189cb153 Log manager erroneously wrote string termination character after every log message.
Added a few trivial test cases.
2013-07-26 15:21:54 +03:00
vraatikka
52564314d4 Documented concurrency control between filewriter thread and log clients.
Changed log header text, replaced SkySQL GAteway with SkySQL MaxScale.
Fixed bug in log flushing. skygw_log_write_flush didn't cause call of fsync, which suspended file writing.
2013-07-25 16:21:13 +03:00
vraatikka
baed0e846f Changed log_manager to use block-siuze buffers instead of small write buffers. Added new test cases and added iterations. Added possibility to test with dummy disk write which sleeps constantly for 5ms instead of performing disk write. 2013-07-23 08:34:43 +03:00
Mark Riddoch
bce6f88efe Add depend make target and resolve conflict in query_classifier.cc 2013-07-19 15:05:21 +02:00
Mark Riddoch
53c6d6daa6 Updates for make install rules 2013-07-17 13:31:18 +02:00
vraatikka
3d3ba6f986 Division by zero :-) 2013-07-09 12:13:34 +03:00
vraatikka
bce2573c21 Fixed bug which tried to access list without mutex. 2013-07-02 18:04:00 +03:00
vraatikka
6abd9dbdaa Added profile counters: writebuffer init, done, total and recycle counts. 2013-07-02 16:53:23 +03:00
vraatikka
d11741130d Changed log manager API to support variable length formatted log strings. New API is as follows:
/** No change in these */
bool skygw_logmanager_init(void** buf, int argc, char* argv[]);
void skygw_logmanager_done(void** buf);
void skygw_logmanager_exit(void);
int  skygw_log_flush(logfile_id_t id);

/** writebuf remains unused, but formatted string is now possible and in case 
 * of formatted string, arbitrary long argument list is supported too. Max 
 * length for a log string is defined to BUFSIZ, whose value depends on the 
 * system but typically is 4/8KB.
 */
int  skygw_log_write(void* writebuf, logfile_id_t id, char* format, ...);
int  skygw_log_write_flush(void* writebuf, logfile_id_t id, char* format, ...);

makefile.inc includes new CFLAG : SS_PROF, which is set if PROF=Y on make command line or in build_gateway.inc .
ss_debug.h includes corresponding ss_prof(exp) macro which equals to exp if SS_PROF is defined and to empty if in other case.

mlist_t now includes datadel function which is a callback and it is executed for mlnode_data on node exit.
2013-07-02 14:46:39 +03:00
vraatikka
1888b8a517 Prevented redefinition of bool, added function void skygw_logmanager_exit(void) to be called by atexit 2013-06-28 11:39:00 +03:00