Commit Graph

6392 Commits

Author SHA1 Message Date
30927455ef Add each DCB to the owning thread's list
Each DCB needs to be added to the owning thread's list so that they can be
iterated through. As sessions always have a client DCB, the sessions don't
need to be added to a similar per thread list.

This change fixes a problem with dcb_hangup_foreach that the removal of
the list manager introduced. Now the hangup events are properly injected
for the DCBs that connect to the server in question.
2016-11-30 10:28:23 +02:00
fe56e65903 Use service capabilities in response processing
The MySQLBackend protocol now only checks for complete packets if the
service requires statement based routing. This should remove unnecessary
processing when data is only streamed from the backend to the client.
2016-11-30 10:28:22 +02:00
abc0681248 Temporarily disable listmanager debug checks
Since the listmanager code isn't used, the debug assertions will always
fail. They should be disabled until the listmanager code can converted to
the per-thread model.
2016-11-30 10:28:22 +02:00
51842333d7 Temporarily take out the listmanager from use
The listmanager code uses a global spinlock which might cause problems
with multiple threads.
2016-11-30 10:28:22 +02:00
68d3fc1092 Remove the DCB write queue locking
Since only one thread can append to the DCBs write queue at a time, there
is no need to lock it.
2016-11-30 10:28:22 +02:00
076b810c1e Create thread specific zombie queues
Because each thread has their own epoll file descriptor and only one
thread can process a DCB, it makes sense to move to a per thread zombie
queue. This removes one of the last restrictions on scalability.
2016-11-30 10:28:22 +02:00
f77f78360e Insert fake events under a lock
The thread-specific spinlock needs to be acquired before a fake event is
inserted from a non-polling thread. The usual situation is when a monitor
thread inserts a hangup event for a DCB.

Other, less common cases are when session timeouts have been enabled and
the DCB needs to be closed. Here it is better to insert a fake hangup
event instead of directly closing the DCB from an external thread.
2016-11-30 10:28:22 +02:00
8efdaa1ea6 Move fake events to a thread-specific queue
The fake poll events are now stored in thread specific queues. This
removes the need for the poll event queue.
2016-11-30 10:28:22 +02:00
b79210c760 Create thread specific epoll instances
Having a unique epoll instance for each thread allows a lot of the locking
from poll.c to be removed. The downside to this is that each session can
have only one thread processing events for it which might reduce
performance with very low client counts.
2016-11-30 10:28:22 +02:00
573faff987 Add module command documentation for avrorouter
The documentation lists the one command it implements.
2016-11-30 10:28:16 +02:00
9175295542 Cache: Now also CacheRules is shared
Unnecessary methods were also removed from CachePT and CacheMT
as it does not make sense to create more than one single instance
of those per filter instance. Consequently there is no need for
them to be able to use an existing StorageFactory (and CacheRules).
2016-11-30 10:22:53 +02:00
93103fd64a Cache: shared_ptr used with CacheRules and StorageFactory
shared_ptr is now used for managing the lifetime of CacheRules
and StorageFactory instances.
2016-11-30 10:22:53 +02:00
d52145054f Cache: Now use CacheRules instead of CACHE_RULES 2016-11-30 10:22:53 +02:00
2e4ac55aa4 cache: Add CacheRules class
Easier to use in a C++ context than the C API
2016-11-30 10:22:53 +02:00
2a2c530857 Cache: Trivial cleanup of inmemory storage 2016-11-30 10:22:53 +02:00
2423031df8 Cache: Use lock guard for locking spinlocks
Now the locks will always be release, also in the presence of an
unantecipated exception.
2016-11-30 10:22:53 +02:00
efa8713b17 Cache: Include inmemory storage in build
Add logging note to storage_rocksdb at the same time.
2016-11-30 10:22:53 +02:00
23bb7c4de7 Cache: Add in memory storage
Storage implementation that simply uses std::unordered_map as
storage structure.
2016-11-30 10:22:53 +02:00
b87ea735fb Cache: Correct examples in documentation 2016-11-30 10:22:53 +02:00
e5a9eceff4 Cache: Fix prev/next twiddling 2016-11-30 10:22:53 +02:00
9c1b9c188e Cache: Enable LRU eviction
The maximum count and maximum size of the cache can now be
specified and a storage can declare what capabilities it has.
If a storage modile cannot enforce the maximum count or maximum
size limits, the storage is decorated with an LRU storage that
can.
2016-11-30 10:22:53 +02:00
e597523c47 Cache: Allow storage modules to specify capabilites
Will be used to decide whether a LRUStorage facade is needed in front
of the actual storage or not.
2016-11-30 10:22:53 +02:00
1701979696 Cache: Add LRU storage
The LRUStorage hierarchy implements the decorator pattern and
is capable of providing LRU behaviour to any underlying storage.
There are two concrete classes - LRUStorageST and LRUStorageMT -
that can be used in single thread and multiple thread contentx,
respectively.

Also tests the convention of placing _ as suffix on member
variables.
2016-11-30 10:22:53 +02:00
5d5b2c9d6c Binlog file is checked at max scale startup if encryption is enabled
Binlog file is checked at max scale startup if encryption is enabled.

The check might fail while calculating next pos or verifying event type.
A message is reported
2016-11-30 09:01:42 +01:00
abd73f3700 blr_read_binlog checks replication header after decryption
blr_read_binlog can now check the  replication header after decryption,
for encrypted events.

Added a small fix for slave server requesting position of
START_ENCRYPTION_EVENT: new pos points to first encrypted event.
2016-11-29 17:57:32 +01:00
c7c68410be Merge branch 'develop' into binlog_server_waitdata_encryption 2016-11-29 17:38:04 +01:00
07c602d81c Add creation and destruction of monitors
Monitors can now be created and destroyed at runtime. The configurations
for new monitors are persisted to disk.
2016-11-29 15:04:19 +02:00
4ff4e69592 Add stopping and starting of listeners to maxadmin
Listeners can now be stopped and started individually.
2016-11-29 15:04:19 +02:00
38e4a5902f MXS-576 Check for negative values for config settings
Previously, negative values were allowed for persistpoolmax and
persistmaxtime. Now they cause an error. Also, monitor_interval
allowed negative (or zero) values, which were then implicitly cast to
unsigned, causing unintended behaviour. Now this causes a warning
and the default value is used.
2016-11-29 14:53:27 +02:00
9ede657a9b Change error message when permissions on .secrets are wrong
If the user running MaxScale could open the .secrets-file and the
file permissions were anything other than owner:read, the
secrets_readkeys() would fail with error message
"Ignoring secrets file <path>, invalid permissions." Now the
message is more accurate in stating the expected permissions.
2016-11-29 14:52:30 +02:00
ecdeb009b3 Fix dbfwfilter regression
The query throttling functionality was broken by the move to the per
thread rule system. One way to fix this is to move the session specific
throttling information into the client session itself. This allows for a
simpler system with no direct dependencies on the rules or users
themselves.
2016-11-29 14:19:08 +02:00
075968e947 Allow avrorouter converstion to be started and stopped
The modulecmd functionality allows the avrorouter to easily control the
conversion process with one command. The conversion can now be started and
stopped by the user.

This also fixes a bug where the conversion would stop if there were no
binlog files present when the service was started.
2016-11-29 14:19:08 +02:00
c01430e3fc Fix module commands with no arguments
Passing zero as the argument count would lead to a malloc for zero
bytes which would always cause a failure.
2016-11-29 14:19:08 +02:00
cc54d80a8b Fix memory leaks in maxinfo (modified for 2.0 and develop-branch)
MXS-1009. This commit adds a gwbuf_free after maxinfo_execute() to
free a buffer with an sql-query after it has been processed. Also,
the parse tree in maxinfo_execute_query() is now freed. The tree_free-
function was renamed to maxinfo_tree_free, since it is now globally
available.

This commit has additional changes (in relation to the 1.4.4 branch)
to remove errors caused by differences in the html and sql-sides of
MaxInfo.
2016-11-29 14:11:12 +02:00
05303f5b7e Fix memory leaks in maxinfo (modified for develop-branch)
MXS-1009. This commit adds a gwbuf_free after maxinfo_execute() to
free a buffer with an sql-query after it has been processed. Also,
the parse tree in maxinfo_execute_query() is now freed. The tree_free-
function was renamed to maxinfo_tree_free, since it is now globally
available.

This commit has additional changes (in relation to the 1.4.4 branch)
to remove errors caused by differences in the html and sql-sides of
MaxInfo.
2016-11-29 13:08:20 +02:00
241fa67cfc AES_CBC is now supported for binlog encryption
AES_CBC can be used for binlog files encryption

The AES_CBC could leaves some not handled bytes in the buffer and those
need a special encoding (ECB and XOR)

This way the output buffer of the whole encoding with AES_CBC will have
same size as the input (AES_CTR does it without any other step)
2016-11-29 11:09:11 +01:00
0bc68742a6 Do error handling in one big locked action
This is not the optimal way to do error handling but it should solve all
problems that could rise from the multi-threaded model of MaxScale.

By taking a lock at the start of handleError, we'll be able to modify the
dcb error handling flag in a thread-safe manner. This should prevent
double error handling for all DCBs.
2016-11-29 12:07:12 +02:00
f8f400bdfd Only close the errored DCB if it is not in use
It was theoretically possible that a DCB was closed while it was still in
use.
2016-11-29 12:07:12 +02:00
8dae8136aa Add more logging to solve MXS-956
If a DCB is about to be closed but is has already been closed, a message
is logged. This will allow us to figure out exactly where the DCB is
closed.
2016-11-29 12:07:12 +02:00
fa58fdcde8 Cache: Make Storage abstract
Storage is now abstract with StorageReal (that uses an actual
storage module) derived from it. This to prepare for a stackable
case where LRU behaviour is implemented in front of the actual
physcal storage.
2016-11-29 11:23:23 +02:00
b4ddf23ba3 Cache: Now uses unordered_map instead of HASHTABLE 2016-11-29 11:23:23 +02:00
02265c5dc2 Cache: All functions now named using snake case
Exception is where the call goes across a plugin boundary, as
is the current MaxScale convention.
2016-11-29 11:23:23 +02:00
a2daacd465 Cache: Add options for choose sharing mode
Now possible to use same cache for all threads or separate cache
for each.
2016-11-29 11:23:23 +02:00
933e09d245 Cache: Add per thread Cache
The per thread cache is implemented so that there is one single
thread cache per thread.

This uses shared_ptr, which is available in different places
depending on which GCC version is default on the target platform.
So, before this is merged to develop, that needs to be sorted out.
2016-11-29 11:23:23 +02:00
910a349f86 Cache: Store name as string and not as char* 2016-11-29 11:23:23 +02:00
4f1f71a05a Cache: Make config and storage factory sharable
In antecipation of a CachePT (cache per thread), rearrange things
so that the storage factory and config can be shared between cache
instances.
2016-11-29 11:23:23 +02:00
c3319b24a6 Cache: Wrongly protected methods made public 2016-11-29 11:23:23 +02:00
124453b756 Cache: Add simple cache
A simple cache is one that does not transparently use multiple
storages in the background. That will be the case when a separate
cache per thread is used. Both CacheST and CacheMT are now derived
from CacheSimple.

A fair amount of what used to be in Cache has now been moved to
CacheSimple. What remains is what surely is common for all cache
types.
2016-11-29 11:23:23 +02:00
5bae5b8b8d Cache: Add cache for single thread 2016-11-29 11:23:23 +02:00
279bf7e2fa Cache: Change key from char ptr to struct 2016-11-29 11:23:23 +02:00