2695 Commits

Author SHA1 Message Date
Markus Mäkelä
e3c4bd7f72 MXS-1220: Add more validation checks to request JSON
The requests that send a body should define at least a `data` member.

Added a simple test that checks that bad requests are rejected. This test
should be expanded to check that the returned error body contains the
correct members.
2017-05-24 15:05:10 +03:00
Markus Mäkelä
75e7ac35ed MXS-1220: Always build libmicrohttpd
This allows MaxScale to use the PATCH functionality that's only in the
newer versions of the library. It also removes some of the problems that
exist with older systems that don't support all of the options.
2017-05-24 15:05:10 +03:00
Esa Korhonen
dbfd631fed Change session registry to a template class
The template class wraps a HashMap such that only a few operations
are allowed. Usage requires specializing a RegistryTraits class
template for each entry type.
2017-05-19 10:16:37 +03:00
Markus Mäkelä
322983a5f4 Merge branch '2.1' into develop 2017-05-17 13:48:52 +03:00
Markus Mäkelä
f1c790db52 MXS-1220: Add logging priorities to logs resoure
The logging priorities can now be queried and altered.
2017-05-17 10:14:57 +03:00
Markus Mäkelä
05cb49d48a MXS-1220: Add error messages to bad request
Requests now contain a JSON API-conforming, human-readable error message.
2017-05-17 10:14:57 +03:00
Markus Mäkelä
ac29abd679 MXS-1220: Add support for X-HTTP-Method-Override
This header allows clients to override the request method to work around
client library limitations.
2017-05-17 10:14:57 +03:00
Markus Mäkelä
adbf537d80 MXS-1220: Add more information to the /maxscale/ resource
The /maxscale/ resource now has values for most of the configuration
parameters in the [maxscale] section as well as the version, commit and
uptime information.
2017-05-17 10:14:56 +03:00
Markus Mäkelä
551f635f50 MXS-1220: Make the http.hh header public
As it contains utility functions for formatting time_t values to HTTP-date
values, there's no real need to make it an internal header.
2017-05-17 10:14:56 +03:00
Markus Mäkelä
56cf06ee08 Don't use explicit struct initialization
Older GCC versions don't seem to support explicit struct member
initialization.
2017-05-16 11:39:57 +03:00
Markus Mäkelä
159c8bb5d1 Listen for only EPOLLIN events with listeners
The example in the epoll(7) manpage only sets the EPOLLIN event flag.

Although it is not explicitly stated that only EPOLLIN events arrive for
sockets that are listening, any other types of events should not be
relevant for listeners.
2017-05-15 19:32:02 +03:00
Markus Mäkelä
4117dcf410 Only store established connections in the pool
If a connection has not been fully established (i.e. authentication has
been completed) then it should not be considered as a connection pool
candidate.
2017-05-15 19:32:02 +03:00
Markus Mäkelä
4050cd20d8 Merge branch '2.1-merge-2.0' into 2.1 2017-05-15 10:36:58 +03:00
Markus Mäkelä
ed653ac729 Merge branch '2.0' into 2.1-merge-2.0 2017-05-14 10:25:12 +03:00
Markus Mäkelä
18c71a8ebd MXS-1220: Fix build failure with older GCC
Older GCC versions seem to have a bug where `struct sockaddr_in` pointers
cannot be accessed directly and need to be dereferenced first.
2017-05-14 10:07:16 +03:00
Markus Mäkelä
ffc6dba720 MXS-1220: Fix error on PUT request with no parameters
Whena server was modified and no parameters were given, the operation was
reported as a failure even though it was successful.
2017-05-14 10:06:02 +03:00
Markus Mäkelä
ca7b24f6fa MXS-1220: Allow admin interface to be disabled
Allowing the admin interface to be disabled completely makes it possible
to remove any security concerns that could arise from its use.
2017-05-14 10:06:02 +03:00
Markus Mäkelä
70db289c60 MXS-1216: Fix DATETIME(n) value interpretation
The DATETIME(n) values generated by a MariaDB 10.0 server were not
interpreted correctly as the wrong algorithm was used to extract the
values.

DATETIME(0) values still do not work properly and they require further
debugging and changes to the code.
2017-05-12 16:50:41 +03:00
Markus Mäkelä
30bd869f1c MXS-1216: Fix crash on MariaDB 10.0 DATETIME(n)
When a MariaDB 10.0 DATETIME field with a custom length was defined, the
field offsets weren't calculated properly.

As there is no metadata for pre-10.1 DATETIME types with decimal
precision, the metadata (i.e. decimal count) needs to be gathered from the
CREATE TABLE statement. This information is then used to calculate the
correct field length when the value is decoded.

This change does not fix the incorrect interpretation of the old DATETIME
value. The converted values are still garbled due to the fact that the
value needs to be shifted out of the decimal format before it can be
properly converted.
2017-05-12 11:22:04 +03:00
Markus Mäkelä
6c583c43dd MXS-1263: Fix timeouts for partially established connections
When the connection timeout was checked for a connection, it assumed that
only valid and fully established sessions should be timed out.

Taking into account the fact that connections can be idle even before the
session is fully established, the check should be expanded to all
connections regardless of the session state.
2017-05-11 09:31:59 +03:00
Esa Korhonen
aebe839990 Change session id to 64bit
The server internal session id may be larger than 4 bytes (MariaDB uses 8)
but only 4 are sent in the handshake. The full value can be queried
from the server, but this query is not supported by MaxScale yet. In any
case, both the protocol and MXS_SESSION now have 64 bit counters. Only the
low 32 bits are sent in the handshake, similar to server.
2017-05-10 13:26:25 +03:00
Markus Mäkelä
8e2c7fd952 Merge branch '2.0' into 2.1 2017-05-10 09:12:42 +03:00
Markus Mäkelä
157b7f853b MXS-1220: Fix resource self links
The top level resource self links pointed to the collection instead of the
resource itself. The individual resoures now also have a links field that
contains the self link to the resource. This should make navigation of the
API easier as all objects have valid links in them.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
9fee283a3f MXS-1220: Add /maxscale/tasks resource
The resource shows all active tasks inside MaxScale.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
613b924f2e Add listener iterator
The listener iterator hides the details of the listener iteration behind a
small set of functions.

The following for-loop demonstrates the main use-case for the iterator:

    LISTENER_ITERATOR iter;

    for (SERV_LISTENER *listener = listener_iterator_init(service, &iter);
         listener; listener = listener_iterator_next(&iter))
    {
        /** Do something with the listener */
    }

As the listeners are mostly iterated to either check a fact about them or
print their information, the functions cater to that use-case. For this
reason, they should always be allocated off the stack.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
c30b817abc MXS-1220: Add warnings for invalid request JSON
Converted info level log messages to warnings and added a missing warning
to server creation.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
778631a860 MXS-1220: Add support for conditional HTTP requests
The REST API now supports the If-Modified-Since, If-Unmodified-Since,
If-Match and If-None-Match headers and returns the correct response if the
conditional fails.

Added tests for the date parsing and expanded the HTTP header tests in the
REST API test suite.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
3deb497394 MXS-1220: Add Last-Modified and ETag headers
The resource system now tracks both the time when a resource was last
modified and the revision number of the resource. This allows working
Last-Modified and ETag headers to be generated by the REST API.

The If-Modified-Since and If-None-Match request headers are not yet
processed and using them will always return the resource instead of a 304
Not Modified response.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
717f883839 Make service ports lock-free
The service port list is now iterated in a safe and lock-free manner. This
makes the handling of service ports somewhat simpler since once an item
has been added to the list it will never be removed. It also removes the
need to lock the service when checking whether a service listens on a port
which caused potential deadlocks.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
461cd6afd9 MXS-1220: Add creation of listeners via REST API
Listeners can now be created via the REST API by doing a POST request to
the service listener resource.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
3e1ff70d7d MXS-1220: Respond with 200 OK to root level requests
If a request to the `/` resource is made, the API responds with an 200
OK. This is done to make it possible to use the HTTP health check
mechanism found in many cloud load balancers.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
e3ff3b56bc MXS-1220: Add test for /maxscale/logs/ endpoint
The test alters logging options and flushes the logs.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
efc5461daa MXS-1220: Return 204 No Content for PUT and POST request
Returning 204 No Content removes the cost of always sending back the
modified resource. If the modified resource is required, a GET request
should be made to retrieve it.

Updated tests to account for this change.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
a384665141 MXS-1220: Allow modification of logging options
The logging options can now be modified with a PUT request.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
1cb2783106 MXS-1220: Implement /maxscale/logs resource
The /maxscale/logs resource exposes information about the state of logging
in MaxScale.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
d4212b9c78 MXS-1220: Add request option test
Currently only the `pretty=true` option is supported but it is also
tested.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
16e597d43e MXS-1220: Expand REST API test suite
Added tests for services and extended the monitor test suite to test
actions on monitors.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
0e57bec4ef MXS-1220: Add threads resource
The threads are now a REST API resource exposed via the /maxscale/threads
resource collection.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
ac21443529 MXS-1220: Make modules proper resources
The /maxscale/modules/ endpoint is now a resource collection and the
/maxscale/modules/:module endpoint exposes the individual resources.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
08dca8c273 MXS-1220: Treat service listeners as service sub-resources
The listeners under the /services/:service/listeners collection are now
fully JSON API compliant resources.

The listeners could also be exposed as a /listeners collection to easily
group all listener type resources in one place. This approach does has
some semantical and practical problems, namely the fact that each listener
has a many-to-one relationship with its service and listeners by
themselves can't exist alone.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
1d98b4b67b MXS-1220: Return 403 Forbidden for invalid requests
The JSON API specification suggests that the API returns the 403 Forbidden
error when the user does an invalid request. The 400 Bad Request isn't the
ideal error for cases where the syntax is correct but the action being
performed is wrong.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
798ee840a8 MXS-1220: Make /maxscale/ resource conform to JSON API
The /maxscale/ resource now conforms to the JSON API and a test checks
that it the returned JSON is valid.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
7ce20b75d7 MXS-1220: Extend and fix REST API tests
Fixed a few broken tests and extended the monitor tests.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
18b52adaeb MXS-1220: Fix minor bug in monitor creation
The JSON validation function wasn't using the JSON Pointer function for
all values.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
c1968aac2f MXS-1220: Migrate create/update monitor to JSON API format
The creation and modification of moitor now supports the JSON API
conforming format generated by the GET endpoints.

Also added tests for creating and altering monitors via the REST API.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
9495438f2b MXS-1220: Add server test
The test creates, alters and destroys servers.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
a3c683ab87 MXS-1220: Simplify test creation
The tests now automatically start MaxScale before each test block and stop
it and perform cleanup after the test. This is done by simply calling the
`before.sh` and `after.sh` scripts before each test block.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
6b0fabf834 MXS-1220: Make REST API locally testable
The `test_rest_api` make target creates a discardable installation of
MaxScale which is used to launch a local instance of MaxScale. This local
instance is then used to test the REST API.

This is definitely not an efficient way to test the MaxScale but it allows
local testing without virtual machines or containers.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
6c220a1151 MXS-1220: Migrate create/update server to JSON API format
The creation and modification of servers now supports the JSON API
conforming format generated by the GET endpoints.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
2f8db4ec1a MXS-1220: Add JSON Pointer support for json_t objects
Using the JSON Pointer syntax specified in RFC 6901
(https://tools.ietf.org/html/rfc6901) allows for a convenient way to
access values deep in a JSON object.
2017-05-09 15:32:41 +03:00