185 Commits

Author SHA1 Message Date
Markus Mäkelä
05cb49d48a MXS-1220: Add error messages to bad request
Requests now contain a JSON API-conforming, human-readable error message.
2017-05-17 10:14:57 +03:00
Markus Mäkelä
551f635f50 MXS-1220: Make the http.hh header public
As it contains utility functions for formatting time_t values to HTTP-date
values, there's no real need to make it an internal header.
2017-05-17 10:14:56 +03:00
Markus Mäkelä
8e2c7fd952 Merge branch '2.0' into 2.1 2017-05-10 09:12:42 +03:00
Markus Mäkelä
778631a860 MXS-1220: Add support for conditional HTTP requests
The REST API now supports the If-Modified-Since, If-Unmodified-Since,
If-Match and If-None-Match headers and returns the correct response if the
conditional fails.

Added tests for the date parsing and expanded the HTTP header tests in the
REST API test suite.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
3deb497394 MXS-1220: Add Last-Modified and ETag headers
The resource system now tracks both the time when a resource was last
modified and the revision number of the resource. This allows working
Last-Modified and ETag headers to be generated by the REST API.

The If-Modified-Since and If-None-Match request headers are not yet
processed and using them will always return the resource instead of a 304
Not Modified response.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
461cd6afd9 MXS-1220: Add creation of listeners via REST API
Listeners can now be created via the REST API by doing a POST request to
the service listener resource.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
efc5461daa MXS-1220: Return 204 No Content for PUT and POST request
Returning 204 No Content removes the cost of always sending back the
modified resource. If the modified resource is required, a GET request
should be made to retrieve it.

Updated tests to account for this change.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
a384665141 MXS-1220: Allow modification of logging options
The logging options can now be modified with a PUT request.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
0e57bec4ef MXS-1220: Add threads resource
The threads are now a REST API resource exposed via the /maxscale/threads
resource collection.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
ac21443529 MXS-1220: Make modules proper resources
The /maxscale/modules/ endpoint is now a resource collection and the
/maxscale/modules/:module endpoint exposes the individual resources.
2017-05-09 15:32:42 +03:00
Markus Mäkelä
624434a6d4 MXS-1220: Compare headers case-insensitively
The header names are not case sensitive.
2017-05-09 15:32:41 +03:00
Markus Mäkelä
b2e94fc73c MXS-1220: Add versioned URLs
Added versioning to URLs. This should allow somewhat safe modification to
the API after it has been finished.
2017-05-09 15:32:41 +03:00
Esa Korhonen
ee20191645 KILL [CONNECTION | QUERY] support, part2B
Various small changes to part2, as suggested by comments and otherwise.
Mostly renaming, working logic should not change.

Exception: session id changed to 64bit in the container and associated
functions. Another commit will change it to 64bit in the session itself.
2017-05-08 09:58:02 +03:00
Esa Korhonen
17f6e94cba KILL [CONNECTION | QUERY] support, part2
MySQL sessions are added to a hasmap when created, removed when closed.
MYSQL_COM_PROCESS_KILL is now detected, the thread_id is read and the kill
command sent to all worker threads to find the correct session. If found, a
fake hangup even is created for the client dcb.

As is, this function is of little use since the client could just disconnect
itself instead. Later on, additional commands of this nature will be  added.
2017-05-08 09:51:07 +03:00
Johan Wikman
f66620c89c Accept auto_ptr<T> where T is derived type
Without the member template it is not possible to pass an auto_ptr
instantiated with a derived type to post() or broadcast().

The reason is that the conversion constructor and conversion
operator of auto_ptr are equally good for that purpose, and hence
the compilation ends with an error.
2017-05-08 09:36:16 +03:00
Johan Wikman
6c4a4a3ee0 Do not inherit WorkerDisposableTask from WorkerTask
WorkerDisposableTask is intended to be passed using auto_ptr, to
make it plain that the ownership is moved. If it's inherited from
WorkerTask it can be passed as a WorkerTask, which is confusing as
it in that case will not be disposed of.
2017-05-04 14:06:44 +03:00
Johan Wikman
5de5609692 Ensure messages are a multiple of 8
The compiler will anyway insert padding to ensure the size of the
structure is a multiple of 8. That will under Valgrind show up as
writes of uninitialized data when the messages are sent over the
pipe.

Rather than forcing the structure to be aligned on 4 byte boundary
we simply expand the id field to an 8 byte type.
2017-05-04 13:09:19 +03:00
Markus Mäkelä
dcbd91deee Fix filters parameter processing
The filter names for the service parameter `filters` weren't converted
into the new format. This caused a configuration error when a filter with
significant whitespace in its name was used in a service.
2017-05-04 09:17:53 +03:00
Markus Mäkelä
afff5e98b3 MXS-1220: Expose module information via the REST API
The modules, their types and default values are exposed via the
/maxscale/modules resource. Currently, only a list of resources can be
exposed as the externally exposed module object (MXS_MODULE) does not have
the name and type information in it.
2017-05-04 09:17:51 +03:00
Markus Mäkelä
bf44cd0d14 MXS-1220: Add HTTPS support
The REST API now supports encryption. The user needs to configure
certificates for the REST API before encryption is used.
2017-05-04 09:17:51 +03:00
Markus Mäkelä
d248c7e081 MXS-1220: Add PUT support for services
Service parameters can now be altered with a PUT request to the REST
API. This allows general level parameters to be altered.

Module specific parameters need to be altered with a different mechanism,
namely the module command system. This requires that a generic way to call
a function needs to be devised.
2017-05-04 09:17:49 +03:00
Markus Mäkelä
d282b14b36 Allow runtime changes to service parameters
General service parameters can now be altered at runtime.
2017-05-04 09:17:49 +03:00
Markus Mäkelä
f7fefad2e6 Improve persisted configuration handling
When a persisted configuration file is read, the values in it are
considered to be more up-to-date than the ones in the main configuration
file. This allows all objects to be persisted in a more complete form
making it easier to change configuration values at runtime.

This change is intended to help make runtime alterations to services
possible.
2017-05-04 09:17:49 +03:00
Markus Mäkelä
1e1c4abcb7 MXS-1220: Add PUT support for monitors
Monitor resources can now be altered with a PUT request. The method allows
alterations on all parameters that the maxadmin `alter monitor` command
allows.
2017-05-04 09:17:48 +03:00
Markus Mäkelä
cd6e0ab5e9 MXS-1220: Show parameters for services, filters and monitors
The service, filter and monitor resources now have a "parameters" value
which contains a set of all configuration parameters for that object. This
set contains both standard and non-standard parameters.

Also fixed a mistake in the constant name definitions for the monitor
parameters "events" and "script".
2017-05-04 09:17:47 +03:00
Markus Mäkelä
50eafe19fe MXS-1220: Add self links to all resources
A self link to the resource itself provides a convenient way for the
client to request a resource, modify it and call the self link to update
it. This removes some of the burden on the client to keep track of the
resource links by placing these in the resource itself.
2017-05-04 09:14:04 +03:00
Markus Mäkelä
47d819b008 MXS-1220: Make HTTP request handler parameters const
The HttpRequest can be const as the request itself will not be modified.
2017-05-04 09:14:04 +03:00
Markus Mäkelä
043ed95bdf MXS-1220: Add PUT support for servers
The server can now be modified with a PUT request of a modified server
resource. The server resource was reorganized to have the parameters as a
separate entity from the other more general entities of the resource.

The PUT/POST functions return a more appropriate error message when no
request body is provided.

Moved some of the constant names used in server.cc into the config.h
header.
2017-05-04 09:14:04 +03:00
Markus Mäkelä
9aa4138705 MXS-1220: Add POST handling for monitors
Doing a POST on the /monitors/ resource now creates a new monitor.
2017-05-04 09:14:03 +03:00
Markus Mäkelä
c17c451fb5 MXS-1220: Move header generation back to HttpResponse
The actual list of headers is not known when the request is first
generated. This prevents the headers from being generated in admin.cc
which handles things on a lower level.

The moving of the header generation is done with the OPTIONS method in
mind. This header needs to be generated inside the RootResource class
which manages the navigation of the resources.
2017-05-04 09:14:03 +03:00
Markus Mäkelä
ebc9e4bd3b MXS-1220: Clean up resource, request and response headers
Cleaned up various parts of the resource, request and response class
headers.

Moved `using` declarations into .cc files.

Made the Resource class non-copyable as it isn't really meant to be
copied.
2017-05-04 09:14:03 +03:00
Markus Mäkelä
80104d6dad MXS-1220: Add POST handling for servers
New servers can now be created by POSTing a new server definition to the
/servers/ resource.
2017-05-04 09:14:03 +03:00
Markus Mäkelä
46344b204a MXS-1220: Properly handle request with data
The data was not processed correctly and instead an error was sent to the
client as soon as the request arrived.

Created a class that somewhat abstracts the internals of the client
request processing.
2017-05-04 09:14:03 +03:00
Markus Mäkelä
978af12a91 MXS-1220: Simplify resource path handling
The resources are now declared by simply typing the path that they match
and providing a callback for that path. This makes it easier to add new
resources without having to create a new class for each added resource
endpoint. It also removes some of the pathname processing that was done
inside the child class handler functions.
2017-05-04 09:12:16 +03:00
Markus Mäkelä
52e075963e MXS-1220: Reorganize request and response processing
The standard response headers are now generated at a higher level. This
reduces the scope of the HttpResponse class making it a leaner wrapper
around a few simple variables, namely the JSON body of the response.

The HttpRequest now exposes the Host header that the client sent. This
allows resource relations to be real links that work without modification.
2017-05-04 09:12:16 +03:00
Markus Mäkelä
d242203279 MXS-1220: Use libmicrohttpd for the HTTP handling
The HTTP side of the REST API is better handled with an actual
library. The libmicrohttpd library provides a convenient way of handling
the HTTP traffic between the clients and MaxScale.
2017-05-04 09:12:15 +03:00
Markus Mäkelä
caf2172677 MXS-1220: Fix minor problems with admin thread
The admin thread now uses blocking IO. This is not optimal but it
simplifies the code by some amount.

Fixed option processing removing one extra character from key name.

Use correct member variable when checking for the option map end.
2017-05-04 09:11:18 +03:00
Markus Mäkelä
900bf2db5a MXS-1220: Take the resource handler into use
The resource handler system is now usable but it doesn't perform anything
useful. Although, this will allows it to be tested for correctness.

Minor fixes to HttpResponse output and renaming of functions.
2017-05-04 09:10:33 +03:00
Markus Mäkelä
8b1c0cd1a1 MXS-1220: Add REST resource handler class
The Resource class is intended to be an abstraction of a resource
tree. Each node in the tree can perform actions. The tree is traversed
depth first so that deeper command paths resolve to the correct nodes.

Currently all the base resources defined in the REST API documents are
implemented in a way that they return a 200 OK response to all
requests. When the internal data can be represented as JSON, the resources
can be hooked up to functions that generate JSON.
2017-05-04 09:10:33 +03:00
Markus Mäkelä
e248178349 MXS-1220: Split requested resource into parts
When a client requests a resource, the HttpRequest class now splits the
requested resource into parts. This should help with the resource
validation and navigation.

Added test that checks that the resources are correctly split into the
correct number of arguments and that the argument contents are correct.
2017-05-04 09:10:33 +03:00
Markus Mäkelä
34ee4a1997 MXS-1220: Add option parsing
The options of a request are now parsed and exposed by the HttpRequest
class.

Added tests for the request options.

Also added missing error handling of invalid requests.
2017-05-04 09:10:33 +03:00
Markus Mäkelä
9d0d394361 MXS-1220: Expand HttpResponse class
The class now generates default headers. The ETag and Last-Modified tags
do not represent any actual modification time or resource hash.

The basic functionality of the HTTP responses is tested by the core test
suite. More advanced testing of the whole REST API is still required.

Removed the static `create` functions as only the JSON parsing version
could generated errors and even then the errors were unlikely. By
replacing the static creator function with a normal constructor, the
HttpResponse class can now also be created on the stack making its use
easier.
2017-05-04 09:10:33 +03:00
Markus Mäkelä
a73d3e9276 MXS-1220: Create HTTP response class
The HTTP response class simplifies the response creation. The next step is
to add generation of all the default headers that are needed by the REST
API.
2017-05-04 09:10:32 +03:00
Markus Mäkelä
55b52b8ab1 MXS-1220: Add HEAD method support
The HEAD method was not in the list of supported methods.
2017-05-04 09:10:32 +03:00
Markus Mäkelä
c937457738 MXS-1220: Add request body parsing
The HTTP request body is expected to be a valid JSON object. All other
requests are considered malformed requests and result in a HTTP 400 error.

Added the Jansson license to the LICENSE-THIRDPARTY.TXT file. Imported
some of the tests from the Jansson test suite to the HttpParser test.
2017-05-04 09:10:32 +03:00
Markus Mäkelä
4eb121ce35 MXS-1220: Rename and reorganize HttpParser
The HttpParser class was renamed to HttpRequest as it parses and processes
only HTTP requests. A second class that creates a HTTP response needs to
be created to handle the response generation.

Moved some of the HTTP constants and helper functions to a separate
http.hh header.
2017-05-04 09:10:32 +03:00
Markus Mäkelä
e34b65658e MXS-1220: Simplify admin request handling
The admin requests are now processed in blocking mode. The timing out of
connecttions is handled by a specific timeout thread that checks the state
of each admin request.

The simplification will help with the JSON parsing with PUT/POST
commands. If non-blocking IO is used, the network reading code and JSON
parsing needs a lot more work to handle partial reads.

If the administrative interface requires higher performance and
concurrency, a multi-threaded solution could be created.
2017-05-04 09:10:32 +03:00
Markus Mäkelä
605fed7839 MXS-1220: Add HTTP parser
The HTTP parser parses HTTP/1.1 messages into easily manageable data
structures. This should make it easier to map the HTTP requests into
actual commands in MaxScale.
2017-05-04 09:10:31 +03:00
Markus Mäkelä
23b6fb3e6d MXS-1220: Create separate admin thread
When MaxScale is started, a separate thread for the administrative
interface is started. This allows the worker threads to handle client
requests while the administrative thread handles the lower priority
administrative requests.

The administrative interface responds to all request with a 200 OK HTTP
response. This allows the administrative interface itself to be tested.
2017-05-04 09:10:31 +03:00
Markus Mäkelä
dd8a10f466 Remove old polling message system
The old polling message system is obsolete now that the worker messages
are implemented. The old system was only used to clean up the persistent
connection pool of a server.
2017-05-03 14:16:35 +03:00