7466 Commits

Author SHA1 Message Date
Markus Mäkelä
695a4032ca Use common constants for monitor and server parameters
The parameter names for monitors and servers now use a set of constant
names. This removes some of the errors caused by spelling mistakes when
the same parameter name is repeated in multiple places.

The service, filter and listener parameters should also be converted to
constants. This allows for a consistent user experience.
2017-05-04 09:14:04 +03:00
Markus Mäkelä
9aa4138705 MXS-1220: Add POST handling for monitors
Doing a POST on the /monitors/ resource now creates a new monitor.
2017-05-04 09:14:03 +03:00
Markus Mäkelä
076599ee5e MXS-1220: Make the parameters of the diagnostic entry points const
The diagnostic entry points should not modify the state of the object
being diagnosed.
2017-05-04 09:14:03 +03:00
Markus Mäkelä
c17c451fb5 MXS-1220: Move header generation back to HttpResponse
The actual list of headers is not known when the request is first
generated. This prevents the headers from being generated in admin.cc
which handles things on a lower level.

The moving of the header generation is done with the OPTIONS method in
mind. This header needs to be generated inside the RootResource class
which manages the navigation of the resources.
2017-05-04 09:14:03 +03:00
Markus Mäkelä
bc3cfe0221 MXS-1220: Fix memory leaks
Some of the JSON objects created in the diagnostic functions leaked
memory.
2017-05-04 09:14:03 +03:00
Markus Mäkelä
ebc9e4bd3b MXS-1220: Clean up resource, request and response headers
Cleaned up various parts of the resource, request and response class
headers.

Moved `using` declarations into .cc files.

Made the Resource class non-copyable as it isn't really meant to be
copied.
2017-05-04 09:14:03 +03:00
Markus Mäkelä
80104d6dad MXS-1220: Add POST handling for servers
New servers can now be created by POSTing a new server definition to the
/servers/ resource.
2017-05-04 09:14:03 +03:00
Markus Mäkelä
46344b204a MXS-1220: Properly handle request with data
The data was not processed correctly and instead an error was sent to the
client as soon as the request arrived.

Created a class that somewhat abstracts the internals of the client
request processing.
2017-05-04 09:14:03 +03:00
Markus Mäkelä
3d2219e8ef MXS-1220: Add missing relationships
The relationships from servers to services and monitors and filters to
services were not implemented. Now each server lists the services and
monitors that use it and each filter lists the services that use the
filter.

This enables the creation of a server and linking of that server to
services and monitors in one atomic operation.
2017-05-04 09:14:03 +03:00
Markus Mäkelä
b6add43bd2 MXS-1220: Express relations in JSON API style
The JSON API (http://jsonapi.org/) specifies a way to express relations in
a somewhat generic way. By moving towards a more generic schema for the
resources, it will be easier to handle the modification of the relations
between these resources.
2017-05-04 09:14:03 +03:00
Markus Mäkelä
5b9c276123 MXS-1220: Create corrent relation links
When a resource has a relation to another resource, it should be expressed
as a working link to the resource. By passing the hostname of the server
to the functions, we are able to generate working relation links.
2017-05-04 09:14:01 +03:00
Markus Mäkelä
978af12a91 MXS-1220: Simplify resource path handling
The resources are now declared by simply typing the path that they match
and providing a callback for that path. This makes it easier to add new
resources without having to create a new class for each added resource
endpoint. It also removes some of the pathname processing that was done
inside the child class handler functions.
2017-05-04 09:12:16 +03:00
Markus Mäkelä
52e075963e MXS-1220: Reorganize request and response processing
The standard response headers are now generated at a higher level. This
reduces the scope of the HttpResponse class making it a leaner wrapper
around a few simple variables, namely the JSON body of the response.

The HttpRequest now exposes the Host header that the client sent. This
allows resource relations to be real links that work without modification.
2017-05-04 09:12:16 +03:00
Markus Mäkelä
8c77e62872 MXS-1220: Make nested resources links
If a resource can be accessed from more than one place, it should be a
link. The previous change that duplicated the content for nested resources
wasn't a very clean solution. With links, the content is easier to browse
using a normal browser.
2017-05-04 09:12:16 +03:00
Markus Mäkelä
ec6f3d566b MXS-1220: Implement JSON diagnostics in binlogrouter
The binlogrouter now implements the JSON diagnistics entry point.
2017-05-04 09:12:16 +03:00
Markus Mäkelä
52e0cc8e16 MXS-1220: Add HTTP BA authentication
The admin interface now supports Basic Access authentication. This is not
a secure method of authentication and it should not be used without
unencrypted connections.

Made the admin interface port, authentication, username and password
configurable.
2017-05-04 09:12:15 +03:00
Markus Mäkelä
d242203279 MXS-1220: Use libmicrohttpd for the HTTP handling
The HTTP side of the REST API is better handled with an actual
library. The libmicrohttpd library provides a convenient way of handling
the HTTP traffic between the clients and MaxScale.
2017-05-04 09:12:15 +03:00
Markus Mäkelä
eb3ff1cc7b MXS-1220: Implement JSON diagnostics for most routers
All routers except the binlogrouter now fully implement the JSON
diagnostic entry point. The binlogrouter needs to be handled in a separate
commit as it produces a large amount of diagnostic output.
2017-05-04 09:12:15 +03:00
Markus Mäkelä
25c8fb8556 MXS-1220: Implement JSON diagnostics entry point all filters
The second and final part of the filters now implement the JSON version of
the diagnostics function.
2017-05-04 09:12:15 +03:00
Markus Mäkelä
12baa304e6 MXS-1220: Implement JSON diagnostics entry point in first part of filters
First part of the filters now implement the JSON version of the
diagnostics function. The rest are converted in a followup commit.
2017-05-04 09:12:15 +03:00
Markus Mäkelä
4804c975ad MXS-1220: Implement JSON diagnostics entry point in autheticators
All autheticators now implement the JSON version of the diagnostics
function.
2017-05-04 09:12:15 +03:00
Markus Mäkelä
5e679aa167 MXS-1220: Implement JSON diagnostics entry point in monitors
All monitors now implement the JSON version of the diagnostics function.
2017-05-04 09:12:15 +03:00
Markus Mäkelä
9d108a58de MXS-1220: Use new diagnostics in avrorouter
The avrorouter now implements the new diagnostics API entry point.
2017-05-04 09:12:15 +03:00
Markus Mäkelä
bbe0620944 MXS-1220: Add JSON return value to diagnostics entry points
The modules that implement a diagnostics entry point now return a JSON
type object. This removes the need to format data inside the modules.

The module implementations of these are not yet complete which means that
MaxScale will fail to compile.
2017-05-04 09:12:15 +03:00
Markus Mäkelä
94ebef0703 MXS-1220: Expand nested service and monitor resources
As a service is a collection of other resources, it makes sense to expand
those resources that were previously expressed as relative paths. This way
the API can be readily consumed without altering the pathnames to
links. The downside to this is the increased bandwidth consumption and
possible excess data.
2017-05-04 09:12:15 +03:00
Markus Mäkelä
47c1e9d533 MXS-1220: Deprecate whitespace in object names
All whitespace in object names is now converted to a compacted format with
whitespace replaced with hyphens. If a conversion takes place, a warning
is logged.

Converted some of the tests into C++ as they directly include .cc files.
2017-05-04 09:12:15 +03:00
Markus Mäkelä
aa16980cec MXS-1220: Fix date formatting
Some of the dates were printed with a trailing newline. This was added by
asctime_r.

Fixed `/services` resource returning an empty array.
2017-05-04 09:12:15 +03:00
Markus Mäkelä
75b44198ba MXS-1220: Add monitor to JSON conversion
Monitors can now be printed in JSON format. The REST API resource
`/monitors` accepts GET requests and returns a JSON representation of the
monitors as a response.
2017-05-04 09:12:15 +03:00
Markus Mäkelä
d2e4d9cc64 MXS-1220: Make filter to JSON functions const correct
The functions don't modify the filters when the JSON is generated. This
means that the parameters can be const.

Minor formatting fixes to service sources.
2017-05-04 09:12:14 +03:00
Markus Mäkelä
b656f2b300 MXS-1220: Add JSON formatting for servers and filters
Both the filters and services can be queried via the REST API now that
these resources can be expressed in JSON format.

As with the other resources, these directly call the functions that
generate the data. This will be done via the inter-thread messaging system
once it's in place.
2017-05-04 09:12:14 +03:00
Markus Mäkelä
9468893048 MXS-1220: Add server and session printing to REST API
The REST API now prints individual sessions and servers. It also lists all
servers if no specific server is given.

The functions directly call the printing functions when they should be
using the inter-thread messaging system. When the messaging system is
ready, these functions should be updated.
2017-05-04 09:12:14 +03:00
Markus Mäkelä
fd680544d6 MXS-1220: Add session JSON output
Sessions can now be printed in JSON.
2017-05-04 09:12:12 +03:00
Markus Mäkelä
13cba2cb75 MXS-1220: Make server JSON functions const correct
The functions now take const parameters.

Also fixed a memory leak.
2017-05-04 09:11:18 +03:00
Markus Mäkelä
caf2172677 MXS-1220: Fix minor problems with admin thread
The admin thread now uses blocking IO. This is not optimal but it
simplifies the code by some amount.

Fixed option processing removing one extra character from key name.

Use correct member variable when checking for the option map end.
2017-05-04 09:11:18 +03:00
Markus Mäkelä
c4a8f8c8a6 MXS-1220: Add JSON functions for servers
The servers can now be printed in JSON format.
2017-05-04 09:11:17 +03:00
Markus Mäkelä
b975518996 MXS-1220: Add simple functionality to resources
The resources now properly process parts of the uri. This allows, for
example, certain sessions to be inspected. The current functionality is
only intended for testing and provides no useful functionality.

The actions taken by the resource manager are not done via the
inter-thread messaging system. When the implementation of the messages and
the JSON representation of the resources is done, the REST API resource
can actually be used.
2017-05-04 09:11:15 +03:00
Markus Mäkelä
900bf2db5a MXS-1220: Take the resource handler into use
The resource handler system is now usable but it doesn't perform anything
useful. Although, this will allows it to be tested for correctness.

Minor fixes to HttpResponse output and renaming of functions.
2017-05-04 09:10:33 +03:00
Markus Mäkelä
8b1c0cd1a1 MXS-1220: Add REST resource handler class
The Resource class is intended to be an abstraction of a resource
tree. Each node in the tree can perform actions. The tree is traversed
depth first so that deeper command paths resolve to the correct nodes.

Currently all the base resources defined in the REST API documents are
implemented in a way that they return a 200 OK response to all
requests. When the internal data can be represented as JSON, the resources
can be hooked up to functions that generate JSON.
2017-05-04 09:10:33 +03:00
Markus Mäkelä
e248178349 MXS-1220: Split requested resource into parts
When a client requests a resource, the HttpRequest class now splits the
requested resource into parts. This should help with the resource
validation and navigation.

Added test that checks that the resources are correctly split into the
correct number of arguments and that the argument contents are correct.
2017-05-04 09:10:33 +03:00
Markus Mäkelä
4b4a87fb41 MXS-1220: Clean up requested resource paths
Remove trailing slashes in requested resource paths.
2017-05-04 09:10:33 +03:00
Markus Mäkelä
34ee4a1997 MXS-1220: Add option parsing
The options of a request are now parsed and exposed by the HttpRequest
class.

Added tests for the request options.

Also added missing error handling of invalid requests.
2017-05-04 09:10:33 +03:00
Markus Mäkelä
9d0d394361 MXS-1220: Expand HttpResponse class
The class now generates default headers. The ETag and Last-Modified tags
do not represent any actual modification time or resource hash.

The basic functionality of the HTTP responses is tested by the core test
suite. More advanced testing of the whole REST API is still required.

Removed the static `create` functions as only the JSON parsing version
could generated errors and even then the errors were unlikely. By
replacing the static creator function with a normal constructor, the
HttpResponse class can now also be created on the stack making its use
easier.
2017-05-04 09:10:33 +03:00
Markus Mäkelä
a73d3e9276 MXS-1220: Create HTTP response class
The HTTP response class simplifies the response creation. The next step is
to add generation of all the default headers that are needed by the REST
API.
2017-05-04 09:10:32 +03:00
Markus Mäkelä
55b52b8ab1 MXS-1220: Add HEAD method support
The HEAD method was not in the list of supported methods.
2017-05-04 09:10:32 +03:00
Markus Mäkelä
c937457738 MXS-1220: Add request body parsing
The HTTP request body is expected to be a valid JSON object. All other
requests are considered malformed requests and result in a HTTP 400 error.

Added the Jansson license to the LICENSE-THIRDPARTY.TXT file. Imported
some of the tests from the Jansson test suite to the HttpParser test.
2017-05-04 09:10:32 +03:00
Markus Mäkelä
4eb121ce35 MXS-1220: Rename and reorganize HttpParser
The HttpParser class was renamed to HttpRequest as it parses and processes
only HTTP requests. A second class that creates a HTTP response needs to
be created to handle the response generation.

Moved some of the HTTP constants and helper functions to a separate
http.hh header.
2017-05-04 09:10:32 +03:00
Markus Mäkelä
e34b65658e MXS-1220: Simplify admin request handling
The admin requests are now processed in blocking mode. The timing out of
connecttions is handled by a specific timeout thread that checks the state
of each admin request.

The simplification will help with the JSON parsing with PUT/POST
commands. If non-blocking IO is used, the network reading code and JSON
parsing needs a lot more work to handle partial reads.

If the administrative interface requires higher performance and
concurrency, a multi-threaded solution could be created.
2017-05-04 09:10:32 +03:00
Markus Mäkelä
439d67d129 MXS-1220: Add test for HttpParser
The test simulates various HTTP requests and checks that they are parsed
properly.
2017-05-04 09:10:32 +03:00
Markus Mäkelä
605fed7839 MXS-1220: Add HTTP parser
The HTTP parser parses HTTP/1.1 messages into easily manageable data
structures. This should make it easier to map the HTTP requests into
actual commands in MaxScale.
2017-05-04 09:10:31 +03:00
Markus Mäkelä
23b6fb3e6d MXS-1220: Create separate admin thread
When MaxScale is started, a separate thread for the administrative
interface is started. This allows the worker threads to handle client
requests while the administrative thread handles the lower priority
administrative requests.

The administrative interface responds to all request with a 200 OK HTTP
response. This allows the administrative interface itself to be tested.
2017-05-04 09:10:31 +03:00