Commit Graph

7 Commits

Author SHA1 Message Date
dcd5e5df12 [AuditPlugin] Modify load label of audit plugin to avoid load confliction (#3681)
Change the load label of audit plugin as:

`audit_yyyyMMdd_HHmmss_feIdentity`.

The `feIdentity` is got from the FE which run this plugin, currently just use FE's IP_editlog_port.
2020-05-26 18:23:07 +08:00
8e21827505 [Bug][AuditPlugin] Fix bug that length of query stmt in audit should less than limit. (#3444)
Use `String.getBytes().length` instead of `String.length()` to get the real byte length of a string.
Fix: #3443 

Also fix the typo of `max_filter_ratio` in audit log plugin.
2020-05-01 21:17:10 +08:00
wyb
2de78e50e2 [Bug] Fix authorization missing when auditloader plugin redirect stream load (#3367)
HttpURLConnection can automatically redirect stream load to BE, but there is no authorization 
information in http request headers after redirect.

Maybe HttpURLConnection remove authorization info when do followRedirect.

The solution is set the followRedirect property to false on the connection object and do the 
redirect request manually.

#3364
2020-04-21 22:03:18 +08:00
c6ac60bab9 [SegmentV2] Optimize the upgrade logic of SegmentV2 (#3340)
This CL mainly made the following modifications:

1. Reorganized SegmentV2 upgrade document.
2. When the variable `use_v2_rollup` is set to true, the base rollup in v2 format is forcibly queried for verifying the data.
3. Fix a problem that there is no persistent storage format information in the schema change operation that performs v2 conversion.
4. Allow users to directly create v2 format tables.
2020-04-21 10:45:29 +08:00
ce1d5ab9ab [Bug] Fix some bugs of install/uninstall plugins (#3267)
1. Avoid losing plugin if plugin failed to load when replaying
    When in replay process, the plugin should always be added to the plugin manager,
    even if that plugin failed to be loaded.

2. `show plugin` statement should show all plugins, not only the successfully installed plugins.

3. plugin's name should be unique globally and case insensitive.

4. Avoid creating new instances of plugins when doing metadata checkpoint.

5. Add a __builtin_ prefix for builtin plugins.
2020-04-09 23:04:28 +08:00
c9c58342b2 [License] Add License to codes (#3272) 2020-04-07 16:35:13 +08:00
fcb651329c [Plugin] Making FE audit module pluggable (#3219)
Currently we have implemented the plugin framework in FE. 
This CL make the original audit log logic pluggable.
The following classes are mainly implemented:

1. AuditPlugin
    The interface of audit plugin

2. AuditEvent
    An AuditEvent contains all information about an audit event, such as a query, or a connection.

3. AuditEventProcessor
    Audit event processor receive all audit events and deliver them to all installed audit plugins.

This CL implements two audit module plugins:

1. The builtin plugin `AuditLogBuilder`, which act same as the previous logic, to save the 
    audit log to the `fe.audit.log`

2. An optional plugin `AuditLoader`, which will periodically inserts the audit log into a Doris table
    specified by the user. In this way, users can conveniently use SQL to query and analyze this
    audit log table.

Some documents are added:

1. HELP docs of install/uninstall/show plugin.
2. Rename the `README.md` in `fe_plugins/` dir to `plugin-development-manual.md` and move
    it to the `docs/` dir
3. `audit-plugin.md` to introduce the usage of `AuditLoader` plugin.

ISSUE: #3226
2020-04-03 09:53:50 +08:00