Commit Graph

12 Commits

Author SHA1 Message Date
ad1f635070 [Feature](auditloader) Plugin auditloader use auth token to avoid using cleartext passwords in config (#26278)
Doris FE will check if stream load http request has auth token after checking password failed;
Plugin audit-log loader can use auth token if plugin config set use_auth_token to true

Co-authored-by: Mingyu Chen <morningman.cmy@gmail.com>
2023-11-07 19:14:57 +08:00
6297ef10e9 [enhancement](plugin) import audit logs for slow queries into a separate table (#14100)
* import audit logs for slow queries into a separate table
2022-11-11 09:06:01 +08:00
wxy
c4b6d4d839 [enhancement](AuditLoaderPlugin): add audit queue capacity configurat… (#12887) 2022-09-27 08:50:30 +08:00
d9066440f6 [fix](fe_plugins) update fe plugins' dependency to fe 1.0-SNAPSHOT (#11141) 2022-07-25 09:13:15 +08:00
4bdeef3b64 [chore][fix][doc](fe-plugin)(mysqldump) fix build auditlog plugin error (#7804)
1. fix problems when build fe_plugins
2. format
3. add docs about dump data using mysql dump
2022-01-26 09:11:23 +08:00
dcad6ff5e5 [License] Add License header for missing files (#7130)
1. Add License header for missing files
2. Modify the spark pom.xml to correct the location of `thrift`
2021-11-16 18:37:54 +08:00
6f46c52f1d Make the max stmt length to be loaded in audit table can be defined by user. (#5673) 2021-04-21 09:14:32 +08:00
67b0631257 [Enhancement] Fix bug for auditloader plugin that audit event cannot be processed in time (#5194)
* [Enhancement] Fix bug that audit event cannot be processed in time

Co-authored-by: caiconghui [蔡聪辉] <caiconghui@xiaomi.com>
2021-01-28 10:53:32 +08:00
67b842ce04 [License] Organize and modify the license of the code (#4371)
1. Disable the MySQL client and LZO library by default when building the Doris.

    MySQL client library is used for MySQL external table feature.
    This feature will be replaced by the new ODBC external table soon.

    LZO library is used to compress/decompress data of some old data format of Doris,
    which is no longer used anymore.

2. Add missing license to some files.

3. For all non-Apache-License code, all are explained in NOTICE file and the corresponding license is declared.

4. Remove the js source code from webroot, it will be downloaded as thirdparty
2020-08-24 21:51:55 +08:00
dcd5e5df12 [AuditPlugin] Modify load label of audit plugin to avoid load confliction (#3681)
Change the load label of audit plugin as:

`audit_yyyyMMdd_HHmmss_feIdentity`.

The `feIdentity` is got from the FE which run this plugin, currently just use FE's IP_editlog_port.
2020-05-26 18:23:07 +08:00
c9c58342b2 [License] Add License to codes (#3272) 2020-04-07 16:35:13 +08:00
fcb651329c [Plugin] Making FE audit module pluggable (#3219)
Currently we have implemented the plugin framework in FE. 
This CL make the original audit log logic pluggable.
The following classes are mainly implemented:

1. AuditPlugin
    The interface of audit plugin

2. AuditEvent
    An AuditEvent contains all information about an audit event, such as a query, or a connection.

3. AuditEventProcessor
    Audit event processor receive all audit events and deliver them to all installed audit plugins.

This CL implements two audit module plugins:

1. The builtin plugin `AuditLogBuilder`, which act same as the previous logic, to save the 
    audit log to the `fe.audit.log`

2. An optional plugin `AuditLoader`, which will periodically inserts the audit log into a Doris table
    specified by the user. In this way, users can conveniently use SQL to query and analyze this
    audit log table.

Some documents are added:

1. HELP docs of install/uninstall/show plugin.
2. Rename the `README.md` in `fe_plugins/` dir to `plugin-development-manual.md` and move
    it to the `docs/` dir
3. `audit-plugin.md` to introduce the usage of `AuditLoader` plugin.

ISSUE: #3226
2020-04-03 09:53:50 +08:00