Commit Graph

2236 Commits

Author SHA1 Message Date
a7376bf109 [typo](doc)flink and spark connector remove thrift compiled documents (#19794)
* [typo](doc)flink and spark connector remove thrift compiled documents

* delete enable_http_server_v2
2023-05-19 14:12:07 +08:00
66f53a723e [typo](docs) Add a previously missed configuration item and other note (#19814)
fix enable_query_memory_overcommit config doc
2023-05-19 08:38:10 +08:00
40ab4ce305 fix select resource groups bug (#19808) 2023-05-18 21:54:31 +08:00
294599ee45 [feature](jsonb) rename JSONB type name and function name to JSON (#19774)
To be more compatible with MySQL, rename JSONB type name and function name to JSON.

The old JSONB type name and jsonb_xx function can still be used for backward compatibility.

There is a function jsonb_extract remained since json_extract is used by json string function and more work need to change it. It will be changed further.
2023-05-18 16:16:52 +08:00
9a99d6cd7e [improvement-#19739]statement error in data-model (#19740)
Co-authored-by: fuchanghai <‘2875334588@qq.com’>
2023-05-18 12:07:01 +08:00
5707ecd0e5 [typo](docs) modify the dynamic table document (#19750) 2023-05-18 09:24:17 +08:00
51a12f4f11 [Doc](map-type) add map sql doc (#19612) 2023-05-18 09:09:41 +08:00
7f9cdc41f5 [typo](docs)add new describe for grant (#19656) 2023-05-18 09:00:39 +08:00
fcd85cfc14 [typo](doc)fix typo in BROKER-LOAD.md #19737 2023-05-18 08:38:12 +08:00
ba1bccc518 [typo](docs)Fix typo in monitor-alert (#19743) 2023-05-18 08:37:55 +08:00
b32ee4facc [typo](doc)fix audit plugin and spark load kerberos for yarn #19749 2023-05-18 08:37:41 +08:00
40909f49cb [typo](docs) spark load example revise and some other revise (#19756)
Co-authored-by: zhuwei <zhuwei8421@gmail.com>
2023-05-18 08:37:23 +08:00
35ad081784 [typo](doc)Example of adding stream load import to current time (#19760) 2023-05-18 08:36:39 +08:00
4566281cc3 [fix](sink) disable lazy-open partition by default (#19769)
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>
2023-05-18 07:28:04 +08:00
97d4778ecf [enhancement](schema) dynamic_partition.time_unit support year (#19551)
dynamic_partition.time_unit support year
2023-05-17 23:49:15 +08:00
8aa7f0e188 [fix](catalog) fix the include_database_list not in effect (#19589) 2023-05-17 22:56:21 +08:00
60d5c82f44 [fix](tvf) fix the inconsistency between tvf backends function and show backends result (#19697) 2023-05-17 22:55:46 +08:00
4ecd1c683d [doc](fix) cold hot separation doc fix (#19757) 2023-05-17 22:39:31 +08:00
2536b57590 [doc](catalog) optimize catalog doc (#19601) 2023-05-17 21:45:08 +08:00
1eb929e1ca [Bugfix](Jdbc Catalog) fix data type mapping of SQLServer Catalog (#19525)
We map `money/smallmoney` types of SQLSERVER into decimal type of doris.
2023-05-17 21:02:42 +08:00
f95c1d7cb6 [feat](profile) Add a new rest api to query instance host and ip information for query profile action in branch master(#18668) (#19643) 2023-05-17 10:52:47 +08:00
9cc7af6062 [doc](doris future) Add mentor doc for doris future in community page (#19690) 2023-05-17 08:20:35 +08:00
3a7bc3a7a8 [doc](retention) optimize retention doc (#19692) 2023-05-17 08:17:45 +08:00
0d11c4207a [docs](struct-type) add docs for struct and named_struct function (#19700) 2023-05-17 08:16:33 +08:00
12c21287a5 [docs](struct-type) Add docs for struct type (#19694) 2023-05-16 19:13:27 +08:00
325a1d4b28 [vectorized](function) support array_count function (#18557)
support array_count function.
array_count:Returns the number of non-zero and non-null elements in the given array.
2023-05-16 17:00:01 +08:00
9535ed01aa [feature](tvf) Support compress file for tvf hdfs() and s3() (#19530)
We can support this by add a new properties for tvf, like :

`select * from hdfs("uri" = "xxx", ..., "compress_type" = "lz4", ...)`

User can:

Specify compression explicitly by setting `"compression" = "xxx"`.
Doris can infer the compression type by the suffix of file name(e.g. `file1.gz`)
Currently, we only support reading compress file in `csv` format, and on BE side, we already support.
All need to do is to analyze the `"compress_type"` on FE side and pass it to BE.
2023-05-16 08:50:43 +08:00
e48524009d [doc](fqdn)fqdn doc en (#19634) 2023-05-16 08:48:34 +08:00
ac9e92e1aa [typo](docs) Optimize mac compilation documentation (#19629) 2023-05-15 20:34:47 +08:00
c87e78dc35 [bug](jsonb) fix jsonb query bug When the json key value contains "." (#19185)
Issue Number: close #19173

mysql> SELECT jsonb_extract('{"a.b.c":{"k1":"v31", "k2.a1": 300},"a":"opentelemetry"}', '$."a.b.c".k1');
+-------------------------------------------------------------------------------------------+
| jsonb_extract('{"a.b.c":{"k1":"v31", "k2.a1": 300},"a":"opentelemetry"}', '$."a.b.c".k1') |
+-------------------------------------------------------------------------------------------+
| "v31" |
+-------------------------------------------------------------------------------------------+
1 row in set (0.06 sec)
2023-05-15 15:43:12 +08:00
6748ae4a57 [Feature] Collect the information statistics of the query hit (#18805)
1. Show the query hit statistics for `baseall`

   ```sql
    MySQL [test_query_db]> show query stats from baseall;
    +-------+------------+-------------+
    | Field | QueryCount | FilterCount |
    +-------+------------+-------------+
    | k0    | 0          | 0           |
    | k1    | 0          | 0           |
    | k2    | 0          | 0           |
    | k3    | 0          | 0           |
    | k4    | 0          | 0           |
    | k5    | 0          | 0           |
    | k6    | 0          | 0           |
    | k10   | 0          | 0           |
    | k11   | 0          | 0           |
    | k7    | 0          | 0           |
    | k8    | 0          | 0           |
    | k9    | 0          | 0           |
    | k12   | 0          | 0           |
    | k13   | 0          | 0           |
    +-------+------------+-------------+
    14 rows in set (0.002 sec)

    MySQL [test_query_db]> select k0, k1,k2, sum(k3) from baseall  where k9 > 1 group by k0,k1,k2;
    +------+------+--------+-------------+
    | k0   | k1   | k2     | sum(`k3`)   |
    +------+------+--------+-------------+
    |    0 |    6 |  32767 |        3021 |
    |    1 |   12 |  32767 | -2147483647 |
    |    0 |    3 |   1989 |        1002 |
    |    0 |    7 | -32767 |        1002 |
    |    1 |    8 |    255 |  2147483647 |
    |    1 |    9 |   1991 | -2147483647 |
    |    1 |   11 |   1989 |       25699 |
    |    1 |   13 | -32767 |  2147483647 |
    |    1 |   14 |    255 |         103 |
    |    0 |    1 |   1989 |        1001 |
    |    0 |    2 |   1986 |        1001 |
    |    1 |   15 |   1992 |        3021 |
    +------+------+--------+-------------+
    12 rows in set (0.050 sec)

    MySQL [test_query_db]> show query stats from baseall;
    +-------+------------+-------------+
    | Field | QueryCount | FilterCount |
    +-------+------------+-------------+
    | k0    | 1          | 0           |
    | k1    | 1          | 0           |
    | k2    | 1          | 0           |
    | k3    | 1          | 0           |
    | k4    | 0          | 0           |
    | k5    | 0          | 0           |
    | k6    | 0          | 0           |
    | k10   | 0          | 0           |
    | k11   | 0          | 0           |
    | k7    | 0          | 0           |
    | k8    | 0          | 0           |
    | k9    | 1          | 1           |
    | k12   | 0          | 0           |
    | k13   | 0          | 0           |
    +-------+------------+-------------+
    14 rows in set (0.001 sec)
   ```

2. Show the query hit statistics summary for all the mv in a table

   ```sql
   MySQL [test_query_db]> show query stats from baseall all;
    +-----------+------------+
    | IndexName | QueryCount |
    +-----------+------------+
    | baseall   | 1          |
    +-----------+------------+
    1 row in set (0.005 sec)
   ```

3. Show the query hit statistics detail info for all the mv in a table

   ```sql
    MySQL [test_query_db]> show query stats from baseall all verbose;
    +-----------+-------+------------+-------------+
    | IndexName | Field | QueryCount | FilterCount |
    +-----------+-------+------------+-------------+
    | baseall   | k0    | 1          | 0           |
    |           | k1    | 1          | 0           |
    |           | k2    | 1          | 0           |
    |           | k3    | 1          | 0           |
    |           | k4    | 0          | 0           |
    |           | k5    | 0          | 0           |
    |           | k6    | 0          | 0           |
    |           | k10   | 0          | 0           |
    |           | k11   | 0          | 0           |
    |           | k7    | 0          | 0           |
    |           | k8    | 0          | 0           |
    |           | k9    | 1          | 1           |
    |           | k12   | 0          | 0           |
    |           | k13   | 0          | 0           |
    +-----------+-------+------------+-------------+
    14 rows in set (0.017 sec)
   ```

4. Show the query hit for a database

   ```sql
    MySQL [test_query_db]> show query stats for test_query_db;
    +----------------------------+------------+
    | TableName                  | QueryCount |
    +----------------------------+------------+
    | compaction_tbl             | 0          |
    | bigtable                   | 0          |
    | empty                      | 0          |
    | tempbaseall                | 0          |
    | test                       | 0          |
    | test_data_type             | 0          |
    | test_string_function_field | 0          |
    | baseall                    | 1          |
    | nullable                   | 0          |
    +----------------------------+------------+
    9 rows in set (0.005 sec)
   ```

5. Show query hit statistics for all the databases

   ```sql
    MySQL [(none)]> show query stats;
    +-----------------+------------+
    | Database        | QueryCount |
    +-----------------+------------+
    | test_query_db   | 1          |
    +-----------------+------------+
    1 rows in set (0.005 sec)
   ```
2023-05-15 10:56:34 +08:00
91d5e956a0 [typo](doc) Fixed typos in cluster-action.md (#19549) 2023-05-14 23:52:41 +08:00
80886af828 [doc](grant)add the version for grant for user; (#19556) 2023-05-14 23:52:18 +08:00
859b203b1d [typo](doc) Fixed typos in query-profile-action.md (#19552) 2023-05-14 23:51:58 +08:00
2b402483a9 add release shade and sdk doc (#19576) 2023-05-14 23:51:17 +08:00
f4aea2a6db [Doc](binlog-load) delete binlog-load doc side bar (#19593) 2023-05-14 23:50:55 +08:00
be0f4abc71 [doc](doris-future)Add doc for doris future (#19617) 2023-05-14 20:22:05 +08:00
f8ef25bb10 [enhancement](load) lazy-open necessary partitions when load (#18874) 2023-05-14 16:09:55 +08:00
cd9d633c1b [doc](multi-catalog)add properties converter docs (#18287)
update doc for #18005
2023-05-12 21:03:30 +08:00
26d1eb64d2 [Doc](statistics) add statistics documents (#19323)
The stats feature will continue to be refined, and the documentation will change over time.
2023-05-12 20:11:29 +08:00
feef5afa0b [typo](doc) Fixed typos in SHOW-ROUTINE-LOAD.md (#19573) 2023-05-12 14:37:28 +08:00
e8f1ce4eaa [typo](docs) fix doc bug of 'collect_list' and 'deploy' (#19531) 2023-05-11 16:53:29 +08:00
834bf2eab7 [feature](array) Add array_last lambda function (#18388)
Add array_last lambda function
2023-05-11 13:15:54 +08:00
41d4ed8367 [Improvement](multicatalog) support show_partitions for hms catalog (#19242)
* [Improvement](multicatalog) support show_partitions for hms catalog

* update according review advice
2023-05-11 01:17:23 +08:00
840dbdc7c0 [typo](docs) add comment of partition and key/value column (#19448)
* change docker compose to 'docker-compose'

* modify sql of mysql

* fix docker start and stop cmd

* new commit

* add comment of partition and key/value column

* Update cn doc format

---------

Co-authored-by: Luzhijing <82810928+luzhijing@users.noreply.github.com>
2023-05-11 01:14:17 +08:00
d20b5f90d8 [feature](executor) Automatically set the instance_num using the info from be. (#19345)
1. fixed some error regressions (results error with big nstance_num due to incorrect order by).
2. if set parallel_fragment_exec_instance_num to 0, the concurrency in the Pipeline execution engine will automatically be set to half of the number of CPU cores.
3. add limit to parallel_fragment_exec_instance_num that it cannot be set to more than fe.conf::max_instance_num(Default: 128)
```
mysql [(none)]>set parallel_fragment_exec_instance_num = 514;
ERROR 1231 (42000): errCode = 2, detailMessage = Variable 'parallel_fragment_exec_instance_num' can't be set to the value of '514(Should not be set to more than 128)'
```
2023-05-10 17:07:41 +08:00
0dd35c81b4 [docs](data-model):add sql statements to import data (#19390)
* [docs](data-model):add sql statements to import data
* [docs](data-model)synchronize documents in English
2023-05-10 17:06:50 +08:00
Pxl
9b7a419aed [Chore](build) update some doc about build enviroment (#19325)
update some doc about build enviroment
2023-05-10 16:18:44 +08:00
bdf54963ae Update query-analysis.md (#19456) 2023-05-10 16:05:51 +08:00
7631c82eff [typo](doc) Fixed typos in native-user-defined-function.md (#19459) 2023-05-10 16:05:30 +08:00