Commit Graph

12780 Commits

Author SHA1 Message Date
6aeacf252e [fix](pipeline) if pr addded files, also need run regresion (#23332)
Co-authored-by: stephen <hello-stephen@qq.com>
2023-08-23 16:45:44 +08:00
4d60418c58 Update load.groovy (#23333) 2023-08-23 16:33:40 +08:00
7f1857b4e7 [fix](regression-test) fix unstable case load_p0/insert/test_insert.groovy (#23326) 2023-08-23 16:22:49 +08:00
c3327b51b9 [Fix](Nereids) add nereids load function in read fields of GlobalFunctionMgr and Database (#23248)
add nereids load function in read fields of GlobalFunctionMgr and Database
to fix some udf is lost when restart FE and query with Nereids.
2023-08-23 15:59:50 +08:00
22e373a799 [feature](vector-search) add 4 distance functions to support vector search (#23129) 2023-08-23 15:51:15 +08:00
2dda44d7b5 [fix](csv-reader)fix bug of multi-char delimiter in csv reader
fix bug that csv_reader parse line in order to get column.
2023-08-23 15:19:13 +08:00
Pxl
ab7b45a9e2 [Bug](map) support replicate for column map and remove some unused code #23356 2023-08-23 15:06:04 +08:00
c7b9eb5f9c [enhancement](bitmap)support bitmap type for non-key column in unique table (#23228) 2023-08-23 14:21:22 +08:00
527293aa41 [refactor](dynamic table) remove dynamic table (#23298) 2023-08-23 14:15:14 +08:00
d8a31405c4 [refactor](Nereids): remove Penalty (#23321) 2023-08-23 13:31:01 +08:00
ba882dea21 [pipelineX](dependency) Build DAG between pipelines (#23355) 2023-08-23 13:21:32 +08:00
450b90c082 [Bug](compile) fix compiling error due to merge conflicts (#23351)
fix compiling error due to merge conflicts
2023-08-23 11:57:57 +08:00
78849f6e64 [refactor](predicate) refactor function parse_to_predicte (#23343)
Refactored the interface of the parse_to_predicte function, we will be utilizing this interface to support the decomposition of variant columns.

The "column" parameter might represent a column resulting from the decomposition of a variant column. Instead of using a "unique_id" for identification, we are utilizing a "path" to denote this column.
2023-08-23 11:52:20 +08:00
14296ee87f [fix](window_function) wrong order by range (#23346) 2023-08-23 11:23:00 +08:00
ab9b29f11f [fix](multi-catalog)fix broker and tvf params (#23252)
fix fs specifics, broker and tvf params, and NPE
2023-08-23 10:33:24 +08:00
a82c448910 [test](fix) fix test_clickhouse_jdbc_catalog.groovy (#23342) 2023-08-23 10:31:54 +08:00
35d0c9e71e [refactor](nereids) Refactor stats collection framework (#22963)
* remove auto analyze grammer
* refactor ResultRow
2023-08-23 10:05:57 +08:00
78c6b115c3 [fix](planner)avg function need support large int param (#23254)
* [fix](planner)avg function need support large int param
2023-08-23 10:05:08 +08:00
a5382043d7 [chore](third-party) Fix errors while building brpc (#23339)
close #23340
2023-08-23 09:13:06 +08:00
391355eeae [pipelineX](agg) Support streaming agg (#23341) 2023-08-23 08:38:12 +08:00
Pxl
8ed4045df9 [Chore](primitive-type) remove VecPrimitiveTypeTraits (#22842) 2023-08-23 08:37:40 +08:00
d9993a7790 [doc](multi-catalog)add gcs broker load faq (#23229)
add gcs broker load faq

When the '_' is used  at buckt name, will report an error:  Missing required property: uri

the gcs bucket not match the s3 bucket name format: https://docs.aws.amazon.com/zh_cn/AmazonS3/latest/userguide/bucketnamingrules.html

we'd better not use  '_' in bucket.

<!--Describe your changes.-->
2023-08-22 22:48:31 +08:00
Pxl
e6d20f842c [Bug](compile) fix compile failed on function case (#23335) 2023-08-22 22:10:53 +08:00
5c2fae7ce5 [pipeline](exec) Refactor the table sink code in remove unless code (#23223)
Refactor the table sink code in remove unless code
2023-08-22 20:42:14 +08:00
2ad46c5826 [fix](show) show load warning support load v2 (#22759) 2023-08-22 20:08:19 +08:00
eeec26d68d [refactor](load) rename flush_memtable_and_wait to flush_async (#23204) 2023-08-22 20:07:50 +08:00
1609b6cbf2 [pipelineX](sort) Support sort operator (#23322) 2023-08-22 19:36:50 +08:00
6124eea55e [Fix](regression-test)fix unstable test_default_limit case. (#23270)
[Fix](regression-test)fix unstable test_default_limit case. #23270
2023-08-22 18:59:48 +08:00
8819d73abd [fix](be) fix the crash when there is no tzfile in docker env (#23071) 2023-08-22 18:56:36 +08:00
6643d147c6 forbid: nereids_scalar_fn_Array (#23328) 2023-08-22 18:50:34 +08:00
Pxl
1a1f86486d [Improvement](function) opt for case when (#23068)
opt for case when
2023-08-22 18:31:40 +08:00
0b51e6d8e1 [refractor](FunctionArrayIndex) make the codes more simple 2023-08-22 17:48:59 +08:00
9a7d4f906b [enhance](Tools): update tpcds tools #23308
Update tpcds tools: add sf1000 & sf10000 related sql and config.
2023-08-22 17:10:50 +08:00
5184246804 [fix](case) disable unstable case (#23314)
Co-authored-by: stephen <hello-stephen@qq.com>
2023-08-22 16:34:04 +08:00
a4e041ea55 [improve](alter-job) Add a config for forbiding doing alter job (#23294) 2023-08-22 16:28:36 +08:00
4a690f8b13 fix compile error with gcc12 (#23190) 2023-08-22 16:05:03 +08:00
bcdb481374 [refactor](fragment) refactor non pipeline fragment executor (#23281)
---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
2023-08-22 16:00:34 +08:00
820d328ad7 [feature](Nereids) support handle query in fe (#23263)
1. fix select variable with alias parsed as two level alias expression.
2. support handle query in fe
2023-08-22 15:28:29 +08:00
a7675243d9 [fix](jdbc catalog) fix adaptation to Oracle special character / table names (#23080)
The changes of this PR for JdbcOracleClient are as follows:

#### bug fixes:
  1. Fix the problem that if there is an approximate table name for Schema synchronization with a table name with `/` characters, the synchronization Column will be confused
  2. Fix the NPE problem of metadata synchronization after enabling lower_case_table_names configuration

#### improvement:
  1. Modify the method of synchronizing Oracle User to Doris Database mapping, use `metadata.getSchemas` instead of `SELECT DISTINCT OWNER FROM all_tables`
  2. When synchronizing metadata, change `null` at the catalog level to `conn.getcatalog`
2023-08-22 15:25:42 +08:00
5b05d5167d [FIX](regress-test) fix local_tvf with complex-type in cluster (#23299)
* fixed

* update

* fixed
2023-08-22 15:25:10 +08:00
da2eb69eba [test](Nereids) add array scalar function test cases (#23303) 2023-08-22 15:05:28 +08:00
9d2e23b1aa [fix](parquet) A row of complex type may be stored across more pages (#23277)
A row of complex type may be stored across two(or more) pages, and the parameter `align_rows` indicates that whether the reader should read the remaining value of the last row in previous page.
2023-08-22 14:47:10 +08:00
fbccd5cfeb disable nereids_delete_mow_partial_update (#23273) 2023-08-22 14:46:51 +08:00
6c8af92175 [fix])(nereids)Support select catalog.db.table.column from xxx for nereids planner. #23221
Nereids doesn't support select table.* from table, this pr is to fix this bug.
Support three layer qualifier. (catalog.database.table)
2023-08-22 13:58:25 +08:00
5ff7b57fc1 [fix](parquet) parquet reader confuses logical/physical/slot id of columns (#23198)
`ParquetReader` confuses logical/physical/slot id of columns. If only reading the scalar types, there's nothing wrong, but when reading complex types, `RowGroup` and `PageIndex` will get wrong statistics. Therefore, if the query contains complex types and pushed-down predicates, the probability of the result set is incorrect.
2023-08-22 13:35:29 +08:00
51db11ed0b [improve](jdbc catalog) Add a variable to accommodate the final keyword in ClickHouse Jdbc Catalog queries (#23282) 2023-08-22 12:13:36 +08:00
b471862dba [Fix](regression-test) fix es regression test (#23160) 2023-08-22 11:52:37 +08:00
7450bd504d [improvement](scan) Reduce time to fetch function calls (#23098) 2023-08-22 11:24:55 +08:00
438010a214 [fix](merge-on-write) process error of delete bitmap calculation (#23285) 2023-08-22 10:53:50 +08:00
2a5aed46f7 add CloudCanal en doc (#22731) 2023-08-22 10:49:54 +08:00