Commit Graph

5908 Commits

Author SHA1 Message Date
ffe7af49c8 [fix](array-type) run 'show create table' return null (#11912)
Co-authored-by: hucheng01 <hucheng01@baidu.com>
2022-08-19 21:28:15 +08:00
6ca90afbfa [Bugfix](datetime) fix DateLiteral range check is no longer valid (#11917)
* [Bugfix](datetime) fix DateLiteral range check is no longer valid
2022-08-19 21:27:32 +08:00
be7a38e170 [refactor](planner): refactor and replace use NIO (#11645)
* [refactor](planner): refactor equals code in Catalog dir.
2022-08-19 21:26:39 +08:00
Pxl
64dc3b360f [Bug](function) fix dcheck fail on close vexpr ctx (#11908) 2022-08-19 19:11:10 +08:00
c82d7687b4 [Bug](shell) fix wrong condition expression in script (#11913)
* [Bug](shell) fix wrong condition expression in script
2022-08-19 19:10:11 +08:00
f66e42f848 [optimization](array-type) support the decimal/datetime as the nest type of array in print_value (#11784)
Co-authored-by: hucheng01 <hucheng01@baidu.com>
2022-08-19 17:59:09 +08:00
e63c83e8e1 [fix](script) Support starting BE without Java environment (#11910) 2022-08-19 17:58:40 +08:00
0728f08c65 [improvement](dependency) Use release mode to build cctz. Improve performance. 2022-08-19 17:56:52 +08:00
Pxl
d1e97be37e [doc](developer-guide) add vscode-lldb usage to developer guide (#11923)
* add codelldb usage

* update
2022-08-19 17:56:17 +08:00
67ad8a81eb [docs](update)udaf now supported (#11929) 2022-08-19 17:56:02 +08:00
1a8a889d56 [refactor](planner): improve enfocer job. (#11922)
- handle enforce distribution when meet sort.
- calculate stats in enforcer job
- refactor calculate stats.
2022-08-19 17:55:43 +08:00
788114c89c [docs](fix) updata pictures licks (#11890)
updata pictures licks
2022-08-19 16:35:50 +08:00
a98d808080 [Chore](benchmark) Fix benchmark scripts, cover case that $PASSWORD not empty (#11486)
Fix benchmark scripts, cover case that $PASSWORD not empty
2022-08-19 15:40:18 +08:00
1b0b5b5f09 [Enhancement](load) add hidden_columns in stream load param (#11625)
Stream load will ignore invisible columns if no http header columns
specified, but in some case user cannot get all columns if columns
changed frequently。
Add a hidden_columns header to support hidden columns import。User can
set hidden_columns such as __DORIS_DELETE_SIGN__ and add this column
in stream load data so we can delete this line.
For example:
curl -u root -v --location-trusted -H "hidden_columns: __DORIS_DELETE_SIGN__" -H
"format: json" -H "strip_outer_array: true" -H "jsonpaths: [\"$.id\",
\"$.name\",\"$.__DORIS_DELETE_SIGN__\"]" -T 1.json
http://{beip}:{be_port}/api/test/test1/_stream_load

Co-authored-by: yixiutt <yixiu@selectdb.com>
2022-08-19 14:57:11 +08:00
01bd7f224b [bugifx](compaction) fix filter_delete if schema has sequence column (#11909)
introduced in #11721. Use last column as delete sign, but if sequence column
exist, it's wrong.

Co-authored-by: yixiutt <yixiu@selectdb.com>
2022-08-19 14:56:06 +08:00
1f9eec5462 [Regression](datev2) Add test cases for datev2/datetimev2 (#11831) 2022-08-19 10:57:55 +08:00
Pxl
089fe01aea [Feature](vectorized alter table) set vectorized alter table to default open (#11897) 2022-08-19 10:57:00 +08:00
7a505cf040 [remote-udaf](optimize) Optimize RPC exception handling logic (#11680) 2022-08-19 10:25:01 +08:00
fcae979798 [fix](memtracker) Fix PartitionedAggregationNode DCHECK when mem exceed limit (#11902) 2022-08-19 09:56:49 +08:00
f1ede2aa9d [fix](function) Fix semantic analysis error in window function at first_value (#11855) 2022-08-19 09:13:29 +08:00
8eb9ac3b04 [impovement](sink) print load_id when sink fails (#11893) 2022-08-19 08:48:02 +08:00
40bff30778 [doc](fix) add STRING conversion notice (#11895) 2022-08-18 21:31:04 +08:00
b15e2ddeaa [fix](Nereids): fix and enable stats derive job (#11755)
fix and enable statistics derive job
Add mock for statistics in computeScan().
2022-08-18 21:26:35 +08:00
124b4f7694 [feature-wip](parquet-reader) row group reader ut finish (#11887)
Co-authored-by: jinzhe <jinzhe@selectdb.com>
2022-08-18 17:18:14 +08:00
4fa53b4cdb [chore](workflow) Add shellcheck to check shell scripts (#11744) 2022-08-18 16:07:28 +08:00
355b7c0a92 [Enhancement](docs) fix document typo #11818
Co-authored-by: ninjazhou <ninjazhou@tencent.com>
2022-08-18 16:06:20 +08:00
Pxl
c0dc51b453 [Bug](Vectorzed alter table)modify schema change cast validate (#11864) 2022-08-18 16:05:48 +08:00
b9dcb60172 [Planner](fix)Fix unexpected index out of bound exception (#11819) 2022-08-18 15:52:54 +08:00
066bc7693e [fix](orderby)disallow hll and bitmap data type in order by list (#11837) 2022-08-18 14:50:25 +08:00
1da39771e3 [Bug](runtime filter) Fix bug for runtime filter in concurrent scanners (#11848) 2022-08-18 14:47:08 +08:00
b8a33d2629 [Improvement](load) turn enable_vectorized_load on by default (#11833) 2022-08-18 14:43:09 +08:00
Pxl
cac317430f [Bug](aggregation) fix core dump on 2nd phase aggregate (#11843) 2022-08-18 14:42:34 +08:00
0637c339b1 [fix](array-type) support to insert the largeint in array (#11868)
Co-authored-by: hucheng01 <hucheng01@baidu.com>
2022-08-18 14:41:07 +08:00
b300b4faa0 [enhancement](memtracker) Optimize readability of mem exceed limit error message #11877 2022-08-18 14:39:41 +08:00
4c3f72d019 [improvement](meta) sort result by tablename when show table status like 'show data' (#11885) 2022-08-18 14:23:45 +08:00
d505d1a5ae [Vectorized](compaction) filter delete data in base compaction (#11721)
* [Vectorized](compaction) filter delete data in base compaction


Co-authored-by: lihaopeng <lihaopeng@baidu.com>
2022-08-18 14:22:59 +08:00
3eeaa8e65b [typo](fix) Fix spelling errors in comments (#11810) 2022-08-18 13:55:41 +08:00
0903dd61f3 [Enhancement](Planner) Improve error message when columns order and keys orders don't match. (#11724)
When creating table like this:
```
CREATE TABLE `test`.`test_key_order` (
  `k1` tinyint(4) NULL COMMENT "",
  `k2` smallint(6) NULL COMMENT "",
  `k3` int(11) NULL COMMENT "",
  `v1` double MAX NULL COMMENT "",
  `v2` float SUM NULL COMMENT ""
) ENGINE=OLAP
AGGREGATE KEY(`k1`, `k3`, `k2`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`k1`) BUCKETS 5
PROPERTIES (
"replication_num" = "1"
);
```

The error message before is:
```
Key columns should be a ordered prefix of the schema.
```

With this PR, the error message is:
```
Key columns should be a ordered prefix of the schema. KeyColumns[1] (starts from zero) is k3, but corresponding column is k2 in the previous columns declaration.
```
2022-08-18 13:28:54 +08:00
ca77824857 [typo](doc)Add the actual hive bitmap udf documentation (#11883)
add miss hive bitmap udf
2022-08-18 12:20:24 +08:00
e1a1a04c2f [Enhancement](Doe) Be query es use fe generate dsl. (#11840) 2022-08-18 10:31:17 +08:00
cfb90b39c7 (vec-stream-load-json) simdjson throw execption lead to core dump (#11880)
when config::enable_simdjson_parser=true in vec streamload, may lead to core dump when json input invalid format string like '{ "a', or all the fields is null like '{}', this may lead to simdjson lib throw some unhandled expection like `Objects and arrays can only be iterated when they are first encountered`.We should take care of these cases

Signed-off-by: eldenmoon <15605149486@163.com>
2022-08-18 10:27:34 +08:00
6c66bdbf30 [fix](orderby)remove useless null literal in order by (#11821) 2022-08-18 10:10:25 +08:00
881670566c [fix]Fix the coredump when an IOError occurs in be (#11857) 2022-08-18 09:13:41 +08:00
8b10a1a3f7 [enhancement](VSlotRef) enhance column_id check in execute function during runtime (#11862)
The column id check in VSlotRef::execute function before is too strict for fuzzy test to continuously produce random query. Temporarily loosen the check logic.
Moreover, there exists some careless call to VExpr::get_const_col, it might return a nullptr but not every function call checks if it's valid. It's an underlying problem.
2022-08-18 09:12:26 +08:00
582be130dd [Feature] (ODBC) support read/write emoji of utf16 via odbc table (#11863)
Co-authored-by: lihaopeng <lihaopeng@baidu.com>
2022-08-18 09:09:02 +08:00
ff1971f916 [improvement](test) add dryRun option and group all cases into either p0 or p1 (#11576)
1. add dryRun option to list tests
2. group all cases into p0 p1 p2
2022-08-17 22:45:53 +08:00
4cdf9f2a23 [Enhancement](Nereids) Refine nereids parser. (#11839)
1. Use ParseException in nereids parser.
2. Add check utils in the parser test.
3. Distinguish matchesFromRoot and matches when checking plans.
2022-08-17 20:17:26 +08:00
11dc5cad83 [feature-wip](unique-key-merge-on-write) add min/max key in segment (#11830)
some feature:
1. add min max key in segment footer to speed up get_row_ranges_by_keys
2. do not load pk bloom filter in query

Co-authored-by: yixiutt <yixiu@selectdb.com>
2022-08-17 18:11:39 +08:00
000253b6aa [doc] Fix typos (#11852)
fix a typo in get-starting doc
2022-08-17 17:52:56 +08:00
50ef6e35be [enhancement](RowDescriptor) enhance tuple_idx check during runtime (#11835) 2022-08-17 17:50:48 +08:00