Commit Graph

17549 Commits

Author SHA1 Message Date
343d58123d [Fix](nereids)Fix nereids fail to parse tablesample rows bug (#26981) 2023-11-16 12:23:37 +08:00
bf6a9383bc [fix](stats) table not exists error msg not print objects name (#27074) 2023-11-15 22:10:50 -06:00
6be74d22ea [fix](nereids)fix bug that query infomation_schema.rowsets fe send fragment to one of muilti be. (#27025)
Fixed the bug of incomplete query results when querying information_schema.rowsets in the case of multiple BEs.

The reason is that the schema scanner sends the scan fragment to one of multiple bes, and be queries the information of fe through rpc. Since the rowsets information requires information about all BEs, the scan fragment needs to be sent to all BEs.
2023-11-16 12:08:22 +08:00
7e82e7651a [Improve](txn) Add some fuzzy test stub in txn (#26712) 2023-11-16 11:50:06 +08:00
624d372dcd [FIX](map)fix map element_at with decimal value (#27030) 2023-11-16 11:49:51 +08:00
230d8af777 [regression test](temporary_partitions) add case for temporary_partitions #27063 2023-11-16 11:49:37 +08:00
7fbc6d26a7 [debug](log) add some log to debug issue about insert (#27045) 2023-11-16 11:46:47 +08:00
f3ee6dd55a [feature](Nereids): eliminate sort under subquery (#26993) 2023-11-16 10:30:28 +08:00
a54cfb7558 [fix](backup) return if status not ok and reduce summit job (#26940)
when backup is prepareAndSendSnapshotTask(), if some table has error, return status not ok, but not return, and other tables continue put snapshot job into batchTask and summit jobs to be while these jobs need cancel. so when status is not ok, return and do not summit jobs
2023-11-16 10:16:56 +08:00
612d9dd7c6 [fix](errmsg) fix multiple FE processes start err msg (#27009) 2023-11-16 10:16:35 +08:00
042f6e8458 [cleanup](move-memtable) cleanup unused fields in rowset writer v2 (#27073) 2023-11-16 10:13:00 +08:00
xy
b8b86a7262 [enhance](cooldown) Reduce the locking interval for cooldown task (#26984)
Co-authored-by: xingying01 <xingying01@corp.netease.com>
2023-11-16 10:02:32 +08:00
0eabe9a651 [Test](orc-reader) Add orc submodule's unit tests. (#26878) 2023-11-16 09:53:42 +08:00
7ef1f7e511 [Bug](pipeline) try fix the exchange sink buffer result error (#27052) 2023-11-16 09:20:56 +08:00
02f3762ab3 [refactor](status) define error code and enable stacktrace in same place (#27065) 2023-11-16 08:41:40 +08:00
3ad865fef9 [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits (#26191) 2023-11-15 21:34:49 +08:00
76d530e349 [fix](tablet sched) fix sched delete stale remain replica (#27050) 2023-11-15 21:26:22 +08:00
10ee48bb6f [fix](publish version) publish version task no need return VERSION_NOT_EXIST #27005
if BE's tablet not contains a txn, publish txn on them will no error, when check version exists it will indicate the tablet as error_tablet_id in task's response, so FE can know this tablet has fail.

Also for task, it's no need to set its status as "VERSION_NOT_EXIST". Because if set it as not ok, the BE will try this task two times. Since not contains this tablet's txn, the retry is in vain.
2023-11-15 21:09:54 +08:00
4dda5aad09 [fix](fe ut) fix unstable test DecommissionBackendTest (#26891) 2023-11-15 21:02:59 +08:00
26480b57ff [fix](fe ut) fix unstable ut TabletRepairAndBalanceTest (#27044) 2023-11-15 20:47:50 +08:00
035e593b26 remove useless hash function (#26955) 2023-11-15 20:37:21 +08:00
718ba41cd1 [Enhance](regression) skip case test_information_schema_external (#27056) 2023-11-15 20:29:46 +08:00
83edcdead9 [enhancement](random_sink) change tablet search algorithm from random to round-robin for random distribution table (#26611)
1. fix race condition problem when get tablet load index
2. change tablet search algorithm from random to round-robin for random distribution table when load_to_single_tablet set to false
2023-11-15 19:55:31 +08:00
4e105e94a2 [fix](statistics) fix updated rows incorrect due to typo in code (#26979) 2023-11-15 05:25:46 -06:00
ad7dfd75d5 [enhancement](Nereids): make HyperGraph can be built from plan (#26592) 2023-11-15 19:06:14 +08:00
xy
ea4b6e9975 [Fix](doc) Fixed some errors in the documentation (#26958)
Modify schedule_slot_num_per_path to schedule_slot_num_per_hdd_path and schedule_slot_num_per_ssd_path in the documentation

Co-authored-by: xingying01 <xingying01@corp.netease.com>
2023-11-15 18:25:50 +08:00
0491437a86 [Opt](scanner-scheduler) Optimize BlockingQueue, BlockingPriorityQueue and change remote scan thread pool. (#26784)
## Proposed changes
- Optimize `BlockingQueue`, `BlockingPriorityQueue` by swapping `notify` and `unlock` to reduce lock competition. Ref: https://www.boost.org/doc/libs/1_54_0/boost/thread/sync_bounded_queue.hpp
- Change remote scan thread pool to `PriorityQueue`.

### Test result
Before:
```
mysql> select  sum(lo_partkey)  from  lineorder;
+-----------------+
| sum(lo_partkey) |
+-----------------+
| 300021444265405 |
+-----------------+
1 row in set (1.11 sec)
```

After:
```
mysql> select  sum(lo_partkey)  from  lineorder;
+-----------------+
| sum(lo_partkey) |
+-----------------+
| 300021444265405 |
+-----------------+
1 row in set (0.80 sec)
```
2023-11-15 18:24:36 +08:00
c1c8376eb2 [fix](thirdparty) brpc stream crash on error stream (#27016)
* [fix](thirdparty) brpc stream crash on error stream

* rename
2023-11-15 18:24:01 +08:00
52d7725b36 [fix](auth) fix overwrite logic of user with domain (#27002)
Reproduce:
DBA do following operations:
1. create user user1@['domain'];   // the domain will be resolved as 2 ip: ip1 and ip2;
2. create user user1@'ip1';
3. wait at least 10 second
4. grant all on *.*.* to user1@'ip1';  // will return error: user1@'ip1' does not exist

This is because the daemon thread DomainResolver resolve the "domain" and overwrite the `user1@'ip1'`
which is created by DBA.

This PR fix it.
2023-11-15 18:19:54 +08:00
d3fd923447 [opt](pipeline) Return InternalError to FE instead of doing a useless DCHECK in ExecNode #27035
Effect: Client will see error message like below when BE meeting plan logical error.

RROR 1105 (HY000): errCode = 2, detailMessage = ([xxx]())[CANCELLED]Logical error during processing VNewOlapScanNode(dr_case_tag), output of projections 2 mismatches with exec node output 3
2023-11-15 18:15:21 +08:00
00896d8954 [fix](agg) fix coredump of multi distinct of decimal128I (#27014)
* [fix](agg) fix coredump of multi distinct of decimal128

* fix
2023-11-15 17:37:20 +08:00
2f529c1c7b [feature](Nereids): remove True in Join condition (#26951)
Remove `True` in Join Condition like `SELECT * FROM t1 JOIN t2 on True`;
2023-11-15 15:58:33 +08:00
760c6cdeab [fix](ut) enable running ut BDBJEJournalTest inside IDE #27012 2023-11-15 14:55:22 +08:00
a1d139080d [fix](nereids) patition prune is affected by non-paritition-key condition (#26873)
stop propagate Context.childrenContainsNonInterestedSlots if expr changed to TRUE
2023-11-15 14:46:59 +08:00
5dbc3cbba4 [test](information_schema)append information_schema external_table_p0 case. (#26846) 2023-11-15 14:30:16 +08:00
e1ba471727 [fix](send_batch_parallelism) add test case for send_batch_parallelism (#26908) 2023-11-15 14:21:58 +08:00
dbac12bae8 [fix](memory)Modify the default conf values of mem_limit and cache_last_version_interval_second (#26945)
mem_limit from 80% to 90%
cache_last_version_interval_second from 900 to 30
2023-11-15 14:02:58 +08:00
f10ebc4392 [opt](docs)add the best practices for hive catalog with kerberos (#26729) 2023-11-15 13:55:29 +08:00
2c6d2255c3 [fix](Nereids) nested type literal type coercion and insert values with map (#26669) 2023-11-14 21:13:26 -06:00
febf4bcb23 [chore](clang-tidy) Remove some confusing check of clang-tidy #26953 2023-11-15 10:39:45 +08:00
f87c807979 [enhancement](jdk) support doris fe running in jvm with jdk16+ (#26889) 2023-11-15 10:27:30 +08:00
df867a1531 [fix](catalog) Fix ClickHouse DataTime64 precision parsing (#26977) 2023-11-15 10:23:21 +08:00
15c43d8b8a [BugFix](JDBC Catalog) fix jdbc catalog query bitmap may cause be core sometimes (#26933)
BitmapValue::write_to will get a string with size 1 for empty BitmapValue, however the size 1 string will reinterpret to BitmapValue* back in ColumnComplexType::insert:
void insert(const Field& x) override {
const String& s = doris::vectorized::get<const String&>(x);
data.push_back(reinterpret_cast<const T>(s.c_str()));
}

in data.push_back will goto BitmapValue copy constructor, as the _type is not first member in BitmapValue, cause access to an unknown memory location.
2023-11-15 10:20:42 +08:00
6183b298e1 [refactor](data_type) remove some unused functions (#26966) 2023-11-15 09:23:53 +08:00
cdc5626930 [regression](partial update) Add row store cases for all existing partial update cases (#26924) 2023-11-15 08:20:55 +08:00
89215306d3 [improve](load) add switch for vertical segment writer (#26996) 2023-11-15 08:19:12 +08:00
82a88366f1 [fix](fe ut) make colcoate test wait time longer (#26999) 2023-11-15 08:18:48 +08:00
add160b768 [improvement](regression-test) add more group commit regression-test (#26952) 2023-11-15 00:01:13 +08:00
88d909b4dd [test](regression) Add more alter stmt regression case (#26988) 2023-11-14 23:58:15 +08:00
08044a35ae (fix)[schema change] fix incorrect setting of schema change jobstate when replay editlog (#26992) 2023-11-14 23:53:22 +08:00