Commit Graph

10668 Commits

Author SHA1 Message Date
da66a64e09 [fix](merge-on-write) return error st if check_pk_in_pre_segments failed (#19736) 2023-05-23 11:04:15 +08:00
8d5269542b [fix](testcase) fix wrong use of same table in different case 2023-05-23 10:24:33 +08:00
4398b91576 [Fix](multi catalog)Change all partition names to lower case (#19816)
Iceberg table partition name may contain upper case characters, for example: City=xxx, Nation=xxx.
But in Doris, all column names are in lower case. Here we transfer the partition name to lower case to keep consist with column name.
2023-05-23 09:31:31 +08:00
633989c78e [fix](Nereids): commute non-inner join for DPHyp (#19929) 2023-05-23 09:30:50 +08:00
d2e3fb097b [docs](struct-map-type) update user docs for struct and map type (#19939)
We have already supported decimalV3.
2023-05-23 09:30:26 +08:00
bd74890cf7 [fix](multi-catalog) JDBC Catalog Unknown UNSIGNED type of mysql, type: [DOUBLE] (#19912) 2023-05-23 09:29:57 +08:00
fe111207a9 [Fix](lazy_open) Fix lazy open null point (#19829) 2023-05-23 09:17:46 +08:00
30417e06d4 [enhance](fs) use bvar to monitor s3 file reader& writer (#19607)
remove useless prefix
2023-05-22 23:01:21 +08:00
3dcdadcea6 [Improvement](function) support decimalv3 for function least and greatest (#19931) 2023-05-22 22:48:44 +08:00
53ba46e404 [Fix][Refactor] Fix 'not member call on null pointer of type 'doris::TextConverter' error in ubsan env and refactor text converter. (#19849)
Fix 'not member call on null pointer of type doris::TextConverter' error in ubsan env and refactor text converter.
2023-05-22 21:00:19 +08:00
6762af3c9b [Improve](struct)improve struct support into outfile (#19894)
support select into outfile for struct type
2023-05-22 18:45:56 +08:00
Pxl
9945067e3c [Bug](function) make VcompoundPred optimization work well (#19870)
make VcompoundPred optimization work well
#19818 this pr try to enable VcompoundPred optimization but get wrong result on tpcds q28.
The reason is some nullable logic on mysql need special handling.

mysql [regression_test_tpcds_sf1_p1]>select null and false;
+----------------+
| NULL AND FALSE |
+----------------+
|              0 |
+----------------+
1 row in set (0.00 sec)

mysql [regression_test_tpcds_sf1_p1]>select null and true;
+---------------+
| NULL AND TRUE |
+---------------+
| NULL          |
+---------------+
1 row in set (0.00 sec)

mysql [regression_test_tpcds_sf1_p1]>select null or false;
+---------------+
| NULL OR FALSE |
+---------------+
| NULL          |
+---------------+
1 row in set (0.00 sec)

mysql [regression_test_tpcds_sf1_p1]>select null or true;
+--------------+
| NULL OR TRUE |
+--------------+
|            1 |
+--------------+
1 row in set (0.00 sec)
2023-05-22 18:32:17 +08:00
7ae3724acd [typo](doc)fix typo in readme #19895
miliseconds -> milliseconds
2023-05-22 18:31:19 +08:00
750a3ea1b4 [doc](fqdn)broker fqdn doc #19910 2023-05-22 18:30:57 +08:00
Pxl
e9223f6a19 [Feature](aggregation) add agg_state define and ddl support (#19824)
add agg_state define and ddl support
2023-05-22 11:45:53 +08:00
Pxl
d64be9565d [Bug](function) fix function in get wrong result when input const column (#19791)
fix function in get wrong result when input const column
2023-05-22 10:58:29 +08:00
f57b6adba9 [improvement](libhdfs) Use keytab and principal to login kerberos (#19841)
User keytab and princpal to login kerberos.
And user does not need to execute kinit manually anymore.
2023-05-22 10:54:21 +08:00
76dc5841dc [opt](nereids)compute runtime filter size by the ndv of build side (#18803) 2023-05-22 10:38:11 +08:00
060fbe2b1f [typo](docs) replace the wrong field "json_paths" in s3 tvf related docs with the correct one "jsonpaths" #19900 2023-05-22 09:03:56 +08:00
a6bd014b8a [FIX](serde)pb ut is not stable #19907 2023-05-22 09:03:02 +08:00
7c539575c7 [refactor](hudi) remove hudi external table (#19908)
Hudi external table is deprecated since 1.2.
We should remove it now.
Recommend to use "multi-catalog" feature to connect to Hudi.

User can not create Hudi external table.
When restarting FE, all hudi external table will still be replayed but can not be read. And when doing checkpoint, all these tables will be discarded.
2023-05-22 09:02:34 +08:00
1c950d6930 [fix](config) fix memory config enable_query_memroy_overcommit spell problem #19898 2023-05-22 00:32:20 +08:00
76c358b3e3 [revert](memory) revert page no use Allocator && default disable ChunkAllocator (#19905)
default chunk allocator reserve is 0. At this time, it is meaningless to enable chunk allocator, it will only waste memory.
2023-05-21 22:16:41 +08:00
33fd965b5c [feature-wip](resouce-group) Supports memory soft isolation of resource group (#19802)
create resource groups name properties(
    'enable_memory_overcommit' = 'true' // whether to enable memory soft isolation
)
2023-05-21 19:33:57 +08:00
a7f3bfec89 [refactor](cluster)(step-2) remove cluster related to Backend (#19842) 2023-05-21 09:00:35 +08:00
777bdce5a5 [minor](clone) add more debug log for tablet scheduler (#19892)
Sometimes I find that the tablet scheduler can not schedule tablet, and with no more info for debugging.
So I add some debug log for this process.
No logic is changed.
2023-05-20 15:59:26 +08:00
8b9813663d [test](executor)add crud regression test for resource group (#19659)
dd crud regression test for resource group (#19659)
2023-05-20 13:49:02 +08:00
499f443779 [feature](iceberg) Support read iceberg data on gcs (#19815) 2023-05-20 12:40:03 +08:00
HB
1b119704f8 [Enhancement] show total transactions in show proc "/transactions" (#19492)
In a scenario where multiple DBs are simultaneously imported with high concurrency, a significant number of transactions will be generated. Without a summary field, we cannot clearly see how many transactions there are in the current cluster. Therefore, I have enhanced this point.

```
mysql> show proc "/transactions";
+-------+-----------------------------------+-----------------------+
| DbId     | DbName                                             | RunningTransactionNum |
+-------+-----------------------------------+-----------------------+
| 10002   | default_cluster:xxxx                         | 0                                     |
| 14005   | default_cluster:__internal_schema | 0                                     |
| Total     |  2                                                         | 0                                     |
+-------+-----------------------------------+-----------------------+
3 rows in set (0.02 sec)
```
2023-05-20 11:26:28 +08:00
a81db3e984 [improvement](FQDN) broker support fqdn (#19821)
1.broker support fqdn
2.change 'master_only' attr of 'enable_fqdn_mode'
2023-05-20 11:25:58 +08:00
178d6cc529 [improvement](multi-catalog)hms sync event log more info #19887 2023-05-20 08:25:14 +08:00
512806f902 [fix](ubsan) UBSAN avoid thread local switch 2023-05-20 07:14:43 +08:00
3e02b0e7ea [Doc](fix) Fixed two documentation errors (#19879) 2023-05-20 00:25:01 +08:00
ca737c37ee add testcases for inverted index on different datatypes (#19843) 2023-05-20 00:21:34 +08:00
6f21cd8a5f update create table doc (#19871) 2023-05-20 00:18:38 +08:00
e13e96473f Modify the jdk version requirements in the documentation (#19874) 2023-05-20 00:18:24 +08:00
0b367888f8 Modify the usage of Doriswriter parameters (#19875) 2023-05-20 00:18:10 +08:00
0d956da927 [doc](multi-catalog)change hms event doc (#19886)
change hms event doc
need restart more service
2023-05-19 22:48:41 +08:00
77dfdfdd50 [Bug][pipeline] Fix regression tpcds failed in nereid planner (#19885) 2023-05-19 22:30:48 +08:00
24b2fab943 [fix](Nereids): BuildAggForUnion forgot to convert Qualifier Type. (#19883) 2023-05-19 22:18:38 +08:00
88d09ec263 [fix](community): remove some collaborators. (#19884) 2023-05-19 22:08:01 +08:00
47509e65c9 [fix](memory)page no use Allocator, avoid ckbench oom (#19877) 2023-05-19 21:23:31 +08:00
5547bbbaef [decimalv3](function) support function width_bucket (#19806) 2023-05-19 20:28:59 +08:00
78bcc68ab8 [Fix](Nereids) fix serialize colocate table index concurrent bug (#19862)
When doing serialization of minidump input, we can find that when serializing colocate table index, the size and entry get by the hash map always unmatched when concurrent occur. So a write lock be added to ensure concurrency.
2023-05-19 19:51:22 +08:00
ae1577e95c [improvement](jdbc catalog) set oceanbase mysql mode jdbc param useCursorFetch default true (#19856) 2023-05-19 19:45:22 +08:00
68be81363b [enhance](Nereids): Pushdown Filter Through Project in Post Processor. (#19873)
Originally, PushdownFilterThroughProject is in CBO phase, but it will increase Memo size.
So, we move it into PostProcessor
2023-05-19 19:27:52 +08:00
abde8bf26a [chore](build) Fix the compilation errors on macOS (arm64) (#19859)
Some errors raise when building the codebase on macOS (arm64).
2023-05-19 18:50:47 +08:00
67dc68630b [Improve](complex-type)improve array/map/struct creating and function with decimalv3 (#19830) 2023-05-19 17:43:36 +08:00
d8d6880ef1 [Doc](point query) Modify the document to add constraints and limitations for optimizing high-concurrency point queries. (#19852) 2023-05-19 17:29:10 +08:00
2ab844550f [feature-wip](MTMV) support multi catalog (#19854)
* mtmv support multi catalog

* mtmv support multi catalog
2023-05-19 16:44:55 +08:00