Commit Graph

18263 Commits

Author SHA1 Message Date
76767f2867 [fix](fe) Fix UnsetVariableStmt write editlog in non master node (#31080)
* Problem introduced by https://github.com/apache/doris/pull/27552
2024-02-20 09:18:33 +08:00
689b2c7bc5 [fix](s3client) fix unable to access COS (#31111) 2024-02-20 09:16:28 +08:00
02bded2688 [Improve](common)Optimize logging performance with LOG.isDebugEnabled() (#31091)
* [Improve](common)Optimize logging performance with LOG.isDebugEnabled()

* fix error ut
2024-02-20 09:16:14 +08:00
4c34ebb1cf [fix](mtmv)Fix the case failure issue caused by the same catalog name #31058 2024-02-20 09:12:38 +08:00
faaa51fa9d [fix](regression-test) test_nested_type_with_rowstore is flaky (#31121) 2024-02-20 09:12:38 +08:00
7ca3be6d51 [fix](parquet) return error if schema changed in complex types (#31128)
Check the column type of complex type to prevent core dump in BE. ColumnReader will throw segmentation fault in the following case:
Change complex types in hive:

hive> create table struct_test(
           id int,
           sf struct<f1: int, f2: map<string, string>>) stored as parquet;

hive> insert into struct_test values
          (1, named_struct('f1', 1, 'f2', str_to_map('1:s2,2:s2'))),
          (2, named_struct('f1', 2, 'f2', str_to_map('k1:s3,k2:s4'))),
          (3, named_struct('f1', 3, 'f2', str_to_map('k1:s5,k2:s6')));

hive> alter table struct_test change sf sf struct<f1:int, f2: string>;
2024-02-20 09:12:38 +08:00
8f70c00a26 [fix](routine-load) add lock to avoid editlog out of order when concurrent update job (#31095) 2024-02-20 09:12:38 +08:00
7a1bd6abb0 [improvment](group_commit) Refector scan wal function (#30939)
Co-authored-by: Yongqiang YANG <98214048+dataroaring@users.noreply.github.com>
2024-02-20 09:12:38 +08:00
066d674358 [Fix](inverted index) fix inverted index read data opt not work on MOW (#31075) 2024-02-20 09:12:38 +08:00
4a33d9820a [fix](multi-catalog)fix getting ugi methods and unify them (#30844)
put all ugi login methods to HadoopUGI
2024-02-20 09:12:38 +08:00
d55d796c74 [branch-2.1](multi-table-load) reduce routine load job transaction info storage in memory #31127 2024-02-20 09:08:56 +08:00
9a708806e0 [fix](segcompaction) enable segcompaction by default (#30810) 2024-02-19 19:04:22 +08:00
56ebff8f74 Doris doc (#31099) 2024-02-19 18:35:29 +08:00
e892b967da [docs](functions) fix and complete agg functions docs #31114 2024-02-19 18:35:29 +08:00
3f9b597b6b [fix](auto-partition) Fix concurrent load same value of auto partition #31107 2024-02-19 18:35:29 +08:00
2bb184f43b [fix](docs) fix kafka-load error word #31116 2024-02-19 18:35:29 +08:00
7607bfc78d [bugfix](performance) fix performance problem (#31093) 2024-02-19 17:48:29 +08:00
180fc13f6b [fix](nereids) disable PushDownJoinOtherCondition rule for mark join (#31084) 2024-02-19 17:48:29 +08:00
8a3e6644d4 [fix](udf) fix java-udf coredump as get env return nullptr (#30986) 2024-02-19 17:23:24 +08:00
277de979c3 [fix](mysql-channel) initialize mysql serializer by default (#31083) 2024-02-19 17:22:03 +08:00
2f9bd3e3bb (enhance)(S3) Change s3 metric from bvar adder to latency recorder (#28861) 2024-02-19 17:22:03 +08:00
5ac4b6a137 [opt](Nereids) refine group by elimination column prune (#30953) 2024-02-19 17:22:03 +08:00
Pxl
bb4575a392 [Improvement](join) optimization for build_side_output_column (#30826)
optimization for build_side_output_column
2024-02-19 17:22:03 +08:00
ed30b658bb fix pick_pr.sh push_id (#31041) 2024-02-19 17:22:03 +08:00
b3ac2128dd [Refactor](catalog) Refactor Jdbc Catalog external name case mapping rules (#28414) 2024-02-19 17:22:03 +08:00
8db2824c44 [bugfix](es catalog) add constant_keyword wildcard data type (#30947) 2024-02-19 17:20:21 +08:00
bda2042c78 [enhancement](paimon)add obs filesystem for paimon (#31061) 2024-02-19 17:20:21 +08:00
2f960c49f5 [Fix](executor)Fix query runtime statistics report failed #31064 2024-02-19 17:20:21 +08:00
e29d1ff59d [Fix](Docs) add thrift brew command #31077 2024-02-19 17:20:21 +08:00
ae809cd900 Fix hive p2 case. (#31072) 2024-02-19 17:20:21 +08:00
bb91c3a1cf [enhancement](neredis)add PushDownFilterThroughGenerate rule (#31057) 2024-02-19 17:20:21 +08:00
6504f6de74 [pipelineX](bug) Fix ConcurrentModificationException for profile reporting (#31040) 2024-02-19 17:20:21 +08:00
3f126fdd63 [fix](repository) fix NullPointerException when execute create repository statement without properties (#30972)
Co-authored-by: caiconghui1 <caiconghui1@jd.com>
2024-02-19 17:20:21 +08:00
a7037df9a1 [fix](iceberg) iceberg hms support hive1 (#30985) 2024-02-19 17:20:21 +08:00
6e4f76de54 [improvement](jdbc catalog) Delete unnecessary schema and optimize insert logic (#30880)
In the previous design, we were compatible with MySQL's auto-increment column and default value to bypass the null value check when writing back Jdbc External Table. However, because MySQL's default value is not completely unified with Doris, this resulted in The unsuitable default value is wrong. In response to this situation, I made the following optimizations
1. For JDBC External Table, we always allow certain columns to be missing during insertion. Even if these columns are not allowed to be empty at the source end, the error should be generated by the source end, not Doris herself.
2. When the target column is non-nullable and the insertion is done via `INSERT INTO tbl VALUES()` or `INSERT INTO tbl SELECT constants`, Doris should verify any inconsistency between them and throw an exception. This check is not applied for `INSERT INTO tbl SELECT ... FROM tbl` operations.
2024-02-19 17:20:21 +08:00
3da168afc9 Fix hive sample case ndv value. (#31043) 2024-02-19 17:20:21 +08:00
d66559f693 [bugfix](sessionVariable) Fix sessionVariable has sessionOriginValue property, and execute cloneSessionVariable method will throw java.io.NotSerializableException: java.lang.reflect.Field (#31029) 2024-02-18 14:45:25 +08:00
5ea46f210c [pipelineX](bug) Fix use-after-free when BE exits (#31042) 2024-02-18 14:45:25 +08:00
edfa5e750b Remove slow auto analyze test case. (#31037) 2024-02-18 14:45:25 +08:00
870a9342b7 [fix](function) fix extract_url_parameter's bug then get the last key (#30929)
fix extract_url_parameter's bug then get the last key
2024-02-18 14:45:25 +08:00
6cf7468073 [enhancement](function) change some function nullable mode (#30991)
change some function nullable mode
2024-02-18 14:45:25 +08:00
68102fd531 [Fix](auto-partition) fix a concurrent bug of extremely long values (#31005) 2024-02-18 14:45:25 +08:00
753ff10840 [bugfix](export command) Export exceptions when use legacy optimizer (#31020) 2024-02-18 14:45:25 +08:00
33aaacf2fa [Bug] (plsql) procedure call to a non existing procedure, returning success (#31019)
Issue Number: close. #31018

state was set OK even in failure case. changed it.
2024-02-18 11:50:17 +08:00
a3c78dd21a [chore](refactor) refactor some rf code and delete rpc file (#31031)
---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-02-18 11:50:17 +08:00
d70776af55 [feature](agg-func) support covar and covar_samp function (#30983) 2024-02-18 11:50:17 +08:00
7b79b77cc9 [Optimize](Variant) make tablet schema more well-organized (#99) (#30922) 2024-02-18 11:50:17 +08:00
b0d2ecbf52 [Improve](Tablet Schema) Use deterministic way to serialize protobuf (#101) (#30906) 2024-02-18 11:50:17 +08:00
f9e16e08eb [fix](script) Fix start_fe.sh --image path check image not work (#30695) 2024-02-18 11:50:17 +08:00
b5012dc55a [Enhancement](group commit) optimize pre allocated calculation (#30893) 2024-02-18 11:50:17 +08:00