b445c783eb
[test](tvf) move p2 tvf tests from p2 to p0 ( #37081 ) ( #37152 )
...
bp: #37081
2024-07-02 22:38:22 +08:00
74086189d3
[test](tvf) move p2 tvf tests from p2 to p0 ( #36871 ) ( #37150 )
...
bp: #36871
2024-07-02 22:37:43 +08:00
bd24a8bdd9
[Fix](csv_reader) Add a session variable to control whether empty rows in CSV files are read as NULL values ( #37153 )
...
bp: #36668
2024-07-02 22:12:17 +08:00
e25717458e
[opt](catalog) add some profile for parquet reader and change meta cache config ( #37040 ) ( #37146 )
...
bp #37040
2024-07-02 20:58:43 +08:00
c7549acf29
[Fix](Nereids) fix leading with different be instance number ( #36613 ) ( #36967 )
...
cherry-pick: #36613
Problem:
When use different be number to test leading explain shape plan,
physical distribute plan would differ due to different be numbers
Solved:
Disable physical distribute node showing in fix leading cases
2024-07-02 19:56:17 +08:00
42f4271e9d
[enhancement](nereids) speedup sql cache with variable ( #37090 ) ( #37119 )
...
cherry pick from #37090
2024-07-02 19:25:22 +08:00
f5572ac732
[pick]reset memtable flush thread num ( #37092 )
...
## Proposed changes
pick #37028
2024-07-02 19:20:17 +08:00
b63e6ac350
[fix](ES Catalog)Add array types support in esquery function ( #36936 ) ( #37054 )
...
backport #36936
2024-07-02 17:48:35 +08:00
239bc1a7e0
[fix](compile) fix compile failed on MacOS due to ambiguous std::abs ( #37136 )
...
cherry-pick #35125 to branch-2.1
Co-authored-by: morrySnow <101034200+morrySnow@users.noreply.github.com >
2024-07-02 17:45:33 +08:00
f5d0cdeeb4
[branch-2.1] avoid glog coredump when running with ASAN ( #37134 )
...
## Proposed changes
This is just a workround try avoid coredump like this:
```
#0 0x56414f0e8ed1 in __asan::CheckUnwind() crtstuff.c
#1 0x56414f1009a2 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) crtstuff.c
#2 0x56414f0ecbf3 in __asan::AsanThread::GetStackFrameAccessByAddr(unsigned long, __asan::AsanThread::StackFrameAccess*) crtstuff.c
#3 0x56414f050d87 in __asan::AddressDescription::AddressDescription(unsigned long, unsigned long, bool) crtstuff.c
#4 0x56414f052a73 in __asan::ErrorGeneric::ErrorGeneric(unsigned int, unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long) crtstuff.c
#5 0x56414f0e6a9e in __asan::ReportGenericError(unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long, unsigned int, bool) crtstuff.c
#6 0x56414f066885 in gmtime_r (/mnt/hdd01/ci/branch21-deploy/be/lib/doris_be+0x17ef3885) (BuildId: f58eb5e327529636)
#7 0x564177940521 in google::LogMessage::Init(char const*, int, int, void (google::LogMessage::*)()) crtstuff.c
#8 0x564151de36fc in doris::Status doris::ThriftRpcHelper::rpc(std::__cxx11::basic_string, std::allocator> const&, int, std::function&)>, int) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/util/thrift_rpc_helper.cpp:76:13
#9 0x56417603cda7 in doris::vectorized::VRowDistribution::automatic_create_partition() /home/zcp/repo_center/doris_branch-2.1/doris/be/src/vec/sink/vrow_distribution.cpp:99:5
#10 0x56417614cffa in doris::vectorized::VTabletWriter::_send_new_partition_batch() /home/zcp/repo_center/doris_branch-2.1/doris/be/src/vec/sink/writer/vtablet_writer.cpp:1346:9
....
```
2024-07-02 17:45:04 +08:00
cf86eb8647
[test](migrate) move test_hive_text_complex_type from p2 to p0 ( #37007 ) ( #37123 )
...
bp: #37007
2024-07-02 17:36:37 +08:00
d0eea3886d
[fix](multi-catalog) Revert #36575 and check nullptr of data column ( #37086 )
...
Revert #36575 , because `VScanner::get_block` will check
`DCHECK(block->rows() == 0)`, so block should be cleared when `eof =
true`.
2024-07-02 15:32:52 +08:00
7443e8fcf2
[cherry-pick](branch-2.1) fix single compaction test p2 #34568 #36881 ( #37075 )
2024-07-02 15:22:04 +08:00
859a7c80b8
[fix](parquet) prevent parquet page reader print much warning logs ( #37012 )
...
bp #37011
2024-07-02 14:33:01 +08:00
496024ffb5
[fix](mtmv) mtmv add case and fix bug ( #36019 ) ( #36600 )
...
pick from: #36019
2024-07-02 14:24:53 +08:00
ebc1afed4a
[enhance](mtmv)when calculating the availability of MTMV, no longer c… ( #36617 )
...
pick: #36507
2024-07-02 14:24:33 +08:00
0c8934642e
[opt](nereids)using mv's derived stats #35721 ( #37098 )
...
pick from #35721
(cherry picked from commit a481d79b1e258f7b6711157bb0d5e4c224e680e5)
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
2024-07-02 13:54:05 +08:00
c66975b244
[branch-2.1] Pick "[Fix](regression) Fix p0 case test_unique_table_auto_inc_concurrent #37048 " ( #37050 )
...
## Proposed changes
pick https://github.com/apache/doris/pull/37048
2024-07-02 11:15:50 +08:00
a9764b9d39
[chore](multi catalog) Print serde properties when show create hive-external-table ( #34966 ) ( #37096 )
...
bp #34966
Co-authored-by: xy720 <22125576+xy720@users.noreply.github.com >
2024-07-02 11:07:10 +08:00
6789f5bc80
[fix](null safe equal join) fix coredump if both sides of the conjunct is not nullable #36263 ( #37073 )
2024-07-02 11:01:55 +08:00
011f203d71
[opt](log) Remove unnecessary warning log ( #37093 )
...
When enable_profile = true or report_succeed=true, it is very likely
fe/fe-core/src/main/java/org/apache/doris/qe/QeProcessorImpl.java::reportExecStatus
will print much warning log. It is not necesary.
2024-07-02 10:53:36 +08:00
3f382b797a
[branch-2.1][improvement](sqlserver catalog) Configurable whether to use encrypt when connecting to SQL Server using the catalog ( #36971 )
...
pick (#36659 )
pick #37015
In previous versions, we used druid as the default JDBC connection pool,
which can use custom decryption to parse the certificate when SQL Server
encryption is turned on. However, in the new version, after changing
HikariCP as the default connection pool, the SQLServer certificate
cannot be parsed, so encryption needs to be turned off for normal use.
Therefore, a parameter is added to decide whether to disable SQLServer
encryption. It is not disabled by default.
2024-07-02 10:14:43 +08:00
6f944549d1
[fix](regression)fix case failure ( #37058 )
2024-07-02 09:55:18 +08:00
6425ce8a89
[Fix](Prepared Statment) use fixed charset to init StringLiteral ( #37084 )
...
picked from #36860
2024-07-01 23:11:13 +08:00
e686e85f27
[opt](split) add max wait time of getting splits ( #36842 )
...
bp: #36843
2024-07-01 22:05:25 +08:00
72c20d3ccc
[branch-2.1](function) fix date_format and from_unixtime core when meet long format string ( #35883 ) ( #36158 )
...
pick #35883
2024-07-01 20:35:31 +08:00
89b4918e88
[branch-2.1](stmt) fix show create table consistency #37074 ( #37079 )
...
pick #37074
2024-07-01 20:35:05 +08:00
798d9d6fc6
[pick21][opt](mow) reduce memory usage for mow table compaction ( #36865 ) ( #36968 )
...
cherry-pick https://github.com/apache/doris/pull/36865 to branch-2.1
2024-07-01 15:33:18 +08:00
24d236b210
[feat](Nereids) Optimize Sum Literal Rewriting by Excluding Single Instances ( #35559 ) ( #37047 )
...
pick from master #35559
This PR introduces a change in the method removeOneSumLiteral to enhance
the performance of sum literal rewriting in SQL queries. The
modification ensures that sum literals appearing only once, such as in
expressions like select count(id1 + 1), count(id2 + 1) from t, are not
rewritten.
2024-07-01 14:57:15 +08:00
14c991f09b
[Feature](Prepared Statement) fix and enable enable_server_side_prepared_statement by default #36581 ( #36818 )
...
picked from #36581
2024-07-01 14:35:17 +08:00
62c4451c97
[branch-2.1][improvement](jdbc catalog) Modify the maximum number of connections in the connection pool to 30 by default ( #37023 )
...
pick (#36720 )
In many cases, we found that users would use JDBC Catalog to perform a
large number of queries, which resulted in the maximum of 10 connections
being insufficient, so I adjusted it to 30, which covered most needs.
2024-07-01 12:22:20 +08:00
4210a6a8d6
[branch-2.1] PIck "[Fix](autoinc) Hanlde the processing of auto_increment column on exchange node rather than on TabletWriter when using TABLET_SINK_SHUFFLE_PARTITIONED #36836 " ( #37029 )
...
## Proposed changes
pick https://github.com/apache/doris/pull/36836
2024-07-01 09:56:30 +08:00
92cbbd2b75
[fix](clone) Fix clone and alter tablet use same tablet path #34889 ( #36858 )
...
cherry pick from #34889
2024-06-30 20:40:54 +08:00
07278e9dcb
[improvement](segmentcache) limit segment cache by memory or segment … ( #37035 )
...
…num (#37026 )
pick ##37026
2024-06-30 20:34:13 +08:00
12dddfc26c
[branch-2.1] Pick "[Fix](autoinc) try fix concurrent load problem with auto inc column #36421 " ( #37027 )
...
## Proposed changes
pick https://github.com/apache/doris/pull/36421
2024-06-30 13:10:03 +08:00
d237a4d303
[fix](array)fix array_except/union for left const return only one row result #36776 ( #36986 )
2024-06-30 12:25:17 +08:00
021ed1d690
[fix](map)fix upgrade behavior from 1.2 version #36635 ( #36983 )
2024-06-30 12:11:03 +08:00
e25b0d7c37
[branch-2.1][improvement](mysql catalog) disable mysql AbandonedConnectionCleanupThread ( #36970 )
...
pick (#36655 )
2024-06-29 18:35:41 +08:00
a9855ca173
[Improvement](set) enable admin_set_frontend_config can apply to all fe ( #37022 )
...
bp #34685
2024-06-29 09:17:38 +08:00
d0c0a7b9ae
[Fix](variant) ignore serialization of nothing type ( #37006 )
...
picked from #36997
2024-06-28 18:41:40 +08:00
7c6d280def
[Test](manager_interface)append manager interface test. ( #35889 ) ( #36912 )
...
bp #35889
---------
Co-authored-by: daidai <2017501503@qq.com >
2024-06-28 18:38:33 +08:00
cb80ae906f
[Bug](runtime-filter) disable sync filter when pipeline engine is off ( #36994 )
...
## Proposed changes
1. disable sync filter when pipeline engine is off
2. reduce some warning log
2024-06-28 16:59:26 +08:00
4dcceaefea
[test](ES Catalog) Add test cases for ES 5.x ( #34441 ) ( #36993 )
...
backport #34441
2024-06-28 16:58:07 +08:00
4bec08c7e8
[fix](inverted index)Make build index operation only affect base index ( #36869 ) ( #36988 )
...
backport #36869
2024-06-28 16:57:07 +08:00
72de9c1b8f
[fix](autoinc) avoid duplicated auto inc when role of fe changes ( #36961 )
...
## Proposed changes
pick #36960
Issue Number: close #xxx
<!--Describe your changes.-->
2024-06-28 16:54:48 +08:00
b46c86f352
[improvement](partition rebalance) improve partition rebalance choose candidate speed #36509 ( #36976 )
...
cherry pick from #36509
2024-06-28 16:31:50 +08:00
fe75cbc927
[case](udf) Only one backend, skip scp udf file ( #36810 ) ( #36964 )
...
backport #36810
2024-06-28 16:31:30 +08:00
47e56cc326
[fix](fe ut) fix unstable SystemInfoServiceTest #36893 ( #36974 )
...
cherry pick from #36893
2024-06-28 16:30:47 +08:00
65e8bbf0f2
[improvement](statistics)Disable fetch stats for iceberg table through Iceberg api by default. ( #36931 ) ( #36999 )
...
backport https://github.com/apache/doris/pull/36931
2024-06-28 16:23:25 +08:00
816899df41
[improvement](statistics)Use real base index id to fetch stats cache. ( #36914 ) ( #36992 )
...
For historical reason, statistics tables use -1 for OlapTable base index
id. This brings many if/else branch for stats calculate. This pr is to
screen the -1 for Nereids. The stats user could use the real base index
id to fetch stats cache. Will do the id translation inside the get cache
api.
backport: https://github.com/apache/doris/pull/36914
2024-06-28 16:22:20 +08:00
