facb7cf4e2
[fix](spark load)Temp partition with spark load ( #14648 )
...
* [fix](spark load)losing temporary partition item entry
* [fix](spark load)Temp partition with spark load
2022-11-29 15:21:44 +08:00
c5f9fd5619
[fix](spark load)partition column is not duplicate key, spark load IndexOutOfBounds error ( #14661 )
...
* [fix](spark load)partition column is not duplicate key，spark load IndexOutOfBoundsException error
Co-authored-by: 张放(vivianv.zhang) <vivianv.zhang@huolala.cn >
2022-11-29 15:21:21 +08:00
3e8b3658c7
[feature-wip](decimalv3) Support basic agg and arithmetic operations for decimal v3 ( #14513 )
2022-11-29 15:12:41 +08:00
82da071b45
[Chore](format) update clang-format version to 15 ( #13036 )
...
update clang-format version to 15
2022-11-29 14:46:10 +08:00
97f0d3a756
[Improvement](datatype) disable new types if vectorized engine is disabled ( #14561 )
...
* [Imptovement](datatype) disable new types if vectorized engine is disabled
disable datev2/datetimev2/decimalv3 if vectorized engine is disabled
2022-11-29 10:33:46 +08:00
1bddf9ba5c
[docs](readme)update the user numbers ( #14639 )
2022-11-29 10:28:52 +08:00
f7a827c06b
[fix](new-scan) fix some bugs about new scan node and readers ( #14504 )
...
json reader DCHECK fail because of missing TYPE_STRING
fix bug that if no file is found, the tvf will throw NPE.
The predicate conjuncts can not be pushed down to parquet reader if this is a load task.
Because the predicate should be applied on column of dest table, not on column of source file.
Add a temp property "use_new_load_scan_node" of broker load to make regression test happy.
So that we can use new load scan node for a certain job and avoid setting global FE config.
2022-11-29 10:21:41 +08:00
e1f0fa069c
[enhancement](memory) Refactored process memory statistics periodically refresh, and fix catch bad_alloc ( #14580 )
2022-11-29 10:15:25 +08:00
0daebde223
[fix](java-udf) Disable the corresponding configuration if building BE without Java UDF support ( #14303 )
2022-11-29 10:12:00 +08:00
2295ab24b0
[fix](metric) fix jvm_young_size_bytes. ( #14562 )
...
Co-authored-by: wangxiangyu@360shuke.com <wangxiangyu@360shuke.com >
2022-11-29 09:10:48 +08:00
7513c82431
[NLJoin](conjuncts) separate join conjuncts and general conjuncts ( #14608 )
2022-11-29 08:55:54 +08:00
c5eb8ab084
[fix](persiste) make ArithmeticExpr wriable ( #14615 )
...
Fix bug that the ArithmeticExpr's write method is not implement, causing FE crash when creating function like:
CREATE ALIAS FUNCTION IF NOT EXISTS mesh_udf_test1(INT,INT) WITH PARAMETER(n,d) AS ROUND(1+floor(n/d));
Add if exists and if not exists for drop and create function
Fix a minor bug that if file does not exist, hdfs() table valued function will throw NPE
2022-11-29 08:55:18 +08:00
daeabcf053
[improvement](vec) optimize the logic for _has_null in ColumnNullable ( #14633 )
2022-11-29 08:53:30 +08:00
39dd6682f2
[typo](docs)change the metadata directory from palo-meta to doris-meta #14647
2022-11-29 08:52:55 +08:00
e5e94e128d
[docs](macOS) Fix the render ( #14643 )
2022-11-29 08:22:06 +08:00
b51f6ae050
[feature](Nereids)add rule: PruneOlapScanTablet ( #14378 )
2022-11-29 01:06:14 +08:00
a803e75438
[feature](Nereids) add rule: EliminateGroupByConstants ( #14541 )
...
remove group by constants, like:
before apply rule:
select 1, k1, min(k2), max(k3) from t1 group by 1, 2;
after apply rule:
select 1, k1, min(k2), max(k3) from t1 group by k1;
2022-11-28 22:52:24 +08:00
8d1d144eed
[doc](1.2) add version tag for feature in 1.2 ( #14624 )
2022-11-28 20:39:53 +08:00
0702277196
[improvement](tcmalloc) add moderate mode and avoid oom with a lot of cache ( #14374 )
...
ReleaseToSystem aggressively when there are little free memory.
2022-11-28 20:17:51 +08:00
16bc20a357
[opt](nereids)Estimate cost by row, not by data size ( #14471 )
...
Since column data size is not always available, estimate plan cost by row count instead of data size.
2022-11-28 19:58:06 +08:00
c7da050da4
[fix](test) tpch_sf1_p1 and tpch_sf1_p1/tpch_sf1 are confusing ( #14206 )
2022-11-28 19:30:32 +08:00
1e690ea6aa
[fix](bitmapfilter) Set bitmap filter waiting time to the query timeout. ( #14623 )
...
bitmap filter is precise filter and only filter once, so it must be applied.
2022-11-28 18:57:27 +08:00
529bdfb153
[Fix](function) Fix retention function return wrong value type ( #14552 )
...
MySQL [db]> SELECT SUM(a.r[1]) as active_user_num, SUM(a.r[2]) as active_user_num_1day, SUM(a.r[3]) as active_user_num_3day, SUM(a.r[4]) as active_user_num_7day FROM ( SELECT user_id, retention( day = '2022-11-01', day = '2022-11-02', day = '2022-11-04', day = '2022-11-07') as r FROM login_event WHERE (day >= '2022-11-01') AND (day <= '2022-11-21') GROUP BY user_id ) a;
ERROR 1105 (HY000): errCode = 2, detailMessage = sum requires a numeric parameter: sum(%element_extract%(a.r, 1))
2022-11-28 15:56:18 +08:00
73a600fba8
bug fix for outfile ( #14550 )
...
Co-authored-by: cambyzju <zhuxiaoli01@baidu.com >
2022-11-28 15:46:41 +08:00
9d087a01f3
[improvement](startup) not print error stack for OLAP_ERR_TABLE_ALREADY_DELETED_ERROR because there are too much errors during startup ( #14627 )
...
If there are too much deleted tablets in RocksDB, there are many OLAP_ERR_TABLE_ALREADY_DELETED_ERROR during startup and will try to get error stack. It will cost a lot of time and the start process taks very very long.
Co-authored-by: yiguolei <yiguolei@gmail.com >
2022-11-28 15:43:24 +08:00
f6154a65a6
[docs](macOS) Add compilation guides for macOS ( #14634 )
...
* [docs](macOS) Add compilation guides for macOS
* Add to sidebar
2022-11-28 15:13:50 +08:00
d3cb79c629
[regressiontest](fuzzy) modify window function and schema change test to pass fuzzy ( #14632 )
...
enable fe fuzzy mode for P0
set parallel instance num = 1 for window function
sleep 1000ms for schema change test or the data result is wrong.
2022-11-28 14:58:27 +08:00
d712c4efe1
[Enhancement](predicate) move create column predicate to create_predicate_function ( #14588 )
...
move create column predicate to create_predicate_function
use same macro to create_column_predicate and create_predicate_function
2022-11-28 14:13:40 +08:00
c0e25a1c37
[fix](Nereids) diable unstable test in graph simplifier ( #14630 )
2022-11-28 14:07:14 +08:00
b9270dace3
[fix](nereids) after injection, min/max value in columnStats for date/dateV2 type is wrong ( #14605 )
2022-11-28 14:05:33 +08:00
2bc43f9757
[fix](ui) clean npm cache before install ( #14629 )
...
npm ERR! Unexpected end of JSON input while parsing near '...ih/ae0E6HfGdwwO/\r\na'
2022-11-28 12:12:51 +08:00
ed92a8f81e
[feature](jsonb function)change jsonb_extract_string behavior and doc ( #14619 )
...
1. change jsonb_extract_string behavior: convert to string instead of NULL if the type of json path is not string
2. move jsonb tutorial doc to JSONB data type
2022-11-28 11:36:54 +08:00
a0327eca07
[improvement](load) add more log on rpc error ( #14559 )
...
* [improvement](load) add more log on rpc error
* update
2022-11-28 08:32:42 +08:00
39c47d930b
[improvement](load) add more log on rpc error ( #14559 )
...
* [improvement](load) add more log on rpc error
* update
2022-11-28 08:32:20 +08:00
b6605b99aa
[ehancement](nereids) eliminate project in the post process phase ( #14490 )
...
Remove those projects that used for column pruning only and don't do any expression calculation, So that we could avoid some redundant data copy in do_projection of BE side.
2022-11-28 00:39:36 +08:00
280f8be4bd
[test](regression) adjust nereids related regression cases under datev2 ( #14578 )
...
1. revert 14439, recovery dup&unique test cases
2. adjust nereids related case
2022-11-27 23:57:51 +08:00
230ede9085
[opt](nereids) avoid broadcast join if hash table is big ( #14240 )
...
1. when we choose broadcast join, we only consider transferring less data. This may lead OOM, if hash table is big enough.
2. fix a bug in `Stats.computeSize()`. ColumnStats.dataSize is the total size of this column, but we need the byte of one cell.
2022-11-27 23:22:43 +08:00
948ee41632
[opt](planner) let cardinality in explain result more readable ( #14330 )
...
1. add common for big int in explain. for example "1500000" will be printed as "1,500,000"
2. Scan node cardinal is missing
2022-11-27 23:12:41 +08:00
e17519b00f
[docs](docs)Add Docker Image build documentation ( #14622 )
...
Add Docker Image build documentation
2022-11-27 22:43:53 +08:00
cf2de85633
[Document](function) add docs of hll_cardinality/hll_empty/hll_hash ( #14547 )
2022-11-27 13:40:16 +08:00
93b940bc92
[test](jdbc)add new case for mysql jdbc table ( #14581 )
2022-11-27 13:39:59 +08:00
78adecac1b
[enhancemennt](be)optimize mem usage in join and set node ( #14602 )
2022-11-27 13:38:49 +08:00
38b4cbe253
[Bug](regression) regression fail random in fuzzy mode ( #14614 )
2022-11-27 09:23:36 +08:00
b3859e1e1a
[ehancement](fe) Remove unnecessary kill in AutoCloseConnectContext ( #14606 )
...
The invocation in ConnectContext.kill in AutoCloseConnectContext is redundant and caused too many useless logs
2022-11-26 23:54:33 +08:00
36419fae48
[fix](JdbcExecutor) fix that JdbcExecutor did not load the class jar ( #14598 )
...
JdbcExecutor did not load jdbc driver jar, so add classloader to load jdbc jar.
2022-11-26 23:53:05 +08:00
a877c8e50d
[test](docker) delete show table ( #14612 )
2022-11-26 23:44:29 +08:00
6b8f1a1a88
[typo](docs) fix four string functions' document issue #14610
...
Co-authored-by: yuleiyang <yuleiyang@tencent.com >
2022-11-26 19:27:40 +08:00
dd21056a4c
[fix](nereids) delete view in regression-test ( #14607 )
2022-11-26 18:03:21 +08:00
d5d3f7e0b7
[fix](memtracker) Fix thrift BackendService thread local is not initialized, memtracker init fail ( #14589 )
2022-11-26 13:04:39 +08:00
4c60186e87
[test](jdbc)add new case for pg jdbc table ( #14582 )
2022-11-26 13:02:05 +08:00
