d6ff9744c9
[feature](Nereids) covert predicate to SARGABLE ( #25180 )
...
covert predicate to SARGABLE
1. support format like `1 - a`
2. support rearrange `year/month/week/day/minutes/seconds_sub/add` function
2023-10-12 14:46:56 +08:00
c63bf24c84
[Improvement](statistics) Improve sample count accuracy ( #25175 )
...
While doing sample analyze, the result of row count, null number and datasize need to multiply a coefficient based on
the sample percent/rows. This pr is mainly to calculate the coefficient according to the sampled file size over total size.
2023-10-12 14:42:02 +08:00
22684dedff
[pipelineX](pick) pick PRs from pipeline ( #25340 )
2023-10-12 14:35:32 +08:00
80a49ed97a
[fix](nereids)fix some function signature issue ( #25301 )
...
1. remove wrong signature of nvl
2. the promoted type datetimev2 for datetime should be datetimev2(0)
2023-10-12 01:23:20 -05:00
a0d3206d78
[fix](Nereids) support nested complex type literal ( #25287 )
2023-10-12 01:17:38 -05:00
2664d1cffb
[chore](vec) Make this copy constructor of StringRef explicit ( #25337 )
2023-10-12 14:12:46 +08:00
42f8b253aa
[function](nereids) support array_apply/array_repeat/group_uniq_array/ipv4numtostring ( #25249 )
...
nereids support functions: array_apply/array_repeat/group_uniq_array/ipv4numtostring
2023-10-12 11:08:42 +08:00
a0d2b1ec56
[Bug](materialized-view) fix not match mv when some alias on agg ( #25321 )
...
fix not match mv when some alias on agg
2023-10-12 11:02:55 +08:00
f14e4311c4
[Chore](check) add length check for BufferWritable ( #25322 )
...
add length check for BufferWritable
2023-10-12 10:51:50 +08:00
7447ac71b5
[minor](format) fix BE code format ( #25328 )
2023-10-12 10:34:36 +08:00
022762d5f0
[fix](memory) Fix work load group GC and add logs to locate slow GC #24975
...
Fix work load group GC, add cancel load and add logs.
Unify the format and change all to lowercase of GC logs, avoid unnecessary trouble when grep or less
Add logs to help locate the cause of slow GC.
2023-10-12 10:33:56 +08:00
2014e16cfb
[fix](es catalog)fix es http timeout ( #25273 )
2023-10-12 10:21:55 +08:00
c4916184f1
[regression-test](topn opt) fix case that explain with wrong result ( #25316 )
...
`select * from tbl where k = 1 order by k limit 1` will eliminate sort in original planner
2023-10-12 10:16:42 +08:00
7ca63665b4
[fix](agg) garbled characters in result of map_agg ( #25318 )
2023-10-12 10:10:55 +08:00
58d96ecdbf
[Improve](status) avoid print too may stack log for DATA_QUALITY_ERROR code ( #25292 )
2023-10-12 09:58:51 +08:00
46ab4346ca
[Opt](parquet reader) Optimize the performance of reading decimal in parquet reader. ( #25012 )
...
Optimize the performance of reading decimal in parquet reader.
- Static dispatch `DecimalScaleParams`.
- Optimize `memcpy`, static dispatch copy size in fixed length cases.
- Use right shift bit operator to convert decimals.
2023-10-12 09:53:08 +08:00
e41b03e530
[Fix](multi-catalog) delete hdfs hedged configs at BE side. ( #25094 )
...
Issue Number: close #25093
We can set hdfs hedged configs when creating catalog, just like this:
```
CREATE CATALOG `test_ctl` PROPERTIES (
...
"dfs.client.hedged.read.threadpool.size" = "128",
"dfs.client.hedged.read.threshold.millis" = "500",
...
);
```
It is redundant to set these configs at BE side, and it will brings an occasional bug at #25093 .
2023-10-11 23:25:30 +08:00
9a4baf7ccf
[fix](Nereids)Fix the bug that count(*) does not push down for tables with only one column. ( #25222 )
...
after pr #22115 .
Fixed the bug that when selecting count(*) from table, if the table has only one column, the aggregate count is not pushed down.
2023-10-11 23:17:30 +08:00
d1f59a4025
[fix](catalog)fix when modifying comments in property, it will modify the comments in the catalog ( #24857 )
...
- fix when modifying comments in property, it will modify the comments in the catalog
- add `alter catalog modify comment` to modify comment for catalog
- abstract some logic of `alter catalog` to parent class
2023-10-11 23:16:19 +08:00
73c3e3ab55
[Feature](x-load) support config min replica num for loading data ( #21118 )
2023-10-11 21:07:35 +08:00
ba87f7d3a3
[fix](pipelineX) add table sink and some fix in pipelineX ( #25314 )
2023-10-11 20:18:08 +08:00
e94fca4949
[enhancement](bvar) add metrics to monitor load throughput ( #25189 )
...
to monitor realtime load throughput of the BE
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com >
2023-10-11 19:56:33 +08:00
f960b8c989
[bugfix](stream receiver) be will core during stop because receiver is not closed ( #25298 )
...
Co-authored-by: yiguolei <yiguolei@gmail.com >
2023-10-11 19:49:40 +08:00
9d5ff5b55d
[doc](readme)Update 2.0.2 release note in README.md ( #25324 )
2023-10-11 19:49:00 +08:00
b8ba83faac
[Chore](check) ignore license check on .github ( #25323 )
...
ignore license check on .github
2023-10-11 19:44:05 +08:00
f680a2141d
[enhancement](regression-test) add routine load json case ( #25253 )
2023-10-11 19:43:08 +08:00
b17bac6323
[fix](case) use the custom DB explicitly in analyze_stats.groovy ( #25285 )
...
Co-authored-by: stephen <hello-stephen@qq.com >
use the custom DB explicitly in analyze_stats.groovy
2023-10-11 19:28:14 +08:00
46be6c07e1
[opt](Nereids) expose multi distinct functions ( #25309 )
2023-10-11 05:42:39 -05:00
c6b1c903e4
[fix](Regression-test) fix that the String type in a nested type should contain double quotes and add regression-test ( #25115 )
2023-10-11 18:30:26 +08:00
e514d52232
[fix](point-query) Support mow table with sequence column ( #25308 )
2023-10-11 18:22:16 +08:00
1e300d895d
[improvement](checkpoint) checkpoint thread update tablet invert index ( #25098 )
2023-10-11 18:18:03 +08:00
73f632a4e3
[fix](move-memtable) handle error in LoadStreamWriter::close ( #24805 )
2023-10-11 16:54:42 +08:00
3db33207d4
[pipelineX](fix) Fix nullable types for set operator ( #25294 )
2023-10-11 16:50:54 +08:00
0170a63c40
[fix](workflow) checkout the pr, not the base ( #25310 )
...
Co-authored-by: stephen <hello-stephen@qq.com >
checkout the pr, not the base
2023-10-11 16:50:25 +08:00
2d19f2fbfe
[fix](planner)need call materializeSrcExpr for materialized slots in join node ( #25204 )
2023-10-11 16:34:53 +08:00
dabeeb0338
[fix](planner)should always use plan node's getTblRefIds method to get unassigned conjuncts for this node ( #25130 )
2023-10-11 16:34:21 +08:00
2221c8e2ed
[fix](planner)implicit cast should use type member variable instead of targetTypeDef ( #24582 )
2023-10-11 16:33:48 +08:00
e9554e36a8
[fix](nereids)disable parallel scan in some case ( #25089 )
2023-10-11 16:32:09 +08:00
df7724d6cb
[typo](docs)delete wrong description of from_unixtime ( #23897 )
2023-10-11 03:20:13 -05:00
004d3264a6
[typo](docs) add 'order by' when use 'limit m,n' ( #24236 )
2023-10-11 03:15:33 -05:00
193ef79df2
[chore](fix) Update .asf.yaml ( #25300 )
...
fix issue introduced by #25101
...In order to solve this, we’ve added a new pull_request_target event, which behaves in an almost identical way to the pull_request event with the same set of filters and payload.
However, instead of running against the workflow and code from the merge commit, the event runs against the workflow and code from the base of the pull request.
2023-10-11 03:15:16 -05:00
a9b84ae6ee
[test](nereids)add more case in PushdownFilterThroughAggregationTest ( #24927 )
2023-10-11 16:14:36 +08:00
6d999f5b95
[enhancement](nereids)add eliminate filter on one row relation rule ( #24980 )
...
1.simplify PushdownFilterThroughSetOperation rule
2.add eliminate filter on one row relation rule
2023-10-11 16:12:24 +08:00
48b04752a5
[Chore](cases) set enable_nereids_planner on mv cases ( #25277 )
...
set enable_nereids_planner on mv cases
2023-10-11 15:58:06 +08:00
47578c0fc9
[fix](Nereids) fix toSql of date literal ( #25243 )
...
toSql should return '2023-2-1 ' for DateLiteral 2023-2-1
2023-10-11 13:04:05 +08:00
0d603dd4c3
[Bug](delete) Use date as common type for date comparison ( #25262 )
2023-10-11 11:51:43 +08:00
bb670118f5
[coverage](test) Delete unused function to improve test coverage ( #25233 )
2023-10-11 11:50:51 +08:00
cdf5f0fe68
[fix](pipelineX) mark join column should be nullable ( #25275 )
2023-10-11 11:35:43 +08:00
1e6d34d1d0
[Enhancement](sql-cache) Add partition update time for hms table and use it at sql-cache. ( #24491 )
...
Now FE does not record the update time of hms tbl's partitons, so the sql cache may be hit even the hive table's partitions have changed. This pr add a field to record the partition update time, and use it when enable sql-cache.
The cache will be missed if any partition has changed at hive side.
Use System.currentTimeMillis() but not the event time of hms event because we would better keep the same measurement with the schemaUpdateTime of external table. Add this value to ExternalObjectLog and let slave FEs replay it because it is better to keep the same value with all FEs, so the sql-cache can be hit by the querys through different FEs.
2023-10-11 11:05:16 +08:00
2f706cc84b
[compile](simdjson reader) use __AVX2__ macro to decide whether use simdjson to parse ( #25165 )
2023-10-11 10:50:13 +08:00
