Commit Graph

18263 Commits

Author SHA1 Message Date
2ae0bb7f13 [minor](test) remove unused function to improve test coverage (#18598) 2023-04-13 15:30:53 +08:00
Pxl
eb46bcb304 [Bug](materialized-view) fix match wrong index on some scan node (#18561)
fix match wrong index on some scan node
2023-04-13 11:50:14 +08:00
726402b53b [bugfix](topn) fix topn runtime predicate crash in short circuit evaluate for types like string decimal (#18409) 2023-04-13 11:10:59 +08:00
df0aaece1d [Function](test) add some test cases for agg functions (#18610) 2023-04-13 10:23:41 +08:00
4335c9998f [chore](ARM) Add some vectorization compatibility code on aarch64 (#18553)
update sse2noen to support more sse code on arm cpus
2023-04-13 10:15:33 +08:00
6d91635c5b [fix](json_reader) Do not increase the value of read_rows for empty line (#18611)
If read an empty row the row num++, the row num will be larger than actual column size, it will core.
2023-04-13 10:08:11 +08:00
3c3364ba27 [chore](row store) ignore serialize block to row column if no row store column (#18601) 2023-04-13 10:02:33 +08:00
d57371da13 [feature](struct-type) support basic struct constructor function (#18190)
This commit will support struct and named_struct function.
2023-04-13 09:18:00 +08:00
af0cf0c050 [Fix](multi catalog)Refresh table object while refresh external table. (#18592)
Refresh table object while refresh external table. Including:
Refresh catalog, refresh database and refresh table.
Before visiting database, need to guarantee catalog has been initialized.
Before visiting table, need to guarantee catalog and database have been initialized.
2023-04-13 08:49:44 +08:00
c5a55574c6 [fix](HttpServer) Disable https by default (#18608) 2023-04-12 23:35:44 +08:00
f9c6d7791c [typo](docs)Optimize SeaTunnel documentation (#18584) 2023-04-12 21:33:31 +08:00
1f9372558d [improve](regression case) Add more inverted index regression case (#18589)
1. add more inverted index regression case for unique mow
2. add inverted index case with different data types
2023-04-12 20:40:55 +08:00
a9f9366736 [fix](nereids) the data type of compareExpr and listQuery should be the same when creating InSubquery (#18539)
Consider sql

select table_B_alias.b from table_B_alias where table_B_alias.b in ( select a from table_A_alias );

if table_B_alias.b is int and table_A_alias.a is bigint,
we should cast(b as bigint) to make the data type the same as the InSubquery.
2023-04-12 20:02:37 +08:00
3cf4f49444 [FixBug](jdbc Catalog) fix sqlserver column type mapping (#18518)
For type int identity of sqlserver, the column type read from JDBC is called int indentity. So we need deal with this case.
2023-04-12 19:58:30 +08:00
edbe3e40b3 [fix](nereids) remove unused visitDateTimeV2Literal method (#18568)
BE supports date v2 literal and datetime v2 literal now, so remove visitDateTimeV2Literal method
2023-04-12 19:52:22 +08:00
09a4e9fd6b [enhancement](Nereids) Simplify the codes for runtime filter validation (#18571)
Since the goal of `ColumnStatistic#coverage` function is to determine whether the build side range is complete enclosed by the range of probe side, if so, as the comment of `RuntimeFilterPruner` explained, corresponding runtime filter might be thought as useless and get pruned.

Howerver, the original logic of this method is quite confused.

Simplify its logic by this formula:

```java
!(this.maxValue >= other.maxValue && this.maxValue <= other.maxValue)
```
2023-04-12 17:55:29 +08:00
db44970685 [feature](stats) Support sync analyze (#18567)
Gammer:

```
ANALYZE [SYNC] TABLE ....
```

Add this feature so that we could test and tune stats framework conveniently.
2023-04-12 17:49:30 +08:00
b93e04ab66 [test](Nereids) add regression test to check join order for tpch queries (#18543)
by explain shape plan command, with stats injection, we add regression test to check tpch queries' plan shape.
2023-04-12 15:43:21 +08:00
5dbc7e1c0e [fix](fe) add fe isReady check before getMasterIp (#18417)
when fe node is not ready, will get "" for master ip, and redirect will get error

---------

Signed-off-by: nextdreamblue <zxw520blue1@163.com>
2023-04-12 14:33:31 +08:00
34c946bb99 [Bug](date) fix regression test test_date_function (#18564) 2023-04-12 14:16:30 +08:00
75fd4b70fa [improve](fe)Optimize fe binary package packaging (#18554) 2023-04-12 12:58:45 +08:00
39a7a4cc55 [feat](Nereids): a new CBO rule: Eager Split/GroupByCount (#18556) 2023-04-12 12:13:06 +08:00
ecb22ad35e [chore](proto) modify the order of store_row_column and is_dynamic_schema to be compatible with branch-1.2-lts (#18232) 2023-04-12 11:59:56 +08:00
155e4e547b [pipeline](profile) Show each instance profile in FE (#18544) 2023-04-12 11:25:46 +08:00
43392918cd [Optimization](functions)Optimize function call for const columns. (#18310) 2023-04-12 11:11:01 +08:00
49a9956986 [Enhencement](Profile) add profile info for jdbc scanner #18569 2023-04-12 10:47:21 +08:00
2209b714d1 [chore](orc) Update orc lib to third party lib(1.8.3) using git submodule. (#18531) 2023-04-12 10:37:50 +08:00
17e1b6d1b2 [chore](third-party) Porting to GCC-12 (#18560)
The definitions of _mm_cvtsi128_si16 conflicts. This PR resolves this issue.
2023-04-12 10:33:24 +08:00
cbe2e138c3 [Enhancement](HttpServer) Support https in be (#17034)
* [Enhancement](HttpServer) Support https in be
2023-04-12 10:27:07 +08:00
bdd853b951 [typo](docs) enhance docs SHOW-TABLES.md (#18572) 2023-04-12 10:17:01 +08:00
80754edae3 [typo](docs) fix docs multi-catalog.md (#18573) 2023-04-12 10:13:18 +08:00
0290f0f770 [doc](https) Add https certificate docs (#18558)
* add https certificate docs

* add version

* add version

* add version
2023-04-12 09:40:51 +08:00
1161ddd6ea [fix](regression-test) Remove the assert check to prevent wrong results (#18538)
Since set global cannot take effect immediately in the current session.
2023-04-12 09:13:55 +08:00
c45f8c4371 [typo](doc)update hight-concurrent-point-query.md (#18570) 2023-04-12 00:21:50 +08:00
ef462fc74b modify introduction to String Type (#18562) 2023-04-11 23:40:02 +08:00
1238f6de97 [bug](array) fix be core in array_with_constant/array_repeat function when the first argument is nullable (#18404)
fix be core in array_with_constant/array_repeat function when the first argument is nullable
2023-04-11 19:46:41 +08:00
cb644d5bc3 [feature](function) support any type in SQL function (#18392)
Add AnyType to Doris.
Support Inference function in fe SQL function.
2023-04-11 19:45:02 +08:00
876b4efdf1 [fix](nereids)remove redundant session Var ENABLE_NEREIDS_RUNTIME_FILTER (#18523)
remove redundant session Var ENABLE_NEREIDS_RUNTIME_FILTER
2023-04-11 18:48:54 +08:00
5aac346ca4 [minor](refactor) delete unused codes (#18540) 2023-04-11 17:24:50 +08:00
79cd50e1ff [enhancement](statistics) update semi/anti cardinality estimation algorithm (#18524) 2023-04-11 16:51:24 +08:00
25008bbf7f [feat](Nereids): a new CBO rule: Eager Count/GroupBy. (#18511) 2023-04-11 16:37:59 +08:00
0ef958f228 [enhancement](nereids) Forbid analyze view (#18522) 2023-04-11 16:11:06 +08:00
0c5e3df4a3 [optimize](string) optimize split_by_string and substring_index function (#18496)
Use SIMD stringsearcher and SIMD memcmp optimze split_by_string and substring_index function.

split_by_string function has 32%~540% up
substring_index function has 22%~46% up
Performance difference depends on the needle size and whether the needle is constant param. And the longer the needle, the more performance improvement
2023-04-11 15:49:03 +08:00
1b70db54df [ehancement](stats) Stats preheating as FE booted (#18502)
1. Support prefetch some column stats when FE booted, it would load column stats that was got updated recently according to the comment of PR #18460 from @morrySnow  
2. Refactor stats cache, split histogram cache from column stats, so that we could avoid some redundant query for column statistics table,for example, update the histogram or column stats only, in the previous implementation a united cache loader would send query request to both column stats table and histogram table, 
3. Extract some common logic to StatsUtil
4. Remove some useless codes in unit tests, those codes is hard to maintaince and it's not a good idea for testing the accurracy of stats estimation according to the advise from @englefly 
5. Add field type restriction when create analysis tasks to avoid unnecessary failure
2023-04-11 15:19:59 +08:00
463c287361 [chore](orc) Ignore apache-orc checks. (#18537) 2023-04-11 14:08:09 +08:00
bda1b42a4f [fix](streamload) timeout in query option not set properly (#18414) 2023-04-11 14:07:24 +08:00
5f981b0b1f [fix](catalog)Use hive-catalog-shade to solve thrift version compatibility issues (#18504)
`Hive 3` uses the `thrift-0.9.3` package, and `Doris` uses the `thrift-0.16.0` package.
These two packages are not compatible, so we use the `hive-sahde` package to manage hive dependencies
in a unified way. This jar package renames the `thrift` class , so the problem of conflict can be resolved.
2023-04-11 13:19:39 +08:00
a832d24206 [doc](typo) remove the LTS version explanation about 1.2.x (#18459) 2023-04-11 12:01:40 +08:00
Pxl
da2d0bba74 [Chore](case) move some mv case from feut to p0 (#18475)
move some mv case from feut to p0
2023-04-11 11:48:55 +08:00
e562017801 [feature](table-metadata) support altering the property "light_schema_change" for the tables which created before 1.2 (#17704) 2023-04-11 11:09:43 +08:00