Commit Graph

9877 Commits

Author SHA1 Message Date
d28030e1e5 [chore](third-party) Configure the search paths for pkg-config and cmake (#18624)
Currently, our third party libraries are built by autotools or cmake. Under some scenarios, we may use system-wide headers or libraries to build them which may make the build process fail.

We can configure the search paths explicitly to help autotools and cmake find the right dependencies.
2023-04-14 08:43:27 +08:00
b39846c2c7 [Fix](Catalog)Delete duplicate defined dependencies to avoid class loading exceptions (#18628)
`iceberg-hive-metastore` and `hive-storage-api` have been defined in hive-catalog-shade,
and some classes in the shade have been renamed, so we cannot declare them again.
The classes in the shade should be kept.

The `hive-metastore-api` used in `ranger` can also use the jar in the `shade`.
Since we rename the tool class used inside the `hive`, this has no effect.
2023-04-13 22:12:19 +08:00
1d3699a70c [refactor](jdbc) refactor jdbc connection num in datasource (#18563)
now maybe jdbc have problem that there are too many connections and they do not release,
so change the property of datasource: init = 1, min = 1, max = 100, and idle time is 10 minutes.
2023-04-13 22:08:08 +08:00
6c0af24e9d [Improve](simdjson reader) support UTF-8 unicode (with BOM) (#18585) 2023-04-13 21:58:44 +08:00
281ceee3cc [feature-wip](resource-group) Support resource group tvf (#18519)
related: #18098
2023-04-13 20:11:20 +08:00
33eec9096f [Enhancement](FE) use customized grpc threadpool to get better metric for grpc from FE to BE (#13983)
Previously in Doris FE, there is no specific thread pool for grpc-client-channel,
by default the underlying netty logic would use one dynamic unbounded cache threadpool.
The workload for this grpc threadpool is unseen.
Use ThreadpoolMgr to create one customized threadpool to get Prometheus-compatible metric data.
2023-04-13 20:09:26 +08:00
aa6b3cc537 [fix](planner)keep all agg functions if there is any virtual slots in group by list (#18630)
Because of the limitation of ProjectPlanner, we have to keep set agg functions materialized if there is any virtual slots in the group by list, such as 'GROUPING_ID' in the group by list etc.
2023-04-13 19:44:46 +08:00
2519931a04 [vectorized](function) support time_to_sec function (#18354)
support time_to_sec function
2023-04-13 19:31:12 +08:00
05badac053 [Improve](docs)new libraries check (#18634) 2023-04-13 17:57:38 +08:00
40a352959d [Pipeline](exec) Support shared scan in colo agg (#18457) 2023-04-13 17:25:41 +08:00
99558153f5 [minor](Nereids): rename func and add TODO. (#18633) 2023-04-13 17:17:43 +08:00
b72c71dec0 [fix](stats) Analysis jobs didn't get persisted properly (#18602)
In previous implementation, Doris would only persist one task to tract analysis job status. After this PR, each task of column analysis would be persisted.And store a record which task_id is -1 as the job of the user submitted AnalyzeStmt.

AnalyzeStmt <---1-1---> AnalysisJob
AnalysisJob <---1-n---> AnalysisTask
2023-04-13 16:36:06 +08:00
2f64a8b387 [feature](GEO)Support read/write WKB/EWKB to gis types (#18526)
Support mutual conversion from wkb and gis types.also compatible with EWKB format
https://cwiki.apache.org/confluence/display/DORIS/DSIP-033%3A+More+GEO+functions
2023-04-13 16:25:18 +08:00
c4e9808382 [feature](multi-catalog) support trino jdbc catalog and jdbc external table (#18497) 2023-04-13 16:00:09 +08:00
2ae0bb7f13 [minor](test) remove unused function to improve test coverage (#18598) 2023-04-13 15:30:53 +08:00
Pxl
eb46bcb304 [Bug](materialized-view) fix match wrong index on some scan node (#18561)
fix match wrong index on some scan node
2023-04-13 11:50:14 +08:00
726402b53b [bugfix](topn) fix topn runtime predicate crash in short circuit evaluate for types like string decimal (#18409) 2023-04-13 11:10:59 +08:00
df0aaece1d [Function](test) add some test cases for agg functions (#18610) 2023-04-13 10:23:41 +08:00
4335c9998f [chore](ARM) Add some vectorization compatibility code on aarch64 (#18553)
update sse2noen to support more sse code on arm cpus
2023-04-13 10:15:33 +08:00
6d91635c5b [fix](json_reader) Do not increase the value of read_rows for empty line (#18611)
If read an empty row the row num++, the row num will be larger than actual column size, it will core.
2023-04-13 10:08:11 +08:00
3c3364ba27 [chore](row store) ignore serialize block to row column if no row store column (#18601) 2023-04-13 10:02:33 +08:00
d57371da13 [feature](struct-type) support basic struct constructor function (#18190)
This commit will support struct and named_struct function.
2023-04-13 09:18:00 +08:00
af0cf0c050 [Fix](multi catalog)Refresh table object while refresh external table. (#18592)
Refresh table object while refresh external table. Including:
Refresh catalog, refresh database and refresh table.
Before visiting database, need to guarantee catalog has been initialized.
Before visiting table, need to guarantee catalog and database have been initialized.
2023-04-13 08:49:44 +08:00
c5a55574c6 [fix](HttpServer) Disable https by default (#18608) 2023-04-12 23:35:44 +08:00
f9c6d7791c [typo](docs)Optimize SeaTunnel documentation (#18584) 2023-04-12 21:33:31 +08:00
1f9372558d [improve](regression case) Add more inverted index regression case (#18589)
1. add more inverted index regression case for unique mow
2. add inverted index case with different data types
2023-04-12 20:40:55 +08:00
a9f9366736 [fix](nereids) the data type of compareExpr and listQuery should be the same when creating InSubquery (#18539)
Consider sql

select table_B_alias.b from table_B_alias where table_B_alias.b in ( select a from table_A_alias );

if table_B_alias.b is int and table_A_alias.a is bigint,
we should cast(b as bigint) to make the data type the same as the InSubquery.
2023-04-12 20:02:37 +08:00
3cf4f49444 [FixBug](jdbc Catalog) fix sqlserver column type mapping (#18518)
For type int identity of sqlserver, the column type read from JDBC is called int indentity. So we need deal with this case.
2023-04-12 19:58:30 +08:00
edbe3e40b3 [fix](nereids) remove unused visitDateTimeV2Literal method (#18568)
BE supports date v2 literal and datetime v2 literal now, so remove visitDateTimeV2Literal method
2023-04-12 19:52:22 +08:00
09a4e9fd6b [enhancement](Nereids) Simplify the codes for runtime filter validation (#18571)
Since the goal of `ColumnStatistic#coverage` function is to determine whether the build side range is complete enclosed by the range of probe side, if so, as the comment of `RuntimeFilterPruner` explained, corresponding runtime filter might be thought as useless and get pruned.

Howerver, the original logic of this method is quite confused.

Simplify its logic by this formula:

```java
!(this.maxValue >= other.maxValue && this.maxValue <= other.maxValue)
```
2023-04-12 17:55:29 +08:00
db44970685 [feature](stats) Support sync analyze (#18567)
Gammer:

```
ANALYZE [SYNC] TABLE ....
```

Add this feature so that we could test and tune stats framework conveniently.
2023-04-12 17:49:30 +08:00
b93e04ab66 [test](Nereids) add regression test to check join order for tpch queries (#18543)
by explain shape plan command, with stats injection, we add regression test to check tpch queries' plan shape.
2023-04-12 15:43:21 +08:00
5dbc7e1c0e [fix](fe) add fe isReady check before getMasterIp (#18417)
when fe node is not ready, will get "" for master ip, and redirect will get error

---------

Signed-off-by: nextdreamblue <zxw520blue1@163.com>
2023-04-12 14:33:31 +08:00
34c946bb99 [Bug](date) fix regression test test_date_function (#18564) 2023-04-12 14:16:30 +08:00
75fd4b70fa [improve](fe)Optimize fe binary package packaging (#18554) 2023-04-12 12:58:45 +08:00
39a7a4cc55 [feat](Nereids): a new CBO rule: Eager Split/GroupByCount (#18556) 2023-04-12 12:13:06 +08:00
ecb22ad35e [chore](proto) modify the order of store_row_column and is_dynamic_schema to be compatible with branch-1.2-lts (#18232) 2023-04-12 11:59:56 +08:00
155e4e547b [pipeline](profile) Show each instance profile in FE (#18544) 2023-04-12 11:25:46 +08:00
43392918cd [Optimization](functions)Optimize function call for const columns. (#18310) 2023-04-12 11:11:01 +08:00
49a9956986 [Enhencement](Profile) add profile info for jdbc scanner #18569 2023-04-12 10:47:21 +08:00
2209b714d1 [chore](orc) Update orc lib to third party lib(1.8.3) using git submodule. (#18531) 2023-04-12 10:37:50 +08:00
17e1b6d1b2 [chore](third-party) Porting to GCC-12 (#18560)
The definitions of _mm_cvtsi128_si16 conflicts. This PR resolves this issue.
2023-04-12 10:33:24 +08:00
cbe2e138c3 [Enhancement](HttpServer) Support https in be (#17034)
* [Enhancement](HttpServer) Support https in be
2023-04-12 10:27:07 +08:00
bdd853b951 [typo](docs) enhance docs SHOW-TABLES.md (#18572) 2023-04-12 10:17:01 +08:00
80754edae3 [typo](docs) fix docs multi-catalog.md (#18573) 2023-04-12 10:13:18 +08:00
0290f0f770 [doc](https) Add https certificate docs (#18558)
* add https certificate docs

* add version

* add version

* add version
2023-04-12 09:40:51 +08:00
1161ddd6ea [fix](regression-test) Remove the assert check to prevent wrong results (#18538)
Since set global cannot take effect immediately in the current session.
2023-04-12 09:13:55 +08:00
c45f8c4371 [typo](doc)update hight-concurrent-point-query.md (#18570) 2023-04-12 00:21:50 +08:00
ef462fc74b modify introduction to String Type (#18562) 2023-04-11 23:40:02 +08:00
1238f6de97 [bug](array) fix be core in array_with_constant/array_repeat function when the first argument is nullable (#18404)
fix be core in array_with_constant/array_repeat function when the first argument is nullable
2023-04-11 19:46:41 +08:00