320eda78e6
[fix](nereids) remove useless cast in in-predicate ( #23171 )
...
consider sql "select * from test_simplify_in_predicate_t where a in ('1992-01-31', '1992-02-01', '1992-02-02', '1992-02-03', '1992-02-04');"
before:
```
| 0:VOlapScanNode |
| TABLE: default_cluster:bugfix.test_simplify_in_predicate_t(test_simplify_in_predicate_t), PREAGGREGATION: OFF. Reason: No aggregate on scan. |
| PREDICATES: CAST(a[#0 ] AS DATETIMEV2(0)) IN ('1992-01-31 00:00:00', '1992-02-01 00:00:00', '1992-02-02 00:00:00', '1992-02-03 00:00:00', '1992-02-04 00:00:00') AND __DORIS_DELETE_SIGN__[#1 ] = 0 |
| partitions=0/1, tablets=0/0, tabletList= |
| cardinality=1, avgRowSize=0.0, numNodes=1 |
| pushAggOp=NONE |
| projections: a[#0 ] |
| project output tuple id: 1 |
| tuple ids: 0
```
after:
```
| 0:VOlapScanNode |
| TABLE: default_cluster:bugfix.test_simplify_in_predicate_t(test_simplify_in_predicate_t), PREAGGREGATION: OFF. Reason: No aggregate on scan. |
| PREDICATES: a[#0 ] IN ('1992-01-31', '1992-02-01', '1992-02-02', '1992-02-03', '1992-02-04') AND __DORIS_DELETE_SIGN__[#1 ] = 0 |
| partitions=0/1, tablets=0/0, tabletList= |
| cardinality=1, avgRowSize=0.0, numNodes=1 |
| pushAggOp=NONE |
| projections: a[#0 ] |
| project output tuple id: 1 |
| tuple ids: 0
```
2023-08-24 18:14:43 +08:00
6c5072ffc5
[FIX](array-func) fix array index func with decimal ( #23399 )
...
fix array index func with decimal
in old analyzer when sql with array_position or array_contains with decimal , may loss precision to which will make result wrong
2023-08-24 17:58:20 +08:00
044423f902
[fix](Nereids) use Stopwatch to do timeout checker ( #23383 )
...
1. avoid thread leak if exception thrown in planning
2. avoid memory release delays since the timer task hold CascadesContext
object
2023-08-24 09:49:35 +08:00
f0bc2c2eff
[fix](tablet clone) fix partition rebalance apply move exception ( #23222 )
2023-08-23 21:50:50 +08:00
156f7b7699
[improvement](transaction) make commit txn fail hint more understandable ( #23227 )
2023-08-23 21:50:24 +08:00
448b7755c6
[feature](jdbc catalog) support doris jdbc catalog array type ( #23056 )
2023-08-23 21:17:16 +08:00
daa4db097e
[fix](Nereids) array_difference and array_position get wrong result ( #23331 )
...
1. change array_difference signature to let it return same type as arg
2. do not change precision when signature not use wildcard type
2023-08-23 20:38:09 +08:00
dad1587799
[opt](stats) don't load stats when task is killed ( #23354 )
...
Sync cache after analyze task is killed is meaningless and make error message confused
2023-08-23 20:24:08 +08:00
1b7d692d72
[fix](planner & nereids) convert to double if div decimal overflow ( #23272 )
2023-08-23 20:10:53 +08:00
f0c16ab20d
[fix](Nereids): visitLogicalWindow don't override ( #23376 )
...
- visitLogicalWindow don't override
- head index is wrong
2023-08-23 18:50:15 +08:00
711913ee07
[refactor](Nereids): remove penalty in CostModel ( #23375 )
2023-08-23 18:44:42 +08:00
51ac92f65c
Revert "[fix](function) to_bitmap parameter parsing failure returns null instead of bitmap_empty ( #21236 )" ( #23368 )
...
This reverts commit 1c3cc77a54938ed948ad8186b8dea8385977d23c.
2023-08-23 18:27:35 +08:00
4c751202e3
[fix](Es Catalog) Pushdown predicate of Es catalog query in nereids ( #23345 )
2023-08-23 18:03:39 +08:00
8140fc737e
[Fix](inverted index) fix bug when match condition in hash join ( #23105 )
...
* [Fix](inverted index) fix bug when match condition in hash join
2023-08-23 17:48:31 +08:00
c3327b51b9
[Fix](Nereids) add nereids load function in read fields of GlobalFunctionMgr and Database ( #23248 )
...
add nereids load function in read fields of GlobalFunctionMgr and Database
to fix some udf is lost when restart FE and query with Nereids.
2023-08-23 15:59:50 +08:00
c7b9eb5f9c
[enhancement](bitmap)support bitmap type for non-key column in unique table ( #23228 )
2023-08-23 14:21:22 +08:00
527293aa41
[refactor](dynamic table) remove dynamic table ( #23298 )
2023-08-23 14:15:14 +08:00
d8a31405c4
[refactor](Nereids): remove Penalty ( #23321 )
2023-08-23 13:31:01 +08:00
ab9b29f11f
[fix](multi-catalog)fix broker and tvf params ( #23252 )
...
fix fs specifics, broker and tvf params, and NPE
2023-08-23 10:33:24 +08:00
35d0c9e71e
[refactor](nereids) Refactor stats collection framework ( #22963 )
...
* remove auto analyze grammer
* refactor ResultRow
2023-08-23 10:05:57 +08:00
78c6b115c3
[fix](planner)avg function need support large int param ( #23254 )
...
* [fix](planner)avg function need support large int param
2023-08-23 10:05:08 +08:00
2ad46c5826
[fix](show) show load warning support load v2 ( #22759 )
2023-08-22 20:08:19 +08:00
a4e041ea55
[improve](alter-job) Add a config for forbiding doing alter job ( #23294 )
2023-08-22 16:28:36 +08:00
820d328ad7
[feature](Nereids) support handle query in fe ( #23263 )
...
1. fix select variable with alias parsed as two level alias expression.
2. support handle query in fe
2023-08-22 15:28:29 +08:00
a7675243d9
[fix](jdbc catalog) fix adaptation to Oracle special character / table names ( #23080 )
...
The changes of this PR for JdbcOracleClient are as follows:
#### bug fixes:
1. Fix the problem that if there is an approximate table name for Schema synchronization with a table name with `/` characters, the synchronization Column will be confused
2. Fix the NPE problem of metadata synchronization after enabling lower_case_table_names configuration
#### improvement:
1. Modify the method of synchronizing Oracle User to Doris Database mapping, use `metadata.getSchemas` instead of `SELECT DISTINCT OWNER FROM all_tables`
2. When synchronizing metadata, change `null` at the catalog level to `conn.getcatalog`
2023-08-22 15:25:42 +08:00
6c8af92175
[fix])(nereids)Support select catalog.db.table.column from xxx for nereids planner. #23221
...
Nereids doesn't support select table.* from table, this pr is to fix this bug.
Support three layer qualifier. (catalog.database.table)
2023-08-22 13:58:25 +08:00
51db11ed0b
[improve](jdbc catalog) Add a variable to accommodate the final keyword in ClickHouse Jdbc Catalog queries ( #23282 )
2023-08-22 12:13:36 +08:00
5d9678700c
[feature](Nereids) support select tablets with nereids optimizer ( #23164 )
2023-08-22 10:14:27 +08:00
8f48acaab1
[refactor](nereids) convert session var name "beNumForTest" #23255
...
this var is used for test only. So keep "for_test" as suffix.
2023-08-22 10:12:07 +08:00
b670dd0db7
[feature](Nereids) support array type ( #22851 )
...
FEATURE:
1. enable array type in Nereids
2. support generice on function signature
3. support array and map type in type coercion and type check
4. add element_at and element_slice syntax in Nereids parser
REFACTOR:
1. remove AbstractDataType
BUG FIX:
1. remove FROM from nonReserved keyword list
TODO:
1. support lambda expression
2. use Nereids' way do function type coercion
3. use castIfnotSame when do implict cast on BoundFunction
4. let AnyDataType type coercion do same thing as function type coercion
5. add below array function
- array_apply
- array_concat
- array_filter
- array_sortby
- array_exists
- array_first_index
- array_last_index
- array_count
- array_shuffle shuffle
- array_pushfront
- array_pushback
- array_repeat
- array_zip
- reverse
- concat_ws
- split_by_string
- explode
- bitmap_from_array
- bitmap_to_array
- multi_search_all_positions
- multi_match_any
- tokenize
2023-08-22 09:47:55 +08:00
4e63aa052c
[fix](session-var) fix forward parallel fragment instance num, and forward some session by the way ( #23234 )
2023-08-22 09:00:29 +08:00
ae48de70d6
[improve](jdbc catalog) Improve the speed of writing to postgresql databases ( #23279 )
...
reWriteBatchedInserts (boolean) Default false
This will change batch inserts from insert into foo (col1, col2, col3) values (1, 2, 3) into insert into foo (col1, col2, col3) values (1, 2, 3), (4, 5, 6) this provides 2-3x performance improvement
from https://jdbc.postgresql.org/documentation/use/#reWriteBatchedInserts
delete useless parameters useCursorFetch
2023-08-21 23:46:27 +08:00
ea5fd53f22
[enhancement](Nereids): optimize deepEquals ( #22944 )
2023-08-21 23:03:06 +08:00
4c8f269f19
[fix](Nereids): fix LogicalProject equals() ( #23241 )
...
* [fix](Nereids): fix LogicalProject equals()
* [refactor](Nereids): SubqueryExpr is LeafExpression
* remove toArray()
2023-08-21 23:01:22 +08:00
0090b8f1f6
correct a note error ( #23243 )
2023-08-21 21:33:45 +08:00
ae9f04f969
[fix](array) fix typeExtactMatch for array() type ( #23264 )
...
if we write sql with : `select cast(array() as array<varchar(10)>)`
castexpr in fe will call analyze() with `Type.matchExactType(childType, type, true);`
here array type only check contains_null , but should check inner type to make array matchExactType right
2023-08-21 19:41:09 +08:00
c1fb137d17
[improve](jdbc catalog) Add more detailed exception logging to the jdbc catalog ( #22930 )
2023-08-21 17:18:24 +08:00
dcd6c3c022
[pipelineX](refactor) propose a new pipeline execution model ( #22562 )
2023-08-21 15:38:45 +08:00
8411705e36
[fix](nereids)scalar subquery shouldn't be used in mark join ( #22907 )
...
* [fix](nereids)scalar subquery shouldn't be used in mark join
2023-08-21 15:38:22 +08:00
aaf9a51440
[fix](nereids) fix insert timeout loses effect with nereids ( #23217 )
2023-08-21 14:59:27 +08:00
169367495f
[Fix](executor)Fix enable workload group error message #23230
2023-08-21 14:28:25 +08:00
db5a622166
[Fix](planner) fix incorrect nullable in ctas. ( #22770 )
...
ctas with outer join like
```sql
create table a (
id int not null,
name varchar(20) not null
)
distributed by hash(id) buckets 4
properties (
"replication_num"="1"
);
create table b (
id int not null,
age int not null
)
distributed by hash(id) buckets 4
properties (
"replication_num"="1"
);
insert into a values(1, 'ww'), (2, 'zs');
insert into b values(1, 22);
create table c properties("replication_num"="1") as select a.id, a.name, b.age from a left join b on a.id = b.id;
```
the column 'age' in c is not null, but nullable is expected, we fix it by use the nullable mode of the outputs of root plan fragment.
2023-08-21 11:46:27 +08:00
b9127fa847
[Fix](View)Use the element_at function to replace %element_extract% to prevent parsing errors ( #23093 )
2023-08-21 11:19:10 +08:00
0967d7ec04
[improvement](agg) Do not serialize bitmap to string ( #23172 )
2023-08-21 10:10:15 +08:00
a11e0e3bc4
[Bug](agg) fix QUANTILE_UNION many problems ( #23181 )
...
fix QUANTILE_UNION many problems
2023-08-21 10:04:27 +08:00
3d980ae248
[Fix](multi-catalog) Fix NPE when file cache is enabled. ( #23165 )
2023-08-20 23:35:28 +08:00
6ffc26858a
[Improvement](meta) add default_value column & is changed column for result of show_variables stmt ( #23017 )
...
* [Improvement](meta) add default_value column for result of show_variables stmt
* add Changed column to show whether value is modified
* fix code style issue
2023-08-20 20:48:45 +08:00
97fa840324
[feature](multi-catalog)support iceberg hadoop catalog external table query ( #22949 )
...
support iceberg hadoop catalog external table query
2023-08-20 19:29:25 +08:00
5ba505ebf4
[fix](multi-catalog)fix avro and jdbc scanner dependency ( #23015 )
...
add preload-extensions module, put all conflict dependencies to pom.xml in preload-extensions
2023-08-20 19:28:17 +08:00
6bf65253d0
[fix](Nereids): unstable test when run single UT. ( #23189 )
2023-08-18 23:14:56 +08:00
