0d7a61ae8c
[fix](load) fix duplicate register of memtable writer in memory limiter ( #23205 )
2023-08-22 10:05:17 +08:00
f842a5cf37
[fix](case) comment out orthogonal_bitmap_* ( #23259 )
...
comment out orthogonal_bitmap_*
2023-08-22 10:02:53 +08:00
b670dd0db7
[feature](Nereids) support array type ( #22851 )
...
FEATURE:
1. enable array type in Nereids
2. support generice on function signature
3. support array and map type in type coercion and type check
4. add element_at and element_slice syntax in Nereids parser
REFACTOR:
1. remove AbstractDataType
BUG FIX:
1. remove FROM from nonReserved keyword list
TODO:
1. support lambda expression
2. use Nereids' way do function type coercion
3. use castIfnotSame when do implict cast on BoundFunction
4. let AnyDataType type coercion do same thing as function type coercion
5. add below array function
- array_apply
- array_concat
- array_filter
- array_sortby
- array_exists
- array_first_index
- array_last_index
- array_count
- array_shuffle shuffle
- array_pushfront
- array_pushback
- array_repeat
- array_zip
- reverse
- concat_ws
- split_by_string
- explode
- bitmap_from_array
- bitmap_to_array
- multi_search_all_positions
- multi_match_any
- tokenize
2023-08-22 09:47:55 +08:00
4e63aa052c
[fix](session-var) fix forward parallel fragment instance num, and forward some session by the way ( #23234 )
2023-08-22 09:00:29 +08:00
ae48de70d6
[improve](jdbc catalog) Improve the speed of writing to postgresql databases ( #23279 )
...
reWriteBatchedInserts (boolean) Default false
This will change batch inserts from insert into foo (col1, col2, col3) values (1, 2, 3) into insert into foo (col1, col2, col3) values (1, 2, 3), (4, 5, 6) this provides 2-3x performance improvement
from https://jdbc.postgresql.org/documentation/use/#reWriteBatchedInserts
delete useless parameters useCursorFetch
2023-08-21 23:46:27 +08:00
ea5fd53f22
[enhancement](Nereids): optimize deepEquals ( #22944 )
2023-08-21 23:03:06 +08:00
4c8f269f19
[fix](Nereids): fix LogicalProject equals() ( #23241 )
...
* [fix](Nereids): fix LogicalProject equals()
* [refactor](Nereids): SubqueryExpr is LeafExpression
* remove toArray()
2023-08-21 23:01:22 +08:00
12075f9853
[pipelineX](projection) Support projection and blocking agg ( #23256 )
2023-08-21 22:23:02 +08:00
8503d26e5c
[bugfix](vertical-compaction) Only can init the SegmentCacheHandle once ( #23246 )
2023-08-21 21:57:27 +08:00
0090b8f1f6
correct a note error ( #23243 )
2023-08-21 21:33:45 +08:00
3b4cc06a62
[feature](move-memtable)[1/8] support creating segments externally in rowset writer ( #23085 )
2023-08-21 21:02:34 +08:00
d4642b47b4
[fix](InternalService) add short-cut return when offer failed ( #23239 )
...
During offer_failed(), rpc done will be executed so the response will be
sent and released. Further access to such object will cause NPE error.
So let us just return after offer_failed().
2023-08-21 21:00:49 +08:00
ae9f04f969
[fix](array) fix typeExtactMatch for array() type ( #23264 )
...
if we write sql with : `select cast(array() as array<varchar(10)>)`
castexpr in fe will call analyze() with `Type.matchExactType(childType, type, true);`
here array type only check contains_null , but should check inner type to make array matchExactType right
2023-08-21 19:41:09 +08:00
c1fb137d17
[improve](jdbc catalog) Add more detailed exception logging to the jdbc catalog ( #22930 )
2023-08-21 17:18:24 +08:00
dcd51c304a
Update test_csv_with_enclose_and_escape.groovy ( #23173 )
2023-08-21 17:08:25 +08:00
32dc0250ba
[Chore](ColdHeatSeparation) change partition value for cold heat separation case ( #22996 )
2023-08-21 16:14:08 +08:00
dcd6c3c022
[pipelineX](refactor) propose a new pipeline execution model ( #22562 )
2023-08-21 15:38:45 +08:00
8411705e36
[fix](nereids)scalar subquery shouldn't be used in mark join ( #22907 )
...
* [fix](nereids)scalar subquery shouldn't be used in mark join
2023-08-21 15:38:22 +08:00
aaf9a51440
[fix](nereids) fix insert timeout loses effect with nereids ( #23217 )
2023-08-21 14:59:27 +08:00
169367495f
[Fix](executor)Fix enable workload group error message #23230
2023-08-21 14:28:25 +08:00
d4694167a8
[Enhancement](chore) Some Status relevant enhancement ( #23072 )
2023-08-21 14:14:38 +08:00
81dd00f6e4
[Feature](Compaction) Support do full compaction by table id ( #22010 )
2023-08-21 11:54:51 +08:00
f7c99d487a
[test](fix) forbid test_default_limit ( #23231 )
...
forbid test_default_limit
2023-08-21 11:50:57 +08:00
db5a622166
[Fix](planner) fix incorrect nullable in ctas. ( #22770 )
...
ctas with outer join like
```sql
create table a (
id int not null,
name varchar(20) not null
)
distributed by hash(id) buckets 4
properties (
"replication_num"="1"
);
create table b (
id int not null,
age int not null
)
distributed by hash(id) buckets 4
properties (
"replication_num"="1"
);
insert into a values(1, 'ww'), (2, 'zs');
insert into b values(1, 22);
create table c properties("replication_num"="1") as select a.id, a.name, b.age from a left join b on a.id = b.id;
```
the column 'age' in c is not null, but nullable is expected, we fix it by use the nullable mode of the outputs of root plan fragment.
2023-08-21 11:46:27 +08:00
80d83b3a67
[fix](case) nereids_p0/update/update_unique_table.groovy ( #23195 )
...
fix cases in order to run without load data
2023-08-21 11:27:53 +08:00
b9127fa847
[Fix](View)Use the element_at function to replace %element_extract% to prevent parsing errors ( #23093 )
2023-08-21 11:19:10 +08:00
37b49f60b7
[refactor](conf) add be conf for partition topn partitions threshold ( #23220 )
...
add be conf for partition topn partitions threshold
2023-08-21 10:52:41 +08:00
33dfa0c454
[Improve](serde) support text serde for nested type-array/map ( #22738 )
...
Now we can not support nested type array/map
so this pr aim to:
1. add format option for string convert defined datatype to keep with origin from_string
2. support array map can nested array and map
2023-08-21 10:32:28 +08:00
0967d7ec04
[improvement](agg) Do not serialize bitmap to string ( #23172 )
2023-08-21 10:10:15 +08:00
a11e0e3bc4
[Bug](agg) fix QUANTILE_UNION many problems ( #23181 )
...
fix QUANTILE_UNION many problems
2023-08-21 10:04:27 +08:00
3d980ae248
[Fix](multi-catalog) Fix NPE when file cache is enabled. ( #23165 )
2023-08-20 23:35:28 +08:00
4bf055c818
[fix](parquet) the key colum of map type in parquet may be nullable ( #23180 )
...
Fix errors when reading map type with nullable key column in parquet file. `ParquetReader` support to read nullable key column, but add a check to prevent reading nullable key column. Unfortunately, this check error was not thrown correctly, causing the BE to crash, and thrown meaningless error logs in be.out:
```
...
11# doris::vectorized::ParquetReader::get_columns(std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, doris::TypeDescriptor, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, doris::TypeDescriptor> > >*, std::unordered_set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >*) at /root/doris/be/src/vec/exec/format/parquet/vparquet_reader.cpp:508
12# doris::vectorized::VFileScanner::_get_next_reader() in /root/yun_you_external/output/be/lib/doris_be
13# doris::vectorized::VFileScanner::_get_block_impl(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /root/doris/be/src/vec/exec/scan/vfile_scanner.cpp:241
...
```
2023-08-20 22:59:18 +08:00
6ffc26858a
[Improvement](meta) add default_value column & is changed column for result of show_variables stmt ( #23017 )
...
* [Improvement](meta) add default_value column for result of show_variables stmt
* add Changed column to show whether value is modified
* fix code style issue
2023-08-20 20:48:45 +08:00
97fa840324
[feature](multi-catalog)support iceberg hadoop catalog external table query ( #22949 )
...
support iceberg hadoop catalog external table query
2023-08-20 19:29:25 +08:00
5ba505ebf4
[fix](multi-catalog)fix avro and jdbc scanner dependency ( #23015 )
...
add preload-extensions module, put all conflict dependencies to pom.xml in preload-extensions
2023-08-20 19:28:17 +08:00
58b9bce954
[fix](load) add rowset builder init error handling ( #23166 )
2023-08-19 17:13:10 +08:00
ac855e75f1
[fix](doc) fix from_unixtime error range #23176
2023-08-19 12:17:13 +08:00
433a6103ab
[Enhancement](scanner) allocate blocks in scanner_context on demand and free them on close ( #23182 )
...
Introduced #19389 , removed #20785
2023-08-19 12:13:24 +08:00
6bf65253d0
[fix](Nereids): unstable test when run single UT. ( #23189 )
2023-08-18 23:14:56 +08:00
0838ff4bf4
[fix](Outfile) fix bug that the fileSize is not correct when outfile is completed ( #22951 )
2023-08-18 22:31:44 +08:00
10abbd2b62
[Feauture](Export) support parallel export job using Job Schedule ( #22854 )
2023-08-18 22:24:42 +08:00
6847592137
[Fix](RoutineLoad)Fix when Unique (MoW) RoutineLoad imports unspecified Sequence column ( #23167 )
...
[Fix](RoutineLoad)Fix when Unique (MoW) routineload imports unspecified Sequence column
2023-08-18 21:49:09 +08:00
26905e36e5
[fix](load) fix nullptr in memtable limiter flush ( #23149 )
2023-08-18 19:55:53 +08:00
b6dd56fee0
[fix](multi-catalog)fix compability issue for s3 endpoint ( #23175 )
2023-08-18 18:37:21 +08:00
345eaab00b
[refactor](Nereids): remove useless equals()/hashcode() about Id ( #23162 )
2023-08-18 18:31:31 +08:00
7c4870c371
[fix](catalog) fix hive partition prune bug on nereids ( #23026 )
2023-08-18 18:31:01 +08:00
9cee0ecccc
[fix](show-table-status) fix priv error on show table status stmt ( #22918 )
2023-08-18 18:30:09 +08:00
419e922a69
[fix](json)Fix the bug that does not stop when reading json files ( #23062 )
...
* [fix](json)Fix the bug that does not stop when reading json files
2023-08-18 18:23:19 +08:00
477961dc21
[Chore](agg) refactor of hash map ( #22958 )
...
refactor of hash map
2023-08-18 17:59:30 +08:00
f0ad3ef244
[fix](merge-on-write) should use write lock of tablet's header lock in #23047 ( #23161 )
2023-08-18 17:50:44 +08:00
