5276cc4db6
[docker][fix] update routine load cases ( #31553 )
...
Co-authored-by: 胥剑旭 <xujianxu@xujianxudeMacBook-Pro.local >
2024-02-29 16:44:39 +08:00
3c37fb085c
[refactor](jdbc catalog) split jdbc executor for different data sources (step-1) ( #31406 )
2024-02-29 12:38:03 +08:00
e3b4b83bca
[test](regression) add regression test for schange change of complex type ( #31207 )
...
Add regression test for #31128
2024-02-22 19:50:07 +08:00
b3ac2128dd
[Refactor](catalog) Refactor Jdbc Catalog external name case mapping rules ( #28414 )
2024-02-19 17:22:03 +08:00
8db2824c44
[bugfix](es catalog) add constant_keyword wildcard data type ( #30947 )
2024-02-19 17:20:21 +08:00
92cad69fc4
[Fix](parquet-reader) Fix reading fixed length byte array decimal in parquet reader. ( #30535 )
2024-01-31 23:53:40 +08:00
658c869aac
[improvement](mtmv)mtmv support partition by hms table ( #29989 )
2024-01-29 19:02:46 +08:00
9b100207d2
[docker](container)create data path after rm residual data avoiding docker start fail ( #30141 )
2024-01-23 10:09:54 +08:00
fac0580eae
[opt](docker)optimize ES docker compose ( #30068 )
...
1. add volume for es logs
2. optimize health check, waiting for es status to be green
3. fix es6 valume path error
4. optimize disk watermark to avoid es disk watermark error
5. fix es6 create index error
6. add custom elasticsearch.yml for es6
7. add log4j2.properties for es6, es7, es8
2024-01-19 15:48:56 +08:00
44ba9e102c
[feature](statistics)support statistics for iceberg/paimon/hudi table ( #29868 )
2024-01-18 12:03:07 +08:00
5d0732fcf4
[typo](docker thirdparties) Modify script prompt information error ( #29770 )
2024-01-12 11:53:57 +08:00
67986a8a6f
[fix](case) external case, specify networks ( #29699 )
...
Co-authored-by: stephen <hello-stephen@qq.com >
2024-01-12 11:46:29 +08:00
3cd1c7745a
[fix](jdbc catalog) Fix the precision of decimal type mapping to 0 ( #29407 )
2024-01-12 11:39:57 +08:00
2c4e52e44e
[fix](es catalog) only es_query function can push down to ES ( #29320 )
...
Issue Number: close #29318
1. Only push down `es_query` function to ES
2. Add null check where ES query result not have `_source` or `fields` fields.
2023-12-30 09:33:26 +08:00
7da86c37ec
[fix](hive) add support for quoteChar and seperatorChar for hive ( #28613 )
...
add support for quoteChar and seperatorChar .
2023-12-19 19:35:03 +08:00
608baae001
[docker][regregression]update routine load cases #28450
...
Co-authored-by: 胥剑旭 <xujianxu@xujianxudeMacBook-Pro.local >
2023-12-16 17:57:39 +08:00
01c94a554d
[fix](autoinc) Fix broker load when target table has autoinc column ( #28402 )
2023-12-14 18:02:54 +08:00
a271fee3c5
[test](statistics)Add external empty table test case. ( #28267 )
2023-12-13 21:48:01 +08:00
60bc3be8a2
[Opt](Compression) Opt zstd block decompression by ZSTD_decompressDCtx(). ( #27534 )
...
Opt zstd block decompression by `ZSTD_decompressDCtx()` to replace streaming decompression.
It will improve performance but consume more memory.
Test result:
- env: 1 node(16 cores, 64G).
- parquet column: 100 million rows of char(255) column.
- result: 5.2 -> 4.6.
2023-12-01 09:10:32 +08:00
e4149c6e4c
[Fix](parquet-reader) Fix null map issue in parquet reader. ( #27777 )
...
Fix null map issue in parquet reader which cause result incorrect such as `min()`, `max()`.
In order to share null map between parquet converted src column and dst column to avoid copying. It is very tricky that will call mutable function `doris_nullable_column->get_null_map_column_ptr()` which will set `_need_update_has_null = true`. Because some operations such as agg will call `has_null()` to set `_need_update_has_null = false`.
2023-11-30 13:55:37 +08:00
cc395f5428
[Fix](hive-transactional-table) Fix NPE when query empty hive transactional table. ( #27563 )
2023-11-25 10:29:39 +08:00
b477839bce
[enhancement](jdbc catalog) Add lowercase column name mapping to Jdbc data source & optimize database and table mapping ( #27124 )
...
This PR adds the processing of lowercase Column names in Oracle Jdbc Catalog. In the previous behavior, we changed all Oracle columns to uppercase queries by default, but could not handle the lowercase case. This PR can solve this situation and improve All Jdbc Catalog works
2023-11-17 23:51:47 +08:00
df867a1531
[fix](catalog) Fix ClickHouse DataTime64 precision parsing ( #26977 )
2023-11-15 10:23:21 +08:00
1edeacd0a5
[enhance](regression) enhance docker network by add network subnet ( #26862 )
2023-11-14 20:06:20 +08:00
3585c7e216
[test](parquet)append parquet reader byte_array_decimal and rle_bool case ( #26751 )
2023-11-14 15:05:10 +08:00
7ce746654a
[test](jdbc) add doris and sqlserver jdbc catalog test case ( #26656 )
2023-11-10 10:32:09 +08:00
49cffd0bc9
[fix](JdbcCatalog) fix that the predicate column name does not have back quote when querying the JDBC appearance ( #26479 )
2023-11-10 09:54:39 +08:00
8434389358
[fix](jdbc) fix clickhouse catalog arr nullable and add case ( #26639 )
2023-11-09 19:32:05 +08:00
22bf2889e5
[feature](tvf)(jni-avro)jni-avro scanner add complex data types ( #26236 )
...
Support avro's enum, record, union data types
2023-11-09 13:58:49 +08:00
f6b7046a6e
[fix](regression-test) add tests for jdbc catalog ( #26608 )
2023-11-09 11:59:35 +08:00
5bcf6bfd46
[fix](jdbc catalog) fix mysql zero date ( #26569 )
2023-11-08 21:41:56 +08:00
80f654ec2a
[Fix](statistics)Fix analyze min max sql syntax error. #26240
2023-11-02 09:22:32 +08:00
78204f7c92
[Fix](statistics)Fix external couldn't analyze database bug ( #26025 )
2023-10-31 11:32:47 +08:00
501c6096dd
Revert "[Test](multi-catalog) Add tpcds sf100 hive shape. ( #25639 )" ( #26069 )
...
This reverts commit 3beba1764c01b6712b108556433c96429c59cc45.
2023-10-29 12:45:32 +08:00
3beba1764c
[Test](multi-catalog) Add tpcds sf100 hive shape. ( #25639 )
...
Add tpcds sf100 hive shapes.
Disable query64 temporarily because it is not same with emr cluster after collecting metadata by analyze table xxx.
And the root cause need to analyze, will enable in future PR.
2023-10-27 18:39:29 +08:00
c86fad7cbd
[Fix](orc-reader) Fix orc decimal128 scale issue. ( #25977 )
2023-10-26 08:50:18 -05:00
e7a3cb079b
[Enhance](regression)docker hive s3 file address is determined based on the configuration ( #25905 )
...
docker hive s3 file address is determined based on the configuration custom_settings.env
2023-10-26 11:58:33 +08:00
e783ef716f
[fix](multi-table) fix unknown source slot descriptor when load multi table ( #25762 )
2023-10-25 21:52:01 +08:00
10f1957379
[feature](docker)add docker-iceberg init tables ( #25424 )
...
Add some init tables for docker-iceberg.
2023-10-24 19:29:57 +08:00
7de3d9882c
[regresstion-test](jdbc catalog)Mariadb compatible test ( #25664 )
2023-10-23 11:51:03 +08:00
0533911275
[Enhance](regression)change external default docker list ( #25573 )
2023-10-18 02:56:10 -05:00
ce18f1148a
[improvement](catalog)compatible with paimon 0.5 ( #24985 )
...
compatible with paimon 0.5
add p0 for paimon,need set enablePaimonTest=true
2023-10-17 22:07:13 +08:00
b946521a56
[enhancement](regression-test) add single stream multi table case ( #25360 )
2023-10-14 10:59:50 +08:00
1c3ecbbae9
[docker] [fix] add kafka log collector ( #25326 )
...
add kafka log collector
2023-10-12 15:23:10 +08:00
f680a2141d
[enhancement](regression-test) add routine load json case ( #25253 )
2023-10-11 19:43:08 +08:00
b7ac95a970
[enhancement](regression-test) open routine load regression test by default and add data check ( #25122 )
2023-10-11 10:03:16 +08:00
7e9ffad933
[fix](ES catalog)Doris cannot parse ES date field without time zone ( #24864 )
...
1. Add support for Doris to parse ES date field without time zone info. eg: `2023-04-17T23:01:18.151`, this time will be treated as UTC time, since ES assumes that the time zone for time fields without time zones is UTC.
2. Change local time zone convertion from system local time zone to session variable time zone.
2023-10-08 19:28:08 +08:00
541f48a754
[feature](es-catalog) add include_hidden_indexin order to get the hidden index. ( #24826 )
2023-10-08 14:35:08 +08:00
fddef8b473
[fix](es-catalog)fix error when querying the index ,elasticsearch version 8.9.1 ( #24839 )
...
Issue Number: close #24833
2023-10-08 10:19:45 +08:00
74bba4bdaf
[enhancement](regression-test) Add routine load case ( #24536 )
2023-09-22 14:55:01 +08:00
