Commit Graph

16494 Commits

Author SHA1 Message Date
097641b543 [fix](Nereids): fix AssertNumRows StatsCalculator (#30053) 2024-01-19 15:48:15 +08:00
Pxl
a5ca8833d7 [Improvement](aggregate) optimize for small string aggregate (#29919) 2024-01-19 15:48:15 +08:00
f1462f6cf4 [fix](Nereids): eliminate redundant join condition after inferring condition (#30093)
eliminate redundant join when find hashing join condition
such as for plan:
```
T1 join T2 on T1.id = T2.id join T3 on T1.id = T3.id and T2.id = T3.id 
```
we infer a new predicate T1.id = T2.id which is redundant. Therefore we need to eliminate it when find hash condition
2024-01-19 15:48:15 +08:00
7d3a3fee65 [fix](Nereids) update assignment column name should case insensitive (#30071) 2024-01-19 15:48:15 +08:00
Pxl
2ccb69dbed [Feature](materialized-view) support some case unmached to materialized-view (#30036)
same column appears in key and value like select id,count(id) group by id;
complex expr in sum select sum(if(xxx));
2024-01-18 12:03:07 +08:00
0ccd706a30 [Enhancement](Jdbc Catalog) Map Jdbc Catalog JSON Type to String for Improved Performance and Compatibility (#30035)
This PR proposes mapping external catalog JSON types to String instead of JsonB in Apache Doris. This change is motivated by the realization that JDBC retrieves JSON data as a String JSON string, regardless of its storage format (Json(String) or Json(Binary)). Mapping to String streamlines data retrieval, simplifies write-backs, and ensures compatibility with all JSON(String) and JSON(Binary) functions, despite potentially misleading displays of JSON data as Strings in Doris. This approach avoids the performance overhead and complexity of converting each row of data from JsonB to String, making the process more efficient and elegant.

About Upgrade
To ensure query compatibility with existing Catalogs in the upgraded version,we currently still retain the capability to query external JSON types as JSONB. However, once you upgrade to the new version and either refresh the Catalog or create a new one, all external JSON types will be treated as Strings. To ensure consistent behavior,and possible future removal of support for JSON as JSONB query code, it is highly recommended that you manually refresh your Catalog as soon as possible after upgrading to the new version.
2024-01-18 12:03:07 +08:00
c1722a6ab0 [Fix](schema change) fix tablet meta write race (#30081) 2024-01-18 12:03:07 +08:00
d554d5478c [improvement](fe) skip attach profile during send fragment (#30085)
Sometime attach profile may cost a lot of time in large machine with many instances.


---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-01-18 12:03:07 +08:00
4608bcd7bf [fix](ut) fixed test cases failure for showCreateFunction (#28593)
2 tests cases (ShowCreateFunctionTest and ShowFunctionTest) have function name clash;
because of which one of the test fails to complete setup() and eventually fail for not able to create funciton.

Changed the name of the test function in 1 of the file.
2024-01-18 12:03:07 +08:00
7d1b3d4704 [feature](statistics, metadata)Meta data place holder for statistics (#29867)
Meta data place holder for statistics in version 2.1.x. Users could upgrade to this version, but doesn't support rollback.
After this change, statistics related functions doesn't need to change meta data any more in the 2.1 series.
2024-01-18 12:03:07 +08:00
7e821f3d66 [Performance](Join) Support all match one logic (#30019)
Support all match one logic
2024-01-18 12:03:07 +08:00
8a0a6bf856 [Fix](typo) Fix group commit regression test typo (#30057) 2024-01-18 12:03:07 +08:00
990d5d8664 [cleanup](insert-into) clean up some insert into log (#30063) 2024-01-18 12:03:07 +08:00
65b5a5436a [minor](Nereids): remove useless method isMysqlCompatibleDatabase (#30064) 2024-01-18 12:03:07 +08:00
Pxl
30378f9bbc [Chore](config) remove some unused config (#29983)
remove some unused config
2024-01-18 12:03:07 +08:00
44ba9e102c [feature](statistics)support statistics for iceberg/paimon/hudi table (#29868) 2024-01-18 12:03:07 +08:00
ade720470d [Improve](config)delete confused config for nested complex type (#29988) 2024-01-18 12:03:07 +08:00
143df9f414 [typo](docs) fix typo (#30031)
Co-authored-by: hechao <hechao@selectdb.com>
2024-01-18 10:04:21 +08:00
e894911cda [function](char) change char function behaviour same with mysql (#30034)
select char(0) = '\0';
should return true;
2024-01-18 10:04:21 +08:00
dcc9cf9dd3 [config](Nereids): add SessionVariable to control nereids timeout (#30048) 2024-01-18 10:04:21 +08:00
48d7c1b1ed [test](regression-test) fix case, compatible with 3 replicas. (#29905) 2024-01-18 10:04:21 +08:00
7e3ed8a19a [fix](index_tool) fix index_tool compile error (#30049) 2024-01-18 10:04:21 +08:00
e73c0f4077 [fix](fe) Fix drop table write too many times editlog (#29966) 2024-01-18 10:04:21 +08:00
d4195b3cc0 [fix](Nereids): fix enable mv rewrite is useless in dphyp (#30042) 2024-01-18 10:04:21 +08:00
Pxl
b0c49024cb [Feature](materialized-view) support match function with alias in materialized-view (#30025)
support match function with alias in materialized-view
2024-01-18 10:04:21 +08:00
2ca3c82135 [fix](fe ut) fix unstable TabletRepairAndBalanceTest (#30041) 2024-01-18 10:04:21 +08:00
6ef9ed08aa [fix](multi-table-load) fix multi table load can not finish (#29957) 2024-01-18 10:03:35 +08:00
3deee14680 [fix](Nereids): find hash condition after infer predicate (#30026) 2024-01-18 10:03:01 +08:00
b758286b57 [doc](Variant) add doc for Variant type (#28951) 2024-01-18 09:00:42 +08:00
3f2a794c2e [refactor](insert) remove unused insert code in FE #29924 2024-01-18 09:00:32 +08:00
f0a4ec5f85 [pipelineX](fix) Fix runtime filter dependency DCHECK failed (#30038) 2024-01-18 08:56:47 +08:00
9714a3c873 [fix](regression) fault injection may cause fd to be closed twice (#30070) 2024-01-18 08:55:53 +08:00
d20103cad4 [fix](compatibility) remove cluster prefix in user property info (#30078) 2024-01-18 08:55:14 +08:00
2dcdf07dd4 [Feature](profile)Support active_queries TVF (#29999) 2024-01-16 21:25:02 +08:00
d658a44cef [improvement](catalog) Change the push-down parameters of the predicate function of the table query SQL into variables (#30028)
In this PR, we will control whether the external data source query is a push-down function parameter in the filter condition, changing the enable_fun_pushdown of fe conf to the enable_ext_func_pred_pushdown of the variable
2024-01-16 21:14:35 +08:00
2fa511f80e [improve](multi-table-load) avoid plan and execute too many plan at once (#29951) 2024-01-16 21:14:35 +08:00
8090a395bd change sessionvariable.java style 2024-01-16 20:40:36 +08:00
f98f790a80 [chore](test) update delete complex type case to make Nereids happy (#30022) 2024-01-16 20:23:09 +08:00
e693249f5b [fix](Nereids) remove useless substr of string like type cast (#29997) 2024-01-16 20:23:09 +08:00
f7a340a2df [improve](move-memtable) add cancel method to load stream stub (#29994) 2024-01-16 20:23:09 +08:00
d101234be1 [fix](test) fix disableDebugPointForAllBEs do not execute (#30023) 2024-01-16 20:23:09 +08:00
a0355a6580 [opt](scanner) optimize the number of threads of scanners, follow up #28640 (#30030)
follow up #28640, doris_scanner_thread_pool_thread_num may be to large in machines with 128 cores.
2024-01-16 20:23:09 +08:00
4e41e1d797 [feat](Nereids) persist constraint in table (#29767) 2024-01-16 18:49:29 +08:00
2916745cf2 [Chore](arm) Support auto-SIMD on arm and refactor compilation docs (#29956) 2024-01-16 18:49:04 +08:00
168afdb965 [fix](disk balance) Change disk rebalance unpick time to configurable (#28949) 2024-01-16 18:49:04 +08:00
74991c4af2 [bugfix](paimon)support native and jni to read paimon for minio/cos #29933 2024-01-16 18:49:01 +08:00
4bf4239d7a [feature](Nereids): optimize logical group expression in dphyp (#30000) 2024-01-16 18:48:20 +08:00
d11e797d4c [fix](decimal) fix wrong decimal overflow caused by uninitialized nested column of null value (#29960) 2024-01-16 18:48:20 +08:00
6ce9c67383 [fix](Nereids) struct type coercion call wrong api (#30018) 2024-01-16 18:48:20 +08:00
d3bf23d70d [chore](removelogs) remove debug query timeout logs (#30006)
---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-01-16 18:48:18 +08:00