the cold-hot separation feature is still
under development. And seems there are some unsolved feature remains.
So I add a fe config enable_storage_policy, and default is false, to disable the creation and usage of storage policy by default.
So that user can aware that he is using an experimental feature on his own, and it will not be released formally in v1.2.0.
Disable storage policy by default, user can not use or create storage policy. Configured by enable_storage_policy.
Remove property remote_storage_policy, it is duplicate with storage_policy
Change the persist field in DataProperty.java.
And remove remoteCooldownTime from DataProperty, because it can be got from StoragePolicy.
Add tpch 1g orc test case in hive docker
Refactor some suites dir of catalog test cases.
And "-internal" for dlf endpoint, to support access oss with aliyun vpc.
Adjust the field naming rules when creating tables.
The original table field rules are letters or underscores or @ characters as the first letter,
followed by a maximum of 63 characters, and the total cannot exceed 64 characters.
However, in many industries, such as the financial industry, the length of the derived fields often exceeds 64
characters, so adjust the regular The rules are from 64 characters to 128 characters.
Many users load data from Hive to Doris through appearance or BrokerLoad.
Arabic numerals can be used as the first letter in the Hive table, so the regular rules are adjusted
to support Arabic numerals as the first letter.
When required cache hasn't been loaded yet, cache would always return ColumnStatistics.DEFAULT which not define the max/min literal expr, add judge for that.
Losing segmentid info will mess up the _segment_id_to_value_in_dict_flags map
in InListPredicate, causing two distinct segments to collide and crash the BE
at last.
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>
Some of default properties of ES catalog is not persisted in EditLog. So when FE is restarted,
the default properties is lost, such as `elasticsearch.doc_value_scan`, `elasticsearch.keyword_sniff` and so on.
json reader DCHECK fail because of missing TYPE_STRING
fix bug that if no file is found, the tvf will throw NPE.
The predicate conjuncts can not be pushed down to parquet reader if this is a load task.
Because the predicate should be applied on column of dest table, not on column of source file.
Add a temp property "use_new_load_scan_node" of broker load to make regression test happy.
So that we can use new load scan node for a certain job and avoid setting global FE config.
Fix bug that the ArithmeticExpr's write method is not implement, causing FE crash when creating function like:
CREATE ALIAS FUNCTION IF NOT EXISTS mesh_udf_test1(INT,INT) WITH PARAMETER(n,d) AS ROUND(1+floor(n/d));
Add if exists and if not exists for drop and create function
Fix a minor bug that if file does not exist, hdfs() table valued function will throw NPE
remove group by constants, like:
before apply rule:
select 1, k1, min(k2), max(k3) from t1 group by 1, 2;
after apply rule:
select 1, k1, min(k2), max(k3) from t1 group by k1;
MySQL [db]> SELECT SUM(a.r[1]) as active_user_num, SUM(a.r[2]) as active_user_num_1day, SUM(a.r[3]) as active_user_num_3day, SUM(a.r[4]) as active_user_num_7day FROM ( SELECT user_id, retention( day = '2022-11-01', day = '2022-11-02', day = '2022-11-04', day = '2022-11-07') as r FROM login_event WHERE (day >= '2022-11-01') AND (day <= '2022-11-21') GROUP BY user_id ) a;
ERROR 1105 (HY000): errCode = 2, detailMessage = sum requires a numeric parameter: sum(%element_extract%(a.r, 1))
If there are too much deleted tablets in RocksDB, there are many OLAP_ERR_TABLE_ALREADY_DELETED_ERROR during startup and will try to get error stack. It will cost a lot of time and the start process taks very very long.
Co-authored-by: yiguolei <yiguolei@gmail.com>
1. change jsonb_extract_string behavior: convert to string instead of NULL if the type of json path is not string
2. move jsonb tutorial doc to JSONB data type
