* Refactor upgrade documentation
* Optimize Dockerfile content for FE and BE.
* Optimize Dockerfile content for FE and BE.
---------
Co-authored-by: Yijia Su <suyijia@selectdb.com>
The `PARTITION BY` syntax used by external catalogs has been added.
You can specify a column directly, or a partition function as a partition condition.
Like:
`PARTITION BY LIST(col1, col2, func(param), func(param1, param2), func(param1, param2, param3))`
NOTICE:
This PR change the grammar of `AUTO PARTITION`
From
```
AUTO PARTITION BY RANGE date_trunc(`TIME_STAMP`, 'month')
```
To
```
AUTO PARTITION BY RANGE (date_trunc(`TIME_STAMP`, 'month'))
```
* Problem:
Inconsistent behavior occurs when executing partial column update `UPDATE` statements and `INSERT` statements on merge-on-write tables with the Nereids optimizer enabled. The number of columns passed to BE differs; `UPDATE` operations incorrectly pass all columns, while `INSERT` operations correctly pass only the updated columns.
Reason:
The Nereids optimizer does not handle partial column update `UPDATE` statements properly. The processing logic for `UPDATE` statements rewrites them as equivalent `INSERT` statements, which are then processed according to the logic of `INSERT` statements. For example, assuming a MoW table structure with columns k1, k2, v1, v2, the correct rewrite should be:
* `UPDATE` table t1 set v1 = v1 + 1 where k1 = 1 and k2 = 2
* =>
* `INSERT` into table (v1) select v1 + 1 from table t1 where k1 = 1 and k2 = 2
However, the actual rewriting process does not consider the logic for partial column updates, leading to all columns being included in the `INSERT` statement, i.e., the result is:
* `INSERT` into table (k1, k2, v1, v2) select k1, k2, v1 + 1, v2 from table t1 where k1 = 1 and k2 = 2
This results in `UPDATE` operations incorrectly passing all columns to BE.
Solution:
Having analyzed the cause, the solution is straightforward: when rewriting partial column update `UPDATE` statements to `INSERT` statements, only retain the updated columns and all key columns (as partial column updates must include all key columns). Additionally, this PR includes error injection cases to verify the number of columns passed to BE is correct.
* 2
* 3
* 4
* 5
This pr imporve the high QPS query by speed up PartitionPrunner
1. remove useless Date parse/format, use LocalDate instead
2. fast evaluate path for single value partition
3. change Collection.stream() to ImmutableXxx.builderWithExpectedSize(n) to skip useless method call and collection resize
4. change lots of if-else to switch
5. don't parse to string to compare dateLiteral, use int field compare instead
