In some scenarios, users cannot find a suitable hash key to avoid data skew, so we need to provide an additional data distribution for olap table to avoid data skew
example:
CREATE TABLE random_table
(
siteid INT DEFAULT '10',
citycode SMALLINT,
username VARCHAR(32) DEFAULT '',
pv BIGINT SUM DEFAULT '0'
)
AGGREGATE KEY(siteid, citycode, username)
DISTRIBUTED BY random BUCKETS 10
PROPERTIES("replication_num" = "1");
Co-authored-by: caiconghui1 <caiconghui1@jd.com>
There are many old codes in FE for old FE meta version such as `if (FeMetaVersion < VERSION_45) xxxxx`,
but the latest FE meta version is 107, these code maybe never reached,
but we do not remove these code because "sometimes" there are old code.
Add minimum required version check to allow us remove these old codes.
1. Add repeated PBlock.PColumnMeta.children field for complex data type, such as ARRAY;
2. change PBlock.PColumnMeta.name to optional, nested columns do not need to a column name;
Also fix BE ut:
1. fix scheme_change_test memory leak
2. fix mem_pool_test
Do not using DEFAULT_PADDING_SIZE = 0x10 in mem_pool when running ut.
3. remove plugin_test
* make ASAN poisoning work as much as possible
Before this patch a use after poison is reported like below
==19305==ERROR: AddressSanitizer: unknown-crash on address
0x625000137013 at pc 0x561c44bcf6b8 bp 0x7ffb75a00910 sp 0x7ffb75a000b8
After this patch the use after poison is reported like below
==17782==ERROR: AddressSanitizer: use-after-poison on address
0x625000137033 at pc 0x55633c8f56b8 bp 0x7ff3dc437930 sp 0x7ff3dc43
Before this patch, a false memory usage is reported like below
==33080==AddressSanitizer CHECK failed: ../../../../src/libsanitizer/
asan/asan_allocator.cpp:189 "((old)) == ((kAllocBegMagic))"
Because of the latest moving of some code to a new repository, the documentation for release and verification
needs to be reorganized. There are 5 relevant documents as follows.
1. release-prepare.md
General instructions for the release and related preparation work.
2. release-doris-core.md
The Doris Core release process
3. release-doris-connectors
The Doris Connectors release process
4. release-complete.md
Release completion after release polling passed.
5. release-verify.md
Release verification methods.
Add scalable regression testing framework(#7584)
contains
- Test framework making by groovy, and support built-in **readable DSL** named as `Action`
- Demo exists in `${DORIS_HOME}/regression-test/data/demo`
- Chinese doc exist in `${DORIS_HOME}/docs/zh-CN/developer-guide/regression-testing.md`
English document coming soon
