Also fix BE ut:
1. fix scheme_change_test memory leak
2. fix mem_pool_test
Do not using DEFAULT_PADDING_SIZE = 0x10 in mem_pool when running ut.
3. remove plugin_test
* make ASAN poisoning work as much as possible
Before this patch a use after poison is reported like below
==19305==ERROR: AddressSanitizer: unknown-crash on address
0x625000137013 at pc 0x561c44bcf6b8 bp 0x7ffb75a00910 sp 0x7ffb75a000b8
After this patch the use after poison is reported like below
==17782==ERROR: AddressSanitizer: use-after-poison on address
0x625000137033 at pc 0x55633c8f56b8 bp 0x7ff3dc437930 sp 0x7ff3dc43
Before this patch, a false memory usage is reported like below
==33080==AddressSanitizer CHECK failed: ../../../../src/libsanitizer/
asan/asan_allocator.cpp:189 "((old)) == ((kAllocBegMagic))"
Because of the latest moving of some code to a new repository, the documentation for release and verification
needs to be reorganized. There are 5 relevant documents as follows.
1. release-prepare.md
General instructions for the release and related preparation work.
2. release-doris-core.md
The Doris Core release process
3. release-doris-connectors
The Doris Connectors release process
4. release-complete.md
Release completion after release polling passed.
5. release-verify.md
Release verification methods.
Add scalable regression testing framework(#7584)
contains
- Test framework making by groovy, and support built-in **readable DSL** named as `Action`
- Demo exists in `${DORIS_HOME}/regression-test/data/demo`
- Chinese doc exist in `${DORIS_HOME}/docs/zh-CN/developer-guide/regression-testing.md`
English document coming soon
As described in #8120, a large number of rowset meta remain in rocksdb, which may be generated by:
1. drop tablet
The drop tablet task itself just sets the state of the tablet meta to `SHUTDOWN`
and moves the tablet to `_shutdown_tablets` vector then the background thread
will periodically clean up the tablet in `_shutdown_tablets` (that's why even if we execute
the `drop table xx force`, the tablet may be delayed by 10min to 1 hour before it goes into the trash directory).
The regular cleanup thread in the background saves the complete tablet meta as a `.hdr` file
when deleting the tablet, and then moves it to the trash directory along with the data files.
But this process does not process the rowset meta (before doing the checkpoint of the tablet meta,
the rowset meta is stored independently in rocksdb as a key-value). So this results in a residual rowset meta.
2. clone task
The clone task may migrate back and forth between BEs, which may result in a situation
where the tablet id is the same on the BE, but the tablet uuid is different.
This leads to some rowset meta can not find the corresponding tablet, but there is no thread
to process these rowsets, and eventually lead to residual.
This is PR, I handled it in the regular cleanup thread with method `_clean_unused_rowset_metas()`.
I did not delete rowset meta along with "drop tablet" task, because "drop tablet" itself is not a synchronous operation.
It also relies on a background thread to clean up the tablet periodically.
So I put this operation in the background cleanup thread.
1. move `group_concat` from string-functions to aggregate-functions.
2. add `json_array`/`json_object`/`json_quote` to sidebar file.
3. move `json_array`/`json_object`/`json_quote`/`get_json_double`/`get_json_int`/`get_json_string` to json-functions.
4. change `group_concat` document to uppercase
This Bug is introduced by PR #7936 , which change key type of connectionMap from Long to Integer,
which cause connectionMap could not find connectContext by connectionId
if you want test storage layer vectorized, you need modify some codes to let vectorized storage layer working,
it's boring work.
now, you can just change one code (redefine the macro STORAGE_LAYER_VECTORIZED_SWITCH as 1 or 0),
this gets more convenient.
