pick #39412
At present, if the table structure changes, the routine load cannot
perceive it. As a long-running load, it should be able to perceive the
changes in the table structure.
…
## Proposed changes
javax.el is the API for Java Expression Language, which provides a
simple and flexible way for Java Web applications to access and
manipulate data. EL expressions are commonly used in JSP pages, but we
are not involved in their use here, so I removed it.
- upgrade jackson to 2.16.0
(cherry picked from commit 33a0ea5565c2e0d7e5909ca3b512fb0b539f05f4)
#38843
Issue Number: close #xxx
<!--Describe your changes.-->
…
improve the flexibility of the project by decoupling direct dependencies
on the hadoop-cos and hadoop-huaweicloud libraries. These changes allow
users to control whether COS and OBS dependencies are included in the
final build, enabling a more customizable setup.
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
pick (#39975)
We encountered a scenario where a large number of small transactions
were generated, resulting in an impact on query performance: Kafka's
data comes in batches of very small data every very short time, which
leads to tasks being frequently scheduled and ending very quickly,
resulting in a large number of small transactions.
To solve this problem, we delay the scheduling of tasks that perceive
EOF, which would not delay data consumption, for perceiving EOF
indicates that the consumption speed is greater than the production
speed.
## Proposed changes
1. Schema change should rebuild distribution info after modifying
columns, especially distribution columns. Or it may cause dynamic
partition failed when checking distribution columns' equality.
2. Support hash key columns to do light schema change. For unique key or
dup key columns, could not be enabled temporarily due to some historical
reasons. See #39798 .
pick #40367
Meet code dump when using single stream multi table load:
```
SUMMARY: AddressSanitizer: heap-buffer-overflow /root/doris/be/src/io/fs/multi_table_pipe.cpp:99:22 in doris::io::MultiTablePipe::dispatch(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, char const*, unsigned long, doris::Status (doris::io::KafkaConsumerPipe::*)(char const*, unsigned long))
```
1. It is hard to guaranteed that msg is a C-style string ending in '\0'
character. If not, it may cause the core dump to access memory out of
bounds.
2. It is not need to calculate the length of msg twice.
Therefore, deleting the logic that using strlen to calculate the length
of msg.
## Proposed changes
`select count() from table where a + b > 0 or b > 0`
- When _execute_predicates_except_leafnode_of_andnode is executed, an
Expr tree is traversed from bottom to top. When it reaches the leaf node
b, the information of this column b is placed into new_predicate_info.
- However, this step is skipped directly at an ADD node, which leads to
the GT node at the upper level generating a sign equivalent to b > 0,
the same as the sign on the right side b > 0.
- This causes the compound OR calculation to assume that both GT
conditions below have been evaluated, thus prematurely computing this
EXPR, when in fact, the ADD node has not been evaluated.
- If the SQL is written as SELECT COUNT(*) FROM table WHERE b + a > 0 OR
b > 0, the calculation would be correct because the sign generated by
this > node would be equivalent to a > 0, which is different from b > 0
on the right side.
…
## Proposed changes
For internal tasks, such as MTMV, the start time may already be set, or
the time may be adjusted immediately.
<!--Describe your changes.-->
(cherry picked from commit 904a6c0fc1a804520285533de874fe4d0ffff2c1)
## Proposed changes
Issue Number: close#36805
<!--Describe your changes.-->
