Commit Graph

13145 Commits

Author SHA1 Message Date
7709fa5ea5 fix feut failed by mockit.internal.expectations.invocation.MissingInvocation: Missing 1 invocation to: (#23893)
mockit.internal.expectations.invocation.MissingInvocation: Missing 1 invocation to:
org.apache.doris.catalog.Env#isCheckpointThread()
mockit.internal.expectations.invocation.MissingInvocation: 
Missing 1 invocation to:
org.apache.doris.catalog.Env#isCheckpointThread()
Caused by: mockit.internal.expectations.invocation.ExpectationError
resolve http://43.132.222.7:8111/viewLog.html?buildId=213066&buildTypeId=Doris_Doris_FeUt

Issue Number: close #xxx
2023-09-05 14:30:52 +08:00
2bce8bbc66 [fix](spark load) not setting the file format cause null pointer exception (#16202) 2023-09-05 12:14:07 +08:00
e090b83e33 [improvemnt](script) support custom lib dir to save custom libs (#23887)
Sometimes, user need to add some custom libs to the cluster, such lzo.jar, orai18n.jar, etc.
In previous, these lib files are places in fe/lib or be/lib.
But when upgrading cluster, the lib dir will be replaced by new lib dir, so that all custom libs are lost.

In this PR, I add new dir custom_lib for FE and BE, and user can place custom lib files in it.
2023-09-05 11:54:19 +08:00
3f2f2a7218 [typo](docs) remove redundant column data from the insert statement (#23880) 2023-09-05 11:52:18 +08:00
d9461d77ba [Fix](bitmap index) like predicate does not work in bitmap index (#23819) 2023-09-05 11:43:16 +08:00
4dac2d3b94 [Fix](Plan)StreamLoad cannot be parsed correctly when it contains complex where conditions (#23874) 2023-09-05 11:26:59 +08:00
e525e021ee [Enhancement](Load) stream tvf support csv header (#23797)
Co-authored-by: yiguolei <676222867@qq.com>
2023-09-05 11:15:45 +08:00
a91f9fbccb [fix][UT]fix coverage compute fail if run string cmd directly (#23878) 2023-09-05 10:57:54 +08:00
32dd088d30 [fix](inverted)fix inverted index report be hostname error #23872
when inverted index report task status to fe,not put the correct host
delete unused _backend
2023-09-05 10:52:23 +08:00
a36c387a2b [Refactor](transformer) convert to file format writer to transformer (#23888) 2023-09-05 10:50:10 +08:00
a02ee8e1d0 [test](regression) Reduce dependency of other database in test_alter_table_column (#22547) 2023-09-05 10:15:01 +08:00
5853ed385e [pipelineX](join) Support shared hash table (#23876) 2023-09-05 10:14:40 +08:00
b51d8aa7b7 [pipelineX](feature) support empty set operator (#23877) 2023-09-05 10:11:08 +08:00
1d1a9e2bfc [improvement](graceful shutdown) waiting for all query finished when graceful shutdown (#23865)
In some cloud native deployment scenario, BE(especially the Compute Node BE) will be add to cluster and remove from cluster very frequently. User's query will fail if there is a fragment is running on the shutting down BE. Users could use stop_be.sh --grace, then BE will wait all running queries to stop to avoiding running query failure, but if the waiting time exceed the limit, then be will exit directly. During this period, FE will not send any queries to BE and waiting for all running queries to stop
2023-09-05 09:52:28 +08:00
774a771e0c [Improve](be)check swap (#18891)
Co-authored-by: Yongqiang YANG <98214048+dataroaring@users.noreply.github.com>
2023-09-05 09:39:55 +08:00
c9b4749276 [fix](RemoteUDF) fix string type do not set PGenericType #23832 2023-09-05 09:23:20 +08:00
eea9a43d97 [fix](load) add lock for serialize profile (#23856) 2023-09-05 09:11:55 +08:00
2885de1d63 [chore](macOS) Fix invalid option errors in start_be.sh (#23861) 2023-09-05 09:07:53 +08:00
a1db5ef562 [bugfix](regression-test) Repeated create user during parallel testing #23869 2023-09-05 09:06:34 +08:00
babd0430c7 [fix](stacktrace) Fix StackTraceCache initialized before ExecEnv (#23828) 2023-09-05 09:06:16 +08:00
e639454753 [fix](regressionTest) fix test_external_catalog_es p2 regression test #23838 2023-09-05 09:02:16 +08:00
9d758a6c1a [fix](case) disable unstable case in dir nereids_tpcds_shape_sf100_p0 (#23884) 2023-09-04 22:51:21 +08:00
2a3fc92d13 [fix](auth)fix after setting the user password to expire, changing the password again will not take effect (#23426)
Create a jack user and set the password to expire after 10 days.
`CREATE USER 'jack' IDENTIFIED BY '12345' PASSWORD_EXPIRE INTERVAL 10 DAY FAILED_LOGIN_ATTEMPTS 3 PASSWORD_LOCK_TIME 1 DAY;`

After the password has expired, reset the password.
`SET PASSWORD FOR 'jack' = PASSWORD('123');`


Log in with a new password, ERROE ` Your password has expired. To log in you must change it using a client that supports expired passwords`
2023-09-04 21:19:54 +08:00
c1620f9e1a [chore](case) Update regression-conf.groovy (#23873) 2023-09-04 20:54:48 +08:00
7727535b91 [Enhancement](build) try to download commit specific source code when git submodule failed (#23846) 2023-09-04 20:12:46 +08:00
039c76cbc0 [feature-wip] (arrow-flight) (step1) BE support Arrow Flight server, read data only (#23765) 2023-09-04 19:19:55 +08:00
21aea76333 [pipelineX](feature) support assert rows num operator (#23857) 2023-09-04 18:17:26 +08:00
d694f4a946 [doc](bitmap) add docs from bitmap_to_base64 and bitmap_from_base64 (#23805) 2023-09-04 17:57:24 +08:00
Pxl
251e79d9f8 [Bug](compare) fix core dump on decimal128 when be is build by debug mode (#23840)
fix core dump on decimal128 when be is build by debug mode
2023-09-04 17:40:57 +08:00
db306a51b3 [fix](nereids) missing return value of resetLogicalProperties() (#23850)
physicalPlan.resetLogicalProperties(); will not change the origin plan but create a new plan with no logical property. So should update the plan using resetLogicalProperties()'s return value.
2023-09-04 17:37:26 +08:00
72b709d6a9 [opt](stats) split period collector from auto collector (#23622)
1. Split period analyze from auto collector
2. Analyze table incrementally by default
3. Rename StatisticsAutoAnalyzer to StatisticsAutoCollector
2023-09-04 17:04:16 +08:00
1c73fd2687 [pipelineX](ssb) add SSB test cases (#23844) 2023-09-04 16:38:37 +08:00
f37078db99 [fix](testcase) add analyze table in load of tpcds_sf1_index testcase (#23811) 2023-09-04 16:19:48 +08:00
9bc7010639 fix topn be inoperative because Field == Null always return true (#23830)
```if (!new_top.is_null() && new_top != old_top)``` is always false since old_top is Null when init and Field == Null always return true.

We add old_top.is_null() check first to avoid the problem and then issue more carefull discussion about Field == Null semantics.
2023-09-04 16:02:07 +08:00
5dc46efcdf [feature](nereids) eliminate cascading outer join (#23754)
after eliminate outer join, create is-not-null predicate, and then this is-not-null predicate can be used to eliminate descendant outer join. the newly created is-not-null predicate will be eliminated in EliminateNotNull rule.
2023-09-04 16:00:18 +08:00
422159bd94 [pipelineX](refactor) add repeat node in pipelineX (#23750) 2023-09-04 15:55:09 +08:00
301a1d97e1 [fix](row-policy) fix creating row policy with forward issue (#23801)
The `CreateRowPolicyCommand` is implemented with overriding `run()` method.

So when executing `create row policy` in non-master FE, and forward it to Master FE,
it will call `execute(TUniqueId queryId)` method and go through `executeByNereids()`.
And because without `run()` method, it will do nothing and return OK.
So after `show row policy`, user will get empty result.

This PR fix it by implmenting the `run()` method but throw an Exception, so that it will
fallback to old planner, to do the creating row policy command normally.

The full implement of `run()` method should be implemented later.
This is just a tmp fix.
2023-09-04 15:19:37 +08:00
ff92978392 [pipelineX](fix) fix correctness problem (#23823) 2023-09-04 15:04:49 +08:00
9bca52d002 [chore](idea) add back .idea dir under fe (#23821) 2023-09-04 14:01:00 +08:00
Pxl
bb3fadc5d3 [Bug](materialized-view) fix mv not match because cast and alias name (#23580)
fix mv not match because cast and alias name
2023-09-04 12:46:33 +08:00
0179e5c2ba [bugfix](fragment mgr) heap used after free in fragment manager when query is cancelled (#23817)
Co-authored-by: yiguolei <yiguolei@gmail.com>
2023-09-04 12:20:16 +08:00
a1915d13c0 [docs] Correct spelling of words (#23652)
Co-authored-by: 苗宏凯 <9499318+miaohongkai@user.noreply.gitee.com>
2023-09-04 11:55:48 +08:00
0f2bbf5b81 [doc](datetime) description of datetime in DATETIME.md (#23683) 2023-09-04 11:54:16 +08:00
hzq
201c0cc4ef [fix](pipeline) Fix cancel_query locks nothing #23815 2023-09-04 11:50:24 +08:00
3e00d62780 [docs](releasenote) 2.0.1&1.2.7 release note (#23807) 2023-09-04 11:29:03 +08:00
e1f9626564 [regression](inverted index) force nereids open for count on index (#23796) 2023-09-04 11:18:18 +08:00
618f636197 [community](collaborator) add collaborator shuke987 for issue/pr label (#23813)
remove collaborators xiaokang and zy-kkk since they are committer
2023-09-04 11:13:12 +08:00
12ca9d95b6 [opt](Nereids) use avl tree to construct continuous union operand (#23763) 2023-09-04 10:41:48 +08:00
3317909141 [pipelineX](join) support nested loop join operator (#23756) 2023-09-04 10:08:22 +08:00
9da9409bd4 [refactor](join) improve join node output when build table rows is 0 (#23713) 2023-09-04 09:48:38 +08:00