Commit Graph

178 Commits

Author SHA1 Message Date
0728f08c65 [improvement](dependency) Use release mode to build cctz. Improve performance. 2022-08-19 17:56:52 +08:00
4fa53b4cdb [chore](workflow) Add shellcheck to check shell scripts (#11744) 2022-08-18 16:07:28 +08:00
a912910d4b [fix](thirdparty) Fix the errors while building brpc (#11714) 2022-08-12 13:59:44 +08:00
0f555e90cb [deps](brpc) Upgrade Brpc to 1.2.0 fix _dl_sym undefined reference on new verison of glibc (#11688) 2022-08-12 09:07:57 +08:00
c6f520fab4 [thirdparty](brpc) fix _dl_sym undefined reference on Ubuntu22.04 (#11643)
Co-authored-by: qcloud <ubuntu@localhost.localdomain>
2022-08-10 19:23:10 +08:00
8b9d299472 [improvement](thirdparty) Build re2 with release mode (#11578) 2022-08-07 20:50:07 +08:00
95091256b0 [chore](deps) update bdbje tp doris bdbje, update libhdfs3 to improve performance (#11497) 2022-08-04 17:10:56 +08:00
c176ff54a5 [improvement](orc)improve hdfs scan performance (#11501)
Co-authored-by: root <root@bytedance>
2022-08-04 17:10:30 +08:00
36784d9131 [bugfix](compile) Fix compile hyperscan failed when build thirdparty #11481 2022-08-04 11:21:17 +08:00
edf92f10ff [fix](build) The nproc command does not exist in macos by default, use sysctl -n hw.logicalcpu instead to get the number of logical cores available (#11379) 2022-08-02 09:11:14 +08:00
8f85ad1c96 [doc](intro) update the introduce (#11364) 2022-08-01 14:22:17 +08:00
d1d66f7ffc [chore](thirdparty) Build script doesn't interrupt when errors occur. (#11357)
1. Add a comment
2. Remove all &&
2022-07-31 20:53:50 +08:00
ab7e65112c [fix](thirdparty) fix compile error: missing libbrpc.a (#11353)
fix compile error, add fpic to brpc dependency library
2022-07-31 13:26:08 +08:00
Pxl
6c2337483b [compile]remove all fpic on thirdpatry build (#11298) 2022-07-29 14:16:43 +08:00
6a53cf8efa [dependency](be) Add vectorscan for support hypserscan on ARM (#11102)
hyperscan is a high-performance regular expression matching library, but can not be used on ARM.
vectorscan is an ARM port for hyperscan, it can be used as a drop in replacement.

Since hyperscan is original created by Intel and is popular and mature on x86, so we just use vectorscan
only for aarch64 when build thirdparty.
2022-07-26 11:28:25 +08:00
4d158f9050 [dependency](arrow) Add GetRawORCReader function for arrow orc reader (#11069)
Add a new function in arrow adapter to get the raw orc reader which we can get more information
from such offset or min/max value.
And this will be used in #1046
This modify is inspired by Clickhouse
2022-07-21 22:23:05 +08:00
e3c19ded44 [enhancement](thirdparty) Support building thirdparty on macOS (#10677) 2022-07-18 10:50:30 +08:00
e159e748df [chore](dependency) fix opentelemetry-cpp enable o3 optimization will core. (#10675) 2022-07-08 10:08:07 +08:00
b7441ed291 [chore] remove default REPOSITORY_URL link (#10599) 2022-07-05 11:07:18 +08:00
67f341f44e [TLP](step-1) Remove incubator prefix (#10230)
Remove some `incubator-` prefix in source code.
The document is not modified, will be done in next PR.
2022-06-19 19:34:52 +08:00
14d21edf65 [fix] croaringbitmap compile support USE_AVX2=0 (#10140)
* If we disable AVX2 by config USE_AVX2=0, we need to croaringbitmap with ROARING_DISABLE_AVX=ON

* update to trigger regression test again

Co-authored-by: cambyzju <zhuxiaoli01@baidu.com>
2022-06-16 18:17:46 +08:00
3f9436c6a8 [compile]fix simdjson compile flags (#10054) 2022-06-16 11:28:51 +08:00
39a2785ce2 [enhancement] support simd instructions on arm cpus through sse2neon (#10068)
* [enhancement] support simd instructions on arm cpus through sse2neon
2022-06-14 09:17:09 +08:00
7cf0cc7dd6 [deps] update libhdfs3 to fix a uuid set problem (#10092) 2022-06-14 09:16:32 +08:00
e0cf2677a0 [dependency][enhancement] support build libhdfs in arm cpus (#10018)
Supports native hdfs functionality on arm cpu
This pr mainly upgrades libdfs3 and supports running on arm,and make libhdfs3 with kerberos as default
2022-06-10 19:40:41 +08:00
a34d4b55f9 [dependency] Add opentelemetry related third-party library. (#9990) 2022-06-10 09:59:07 +08:00
73a3093539 [dependency] add hyperscan and its dependency ragel to thirdparty (#9964) 2022-06-09 23:36:09 +08:00
Pxl
c0ad1be1bd [Enhancement][Chore] remove breakpad and unused variable (#9937) 2022-06-02 20:52:17 +08:00
9f26c6a34f [chore] fix some compatible on old linux distributions (#9815) 2022-06-02 10:37:10 +08:00
635b5630e3 [fix](third-party) Fix build issues of third-party in ubuntu docker (#9805)
Fixes:
1. Fully static linkage of protoc may lead to core dump when using it.
2. Building libmyclient may fail due to miss-used option `-v` of `make`
3. Building lzma implicitly requires autopoint
4. Building xml2 implicitly requires pkg-config
2022-05-29 17:33:59 +08:00
cbbda7857b [feature-wip](parquet-orc) Support orc scanner in vectorized engine (#9541) 2022-05-26 21:39:12 +08:00
ddda91c89d [doc] Update dev image (#9721) 2022-05-23 11:59:15 +08:00
b3a2a92bf5 [deps] libhdfs3 build enable kerberos support (#9524)
Currently, the libhdfs3 library integrated by doris BE does not support accessing the cluster with kerberos authentication 
enabled, and found that kerberos-related dependencies(gsasl and krb5) were not added when build libhdfs3.

so, this pr will enable kerberos support and rebuild libhdfs3 with dependencies gsasl and krb5:

- gsasl version: 1.8.0
- krb5 version: 1.19
2022-05-22 20:58:19 +08:00
defdae1e7d [improvement](stream-load) adjust read unit of http to optimize stream load (#9154) 2022-05-20 09:52:36 +08:00
0b1b3e225d Revert "[Thirdparty]Add llvm for codegen (#8938)" (#8948)
This reverts commit 32133621c69a2d7544549c5ea54ed6d9de60415e.

Reverts #8938
The LLVM requires GLIBC_2.15. I decided to create a branch for the llvm feature first.
And once we resolve the low version glibc issue, it will be merged back to master.
2022-04-11 13:03:16 +08:00
32133621c6 [Thirdparty]Add llvm for codegen (#8938) 2022-04-10 20:37:09 +08:00
79be81a8a4 [chore] Optimize build_lz4 in build-thirdparty.sh (#8653) 2022-03-28 10:24:32 +08:00
b2861f36c4 [chore] optimize aws thirdparty package download. (#8637) 2022-03-28 09:35:51 +08:00
989e03ddf9 [improvement] Improve sig handler (#8545)
* Refactor glog's default signal handler

Co-authored-by: Zhengguo Yang <780531911@qq.com>
2022-03-22 10:40:31 +08:00
848acec584 [chore](dependency) update Croaring for good performance (#8492)
update Croaring for good performance, according to RoaringBitmap/CRoaring#320
2022-03-17 10:07:55 +08:00
a4b710cb2d [chore](dependency) fix build thirdparty errors (#8456)
1. the patch for aws-c-cal-0.4.5 does not need anymore
2. remove duplicate bit_length document
3. add some debug log for routine load
2022-03-13 22:11:24 +08:00
f3c44bcd75 [chore][fix](librdkafka) disable librdkafka assert and update some thirdparty (#8425)
1. comment  librdkafka `rd_assert(thrd_is_current(rkb->rkb_thread));` to avoid core dump
2. upgrade arrow to 7.0.0
3. upgrade aws sdk to 1.9
4. upgrade orc to 1.7.2
2022-03-12 22:09:06 +08:00
50a59f3f86 [license] Organize third-party dependent licenses for bianry releases (#8350) 2022-03-07 23:18:58 +08:00
7f3564cca6 [chore] Support aarch64 target with ldb_toolchain (#8249) 2022-02-27 10:38:17 +08:00
Pxl
0d5b297cad [chore] add aclocal/automake version check (#8055) 2022-02-22 09:19:14 +08:00
16020cbdf9 [fix](lateral-view) Fix bug that explode_json_array_string return unstable result (#8152)
Co-authored-by: morningman <chenmingyu@baidu.com>
2022-02-21 09:38:36 +08:00
789472a87d [build] change docker image add perf tools and simdjson (#8012) 2022-02-11 18:54:09 +08:00
b1b1a8c64d [chore] Fix the return code of function download_func (#7919) 2022-01-31 22:14:46 +08:00
3904447db8 [fix](toolchain) Allow building when system contains libunwind (#7866) 2022-01-29 12:33:32 +08:00
ea96242529 [chore] add set -e to download_thirdparty.sh (#7867)
Let the build process stop immediately if some command execute failed.
For example, patch failed due to command not found, otherwise it will stop until compile error.
2022-01-27 10:33:08 +08:00