database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
16,187 Commits 1 Branch 0 Tags
e1ff439e638b2e6a02600182dfcea2beecebecbc
Commit Graph

294 Commits

Author SHA1 Message Date
Jerry Hu
e1ff439e63 [chore](thirdparty) update vectorscan to 5.7.11 (#29623) 2024-01-07 11:41:26 +08:00
Kang
dbf61005df change boost thirdparty url to offical archives.boost.io (#29401) 2024-01-03 12:12:43 +08:00
HappenLee
7c5fda1169 [thirdparty](lib) Add streamvbyte thirdparty lib (#29308) 2023-12-29 19:22:07 +08:00
HHoflittlefish777
ca6cb04d16 [fix](routine_load) change librdkafka version from 2.0.2 to 1.9.2 to avoid memory leak (#28528) 2023-12-18 18:35:04 +08:00
Xinyi Zou
4bcde57775 [enhancement](memory) Support jemalloc 64K page size - thirdparty (#28429) 2023-12-18 12:38:09 +08:00
HHoflittlefish777
8a7b0e5485 [upgrade](thirdparty) upgrade librdkafka from 1.8.2 to 2.0.2 (#28210) 
Some error happen when using routine load
```
[INTERNAL_ERROR]Message at offset XXX might be too large to fetch, try increasing receive.message.max.bytes
```
Refer to https://github.com/confluentinc/librdkafka/issues/2993, we should upgrade librdkafka version to avoid this bug.
 2023-12-12 11:12:10 +08:00
Mingyu Chen
aaa44fa3dc [deps](odbc) undefined the BOOL in include/sqltypes.h (#28014) 
When `ENABLE_PCH = false`, this define will be conflict with BOOL in `include/arrow/type_fwd.h`.
The ODBC table will be deprecated in 2.1, so I just simply undefined the BOOL in include/sqltypes.h
to make compile OK
 2023-12-06 00:16:23 +08:00
Qi Chen
65126459bd [deps](compression) Opt gzip decompress by libdeflate on X86 and X86_64 platforms: 1. Add libdeflate lib. (#27542) 
Test result:

- env: 1 node(16 cores, 64G).
- parquet column: 100 million rows of char(255) column.
- result: 9.09 s -> 6.04 s.
 2023-11-28 10:14:48 +08:00
Amos Bird
23ab3333a0 [chore](thirdparty) Fix link errors of librdkafka (#27247) 
Exclude coverage symbol from librdkafka's version script generator to
avoid compatibility issues with recent linkers such as lld-17.
 2023-11-19 21:22:26 +08:00
Yongqiang YANG
c1c8376eb2 [fix](thirdparty) brpc stream crash on error stream (#27016) 
* [fix](thirdparty) brpc stream crash on error stream

* rename
 2023-11-15 18:24:01 +08:00
zhiqiang
a5565f68b2 [Refactor](opentelemetry) Remove opentelemetry (#26605) 2023-11-09 18:05:34 +08:00
zclllyybb
4889841ff7 [thirdparty](import) import avx2neon library (#25383) 
import avx2neon library
 2023-10-23 11:51:54 +08:00
Adonis Ling
f6f1e3b646 [chore](build) Bump the version of hyperscan (#25464) 
The latest version fixed the previous issue (https://github.com/intel/hyperscan/issues/292).
 2023-10-17 08:45:25 -05:00
AlexYue
e1b9854f90 [bugfix](thirdparty) Upgrade aws s3 sdk to prevent mem leak (#25106) 
During the use of the AWS S3 SDK, we found that there is a memory leak. According to the official issue, upgrading the SDK should resolve the issue.
 2023-10-09 16:08:50 +08:00
zhiqqqq
11d03a3ab0 [thirdyparty] new thirdy party dragonbox (#24979) 2023-09-28 13:42:44 +08:00
Yongqiang YANG
94907ca0f2 [improvement](brpc) change parking slot to 32 (#24567) 2023-09-21 21:35:46 +08:00
Qi Chen
57ca7d66d3 [Fix](multi-catalog) Fix zlib init error by using doris's zlib shared library and jni.log does not output. (#23260) 2023-09-02 21:44:14 +08:00
Xinyi Zou
ba351af452 [enhancement](thirdparty) upgrade thirdparty libs - again (#23414) 
submit again #23290 (not upgrade brpc, because bthread local has error)

protobuf 3.15.0 -> 21.11
glog 0.4.0 -> 0.6.0
lz4 1.9.3 -> 1.9.4
curl 7.79.0 -> 8.2.1
zstd 1.5.2 -> 1.5.5
arrow 7.0.0 -> 13.0.0
abseil 20220623.1 -> 20230125.3
orc 1.7.2 -> 1.9.0
jemalloc for arrow 5.2.1 -> 5.3.0
xsimd 7.0.0 -> 13.0.0
opentelemetry-proto 0.19.0 -> 1.0.0
opentelemetry 1.8.3 -> 1.10.0

new:
c-ares -> 1.19.1
grpc -> 1.54.3
 2023-08-26 22:59:10 +08:00
Dongyang Li
303d5ac3f7 Revert "[enhancement](thirdparty) upgrade thirdparty libs (#23290)" (#23420) 
This reverts commit 2185268bab2f5890a2c3c9a3c315375a971c4416.

Co-authored-by: stephen <hello-stephen@qq.com>
 2023-08-24 14:40:51 +08:00
Xinyi Zou
2185268bab [enhancement](thirdparty) upgrade thirdparty libs (#23290) 
protobuf 3.15.0 -> 21.11
glog 0.4.0 -> 0.6.0
lz4 1.9.3 -> 1.9.4
curl 7.79.0 -> 8.2.1
brpc 1.4.0 -> 1.6.0
zstd 1.5.2 -> 1.5.5
arrow 7.0.0 -> 13.0.0
abseil 20220623.1 -> 20230125.3
orc 1.7.2 -> 1.9.0
jemalloc for arrow 5.2.1 -> 5.3.0
xsimd 7.0.0 -> 13.0.0
opentelemetry-proto 0.19.0 -> 1.0.0
opentelemetry 1.8.3 -> 1.10.0

new:
c-ares -> 1.19.1
grpc -> 1.54.3
 2023-08-23 20:25:49 +08:00
Adonis Ling
a5382043d7 [chore](third-party) Fix errors while building brpc (#23339) 
close #23340
 2023-08-23 09:13:06 +08:00
Adonis Ling
3b8981bee7 [chore](third-party) Speed the download up for aws-crt-cpp (#22997) 
The package aws-sdk-cpp was upgraded in #20252. We can speed the download up for aws-crt-cpp.
 2023-08-16 09:47:18 +08:00
Xinyi Zou
c2c01825c1 [opt](stacktrace) Optimize stacktrace output #22467 2023-08-06 15:53:53 +08:00
Adonis Ling
9a95d664b9 [chore](third-party) Fix the build order for libunwind (#22244) 
1. libunwind depends on lzma
2. Fix the missing headers issues reported by GCC-13
 2023-07-27 14:07:08 +08:00
Mingyu Chen
0c811edb78 [deps](hadoop) update hadoop libs to 3.3.4.5 (#22062) 2023-07-23 20:17:16 +08:00
Xinyi Zou
e0705f1149 [chore](third-party) Introduce libunwind (#21938) 2023-07-19 01:55:26 +08:00
Qi Chen
fde73b6cc6 [Fix](multi-catalog) Fix hadoop short circuit reading can not enabled in some environments. (#21516) 
Fix hadoop short circuit reading can not enabled in some environments.
- Revert #21430 because it will cause performance degradation issue.
- Add `$HADOOP_CONF_DIR` to `$CLASSPATH`.
- Remove empty `hdfs-site.xml`. Because in some environments it will cause hadoop short circuit reading can not enabled.
- Copy the hadoop common native libs(which is copied from https://github.com/apache/doris-thirdparty/pull/98
) and add it to `LD_LIBRARY_PATH`. Because in some environments `LD_LIBRARY_PATH` doesn't contain hadoop common native libs, which will cause hadoop short circuit reading can not enabled.
 2023-07-06 15:00:26 +08:00
Mingyu Chen
af51a31c21 [deps](benchmark) bump benchmakr from 1.5.6 -> 1.8.0 (#21121) 
To support some new methods used in #21074
 2023-06-25 23:42:54 +08:00
Xinyi Zou
493f9f563c [chore](third-party) temporary rollback brpc to 1.4 (#21011) 2023-06-20 20:16:51 +08:00
Kaijie Chen
b0bbff0fd1 [performance](load) improve memtable sort performance (#20392) 2023-06-04 20:33:15 +08:00
Adonis Ling
fb730fb653 [chore](third-party) Bump the version of hadoop_libs (#20369) 
Bump the version of hadoop_libs to build HDFS related libraries only.
 2023-06-02 17:18:36 +08:00
Mingyu Chen
d963bf8d79 [deps](aws) upgrade to 1.9.272 to fix non-compliant RFC3986 encoding (#20252) 2023-05-31 18:19:06 +08:00
Adonis Ling
f43282e612 [chore](third-party) Bump the version of hadoop_libs (#20250) 
Fix the issues with the workflow Build Third Party Libraries. See https://github.com/apache/doris-thirdparty/actions/runs/5109407220/jobs/9184234534
 2023-05-31 09:21:43 +08:00
Mingyu Chen
047311171d [deps](libhdfs) change hadoop libhdfs to source compile (#20058) 
compile it as other deps, instead of downloading prebuild binary.
Only for linux platform
 2023-05-26 14:37:49 +08:00
Mingyu Chen
b3ce4593b1 [deps](libhdfs) update hadoop libhdfs to 3.3.4.1 for doris (#19832) 2023-05-19 08:44:32 +08:00
airborne12
f32deb18e9 [Update](build) change clucene from thirdparty to git module (#19352) 2023-05-19 08:25:51 +08:00
luozenglin
f68d3a660e [improvement](opentelemetry) upgrade opentelemetry jar to v1.26.0 and opentelemetry-cpp to v1.8.3 (#19733) 
why upgrade? anything wrong?

Try to fix the problem about opentelemetry::v1::ext::http::client::curl::HttpOperation::Send(), I have updated the pr info.
 2023-05-18 18:46:20 +08:00
Pxl
a2c9ed7be8 [Chore](build) fix some undefined behavior about incomplete type vector #19753 2023-05-18 15:13:45 +08:00
Adonis Ling
fe553f7dfc [chore](third-party) Support specifying packages to build (#19688) 
Usage: ./build-thirdparty.sh [options...] [packages...]
  Optional options:
     -j <num>               build thirdparty parallel
     --clean                clean the extracted data
     --continue <package>   continue to build the remaining packages (starts from the specified package)

Examples:
1. Specify packages to build.
    Build gflags, gtest and glog by executing ./build-thirdparty.sh gflags gtest glog.
2. Continue to build the remaining packages.
    Build the remaining packages (starts from sse2neon) by executing ./build-thirdparty.sh --continue sse2neon.
 2023-05-16 19:24:19 +08:00
Adonis Ling
ddcf7ec1b4 [chore](third-party) Don't link keyutils to krb5 explicitly (#19632) 
We may link system-wide keyutils to krb5 when building krb5 which may introduce an extra dependency to the codebase.
 2023-05-16 10:37:37 +08:00
Adonis Ling
e98f4c4a5e [fix](be) BE UT built against Clang-16 failed (#19610) 
If we use Clang-16 to build the third-party libraries and build doris_be_test against them, we can not run doris_be_test successfully. Some errors with BRPC occur.

I tested this on Linux (x86_64) and macOS (x86_64/arm64), these errors always raised.
 2023-05-13 22:32:29 +08:00
airborne12
2504b243f0 [Fix](build) fix clucene build type (#19376) 
RelWithDebInfo default uses O2 as compile flags which hurt performance for clucene
 2023-05-09 14:29:04 +08:00
Adonis Ling
673cbe3317 [chore](build) Porting to GCC-13 (#19293) 
Support using GCC-13 to build the codebase.
 2023-05-08 10:42:06 +08:00
luozenglin
5c8ecfbf9c [fix](thirdparty) fix opentelemetry error message compiling with ubsan (#18912) 2023-05-05 19:09:43 +08:00
Adonis Ling
2c1a5bb352 Revert "[chore](third-party) Fix the checksums of mysql (#19047)" (#19189) 
This reverts commit c93d6ba3be2f2448b824d36da61835e2cd1235cd.
 2023-05-04 10:09:37 +08:00
airborne12
31c1ebc165 [Chore](thirdparty) update clucene from 2.4.11 to 2.4.12 (#19150) 
fix memory leak in Standard Analyzer
 2023-04-27 23:33:26 +08:00
airborne12
0271886c36 [Chore](thirdparty) update clucene from 2.4.9 to 2.4.11 (#18876) 2023-04-26 20:11:35 +08:00
Adonis Ling
c93d6ba3be [chore](third-party) Fix the checksums of mysql (#19047) 
The checksum of MySQL changed which makes the workflows fail.

See https://github.com/apache/doris-thirdparty/actions/runs/4794208534/jobs/8527425262.
 2023-04-25 23:13:53 +08:00
Adonis Ling
d28030e1e5 [chore](third-party) Configure the search paths for pkg-config and cmake (#18624) 
Currently, our third party libraries are built by autotools or cmake. Under some scenarios, we may use system-wide headers or libraries to build them which may make the build process fail.

We can configure the search paths explicitly to help autotools and cmake find the right dependencies.
 2023-04-14 08:43:27 +08:00
Zhengguo Yang
4335c9998f [chore](ARM) Add some vectorization compatibility code on aarch64 (#18553) 
update sse2noen to support more sse code on arm cpus
 2023-04-13 10:15:33 +08:00