database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
20,619 Commits 1 Branch 0 Tags
443e87e20327eaa5577cc10f08a63ec1694de358
Commit Graph

153 Commits

Author SHA1 Message Date
Mingyu Chen
171258cc31 [deps](hadoop) update hadoop to 3.3.6.3 #39376 (#41748) 
bp #39376
 2024-10-12 16:34:59 +08:00
Xinyi Zou
40500f2b10 [branch-2.1](thirdparty) upgrade arrow to 17.0.0 (#39773) 
pick #38572
 2024-08-22 18:35:32 +08:00
AlexYue
ba0161c8b9 Revert "[feature-wip](Cloud) Introduce azure core C++ sdk (#35208)" (#35826) 
## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->

This reverts commit #35641 because of the compilation of such is not
successful on arm plateform.
 2024-06-03 23:22:56 +08:00
AlexYue
2e4c2a6e6e [feature-wip](Cloud) Introduce azure core C++ sdk (#35208) (#35641) 
bp #35208
 2024-05-30 13:54:53 +08:00
zhiqiang
0ae1b9c70a [chore](remove code) Remove dragonbox related (#34528) 
* Revert "[refactor](mysql result format) use new serde framework to tuple convert (#25006)"

This reverts commit e5ef0aa6d439c3f9b1f1fe5bc89c9ea6a71d4019.

* run buildall

* MORE

* FIX
 2024-05-13 22:16:57 +08:00
Xinyi Zou
39b768b9ea [branch-2.1](thirdparty) upgrade arrow to 15.0.2 #32827 2024-03-27 08:35:40 +08:00
HappenLee
f9ee3ff359 [thirdparty](lib) Add base64 thirdparty lib (#31850) 2024-03-14 09:18:52 +08:00
Jerry Hu
5e60ca0652 [chore](3rd) update CRoaring bitmap version to 2.1.2 (#30697) 2024-02-01 23:14:14 +08:00
Mingyu Chen
7a6475eeee [deps](hadoop) update hadoop on BE side to 3.3.6 (#29939) 
Same as on FE side
 2024-01-14 15:52:04 +08:00
Jerry Hu
e1ff439e63 [chore](thirdparty) update vectorscan to 5.7.11 (#29623) 2024-01-07 11:41:26 +08:00
Kang
dbf61005df change boost thirdparty url to offical archives.boost.io (#29401) 2024-01-03 12:12:43 +08:00
HappenLee
7c5fda1169 [thirdparty](lib) Add streamvbyte thirdparty lib (#29308) 2023-12-29 19:22:07 +08:00
HHoflittlefish777
ca6cb04d16 [fix](routine_load) change librdkafka version from 2.0.2 to 1.9.2 to avoid memory leak (#28528) 2023-12-18 18:35:04 +08:00
HHoflittlefish777
8a7b0e5485 [upgrade](thirdparty) upgrade librdkafka from 1.8.2 to 2.0.2 (#28210) 
Some error happen when using routine load
```
[INTERNAL_ERROR]Message at offset XXX might be too large to fetch, try increasing receive.message.max.bytes
```
Refer to https://github.com/confluentinc/librdkafka/issues/2993, we should upgrade librdkafka version to avoid this bug.
 2023-12-12 11:12:10 +08:00
Qi Chen
65126459bd [deps](compression) Opt gzip decompress by libdeflate on X86 and X86_64 platforms: 1. Add libdeflate lib. (#27542) 
Test result:

- env: 1 node(16 cores, 64G).
- parquet column: 100 million rows of char(255) column.
- result: 9.09 s -> 6.04 s.
 2023-11-28 10:14:48 +08:00
zhiqiang
a5565f68b2 [Refactor](opentelemetry) Remove opentelemetry (#26605) 2023-11-09 18:05:34 +08:00
zclllyybb
4889841ff7 [thirdparty](import) import avx2neon library (#25383) 
import avx2neon library
 2023-10-23 11:51:54 +08:00
Adonis Ling
f6f1e3b646 [chore](build) Bump the version of hyperscan (#25464) 
The latest version fixed the previous issue (https://github.com/intel/hyperscan/issues/292).
 2023-10-17 08:45:25 -05:00
AlexYue
e1b9854f90 [bugfix](thirdparty) Upgrade aws s3 sdk to prevent mem leak (#25106) 
During the use of the AWS S3 SDK, we found that there is a memory leak. According to the official issue, upgrading the SDK should resolve the issue.
 2023-10-09 16:08:50 +08:00
zhiqqqq
11d03a3ab0 [thirdyparty] new thirdy party dragonbox (#24979) 2023-09-28 13:42:44 +08:00
Xinyi Zou
ba351af452 [enhancement](thirdparty) upgrade thirdparty libs - again (#23414) 
submit again #23290 (not upgrade brpc, because bthread local has error)

protobuf 3.15.0 -> 21.11
glog 0.4.0 -> 0.6.0
lz4 1.9.3 -> 1.9.4
curl 7.79.0 -> 8.2.1
zstd 1.5.2 -> 1.5.5
arrow 7.0.0 -> 13.0.0
abseil 20220623.1 -> 20230125.3
orc 1.7.2 -> 1.9.0
jemalloc for arrow 5.2.1 -> 5.3.0
xsimd 7.0.0 -> 13.0.0
opentelemetry-proto 0.19.0 -> 1.0.0
opentelemetry 1.8.3 -> 1.10.0

new:
c-ares -> 1.19.1
grpc -> 1.54.3
 2023-08-26 22:59:10 +08:00
Dongyang Li
303d5ac3f7 Revert "[enhancement](thirdparty) upgrade thirdparty libs (#23290)" (#23420) 
This reverts commit 2185268bab2f5890a2c3c9a3c315375a971c4416.

Co-authored-by: stephen <hello-stephen@qq.com>
 2023-08-24 14:40:51 +08:00
Xinyi Zou
2185268bab [enhancement](thirdparty) upgrade thirdparty libs (#23290) 
protobuf 3.15.0 -> 21.11
glog 0.4.0 -> 0.6.0
lz4 1.9.3 -> 1.9.4
curl 7.79.0 -> 8.2.1
brpc 1.4.0 -> 1.6.0
zstd 1.5.2 -> 1.5.5
arrow 7.0.0 -> 13.0.0
abseil 20220623.1 -> 20230125.3
orc 1.7.2 -> 1.9.0
jemalloc for arrow 5.2.1 -> 5.3.0
xsimd 7.0.0 -> 13.0.0
opentelemetry-proto 0.19.0 -> 1.0.0
opentelemetry 1.8.3 -> 1.10.0

new:
c-ares -> 1.19.1
grpc -> 1.54.3
 2023-08-23 20:25:49 +08:00
Mingyu Chen
0c811edb78 [deps](hadoop) update hadoop libs to 3.3.4.5 (#22062) 2023-07-23 20:17:16 +08:00
Xinyi Zou
e0705f1149 [chore](third-party) Introduce libunwind (#21938) 2023-07-19 01:55:26 +08:00
Mingyu Chen
af51a31c21 [deps](benchmark) bump benchmakr from 1.5.6 -> 1.8.0 (#21121) 
To support some new methods used in #21074
 2023-06-25 23:42:54 +08:00
Xinyi Zou
493f9f563c [chore](third-party) temporary rollback brpc to 1.4 (#21011) 2023-06-20 20:16:51 +08:00
Kaijie Chen
b0bbff0fd1 [performance](load) improve memtable sort performance (#20392) 2023-06-04 20:33:15 +08:00
Adonis Ling
fb730fb653 [chore](third-party) Bump the version of hadoop_libs (#20369) 
Bump the version of hadoop_libs to build HDFS related libraries only.
 2023-06-02 17:18:36 +08:00
Mingyu Chen
d963bf8d79 [deps](aws) upgrade to 1.9.272 to fix non-compliant RFC3986 encoding (#20252) 2023-05-31 18:19:06 +08:00
Adonis Ling
f43282e612 [chore](third-party) Bump the version of hadoop_libs (#20250) 
Fix the issues with the workflow Build Third Party Libraries. See https://github.com/apache/doris-thirdparty/actions/runs/5109407220/jobs/9184234534
 2023-05-31 09:21:43 +08:00
Mingyu Chen
047311171d [deps](libhdfs) change hadoop libhdfs to source compile (#20058) 
compile it as other deps, instead of downloading prebuild binary.
Only for linux platform
 2023-05-26 14:37:49 +08:00
Mingyu Chen
b3ce4593b1 [deps](libhdfs) update hadoop libhdfs to 3.3.4.1 for doris (#19832) 2023-05-19 08:44:32 +08:00
airborne12
f32deb18e9 [Update](build) change clucene from thirdparty to git module (#19352) 2023-05-19 08:25:51 +08:00
luozenglin
f68d3a660e [improvement](opentelemetry) upgrade opentelemetry jar to v1.26.0 and opentelemetry-cpp to v1.8.3 (#19733) 
why upgrade? anything wrong?

Try to fix the problem about opentelemetry::v1::ext::http::client::curl::HttpOperation::Send(), I have updated the pr info.
 2023-05-18 18:46:20 +08:00
Adonis Ling
e98f4c4a5e [fix](be) BE UT built against Clang-16 failed (#19610) 
If we use Clang-16 to build the third-party libraries and build doris_be_test against them, we can not run doris_be_test successfully. Some errors with BRPC occur.

I tested this on Linux (x86_64) and macOS (x86_64/arm64), these errors always raised.
 2023-05-13 22:32:29 +08:00
Adonis Ling
2c1a5bb352 Revert "[chore](third-party) Fix the checksums of mysql (#19047)" (#19189) 
This reverts commit c93d6ba3be2f2448b824d36da61835e2cd1235cd.
 2023-05-04 10:09:37 +08:00
airborne12
31c1ebc165 [Chore](thirdparty) update clucene from 2.4.11 to 2.4.12 (#19150) 
fix memory leak in Standard Analyzer
 2023-04-27 23:33:26 +08:00
airborne12
0271886c36 [Chore](thirdparty) update clucene from 2.4.9 to 2.4.11 (#18876) 2023-04-26 20:11:35 +08:00
Adonis Ling
c93d6ba3be [chore](third-party) Fix the checksums of mysql (#19047) 
The checksum of MySQL changed which makes the workflows fail.

See https://github.com/apache/doris-thirdparty/actions/runs/4794208534/jobs/8527425262.
 2023-04-25 23:13:53 +08:00
Zhengguo Yang
4335c9998f [chore](ARM) Add some vectorization compatibility code on aarch64 (#18553) 
update sse2noen to support more sse code on arm cpus
 2023-04-13 10:15:33 +08:00
Adonis Ling
17e1b6d1b2 [chore](third-party) Porting to GCC-12 (#18560) 
The definitions of _mm_cvtsi128_si16 conflicts. This PR resolves this issue.
 2023-04-12 10:33:24 +08:00
Mingyu Chen
2a301eb437 [deps](arrow) update arrow download link (#18360) 2023-04-04 10:39:04 +08:00
Adonis Ling
f9c4542d04 [chore](build) Porting to Clang-16 (#18196) 
This PR ports the codebase to Clang-16.

Upgrade some third-party libraries:
1. Apache BRPC: 1.2.0 -> 1.4.0 (Some bugs are fixed and all patches for 1.2.0 can be removed.)
2. Boost: 1.73.0 -> 1.81.0 (Porting to Clang-16)
3. libclucene: 2.4.6 -> 2.4.8 (Porting to Clang-16)
 2023-03-30 10:36:29 +08:00
Pxl
1a549edac2 [Chore](third-party) upgrade thrift from 0.13 to 0.16 (#17202) 
upgrade thrift from 0.13 to 0.16
There is thrift's release notes https://github.com/apache/thrift/blob/master/CHANGES.md
 2023-03-10 11:33:16 +08:00
Mingyu Chen
bc5f16dc80 [deps](libhdfs) add official hadoop libhdfs for x86 (#17435) 
This is the first step to introduce official hadoop libhdfs to Doris.
Because the current hdfs client libhdfs3 lacks some important feature and is hard to maintain.

Download the hadoop 3.3.4 binary from hadoop website: https://hadoop.apache.org/releases.html
Extract libs and headers which are used for libhdfs, and pack them into hadoop_lib_3.3.4-x86.tar.gz
Upload it to https://github.com/apache/doris-thirdparty/releases/tag/hadoop-libs-3.3.4

TODO:
The hadoop libs for arm is missing, we need to find a way to build it
 2023-03-06 12:47:53 +08:00
airborne12
ad86b931d4 [Thirdparty](clucene) update clucene to v2.4.6 to fix bthread/pthread context bug (#16982) 
1. change clucene version from 2.4.4->2.4.6
2. update build-thirdparty.sh clucene's build block, adding USE_BTHREAD CMAKE flag, this flag is inherited from doris's USE_BTHREAD_SCANNER.
 2023-02-22 11:24:45 +08:00
Mingyu Chen
cd3dbc33c9 [deps](be) update libhdfs3 and jemalloc (#16894) 
- Modified: libhdfs3 2.3.7 -> 2.3.8
- Modified: jemalloc 5.2.1 -> 5.3.0  (#14871)
 2023-02-19 19:49:27 +08:00
Mingyu Chen
f02aa7e3e2 [thirdparty](libhdfs3) update libhdfs3 to 2.3.7 to fix kms seek bug (#16265) 2023-01-31 08:56:00 +08:00
HappenLee
f0ab71aa84 [Thirdpart](lib) Add lib fast_float to replace std::from_chars() convert float/double (#16204) 2023-01-30 00:41:56 +08:00