Commit Graph

67 Commits

Author SHA1 Message Date
31827e88e3 [fix](merge-cloud) Fix ali open sdk gcc compile issue (#30308) 2024-03-14 10:59:25 +08:00
8ae493e54f [thirdparty](lib) Add Patch for base64 lib (#32059) 2024-03-14 09:20:45 +08:00
b7c86d66fa [improvement](thirdparty) introduce brpc stream patch to make stream write to socket in background bthread (#30458) 2024-01-30 15:33:40 +08:00
a4ccf92fec [fix](thirdparty) patch brpc 1.4.0 to fix stream rpc (#30476) 2024-01-29 19:02:46 +08:00
e1ff439e63 [chore](thirdparty) update vectorscan to 5.7.11 (#29623) 2024-01-07 11:41:26 +08:00
ca6cb04d16 [fix](routine_load) change librdkafka version from 2.0.2 to 1.9.2 to avoid memory leak (#28528) 2023-12-18 18:35:04 +08:00
8a7b0e5485 [upgrade](thirdparty) upgrade librdkafka from 1.8.2 to 2.0.2 (#28210)
Some error happen when using routine load
```
[INTERNAL_ERROR]Message at offset XXX might be too large to fetch, try increasing receive.message.max.bytes
```
Refer to https://github.com/confluentinc/librdkafka/issues/2993, we should upgrade librdkafka version to avoid this bug.
2023-12-12 11:12:10 +08:00
aaa44fa3dc [deps](odbc) undefined the BOOL in include/sqltypes.h (#28014)
When `ENABLE_PCH = false`, this define will be conflict with BOOL in `include/arrow/type_fwd.h`.
The ODBC table will be deprecated in 2.1, so I just simply undefined the BOOL in include/sqltypes.h
to make compile OK
2023-12-06 00:16:23 +08:00
23ab3333a0 [chore](thirdparty) Fix link errors of librdkafka (#27247)
Exclude coverage symbol from librdkafka's version script generator to
avoid compatibility issues with recent linkers such as lld-17.
2023-11-19 21:22:26 +08:00
c1c8376eb2 [fix](thirdparty) brpc stream crash on error stream (#27016)
* [fix](thirdparty) brpc stream crash on error stream

* rename
2023-11-15 18:24:01 +08:00
f6f1e3b646 [chore](build) Bump the version of hyperscan (#25464)
The latest version fixed the previous issue (https://github.com/intel/hyperscan/issues/292).
2023-10-17 08:45:25 -05:00
e1b9854f90 [bugfix](thirdparty) Upgrade aws s3 sdk to prevent mem leak (#25106)
During the use of the AWS S3 SDK, we found that there is a memory leak. According to the official issue, upgrading the SDK should resolve the issue.
2023-10-09 16:08:50 +08:00
94907ca0f2 [improvement](brpc) change parking slot to 32 (#24567) 2023-09-21 21:35:46 +08:00
ba351af452 [enhancement](thirdparty) upgrade thirdparty libs - again (#23414)
submit again #23290 (not upgrade brpc, because bthread local has error)

protobuf 3.15.0 -> 21.11
glog 0.4.0 -> 0.6.0
lz4 1.9.3 -> 1.9.4
curl 7.79.0 -> 8.2.1
zstd 1.5.2 -> 1.5.5
arrow 7.0.0 -> 13.0.0
abseil 20220623.1 -> 20230125.3
orc 1.7.2 -> 1.9.0
jemalloc for arrow 5.2.1 -> 5.3.0
xsimd 7.0.0 -> 13.0.0
opentelemetry-proto 0.19.0 -> 1.0.0
opentelemetry 1.8.3 -> 1.10.0

new:
c-ares -> 1.19.1
grpc -> 1.54.3
2023-08-26 22:59:10 +08:00
303d5ac3f7 Revert "[enhancement](thirdparty) upgrade thirdparty libs (#23290)" (#23420)
This reverts commit 2185268bab2f5890a2c3c9a3c315375a971c4416.

Co-authored-by: stephen <hello-stephen@qq.com>
2023-08-24 14:40:51 +08:00
2185268bab [enhancement](thirdparty) upgrade thirdparty libs (#23290)
protobuf 3.15.0 -> 21.11
glog 0.4.0 -> 0.6.0
lz4 1.9.3 -> 1.9.4
curl 7.79.0 -> 8.2.1
brpc 1.4.0 -> 1.6.0
zstd 1.5.2 -> 1.5.5
arrow 7.0.0 -> 13.0.0
abseil 20220623.1 -> 20230125.3
orc 1.7.2 -> 1.9.0
jemalloc for arrow 5.2.1 -> 5.3.0
xsimd 7.0.0 -> 13.0.0
opentelemetry-proto 0.19.0 -> 1.0.0
opentelemetry 1.8.3 -> 1.10.0

new:
c-ares -> 1.19.1
grpc -> 1.54.3
2023-08-23 20:25:49 +08:00
f68d3a660e [improvement](opentelemetry) upgrade opentelemetry jar to v1.26.0 and opentelemetry-cpp to v1.8.3 (#19733)
why upgrade? anything wrong?

Try to fix the problem about opentelemetry::v1::ext::http::client::curl::HttpOperation::Send(), I have updated the pr info.
2023-05-18 18:46:20 +08:00
Pxl
a2c9ed7be8 [Chore](build) fix some undefined behavior about incomplete type vector #19753 2023-05-18 15:13:45 +08:00
e98f4c4a5e [fix](be) BE UT built against Clang-16 failed (#19610)
If we use Clang-16 to build the third-party libraries and build doris_be_test against them, we can not run doris_be_test successfully. Some errors with BRPC occur.

I tested this on Linux (x86_64) and macOS (x86_64/arm64), these errors always raised.
2023-05-13 22:32:29 +08:00
673cbe3317 [chore](build) Porting to GCC-13 (#19293)
Support using GCC-13 to build the codebase.
2023-05-08 10:42:06 +08:00
5c8ecfbf9c [fix](thirdparty) fix opentelemetry error message compiling with ubsan (#18912) 2023-05-05 19:09:43 +08:00
f9c4542d04 [chore](build) Porting to Clang-16 (#18196)
This PR ports the codebase to Clang-16.

Upgrade some third-party libraries:
1. Apache BRPC: 1.2.0 -> 1.4.0 (Some bugs are fixed and all patches for 1.2.0 can be removed.)
2. Boost: 1.73.0 -> 1.81.0 (Porting to Clang-16)
3. libclucene: 2.4.6 -> 2.4.8 (Porting to Clang-16)
2023-03-30 10:36:29 +08:00
fd0c352144 [fix](thirdparty) patch sqltypes.h in odbc to avoid conflict with clucene macro TCHAR (#15926)
Fix conflit name TCHAR in odbc sqltypes.h and clucene clucene-config.h.
Change TCHAR to TWCHAR in odbc sqltypes.h, because TCHAR in odbc is not found used in doris,
but there are too many places to call clucene's TCHAR.

thirdparty/installed/include/sqltypes.h: 
`typedef char                            TCHAR;`

thirdparty/installed/include/CLucene/clucene-config.h: 
`typedef  wchar_t TCHAR;`
2023-01-15 10:10:00 +08:00
fe02b08e04 [Improvement](thirdparty)upgrade simdjson from 1.0.2 to 3.0.1 (#15412)
Upgrade simdjson from 1.0.2 to latest version 3.0.1 to avoid -mlzcnt compiler flag causing BE UT(macOS) failure.
simdjson is now only used by VJsonScanner and disabled by default. So the impact of upgrade is limited.
2022-12-28 12:24:16 +08:00
7d49ddf50c [bugfix](thirdparty) patch simdjson to avoid conflict with odbc macro BOOL (#15223)
fix conflit name BOOL in odbc sqltypes.h and simdjson element.h. Change BOOL to BOOLEAN in simdjson.

- thirdparty/installed/include/sqltypes.h

> #define	BOOL				int


- thirdparty/src/simdjson-1.0.2/include/simdjson/dom/element.h

> enum class element_type {
>   ARRAY = '[',     ///< dom::array
>   OBJECT = '{',    ///< dom::object
>   INT64 = 'l',     ///< int64_t
>   UINT64 = 'u',    ///< uint64_t: any integer that fits in uint64_t but *not* int64_t
>   DOUBLE = 'd',    ///< double: Any number with a "." or "e" that fits in double.
>   STRING = '"',    ///< std::string_view
>   BOOL = 't',      ///< bool
>   NULL_VALUE = 'n' ///< null
> };
>
2022-12-22 09:40:04 +08:00
Pxl
c816d9d6e5 [Chore](s2geo) disable ABSL_BTREE_ENABLE_GENERATIONS and use CMAKE_BUILD_TYPE to rep… (#15104)
disable ABSL_BTREE_ENABLE_GENERATIONS and use CMAKE_BUILD_TYPE to replace o3
2022-12-15 21:36:28 +08:00
Pxl
decabbb933 [Chore](s2geo) upgrade s2geo to 0.10.0 (#15002)
upgrade s2geo to 0.10.0
2022-12-13 10:34:51 +08:00
7624c80d83 [Feature](Kafka) Add kerberos support for kafka (#14431)
Compile librdkafka with Kerberos SASL GSSAPI support.
2022-11-21 20:45:50 +08:00
015f8ab78d [enhancement](thirdparty) support create stripe reader by column names (#14184)
ORC NextStripeReader now only support read columns by indices, but it is hard to get column indices for complex types.
We patch ORC adapter to support read columns by column names.
Co-authored-by: cambyzju <zhuxiaoli01@baidu.com>
2022-11-11 15:10:20 +08:00
9350c0be80 [Bug](aarch64) Fix the BRPC "sched_to itself" error when building BE by Clang on Linux aarch64 (#13270) 2022-10-11 15:38:28 +08:00
429ac929fb [chore](build) Support building from source on ubuntu-22.04 (aarch64) (#12813)
Support building from source on ubuntu-22.04
2022-09-27 10:29:13 +08:00
cb06e67fba [fix](tracing) Fix opentelemetry log output to be.out (#11856) 2022-09-18 17:40:23 +08:00
47d43b34b3 [enhancement](thirdparty) Compile Jemalloc separately on thirdparty (#12577)
Compile Jemalloc separately and optimize the configuration
2022-09-14 23:31:48 +08:00
a912910d4b [fix](thirdparty) Fix the errors while building brpc (#11714) 2022-08-12 13:59:44 +08:00
0f555e90cb [deps](brpc) Upgrade Brpc to 1.2.0 fix _dl_sym undefined reference on new verison of glibc (#11688) 2022-08-12 09:07:57 +08:00
c6f520fab4 [thirdparty](brpc) fix _dl_sym undefined reference on Ubuntu22.04 (#11643)
Co-authored-by: qcloud <ubuntu@localhost.localdomain>
2022-08-10 19:23:10 +08:00
6a53cf8efa [dependency](be) Add vectorscan for support hypserscan on ARM (#11102)
hyperscan is a high-performance regular expression matching library, but can not be used on ARM.
vectorscan is an ARM port for hyperscan, it can be used as a drop in replacement.

Since hyperscan is original created by Intel and is popular and mature on x86, so we just use vectorscan
only for aarch64 when build thirdparty.
2022-07-26 11:28:25 +08:00
4d158f9050 [dependency](arrow) Add GetRawORCReader function for arrow orc reader (#11069)
Add a new function in arrow adapter to get the raw orc reader which we can get more information
from such offset or min/max value.
And this will be used in #1046
This modify is inspired by Clickhouse
2022-07-21 22:23:05 +08:00
e3c19ded44 [enhancement](thirdparty) Support building thirdparty on macOS (#10677) 2022-07-18 10:50:30 +08:00
e159e748df [chore](dependency) fix opentelemetry-cpp enable o3 optimization will core. (#10675) 2022-07-08 10:08:07 +08:00
e0cf2677a0 [dependency][enhancement] support build libhdfs in arm cpus (#10018)
Supports native hdfs functionality on arm cpu
This pr mainly upgrades libdfs3 and supports running on arm,and make libhdfs3 with kerberos as default
2022-06-10 19:40:41 +08:00
a34d4b55f9 [dependency] Add opentelemetry related third-party library. (#9990) 2022-06-10 09:59:07 +08:00
73a3093539 [dependency] add hyperscan and its dependency ragel to thirdparty (#9964) 2022-06-09 23:36:09 +08:00
b3a2a92bf5 [deps] libhdfs3 build enable kerberos support (#9524)
Currently, the libhdfs3 library integrated by doris BE does not support accessing the cluster with kerberos authentication 
enabled, and found that kerberos-related dependencies(gsasl and krb5) were not added when build libhdfs3.

so, this pr will enable kerberos support and rebuild libhdfs3 with dependencies gsasl and krb5:

- gsasl version: 1.8.0
- krb5 version: 1.19
2022-05-22 20:58:19 +08:00
defdae1e7d [improvement](stream-load) adjust read unit of http to optimize stream load (#9154) 2022-05-20 09:52:36 +08:00
848acec584 [chore](dependency) update Croaring for good performance (#8492)
update Croaring for good performance, according to RoaringBitmap/CRoaring#320
2022-03-17 10:07:55 +08:00
f3c44bcd75 [chore][fix](librdkafka) disable librdkafka assert and update some thirdparty (#8425)
1. comment  librdkafka `rd_assert(thrd_is_current(rkb->rkb_thread));` to avoid core dump
2. upgrade arrow to 7.0.0
3. upgrade aws sdk to 1.9
4. upgrade orc to 1.7.2
2022-03-12 22:09:06 +08:00
7f3564cca6 [chore] Support aarch64 target with ldb_toolchain (#8249) 2022-02-27 10:38:17 +08:00
3904447db8 [fix](toolchain) Allow building when system contains libunwind (#7866) 2022-01-29 12:33:32 +08:00
ca0fac0722 [chore](ldb-toolchain) Support ldb_toolchain on ubuntu 20 (#7846) 2022-01-24 13:16:37 +08:00