989e03ddf9
[improvement] Improve sig handler ( #8545 )
...
* Refactor glog's default signal handler
Co-authored-by: Zhengguo Yang <780531911@qq.com >
2022-03-22 10:40:31 +08:00
848acec584
[chore](dependency) update Croaring for good performance ( #8492 )
...
update Croaring for good performance, according to RoaringBitmap/CRoaring#320
2022-03-17 10:07:55 +08:00
a4b710cb2d
[chore](dependency) fix build thirdparty errors ( #8456 )
...
1. the patch for aws-c-cal-0.4.5 does not need anymore
2. remove duplicate bit_length document
3. add some debug log for routine load
2022-03-13 22:11:24 +08:00
f3c44bcd75
[chore][fix](librdkafka) disable librdkafka assert and update some thirdparty ( #8425 )
...
1. comment librdkafka `rd_assert(thrd_is_current(rkb->rkb_thread));` to avoid core dump
2. upgrade arrow to 7.0.0
3. upgrade aws sdk to 1.9
4. upgrade orc to 1.7.2
2022-03-12 22:09:06 +08:00
50a59f3f86
[license] Organize third-party dependent licenses for bianry releases ( #8350 )
2022-03-07 23:18:58 +08:00
7f3564cca6
[chore] Support aarch64 target with ldb_toolchain ( #8249 )
2022-02-27 10:38:17 +08:00
0d5b297cad
[chore] add aclocal/automake version check ( #8055 )
2022-02-22 09:19:14 +08:00
16020cbdf9
[fix](lateral-view) Fix bug that explode_json_array_string return unstable result ( #8152 )
...
Co-authored-by: morningman <chenmingyu@baidu.com >
2022-02-21 09:38:36 +08:00
789472a87d
[build] change docker image add perf tools and simdjson ( #8012 )
2022-02-11 18:54:09 +08:00
b1b1a8c64d
[chore] Fix the return code of function download_func ( #7919 )
2022-01-31 22:14:46 +08:00
3904447db8
[fix](toolchain) Allow building when system contains libunwind ( #7866 )
2022-01-29 12:33:32 +08:00
ea96242529
[chore] add set -e to download_thirdparty.sh ( #7867 )
...
Let the build process stop immediately if some command execute failed.
For example, patch failed due to command not found, otherwise it will stop until compile error.
2022-01-27 10:33:08 +08:00
ca0fac0722
[chore](ldb-toolchain) Support ldb_toolchain on ubuntu 20 ( #7846 )
2022-01-24 13:16:37 +08:00
f2cbf0a8d2
[chore] Improve the ldb toolchain compilation documentation ( #7829 )
...
Add document for compiling Doris with ldb toolchain
2022-01-21 21:36:43 +08:00
800a36343a
[chore] Prolog of hermetic build with GCC 11 and Clang 13. ( #7712 )
...
Prepare to generate hermetic build using GCC 11 and Clang 13.
The ideal toolchain would be ldb toolchain generated by [ldb_toolchain_gen.sh](https://github.com/amosbird/ldb_toolchain_gen/releases/download/v0.3/ldb_toolchain_gen.sh )
To kick off a clang build, set `DORIS_TOOLCHAIN=clang` before running any build scripts.
2022-01-21 12:12:04 +08:00
0efef1b332
[fix](schema-change) Fix bug that schema change may return -102 error ( #7808 )
...
When using linked schema change, we need to check if all rowsets are of the same type,
ALPHA or BETA. otherwise, we need to use direct schema change to convert the data.
2022-01-21 10:59:54 +08:00
7b13ac5b31
[deps][chore] make openssl works with old glibc version ( #7541 )
...
1. build OpenSSL with --with-rand-seed=devrandom
2. Modified: brpc 1.0.0-rc02 -> 1.0.0
2021-12-31 23:19:04 +08:00
2d72c039ad
[deps](openssl) upgrade openssl to 1.1.1m ( #7446 )
...
upgrade openssl to 1.1.1m, ready for support SM2 / SM3 / SM4 national secret (national commercial password) algorithm
2021-12-21 10:09:36 +08:00
e9536a8cf1
[deps](cyrus_sasl) Add -fPIC for cyrus_sasl ( #7408 )
2021-12-17 13:11:25 +08:00
382351b0ee
[fix](ut) Fix run fe ut failed, be ut memory leak and build thirdparty failed ( #7377 )
2021-12-15 11:00:20 +08:00
92020e6e85
[deps](librdkafka) set --enable-sasl option in rdkafka build to enable plain password auth at routine load ( #7251 )
...
```
create routine load rd_001
on tb1
with append
COLUMNS(user_id, date)
properties (
"desired_concurrent_number" = "3",
"max_batch_interval" = "5",
"max_batch_rows" = "300000",
"max_batch_size" = "209715200",
"max_error_number" = "100",
"format" = "json"
)
from KAFKA (
"kafka_broker_list" = "127.0.0.1:9092",
"kafka_topic" = "topic1",
"property.security.protocol" = "sasl_plaintext",
"property.sasl.mechanism" = "PLAIN",
"property.sasl.username" = "your-username",
"property.sasl.password" = "your-password",
"property.group.id" ="group1",
"property.client.id" = "client-1",
"property.kafka_default_offsets" = "OFFSET_BEGINNING"
);
```
2021-12-02 11:44:37 +08:00
b6a9207a25
[deps](brpc) fix compile bug that could not find protobuf lib during compile ( #7197 )
2021-11-24 10:44:26 +08:00
00e653e812
[deps] add breakpad for minidump ( #7128 )
...
Related #7123
2021-11-18 14:34:38 +08:00
6c6380969b
[refactor] replace boost smart ptr with stl ( #6856 )
...
1. replace all boost::shared_ptr to std::shared_ptr
2. replace all boost::scopted_ptr to std::unique_ptr
3. replace all boost::scoped_array to std::unique<T[]>
4. replace all boost:thread to std::thread
2021-11-17 10:18:35 +08:00
e476e155c6
[Bug] Fix inappropriate Cmake option used to build ZSTD ( #7111 )
2021-11-16 10:04:16 +08:00
aba7d2ccae
[Thirdparty] Fix flatbuffers download url error ( #6968 )
...
Change google flatbuffers download URL in thirdparty.var.sh
2021-11-02 16:34:17 +08:00
51e210869a
[ARM64] Fix some problem when compiling on ARM64 platform ( #6836 ) ( #6872 )
...
With thirdparties 1.4.0 to 1.4.1
1. Add patch for aws-c-cal-0.4.5
2. Add some solutions for `undefined reference libpsl`
3. Move libgsasl to fix link problme of libcurl.
4. Downgrade openssl to 1.0.2k to fix problem of low version glibc
2021-10-19 13:26:02 +08:00
24d38614a0
[Dependency] Upgrade thirdparty libs ( #6766 )
...
Upgrade the following dependecies:
libevent -> 2.1.12
OpenSSL 1.0.2k -> 1.1.1l
thrift 0.9.3 -> 0.13.0
protobuf 3.5.1 -> 3.14.0
gflags 2.2.0 -> 2.2.2
glog 0.3.3 -> 0.4.0
googletest 1.8.0 -> 1.10.0
snappy 1.1.7 -> 1.1.8
gperftools 2.7 -> 2.9.1
lz4 1.7.5 -> 1.9.3
curl 7.54.1 -> 7.79.0
re2 2017-05-01 -> 2021-02-02
zstd 1.3.7 -> 1.5.0
brotli 1.0.7 -> 1.0.9
flatbuffers 1.10.0 -> 2.0.0
apache-arrow 0.15.1 -> 5.0.0
CRoaring 0.2.60 -> 0.3.4
orc 1.5.8 -> 1.6.6
libdivide 4.0.0 -> 5.0
brpc 0.97 -> 1.0.0-rc02
librdkafka 1.7.0 -> 1.8.0
after this pr compile doris should use build-env:1.4.0
2021-10-15 13:03:04 +08:00
5ef3f59928
[Optimize][RoutineLoad] Avoid sending tasks if there is no data to be consumed ( #6805 )
...
1 Avoid sending tasks if there is no data to be consumed
By fetching latest offset of partition before sending tasks.(Fix [Optimize] Avoid too many abort task in routine load job #6803 )
2 Add a preCheckNeedSchedule phase in update() of routine load.
To avoid taking write lock of job for long time when getting all kafka partitions from kafka server.
3 Upgrade librdkafka's version to 1.7.0 to fix a bug of "Local: Unknown partition"
See offsetsForTimes fails with 'Local: Unknown partition' edenhill/librdkafka#3295
4 Avoid unnecessary storage migration task if there is no that storage medium on BE.
Fix [Bug] Too many unnecessary storage migration tasks #6804
2021-10-13 11:39:01 +08:00
83003cc372
[Thirdparty] Change libhdfs3 download url to a stable one( #6744 )
2021-10-03 11:56:36 +08:00
3812cbf117
fix librdkafka depends on high version of glibc ( #6634 )
2021-09-14 09:58:23 +08:00
4dd610c28d
[Feature] Support for storage layer benchmark ( #6506 )
...
* add benchmark tool
2021-09-02 09:57:19 +08:00
abbc9202af
support routine load isolation_level read_committed ( #6191 )
...
Co-authored-by: Geoffrey <gaofeng01@rd.netease.com >
2021-08-30 17:22:08 +08:00
d9cc235d6d
Fix typo pdqosrt ( #6441 )
2021-08-14 10:24:12 +08:00
9216735cfa
[New Featrue] Support Vectorization Execution Engine Interface For Doris ( #6329 )
...
1. FE vectorized plan code
2. Function register vec function
3. Diff function nullable type
4. New thirdparty code and new thrift struct
2021-08-11 14:54:06 +08:00
de2dd2c015
[Thirdparty] Move DataTables download url to a fixed download address ( #6189 )
...
The origin download url is always changing: https://datatables.net/download/builder?bs-3.3.7/jq-3.3.1/dt-1.10.25
So we put it in our own http server.
If someone can offer an official url for DataTables, please update this.
2021-07-10 10:13:26 +08:00
48bd680068
update download info for boost and datatables ( #6008 )
2021-06-18 11:38:41 +08:00
d7e62e361f
[Bug] Fix that build thirdparty of parallel-hashmap-1.33 failed on ubuntu18.04 ( #6033 )
...
Co-authored-by: caiconghui <caiconghui@xiaomi.com >
2021-06-17 14:45:34 +08:00
80220af271
[Enhancement] Use Parallel Hash Map Replace Unordered Map In Dict Encodeing Map And Hyper Set ( #5990 )
...
Use Parallel Hash Map Replace Unordered Map In Dict Encodeing Map And Hyper Set To Improve Ferformance
2021-06-10 17:38:08 +08:00
109b55ee5f
[Shell] Add build parallel option ( #5819 )
...
Add build parallel option then we can build project with a user specified parallel not a fixed value.
2021-05-19 09:32:58 +08:00
2883ebeb0a
[Compile] fix s2 link /usr/local/openssl ( #5810 )
2021-05-17 11:52:21 +08:00
bd88309346
[Refactor] fix warning in gcc8+, fix warning from brpc, s2 ( #5763 )
...
Fix warning from brpc, S2
Fix -Warray-bounds
2021-05-12 10:38:46 +08:00
6a1313594c
[Thirdparty] Fix thirdparty build failed ( #5754 )
2021-05-05 10:20:25 +08:00
29a3fa1084
[Feature] Support read data with format of parquet from hdfs, using libhdfs3 ( #5686 )
...
Add new lib, Backend can read data from hdfs without broker,
this patch include libhdfs3.a which can read file on hdfs.
This patch will make reading the data from hdfs with parquet possible.
By this, we will support more format of file on hdfs in the future,
and we will support other metadata in the future.
2021-04-24 17:41:48 +08:00
7445051174
[Refactor] fix warning in gcc8+, update rapidjson ( #5649 )
2021-04-20 09:14:44 +08:00
2b5e4dc951
Add fmt library to speed up mysql text result serialization ( #5554 )
...
* Add fmt library to speed up mysql text result serialization
* use BUILD_SYSTEM instead of make
Co-authored-by: gaodayue <gaodayue@bytedance.com >
2021-04-08 09:16:58 +08:00
5a0a039026
[refacor] Remove minizip source code ( #5571 )
2021-04-08 09:12:22 +08:00
08bfd1c762
[Bug] Fix some problem with downloading and building thirdparty ( #5439 )
2021-03-30 10:47:45 +08:00
ad67dd34a0
update gcc to gcc 10 and support c++17 ( #5394 )
...
* update gcc to gcc 10 and support c++17
update brpc to 0.9.7
update boost to 1.73
remove third-party boost 1.54 for mysql
* update cmake version
* ignore jdk version
* remove unused patch
* avoid use SYS_getrandom call
2021-03-25 09:30:38 +08:00
d6ac8f4e35
Masking glibc symbols for better portability ( #4180 )
...
* Masking glibc symbols for better portability
* Remove redundant files
2021-03-05 13:15:55 +08:00
