Commit Graph

33 Commits

Author SHA1 Message Date
08ebef2992 [Enhancement] check vm.max_map_count before starting (#11052)
When vectorized engine is enabled, doris uses much more vmas than before,
and it leads to core dump due to memory allocation failure.
2022-07-21 21:16:48 +08:00
7147a7c290 [feature-wip](multi-catalog) Support s3 storage for file scan node (#10977)
This is an example of s3 hms_catalog:
```sql
CREATE CATALOG hms_catalog properties(
"type" = "hms",
"hive.metastore.uris"="thrift://localhost:9083",
"AWS_ACCESS_KEY" = "your access key",
"AWS_SECRET_KEY"="your secret key",
"AWS_ENDPOINT"="s3 endpoint",
"AWS_REGION"="s3-region",
"fs.s3a.paging.maximum"="1000");
```
All these params are necessary;
2022-07-21 17:38:53 +08:00
f78db1d773 release memory allocated in agg function in vec stream load (#10739)
release memory allocated in agg function in vec stream load

When a load is cancelled, memory allocated by agg functions should
be freeed.
2022-07-16 15:32:53 +08:00
89bec9b56a [enhancement](be) be asan add asan_suppr.conf to ignore known leak. (#10768) 2022-07-12 19:51:34 +08:00
6f29a8ac0d [refactor] update stop_be.sh to avoid error message (#10691)
* update stop_be.sh to avoid error message

* update stop_be.sh
2022-07-08 20:49:00 +08:00
bff561c0da [feature](script) add --grace option for stop_be.sh (#10626)
be asan mem leak check needs exit app gracefully.
2022-07-06 17:53:01 +08:00
8a49c7ef04 [chore] Rename Doris binary output format 2022-06-24 15:30:05 +08:00
4df1106e1e [improvement](script) Add jvm parameters and the process will automatically stop when oom occurs in fe. (#9765) 2022-05-30 09:44:12 +08:00
Pxl
e772163b98 [fix](script) meet error on start_fe.sh(#9187)
start_fe.sh: line 174: [: -eq: unary operator expected
2022-04-26 10:21:03 +08:00
3bdfcde8e8 [Improvement] not print logs to fe.out when fe is running under daemon mode (#9195)
Co-authored-by: yiguolei <yiguolei@gmail.com>
2022-04-25 18:29:29 +08:00
4a10b37ca2 [feature](image tool) support image load tool (#8982) 2022-04-23 21:36:58 +08:00
419ec3b96c [Fix Bug] Fix ehco command not found (#9021) 2022-04-15 13:43:47 +08:00
0d761f9909 [feature-wip][UDF][DIP-1] Support variable-size input and output for Java UDF (#8678)
This feature is proposed in DSIP-1. This PR support variable-length input and output Java UDF.
2022-04-11 09:36:16 +08:00
b89e4c7bba [feature-wip](java-udf) support java UDF with fixed-length input and output (#8516)
This feature is propsoed in [DSIP-1](https://cwiki.apache.org/confluence/display/DORIS/DSIP-001%3A+Java+UDF). 
This PR support fixed-length input and output Java UDF. Phase I in DIP-1 is done after this PR.

To support Java UDF effeciently, I use no data copy in JNI call and all compute operations are off-heap in Java.
To achieve that, I use a UdfExecutor instead. 

For users, a UDF class must have a public evaluate method.
2022-03-23 10:32:50 +08:00
a390b766d4 [Improvement] BE could print log foreground when not use daemon mode (#8031) 2022-02-14 09:30:12 +08:00
5bd9fdb8c1 [Improvement] print log foreground if not use --daemon to start fe (#7995) 2022-02-10 17:19:39 +08:00
bc4ceeca44 [improvement] optimize java cmd find (#7428)
* optimize java cmd find, if java_home not set use java in PATH
2021-12-30 10:16:56 +08:00
6a00c68264 Fix a typo ehco -> echo (#5433) 2021-03-03 14:46:54 +08:00
8e9bbfb3ba [Script] Check and create if the log directory not existed before outputing message to the log file. (#4929)
This is a minor issue when we had FE start after a fresh installation,
but it will occur an error about the log directory is missing due to log directory is not existed
before some environment check message outputing to the log file.
the log directory creation code in bin/start_fe.sh is in the wrong place,
only need to put the log directory creation code in the beginning.
2020-11-22 20:52:32 +08:00
a61d0de173 [ODBC SCAN NODE] 4/4 Add ODBC_SCAN_NODE and Odbc_Scanner in BE and add ODBC_SCAN_NODE docs (#4438) 2020-09-25 10:19:50 +08:00
35b09ecd66 [JDK] Support OpenJDK (#2804)
Support compile and running Frontend process and Broker process with OpenJDK.
OpenJDK 13 is tested.
2020-02-20 23:47:02 +08:00
7d58aa530f Add --daemon option to start script (#642)
Add --daemon option to start_fe.sh/start_be.sh/start_broker.sh
If run scripte without --daemon, it will run as a foreground process.
2019-02-20 16:38:28 +08:00
603f4e0ca9 Fix a sending signal error when starting Doris BE (#367)
Redirect output message of kill to /dev/null.

Co-Authored-By: chalsliu <45041955+chalsliu@users.noreply.github.com>

ISSUE #365
2018-12-03 15:38:33 +08:00
2868793b6b Change license to Apache License 2.0 (#262) 2018-11-01 09:06:01 +08:00
06413999e5 Fix some compile and script errors (#259)
1. Fix error class in start_fe.sh and start_broker.sh.
2. Add log4j2.xml in fe/src/test/resources/ to run fe ut without log4j warnings.
3. Reduce the test file size in be ut.
2018-10-31 19:45:21 +08:00
ae9ce81453 Changed: change build.sh to use environment variable to get thirdparty's
path, and change PALO_HOME to DORIS_HOME
2018-10-30 16:29:06 +08:00
bea10e4f06 1. hide password and other sensitive information in log and audit log
2. add 2 new proc '/current_queries' and '/current_backend_instances' to monitor the current running queries.
3. add a manual compaction api on Backend to trigger cumulative or base compaction manually.
4. add Frontend config 'max_bytes_per_broker_scanner' to limit to bytes per one broker scanner. This is to limit the memory cost of a single broker load job
5. add Frontend config 'max_unfinished_load_job' to limit load job number: if number of running load jobs exceed the limit, no more load job is allowed to be submmitted.
6. a log of bug fixed
2018-09-19 20:04:01 +08:00
cc74efb3c5 merge to ddb65b69f9c788e359e191889cb31f15279c41ec (#224)
1. Apache HDFS broker support HDFS HA and Hadoop kerberos authentication.
2. New Backup and Restore function. Use Fs Broker to backup your data to HDFS or restore them from HDFS.
3. Table-Level Privileges. Grant fine-grained privileges on table-level to specified user.
4. A lot of bugs fixed.
5. Performance improvement.
2018-08-24 17:12:26 +08:00
aad32ec8fc merge to 3eb59087fb98ed61655997de525331736c56256a (#215)
1. fixed a bug when cast float to string
2. delay result's cancelled time
3. update type check in CastTo
4. remove unused mysql channel log
5. update stop_be.sh, use kill -0 instead of flock -nx
6. update jprotobuf-rpc to 3.5.15 in Frontend, the previous version will cause too many threads in JVM
2018-07-26 19:22:37 +08:00
585c21fab4 add feature and fix bugs (#148)
Add new features:
1. plugins of Ambari and k8s deploy
2. specified config 'priority_network' to solve some ip problems

Fix bugs:
fix bugs that rebalance does not work in some case.
fix count(*) from union stmt bug
fix some union stmt bugs
fix bugs when try to schema change a clone replica
2017-11-30 16:31:12 +08:00
fde6b39539 modify start and stop script. User can now specified PID file dir and log dir (#71) 2017-09-04 20:56:50 +08:00
6486be64c3 fix license statement (#29)
* change picture to word

* change picture to word

* SHOW FULL TABLES WHERE Table_type != VIEW sql can not execute

* change license description
2017-08-18 19:16:23 +08:00
e2311f656e baidu palo 2017-08-11 17:51:21 +08:00