Commit Graph

291 Commits

Author SHA1 Message Date
cb7e8ff2bb Fix compile failure in ScanNode (#384) 2018-12-04 16:51:48 +08:00
31d1630149 Support 'NO_BACKSLASH_ESCAPES' sql_mode (#382) 2018-12-04 11:33:04 +08:00
d9eb8a2ca1 Fix cast error in BrokerScanNode (#383) 2018-12-04 11:30:03 +08:00
6b4049e21c Unify Slice code path (#380) 2018-12-03 18:11:47 +08:00
603f4e0ca9 Fix a sending signal error when starting Doris BE (#367)
Redirect output message of kill to /dev/null.

Co-Authored-By: chalsliu <45041955+chalsliu@users.noreply.github.com>

ISSUE #365
2018-12-03 15:38:33 +08:00
ff95f23615 Remove OLAP_LOG_DEBUG AND OLAP_LOG_TRACE log format (#378)
Use VLOG(3) and VLOG(10) instead
2018-12-03 10:08:21 +08:00
fb8304123d Support _IP/_HOST in principal of kerberos (#373) 2018-12-03 10:03:57 +08:00
c556ed13f6 Support TRUNCATE TABLE stmt (#377)
* Support TRUNCATE TABLE stmt

User can use TRUNCATE TABLE stmt to empties a table
or partitions completely.
Unlike DELETE, it will drop the tablets directly, and
without any performance impact.

* Fix bugs that new partition should use new ID

* Use equals() to compare Integer

* Fix compile bug

* Fix bug on single range parititon

* Check table's state again after creating partition
2018-12-01 21:18:27 +08:00
5dea8bd3e6 Remove OLAP_LOG_FATAL log format. Use LOG(FATAL) instead (#376) 2018-12-01 19:26:08 +08:00
3d324e38ea Remove OLAP_LOG_INFO log format. Use LOG(INFO) instead (#372) 2018-11-30 20:59:40 +08:00
49302955c8 Revert "Remove OLAP_LOG_INFO log format. Use LOG(INFO) instead (#370)" (#371)
This reverts commit a816925776de06dc7503ea7429802cad9042d0e4.
2018-11-30 20:56:51 +08:00
a816925776 Remove OLAP_LOG_INFO log format. Use LOG(INFO) instead (#370)
* Remove unused row-oriented format flags

* Remove unused row-oriented format flags

* Remove OLAP_LOG_INFO log format. Use LOG(INFO) instead
2018-11-30 20:36:58 +08:00
9447a349ec Subsititue ColumnType to Type (#366)
* Subsititue ColumnType to Type
2018-11-30 16:30:30 +08:00
85d0996b35 Rename Rowset to SegmentGroup (#364)
* Rename Rowset to SegmentGroup

* Modify protobuf related rowset to SegmentGroup
2018-11-29 17:30:41 +08:00
5694bcbd78 Fix stream load failure when target table contains HLL and insert failure when it contains subquery (#359) 2018-11-29 15:40:04 +08:00
aa27d0e056 Fix snapshot's making header bug (#362) 2018-11-28 18:58:21 +08:00
1ffc294833 Ubuntu llvm compile (#361) 2018-11-28 15:22:00 +08:00
f1718578f3 Fix insert error when it contains HLL (#358) 2018-11-27 17:10:41 +08:00
477b6b3d2a Remove unused row-oriented format flags (#357) 2018-11-27 16:34:22 +08:00
cfefa71daa Fix cast error in StreamLoadScanNodeush (#356) 2018-11-27 16:03:33 +08:00
b2d89dfee9 Add connection id to CurrentQueryStatisticsProcDir (#355) 2018-11-27 14:28:39 +08:00
cddd864d83 Avoid 'No more data to read' error when handling stream load RPC (#354)
* Avoid 'No more data to read' error when handling stream load rpc

1. Catch throwable of all stream load rpc.
2. Avoid setting null string as error msg of rpc result status.

* Change setError_msgs to addToError_msgs
2018-11-27 14:18:41 +08:00
dedfccfaf5 Optimize the publish logic of streaming load (#350)
1. Only collect all error replicas if publish task is timeout.
2. Add 2 metrics to monitor the success of failure of txn.
3. Change publish timeout to Config.load_straggler_wait_second
2018-11-26 19:01:50 +08:00
7ac011571f Improve the Backend's disk info report performance (#349)
1. Use the data size save in tablet's header to calculate the disk used capacity.
2. Decrease the default interval of disk and tablet report, from 10 min to 1 min.
2018-11-26 13:42:58 +08:00
33873f2446 Fix wrong query result when column value is Null (#344) 2018-11-26 13:36:23 +08:00
6413518b1e Update README.md (#347)
Move old content of READEME.md to the Overview of Wiki.
Add build guide and user cases to README.md.
2018-11-23 15:40:07 +08:00
bbdf4fba4a Add distributor which schedule task to be fairly, for routine load job (#333)
Step1: updateBeIdTaskMaps, remove unavailable BE and add new alive BE
Step2: process timeout tasks, if a task has already been allocated to BE but not finished before DEFAULT_TASK_TIMEOUT, it will be discarded.
       At the same time, the partitions belong to old tasks will be allocated to a new task. The new task with a signature will be added in the queue of needSchedulerRoutineLoadTask.
Step3: process all needSchedulerRoutineLoadTasks, allocate task to BE. The task will be executed by BE.
2018-11-23 10:35:10 +08:00
be6e9c393d Fix bug of using symbolic link dir as storage path (#340)
* Fix bug of #307

There is a bug to use symbolic link directory as storage root path.
It is a problem that whether the path is canonical

In DownloadAction, checking fails by comparing canonical path with non-canonical path.
So fix the bug by convert all path to canonical path before comparison
2018-11-23 10:12:26 +08:00
8fbc043f93 Change ByteBuffer to StorageByteBuffer in olap/byte_buffer.h (#341)
Class definition of ByteBuffer duplicates between olap/byte_buffer.h and util/byte_buffer.h.
All of the two classes has a function names as remaining().
Some place which want to call remaining() of util/byte_buffer.h is linked to the other remaining() function of olap/byte_buffer.h
2018-11-22 23:18:33 +08:00
cec1b971f6 Add special add_column_statistics method for linked_schema_change (#337) 2018-11-22 15:39:00 +08:00
f9494cbdf2 Fix a compile issue of DORIS_GCC_HOME (#339) 2018-11-22 12:32:14 +08:00
f2df5d523d Fix compile issue of thirdparty library (#338) 2018-11-22 11:15:42 +08:00
485db34f1e Modify partition's version name to what it means (#334)
* Modify partition's version name to what it means.

1. committedVersion(Hash) -> visibleVersion(Hash)
2. currentVersion(Hash) -> committedVersion(Hash)
3. add some comment to make the code more readable

* Check if editlog is null in CatalogIdGenerator
    To avoid unit test failure
2018-11-21 19:21:16 +08:00
fbde6a8636 Support for custom build toolchains (#330)
* Support for custom build toolchains

ISSUE #297

Locate the header file path of DORIS_GCC_HOME, add them to the CPLUS_INCLUDE_PATH environment variable.
So clang can find it.

* Compatible with custom toolchain for clang

ISSUE #297

1. Find default header file paths of DORIS_GCC_HOME and set as enviroment variable CLANG_COMPATIBLE_FLAGS.
2. Explicitly add these paths to CLANG_INCLUDE_FLAGS.
2018-11-21 15:58:12 +08:00
b57dfd4dd3 Fix a compile issue (#331)
* Fix a compile issue

* Fix a compile issue
2018-11-20 20:06:36 +08:00
791e89568e Change PaloMetrics' name and Catalog's Id generator (#329)
* Change PaloMetrics' name and Catalog's Id generator
1. Remove 'Palo' prefix of class Metric.
2. Add a new CatalogIdGenerator to replace the old AtomicLong, to avoid too many edit logs.
3. Add a new histogram to monitor write letency of edit log write.

* modify next id logic

* fix a bug that Metric is not init before using HISTO_EDIT_LOG_WRITE_LATENCY

* fix a problem
2018-11-20 18:59:18 +08:00
9a2ad18428 Add path info of replica in catalog (#327)
Add path info of replica in catalog

Also fix a bug that when calling check_none_row_oriented_table,
store is null, it cannot be used to create table.
Instead, OLAPHeader can be used to get storage type information.
2018-11-19 17:42:46 +08:00
dddaacc2d7 Fix fe.conf audit_log_moudles,sys_log_roll_mode,audit_log_roll_mode settings (#326) 2018-11-17 22:18:19 +08:00
fec3c58655 Change log verbose level to vlog(3) (#325)
* Transform row-oriented table to columnar-oriented table

* Transform row-oriented table to columnar-oriented table

* change log verbose level
2018-11-16 17:17:39 +08:00
b946f20e37 Add docker dev environment (#301)
Add a docker based dev environment.
User can build Doris from source in docker container.
2018-11-16 16:35:08 +08:00
1ba8a4ee4e Transform row-oriented table to columnar-oriented table (#311) 2018-11-16 16:03:56 +08:00
0aea149c0b Fix core local value UT failed (#324)
Issue: #323
2018-11-16 15:27:16 +08:00
e9bf332994 Fix fe.conf audit_log_moudles default value error (#321) 2018-11-16 08:54:57 +08:00
44029937e4 Add scheduler routine load job for stream load (#313)
1. fetch need_scheduler routine load job
2. caculate current concurrent task number of job
3. divide kafka partition into tasks
2018-11-15 21:04:22 +08:00
8ac9492b11 Fix SHOW BACKENDS return ERROR (#320)
In some case, errMsg in Backend maybe null. we change it when check it
nut null

Issue: #317
2018-11-15 20:14:39 +08:00
f1b673503e Fix lose of be's meta data bug (#318)
Modify rocksdb's WriteOptions.sync to true
to force sync WAL to disk every write operation
2018-11-15 19:23:42 +08:00
a2b299e3b9 Reduce UT binary size (#314)
* Reduce UT binary size

Almost every module depend on ExecEnv, and ExecEnv contains all
singleton, which make UT binary contains all object files.

This patch seperate ExecEnv's initial and destory to anthor file to
avoid other file's dependence. And status.cc include debug_util.h which
depend tuple.h tuple_row.h, and I move get_stack_trace() to
stack_util.cpp to reduce status.cc's dependence.

I add USE_RTTI=1 to build rocksdb to avoid linking librocksdb.a

Issue: #292

* Update
2018-11-15 16:17:23 +08:00
d7ee57e881 Optimize quota unit (#309)
Originally, we can only set quota in bytes unit. This commit add quota unit:K/KB/M/MB/G/GB/T/TB/P/PB
for convenience.
2018-11-15 14:03:52 +08:00
3a878fcf09 Improve build script (#305) 2018-11-14 14:10:46 +08:00
c6df9b1e2d Make BE UT success (#304) 2018-11-13 19:17:51 +08:00