Commit Graph

17549 Commits

Author SHA1 Message Date
afa9b6ecab Add meta store service thrift definition (#1783) 2019-09-12 10:30:49 +08:00
a85ffa1c2a Fix FE log error (#1785) 2019-09-11 16:13:34 +08:00
b327643132 Fix bug that failed to limit the mem usage of HLL column when loading (#1778)
Should use arena to allocate mem for HyperLogLog column.
2019-09-11 10:20:46 +08:00
5a12a1d7df Fix compile error (#1780) 2019-09-10 23:48:42 +08:00
dcdfc5faf2 Update .gitignore: ignore cmake dir (#1779) 2019-09-10 21:40:11 +08:00
bf373758b2 Make CpuInfo::get_current_core work (#1773) 2019-09-10 19:35:55 +08:00
235cdb0ecd Commit kafka offset (#1734)
Commit kafka offset in routine load

Kafka will decide whether to delete data based on whether all consumer group is commit offset or not. If there is no commit offset, the kafka server disk may be full
2019-09-10 14:27:06 +08:00
40a11c41a9 Fix BE crash when schema changing with HLL column (#1772) 2019-09-10 13:59:38 +08:00
5653822298 Writer magic number in footer instead of header (#1771) 2019-09-10 09:54:13 +08:00
044489b92f Optimize some kinds of load jobs (#1762)
1. Support specifying label to Insert Into stmt.

    INSERT INTO tbl1 WITH LABEL label1 ...;

2. Return job' state corresponding to the existing label in result of stream load.

    ...
    "Status": "Label Already Exists",
    "ExistingJobStatus": "FINISHED"
    ...

3. Return the recent 2000 transactions in SHOW PROC '/transactions'
2019-09-09 22:11:12 +08:00
8b663bf416 Fix bug: unknown column from the inline view (#1770)
Revert code from PR-1617. The column who belongs to inline view need to be initialized by alias.
2019-09-09 20:57:42 +08:00
0f44ce99ce Fix segment v2 comment (#1769) 2019-09-09 18:26:48 +08:00
cd5cfea5cc Encapsulate HLL logic (#1756) 2019-09-09 15:52:10 +08:00
a349409838 Move compare from RowCursor to row (#1764) 2019-09-09 14:51:13 +08:00
ca23b7a511 Should create init rowset for alter task v2 (#1767) 2019-09-09 13:59:19 +08:00
5acdeee4d2 Assign schema_size from other Schema (#1768) 2019-09-09 13:50:14 +08:00
fd2937360c Get rid of external_sorting when rowsets have already been filtered (#1760) 2019-09-08 21:31:12 +08:00
b85cb0071b Bug-fix: error result of union stmt (#1758)
ISSUES-1725: The result of union stmt whose child is outer join stmt is incorrect.

Example:
sql: (select k1 from empty) union all (select b.k1 k1 from left_table a left join empty  b on a.k2 = b.k2);
context: the empty table has no data.
error result: 0
expect result: null

Reason:
The judgment (columns k1 who belongs to union tuple is nullable ) is incorrect.
It could not be determined by slot attribute of children when the slot is produced by the outer join.
The slot A is not nullable while the result of outer join is nullable which is same as slot A.
So, the judgment needs to consider if the slot is come from the outer join.
2019-09-08 21:26:31 +08:00
f23ac0eadd Planner support push down predicates past agg, win and sort (#1471) 2019-09-08 09:30:46 +08:00
2f52ae7988 Add PreAgg Hint (#1617)
eg:
SELECT xxx FROM tbl /*+ PREAGGOPEN */ 
This will open pre-aggregation forcibly for the specified table
2019-09-06 19:47:18 +08:00
981e0feb99 Check rowset is useful atomicly (#1750)
* Check rowset is useful atomicly

* Only release rowset id when it is added to unused rowset

* remove release rowset id when save rowset meta
2019-09-06 17:21:42 +08:00
65dcabf1df Use crc32c checksum for segment v2 (#1753) 2019-09-06 15:23:57 +08:00
54fd3652e6 Fix bug in BetaRowsetReader which results in empty result (#1754) 2019-09-06 15:07:23 +08:00
da69812f65 Fix compile error (#1749) 2019-09-05 22:30:41 +08:00
3f22238012 Add check for to_bitmap function argument (#1747) 2019-09-05 18:11:38 +08:00
85940a292b RowsetFactory as a single entry for Rowset creation (#1748) 2019-09-05 18:10:18 +08:00
0dc0dadad1 Reduce unnecessary memory allocat and copy in OlapScanNode (#1742) 2019-09-04 21:05:12 +08:00
a84c64785f Shuffle partitioned instance to avoid skew (#1744) 2019-09-04 18:31:18 +08:00
726509e9b9 Add MIN/MAX aggregate function compatible with char/varchar (#1739) 2019-09-04 17:28:27 +08:00
a63989cc61 Use RowsetFactory to create and init RowsetWriter (#1740) 2019-09-04 17:02:43 +08:00
fddfffe4c0 Fix bug that failed to create a new partition when no partition in a table (#1688) 2019-09-04 13:36:17 +08:00
03b399150e Not add alter task to tablet in alter tablet request v2 (#1741) 2019-09-03 21:34:52 +08:00
f76dad289e Basic implementation for BetaRowsetReader (#1718) 2019-09-03 13:52:16 +08:00
9f5e5717d4 Unify the msg of 'Memory exceed limit' (#1737)
The new msg of limit exceed: "Memory exceed limit. %msg, Backend:%ip, fragment:%id Used:% , Limit:%. xxx".
This commit unifies the msg of 'Memory exceed limit' such as check_query_state, RETURN_IF_LIMIT_EXCEEDED and LIMIT_EXCEEDED.
2019-09-03 10:42:16 +08:00
a80e9996a6 Move version to high 8 bit (#1736) 2019-09-02 19:43:04 +08:00
b4f6f755f1 Add exchange in MemPool to reduce alloc/free operation (#1732)
Reuse allocated chunks when storage read operation.
2019-09-02 19:29:30 +08:00
8034d83e20 Add scroll keepalive and http timeout configuration (#1731) 2019-09-02 19:04:30 +08:00
6f4feca3dc Add rowset id generator to FE and BE (#1678) 2019-09-02 18:51:31 +08:00
81ca3e3abf Free olap scanner out of lock (#1733)
Close scanner out of OlapScanner's batch lock,
which will lead all scanners wait for one scanner to finish.
2019-09-02 16:49:28 +08:00
ba170aa9e6 Fix NPE of DataDescription (#1735)
When user does not specify column mapping in BrokerLoadStmt, NPE may be thrown.
2019-09-02 16:03:26 +08:00
f87abd93c8 Modify the website (#1730)
1. Add Apache incubator disclaimer.
2. Add "Edit on Github" button on every page.
3. Add Committer Ling Miao.
4. Modify some English documents.
2019-08-31 19:49:44 +08:00
1164264e9d Add English version Doris website (#1729) 2019-08-30 22:07:24 +08:00
76987275b9 Fix result of unix_timestamp() (#1727) 2019-08-30 21:39:16 +08:00
06b87d998a Error check about column which has no default value (#1728)
This commit check the all of parsed column include hadoop function and other function.
Otherwise, the load will thrown the "Column has no default value" exception while the column also has been defined by a non-hadoop function.
2019-08-30 20:23:32 +08:00
206f5394ee Limit V2 segment file row count (#1647) (#1705) 2019-08-30 18:54:37 +08:00
3a33f3d350 Make bitmap_union agg column support insert into and broker load (#1721) 2019-08-30 14:44:51 +08:00
378ce8ca04 Use double when converting TIME type value (#1722)
TIME type value is saved in DOUBLE, so using int64 can extend the time range.
2019-08-29 21:19:19 +08:00
ecbdfc2cee Avoid consistency problem when has no more data (#1716) 2019-08-29 18:57:49 +08:00
7a0c7f45b2 Add English documents for Doris (#1719)
The english documents is translated by machine. So It may has some mistake.
We will fix them later
2019-08-29 13:47:15 +08:00
056a9fada3 fix delete bug (#1720) 2019-08-29 10:42:04 +08:00