347cceb530
[Feature](inverted index) push count on index down to scan node ( #22687 )
...
Co-authored-by: airborne12 <airborne12@gmail.com >
2023-09-02 22:24:43 +08:00
fe6bae2924
[fix](invert index) supports utf8 and non-utf8 strings ( #22570 )
...
supports utf8 and non-utf8 strings: [fix] compatible with utf8 and invalid utf8 doris-thirdparty#110
2023-08-05 12:52:53 +08:00
7fe08c74fe
[fix](inverted index) return empty result instead of error for empty match query ( #22592 )
...
return empty result instead of error for empty match query as follows:
`SELECT * FROM t WHERE msg MATCH ''`
`SELECT * FROM t WHERE msg MATCH 'stop_word'`
2023-08-04 17:36:32 +08:00
0cd5183556
[Refactor](inverted index) refact tokenize function for inverted index ( #22313 )
2023-08-02 19:12:22 +08:00
fc2b9db0ad
[Feature](inverted index) add tokenize function for inverted index ( #21813 )
...
In this PR, we introduce TOKENIZE function for inverted index, it is used as following:
```
SELECT TOKENIZE('I love my country', 'english');
```
It has two arguments, first is text which has to be tokenized, the second is parser type which can be **english**, **chinese** or **unicode**.
It also can be used with existing table, like this:
```
mysql> SELECT TOKENIZE(c,"chinese") FROM chinese_analyzer_test;
+---------------------------------------+
| tokenize(`c`, 'chinese') |
+---------------------------------------+
| ["来到", "北京", "清华大学"] |
| ["我爱你", "中国"] |
| ["人民", "得到", "更", "实惠"] |
+---------------------------------------+
```
2023-07-25 15:05:35 +08:00
c01230f99a
[fix](match) Optimize the logic for match_phrase function filter ( #21622 )
2023-07-25 14:22:37 +08:00
b656f31cf2
[Enchancement](compatible) show decimalv3 to decimal ( #21782 )
2023-07-18 09:17:14 +08:00
00c48f7d46
[opt](regression case) add more index change case ( #21734 )
2023-07-12 21:52:48 +08:00
c58d5cd81b
[opt](regression case) add more index change regression case ( #21633 )
2023-07-08 22:23:09 +08:00
5506faa7b4
[datetimev2](minor) Add scale parameter for datetimev2 ( #21176 )
2023-06-27 19:55:35 +08:00
1ac8cdec7e
[Fix](inverted index) fix inverted query cache for chinese tokenizer ( #21106 )
...
1. query cache for chinese tokenizer is confusing when just converting w_char to char.
2. seperate query_type from inverted_index_reader to clean code.
2023-06-25 22:04:02 +08:00
6ac0bfeceb
[Feature](inverted index) add unicode parser for inverted index ( #21035 )
2023-06-21 20:14:06 +08:00
cc53391c9a
Revert "[feature](merge-on-write) enable merge on write by default (#… ( #21041 )
2023-06-21 18:36:46 +08:00
656b9ad3da
[enhancement](index) Nereids support no need to read raw data for index column that only in filter conditions ( #20605 )
2023-06-09 21:54:48 +08:00
b62c5a70c7
[fix](match query) fix array column match query failed without inverted index ( #20344 )
2023-06-02 21:10:12 +08:00
adc3acb283
[fix](match) fix match query with compound predicates return -6003 ( #20361 )
2023-06-02 18:25:37 +08:00
6adb3fdf11
[fix](match_phrase) Fix the inconsistent query result for 'match_phrase' after creating index without support_phrase property ( #20258 )
...
if create inverted index without support_phrase property, remaining the match_phrase condition to filter by match function.
2023-05-31 18:09:50 +08:00
90b4e127e3
[Feature](inverted index) add parser_mode properties for inverted index parser ( #20116 )
...
We add parser mode for inverted index, usage like this:
```
CREATE TABLE `inverted` (
`FIELD0` text NULL,
`FIELD1` text NULL,
`FIELD2` text NULL,
`FIELD3` text NULL,
INDEX idx_name1 (`FIELD0`) USING INVERTED PROPERTIES("parser" = "chinese", "parser_mode" = "fine_grained") COMMENT '',
INDEX idx_name2 (`FIELD1`) USING INVERTED PROPERTIES("parser" = "chinese", "parser_mode" = "coarse_grained") COMMENT ''
) ENGINE=OLAP
);
```
2023-05-29 23:21:52 +08:00
55ccddb62c
[Conf](decimalv3) enable decimalv3 by default
2023-05-29 15:38:31 +08:00
ae352997b4
[Enhancement](alter inverted index) Improve alter inverted index performance with light weight add or drop inverted index ( #19063 )
2023-05-28 11:23:07 +08:00
ee34b6de2d
[Refact] (serde) refact mysql serde with data type ( #19543 )
...
refact mysql output (de)serialize with data type serde , avoid accoriding switch case Primitive type writed in mysqlWriter
2023-05-26 14:11:17 +08:00
5167dc1251
[feature](merge-on-write) enable merge on write by default ( #19017 )
2023-05-11 11:10:48 +08:00
98b8bef05b
[bugfix](inverted index) fix inverted index to support NULL value filter ( #18302 )
2023-04-15 13:20:26 +08:00
1f9372558d
[improve](regression case) Add more inverted index regression case ( #18589 )
...
1. add more inverted index regression case for unique mow
2. add inverted index case with different data types
2023-04-12 20:40:55 +08:00
0b8bc51b72
[fix](inverted index) Fix key column match query failed ( #18436 )
...
* [fix](inverted index) Fix key column match query failed
* [chore](regression case) add regression case
* [fix] fix regression case no order by
2023-04-08 15:45:08 +08:00
e77833bfa1
[Bug](materialized-view) fix where clause persistence replay incorrect ( #18228 )
...
fix where clause persistence replay incorrect
2023-04-03 12:49:01 +08:00
5f2dad29ca
[enhancement](inverted index) Support inverted index without specified parser to use match query ( #17110 )
2023-02-24 20:34:55 +08:00
43eca4f209
[Feature-WIP](inverted index) Implementation for alter inverted index. ( #16371 )
...
implementation for add/drop inverted index.
2023-02-10 17:56:17 +08:00
bb179b77f7
[Feature-WIP](inverted index) support array type for inverted index reader ( #16355 )
2023-02-02 16:14:14 +08:00
