This commit replaces the PowerMock/EasyMock in our unit tests, But not all.
PS.(The tests relevant to DescribeStmt are ignored until I find a way to fix it)
This CL changes:
1. add function bitmap_to_string and bitmap_from_string, which will
convert a bitmap to/from string which contains all bit in bitmap
2. add function murmur_hash3_32, which will compute murmur hash for
input strings
3. make the function cast float to string the same with user result
logic
The problem of inconsistence style in Doris code is too big, it's hard to minimize modification when reformatting code.
So here, our aim is to make the style rules, tune the config in .clang-format.
Note: I choose clang-format-8.0+ to support richer sytle options.
In previous versions, if the children of the IN predicate included NULL, all child types would be converted to DOUBLE for calculation.
For example:
select * from t1 where k1 in ('TABLE', NULL);
But children like varchar cannot be converted to double, so the query cannot be executed.
The error is "TABLE is not a number"
The current version, if null exists in the child, it will not enter the calculation of compatibility type.
For the above query, the compatibility type is varchar, so the 'TABLE' is not converted to double, and the query could be executed.
Also, for JDBC. It will convert 'show tables;' to :
```
SELECT
TABLE_SCHEMA AS TABLE_CAT, NULL AS TABLE_SCHEM, TABLE_NAME,
CASE WHEN TABLE_TYPE='BASE TABLE'
THEN CASE WHEN TABLE_SCHEMA = 'mysql' OR TABLE_SCHEMA = 'performance_schema'
THEN 'SYSTEM TABLE' ELSE 'TABLE'END WHEN TABLE_TYPE='TEMPORARY'
THEN 'LOCAL_TEMPORARY' ELSE TABLE_TYPE END AS TABLE_TYPE, TABLE_COMMENT AS REMARKS, NULL AS TYPE_CAT, NULL AS TYPE_SCHEM, NULL AS TYPE_NAME, NULL AS SELF_REFERENCING_COL_NAME, NULL AS REF_GENERATION
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA LIKE 'test_db'
AND TABLE_NAME LIKE '%'
HAVING TABLE_TYPE IN ('TABLE','VIEW',null,null,null)
ORDER BY TABLE_TYPE, TABLE_SCHEMA, TABLE_NAME
```
In previous version, Doris could not return the correct tables to JDBC. It will thrown the error "'TABLE' is not a number".
After this commit, #2729 is fixed. Doris could return the tables schema by JDBC.
This CL make bitmap_count, bitmap_union, and bitmap_union_count accept any expression whose return type is bitmap as input so that we can support flexible bitmap expression such as bitmap_count(bitmap_and(to_bitmap(1), to_bitmap(2))).
This CL also create separate documentation for each bitmap UDF to conform with other functions.
For #2722
In our test environment, Doris cluster used 1 fe and 7 be (32C+128G). When using spakr-doris connecter to query a table containing 67 columns, it took about 1 hour for the query to return 69 million rows of data. After the improvement, the same query condition took 2.5 minutes and the query performance was significantly improved
Row will be scanned mistakenly after cumulative compaction on singleton rowset.
If I have (1, 1), (2, 2), (3, 3) three records.
Now I have read (1, 1), this bug will make return row is (2, 2)
instead of (1, 1).
This commit contains the following changes:
1. Let create/alter view statement support cte sql. (Issue #2625 )
e.g.
```
Alter view test_tbl_view (h1, h2)
as
with testTbl_cte (w1, w2) as
(
select col1, col2 from testDb.testTbl
)
select w1 as c1, sum(w2) as c2 from testTbl_cte
where w1 > 10
group by w1 order by w1
```
2. Fix the bug that view's schema remains unchanged after replaying alter view. (Issue #2624 )
Currently all singleton rowsets with data are considered overlapping upon construction, even when the rowset contains only one segment. In the meanwhile, singleton rowsets could be input to base compaction (when the tablet hasn't been compacted for base_compaction_interval_seconds_since_last_operation) as long as they are converted into non-overlapping rowsets by cumulative compactor.
By making rowset with one segment non-overlapping, we can avoid the work to convert such rowset to non-overlapping rowset in cumulative compaction.
"Type" is a abstract class, it has 4 sub classes:
1. ScalarType
2. ArrayType
3. MapType
4. StructType
This CL only support ScalarType. Other types can be added later.
the num segments should be read from rowset meta pb.
But the previous code error caused this value not to be set in some cases.
So when init the rowset meta and find that the num_segments is 0(not set),
we will try to calculate the num segments from AlphaRowsetExtraMetaPB,
and then set the num_segments field.
This should only happen in some rowsets converted from old version.
and for all newly created rowsets, the num_segments field must be set.
time(NULL) returns second-resolution timestamp, however all compaction related time in Tablet are in millis-resolution. Therefore should use UnixMillis() instead.
1. upgrade gutil code from imapla to new verison， include `cpuinfo`, `spinlock` and `linux_syscall_support `
2. impliments arm version utf8 check code
3. remove incompatible code from stopwatch
This CL supports arrow's zero copy read interface, which can make code
comply with arrow 0.15.
And the schema change unit test has some problem, I disable it in run-ut.sh
In some scenarios, when a user creates an olap table that is range partition by time, the user needs to periodically add and remove partitions to ensure that the data is valid. As a result, adding and removing partitions dynamically can be very useful for users.
