1. this pr is used to fix the be core dump when select the invalid array.
2. before the change, we run "select array_intersect([1, 2, 3, 1, 2, 3], '1[3, 2, 5]');" will cause be core dump.
MySQL [example_db]> select array_intersect([1, 2, 3, 1, 2, 3], '1[3, 2, 5]');
ERROR 1105 (HY000): RpcException, msg: io.grpc.StatusRuntimeException: UNAVAILABLE: Network closed for unknown reason
3. after the change, we run "select array_intersect([1, 2, 3, 1, 2, 3], '1[3, 2, 5]');" will get error message.
MySQL [example_db]> select array_intersect([1, 2, 3, 1, 2, 3], '1[3, 2, 5]');
errCode = 2, detailMessage = No matching function with signature: array_intersect(array<tinyint(4)>, varchar(-1))"
Co-authored-by: hucheng01 <hucheng01@baidu.com>
There are some issues with the compile flag `-Wno-unused-but-set-variable` for clang.
1. `-Wno-unused-but-set-variable` should be set when building source by clang-15 on Linux. (#13000#13016)
2. On macOS Monterey, Apple Clang 13 may treat it as a unknown warning option and the compilation process may interrupt.
This PR introduces a better way to make this compile flag more portable.
1. Test whether the compiler recognizes this flag.
2. Add this flag if the compiler recognizes it.
Issue Number: close#12574
This pr adds `NewJsonReader` which implements GenericReader interface to support read json format file.
TODO:
1. modify `_scann_eof` later.
2. Rename `NewJsonReader` to `JsonReader` when `JsonReader` is deleted.
The index for external table columns from path is incorrect in new scanner. This is a fix for it.
e.g. In the next query, nation and city columns are from path
```
mysql> select nation, city, count(*) from parquet_two_part group by nation, city;
+--------+------------+----------+
| nation | city | count(*) |
+--------+------------+----------+
| cn | beijing | 1199969 |
| cn | shanghai | 1199771 |
| jp | tokyo | 599715 |
| rus | moscow | 600659 |
| us | chicago | 1199805 |
| us | washington | 1201296 |
+--------+------------+----------+
6 rows in set (0.39 sec)
```
From now, we don't support type derivation for array function's arguments.
So that the cases below will return wrong values or even cause be core.
mysql> select array_union([1],[10000000]);
+----------------------------------------+
| array_union(ARRAY(1), ARRAY(10000000)) |
+----------------------------------------+
| [1, -128] |
+----------------------------------------+
1 row in set (0.03 sec)
mysql> select array_union([NULL],[1]);
ERROR 1105 (HY000): RpcException, msg: io.grpc.StatusRuntimeException: UNAVAILABLE: Network closed for unknown reason
mysql> select array_union([],[1]);
ERROR 1105 (HY000): RpcException, msg: io.grpc.StatusRuntimeException: UNAVAILABLE: Network closed for unknown reason
This commit make a small fix to derivate the argument types of the array function
1、 For null type in arguments, cast the null type to boolean type, because null type should not be seen in be.
2、For different types in arguments, cast all arguments type to their compatible type.
Users do not need to set create_table_timeout, it is a ddl command and when encounter a timeout event
users will set a lager timeout and retry.
Stream load 2pc is used by default in flink connector, so we should not disable it by config, the config
item is useless.
* [fix](streamload) report exactly error message when be does not receive heartbeat from fe
Http service is started before hearbeat from fe, so if a streamload comes before heartbeat
from fe, then be report an error like below because master address is not set.
"Couldn't open transport for :0 (Could not resolve host for client socket.)".
1. Missing field and line delimiter
2. When query external table with text(csv) format, we should pass the column position map to BE,
otherwise the column order is wrong.
TODO:
1. For now, if we query csv file with non-exist column, it will return null.
But it should return null or default value of that column.
2. Add regression test after hive docker is ready.
this pr is used to fix the wrong result of array_join function.
before the change, the array_join function will return wrong result.
MySQL [example_db]> select array_join(["", "1", "2"], '');
+--------------------------------------+
| array_join(ARRAY('', '1', '2'), '') |
+--------------------------------------+
| 1_2 |
+--------------------------------------+
3.after the change, the array_join function will return correct result.
MySQL [example_db]> select array_join(["", "1", "2"], '');
+--------------------------------------+
| array_join(ARRAY('', '1', '2'), '') |
+--------------------------------------+
| _1_2 |
+--------------------------------------+
Issue Number: #7570
Fix for SQL with array column:
delete from tbl where c_array is null;
more info please refer to #13360
Co-authored-by: cambyzju <zhuxiaoli01@baidu.com>
We should consider memory which are being flushed from memtable to disk when trying to reduce memory by flushing memtable. Otherwise, we might not release memory space as expected. (e.g. lots of large memtable is in flush, the reduce_mem_usage method picks some small memtables to flush, it can't release enough memory and also can generate lots of small segments, which can cause -238 error)
