Commit Graph

17 Commits

Author SHA1 Message Date
30c4f25cb3 [fix](multi-catalog) verify the precision of datetime types for each data source (#19544)
Fix threes bugs of timestampv2 precision:
1. Hive catalog doesn't set the precision of timestampv2, and can't get the precision from hive metastore, so set the largest precision for timestampv2;
2. Jdbc catalog use datetimev1 to parse timestamp, and convert to timestampv2, so the precision is lost.
3. TVF doesn't use the precision from meta data of file format.
2023-05-17 20:50:15 +08:00
54dbb4af67 [vectorzied](jdbc) refactor jdbc table read array type (#18187)
jdbc read array type get result from Doris is string, PG is java.sql.array, CK is java.lang.object
it's difficult to maintain and read the code,
so change all database's array result to string, then add a cast function from string to doris array type
2023-04-04 11:57:04 +08:00
211cc66d02 [fix](multi-catalog) fix image loading failture when create catalog with resource (#15692)
Bug fix
fix image loading failture when create catalog with resource
When creating jdbc catalog with resource, the metadata image will failed to be loaded.
Because when loading jdbc catalog image, it will try to get resource from ResourceMgr,
but ResourceMgr has not been loaded, so NPE will be thrown.

This PR fix this bug, and refactor some logic about catalog and resource.

When loading jdbc catalog image, it will not get resource from ResourceMgr.
And now user can create catalog with resource and properties, like:

create catalog jdbc_catalog with resource jdbc_resource
properites("user" = "user1");
The properties in "properties" clause will overwrite the properties in "jdbc_resource".

force adding tinyInt1isBit=false to jdbc url
The default value of tinyInt1isBit is true, and it will cause tinyint in mysql to be bit type.
force adding tinyInt1isBit=false to jdbc url so that the tinyint in mysql will be tinyint in Doris.

Avoid calculate checksum of jdbc driver jar multiple times
Refactor
Refactor the notification logic when updating properties in resource.
When updating properties in resource, it will notify the corresponding catalog to update its own properties.
This PR change this logic. After updating properties in resource, it will only uninitialize the catalog's internal
objects such "jdbc client" or "hms client". And this objects will be re-initialized lazily.

And all properties will be got from Resource at runtime, so that it will always get the latest properties

Regression test cases
Because we add tinyInt1isBit=false to jdbc url, some of cases need to be changed.
2023-01-09 09:56:26 +08:00
8b6e4e74e7 [improvement](jdbc) add default jdbc driver's dir (#15346)
Add a new config "jdbc_drivers_dir" for both FE and BE.
User can put jdbc drivers' jar file in this dir, and only specify file name in "driver_url" properties
when creating jdbc resource.
And Doris will find jar files in this dir.

Also modify the logic so that when the jdbc resource is modified, the corresponding jdbc table
will get the latest properties.
2022-12-26 11:51:12 +08:00
ae6a007c4e [test](jdbc)add new extremum case (#14692) 2022-12-02 11:28:11 +08:00
93b940bc92 [test](jdbc)add new case for mysql jdbc table (#14581) 2022-11-27 13:39:59 +08:00
4c60186e87 [test](jdbc)add new case for pg jdbc table (#14582) 2022-11-26 13:02:05 +08:00
608cb6c4ad [test](jdbc)add new case for mysql external table (#14530) 2022-11-24 16:36:44 +08:00
b4d8ae5204 [test](jdbc)add new pg case from other source (#14445) 2022-11-24 16:35:59 +08:00
6fcffd041c [test](jdbc)add new mysql jdbc case from other source (#14495) 2022-11-23 16:23:42 +08:00
1fe9bced25 [test](jdbc)add more mysql jdbc test case (#14475) 2022-11-22 21:14:10 +08:00
e3d764aac5 [test](jdbc) add new jdbc case in other source (#14443) 2022-11-21 21:33:06 +08:00
3f29e3bff6 [bug](test) fix regression test of jdbc postgresql table core (#14417) 2022-11-20 23:03:14 +08:00
02372ca2ea [test](jdbc external table) add new jdbc mysql external table (#14323) 2022-11-19 09:46:48 +08:00
082028b2a2 [test](jdbc postgresql case)add jdbc test case for postgresql (#14162) 2022-11-12 20:43:13 +08:00
78fa167b0a [test](jdbc external table) add jdbc regression test case (#14086) 2022-11-12 20:42:57 +08:00
847b80ebfa [test](jdbc) add jdbc and hive regression test (#13143)
1. Modify default behavior of `build.sh`
    The `BUILD_JAVA_UDF` is default ON, so that jvm is needed for compilation and runtime.

2. Add docker-compose for MySQL 5.7, PostgreSQL 14 and Hive 2
   See `docker/thirdparties/docker-compose`.

3. Add some regression test cases for jdbc query on MySQL, PG and Hive Catalog
   The default is `false`, if set to true, you need first start docker for MySQL/PG/Hive.

4. Support `if not exists` and `if exists` for create/drop resource and create/drop encryptkey
2022-10-21 15:29:27 +08:00