05db6e9b55
[refactor](file-system)(step-2) remove env, file_utils and filesystem_utils ( #18009 )
...
Follow #17586 .
This PR mainly changes:
Remove env/
Remove FileUtils/FilesystemUtils
Some methods are moved to LocalFileSystem
Remove olap/file_cache
Add s3 client cache for s3 file system
In my test, the time of open s3 file can be reduced significantly
Fix cold/hot separation bug for s3 fs.
This is the last PR of #17764 .
After this, all IO operation should be in io/fs.
Except for tests in #17586 , I also tested some case related to fs io:
clone
concurrency query on local/s3/hdfs
load error log create and clean
disk metrics
2023-03-29 09:00:52 +08:00
d8f0ca7108
[Chore](schema change) remove some unused code in schema change ( #17459 )
...
remove some unused code in schema change.
remove some row-based config and code.
2023-03-07 09:18:34 +08:00
0b624d282d
[enhancement](ut) add merge-on-write ut code back ( #16939 )
2023-02-22 16:29:15 +08:00
2bee26b05a
[fix](merge-on-write) fix that the query result has duplicate keys ( #16336 )
...
* [fix](merge-on-write) fix that the query result has duplicate keys
* add ut
2023-02-06 17:09:53 +08:00
00a598a839
[feature](cooldown) Decouple storage policy and resource ( #15873 )
2023-01-31 14:13:47 +08:00
365c3eec16
[enhancement](compaction) vertical compaction support unique-key mow ( #15353 )
2023-01-02 22:53:04 +08:00
ad68764977
[enhancement](tablet) Unify redundant create_rowset_writer methods ( #15519 )
...
* Remove redundant create_rowset_writer methods
* Set resource id when setting FS in rowset meta
* fix
* fix ut
2022-12-30 22:57:12 +08:00
821c12a456
[chore](BE) remove all useless segment group related code #15193
...
The segment group is useless in current codebase, remove all the related code inside Doris. As for the related protobuf code, use reserved flag to prevent any future user from using that field.
2022-12-20 17:11:47 +08:00
f3aea7f0f0
[Enhancement](status) Unify error code and enable customed err msg for BE internal errors ( #14744 )
2022-12-11 23:33:18 +08:00
94a6ffb906
[feature](compaction) support vertical_compaction & ordered_data_compaction ( #14524 )
2022-12-01 22:15:41 +08:00
eab8876abc
[Feature](remote) Using heavy schema change if the table is not enable light weight schema change ( #13487 )
2022-10-28 15:48:22 +08:00
ccff3f5711
[bugfix](light weight schema change) support delete condition in schema change ( #11869 )
...
* [bugfix](light weight schema change) support delete condition in schema change
Co-authored-by: yiguolei <yiguolei@gmail.com >
2022-08-26 11:45:55 +08:00
d505d1a5ae
[Vectorized](compaction) filter delete data in base compaction ( #11721 )
...
* [Vectorized](compaction) filter delete data in base compaction
Co-authored-by: lihaopeng <lihaopeng@baidu.com >
2022-08-18 14:22:59 +08:00
ea57bf6370
[refactor](delete predicate) Unify delete to segmentiterator ( #11650 )
...
* remove seek columns and unify delete columns in rowset reader
Co-authored-by: yiguolei <yiguolei@gmail.com >
2022-08-11 15:12:43 +08:00
aaaf6915e4
[feature-wip](unique-key-merge-on-write) fix rowid conversion ut that may create a directory under an incorrect path ( #11628 )
2022-08-10 08:17:47 +08:00
1e6a3610a7
[feature-wip](unique-key-merge-on-write) optimize rowid conversion and add ut ( #11541 )
2022-08-08 10:41:44 +08:00
d4fb27125a
[feature-wip](unique-key-merge-on-write) row id conversion for compaction ( #11149 )
2022-07-27 16:32:13 +08:00
