97 Commits

Author SHA1 Message Date
j2rong4cn
ddffacf07b
perf: optimize IO read/write usage (#8243)
* perf: optimize IO read/write usage

* .

* Update drivers/139/driver.go

Co-authored-by: MadDogOwner <xiaoran@xrgzs.top>

---------

Co-authored-by: MadDogOwner <xiaoran@xrgzs.top>
2025-04-12 16:55:31 +08:00
KirCute
1335f80362
feat(archive): support multipart archives (#8184 close #8015)
* feat(archive): multipart support & sevenzip tool

* feat(archive): rardecode tool

* feat(archive): support decompress multi-selected

* fix(archive): decompress response filter internal

* feat(archive): support multipart zip

* fix: more applicable AcceptedMultipartExtensions interface
2025-03-27 23:20:44 +08:00
KirCute
9a9aee9ac6
feat(alias): support writing to non-ambiguous paths (#8216)
* feat(alias): support writing to non-ambiguous paths

* feat(alias): support extract concurrency

* fix(alias): extract url no pass query
2025-03-27 23:17:45 +08:00
KirCute_ECT
d5ec998699
feat(task): allow retry canceled (#7852) 2025-01-27 20:18:10 +08:00
KirCute_ECT
11b6a6012f
fix(copy): use Link and Put when the driver does not support copying (#7834)
Some checks failed
auto_lang / auto generate lang.json (1.21, ubuntu-latest) (push) Has been cancelled
beta release / Beta Release Changelog (1.21, ubuntu-latest) (push) Has been cancelled
build / Build (1.21, ubuntu-latest) (push) Has been cancelled
release_docker / Build Binaries for Docker Release (push) Has been cancelled
beta release / Beta Release (md5, !(*musl*|*windows-arm64*|*android*|*freebsd*)) (push) Has been cancelled
beta release / Beta Release (md5-android, android-*) (push) Has been cancelled
beta release / Beta Release (md5-freebsd, freebsd-*) (push) Has been cancelled
beta release / Beta Release (md5-linux-musl, linux-!(arm*)-musl*) (push) Has been cancelled
beta release / Beta Release (md5-linux-musl-arm, linux-arm*-musl*) (push) Has been cancelled
beta release / Beta Release (md5-windows-arm64, windows-arm64) (push) Has been cancelled
beta release / Beta Release Desktop (push) Has been cancelled
release_docker / Release Docker image (, latest, ) (push) Has been cancelled
release_docker / Release Docker image (INSTALL_ARIA2=true, aria2, suffix=-aria2,onlatest=true) (push) Has been cancelled
release_docker / Release Docker image (INSTALL_FFMPEG=true INSTALL_ARIA2=true , aio, suffix=-aio,onlatest=true) (push) Has been cancelled
release_docker / Release Docker image (INSTALL_FFMPEG=true, ffmpeg, suffix=-ffmpeg,onlatest=true) (push) Has been cancelled
Close inactive / close-inactive (push) Has been cancelled
2025-01-18 23:52:02 +08:00
KirCute_ECT
bb40e2e2cd
feat(archive): archive manage (#7817)
* feat(archive): archive management

* fix(ftp-server): remove duplicate ReadAtSeeker realization

* fix(archive): bad seeking of SeekableStream

* fix(archive): split internal and driver extraction api

* feat(archive): patch

* fix(shutdown): clear decompress upload tasks

* chore

* feat(archive): support .iso format

* chore
2025-01-18 23:28:12 +08:00
KirCute_ECT
d7aa1608ac
feat(task): add speed monitor (#7655) 2024-12-25 21:09:54 +08:00
KirCute_ECT
64ceb5afb6
feat: support general users view and cancel own tasks (#7416 close #7398)
* feat: support general users view and cancel own tasks

Add a creator attribute to the upload, copy and offline download
tasks, so that a GENERAL task creator can view and cancel them.

BREAKING CHANGE:

1. A new internal package `task` including the struct `TaskWithCreator`
   which embeds `tache.Base` is created, and the past dependence on
   `tache.Task` will all be transferred to dependence on this package.
2. The API `/admin/task` can now also be accessed via `/task`, and the
   old endpoint is retained to ensure compatibility with legacy
   automation scripts.

Closes #7398

* fix(deps): update github.com/xhofe/tache to v0.1.3
2024-11-01 23:32:26 +08:00
itsHenry
74f8295960
feat: persistant Task (#6925 close #5313) 2024-08-07 12:16:21 +08:00
j2rong4cn
29fe49fb87
fix(alias): Support forced refresh of file list (#6562) 2024-06-16 16:59:10 +08:00
Andy Hsu
026e944cbb feat: add task info to resp of add task api (close #5579) 2023-12-03 14:44:20 +08:00
Andy Hsu
7583c4d734 feat: customize workers and retry of task (close #5493 fix #5274) 2023-11-21 15:51:57 +08:00
Andy Hsu
11a30c5044 feat: refactor task module 2023-11-20 18:01:51 +08:00
Andy Hsu
352a6a741a feat(webdav): support copy directly without task (close #5206) 2023-09-13 15:45:57 +08:00
Sean
a3748af772
feat: misc improvements about upload/copy/hash (#5045)
general: add createTime/updateTime support in webdav and some drivers
general: add hash support in some drivers
general: cross-storage rapid-upload support
general: enhance upload to avoid local temp file if possible
general: replace readseekcloser with File interface to speed upstream operations
feat(aliyun_open): same as above
feat(crypt): add hack for 139cloud

Close #4934 
Close #4819 

baidu_netdisk needs to improve the upload code to support rapid-upload
2023-08-27 21:14:23 +08:00
Andy Hsu
5606c23768 perf(copy): use multi-thread downloader (close #5000) 2023-08-13 15:31:49 +08:00
Sean
7877184bee
feat(baidu_netdisk): add retry to most operations (close #4863 in #4939) 2023-08-07 13:44:28 +08:00
Sean
3c21a9a520
feat: Crypt driver, improve http/webdav handling (#4884)
this PR has several enhancements, fixes, and features:
- [x] Crypt: a transparent encryption driver. Anyone can easily, and safely store encrypted data on the remote storage provider.  Consider your data is safely stored in the safe, and the storage provider can only see the safe, but not your data.
  - [x] Optional: compatible with [Rclone Crypt](https://rclone.org/crypt/). More ways to manipulate the encrypted data.
  - [x] directory and filename encryption
  - [x] server-side encryption mode (server encrypts & decrypts all data, all data flows thru the server)
- [x] obfuscate sensitive information internally
- [x] introduced a server memory-cached multi-thread downloader.
  - [x] Driver: **Quark** enabled this feature, faster load in any single thread scenario. e.g. media player directly playing from the link, now it's faster.
- [x] general improvement on HTTP/WebDAV stream processing & header handling & response handling
  - [x] Driver: **Mega** driver support ranged http header
  - [x] Driver: **Quark** fix bug of not closing HTTP request to Quark server while user end has closed connection to alist

## Crypt, a transparent Encrypt/Decrypt Driver. (Rclone Crypt compatible)

e.g.  
Crypt mount path ->  /vault 
Crypt remote path -> /ali/encrypted
Aliyun mount paht -> /ali

when the user uploads a.jpg to /vault, the data will be encrypted and saved to /ali/encrypted/xxxxx. And when the user wants to access a.jpg,  it's automatically decrypted, and the user can do anything with it.
Since it's Rclone Crypt compatible, users can download /ali/encrypted/xxxxx  and decrypt it with rclone crypt tool. Or the user can mount this folder using rclone, then mount the decrypted folder in Linux...

NB.  Some breaking changes is made to make it follow global standard, e.g. processing the HTTP header properly.

close #4679 
close #4827 

Co-authored-by: Sean He <866155+seanhe26@users.noreply.github.com>
Co-authored-by: Andy Hsu <i@nn.ci>
2023-08-02 14:40:36 +08:00
Andy Hsu
e700ce15e5 fix: missed progress in upload task 2023-07-29 17:09:26 +08:00
Andy Hsu
b2f5757f8d fix(copy): copy from driver that return writer (close #4291) 2023-05-26 21:57:43 +08:00
Noah Hsu
63de65be45 fix: increase timeout for http_client (close #4409) 2023-05-18 23:32:05 +08:00
XYUU
a3446720a2
fix: make TlsInsecureSkipVerify enable for all request (#4386) 2023-05-14 17:05:47 +08:00
Andy Hsu
7e3b13ea2d fix: fs/list interface conversion from copy alias (close #4279) 2023-05-01 15:45:45 +08:00
Andy Hsu
cc43238bd1 fix(alias): disable log completely (#4054) 2023-04-09 15:46:26 +08:00
Andy Hsu
c0a6beecea fix(alias): panic on nil pointer (close #4093) 2023-04-09 14:06:04 +08:00
Andy Hsu
0f8a84f67e perf(alias): disabled log on fs call (close #4054) 2023-04-07 00:02:07 +08:00
Andy Hsu
d9795ff22f feat(alias): support proxy and direct together 2023-03-14 13:46:27 +08:00
NewbieOrange
22843ffc70
fix(fs): copy file if symlink failed (#3368) 2023-02-13 14:41:35 +08:00
NewbieOrange
62dae50d70
feat(fs): create symbolic link instead of copy local files (close #2186 in #3354) 2023-02-12 16:03:11 +08:00
Noah Hsu
99d7105357 fix: move virtual files to end (close #3052) 2023-01-18 10:23:54 +08:00
Noah Hsu
9d9377f65d fix(local): incorrect path of thumbnail (for 6453ae0) 2023-01-16 20:02:30 +08:00
Noah Hsu
6453ae0968 fix(search): empty parent where update (close #2810) 2023-01-16 17:33:24 +08:00
foxxorcat
2f19d4a834
perf(lanzou): optimize the use of list cache (#2956)
* fix:local sort not cache

* perf(lanzou): Optimize the use of list cache
2023-01-08 21:31:35 +08:00
foxxorcat
62a06fa0f9
feat: optimize file operation interface (#2757)
* feat: optimize file operation interface

* chore: fix typo

Co-authored-by: Noah Hsu <i@nn.ci>
2022-12-20 15:02:40 +08:00
foxxorcat
6024e8d832
refactor: split the db package hook and cache to the op package (#2747)
* refactor:separate the setting method from the db package to the op package and add the cache

* refactor:separate the meta method from the db package to the op package

* fix:setting not load database data

* refactor:separate the user method from the db package to the op package

* refactor:remove user JoinPath error

* fix:op package user cache

* refactor:fs package list method

* fix:tile virtual paths (close #2743)

* Revert "refactor:remove user JoinPath error"

This reverts commit 4e20daaf9e700da047000d4fd4900abbe05c3848.

* clean path directly may lead to unknown behavior

* fix: The path of the meta passed in must be prefix of reqPath

* chore: rename all virtualPath to mountPath

* fix: `getStoragesByPath` and `GetStorageVirtualFilesByPath`

is_sub_path:

/a/b isn't subpath of /a/bc

* fix: don't save setting if hook error

Co-authored-by: Noah Hsu <i@nn.ci>
2022-12-18 19:51:20 +08:00
foxxorcat
fb64f00640
refactor: obj name mapping and internal path processing (#2733)
* refactor:Prepare to remove the get interface

* feat:add obj Unwarp interface

* refactor:obj name mapping and program internal path processing

* chore: fix typo

* feat: unwrap get

* fix: no use op.Get to get parent id

* fix: set the path uniformly

Co-authored-by: Noah Hsu <i@nn.ci>
2022-12-17 19:49:05 +08:00
BoYanZh
8c0dfe2f3d
feat: Search enhancement (#2562)
* feat: ignore AList storage on indexing

* fix: remove unused err in `walkFn`

* chore(ci): fix auto_lang trigger and run it

* feat: batch index

* feat: quit index & init index

* feat: set DocType for bleve data

* fix: build index cleanup check origin err
2022-12-05 13:28:39 +08:00
Noah Hsu
4980b71ba3 fix: add hide check to canAccess (close #2532) 2022-11-30 22:01:33 +08:00
Noah Hsu
ddcba93eea
feat: multiple search indexes (#2514)
* refactor: abstract search interface

* wip: ~

* fix cycle import

* objs update hook

* wip: ~

* Delete search/none

* auto update index while cache changed

* db searcher

TODO: bleve init issue

cannot open index, metadata missing

* fix size type

why float64??

* fix typo

* fix nil pointer using

* api adapt ui

* bleve: fix clear & change struct
2022-11-28 13:45:25 +08:00
Noah Hsu
4dcaa24758 fix: cache is modified while sorting (close #2340) 2022-11-15 14:38:23 +08:00
Noah Hsu
8c69260972 fix(webdav): set mime by ext if it's empty 2022-10-09 19:29:55 +08:00
Noah Hsu
bec3a327a7 fix: hide objs if only virtual files 2022-09-17 15:31:30 +08:00
Noah Hsu
d9f0603271 fix: copy folder between two storage (fix #1670) 2022-09-15 17:58:32 +08:00
Noah Hsu
0f2425ce53 feat: add teambition driver 2022-09-02 18:24:14 +08:00
Noah Hsu
d31faabc24 chore: fix typo 2022-08-31 22:08:12 +08:00
Noah Hsu
7ac1d14eeb style: shorten name operations to op 2022-08-31 21:01:15 +08:00
Noah Hsu
68a125491b chore: add refresh arg in list func 2022-08-29 19:15:52 +08:00
Noah Hsu
97d4114e38 fix: check err before check upload 2022-08-29 14:18:43 +08:00
Noah Hsu
af884010d1 feat: local storage image thumbnail 2022-08-11 20:32:17 +08:00
Noah Hsu
b51e664543 chore: go fmt 2022-08-03 14:26:59 +08:00