support for gocryptfs, ecryptfs, borg, duplicity, and rsync#374

Open

bgemmill wants to merge 82 commits intoyadayada:masterfrom

bgemmill:master

Commits on Aug 7, 2016

support for ecryptfs; this requires sequential file writing using multiple file descriptors, as well as support for getting and setting xattrs on files.
bgemmill
committed

Commits on Aug 8, 2016

Commits on Aug 10, 2016

Commits on Aug 11, 2016

bugfix: turns out reinterpret casting floats to bytes and back via structs leads to epsilon problems, causing rsync to think that mtime is different when it isn't.
benjamin.gemmill@gmail.com
committed

Commits on Aug 13, 2016

workaround: due to ecryptfs' bug, reporting incorrect file sizes when using xattrs for crypto headers, we have to allow re-writing the first bytes of a file to make ecryptfs happy. once they fix th…
bgemmill
committed

Commits on Aug 15, 2016

Commits on Aug 16, 2016

The corner cases around implementing truncate when we only have a (partial) buffer in the middle of writing is hard enough that we bail on it. We don't care about pre-allocating files since we have…
bgemmill
committed

Commits on Aug 17, 2016

Use memory as a backing for buffers only if the writing size is 1G (configurable) or smaller, disk otherwise.
bgemmill
committed

Commits on Aug 18, 2016

expand the path -> node caching to solve large delays with readdir + many getattr calls for very full directories
bgemmill
committed

Commits on Aug 21, 2016

Commits on Aug 22, 2016

duplicity support: opportunistic reading and file size query if we're writing at the same time but have not yet flushed to amazon
bgemmill
committed

Commits on Sep 11, 2016

Commits on Sep 12, 2016

retry fuse operations if there's a failure on amazon's side and the return code says to retry.
bgemmill
committed

Commits on Sep 13, 2016

Commits on Sep 14, 2016

turns out the 1000 response is thrown rather then returned
bgemmill
committed

Commits on Sep 24, 2016

Merge branch 'master' of https://github.com/yadayada/acd_cli
bgemmill
committed

Commits on Oct 13, 2016

Merge branch 'master' of https://github.com/yadayada/acd_cli
bgemmill
committed

Commits on Nov 6, 2016

Merge branch 'master' of https://github.com/yadayada/acd_cli
bgemmill
committed

Commits on Nov 7, 2016

we can get 1000 error codes on writing too.
bgemmill
committed

Commits on Nov 9, 2016

Commits on Nov 22, 2016

Merge branch 'master' of https://github.com/yadayada/acd_cli
bgemmill
committed

Commits on Dec 2, 2016

Commits on Dec 25, 2016

Merge branch 'master' of https://github.com/yadayada/acd_cli
bgemmill
committed

Commits on Dec 26, 2016

verify that file renames and moves happen since amazon can drop some despite returning success
bgemmill
committed

Commits on Jan 8, 2017

Commits on Jan 12, 2017

Commits on Jan 16, 2017

Merge branch 'master' of https://github.com/yadayada/acd_cli
bgemmill
committed

Commits on Jan 19, 2017

Commits on Jan 21, 2017

Commits on Jan 22, 2017

add elements to the resolve cache on creation
bgemmill
committed

Commits on Jan 24, 2017

The entirety of resolve caching can be removed if we apply gerph's PR. Thanks gerph!
bgemmill
committed

Commits on Jan 25, 2017

turns out it's faster with both
bgemmill
committed

Commits on Jan 29, 2017

cache node ids instead of nodes to fix 0 file size issues
bgemmill
committed

Commits on Feb 4, 2017

cache write buffer length so long uploads won't hold up getattr calls
bgemmill
committed

Commits on Feb 6, 2017

hit sqlite less
bgemmill
committed

Commits on Feb 10, 2017

Commits on Feb 14, 2017

Commits on Feb 15, 2017

store symlinks targets in file contents to ultimately allow for longer targets than amazon's max xattr size
bgemmill
committed

Commits on Feb 19, 2017

cache files' content, starting with small files and symlinks
bgemmill
committed

Commits on Feb 21, 2017

retry on 429 rate throttling errors
bgemmill
committed

Commits on Feb 23, 2017

release file handle lock during file writes to amazon
bgemmill
committed

Commits on Feb 26, 2017

Commits on Feb 27, 2017

turns out rsync needs this for proper change detection in some cases.
bgemmill
committed

Commits on Mar 1, 2017

releasing the fh lock can lead to 409 concurrent modification errors in huge directory trees with rsync
bgemmill
committed

Commits on Mar 3, 2017

sparse file support
bgemmill
committed

Commits on Mar 4, 2017

sparse file support at ends of files too
bgemmill
committed

Commits on Mar 6, 2017

put flush back to try to solve plex issues
bgemmill
committed

Commits on Mar 7, 2017

Commits on Mar 15, 2017

tidy
bgemmill
committed

Commits on Mar 16, 2017

Commits on Mar 21, 2017

prevent xattr cache from falling out of sync on failed acd calls
bgemmill
committed

Commits on Mar 29, 2017

the fuse subdir module sometimes leaves trailing slashes on directory paths
bgemmill
committed

Commits on Apr 16, 2017

Merge branch 'master' of https://github.com/yadayada/acd_cli
bgemmill
committed

Commits on May 13, 2017

Merge branch 'master' of https://github.com/yadayada/acd_cli
bgemmill
committed

Commits on May 26, 2017

Commits on May 27, 2017

Commits on May 29, 2017

Merge branch 'master' of https://github.com/yadayada/acd_cli
bgemmill
committed

Commits on May 30, 2017