Storage

A user's virtual_root picks the backend. Every protocol works the same on both.

`virtual_root`	Backend
`/srv/siftp/alice` (or empty, or relative)	Local directory
`s3://bucket/prefix`	S3-compatible object storage

Local directories

Paths resolve inside the root; .. past it is refused before any filesystem call.
Symlinks are followed only when they resolve inside the root. Links pointing outside are listed and can be deleted or renamed, never read through or written through.
SFTP SYMLINK refuses a target outside the root (invalid_path). An absolute target is a path in the user's own root (/docs is <root>/docs), and a target with .. is stored resolved, so readlink shows /docs for ../docs created in /sub.
Downloads use sendfile, uploads splice (Linux), when the transfer is plain FTP in binary mode and the user has no rate limit.
The root directory is created on first login.

S3

[s3]
endpoint = "https://s3.eu-central-1.amazonaws.com"
region = "eu-central-1"
access_key_id = "AKIA..."
secret_access_key = "${ENV:SIFTP_S3_SECRET}"
path_style = false            # true for MinIO, VersityGW, SeaweedFS, Ceph
spool_dir = "/var/lib/siftp/spool"

[[users]]
name = "alice"
password_hash = "$argon2id$..."
virtual_root = "s3://company-files/users/alice"
permissions = "rwdnml"

Set [server].root = "s3://bucket/base" to put every default user root in S3. Tested against AWS-style SigV4 servers VersityGW and SeaweedFS.

Operation	On S3
File `/a/b.txt`	Object `prefix/a/b.txt`
Directory	A key prefix; `mkdir` writes an empty `dir/` marker
Download	Ranged GETs, 8 MiB at a time
Upload	Staged in `spool_dir`, published when the transfer ends (multipart above `part_size_bytes`)
Failed upload	Nothing is published; the previous object stays
Rename	Copy, then delete (directories up to 10 000 objects)
Append, `REST` resume	The existing object is downloaded into the spool first
Symlinks	Not supported (`invalid_command`)
Times and modes	Server-assigned `Last-Modified`; `MFMT`/`SETSTAT` times are accepted and ignored
Free space	Reported as 1 PiB; quotas still apply

The spool needs room for the largest concurrent uploads. Requests are presigned (SigV4) and sent over HTTPS with rustls.

With [tier].remote set, every file under [server].root is also copied to S3, and the local disk keeps only what is in use. Linux and other Unix hosts; the filesystem needs user extended attributes and sparse files (ext4, XFS, Btrfs).

[s3]
endpoint = "https://s3.us-east-2.amazonaws.com"
region = "us-east-2"
path_style = false
credentials = "instance"          # the EC2 instance role

[tier]
remote = "s3://example-siftp-tier/ftp"
storage_class = "INTELLIGENT_TIERING"
evict_after_days = 30
high_watermark_percent = 80
low_watermark_percent = 70

Event	What happens
A file has not changed for `upload_after_secs`	Uploaded under a random key in the chosen storage class
It changes later	Uploaded again under a new key
Nobody reads or writes it for `evict_after_days`	The local copy becomes a placeholder: same name, size, time, and mode, no data on disk
The disk passes `high_watermark_percent`	Least recently used copies become placeholders until usage is below `low_watermark_percent`
A client reads a placeholder (or appends, resumes, or truncates it to a new size)	The file is downloaded first, checked, and put back in place
A client overwrites a placeholder	The new upload replaces it; nothing is downloaded
Rename, delete, `MFMT`, `SETSTAT` times and modes	Work on placeholders without downloading

Listings, sizes, times, and quotas are the same whether a file is local or a placeholder. The first read of a placeholder waits for the whole file to download. Storage classes are limited to those that serve reads immediately: STANDARD, STANDARD_IA, ONEZONE_IA, INTELLIGENT_TIERING, GLACIER_IR.

Each file's state is an extended attribute (user.siftp.tier). Copy files in or out with siftp (FTP, SFTP) rather than on the host: a placeholder copied with cp is an empty sparse file. A placeholder changed on the host is quarantined (never uploaded, refused on read, counted in tier_quarantined). Setup, bucket policy, and restore: Operations.

Storage

Local directories

S3

Tiered storage