# Storage

A user's `virtual_root` picks the backend. Every protocol works the
same on both.

| `virtual_root` | Backend |
| --- | --- |
| `/srv/siftp/alice` (or empty, or relative) | Local directory |
| `s3://bucket/prefix` | S3-compatible object storage |

## Local directories

- Paths resolve inside the root; `..` past it is refused before any
  filesystem call.
- Symlinks are followed only when they resolve inside the root. Links
  pointing outside are listed and can be deleted or renamed, never
  read through or written through.
- SFTP `SYMLINK` refuses a target outside the root (`invalid_path`). An
  absolute target is a path in the user's own root (`/docs` is
  `<root>/docs`), and a target with `..` is stored resolved, so `readlink`
  shows `/docs` for `../docs` created in `/sub`.
- Downloads use `sendfile`, uploads `splice` (Linux), when the transfer
  is plain FTP in binary mode and the user has no rate limit.
- The root directory is created on first login.

## S3

```toml
[s3]
endpoint = "https://s3.eu-central-1.amazonaws.com"
region = "eu-central-1"
access_key_id = "AKIA..."
secret_access_key = "${ENV:SIFTP_S3_SECRET}"
path_style = false            # true for MinIO, VersityGW, SeaweedFS, Ceph
spool_dir = "/var/lib/siftp/spool"

[[users]]
name = "alice"
password_hash = "$argon2id$..."
virtual_root = "s3://company-files/users/alice"
permissions = "rwdnml"
```

Set `[server].root = "s3://bucket/base"` to put every default user root
in S3. Tested against AWS-style SigV4 servers VersityGW and SeaweedFS.

| Operation | On S3 |
| --- | --- |
| File `/a/b.txt` | Object `prefix/a/b.txt` |
| Directory | A key prefix; `mkdir` writes an empty `dir/` marker |
| Download | Ranged GETs, 8 MiB at a time |
| Upload | Staged in `spool_dir`, published when the transfer ends (multipart above `part_size_bytes`) |
| Failed upload | Nothing is published; the previous object stays |
| Rename | Copy, then delete (directories up to 10 000 objects) |
| Append, `REST` resume | The existing object is downloaded into the spool first |
| Symlinks | Not supported (`invalid_command`) |
| Times and modes | Server-assigned `Last-Modified`; `MFMT`/`SETSTAT` times are accepted and ignored |
| Free space | Reported as 1 PiB; quotas still apply |

The spool needs room for the largest concurrent uploads. Requests are
presigned (SigV4) and sent over HTTPS with rustls.

## Tiered storage

With `[tier].remote` set, every file under `[server].root` is also
copied to S3, and the local disk keeps only what is in use. Linux and
other Unix hosts; the filesystem needs user extended attributes and
sparse files (ext4, XFS, Btrfs).

```toml
[s3]
endpoint = "https://s3.us-east-2.amazonaws.com"
region = "us-east-2"
path_style = false
credentials = "instance"          # the EC2 instance role

[tier]
remote = "s3://example-siftp-tier/ftp"
storage_class = "INTELLIGENT_TIERING"
evict_after_days = 30
high_watermark_percent = 80
low_watermark_percent = 70
```

| Event | What happens |
| --- | --- |
| A file has not changed for `upload_after_secs` | Uploaded under a random key in the chosen storage class |
| It changes later | Uploaded again under a new key |
| Nobody reads or writes it for `evict_after_days` | The local copy becomes a placeholder: same name, size, time, and mode, no data on disk |
| The disk passes `high_watermark_percent` | Least recently used copies become placeholders until usage is below `low_watermark_percent` |
| A client reads a placeholder (or appends, resumes, or truncates it to a new size) | The file is downloaded first, checked, and put back in place |
| A client overwrites a placeholder | The new upload replaces it; nothing is downloaded |
| Rename, delete, `MFMT`, `SETSTAT` times and modes | Work on placeholders without downloading |

Listings, sizes, times, and quotas are the same whether a file is local
or a placeholder. The first read of a placeholder waits for the whole
file to download. Storage classes are limited to those that serve reads
immediately: `STANDARD`, `STANDARD_IA`, `ONEZONE_IA`,
`INTELLIGENT_TIERING`, `GLACIER_IR`.

Each file's state is an extended attribute (`user.siftp.tier`). Copy
files in or out with siftp (FTP, SFTP) rather than on the host: a
placeholder copied with `cp` is an empty sparse file. A placeholder
changed on the host is quarantined (never uploaded, refused on read,
counted in `tier_quarantined`). Setup, bucket policy, and restore:
[Operations](/docs/operations.html#tiered-storage).