# siftp 0.2.1 — full documentation
Generated by website/build.py from the Markdown sources. Each section names its source URL.

---

<!-- Source: https://siftp.org/ (Markdown: https://siftp.org/index.md) -->

# siftp

An FTP, FTPS, and SFTP server in Rust. One daemon, one config file,
the same users and permissions on every protocol.

```sh
siftpd hash-password 'a strong password'      # argon2id hash for the config
siftpd --check-config --config siftp.toml     # validate; errors are JSON
siftpd --config siftp.toml                    # serve
curl -T report.pdf ftp://alice:'a strong password'@127.0.0.1:2121/
```

[Quickstart](/docs/quickstart.html) · [Download](/download.html) ·
[Specification](/spec.html)

Built by [SiteVue](https://sitevue.ai).

## Install

Debian 12 or later and Ubuntu 22.04 or later, amd64 and arm64
(including AWS Graviton):

```sh
sudo curl -fsSLo /etc/apt/keyrings/siftp.gpg https://apt.siftp.org/siftp-archive-keyring.gpg
echo "deb [signed-by=/etc/apt/keyrings/siftp.gpg] https://apt.siftp.org stable main" | sudo tee /etc/apt/sources.list.d/siftp.list
sudo apt update && sudo apt install siftpd
```

Key fingerprint, release files, and other platforms:
[Download](/download.html).

## What it does

| Area | Details |
| --- | --- |
| Protocols | FTP (RFC 959, 2428, 3659, 7151), FTPS explicit and implicit (TLS 1.2/1.3), SSH/SFTP v3 with OpenSSH extensions, read-only WebDAV |
| Storage | Local directories with a symlink-proof jail, or S3-compatible object storage, behind the same interface; optional tiering keeps every file in S3 and only what is in use on local disk |
| Access | argon2id passwords, SSH keys with expiry and revocation, groups, per-directory permissions, quotas, rate limits, login bans, LDAP |
| Data path | `sendfile` and `splice` on Linux, optional io_uring, a bounded buffer pool for backpressure, assembly CRC32C and scanning kernels |
| Operations | Durable user store, SIGHUP reload, graceful drain, `/healthz` and `/readyz`, Prometheus metrics, hash-chained audit log, systemd unit, container image, .deb |
| Automation | REST API with OpenAPI 3.1, MCP tools, a capability document, and ten stable error codes shared by every protocol |

## Measured

Against vsftpd 3.0.5, ProFTPD 1.3.8c, and OpenSSH 10.0p2 on one
bare-metal Linux host (loopback, tmpfs, median of 5 runs):

| | siftp | vsftpd | ProFTPD | OpenSSH |
| --- | ---: | ---: | ---: | ---: |
| FTPS download, MiB/s | 2357 | 1040 | 1391 | – |
| FTPS upload, MiB/s | 1752 | 683 | 1520 | – |
| SFTP download, MiB/s | 954 | – | – | 763 |
| SFTP upload, MiB/s | 900 | – | – | 811 |
| Memory with 200 FTP sessions, MiB | 17.1 | 65.5 | 174.0 | – |
| First FTP login, 32 clients, logins/s | 142 | 393 | 106 | – |
| Repeat FTP login, 32 clients, logins/s | 97173 | 393 | 106 | – |

Plaintext FTP is the same for all three FTP servers: each uses
`sendfile`, and the client or the host is the limit. A first login in
siftp runs argon2id, which is deliberately expensive; a repeat login by
the same user is served from a keyed-digest cache without recomputing
the hash.

Method, full table, and how to reproduce it: [Performance](/docs/performance.html).

## Errors are data

Every failure carries one of ten stable codes. The admin API, MCP, and
WebDAV return the whole object: code, message, hint, and a link to the
docs. FTP replies and SFTP status messages start with the code,
followed by the message.

```text
550 permission_denied: permission denied: write is not allowed for this user
```

```json
{"code":"permission_denied","message":"permission denied: write is not allowed for this user","hint":"grant the 'w' permission bit or choose a different operation","docs_url":"https://siftp.org/docs/errors#permission_denied"}
```

Ten codes, append-only: [Error codes](/docs/errors.html).

## For agents

A running server describes itself at `GET /.well-known/siftp.json` on
its admin port. Start there, then read [`/llms.txt`](/llms.txt) and the
OpenAPI document it points to. [Agents and MCP](/docs/llm.html) has the
full sequence.

## License

MIT or Apache-2.0, at your option.

---

<!-- Source: https://siftp.org/docs/quickstart.html (Markdown: https://siftp.org/docs/quickstart.md) -->

# Quickstart

From nothing to FTP, FTPS, and SFTP transfers on loopback ports that
need no privileges. Production installs (package, container, systemd)
are in [Operations](/docs/operations.html).

## 1. Get the binary

Download a release for Linux x86_64 or aarch64 from
[Download](/download.html), or build it:

```sh
cargo build --release -p siftpd     # Rust 1.89+; a C compiler for ring and the asm kernels
./target/release/siftpd --version
```

## 2. Write a config

Hash a password, then put it in `siftp.toml`:

```sh
./target/release/siftpd hash-password 'choose a strong password'
# $argon2id$v=19$m=65536,t=3,p=4$...
```

```toml
[server]
root = "/tmp/siftp-demo/srv"
user_store = "/tmp/siftp-demo/users.toml"

[ftp]
bind = "127.0.0.1:2121"
passive_port_range = [40000, 40100]

[ftps]
# FTPS stays off until these files exist (step 4).
cert_file = "/tmp/siftp-demo/tls.crt"
key_file = "/tmp/siftp-demo/tls.key"
implicit_bind = "127.0.0.1:2990"

[sftp]
bind = "127.0.0.1:2222"
host_key_file = "/tmp/siftp-demo/ssh_host_ed25519_key"

[admin]
bind = "127.0.0.1:8080"
token = "${ENV:SIFTP_ADMIN_TOKEN}"

[logging]
audit_file = "/tmp/siftp-demo/audit.jsonl"

[[users]]
name = "alice"
password_hash = "$argon2id$v=19$m=65536,t=3,p=4$...paste the hash..."
permissions = "rwdnml"
```

`alice`'s root defaults to `/tmp/siftp-demo/srv/alice` and is created on
first login. Check the file; problems come back as JSON naming the
field:

```sh
export SIFTP_ADMIN_TOKEN=$(openssl rand -hex 32)
./target/release/siftpd --check-config --config siftp.toml
```

## 3. Run and transfer

```sh
./target/release/siftpd --config siftp.toml
```

In another terminal:

```sh
echo hello > hello.txt
curl -T hello.txt ftp://alice:'choose a strong password'@127.0.0.1:2121/
curl ftp://alice:'choose a strong password'@127.0.0.1:2121/hello.txt
sftp -P 2222 alice@127.0.0.1                                  # interactive
```

For SFTP in a script, pin the host key instead of trusting the first
connection. The fingerprint comes from the key file:

```sh
fp=$(ssh-keygen -lf /tmp/siftp-demo/ssh_host_ed25519_key | awk '{print $2}' | cut -d: -f2)
curl --hostpubsha256 "$fp" -u alice:'choose a strong password' \
  sftp://127.0.0.1:2222/hello.txt
```

## 4. Add FTPS

Put a PEM certificate and key at the paths in `[ftps]` and restart. For
a local test, a self-signed pair:

```sh
openssl req -x509 -newkey rsa:2048 -nodes -days 30 -subj /CN=localhost \
  -keyout /tmp/siftp-demo/tls.key -out /tmp/siftp-demo/tls.crt
```

Explicit FTPS uses the FTP port (`curl --ssl-reqd ftp://...`), implicit
FTPS the `implicit_bind` port (`curl ftps://...`). Set
`require_tls = true` under `[ftp]` to refuse plaintext logins. A new
certificate is picked up on `kill -HUP` without a restart.

## 5. Manage users through the API

```sh
curl -H "Authorization: Bearer $SIFTP_ADMIN_TOKEN" \
  -H 'Content-Type: application/json' \
  -d '{"name":"bob","password":"bob-password","permissions":"rwl"}' \
  http://127.0.0.1:8080/api/v1/users
curl -H "Authorization: Bearer $SIFTP_ADMIN_TOKEN" http://127.0.0.1:8080/api/v1/users
```

Without `permissions`, a user gets the union of their groups'
permissions, or `rl` (read and list) if they have no groups. API users
are written to `user_store` and survive restarts. Users in the config
file stay read-only through the API.

## Next

- [Configuration](/docs/configuration.html): every key and override.
- [Users and access](/docs/identity.html): groups, directory rules,
  keys, LDAP.
- [Security](/docs/security.html) before exposing a listener.

---

<!-- Source: https://siftp.org/docs/configuration.html (Markdown: https://siftp.org/docs/configuration.md) -->

# Configuration

One TOML file. Every key and its default is under
[Reference](#reference). The package installs a commented example as
`/usr/share/doc/siftpd/siftp.example.toml`.

## Loading

| Step | Rule |
| --- | --- |
| File | `--config <path>`, else `./siftp.toml`, else `configs/siftp.toml` |
| Unknown keys | An error, never ignored |
| `${ENV:NAME}` | Replaced inside any string value (tokens, LDAP and S3 secrets, hashes) |
| `SIFTP_<SECTION>_<KEY>` | Overrides the file, e.g. `SIFTP_FTP_BIND=0.0.0.0:2121` |
| `RUST_LOG` | Overrides `[logging].level` |
| Validation | `siftpd --check-config` prints a JSON summary with warnings, or one JSON error naming the field |

Reload with `kill -HUP` or `POST /api/v1/reload`: config, users, the
revoked-key list, and the FTPS certificate are re-read and the audit
file is reopened. Live sessions keep their permissions. Changing a bind
address needs a restart.

## Reference

Every key, with its default. A section you leave out takes all its
defaults. `configs/siftp.example.toml` in the source tree is the same
reference as a commented file.

### `[server]`

| Key | Default | Meaning |
| --- | --- | --- |
| `root` | `/srv/siftp` | Parent of default user roots (`<root>/<name>`). `s3://bucket/prefix` puts them in S3 |
| `max_connections` | `512` | Open connections across all protocols; more are refused with `rate_limited` |
| `max_connections_per_ip` | `32` | The same, per client address |
| `idle_timeout_secs` | `300` | Idle control or data connections are closed after this long |
| `max_failed_logins` | `5` | Failed logins from one address within `ban_window_secs` that ban it; `0` turns bans off |
| `ban_window_secs` | `600` | Window for counting failures |
| `ban_duration_secs` | `900` | How long a ban lasts |
| `shutdown_grace_secs` | `30` | How long SIGTERM waits for transfers before closing them |
| `user_store` | `/var/lib/siftp/users.toml` | Where API- and MCP-created users are kept. Empty keeps them in memory only |
| `revoked_keys_file` | empty | SSH keys refused for every user: OpenSSH lines or `SHA256:` fingerprints, one per line |
| `password_max_age_days` | `0` | Passwords older than this stop working; `0` means never |
| `login_cache_ttl_secs` | `300` | How long a password that argon2id accepted is remembered, so the same user logging in again skips the hash. The server keeps a keyed digest under a key that exists only in process memory, never the password. Wrong passwords always pay the full hash. `0` turns this off |
| `max_concurrent_password_checks` | `0` | argon2id checks running at once; others wait. Each uses the hash's memory (64 MiB by default) while it runs. `0` means one per CPU |

### `[ftp]`

| Key | Default | Meaning |
| --- | --- | --- |
| `bind` | `0.0.0.0:21` | FTP listener; explicit FTPS (`AUTH TLS`) uses it too |
| `passive_port_range` | `[40000, 40100]` | Ports for passive data connections. Environment form: `40000-40100` |
| `passive_external_ip` | empty | Address sent in `PASV` replies behind NAT; empty sends the local address |
| `passive_promiscuous` | `false` | Accept data connections from any address. Off: only from the control connection's address |
| `allow_anonymous` | `false` | Allow `anonymous` and `ftp` logins, read-only (`rl`), in `anonymous_root` |
| `anonymous_root` | `/srv/siftp/anonymous` | Root for anonymous logins |
| `greeting` | `siftp ready` | Text of the `220` greeting |
| `require_tls` | `false` | Refuse login and data transfers until `AUTH TLS` (`tls_required`) |
| `virtual_hosts` | `[]` | Names accepted by `HOST`; empty accepts any name |

### `[ftps]`

| Key | Default | Meaning |
| --- | --- | --- |
| `cert_file` | `/etc/siftp/tls.crt` | PEM certificate chain. If it or the key is missing, FTPS is off: `AUTH TLS` is refused, the implicit listener does not start, and `--check-config` warns |
| `key_file` | `/etc/siftp/tls.key` | PEM private key |
| `implicit_bind` | `0.0.0.0:990` | Implicit FTPS listener |
| `min_tls_version` | `1.2` | `1.2` or `1.3` |
| `require_session_reuse` | `false` | Refuse TLS data connections that do not resume the control connection's TLS session |

The certificate and key are re-read on reload.

### `[sftp]`

| Key | Default | Meaning |
| --- | --- | --- |
| `bind` | `0.0.0.0:22` | SSH listener |
| `host_key_file` | `/etc/siftp/ssh_host_ed25519_key` | OpenSSH-format private key. Missing: an Ed25519 key is generated (mode 0600). ECDSA works; RSA is refused |
| `ciphers` | empty | SSH ciphers to offer, in order: any of `aes256-gcm@openssh.com`, `aes128-gcm@openssh.com`, `chacha20-poly1305@openssh.com`. Empty: the two AES-GCM ciphers when the CPU has AES instructions, otherwise `chacha20-poly1305@openssh.com` first and then AES-GCM. Read at startup |

### `[admin]`

| Key | Default | Meaning |
| --- | --- | --- |
| `bind` | `127.0.0.1:8080` | Admin HTTP: the API, `/metrics`, `/mcp`, health checks, capability document |
| `token` | empty | Bearer token. Empty refuses every request except the public bootstrap routes |

### `[mcp]`

| Key | Default | Meaning |
| --- | --- | --- |
| `transport` | `http` | `http` serves MCP at `POST /mcp` on the admin listener. `stdio` turns that off; `siftpd mcp-stdio` works either way |

### `[logging]`

| Key | Default | Meaning |
| --- | --- | --- |
| `level` | `info` | `error`, `warn`, `info`, `debug`, or `trace`. `RUST_LOG` overrides it |
| `format` | `text` | `text` or `json`, written to stderr |
| `audit_file` | `/var/log/siftp/audit.jsonl` | Hash-chained audit log ([Operations](/docs/operations.html#audit-log)) |

### `[performance]`

| Key | Default | Meaning |
| --- | --- | --- |
| `sendfile` | `true` | Linux downloads without a user-space copy |
| `splice` | `true` | Linux uploads without a user-space copy |
| `io_uring` | `false` | io_uring data path; needs a build with the `io-uring` feature |
| `buffer_pool_bytes` | `67108864` | Memory for all pooled (TLS, S3, non-Linux) transfers together |
| `transfer_chunk_bytes` | `262144` | Size of one pooled chunk |

### `[s3]`

| Key | Default | Meaning |
| --- | --- | --- |
| `endpoint` | empty | S3 endpoint URL. Empty disables `s3://` roots |
| `region` | `us-east-1` | Signing region |
| `access_key_id` | empty | Access key |
| `secret_access_key` | empty | Secret key; use `${ENV:NAME}` |
| `path_style` | `true` | `endpoint/bucket/key` URLs; most self-hosted S3 needs this |
| `spool_dir` | `/var/lib/siftp/spool` | Uploads are staged here until the transfer completes. Created when an `s3://` root is first used; never touched otherwise |
| `part_size_bytes` | `16777216` | Multipart upload part size |
| `timeout_secs` | `30` | Per-request timeout |
| `credentials` | `static` | `static`: the keys above (unsigned requests when empty). `env`: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN`. `instance`: the EC2 instance role through IMDSv2, refreshed before it expires |

Details: [Storage](/docs/storage.html).

### `[tier]`

Tiered storage for local roots: every file under `[server].root` is
copied to object storage, and local copies that are not being used are
replaced by placeholders that are fetched back on the next read.

| Key | Default | Meaning |
| --- | --- | --- |
| `remote` | empty | `s3://bucket/prefix` for the copies, using the `[s3]` connection settings. Empty disables tiering. Read at startup |
| `storage_class` | `INTELLIGENT_TIERING` | S3 class of the copies: `STANDARD`, `STANDARD_IA`, `ONEZONE_IA`, `INTELLIGENT_TIERING`, or `GLACIER_IR` (classes that serve reads immediately) |
| `upload_after_secs` | `60` | Upload a file once it has not changed for this long |
| `evict_after_days` | `30` | Replace the local copy of a file not read or written for this many days |
| `min_evict_bytes` | `1048576` | Smaller files always stay local |
| `high_watermark_percent` | `80` | Above this disk usage, evict least recently used files first ... |
| `low_watermark_percent` | `70` | ... until usage is below this |
| `scan_interval_secs` | `300` | Time between scans of `[server].root` (minimum 10) |

The policy keys apply on reload. Details: [Storage](/docs/storage.html#tiered-storage).

### `[ldap]`

| Key | Default | Meaning |
| --- | --- | --- |
| `url` | empty | `ldap://` or `ldaps://` URL. Empty disables LDAP |
| `starttls` | `false` | Upgrade an `ldap://` connection with StartTLS |
| `ca_file` | empty | PEM CA bundle for the LDAP server's certificate |
| `bind_dn` | empty | Service account for search-then-bind |
| `bind_password` | empty | Its password; use `${ENV:NAME}` |
| `base_dn` | empty | Where to search for users |
| `user_filter` | `(uid={user})` | Search filter; `{user}` is the login name, escaped |
| `user_dn_template` | empty | Direct bind instead of search, e.g. `uid={user},ou=people,dc=example,dc=org` |
| `group_attribute` | `memberOf` | Attribute holding the user's group DNs |
| `group_map` | `{}` | Table of group DN to `[[groups]]` name |
| `default_permissions` | `rl` | Permissions for users in no mapped group |
| `require_group` | `false` | Refuse users in no mapped group |
| `virtual_root_template` | empty | Root for LDAP users with `{user}`; empty: `<server.root>/<user>` |
| `timeout_secs` | `5` | Connect and operation timeout |

Details: [Users and access](/docs/identity.html#ldap).

### `[webdav]`

| Key | Default | Meaning |
| --- | --- | --- |
| `bind` | empty | Read-only WebDAV gateway listener. Empty disables it |
| `tls` | `false` | Serve HTTPS with the `[ftps]` certificate |

### `[[groups]]`

| Key | Meaning |
| --- | --- |
| `name` | Group name, referenced from users' `groups` and `[ldap.group_map]` |
| `permissions` | Bits `r w d n m l` |
| `directories` | `[[groups.directories]]` entries with `path` and `permissions` |

## Users

```toml
[[users]]
name = "backup"
ssh_public_keys = ["ssh-ed25519 AAAAC3... backup@host"]
virtual_root = "backup"          # relative: under [server].root
permissions = "rwl"
quota_bytes = 107374182400
rate_limit_bps = 50000000
max_sessions = 4
```

| Key | Meaning |
| --- | --- |
| `name` | 1-255 characters, not `.` or `..`, no whitespace, `/`, `\`, `:` |
| `password_hash` | argon2id PHC string from `siftpd hash-password`; plaintext and weak parameters are rejected |
| `password_set_unix` | When the password was set; used by `password_max_age_days` |
| `ssh_public_keys` | OpenSSH lines, optional `expiry-time="YYYYMMDD"` prefix |
| `virtual_root` | Empty: `<root>/<name>`. Relative: under `[server].root`. Absolute path or `s3://bucket/prefix` as written |
| `permissions` | Bits `r w d n m l`; omitted: union of the user's groups, else `rl` |
| `groups`, `directories` | See [Users and access](/docs/identity.html) |
| `hosts` | FTP `HOST` names this user may log in under |
| `max_sessions`, `quota_bytes`, `rate_limit_bps` | 0 means unlimited |

Users created with the admin API or MCP go to `[server].user_store`
(default `/var/lib/siftp/users.toml`) and survive restarts. Users in the
config file are read-only through the API; a store user with the same
name is ignored and reported by `--check-config`.

## Environment overrides

Every key in the tables above, except `group_map` and the
`[[groups]]` and `[[users]]` arrays, has a variable named
`SIFTP_<SECTION>_<KEY>` in upper case:

| Variable | Example |
| --- | --- |
| `SIFTP_ADMIN_TOKEN` | a long random string |
| `SIFTP_FTP_BIND`, `SIFTP_SFTP_BIND`, `SIFTP_ADMIN_BIND` | `0.0.0.0:2121` |
| `SIFTP_FTP_PASSIVE_PORT_RANGE` | `40000-40100` |
| `SIFTP_FTP_PASSIVE_EXTERNAL_IP` | `203.0.113.10` |
| `SIFTP_FTP_VIRTUAL_HOSTS` | `files.example.com,backup.example.com` |
| `SIFTP_LOGGING_FORMAT` | `json` |
| `SIFTP_PERFORMANCE_IO_URING` | `true` |
| `SIFTP_S3_SECRET_ACCESS_KEY`, `SIFTP_LDAP_BIND_PASSWORD` | secrets |

Booleans accept `true`, `false`, `1`, `0`. Lists take comma-separated
values.

---

<!-- Source: https://siftp.org/docs/protocols.html (Markdown: https://siftp.org/docs/protocols.md) -->

# Protocols

All protocols share users, roots, permissions, quotas, rate limits,
bans, and the audit log. Paths are virtual: `/` is the user's root, and
`..` past it is refused with `invalid_path` everywhere.

## FTP and FTPS

| Group | Commands |
| --- | --- |
| Session | `USER` `PASS` `ACCT` `QUIT` `SYST` `NOOP` `HELP` `FEAT` `OPTS` `SITE HELP` `HOST` |
| Navigation | `PWD` `CWD` `CDUP` (and `X` variants) |
| Transfer setup | `TYPE A/I` `MODE S` `STRU F` `PASV` `EPSV` `EPSV ALL` `PORT` `EPRT` `REST` `ABOR` |
| Transfers | `RETR` `STOR` `STOU` `APPE` `LIST` `NLST` `MLSD` |
| Files | `DELE` `MKD` `RMD` `RNFR` `RNTO` `SIZE` `MDTM` `MFMT` `MLST` |
| TLS | `AUTH TLS` `PBSZ 0` `PROT P/C` (`CCC` refused) |

- Replies carry the error code first: `550 permission_denied: ...`.
- `MLSD`/`MLST` facts: `type`, `size`, `modify`, `perm`, `unique`;
  `OPTS MLST type;size;` selects a subset. `perm` reflects each entry's
  own directory rules.
- `HOST name` (RFC 7151) must come before `USER`. With
  `[ftp].virtual_hosts` set, other names get `504`; a user with `hosts`
  logs in only under those names.
- `MFMT YYYYMMDDHHMMSS path` sets the modification time (UTC, needs `w`).
- `RNTO` replaces an existing target. `TYPE A` translates line endings;
  `TYPE I` is byte-exact.
- Passive data connections must come from the client's address
  (`passive_promiscuous = true` turns this off for exotic NATs). Active
  mode connects only to the client's own address, port 1024 or higher.
- Behind NAT, set `[ftp].passive_external_ip` and open
  `passive_port_range`.
- FTPS data connections resume the control connection's TLS session;
  `[ftps].require_session_reuse = true` refuses those that do not.
  The data channel never issues TLS tickets (upload-only clients would
  leave them unread and reset the connection).

## SFTP

SFTP version 3 over SSH-2, OpenSSH wire numbering.

| Item | Value |
| --- | --- |
| Requests | `OPEN` `CLOSE` `READ` `WRITE` `STAT` `LSTAT` `FSTAT` `SETSTAT` `FSETSTAT` `OPENDIR` `READDIR` `REMOVE` `MKDIR` `RMDIR` `RENAME` `READLINK` `SYMLINK` `REALPATH` |
| Extensions | `posix-rename` `statvfs` `fstatvfs` `expand-path` `limits` `fsync` (all `@openssh.com`) |
| Key exchange | `mlkem768x25519-sha256` (post-quantum hybrid), `curve25519-sha256`, `ecdh-sha2-nistp256`; strict KEX |
| Ciphers | `aes256-gcm@openssh.com`, `aes128-gcm@openssh.com`; `chacha20-poly1305@openssh.com` first only on CPUs without AES instructions. `[sftp].ciphers` sets the list |
| Host keys | Ed25519 (generated) or ECDSA; RSA host keys are refused |
| Client auth | Password, or public key: Ed25519, ECDSA, RSA (SHA-2) |
| Limits | 256 KiB packets, 255 KiB reads/writes, 256 open handles |

- Plain `RENAME` will not overwrite (draft v3); `posix-rename` does.
  OpenSSH's `sftp` uses `posix-rename` when offered.
- `df` in `sftp` shows the quota as the disk size when one is set.
- `SETSTAT` applies size, time, and permission bits; setuid, setgid, and
  sticky bits are stripped.
- `scp` works in its default SFTP mode. Shell, exec, and forwarding are
  refused.

## WebDAV/HTTP (read-only)

Set `[webdav].bind` (and `tls = true` for HTTPS with the FTPS
certificate). Clients: Finder, Windows Explorer, `rclone`, `cadaver`,
browsers, `curl`.

| Method | Behaviour |
| --- | --- |
| `OPTIONS` | `DAV: 1`, allowed methods |
| `PROPFIND` | Depth 0 or 1 (infinity is treated as 1); needs `l` |
| `GET`, `HEAD` | Files with single `Range` (needs `r`); directories as HTML, or JSON with `Accept: application/json` (needs `l`) |
| Anything else | `403 permission_denied` |

```sh
curl -u alice:password http://127.0.0.1:8081/docs/report.pdf -o report.pdf
curl -u alice:password -X PROPFIND -H 'Depth: 1' http://127.0.0.1:8081/docs/
```

Authentication is HTTP Basic against the same users (bans and LDAP
apply); verified credentials are cached for 60 seconds.

---

<!-- Source: https://siftp.org/docs/storage.html (Markdown: https://siftp.org/docs/storage.md) -->

# Storage

A user's `virtual_root` picks the backend. Every protocol works the
same on both.

| `virtual_root` | Backend |
| --- | --- |
| `/srv/siftp/alice` (or empty, or relative) | Local directory |
| `s3://bucket/prefix` | S3-compatible object storage |

## Local directories

- Paths resolve inside the root; `..` past it is refused before any
  filesystem call.
- Symlinks are followed only when they resolve inside the root. Links
  pointing outside are listed and can be deleted or renamed, never
  read through or written through.
- SFTP `SYMLINK` refuses a target outside the root (`invalid_path`). An
  absolute target is a path in the user's own root (`/docs` is
  `<root>/docs`), and a target with `..` is stored resolved, so `readlink`
  shows `/docs` for `../docs` created in `/sub`.
- Downloads use `sendfile`, uploads `splice` (Linux), when the transfer
  is plain FTP in binary mode and the user has no rate limit.
- The root directory is created on first login.

## S3

```toml
[s3]
endpoint = "https://s3.eu-central-1.amazonaws.com"
region = "eu-central-1"
access_key_id = "AKIA..."
secret_access_key = "${ENV:SIFTP_S3_SECRET}"
path_style = false            # true for MinIO, VersityGW, SeaweedFS, Ceph
spool_dir = "/var/lib/siftp/spool"

[[users]]
name = "alice"
password_hash = "$argon2id$..."
virtual_root = "s3://company-files/users/alice"
permissions = "rwdnml"
```

Set `[server].root = "s3://bucket/base"` to put every default user root
in S3. Tested against AWS-style SigV4 servers VersityGW and SeaweedFS.

| Operation | On S3 |
| --- | --- |
| File `/a/b.txt` | Object `prefix/a/b.txt` |
| Directory | A key prefix; `mkdir` writes an empty `dir/` marker |
| Download | Ranged GETs, 8 MiB at a time |
| Upload | Staged in `spool_dir`, published when the transfer ends (multipart above `part_size_bytes`) |
| Failed upload | Nothing is published; the previous object stays |
| Rename | Copy, then delete (directories up to 10 000 objects) |
| Append, `REST` resume | The existing object is downloaded into the spool first |
| Symlinks | Not supported (`invalid_command`) |
| Times and modes | Server-assigned `Last-Modified`; `MFMT`/`SETSTAT` times are accepted and ignored |
| Free space | Reported as 1 PiB; quotas still apply |

The spool needs room for the largest concurrent uploads. Requests are
presigned (SigV4) and sent over HTTPS with rustls.

## Tiered storage

With `[tier].remote` set, every file under `[server].root` is also
copied to S3, and the local disk keeps only what is in use. Linux and
other Unix hosts; the filesystem needs user extended attributes and
sparse files (ext4, XFS, Btrfs).

```toml
[s3]
endpoint = "https://s3.us-east-2.amazonaws.com"
region = "us-east-2"
path_style = false
credentials = "instance"          # the EC2 instance role

[tier]
remote = "s3://example-siftp-tier/ftp"
storage_class = "INTELLIGENT_TIERING"
evict_after_days = 30
high_watermark_percent = 80
low_watermark_percent = 70
```

| Event | What happens |
| --- | --- |
| A file has not changed for `upload_after_secs` | Uploaded under a random key in the chosen storage class |
| It changes later | Uploaded again under a new key |
| Nobody reads or writes it for `evict_after_days` | The local copy becomes a placeholder: same name, size, time, and mode, no data on disk |
| The disk passes `high_watermark_percent` | Least recently used copies become placeholders until usage is below `low_watermark_percent` |
| A client reads a placeholder (or appends, resumes, or truncates it to a new size) | The file is downloaded first, checked, and put back in place |
| A client overwrites a placeholder | The new upload replaces it; nothing is downloaded |
| Rename, delete, `MFMT`, `SETSTAT` times and modes | Work on placeholders without downloading |

Listings, sizes, times, and quotas are the same whether a file is local
or a placeholder. The first read of a placeholder waits for the whole
file to download. Storage classes are limited to those that serve reads
immediately: `STANDARD`, `STANDARD_IA`, `ONEZONE_IA`,
`INTELLIGENT_TIERING`, `GLACIER_IR`.

Each file's state is an extended attribute (`user.siftp.tier`). Copy
files in or out with siftp (FTP, SFTP) rather than on the host: a
placeholder copied with `cp` is an empty sparse file. A placeholder
changed on the host is quarantined (never uploaded, refused on read,
counted in `tier_quarantined`). Setup, bucket policy, and restore:
[Operations](/docs/operations.html#tiered-storage).

---

<!-- Source: https://siftp.org/docs/identity.html (Markdown: https://siftp.org/docs/identity.md) -->

# Users and access

## Where users come from

| Source | Managed by | Durable |
| --- | --- | --- |
| `[[users]]` in the config file | You, in the file | Yes; read-only through the API |
| User store (`[server].user_store`) | Admin API and MCP | Yes; atomic writes, survives restart and crash |
| LDAP | Your directory | Session only; nothing is stored |

Lookup order at login: config file, then store, then LDAP.

## Permissions

| Bit | Allows |
| --- | --- |
| `r` | Download, SFTP read, WebDAV `GET` |
| `w` | Upload, SFTP write, `SETSTAT`, `MFMT`, symlinks |
| `d` | Delete files and directories |
| `n` | Rename (needed on both ends) |
| `m` | Create directories |
| `l` | List, stat, WebDAV `PROPFIND` |

A missing bit is `permission_denied` before any file is touched.

## Groups and directory rules

```toml
[[groups]]
name = "staff"
permissions = "rwdnml"

[[groups.directories]]
path = "/archive"
permissions = "rl"

[[users]]
name = "carol"
password_hash = "$argon2id$..."
groups = ["staff"]

[[users.directories]]
path = "/incoming"
permissions = "wl"
```

- Base bits: the user's `permissions`; if absent, the union of their
  groups' `permissions`; if no groups, `rl`.
- A rule covers its path and everything below it. The longest matching
  rule wins. At the same path a user rule beats group rules; group rules
  at the same path are combined.
- Rules replace the base for their subtree, so they can grant
  (`/incoming = "wl"` for a read-only user) or restrict
  (`/archive = "rl"`).
- Symlinks do not launder rights: an operation through a link is
  checked against both the link's path and the path it resolves to.

## SSH keys

```toml
ssh_public_keys = [
  'expiry-time="20271231" ssh-ed25519 AAAAC3... laptop',
  "no-pty ecdsa-sha2-nistp256 AAAAE2... ci",
]
```

- `expiry-time` (UTC, `YYYYMMDD[HHMM[SS]]`) stops the key at that time.
- `restrict`, `no-pty`, `no-port-forwarding`, `no-agent-forwarding`,
  `no-x11-forwarding`, `no-user-rc`, `no-touch-required` are accepted as
  no-ops (siftp never grants those). Any other option is a config error.
- Revoke keys server-wide in `[server].revoked_keys_file`: OpenSSH key
  lines or `SHA256:` fingerprints, `#` comments. It is re-read on reload.

## Passwords

- argon2id only: `siftpd hash-password`, or plaintext through the API
  (hashed server-side, never stored or echoed).
- Rotation:

```sh
curl -X POST -H "Authorization: Bearer $SIFTP_ADMIN_TOKEN" \
  http://127.0.0.1:8080/api/v1/users/bob/password
# {"name":"bob","generated":true,"password":"Xk4...","password_set_unix":1791849600}
```

  Without a body a 24-character password is generated and shown once.
  With `{"password":"..."}` the given one is set.
- Expiry: `[server].password_max_age_days` rejects passwords older than
  that, counted from `password_set_unix` (set by the API; add it by hand
  for config users). Clients see the normal `auth_failed`.

## Login protection

| Setting | Default | Effect |
| --- | --- | --- |
| `max_failed_logins` / `ban_window_secs` | 5 / 600 | Failures from one IP in the window ban it |
| `ban_duration_secs` | 900 | Ban length; covers every listener and the admin API |
| `max_connections` / `max_connections_per_ip` | 512 / 32 | Shared by FTP, FTPS, SFTP, WebDAV |
| `max_sessions` (per user) | unlimited | Concurrent logins per user |

Wrong admin or MCP tokens count as failed logins.

## LDAP

```toml
[ldap]
url = "ldaps://ldap.example.com"
bind_dn = "cn=siftp,ou=services,dc=example,dc=com"
bind_password = "${ENV:SIFTP_LDAP_PASSWORD}"
base_dn = "ou=people,dc=example,dc=com"
user_filter = "(uid={user})"
group_attribute = "memberOf"
require_group = true

[ldap.group_map]
"cn=ftp-staff,ou=groups,dc=example,dc=com" = "staff"
```

- Search-then-bind as above, or direct bind with
  `user_dn_template = "uid={user},ou=people,dc=example,dc=com"`.
- Use `ldaps://`, or `ldap://` with `starttls = true`; `ca_file` adds a
  private CA.
- Mapped groups give the user that group's permissions and directory
  rules; unmapped users get `default_permissions`, or are refused with
  `require_group = true`.
- Roots: `virtual_root_template` (`{user}` substituted), else
  `<server.root>/<user>`; same jail as everyone else.
- Empty passwords are refused before contacting the directory. If the
  directory is unreachable, logins fail as `auth_failed` and the audit
  log records `internal`.

---

<!-- Source: https://siftp.org/docs/errors.html (Markdown: https://siftp.org/docs/errors.md) -->

# Error codes

Every failure carries `{code, message, hint, docs_url}`. Branch on
`code`. Codes are never renamed or reused; new ones may be added, so
treat an unknown code as `internal`.

| Code | HTTP | FTP | SFTP status | Meaning |
| --- | ---: | ---: | --- | --- |
| `auth_failed` | 401 | 530 | auth refused | Missing or wrong credentials |
| `permission_denied` | 403 | 550 | PERMISSION_DENIED (3) | Logged in, but not allowed |
| `not_found` | 404 | 550 | NO_SUCH_FILE (2) | No such file, directory, user, or session |
| `already_exists` | 409 | 550 | FAILURE (4) | The name is taken |
| `invalid_path` | 400 | 553 | PERMISSION_DENIED (3) | The path leaves the root or is malformed |
| `invalid_command` | 400 | 500 | BAD_MESSAGE (5) | Malformed command, packet, body, or config |
| `rate_limited` | 429 | 421 | FAILURE (4) | A limit fired: ban, cap, timeout, drain |
| `quota_exceeded` | 507 | 552 | FAILURE (4) | The write would exceed the user's quota |
| `tls_required` | 426 | 534 | n/a | TLS is required for this |
| `internal` | 500 | 451 | FAILURE (4) | Server-side fault |

FTP may use a protocol-specific number where an RFC requires one (for
example `504` for an unknown `HOST`, `503` for `HOST` after `USER`); the
code after the number is always one of the ten.

## auth_failed

Credentials were missing or wrong, the password expired, the key is
expired or revoked, the user may not log in under this FTP `HOST`, or
(admin API, MCP) the token is wrong or not configured. All of these look
the same on purpose. Repeated failures from one address lead to a ban
(`rate_limited`).

## permission_denied

The user lacks the permission bit for this path (`r w d n m l`),
possibly because of a directory rule; a config-file user was changed
through the API (edit the file instead); a WebDAV client tried to write
(the gateway is read-only); or an active-mode `PORT` named another host
or a port below 1024. The `hint` says which bit is missing.

## not_found

The file, directory, user, or session does not exist. List the parent
(`MLSD`, SFTP `READDIR`, `GET /api/v1/files`) or the collection
(`GET /api/v1/users`, `GET /api/v1/sessions`).

## already_exists

A create collided with an existing name: `MKD`, an exclusive SFTP open,
SFTP `RENAME` onto an existing file (use `posix-rename@openssh.com` to
replace), or a user name that is taken.

## invalid_path

The path contains NUL, a `..` that leaves the root (also when
percent-encoded), a symlink that points outside, or a new symlink whose
target is outside the root. The same request is
refused identically on every protocol.

## invalid_command

The request is malformed: unknown FTP verb, bad argument, oversized
line or packet, unsupported SFTP extension, invalid JSON, unknown config
key, weak password hash, unsupported SSH key option. The `hint` names
the expected form; `HELP`, `tools/list`, and `/api/v1/openapi.json` list
what is valid.

## rate_limited

A protective limit fired: login ban, connection cap (global or per
address), per-user session cap, too many open SFTP handles, too many
concurrent admin requests, an idle or handshake timeout, or the server
is draining for a restart. Wait and retry; operators tune these under
`[server]`.

## quota_exceeded

The write (upload, SFTP write, or `SETSTAT` that grows a file) would
exceed `quota_bytes`. The partial file is removed unless the client was
extending an existing one (`REST`, `APPE`). Free space or raise the
quota.

## tls_required

The server requires TLS (`require_tls`), `PROT` was sent without `AUTH
TLS`, `CCC` was attempted, active mode was combined with `PROT P`, or a
TLS data connection did not resume the control session while
`require_session_reuse` is on. Connect with `AUTH TLS`, `PBSZ 0`,
`PROT P`, or use the implicit FTPS port.

## internal

A server-side fault: disk error, unreadable TLS material, an S3 or LDAP
server that is down or refusing access. Retry once; if it persists, the
`message` and the daemon log name the cause.

---

<!-- Source: https://siftp.org/docs/operations.html (Markdown: https://siftp.org/docs/operations.md) -->

# Operations

## Install

### Debian or Ubuntu package

From the apt repository ([Download](/download.html#install-with-apt)
adds it), or from a downloaded package:

```sh
sudo apt install siftpd                           # from apt.siftp.org
sudo apt install ./siftpd_0.2.1-1_amd64.deb       # or _arm64
sudo systemctl status siftpd
```

The package creates the `siftp` system user, `/srv/siftp`,
`/var/lib/siftp`, `/var/log/siftp`, a random admin token in
`/etc/siftp/siftpd.env`, and enables the hardened unit. The default
config (`/etc/siftp/siftp.toml`) serves FTP on 21, implicit FTPS on 990
(once a certificate exists), SFTP on 2222 (so the host's OpenSSH keeps
22), and the admin API on `127.0.0.1:8080`.

```sh
sudo -u siftp siftpd hash-password 'a strong password'
sudoedit /etc/siftp/siftp.toml          # or create users through the API
sudo systemctl reload siftpd
```

### Container

Release images are published to the project's GitHub Container
Registry (`ghcr.io/<owner>/siftp:<version>`, multi-arch, signed). To
build the same image locally:

```sh
docker build -f packaging/container/Dockerfile -t siftp:0.2.1 .
```

```sh
docker run -d --name siftp \
  -e SIFTP_ADMIN_TOKEN="$(openssl rand -hex 32)" \
  -e SIFTP_FTP_PASSIVE_EXTERNAL_IP=203.0.113.10 \
  -p 2121:2121 -p 2222:2222 -p 127.0.0.1:8080:8080 -p 40000-40019:40000-40019 \
  -v siftp-data:/srv/siftp -v siftp-state:/var/lib/siftp \
  siftp:0.2.1
```

The image is distroless, runs as UID 65532, and has a `HEALTHCHECK`
(`siftpd health`). Mount a certificate pair at `/etc/siftp/tls/` to
enable FTPS on 2990. Create users through the admin API:

```sh
curl -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{"name":"alice","password":"...","permissions":"rwdnml"}' \
  http://127.0.0.1:8080/api/v1/users
```

### Binary and systemd

```sh
sudo install -m 755 siftpd /usr/bin/siftpd
sudo install -m 644 packaging/systemd/siftpd.service /lib/systemd/system/
sudo useradd --system --home /var/lib/siftp --shell /usr/sbin/nologin siftp
sudo install -d -o siftp -g siftp -m 750 /srv/siftp
sudo install -d -o root -g siftp -m 750 /etc/siftp
sudo install -m 640 -g siftp packaging/siftp.toml /etc/siftp/siftp.toml
echo "SIFTP_ADMIN_TOKEN=$(openssl rand -hex 32)" | sudo install -m 640 -g siftp /dev/stdin /etc/siftp/siftpd.env
sudo systemctl daemon-reload && sudo systemctl enable --now siftpd
```

macOS: `packaging/launchd/org.siftp.siftpd.plist` (same signals; the
pooled data path is used on macOS).

## Verify a release

Every release asset has a Sigstore signature, certificate, and a build
provenance attestation; `SHA256SUMS` covers all files.

```sh
sha256sum -c SHA256SUMS --ignore-missing
cosign verify-blob siftpd-v0.2.1-x86_64-linux.tar.gz \
  --signature siftpd-v0.2.1-x86_64-linux.tar.gz.sig \
  --certificate siftpd-v0.2.1-x86_64-linux.tar.gz.pem \
  --certificate-identity https://github.com/sitevue/siftp/.github/workflows/release.yml@refs/tags/v0.2.1 \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com
gh attestation verify siftpd-v0.2.1-x86_64-linux.tar.gz --owner sitevue
cosign verify ghcr.io/sitevue/siftp:v0.2.1 \
  --certificate-identity https://github.com/sitevue/siftp/.github/workflows/release.yml@refs/tags/v0.2.1 \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com
```

The identity is the release workflow in the `sitevue/siftp` repository
at the tag you are verifying. A signature from any other repository,
workflow, or tag fails the check.

Each release also ships a CycloneDX SBOM (`*.cdx.json`).

## Service control

| Action | Command | Effect |
| --- | --- | --- |
| Reload | `systemctl reload siftpd` (SIGHUP) | Re-reads config, user store, revoked keys, TLS certificate; reopens the audit log; sessions keep running |
| Stop | `systemctl stop siftpd` (SIGTERM) | Drain: stop accepting, 421 to idle sessions, transfers finish, exit 0 within `shutdown_grace_secs` (30 s) |
| Stop now | second SIGTERM | Skip the wait |
| Validate | `siftpd --check-config --config /etc/siftp/siftp.toml` | JSON summary with warnings, or the error |

Bind address changes need a restart. Everything else reloads.

## Health and readiness

| Endpoint | Auth | 200 when | 503 when |
| --- | --- | --- | --- |
| `GET /healthz` | none | process is up | never |
| `GET /readyz` | none | all listeners bound, not draining | starting or draining |

`siftpd health --config <file>` checks `/readyz` and exits 0 or 1, for
container probes. Point load balancers at `/readyz` so a draining node
leaves rotation before it stops accepting.

## Metrics and alerts

`GET /metrics` on the admin listener, with the admin token:

```yaml
scrape_configs:
  - job_name: siftp
    authorization: { credentials_file: /etc/prometheus/siftp-token }
    static_configs: [{ targets: ["127.0.0.1:8080"] }]
```

| Metric | Type |
| --- | --- |
| `siftp_bytes_sent_total`, `siftp_bytes_received_total` | counter |
| `siftp_transfers_total{path="sendfile|splice|pool|uring"}` | counter |
| `siftp_transfers_failed_total` | counter |
| `siftp_logins_ok_total`, `siftp_logins_failed_total`, `siftp_logins_cached_total`, `siftp_bans_total` | counter |
| `siftp_connections_total`, `siftp_connections_refused_total` | counter |
| `siftp_tls_data_resumed_total`, `siftp_tls_data_full_handshake_total` | counter |
| `siftp_active_sessions`, `siftp_active_connections`, `siftp_active_transfers` | gauge |
| `siftp_banned_addresses`, `siftp_buffer_pool_outstanding_buffers` | gauge |
| `siftp_ready`, `siftp_start_time_seconds`, `siftp_build_info` | gauge |

Alert rules: `packaging/prometheus/alerts.yml` (down, not ready, login
failure spikes, bans, refused connections, transfer failures, buffer
pool exhaustion, connection count).

## Logs

Logs go to stderr (journald under systemd). `[logging].format = "json"`
gives one JSON object per line with the session span: `peer`,
`session`, `user`, and `host` (after FTP `HOST`). Every transfer logs
`transfer complete` or `transfer failed` with verb, path, bytes, data
path, and milliseconds.

## Audit log

`[logging].audit_file` holds one JSON line per login (success or
failure), ban, transfer, file change, and admin change, with `seq` and a
SHA-256 `chain` linking each line to the previous one. Lookups such as
`CWD`, `SIZE`, `LIST`, and SFTP `STAT` are not logged. SPEC §8.5 lists
every `action` value.

```json
{"ts":1760320000,"user":"alice","action":"STOR","path":"/report.pdf","result":"ok","remote":"203.0.113.7:51522","seq":42,"chain":"9f2c…"}
```

```sh
siftpd audit-verify /var/log/siftp/audit.jsonl           # exit 2 if altered
siftpd audit-verify audit.jsonl --anchor <last chain of the rotated file>
curl -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8080/api/v1/audit/verify
```

Rotation: the package installs `/etc/logrotate.d/siftpd` (daily, 90
days, `create`, then `systemctl reload`). The chain continues into the
new file. Ship lines off the host if you need to detect truncation of
the newest lines.

## SSH host key

siftpd generates an Ed25519 host key at `[sftp].host_key_file` on
first start. Publish its fingerprint so SFTP clients can check it
instead of trusting the first connection:

```sh
ssh-keygen -lf /var/lib/siftp/ssh_host_ed25519_key   # SHA256:... to publish
ssh-keygen -yf /var/lib/siftp/ssh_host_ed25519_key   # public key line
```

On a client, put the public key in `known_hosts` with the server's
name and port, then connect with strict checking:

```sh
echo "[files.example.com]:2222 ssh-ed25519 AAAAC3..." >> ~/.ssh/known_hosts
sftp -o StrictHostKeyChecking=yes -P 2222 alice@files.example.com
curl --hostpubsha256 '<base64 part of the SHA256 fingerprint>' sftp://files.example.com:2222/
```

Back the key up with the user store. A new key makes every client
warn until it updates `known_hosts`.

## Backups

| Path | Contents |
| --- | --- |
| `/var/lib/siftp/users.toml` | API-managed users (argon2id hashes) |
| `/var/lib/siftp/ssh_host_ed25519_key` | SSH host key (clients pin it) |
| `/etc/siftp/` | Config, TLS certificate and key, admin token |
| `/var/log/siftp/audit.jsonl*` | Audit trail |
| `/srv/siftp/` | User files |

The user store is replaced atomically on every change, so copying it
at any time yields a complete file.

## Tiered storage

[Tiered storage](/docs/storage.html#tiered-storage) needs a bucket that
nothing else writes to, and a role or keys that can use it.

```sh
aws s3api create-bucket --bucket example-siftp-tier \
  --create-bucket-configuration LocationConstraint=us-east-2
aws s3api put-public-access-block --bucket example-siftp-tier \
  --public-access-block-configuration BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true
aws s3api put-bucket-versioning --bucket example-siftp-tier \
  --versioning-configuration Status=Enabled
aws s3api put-bucket-lifecycle-configuration --bucket example-siftp-tier \
  --lifecycle-configuration '{"Rules":[{"ID":"siftp","Status":"Enabled","Filter":{},
    "NoncurrentVersionExpiration":{"NoncurrentDays":30},
    "AbortIncompleteMultipartUpload":{"DaysAfterInitiation":7}}]}'
```

Versioning keeps an overwritten or deleted object for 30 days. The
instance role (or the keys in `[s3]`) needs:

```json
{"Version": "2012-10-17", "Statement": [
  {"Effect": "Allow", "Action": ["s3:ListBucket"],
   "Resource": "arn:aws:s3:::example-siftp-tier"},
  {"Effect": "Allow", "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject",
     "s3:AbortMultipartUpload"],
   "Resource": "arn:aws:s3:::example-siftp-tier/*"}]}
```

- siftp writes `objects/` (file copies), `manifests/` (the newest 48
  lists of every file, directory, and link), and `root-id`. The data
  disk holds `.siftp-tier/` with the matching `root-id`, a lock, and
  temporary files.
- If the two `root-id`s differ, for example after the data disk was
  replaced or emptied, tiering turns read-only and logs an error. Stop
  siftpd and rebuild the root as placeholders:

  ```sh
  sudo systemctl stop siftpd
  sudo -u siftp siftpd tier-restore --config /etc/siftp/siftp.toml
  sudo systemctl start siftpd
  ```

  It prints a JSON report. Existing files are never replaced, and
  objects no manifest lists are placed under `siftp-recovered/`.
- Metrics: `siftp_tier_uploads_total`, `siftp_tier_evictions_total`,
  `siftp_tier_fetches_total` (with `_bytes_total` companions),
  `siftp_tier_errors_total`, and the gauges `siftp_tier_pending`,
  `siftp_tier_quarantined`, `siftp_tier_orphans`. Objects no file refers
  to are counted, not deleted.
- In a container, put `[server].root` on a volume: the overlay
  filesystem may lack extended attributes, and siftpd refuses to start
  tiering without them.

## Upgrades and rollback

1. Read the release notes; check the config with the new binary before
   switching: `siftpd-new --check-config --config /etc/siftp/siftp.toml`.
2. Install the package (or replace the binary), then
   `systemctl restart siftpd`. Restart drains first; clients reconnect.
3. Roll back by installing the previous package. The config, user
   store, host key, and audit chain are compatible within 0.x minor
   releases; new config keys default to off.
4. Versioned docs for each release stay at `/v/<version>/`.

---

<!-- Source: https://siftp.org/docs/performance.html (Markdown: https://siftp.org/docs/performance.md) -->

# Performance

How siftp moves bytes, and how it compares with vsftpd, ProFTPD, and
OpenSSH's sftp-server on the same machine.

## Benchmark

Measured 2026-10-16 with `bench/run.sh` on a bare-metal Linux host (see
[Method](#method)). All four servers ran one at a time in the same
Debian 13 container, on loopback, with files on tmpfs. Each number is
the median of 5 runs; the full results with min and max are in
`bench/results/amd64.json` in the repository.

| FTP, plaintext | siftp | vsftpd 3.0.5 | ProFTPD 1.3.8c |
| --- | ---: | ---: | ---: |
| Download 1 GiB, MiB/s | 4013 | 4019 | 3910 |
| Upload 1 GiB, MiB/s | 2798 | 2759 | 2697 |
| 16 parallel downloads of 1 GiB, total MiB/s | 73715 | 74625 | 74198 |
| Small files up (1000 x 16 KiB), files/s | 5806 | 4359 | 4297 |
| Small files down, files/s | 6626 | 4859 | 5036 |
| Logins per second, first login, sequential | 11.7 | 19.0 | 10.4 |
| Logins per second, first login, 32 concurrent clients | 142 | 393 | 106 |
| Logins per second, repeat login, sequential | 7325 | 19 | 10 |
| Logins per second, repeat login, 32 concurrent clients | 97173 | 393 | 106 |
| Server CPU per GiB downloaded, s | 0.049 | 0.050 | 0.051 |
| Memory, idle, MiB | 16.4 | 2.7 | 10.2 |
| Memory with 200 logged-in sessions, MiB | 17.1 | 65.5 | 174.0 |

| FTPS, explicit TLS, PROT P | siftp | vsftpd 3.0.5 | ProFTPD 1.3.8c |
| --- | ---: | ---: | ---: |
| Download 1 GiB, MiB/s | 2357 | 1040 | 1391 |
| Upload 1 GiB, MiB/s | 1752 | 683 | 1520 |

| SFTP | siftp | OpenSSH 10.0p2 sftp-server |
| --- | ---: | ---: |
| Download 1 GiB, MiB/s | 954 | 763 |
| Upload 1 GiB, MiB/s | 900 | 811 |
| Session setup (key exchange, key auth, subsystem), ms | 51 | 80 |

### What the numbers say

- **Plaintext FTP:** large files move at about the same speed on all
  three, within 4%. Every server downloads with `sendfile`, so the
  kernel does the work, and the client and loopback set the limit.
  Sixteen parallel streams reach about 72 GiB/s on every server, which
  is the host's limit, not the servers'. With 1000 small files, siftp
  is about a third faster than both in each direction.
- **FTPS:** siftp downloads at 2.3x vsftpd and 1.7x ProFTPD, and
  uploads at 2.6x vsftpd and 1.15x ProFTPD. TLS rules out `sendfile`,
  so these transfers use the buffer pool, with file reads and writes
  overlapping the socket work (see [Data path](#data-path)).
- **SFTP:** siftp downloads 25% faster and uploads 11% faster than
  OpenSSH's sftp-server, and sets up a session in 51 ms against 80.
  Both negotiated `chacha20-poly1305@openssh.com` with
  `mlkem768x25519-sha256`.
- **Many sessions:** siftp runs every session in one process, so 200
  logged-in sessions add under 1 MiB. vsftpd and ProFTPD fork one
  process per session. siftp's idle figure is higher than theirs
  because the runtime starts one worker thread per CPU (96 on this
  host), and because it is measured after the login tests, so it
  includes heap the allocator kept after about 600,000 logins.
- **First logins:** each one runs argon2id (64 MiB, 3 passes), which
  is deliberately expensive, so siftp checks fewer new passwords per
  second than vsftpd, which checks the system's yescrypt hashes (`$y$`
  in `/etc/shadow`). With 32 clients, siftp completes 142 checks per
  second, each taking 2.6x as long as one check alone. Unknown user
  names cost siftp the same argon2id work, so login timing does not
  reveal which accounts exist.
- **Repeat logins:** a credential that argon2id has accepted is served
  from siftp's verification cache for `login_cache_ttl_secs` (default
  300 s) without recomputing the hash: 7325 logins per second from one
  client, 97173 from 32, limited by connection handling. vsftpd and
  ProFTPD hash every time, so their repeat and first rows are equal.

### Login throughput

A password check costs what argon2id costs, and that is the point: a
copy of the user store costs an attacker the same per guess. On the
benchmark host one login takes about 85 ms, and 32 at once complete
about 140 per second, each taking 2.6x as long as one alone, with most
of the 96 vCPUs idle. Memory bandwidth sets that ceiling, not the
implementation: on an earlier Apple silicon test VM, Debian's reference
C library took twice as long per lane as the Rust implementation siftp
uses. A faster kernel would shorten a single login, not raise the
ceiling.

siftp keeps logins fast without making hashes cheaper:

- **Verification cache.** After argon2id accepts a password, the server
  keeps a BLAKE2b keyed digest of the user name, the stored hash, and
  the password under a 256-bit key drawn from the OS at start. The key
  is never written anywhere, so neither the user store nor the cache
  contents help an attacker, and only credentials that passed are ever
  stored: a wrong password misses the cache and pays a full argon2id
  check every time. A changed password misses because the stored hash
  is part of the digest, and the cache is cleared whenever the user
  table changes. `login_cache_ttl_secs = 0` turns it off.
- **Concurrency.** Checks run on all cores. `max_concurrent_password_checks`
  (default one per CPU) caps how many run at once, so a burst of logins
  queues instead of allocating 64 MiB per connection.
- **Keys.** SSH public-key logins do not involve argon2id at all; the
  51 ms SFTP session setup is key exchange and authentication.

If you need more distinct first logins per second than that on similar
hardware, add cores and memory bandwidth, or lengthen the cache TTL.
siftp accepts argon2id hashes down to the OWASP minimum (19 MiB, 2
passes), which costs about a third of the default, but `siftpd
hash-password` only produces default-cost hashes: a cheaper hash makes
a stolen user store exactly that much cheaper to attack, and that is
not a trade the server will make for you.

### Method

- **Host:** AWS `c7i.metal-24xl`: Intel Xeon (Sapphire Rapids), 96
  vCPUs, 188 GiB, bare metal with no hypervisor. Ubuntu 24.04 with
  kernel 7.0, CPU frequency governor `performance`, nothing else
  running. The five runs behind each siftp number fell within 5% of
  each other; the widest spread for another server was 10% (vsftpd's
  FTPS download). An earlier run in Docker Desktop's VM on an Apple
  silicon Mac, before the current FTPS data path, is in
  `bench/results/arm64.json`. Run the harness on your own hardware
  before sizing a deployment.
- **Storage and network:** tmpfs and loopback. This measures each
  server's data path rather than a disk or a NIC.
- **Clients:** curl 8.14.1 for large FTP and FTPS transfers, OpenSSH's
  `sftp` for SFTP, and Python's `ftplib` for small files, logins,
  parallel downloads, and sessions. The same client is used against
  every server.
- **Throughput counts only the data phase.** For curl, that is
  `time_total - time_pretransfer`. For SFTP, it is wall-clock time minus
  the median time of a session that transfers nothing (the setup row).
  Parallel sessions all log in first, wait at a barrier, then start
  `RETR` together. Login speed has its own rows.
- **Logins:** one login is connect, `USER`, `PASS`, `QUIT` with
  `ftplib`. Sequential runs at least 100 logins and at least 3 s;
  concurrent runs 32 processes for 6 s after a barrier. First-login
  rows run siftp with `login_cache_ttl_secs = 0`, so every login hashes.
  vsftpd and ProFTPD have no cache, so their first and repeat rows are
  the same measurement.
- **CPU:** the sum of user and system time across the server's process
  tree during eight 1 GiB `RETR`s on one logged-in session.
- **Memory:** PSS summed across the server's process tree, idle and
  then with 200 sessions logged in and idle.
- **Configuration:** each server has one local user with a home
  directory; vsftpd and ProFTPD have `sendfile` on. FTPS uses an
  RSA-2048 certificate; all three negotiated TLS 1.3 with
  `TLS_AES_256_GCM_SHA384`. siftp is the release build with the
  `io-uring` feature; `io_uring` is off (the default), so the
  `sendfile` and `splice` paths served these transfers.

### Reproduce

```sh
bench/run.sh            # full run, about 10 minutes; needs Docker
bench/run.sh --quick    # smaller files, 3 runs
```

The script builds `siftpd` for Linux on your machine's architecture and
runs `bench/bench.py` in a Debian container that has vsftpd, ProFTPD,
and OpenSSH from the distribution. It writes
`bench/results/<arch>.json` and prints the tables above.

## Data path

Each transfer takes the fastest path that is correct for it. The
`GET /api/v1/stats` and `/metrics` counters show which path served
each transfer.

| Path | When | Notes |
| --- | --- | --- |
| `sendfile` | Linux, plaintext binary download | File to socket in the kernel |
| `splice` | Linux, plaintext binary upload | Socket to pipe to file, with the pipe grown to 1 MiB where the kernel allows; the file is created only after the first splice works |
| `io_uring` | Linux, build feature `io-uring` and `io_uring = true` | Used when `sendfile` or `splice` is off or refused |
| Pooled copy | Everything else: TLS, ASCII mode, S3 roots, rate-limited users, macOS | Chunks from a shared buffer pool with a fixed byte budget. Socket and file work overlap: an upload fills a whole chunk before one file write, which runs while the next chunk fills; a download reads the next chunk while the current one is sent, when the pool has a spare buffer |

If a kernel path is unavailable (for example, seccomp blocks
`io_uring` under Docker's default profile), the transfer falls back to
the pooled copy before any byte moves. Clients see no difference.

When every pooled buffer is in use, new chunks wait for one to be
returned. That wait is how backpressure works, and it caps transfer
memory at `buffer_pool_bytes` however many sessions are active.

## Settings

```toml
[performance]
sendfile = true                  # Linux downloads
splice = true                    # Linux uploads
io_uring = false                 # needs the io-uring build feature
buffer_pool_bytes = 67108864     # total memory for pooled transfers
transfer_chunk_bytes = 262144    # size of one pooled chunk
```

Raise `buffer_pool_bytes` if many concurrent TLS or S3 transfers wait
on the pool. `siftp_buffer_pool_waits_total` in `/metrics` counts
those waits.

## Assembly kernels

`siftp-asm` has hand-written x86_64 and aarch64 versions of the hot
byte loops, each next to a pure-Rust version that defines the expected
result.

| Kernel | Used for |
| --- | --- |
| `crc32c` | Checksums (SSE4.2 `crc32`; aarch64 `crc32c` after a CPU feature check) |
| `adler32` | Weak checksums |
| `find_byte`, `find_newline`, `find_crlf` | FTP command framing |
| `ct_eq` | Constant-time token comparison |

Differential tests compare each kernel with its Rust version on every
build. `--no-default-features` builds without assembly and gives the
same results.

```sh
cargo test  -p siftp-asm --features asm   # differential tests
cargo bench -p siftp-asm --features asm   # both versions side by side
```

---

<!-- Source: https://siftp.org/docs/security.html (Markdown: https://siftp.org/docs/security.md) -->

# Security

## Model

siftp assumes clients are untrusted, the network is hostile, and the
operator controls the host and config. Every client path is resolved
inside the user's root and permission-checked before any filesystem or
object-storage call. The daemon runs as an unprivileged user; the
systemd unit grants only `CAP_NET_BIND_SERVICE`.

## Controls

| Area | Control |
| --- | --- |
| Paths | `..` past the root, NUL bytes, and encoded variants refused on every protocol; symlinks followed only inside the root, and created only with targets inside it; directory rules re-checked against symlink targets |
| Passwords | argon2id only (m≥19 MiB, t≥2), constant-time comparison, dummy verification for unknown users, identical failures for every cause, optional maximum age. Repeat logins within `login_cache_ttl_secs` are served from a cache of keyed digests (process-only key, never the password); wrong passwords always pay the full hash |
| SSH | Ed25519/ECDSA host keys (RSA refused, see below), post-quantum hybrid kex `mlkem768x25519-sha256`, strict KEX (Terrapin), AEAD ciphers only, key expiry and revocation, no shell/exec/forwarding |
| TLS | rustls, TLS 1.2+, optional `require_tls`, `CCC` refused, optional enforced session reuse on FTPS data channels, certificate hot reload |
| FTP data | Passive connections must come from the client's address; active mode only to the client, port ≥ 1024 |
| Abuse | Login bans across all listeners and the admin API, global and per-IP connection caps, per-user session caps, command/packet/handle limits, idle and handshake timeouts, rate limits on every data path |
| Admin API | Loopback by default; bearer token compared in constant time, fail-closed when unset; 1 MiB bodies, 64 concurrent requests, 30 s per request; `nosniff`, `no-store`, `DENY` framing |
| WebDAV | Read-only; files served with `nosniff` and `Content-Security-Policy: sandbox` |
| Storage | User store written atomically with mode 0600; uploads to S3 are published only on success |
| Audit | Hash-chained JSON lines; `siftpd audit-verify` detects edits, insertions, deletions |
| Secrets | Passwords, keys, tokens, and file contents never appear in logs, errors, or API responses |
| Code | `#![forbid(unsafe_code)]` everywhere except the assembly FFI shims and one documented io_uring submission helper |

## Deploying the admin API

The admin listener speaks plain HTTP and defaults to `127.0.0.1:8080`.
To reach it remotely, keep it on loopback and put a TLS proxy in front:

```text
# Caddyfile
admin.example.com {
    reverse_proxy 127.0.0.1:8080
}
```

```nginx
server {
    listen 443 ssl;
    server_name admin.example.com;
    ssl_certificate     /etc/ssl/admin.crt;
    ssl_certificate_key /etc/ssl/admin.key;
    location / { proxy_pass http://127.0.0.1:8080; }
}
```

`siftpd --check-config` and startup both warn when the admin bind is
not loopback. Use a token of at least 32 random bytes.

## Dependencies

`cargo deny` (advisories, licenses, banned crates — no OpenSSL —
and sources) and `cargo audit` run on every change. One advisory is
accepted with a reason:

- **RUSTSEC-2023-0071** (`rsa`, timing side channel on private-key
  operations). siftp performs no RSA private-key operations: RSA host
  keys are refused at startup, and client RSA keys are used only to
  verify signatures.

## Fuzzing

cargo-fuzz targets cover FTP command lines, SFTP packets and ATTRS, the
config loader and key parser, and path/Range handling (`fuzz/`). CI runs
each against its seed corpus; `cargo test` replays the corpus plus
thousands of deterministic mutations on every build.

## Reporting a vulnerability

Report privately through the project repository's security advisory
flow, with steps to reproduce against a local instance. Please do not
open a public issue first. Expect an acknowledgement, a fix or
mitigation plan, and credit in the release notes unless you prefer
otherwise.

---

<!-- Source: https://siftp.org/docs/llm.html (Markdown: https://siftp.org/docs/llm.md) -->

# Agents and MCP

Everything an operator can do through the admin API, an agent can do
through MCP, with the same JSON payloads and the same error object.

## Boot sequence

1. `GET http://<host>:<admin-port>/.well-known/siftp.json` (no auth):
   version, protocols this server serves, and links to the docs for
   that exact version.
2. Read the `llms_txt` URL it gives (or [`/llms.txt`](/llms.txt) for
   the latest release).
3. `GET /api/v1/openapi.json` (no auth): every route and schema,
   including the error-code enum.
4. Act with the admin token: REST, or MCP over `POST /mcp` or
   `siftpd mcp-stdio`.

```json
{"name":"siftp","version":"0.2.1","protocols":["ftp","ftps-explicit","ftps-implicit","sftp","webdav-readonly"],"docs":"https://siftp.org/v/0.2.1/docs/","llms_txt":"https://siftp.org/v/0.2.1/llms.txt","docs_latest":"https://siftp.org/docs","openapi":"/api/v1/openapi.json","mcp":"/mcp","metrics":"/metrics","health":"/healthz","ready":"/readyz"}
```

## Errors

```json
{"code":"permission_denied","message":"user 'ops' is defined in the config file /etc/siftp/siftp.toml and is read-only through the API","hint":"edit the [[users]] entry in the config file and reload, or manage a different user name through the API","docs_url":"https://siftp.org/docs/errors#permission_denied"}
```

Branch on `code` (ten values, [Error codes](/docs/errors.html)); show
`message`; do what `hint` says. FTP replies start with the code
(`550 permission_denied: ...`), SFTP status messages too.

## Admin API

```sh
H="Authorization: Bearer $SIFTP_ADMIN_TOKEN"
curl -H "$H" http://127.0.0.1:8080/api/v1/users
curl -H "$H" -H 'Content-Type: application/json' \
  -d '{"name":"erin","password":"...","permissions":"rwl"}' \
  http://127.0.0.1:8080/api/v1/users
curl -H "$H" -X POST http://127.0.0.1:8080/api/v1/users/erin/password
curl -H "$H" 'http://127.0.0.1:8080/api/v1/files?user=erin&path=/'
```

| Route | Purpose |
| --- | --- |
| `GET`/`POST /api/v1/users` | List, create (stored durably) |
| `GET`/`PUT`/`DELETE /api/v1/users/{name}` | Read, replace (omitted fields kept), delete (files kept) |
| `POST /api/v1/users/{name}/password` | Set, or generate and return once |
| `GET /api/v1/sessions`, `DELETE /api/v1/sessions/{id}` | Live sessions, terminate one |
| `GET /api/v1/files?user=&path=` | Listing inside a user's root |
| `GET /api/v1/audit?lines=`, `GET /api/v1/audit/verify` | Audit tail as `{"lines": [...]}`, oldest first, each line a JSON string to parse; chain check |
| `GET /api/v1/stats`, `GET /metrics` | Counters as JSON, Prometheus text |
| `POST /api/v1/reload` | Reload config, users, keys, certificate |

The OpenAPI document gives each route's parameters, request body, and
response schema. `password_hash` is always `null` in responses. Users with
`"source": "config"` are read-only; manage `"source": "store"` users.

## MCP

| Tool | Does |
| --- | --- |
| `siftp_capabilities` | The capability document |
| `siftp_list_users`, `siftp_get_user` | Read users |
| `siftp_create_user`, `siftp_update_user`, `siftp_delete_user` | Manage users |
| `siftp_set_password` | Set or rotate a password |
| `siftp_list_files` | List a directory in a user's root |
| `siftp_list_sessions`, `siftp_kill_session` | Sessions |
| `siftp_tail_audit`, `siftp_verify_audit` | Audit log |
| `siftp_stats` | Counters |
| `siftp_reload_config` | Reload |

```sh
printf '%s\n' \
  '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}' \
  '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"siftp_stats","arguments":{}}}' \
  | siftpd mcp-stdio --config /etc/siftp/siftp.toml
```

Tool results are the admin JSON as text content; failures come back
with `isError: true` and the error object as text. Over HTTP, `POST
/mcp` takes one JSON-RPC message per request with the admin token;
wrong tokens count toward the login ban.

## Checking your work

- Read back what you changed: `siftp_get_user`, `siftp_list_files`.
- Logins, transfers, file changes, and admin changes are in the audit
  log with `user`, `action`, `path`, `result` (lookups are not);
  `siftp_verify_audit` confirms the log is intact.
- `siftp_stats` shows which data path served transfers
  (`transfers_sendfile`, `transfers_pool`, …).

---

<!-- Source: https://siftp.org/download.html (Markdown: https://siftp.org/download.md) -->

# Download

## Install with apt

Debian 12 or later and Ubuntu 22.04 or later, on amd64 and arm64
(including AWS Graviton). Needs `curl`:

```sh
sudo curl -fsSLo /etc/apt/keyrings/siftp.gpg https://apt.siftp.org/siftp-archive-keyring.gpg
echo "deb [signed-by=/etc/apt/keyrings/siftp.gpg] https://apt.siftp.org stable main" | sudo tee /etc/apt/sources.list.d/siftp.list
sudo apt update && sudo apt install siftpd
```

The repository is signed with an Ed25519 key, fingerprint
`705C FD11 EA2D 784D 8110 F551 B34A 5D63 8C91 9943`. Compare it with
`gpg --show-keys /etc/apt/keyrings/siftp.gpg`. The package starts the
service with the hardened systemd unit; what it sets up and how to add
users: [Operations](/docs/operations.html#debian-or-ubuntu-package).

## Releases

Each release has, for Linux x86_64 and aarch64:

| File | Contents |
| --- | --- |
| `siftpd-<version>-<arch>-linux.tar.gz` | Binary, example config, systemd unit, logrotate and Prometheus files, licenses |
| `siftpd_<version>-1_<arch>.deb` | Debian/Ubuntu package with a hardened systemd unit |
| `siftpd-<version>-<arch>.cdx.json` | CycloneDX SBOM |
| `SHA256SUMS` | Checksums for all files |
| `*.sig`, `*.pem` | Sigstore signatures and certificates |

Plus a multi-arch container image, signed with cosign. How to verify:
[Operations](/docs/operations.html#verify-a-release).

The binaries link glibc 2.35 or newer (Ubuntu 22.04, Debian 12, RHEL 9
and later).

## Build from source

```sh
cargo build --release -p siftpd                     # assembly kernels on
cargo build --release -p siftpd --features io-uring # plus the io_uring data path (Linux)
cargo build --release -p siftpd --no-default-features # pure Rust
```

Requirements: Rust 1.89 or newer and a C compiler (for `ring` and the
assembly kernels). Packages:

```sh
cargo deb -p siftpd                                  # .deb (cargo install cargo-deb)
docker build -f packaging/container/Dockerfile -t siftp .
```

## Platforms

| Platform | Status |
| --- | --- |
| Linux x86_64, aarch64 | Supported; all fast paths |
| macOS (Apple silicon, Intel) | Supported for small deployments; pooled data path; launchd plist in `packaging/launchd/` |
| Other Unix | Builds with the pure-Rust paths; not tested in CI |
| Windows | Builds (CI lane, `tests/windows/cross-check.sh`) and passes a smoke test under Wine. Not supported for deployment: no service integration, and secret files do not get owner-only permissions |

## Verify a build

```sh
cargo test --workspace
tests/interop/run.sh target/release/siftpd     # needs curl, lftp, OpenSSH, python3
bench/run.sh --quick                           # needs Docker
```

MIT or Apache-2.0, at your option.

---

<!-- Source: https://siftp.org/spec.html (Markdown: https://siftp.org/spec.md) -->

# siftp Specification — Super Intelligent FTP

- Project: **siftp** (Super Intelligent FTP), https://siftp.org
- Version: 0.2.1
- License: MIT OR Apache-2.0
- Platforms: Linux x86_64 and aarch64 (primary: assembly kernels,
  `sendfile`/`splice`, optional io_uring). macOS runs the same code with
  the pooled data path and is supported for small deployments. Other
  Unix targets build with the pure-Rust fallbacks; Windows compiles but
  is not a supported deployment target (§12).
- Language/runtime: Rust (edition 2021, MSRV 1.89), tokio multi-thread
  runtime; the `io-uring` cargo feature enables the io_uring data path.

This document is normative. Code, `configs/siftp.example.toml`, the admin
OpenAPI document, the MCP tool schemas, `llms.txt`, and the website must
not contradict it; when they disagree, fix the code or amend this spec in
the same change.

## 1. Goals and non-goals

### 1.1 Goals

1. One daemon, `siftpd`, serving FTP, explicit and implicit FTPS,
   SSH/SFTP v3, and a read-only WebDAV/HTTP gateway against the same
   users, virtual roots, and permissions.
2. Throughput close to kernel limits: zero-copy file transfer
   (`sendfile`/`splice`), pooled buffers, backpressure everywhere, an
   optional io_uring data plane, and hand-written assembly kernels for
   checksum/scanning hot paths with bit-identical Rust fallbacks.
3. Secure by default: argon2id password hashing, SSH public-key auth,
   per-user virtual-root jails with per-directory permissions, rate
   limits, login bans, TLS 1.2+, post-quantum SSH key exchange.
4. LLM-operable ("Super Intelligent"): every error is stable structured
   JSON `{code,message,hint,docs_url}`; a capability document at
   `/.well-known/siftp.json`; an OpenAPI 3.1 admin REST API; MCP tools
   mirroring the admin API; `llms.txt`; a JSON-lines audit log.
5. Boring operations: one TOML file with `SIFTP_*` environment
   overrides, a durable store for API-managed users, graceful reload and
   drain, health and readiness probes, Prometheus metrics.

### 1.2 Non-goals

- FTP extensions beyond §4 (no `MFCT`, no `COMB`, no mode B/C, no FXP
  server-to-server transfers — active-mode `PORT` to a third party is
  rejected as an anti-bounce measure).
- SFTP versions other than 3; legacy SCP protocol (OpenSSH `scp` in its
  default SFTP mode works); shell access; port or agent forwarding; SSH
  subsystems other than `sftp`.
- Writes over WebDAV; clustering/HA; OIDC (§12).
- A web admin GUI. The website is static documentation only.

## 2. Architecture

```
            ┌──────────────────────────────────────────────────────┐
 clients ──▶│ siftpd (crates/siftpd) — tokio runtime               │
            │  ┌──────────┐ ┌───────────┐ ┌────────────┐ ┌───────┐ │
            │  │siftp-ftp │ │siftp-sftp │ │siftp-webdav│ │siftp- │ │
            │  │FTP/FTPS  │ │SSH/SFTP v3│ │read-only   │ │admin  │ │
            │  └────┬─────┘ └─────┬─────┘ └─────┬──────┘ └───┬───┘ │
            │       └─────────────┴──────┬──────┘     siftp-mcp│     │
            │                       siftp-core ◀───────────────┘     │
            │   (auth, jail, Storage trait, errors, config, audit)   │
            │   siftp-s3 (S3 Storage)   siftp-ldap (identity source)  │
            │   siftp-asm (kernels + Rust fallback)                   │
            └──────────────────────────────────────────────────────┘
```

- All protocol front-ends share one `siftp-core` session/authorisation
  model: authenticate → bind a `Storage` (virtual root) + `Access`
  (permissions per path) → every file operation is resolved inside the
  root, permission-checked, and audited.
- The data plane never copies file bytes through user space on Linux
  when TLS is off and the root is local (`sendfile`/`splice`); with TLS,
  SSH, rate limits, or remote storage it uses pooled buffers.
  Backpressure: a transfer reads only when the socket (or ring) accepts
  more data; bounded queues everywhere.
- Admin and MCP are *control plane only*: they never move file bytes
  except directory listings and audit tails.

### 2.1 Crate layout

| Crate | Responsibility | Key public surface |
| --- | --- | --- |
| `siftp-core` | Error contract, jail and `Storage` trait, permissions and `Access`, users, durable store, config, context, audit, stats | `ApiError`, `ErrorCode`, `Storage`, `JailedFs`, `Access`, `ServerContext`, `AuditLog`, `capability_document` |
| `siftp-asm` | Assembly kernels + pure-Rust fallbacks, backend selection | `crc32c`, `find_newline`, `ct_eq`, `backend_name`; `asm/<arch>/*.s` |
| `siftp-ftp` | FTP/FTPS control + data channels, TLS context | `Command`, `SUPPORTED_COMMANDS`, `serve_ftp`, `serve_implicit_ftps`, `FtpsTls` |
| `siftp-sftp` | SSH transport + SFTP v3 subsystem | `PacketType`, `StatusCode`, `serve_sftp`, `load_or_generate_host_key`, `EXTENSIONS` |
| `siftp-webdav` | Read-only WebDAV/HTTP gateway | `serve_webdav`, `router` |
| `siftp-s3` | S3-compatible `Storage` backend and tier object store (+ test mock) | `S3Storage`, `S3Factory`, `S3ObjectStore`, `CredentialProvider` |
| `siftp-tier` | Tiered storage: local roots as a cache over an object store (§3.2) | `TierManager`, `TieredStorage`, `restore` |
| `siftp-ldap` | LDAP identity source | `LdapProvider` |
| `siftp-admin` | Admin REST API, OpenAPI, health/readiness, metrics | `ROUTES`, `openapi_document`, `router`, `service::*` |
| `siftp-mcp` | MCP tool registry and dispatch over the admin service | `TOOLS`, `call_tool`, `serve_stdio`, `http_router` |
| `siftpd` | Binary: config, listeners, signals, subcommands | `serve`, `--check-config`, `hash-password`, `mcp-stdio`, `audit-verify`, `health`, `tier-restore` |

Dependency rule: protocol crates (`ftp`, `sftp`, `webdav`), backends
(`s3`, `ldap`), and `siftp-tier` depend on `siftp-core` (+ `siftp-asm`)
only; `siftp-mcp`
depends on `siftp-admin`; only `siftpd` depends on everything and wires
backends into the context (`register_storage`, `set_identity_provider`).
No crate may depend "upwards" or sideways (dev-dependencies excepted).

### 2.2 Concurrency model

- tokio multi-thread runtime, one task per control connection, one task
  per data transfer. Admission is shared by FTP, FTPS, SFTP, and WebDAV:
  `server.max_connections` total and `server.max_connections_per_ip`
  per source address, checked at accept together with the ban list;
  per-user `max_sessions` is enforced at login.
- Blocking filesystem and object-storage calls run on `spawn_blocking`
  or io_uring (feature `io-uring`) so runtime threads never stall.
- Shared state (config, user table, session registry, ban list,
  counters, drain flag) lives in `ServerContext`; reload swaps `Arc`
  snapshots atomically without dropping sessions.

## 3. Virtual filesystem and jail semantics

- Every user has a `virtual_root`. Client paths are *virtual*: `/` is the
  virtual root. Resolution splits on `/` (and `\` for FTP clients), drops
  empty/`.` segments, and rejects any `..` segment that would leave the
  root and any NUL byte with `invalid_path` (FTP 553 /
  SSH_FX_PERMISSION_DENIED) *before* touching storage. Reference:
  `siftp_core::Jail`.
- Local roots: after resolution, the canonical host path must still be
  inside the canonical root (symlink escape check) at use time; symlinks
  pointing outside are followed only if they land back inside.
- Creating a symlink (SFTP `SYMLINK`, local roots): a relative target
  without `..` is stored as given. Any other target is resolved as a
  virtual path, a relative one from the link's directory; a target that
  leaves the root is `invalid_path`, and one inside is stored as its host
  path under the canonical root. `READLINK` returns such targets as
  virtual paths. No link created through siftp points outside its root,
  also after the link or a directory above it is renamed, so a host
  process that follows links under the root cannot reach another user's
  files through one. Links placed on the host by other means are still
  checked at use.
- Filenames are UTF-8 (FTP `OPTS UTF8 ON` is a no-op success; FEAT
  advertises `UTF8`). Non-UTF-8 host filenames are listed lossily.
- Quotas (`quota_bytes`, 0 = unlimited) are checked on `STOR`/`APPE`,
  SFTP write, and SFTP `SETSTAT` size growth: exceeding the quota fails
  with `quota_exceeded` and removes the partial file unless the client
  was extending an existing file (`REST`, `APPE`).

### 3.1 Storage backends

Protocols program against `siftp_core::Storage`; a backend implements
metadata, listing, positional open/read/write with explicit
`commit`/`abort`, rename (with and without overwrite), mkdir/rmdir,
remove, symlinks, usage, attribute changes, free space, and a canonical
virtual path for permission checks.

| `virtual_root` | Backend | Notes |
| --- | --- | --- |
| a host path | `JailedFs` (local) | symlink-proof jail; kernel fast paths available |
| `s3://bucket/prefix` | `S3Storage` (`siftp-s3`) | `[s3]` endpoint and credentials; SigV4 presigned requests over rustls |

S3 semantics: virtual `/a/b` is object `prefix/a/b`; directories are key
prefixes, with an empty `dir/` marker object for directories created by
mkdir; reads are ranged GETs with an 8 MiB window; writes are staged in
`[s3].spool_dir` and published on commit (single PUT up to
`part_size_bytes`, multipart above) — an aborted upload publishes
nothing and the previous object survives; rename is copy + delete
(directories up to 10 000 objects); symlinks are `invalid_command`;
modification times and mode bits cannot be set (requests succeed and
are ignored); free space is reported as 1 PiB (quotas still apply).
`[s3].credentials` selects static keys, the `AWS_*` environment, or the
EC2 instance role (IMDSv2, refreshed five minutes before expiry).

### 3.2 Tiered storage

With `[tier].remote` set (`s3://bucket/prefix`, Unix only), the local
roots under `[server].root` are a cache over an object store
(`siftp-tier`).

- **Copies.** A regular file unchanged for `upload_after_secs` is
  uploaded to `objects/<32 hex>`, a random key, so renames never touch
  the store. It carries `[tier].storage_class` (instant-retrieval
  classes only) and its root-relative path as `x-amz-meta-siftp-path`.
  Hard-linked files, files open for writing, and directories on another
  filesystem are skipped.
- **State.** Each file carries one extended attribute, `user.siftp.tier`
  (58 bytes: local or remote, object id, size, mtime in ns, inode, a
  SHA-256 prefix, and the remote's id), written only through an open
  descriptor; `user.siftp.tier.read` records the last read. A file whose
  size, mtime, or inode no longer match its attribute has changed and is
  uploaded again under a new key.
- **Eviction.** A copied file nobody read or wrote for
  `evict_after_days` is replaced by a placeholder, and so are the least
  recently used ones while disk usage is above `high_watermark_percent`,
  until it is below `low_watermark_percent`. A placeholder is a sparse
  file with the same name, size, mtime, and mode and no data. The file is
  hashed again and the object's size checked first. Files below
  `min_evict_bytes`, files written within `upload_after_secs`, and files
  leased by an open writer or a pending sendfile or splice are kept.
  Listings, `SIZE`, `MDTM`, MLSD facts (except `unique`, the inode),
  quotas, rename, delete, and WebDAV ETags behave as for local files.
- **Fetch.** Opening a placeholder to read, append, or write in place,
  `local_path` (sendfile), and a SETSTAT that changes its size first
  download the object to a temporary file on the same filesystem, check
  length and hash, fsync, and rename it over the placeholder under a
  per-path lock. A truncating open replaces a placeholder without a
  fetch; STOR onto a placeholder takes the pooled path. An mtime change
  on a placeholder updates the placeholder and its attribute together.
- **No zeros uploaded.** A placeholder is never uploaded. One whose size,
  mtime, or inode changed outside siftp is quarantined: never uploaded or
  fetched over, refused with `internal` when opened, and counted. A
  non-empty file without data extents (`SEEK_DATA`) is never uploaded.
- **Guards.** `<root>/.siftp-tier/` holds an exclusive lock (one daemon
  or restore at a time), temporary files (cleared at start), and
  `root-id`, which must equal the store's `root-id`. A mismatch (an
  empty or replaced disk, or the wrong store) makes tiering read-only:
  fetches work; nothing is uploaded, evicted, or written to manifests.
  The directory is hidden from users rooted at `[server].root`. Startup
  fails if the filesystem lacks user extended attributes or, on Linux,
  hole reporting.
- **Manifests and restore.** Each scan that found a change writes
  `manifests/<time>-<hash>.jsonl` (files with keys, sizes, mtimes, modes
  and hashes; directories; symlinks); the newest 48 are kept.
  `siftpd tier-restore`, with the daemon stopped, recreates the newest
  manifest as placeholders without replacing anything that exists, puts
  objects no manifest names under `siftp-recovered/<recorded path>`, and
  adopts the store's `root-id`.
- **Nothing deleted.** Objects no file refers to are counted
  (`tier_orphans`), not deleted; the bucket's lifecycle rules expire
  noncurrent versions and incomplete uploads.
- **Worker.** One scan every `scan_interval_secs`, sooner when a
  10-second check finds disk usage above the high watermark; it stops
  between files on drain. Data movement is reported in tracing and the
  `tier_*` counters, not the audit log: it changes nothing a user sees.

## 4. Protocol matrix

| Area | Standard | Scope |
| --- | --- | --- |
| FTP core | RFC 959 | USER PASS ACCT QUIT SYST NOOP HELP SITE; PWD/XPWD CWD/XCWD CDUP/XCUP; TYPE A/I, MODE S, STRU F; PASV, PORT; RETR STOR STOU APPE REST ABOR; DELE RMD/XRMD MKD/XMKD RNFR RNTO; LIST NLST |
| FTP extensions | RFC 3659, RFC 7151, draft-somers-ftp-mfxx | MLSD MLST (facts type, size, modify, perm, unique; `OPTS MLST` selects facts), SIZE, MDTM, MFMT, TVFS; FEAT, OPTS (UTF8, MLST); HOST (virtual hosts) |
| IPv6 / NAT | RFC 2428 | EPSV (incl. `EPSV ALL`), EPRT; passive port range + external IP config |
| FTPS | RFC 4217 (+ RFC 2228 verbs) | Explicit: AUTH TLS, PBSZ 0, PROT P/C (CCC refused by policy, §10); implicit FTPS listener; TLS 1.2/1.3 via rustls; `require_tls` rejects plaintext PASS; data connections resume the control session's TLS session, enforced with `require_session_reuse` |
| SSH | RFC 4251–4254 | SSH-2.0; host keys Ed25519 (generated) or ECDSA, RSA host keys refused (§10); kex mlkem768x25519-sha256, curve25519-sha256, ecdh-sha2-nistp256, with ext-info and OpenSSH strict KEX; ciphers aes256-gcm and aes128-gcm, with chacha20-poly1305 offered first only when the CPU lacks AES and carry-less multiply instructions (clients pick it whenever it is offered, and AES-GCM is several times faster with them), or the `[sftp].ciphers` list; auth publickey (Ed25519, ECDSA, RSA-SHA2 verify-only) and password; only `session` channels with subsystem `sftp` |
| SFTP | draft-ietf-secsh-filexfer-02 (v3) | INIT/VERSION, OPEN CLOSE READ WRITE, LSTAT FSTAT STAT SETSTAT FSETSTAT, OPENDIR READDIR, REMOVE MKDIR RMDIR RENAME, READLINK SYMLINK, REALPATH; extensions posix-rename, statvfs, fstatvfs, expand-path, limits, fsync (`@openssh.com`); OpenSSH wire numbering (REALPATH 16, STAT 17, SYMLINK 20, NAME 104, ATTRS 105), SYMLINK target first; exit-status 0 sent when the subsystem ends |
| WebDAV/HTTP | RFC 4918 (class 1, read-only) | OPTIONS, PROPFIND (Depth 0/1), GET/HEAD with single Range; HTTP Basic auth; write methods refused (§4.1) |
| Admin API | OpenAPI 3.1 (§8.3) | REST JSON over HTTP/1.1 on the admin listener (loopback by default), bearer token |
| MCP | Model Context Protocol (2025-06-18) | `tools/list`, `tools/call` over stdio or streamable HTTP at `/mcp`; tools in §8.4 |

FTP details:

- Greeting `220` uses `[ftp].greeting`; multi-line replies use the
  `220-`…`220 ` convention. Command lines are capped at 4 KiB
  (`invalid_command` beyond that); verbs are case-insensitive.
- `LIST` emits Unix-style lines; `MLSD`/`MLST` are the machine-readable
  formats. `perm` facts reflect the rights for each entry's own path.
- `REST` applies to the next RETR/STOR only and resets after any transfer
  command. `TYPE A` performs CRLF translation; `TYPE I` is byte-exact and
  the only mode eligible for kernel fast paths.
- `HOST <name>` must precede `USER` (503 otherwise). With
  `[ftp].virtual_hosts` set, unknown names get 504 (`not_found`); users
  with `hosts = [...]` may log in only under one of those names.
- `MFMT YYYYMMDDHHMMSS <path>` (UTC) sets the modification time; it needs
  the `w` bit and replies `213 Modify=<time>; <path>`.
- `RNTO` replaces an existing target (rename(2) semantics).
- Passive data connections must come from the control connection's
  address unless `[ftp].passive_promiscuous = true`; others are refused
  and audited (`DATA_REFUSED`). Active mode targets only the control
  peer, on a port ≥ 1024.
- Control idle: `[server].idle_timeout_secs`; the same limit applies to
  each data-channel read and write (stalled transfers end with
  `rate_limited`), and TLS handshakes are bounded at 15 s.
- Anonymous FTP is off unless `allow_anonymous = true`; the anonymous
  user maps to `anonymous_root` with read+list permissions only.
- TLS data channels resume sessions issued on the control connection
  but never issue tickets themselves (an upload-only client would leave
  them unread and its close would reset the connection mid-file).

SFTP details:

- RENAME fails with `already_exists` if the target exists;
  `posix-rename@openssh.com` replaces it.
- `statvfs`/`fstatvfs` report the user's quota as the filesystem size
  when a quota is set (so `df` in `sftp` shows what is left).
- At most 256 open handles per session (`rate_limited` beyond);
  `limits@openssh.com` advertises 256 KiB packets and 255 KiB reads/writes.
- SETSTAT/FSETSTAT apply size, mtime, and permission bits; setuid,
  setgid, and sticky bits are stripped; uid/gid are ignored.

### 4.1 Read-only WebDAV/HTTP gateway

Enabled by `[webdav].bind` (HTTPS with `[webdav].tls = true`, using the
FTPS certificate). Users, credentials (HTTP Basic, verified through the
shared authentication path), storage, per-path permissions, rate
limits, bans, and audit are the same as for FTP/SFTP. `GET` of a file
needs `r`; `PROPFIND` and directory `GET` need `l` (HTML index, or JSON
with `Accept: application/json`). Every write method gets
`permission_denied` (403). File responses carry
`X-Content-Type-Options: nosniff` and `Content-Security-Policy: sandbox`.
Verified credentials are cached for 60 s under a keyed digest from the
login verification cache's process-only key (§5.1), with the stored
hash in the input, so a changed password or a deleted user stops
working at once.

## 5. Authentication and authorisation

### 5.1 Passwords

- Stored only as argon2id PHC strings (`$argon2id$v=19$…`), generated
  with `siftpd hash-password` or server-side by the admin API (defaults
  m=64 MiB, t=3, p=4; never below m=19 MiB, t=2, p=1 — config load
  rejects weaker hashes and `--check-config` fails).
- Verification is constant-time (argon2's verify; bearer tokens via
  `siftp_asm::ct_eq` over SHA-256 digests). Unknown and key-only users
  run a dummy verification against a hash with the default parameters,
  so response timing does not enumerate accounts whose hashes use the
  defaults. Custom-cost hashes are faster or slower to reject than
  unknown names, in proportion to their cost.
- Verification cache: after argon2id accepts a password, the server
  keeps a BLAKE2b keyed digest of `(user name, stored PHC string,
  password)` for `server.login_cache_ttl_secs` (default 300; 0 turns
  the cache off), and a repeat of exactly that credential within the
  window is accepted without recomputing argon2id. The 256-bit key is
  drawn from the OS RNG at start and lives only in process memory, so
  neither the user store nor the cache contents can be used to test
  guesses offline. Only correct credentials are ever stored; a wrong
  password, an unknown name, or a key-only user always pays one full
  argon2id computation. The cache holds at most 8192 entries and is
  cleared whenever the user table is replaced (reload or admin change);
  the PHC string in the digest means a changed password misses anyway.
  The password-age policy is applied to cache hits too. Cached logins
  are counted in `logins_cached` as well as `logins_ok`.
- Concurrency: at most `server.max_concurrent_password_checks` argon2id
  computations run at once (0, the default, means one per CPU); further
  logins and admin hash requests wait for a slot. Each computation
  holds the hash's memory while it runs, so this is the memory cap for
  password checking.
- Age: with `server.password_max_age_days` > 0, a password older than
  that (measured from the user's `password_set_unix`) fails like a wrong
  password on the wire and is logged as expired. Users without
  `password_set_unix` never expire (`--check-config` warns).
- Rotation: `POST /api/v1/users/{name}/password` (MCP
  `siftp_set_password`) sets a given password or generates a 24-character
  one and returns it once; `password_set_unix` is updated.
- Plaintext passwords are never logged or echoed, including in audit
  events and API responses.

### 5.2 SSH public keys

- `ssh_public_keys` holds OpenSSH lines (`ssh-ed25519`, `ssh-rsa`,
  `ecdsa-sha2-nistp256/384/521`), optionally prefixed by options:
  `expiry-time="YYYYMMDD[HHMM[SS]]"` (UTC; the key stops working at that
  time) and the no-op restrictions `restrict`, `no-pty`,
  `no-port-forwarding`, `no-agent-forwarding`, `no-x11-forwarding`,
  `no-user-rc`, `no-touch-required`. Any other option (`command=`,
  `from=`, …) is a config error rather than silently ignored.
- `server.revoked_keys_file` lists revoked keys (OpenSSH lines or
  `SHA256:` fingerprints, `#` comments); it is re-read on reload.
  Revoked and expired keys are refused at both the offer and the signed
  request.
- Key offers are answered without a signature check per RFC 4252 §7; the
  signature is verified on the signed request. Either a password or a
  key authenticates (no MFA).

### 5.3 Permissions, groups, and directory rules

- Permission bits: `r` read (RETR/SFTP read/WebDAV GET), `w` write
  (STOR/APPE/SFTP write/SETSTAT/MFMT/SYMLINK), `d` delete (DELE/RMD/
  REMOVE), `n` rename (both ends), `m` mkdir, `l` list (LIST/MLSD/
  READDIR/STAT/PROPFIND). Missing the bit ⇒ `permission_denied` before
  any I/O.
- Base bits: the user's `permissions`, else the union of their
  `[[groups]]` permissions, else `rl`.
- Directory rules (`[[users.directories]]`, `[[groups.directories]]`,
  `{path, permissions}`) cover a virtual path and everything below it.
  The longest matching rule wins; a user rule beats group rules at the
  same path; group rules at the same path are unioned; rules replace the
  base for their subtree (they can grant or restrict).
- The check runs on the lexical path first (no I/O); when a user has
  rules, the operation is re-checked against the symlink-resolved path,
  so a link cannot reach a restricted directory with another directory's
  rights. Reference: `siftp_core::Access`.
- A session is bound to one root and one `Access` snapshot for its
  lifetime; `CWD` changes a virtual prefix only.

### 5.4 Rate limits and throttling

- `rate_limit_bps` (per user, 0 = unlimited) token-buckets data bytes in
  both directions on every protocol; rate-limited users never take the
  kernel fast paths (which cannot pace bytes).
- Failed logins (any protocol, including wrong admin/MCP bearer tokens):
  after `max_failed_logins` failures from one IP within
  `ban_window_secs`, the IP is refused at accept time (and by the admin
  API with 429) for `ban_duration_secs`, logged as `auth_ban`. Success
  clears the counter.
- `max_connections` and `max_connections_per_ip` (default 512 / 32)
  refuse excess connections with FTP 421 `rate_limited` (SSH: closed
  before the banner).

### 5.5 Users from the durable store

See §7.1. Store users are validated exactly like config users.

### 5.6 External identity: LDAP

With `[ldap].url` set, a login name that is not a local user (config or
store) is checked against LDAP: search-then-bind (`bind_dn` + `base_dn`
+ `user_filter`, filter-escaped) or direct bind (`user_dn_template`,
DN-escaped), over `ldaps://` or StartTLS. Empty passwords are refused
before any network I/O (anonymous-bind bypass). Group DNs from
`group_attribute` map onto `[[groups]]` through `group_map`; without a
mapped group the user gets `default_permissions`, or is refused with
`require_group = true`. The virtual root is `virtual_root_template`
(`{user}` substituted) or `<server.root>/<user>`. LDAP users are
session-only (never stored) and go through the same jail, permissions,
quotas, and audit. An unreachable directory fails logins as
`auth_failed` on the wire (audited as `internal`).

## 6. Performance design

### 6.1 Zero-copy data plane (Linux)

- Plaintext `RETR`/`TYPE I` on a local root: `sendfile64` loop with
  explicit offsets on a blocking thread. `STOR`: `splice` socket→pipe→
  file, with the pipe grown to 1 MiB (`F_SETPIPE_SZ`) where the kernel
  allows it and the default 64 KiB otherwise. TLS/SSH channels and
  remote storage use the buffer pool. On a tiered root (§3.2), `RETR`
  of a placeholder fetches it before sendfile, and `STOR`/`APPE` onto a
  placeholder use the buffer pool.
- `[performance].sendfile`/`splice` disable each path; every path falls
  back automatically on `EINVAL`/`ENOSYS`/`EOPNOTSUPP` (and `EPERM` from
  seccomp) before any byte moves. Blocking sockets carry
  `SO_SNDTIMEO`/`SO_RCVTIMEO` equal to the idle timeout. The chosen path
  is counted in `/api/v1/stats` and `/metrics`.

### 6.2 io_uring (feature `io-uring`, Linux ≥ 5.6)

With the feature compiled and `[performance].io_uring = true`:
downloads read the file with `IORING_OP_READ` (after sendfile in
preference) and uploads write it with `IORING_OP_WRITE` (after splice),
one ring per transfer, one entry in flight. If ring setup fails (old
kernel, seccomp — Docker's default profile blocks io_uring), that
transfer falls back. The feature never changes wire behaviour.

### 6.3 Buffer pools and backpressure

- A byte-budget pool (`buffer_pool_bytes`, default 64 MiB) hands out
  `transfer_chunk_bytes` (default 256 KiB) buffers; acquiring waits when
  exhausted — that wait *is* the backpressure signal.
- The pooled FTP paths overlap socket and file work. An upload fills a
  whole buffer from the socket (a TLS read returns one 16 KiB record)
  before one blocking write, and that write runs while the next buffer
  fills. A download reads the next chunk on the blocking pool while the
  current one is written (and encrypted) to the socket, but only when a
  second buffer is free at once; otherwise the current chunk is sent
  first. A pooled transfer holds up to two buffers and never waits for
  one while holding another, so any pool of at least one buffer makes
  progress.
- Control channels are line-buffered with a 4 KiB cap; SFTP packets are
  capped at 256 KiB (`siftp_sftp::MAX_PACKET`) before allocation.

### 6.4 Assembly kernels (`siftp-asm`)

Sources are `.s` files under `crates/siftp-asm/asm/<arch>/`, assembled
via `build.rs` when the `asm` feature is on, `target_os ∈ {linux,
macos}`, and `target_arch ∈ {x86_64, aarch64}`. Every kernel has a pure-
Rust twin that is always compiled and tested and is the reference for
differential tests. Runtime feature detection (SSE4.2 `cpuid`, aarch64
`HWCAP_CRC32`) gates each kernel; absent the feature, Rust serves.

| Kernel | Signature `extern "C"` | Use | Rust reference |
| --- | --- | --- | --- |
| `siftp_crc32c_<arch>` | `fn(*const u8, usize, u32) -> u32` | integrity helpers | `crc32c_rust` |
| `siftp_find_newline_<arch>` | `fn(*const u8, usize) -> usize` | FTP command-line scanning | `find_newline_rust` |
| `siftp_ct_eq_<arch>` | `fn(*const u8, *const u8, usize) -> u8` | token comparison | `ct_eq_rust` |
| `siftp_find_byte_<arch>` | `fn(*const u8, usize, u8) -> usize` | byte scanning | `find_byte_rust` |
| `siftp_adler32_<arch>` | `fn(*const u8, usize, u32) -> u32` | weak checksums | `adler32_rust` |

Rules: (1) assembly is bit-identical to the Rust reference on all
inputs; (2) no kernel allocates, panics, or reads past `len`; (3)
without the feature `backend_name()` returns `"rust"`; (4) `unsafe` is
confined to the FFI shims in `siftp-asm` and the one io_uring submission
helper in `siftp-ftp::fastpath`, with its safety argument written out.
Everything else is `#![forbid(unsafe_code)]` (the FTP crate uses `deny`).

## 7. Configuration

- File format: TOML. Discovery: `--config`, else `./siftp.toml`, else
  `configs/siftp.toml`. `configs/siftp.example.toml` covers every key.
  Unknown keys are an error.
- Environment overrides: `SIFTP_<SECTION>_<KEY>`, one for every scalar
  key (list: `siftp_core::config::ENV_OVERRIDE_NAMES`; a test checks it
  against the config structs, and that `website/docs/configuration.md`
  documents every key). Precedence: env > file >
  default. `${ENV:NAME}` interpolation works inside string values
  (tokens, LDAP and S3 secrets, hashes).
- Sections: `[server]`, `[ftp]`, `[ftps]`, `[sftp]`, `[admin]`, `[mcp]`,
  `[logging]`, `[performance]`, `[s3]`, `[ldap]`, `[webdav]`,
  `[[groups]]`, `[[users]]`.
- `[[users]]` keys: `name`, `password_hash`, `password_set_unix`,
  `ssh_public_keys`, `virtual_root`, `permissions`, `groups`,
  `directories`, `hosts`, `max_sessions`, `quota_bytes`,
  `rate_limit_bps`. User names are 1–255 characters, not `.`/`..`, with
  no whitespace, `/`, `\`, `:`, or control characters.

### 7.1 Durable user store

- `[server].user_store` (default `/var/lib/siftp/users.toml`) holds users
  created or changed through the admin API or MCP, in the same
  `[[users]]` format (file mode 0600). An empty value keeps API users in
  memory only (warned).
- Precedence: config-file users are operator-owned and read-only through
  the API (`permission_denied`); a store entry with a config user's name
  is shadowed and reported by `--check-config`.
- Writes are atomic: temp file in the same directory, `fsync`, rename,
  directory `fsync`. A crash leaves the old or the new file, never a torn
  one; stray temp files are swept on the next write.
- Mutations take an exclusive OS lock on `<store>.lock`, re-read the
  file, apply the change, and write, so concurrent writers (the daemon
  and `siftpd mcp-stdio`) cannot lose updates.

### 7.2 Lifecycle

- Reload (`SIGHUP` or `POST /api/v1/reload`): re-reads config, the user
  store, and the revoked-key list; swaps snapshots; reopens the audit
  file (log rotation); reloads the FTPS certificate (a bad pair keeps the
  old one and the reload reports it). Live sessions keep their snapshot;
  listener address changes need a restart.
- Readiness: `GET /readyz` is 200 once every listener is bound and 503
  while starting or draining; `GET /healthz` is 200 while the process
  runs.
- Drain (`SIGTERM`/`SIGINT`): stop accepting on every listener, mark not
  ready, tell idle sessions `421` and close them, let transfers and open
  SFTP uploads finish, then exit 0 — bounded by
  `server.shutdown_grace_secs` (default 30). A second signal skips the
  wait.

## 8. LLM contracts

### 8.1 Structured errors — everywhere

Every error on every surface derives from `siftp_core::ApiError` and
serializes to exactly:

```json
{"code":"permission_denied","message":"user bob lacks write permission","hint":"grant the 'w' permission bit or choose a writable path","docs_url":"https://siftp.org/docs/errors#permission_denied"}
```

- `code` is from a closed, append-only set (`auth_failed`,
  `permission_denied`, `not_found`, `already_exists`, `invalid_path`,
  `invalid_command`, `rate_limited`, `quota_exceeded`, `tls_required`,
  `internal`). Clients treat unknown codes as `internal`.
- `message` may change; `hint` says what to do next; `docs_url` anchors
  into the (unversioned) error reference.
- Mapping: admin API, MCP, and WebDAV return the JSON object (HTTP 401
  auth_failed, 403 permission_denied, 404 not_found, 409 already_exists,
  400 invalid_*, 429 rate_limited, 507 quota_exceeded, 426 tls_required,
  500 internal; 413 for oversized admin bodies). FTP sends
  `ErrorCode::ftp_reply` with text `"<code>: <message>"`; SFTP sends the
  mapped `SSH_FX_*` status with `"<code>: …"`.

### 8.2 Capability document

`GET /.well-known/siftp.json` on the admin listener (unauthenticated):

```json
{"name":"siftp","version":"0.2.1","protocols":["ftp","ftps-explicit","ftps-implicit","sftp","webdav-readonly"],"docs":"https://siftp.org/v/0.2.1/docs/","llms_txt":"https://siftp.org/v/0.2.1/llms.txt","docs_latest":"https://siftp.org/docs","openapi":"/api/v1/openapi.json","mcp":"/mcp","metrics":"/metrics","health":"/healthz","ready":"/readyz"}
```

`protocols` lists what this server serves with its current config
(FTPS only with a certificate, WebDAV only when bound). `docs` and
`llms_txt` point at the documentation for this exact version (§9.1).
`mcp` is null when `[mcp].transport = "stdio"`.
An agent's boot sequence: this document → `llms_txt` → `openapi` →
operate.

### 8.3 Admin REST API + OpenAPI

Base: `[admin].bind` (default `127.0.0.1:8080`), JSON bodies up to 1 MiB,
`Authorization: Bearer <token>` on everything except
`/.well-known/siftp.json`, `/healthz`, `/readyz`, and
`/api/v1/openapi.json`. At most 64 concurrent requests (429 beyond) and
30 s per request. Responses carry `X-Content-Type-Options: nosniff`,
`Cache-Control: no-store`, `X-Frame-Options: DENY`. The route table is
`siftp_admin::ROUTES`: each route carries its parameters, request body,
success status, and response schema, and the OpenAPI document is
generated from it (every `$ref` resolves; a test enforces it).

| Method & path | Purpose |
| --- | --- |
| `GET /api/v1/users` · `POST /api/v1/users` | List / create users (create hashes a plaintext password server-side; persisted in the store) |
| `GET/PUT/DELETE /api/v1/users/{name}` | Get / replace / delete one user (`password_hash` is always `null`; config users are read-only) |
| `POST /api/v1/users/{name}/password` | Set or rotate a password (§5.1) |
| `GET /api/v1/sessions` · `DELETE /api/v1/sessions/{id}` | List / terminate sessions |
| `GET /api/v1/files?user=&path=` | Directory listing inside a user's root |
| `GET /api/v1/audit?lines=` · `GET /api/v1/audit/verify` | Tail the audit log / verify its hash chain |
| `GET /api/v1/stats` · `GET /metrics` | Counters as JSON / Prometheus text (§8.7) |
| `POST /api/v1/reload` | Reload (§7.2) |
| `GET /healthz` · `GET /readyz` | Liveness / readiness |

### 8.4 MCP tools

Transports: stdio (`siftpd mcp-stdio`, always available) and streamable
HTTP at `POST /mcp` (same bearer token and ban accounting), which
`[mcp].transport = "stdio"` turns off: requests then get `not_found`.
The setting is read per request, so a reload applies it. Results are the admin JSON
payloads; errors are the §8.1 object in an error result. Registry:
`siftp_mcp::TOOLS` (names are stable API).

| Tool | Mirrors |
| --- | --- |
| `siftp_capabilities` | §8.2 document |
| `siftp_list_users` · `siftp_get_user` | `GET /api/v1/users[/{name}]` |
| `siftp_create_user` · `siftp_update_user` · `siftp_delete_user` | `POST`/`PUT`/`DELETE` users |
| `siftp_set_password` | `POST /api/v1/users/{name}/password` |
| `siftp_list_files` | `GET /api/v1/files` |
| `siftp_list_sessions` · `siftp_kill_session` | sessions |
| `siftp_tail_audit` · `siftp_verify_audit` | audit tail / verify |
| `siftp_stats` | `GET /api/v1/stats` |
| `siftp_reload_config` | `POST /api/v1/reload` |

### 8.5 Audit log

`[logging].audit_file` is append-only JSON-lines (mode 0640):
`{"ts":<unix>,"user":…,"action":"RETR|STOR|LOGIN|…","path":…,
"result":"ok|<error code>","remote":"ip:port","seq":<n>,"chain":"<hex>"}`.
Passwords, keys, tokens, and file contents never appear. One line per:

| Event | `action` values |
| --- | --- |
| Login, success or failure | `LOGIN` (FTP, SFTP, WebDAV), `ADMIN_AUTH` (wrong admin token) |
| Ban | `auth_ban` |
| Transfer | FTP `RETR`, `STOR`, `STOU`, `APPE`; SFTP `READ`, `WRITE` (at CLOSE); WebDAV `GET` |
| File change | FTP `MKD`, `RMD`, `DELE`, `RNTO`, `MFMT`; SFTP `MKDIR`, `RMDIR`, `REMOVE`, `RENAME`, `SYMLINK`, `SETSTAT` |
| Admin change (REST or MCP; `remote` is `admin-api`) | `USER_CREATE`, `USER_REPLACE`, `USER_DELETE`, `PASSWORD_SET`, `SESSION_KILL`, `RELOAD` |
| Other | `RELOAD` on SIGHUP (`remote` is `signal`), `DATA_REFUSED` (FTP data connection from a foreign address), `PROT` with `path` `C` (FTPS data channel set to clear) |

Failed transfers and changes are logged with their error code as
`result`. `path` is the resolved virtual path (absolute, starting with
`/`) on every protocol; when a path does not resolve (an escape
attempt), it is the text the client sent. Lookups (FTP `CWD`, `SIZE`, `MDTM`, `LIST`, `MLSD`, `STAT`;
SFTP `STAT`, `REALPATH`, `OPENDIR`, `READDIR`; admin `GET` routes) are
not logged, including when they fail.

- `seq` increases by one per line for the life of the chain (across
  restarts and rotations); `chain` is
  `hex(sha256(previous chain || "\n" || line up to ",\"chain\""))`,
  starting from 64 zeros. Editing, inserting, or deleting a line breaks
  every later value; `siftpd audit-verify <file> [--anchor <chain>]`
  (exit 2 on tampering) and `GET /api/v1/audit/verify` check it.
  Truncating the end cannot be detected from the file alone.
- Rotation: logrotate `create` + reload; the chain continues into the new
  file.

### 8.6 `llms.txt` rules

`website/llms.txt` (llmstxt.org format): an H1, a blockquote summary,
then H2 sections of links to the spec, config, error, OpenAPI, and MCP
references, each with one line. Every link resolves; no marketing copy;
updated with any contract change; under ~200 lines. A copy is published
per version (§9.1).

### 8.7 Observability

- `GET /metrics` (bearer) exposes Prometheus text: every counter in
  `/api/v1/stats` as `siftp_*_total` (tiered storage under `tier` in
  the JSON, as `siftp_tier_*` in Prometheus), transfers by data path
  (`siftp_transfers_total{path=…}`), gauges for sessions, connections,
  transfers in flight, banned addresses, buffer pool use, readiness,
  start time, and `siftp_build_info`. Example alerts:
  `packaging/prometheus/alerts.yml`.
- Logs go to stderr, as text or JSON (`[logging].format`); every event
  carries its span: per FTP/SFTP connection `peer`, `session`, `user`
  (and `host` after FTP `HOST`), plus one `transfer complete|failed`
  event per transfer with verb, path, bytes, data path, and duration.

## 9. Website information architecture (siftp.org)

Static files only. Markdown files are the canonical sources and the
served `.md` mirrors; `website/build.py` (stdlib-only) renders HTML,
`sitemap.xml`, and `llms-full.txt`:

```
website/
  index.md            what siftp is, quick start, measured numbers
  llms.txt, llms-full.txt
  docs/               quickstart, configuration, protocols, storage,
                      identity, operations, llm, performance, security,
                      errors
  spec.md             verbatim copy of SPEC.md
  download.md         releases, verification, source builds
  v/<version>/        frozen copy of the docs for each release (§9.1)
```

Rules: canonical URL per topic; every error code anchorable at
`/docs/errors#<code>`; no JavaScript required; code samples runnable.

### 9.1 Versioned documentation

Each release freezes the docs under `/v/<version>/` (HTML, Markdown
mirrors, and its own `llms.txt`/`llms-full.txt`); `versions.json` lists
them. Pages under `/v/<version>/` that are not the latest carry a banner
linking to the current page. The capability document (§8.2) points
agents at the version they are talking to.

## 10. Security considerations

- **Jail first**: path resolution and permission checks precede all I/O;
  symlink escapes and symlink laundering across directory rules are
  re-checked at use (§3, §5.3). Run unprivileged; bind low ports with
  `CAP_NET_BIND_SERVICE` (the systemd unit does this).
- **Credentials**: argon2id only; SSH keys from a strict allow-list with
  expiry and revocation; admin token compared in constant time,
  required on every non-public route, and failing closed when unset;
  wrong tokens count toward bans.
- **Transport**: rustls TLS ≥ 1.2, no plaintext fallback with
  `require_tls`; optional enforced TLS session reuse on data channels;
  SSH with AEAD ciphers, post-quantum hybrid kex, and strict KEX
  (Terrapin countermeasure). RSA host keys are refused: the available
  RSA implementation is not constant-time for private-key operations
  (RUSTSEC-2023-0071); client RSA keys are verify-only.
- **Abuse resistance**: bans, shared connection caps, command/packet/
  handle caps, idle and handshake timeouts, data-peer pinning, anti-bounce
  `PORT`/`EPRT`, admin request limits, transient accept errors retried
  rather than fatal.
- **No secret leakage**: passwords, keys, tokens, and file bytes never
  enter logs, errors, or API responses; WebDAV caches only credential
  digests.
- **Supply chain**: `Cargo.lock` committed; `cargo deny` (advisories,
  licenses, bans — OpenSSL banned, sources) and `cargo audit` in CI;
  cargo-fuzz targets for FTP lines, SFTP packets, config, and paths run
  in CI and their seed corpora replay in `cargo test`; releases are
  signed (Sigstore) with SBOMs and provenance attestations.

## 11. Acceptance tests

Run in CI on Linux x86_64 and aarch64 (and macOS for the dev lane):

1. `cargo build/test/clippy -D warnings` on the workspace, also with
   `--features asm` (siftp-asm) and `--features io-uring` (siftp-ftp).
2. FTP with curl, lftp, and Python ftplib (`tests/interop/run.sh`):
   login, PASV/EPSV transfers byte-identical, MLSD/SIZE/MDTM/MFMT, REST
   resume, mirror both ways, `..` refused 553, read-only STOR 550.
3. FTPS explicit (curl, lftp, ftplib with PROT P) and implicit; with
   `require_tls`, plaintext PASS gets 534; session reuse enforced when
   configured.
4. SFTP with OpenSSH `sftp`/`scp` and a russh client: pubkey + password,
   put/get byte-identical, readdir, rename semantics, statvfs, symlinks,
   jail escape refused, oversized packet rejected, expired/revoked keys
   refused.
5. Auth/abuse: bans trip and expire; quota exceeded removes the partial
   file; caps refuse; drain completes an in-flight upload.
6. LLM surface: capability document; create a user via the admin API and
   log in over FTP; every MCP tool once over stdio; error bodies match
   §8.1; audit lines parse and the chain verifies.
7. Performance smoke (Linux): sendfile/splice/io_uring counters advance
   on real transfers; asm kernels match references; `bench/run.sh`
   reproduces the published table.
8. Config: `--check-config` accepts the substituted example and rejects
   unknown keys and weak hashes; env overrides win; reload adds a user
   without dropping a session; API users survive restart, reload, and
   SIGKILL mid-write.
9. Storage: the FTP and SFTP suites pass on S3 roots (in-process mock in
   `cargo test`; VersityGW in CI).

## 12. Roadmap

- OIDC for the admin API; writes over WebDAV; per-user bandwidth pools
  shared across sessions.
- Clustering (shared user store and ban list).
- Windows service integration (the workspace compiles for Windows; it is
  not a supported deployment target).

## 13. Stage map

Stage 1 (spec + scaffold) → config/auth core → FTP/FTPS → SSH/SFTP →
data-plane fast paths → admin API → MCP → website → production (storage
backends, identity, durability, lifecycle, observability, hardening,
packaging, CI, benchmarks). No stage merges with a red build or a
placeholder where this spec defines behaviour.