# Operations

## Install

### Debian or Ubuntu package

From the apt repository ([Download](/download.html#install-with-apt)
adds it), or from a downloaded package:

```sh
sudo apt install siftpd                           # from apt.siftp.org
sudo apt install ./siftpd_0.2.1-1_amd64.deb       # or _arm64
sudo systemctl status siftpd
```

The package creates the `siftp` system user, `/srv/siftp`,
`/var/lib/siftp`, `/var/log/siftp`, a random admin token in
`/etc/siftp/siftpd.env`, and enables the hardened unit. The default
config (`/etc/siftp/siftp.toml`) serves FTP on 21, implicit FTPS on 990
(once a certificate exists), SFTP on 2222 (so the host's OpenSSH keeps
22), and the admin API on `127.0.0.1:8080`.

```sh
sudo -u siftp siftpd hash-password 'a strong password'
sudoedit /etc/siftp/siftp.toml          # or create users through the API
sudo systemctl reload siftpd
```

### Container

Release images are published to the project's GitHub Container
Registry (`ghcr.io/<owner>/siftp:<version>`, multi-arch, signed). To
build the same image locally:

```sh
docker build -f packaging/container/Dockerfile -t siftp:0.2.1 .
```

```sh
docker run -d --name siftp \
  -e SIFTP_ADMIN_TOKEN="$(openssl rand -hex 32)" \
  -e SIFTP_FTP_PASSIVE_EXTERNAL_IP=203.0.113.10 \
  -p 2121:2121 -p 2222:2222 -p 127.0.0.1:8080:8080 -p 40000-40019:40000-40019 \
  -v siftp-data:/srv/siftp -v siftp-state:/var/lib/siftp \
  siftp:0.2.1
```

The image is distroless, runs as UID 65532, and has a `HEALTHCHECK`
(`siftpd health`). Mount a certificate pair at `/etc/siftp/tls/` to
enable FTPS on 2990. Create users through the admin API:

```sh
curl -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{"name":"alice","password":"...","permissions":"rwdnml"}' \
  http://127.0.0.1:8080/api/v1/users
```

### Binary and systemd

```sh
sudo install -m 755 siftpd /usr/bin/siftpd
sudo install -m 644 packaging/systemd/siftpd.service /lib/systemd/system/
sudo useradd --system --home /var/lib/siftp --shell /usr/sbin/nologin siftp
sudo install -d -o siftp -g siftp -m 750 /srv/siftp
sudo install -d -o root -g siftp -m 750 /etc/siftp
sudo install -m 640 -g siftp packaging/siftp.toml /etc/siftp/siftp.toml
echo "SIFTP_ADMIN_TOKEN=$(openssl rand -hex 32)" | sudo install -m 640 -g siftp /dev/stdin /etc/siftp/siftpd.env
sudo systemctl daemon-reload && sudo systemctl enable --now siftpd
```

macOS: `packaging/launchd/org.siftp.siftpd.plist` (same signals; the
pooled data path is used on macOS).

## Verify a release

Every release asset has a Sigstore signature, certificate, and a build
provenance attestation; `SHA256SUMS` covers all files.

```sh
sha256sum -c SHA256SUMS --ignore-missing
cosign verify-blob siftpd-v0.2.1-x86_64-linux.tar.gz \
  --signature siftpd-v0.2.1-x86_64-linux.tar.gz.sig \
  --certificate siftpd-v0.2.1-x86_64-linux.tar.gz.pem \
  --certificate-identity https://github.com/sitevue/siftp/.github/workflows/release.yml@refs/tags/v0.2.1 \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com
gh attestation verify siftpd-v0.2.1-x86_64-linux.tar.gz --owner sitevue
cosign verify ghcr.io/sitevue/siftp:v0.2.1 \
  --certificate-identity https://github.com/sitevue/siftp/.github/workflows/release.yml@refs/tags/v0.2.1 \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com
```

The identity is the release workflow in the `sitevue/siftp` repository
at the tag you are verifying. A signature from any other repository,
workflow, or tag fails the check.

Each release also ships a CycloneDX SBOM (`*.cdx.json`).

## Service control

| Action | Command | Effect |
| --- | --- | --- |
| Reload | `systemctl reload siftpd` (SIGHUP) | Re-reads config, user store, revoked keys, TLS certificate; reopens the audit log; sessions keep running |
| Stop | `systemctl stop siftpd` (SIGTERM) | Drain: stop accepting, 421 to idle sessions, transfers finish, exit 0 within `shutdown_grace_secs` (30 s) |
| Stop now | second SIGTERM | Skip the wait |
| Validate | `siftpd --check-config --config /etc/siftp/siftp.toml` | JSON summary with warnings, or the error |

Bind address changes need a restart. Everything else reloads.

## Health and readiness

| Endpoint | Auth | 200 when | 503 when |
| --- | --- | --- | --- |
| `GET /healthz` | none | process is up | never |
| `GET /readyz` | none | all listeners bound, not draining | starting or draining |

`siftpd health --config <file>` checks `/readyz` and exits 0 or 1, for
container probes. Point load balancers at `/readyz` so a draining node
leaves rotation before it stops accepting.

## Metrics and alerts

`GET /metrics` on the admin listener, with the admin token:

```yaml
scrape_configs:
  - job_name: siftp
    authorization: { credentials_file: /etc/prometheus/siftp-token }
    static_configs: [{ targets: ["127.0.0.1:8080"] }]
```

| Metric | Type |
| --- | --- |
| `siftp_bytes_sent_total`, `siftp_bytes_received_total` | counter |
| `siftp_transfers_total{path="sendfile|splice|pool|uring"}` | counter |
| `siftp_transfers_failed_total` | counter |
| `siftp_logins_ok_total`, `siftp_logins_failed_total`, `siftp_logins_cached_total`, `siftp_bans_total` | counter |
| `siftp_connections_total`, `siftp_connections_refused_total` | counter |
| `siftp_tls_data_resumed_total`, `siftp_tls_data_full_handshake_total` | counter |
| `siftp_active_sessions`, `siftp_active_connections`, `siftp_active_transfers` | gauge |
| `siftp_banned_addresses`, `siftp_buffer_pool_outstanding_buffers` | gauge |
| `siftp_ready`, `siftp_start_time_seconds`, `siftp_build_info` | gauge |

Alert rules: `packaging/prometheus/alerts.yml` (down, not ready, login
failure spikes, bans, refused connections, transfer failures, buffer
pool exhaustion, connection count).

## Logs

Logs go to stderr (journald under systemd). `[logging].format = "json"`
gives one JSON object per line with the session span: `peer`,
`session`, `user`, and `host` (after FTP `HOST`). Every transfer logs
`transfer complete` or `transfer failed` with verb, path, bytes, data
path, and milliseconds.

## Audit log

`[logging].audit_file` holds one JSON line per login (success or
failure), ban, transfer, file change, and admin change, with `seq` and a
SHA-256 `chain` linking each line to the previous one. Lookups such as
`CWD`, `SIZE`, `LIST`, and SFTP `STAT` are not logged. SPEC §8.5 lists
every `action` value.

```json
{"ts":1760320000,"user":"alice","action":"STOR","path":"/report.pdf","result":"ok","remote":"203.0.113.7:51522","seq":42,"chain":"9f2c…"}
```

```sh
siftpd audit-verify /var/log/siftp/audit.jsonl           # exit 2 if altered
siftpd audit-verify audit.jsonl --anchor <last chain of the rotated file>
curl -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8080/api/v1/audit/verify
```

Rotation: the package installs `/etc/logrotate.d/siftpd` (daily, 90
days, `create`, then `systemctl reload`). The chain continues into the
new file. Ship lines off the host if you need to detect truncation of
the newest lines.

## SSH host key

siftpd generates an Ed25519 host key at `[sftp].host_key_file` on
first start. Publish its fingerprint so SFTP clients can check it
instead of trusting the first connection:

```sh
ssh-keygen -lf /var/lib/siftp/ssh_host_ed25519_key   # SHA256:... to publish
ssh-keygen -yf /var/lib/siftp/ssh_host_ed25519_key   # public key line
```

On a client, put the public key in `known_hosts` with the server's
name and port, then connect with strict checking:

```sh
echo "[files.example.com]:2222 ssh-ed25519 AAAAC3..." >> ~/.ssh/known_hosts
sftp -o StrictHostKeyChecking=yes -P 2222 alice@files.example.com
curl --hostpubsha256 '<base64 part of the SHA256 fingerprint>' sftp://files.example.com:2222/
```

Back the key up with the user store. A new key makes every client
warn until it updates `known_hosts`.

## Backups

| Path | Contents |
| --- | --- |
| `/var/lib/siftp/users.toml` | API-managed users (argon2id hashes) |
| `/var/lib/siftp/ssh_host_ed25519_key` | SSH host key (clients pin it) |
| `/etc/siftp/` | Config, TLS certificate and key, admin token |
| `/var/log/siftp/audit.jsonl*` | Audit trail |
| `/srv/siftp/` | User files |

The user store is replaced atomically on every change, so copying it
at any time yields a complete file.

## Tiered storage

[Tiered storage](/docs/storage.html#tiered-storage) needs a bucket that
nothing else writes to, and a role or keys that can use it.

```sh
aws s3api create-bucket --bucket example-siftp-tier \
  --create-bucket-configuration LocationConstraint=us-east-2
aws s3api put-public-access-block --bucket example-siftp-tier \
  --public-access-block-configuration BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true
aws s3api put-bucket-versioning --bucket example-siftp-tier \
  --versioning-configuration Status=Enabled
aws s3api put-bucket-lifecycle-configuration --bucket example-siftp-tier \
  --lifecycle-configuration '{"Rules":[{"ID":"siftp","Status":"Enabled","Filter":{},
    "NoncurrentVersionExpiration":{"NoncurrentDays":30},
    "AbortIncompleteMultipartUpload":{"DaysAfterInitiation":7}}]}'
```

Versioning keeps an overwritten or deleted object for 30 days. The
instance role (or the keys in `[s3]`) needs:

```json
{"Version": "2012-10-17", "Statement": [
  {"Effect": "Allow", "Action": ["s3:ListBucket"],
   "Resource": "arn:aws:s3:::example-siftp-tier"},
  {"Effect": "Allow", "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject",
     "s3:AbortMultipartUpload"],
   "Resource": "arn:aws:s3:::example-siftp-tier/*"}]}
```

- siftp writes `objects/` (file copies), `manifests/` (the newest 48
  lists of every file, directory, and link), and `root-id`. The data
  disk holds `.siftp-tier/` with the matching `root-id`, a lock, and
  temporary files.
- If the two `root-id`s differ, for example after the data disk was
  replaced or emptied, tiering turns read-only and logs an error. Stop
  siftpd and rebuild the root as placeholders:

  ```sh
  sudo systemctl stop siftpd
  sudo -u siftp siftpd tier-restore --config /etc/siftp/siftp.toml
  sudo systemctl start siftpd
  ```

  It prints a JSON report. Existing files are never replaced, and
  objects no manifest lists are placed under `siftp-recovered/`.
- Metrics: `siftp_tier_uploads_total`, `siftp_tier_evictions_total`,
  `siftp_tier_fetches_total` (with `_bytes_total` companions),
  `siftp_tier_errors_total`, and the gauges `siftp_tier_pending`,
  `siftp_tier_quarantined`, `siftp_tier_orphans`. Objects no file refers
  to are counted, not deleted.
- In a container, put `[server].root` on a volume: the overlay
  filesystem may lack extended attributes, and siftpd refuses to start
  tiering without them.

## Upgrades and rollback

1. Read the release notes; check the config with the new binary before
   switching: `siftpd-new --check-config --config /etc/siftp/siftp.toml`.
2. Install the package (or replace the binary), then
   `systemctl restart siftpd`. Restart drains first; clients reconnect.
3. Roll back by installing the previous package. The config, user
   store, host key, and audit chain are compatible within 0.x minor
   releases; new config keys default to off.
4. Versioned docs for each release stay at `/v/<version>/`.