Initial commit: _deploy_app skill

Deploy new apps or push updates to existing deployments via Docker Compose + Caddy + Gitea webhooks. Multi-server profiles, auto-detection of deployment status, full infrastructure provisioning. - SKILL.md: 715-line workflow documentation - scripts/detect_deployment.py: deployment status detection - scripts/validate_compose.py: compose file validation - references/: infrastructure, compose patterns, Caddy patterns - assets/: Makefile and compose templates - config.json: mew server profile
2026-03-25 21:12:30 -04:00
commit 994332a3f0
11 changed files with 3006 additions and 0 deletions
--- a/references/infrastructure.md
+++ b/references/infrastructure.md
@@ -0,0 +1,431 @@
+# Infrastructure Reference — mew Server (155.94.170.136)
+
+This document describes every infrastructure component on the mew server relevant to deploying Docker Compose applications behind Caddy with automated Gitea-triggered deployments.
+
+---
+
+## 1. Deploy Listener
+
+### Overview
+
+A Python webhook listener that receives push events from Gitea/Forgejo and automatically deploys the corresponding Docker Compose stack.
+
+### Filesystem Locations
+
+| Item | Path |
+|------|------|
+| Script | `/usr/local/bin/deploy-listener.py` |
+| Systemd unit | `deploy-listener.service` |
+| Deploy map | `/etc/deploy-listener/deploy-map.json` |
+| Environment file | `/etc/deploy-listener/deploy-listener.env` |
+| Service user home | `/var/lib/deploy` |
+
+### Service User
+
+- **User**: `deploy`
+- **Groups**: `docker`, `git`
+- **Home directory**: `/var/lib/deploy`
+
+The `deploy` user has Docker socket access through the `docker` group and repository access through the `git` group.
+
+### Network Binding
+
+- **Port**: 50500
+- **Bind address**: 0.0.0.0
+- **Firewall**: UFW blocks external access to port 50500. Only Docker's internal 10.0.0.0/8 range is allowed. Caddy reaches the listener at `10.0.12.1:50500` (the proxy network gateway).
+
+### Deploy Map
+
+Location: `/etc/deploy-listener/deploy-map.json`
+
+Format — a JSON object mapping `owner/repo` to the absolute path of the compose directory:
+
+```json
+{
+  "darren/compose-bookstack": "/srv/git/compose-bookstack",
+  "darren/compose-linkstack": "/srv/git/compose-linkstack",
+  "darren/my-app": "/srv/git/my-app"
+}
+```
+
+Add a new entry to this file for every application that should be auto-deployed on push.
+
+### Environment File
+
+Location: `/etc/deploy-listener/deploy-listener.env`
+
+```env
+WEBHOOK_SECRET=<the-shared-secret>
+LISTEN_PORT=50500
+```
+
+The `WEBHOOK_SECRET` value must match the secret configured in each Gitea/Forgejo webhook.
+
+### Request Validation & Behavior
+
+1. **HMAC-SHA256 validation**: The listener reads the `X-Gitea-Signature` or `X-Forgejo-Signature` header and validates the request body against the `WEBHOOK_SECRET` using HMAC-SHA256. Requests that fail validation are rejected.
+2. **Branch filter**: Only pushes to `main` or `master` (checked via the `ref` field) trigger a deploy. All other branches are ignored.
+3. **Deploy map lookup**: The `repository.full_name` field (e.g., `darren/my-app`) is looked up in the deploy map. If not found, the request is ignored.
+4. **Deploy sequence**: On a valid push, the listener executes:
+   ```bash
+   cd /srv/git/my-app
+   git pull
+   docker compose pull
+   docker compose up -d
+   ```
+5. **Concurrency control**: A file lock prevents concurrent deploys. If a deploy is already running, the incoming request is queued or rejected.
+
+### Health Check
+
+Verify the listener is running:
+
+```bash
+curl https://deploy.lavender.spl.tech/health
+```
+
+A successful response confirms the listener is reachable through Caddy and functioning.
+
+### Systemd Management
+
+```bash
+# Check status
+sudo systemctl status deploy-listener
+
+# Restart
+sudo systemctl restart deploy-listener
+
+# View logs
+sudo journalctl -u deploy-listener -f
+```
+
+---
+
+## 2. Caddy Reverse Proxy
+
+### Overview
+
+Caddy serves as the TLS-terminating reverse proxy for all applications on mew. It automatically provisions and renews certificates via Let's Encrypt.
+
+### Filesystem Locations
+
+| Item | Path |
+|------|------|
+| Caddyfile | `/data/docker/caddy/Caddyfile` |
+| Compose file | `/data/docker/caddy/docker-compose.yaml` |
+| Container name | `caddy` |
+| Image | `caddy:2-alpine` |
+
+### Network
+
+- **Network name**: `proxy`
+- **Type**: external Docker network
+- **Subnet**: 10.0.12.0/24
+- **Gateway**: 10.0.12.1
+- All application containers MUST join the `proxy` network for Caddy to reach them by container name.
+
+### TLS
+
+- **Method**: Automatic via Let's Encrypt
+- **Email**: `postmaster@lavender-daydream.com`
+- No manual certificate management required. Caddy handles provisioning, renewal, and OCSP stapling automatically.
+
+### Deploy Endpoint
+
+The deploy listener is exposed externally through Caddy:
+
+```
+deploy.lavender.spl.tech → 10.0.12.1:50500
+```
+
+This routes through the proxy network gateway to the host-bound deploy listener.
+
+### Reloading the Caddyfile
+
+**Standard reload** (when Caddyfile content changed but inode is the same):
+
+```bash
+docker exec caddy caddy reload --config /etc/caddy/Caddyfile
+```
+
+**Full restart** (required when the Caddyfile inode changed, e.g., after replacing the file rather than editing in-place):
+
+```bash
+cd /data/docker/caddy && docker compose restart caddy
+```
+
+Always check whether the file was edited in-place or replaced. If replaced, you MUST restart rather than reload.
+
+### Site Block Format
+
+Follow this exact format when adding new site blocks to the Caddyfile:
+
+```
+# === App Name ===
+domain.example.com {
+    encode zstd gzip
+    reverse_proxy container_name:port
+}
+```
+
+- Place the comment header (`# === App Name ===`) above each block for readability.
+- Always include `encode zstd gzip` for compression.
+- Use the container name (not IP) in the `reverse_proxy` directive — Caddy resolves container names on the proxy network.
+
+---
+
+## 3. Gitea API
+
+### Connection Details
+
+| Item | Value |
+|------|-------|
+| Internal URL (from mew host) | `http://10.0.12.5:3000` |
+| External URL | `https://git.lavender-daydream.com` |
+| API base path | `/api/v1` |
+| Token location | `~/.claude/secrets/gitea.json` |
+
+### Authentication
+
+Include the token as a header on every API request:
+
+```
+Authorization: token {GITEA_TOKEN}
+```
+
+### Key Endpoints
+
+#### Check if a repo exists
+
+```
+GET /api/v1/repos/{owner}/{repo}
+```
+
+- **200**: Repo exists (response includes repo details).
+- **404**: Repo does not exist.
+
+#### Create a new repo
+
+```
+POST /api/v1/user/repos
+Content-Type: application/json
+
+{
+  "name": "my-app",
+  "private": false,
+  "auto_init": false
+}
+```
+
+Set `auto_init` to `false` when pushing an existing local repo. Set to `true` if you want Gitea to create an initial commit.
+
+#### Add a webhook
+
+```
+POST /api/v1/repos/{owner}/{repo}/hooks
+Content-Type: application/json
+
+{
+  "type": "gitea",
+  "active": true,
+  "branch_filter": "main master",
+  "config": {
+    "url": "https://deploy.lavender.spl.tech/webhook",
+    "content_type": "json",
+    "secret": "<WEBHOOK_SECRET>"
+  },
+  "events": ["push"]
+}
+```
+
+The `secret` in the webhook config MUST match the `WEBHOOK_SECRET` in `/etc/deploy-listener/deploy-listener.env`.
+
+#### List repos
+
+```
+GET /api/v1/repos/search?limit=50
+```
+
+Returns up to 50 repositories. Use `page` parameter for pagination.
+
+---
+
+## 4. Forgejo API
+
+### Connection Details
+
+| Item | Value |
+|------|-------|
+| Container name | `forgejo` |
+| Internal port | 3000 |
+| External URL | `https://forgejo.lavender-daydream.com` |
+| SSH port | 2223 |
+
+### API Compatibility
+
+Forgejo is a fork of Gitea. The API format, endpoints, authentication, and request/response structures are identical to those documented in the Gitea section above. Use the same patterns — just substitute the Forgejo base URL.
+
+### SSH Access
+
+```bash
+git remote add forgejo ssh://git@forgejo.lavender-daydream.com:2223/owner/repo.git
+```
+
+---
+
+## 5. Cloudflare DNS
+
+### Token & Zone Configuration
+
+Location: `~/.claude/secrets/cloudflare.json`
+
+Format:
+
+```json
+{
+  "CLOUDFLARE_API_TOKEN": "your-api-token-here",
+  "zones": {
+    "lavender-daydream.com": "zone_id_for_lavender_daydream",
+    "spl.tech": "zone_id_for_spl_tech"
+  }
+}
+```
+
+### Authentication
+
+Include the token as a Bearer header:
+
+```
+Authorization: Bearer {CLOUDFLARE_API_TOKEN}
+```
+
+### Create an A Record
+
+```
+POST https://api.cloudflare.com/client/v4/zones/{zone_id}/dns_records
+Content-Type: application/json
+
+{
+  "type": "A",
+  "name": "{subdomain}",
+  "content": "155.94.170.136",
+  "ttl": 1,
+  "proxied": false
+}
+```
+
+- **`name`**: The subdomain portion (e.g., `myapp` for `myapp.lavender-daydream.com`, or the full FQDN).
+- **`content`**: Always `155.94.170.136` (mew's public IP).
+- **`ttl`**: `1` means automatic TTL.
+- **`proxied`**: Set to `false` so Caddy handles TLS directly. Setting to `true` would route through Cloudflare's proxy and interfere with Let's Encrypt.
+
+### Choosing the Zone
+
+Pick the zone based on the desired domain suffix:
+
+- `*.lavender-daydream.com` → use the `lavender-daydream.com` zone ID
+- `*.spl.tech` → use the `spl.tech` zone ID
+
+---
+
+## 6. Docker Networking
+
+### The `proxy` Network
+
+| Property | Value |
+|----------|-------|
+| Name | `proxy` |
+| Subnet | 10.0.12.0/24 |
+| Gateway | 10.0.12.1 |
+| Type | External (created once, referenced by all stacks) |
+
+### Requirements
+
+- **Every application container** that Caddy must reach MUST join the `proxy` network.
+- Caddy resolves container names to IPs on this network — use container names (not IPs) in `reverse_proxy` directives.
+- The network is created externally (not by any single compose file). If it does not exist, create it:
+
+```bash
+docker network create --subnet=10.0.12.0/24 --gateway=10.0.12.1 proxy
+```
+
+### Compose Configuration
+
+Every compose file that needs Caddy access must include:
+
+```yaml
+networks:
+  proxy:
+    name: proxy
+    external: true
+```
+
+And each service that Caddy proxies to must list `proxy` in its `networks` key:
+
+```yaml
+services:
+  app:
+    # ...
+    networks:
+      - proxy
+```
+
+If the stack also has internal-only services (e.g., a database), create an additional internal network:
+
+```yaml
+networks:
+  proxy:
+    name: proxy
+    external: true
+  internal:
+    driver: bridge
+```
+
+---
+
+## 7. Compose Stack Locations
+
+### Core Infrastructure Stacks
+
+Location: `/data/docker/`
+
+These are foundational services that support the entire server:
+
+| Directory | Service |
+|-----------|---------|
+| `/data/docker/caddy/` | Caddy reverse proxy |
+| `/data/docker/gitea/` | Gitea git forge |
+| `/data/docker/forgejo/` | Forgejo git forge |
+| `/data/docker/email/` | Email services |
+| `/data/docker/website/` | Main website |
+| `/data/docker/linkstack-berlyn/` | Berlyn's linkstack |
+
+### Application Stacks
+
+Location: `/srv/git/`
+
+These are deployed applications managed by the deploy listener:
+
+| Directory | Application |
+|-----------|-------------|
+| `/srv/git/compose-bookstack/` | BookStack wiki |
+| `/srv/git/compose-linkstack/` | LinkStack |
+| `/srv/git/compose-portainer/` | Portainer |
+| `/srv/git/compose-wishthis/` | WishThis |
+| `/srv/git/compose-anythingllm/` | AnythingLLM |
+
+### Ownership & Permissions
+
+- **Owner**: `root:git`
+- **Permissions**: `2775` (setgid)
+- The setgid bit ensures new files and directories inherit the `git` group, so both `root` and members of the `git` group (including `deploy` and `darren`) can read/write.
+
+### Standard Stack Contents
+
+Each compose stack directory should contain:
+
+| File | Purpose |
+|------|---------|
+| `docker-compose.yaml` | Service definitions |
+| `.env` | Environment variables (secrets, config) |
+| `Makefile` | Convenience targets (`make up`, `make down`, `make logs`) |
+| `README.md` | Stack documentation |