record-asr/README.md

# record-asr

A self-contained **Go** HTTP transcription service built on
[sherpa-onnx](https://github.com/k2-fsa/sherpa-onnx) Go bindings, running
**SenseVoiceSmall (int8)** for recognition and a VAD for segmentation.

- Runs on **linux/amd64** and **linux/arm64**.
- Models download **at build time into the Docker image** (see [Docker](#docker))
  — or, for the standalone binary, **on first run**.
- All runtime knobs are **environment variables**; an **access password** can
  be set with `ASR_PASSWORD`.
- Uploaded tasks are **persisted to disk** (`data/tasks.json` +
  `data/uploads/`) so they survive restarts, and can be **deleted**.
- On error a task **auto-retries with exponential backoff**, up to
  `ASR_MAX_RETRIES`; the worker then **continues** to the next task. You can
  also manually **retry** or **skip (continue)** any task via the API.
- Ships a `Dockerfile` + `docker-compose.yml`; `docker buildx` produces a
  multi-arch image with no cross-compiler needed.
- Bundles an **embedded web console** (`GET /`) — drag-drop upload, live task
  queue, transcript view with per-segment timestamps, and retry/skip/delete — no
  separate frontend to deploy.

> **Note on the VAD.** The original request asked for **FSMN-VAD**. The
> current `sherpa-onnx` C API (v1.13.7) **no longer exposes FSMN-VAD** — its
> `VadModelConfig` only supports `silero_vad` and `ten_vad` (verified in the
> shipped `c-api.h` and across older tags). After confirming this with you,
> this project uses **silero-vad** as the maintained replacement. The rest of
> the spec (SenseVoiceSmall int8, Go, both arches, password, task save/delete,
> retry/continue) is implemented exactly as requested.

## Build

Requires Go ≥ 1.22 and a C compiler (cgo is used by the sherpa-onnx bindings).

```sh
make build          # portable native build -> dist/<arch>/{record-asr, lib/*.so}
```

The build bakes an `$ORIGIN/lib` rpath into the binary and copies the
sherpa-onnx / onnxruntime `.so` files next to it, so `dist/<arch>/` is a
**self-contained bundle** — copy the folder anywhere and run it.

### Both architectures

```sh
make build-amd64    # linux/amd64  (native toolchain)
make build-arm64    # linux/arm64  (cross-compile)
make dist           # build both + tarballs in dist/
```

`build-arm64` cross-compiles from amd64 and needs a C cross-compiler:

```sh
sudo apt-get install gcc-aarch64-linux-gnu   # Debian/Ubuntu
```

The arm64 `.so` files are already present in the sherpa-onnx-go-linux module,
so no extra download is needed; only the cross-compiler is required. (This
environment has no `aarch64-linux-gnu-gcc`/qemu, so only the amd64 build was
runtime-tested here; the arm64 path uses identical build tags selecting the
arm64 `.so`.)

## Docker

The multi-stage `Dockerfile` builds the binary, downloads the models **at build
time**, and bakes both into the image — so a container starts instantly and
needs no network at runtime.

```sh
docker build -t record-asr .
# Restricted-network build args (optional):
#   GOPROXY    Go module proxy            (default proxy.golang.org)
#   APT_MIRROR swap deb.debian.org for a faster mirror (e.g. mirrors.tuna.tsinghua.edu.cn)
docker build --build-arg GOPROXY=https://goproxy.cn,direct \
             --build-arg APT_MIRROR=mirrors.tuna.tsinghua.edu.cn \
             -t record-asr .
```

The image bundles **ffmpeg** (so `.m4a`/`.aac`/`.opus`/… work out of the box)
and the models (downloaded at build time).

Run a single container:

```sh
docker run -d -p 8080:8080 -e ASR_PASSWORD=changeme \
  -v record-asr-data:/app/data record-asr
```

Or with `docker compose` (sets sensible defaults; only the password is
required):

```sh
ASR_PASSWORD=changeme docker compose up -d
```

The image is **multi-arch**. Publish both platforms at once with `buildx` —
each platform builds natively, so **no cross-compiler is required** (unlike
`make build-arm64`):

```sh
docker buildx build --platform linux/amd64,linux/arm64 \
  -t record-asr:latest --push .
```

> Models (~230 MB SenseVoice + silero VAD) live at `/models` in the image.
> Only `/app/data` (tasks + uploads) is a volume; models are not on the
> volume, so they're not re-downloaded on restart. To upgrade a model, rebuild.

## Run

```sh
# Set a password (recommended); pick a data dir; launch.
ASR_PASSWORD=changeme ASR_DATA_DIR=./data ./dist/$(go env GOARCH)/record-asr
```

On first start it downloads the models into `<ASR_MODEL_DIR>/` (~230 MB
SenseVoice + ~2 MB silero VAD); subsequent starts load them from disk.

### Environment variables

| Variable | Default | Description |
|---|---|---|
| `ASR_PASSWORD` | _empty_ | Shared secret required on `/api/*`. If unset, the API is open and a warning is logged. |
| `ASR_ADDR` | `:8080` | Listen address. |
| `ASR_DATA_DIR` | `./data` | Root for `tasks.json` and `uploads/`. |
| `ASR_MODEL_DIR` | `<DataDir>/models` | Where models live / are downloaded. |
| `ASR_NUM_THREADS` | `min(4, NumCPU)` | ONNX compute threads. |
| `ASR_LANGUAGE` | `auto` | SenseVoice hint: `auto`/`zh`/`en`/`ja`/`ko`/`yue`. |
| `ASR_USE_ITN` | `1` | Inverse text normalization (numbers, punctuation). |
| `ASR_MAX_RETRIES` | `3` | Auto-retry attempts before a task is marked failed. |
| `ASR_MAX_UPLOAD_MB` | `512` | Per-upload size cap. |
| `ASR_CHUNK_SIZE_MB` | `50` | Slice size for resumable chunked uploads; kept under CDN body limits (e.g. Cloudflare's 100MB). |
| `ASR_WORKERS` | `1` | Concurrent transcription workers. |
| `ASR_VAD_THRESHOLD` | `0.5` | Silero speech probability threshold. |
| `ASR_VAD_MIN_SPEECH` | `0.5` | Min speech segment (s). |
| `ASR_VAD_MIN_SILENCE` | `0.5` | Min silence to split (s). |
| `ASR_VAD_MAX_SPEECH` | `20` | Max segment length (s); keeps SenseVoice within its window. |
| `ASR_VAD_WINDOW` | `512` | VAD window (samples @16k). |
| `ASR_VAD_BUFFER_SECS` | `60` | VAD internal buffer (s). |
| `ASR_DOWNLOAD_TIMEOUT` | `1800` | Model download timeout (s). |
| `ASR_FFMPEG` | `ffmpeg` | ffmpeg binary used to decode m4a/aac/opus/wma (beep handles wav/mp3/flac/ogg). Only needed for those formats. |
| `ASR_DECODE_TIMEOUT` | `1800` | Per-file decode timeout (beep + ffmpeg fallback) (s). |

## Web UI

Open `http://<host>:<port>/` in a browser. The page itself is public; enter the
`ASR_PASSWORD` value in the top-bar field (stored in `localStorage`, sent as
`X-Api-Key` on every API call). The console is a single embedded SPA (no
separate frontend to deploy) and covers the whole API:

- **Upload** — drag & drop or click; audio (`.wav/.mp3/.flac/.ogg/.m4a/.aac/.opus/…`) **and video** (`.mp4/.mkv/.mov/.avi/.ts/…` — the audio track is auto-extracted via ffmpeg).
- **Task list** — auto-refreshing; status dots (pending/processing/completed/
  failed/skipped), attempts, timestamps.
- **Detail** — transcript (full text + per-segment timestamps), detected
  language/emotion, duration, and Retry / Skip / Delete actions.
- **Copy** — one-click copy for the full transcript, each segment's text, and
  each segment's timestamp (时间轴).
- **Theme** — light (default) / dark toggle in the top bar; choice persisted.
- A live health dot in the top bar reflects model readiness.

## HTTP API

Auth is required on `/api/*`: send the password as `X-Api-Key`, as
`Authorization: Bearer <password>`, or as `?token=<password>`. `/`, `/healthz`
are public.

| Method | Path | Description |
|---|---|---|
| `GET` | `/` | Web console (embedded SPA). |
| `GET` | `/healthz` | Liveness + model readiness. |
| `POST` | `/api/tasks` | Upload audio (`multipart: file=`). → `202` + task. |
| `GET` | `/api/tasks` | List tasks (newest-first). |
| `GET` | `/api/tasks/{id}` | Task detail (includes `result` when done). |
| `DELETE` | `/api/tasks/{id}` | Delete task + its audio. → `204`. |
| `POST` | `/api/tasks/{id}/retry` | Requeue a failed/skipped task. → `202`. |
| `POST` | `/api/tasks/{id}/skip` | Abandon a failed task (queue moves on). → `202`. |
| `GET` | `/api/tasks/{id}/transcript` | Just the transcript (when `completed`). |

Supported audio: `.wav`, `.mp3`, `.flac`, `.ogg` (pure-Go, no external tool)
plus `.m4a`, `.aac`, `.opus`, `.wma`, `.aiff`, … and **video** `.mp4`, `.mkv`,
`.mov`, `.avi`, `.ts`, … — all decoded via **ffmpeg** (which drops the video
stream and extracts the audio track). The Docker image ships ffmpeg; the
standalone binary needs ffmpeg on `PATH` (or `ASR_FFMPEG`).

### Examples

```sh
# Upload
curl -H "X-Api-Key: changeme" -F "[email protected]" http://localhost:8080/api/tasks

# Poll
curl -H "X-Api-Key: changeme" http://localhost:8080/api/tasks/<id>

# Transcript
curl -H "X-Api-Key: changeme" http://localhost:8080/api/tasks/<id>/transcript
```

### Chunked uploads (bypassing a 100MB proxy limit)

When the service sits behind a size-capping proxy — Cloudflare's free plan
rejects request bodies over **100MB** — a single `POST /api/tasks` upload of a
large recording never reaches the origin. The resumable chunked-upload API
splits the file into slices no bigger than `ASR_CHUNK_SIZE_MB` (default **50MB**,
well under the cap) and reassembles them server-side into one task:

| Method | Path | Description |
|---|---|---|
| `POST` | `/api/uploads` | Open a session: JSON `{filename,size}` → `{upload_id, chunk_size, total_chunks}`. |
| `POST` | `/api/uploads/{id}?index=N` | Send slice `N` (0-based, raw body ≤ chunk_size). Order is free; resending is idempotent. |
| `POST` | `/api/uploads/{id}/complete` | Once all slices arrive: merge → enqueue → `202` + task (same shape as `/api/tasks`). |
| `DELETE` | `/api/uploads/{id}` | Abort the session and drop the partial file → `204`. |

```sh
# Open a session for a 180MB file (4 slices of 50MB)
UID=$(curl -sH "X-Api-Key: $PW" -H "Content-Type: application/json" \
  -d '{"filename":"big.mp4","size":188743680}' \
  http://localhost:8080/api/uploads | jq -r .upload_id)

# Send each slice (split with dd/split beforehand)
for i in 0 1 2 3; do
  curl -sH "X-Api-Key: $PW" --data-binary @"slice_$i.bin" \
    "http://localhost:8080/api/uploads/$UID?index=$i" >/dev/null
done

# Merge → enqueue (returns the task, just like /api/tasks)
curl -H "X-Api-Key: $PW" -X POST http://localhost:8080/api/uploads/$UID/complete
```

The web console handles this automatically: files larger than one chunk go
through the chunked path (with a single progress bar across all slices);
smaller files keep the simple single-request upload. Sessions live in memory,
so an interrupted upload is retried from scratch on reload — and any orphaned
partial files are swept on the next start.

## Task lifecycle & error handling

States: `pending` → `processing` → `completed` | `failed` | `skipped`.

- A background worker claims `pending` tasks, transcribes them, and records the
  result. Tasks left `processing` at crash time are reset to `pending` on next
  start (so they re-run — crash recovery).
- **Retry:** on failure, the task is requeued with exponential backoff
  (`5s, 10s, 20s, …`, capped at `60s`), up to `ASR_MAX_RETRIES` attempts; the
  worker **continues** to other tasks meanwhile. After exhausting retries the
  task is `failed`.
- **Continue / retry (manual):** `POST /api/tasks/{id}/skip` abandons a failed
  task (terminal `skipped`); `POST /api/tasks/{id}/retry` requeues it
  (`pending`, attempts reset).

## Project layout

```
main.go                      entrypoint + signal handling
internal/
  config/                    env-based config
  models/                    auto-download + load recognizer/VAD
  audio/                     wav/mp3/flac/ogg decode -> 16k mono float32
  transcribe/                VAD segmentation + SenseVoice recognition
  store/                     task persistence (tasks.json + uploads/)
  server/                    HTTP API, auth middleware, worker loop
Makefile                     portable builds for amd64 + arm64
```