Skip to content

Media pipeline

How a recording gets from the browser to a seekable, streamable video.

1. Upload

There are three ways in, and all end with POST /api/videos/{id}/complete. The recorder uses the first two; the third is for S3 storage (the command-line client uses it).

Live, while recording (multipart)

  1. POST /api/videos with live: true reserves a video:
    • status uploading;
    • a multipart upload is started in the object store;
    • the response includes part_url.
  2. The recorder sends 5 MiB parts with PUT /api/videos/{id}/parts/{n}:
    • n runs from 1 to 10,000;
    • at most 64 MiB per part;
    • the running total is checked against the recording limit, VISTILLA_MAX_RECORDING_BYTES (4 GiB by default);
    • each part is recorded in upload_parts, and touching the video keeps it "alive".
  3. On complete, the parts must run from 1 with no gaps. They are joined in the store and the part rows are deleted.

Whole file

PUT /api/videos/{id}/blob streams the whole file:

  • S3: the file is spooled to a temporary file, then uploaded with retries.
  • Filesystem: the file is written next to its final name with a .partial extension, synced, and renamed when complete.

A whole-file upload replaces any parts already sent.

Straight to the bucket (S3 only)

POST /api/videos also returns direct_put_url, a presigned PUT URL valid for an hour. The file then goes from the client to the bucket without passing through the server.

Accepted types and limits

  • video/webm, video/mp4, video/quicktime, video/x-matroska, video/ogg.
  • Uploads in parts (live: true) are limited by VISTILLA_MAX_RECORDING_BYTES (4 GiB by default). Each part is a separate request, so a proxy's per-request limit does not apply to the whole.
  • A file sent in one request is limited by VISTILLA_MAX_UPLOAD_BYTES (2 GiB by default; 100 MB on the hosted instance, matching Cloudflare's per-request limit). The library sends files over 64 MiB in 16 MiB parts instead.

Complete

POST /api/videos/{id}/complete:

  • joins the parts of a live upload. The store's own list of parts (ListParts) is used when it holds at least as many parts as the upload_parts table (the table is the fallback if listing fails), so a part whose database row was lost (a write during a server restart) is still joined. A missing part number is refused, unless partial: true is sent: then the unbroken run from part 1 is joined and the rest left out;
  • stores the length and size (duration_ms, width, height and size_bytes), which come from the browser;
  • applies a trim, if one was asked for and is meaningful:
    • at least 1 second must be kept;
    • a trim within 250 ms of both ends is ignored;
  • sets the status to processing (trim pending) or ready;
  • queues a transcript, if the workspace has AI on. Trimmed videos are queued later, by the backfill, once the cut is done. Video replies are never transcribed;
  • returns the video and its page URL.

A poster image is uploaded separately with PUT …/thumbnail (JPEG, PNG or WebP, up to 8 MiB).

Abandoned uploads

  • Live uploads that get no new part for 7 days are discarded by the jobs worker: it picks a video that is not ready, still has a multipart_id, and was last updated over 7 days ago, aborts its upload in parts, and marks it failed and deleted.
  • Until then, the owner sees "This recording did not finish publishing" on the video page, with how much arrived and a Publish what arrived button (complete with partial: true). The length is measured by the worker, since the page that knew it is gone.
  • The recorder also deletes its reservation if the page is closed before publishing.

2. Object keys

Object Key
Video videos/{team_id}/{video_id}.{webm\|mp4\|mov\|mkv\|ogv}
Poster thumbs/{team_id}/{video_id}.{jpg\|png\|webp}
Database replica litestream/loom.db/… (hosted instance)

The API never returns the storage key as such. It does appear inside presigned URLs: direct_put_url, and the redirect Location when streams are redirected.

3. Make it seekable (seekable.rs)

Browsers' MediaRecorder writes WebM with no index ("Cues") and no duration, so the video cannot be seeked until it has fully downloaded. MP4 files often have their index at the end.

The jobs worker fixes each new video with ffmpeg, copying the streams without re-encoding:

  • WebM: -c copy -cues_to_front 1 -f webm, which puts the index at the front.
  • MP4: -c copy -movflags +faststart.

The rewritten file replaces the original under the same key. Then videos.seekable is set to 1, or to -1 if it failed; a failed video is not retried. If the recording was published without its length (duration_ms 0), the length the rewritten file states is saved.

With S3 storage, ffmpeg reads the original straight from the bucket over a signed link (re-opened if it drops), so only the rewritten copy is written to the temporary disk. That disk is small (2 GB on the hosted instance, which stops the whole server when it overflows), so a recording over 1.5 GB (750 MB if it is also trimmed, which needs two copies) is not rewritten by ffmpeg, and its trim is skipped.

Such a WebM (and one ffmpeg failed to copy) gets its index as it streams instead (webm_index.rs), with nothing written to disk. The recording is read once, front to back; from its first Cluster on, the bytes are sent straight back to the same key as the 16 MB parts of a multipart upload, while the reader notes where each cluster starts and the time of its first video keyframe. At the end a new front is built (the same EBML header and Tracks, the Info with the Duration added, a SeekHead, and Cues with one point per cluster) and sent as part 1, together with the first 16 MB of the body (the store wants every part but the last to be at least 5 MB). The store then joins the parts into the indexed file. If anything goes wrong the upload is abandoned and the recording stays as it was. A rewrite cut short by a restart leaves its parts in the store, so each run first abandons any upload still open at the recording's key (ListMultipartUploads, then AbortMultipartUpload). With folder storage the original is downloaded first, in a stream (and the guard does not count that copy). Signed links for the server's own reads last 6 hours and, like every download, go through VISTILLA_S3_DOWNLOAD_PROXY when it is set.

A download that gets no bytes for 2 minutes is given up, and the upload back has a time limit of 10 minutes plus a second per megabyte, so a stalled connection cannot hold the worker, and every video queued behind it, for ever.

Trimming

A trimmed video is cut in a second ffmpeg pass:

  • -ss start before -i, then -t length -avoid_negative_ts make_zero, without re-encoding, so the cut starts at the keyframe at or before the requested start;
  • ffprobe then finds that keyframe and measures the result: if the length differs from what is expected by more than 1.5 s, the trim is abandoned and the video is published untrimmed.

Afterwards the status becomes ready, and the trim columns are cleared.

If ffmpeg is missing, videos waiting for a trim are released untrimmed.

4. Find silences (silences.rs)

For skip silences, the worker runs:

ffmpeg -map 0:a:0 -af silencedetect=noise=-40dB:d=2 -f null -
  • Pauses of 2 seconds or more are stored as JSON [[start_ms, end_ms], …] in videos.silences.
  • If the pauses add up to more than 40% of the video, an empty list is stored, since such a video is mostly quiet on purpose. (NULL means "not checked yet".)
  • The player skips from 0.4 s after a pause starts to 0.4 s before it ends.

5. Streaming (/media/…)

GET /media/videos/{id} (signed-in, with access) and GET /media/share/{token} (share link):

  • return 409 while the video is not ready;
  • otherwise there are two modes:

Redirect (VISTILLA_STREAM_REDIRECT=1 with S3):

  • a 307 to a presigned GET URL on the bucket, valid for at least 4 hours;
  • the link is signed as of the start of the hour, so it is the same for an hour: the Worker marks what it passes on Cache-Control: private, max-age (at most an hour), and the browser answers a replay or a jump back from its own cache; the app keeps watched video too (400 MB);
  • the bytes go straight from the bucket to the browser;
  • the hosted instance uses this.

Proxy (default):

  • the server streams the object itself;
  • it supports Range requests (206 Partial Content, Accept-Ranges: bytes), so seeking works;
  • it sends Cache-Control: private, max-age=3600.

Downloads use the same paths with /download, and add a Content-Disposition file name made from the title.

Posters (/media/thumbs/{id}) are always proxied and cached for a day. With S3 the server keeps each poster it has read on its own disk (up to 256 MB, until it restarts), so a library page does not read the bucket for every card each time it opens.

6. Storage back ends (storage/)

Back end When Notes
Filesystem VISTILLA_STORAGE=fs (default) Objects under VISTILLA_FS_ROOT. Writes go to a .partial file, are synced, then renamed. Paths are validated (no ..).
S3-compatible VISTILLA_STORAGE=s3 or r2 Any S3 API: Backblaze B2, Cloudflare R2, MinIO, AWS. Requests are signed with a small built-in SigV4 implementation (no AWS SDK). Uploads, and starting, listing and finishing uploads in parts, get 4 tries in all, waiting 0.5, 1 and 2 s, on 5xx, 429, 408 and network errors. Downloads go through VISTILLA_S3_DOWNLOAD_PROXY when it is set.