evanescere/README.md

# Evanescere

Evanescere turns recorded livestreams into suggested, rendered clips:

1. Poll IIS WebDAV for finished recordings.
2. Record files visible during the first scan as a manual-only historical baseline.
3. For files first seen later, wait until file size is unchanged across polling cycles.
4. Download the stable source file to Framework-local storage.
5. Remux FLV/H264 to MP4 and extract ASR-ready audio.
6. Transcribe Mandarin audio through a FunASR-compatible API.
7. Ask DeepSeek for ranked timeline-aware clip suggestions.
8. Generate subtitles, render clips, generate thumbnails with optional local image generation and VTuber overlay, and optionally upload.

The Framework Arch server is the intended production host. The Hyper-V Arch VM can be used for development, API testing, and database/control-plane work.

## Architecture

Evanescere is split into a static frontend and an API/worker backend.

- `frontend`: React/Vite UI. It builds to static files and is served by nginx in Docker.
- `src/evanescere/api.py`: FastAPI backend. It serves JSON only.
- `src/evanescere/scheduler.py`: polling loop for IIS WebDAV.
- `src/evanescere/jobs.py`: Dramatiq queue actors backed by Redis.
- `src/evanescere/pipeline.py`: orchestration for media, ASR, LLM, render, thumbnail, and upload stages.
- `src/evanescere/services`: adapters for WebDAV, ffmpeg, FunASR, DeepSeek, subtitles, thumbnails, artifacts, and upload.
- `migrations`: Alembic migrations for PostgreSQL.

Runtime data flow:

```text
IIS WebDAV recordings
  -> first scan records existing files as manual-only baseline entries
  -> later scans poll new file sizes
  -> PostgreSQL records video state
  -> Redis queues pipeline job
  -> worker downloads source to configured storage.local_root
  -> ffmpeg remuxes/extracts audio
  -> FunASR creates timestamped transcript
  -> DeepSeek creates clip suggestions
  -> ffmpeg renders clips/subtitles/thumbnails
  -> thumbnail service composes frame/generated image + VTuber overlay + title
  -> uploader adapter runs, or noop for testing
```

The backend tracks durable state in PostgreSQL. Large media artifacts stay on the Framework SSD under `[storage].local_root`; WebDAV is used for source ingestion and can later be used for pushing final artifacts back to the Windows server.

## Configuration

Runtime configuration is TOML-based. Copy the commented example and edit your private config:

```bash
cp config.example.toml config.toml
```

The application looks for config in this order:

1. `/etc/evanescere/config.toml`
2. `./config.toml`
3. Built-in development defaults, only if no config file exists.

For Docker/Compose, mount your config file to:

```text
/etc/evanescere/config.toml
```

The checked-in `docker-compose.yml` mounts local `./config.toml` there for the API, workers, and scheduler.

## Services And Ports

- Frontend: `http://localhost:3000`
- Backend API: `http://localhost:8080`
- API docs: `http://localhost:8080/docs`
- Redis: `localhost:6379` when exposed by Compose
- PostgreSQL: only started by Compose when using the `local-db` profile

## Test Environment

Create and edit `config.toml`:

```bash
cp config.example.toml config.toml
```

For a self-contained local test stack, use the Compose PostgreSQL profile:

```bash
docker compose --profile local-db up --build
```

Run database migrations:

```bash
docker compose run --rm api alembic upgrade head
```

The scheduler automatically records files visible during its first scan as `existing_done`. Those historical recordings appear in the UI but are not automatically processed. To establish that baseline manually before starting the scheduler:

```bash
docker compose run --rm api evanescere bootstrap-existing
```

In the frontend, click `Sync WebDAV` to perform the same safe first-time baseline import or to discover later files. Select any historical FLV row and click its play button to manually queue a test pipeline run.

Useful test commands:

```bash
docker compose run --rm api evanescere scan
docker compose run --rm api evanescere videos
docker compose run --rm api evanescere run-video VIDEO_ID
docker compose logs -f api scheduler worker-media worker-ai
```

Automated tests are written with `pytest`. After installing Python dev dependencies:

```bash
pytest -q
```

Without installing dependencies locally, the available lightweight checks are:

```bash
python -m compileall src tests migrations
docker compose config --quiet
```

For safe test/debug runs, set these in `config.toml`:

```toml
[app]
log_level = "DEBUG"

[defaults]
upload_enabled = false

[upload]
adapter = "noop"
```

`adapter = "noop"` prevents accidental real uploads. `upload_enabled = false` lets you inspect rendered artifacts before adding an uploader.

## Frontend Development

The frontend reads the backend URL from `frontend/public/config.js` at runtime:

```js
window.__EVANESCERE_FRONTEND_CONFIG__ = {
  apiBaseUrl: "http://192.168.1.44:8080"
};
```

Manual frontend development requires npm packages. Install only after you approve package installation:

```bash
cd frontend
npm install
npm run dev
```

The Vite dev server runs on `http://localhost:5173`. Make sure `[app].cors_origins` includes that URL.

Static build:

```bash
cd frontend
npm run build
```

The production Compose frontend service builds the static app and serves `dist/` through nginx.

## Thumbnail Generation

Each rendered clip can produce three thumbnail artifacts:

- `thumbnail_base`: a representative frame extracted from the source MP4 near the middle of the clip.
- `thumbnail_generated`: optional output from a local image generation command.
- `thumbnail_final`: the final composed JPG with optional title and VTuber character overlay.

Default provider:

```toml
[thumbnail]
provider = "frame_overlay"
```

This uses the extracted frame as the background, then programmatically overlays the title and character PNG using Pillow.

To overlay the VTuber character, place transparent PNG variants in a directory. Evanescere picks one randomly for each thumbnail:

```toml
[thumbnail]
character_overlay_dir = "/data/evanescere/assets/"
character_scale = 0.42
character_position = "bottom-right"
```

Supported positions are `bottom-right`, `bottom-left`, `center-right`, and `center-left`.

For a local image generation model, configure a command provider:

```toml
[thumbnail]
provider = "command"
command = "/opt/evanescere-thumbnail/generate-thumbnail"
```

The command receives JSON on stdin and must create the file named by `output_path`. This makes it easy to wrap ComfyUI, Stable Diffusion, Flux, or any other local generator. Payload shape:

```json
{
  "video_id": 123,
  "clip_id": 456,
  "source_frame": "/data/evanescere/videos/123/clips/456/thumbnail_base.jpg",
  "output_path": "/data/evanescere/videos/123/clips/456/thumbnail_generated.png",
  "width": 1920,
  "height": 1080,
  "title_zh": "标题",
  "summary_zh": "摘要",
  "reason": "推荐原因",
  "tags": ["tag"],
  "transcript": [
    {"start_sec": 10.0, "end_sec": 15.0, "text": "片段台词"}
  ]
}
```

After the command writes `thumbnail_generated.png`, Evanescere still applies the same title and VTuber overlay to produce `thumbnail_final.jpg`.

## Production Deployment

On the Framework Arch server:

1. Install Docker/Compose.
2. Copy `config.example.toml` to `config.toml`.
3. Set `[database].url` to the reachable PostgreSQL server.
4. Set `[redis].url = "redis://redis:6379/0"` if using Compose Redis.
5. Set `[storage].local_root = "/data/evanescere"` and mount a Framework SSD directory there.
6. Set `[webdav]` credentials and `base_url`.
7. Set `[funasr].base_url` to the local FunASR service.
8. Set `[deepseek].api_key`.
9. Keep `[upload].adapter = "noop"` until the full pipeline has been tested.

Start production services:

```bash
docker compose up -d --build
docker compose run --rm api alembic upgrade head
docker compose run --rm api evanescere bootstrap-existing
```

Then watch logs:

```bash
docker compose logs -f api scheduler worker-media worker-ai
```

If you use your existing PostgreSQL server instead of the Compose `local-db` profile, do not start the profile. Ensure PostgreSQL accepts connections from the Framework host and that `[database].url` uses the reachable host/IP.

## Config Reference

The full commented example lives in `config.example.toml`. These are the keys the backend reads:

| Section | Key | Default | Meaning |
| --- | --- | --- | --- |
| `app` | `env` | `dev` | Free-form environment label included in logs. Use `prod` on production. |
| `app` | `log_level` | `INFO` | Python logging level. Use `DEBUG` for pipeline bring-up. |
| `app` | `log_sql` | `false` | Enables SQLAlchemy SQL logging and engine echo. Noisy in production. |
| `app` | `cors_origins` | local frontend URLs | Array of frontend origins allowed to call FastAPI. |
| `database` | `url` | local PostgreSQL URL | SQLAlchemy PostgreSQL connection string. |
| `redis` | `url` | local Redis URL | Redis broker URL for Dramatiq jobs. |
| `storage` | `local_root` | `./storage` | Root directory for downloaded sources, remuxed MP4s, audio, transcripts, subtitles, clips, and thumbnails. |
| `webdav` | `base_url` | local placeholder | IIS WebDAV directory containing recording files. |
| `webdav` | `username` | empty | WebDAV basic-auth username, if required. |
| `webdav` | `password` | empty | WebDAV basic-auth password, if required. |
| `webdav` | `verify_tls` | `true` | Whether WebDAV HTTPS certificates are verified. |
| `webdav` | `poll_interval_seconds` | `60` | Scheduler interval. A file is stable after two equal positive size samples. |
| `funasr` | `base_url` | local FunASR URL | FunASR-compatible API base URL. The client calls `/audio/transcriptions`. |
| `funasr` | `api_key` | empty | Optional bearer token for the FunASR service. |
| `funasr` | `model` | `paraformer-zh` | Model name sent to FunASR. |
| `deepseek` | `base_url` | `https://api.deepseek.com` | OpenAI-compatible DeepSeek API base URL. |
| `deepseek` | `api_key` | empty | DeepSeek API key. Required when clip suggestion is enabled. |
| `deepseek` | `model` | `deepseek-v4-pro` | Model used for clip suggestion. |
| `deepseek` | `temperature` | `0.2` | Sampling temperature for clip suggestion. |
| `llm_prompt` | `system` | VTuber clip editor prompt | Custom system prompt for clip selection. The required JSON schema is appended by code. |
| `llm_prompt` | `user` | timestamped transcript prompt | Custom user prompt template. Supports documented placeholders. |
| `defaults` | `suggest_enabled` | `true` | Initial setting for automatic LLM clip suggestion. |
| `defaults` | `render_enabled` | `true` | Initial setting for automatic rendering. |
| `defaults` | `upload_enabled` | `true` | Initial setting for automatic upload after render. For testing, set false. |
| `defaults` | `preserve_final_artifacts` | `true` | Whether rendered final artifacts are marked for preservation. |
| `defaults` | `bake_subtitles` | `true` | Whether rendered clips include hard-baked subtitles by default. |
| `clip` | `min_seconds` | `30` | Minimum LLM clip duration accepted by backend. |
| `clip` | `max_seconds` | `360` | Maximum LLM clip duration accepted by backend. |
| `clip` | `transcript_chunk_seconds` | `900` | Transcript seconds sent to DeepSeek per request. |
| `clip` | `max_candidates_total` | `20` | Maximum suggestions kept across the entire transcript after overlap deduplication. |
| `thumbnail` | `enabled` | `true` | Enables thumbnail generation during clip render. |
| `thumbnail` | `provider` | `frame_overlay` | `frame_overlay` or `command`. |
| `thumbnail` | `width` | `1920` | Final thumbnail width in pixels. |
| `thumbnail` | `height` | `1080` | Final thumbnail height in pixels. |
| `thumbnail` | `character_overlay_dir` | `/data/evanescere/assets/` | Directory of transparent VTuber PNG variants. One is selected randomly per thumbnail. |
| `thumbnail` | `character_scale` | `0.42` | Character overlay height as a fraction of final thumbnail height. |
| `thumbnail` | `character_position` | `bottom-right` | Character placement. |
| `thumbnail` | `title_enabled` | `true` | Draws the clip title onto the final thumbnail. |
| `thumbnail` | `command` | empty | Local image generation command used when `provider = "command"`. |
| `upload` | `adapter` | `noop` | `noop` or `command`. |
| `upload` | `command` | empty | Command invoked when `adapter = "command"`; receives JSON metadata on stdin. |

Frontend runtime config:

| File | Key | Meaning |
| --- | --- | --- |
| `frontend/public/config.js` | `apiBaseUrl` | URL the static frontend uses to call the backend. Change this when hosting frontend/backend on different hosts. |

## CLI

```bash
evanescere scan
evanescere bootstrap-existing
evanescere videos
evanescere run-video VIDEO_ID
evanescere transcribe VIDEO_ID
evanescere suggest VIDEO_ID
evanescere render CLIP_ID
evanescere upload CLIP_ID
```

Inside Compose, prefix commands with:

```bash
docker compose run --rm api
```

Useful API endpoints for generated files:

- `POST /webdav/scan`
- `GET /videos/{video_id}/artifacts`
- `GET /clips/{clip_id}/artifacts`
- `GET /artifacts/{artifact_id}`

Thumbnail outputs are visible through the artifact endpoints. Upload command payloads include `thumbnail` when a `thumbnail_final` artifact exists.

## Custom LLM Prompt

Edit `[llm_prompt]` in your private `config.toml` to tune clip selection without rebuilding containers:

```toml
[llm_prompt]
system = """
Your custom editorial guidance.
"""

user = """
Choose up to {max_candidates} clips between {min_clip_seconds} and {max_clip_seconds} seconds.
Stream range: {chunk_start_sec}-{chunk_end_sec}
Transcript:
{transcript}
"""
```

Available user-prompt placeholders:

- `{max_candidates}`
- `{min_clip_seconds}`
- `{max_clip_seconds}`
- `{chunk_start_sec}`
- `{chunk_end_sec}`
- `{transcript}`

The backend appends its strict JSON response schema to the system message so custom prompt wording cannot accidentally remove the machine-readable output contract.

## Debugging

Set verbose logs in `config.toml`:

```toml
[app]
log_level = "DEBUG"
log_sql = false
```

Then inspect service logs:

```bash
docker compose logs -f scheduler
docker compose logs -f worker-media
docker compose logs -f worker-ai
docker compose logs -f api
```

What to look for:

- Scheduler logs `webdav propfind done`, `new webdav file observed`, and `webdav file stable`.
- On first startup, scheduler logs `webdav baseline missing` and stores all currently visible recordings as `existing_done`.
- Worker logs `pipeline start`, `prepare media`, `transcription start`, `clip suggestion start`, `render start`, and `upload start`.
- ffmpeg command failures include the failing command and stderr tail.
- DeepSeek logs include transcript chunk bounds, character counts, candidate counts, and usage when returned by the SDK.
- Thumbnail logs include frame extraction, local image generation command execution, VTuber overlay path, and final artifact path.
- API logs include method, path, status code, and elapsed time for each frontend request.

Enable SQL logs only when diagnosing database behavior:

```toml
[app]
log_sql = true
```

This is intentionally noisy and should usually stay off in production.

## Notes

- FunASR/ROCm setup is intentionally outside the main Compose file for now. Benchmark the Framework host manually before binding the project to a specific GPU runtime.
- The uploader adapter is deliberately small. `[upload].adapter = "command"` is enough to integrate a Bilibili uploader later without changing the pipeline core.
- Existing recordings are automatically baselined on the first scheduler scan. The explicit `evanescere bootstrap-existing` command remains available when you want to establish or refresh the baseline before starting services.