initial commit

This commit is contained in:
2026-05-30 23:29:44 -07:00
commit 431ffdff06
48 changed files with 4221 additions and 0 deletions
+13
View File
@@ -0,0 +1,13 @@
.env
.venv
__pycache__
.pytest_cache
.ruff_cache
.mypy_cache
*.pyc
storage
tmp
dist
build
frontend/node_modules
frontend/dist
+16
View File
@@ -0,0 +1,16 @@
.env
config.toml
.venv/
__pycache__/
.pytest_cache/
.ruff_cache/
.mypy_cache/
*.pyc
*.pyo
*.pyd
*.egg-info/
dist/
build/
storage/
tmp/
.DS_Store
+17
View File
@@ -0,0 +1,17 @@
FROM python:3.12-slim
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1
RUN apt-get update \
&& apt-get install -y --no-install-recommends ffmpeg curl fonts-noto-cjk \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY pyproject.toml README.md ./
COPY src ./src
COPY alembic.ini ./alembic.ini
COPY migrations ./migrations
RUN pip install --no-cache-dir .
CMD ["uvicorn", "evanescere.api:app", "--host", "0.0.0.0", "--port", "8000"]
+374
View File
@@ -0,0 +1,374 @@
# Evanescere
Evanescere turns recorded livestreams into suggested, rendered clips:
1. Poll IIS WebDAV for finished recordings.
2. Wait until file size is unchanged across polling cycles.
3. Download the stable source file to Framework-local storage.
4. Remux FLV/H264 to MP4 and extract ASR-ready audio.
5. Transcribe Mandarin audio through a FunASR-compatible API.
6. Ask DeepSeek for ranked timeline-aware clip suggestions.
7. Generate subtitles, render clips, generate thumbnails with optional local image generation and VTuber overlay, and optionally upload.
The Framework Arch server is the intended production host. The Hyper-V Arch VM can be used for development, API testing, and database/control-plane work.
## Architecture
Evanescere is split into a static frontend and an API/worker backend.
- `frontend`: React/Vite UI. It builds to static files and is served by nginx in Docker.
- `src/evanescere/api.py`: FastAPI backend. It serves JSON only.
- `src/evanescere/scheduler.py`: polling loop for IIS WebDAV.
- `src/evanescere/jobs.py`: Dramatiq queue actors backed by Redis.
- `src/evanescere/pipeline.py`: orchestration for media, ASR, LLM, render, thumbnail, and upload stages.
- `src/evanescere/services`: adapters for WebDAV, ffmpeg, FunASR, DeepSeek, subtitles, thumbnails, artifacts, and upload.
- `migrations`: Alembic migrations for PostgreSQL.
Runtime data flow:
```text
IIS WebDAV recordings
-> scheduler polls file size
-> PostgreSQL records video state
-> Redis queues pipeline job
-> worker downloads source to configured storage.local_root
-> ffmpeg remuxes/extracts audio
-> FunASR creates timestamped transcript
-> DeepSeek creates clip suggestions
-> ffmpeg renders clips/subtitles/thumbnails
-> thumbnail service composes frame/generated image + VTuber overlay + title
-> uploader adapter runs, or noop for testing
```
The backend tracks durable state in PostgreSQL. Large media artifacts stay on the Framework SSD under `[storage].local_root`; WebDAV is used for source ingestion and can later be used for pushing final artifacts back to the Windows server.
## Configuration
Runtime configuration is TOML-based. Copy the commented example and edit your private config:
```bash
cp config.example.toml config.toml
```
The application looks for config in this order:
1. `/etc/evanescere/config.toml`
2. `./config.toml`
3. Built-in development defaults, only if no config file exists.
For Docker/Compose, mount your config file to:
```text
/etc/evanescere/config.toml
```
The checked-in `docker-compose.yml` mounts local `./config.toml` there for the API, workers, and scheduler.
## Services And Ports
- Frontend: `http://localhost:3000`
- Backend API: `http://localhost:8000`
- API docs: `http://localhost:8000/docs`
- Redis: `localhost:6379` when exposed by Compose
- PostgreSQL: only started by Compose when using the `local-db` profile
## Test Environment
Create and edit `config.toml`:
```bash
cp config.example.toml config.toml
```
For a self-contained local test stack, use the Compose PostgreSQL profile:
```bash
docker compose --profile local-db up --build
```
Run database migrations:
```bash
docker compose run --rm api alembic upgrade head
```
Bootstrap existing WebDAV files so old recordings are marked `existing_done` instead of auto-processed:
```bash
docker compose run --rm api evanescere bootstrap-existing
```
Useful test commands:
```bash
docker compose run --rm api evanescere scan
docker compose run --rm api evanescere videos
docker compose run --rm api evanescere run-video VIDEO_ID
docker compose logs -f api scheduler worker-media worker-ai
```
Automated tests are written with `pytest`. After installing Python dev dependencies:
```bash
pytest -q
```
Without installing dependencies locally, the available lightweight checks are:
```bash
python -m compileall src tests migrations
docker compose config --quiet
```
For safe test/debug runs, set these in `config.toml`:
```toml
[app]
log_level = "DEBUG"
[defaults]
upload_enabled = false
[upload]
adapter = "noop"
```
`adapter = "noop"` prevents accidental real uploads. `upload_enabled = false` lets you inspect rendered artifacts before adding an uploader.
## Frontend Development
The frontend reads the backend URL from `frontend/public/config.js` at runtime:
```js
window.__EVANESCERE_FRONTEND_CONFIG__ = {
apiBaseUrl: "http://localhost:8000"
};
```
Manual frontend development requires npm packages. Install only after you approve package installation:
```bash
cd frontend
npm install
npm run dev
```
The Vite dev server runs on `http://localhost:5173`. Make sure `[app].cors_origins` includes that URL.
Static build:
```bash
cd frontend
npm run build
```
The production Compose frontend service builds the static app and serves `dist/` through nginx.
## Thumbnail Generation
Each rendered clip can produce three thumbnail artifacts:
- `thumbnail_base`: a representative frame extracted from the source MP4 near the middle of the clip.
- `thumbnail_generated`: optional output from a local image generation command.
- `thumbnail_final`: the final composed JPG with optional title and VTuber character overlay.
Default provider:
```toml
[thumbnail]
provider = "frame_overlay"
```
This uses the extracted frame as the background, then programmatically overlays the title and character PNG using Pillow.
To overlay the VTuber character, use a transparent PNG:
```toml
[thumbnail]
character_overlay_path = "/data/evanescere/assets/vtuber.png"
character_scale = 0.42
character_position = "bottom-right"
```
Supported positions are `bottom-right`, `bottom-left`, `center-right`, and `center-left`.
For a local image generation model, configure a command provider:
```toml
[thumbnail]
provider = "command"
command = "/opt/evanescere-thumbnail/generate-thumbnail"
```
The command receives JSON on stdin and must create the file named by `output_path`. This makes it easy to wrap ComfyUI, Stable Diffusion, Flux, or any other local generator. Payload shape:
```json
{
"video_id": 123,
"clip_id": 456,
"source_frame": "/data/evanescere/videos/123/clips/456/thumbnail_base.jpg",
"output_path": "/data/evanescere/videos/123/clips/456/thumbnail_generated.png",
"width": 1920,
"height": 1080,
"title_zh": "标题",
"summary_zh": "摘要",
"reason": "推荐原因",
"tags": ["tag"],
"transcript": [
{"start_sec": 10.0, "end_sec": 15.0, "text": "片段台词"}
]
}
```
After the command writes `thumbnail_generated.png`, Evanescere still applies the same title and VTuber overlay to produce `thumbnail_final.jpg`.
## Production Deployment
On the Framework Arch server:
1. Install Docker/Compose.
2. Copy `config.example.toml` to `config.toml`.
3. Set `[database].url` to the reachable PostgreSQL server.
4. Set `[redis].url = "redis://redis:6379/0"` if using Compose Redis.
5. Set `[storage].local_root = "/data/evanescere"` and mount a Framework SSD directory there.
6. Set `[webdav]` credentials and `base_url`.
7. Set `[funasr].base_url` to the local FunASR service.
8. Set `[deepseek].api_key`.
9. Keep `[upload].adapter = "noop"` until the full pipeline has been tested.
Start production services:
```bash
docker compose up -d --build
docker compose run --rm api alembic upgrade head
docker compose run --rm api evanescere bootstrap-existing
```
Then watch logs:
```bash
docker compose logs -f api scheduler worker-media worker-ai
```
If you use your existing PostgreSQL server instead of the Compose `local-db` profile, do not start the profile. Ensure PostgreSQL accepts connections from the Framework host and that `[database].url` uses the reachable host/IP.
## Config Reference
The full commented example lives in `config.example.toml`. These are the keys the backend reads:
| Section | Key | Default | Meaning |
| --- | --- | --- | --- |
| `app` | `env` | `dev` | Free-form environment label included in logs. Use `prod` on production. |
| `app` | `log_level` | `INFO` | Python logging level. Use `DEBUG` for pipeline bring-up. |
| `app` | `log_sql` | `false` | Enables SQLAlchemy SQL logging and engine echo. Noisy in production. |
| `app` | `cors_origins` | local frontend URLs | Array of frontend origins allowed to call FastAPI. |
| `database` | `url` | local PostgreSQL URL | SQLAlchemy PostgreSQL connection string. |
| `redis` | `url` | local Redis URL | Redis broker URL for Dramatiq jobs. |
| `storage` | `local_root` | `./storage` | Root directory for downloaded sources, remuxed MP4s, audio, transcripts, subtitles, clips, and thumbnails. |
| `webdav` | `base_url` | local placeholder | IIS WebDAV directory containing recording files. |
| `webdav` | `username` | empty | WebDAV basic-auth username, if required. |
| `webdav` | `password` | empty | WebDAV basic-auth password, if required. |
| `webdav` | `verify_tls` | `true` | Whether WebDAV HTTPS certificates are verified. |
| `webdav` | `poll_interval_seconds` | `60` | Scheduler interval. A file is stable after two equal positive size samples. |
| `funasr` | `base_url` | local FunASR URL | FunASR-compatible API base URL. The client calls `/audio/transcriptions`. |
| `funasr` | `api_key` | empty | Optional bearer token for the FunASR service. |
| `funasr` | `model` | `paraformer-zh` | Model name sent to FunASR. |
| `deepseek` | `base_url` | `https://api.deepseek.com` | OpenAI-compatible DeepSeek API base URL. |
| `deepseek` | `api_key` | empty | DeepSeek API key. Required when clip suggestion is enabled. |
| `deepseek` | `model` | `deepseek-v4-pro` | Model used for clip suggestion. |
| `deepseek` | `temperature` | `0.2` | Sampling temperature for clip suggestion. |
| `defaults` | `suggest_enabled` | `true` | Initial setting for automatic LLM clip suggestion. |
| `defaults` | `render_enabled` | `true` | Initial setting for automatic rendering. |
| `defaults` | `upload_enabled` | `true` | Initial setting for automatic upload after render. For testing, set false. |
| `defaults` | `preserve_final_artifacts` | `true` | Whether rendered final artifacts are marked for preservation. |
| `defaults` | `bake_subtitles` | `true` | Whether rendered clips include hard-baked subtitles by default. |
| `clip` | `min_seconds` | `30` | Minimum LLM clip duration accepted by backend. |
| `clip` | `max_seconds` | `360` | Maximum LLM clip duration accepted by backend. |
| `clip` | `transcript_chunk_seconds` | `900` | Transcript seconds sent to DeepSeek per request. |
| `thumbnail` | `enabled` | `true` | Enables thumbnail generation during clip render. |
| `thumbnail` | `provider` | `frame_overlay` | `frame_overlay` or `command`. |
| `thumbnail` | `width` | `1920` | Final thumbnail width in pixels. |
| `thumbnail` | `height` | `1080` | Final thumbnail height in pixels. |
| `thumbnail` | `character_overlay_path` | empty | Optional transparent PNG of the VTuber character. |
| `thumbnail` | `character_scale` | `0.42` | Character overlay height as a fraction of final thumbnail height. |
| `thumbnail` | `character_position` | `bottom-right` | Character placement. |
| `thumbnail` | `title_enabled` | `true` | Draws the clip title onto the final thumbnail. |
| `thumbnail` | `command` | empty | Local image generation command used when `provider = "command"`. |
| `upload` | `adapter` | `noop` | `noop` or `command`. |
| `upload` | `command` | empty | Command invoked when `adapter = "command"`; receives JSON metadata on stdin. |
Frontend runtime config:
| File | Key | Meaning |
| --- | --- | --- |
| `frontend/public/config.js` | `apiBaseUrl` | URL the static frontend uses to call the backend. Change this when hosting frontend/backend on different hosts. |
## CLI
```bash
evanescere scan
evanescere bootstrap-existing
evanescere videos
evanescere run-video VIDEO_ID
evanescere transcribe VIDEO_ID
evanescere suggest VIDEO_ID
evanescere render CLIP_ID
evanescere upload CLIP_ID
```
Inside Compose, prefix commands with:
```bash
docker compose run --rm api
```
Useful API endpoints for generated files:
- `GET /videos/{video_id}/artifacts`
- `GET /clips/{clip_id}/artifacts`
- `GET /artifacts/{artifact_id}`
Thumbnail outputs are visible through the artifact endpoints. Upload command payloads include `thumbnail` when a `thumbnail_final` artifact exists.
## Debugging
Set verbose logs in `config.toml`:
```toml
[app]
log_level = "DEBUG"
log_sql = false
```
Then inspect service logs:
```bash
docker compose logs -f scheduler
docker compose logs -f worker-media
docker compose logs -f worker-ai
docker compose logs -f api
```
What to look for:
- Scheduler logs `webdav propfind done`, `new webdav file observed`, and `webdav file stable`.
- Worker logs `pipeline start`, `prepare media`, `transcription start`, `clip suggestion start`, `render start`, and `upload start`.
- ffmpeg command failures include the failing command and stderr tail.
- DeepSeek logs include transcript chunk bounds, character counts, candidate counts, and usage when returned by the SDK.
- Thumbnail logs include frame extraction, local image generation command execution, VTuber overlay path, and final artifact path.
- API logs include method, path, status code, and elapsed time for each frontend request.
Enable SQL logs only when diagnosing database behavior:
```toml
[app]
log_sql = true
```
This is intentionally noisy and should usually stay off in production.
## Notes
- FunASR/ROCm setup is intentionally outside the main Compose file for now. Benchmark the Framework host manually before binding the project to a specific GPU runtime.
- The uploader adapter is deliberately small. `[upload].adapter = "command"` is enough to integrate a Bilibili uploader later without changing the pipeline core.
- Existing recordings should be bootstrapped before scheduler-driven production runs, otherwise old stable files may be queued as new work.
+39
View File
@@ -0,0 +1,39 @@
[alembic]
script_location = migrations
prepend_sys_path = .
sqlalchemy.url = postgresql+psycopg://evanescere:evanescere@localhost:5432/evanescere
[loggers]
keys = root,sqlalchemy,alembic
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARN
handlers = console
qualname =
[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S
+126
View File
@@ -0,0 +1,126 @@
# Evanescere configuration file.
#
# Copy this file to config.toml for local Compose runs:
#
# cp config.example.toml config.toml
#
# In containers, mount your config to:
#
# /etc/evanescere/config.toml
#
# Secrets such as DeepSeek and WebDAV credentials belong in your private config.toml.
[app]
# Free-form environment label included in logs.
env = "dev"
# Python logging level: DEBUG, INFO, WARNING, ERROR.
log_level = "DEBUG"
# Enable SQLAlchemy SQL logging. Very noisy; keep false unless debugging DB behavior.
log_sql = false
# Frontend origins allowed to call FastAPI. Include the Vite dev server while developing.
cors_origins = ["http://localhost:3000", "http://localhost:5173"]
[database]
# PostgreSQL connection string used by SQLAlchemy.
# For Compose local-db profile: postgresql+psycopg://evanescere:evanescere@postgres:5432/evanescere
# For an existing DB: postgresql+psycopg://USER:PASSWORD@HOST:5432/DBNAME
url = "postgresql+psycopg://evanescere:evanescere@postgres:5432/evanescere"
[redis]
# Redis URL used by Dramatiq workers.
url = "redis://redis:6379/0"
[storage]
# Container path for working media. Mount a Framework SSD directory here in Compose.
local_root = "/data/evanescere"
[webdav]
# IIS WebDAV directory containing recording files. Point at the collection, not one file.
base_url = "https://windows-server.example.local/webdav/recordings"
# Leave username/password empty if your WebDAV endpoint does not require basic auth.
username = ""
password = ""
# Set false only for trusted internal/self-signed testing.
verify_tls = true
# Scheduler interval. A file becomes stable after two equal positive size samples.
poll_interval_seconds = 60
[funasr]
# FunASR-compatible API base URL. Evanescere calls /audio/transcriptions.
base_url = "http://funasr:10096/v1"
# Optional bearer token for the FunASR service.
api_key = ""
# Model name sent to FunASR. Adjust this to match your deployed service.
model = "paraformer-zh"
[deepseek]
# OpenAI-compatible DeepSeek API base URL.
base_url = "https://api.deepseek.com"
# Required when clip suggestion is enabled.
api_key = ""
# Initial MVP model.
model = "deepseek-v4-pro"
# Lower values improve consistency.
temperature = 0.2
[defaults]
# Initial automatic pipeline settings. These can later be changed through the API/UI.
suggest_enabled = true
render_enabled = true
# For testing, consider false until uploads are wired and verified.
upload_enabled = true
preserve_final_artifacts = true
bake_subtitles = true
[clip]
# LLM clip duration bounds accepted by the backend.
min_seconds = 30
max_seconds = 360
# Transcript seconds per DeepSeek request. Larger chunks use more tokens.
transcript_chunk_seconds = 900
[thumbnail]
# Generate thumbnails during clip render.
enabled = true
# frame_overlay: use extracted video frame as background.
# command: run command below to call a local image generation workflow.
provider = "frame_overlay"
width = 1920
height = 1080
# Optional transparent PNG of the VTuber character.
character_overlay_path = ""
# Character height as a fraction of thumbnail height.
character_scale = 0.42
# Supported: bottom-right, bottom-left, center-right, center-left.
character_position = "bottom-right"
# Draw the clip title onto the final thumbnail.
title_enabled = true
# Used only when provider = "command". Receives JSON on stdin and must write output_path.
command = ""
[upload]
# noop: do not upload, only record what would have happened.
# command: run command below with JSON metadata on stdin.
adapter = "noop"
command = ""
+65
View File
@@ -0,0 +1,65 @@
services:
frontend:
build: ./frontend
ports:
- "3000:80"
depends_on:
- api
api:
build: .
command: uvicorn evanescere.api:app --host 0.0.0.0 --port 8000
ports:
- "8000:8000"
volumes:
- ./config.toml:/etc/evanescere/config.toml:ro
- ./storage:/data/evanescere
depends_on:
- redis
worker-media:
build: .
command: dramatiq evanescere.jobs
volumes:
- ./config.toml:/etc/evanescere/config.toml:ro
- ./storage:/data/evanescere
depends_on:
- redis
worker-ai:
build: .
command: dramatiq evanescere.jobs
volumes:
- ./config.toml:/etc/evanescere/config.toml:ro
- ./storage:/data/evanescere
depends_on:
- redis
scheduler:
build: .
command: python -m evanescere.scheduler
volumes:
- ./config.toml:/etc/evanescere/config.toml:ro
- ./storage:/data/evanescere
depends_on:
- redis
redis:
image: redis:7-alpine
ports:
- "6379:6379"
postgres:
image: postgres:17-alpine
profiles: ["local-db"]
environment:
POSTGRES_USER: evanescere
POSTGRES_PASSWORD: evanescere
POSTGRES_DB: evanescere
ports:
- "5432:5432"
volumes:
- pgdata:/var/lib/postgresql/data
volumes:
pgdata:
+5
View File
@@ -0,0 +1,5 @@
node_modules
dist
.vite
npm-debug.log
+16
View File
@@ -0,0 +1,16 @@
FROM node:22-alpine AS build
WORKDIR /app
COPY package.json ./
RUN npm install
COPY index.html tsconfig.json vite.config.ts ./
COPY public ./public
COPY src ./src
RUN npm run build
FROM nginx:1.27-alpine
COPY nginx.conf /etc/nginx/conf.d/default.conf
COPY --from=build /app/dist /usr/share/nginx/html
EXPOSE 80
+14
View File
@@ -0,0 +1,14 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Evanescere</title>
<script src="/config.js"></script>
</head>
<body>
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
</body>
</html>
+16
View File
@@ -0,0 +1,16 @@
server {
listen 80;
server_name _;
root /usr/share/nginx/html;
index index.html;
location / {
try_files $uri $uri/ /index.html;
}
location = /config.js {
add_header Cache-Control "no-store";
try_files $uri =404;
}
}
+23
View File
@@ -0,0 +1,23 @@
{
"name": "evanescere-frontend",
"private": true,
"version": "0.1.0",
"type": "module",
"scripts": {
"dev": "vite",
"build": "tsc -b && vite build",
"preview": "vite preview --host 0.0.0.0"
},
"dependencies": {
"lucide-react": "^0.468.0",
"react": "^19.0.0",
"react-dom": "^19.0.0"
},
"devDependencies": {
"@types/react": "^19.0.0",
"@types/react-dom": "^19.0.0",
"@vitejs/plugin-react": "^4.3.4",
"typescript": "^5.7.0",
"vite": "^6.0.0"
}
}
+3
View File
@@ -0,0 +1,3 @@
window.__EVANESCERE_FRONTEND_CONFIG__ = {
apiBaseUrl: "http://localhost:8000"
};
+304
View File
@@ -0,0 +1,304 @@
import {
Check,
Clapperboard,
Play,
RefreshCw,
Save,
Send,
Settings2,
Upload,
} from "lucide-react";
import { useCallback, useEffect, useMemo, useState } from "react";
import {
apiBaseUrl,
approveClip,
getClips,
getSettings,
getTranscript,
getVideos,
patchSettings,
renderClip,
runVideo,
uploadClip,
} from "./api";
import type { ClipSuggestion, PipelineSettings, TranscriptSegment, Video } from "./types";
const settingLabels: Record<keyof PipelineSettings, string> = {
suggest_enabled: "Suggest",
render_enabled: "Render",
upload_enabled: "Upload",
preserve_final_artifacts: "Preserve",
bake_subtitles: "Bake subs",
};
function formatDuration(seconds: number | null) {
if (seconds === null) return "";
const total = Math.round(seconds);
const hours = Math.floor(total / 3600);
const minutes = Math.floor((total % 3600) / 60);
const secs = total % 60;
return hours > 0
? `${hours}:${minutes.toString().padStart(2, "0")}:${secs.toString().padStart(2, "0")}`
: `${minutes}:${secs.toString().padStart(2, "0")}`;
}
function formatTimeRange(start: number, end: number) {
return `${formatDuration(start)}-${formatDuration(end)}`;
}
function statusTone(value: string) {
if (["done", "stable", "approved", "auto_approved"].includes(value)) return "good";
if (["failed", "error"].includes(value)) return "bad";
if (["running", "queued", "observing", "pending"].includes(value)) return "busy";
return "neutral";
}
export function App() {
const [videos, setVideos] = useState<Video[]>([]);
const [settings, setSettings] = useState<PipelineSettings | null>(null);
const [selectedVideoId, setSelectedVideoId] = useState<number | null>(null);
const [transcript, setTranscript] = useState<TranscriptSegment[]>([]);
const [clips, setClips] = useState<ClipSuggestion[]>([]);
const [loading, setLoading] = useState(false);
const [error, setError] = useState<string | null>(null);
const selectedVideo = useMemo(
() => videos.find((video) => video.id === selectedVideoId) ?? null,
[selectedVideoId, videos],
);
const loadSelected = useCallback(async (videoId: number) => {
const [nextTranscript, nextClips] = await Promise.all([getTranscript(videoId), getClips(videoId)]);
setTranscript(nextTranscript);
setClips(nextClips);
}, []);
const refresh = useCallback(async () => {
setLoading(true);
setError(null);
try {
const [nextSettings, nextVideos] = await Promise.all([getSettings(), getVideos()]);
setSettings(nextSettings);
setVideos(nextVideos);
const targetVideoId = selectedVideoId ?? nextVideos[0]?.id ?? null;
setSelectedVideoId(targetVideoId);
if (targetVideoId !== null) {
await loadSelected(targetVideoId);
} else {
setTranscript([]);
setClips([]);
}
} catch (caught) {
setError(caught instanceof Error ? caught.message : "Unknown error");
} finally {
setLoading(false);
}
}, [loadSelected, selectedVideoId]);
useEffect(() => {
void refresh();
}, [refresh]);
async function handleSelect(videoId: number) {
setSelectedVideoId(videoId);
setLoading(true);
setError(null);
try {
await loadSelected(videoId);
} catch (caught) {
setError(caught instanceof Error ? caught.message : "Unknown error");
} finally {
setLoading(false);
}
}
async function withRefresh(action: () => Promise<unknown>) {
setLoading(true);
setError(null);
try {
await action();
await refresh();
} catch (caught) {
setError(caught instanceof Error ? caught.message : "Unknown error");
setLoading(false);
}
}
async function saveSettings() {
if (!settings) return;
await withRefresh(() => patchSettings(settings));
}
return (
<main className="app-shell">
<header className="topbar">
<div>
<h1>Evanescere</h1>
<p>{apiBaseUrl}</p>
</div>
<div className="toolbar">
<button type="button" className="icon-button" onClick={() => void refresh()} disabled={loading}>
<RefreshCw size={18} />
Refresh
</button>
<button type="button" className="icon-button primary" onClick={() => void saveSettings()} disabled={loading || !settings}>
<Save size={18} />
Save
</button>
</div>
</header>
{error && <div className="error-strip">{error}</div>}
<section className="band controls-band">
<div className="section-title">
<Settings2 size={18} />
<h2>Pipeline</h2>
</div>
<div className="toggle-row">
{settings &&
(Object.keys(settingLabels) as Array<keyof PipelineSettings>).map((key) => (
<label className="switch" key={key}>
<input
type="checkbox"
checked={settings[key]}
onChange={(event) =>
setSettings((current) =>
current ? { ...current, [key]: event.currentTarget.checked } : current,
)
}
/>
<span>{settingLabels[key]}</span>
</label>
))}
</div>
</section>
<section className="workbench">
<aside className="video-list">
<div className="section-title">
<Clapperboard size={18} />
<h2>Videos</h2>
</div>
<div className="table-scroll">
<table>
<thead>
<tr>
<th>ID</th>
<th>File</th>
<th>Status</th>
<th></th>
</tr>
</thead>
<tbody>
{videos.map((video) => (
<tr key={video.id} className={video.id === selectedVideoId ? "selected" : ""}>
<td>{video.id}</td>
<td>
<button type="button" className="link-button" onClick={() => void handleSelect(video.id)}>
{video.filename}
</button>
<div className="muted">{formatDuration(video.duration_sec)}</div>
</td>
<td>
<StatusPill value={video.ingest_status} />
<StatusPill value={video.processing_status} />
</td>
<td>
<button
type="button"
className="square-button"
title="Run pipeline"
onClick={() => void withRefresh(() => runVideo(video.id))}
>
<Play size={17} />
</button>
</td>
</tr>
))}
</tbody>
</table>
</div>
</aside>
<section className="detail-pane">
<div className="detail-header">
<div>
<h2>{selectedVideo?.filename ?? "No video selected"}</h2>
<p>{selectedVideo?.source_url ?? ""}</p>
</div>
<div className="metric-row">
<Metric label="Transcript" value={transcript.length.toString()} />
<Metric label="Clips" value={clips.length.toString()} />
</div>
</div>
<div className="split">
<div className="transcript-pane">
<h3>Transcript</h3>
<div className="transcript-lines">
{transcript.slice(0, 180).map((segment) => (
<div className="transcript-line" key={segment.id}>
<span>{formatTimeRange(segment.start_sec, segment.end_sec)}</span>
<p>{segment.text}</p>
</div>
))}
</div>
</div>
<div className="clips-pane">
<h3>Clips</h3>
<div className="clip-list">
{clips.map((clip) => (
<article className="clip-item" key={clip.id}>
<div className="clip-main">
<div>
<h4>{clip.title_zh}</h4>
<p>{clip.summary_zh}</p>
</div>
<strong>{Math.round(clip.score * 100)}</strong>
</div>
<div className="clip-meta">
<span>{formatTimeRange(clip.start_sec, clip.end_sec)}</span>
<StatusPill value={clip.approval_status} />
<StatusPill value={clip.render_status} />
<StatusPill value={clip.upload_status} />
</div>
<div className="toolbar compact">
<button type="button" className="icon-button" onClick={() => void withRefresh(() => approveClip(clip.id))}>
<Check size={16} />
Approve
</button>
<button type="button" className="icon-button" onClick={() => void withRefresh(() => renderClip(clip.id))}>
<Send size={16} />
Render
</button>
<button type="button" className="icon-button" onClick={() => void withRefresh(() => uploadClip(clip.id))}>
<Upload size={16} />
Upload
</button>
</div>
</article>
))}
</div>
</div>
</div>
</section>
</section>
</main>
);
}
function StatusPill({ value }: { value: string }) {
return <span className={`status-pill ${statusTone(value)}`}>{value}</span>;
}
function Metric({ label, value }: { label: string; value: string }) {
return (
<div className="metric">
<span>{label}</span>
<strong>{value}</strong>
</div>
);
}
+66
View File
@@ -0,0 +1,66 @@
import type { Artifact, ClipSuggestion, PipelineSettings, TranscriptSegment, Video } from "./types";
const runtimeApiBase = window.__EVANESCERE_FRONTEND_CONFIG__?.apiBaseUrl;
export const apiBaseUrl = (runtimeApiBase || "http://localhost:8000").replace(
/\/$/,
"",
);
async function request<T>(path: string, init: RequestInit = {}): Promise<T> {
const response = await fetch(`${apiBaseUrl}${path}`, {
...init,
headers: {
"Content-Type": "application/json",
...init.headers,
},
});
if (!response.ok) {
const text = await response.text();
throw new Error(text || `${response.status} ${response.statusText}`);
}
return response.json() as Promise<T>;
}
export function getSettings(): Promise<PipelineSettings> {
return request<PipelineSettings>("/settings");
}
export function patchSettings(settings: PipelineSettings): Promise<PipelineSettings> {
return request<PipelineSettings>("/settings", {
method: "PATCH",
body: JSON.stringify(settings),
});
}
export function getVideos(): Promise<Video[]> {
return request<Video[]>("/videos");
}
export function runVideo(videoId: number) {
return request(`/videos/${videoId}/run`, { method: "POST" });
}
export function getTranscript(videoId: number): Promise<TranscriptSegment[]> {
return request<TranscriptSegment[]>(`/videos/${videoId}/transcript`);
}
export function getClips(videoId: number): Promise<ClipSuggestion[]> {
return request<ClipSuggestion[]>(`/videos/${videoId}/clips`);
}
export function getClipArtifacts(clipId: number): Promise<Artifact[]> {
return request<Artifact[]>(`/clips/${clipId}/artifacts`);
}
export function approveClip(clipId: number): Promise<ClipSuggestion> {
return request<ClipSuggestion>(`/clips/${clipId}/approve`, { method: "POST" });
}
export function renderClip(clipId: number): Promise<ClipSuggestion> {
return request<ClipSuggestion>(`/clips/${clipId}/render`, { method: "POST" });
}
export function uploadClip(clipId: number): Promise<ClipSuggestion> {
return request<ClipSuggestion>(`/clips/${clipId}/upload`, { method: "POST" });
}
+11
View File
@@ -0,0 +1,11 @@
import { StrictMode } from "react";
import { createRoot } from "react-dom/client";
import { App } from "./App";
import "./styles.css";
createRoot(document.getElementById("root")!).render(
<StrictMode>
<App />
</StrictMode>,
);
+434
View File
@@ -0,0 +1,434 @@
* {
box-sizing: border-box;
}
:root {
color: #1f252b;
background: #f4f6f8;
font-family:
Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
font-synthesis: none;
text-rendering: optimizeLegibility;
}
body {
margin: 0;
min-width: 320px;
min-height: 100vh;
background: #f4f6f8;
}
button,
input {
font: inherit;
}
button {
white-space: nowrap;
}
.app-shell {
width: min(1440px, 100%);
margin: 0 auto;
padding: 20px;
}
.topbar {
display: flex;
align-items: center;
justify-content: space-between;
gap: 16px;
padding: 8px 0 20px;
}
.topbar h1 {
margin: 0;
color: #18222b;
font-size: 28px;
font-weight: 720;
}
.topbar p,
.detail-header p,
.muted {
margin: 4px 0 0;
color: #66737f;
font-size: 13px;
overflow-wrap: anywhere;
}
.toolbar {
display: flex;
align-items: center;
gap: 8px;
flex-wrap: wrap;
}
.toolbar.compact {
margin-top: 12px;
}
.icon-button,
.square-button {
display: inline-flex;
align-items: center;
justify-content: center;
gap: 7px;
min-height: 36px;
border: 1px solid #b7c0ca;
border-radius: 6px;
background: #ffffff;
color: #1f252b;
cursor: pointer;
}
.icon-button {
padding: 0 12px;
}
.square-button {
width: 36px;
padding: 0;
}
.icon-button:hover,
.square-button:hover {
border-color: #21756b;
background: #e9f3f1;
}
.icon-button.primary {
border-color: #21756b;
background: #21756b;
color: #ffffff;
}
.icon-button:disabled,
.square-button:disabled {
cursor: progress;
opacity: 0.6;
}
.error-strip {
margin-bottom: 14px;
border-left: 4px solid #b42318;
background: #fff3f0;
color: #76180f;
padding: 10px 12px;
border-radius: 6px;
font-size: 14px;
overflow-wrap: anywhere;
}
.band {
border-top: 1px solid #d8dee6;
padding: 16px 0;
}
.controls-band {
display: flex;
justify-content: space-between;
align-items: center;
gap: 18px;
}
.section-title {
display: flex;
align-items: center;
gap: 8px;
color: #2f3b46;
}
.section-title h2,
.detail-header h2,
.transcript-pane h3,
.clips-pane h3 {
margin: 0;
font-size: 16px;
font-weight: 690;
}
.toggle-row {
display: flex;
gap: 10px;
flex-wrap: wrap;
}
.switch {
display: inline-flex;
align-items: center;
gap: 7px;
min-height: 34px;
padding: 0 10px;
border: 1px solid #ccd3db;
border-radius: 6px;
background: #ffffff;
font-size: 13px;
color: #2f3b46;
}
.switch input {
width: 16px;
height: 16px;
accent-color: #21756b;
}
.workbench {
display: grid;
grid-template-columns: minmax(340px, 0.36fr) minmax(0, 1fr);
gap: 18px;
align-items: start;
}
.video-list,
.detail-pane {
min-width: 0;
}
.video-list {
border-top: 1px solid #d8dee6;
padding-top: 16px;
}
.table-scroll {
margin-top: 12px;
overflow: auto;
background: #ffffff;
border: 1px solid #d8dee6;
border-radius: 8px;
}
table {
width: 100%;
min-width: 520px;
border-collapse: collapse;
font-size: 13px;
}
th,
td {
padding: 10px;
border-bottom: 1px solid #e4e8ed;
text-align: left;
vertical-align: top;
}
th {
color: #586674;
font-weight: 650;
background: #fbfcfd;
}
tr.selected td {
background: #eef6f5;
}
.link-button {
display: inline;
border: 0;
padding: 0;
background: transparent;
color: #174b91;
cursor: pointer;
text-align: left;
overflow-wrap: anywhere;
}
.detail-pane {
border-top: 1px solid #d8dee6;
padding-top: 16px;
}
.detail-header {
display: flex;
align-items: flex-start;
justify-content: space-between;
gap: 16px;
margin-bottom: 14px;
}
.metric-row {
display: flex;
gap: 10px;
}
.metric {
min-width: 88px;
border: 1px solid #d8dee6;
border-radius: 8px;
background: #ffffff;
padding: 8px 10px;
}
.metric span {
display: block;
color: #66737f;
font-size: 12px;
}
.metric strong {
display: block;
margin-top: 3px;
font-size: 20px;
font-weight: 720;
}
.split {
display: grid;
grid-template-columns: minmax(280px, 0.95fr) minmax(320px, 1.05fr);
gap: 18px;
}
.transcript-pane,
.clips-pane {
min-width: 0;
}
.transcript-lines,
.clip-list {
margin-top: 12px;
max-height: 68vh;
overflow: auto;
}
.transcript-lines {
border: 1px solid #d8dee6;
border-radius: 8px;
background: #ffffff;
}
.transcript-line {
display: grid;
grid-template-columns: 112px minmax(0, 1fr);
gap: 10px;
padding: 9px 10px;
border-bottom: 1px solid #edf0f3;
}
.transcript-line span {
color: #66737f;
font-size: 12px;
font-variant-numeric: tabular-nums;
}
.transcript-line p {
margin: 0;
overflow-wrap: anywhere;
line-height: 1.45;
}
.clip-list {
display: grid;
gap: 10px;
}
.clip-item {
border: 1px solid #d8dee6;
border-radius: 8px;
background: #ffffff;
padding: 12px;
}
.clip-main {
display: flex;
justify-content: space-between;
gap: 12px;
}
.clip-main h4 {
margin: 0;
color: #18222b;
font-size: 15px;
}
.clip-main p {
margin: 5px 0 0;
color: #4f5c68;
line-height: 1.45;
}
.clip-main strong {
display: grid;
place-items: center;
flex: 0 0 44px;
width: 44px;
height: 44px;
border-radius: 50%;
background: #eef6f5;
color: #155c54;
}
.clip-meta {
display: flex;
align-items: center;
gap: 7px;
flex-wrap: wrap;
margin-top: 10px;
color: #66737f;
font-size: 12px;
}
.status-pill {
display: inline-flex;
align-items: center;
min-height: 22px;
padding: 0 7px;
border-radius: 999px;
font-size: 12px;
font-variant-numeric: tabular-nums;
background: #edf0f3;
color: #4f5c68;
}
.status-pill.good {
background: #e5f5ec;
color: #17623b;
}
.status-pill.bad {
background: #fff0ec;
color: #a02717;
}
.status-pill.busy {
background: #fff4d6;
color: #6f4d00;
}
@media (max-width: 1100px) {
.workbench,
.split {
grid-template-columns: 1fr;
}
.transcript-lines,
.clip-list {
max-height: none;
}
}
@media (max-width: 720px) {
.app-shell {
padding: 14px;
}
.topbar,
.controls-band,
.detail-header {
align-items: stretch;
flex-direction: column;
}
.metric-row {
width: 100%;
}
.metric {
flex: 1;
}
.transcript-line {
grid-template-columns: 1fr;
}
}
+61
View File
@@ -0,0 +1,61 @@
export interface Video {
id: number;
source_url: string;
filename: string;
size_bytes: number | null;
duration_sec: number | null;
codec_metadata: Record<string, unknown>;
ingest_status: string;
processing_status: string;
created_at: string;
updated_at: string;
}
export interface TranscriptSegment {
id: number;
video_id: number;
start_sec: number;
end_sec: number;
text: string;
speaker: string | null;
confidence: number | null;
segment_metadata: Record<string, unknown>;
}
export interface ClipSuggestion {
id: number;
video_id: number;
start_sec: number;
end_sec: number;
title_zh: string;
summary_zh: string;
reason: string;
score: number;
tags: string[];
subtitle_priority: string;
approval_status: string;
render_status: string;
upload_status: string;
created_at: string;
updated_at: string;
}
export interface PipelineSettings {
suggest_enabled: boolean;
render_enabled: boolean;
upload_enabled: boolean;
preserve_final_artifacts: boolean;
bake_subtitles: boolean;
}
export interface Artifact {
id: number;
video_id: number | null;
clip_id: number | null;
artifact_type: string;
local_path: string;
webdav_url: string | null;
preserve: boolean;
artifact_metadata: Record<string, unknown>;
created_at: string;
}
+7
View File
@@ -0,0 +1,7 @@
/// <reference types="vite/client" />
interface Window {
__EVANESCERE_FRONTEND_CONFIG__?: {
apiBaseUrl?: string;
};
}
+22
View File
@@ -0,0 +1,22 @@
{
"compilerOptions": {
"composite": true,
"target": "ES2022",
"useDefineForClassFields": true,
"lib": ["ES2022", "DOM", "DOM.Iterable"],
"allowJs": false,
"skipLibCheck": true,
"esModuleInterop": true,
"allowSyntheticDefaultImports": true,
"strict": true,
"forceConsistentCasingInFileNames": true,
"module": "ESNext",
"moduleResolution": "Bundler",
"resolveJsonModule": true,
"isolatedModules": true,
"noEmit": true,
"jsx": "react-jsx"
},
"include": ["src"]
}
+11
View File
@@ -0,0 +1,11 @@
import { defineConfig } from "vite";
import react from "@vitejs/plugin-react";
export default defineConfig({
plugins: [react()],
server: {
host: "0.0.0.0",
port: 5173,
},
});
+44
View File
@@ -0,0 +1,44 @@
from logging.config import fileConfig
from alembic import context
from sqlalchemy import engine_from_config, pool
from evanescere.config import get_settings
from evanescere.models import Base
config = context.config
if config.config_file_name is not None:
fileConfig(config.config_file_name)
config.set_main_option("sqlalchemy.url", get_settings().database_url)
target_metadata = Base.metadata
def run_migrations_offline() -> None:
url = config.get_main_option("sqlalchemy.url")
context.configure(url=url, target_metadata=target_metadata, literal_binds=True)
with context.begin_transaction():
context.run_migrations()
def run_migrations_online() -> None:
connectable = engine_from_config(
config.get_section(config.config_ini_section, {}),
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)
with connectable.connect() as connection:
context.configure(connection=connection, target_metadata=target_metadata)
with context.begin_transaction():
context.run_migrations()
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()
+113
View File
@@ -0,0 +1,113 @@
"""Initial schema.
Revision ID: 0001_initial
Revises:
Create Date: 2026-05-29
"""
from collections.abc import Sequence
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
revision: str = "0001_initial"
down_revision: str | None = None
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def upgrade() -> None:
op.create_table(
"videos",
sa.Column("id", sa.Integer(), primary_key=True),
sa.Column("source_url", sa.Text(), nullable=False, unique=True),
sa.Column("filename", sa.Text(), nullable=False),
sa.Column("size_bytes", sa.BigInteger()),
sa.Column("size_samples", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("duration_sec", sa.Float()),
sa.Column("codec_metadata", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("ingest_status", sa.String(length=32), nullable=False),
sa.Column("processing_status", sa.String(length=32), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
)
op.create_index("ix_videos_ingest_status", "videos", ["ingest_status"])
op.create_table(
"pipeline_runs",
sa.Column("id", sa.Integer(), primary_key=True),
sa.Column("video_id", sa.Integer(), sa.ForeignKey("videos.id", ondelete="CASCADE"), nullable=False),
sa.Column("trigger", sa.String(length=32), nullable=False),
sa.Column("stage", sa.String(length=64), nullable=False),
sa.Column("status", sa.String(length=32), nullable=False),
sa.Column("error", sa.Text()),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
)
op.create_table(
"artifacts",
sa.Column("id", sa.Integer(), primary_key=True),
sa.Column("video_id", sa.Integer(), sa.ForeignKey("videos.id", ondelete="CASCADE")),
sa.Column("clip_id", sa.Integer()),
sa.Column("artifact_type", sa.String(length=64), nullable=False),
sa.Column("local_path", sa.Text(), nullable=False),
sa.Column("webdav_url", sa.Text()),
sa.Column("preserve", sa.Boolean(), nullable=False),
sa.Column("metadata", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
)
op.create_table(
"transcript_segments",
sa.Column("id", sa.Integer(), primary_key=True),
sa.Column("video_id", sa.Integer(), sa.ForeignKey("videos.id", ondelete="CASCADE"), nullable=False),
sa.Column("start_sec", sa.Float(), nullable=False),
sa.Column("end_sec", sa.Float(), nullable=False),
sa.Column("text", sa.Text(), nullable=False),
sa.Column("speaker", sa.Text()),
sa.Column("confidence", sa.Float()),
sa.Column("metadata", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
)
op.create_index("ix_transcript_segments_video_id", "transcript_segments", ["video_id"])
op.create_table(
"clip_suggestions",
sa.Column("id", sa.Integer(), primary_key=True),
sa.Column("video_id", sa.Integer(), sa.ForeignKey("videos.id", ondelete="CASCADE"), nullable=False),
sa.Column("start_sec", sa.Float(), nullable=False),
sa.Column("end_sec", sa.Float(), nullable=False),
sa.Column("title_zh", sa.Text(), nullable=False),
sa.Column("summary_zh", sa.Text(), nullable=False),
sa.Column("reason", sa.Text(), nullable=False),
sa.Column("score", sa.Float(), nullable=False),
sa.Column("tags", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("subtitle_priority", sa.String(length=32), nullable=False),
sa.Column("llm_raw", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("approval_status", sa.String(length=32), nullable=False),
sa.Column("render_status", sa.String(length=32), nullable=False),
sa.Column("upload_status", sa.String(length=32), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
)
op.create_table(
"settings",
sa.Column("key", sa.String(length=128), primary_key=True),
sa.Column("value", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
)
def downgrade() -> None:
op.drop_table("settings")
op.drop_table("clip_suggestions")
op.drop_index("ix_transcript_segments_video_id", table_name="transcript_segments")
op.drop_table("transcript_segments")
op.drop_table("artifacts")
op.drop_table("pipeline_runs")
op.drop_index("ix_videos_ingest_status", table_name="videos")
op.drop_table("videos")
+42
View File
@@ -0,0 +1,42 @@
[project]
name = "evanescere"
version = "0.1.0"
description = "Livestream transcription, clip suggestion, rendering, and upload orchestration."
requires-python = ">=3.12"
dependencies = [
"alembic>=1.13",
"dramatiq[redis]>=1.17",
"fastapi>=0.115",
"httpx>=0.27",
"openai>=1.55",
"pillow>=11.0",
"psycopg[binary]>=3.2",
"pydantic>=2.9",
"python-multipart>=0.0.12",
"sqlalchemy>=2.0",
"typer>=0.12",
"uvicorn[standard]>=0.32",
]
[project.optional-dependencies]
dev = [
"pytest>=8.3",
"ruff>=0.8",
]
[project.scripts]
evanescere = "evanescere.cli:app"
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.hatch.build.targets.wheel]
packages = ["src/evanescere"]
[tool.ruff]
line-length = 100
target-version = "py312"
[tool.ruff.lint]
select = ["E", "F", "I", "UP", "B"]
+4
View File
@@ -0,0 +1,4 @@
__all__ = ["__version__"]
__version__ = "0.1.0"
+180
View File
@@ -0,0 +1,180 @@
import logging
import time
from fastapi import Depends, FastAPI, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from fastapi import Request
from sqlalchemy import select
from sqlalchemy.orm import Session
from evanescere import __version__
from evanescere.config import get_settings
from evanescere.db import get_db
from evanescere.jobs import enqueue_pipeline, enqueue_render, enqueue_upload
from evanescere.logging_config import configure_logging
from evanescere.models import Artifact, ClipSuggestion, PipelineRun, TranscriptSegment, Video
from evanescere.schemas import (
ArtifactRead,
ClipSuggestionRead,
RunRead,
SettingsPatch,
SettingsRead,
TranscriptSegmentRead,
VideoRead,
)
from evanescere.settings_store import get_pipeline_settings, patch_pipeline_settings
configure_logging()
app = FastAPI(title="Evanescere", version=__version__)
logger = logging.getLogger(__name__)
settings = get_settings()
app.add_middleware(
CORSMiddleware,
allow_origins=settings.cors_origin_list,
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
@app.middleware("http")
async def log_requests(request: Request, call_next):
start = time.perf_counter()
response = await call_next(request)
elapsed_ms = (time.perf_counter() - start) * 1000
logger.info(
"api request method=%s path=%s status=%s elapsed_ms=%.1f",
request.method,
request.url.path,
response.status_code,
elapsed_ms,
)
return response
@app.get("/")
def root() -> dict[str, str]:
return {"service": "evanescere-api", "version": __version__}
@app.get("/health")
def health() -> dict[str, str]:
return {"status": "ok", "version": __version__}
@app.get("/videos", response_model=list[VideoRead])
def list_videos(db: Session = Depends(get_db)) -> list[Video]:
return list(db.scalars(select(Video).order_by(Video.created_at.desc())).all())
@app.post("/videos/{video_id}/run", response_model=RunRead)
def run_video(video_id: int, db: Session = Depends(get_db)) -> PipelineRun:
video = db.get(Video, video_id)
if video is None:
raise HTTPException(status_code=404, detail="Video not found")
run = PipelineRun(video_id=video_id, trigger="manual", stage="queued", status="queued")
video.processing_status = "queued"
db.add(run)
db.flush()
enqueue_pipeline(video_id, run.id)
return run
@app.get("/videos/{video_id}/transcript", response_model=list[TranscriptSegmentRead])
def video_transcript(video_id: int, db: Session = Depends(get_db)) -> list[TranscriptSegment]:
return list(
db.scalars(
select(TranscriptSegment)
.where(TranscriptSegment.video_id == video_id)
.order_by(TranscriptSegment.start_sec)
).all()
)
@app.get("/videos/{video_id}/clips", response_model=list[ClipSuggestionRead])
def video_clips(video_id: int, db: Session = Depends(get_db)) -> list[ClipSuggestion]:
return list(
db.scalars(
select(ClipSuggestion)
.where(ClipSuggestion.video_id == video_id)
.order_by(ClipSuggestion.score.desc(), ClipSuggestion.start_sec)
).all()
)
@app.get("/videos/{video_id}/artifacts", response_model=list[ArtifactRead])
def video_artifacts(video_id: int, db: Session = Depends(get_db)) -> list[Artifact]:
return list(
db.scalars(
select(Artifact)
.where(Artifact.video_id == video_id)
.order_by(Artifact.created_at.desc())
).all()
)
@app.get("/clips/{clip_id}/artifacts", response_model=list[ArtifactRead])
def clip_artifacts(clip_id: int, db: Session = Depends(get_db)) -> list[Artifact]:
return list(
db.scalars(
select(Artifact)
.where(Artifact.clip_id == clip_id)
.order_by(Artifact.created_at.desc())
).all()
)
@app.post("/clips/{clip_id}/approve", response_model=ClipSuggestionRead)
def approve_clip(clip_id: int, db: Session = Depends(get_db)) -> ClipSuggestion:
clip = db.get(ClipSuggestion, clip_id)
if clip is None:
raise HTTPException(status_code=404, detail="Clip not found")
clip.approval_status = "approved"
return clip
@app.post("/clips/{clip_id}/render", response_model=ClipSuggestionRead)
def render_clip(clip_id: int, db: Session = Depends(get_db)) -> ClipSuggestion:
clip = db.get(ClipSuggestion, clip_id)
if clip is None:
raise HTTPException(status_code=404, detail="Clip not found")
clip.render_status = "queued"
enqueue_render(clip_id)
return clip
@app.post("/clips/{clip_id}/upload", response_model=ClipSuggestionRead)
def upload_clip(clip_id: int, db: Session = Depends(get_db)) -> ClipSuggestion:
clip = db.get(ClipSuggestion, clip_id)
if clip is None:
raise HTTPException(status_code=404, detail="Clip not found")
clip.upload_status = "queued"
enqueue_upload(clip_id)
return clip
@app.get("/runs/{run_id}", response_model=RunRead)
def get_run(run_id: int, db: Session = Depends(get_db)) -> PipelineRun:
run = db.get(PipelineRun, run_id)
if run is None:
raise HTTPException(status_code=404, detail="Run not found")
return run
@app.get("/artifacts/{artifact_id}", response_model=ArtifactRead)
def get_artifact(artifact_id: int, db: Session = Depends(get_db)) -> Artifact:
artifact = db.get(Artifact, artifact_id)
if artifact is None:
raise HTTPException(status_code=404, detail="Artifact not found")
return artifact
@app.get("/settings", response_model=SettingsRead)
def read_settings(db: Session = Depends(get_db)) -> SettingsRead:
return get_pipeline_settings(db)
@app.patch("/settings", response_model=SettingsRead)
def update_settings(patch: SettingsPatch, db: Session = Depends(get_db)) -> SettingsRead:
return patch_pipeline_settings(db, patch)
+75
View File
@@ -0,0 +1,75 @@
import typer
from sqlalchemy import select
from evanescere.db import session_scope
from evanescere.jobs import enqueue_pipeline, enqueue_render, enqueue_transcribe, enqueue_suggest, enqueue_upload
from evanescere.logging_config import configure_logging
from evanescere.models import PipelineRun, Video
from evanescere.services.webdav import WebDavClient, bootstrap_existing as bootstrap_webdav_existing, scan_once
app = typer.Typer(no_args_is_help=True)
configure_logging()
@app.command()
def bootstrap_existing() -> None:
with session_scope() as session:
count = bootstrap_webdav_existing(WebDavClient.from_settings(), session)
typer.echo(f"Marked {count} existing WebDAV recordings as existing_done.")
@app.command()
def scan() -> None:
with session_scope() as session:
observed, stable = scan_once(WebDavClient.from_settings(), session)
typer.echo(f"Observed {observed} files; {stable} newly stable.")
@app.command("run-video")
def run_video(video_id: int) -> None:
with session_scope() as session:
video = session.get(Video, video_id)
if video is None:
raise typer.BadParameter("Video not found")
run = PipelineRun(video_id=video_id, trigger="manual", stage="queued", status="queued")
video.processing_status = "queued"
session.add(run)
session.flush()
enqueue_pipeline(video_id, run.id)
typer.echo(f"Queued pipeline for video {video_id}.")
@app.command()
def transcribe(video_id: int) -> None:
enqueue_transcribe(video_id)
typer.echo(f"Queued transcription for video {video_id}.")
@app.command()
def suggest(video_id: int) -> None:
enqueue_suggest(video_id)
typer.echo(f"Queued clip suggestion for video {video_id}.")
@app.command()
def render(clip_id: int) -> None:
enqueue_render(clip_id)
typer.echo(f"Queued render for clip {clip_id}.")
@app.command()
def upload(clip_id: int) -> None:
enqueue_upload(clip_id)
typer.echo(f"Queued upload for clip {clip_id}.")
@app.command()
def videos() -> None:
with session_scope() as session:
rows = session.scalars(select(Video).order_by(Video.created_at.desc())).all()
for video in rows:
typer.echo(f"{video.id}\t{video.ingest_status}\t{video.processing_status}\t{video.filename}")
if __name__ == "__main__":
app()
+150
View File
@@ -0,0 +1,150 @@
from __future__ import annotations
import tomllib
from functools import lru_cache
from pathlib import Path
from typing import Any
from pydantic import AnyHttpUrl, BaseModel, Field
DEFAULT_CONFIG_PATHS = (
Path("/etc/evanescere/config.toml"),
Path("config.toml"),
)
class Settings(BaseModel):
app_env: str = "dev"
log_level: str = "INFO"
log_sql: bool = False
cors_origins: list[str] = Field(default_factory=lambda: ["http://localhost:3000", "http://localhost:5173"])
database_url: str = "postgresql+psycopg://evanescere:evanescere@localhost:5432/evanescere"
redis_url: str = "redis://localhost:6379/0"
local_storage_root: Path = Path("./storage")
webdav_base_url: AnyHttpUrl | str = "http://localhost/webdav/recordings"
webdav_username: str | None = None
webdav_password: str | None = None
webdav_verify_tls: bool = True
webdav_poll_interval_seconds: int = Field(default=60, ge=5)
funasr_base_url: str = "http://localhost:10096/v1"
funasr_api_key: str | None = None
funasr_model: str = "paraformer-zh"
deepseek_base_url: str = "https://api.deepseek.com"
deepseek_api_key: str | None = None
deepseek_model: str = "deepseek-v4-pro"
deepseek_temperature: float = 0.2
default_suggest_enabled: bool = True
default_render_enabled: bool = True
default_upload_enabled: bool = True
default_preserve_final_artifacts: bool = True
default_bake_subtitles: bool = True
upload_adapter: str = "noop"
upload_command: str | None = None
clip_min_seconds: int = 30
clip_max_seconds: int = 360
transcript_chunk_seconds: int = 900
thumbnail_enabled: bool = True
thumbnail_provider: str = "frame_overlay"
thumbnail_width: int = Field(default=1920, ge=320)
thumbnail_height: int = Field(default=1080, ge=180)
thumbnail_character_overlay_path: str | None = None
thumbnail_character_scale: float = Field(default=0.42, gt=0, le=1)
thumbnail_character_position: str = "bottom-right"
thumbnail_title_enabled: bool = True
thumbnail_command: str | None = None
@property
def cors_origin_list(self) -> list[str]:
return self.cors_origins
def configured_path() -> Path | None:
for path in DEFAULT_CONFIG_PATHS:
if path.exists():
return path
return None
def load_toml_settings(path: Path) -> dict[str, Any]:
with path.open("rb") as config_file:
raw = tomllib.load(config_file)
return flatten_config(raw)
def blank_to_none(value: Any) -> Any:
if value == "":
return None
return value
def flatten_config(raw: dict[str, Any]) -> dict[str, Any]:
app = raw.get("app", {})
database = raw.get("database", {})
redis = raw.get("redis", {})
storage = raw.get("storage", {})
webdav = raw.get("webdav", {})
funasr = raw.get("funasr", {})
deepseek = raw.get("deepseek", {})
defaults = raw.get("defaults", {})
clip = raw.get("clip", {})
thumbnail = raw.get("thumbnail", {})
upload = raw.get("upload", {})
return {
"app_env": app.get("env", "dev"),
"log_level": app.get("log_level", "INFO"),
"log_sql": app.get("log_sql", False),
"cors_origins": app.get("cors_origins", ["http://localhost:3000", "http://localhost:5173"]),
"database_url": database.get("url", Settings.model_fields["database_url"].default),
"redis_url": redis.get("url", Settings.model_fields["redis_url"].default),
"local_storage_root": storage.get("local_root", Settings.model_fields["local_storage_root"].default),
"webdav_base_url": webdav.get("base_url", Settings.model_fields["webdav_base_url"].default),
"webdav_username": blank_to_none(webdav.get("username")),
"webdav_password": blank_to_none(webdav.get("password")),
"webdav_verify_tls": webdav.get("verify_tls", True),
"webdav_poll_interval_seconds": webdav.get("poll_interval_seconds", 60),
"funasr_base_url": funasr.get("base_url", Settings.model_fields["funasr_base_url"].default),
"funasr_api_key": blank_to_none(funasr.get("api_key")),
"funasr_model": funasr.get("model", "paraformer-zh"),
"deepseek_base_url": deepseek.get("base_url", "https://api.deepseek.com"),
"deepseek_api_key": blank_to_none(deepseek.get("api_key")),
"deepseek_model": deepseek.get("model", "deepseek-v4-pro"),
"deepseek_temperature": deepseek.get("temperature", 0.2),
"default_suggest_enabled": defaults.get("suggest_enabled", True),
"default_render_enabled": defaults.get("render_enabled", True),
"default_upload_enabled": defaults.get("upload_enabled", True),
"default_preserve_final_artifacts": defaults.get("preserve_final_artifacts", True),
"default_bake_subtitles": defaults.get("bake_subtitles", True),
"clip_min_seconds": clip.get("min_seconds", 30),
"clip_max_seconds": clip.get("max_seconds", 360),
"transcript_chunk_seconds": clip.get("transcript_chunk_seconds", 900),
"thumbnail_enabled": thumbnail.get("enabled", True),
"thumbnail_provider": thumbnail.get("provider", "frame_overlay"),
"thumbnail_width": thumbnail.get("width", 1920),
"thumbnail_height": thumbnail.get("height", 1080),
"thumbnail_character_overlay_path": blank_to_none(thumbnail.get("character_overlay_path")),
"thumbnail_character_scale": thumbnail.get("character_scale", 0.42),
"thumbnail_character_position": thumbnail.get("character_position", "bottom-right"),
"thumbnail_title_enabled": thumbnail.get("title_enabled", True),
"thumbnail_command": blank_to_none(thumbnail.get("command")),
"upload_adapter": upload.get("adapter", "noop"),
"upload_command": blank_to_none(upload.get("command")),
}
@lru_cache
def get_settings() -> Settings:
path = configured_path()
values = load_toml_settings(path) if path else {}
settings = Settings(**values)
settings.local_storage_root.mkdir(parents=True, exist_ok=True)
return settings
+29
View File
@@ -0,0 +1,29 @@
from collections.abc import Iterator
from contextlib import contextmanager
from sqlalchemy import create_engine
from sqlalchemy.orm import Session, sessionmaker
from evanescere.config import get_settings
settings = get_settings()
engine = create_engine(settings.database_url, pool_pre_ping=True, echo=settings.log_sql)
SessionLocal = sessionmaker(bind=engine, autoflush=False, expire_on_commit=False)
@contextmanager
def session_scope() -> Iterator[Session]:
session = SessionLocal()
try:
yield session
session.commit()
except Exception:
session.rollback()
raise
finally:
session.close()
def get_db() -> Iterator[Session]:
with session_scope() as session:
yield session
+87
View File
@@ -0,0 +1,87 @@
import logging
import dramatiq
from dramatiq.brokers.redis import RedisBroker
from evanescere.config import get_settings
from evanescere.db import session_scope
from evanescere.logging_config import configure_logging
from evanescere.models import Video
from evanescere.pipeline import render_clip_by_id, run_pipeline, suggest_clips, transcribe_video, upload_clip_by_id
configure_logging()
logger = logging.getLogger(__name__)
redis_broker = RedisBroker(url=get_settings().redis_url)
dramatiq.set_broker(redis_broker)
def enqueue_pipeline(video_id: int, run_id: int | None = None) -> None:
logger.info("enqueue pipeline video_id=%s run_id=%s", video_id, run_id)
pipeline_job.send(video_id, run_id)
def enqueue_transcribe(video_id: int) -> None:
logger.info("enqueue transcribe video_id=%s", video_id)
transcribe_job.send(video_id)
def enqueue_suggest(video_id: int) -> None:
logger.info("enqueue suggest video_id=%s", video_id)
suggest_job.send(video_id)
def enqueue_render(clip_id: int) -> None:
logger.info("enqueue render clip_id=%s", clip_id)
render_job.send(clip_id)
def enqueue_upload(clip_id: int) -> None:
logger.info("enqueue upload clip_id=%s", clip_id)
upload_job.send(clip_id)
@dramatiq.actor(max_retries=1)
def pipeline_job(video_id: int, run_id: int | None = None) -> None:
logger.info("job start pipeline video_id=%s run_id=%s", video_id, run_id)
with session_scope() as session:
run_pipeline(session, video_id, run_id)
logger.info("job done pipeline video_id=%s run_id=%s", video_id, run_id)
@dramatiq.actor(max_retries=1)
def transcribe_job(video_id: int) -> None:
logger.info("job start transcribe video_id=%s", video_id)
with session_scope() as session:
video = session.get(Video, video_id)
if video is None:
raise RuntimeError(f"Video {video_id} not found.")
transcribe_video(session, video)
logger.info("job done transcribe video_id=%s", video_id)
@dramatiq.actor(max_retries=1)
def suggest_job(video_id: int) -> None:
logger.info("job start suggest video_id=%s", video_id)
with session_scope() as session:
video = session.get(Video, video_id)
if video is None:
raise RuntimeError(f"Video {video_id} not found.")
suggest_clips(session, video)
logger.info("job done suggest video_id=%s", video_id)
@dramatiq.actor(max_retries=1)
def render_job(clip_id: int) -> None:
logger.info("job start render clip_id=%s", clip_id)
with session_scope() as session:
render_clip_by_id(session, clip_id)
logger.info("job done render clip_id=%s", clip_id)
@dramatiq.actor(max_retries=1)
def upload_job(clip_id: int) -> None:
logger.info("job start upload clip_id=%s", clip_id)
with session_scope() as session:
upload_clip_by_id(session, clip_id)
logger.info("job done upload clip_id=%s", clip_id)
+27
View File
@@ -0,0 +1,27 @@
from __future__ import annotations
import logging
import sys
from evanescere.config import get_settings
def configure_logging() -> None:
settings = get_settings()
level = getattr(logging, settings.log_level.upper(), logging.INFO)
logging.basicConfig(
level=level,
format="%(asctime)s %(levelname)s [%(name)s] %(message)s",
datefmt="%Y-%m-%dT%H:%M:%S%z",
stream=sys.stdout,
force=True,
)
logging.getLogger("httpx").setLevel(logging.DEBUG if level <= logging.DEBUG else logging.WARNING)
logging.getLogger("httpcore").setLevel(logging.INFO if level <= logging.DEBUG else logging.WARNING)
logging.getLogger("sqlalchemy.engine").setLevel(logging.INFO if settings.log_sql else logging.WARNING)
logging.getLogger("evanescere").debug(
"logging configured level=%s sql=%s env=%s",
settings.log_level,
settings.log_sql,
settings.app_env,
)
+113
View File
@@ -0,0 +1,113 @@
from datetime import UTC, datetime
from typing import Any
from sqlalchemy import BigInteger, Boolean, DateTime, Float, ForeignKey, Integer, String, Text
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column, relationship
def utcnow() -> datetime:
return datetime.now(UTC)
class Base(DeclarativeBase):
pass
class TimestampMixin:
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=utcnow, onupdate=utcnow
)
class Video(Base, TimestampMixin):
__tablename__ = "videos"
id: Mapped[int] = mapped_column(Integer, primary_key=True)
source_url: Mapped[str] = mapped_column(Text, unique=True, nullable=False)
filename: Mapped[str] = mapped_column(Text, nullable=False)
size_bytes: Mapped[int | None] = mapped_column(BigInteger)
size_samples: Mapped[list[dict[str, Any]]] = mapped_column(JSONB, default=list)
duration_sec: Mapped[float | None] = mapped_column(Float)
codec_metadata: Mapped[dict[str, Any]] = mapped_column(JSONB, default=dict)
ingest_status: Mapped[str] = mapped_column(String(32), default="observing", index=True)
processing_status: Mapped[str] = mapped_column(String(32), default="pending")
runs: Mapped[list["PipelineRun"]] = relationship(back_populates="video")
transcript_segments: Mapped[list["TranscriptSegment"]] = relationship(back_populates="video")
clip_suggestions: Mapped[list["ClipSuggestion"]] = relationship(back_populates="video")
class PipelineRun(Base, TimestampMixin):
__tablename__ = "pipeline_runs"
id: Mapped[int] = mapped_column(Integer, primary_key=True)
video_id: Mapped[int] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"))
trigger: Mapped[str] = mapped_column(String(32), default="manual")
stage: Mapped[str] = mapped_column(String(64), default="created")
status: Mapped[str] = mapped_column(String(32), default="queued")
error: Mapped[str | None] = mapped_column(Text)
video: Mapped[Video] = relationship(back_populates="runs")
class Artifact(Base):
__tablename__ = "artifacts"
id: Mapped[int] = mapped_column(Integer, primary_key=True)
video_id: Mapped[int | None] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"))
clip_id: Mapped[int | None] = mapped_column(Integer)
artifact_type: Mapped[str] = mapped_column(String(64), nullable=False)
local_path: Mapped[str] = mapped_column(Text, nullable=False)
webdav_url: Mapped[str | None] = mapped_column(Text)
preserve: Mapped[bool] = mapped_column(Boolean, default=True)
artifact_metadata: Mapped[dict[str, Any]] = mapped_column("metadata", JSONB, default=dict)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow)
class TranscriptSegment(Base):
__tablename__ = "transcript_segments"
id: Mapped[int] = mapped_column(Integer, primary_key=True)
video_id: Mapped[int] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"), index=True)
start_sec: Mapped[float] = mapped_column(Float, nullable=False)
end_sec: Mapped[float] = mapped_column(Float, nullable=False)
text: Mapped[str] = mapped_column(Text, nullable=False)
speaker: Mapped[str | None] = mapped_column(Text)
confidence: Mapped[float | None] = mapped_column(Float)
segment_metadata: Mapped[dict[str, Any]] = mapped_column("metadata", JSONB, default=dict)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow)
video: Mapped[Video] = relationship(back_populates="transcript_segments")
class ClipSuggestion(Base, TimestampMixin):
__tablename__ = "clip_suggestions"
id: Mapped[int] = mapped_column(Integer, primary_key=True)
video_id: Mapped[int] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"))
start_sec: Mapped[float] = mapped_column(Float, nullable=False)
end_sec: Mapped[float] = mapped_column(Float, nullable=False)
title_zh: Mapped[str] = mapped_column(Text, nullable=False)
summary_zh: Mapped[str] = mapped_column(Text, nullable=False)
reason: Mapped[str] = mapped_column(Text, nullable=False)
score: Mapped[float] = mapped_column(Float, nullable=False)
tags: Mapped[list[str]] = mapped_column(JSONB, default=list)
subtitle_priority: Mapped[str] = mapped_column(String(32), default="normal")
llm_raw: Mapped[dict[str, Any]] = mapped_column(JSONB, default=dict)
approval_status: Mapped[str] = mapped_column(String(32), default="pending")
render_status: Mapped[str] = mapped_column(String(32), default="pending")
upload_status: Mapped[str] = mapped_column(String(32), default="pending")
video: Mapped[Video] = relationship(back_populates="clip_suggestions")
class Setting(Base):
__tablename__ = "settings"
key: Mapped[str] = mapped_column(String(128), primary_key=True)
value: Mapped[dict[str, Any]] = mapped_column(JSONB, nullable=False)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=utcnow, onupdate=utcnow
)
+251
View File
@@ -0,0 +1,251 @@
from __future__ import annotations
import json
import logging
from pathlib import Path
from sqlalchemy import select
from sqlalchemy.orm import Session
from evanescere.config import get_settings
from evanescere.models import Artifact, ClipSuggestion, PipelineRun, TranscriptSegment, Video
from evanescere.services.artifacts import latest_artifact, register_artifact, video_work_dir
from evanescere.services.asr import FunAsrClient, normalize_segments
from evanescere.services.llm import DeepSeekClipClient, chunk_transcript
from evanescere.services.media import extract_audio, remux_to_mp4, render_clip
from evanescere.services.subtitles import generate_clip_subtitles
from evanescere.services.thumbnail import generate_thumbnail
from evanescere.services.uploader import choose_upload_artifact, upload_artifact
from evanescere.services.webdav import WebDavClient
from evanescere.settings_store import get_pipeline_settings
logger = logging.getLogger(__name__)
def update_run(run: PipelineRun | None, stage: str, status: str, error: str | None = None) -> None:
if run is not None:
logger.info("run update run_id=%s video_id=%s stage=%s status=%s", run.id, run.video_id, stage, status)
run.stage = stage
run.status = status
run.error = error
def source_artifact(session: Session, video: Video) -> Artifact | None:
return latest_artifact(session, video.id, "raw_source")
def download_source(session: Session, video: Video) -> Path:
existing = source_artifact(session, video)
if existing and Path(existing.local_path).exists():
logger.info("source already downloaded video_id=%s path=%s", video.id, existing.local_path)
return Path(existing.local_path)
destination = video_work_dir(video.id) / video.filename
logger.info("downloading source video_id=%s url=%s destination=%s", video.id, video.source_url, destination)
WebDavClient.from_settings().download(video.source_url, destination)
register_artifact(
session,
video_id=video.id,
artifact_type="raw_source",
local_path=destination,
preserve=False,
metadata={"source_url": video.source_url},
)
return destination
def prepare_media(session: Session, video: Video) -> tuple[Path, Path]:
logger.info("prepare media start video_id=%s file=%s", video.id, video.filename)
source = download_source(session, video)
mp4_artifact = latest_artifact(session, video.id, "remux_mp4")
if mp4_artifact and Path(mp4_artifact.local_path).exists():
logger.info("using existing remux video_id=%s path=%s", video.id, mp4_artifact.local_path)
mp4 = Path(mp4_artifact.local_path)
else:
mp4 = remux_to_mp4(session, video, source)
audio_artifact = latest_artifact(session, video.id, "audio_wav")
if audio_artifact and Path(audio_artifact.local_path).exists():
logger.info("using existing audio video_id=%s path=%s", video.id, audio_artifact.local_path)
audio = Path(audio_artifact.local_path)
else:
audio = extract_audio(session, video, mp4)
logger.info("prepare media done video_id=%s mp4=%s audio=%s", video.id, mp4, audio)
return mp4, audio
def transcribe_video(session: Session, video: Video) -> int:
logger.info("transcription start video_id=%s", video.id)
audio_artifact = latest_artifact(session, video.id, "audio_wav")
if audio_artifact is None:
raise RuntimeError("Cannot transcribe before audio_wav artifact exists.")
payload = FunAsrClient.from_settings().transcribe(Path(audio_artifact.local_path))
transcript_path = video_work_dir(video.id) / "funasr_transcript.json"
transcript_path.write_text(json.dumps(payload, ensure_ascii=False, indent=2), encoding="utf-8")
register_artifact(
session,
video_id=video.id,
artifact_type="transcript_json",
local_path=transcript_path,
metadata={"source": "funasr"},
)
session.query(TranscriptSegment).filter(TranscriptSegment.video_id == video.id).delete()
for item in normalize_segments(payload):
session.add(
TranscriptSegment(
video_id=video.id,
start_sec=item["start_sec"],
end_sec=item["end_sec"],
text=item["text"],
speaker=item["speaker"],
confidence=item["confidence"],
segment_metadata=item["metadata"],
)
)
session.flush()
count = session.query(TranscriptSegment).filter(TranscriptSegment.video_id == video.id).count()
logger.info("transcription done video_id=%s segments=%s transcript=%s", video.id, count, transcript_path)
return count
def suggest_clips(session: Session, video: Video) -> int:
settings = get_settings()
logger.info("clip suggestion start video_id=%s chunk_seconds=%s", video.id, settings.transcript_chunk_seconds)
segments = list(
session.scalars(
select(TranscriptSegment)
.where(TranscriptSegment.video_id == video.id)
.order_by(TranscriptSegment.start_sec)
).all()
)
if not segments:
raise RuntimeError("Cannot suggest clips without transcript segments.")
chunks = chunk_transcript(segments, settings.transcript_chunk_seconds)
logger.info("clip suggestion chunks video_id=%s segments=%s chunks=%s", video.id, len(segments), len(chunks))
candidates = DeepSeekClipClient.from_settings().suggest(chunks)
session.query(ClipSuggestion).filter(ClipSuggestion.video_id == video.id).delete()
for candidate in candidates:
session.add(
ClipSuggestion(
video_id=video.id,
start_sec=candidate.start_sec,
end_sec=candidate.end_sec,
title_zh=candidate.title_zh,
summary_zh=candidate.summary_zh,
reason=candidate.reason,
score=candidate.score,
tags=candidate.tags,
subtitle_priority=candidate.subtitle_priority,
llm_raw=candidate.model_dump(),
)
)
session.flush()
logger.info("clip suggestion done video_id=%s candidates=%s", video.id, len(candidates))
return len(candidates)
def render_clip_by_id(session: Session, clip_id: int) -> None:
logger.info("render start clip_id=%s", clip_id)
clip = session.get(ClipSuggestion, clip_id)
if clip is None:
raise RuntimeError(f"Clip {clip_id} not found.")
video = session.get(Video, clip.video_id)
if video is None:
raise RuntimeError(f"Video {clip.video_id} not found.")
mp4_artifact = latest_artifact(session, video.id, "remux_mp4")
if mp4_artifact is None:
raise RuntimeError("Cannot render before remux_mp4 artifact exists.")
settings = get_pipeline_settings(session)
segments = list(
session.scalars(
select(TranscriptSegment)
.where(TranscriptSegment.video_id == video.id)
.order_by(TranscriptSegment.start_sec)
).all()
)
clip.render_status = "running"
srt, ass = generate_clip_subtitles(
session,
video=video,
clip=clip,
segments=segments,
preserve=settings.preserve_final_artifacts,
)
render_clip(
session,
video=video,
clip=clip,
source_mp4=Path(mp4_artifact.local_path),
srt_path=srt,
ass_path=ass,
bake_subtitles=settings.bake_subtitles,
preserve=settings.preserve_final_artifacts,
)
generate_thumbnail(
session,
video=video,
clip=clip,
source_mp4=Path(mp4_artifact.local_path),
transcript_segments=segments,
)
clip.render_status = "done"
logger.info("render done clip_id=%s video_id=%s", clip.id, video.id)
def upload_clip_by_id(session: Session, clip_id: int) -> None:
logger.info("upload start clip_id=%s", clip_id)
clip = session.get(ClipSuggestion, clip_id)
if clip is None:
raise RuntimeError(f"Clip {clip_id} not found.")
clip.upload_status = "running"
artifacts = (
session.query(Artifact)
.filter(Artifact.clip_id == clip_id)
.order_by(Artifact.created_at.desc())
.all()
)
artifact = choose_upload_artifact(artifacts)
if artifact is None:
raise RuntimeError("No rendered clip artifact found for upload.")
result = upload_artifact(session, clip, artifact)
artifact.artifact_metadata = artifact.artifact_metadata | {"upload": dict(result)}
clip.upload_status = "done"
logger.info("upload done clip_id=%s result=%s", clip.id, dict(result))
def run_pipeline(session: Session, video_id: int, run_id: int | None = None) -> None:
logger.info("pipeline start video_id=%s run_id=%s", video_id, run_id)
video = session.get(Video, video_id)
if video is None:
raise RuntimeError(f"Video {video_id} not found.")
run = session.get(PipelineRun, run_id) if run_id is not None else None
settings = get_pipeline_settings(session)
try:
video.processing_status = "running"
update_run(run, "media", "running")
prepare_media(session, video)
update_run(run, "transcribe", "running")
transcribe_video(session, video)
if settings.suggest_enabled:
update_run(run, "suggest", "running")
suggest_clips(session, video)
if settings.render_enabled:
clips = session.scalars(
select(ClipSuggestion).where(ClipSuggestion.video_id == video.id)
).all()
logger.info("auto render stage video_id=%s clips=%s upload_enabled=%s", video.id, len(clips), settings.upload_enabled)
for clip in clips:
clip.approval_status = "auto_approved"
render_clip_by_id(session, clip.id)
if settings.upload_enabled:
upload_clip_by_id(session, clip.id)
video.processing_status = "done"
update_run(run, "done", "done")
logger.info("pipeline done video_id=%s run_id=%s", video_id, run_id)
except Exception as exc:
video.processing_status = "failed"
update_run(run, "failed", "failed", str(exc))
logger.exception("pipeline failed video_id=%s run_id=%s", video_id, run_id)
raise
+57
View File
@@ -0,0 +1,57 @@
from __future__ import annotations
import logging
import time
from sqlalchemy import select
from evanescere.config import get_settings
from evanescere.db import session_scope
from evanescere.jobs import enqueue_pipeline
from evanescere.logging_config import configure_logging
from evanescere.models import PipelineRun, Video
from evanescere.services.webdav import WebDavClient, scan_once
configure_logging()
logger = logging.getLogger(__name__)
def enqueue_stable_videos() -> int:
queued = 0
with session_scope() as session:
videos = session.scalars(
select(Video).where(Video.ingest_status == "stable", Video.processing_status == "pending")
).all()
for video in videos:
run = PipelineRun(video_id=video.id, trigger="automatic", stage="queued", status="queued")
video.ingest_status = "queued"
video.processing_status = "queued"
session.add(run)
session.flush()
logger.info("queueing stable video video_id=%s run_id=%s file=%s", video.id, run.id, video.filename)
enqueue_pipeline(video.id, run.id)
queued += 1
return queued
def run_forever() -> None:
settings = get_settings()
client = WebDavClient.from_settings()
logger.info(
"scheduler starting webdav_base=%s poll_interval_seconds=%s",
settings.webdav_base_url,
settings.webdav_poll_interval_seconds,
)
while True:
try:
with session_scope() as session:
observed, stable = scan_once(client, session)
queued = enqueue_stable_videos()
logger.info("scan observed=%s newly_stable=%s queued=%s", observed, stable, queued)
except Exception:
logger.exception("scheduler scan failed")
time.sleep(settings.webdav_poll_interval_seconds)
if __name__ == "__main__":
run_forever()
+111
View File
@@ -0,0 +1,111 @@
from datetime import datetime
from typing import Any
from pydantic import BaseModel, ConfigDict, Field
class VideoRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: int
source_url: str
filename: str
size_bytes: int | None
duration_sec: float | None
codec_metadata: dict[str, Any]
ingest_status: str
processing_status: str
created_at: datetime
updated_at: datetime
class RunRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: int
video_id: int
trigger: str
stage: str
status: str
error: str | None
created_at: datetime
updated_at: datetime
class ArtifactRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: int
video_id: int | None
clip_id: int | None
artifact_type: str
local_path: str
webdav_url: str | None
preserve: bool
artifact_metadata: dict[str, Any]
created_at: datetime
class TranscriptSegmentRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: int
video_id: int
start_sec: float
end_sec: float
text: str
speaker: str | None
confidence: float | None
segment_metadata: dict[str, Any]
class ClipSuggestionRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: int
video_id: int
start_sec: float
end_sec: float
title_zh: str
summary_zh: str
reason: str
score: float
tags: list[str]
subtitle_priority: str
approval_status: str
render_status: str
upload_status: str
created_at: datetime
updated_at: datetime
class SettingsPatch(BaseModel):
suggest_enabled: bool | None = None
render_enabled: bool | None = None
upload_enabled: bool | None = None
preserve_final_artifacts: bool | None = None
bake_subtitles: bool | None = None
class SettingsRead(BaseModel):
suggest_enabled: bool = True
render_enabled: bool = True
upload_enabled: bool = True
preserve_final_artifacts: bool = True
bake_subtitles: bool = True
class ClipCandidate(BaseModel):
start_sec: float = Field(ge=0)
end_sec: float = Field(gt=0)
title_zh: str
summary_zh: str
reason: str
score: float = Field(ge=0, le=1)
tags: list[str] = Field(default_factory=list)
subtitle_priority: str = "normal"
class ClipCandidateResponse(BaseModel):
clips: list[ClipCandidate]
+2
View File
@@ -0,0 +1,2 @@
"""Service adapters for storage, AI, media, subtitles, and upload."""
+60
View File
@@ -0,0 +1,60 @@
from pathlib import Path
from sqlalchemy.orm import Session
from evanescere.config import get_settings
from evanescere.models import Artifact
def video_work_dir(video_id: int) -> Path:
root = get_settings().local_storage_root / "videos" / str(video_id)
root.mkdir(parents=True, exist_ok=True)
return root
def clip_work_dir(video_id: int, clip_id: int) -> Path:
root = video_work_dir(video_id) / "clips" / str(clip_id)
root.mkdir(parents=True, exist_ok=True)
return root
def register_artifact(
session: Session,
*,
artifact_type: str,
local_path: Path,
video_id: int | None = None,
clip_id: int | None = None,
preserve: bool = True,
metadata: dict | None = None,
) -> Artifact:
artifact = Artifact(
video_id=video_id,
clip_id=clip_id,
artifact_type=artifact_type,
local_path=str(local_path),
preserve=preserve,
artifact_metadata=metadata or {},
)
session.add(artifact)
session.flush()
return artifact
def latest_artifact(session: Session, video_id: int, artifact_type: str) -> Artifact | None:
return (
session.query(Artifact)
.filter(Artifact.video_id == video_id, Artifact.artifact_type == artifact_type)
.order_by(Artifact.created_at.desc())
.first()
)
def latest_clip_artifact(session: Session, clip_id: int, artifact_type: str) -> Artifact | None:
return (
session.query(Artifact)
.filter(Artifact.clip_id == clip_id, Artifact.artifact_type == artifact_type)
.order_by(Artifact.created_at.desc())
.first()
)
+85
View File
@@ -0,0 +1,85 @@
from __future__ import annotations
import logging
from pathlib import Path
from typing import Any
import httpx
from evanescere.config import get_settings
logger = logging.getLogger(__name__)
class FunAsrClient:
def __init__(self, base_url: str, api_key: str | None, model: str) -> None:
self.base_url = base_url.rstrip("/")
self.api_key = api_key
self.model = model
@classmethod
def from_settings(cls) -> FunAsrClient:
settings = get_settings()
return cls(settings.funasr_base_url, settings.funasr_api_key, settings.funasr_model)
def transcribe(self, audio_path: Path) -> dict[str, Any]:
logger.info("funasr request start base_url=%s model=%s audio=%s", self.base_url, self.model, audio_path)
headers = {}
if self.api_key:
headers["Authorization"] = f"Bearer {self.api_key}"
with open(audio_path, "rb") as audio_file:
response = httpx.post(
f"{self.base_url}/audio/transcriptions",
headers=headers,
data={
"model": self.model,
"language": "zh",
"response_format": "verbose_json",
"timestamp_granularities[]": "segment",
},
files={"file": (audio_path.name, audio_file, "audio/wav")},
timeout=None,
)
response.raise_for_status()
payload = response.json()
logger.info("funasr request done audio=%s keys=%s", audio_path, sorted(payload.keys()))
return payload
def normalize_segments(payload: dict[str, Any]) -> list[dict[str, Any]]:
raw_segments = payload.get("segments") or payload.get("sentence_info") or []
logger.debug("normalizing asr segments raw_count=%s", len(raw_segments))
normalized: list[dict[str, Any]] = []
for item in raw_segments:
start = item.get("start") or item.get("start_sec") or item.get("timestamp", [0, 0])[0]
end = item.get("end") or item.get("end_sec") or item.get("timestamp", [start, start])[1]
if isinstance(start, int) and start > 10_000:
start = start / 1000
if isinstance(end, int) and end > 10_000:
end = end / 1000
text = item.get("text") or item.get("sentence") or item.get("raw_text") or ""
if not text.strip():
continue
normalized.append(
{
"start_sec": float(start),
"end_sec": float(end),
"text": text.strip(),
"speaker": item.get("speaker"),
"confidence": item.get("confidence"),
"metadata": item,
}
)
if not normalized and payload.get("text"):
normalized.append(
{
"start_sec": 0.0,
"end_sec": 0.1,
"text": str(payload["text"]).strip(),
"speaker": None,
"confidence": None,
"metadata": payload,
}
)
logger.info("normalized asr segments count=%s", len(normalized))
return normalized
+157
View File
@@ -0,0 +1,157 @@
from __future__ import annotations
import json
import logging
from dataclasses import dataclass
from openai import OpenAI
from pydantic import ValidationError
from evanescere.config import get_settings
from evanescere.models import TranscriptSegment
from evanescere.schemas import ClipCandidate, ClipCandidateResponse
logger = logging.getLogger(__name__)
@dataclass(frozen=True)
class TranscriptChunk:
start_sec: float
end_sec: float
text: str
class DeepSeekClipClient:
def __init__(self, api_key: str, base_url: str, model: str, temperature: float) -> None:
self.client = OpenAI(api_key=api_key, base_url=base_url)
self.model = model
self.temperature = temperature
@classmethod
def from_settings(cls) -> DeepSeekClipClient:
settings = get_settings()
if not settings.deepseek_api_key:
raise RuntimeError("[deepseek].api_key is required for clip suggestion.")
return cls(
settings.deepseek_api_key,
settings.deepseek_base_url,
settings.deepseek_model,
settings.deepseek_temperature,
)
def suggest(self, chunks: list[TranscriptChunk]) -> list[ClipCandidate]:
logger.info("deepseek suggestion start chunks=%s model=%s", len(chunks), self.model)
candidates: list[ClipCandidate] = []
for chunk in chunks:
candidates.extend(self._suggest_for_chunk(chunk))
ranked = dedupe_and_rank_candidates(candidates)
logger.info("deepseek suggestion done raw_candidates=%s ranked_candidates=%s", len(candidates), len(ranked))
return ranked
def _suggest_for_chunk(self, chunk: TranscriptChunk) -> list[ClipCandidate]:
logger.info(
"deepseek chunk request start start=%.1f end=%.1f chars=%s",
chunk.start_sec,
chunk.end_sec,
len(chunk.text),
)
response = self.client.chat.completions.create(
model=self.model,
temperature=self.temperature,
response_format={"type": "json_object"},
messages=[
{
"role": "system",
"content": (
"You select interesting short clips from Mandarin livestream transcripts. "
"Return strict JSON only. Prefer complete moments with context, funny reactions, "
"surprising reveals, emotional peaks, or strong standalone discussion."
),
},
{
"role": "user",
"content": (
"Find up to 5 clip candidates in this transcript chunk. "
"Each clip must be 30-360 seconds and use absolute stream seconds. "
"JSON schema: {\"clips\":[{\"start_sec\":number,\"end_sec\":number,"
"\"title_zh\":string,\"summary_zh\":string,\"reason\":string,"
"\"score\":number,\"tags\":[string],\"subtitle_priority\":string}]}.\n\n"
f"Chunk bounds: {chunk.start_sec:.1f}-{chunk.end_sec:.1f}\n"
f"Transcript:\n{chunk.text}"
),
},
],
)
content = response.choices[0].message.content or "{}"
candidates = parse_clip_response(content)
usage = getattr(response, "usage", None)
logger.info(
"deepseek chunk request done start=%.1f end=%.1f candidates=%s usage=%s",
chunk.start_sec,
chunk.end_sec,
len(candidates),
usage,
)
return candidates
def segment_line(segment: TranscriptSegment) -> str:
return f"[{segment.start_sec:.1f}-{segment.end_sec:.1f}] {segment.text}"
def chunk_transcript(
segments: list[TranscriptSegment], chunk_seconds: int
) -> list[TranscriptChunk]:
chunks: list[TranscriptChunk] = []
current: list[str] = []
current_start: float | None = None
current_end: float | None = None
for segment in segments:
if current_start is None:
current_start = segment.start_sec
if current_end is not None and segment.end_sec - current_start > chunk_seconds:
chunks.append(
TranscriptChunk(current_start, current_end, "\n".join(current))
)
current = []
current_start = segment.start_sec
current.append(segment_line(segment))
current_end = segment.end_sec
if current and current_start is not None and current_end is not None:
chunks.append(TranscriptChunk(current_start, current_end, "\n".join(current)))
logger.debug("chunked transcript segments=%s chunk_seconds=%s chunks=%s", len(segments), chunk_seconds, len(chunks))
return chunks
def parse_clip_response(content: str) -> list[ClipCandidate]:
try:
payload = json.loads(content)
parsed = ClipCandidateResponse.model_validate(payload)
except (json.JSONDecodeError, ValidationError) as exc:
logger.error("invalid clip json content_prefix=%s", content[:1000])
raise ValueError(f"Invalid clip JSON from LLM: {exc}") from exc
return parsed.clips
def overlap_ratio(left: ClipCandidate, right: ClipCandidate) -> float:
overlap = max(0, min(left.end_sec, right.end_sec) - max(left.start_sec, right.start_sec))
shortest = min(left.end_sec - left.start_sec, right.end_sec - right.start_sec)
if shortest <= 0:
return 0
return overlap / shortest
def dedupe_and_rank_candidates(candidates: list[ClipCandidate]) -> list[ClipCandidate]:
valid = [
candidate
for candidate in candidates
if candidate.end_sec > candidate.start_sec
and 30 <= candidate.end_sec - candidate.start_sec <= 360
]
ranked = sorted(valid, key=lambda item: item.score, reverse=True)
chosen: list[ClipCandidate] = []
for candidate in ranked:
if all(overlap_ratio(candidate, existing) < 0.5 for existing in chosen):
chosen.append(candidate)
logger.debug("dedupe candidates input=%s valid=%s chosen=%s", len(candidates), len(valid), len(chosen))
return chosen
+201
View File
@@ -0,0 +1,201 @@
from __future__ import annotations
import json
import logging
import subprocess
from pathlib import Path
from sqlalchemy.orm import Session
from evanescere.models import ClipSuggestion, Video
from evanescere.services.artifacts import clip_work_dir, register_artifact, video_work_dir
logger = logging.getLogger(__name__)
def run_command(args: list[str]) -> None:
logger.debug("command start args=%s", args)
completed = subprocess.run(args, check=False, text=True, capture_output=True)
if completed.returncode != 0:
logger.error("command failed returncode=%s stderr=%s", completed.returncode, completed.stderr[-4000:])
raise RuntimeError(
f"Command failed ({completed.returncode}): {' '.join(args)}\n{completed.stderr}"
)
logger.debug("command done args=%s stderr_tail=%s", args, completed.stderr[-1000:])
def ffprobe(path: Path) -> dict:
logger.debug("ffprobe start path=%s", path)
completed = subprocess.run(
[
"ffprobe",
"-v",
"error",
"-print_format",
"json",
"-show_format",
"-show_streams",
str(path),
],
check=True,
text=True,
capture_output=True,
)
metadata = json.loads(completed.stdout)
logger.debug("ffprobe done path=%s streams=%s", path, len(metadata.get("streams", [])))
return metadata
def duration_from_probe(metadata: dict) -> float | None:
duration = metadata.get("format", {}).get("duration")
return float(duration) if duration else None
def remux_to_mp4(session: Session, video: Video, source_path: Path) -> Path:
output = video_work_dir(video.id) / f"{Path(video.filename).stem}.mp4"
logger.info("remux start video_id=%s source=%s output=%s", video.id, source_path, output)
try:
run_command(["ffmpeg", "-y", "-i", str(source_path), "-map", "0", "-c", "copy", str(output)])
except RuntimeError:
logger.warning("remux copy failed, falling back to transcode video_id=%s", video.id)
run_command(
[
"ffmpeg",
"-y",
"-i",
str(source_path),
"-c:v",
"libx264",
"-preset",
"veryfast",
"-crf",
"20",
"-c:a",
"aac",
str(output),
]
)
metadata = ffprobe(output)
video.duration_sec = duration_from_probe(metadata)
video.codec_metadata = metadata
register_artifact(session, video_id=video.id, artifact_type="remux_mp4", local_path=output)
logger.info("remux done video_id=%s duration=%s output=%s", video.id, video.duration_sec, output)
return output
def extract_audio(session: Session, video: Video, media_path: Path) -> Path:
output = video_work_dir(video.id) / "audio_16k_mono.wav"
logger.info("audio extraction start video_id=%s source=%s output=%s", video.id, media_path, output)
run_command(
[
"ffmpeg",
"-y",
"-i",
str(media_path),
"-vn",
"-ac",
"1",
"-ar",
"16000",
"-c:a",
"pcm_s16le",
str(output),
]
)
register_artifact(session, video_id=video.id, artifact_type="audio_wav", local_path=output)
logger.info("audio extraction done video_id=%s output=%s", video.id, output)
return output
def render_clip(
session: Session,
*,
video: Video,
clip: ClipSuggestion,
source_mp4: Path,
srt_path: Path,
ass_path: Path,
bake_subtitles: bool,
preserve: bool,
) -> list[Path]:
output_dir = clip_work_dir(video.id, clip.id)
logger.info(
"render clip media start video_id=%s clip_id=%s start=%.3f end=%.3f bake=%s",
video.id,
clip.id,
clip.start_sec,
clip.end_sec,
bake_subtitles,
)
base_args = [
"ffmpeg",
"-y",
"-ss",
f"{clip.start_sec:.3f}",
"-to",
f"{clip.end_sec:.3f}",
"-i",
str(source_mp4),
]
rendered: list[Path] = []
soft_path = output_dir / "clip_soft_sub.mp4"
run_command(base_args + ["-c", "copy", str(soft_path)])
register_artifact(
session,
video_id=video.id,
clip_id=clip.id,
artifact_type="clip_soft_sub_mp4",
local_path=soft_path,
preserve=preserve,
metadata={"srt": str(srt_path), "ass": str(ass_path)},
)
rendered.append(soft_path)
if bake_subtitles:
baked_path = output_dir / "clip_baked_sub.mp4"
run_command(
base_args
+ [
"-vf",
f"ass={ass_path}",
"-c:v",
"libx264",
"-preset",
"veryfast",
"-crf",
"20",
"-c:a",
"aac",
str(baked_path),
]
)
register_artifact(
session,
video_id=video.id,
clip_id=clip.id,
artifact_type="clip_baked_sub_mp4",
local_path=baked_path,
preserve=preserve,
metadata={"srt": str(srt_path), "ass": str(ass_path)},
)
rendered.append(baked_path)
logger.info("render clip media done video_id=%s clip_id=%s outputs=%s", video.id, clip.id, rendered)
return rendered
def extract_thumbnail_frame(session: Session, video: Video, clip: ClipSuggestion, source_mp4: Path) -> Path:
output = clip_work_dir(video.id, clip.id) / "thumbnail_base.jpg"
timestamp = max(clip.start_sec, (clip.start_sec + clip.end_sec) / 2)
logger.info("thumbnail extraction start video_id=%s clip_id=%s timestamp=%.3f", video.id, clip.id, timestamp)
run_command(["ffmpeg", "-y", "-ss", f"{timestamp:.3f}", "-i", str(source_mp4), "-frames:v", "1", str(output)])
register_artifact(
session,
video_id=video.id,
clip_id=clip.id,
artifact_type="thumbnail_base",
local_path=output,
)
logger.info("thumbnail extraction done video_id=%s clip_id=%s output=%s", video.id, clip.id, output)
return output
+109
View File
@@ -0,0 +1,109 @@
from pathlib import Path
import logging
from sqlalchemy.orm import Session
from evanescere.models import ClipSuggestion, TranscriptSegment, Video
from evanescere.services.artifacts import clip_work_dir, register_artifact
logger = logging.getLogger(__name__)
def format_srt_time(seconds: float) -> str:
millis = round(seconds * 1000)
hours, remainder = divmod(millis, 3_600_000)
minutes, remainder = divmod(remainder, 60_000)
secs, millis = divmod(remainder, 1000)
return f"{hours:02}:{minutes:02}:{secs:02},{millis:03}"
def format_ass_time(seconds: float) -> str:
centis = round(seconds * 100)
hours, remainder = divmod(centis, 360_000)
minutes, remainder = divmod(remainder, 6_000)
secs, centis = divmod(remainder, 100)
return f"{hours}:{minutes:02}:{secs:02}.{centis:02}"
def clip_segments(segments: list[TranscriptSegment], clip: ClipSuggestion) -> list[TranscriptSegment]:
return [segment for segment in segments if segment.end_sec > clip.start_sec and segment.start_sec < clip.end_sec]
def write_srt(path: Path, segments: list[TranscriptSegment], offset_sec: float = 0) -> None:
lines: list[str] = []
for index, segment in enumerate(segments, start=1):
start = max(0, segment.start_sec - offset_sec)
end = max(start + 0.1, segment.end_sec - offset_sec)
lines.extend(
[
str(index),
f"{format_srt_time(start)} --> {format_srt_time(end)}",
segment.text.strip(),
"",
]
)
path.write_text("\n".join(lines), encoding="utf-8")
def write_ass(path: Path, segments: list[TranscriptSegment], offset_sec: float = 0) -> None:
header = """[Script Info]
ScriptType: v4.00+
WrapStyle: 0
ScaledBorderAndShadow: yes
[V4+ Styles]
Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding
Style: Default,Noto Sans CJK SC,48,&H00FFFFFF,&H000000FF,&H00000000,&H80000000,0,0,0,0,100,100,0,0,1,3,1,2,80,80,60,1
[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
"""
events = []
for segment in segments:
start = max(0, segment.start_sec - offset_sec)
end = max(start + 0.1, segment.end_sec - offset_sec)
text = segment.text.replace("\n", r"\N").replace(",", ",")
events.append(
f"Dialogue: 0,{format_ass_time(start)},{format_ass_time(end)},Default,,0,0,0,,{text}"
)
path.write_text(header + "\n".join(events) + "\n", encoding="utf-8")
def generate_clip_subtitles(
session: Session,
*,
video: Video,
clip: ClipSuggestion,
segments: list[TranscriptSegment],
preserve: bool,
) -> tuple[Path, Path]:
selected = clip_segments(segments, clip)
logger.info(
"subtitle generation start video_id=%s clip_id=%s selected_segments=%s",
video.id,
clip.id,
len(selected),
)
output_dir = clip_work_dir(video.id, clip.id)
srt = output_dir / "clip.srt"
ass = output_dir / "clip.ass"
write_srt(srt, selected, offset_sec=clip.start_sec)
write_ass(ass, selected, offset_sec=clip.start_sec)
register_artifact(
session,
video_id=video.id,
clip_id=clip.id,
artifact_type="subtitle_srt",
local_path=srt,
preserve=preserve,
)
register_artifact(
session,
video_id=video.id,
clip_id=clip.id,
artifact_type="subtitle_ass",
local_path=ass,
preserve=preserve,
)
logger.info("subtitle generation done video_id=%s clip_id=%s srt=%s ass=%s", video.id, clip.id, srt, ass)
return srt, ass
+270
View File
@@ -0,0 +1,270 @@
from __future__ import annotations
import json
import logging
import shlex
import subprocess
from pathlib import Path
from sqlalchemy.orm import Session
from evanescere.config import get_settings
from evanescere.models import ClipSuggestion, TranscriptSegment, Video
from evanescere.services.artifacts import clip_work_dir, register_artifact
from evanescere.services.media import extract_thumbnail_frame
logger = logging.getLogger(__name__)
FONT_CANDIDATES = [
"/usr/share/fonts/opentype/noto/NotoSansCJK-Bold.ttc",
"/usr/share/fonts/opentype/noto/NotoSansCJK-Regular.ttc",
"/usr/share/fonts/truetype/noto/NotoSansCJK-Regular.ttc",
"/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf",
]
def generate_thumbnail(
session: Session,
*,
video: Video,
clip: ClipSuggestion,
source_mp4: Path,
transcript_segments: list[TranscriptSegment],
) -> Path | None:
settings = get_settings()
if not settings.thumbnail_enabled:
logger.info("thumbnail disabled video_id=%s clip_id=%s", video.id, clip.id)
return None
output_dir = clip_work_dir(video.id, clip.id)
base_frame = extract_thumbnail_frame(session, video, clip, source_mp4)
provider = settings.thumbnail_provider.lower()
logger.info("thumbnail generation start video_id=%s clip_id=%s provider=%s", video.id, clip.id, provider)
if provider == "frame_overlay":
background = base_frame
elif provider == "command":
background = run_thumbnail_command(
output_dir=output_dir,
source_frame=base_frame,
video=video,
clip=clip,
transcript_segments=transcript_segments,
)
register_artifact(
session,
video_id=video.id,
clip_id=clip.id,
artifact_type="thumbnail_generated",
local_path=background,
metadata={"provider": "command"},
)
else:
raise RuntimeError(f"Unsupported [thumbnail].provider={settings.thumbnail_provider!r}")
final_path = output_dir / "thumbnail_final.jpg"
character_path = (
Path(settings.thumbnail_character_overlay_path)
if settings.thumbnail_character_overlay_path
else None
)
compose_thumbnail(
background_path=background,
output_path=final_path,
title=clip.title_zh,
character_overlay_path=character_path,
character_scale=settings.thumbnail_character_scale,
character_position=settings.thumbnail_character_position,
title_enabled=settings.thumbnail_title_enabled,
width=settings.thumbnail_width,
height=settings.thumbnail_height,
)
register_artifact(
session,
video_id=video.id,
clip_id=clip.id,
artifact_type="thumbnail_final",
local_path=final_path,
preserve=True,
metadata={
"provider": provider,
"source_frame": str(base_frame),
"background": str(background),
"character_overlay_path": settings.thumbnail_character_overlay_path or "",
},
)
logger.info("thumbnail generation done video_id=%s clip_id=%s output=%s", video.id, clip.id, final_path)
return final_path
def run_thumbnail_command(
*,
output_dir: Path,
source_frame: Path,
video: Video,
clip: ClipSuggestion,
transcript_segments: list[TranscriptSegment],
) -> Path:
settings = get_settings()
if not settings.thumbnail_command:
raise RuntimeError('[thumbnail].command must be set when [thumbnail].provider = "command"')
output_path = output_dir / "thumbnail_generated.png"
nearby_transcript = [
{
"start_sec": segment.start_sec,
"end_sec": segment.end_sec,
"text": segment.text,
}
for segment in transcript_segments
if segment.end_sec > clip.start_sec and segment.start_sec < clip.end_sec
][:80]
payload = {
"video_id": video.id,
"clip_id": clip.id,
"source_frame": str(source_frame),
"output_path": str(output_path),
"width": settings.thumbnail_width,
"height": settings.thumbnail_height,
"title_zh": clip.title_zh,
"summary_zh": clip.summary_zh,
"reason": clip.reason,
"tags": clip.tags,
"transcript": nearby_transcript,
}
logger.info("thumbnail command start clip_id=%s command=%s output=%s", clip.id, settings.thumbnail_command, output_path)
completed = subprocess.run(
shlex.split(settings.thumbnail_command),
input=json.dumps(payload, ensure_ascii=False),
text=True,
capture_output=True,
check=False,
)
if completed.returncode != 0:
logger.error("thumbnail command failed clip_id=%s stderr=%s", clip.id, completed.stderr[-4000:])
raise RuntimeError(completed.stderr)
if not output_path.exists():
raise RuntimeError(f"Thumbnail command did not create {output_path}")
logger.info("thumbnail command done clip_id=%s stdout=%s", clip.id, completed.stdout[-1000:])
return output_path
def compose_thumbnail(
*,
background_path: Path,
output_path: Path,
title: str,
character_overlay_path: Path | None,
character_scale: float,
character_position: str,
title_enabled: bool,
width: int,
height: int,
) -> None:
from PIL import Image, ImageDraw, ImageEnhance, ImageFont, ImageOps
background = Image.open(background_path).convert("RGB")
canvas = ImageOps.fit(background, (width, height), method=Image.Resampling.LANCZOS)
canvas = ImageEnhance.Color(canvas).enhance(1.12)
canvas = ImageEnhance.Contrast(canvas).enhance(1.08)
rgba = canvas.convert("RGBA")
if title_enabled and title.strip():
draw_title(rgba, title)
if character_overlay_path:
overlay_character(rgba, character_overlay_path, character_scale, character_position)
output_path.parent.mkdir(parents=True, exist_ok=True)
rgba.convert("RGB").save(output_path, quality=92, optimize=True)
def draw_title(image, title: str) -> None:
from PIL import Image, ImageDraw
draw = ImageDraw.Draw(image)
width, height = image.size
font = load_font(max(48, width // 18))
max_text_width = int(width * 0.62)
lines = wrap_text(draw, title, font, max_text_width, max_lines=3)
line_boxes = [draw.textbbox((0, 0), line, font=font, stroke_width=2) for line in lines]
line_height = max((box[3] - box[1] for box in line_boxes), default=font.size)
block_height = line_height * len(lines) + 18 * max(0, len(lines) - 1)
left = int(width * 0.045)
top = int(height * 0.075)
padding_x = 34
padding_y = 26
block_width = min(max_text_width + padding_x * 2, int(width * 0.72))
panel = Image.new("RGBA", (block_width, block_height + padding_y * 2), (0, 0, 0, 150))
image.alpha_composite(panel, (left - padding_x, top - padding_y))
y = top
for line in lines:
draw.text(
(left, y),
line,
font=font,
fill=(255, 255, 255, 255),
stroke_width=3,
stroke_fill=(0, 0, 0, 210),
)
y += line_height + 18
def overlay_character(image, overlay_path: Path, scale: float, position: str) -> None:
from PIL import Image
if not overlay_path.exists():
logger.warning("thumbnail character overlay missing path=%s", overlay_path)
return
overlay = Image.open(overlay_path).convert("RGBA")
width, height = image.size
target_height = int(height * scale)
ratio = target_height / overlay.height
target_size = (max(1, int(overlay.width * ratio)), target_height)
overlay = overlay.resize(target_size, Image.Resampling.LANCZOS)
margin_x = int(width * 0.035)
margin_y = int(height * 0.02)
positions = {
"bottom-right": (width - overlay.width - margin_x, height - overlay.height - margin_y),
"bottom-left": (margin_x, height - overlay.height - margin_y),
"center-right": (width - overlay.width - margin_x, (height - overlay.height) // 2),
"center-left": (margin_x, (height - overlay.height) // 2),
}
x, y = positions.get(position, positions["bottom-right"])
image.alpha_composite(overlay, (max(0, x), max(0, y)))
def load_font(size: int):
from PIL import ImageFont
for candidate in FONT_CANDIDATES:
path = Path(candidate)
if path.exists():
try:
return ImageFont.truetype(str(path), size=size)
except OSError:
continue
return ImageFont.load_default(size=size)
def wrap_text(draw, text: str, font, max_width: int, max_lines: int) -> list[str]:
lines: list[str] = []
current = ""
for char in text.strip():
candidate = current + char
bbox = draw.textbbox((0, 0), candidate, font=font, stroke_width=2)
if bbox[2] - bbox[0] <= max_width or not current:
current = candidate
continue
lines.append(current)
current = char
if len(lines) == max_lines:
break
if current and len(lines) < max_lines:
lines.append(current)
if len(lines) == max_lines and len("".join(lines)) < len(text.strip()):
lines[-1] = lines[-1].rstrip(",。,. ") + "..."
return lines or [text.strip()]
+86
View File
@@ -0,0 +1,86 @@
from __future__ import annotations
import json
import logging
import subprocess
from pathlib import Path
from sqlalchemy.orm import Session
from evanescere.config import get_settings
from evanescere.models import Artifact, ClipSuggestion
logger = logging.getLogger(__name__)
class UploadResult(dict):
pass
def upload_artifact(session: Session, clip: ClipSuggestion, artifact: Artifact) -> UploadResult:
settings = get_settings()
thumbnail = (
session.query(Artifact)
.filter(Artifact.clip_id == clip.id, Artifact.artifact_type == "thumbnail_final")
.order_by(Artifact.created_at.desc())
.first()
)
if settings.upload_adapter == "noop":
logger.info(
"upload noop clip_id=%s artifact_id=%s path=%s thumbnail=%s",
clip.id,
artifact.id,
artifact.local_path,
thumbnail.local_path if thumbnail else None,
)
return UploadResult(
{
"adapter": "noop",
"uploaded": False,
"path": artifact.local_path,
"thumbnail_path": thumbnail.local_path if thumbnail else None,
}
)
if settings.upload_adapter != "command":
raise RuntimeError(f"Unknown upload adapter: {settings.upload_adapter}")
if not settings.upload_command:
raise RuntimeError('[upload].command must be set when [upload].adapter = "command"')
payload = {
"clip_id": clip.id,
"video_id": clip.video_id,
"title": clip.title_zh,
"summary": clip.summary_zh,
"tags": clip.tags,
"file": artifact.local_path,
"thumbnail": thumbnail.local_path if thumbnail else None,
}
logger.info("upload command start clip_id=%s artifact_id=%s command=%s", clip.id, artifact.id, settings.upload_command)
completed = subprocess.run(
settings.upload_command.split(),
input=json.dumps(payload, ensure_ascii=False),
text=True,
capture_output=True,
check=False,
)
if completed.returncode != 0:
logger.error("upload command failed clip_id=%s stderr=%s", clip.id, completed.stderr)
raise RuntimeError(completed.stderr)
logger.info("upload command done clip_id=%s stdout=%s", clip.id, completed.stdout[-1000:])
return UploadResult(
{
"adapter": "command",
"uploaded": True,
"path": artifact.local_path,
"stdout": completed.stdout,
}
)
def choose_upload_artifact(artifacts: list[Artifact]) -> Artifact | None:
by_type = {artifact.artifact_type: artifact for artifact in artifacts}
return by_type.get("clip_baked_sub_mp4") or by_type.get("clip_soft_sub_mp4")
def artifact_path(artifact: Artifact) -> Path:
return Path(artifact.local_path)
+177
View File
@@ -0,0 +1,177 @@
from __future__ import annotations
import logging
from dataclasses import dataclass
from datetime import UTC, datetime
from pathlib import PurePosixPath
from typing import Any
from urllib.parse import quote, urljoin, urlparse
from xml.etree import ElementTree
import httpx
from sqlalchemy.orm import Session
from evanescere.config import get_settings
from evanescere.models import Video
VIDEO_SUFFIXES = {".flv", ".mp4", ".mkv", ".mov"}
logger = logging.getLogger(__name__)
@dataclass(frozen=True)
class WebDavFile:
url: str
filename: str
size_bytes: int
modified_at: str | None = None
class WebDavClient:
def __init__(
self,
base_url: str,
username: str | None = None,
password: str | None = None,
verify_tls: bool = True,
) -> None:
self.base_url = str(base_url).rstrip("/") + "/"
self.auth = (username, password) if username and password else None
self.verify_tls = verify_tls
@classmethod
def from_settings(cls) -> WebDavClient:
settings = get_settings()
return cls(
str(settings.webdav_base_url),
settings.webdav_username,
settings.webdav_password,
settings.webdav_verify_tls,
)
def list_files(self) -> list[WebDavFile]:
logger.debug("webdav propfind start base_url=%s", self.base_url)
body = """<?xml version="1.0" encoding="utf-8" ?>
<D:propfind xmlns:D="DAV:">
<D:prop>
<D:getcontentlength/>
<D:getlastmodified/>
<D:resourcetype/>
</D:prop>
</D:propfind>"""
with httpx.Client(auth=self.auth, verify=self.verify_tls, timeout=30) as client:
response = client.request("PROPFIND", self.base_url, headers={"Depth": "1"}, content=body)
response.raise_for_status()
files = parse_propfind_response(response.text, self.base_url)
logger.info("webdav propfind done base_url=%s files=%s", self.base_url, len(files))
return files
def download(self, url: str, destination) -> None:
logger.info("webdav download start url=%s destination=%s", url, destination)
with httpx.stream("GET", url, auth=self.auth, verify=self.verify_tls, timeout=None) as response:
response.raise_for_status()
with open(destination, "wb") as out_file:
for chunk in response.iter_bytes():
out_file.write(chunk)
logger.info("webdav download done url=%s destination=%s", url, destination)
def upload(self, source, relative_path: str) -> str:
destination = urljoin(self.base_url, quote(relative_path.lstrip("/")))
logger.info("webdav upload start source=%s destination=%s", source, destination)
with open(source, "rb") as in_file:
response = httpx.put(
destination,
content=in_file,
auth=self.auth,
verify=self.verify_tls,
timeout=None,
)
response.raise_for_status()
logger.info("webdav upload done source=%s destination=%s", source, destination)
return destination
def parse_propfind_response(xml_text: str, base_url: str) -> list[WebDavFile]:
ns = {"d": "DAV:"}
root = ElementTree.fromstring(xml_text)
base_path = urlparse(base_url).path.rstrip("/")
files: list[WebDavFile] = []
for response in root.findall("d:response", ns):
href = response.findtext("d:href", default="", namespaces=ns)
href_path = urlparse(href).path
if href_path.rstrip("/") == base_path:
continue
filename = PurePosixPath(href_path).name
if not filename or PurePosixPath(filename).suffix.lower() not in VIDEO_SUFFIXES:
continue
resource_type = response.find(".//d:resourcetype", ns)
if resource_type is not None and list(resource_type):
continue
size_text = response.findtext(".//d:getcontentlength", default="0", namespaces=ns)
modified = response.findtext(".//d:getlastmodified", default=None, namespaces=ns)
file_url = urljoin(base_url.rstrip("/") + "/", quote(filename))
files.append(WebDavFile(file_url, filename, int(size_text or 0), modified))
return files
def add_size_sample(video: Video, size_bytes: int) -> None:
samples: list[dict[str, Any]] = list(video.size_samples or [])
samples.append({"at": datetime.now(UTC).isoformat(), "size_bytes": size_bytes})
video.size_samples = samples[-10:]
video.size_bytes = size_bytes
def has_stable_size(video: Video, required_equal_samples: int = 2) -> bool:
samples = video.size_samples or []
if len(samples) < required_equal_samples:
return False
recent = samples[-required_equal_samples:]
sizes = {sample["size_bytes"] for sample in recent}
return len(sizes) == 1 and next(iter(sizes)) > 0
def bootstrap_existing(client: WebDavClient, session: Session) -> int:
count = 0
for file in client.list_files():
video = session.query(Video).filter(Video.source_url == file.url).one_or_none()
if video is None:
video = Video(
source_url=file.url,
filename=file.filename,
size_bytes=file.size_bytes,
size_samples=[{"at": datetime.now(UTC).isoformat(), "size_bytes": file.size_bytes}],
ingest_status="existing_done",
processing_status="done",
)
session.add(video)
count += 1
logger.debug("bootstrap existing file=%s size=%s", file.filename, file.size_bytes)
session.flush()
logger.info("bootstrap existing done inserted=%s", count)
return count
def scan_once(client: WebDavClient, session: Session) -> tuple[int, int]:
observed = 0
newly_stable = 0
for file in client.list_files():
observed += 1
video = session.query(Video).filter(Video.source_url == file.url).one_or_none()
if video is None:
logger.info("new webdav file observed file=%s size=%s", file.filename, file.size_bytes)
video = Video(
source_url=file.url,
filename=file.filename,
ingest_status="observing",
processing_status="pending",
)
session.add(video)
if video.ingest_status in {"existing_done", "stable", "queued"}:
continue
add_size_sample(video, file.size_bytes)
logger.debug("webdav size sample video_id=%s file=%s size=%s", video.id, file.filename, file.size_bytes)
if has_stable_size(video):
video.ingest_status = "stable"
newly_stable += 1
logger.info("webdav file stable video_id=%s file=%s size=%s", video.id, file.filename, file.size_bytes)
session.flush()
return observed, newly_stable
+39
View File
@@ -0,0 +1,39 @@
from sqlalchemy.orm import Session
from evanescere.config import get_settings
from evanescere.models import Setting
from evanescere.schemas import SettingsPatch, SettingsRead
SETTINGS_KEY = "pipeline"
def default_settings() -> SettingsRead:
settings = get_settings()
return SettingsRead(
suggest_enabled=settings.default_suggest_enabled,
render_enabled=settings.default_render_enabled,
upload_enabled=settings.default_upload_enabled,
preserve_final_artifacts=settings.default_preserve_final_artifacts,
bake_subtitles=settings.default_bake_subtitles,
)
def get_pipeline_settings(session: Session) -> SettingsRead:
row = session.get(Setting, SETTINGS_KEY)
if row is None:
return default_settings()
return SettingsRead(**(default_settings().model_dump() | row.value))
def patch_pipeline_settings(session: Session, patch: SettingsPatch) -> SettingsRead:
current = get_pipeline_settings(session).model_dump()
current.update({k: v for k, v in patch.model_dump().items() if v is not None})
row = session.get(Setting, SETTINGS_KEY)
if row is None:
row = Setting(key=SETTINGS_KEY, value=current)
session.add(row)
else:
row.value = current
session.flush()
return SettingsRead(**current)
+64
View File
@@ -0,0 +1,64 @@
from evanescere.schemas import ClipCandidate
from evanescere.services.llm import dedupe_and_rank_candidates, parse_clip_response
def test_parse_clip_response():
candidates = parse_clip_response(
"""
{
"clips": [
{
"start_sec": 10,
"end_sec": 60,
"title_zh": "标题",
"summary_zh": "摘要",
"reason": "有趣",
"score": 0.9,
"tags": ["反应"],
"subtitle_priority": "high"
}
]
}
"""
)
assert candidates[0].title_zh == "标题"
def test_dedupe_and_rank_candidates_filters_short_and_overlapping():
candidates = [
ClipCandidate(
start_sec=0,
end_sec=20,
title_zh="too short",
summary_zh="",
reason="",
score=1,
),
ClipCandidate(
start_sec=0,
end_sec=60,
title_zh="best",
summary_zh="",
reason="",
score=0.9,
),
ClipCandidate(
start_sec=10,
end_sec=65,
title_zh="overlap",
summary_zh="",
reason="",
score=0.8,
),
ClipCandidate(
start_sec=120,
end_sec=180,
title_zh="second",
summary_zh="",
reason="",
score=0.7,
),
]
ranked = dedupe_and_rank_candidates(candidates)
assert [candidate.title_zh for candidate in ranked] == ["best", "second"]
+10
View File
@@ -0,0 +1,10 @@
from evanescere.services.subtitles import format_ass_time, format_srt_time
def test_format_srt_time():
assert format_srt_time(3661.234) == "01:01:01,234"
def test_format_ass_time():
assert format_ass_time(3661.23) == "1:01:01.23"
+35
View File
@@ -0,0 +1,35 @@
from evanescere.services.webdav import WebDavFile, has_stable_size, parse_propfind_response
class DummyVideo:
def __init__(self, samples):
self.size_samples = samples
def test_has_stable_size_requires_two_equal_positive_samples():
assert not has_stable_size(DummyVideo([]))
assert not has_stable_size(DummyVideo([{"size_bytes": 100}]))
assert not has_stable_size(DummyVideo([{"size_bytes": 100}, {"size_bytes": 101}]))
assert has_stable_size(DummyVideo([{"size_bytes": 100}, {"size_bytes": 100}]))
def test_parse_propfind_response_filters_video_files():
xml = """<?xml version="1.0"?>
<D:multistatus xmlns:D="DAV:">
<D:response>
<D:href>/webdav/recordings/</D:href>
<D:propstat><D:prop><D:resourcetype><D:collection/></D:resourcetype></D:prop></D:propstat>
</D:response>
<D:response>
<D:href>/webdav/recordings/stream.flv</D:href>
<D:propstat><D:prop><D:getcontentlength>123</D:getcontentlength></D:prop></D:propstat>
</D:response>
<D:response>
<D:href>/webdav/recordings/readme.txt</D:href>
<D:propstat><D:prop><D:getcontentlength>12</D:getcontentlength></D:prop></D:propstat>
</D:response>
</D:multistatus>"""
assert parse_propfind_response(xml, "https://host/webdav/recordings") == [
WebDavFile("https://host/webdav/recordings/stream.flv", "stream.flv", 123, None)
]