This commit is contained in:
2026-05-31 23:30:35 -07:00
parent 431ffdff06
commit 3e50191530
20 changed files with 539 additions and 78 deletions
+53 -15
View File
@@ -3,12 +3,13 @@
Evanescere turns recorded livestreams into suggested, rendered clips: Evanescere turns recorded livestreams into suggested, rendered clips:
1. Poll IIS WebDAV for finished recordings. 1. Poll IIS WebDAV for finished recordings.
2. Wait until file size is unchanged across polling cycles. 2. Record files visible during the first scan as a manual-only historical baseline.
3. Download the stable source file to Framework-local storage. 3. For files first seen later, wait until file size is unchanged across polling cycles.
4. Remux FLV/H264 to MP4 and extract ASR-ready audio. 4. Download the stable source file to Framework-local storage.
5. Transcribe Mandarin audio through a FunASR-compatible API. 5. Remux FLV/H264 to MP4 and extract ASR-ready audio.
6. Ask DeepSeek for ranked timeline-aware clip suggestions. 6. Transcribe Mandarin audio through a FunASR-compatible API.
7. Generate subtitles, render clips, generate thumbnails with optional local image generation and VTuber overlay, and optionally upload. 7. Ask DeepSeek for ranked timeline-aware clip suggestions.
8. Generate subtitles, render clips, generate thumbnails with optional local image generation and VTuber overlay, and optionally upload.
The Framework Arch server is the intended production host. The Hyper-V Arch VM can be used for development, API testing, and database/control-plane work. The Framework Arch server is the intended production host. The Hyper-V Arch VM can be used for development, API testing, and database/control-plane work.
@@ -28,7 +29,8 @@ Runtime data flow:
```text ```text
IIS WebDAV recordings IIS WebDAV recordings
-> scheduler polls file size -> first scan records existing files as manual-only baseline entries
-> later scans poll new file sizes
-> PostgreSQL records video state -> PostgreSQL records video state
-> Redis queues pipeline job -> Redis queues pipeline job
-> worker downloads source to configured storage.local_root -> worker downloads source to configured storage.local_root
@@ -67,8 +69,8 @@ The checked-in `docker-compose.yml` mounts local `./config.toml` there for the A
## Services And Ports ## Services And Ports
- Frontend: `http://localhost:3000` - Frontend: `http://localhost:3000`
- Backend API: `http://localhost:8000` - Backend API: `http://localhost:8080`
- API docs: `http://localhost:8000/docs` - API docs: `http://localhost:8080/docs`
- Redis: `localhost:6379` when exposed by Compose - Redis: `localhost:6379` when exposed by Compose
- PostgreSQL: only started by Compose when using the `local-db` profile - PostgreSQL: only started by Compose when using the `local-db` profile
@@ -92,12 +94,14 @@ Run database migrations:
docker compose run --rm api alembic upgrade head docker compose run --rm api alembic upgrade head
``` ```
Bootstrap existing WebDAV files so old recordings are marked `existing_done` instead of auto-processed: The scheduler automatically records files visible during its first scan as `existing_done`. Those historical recordings appear in the UI but are not automatically processed. To establish that baseline manually before starting the scheduler:
```bash ```bash
docker compose run --rm api evanescere bootstrap-existing docker compose run --rm api evanescere bootstrap-existing
``` ```
In the frontend, click `Sync WebDAV` to perform the same safe first-time baseline import or to discover later files. Select any historical FLV row and click its play button to manually queue a test pipeline run.
Useful test commands: Useful test commands:
```bash ```bash
@@ -141,7 +145,7 @@ The frontend reads the backend URL from `frontend/public/config.js` at runtime:
```js ```js
window.__EVANESCERE_FRONTEND_CONFIG__ = { window.__EVANESCERE_FRONTEND_CONFIG__ = {
apiBaseUrl: "http://localhost:8000" apiBaseUrl: "http://192.168.1.44:8080"
}; };
``` ```
@@ -181,11 +185,11 @@ provider = "frame_overlay"
This uses the extracted frame as the background, then programmatically overlays the title and character PNG using Pillow. This uses the extracted frame as the background, then programmatically overlays the title and character PNG using Pillow.
To overlay the VTuber character, use a transparent PNG: To overlay the VTuber character, place transparent PNG variants in a directory. Evanescere picks one randomly for each thumbnail:
```toml ```toml
[thumbnail] [thumbnail]
character_overlay_path = "/data/evanescere/assets/vtuber.png" character_overlay_dir = "/data/evanescere/assets/"
character_scale = 0.42 character_scale = 0.42
character_position = "bottom-right" character_position = "bottom-right"
``` ```
@@ -277,6 +281,8 @@ The full commented example lives in `config.example.toml`. These are the keys th
| `deepseek` | `api_key` | empty | DeepSeek API key. Required when clip suggestion is enabled. | | `deepseek` | `api_key` | empty | DeepSeek API key. Required when clip suggestion is enabled. |
| `deepseek` | `model` | `deepseek-v4-pro` | Model used for clip suggestion. | | `deepseek` | `model` | `deepseek-v4-pro` | Model used for clip suggestion. |
| `deepseek` | `temperature` | `0.2` | Sampling temperature for clip suggestion. | | `deepseek` | `temperature` | `0.2` | Sampling temperature for clip suggestion. |
| `llm_prompt` | `system` | VTuber clip editor prompt | Custom system prompt for clip selection. The required JSON schema is appended by code. |
| `llm_prompt` | `user` | timestamped transcript prompt | Custom user prompt template. Supports documented placeholders. |
| `defaults` | `suggest_enabled` | `true` | Initial setting for automatic LLM clip suggestion. | | `defaults` | `suggest_enabled` | `true` | Initial setting for automatic LLM clip suggestion. |
| `defaults` | `render_enabled` | `true` | Initial setting for automatic rendering. | | `defaults` | `render_enabled` | `true` | Initial setting for automatic rendering. |
| `defaults` | `upload_enabled` | `true` | Initial setting for automatic upload after render. For testing, set false. | | `defaults` | `upload_enabled` | `true` | Initial setting for automatic upload after render. For testing, set false. |
@@ -285,11 +291,12 @@ The full commented example lives in `config.example.toml`. These are the keys th
| `clip` | `min_seconds` | `30` | Minimum LLM clip duration accepted by backend. | | `clip` | `min_seconds` | `30` | Minimum LLM clip duration accepted by backend. |
| `clip` | `max_seconds` | `360` | Maximum LLM clip duration accepted by backend. | | `clip` | `max_seconds` | `360` | Maximum LLM clip duration accepted by backend. |
| `clip` | `transcript_chunk_seconds` | `900` | Transcript seconds sent to DeepSeek per request. | | `clip` | `transcript_chunk_seconds` | `900` | Transcript seconds sent to DeepSeek per request. |
| `clip` | `max_candidates_total` | `20` | Maximum suggestions kept across the entire transcript after overlap deduplication. |
| `thumbnail` | `enabled` | `true` | Enables thumbnail generation during clip render. | | `thumbnail` | `enabled` | `true` | Enables thumbnail generation during clip render. |
| `thumbnail` | `provider` | `frame_overlay` | `frame_overlay` or `command`. | | `thumbnail` | `provider` | `frame_overlay` | `frame_overlay` or `command`. |
| `thumbnail` | `width` | `1920` | Final thumbnail width in pixels. | | `thumbnail` | `width` | `1920` | Final thumbnail width in pixels. |
| `thumbnail` | `height` | `1080` | Final thumbnail height in pixels. | | `thumbnail` | `height` | `1080` | Final thumbnail height in pixels. |
| `thumbnail` | `character_overlay_path` | empty | Optional transparent PNG of the VTuber character. | | `thumbnail` | `character_overlay_dir` | `/data/evanescere/assets/` | Directory of transparent VTuber PNG variants. One is selected randomly per thumbnail. |
| `thumbnail` | `character_scale` | `0.42` | Character overlay height as a fraction of final thumbnail height. | | `thumbnail` | `character_scale` | `0.42` | Character overlay height as a fraction of final thumbnail height. |
| `thumbnail` | `character_position` | `bottom-right` | Character placement. | | `thumbnail` | `character_position` | `bottom-right` | Character placement. |
| `thumbnail` | `title_enabled` | `true` | Draws the clip title onto the final thumbnail. | | `thumbnail` | `title_enabled` | `true` | Draws the clip title onto the final thumbnail. |
@@ -324,12 +331,42 @@ docker compose run --rm api
Useful API endpoints for generated files: Useful API endpoints for generated files:
- `POST /webdav/scan`
- `GET /videos/{video_id}/artifacts` - `GET /videos/{video_id}/artifacts`
- `GET /clips/{clip_id}/artifacts` - `GET /clips/{clip_id}/artifacts`
- `GET /artifacts/{artifact_id}` - `GET /artifacts/{artifact_id}`
Thumbnail outputs are visible through the artifact endpoints. Upload command payloads include `thumbnail` when a `thumbnail_final` artifact exists. Thumbnail outputs are visible through the artifact endpoints. Upload command payloads include `thumbnail` when a `thumbnail_final` artifact exists.
## Custom LLM Prompt
Edit `[llm_prompt]` in your private `config.toml` to tune clip selection without rebuilding containers:
```toml
[llm_prompt]
system = """
Your custom editorial guidance.
"""
user = """
Choose up to {max_candidates} clips between {min_clip_seconds} and {max_clip_seconds} seconds.
Stream range: {chunk_start_sec}-{chunk_end_sec}
Transcript:
{transcript}
"""
```
Available user-prompt placeholders:
- `{max_candidates}`
- `{min_clip_seconds}`
- `{max_clip_seconds}`
- `{chunk_start_sec}`
- `{chunk_end_sec}`
- `{transcript}`
The backend appends its strict JSON response schema to the system message so custom prompt wording cannot accidentally remove the machine-readable output contract.
## Debugging ## Debugging
Set verbose logs in `config.toml`: Set verbose logs in `config.toml`:
@@ -352,6 +389,7 @@ docker compose logs -f api
What to look for: What to look for:
- Scheduler logs `webdav propfind done`, `new webdav file observed`, and `webdav file stable`. - Scheduler logs `webdav propfind done`, `new webdav file observed`, and `webdav file stable`.
- On first startup, scheduler logs `webdav baseline missing` and stores all currently visible recordings as `existing_done`.
- Worker logs `pipeline start`, `prepare media`, `transcription start`, `clip suggestion start`, `render start`, and `upload start`. - Worker logs `pipeline start`, `prepare media`, `transcription start`, `clip suggestion start`, `render start`, and `upload start`.
- ffmpeg command failures include the failing command and stderr tail. - ffmpeg command failures include the failing command and stderr tail.
- DeepSeek logs include transcript chunk bounds, character counts, candidate counts, and usage when returned by the SDK. - DeepSeek logs include transcript chunk bounds, character counts, candidate counts, and usage when returned by the SDK.
@@ -371,4 +409,4 @@ This is intentionally noisy and should usually stay off in production.
- FunASR/ROCm setup is intentionally outside the main Compose file for now. Benchmark the Framework host manually before binding the project to a specific GPU runtime. - FunASR/ROCm setup is intentionally outside the main Compose file for now. Benchmark the Framework host manually before binding the project to a specific GPU runtime.
- The uploader adapter is deliberately small. `[upload].adapter = "command"` is enough to integrate a Bilibili uploader later without changing the pipeline core. - The uploader adapter is deliberately small. `[upload].adapter = "command"` is enough to integrate a Bilibili uploader later without changing the pipeline core.
- Existing recordings should be bootstrapped before scheduler-driven production runs, otherwise old stable files may be queued as new work. - Existing recordings are automatically baselined on the first scheduler scan. The explicit `evanescere bootstrap-existing` command remains available when you want to establish or refresh the baseline before starting services.
+30 -3
View File
@@ -74,6 +74,30 @@ model = "deepseek-v4-pro"
# Lower values improve consistency. # Lower values improve consistency.
temperature = 0.2 temperature = 0.2
[llm_prompt]
# Customize these prompts freely. The backend appends the required JSON response schema.
system = """
You are an expert editor for Mandarin VTuber livestream clips.
Identify moments that work as entertaining standalone clips for Bilibili viewers.
Prioritize strong reactions, jokes, surprising turns, memorable conversations, and moments with a clear payoff.
Avoid repetitive stretches, dead air, and segments that require too much missing context.
Return only the requested JSON object.
"""
# Available placeholders:
# {max_candidates}, {min_clip_seconds}, {max_clip_seconds},
# {chunk_start_sec}, {chunk_end_sec}, {transcript}
user = """
Review the timestamped livestream transcript below.
Choose up to {max_candidates} compelling clip candidates.
Each clip must be between {min_clip_seconds} and {max_clip_seconds} seconds long.
Use absolute stream timestamps and give each candidate a concise Chinese title and summary.
Transcript range: {chunk_start_sec}-{chunk_end_sec}
Transcript:
{transcript}
"""
[defaults] [defaults]
# Initial automatic pipeline settings. These can later be changed through the API/UI. # Initial automatic pipeline settings. These can later be changed through the API/UI.
suggest_enabled = true suggest_enabled = true
@@ -91,7 +115,10 @@ min_seconds = 30
max_seconds = 360 max_seconds = 360
# Transcript seconds per DeepSeek request. Larger chunks use more tokens. # Transcript seconds per DeepSeek request. Larger chunks use more tokens.
transcript_chunk_seconds = 900 transcript_chunk_seconds = 14400
# Maximum number of suggestions kept across the entire transcript.
max_candidates_total = 20
[thumbnail] [thumbnail]
# Generate thumbnails during clip render. # Generate thumbnails during clip render.
@@ -104,8 +131,8 @@ provider = "frame_overlay"
width = 1920 width = 1920
height = 1080 height = 1080
# Optional transparent PNG of the VTuber character. # Directory of transparent VTuber character PNGs. One is selected randomly per thumbnail.
character_overlay_path = "" character_overlay_dir = "/data/evanescere/assets/"
# Character height as a fraction of thumbnail height. # Character height as a fraction of thumbnail height.
character_scale = 0.42 character_scale = 0.42
+1 -1
View File
@@ -10,7 +10,7 @@ services:
build: . build: .
command: uvicorn evanescere.api:app --host 0.0.0.0 --port 8000 command: uvicorn evanescere.api:app --host 0.0.0.0 --port 8000
ports: ports:
- "8000:8000" - "8080:8000"
volumes: volumes:
- ./config.toml:/etc/evanescere/config.toml:ro - ./config.toml:/etc/evanescere/config.toml:ro
- ./storage:/data/evanescere - ./storage:/data/evanescere
+1 -1
View File
@@ -1,3 +1,3 @@
window.__EVANESCERE_FRONTEND_CONFIG__ = { window.__EVANESCERE_FRONTEND_CONFIG__ = {
apiBaseUrl: "http://localhost:8000" apiBaseUrl: "http://192.168.1.44:8080"
}; };
+31 -2
View File
@@ -1,6 +1,7 @@
import { import {
Check, Check,
Clapperboard, Clapperboard,
FolderSync,
Play, Play,
RefreshCw, RefreshCw,
Save, Save,
@@ -19,6 +20,7 @@ import {
patchSettings, patchSettings,
renderClip, renderClip,
runVideo, runVideo,
scanWebDav,
uploadClip, uploadClip,
} from "./api"; } from "./api";
import type { ClipSuggestion, PipelineSettings, TranscriptSegment, Video } from "./types"; import type { ClipSuggestion, PipelineSettings, TranscriptSegment, Video } from "./types";
@@ -47,7 +49,7 @@ function formatTimeRange(start: number, end: number) {
} }
function statusTone(value: string) { function statusTone(value: string) {
if (["done", "stable", "approved", "auto_approved"].includes(value)) return "good"; if (["done", "stable", "existing_done", "approved", "auto_approved"].includes(value)) return "good";
if (["failed", "error"].includes(value)) return "bad"; if (["failed", "error"].includes(value)) return "bad";
if (["running", "queued", "observing", "pending"].includes(value)) return "busy"; if (["running", "queued", "observing", "pending"].includes(value)) return "busy";
return "neutral"; return "neutral";
@@ -61,6 +63,7 @@ export function App() {
const [clips, setClips] = useState<ClipSuggestion[]>([]); const [clips, setClips] = useState<ClipSuggestion[]>([]);
const [loading, setLoading] = useState(false); const [loading, setLoading] = useState(false);
const [error, setError] = useState<string | null>(null); const [error, setError] = useState<string | null>(null);
const [notice, setNotice] = useState<string | null>(null);
const selectedVideo = useMemo( const selectedVideo = useMemo(
() => videos.find((video) => video.id === selectedVideoId) ?? null, () => videos.find((video) => video.id === selectedVideoId) ?? null,
@@ -124,6 +127,26 @@ export function App() {
} }
} }
async function syncWebDav() {
setLoading(true);
setError(null);
setNotice(null);
try {
const result = await scanWebDav();
if (result.baseline_initialized) {
setNotice(
`Imported ${result.baseline_inserted + result.baseline_marked_existing} existing files as manual-only baseline entries.`,
);
} else {
setNotice(`Observed ${result.observed} files; ${result.newly_stable} newly stable.`);
}
await refresh();
} catch (caught) {
setError(caught instanceof Error ? caught.message : "Unknown error");
setLoading(false);
}
}
async function saveSettings() { async function saveSettings() {
if (!settings) return; if (!settings) return;
await withRefresh(() => patchSettings(settings)); await withRefresh(() => patchSettings(settings));
@@ -149,6 +172,7 @@ export function App() {
</header> </header>
{error && <div className="error-strip">{error}</div>} {error && <div className="error-strip">{error}</div>}
{notice && <div className="notice-strip">{notice}</div>}
<section className="band controls-band"> <section className="band controls-band">
<div className="section-title"> <div className="section-title">
@@ -176,10 +200,16 @@ export function App() {
<section className="workbench"> <section className="workbench">
<aside className="video-list"> <aside className="video-list">
<div className="list-header">
<div className="section-title"> <div className="section-title">
<Clapperboard size={18} /> <Clapperboard size={18} />
<h2>Videos</h2> <h2>Videos</h2>
</div> </div>
<button type="button" className="icon-button" onClick={() => void syncWebDav()} disabled={loading}>
<FolderSync size={16} />
Sync WebDAV
</button>
</div>
<div className="table-scroll"> <div className="table-scroll">
<table> <table>
<thead> <thead>
@@ -301,4 +331,3 @@ function Metric({ label, value }: { label: string; value: string }) {
</div> </div>
); );
} }
+13 -2
View File
@@ -1,8 +1,15 @@
import type { Artifact, ClipSuggestion, PipelineSettings, TranscriptSegment, Video } from "./types"; import type {
Artifact,
ClipSuggestion,
PipelineSettings,
TranscriptSegment,
Video,
WebDavScanResult,
} from "./types";
const runtimeApiBase = window.__EVANESCERE_FRONTEND_CONFIG__?.apiBaseUrl; const runtimeApiBase = window.__EVANESCERE_FRONTEND_CONFIG__?.apiBaseUrl;
export const apiBaseUrl = (runtimeApiBase || "http://localhost:8000").replace( export const apiBaseUrl = (runtimeApiBase || "http://localhost:8080").replace(
/\/$/, /\/$/,
"", "",
); );
@@ -37,6 +44,10 @@ export function getVideos(): Promise<Video[]> {
return request<Video[]>("/videos"); return request<Video[]>("/videos");
} }
export function scanWebDav(): Promise<WebDavScanResult> {
return request<WebDavScanResult>("/webdav/scan", { method: "POST" });
}
export function runVideo(videoId: number) { export function runVideo(videoId: number) {
return request(`/videos/${videoId}/run`, { method: "POST" }); return request(`/videos/${videoId}/run`, { method: "POST" });
} }
+17 -1
View File
@@ -120,6 +120,16 @@ button {
overflow-wrap: anywhere; overflow-wrap: anywhere;
} }
.notice-strip {
margin-bottom: 14px;
border-left: 4px solid #21756b;
background: #e9f3f1;
color: #155c54;
padding: 10px 12px;
border-radius: 6px;
font-size: 14px;
}
.band { .band {
border-top: 1px solid #d8dee6; border-top: 1px solid #d8dee6;
padding: 16px 0; padding: 16px 0;
@@ -190,6 +200,13 @@ button {
padding-top: 16px; padding-top: 16px;
} }
.list-header {
display: flex;
align-items: center;
justify-content: space-between;
gap: 12px;
}
.table-scroll { .table-scroll {
margin-top: 12px; margin-top: 12px;
overflow: auto; overflow: auto;
@@ -431,4 +448,3 @@ tr.selected td {
grid-template-columns: 1fr; grid-template-columns: 1fr;
} }
} }
+8
View File
@@ -59,3 +59,11 @@ export interface Artifact {
artifact_metadata: Record<string, unknown>; artifact_metadata: Record<string, unknown>;
created_at: string; created_at: string;
} }
export interface WebDavScanResult {
observed: number;
newly_stable: number;
baseline_initialized: boolean;
baseline_inserted: number;
baseline_marked_existing: number;
}
+8
View File
@@ -21,7 +21,9 @@ from evanescere.schemas import (
SettingsRead, SettingsRead,
TranscriptSegmentRead, TranscriptSegmentRead,
VideoRead, VideoRead,
WebDavScanRead,
) )
from evanescere.services.webdav import WebDavClient, safe_scan_once
from evanescere.settings_store import get_pipeline_settings, patch_pipeline_settings from evanescere.settings_store import get_pipeline_settings, patch_pipeline_settings
configure_logging() configure_logging()
@@ -68,6 +70,12 @@ def list_videos(db: Session = Depends(get_db)) -> list[Video]:
return list(db.scalars(select(Video).order_by(Video.created_at.desc())).all()) return list(db.scalars(select(Video).order_by(Video.created_at.desc())).all())
@app.post("/webdav/scan", response_model=WebDavScanRead)
def scan_webdav(db: Session = Depends(get_db)) -> WebDavScanRead:
result = safe_scan_once(WebDavClient.from_settings(), db)
return WebDavScanRead(**result.__dict__)
@app.post("/videos/{video_id}/run", response_model=RunRead) @app.post("/videos/{video_id}/run", response_model=RunRead)
def run_video(video_id: int, db: Session = Depends(get_db)) -> PipelineRun: def run_video(video_id: int, db: Session = Depends(get_db)) -> PipelineRun:
video = db.get(Video, video_id) video = db.get(Video, video_id)
+15 -5
View File
@@ -5,7 +5,11 @@ from evanescere.db import session_scope
from evanescere.jobs import enqueue_pipeline, enqueue_render, enqueue_transcribe, enqueue_suggest, enqueue_upload from evanescere.jobs import enqueue_pipeline, enqueue_render, enqueue_transcribe, enqueue_suggest, enqueue_upload
from evanescere.logging_config import configure_logging from evanescere.logging_config import configure_logging
from evanescere.models import PipelineRun, Video from evanescere.models import PipelineRun, Video
from evanescere.services.webdav import WebDavClient, bootstrap_existing as bootstrap_webdav_existing, scan_once from evanescere.services.webdav import (
WebDavClient,
bootstrap_existing as bootstrap_webdav_existing,
safe_scan_once,
)
app = typer.Typer(no_args_is_help=True) app = typer.Typer(no_args_is_help=True)
configure_logging() configure_logging()
@@ -14,15 +18,21 @@ configure_logging()
@app.command() @app.command()
def bootstrap_existing() -> None: def bootstrap_existing() -> None:
with session_scope() as session: with session_scope() as session:
count = bootstrap_webdav_existing(WebDavClient.from_settings(), session) result = bootstrap_webdav_existing(WebDavClient.from_settings(), session)
typer.echo(f"Marked {count} existing WebDAV recordings as existing_done.") typer.echo(
f"Observed {result.observed} recordings; inserted {result.inserted} and marked "
f"{result.marked_existing} tracked recordings as existing_done."
)
@app.command() @app.command()
def scan() -> None: def scan() -> None:
with session_scope() as session: with session_scope() as session:
observed, stable = scan_once(WebDavClient.from_settings(), session) result = safe_scan_once(WebDavClient.from_settings(), session)
typer.echo(f"Observed {observed} files; {stable} newly stable.") typer.echo(
f"Observed {result.observed} files; {result.newly_stable} newly stable; "
f"baseline initialized: {result.baseline_initialized}."
)
@app.command("run-video") @app.command("run-video")
+28 -2
View File
@@ -12,6 +12,25 @@ DEFAULT_CONFIG_PATHS = (
Path("config.toml"), Path("config.toml"),
) )
DEFAULT_LLM_SYSTEM_PROMPT = """\
You are an expert editor for Mandarin VTuber livestream clips.
Identify moments that work as entertaining standalone clips for Bilibili viewers.
Prioritize strong reactions, jokes, surprising turns, memorable conversations, and moments with a clear payoff.
Avoid repetitive stretches, dead air, and segments that require too much missing context.
Return only the requested JSON object.
"""
DEFAULT_LLM_USER_PROMPT = """\
Review the timestamped livestream transcript below.
Choose up to {max_candidates} compelling clip candidates.
Each clip must be between {min_clip_seconds} and {max_clip_seconds} seconds long.
Use absolute stream timestamps and give each candidate a concise Chinese title and summary.
Transcript range: {chunk_start_sec}-{chunk_end_sec}
Transcript:
{transcript}
"""
class Settings(BaseModel): class Settings(BaseModel):
app_env: str = "dev" app_env: str = "dev"
@@ -38,6 +57,8 @@ class Settings(BaseModel):
deepseek_api_key: str | None = None deepseek_api_key: str | None = None
deepseek_model: str = "deepseek-v4-pro" deepseek_model: str = "deepseek-v4-pro"
deepseek_temperature: float = 0.2 deepseek_temperature: float = 0.2
llm_system_prompt: str = DEFAULT_LLM_SYSTEM_PROMPT
llm_user_prompt: str = DEFAULT_LLM_USER_PROMPT
default_suggest_enabled: bool = True default_suggest_enabled: bool = True
default_render_enabled: bool = True default_render_enabled: bool = True
@@ -51,12 +72,13 @@ class Settings(BaseModel):
clip_min_seconds: int = 30 clip_min_seconds: int = 30
clip_max_seconds: int = 360 clip_max_seconds: int = 360
transcript_chunk_seconds: int = 900 transcript_chunk_seconds: int = 900
clip_max_candidates_total: int = Field(default=20, ge=1)
thumbnail_enabled: bool = True thumbnail_enabled: bool = True
thumbnail_provider: str = "frame_overlay" thumbnail_provider: str = "frame_overlay"
thumbnail_width: int = Field(default=1920, ge=320) thumbnail_width: int = Field(default=1920, ge=320)
thumbnail_height: int = Field(default=1080, ge=180) thumbnail_height: int = Field(default=1080, ge=180)
thumbnail_character_overlay_path: str | None = None thumbnail_character_overlay_dir: str | None = None
thumbnail_character_scale: float = Field(default=0.42, gt=0, le=1) thumbnail_character_scale: float = Field(default=0.42, gt=0, le=1)
thumbnail_character_position: str = "bottom-right" thumbnail_character_position: str = "bottom-right"
thumbnail_title_enabled: bool = True thumbnail_title_enabled: bool = True
@@ -94,6 +116,7 @@ def flatten_config(raw: dict[str, Any]) -> dict[str, Any]:
webdav = raw.get("webdav", {}) webdav = raw.get("webdav", {})
funasr = raw.get("funasr", {}) funasr = raw.get("funasr", {})
deepseek = raw.get("deepseek", {}) deepseek = raw.get("deepseek", {})
llm_prompt = raw.get("llm_prompt", {})
defaults = raw.get("defaults", {}) defaults = raw.get("defaults", {})
clip = raw.get("clip", {}) clip = raw.get("clip", {})
thumbnail = raw.get("thumbnail", {}) thumbnail = raw.get("thumbnail", {})
@@ -119,6 +142,8 @@ def flatten_config(raw: dict[str, Any]) -> dict[str, Any]:
"deepseek_api_key": blank_to_none(deepseek.get("api_key")), "deepseek_api_key": blank_to_none(deepseek.get("api_key")),
"deepseek_model": deepseek.get("model", "deepseek-v4-pro"), "deepseek_model": deepseek.get("model", "deepseek-v4-pro"),
"deepseek_temperature": deepseek.get("temperature", 0.2), "deepseek_temperature": deepseek.get("temperature", 0.2),
"llm_system_prompt": llm_prompt.get("system", DEFAULT_LLM_SYSTEM_PROMPT),
"llm_user_prompt": llm_prompt.get("user", DEFAULT_LLM_USER_PROMPT),
"default_suggest_enabled": defaults.get("suggest_enabled", True), "default_suggest_enabled": defaults.get("suggest_enabled", True),
"default_render_enabled": defaults.get("render_enabled", True), "default_render_enabled": defaults.get("render_enabled", True),
"default_upload_enabled": defaults.get("upload_enabled", True), "default_upload_enabled": defaults.get("upload_enabled", True),
@@ -127,11 +152,12 @@ def flatten_config(raw: dict[str, Any]) -> dict[str, Any]:
"clip_min_seconds": clip.get("min_seconds", 30), "clip_min_seconds": clip.get("min_seconds", 30),
"clip_max_seconds": clip.get("max_seconds", 360), "clip_max_seconds": clip.get("max_seconds", 360),
"transcript_chunk_seconds": clip.get("transcript_chunk_seconds", 900), "transcript_chunk_seconds": clip.get("transcript_chunk_seconds", 900),
"clip_max_candidates_total": clip.get("max_candidates_total", 20),
"thumbnail_enabled": thumbnail.get("enabled", True), "thumbnail_enabled": thumbnail.get("enabled", True),
"thumbnail_provider": thumbnail.get("provider", "frame_overlay"), "thumbnail_provider": thumbnail.get("provider", "frame_overlay"),
"thumbnail_width": thumbnail.get("width", 1920), "thumbnail_width": thumbnail.get("width", 1920),
"thumbnail_height": thumbnail.get("height", 1080), "thumbnail_height": thumbnail.get("height", 1080),
"thumbnail_character_overlay_path": blank_to_none(thumbnail.get("character_overlay_path")), "thumbnail_character_overlay_dir": blank_to_none(thumbnail.get("character_overlay_dir")),
"thumbnail_character_scale": thumbnail.get("character_scale", 0.42), "thumbnail_character_scale": thumbnail.get("character_scale", 0.42),
"thumbnail_character_position": thumbnail.get("character_position", "bottom-right"), "thumbnail_character_position": thumbnail.get("character_position", "bottom-right"),
"thumbnail_title_enabled": thumbnail.get("title_enabled", True), "thumbnail_title_enabled": thumbnail.get("title_enabled", True),
+12 -2
View File
@@ -108,7 +108,12 @@ def transcribe_video(session: Session, video: Video) -> int:
def suggest_clips(session: Session, video: Video) -> int: def suggest_clips(session: Session, video: Video) -> int:
settings = get_settings() settings = get_settings()
logger.info("clip suggestion start video_id=%s chunk_seconds=%s", video.id, settings.transcript_chunk_seconds) logger.info(
"clip suggestion start video_id=%s chunk_seconds=%s max_candidates_total=%s",
video.id,
settings.transcript_chunk_seconds,
settings.clip_max_candidates_total,
)
segments = list( segments = list(
session.scalars( session.scalars(
select(TranscriptSegment) select(TranscriptSegment)
@@ -120,7 +125,12 @@ def suggest_clips(session: Session, video: Video) -> int:
raise RuntimeError("Cannot suggest clips without transcript segments.") raise RuntimeError("Cannot suggest clips without transcript segments.")
chunks = chunk_transcript(segments, settings.transcript_chunk_seconds) chunks = chunk_transcript(segments, settings.transcript_chunk_seconds)
logger.info("clip suggestion chunks video_id=%s segments=%s chunks=%s", video.id, len(segments), len(chunks)) logger.info("clip suggestion chunks video_id=%s segments=%s chunks=%s", video.id, len(segments), len(chunks))
candidates = DeepSeekClipClient.from_settings().suggest(chunks) candidates = DeepSeekClipClient.from_settings().suggest(
chunks,
max_candidates_total=settings.clip_max_candidates_total,
min_clip_seconds=settings.clip_min_seconds,
max_clip_seconds=settings.clip_max_seconds,
)
session.query(ClipSuggestion).filter(ClipSuggestion.video_id == video.id).delete() session.query(ClipSuggestion).filter(ClipSuggestion.video_id == video.id).delete()
for candidate in candidates: for candidate in candidates:
session.add( session.add(
+9 -3
View File
@@ -10,7 +10,7 @@ from evanescere.db import session_scope
from evanescere.jobs import enqueue_pipeline from evanescere.jobs import enqueue_pipeline
from evanescere.logging_config import configure_logging from evanescere.logging_config import configure_logging
from evanescere.models import PipelineRun, Video from evanescere.models import PipelineRun, Video
from evanescere.services.webdav import WebDavClient, scan_once from evanescere.services.webdav import WebDavClient, safe_scan_once
configure_logging() configure_logging()
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -45,9 +45,15 @@ def run_forever() -> None:
while True: while True:
try: try:
with session_scope() as session: with session_scope() as session:
observed, stable = scan_once(client, session) result = safe_scan_once(client, session)
queued = enqueue_stable_videos() queued = enqueue_stable_videos()
logger.info("scan observed=%s newly_stable=%s queued=%s", observed, stable, queued) logger.info(
"scan observed=%s newly_stable=%s baseline_initialized=%s queued=%s",
result.observed,
result.newly_stable,
result.baseline_initialized,
queued,
)
except Exception: except Exception:
logger.exception("scheduler scan failed") logger.exception("scheduler scan failed")
time.sleep(settings.webdav_poll_interval_seconds) time.sleep(settings.webdav_poll_interval_seconds)
+8 -1
View File
@@ -95,6 +95,14 @@ class SettingsRead(BaseModel):
bake_subtitles: bool = True bake_subtitles: bool = True
class WebDavScanRead(BaseModel):
observed: int
newly_stable: int
baseline_initialized: bool = False
baseline_inserted: int = 0
baseline_marked_existing: int = 0
class ClipCandidate(BaseModel): class ClipCandidate(BaseModel):
start_sec: float = Field(ge=0) start_sec: float = Field(ge=0)
end_sec: float = Field(gt=0) end_sec: float = Field(gt=0)
@@ -108,4 +116,3 @@ class ClipCandidate(BaseModel):
class ClipCandidateResponse(BaseModel): class ClipCandidateResponse(BaseModel):
clips: list[ClipCandidate] clips: list[ClipCandidate]
+99 -21
View File
@@ -13,6 +13,11 @@ from evanescere.schemas import ClipCandidate, ClipCandidateResponse
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
RESPONSE_SCHEMA_INSTRUCTION = """\
Return strict JSON only using this schema:
{"clips":[{"start_sec":number,"end_sec":number,"title_zh":string,"summary_zh":string,"reason":string,"score":number,"tags":[string],"subtitle_priority":string}]}
"""
@dataclass(frozen=True) @dataclass(frozen=True)
class TranscriptChunk: class TranscriptChunk:
@@ -22,10 +27,20 @@ class TranscriptChunk:
class DeepSeekClipClient: class DeepSeekClipClient:
def __init__(self, api_key: str, base_url: str, model: str, temperature: float) -> None: def __init__(
self,
api_key: str,
base_url: str,
model: str,
temperature: float,
system_prompt: str,
user_prompt: str,
) -> None:
self.client = OpenAI(api_key=api_key, base_url=base_url) self.client = OpenAI(api_key=api_key, base_url=base_url)
self.model = model self.model = model
self.temperature = temperature self.temperature = temperature
self.system_prompt = system_prompt
self.user_prompt = user_prompt
@classmethod @classmethod
def from_settings(cls) -> DeepSeekClipClient: def from_settings(cls) -> DeepSeekClipClient:
@@ -37,18 +52,52 @@ class DeepSeekClipClient:
settings.deepseek_base_url, settings.deepseek_base_url,
settings.deepseek_model, settings.deepseek_model,
settings.deepseek_temperature, settings.deepseek_temperature,
settings.llm_system_prompt,
settings.llm_user_prompt,
) )
def suggest(self, chunks: list[TranscriptChunk]) -> list[ClipCandidate]: def suggest(
logger.info("deepseek suggestion start chunks=%s model=%s", len(chunks), self.model) self,
chunks: list[TranscriptChunk],
*,
max_candidates_total: int,
min_clip_seconds: int,
max_clip_seconds: int,
) -> list[ClipCandidate]:
logger.info(
"deepseek suggestion start chunks=%s model=%s max_candidates_total=%s",
len(chunks),
self.model,
max_candidates_total,
)
candidates: list[ClipCandidate] = [] candidates: list[ClipCandidate] = []
quota = candidate_quota_per_chunk(len(chunks), max_candidates_total)
for chunk in chunks: for chunk in chunks:
candidates.extend(self._suggest_for_chunk(chunk)) candidates.extend(
ranked = dedupe_and_rank_candidates(candidates) self._suggest_for_chunk(
chunk,
max_candidates=quota,
min_clip_seconds=min_clip_seconds,
max_clip_seconds=max_clip_seconds,
)
)
ranked = dedupe_and_rank_candidates(
candidates,
min_clip_seconds=min_clip_seconds,
max_clip_seconds=max_clip_seconds,
max_candidates_total=max_candidates_total,
)
logger.info("deepseek suggestion done raw_candidates=%s ranked_candidates=%s", len(candidates), len(ranked)) logger.info("deepseek suggestion done raw_candidates=%s ranked_candidates=%s", len(candidates), len(ranked))
return ranked return ranked
def _suggest_for_chunk(self, chunk: TranscriptChunk) -> list[ClipCandidate]: def _suggest_for_chunk(
self,
chunk: TranscriptChunk,
*,
max_candidates: int,
min_clip_seconds: int,
max_clip_seconds: int,
) -> list[ClipCandidate]:
logger.info( logger.info(
"deepseek chunk request start start=%.1f end=%.1f chars=%s", "deepseek chunk request start start=%.1f end=%.1f chars=%s",
chunk.start_sec, chunk.start_sec,
@@ -62,22 +111,16 @@ class DeepSeekClipClient:
messages=[ messages=[
{ {
"role": "system", "role": "system",
"content": ( "content": f"{self.system_prompt.rstrip()}\n\n{RESPONSE_SCHEMA_INSTRUCTION}",
"You select interesting short clips from Mandarin livestream transcripts. "
"Return strict JSON only. Prefer complete moments with context, funny reactions, "
"surprising reveals, emotional peaks, or strong standalone discussion."
),
}, },
{ {
"role": "user", "role": "user",
"content": ( "content": render_user_prompt(
"Find up to 5 clip candidates in this transcript chunk. " self.user_prompt,
"Each clip must be 30-360 seconds and use absolute stream seconds. " chunk=chunk,
"JSON schema: {\"clips\":[{\"start_sec\":number,\"end_sec\":number," max_candidates=max_candidates,
"\"title_zh\":string,\"summary_zh\":string,\"reason\":string," min_clip_seconds=min_clip_seconds,
"\"score\":number,\"tags\":[string],\"subtitle_priority\":string}]}.\n\n" max_clip_seconds=max_clip_seconds,
f"Chunk bounds: {chunk.start_sec:.1f}-{chunk.end_sec:.1f}\n"
f"Transcript:\n{chunk.text}"
), ),
}, },
], ],
@@ -95,6 +138,27 @@ class DeepSeekClipClient:
return candidates return candidates
def render_user_prompt(
template: str,
*,
chunk: TranscriptChunk,
max_candidates: int,
min_clip_seconds: int,
max_clip_seconds: int,
) -> str:
try:
return template.format(
max_candidates=max_candidates,
min_clip_seconds=min_clip_seconds,
max_clip_seconds=max_clip_seconds,
chunk_start_sec=f"{chunk.start_sec:.1f}",
chunk_end_sec=f"{chunk.end_sec:.1f}",
transcript=chunk.text,
)
except KeyError as exc:
raise ValueError(f"Unknown placeholder in [llm_prompt].user: {exc.args[0]}") from exc
def segment_line(segment: TranscriptSegment) -> str: def segment_line(segment: TranscriptSegment) -> str:
return f"[{segment.start_sec:.1f}-{segment.end_sec:.1f}] {segment.text}" return f"[{segment.start_sec:.1f}-{segment.end_sec:.1f}] {segment.text}"
@@ -141,17 +205,31 @@ def overlap_ratio(left: ClipCandidate, right: ClipCandidate) -> float:
return overlap / shortest return overlap / shortest
def dedupe_and_rank_candidates(candidates: list[ClipCandidate]) -> list[ClipCandidate]: def candidate_quota_per_chunk(chunk_count: int, max_candidates_total: int) -> int:
if chunk_count <= 0:
return 0
return max(1, (max_candidates_total + chunk_count - 1) // chunk_count)
def dedupe_and_rank_candidates(
candidates: list[ClipCandidate],
*,
min_clip_seconds: int = 30,
max_clip_seconds: int = 360,
max_candidates_total: int | None = None,
) -> list[ClipCandidate]:
valid = [ valid = [
candidate candidate
for candidate in candidates for candidate in candidates
if candidate.end_sec > candidate.start_sec if candidate.end_sec > candidate.start_sec
and 30 <= candidate.end_sec - candidate.start_sec <= 360 and min_clip_seconds <= candidate.end_sec - candidate.start_sec <= max_clip_seconds
] ]
ranked = sorted(valid, key=lambda item: item.score, reverse=True) ranked = sorted(valid, key=lambda item: item.score, reverse=True)
chosen: list[ClipCandidate] = [] chosen: list[ClipCandidate] = []
for candidate in ranked: for candidate in ranked:
if all(overlap_ratio(candidate, existing) < 0.5 for existing in chosen): if all(overlap_ratio(candidate, existing) < 0.5 for existing in chosen):
chosen.append(candidate) chosen.append(candidate)
if max_candidates_total is not None and len(chosen) >= max_candidates_total:
break
logger.debug("dedupe candidates input=%s valid=%s chosen=%s", len(candidates), len(valid), len(chosen)) logger.debug("dedupe candidates input=%s valid=%s chosen=%s", len(candidates), len(valid), len(chosen))
return chosen return chosen
+21 -6
View File
@@ -2,6 +2,7 @@ from __future__ import annotations
import json import json
import logging import logging
import random
import shlex import shlex
import subprocess import subprocess
from pathlib import Path from pathlib import Path
@@ -63,11 +64,7 @@ def generate_thumbnail(
raise RuntimeError(f"Unsupported [thumbnail].provider={settings.thumbnail_provider!r}") raise RuntimeError(f"Unsupported [thumbnail].provider={settings.thumbnail_provider!r}")
final_path = output_dir / "thumbnail_final.jpg" final_path = output_dir / "thumbnail_final.jpg"
character_path = ( character_path = select_character_overlay(settings.thumbnail_character_overlay_dir)
Path(settings.thumbnail_character_overlay_path)
if settings.thumbnail_character_overlay_path
else None
)
compose_thumbnail( compose_thumbnail(
background_path=background, background_path=background,
output_path=final_path, output_path=final_path,
@@ -90,7 +87,7 @@ def generate_thumbnail(
"provider": provider, "provider": provider,
"source_frame": str(base_frame), "source_frame": str(base_frame),
"background": str(background), "background": str(background),
"character_overlay_path": settings.thumbnail_character_overlay_path or "", "character_overlay_path": str(character_path or ""),
}, },
) )
logger.info("thumbnail generation done video_id=%s clip_id=%s output=%s", video.id, clip.id, final_path) logger.info("thumbnail generation done video_id=%s clip_id=%s output=%s", video.id, clip.id, final_path)
@@ -237,6 +234,24 @@ def overlay_character(image, overlay_path: Path, scale: float, position: str) ->
image.alpha_composite(overlay, (max(0, x), max(0, y))) image.alpha_composite(overlay, (max(0, x), max(0, y)))
def select_character_overlay(overlay_dir: str | None) -> Path | None:
if not overlay_dir:
return None
directory = Path(overlay_dir)
if not directory.is_dir():
logger.warning("thumbnail character overlay directory missing path=%s", directory)
return None
candidates = sorted(
path for path in directory.iterdir() if path.is_file() and path.suffix.lower() == ".png"
)
if not candidates:
logger.warning("thumbnail character overlay directory has no png files path=%s", directory)
return None
selected = random.choice(candidates)
logger.info("thumbnail character overlay selected path=%s candidates=%s", selected, len(candidates))
return selected
def load_font(size: int): def load_font(size: int):
from PIL import ImageFont from PIL import ImageFont
+84 -9
View File
@@ -5,16 +5,17 @@ from dataclasses import dataclass
from datetime import UTC, datetime from datetime import UTC, datetime
from pathlib import PurePosixPath from pathlib import PurePosixPath
from typing import Any from typing import Any
from urllib.parse import quote, urljoin, urlparse from urllib.parse import quote, unquote, urljoin, urlparse
from xml.etree import ElementTree from xml.etree import ElementTree
import httpx import httpx
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from evanescere.config import get_settings from evanescere.config import get_settings
from evanescere.models import Video from evanescere.models import Setting, Video
VIDEO_SUFFIXES = {".flv", ".mp4", ".mkv", ".mov"} VIDEO_SUFFIXES = {".flv", ".mp4", ".mkv", ".mov"}
WEBDAV_BASELINE_KEY = "webdav_initial_baseline"
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -26,6 +27,22 @@ class WebDavFile:
modified_at: str | None = None modified_at: str | None = None
@dataclass(frozen=True)
class BootstrapResult:
observed: int
inserted: int
marked_existing: int
@dataclass(frozen=True)
class ScanResult:
observed: int
newly_stable: int
baseline_initialized: bool = False
baseline_inserted: int = 0
baseline_marked_existing: int = 0
class WebDavClient: class WebDavClient:
def __init__( def __init__(
self, self,
@@ -100,7 +117,7 @@ def parse_propfind_response(xml_text: str, base_url: str) -> list[WebDavFile]:
href_path = urlparse(href).path href_path = urlparse(href).path
if href_path.rstrip("/") == base_path: if href_path.rstrip("/") == base_path:
continue continue
filename = PurePosixPath(href_path).name filename = unquote(PurePosixPath(href_path).name)
if not filename or PurePosixPath(filename).suffix.lower() not in VIDEO_SUFFIXES: if not filename or PurePosixPath(filename).suffix.lower() not in VIDEO_SUFFIXES:
continue continue
resource_type = response.find(".//d:resourcetype", ns) resource_type = response.find(".//d:resourcetype", ns)
@@ -129,9 +146,29 @@ def has_stable_size(video: Video, required_equal_samples: int = 2) -> bool:
return len(sizes) == 1 and next(iter(sizes)) > 0 return len(sizes) == 1 and next(iter(sizes)) > 0
def bootstrap_existing(client: WebDavClient, session: Session) -> int: def has_webdav_baseline(session: Session) -> bool:
count = 0 return session.get(Setting, WEBDAV_BASELINE_KEY) is not None
for file in client.list_files():
def mark_webdav_baseline(session: Session, result: BootstrapResult) -> None:
value = {
"initialized_at": datetime.now(UTC).isoformat(),
"observed": result.observed,
"inserted": result.inserted,
"marked_existing": result.marked_existing,
}
row = session.get(Setting, WEBDAV_BASELINE_KEY)
if row is None:
session.add(Setting(key=WEBDAV_BASELINE_KEY, value=value))
else:
row.value = value
def bootstrap_existing(client: WebDavClient, session: Session) -> BootstrapResult:
files = client.list_files()
inserted = 0
marked_existing = 0
for file in files:
video = session.query(Video).filter(Video.source_url == file.url).one_or_none() video = session.query(Video).filter(Video.source_url == file.url).one_or_none()
if video is None: if video is None:
video = Video( video = Video(
@@ -143,11 +180,35 @@ def bootstrap_existing(client: WebDavClient, session: Session) -> int:
processing_status="done", processing_status="done",
) )
session.add(video) session.add(video)
count += 1 inserted += 1
logger.debug("bootstrap existing file=%s size=%s", file.filename, file.size_bytes) logger.debug("bootstrap existing file=%s size=%s", file.filename, file.size_bytes)
elif video.ingest_status == "observing" and video.processing_status == "pending":
video.size_bytes = file.size_bytes
video.ingest_status = "existing_done"
video.processing_status = "done"
marked_existing += 1
logger.debug("bootstrap marked existing video_id=%s file=%s", video.id, file.filename)
result = BootstrapResult(
observed=len(files),
inserted=inserted,
marked_existing=marked_existing,
)
mark_webdav_baseline(session, result)
session.flush() session.flush()
logger.info("bootstrap existing done inserted=%s", count) logger.info(
return count "bootstrap existing done observed=%s inserted=%s marked_existing=%s",
result.observed,
result.inserted,
result.marked_existing,
)
return result
def ensure_initial_baseline(client: WebDavClient, session: Session) -> BootstrapResult | None:
if has_webdav_baseline(session):
return None
logger.info("webdav baseline missing; recording currently visible files as existing_done")
return bootstrap_existing(client, session)
def scan_once(client: WebDavClient, session: Session) -> tuple[int, int]: def scan_once(client: WebDavClient, session: Session) -> tuple[int, int]:
@@ -175,3 +236,17 @@ def scan_once(client: WebDavClient, session: Session) -> tuple[int, int]:
logger.info("webdav file stable video_id=%s file=%s size=%s", video.id, file.filename, file.size_bytes) logger.info("webdav file stable video_id=%s file=%s size=%s", video.id, file.filename, file.size_bytes)
session.flush() session.flush()
return observed, newly_stable return observed, newly_stable
def safe_scan_once(client: WebDavClient, session: Session) -> ScanResult:
baseline = ensure_initial_baseline(client, session)
if baseline is not None:
return ScanResult(
observed=baseline.observed,
newly_stable=0,
baseline_initialized=True,
baseline_inserted=baseline.inserted,
baseline_marked_existing=baseline.marked_existing,
)
observed, newly_stable = scan_once(client, session)
return ScanResult(observed=observed, newly_stable=newly_stable)
+59 -1
View File
@@ -1,5 +1,11 @@
from evanescere.schemas import ClipCandidate from evanescere.schemas import ClipCandidate
from evanescere.services.llm import dedupe_and_rank_candidates, parse_clip_response from evanescere.services.llm import (
TranscriptChunk,
candidate_quota_per_chunk,
dedupe_and_rank_candidates,
parse_clip_response,
render_user_prompt,
)
def test_parse_clip_response(): def test_parse_clip_response():
@@ -62,3 +68,55 @@ def test_dedupe_and_rank_candidates_filters_short_and_overlapping():
ranked = dedupe_and_rank_candidates(candidates) ranked = dedupe_and_rank_candidates(candidates)
assert [candidate.title_zh for candidate in ranked] == ["best", "second"] assert [candidate.title_zh for candidate in ranked] == ["best", "second"]
def test_candidate_quota_uses_total_budget_across_chunks():
assert candidate_quota_per_chunk(1, 20) == 20
assert candidate_quota_per_chunk(4, 20) == 5
assert candidate_quota_per_chunk(0, 20) == 0
def test_dedupe_respects_configured_duration_and_total_cap():
candidates = [
ClipCandidate(
start_sec=index * 240,
end_sec=index * 240 + duration,
title_zh=f"clip-{index}",
summary_zh="",
reason="",
score=1 - index / 100,
)
for index, duration in enumerate([60, 180, 181, 90])
]
ranked = dedupe_and_rank_candidates(
candidates,
min_clip_seconds=30,
max_clip_seconds=180,
max_candidates_total=2,
)
assert [candidate.title_zh for candidate in ranked] == ["clip-0", "clip-1"]
def test_render_user_prompt_inserts_configured_values():
prompt = render_user_prompt(
"{max_candidates}|{min_clip_seconds}|{max_clip_seconds}|{chunk_start_sec}|{chunk_end_sec}|{transcript}",
chunk=TranscriptChunk(12.25, 42.75, "hello"),
max_candidates=20,
min_clip_seconds=30,
max_clip_seconds=180,
)
assert prompt == "20|30|180|12.2|42.8|hello"
def test_render_user_prompt_rejects_unknown_placeholder():
try:
render_user_prompt(
"{unknown}",
chunk=TranscriptChunk(0, 1, "hello"),
max_candidates=20,
min_clip_seconds=30,
max_clip_seconds=180,
)
except ValueError as exc:
assert "Unknown placeholder" in str(exc)
else:
raise AssertionError("Expected unknown placeholder to fail")
+22
View File
@@ -0,0 +1,22 @@
from pathlib import Path
from unittest.mock import patch
from evanescere.services.thumbnail import select_character_overlay
def test_select_character_overlay_returns_none_for_empty_directory(tmp_path: Path):
assert select_character_overlay(str(tmp_path)) is None
def test_select_character_overlay_filters_png_files(tmp_path: Path):
first = tmp_path / "first.PNG"
second = tmp_path / "second.png"
first.write_bytes(b"")
second.write_bytes(b"")
(tmp_path / "notes.txt").write_text("ignore me")
with patch("evanescere.services.thumbnail.random.choice", return_value=second) as choice:
assert select_character_overlay(str(tmp_path)) == second
assert choice.call_args.args[0] == [first, second]
+17
View File
@@ -33,3 +33,20 @@ def test_parse_propfind_response_filters_video_files():
WebDavFile("https://host/webdav/recordings/stream.flv", "stream.flv", 123, None) WebDavFile("https://host/webdav/recordings/stream.flv", "stream.flv", 123, None)
] ]
def test_parse_propfind_response_preserves_encoded_chinese_filename():
xml = """<?xml version="1.0"?>
<D:multistatus xmlns:D="DAV:">
<D:response>
<D:href>/Hirumi/hirumi-2-06%E6%9C%8801%E6%97%A501%E6%97%B602%E5%88%8641%E7%A7%92.flv</D:href>
<D:propstat><D:prop><D:getcontentlength>123</D:getcontentlength></D:prop></D:propstat>
</D:response>
</D:multistatus>"""
assert parse_propfind_response(xml, "http://recording.home.arpa/Hirumi") == [
WebDavFile(
"http://recording.home.arpa/Hirumi/hirumi-2-06%E6%9C%8801%E6%97%A501%E6%97%B602%E5%88%8641%E7%A7%92.flv",
"hirumi-2-06月01日01时02分41秒.flv",
123,
None,
)
]