Remotion + Excalidraw + TTS (Local)

Use this skill to turn an Excalidraw diagram + a voiceover script into a rendered MP4 using:

• Remotion (render)
• Excalidraw (render the .excalidraw JSON directly)
• TTS via macOS say (offline)

Quick start (one command)

Run:

python3 skills/remotion-excalidraw-tts/scripts/make_video.py \
  --diagram /absolute/path/to/diagram.excalidraw \
  --voiceover-text /absolute/path/to/voiceover.txt \
  --out /absolute/path/to/out.mp4

Optional: drive camera/focus/subtitles via storyboard JSON:

python3 skills/remotion-excalidraw-tts/scripts/make_video.py \
  --diagram /absolute/path/to/diagram.excalidraw \
  --voiceover-text /absolute/path/to/voiceover.txt \
  --storyboard-json /absolute/path/to/storyboard.json \
  --out /absolute/path/to/out.mp4

What it does:

1) copies the Remotion template project from assets/template/remotion-project/ into a temp workdir

2) writes public/diagram.excalidraw

3) generates public/voiceover.mp3 via say + ffmpeg

4) sets composition duration to match the voiceover length

5) renders MP4 with npx remotion render

Inputs

• --diagram: .excalidraw JSON file (from Excalidraw export)
• --voiceover-text: plain text file (Chinese supported)

Optional:

• --voiceover-mp3: if you already have audio, skip TTS
• --tts-backend: say (default) | openai | elevenlabs
• --fps: default 30

TTS backends:

• macOS say: --tts-backend say --voice Tingting --rate 220
• OpenAI: --tts-backend openai --openai-model gpt-4o-mini-tts --openai-voice alloy (requires OPENAI_API_KEY)
• ElevenLabs: --tts-backend elevenlabs --elevenlabs-voice-id --elevenlabs-model eleven_multilingual_v2 (requires ELEVENLABS_API_KEY)

Customizing scenes (pan/zoom/highlights)

Option A: edit TypeScript storyboard

Template code lives in:

• assets/template/remotion-project/src/video/storyboard/storyboard.ts

Edit scenes:

• cameraFrom/cameraTo (x/y/scale)
• focus rectangle (x/y/width/height + label)
• subtitle

Option B (recommended): provide storyboard.json

Pass --storyboard-json /abs/path/storyboard.json.

Schema reference:

• references/storyboard.schema.json

Minimal example:

{
  "scenes": [
    {
      "name": "intro",
      "durationSec": 10,
      "subtitle": "很多智能体隔天就失忆。",
      "cameraFrom": {"x": 0, "y": 0, "scale": 1},
      "cameraTo": {"x": 0, "y": 0, "scale": 1},
      "focus": {"x": 140, "y": 120, "width": 1640, "height": 340, "label": "问题"}
    }
  ]
}

Requirements

• macOS (for say)
• ffmpeg + ffprobe
• Node.js + npm (the script will run npm i in the temp project)