notivn/AIEV: aI Edit Video by noti.vn

🇬🇧 English · 🇻🇳 Tiếng Việt

Automatic AI video editing. Claude acts as the director - driving HyperFrames (motion-graphics scenes built with HTML + GSAP) and Remotion (timeline assembly) - while you supervise everything through the web dashboard at http://localhost:6868.

Drop in a clip, briefly describe what you want, click "Start editing with AI" - the system automatically transcribes, writes the editing script, creates kinetic-typography scenes, karaoke subtitles, beat-synced zooms, timestamped sound effects, assembles the timeline and exports an MP4.

Features

Architecture

┌─────────────────────────────────────────────────────┐
│  Web UI (Next.js, port 6868)                        │
│  Dashboard · Videos/Images Project · Style Design   │
│  Render Queue · Sound Effects · Skills · Settings   │
└──────────────────────┬──────────────────────────────┘
                       │ REST + SSE
┌──────────────────────┴──────────────────────────────┐
│  Backend (Express, port 6869)                       │
│  Claude Agent SDK · Render queue · SQLite           │
└──────┬──────────────────────────────┬───────────────┘
┌──────┴────────��──┐        ┌─────────┴────────────┐
│ HyperFrames      │  MP4   │ Remotion             │
│ SCENE ENGINE     │───────▶│ ASSEMBLER            │
│ HTML + GSAP      │        │ scene + audio + sub  │
└──────────────────┘        └──────────────────────┘

Full API contract: docs/API.md. Production workflow + know-how: .claude/skills/.

CPU or GPU, step by step

Making one video runs through several stages, and each one picks its processor differently. Nothing here is automatic magic: the defaults are chosen so a draft is fast and a final is high quality, and you can change every one of them in Settings.

Three things worth knowing:

• Final encoding is deliberately on the CPU. NVENC is much faster but at the same file size libx264 looks slightly better. Draft quality does not matter, so draft encoding defaults to the GPU and final defaults to the CPU. If you would rather have speed than the last few percent of quality, turn on GPU encode for FINAL.
• Remotion's own encoding never uses the GPU here, only its rasterizing does. Without --gl angle Remotion falls back to SwANGLE, which is pure software rendering on the CPU and is noticeably slower.
• NVENC is detected through nvidia-smi. If it fails, or the GPU runs out of encoding sessions, or it rejects a frame size, the job re-runs on libx264 instead of failing. On macOS the auto-cut reframe currently always uses libx264, because that detection only looks for NVIDIA; HyperFrames still uses VideoToolbox there.

Parallelism is separate from all of this: Concurrent render jobs (queue) (2 by default) sets how many jobs run at once, Chrome workers (HyperFrames) sets the HyperFrames worker count, and Remotion concurrency sets how many frames Remotion renders in parallel. Two jobs belonging to the same project never run at the same time.

Requirements

• Node.js 22+ (HyperFrames requires >=22)
• FFmpeg on PATH (macOS: brew install ffmpeg)
• Google Chrome (HyperFrames and Remotion render through headless Chromium)
• Claude: sign in to Claude Code on this machine (uses subscription OAuth - recommended) or put ANTHROPIC_API_KEY in .env
• Gemini (only needed for image generation): GEMINI_API_KEY in .env - get one at aistudio.google.com/apikey
• Optional: an NVIDIA GPU (NVENC) or Apple Silicon Mac (VideoToolbox) for faster rendering; Python + faster-whisper for subtitles

You do not have to install these by hand. The start script checks every item above and offers to install whatever it can - see Environment check below.

Getting started

git clone https://github.com/notivn/AIEV.git
cd AIEV

Windows - double-click start\start.bat (or run start\start.ps1).

macOS - double-click start/start.command in Finder. A .sh file cannot be double-clicked on macOS, it just opens in an editor. If macOS blocks it as coming from an unidentified developer, right-click the file → Open → Open.

Linux - run ./start/start.sh.

If you downloaded a ZIP instead of cloning, make the scripts executable once:

chmod +x start/*.sh start/*.command update/*.sh update/*.command

The script handles everything: checks the environment → npm install (first run) → build → creates .env → starts server + web → opens http://localhost:6868. You normally never run the update script by hand; the update button on the dashboard does it.

Manual dev run: npm install then npm run dev.

User guide

Your first video (Videos Project)

1. Create a project - open Videos Project → New project: name, frame size (9:16 vertical for TikTok/Reels, 16:9 for YouTube) and fps.
2. Add the source - in the Sources & Assets card upload your clip, or click Connect phone and scan the QR to send files straight from your phone.
3. Fill in the edit brief - the "Editing script" card is the AI's instruction sheet:
   • Source description - one or two sentences about what the clip is.
   • Auto-trim - cuts silences, fillers and repeated takes before editing.
   • Karaoke subtitles with keyword highlighting.
   • Key layout - the main key sits in the top band of the video, related keys in the bottom band, synced to what is being said.
   • AI illustrations (Gemini) - style-matched images composited into the video. Pick the drawing model, the image density (how many images per minute of video; leave empty and the AI decides), the subject position (3x3 grid picker, like the logo picker) and whether Gemini may draw text inside images.
   • Sound effects and background music - curated SFX set or the full library; music auto-ducks under speech.
   • Style Design - the brand kit (colors, fonts, logo) enforced 100% on everything; Video style - the visual language of this one video (paper fold, ink wash, stick figures...), leave unset and the AI decides.
   • Skill - the editing format to follow (Noti TikTok style, YouTube landscape...).
   • Notes - your free-form request; drop in a template from the Prompts page if you like.
4. Start editing with AI - Claude transcribes the clip, plans the edit, builds HyperFrames scenes, generates the illustrations, wires sound effects and assembles a draft. Progress streams live on the project page; all renders go through the Render queue.
5. Review and finish - watch the draft right on the project page, ask for changes in the review chat ("make the hook bigger", "cut the intro"...), then run QC and the final render. The MP4 lands in outputs/, together with an auto-generated thumbnail and a publish pack (title, description, hashtags).

Text to video (article → video)

Text to video → New session: paste a URL or raw text (the name is optional - it is taken from the article title). The pipeline runs in stages you can review between: Extract the article → Script - the AI rewrites it into a spoken script in chunks (set the target length in seconds, edit any chunk by hand) → Voice - pick the engine (Gemini TTS online or VieNeu-TTS on-device, including your cloned voices), the voice and the reading speed, with per-chunk preview → Build - the narration is synthesized and a video project is created and edited by the AI using the brief you configured. Setting an illustration density here is the easy way to keep a long article video visually alive.

Auto cut (long video → many shorts)

Auto cut → pick a long talking-head video → choose the mode: by time, by AI (finds the highlights) or by prompt. Plan proposes segments with titles - tick/untick and edit them - then Cut creates one child project per segment, all sharing the brief you configured once (aspect, layout, background, style). Each child project can then be edited by the AI like any other project.

Images Project (posters & thumbnails)

Images Project → new image: describe the scene, pick the kind, aspect and model. Gemini paints the background only (never any text), then Remotion overlays the title, subtitle, stats, CTA and logo per the Style Design - so Vietnamese text is never misspelled. Choose the text-block position with the 3x3 grid and re-render at will.

Style Design & Video styles

Style Design holds your brand kits: colors, fonts (type a Google Fonts name and it downloads with full Vietnamese diacritics), logo, tone and effects. The selected style is enforced on every output; when the style has a logo, the assembler stamps it on the top-left of every video automatically - never add your own. Video styles (picked in the brief) define the visual language of a single video: materials and motion.

Voices

The Voices page manages narration voices: 30 Gemini preset voices (online, per-use cost) and VieNeu-TTS (offline, free, Vietnamese with regional accents) - the only engine that can clone your own voice from a short recording or phone video. Preview any voice and reading speed before using it in Text to video.

Sound effects & music

Sound Effects hosts the library: 100+ files with tags and a curated "recommended" set the AI reaches for first; upload and tag your own. Background music lives in assets/music/, which ships empty on purpose - add your own tracks (Sound Effects page → Music tab → upload) and the AI picks one by mood and ducks it automatically under speech. See assets/music/README.md.

Render queue & Settings

Every render goes through the queue: drafts must pass before finals, and the automated QC (safe areas, audio loudness, black frames, Vietnamese diacritics) gates the final render - toggleable in Settings. Settings also holds the GPU switches, worker counts and queue concurrency; Connections manages the Claude / Gemini / OpenAI credentials.

Skills & Prompts

Skills are the production know-how the AI follows - browse, edit, clone or create them (including AI-generated skills from a Q&A form) right in the web UI. Prompts stores reusable request templates for the brief's notes field.

Environment check

start/doctor.mjs probes everything the pipeline needs - Node.js, FFmpeg, Google Chrome, Claude credentials, faster-whisper, the Gemini key, cloudflared, GPU - and installs what it can:

It runs automatically inside start.bat / start.command, and the same list appears as the System check card in the Settings tab, where each missing item gets a one-click install button (or a copyable command when it cannot be automated). Missing pieces never block startup - the dashboard comes up regardless so you can fix things from the UI.

node start/doctor.mjs              # just look
node start/doctor.mjs --fix        # ask before installing each missing item
node start/doctor.mjs --fix --yes  # install without asking
node start/doctor.mjs --lang en    # English output (default follows the script)

One file feeds the terminal, both start scripts and the web UI, so a new check is added in exactly one place.

Upload from your phone

On a project page, in the Sources & Assets card click Connect phone - scan the QR code with your phone camera (same WiFi as the machine running the system) to open the upload page http://<machine-ip>:6868/m/<project>. Videos/photos picked on the phone upload straight into the project's assets. The first time Windows asks about the firewall, choose Allow (the start script adds the rule automatically when it has admin rights).

Remote over 4G/5G (not on the same WiFi):

• Cloudflare Tunnel - fill TUNNEL_DOMAIN=<your-domain> (e.g. aiev.noti.vn) into .env, then the Connect phone QR automatically uses https://<domain>/m/<project> - works over 4G/5G.
• Start the tunnel with start\tunnel.bat (Windows) / ./start/tunnel.sh (macOS) - no TUNNEL_DOMAIN yet and it falls back to a quick tunnel with a random trycloudflare.com URL.
• ⚠️ Warning: the dashboard has no login yet - only expose it publicly behind Cloudflare Access, or never share the link.

Folder structure

├── apps/web/          # Next.js dashboard (port 6868)
├── apps/server/       # Express backend: Agent SDK + render queue + SQLite (port 6869)
├── engines/remotion/  # Remotion: Assemble (video) + Poster (image) compositions
├── .claude/skills/    # Skills - production know-how, manageable from the web UI
├── assets/
│   ├── sound-effects/ # Sound-effect library + library.json
│   ├── styles/        # Style Design (styles.json + fonts/logos)
│   └── prompts/       # Prompt templates
├── video-projects/    # One folder per video (not committed)
├── image-projects/    # Image-generation projects (not committed)
├── outputs/           # Final videos (not committed)
├── start/             # Startup scripts for Windows (.bat/.ps1) + macOS/Linux (.sh)
└── docs/API.md        # API contract - the single source of truth

Tech stack

Next.js 16 · React 19 · Tailwind 4 · Express 5 · better-sqlite3 · Claude Agent SDK · HyperFrames · Remotion · Gemini API · faster-whisper · FFmpeg