HN Showcase
Back to browse
Got VACE working in real-time – 30fps on a 5090

Got VACE working in real-time – 30fps on a 5090

by cmuir·Feb 12, 2026·10 points·0 comments
Visit ProjectView on HN

AI Analysis

●●SolidWizardryEye CandyShip It

VACE ported to real-time streaming without retraining—but limited to proprietary Daydream Scope ecosystem.

Strengths
  • •Training-free adaptation preserves VACE's full control API (R2V, V2V, inpainting, composition) in streaming context via causal attention.
  • •Demonstrates 20–28 fps on consumer hardware (5090); real-time interactivity proof with dual LoRA + scribble stacking.
  • •Technical dive post explains causal attention constraints and masking system; well-reasoned engineering decision.
Weaknesses
  • •Reference-to-video quality described as poor compared to batch; effectively gimped control mode in real-time.
  • •Locked into Daydream Scope + API; early ComfyUI support and no standalone model weights—vendor lock-in risk.
Category
AI/ML
Target Audience

Video AI practitioners, creative technologists, game/interactive media developers

Similar To

Runway Gen-3 · Krea Real-Time · ControlNet

Post Description

I adapted VACE to work with real-time autoregressive video generation.

Here's what it can do right now in real time:

- Depth, pose, optical flow, scribble, edge maps — all the v2v control stuff - First frame animation / last frame lead-in / keyframe interpolation - Inpainting with static or dynamic masks - Stacking stuff together (e.g. depth + LoRA, inpainting + reference images) - Reference-to-video is in there too but honestly quality isn't great yet compared to batch

Getting ~20 fps for most control modes on a 5090 at 368x640 with the 1.3B models. Image-to-video hits ~28 fps. Works with 14b models as well, but doesnt fit on 5090 with VACE.

This is all part of Daydream Scope (https://github.com/daydreamlive/scope), which is an open source tool for running real-time interactive video generation pipelines. The demo was created in scope, and is a combination of Longlive, VACE+Scribble, Custom LoRA.

There's also a very early WIP ComfyUI node pack wrapping scope: https://github.com/daydreamlive/ComfyUI-Daydream-Scope

Curious what people think.

Similar Projects