Frame interpolation

Also known as: video frame interpolation

On this page 7

Frame interpolation, also called video frame interpolation, is the process of synthesizing new intermediate frames between two existing frames of a video so that motion appears continuous. In generative AI, it is handled by neural networks that estimate motion and generate plausible pixels for moments the camera never captured.

The technique raises frame rates, for example from 24 to 60 frames per second, creates slow motion from normal footage and fills gaps in AI-generated video. Its central difficulty is that the missing frame is not uniquely determined by its neighbors, so the model must infer motion, occlusion and appearance at the same time.

How frame interpolation estimates motion

Frame interpolation works by estimating where each pixel moves between two input frames and then building the intermediate frame at a chosen time step, usually written as t between 0 and 1. A value of t equal to 0.5 produces the frame exactly halfway between the inputs.

Classical and many neural methods compute optical flow, a dense field of motion vectors, in both directions. The input frames are then warped toward the target time along those vectors, and a learned fusion step blends the warped images while masking areas that one frame cannot see.

The hardest regions are occlusions and disocclusions, where an object covers or reveals background. Here one input frame has no valid information, so the network must decide which frame to trust or, in generative approaches, invent consistent content. The table below summarizes the stages of a typical flow-based pipeline.

StagePurposeCommon failure
Flow estimationFind motion between input framesErrors on fast or thin objects
WarpingMove pixels to the target timeHoles and stretched textures
Occlusion reasoningChoose the visible source per pixelGhosting at object edges
Fusion and refinementBlend and correct the resultBlur in uncertain regions
Stages of a flow-based interpolation pipeline

Approaches to video frame interpolation

Frame interpolation methods fall into a few families that differ in how explicitly they model motion. The choice affects speed, sharpness and behavior on difficult scenes.

Flow-based networks remain the most widely deployed because they are fast and accurate on moderate motion. Kernel-based methods predict a small convolution kernel for every output pixel, which handles subtle motion well but limits the size of displacement. Generative methods, especially diffusion models, treat the missing frame as a sample conditioned on its neighbors and can produce sharp detail where deterministic methods blur.

ApproachCore ideaStrengthWeakness
Frame blendingAverage neighboring framesTrivial and fastDouble images on motion
Motion-compensated (classical)Block or flow vectors plus warpingRuns in real time on hardwareArtifacts on complex motion
Flow-based neuralLearned flow, warping and fusionAccurate on moderate motionStruggles with large displacement
Kernel-based neuralPer-pixel adaptive kernelsFine detail on small motionLimited motion range
Diffusion-based generativeSample frame conditioned on inputsSharp, plausible contentSlow and may hallucinate detail
Families of frame interpolation methods

Deterministic networks trained with pixel losses tend to output the average of all plausible frames, which looks blurry when motion is ambiguous. Generative models instead commit to one plausible outcome, trading exact fidelity for perceptual sharpness.

Frame interpolation in generative video pipelines

In generative AI video, frame interpolation is often a separate stage after generation. Text-to-video and image-to-video models are expensive per frame, so many pipelines generate a short clip at 8 to 16 frames per second and then interpolate to 24, 30 or 60 frames per second.

Video diffusion models can also perform interpolation directly when they are conditioned on a first and last frame, sometimes called keyframe interpolation. This differs from latent interpolation, which blends points in latent space to morph between two images rather than to reconstruct physical motion over time.

TechniqueInputOutputGoal
Frame interpolationTwo or more neighboring framesIntermediate framesSmooth, physically plausible motion
Image-to-videoOne still image and a promptNew video clipCreate motion from scratch
Latent interpolationTwo latent codesBlended imagesMorph between concepts
Video super-resolutionLow-resolution framesHigher-resolution framesAdd spatial detail, not time
Frame interpolation compared with related techniques

Interpolation also supports temporal consistency: inserting frames that respect motion can hide small flicker between generated frames, although it cannot repair objects that change identity from one keyframe to the next.

Choosing and configuring frame interpolation

The right frame interpolation setup depends on motion magnitude, latency budget and tolerance for artifacts. Key parameters include the interpolation factor, such as 2x or 4x, and whether the method supports arbitrary time steps or only the midpoint.

SituationRecommended choiceReason
Live playback or gamesHardware or lightweight flow modelLatency must stay within milliseconds
Slow motion from sports footageHigh-quality flow network, 4x to 8xLarge, fast motion needs strong flow
AI-generated clips at low frame rateFlow model or keyframe-conditioned diffusionFills gaps without regenerating content
Archival film restorationConservative flow method with reviewFidelity matters more than sharpness
Decision guide for frame interpolation

Recursive midpoint interpolation, which repeatedly inserts frames halfway, accumulates errors at high factors, so methods with direct arbitrary-time prediction usually perform better for 8x or more.

Artifacts, limits and ethical concerns

Frame interpolation produces characteristic artifacts on difficult content. Ghosting and halos appear at occlusion boundaries, thin structures such as fences or wires break apart, and repeating patterns confuse flow estimation. Scene cuts must be detected first, because interpolating across a cut blends two unrelated shots.

  • Motion blur in the source cannot be removed by inserting frames, so interpolated slow motion of blurred footage still looks soft.
  • Cinematic intent matters: many viewers perceive interpolated 24 fps film as the so-called soap opera effect.
  • Evidentiary video must not be interpolated without disclosure, because synthesized frames show moments that were never recorded.

Quality is commonly measured with PSNR, SSIM and perceptual metrics such as LPIPS on benchmark datasets, but these scores do not fully capture temporal smoothness, so visual review remains necessary.

Put simply

A video is a fast series of still pictures. Frame interpolation creates extra pictures in between the real ones, so movement looks smoother or can be slowed down.

It is like a flip-book artist drawing additional pages between two existing drawings. The computer works out where things are moving and paints the missing moment. When motion is simple, the result looks natural. When objects move very fast or hide behind each other, the guesswork can show as blurry edges or ghostly shapes.

In practice

A small studio produces a 20-second product animation with a text-to-video model. Generating at full quality is slow, so the team renders the clip at 12 frames per second, which keeps generation time manageable but makes the rotating product look choppy.

The editor runs a neural flow-based interpolation model with a 4x factor, producing 48 frames per second, and then conforms the result to 24 frames per second for delivery. Smooth surfaces and slow camera moves look clean, but a spinning logo with fine lettering shows ghosting in a few frames.

The editor masks that segment, regenerates it with a first-and-last-frame conditioned video model, and reviews the cut frame by frame. The final clip plays smoothly, and the studio cuts total rendering time roughly in half compared with generating every frame directly.