Photon Frame Generation

Frame generation for Minecraft Java 1.20.1 (Forge). PhotonFrameGen inserts generated frames between the frames Minecraft actually renders. At 40 real FPS you see 80 (2X), 120 (3X) or 160 (4X) frames on screen. It is built for heavy shader packs, where the real frame rate is low and every extra frame of smoothness counts.

These are real presented frames, not motion blur, not a fake FPS counter and not frame duplication. Each generated frame is a separate buffer swap that reaches your monitor. The overlay shows real, generated and displayed FPS separately.

How it works

• Motion estimation on the GPU. Bidirectional optical flow between two real frames (coarse-to-fine block matching, PatchMatch propagation, forward/backward consistency check).
• Depth reprojection (with PhotonEngine). Static geometry (terrain, walls, trees) is reprojected exactly from camera motion and per-pixel depth, using both frames. Block edges stay sharp, walls don’t melt, and the nearer surface correctly hides the farther one at silhouettes.
• Disocclusion handling. Areas that become visible during movement are detected and filled only from the frame that actually sees them. Nothing is invented.
• Optical flow only where things really move (mobs, swinging items), with confidence-based fallbacks everywhere else.
• Sharp HUD. Crosshair, hotbar, chat, F3 screen and modded overlays are copied unchanged into every generated frame. No warped text.
• Undistorted first-person hand (with PhotonEngine). The hand and held item are kept out of the motion estimation and are never warped.

Features

• Modes: Off, 2X, 3X, 4X, Adaptive. Adaptive picks the multiplier that fills your monitor’s refresh rate and turns itself off when it wouldn’t help.
• Frame pacing: GPU timestamps measure where each render stage happens, and held frames are shown at evenly spaced moments.
• In-game settings: a page in Embeddium’s Video Settings (next to Oculus’ Shader Packs), the Mods menu → Config, or F6.
• Stats overlay (F7):
  • real / generated / displayed FPS;
  • GPU cost per stage;
  • presentation interval and jitter;
  • estimated added latency;
  • VRAM use.
• 12 debug views (F9): motion vectors, confidence, disocclusion, depth, hand mask, HUD mask, split screen, and a generated-frame marker for checking frame alternation in recordings.
• Fail-safe: if anything goes wrong, frame generation switches itself off and logs the error instead of crashing the game.

Requirements

• Minecraft 1.20.1, Forge 47.x
• Client-side only. Not needed on servers.
• An OpenGL 3.3+ GPU. Tested target: RTX 3060.
• Recommended: Oculus + Embeddium with the PhotonEngine shader pack (option PhotonFrameGen Data is on by default). This enables depth reprojection and the separate hand handling. Other shader packs and vanilla work too, but use image-only motion estimation, with more artifacts.

Recommended settings

• 2X, Motion quality: Medium. Best image quality per millisecond.
• Use a base of at least ~30 real FPS. Frame generation can’t fix very low frame rates; the lower the base, the larger the motion between frames.
• 3X/4X work best with 50+ real FPS on 144–240 Hz monitors.

Honest limitations

• Latency: interpolation needs the next real frame, so input latency rises by about half a real frame at 2X (≈8 ms at 60 FPS, ≈17 ms at 30 FPS). The overlay shows the estimate.
• Hard cases: particles, rain, fast mobs on similar-looking textures and very fast mouse flicks can still show artifacts, especially at 3X/4X.
• Without PhotonEngine data (other shader packs or vanilla), the hand and walls are reconstructed from the image alone and look noticeably worse.
• Experimental: reports and screenshots are welcome. Please include the F7 overlay and the PhotonFrameGen diag lines from latest.log.