Nido – ITP Thesis Archive 2026

Project Description

This project, Nido, seeks to make 3D room design more accessible to people of all kinds, not just technical experts. There is a huge gap currently in how people engage with planning real 3D spaces. Either someone is a master user of an intricate and expensive 3D software, or they have to go off 2D information—photos and text—for a space. With the recent improvements in graphical compute and 3D capability on internet browsers through libraries like Three.js and SparkJS, we have the technical ability to close this gap but no cheap solutions currently.
 
Inspired by open source projects like Pascal and SuperSplat, I wanted to design a 3D scanning and receiving website that anyone could use. First, someone scans their room and objects with my iOS app. After processing the scans on the server, the results get read from the database to the website, where people can see their room with real dimensions in their browser.
 
Also on the browser, people can upload floorplans that become editable 3D structures using AI Agents, turn images of products into 3D objects with AI, and design iterations of their space with real measurements, without getting out a tape measure.

Technical Details

Nido stands on the shoulders of a lot of open-source work, academic research, and commercial APIs. Credit where it's due:
 
Primary Architectural Reference
 
- Pascal Editor — https://github.com/pascalorg/editor — flat-node store, dirty-node systems, scene registry, and viewer/editor separation patterns
 
Core Web Stack
 
- Next.js — https://github.com/vercel/next.js
 
- React — https://github.com/facebook/react
 
- Three.js — https://github.com/mrdoob/three.js
 
- React Three Fiber — https://github.com/pmndrs/react-three-fiber
 
- drei — https://github.com/pmndrs/drei
 
- camera-controls (via drei) — https://github.com/yomotsu/camera-controls
 
- Zustand — https://github.com/pmndrs/zustand
 
- Zundo (undo/redo middleware) — https://github.com/charkour/zundo
 
- Zod — https://github.com/colinhacks/zod
 
- three-bvh-csg — https://github.com/gkjohnson/three-bvh-csg
 
- three-mesh-bvh — https://github.com/gkjohnson/three-mesh-bvh
 
- Tailwind CSS — https://github.com/tailwindlabs/tailwindcss
 
- pdf.js — https://github.com/mozilla/pdf.js
 
- sharp — https://github.com/lovell/sharp
 

Backend & Infrastructure
 
- Supabase — https://github.com/supabase/supabase
 
- Supabase Swift SDK — https://github.com/supabase/supabase-swift
 
- Modal (GPU compute for splat training) — https://modal.com
 

Mobile Stack

- RoomPlan (Apple) — https://developer.apple.com/augmented-reality/roomplan/
 
- Object Capture (Apple) — https://developer.apple.com/augmented-reality/object-capture/
 
- ARKit (Apple) — https://developer.apple.com/augmented-reality/arkit/
 

Splat Training Pipeline
 
- gsplat — https://github.com/nerfstudio-project/gsplat
 
- Nerfstudio — https://github.com/nerfstudio-project/nerfstudio
 
- 3D Gaussian Splatting (original, Kerbl et al., SIGGRAPH 2023) —
https://github.com/graphdeco-inria/gaussian-splatting
 
- SuperSplat (format conversion tool) — https://playcanvas.com/supersplat/editor
 

- TRELLIS (Microsoft) — https://github.com/microsoft/TRELLIS
 
- Splat Labs (splat ↔ floorplan registration workflow reference) — https://splat-labs.com
 

AI / ML Services
 
- TRELLIS 2 via fal.ai (image-to-3D with PBR) — https://fal.ai
 
- firtoz/trellis on Replicate (current image-to-3D) — https://replicate.com/firtoz/trellis
 
- Google Gemini 3 Pro (floorplan vision) — https://ai.google.dev
 
- Anthropic Claude (development assistant + API usage) — https://anthropic.com

Research/Context

Nido situates itself at the intersection of three established stacks: spatial design software, consumer 3D capture, and browser-based real-time rendering. Professional CAD and BIM tools — Vectorworks, AutoCAD, Revit — offer dimensional rigor but impose a steep learning curve and licensing cost that excludes non-specialists. Consumer-facing alternatives such as SketchUp Free and Floorplanner compromise scale accuracy for accessibility. Nido's main argument is that recent advances in WebGPU rendering, feed-forward gaussian splatting, and on-device LiDAR have collapsed the technical barriers that previously justified this trade-off, and that the remaining gap is one of interaction design.
 
Architecturally, the project draws most heavily on Pascal Editor (pascalorg/editor, MIT), whose flat-node store, scene registry, and strict viewer/editor separation provide a proven pattern for managing large editable scenes in React Three Fiber. The rendering stack — Three.js (WebGPU), drei, Zustand with Zundo for undo history, and Zod for runtime schema validation — was selected to mirror this forked reference implementation.
 
The capture pipeline synthesises findings from three 2025 feed-forward 3DGS papers — LighthouseGS, Flash-Mono, and FastGS — with Apple's RoomPlan and Object Capture APIs, producing a two-tier strategy that preserves a unified guided-sweep UX across LiDAR and non-LiDAR devices. PanSplat (Zhang et al., CVPR 2025) informs the panoramic "quick scan" mode, while ZInD (Cruz et al., CVPR 2021) and the Splat Labs registration workflow ground the floorplan-to-geometry alignment approach. For single-image object reconstruction, TRELLIS 2 (Microsoft, via fal.ai) was chosen over Meshy and Tripo for its native PBR output and licensing terms, complemented by Gemini 3 Pro for vision-based floorplan parsing and SAM / Metric3D as candidate components in a future hybrid reconstruction pipeline when splats can transform into solid 3D textures.

Foundational Gaussian Splatting

- 3D Gaussian Splatting for Real-Time Radiance Field Rendering — Kerbl, Kopanas, Leimkühler, Drettakis (SIGGRAPH 2023) — https://arxiv.org/abs/2308.04079
- Mip-Splatting: Alias-free 3D Gaussian Splatting — Yu et al. (CVPR 2024) — https://arxiv.org/abs/2311.16493
- 2D Gaussian Splatting for Geometrically Accurate Radiance Fields — Huang, Yu, Chen, Geiger, Gao (SIGGRAPH 2024) — https://arxiv.org/abs/2403.17888
- Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering — Lu et al. (CVPR 2024) — https://arxiv.org/abs/2312.00109
- 3D Gaussian Splatting as Markov Chain Monte Carlo — Kheradmand et al. (NeurIPS 2024) — https://arxiv.org/abs/2404.09591

Feed-Forward / Generalizable 3DGS (drives High & Pro tiers)

- pixelSplat: 3D Gaussian Splats from Image Pairs — Charatan, Li, Tagliasacchi, Sitzmann (CVPR 2024) — https://arxiv.org/abs/2312.12337
- MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images — Chen et al. (ECCV 2024) — https://arxiv.org/abs/2403.14627
- Splatter Image: Ultra-Fast Single-View 3D Reconstruction — Szymanowicz, Rupprecht, Vedaldi (CVPR 2024) — https://arxiv.org/abs/2312.13150
- LGM: Large Multi-View Gaussian Model — Tang et al. (ECCV 2024) — https://arxiv.org/abs/2402.05054
- InstantSplat: Sparse-view SfM-free Gaussian Splatting in Seconds — Fan et al. (2024) — https://arxiv.org/abs/2403.20309
- NoPoSplat: No Pose, No Problem — Ye et al. (ICLR 2025) — https://arxiv.org/abs/2410.24207
- LighthouseGS (2025) — https://arxiv.org/abs/2504.05517
- Flash-Mono (2025) — feed-forward monocular 3DGS
- FastGS (2025) — real-time generalizable gaussian splatting

Panoramic / 360° Splatting

- PanSplat: 4K Panorama Synthesis with Feed-Forward Gaussian Splatting — Zhang et al. (CVPR 2025) — https://arxiv.org/abs/2412.12096
- 360-GS: Layout-guided Panoramic Gaussian Splatting — Bai et al. (2024) — https://arxiv.org/abs/2402.00763
- OmniGS: Omnidirectional Gaussian Splatting for Fast Radiance Field Reconstruction — Li et al. (WACV 2025) — https://arxiv.org/abs/2404.03202

Monocular Depth & Geometry Priors

- Metric3D: Towards Zero-shot Metric 3D Prediction — Yin et al. (ICCV 2023) — https://arxiv.org/abs/2307.10984
- Metric3D v2: A Versatile Monocular Geometric Foundation Model — Hu et al. (TPAMI 2024) — https://arxiv.org/abs/2404.15506
- Depth Anything V2 — Yang et al. (NeurIPS 2024) — https://arxiv.org/abs/2406.09414
- MoGe: Unlocking Accurate Monocular Geometry Estimation — Wang et al. (2024) — https://arxiv.org/abs/2410.19115
- DUSt3R: Geometric 3D Vision Made Easy — Wang et al. (CVPR 2024) — https://arxiv.org/abs/2312.14132
- MASt3R: Grounding Image Matching in 3D — Leroy, Cabon, Revaud (ECCV 2024) — https://arxiv.org/abs/2406.09756

Segmentation & Editing for Splats

- Segment Anything (SAM) — Kirillov et al. (ICCV 2023) — https://arxiv.org/abs/2304.02643
- SAM 2: Segment Anything in Images and Videos — Ravi et al. (2024) — https://arxiv.org/abs/2408.00714
- Gaussian Grouping: Segment and Edit Anything in 3D Scenes — Ye et al. (ECCV 2024) — https://arxiv.org/abs/2312.00732
- SAGA: Segment Any 3D Gaussians — Cen et al. (2023) — https://arxiv.org/abs/2312.00860
- Feature 3DGS: Supercharging 3DGS with Foundation Feature Fields — Zhou et al. (CVPR 2024) — https://arxiv.org/abs/2312.03203

Room Layout & Floorplan Estimation

- ZInD — Zillow Indoor Dataset — Cruz et al. (CVPR 2021) — https://arxiv.org/abs/2109.13403
- HorizonNet: Learning Room Layout with 1D Representation — Sun et al. (CVPR 2019) — https://arxiv.org/abs/1901.03861
- LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image — Zou et al. (CVPR 2018) — https://arxiv.org/abs/1803.08999
- RoomFormer: Two-level Queries for Single-Stage Floorplan Reconstruction — Yue et al. (CVPR 2023) — https://arxiv.org/abs/2211.15658
- HoHoNet: 360 Indoor Holistic Understanding with Latent Horizontal Features — Sun et al. (CVPR 2021) — https://arxiv.org/abs/2011.11498

Image/Text-to-3D (furniture & object generation)

- TRELLIS: Structured 3D Latents for Scalable and Versatile 3D Generation — Xiang et al. (2024) — https://arxiv.org/abs/2412.01506
- DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation — Tang et al. (ICLR 2024) — https://arxiv.org/abs/2309.16653
- LRM: Large Reconstruction Model for Single Image to 3D — Hong et al. (ICLR 2024) — https://arxiv.org/abs/2311.04400
- InstantMesh: Efficient 3D Mesh Generation from a Single Image — Xu et al. (2024) — https://arxiv.org/abs/2404.07191
- CAST: Component-Aligned 3D Scene Reconstruction from an RGB Image — Yao et al. (SIGGRAPH 2025) — https://arxiv.org/abs/2502.12894

SfM / MVS Foundations

- Structure-from-Motion Revisited (COLMAP) — Schönberger, Frahm (CVPR 2016) — https://colmap.github.io
- Pixelwise View Selection for Unstructured Multi-View Stereo — Schönberger et al. (ECCV 2016)
- GLOMAP: Global Structure-from-Motion Revisited — Pan et al. (ECCV 2024) — https://arxiv.org/abs/2407.20219

NeRF Predecessors (context)

- NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis — Mildenhall et al. (ECCV 2020) — https://arxiv.org/abs/2003.08934
- Instant Neural Graphics Primitives with a Multiresolution Hash Encoding — Müller et al. (SIGGRAPH 2022) — https://arxiv.org/abs/2201.05989
- Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields — Barron et al. (ICCV 2023) — https://arxiv.org/abs/2304.06706

Dynamic Scenes (future-work context)

- 4D Gaussian Splatting for Real-Time Dynamic Scene Rendering — Wu et al. (CVPR 2024) — https://arxiv.org/abs/2310.08528

Project Description

Technical Details

Research/Context

Further Reading