\n This project, Nido, seeks to make 3D room design more accessible to people of all kinds, not just technical experts. There is a huge gap currently in how people engage with planning real 3D spaces. Either someone is a master user of an intricate and expensive 3D software, or they have to go off 2D information\u2014photos and text\u2014for a space. With the recent improvements in graphical compute and 3D capability on internet browsers through libraries like Three.js and SparkJS, we have the technical ability to close this gap but no cheap solutions currently.
\n <br\/> <br\/>
\nInspired by open source projects like Pascal and SuperSplat, I wanted to design a 3D scanning and receiving website that anyone could use. First, someone scans their room and objects with my iOS app. After processing the scans on the server, the results get read from the database to the website, where people can see their room with real dimensions in their browser.
\n <br\/> <br\/>
\nAlso on the browser, people can upload floorplans that become editable 3D structures using AI Agents, turn images of products into 3D objects with AI, and design iterations of their space with real measurements, without getting out a tape measure. <\/p>\n
\n Nido stands on the shoulders of a lot of open-source work, academic research, and commercial APIs. Credit where it's due:
\n <br\/> <br\/>
\n Primary Architectural Reference
\n <br\/> <br\/>
\n – Pascal Editor \u2014 https:\/\/github.com\/pascalorg\/editor \u2014 flat-node store, dirty-node systems, scene registry, and viewer\/editor separation patterns
\n <br\/> <br\/>
\n Core Web Stack
\n <br\/> <br\/>
\n – Next.js \u2014 https:\/\/github.com\/vercel\/next.js
\n <br\/> <br\/>
\n – React \u2014 https:\/\/github.com\/facebook\/react
\n <br\/> <br\/>
\n – Three.js \u2014 https:\/\/github.com\/mrdoob\/three.js
\n <br\/> <br\/>
\n – React Three Fiber \u2014 https:\/\/github.com\/pmndrs\/react-three-fiber
\n <br\/> <br\/>
\n – drei \u2014 https:\/\/github.com\/pmndrs\/drei
\n <br\/> <br\/>
\n – camera-controls (via drei) \u2014 https:\/\/github.com\/yomotsu\/camera-controls
\n <br\/> <br\/>
\n – Zustand \u2014 https:\/\/github.com\/pmndrs\/zustand
\n <br\/> <br\/>
\n – Zundo (undo\/redo middleware) \u2014 https:\/\/github.com\/charkour\/zundo
\n <br\/> <br\/>
\n – Zod \u2014 https:\/\/github.com\/colinhacks\/zod
\n <br\/> <br\/>
\n – three-bvh-csg \u2014 https:\/\/github.com\/gkjohnson\/three-bvh-csg
\n <br\/> <br\/>
\n – three-mesh-bvh \u2014 https:\/\/github.com\/gkjohnson\/three-mesh-bvh
\n <br\/> <br\/>
\n – Tailwind CSS \u2014 https:\/\/github.com\/tailwindlabs\/tailwindcss
\n <br\/> <br\/>
\n – pdf.js \u2014 https:\/\/github.com\/mozilla\/pdf.js
\n <br\/> <br\/>
\n – sharp \u2014 https:\/\/github.com\/lovell\/sharp
\n <br\/> <br\/> <\/p>\n
Backend & Infrastructure
\n <br\/> <br\/>
\n – Supabase \u2014 https:\/\/github.com\/supabase\/supabase
\n <br\/> <br\/>
\n – Supabase Swift SDK \u2014 https:\/\/github.com\/supabase\/supabase-swift
\n <br\/> <br\/>
\n – Modal (GPU compute for splat training) \u2014 https:\/\/modal.com
\n <br\/> <br\/> <\/p>\n
Mobile Stack <\/p>\n
– RoomPlan (Apple) \u2014 https:\/\/developer.apple.com\/augmented-reality\/roomplan\/
\n <br\/> <br\/>
\n – Object Capture (Apple) \u2014 https:\/\/developer.apple.com\/augmented-reality\/object-capture\/
\n <br\/> <br\/>
\n – ARKit (Apple) \u2014 https:\/\/developer.apple.com\/augmented-reality\/arkit\/
\n <br\/> <br\/> <\/p>\n
Splat Training Pipeline
\n <br\/> <br\/>
\n – gsplat \u2014 https:\/\/github.com\/nerfstudio-project\/gsplat
\n <br\/> <br\/>
\n – Nerfstudio \u2014 https:\/\/github.com\/nerfstudio-project\/nerfstudio
\n <br\/> <br\/>
\n – 3D Gaussian Splatting (original, Kerbl et al., SIGGRAPH 2023) \u2014
\n https:\/\/github.com\/graphdeco-inria\/gaussian-splatting
\n <br\/> <br\/>
\n – SuperSplat (format conversion tool) \u2014 https:\/\/playcanvas.com\/supersplat\/editor
\n <br\/> <br\/> <\/p>\n
– TRELLIS (Microsoft) \u2014 https:\/\/github.com\/microsoft\/TRELLIS
\n <br\/> <br\/>
\n – Splat Labs (splat \u2194 floorplan registration workflow reference) \u2014 https:\/\/splat-labs.com
\n <br\/> <br\/> <\/p>\n
AI \/ ML Services
\n <br\/> <br\/>
\n – TRELLIS 2 via fal.ai (image-to-3D with PBR) \u2014 https:\/\/fal.ai
\n <br\/> <br\/>
\n – firtoz\/trellis on Replicate (current image-to-3D) \u2014 https:\/\/replicate.com\/firtoz\/trellis
\n <br\/> <br\/>
\n – Google Gemini 3 Pro (floorplan vision) \u2014 https:\/\/ai.google.dev
\n <br\/> <br\/>
\n – Anthropic Claude (development assistant + API usage) \u2014 https:\/\/anthropic.com
\n <br\/> <br\/> \n <\/p>\n
\n Nido situates itself at the intersection of three established stacks: spatial design software, consumer 3D capture, and browser-based real-time rendering. Professional CAD and BIM tools \u2014 Vectorworks, AutoCAD, Revit \u2014 offer dimensional rigor but impose a steep learning curve and licensing cost that excludes non-specialists. Consumer-facing alternatives such as SketchUp Free and Floorplanner compromise scale accuracy for accessibility. Nido's main argument is that recent advances in WebGPU rendering, feed-forward gaussian splatting, and on-device LiDAR have collapsed the technical barriers that previously justified this trade-off, and that the remaining gap is one of interaction design.
\n <br\/> <br\/>
\nArchitecturally, the project draws most heavily on Pascal Editor (pascalorg\/editor, MIT), whose flat-node store, scene registry, and strict viewer\/editor separation provide a proven pattern for managing large editable scenes in React Three Fiber. The rendering stack \u2014 Three.js (WebGPU), drei, Zustand with Zundo for undo history, and Zod for runtime schema validation \u2014 was selected to mirror this forked reference implementation.
\n <br\/> <br\/>
\nThe capture pipeline synthesises findings from three 2025 feed-forward 3DGS papers \u2014 LighthouseGS, Flash-Mono, and FastGS \u2014 with Apple's RoomPlan and Object Capture APIs, producing a two-tier strategy that preserves a unified guided-sweep UX across LiDAR and non-LiDAR devices. PanSplat (Zhang et al., CVPR 2025) informs the panoramic "quick scan" mode, while ZInD (Cruz et al., CVPR 2021) and the Splat Labs registration workflow ground the floorplan-to-geometry alignment approach. For single-image object reconstruction, TRELLIS 2 (Microsoft, via fal.ai) was chosen over Meshy and Tripo for its native PBR output and licensing terms, complemented by Gemini 3 Pro for vision-based floorplan parsing and SAM \/ Metric3D as candidate components in a future hybrid reconstruction pipeline when splats can transform into solid 3D textures. <\/p>\n
\n Foundational Gaussian Splatting <\/p>\n
– 3D Gaussian Splatting for Real-Time Radiance Field Rendering \u2014 Kerbl, Kopanas, Leimk\u00fchler, Drettakis (SIGGRAPH 2023) \u2014 https:\/\/arxiv.org\/abs\/2308.04079
\n – Mip-Splatting: Alias-free 3D Gaussian Splatting \u2014 Yu et al. (CVPR 2024) \u2014 https:\/\/arxiv.org\/abs\/2311.16493
\n – 2D Gaussian Splatting for Geometrically Accurate Radiance Fields \u2014 Huang, Yu, Chen, Geiger, Gao (SIGGRAPH 2024) \u2014 https:\/\/arxiv.org\/abs\/2403.17888
\n – Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering \u2014 Lu et al. (CVPR 2024) \u2014 https:\/\/arxiv.org\/abs\/2312.00109
\n – 3D Gaussian Splatting as Markov Chain Monte Carlo \u2014 Kheradmand et al. (NeurIPS 2024) \u2014 https:\/\/arxiv.org\/abs\/2404.09591 <\/p>\n
Feed-Forward \/ Generalizable 3DGS (drives High & Pro tiers) <\/p>\n
– pixelSplat: 3D Gaussian Splats from Image Pairs \u2014 Charatan, Li, Tagliasacchi, Sitzmann (CVPR 2024) \u2014 https:\/\/arxiv.org\/abs\/2312.12337
\n – MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images \u2014 Chen et al. (ECCV 2024) \u2014 https:\/\/arxiv.org\/abs\/2403.14627
\n – Splatter Image: Ultra-Fast Single-View 3D Reconstruction \u2014 Szymanowicz, Rupprecht, Vedaldi (CVPR 2024) \u2014 https:\/\/arxiv.org\/abs\/2312.13150
\n – LGM: Large Multi-View Gaussian Model \u2014 Tang et al. (ECCV 2024) \u2014 https:\/\/arxiv.org\/abs\/2402.05054
\n – InstantSplat: Sparse-view SfM-free Gaussian Splatting in Seconds \u2014 Fan et al. (2024) \u2014 https:\/\/arxiv.org\/abs\/2403.20309
\n – NoPoSplat: No Pose, No Problem \u2014 Ye et al. (ICLR 2025) \u2014 https:\/\/arxiv.org\/abs\/2410.24207
\n – LighthouseGS (2025) \u2014 https:\/\/arxiv.org\/abs\/2504.05517
\n – Flash-Mono (2025) \u2014 feed-forward monocular 3DGS
\n – FastGS (2025) \u2014 real-time generalizable gaussian splatting <\/p>\n
Panoramic \/ 360\u00b0 Splatting<\/p>\n
– PanSplat: 4K Panorama Synthesis with Feed-Forward Gaussian Splatting \u2014 Zhang et al. (CVPR 2025) \u2014 https:\/\/arxiv.org\/abs\/2412.12096
\n – 360-GS: Layout-guided Panoramic Gaussian Splatting \u2014 Bai et al. (2024) \u2014 https:\/\/arxiv.org\/abs\/2402.00763
\n – OmniGS: Omnidirectional Gaussian Splatting for Fast Radiance Field Reconstruction \u2014 Li et al. (WACV 2025) \u2014 https:\/\/arxiv.org\/abs\/2404.03202 <\/p>\n
Monocular Depth & Geometry Priors <\/p>\n
– Metric3D: Towards Zero-shot Metric 3D Prediction \u2014 Yin et al. (ICCV 2023) \u2014 https:\/\/arxiv.org\/abs\/2307.10984
\n – Metric3D v2: A Versatile Monocular Geometric Foundation Model \u2014 Hu et al. (TPAMI 2024) \u2014 https:\/\/arxiv.org\/abs\/2404.15506
\n – Depth Anything V2 \u2014 Yang et al. (NeurIPS 2024) \u2014 https:\/\/arxiv.org\/abs\/2406.09414
\n – MoGe: Unlocking Accurate Monocular Geometry Estimation \u2014 Wang et al. (2024) \u2014 https:\/\/arxiv.org\/abs\/2410.19115
\n – DUSt3R: Geometric 3D Vision Made Easy \u2014 Wang et al. (CVPR 2024) \u2014 https:\/\/arxiv.org\/abs\/2312.14132
\n – MASt3R: Grounding Image Matching in 3D \u2014 Leroy, Cabon, Revaud (ECCV 2024) \u2014 https:\/\/arxiv.org\/abs\/2406.09756 <\/p>\n
Segmentation & Editing for Splats <\/p>\n
– Segment Anything (SAM) \u2014 Kirillov et al. (ICCV 2023) \u2014 https:\/\/arxiv.org\/abs\/2304.02643
\n – SAM 2: Segment Anything in Images and Videos \u2014 Ravi et al. (2024) \u2014 https:\/\/arxiv.org\/abs\/2408.00714
\n – Gaussian Grouping: Segment and Edit Anything in 3D Scenes \u2014 Ye et al. (ECCV 2024) \u2014 https:\/\/arxiv.org\/abs\/2312.00732
\n – SAGA: Segment Any 3D Gaussians \u2014 Cen et al. (2023) \u2014 https:\/\/arxiv.org\/abs\/2312.00860
\n – Feature 3DGS: Supercharging 3DGS with Foundation Feature Fields \u2014 Zhou et al. (CVPR 2024) \u2014 https:\/\/arxiv.org\/abs\/2312.03203 <\/p>\n
Room Layout & Floorplan Estimation <\/p>\n
– ZInD \u2014 Zillow Indoor Dataset \u2014 Cruz et al. (CVPR 2021) \u2014 https:\/\/arxiv.org\/abs\/2109.13403
\n – HorizonNet: Learning Room Layout with 1D Representation \u2014 Sun et al. (CVPR 2019) \u2014 https:\/\/arxiv.org\/abs\/1901.03861
\n – LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image \u2014 Zou et al. (CVPR 2018) \u2014 https:\/\/arxiv.org\/abs\/1803.08999
\n – RoomFormer: Two-level Queries for Single-Stage Floorplan Reconstruction \u2014 Yue et al. (CVPR 2023) \u2014 https:\/\/arxiv.org\/abs\/2211.15658
\n – HoHoNet: 360 Indoor Holistic Understanding with Latent Horizontal Features \u2014 Sun et al. (CVPR 2021) \u2014 https:\/\/arxiv.org\/abs\/2011.11498 <\/p>\n
Image\/Text-to-3D (furniture & object generation) <\/p>\n
– TRELLIS: Structured 3D Latents for Scalable and Versatile 3D Generation \u2014 Xiang et al. (2024) \u2014 https:\/\/arxiv.org\/abs\/2412.01506
\n – DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation \u2014 Tang et al. (ICLR 2024) \u2014 https:\/\/arxiv.org\/abs\/2309.16653
\n – LRM: Large Reconstruction Model for Single Image to 3D \u2014 Hong et al. (ICLR 2024) \u2014 https:\/\/arxiv.org\/abs\/2311.04400
\n – InstantMesh: Efficient 3D Mesh Generation from a Single Image \u2014 Xu et al. (2024) \u2014 https:\/\/arxiv.org\/abs\/2404.07191
\n – CAST: Component-Aligned 3D Scene Reconstruction from an RGB Image \u2014 Yao et al. (SIGGRAPH 2025) \u2014 https:\/\/arxiv.org\/abs\/2502.12894 <\/p>\n
SfM \/ MVS Foundations <\/p>\n
– Structure-from-Motion Revisited (COLMAP) \u2014 Sch\u00f6nberger, Frahm (CVPR 2016) \u2014 https:\/\/colmap.github.io
\n – Pixelwise View Selection for Unstructured Multi-View Stereo \u2014 Sch\u00f6nberger et al. (ECCV 2016)
\n – GLOMAP: Global Structure-from-Motion Revisited \u2014 Pan et al. (ECCV 2024) \u2014 https:\/\/arxiv.org\/abs\/2407.20219 <\/p>\n
NeRF Predecessors (context) <\/p>\n
– NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis \u2014 Mildenhall et al. (ECCV 2020) \u2014 https:\/\/arxiv.org\/abs\/2003.08934
\n – Instant Neural Graphics Primitives with a Multiresolution Hash Encoding \u2014 M\u00fcller et al. (SIGGRAPH 2022) \u2014 https:\/\/arxiv.org\/abs\/2201.05989
\n – Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields \u2014 Barron et al. (ICCV 2023) \u2014 https:\/\/arxiv.org\/abs\/2304.06706 <\/p>\n
Dynamic Scenes (future-work context) <\/p>\n
– 4D Gaussian Splatting for Real-Time Dynamic Scene Rendering \u2014 Wu et al. (CVPR 2024) \u2014 https:\/\/arxiv.org\/abs\/2310.08528\n <\/p>\n","protected":false},"excerpt":{"rendered":"
Nido is a mobile scanning app and room editing website. Scan your room, bring it to the web, and design your space without any fuss.<\/p>\n","protected":false},"author":0,"featured_media":199,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[14,27,15,3],"class_list":["post-67","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-2","tag-machine-learning","tag-mobile","tag-product-design","tag-speculative-futures"],"_links":{"self":[{"href":"https:\/\/itp.nyu.edu\/thesis\/archive\/2026\/wp-json\/wp\/v2\/posts\/67","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/itp.nyu.edu\/thesis\/archive\/2026\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/itp.nyu.edu\/thesis\/archive\/2026\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/itp.nyu.edu\/thesis\/archive\/2026\/wp-json\/wp\/v2\/comments?post=67"}],"version-history":[{"count":6,"href":"https:\/\/itp.nyu.edu\/thesis\/archive\/2026\/wp-json\/wp\/v2\/posts\/67\/revisions"}],"predecessor-version":[{"id":145,"href":"https:\/\/itp.nyu.edu\/thesis\/archive\/2026\/wp-json\/wp\/v2\/posts\/67\/revisions\/145"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/itp.nyu.edu\/thesis\/archive\/2026\/wp-json\/wp\/v2\/media\/199"}],"wp:attachment":[{"href":"https:\/\/itp.nyu.edu\/thesis\/archive\/2026\/wp-json\/wp\/v2\/media?parent=67"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/itp.nyu.edu\/thesis\/archive\/2026\/wp-json\/wp\/v2\/categories?post=67"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/itp.nyu.edu\/thesis\/archive\/2026\/wp-json\/wp\/v2\/tags?post=67"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}