Stop Coding 3D UIs! Build Shapes With Bare Hands Using This Repo

Stop Coding 3D UIs! Build Shapes With Bare Hands Using This Repo

What if I told you that everything you know about 3D interface design is about to become obsolete? No more wrestling with complex mouse controls, no more endless parameter tweaking in clunky panels, no more context-switching between your creative vision and the mechanical input devices that stand between you and your art.

Here's the uncomfortable truth: we've been building 3D content the wrong way for decades. We've accepted friction as inevitable. We've normalized the disconnect between human intention and digital output. But what if your hands—those same hands that sculpt clay, mold dough, and express ideas through gesture—could become your primary creative instrument in the browser?

Enter shape-creator-tutorial, the open-source project that's making developers everywhere rethink their entire approach to interactive 3D. Built by the visionary creator behind Fun With Computer Vision, this repository doesn't just demonstrate hand tracking—it weaponizes it. Using nothing more than your webcam, MediaPipe's blazing-fast computer vision pipeline, and Three.js's industrial-strength rendering engine, you can spawn, manipulate, and destroy 3D geometry in real-time using natural hand gestures.

The implications are staggering. Prototyping becomes performance art. Education becomes embodied experience. Accessibility barriers crumble. And the best part? It's all happening in your browser, with zero installations, no proprietary hardware, and complete transparency under the MIT license.

Ready to see how deep this rabbit hole goes? Let's dismantle everything you thought you knew about 3D interaction.

What is shape-creator-tutorial?

shape-creator-tutorial is a real-time hand-controlled 3D shape creation system that transforms your bare hands into precision instruments for browser-based geometry manipulation. Created by Alan (@measure_plan on X/Twitter, @stereo.drift on Instagram), this project sits at the explosive intersection of computer vision, WebGL rendering, and human-computer interaction design.

The repository emerged from a broader mission: making computer vision accessible, tangible, and genuinely fun. Alan's platform Fun With Computer Vision hosts multiple open-source experiments, but shape-creator-tutorial stands out for its elegant simplicity and devastating effectiveness. Where other gesture systems drown users in calibration rituals and unreliable detection, this project achieves remarkable stability through MediaPipe's battle-tested hand landmark model—21 precise tracking points per hand, running at interactive frame rates directly in the browser.

Why it's trending now: The convergence of three massive forces makes this project irresistibly timely. First, MediaPipe's JavaScript^{↗ Bright Coding Blog} Solutions API matured dramatically, bringing production-grade hand tracking to web developers without machine learning expertise. Second, Three.js r150+ releases dramatically simplified modern WebGL workflows with improved renderer defaults and streamlined scene management. Third, and most critically, the post-pandemic explosion of spatial computing interest—Apple Vision Pro, Meta's Quest ecosystem, WebXR standardization—has primed developers to think beyond flat screens.

But here's what separates shape-creator-tutorial from the hype cycle: it works today, in browsers people actually use, with hardware they already own. No $3,500 headset. No app store gatekeepers. Just a URL, a webcam, and your hands.

Key Features That Separate It From the Pack

Let's dissect what makes this repository genuinely special, not just technically interesting.

Sub-20ms Hand Landmark Detection MediaPipe's hand solution runs the full pipeline—detection, landmark localization, and world coordinate estimation—at speeds that feel instantaneous. The 21 landmarks include metacarpophalangeal, proximal interphalangeal, and distal interphalangeal joints, plus the wrist and finger tips. This granularity enables nuanced gesture differentiation: pinch to grab, spread to scale, swipe to spawn.

Zero-Dependency Three.js Integration The project uses vanilla Three.js without abstractions that bloat bundle size or obscure rendering behavior. You get direct access to the scene graph, the WebGLRenderer with its physically-based material system, and the animation loop. This matters because performance-critical 3D applications need surgical control, not framework hand-holding.

Immediate Mode Feedback via HTML5 Canvas A secondary canvas layer provides real-time visual feedback on hand detection confidence, landmark positions, and gesture state. This dual-canvas architecture—WebGL for 3D, 2D canvas for HUD elements—prevents GPU pipeline stalls and keeps frame rates buttery.

MIT Licensed, Zero Attribution Friction Commercial project? Educational fork? Art installation? The license says yes to everything. Alan explicitly encourages experimentation, modification, and redistribution.

Progressive Enhancement Philosophy The codebase degrades gracefully. No camera? The Three.js scene still renders. No WebGL? A clear error message appears. This isn't demo-ware built for perfect conditions; it's engineered for real-world deployment.

Tutorial-First Architecture Every commit, every module boundary, every naming convention serves pedagogical clarity. The repository isn't just a product—it's a curriculum. Compare this to most computer vision repos, which are impenetrable research artifacts or brittle production systems with zero educational scaffolding.

Use Cases Where This Changes Everything

Still wondering if hand-controlled 3D matters for your work? Consider these scenarios where shape-creator-tutorial becomes transformative:

Rapid 3D Prototyping for Designers Imagine walking into a client presentation and sculpting their product vision in mid-air, in real-time, projected on a conference room screen. No Maya license. No import/export pipeline. Just gesture, create, iterate. The spatial intuition that designers develop through years of physical model-making finally translates directly to digital workflow.

Accessible Creative Tools for Motor Impairments Traditional 3D software assumes fine motor control for mouse manipulation and keyboard shortcuts. Hand tracking—especially when extended with eye-gaze or voice integration—opens 3D creation to users with tremors, limited grip strength, or limb differences. The open-source nature means assistive technology developers can adapt the gesture vocabulary to individual capabilities.

Immersive Educational Demonstrations Physics teachers manipulating molecular models. Architecture professors walking students through structural stress visualizations. Surgeons rehearsing procedures with patient-specific anatomy. Each scenario benefits from embodied interaction that kinesthetic learners crave. The browser-based deployment means zero IT friction in school environments.

Interactive Installation Art Gallery visitors become co-creators as their gestures generate persistent 3D sculptures. The project's MIT license and clean architecture make it ideal for artistic extension—add persistence, multi-user synchronization via WebRTC, or projection mapping integration. Alan's own Particular Drift and Liquid Logo projects demonstrate this aesthetic potential.

Remote Collaboration and VR Precursors Before your team commits to full VR development cycles, prototype spatial interaction concepts in shape-creator-tutorial. Validate gesture vocabularies, test user comfort zones, and gather behavioral data—all through a URL shared in Slack, no headset logistics required.

Step-by-Step Installation & Setup Guide

Getting started borders on trivial, which is itself a feature. Here's the complete path from zero to gesturing:

Prerequisites

• Modern browser: Chrome 90+, Firefox 88+, Safari 15+, or Edge 90+
• WebGL support: Verify at webglreport.com
• Camera access: Built-in webcam or USB camera, minimum 720p recommended
• Local server capability: Any static file server works

Installation Commands

# Clone the repository from GitHub
git clone https://github.com/collidingScopes/shape-creator-tutorial

# Enter the project directory
cd shape-creator-tutorial

# Launch a local server. Python↗ Bright Coding Blog's built-in module is simplest:
python -m http.server

# Alternative: Node.js http-server (install globally if needed)
# npm install -g http-server
# http-server -p 8000

# Alternative: PHP↗ Bright Coding Blog built-in server
# php -S localhost:8000

Browser Configuration

Navigate to http://localhost:8000. On first load, your browser will request camera permissions—allow access for the domain. If permission was previously denied, reset via:

• Chrome: Click lock icon in address bar → Site settings → Camera → Allow
• Firefox: Click lock icon → Permissions → Access Your Camera → Allow
• Safari: Preferences → Websites → Camera → Allow for localhost

Performance Optimization

For smooth 30fps+ hand tracking:

1. Close competing camera applications (Zoom, Teams, OBS) that may lock the video device
2. Ensure adequate lighting—MediaPipe's detection degrades significantly in shadow
3. Position hands 0.5-2 meters from camera for optimal landmark stability
4. Disable browser extensions that inject content scripts into all pages

Troubleshooting Common Issues

REAL Code Examples From the Repository

The repository's elegance lies in its architectural clarity. Let's examine the actual implementation patterns that make this magic possible.

Example 1: Development Server Setup

The README's setup instructions are intentionally minimal, reflecting the project's zero-build-step philosophy:

# Clone this repository
git clone https://github.com/collidingScopes/shape-creator-tutorial

# Navigate to the project directory
cd shape-creator-tutorial
# Serve with your preferred method (example using Python)
python -m http.server

Why this matters: No npm install. No webpack.config.js. No dependency hell. The entire application runs as static files, meaning deployment to GitHub Pages, Netlify, or any CDN is instantaneous. This architectural choice prioritizes accessibility over toolchain sophistication—a deliberate inversion of modern JavaScript's complexity obsession. The Python server serves files with correct MIME types; browsers refuse to execute JavaScript modules from file:// origins due to CORS policies, making any local server mandatory.

Example 2: Core Technology Stack Declaration

The README documents the technology stack with unusual precision:

- **Three.js** for 3D rendering
- **MediaPipe** for hand tracking and gesture recognition
- **HTML5 Canvas** for visual feedback
- **JavaScript** for real-time interaction

Deep dive: This isn't mere documentation—it's a manifesto for browser-native capabilities. Three.js abstracts WebGL's verbose API without sacrificing power; its scene graph, geometry primitives, and material system enable the procedural shape generation that responds to gestures. MediaPipe's @mediapipe/hands JavaScript package runs the same TensorFlow Lite model that powers Google's production vision products, but compiled to WebAssembly for near-native speed. The HTML5 Canvas overlay provides immediate-mode 2D drawing for debug visualization without polluting the WebGL context state. Pure JavaScript—no TypeScript compilation, no framework reactivity systems—means the code executes exactly as written, with zero abstraction penalty.

Example 3: Project Metadata and Ecosystem Linkage

The README's related projects section reveals Alan's systematic exploration of computer vision aesthetics:

You might also like some of my other open source projects:

- [Threejs hand tracking tutorial](https://collidingScopes.github.io/threejs-handtracking-101) - Basic hand tracking setup with threejs and MediaPipe computer vision
- [Particular Drift](https://collidingScopes.github.io/particular-drift) - Turn photos into flowing particle animations
- [Liquid Logo](https://collidingScopes.github.io/liquid-logo) - Transform logos and icons into liquid metal animations
- [Video-to-ASCII](https://collidingScopes.github.io/ascii) - Convert videos into ASCII pixel art

Implementation insight: This ecosystem architecture is brilliant for learning. The threejs-handtracking-101 project isolates the hand tracking setup—camera initialization, MediaPipe configuration, landmark extraction—without the 3D interaction complexity. Developers can master fundamentals before tackling shape-creator-tutorial's integrated system. The progression from 101 → shape-creator → Particular Drift represents a deliberate curriculum: input handling → creative application → generative art. Each repository's GitHub Pages deployment means learners can inspect running code, then git clone to modify.

Example 4: Live Demo and Video Resources

The README's resource linking demonstrates effective project presentation:

[Video](https://youtu.be/oE3a0ghsrBk?si=UCcnjHjpWj21bBA0) | [Live Demo](https://collidingscopes.github.io/shape-creator-tutorial/) | [More Code & Tutorials](https://www.funwithcomputervision.com/)

Technical note: The live demo deploys via GitHub Pages from the repository's gh-pages branch or root directory. This serverless hosting handles global CDN distribution, HTTPS termination, and automatic updates on push—zero infrastructure cost for unlimited scale. The YouTube video provides asynchronous learning for developers who prefer demonstration over documentation, while the consolidated tutorial site funwithcomputervision.com captures email subscribers for monetized lifetime access. This funnel—open-source discovery → free demo engagement → premium educational content—is a sustainable model for independent creator-educators.

Advanced Usage & Best Practices

Once you've mastered the basics, these strategies extract maximum value from shape-creator-tutorial:

Gesture Vocabulary Extension The base project demonstrates core interactions, but MediaPipe's 21 landmarks enable sophisticated gesture recognition. Implement pinch detection by measuring Euclidean distance between thumb tip (landmark 4) and index tip (landmark 8). Add swipe gestures by tracking palm centroid velocity across frames. Create custom gesture classifiers using the landmark angles rather than raw positions—they're rotation-invariant.

Performance Profiling Use Chrome DevTools' Performance panel to identify bottlenecks. MediaPipe inference typically consumes 8-15ms per frame; Three.js rendering should target 16ms (60fps) or 33ms (30fps). If requestAnimationFrame callbacks exceed your target, consider reducing MediaPipe's modelComplexity from 1 to 0, or capping the camera resolution to 640×480.

Multi-Hand Orchestration The base project likely uses single-hand detection for simplicity. Enable maxNumHands: 2 in MediaPipe configuration for bimanual interaction: one hand positions, the other sculpts. This mirrors professional 3D software's multi-input workflows but with natural gestures.

Persistence and Serialization Extend the project by serializing created shapes to JSON (Three.js object's toJSON() method) and storing in localStorage or a backend. Users build persistent galleries; you build a community platform.

Accessibility Augmentation Add Web Speech API integration for voice commands complementing gestures. "Cube," "sphere," "delete"—spoken instructions for users who can't form precise hand configurations. The multimodal approach serves broader user populations.

Comparison with Alternatives

The verdict: For developers prioritizing immediate productivity, educational clarity, and deployment simplicity, shape-creator-tutorial dominates. TensorFlow.js Handpose offers equivalent accuracy but demands significant integration work. ManoMotion's commercial model introduces licensing uncertainty. Leap Motion's hardware dependency kills the "works everywhere" advantage that makes web-based hand tracking revolutionary.

FAQ: What Developers Actually Ask

Q: Does this work on mobile browsers? A: Yes, with caveats. iOS Safari and Chrome Android both support MediaPipe and WebGL, but camera access in WebView contexts (Instagram in-app browser, etc.) may be restricted. For best results, use standalone browser apps with explicit camera permissions.

Q: How accurate is the hand tracking in poor lighting? A: MediaPipe's detection degrades gracefully but noticeably below approximately 50 lux. The model was trained on diverse lighting conditions, but extreme backlighting or point-source shadows cause landmark jitter. Add soft, diffuse lighting for production deployments.

Q: Can I use this for commercial projects? A: Absolutely. The MIT license permits unrestricted commercial use, modification, and distribution. No attribution required, though linking to Alan's work is ethically encouraged.

Q: What's the minimum viable hardware? A: Any laptop from 2018 onward with integrated graphics handles this comfortably. The MediaPipe WASM backend uses CPU inference by default; GPU acceleration via WebGL is available but not mandatory. A 720p webcam suffices; 1080p improves tracking stability at arm's length.

Q: How do I extend this to track body pose, not just hands? A: MediaPipe's Pose solution uses an identical API pattern. Replace @mediapipe/hands with @mediapipe/pose, adjust landmark indices (33 body landmarks versus 21 hand landmarks), and map pose keypoints to 3D scene interactions. Alan's tutorial ecosystem likely expands in this direction.

Q: Is the live demo collecting my biometric data? A: No. All processing occurs client-side in your browser. Video frames never leave your device; no server receives hand landmark data. Verify this by inspecting Network tab during usage—zero outbound video or landmark traffic.

Q: How does this compare to WebXR hand tracking? A: WebXR's hand tracking requires VR/AR hardware with dedicated hand sensors. shape-creator-tutorial achieves comparable interaction vocabulary using ubiquitous webcams, dramatically lowering access barriers. For immersive VR, WebXR is superior; for broad accessibility, this project wins decisively.

Conclusion: Your Hands Are the Interface We've Been Waiting For

We've spent decades contorting human creativity through mechanical intermediaries—mice, keyboards, stylus tablets—accepting the translation loss as inevitable. shape-creator-tutorial exposes this compromise as exactly that: a compromise, not a necessity.

The repository isn't merely functional; it's prophetic. It demonstrates that the technical barriers to natural spatial interaction have collapsed. MediaPipe's computer vision, Three.js's rendering power, and the web's distribution model converge into something that would have seemed magical five years ago and now feels obvious-in-retrospect.

Alan's work matters because he doesn't hoard this capability behind proprietary SDKs or expensive hardware ecosystems. He publishes complete implementations, narrates video tutorials, and maintains living demos anyone can experience in seconds. This is open source at its most transformative—not just code shared, but possibilities democratized.

My recommendation? Don't just read this article. Clone the repository. Run the demo. Wave your hand and watch geometry materialize. Then imagine what you could build when that gesture vocabulary becomes yours to extend, modify, and deploy.

The future of 3D interaction isn't coming. It's already here, running in your browser, waiting for your hands.

→ Star shape-creator-tutorial on GitHub

→ Explore more computer vision projects

→ Watch the video tutorial