AR/VRMedical VisualizationVR Simulation
Medical Surgery VRBrain & Anatomy Trainer
An immersive VR medical simulation built in Unity 6 featuring real-time anatomical visualization, holographic X-ray rendering, surgical scalpel slicing, and a dedicated brain deep-dive mode where users orbit, zoom, and inspect a 3D brain with on-demand hologram toggling.

- Category
- AR/VR
- Built with
- Unity 6, Universal Render Pipeline (URP), XR Interaction Toolkit 3.3 +10 more
Medical Surgery VR is a cutting-edge virtual reality training and visualization application built in Unity 6 with the Universal Render Pipeline (URP), designed to bridge the gap between medical education and immersive technology. The experience is structured around two purpose-built scenes: a fully dressed hospital operating room for hands-on surgical simulation, and a dedicated Brain Deep-Dive environment for in-depth neurological inspection.
In the main surgical scene, users interact with anatomical models placed on an operating table inside a richly detailed hospital room. A fully custom Bounding Box Manipulation system—implemented from scratch with 8 corner scaling handles and 12 edge rotation handles rendered via a real-time LineRenderer wireframe—lets physicians grab, scale, and rotate anatomy models in 3D using XR Interaction Toolkit controllers. A Hologram / X-Ray toggle, backed by a custom URP shader and UI Toggle integration, can instantly convert any mesh into a semi-transparent hologram, revealing internal structures on demand.
The surgical component uses EzySlice for real-time mesh cutting, driven by a physics-accurate ScalpelSlicer that detects trigger contact with sliceable anatomy and splits the mesh along the scalpel’s cut plane. Each resulting hull receives a procedurally generated cross-section material, a convex MeshCollider, and a configured Rigidbody; the lower piece is kept kinematic while the upper piece is released to gravity after a brief delay to avoid interpenetration. A SliceManager supports fully resetting the specimen back to its original state via a simple UI action.
The Brain Deep-Dive scene provides a dedicated neurological inspection mode. Users orbit freely around a high-fidelity 3D brain using the left joystick for horizontal and vertical orbit around the brain’s pivot, while the right joystick controls zoom distance. On the right controller, button A activates holographic scan view—swapping all brain materials to a glowing hologram shader—while button B restores the original anatomical shading, enabling rapid switching between modes for teaching and demonstration.
Built on OpenXR for cross-platform compatibility and AR Foundation for future AR expansion, Medical Surgery VR showcases the convergence of medical science and real-time 3D technology, delivering a training tool that is both technically rigorous and deeply immersive.
See it in action
What makes it work
Dual-scene architecture with a hospital OR suite and a dedicated Brain Deep-Dive inspection mode
Real-time surgical scalpel slicing powered by EzySlice with procedural cross-section materials
On-demand holographic / X-Ray visualization via custom URP shader material swapping
Custom Bounding Box Manipulator featuring 8 corner (scale) and 12 edge (rotate) VR handles
Orbital brain inspection with 360° free orbit, vertical tilt, and joystick-driven zoom
Physics-accurate slice separation with kinematic locking and gravity-based top hull release
VR controller-driven hologram toggle (A = activate, B = restore) with persistent state handling
Wireframe bounding box rendered via LineRenderer for clear spatial context during manipulation
Hospital room environment with operating table, surgical tools, and detailed anatomy models
OpenXR and AR Foundation integration preparing the experience for future AR workflows
The challenge
The hard parts of building Medical Surgery VR
5 problems we had to solve, from physics, interaction and 3D math.
- 01 / 05Physics
Slicing that feels surgical
The most complex technical challenge was building a stable real-time mesh-slicing pipeline that feels surgical and precise. EzySlice outputs raw hull geometry with no physics setup, so a custom post-slice pipeline was required: assigning cross-section materials, generating convex MeshColliders, configuring Rigidbodies with continuous collision detection, and running a coroutine-based delay so the two halves separate cleanly without interpenetrating the table surface.
- 02 / 05Physics
Halves that separate cleanly
Keeping the bottom half perfectly fixed at its original world pose while the top half peels away under gravity—without either clipping through the table or each other—demanded careful offset tuning and multi-frame monitoring.
- 03 / 05Interaction
A 3D gizmo built for VR
Creating the Bounding Box Manipulator inside VR was equally demanding. Without screen-space gizmos, scaling had to be derived from real-world distances between the interactor and the object’s center, while rotation was computed from the projected angle of controller positions onto local axis planes (XZ for Y-rotation, YZ for X-rotation, XY for Z-rotation) using Atan2 in local space.
- 04 / 053D math
Rotation without drift
Preventing rotational drift required basing each frame’s rotation on the original quaternion rather than stacking incremental deltas.
- 05 / 053D math
A camera that glides over the poles
Designing the orbital Brain Deep-Dive camera required avoiding gimbal lock and pole snapping. Instead of relying on LookAt, the camera orbits around the brain’s right axis for vertical motion and a world-up axis for horizontal rotation, allowing smooth traversal over the poles without sudden flips or loss of control.
5 of 5 solved
Got a problem like these? We’d like to hear about it.
Inside Medical Surgery VR
Built with
- Unity 6
- Universal Render Pipeline (URP)
- XR Interaction Toolkit 3.3
- OpenXR
- AR Foundation 6.3
- EzySlice (Real-time Mesh Cutting)
- C# Scripting
- Custom URP Hologram Shader
- New Input System
- Real-time Physics (Rigidbody)
- TextMeshPro UI
- LineRenderer Wireframe
- XR Controller Input API
Have a project like Medical Surgery VR?
Tell us what you want to build. You’ll get a free discovery call and a clear, written proposal.
More work like this
All projects


