AR/VRMedical VisualizationVR Simulation

Medical Surgery VRBrain & Anatomy Trainer

An immersive VR medical simulation built in Unity 6 featuring real-time anatomical visualization, holographic X-ray rendering, surgical scalpel slicing, and a dedicated brain deep-dive mode where users orbit, zoom, and inspect a 3D brain with on-demand hologram toggling.

Medical Surgery VR - Brain & Anatomy Trainer: device mockup
Category
AR/VR
Built with
Unity 6, Universal Render Pipeline (URP), XR Interaction Toolkit 3.3 +10 more

Medical Surgery VR is a cutting-edge virtual reality training and visualization application built in Unity 6 with the Universal Render Pipeline (URP), designed to bridge the gap between medical education and immersive technology. The experience is structured around two purpose-built scenes: a fully dressed hospital operating room for hands-on surgical simulation, and a dedicated Brain Deep-Dive environment for in-depth neurological inspection.

In the main surgical scene, users interact with anatomical models placed on an operating table inside a richly detailed hospital room. A fully custom Bounding Box Manipulation system—implemented from scratch with 8 corner scaling handles and 12 edge rotation handles rendered via a real-time LineRenderer wireframe—lets physicians grab, scale, and rotate anatomy models in 3D using XR Interaction Toolkit controllers. A Hologram / X-Ray toggle, backed by a custom URP shader and UI Toggle integration, can instantly convert any mesh into a semi-transparent hologram, revealing internal structures on demand.

The surgical component uses EzySlice for real-time mesh cutting, driven by a physics-accurate ScalpelSlicer that detects trigger contact with sliceable anatomy and splits the mesh along the scalpel’s cut plane. Each resulting hull receives a procedurally generated cross-section material, a convex MeshCollider, and a configured Rigidbody; the lower piece is kept kinematic while the upper piece is released to gravity after a brief delay to avoid interpenetration. A SliceManager supports fully resetting the specimen back to its original state via a simple UI action.

The Brain Deep-Dive scene provides a dedicated neurological inspection mode. Users orbit freely around a high-fidelity 3D brain using the left joystick for horizontal and vertical orbit around the brain’s pivot, while the right joystick controls zoom distance. On the right controller, button A activates holographic scan view—swapping all brain materials to a glowing hologram shader—while button B restores the original anatomical shading, enabling rapid switching between modes for teaching and demonstration.

Built on OpenXR for cross-platform compatibility and AR Foundation for future AR expansion, Medical Surgery VR showcases the convergence of medical science and real-time 3D technology, delivering a training tool that is both technically rigorous and deeply immersive.

Demo

See it in action

Highlights

What makes it work

  1. Dual-scene architecture with a hospital OR suite and a dedicated Brain Deep-Dive inspection mode

  2. Real-time surgical scalpel slicing powered by EzySlice with procedural cross-section materials

  3. On-demand holographic / X-Ray visualization via custom URP shader material swapping

  4. Custom Bounding Box Manipulator featuring 8 corner (scale) and 12 edge (rotate) VR handles

  5. Orbital brain inspection with 360° free orbit, vertical tilt, and joystick-driven zoom

  6. Physics-accurate slice separation with kinematic locking and gravity-based top hull release

  7. VR controller-driven hologram toggle (A = activate, B = restore) with persistent state handling

  8. Wireframe bounding box rendered via LineRenderer for clear spatial context during manipulation

  9. Hospital room environment with operating table, surgical tools, and detailed anatomy models

  10. OpenXR and AR Foundation integration preparing the experience for future AR workflows

The challenge

The hard parts of building Medical Surgery VR

5 problems we had to solve, from physics, interaction and 3D math.

  1. 01 / 05Physics

    Slicing that feels surgical

    The most complex technical challenge was building a stable real-time mesh-slicing pipeline that feels surgical and precise. EzySlice outputs raw hull geometry with no physics setup, so a custom post-slice pipeline was required: assigning cross-section materials, generating convex MeshColliders, configuring Rigidbodies with continuous collision detection, and running a coroutine-based delay so the two halves separate cleanly without interpenetrating the table surface.

  2. 02 / 05Physics

    Halves that separate cleanly

    Keeping the bottom half perfectly fixed at its original world pose while the top half peels away under gravity—without either clipping through the table or each other—demanded careful offset tuning and multi-frame monitoring.

  3. 03 / 05Interaction

    A 3D gizmo built for VR

    Creating the Bounding Box Manipulator inside VR was equally demanding. Without screen-space gizmos, scaling had to be derived from real-world distances between the interactor and the object’s center, while rotation was computed from the projected angle of controller positions onto local axis planes (XZ for Y-rotation, YZ for X-rotation, XY for Z-rotation) using Atan2 in local space.

  4. 04 / 053D math

    Rotation without drift

    Preventing rotational drift required basing each frame’s rotation on the original quaternion rather than stacking incremental deltas.

  5. 05 / 053D math

    A camera that glides over the poles

    Designing the orbital Brain Deep-Dive camera required avoiding gimbal lock and pole snapping. Instead of relying on LookAt, the camera orbits around the brain’s right axis for vertical motion and a world-up axis for horizontal rotation, allowing smooth traversal over the poles without sudden flips or loss of control.

  6. 5 of 5 solved

    Got a problem like these? We’d like to hear about it.

Screens

Inside Medical Surgery VR

1 / 5

Built with

  • Unity 6
  • Universal Render Pipeline (URP)
  • XR Interaction Toolkit 3.3
  • OpenXR
  • AR Foundation 6.3
  • EzySlice (Real-time Mesh Cutting)
  • C# Scripting
  • Custom URP Hologram Shader
  • New Input System
  • Real-time Physics (Rigidbody)
  • TextMeshPro UI
  • LineRenderer Wireframe
  • XR Controller Input API

Have a project like Medical Surgery VR?

Tell us what you want to build. You’ll get a free discovery call and a clear, written proposal.

Start a similar project
Next projectMinecraft Mode Welding Workshop Education