19036 – Augmented Reality System for Surgical Planning, Intervention, Guidance, and Education

Augmented reality system with data streaming and synchronization than can be deployed across multiple device platforms and locations for surgical planning, intervention, guidance and education


Traditional surgical navigation systems, present virtual models on a separate display away from the surgical field. Our AR system provides enhanced guidance directly on surgical field or fused with physical objects. The AR system also provides real-time metrics on pre-operative plans for procedures such as guided osteotomies and out of plane ultrasound guided ablations, not currently available in commercial systems. Multi-client connectivity also enables multiple clinical users to collaborate, either in the same physical space or remotely.

The system comprises of various AR applications with data streaming and synchronization that can be deployed across multiple devices (mobile devices, computers, AR headsets or projector systems), locally or remotely. Applications may be deployed native or through web standards conforming browsers supporting technologies such as WebGL and WebXR. The system, library, and data are agnostic to the engine and rendering architecture of the applications across client devices due to conformance to device application programming interfaces (APIs), platform software development kits (SDKs), data, and technology standards. For example, one client device may deploy an application on a tablet via Unity, a popular game engine, whereas another iPhone device may run an application built via Apple’s native toolchain and ARKit.

Potential Applications

  • Planning: Multisite collaborative walkthroughs, tele mentoring, difficult case practice
  • Education: Anatomic visualization, skills translation and evaluation, scientific output augmentation
  • Intervention/Guidance: Geometric resection of tumors, needle procedure planning, critical structure avoidance, additional data overlays (vital signs or equipment data)

Contact Information

Name: Noah Schwartz

Email: Noah.Schwartz@uhnresearch.ca

Phone: 0000000000