Politecnico di Milano, in collaboration with NGRSys, is head of the REFINE (ImmeRsive surgEry through artiFicial intelligence, mixed reality, Integration of multi-modal data and augmeNted intEraction) project, financed by Regione Lombardia’s European Regional Development Fund 2014-2020 with the contribution of the European Union.

The project, coordinated by Prof. Pietro Cerveri (Department of Electronics, Information and Bioengineering), addresses virtual and mixed reality in planning and surgery with a view to making accessible to the surgeon digital patient personal information, past clinical history, laboratory reports and pre-operative and intra-operative imaging into a unified perception/interaction framework, allowing for the management of the main instruments used in the room. Exploiting recent head mounted displays, enabled voice commands, eye motion tracking and hand gesture recognition, unprecedented interaction with the virtual scene is enabled.

REFINE proposes an innovative solution which extends the traditional paradigm of mixed reality by integrating the fundamental facilities based 4 main technical pillars.

1. data sharing by 5G-based communication infrastructure for low latency information transmission.
2. interoperability of data/stream formats based on vendor neutral principle for transparent information exchange.
3. tools based on artificial intelligence techniques for advanced multi-modal and heterogeneous data processing.
4. application-dependent hybrid patient model for automated view customization.

In addition to the Department of Electronics, Information and Bioengineering, REFINE also involves the Department of Chemistry, Materials and Chemical Engineering "Giulio Natta" and the Department of Management Engineering.