Skip Section Navigation

MARA & Digital Twin: Real-time Indoor Navigation using AR Foundation Image Targets for user location

Current Students: Andrew Summitt

Status: Current

We have implemented a MARA for indoor navigation that uses Augmented Reality for localization and navigation. The location of the user is determined by their device camera and translated to a position within a Digital Twin of the Discovery Park Building at UNT. For localization we have used AI generated Navigation Meshes and AR Foundation Image Tracking technology. Navigation can be started from any location within the supported area, which covers both floors of the building. The app functionality can be extended to support much larger areas by adding more Area Targets. We have implemented the following:

- Digital Twin of Discovery Park
- Mini Map with Position and Rotation Indicator
- AI Navigation Meshes
- Multiple Destination Targets
- First and Second Floor Navigation
- AR Foundation Image Detection (Room Numbers)

MARA is designed specifically for acquiring spatial analysis, situational awareness, and visual communication. Through the visualization of integrated geographic information systems and real-time data analysis, our mobile AR application provides the current location of the person, the number of exits, and user-specific personalized evacuation routes. The Mobile AR application provides information to support effective decision-making during emergencies for both building occupants and emergency responders.

Augmented Reality Application for Indoor Navigation using HoloLens, Smart Phones, and Tablets

   
HoloLens Version (Video) HoloLens Version  
HoloLens HoloLens (Shelter Area) HoloLens (nearest Exit)
Tablet Demo (Version 4) Phone Demo HoloLens (Shelter Area)
Floor Floor Floor

A 3D model of the building overlaid on top a map to ensure accurate navigation

The highlighted area indicates the walkable path. The navigation references this surface to dynamically guide the user to the desired destination

 Several location targets have been added to the map to allow users to navigate to key locations within the building
Floor Floor Floor

Users select a desired destination, and the shortest possible path is calculated for them. As they approach the target location, the line updates to provide a real time navigation experience.

GUI: The distance between the user’s current and end location is updated in real time

The GUI provides an option for users to recenter their location on the map if they lose their way
Information Gathering Module Information Gathering Module Time Series Analysis Module
     
AR AR AR
     
     
H
     

Videos (Video 1, Video 2 )

Publications

  1. Sharma, S, Summitt, A., "A Mobile Augmented Reality Application for Indoor Emergency Evacuation and Navigation", Proceedings of the IS&T International Symposium on Electronic Imaging (EI 2025) in the Engineering Reality of Virtual Reality Conference, February 2-6, 2025.
  2. Summitt, A., Bhatt, B.J., Sharma, S.,"Emergency Assistive Mobile Application with Digital Twin for Real-Time 3D Navigation in Indoor Environments", Proceedings of the 23rd IEEE/ACIS International Conference on Software Engineering, Management and Applications (SERA 2025), Las Vegas, USA, May 29-31, 2025. (Submitted)
  3. Bhatt, B.J., Summitt, A., Sharma, S.,"Immersive Virtual Reality Data Visualization for Urban Parking Management", Proceedings of the 23rd IEEE/ACIS International Conference on Software Engineering, Management and Applications (SERA 2025), Las Vegas, USA, May 29-31, 2025. (Accepted)
  4. Sharma, S., Pesaladinne R., "Spatial Analysis and Visual Communication of Emergency Information through Augmented Reality", Proceedings of the IS&T International Symposium on Electronic Imaging (EI 2024) in the Engineering Reality of Virtual Reality Conference, DOI: 10.2352/J.ImagingSci.Technol.2023.67.6.060401, January 21-25, 2024.
  5. Chellatore, M.P., Sharma, S, "Mobile application for identifying anomalous behavior and conducting time series analysis using heterogeneous data", Springer Nature Switzerland AG, in J. Wei and G. Margetis (Eds.): HCII 2024, https://doi.org/10.1007/978-3-031-60458-4_12, Lecture Notes in Computer Science (LNCS) 14737, pp. 1–16, 2024.
  6. Sharma, S., Pesaladinne R., "Spatial Analysis and Visual Communication of Emergency Information through Augmented Reality"  in Journal of Imaging Science and Technology,  https://doi.org/10.2352/J.ImagingSci.Technol.2023.67.6.060401, pp 1 - 9,  2023.
  7. Pesaladinne, R., Chellatore, M.P.,Dronavalli,S., Sharma, S., "Situational awareness and feature extraction for indoor building navigation using mixed reality", Proceedings of the IEEE International Conference on Computational Science and Computational Intelligence, (IEEE-CSCI), Research Track on Big Data and Data Science (CSCI-RTBD), Las Vegas, USA, December 13-15, 2023.
  8. Sharma, S, "Mobile Augmented Reality System for Emergency Response", Proceedings of the 21st IEEE/ACIS International Conference on Software Engineering, Management and Applications (SERA 2023), Orlando, USA, May 23-25, 2023.
  9. Sharma, S., Engel, D., "Mobile augmented reality system for object detection, alert, and safety", Proceedings of the IS&T International Symposium on Electronic Imaging (EI 2023) in the Engineering Reality of Virtual Reality Conference, January 15-19, 2023.
  10. Mannuru, N. R., Kanumuru, M., and Sharma, S., "Mobile AR application for navigation and emergency response", Proceedings of the IEEE International Conference on Computational Science and Computational Intelligence , (IEEE-CSCI-RTMC), Las Vegas, USA, December 14-16, 2022.

DVXR LABORATORY (C) 2022-2024, ALL RIGHTS RESERVED