Skip Section Navigation

MARA & Digital Twin: Real-time Indoor Navigation using AR Foundation Image Targets for user location

Current Students: Andrew Summitt

Status: Current

We have implemented a MARA for indoor navigation that uses Augmented Reality for localization and navigation. The location of the user is determined by their device camera and translated to a position within a Digital Twin of the Discovery Park Building at UNT. For localization we have used AI generated Navigation Meshes and AR Foundation Image Tracking technology. Navigation can be started from any location within the supported area, which covers both floors of the building. The app functionality can be extended to support much larger areas by adding more Area Targets. We have implemented the following:

- Digital Twin of Discovery Park
- Mini Map with Position and Rotation Indicator
- AI Navigation Meshes
- Multiple Destination Targets
- First and Second Floor Navigation
- AR Foundation Image Detection (Room Numbers)

MARA is designed specifically for acquiring spatial analysis, situational awareness, and visual communication. Through the visualization of integrated geographic information systems and real-time data analysis, our mobile AR application provides the current location of the person, the number of exits, and user-specific personalized evacuation routes. The Mobile AR application provides information to support effective decision-making during emergencies for both building occupants and emergency responders.

 
Tablet Demo (Version 4) Phone Demo  
Floor Floor Floor

A 3D model of the building overlaid on top a map to ensure accurate navigation

The highlighted area indicates the walkable path. The navigation references this surface to dynamically guide the user to the desired destination

 Several location targets have been added to the map to allow users to navigate to key locations within the building
Floor Floor Floor

Users select a desired destination, and the shortest possible path is calculated for them. As they approach the target location, the line updates to provide a real time navigation experience.

GUI: The distance between the user’s current and end location is updated in real time

The GUI provides an option for users to recenter their location on the map if they lose their way
Information Gathering Module Information Gathering Module Time Series Analysis Module
     
AR AR AR
     

Videos (Video 1, Video 2 )

Publications

  1. Sharma, S., Pesaladinne R., "Spatial Analysis and Visual Communication of Emergency Information through Augmented Reality", Proceedings of the IS&T International Symposium on Electronic Imaging (EI 2024) in the Engineering Reality of Virtual Reality Conference, DOI: 10.2352/J.ImagingSci.Technol.2023.67.6.060401, January 21-25, 2024.
  2. Chellatore, M.P., Sharma, S, "Mobile application for identifying anomalous behavior and conducting time series analysis using heterogeneous data", Springer Nature Switzerland AG, in J. Wei and G. Margetis (Eds.): HCII 2024, https://doi.org/10.1007/978-3-031-60458-4_12, Lecture Notes in Computer Science (LNCS) 14737, pp. 1–16, 2024.
  3. Sharma, S., Pesaladinne R., "Spatial Analysis and Visual Communication of Emergency Information through Augmented Reality"  in Journal of Imaging Science and Technology,  https://doi.org/10.2352/J.ImagingSci.Technol.2023.67.6.060401, pp 1 - 9,  2023.
  4. Pesaladinne, R., Chellatore, M.P.,Dronavalli,S., Sharma, S., "Situational awareness and feature extraction for indoor building navigation using mixed reality", Proceedings of the IEEE International Conference on Computational Science and Computational Intelligence, (IEEE-CSCI), Research Track on Big Data and Data Science (CSCI-RTBD), Las Vegas, USA, December 13-15, 2023.
  5. Sharma, S, "Mobile Augmented Reality System for Emergency Response", Proceedings of the 21st IEEE/ACIS International Conference on Software Engineering, Management and Applications (SERA 2023), Orlando, USA, May 23-25, 2023.
  6. Sharma, S., Engel, D., "Mobile augmented reality system for object detection, alert, and safety", Proceedings of the IS&T International Symposium on Electronic Imaging (EI 2023) in the Engineering Reality of Virtual Reality Conference, January 15-19, 2023.
  7. Mannuru, N. R., Kanumuru, M., and Sharma, S., "Mobile AR application for navigation and emergency response", Proceedings of the IEEE International Conference on Computational Science and Computational Intelligence , (IEEE-CSCI-RTMC), Las Vegas, USA, December 14-16, 2022.
  8. Sharma, S, Bodempudi, S.T., Scribner, D., Grynovicki, J., Grazaitis, P., "Emergency response using HoloLens for building evacuation", Proceeding of 21st International Conference on Human-Computer Interaction, DOI: https://doi.org/10.1007/978-3-030-21607-8_23 , Online ISBN 978-3-030-21607-8, Orlando, Florida, USA, 26-31 July, HCII 2019, Lecture Notes in Computer Science, vol 11574, pp. 299–311, 2019.
  9. Stigall, J., Bodempudi, S.T., Sharma, S, Scribner, D., Grynovicki, J., Grazaitis, P., "Use of Microsoft HoloLens in indoor evacuation", International Journal of Computers and their Applications, IJCA, Vol. 26, No. 1, March 2019.
  10. Stigall, J., Sharma, S, "Evaluation of Mobile Augmented Reality Application for Building Evacuation", Proceedings of 28th International Conference on Software Engineering and Data Engineering (SEDE 2019) in San Diego, CA, USA on September 30 - October 2, 2019.
  11. Stigall, J., Bodempudi, S.T., Sharma, S, Scribner, D., Grynovicki, J., Grazaitis, P., "Building Evacuation using Microsoft HoloLens", Proceedings of 27th International Conference on Software Engineering and Data Engineering (SEDE 2018), New Orleans, Louisiana, USA, pp. 20-25, October 8-10, 2018. [Best paper award finalist]
  12. James Stigall and Dr. Sharad Sharma, "Mobile augmented reality application for building evacuation" , Poster Presentation, 11th Annual Grant Expo and Research day, Office of Research and Sponsored Programs (ORSP), Bowie State University, April 10, 2018. [Won first Prize]
  13. Stigall, J., Sharma, S, "Mobile Augmented Reality Application for Building Evacuation Using Intelligent Signs", proceedings of ISCA 26th International Conference on Software Engineering and Data Engineering (SEDE-2017), pp. 19-24, San Diego,CA, USA, October 2-4, 2017.
  14. Sharma, S, Jerripothula, S., "An indoor augmented reality mobile application for simulation of building evacuation", proceedings of SPIE conference on the Engineering Reality of Virtual Reality 2015, paper 9392-7, San Francisco, CA, USA, 9 - 10 February 2015.

DVXR LABORATORY (C) 2022-2024, ALL RIGHTS RESERVED