Overview
In the last decades, interconnected devices of several types started to be used in the most diverse contexts, creating intelligent environments, and improving the population’s life. New experiences that merge real-world reality with objects and virtual realities have become available through many applications and devices, and with application in several areas, such as entertainment, industry and manufacturing, education and training, medicine and health care, and militarism, among others. On the other hand, the growing use of devices that monitor and collect data about their users and the surrounding environment, the interconnection and sharing of data by such devices, and the rapid dissemination of information through the Internet increased the concerns on maintaining privacy and confidentiality of individuals and companies. Then, several data protection and privacy legislations were created, such as the European General Data Protection Regulation. But identifying risks to privacy and data confidentiality in images captured by devices that freely use cameras to scan the environment around them – to identify objects and people – is a challenging and open issue.
In this project, we intend to provide automatic tools for guaranteeing the privacy and confidentiality of individuals and companies while maintaining seamless, real-time, and persistent physical-digital Augmented Reality (AR) experiences. We will identify the main classes of privacy and confidentiality risks in the AR context and apply machine learning to automatically identify occurrences of such issues in raw data captured by AR applications. For each class, we will research the adequate countermeasure. The identification methods and countermeasures must consider the specific requirements (e.g., response time) and resource limitations of AR applications. To validate our proposals, we will build two use AR applications. The first scenario is on a location-based AR game (LBARG), while the second evaluation scenario is on using AR to aid training and manufacturing.
Publications:
- Costa, RLC., Marto, A., Santos, L., Gonçalves, A., & Rabadão, C.. (2025). SafeAR: Privacy-maintaince in augmented reality applications.In 11th European Conference On Service-Oriented And Cloud Computing (ESOCC 2025). Lecture Notes in Computer Science (LNCS). volume 15547, pp 146–151. DOI: https://doi.org/10.1007/978-3-031-84617-5_12
Link to the paper - Alves, M., Ribeiro, T., Marto, A., Gonçalves, A., Rabadão, C., Santos, L., & de C. Costa, R. L. (2025). Evaluating Service-based Privacy-Protection for Augmented Reality Applications. In 16th International Conference on Management of Digital EcoSystems (MEDES 2024). Communications in Computer and Information Science (CCIS), volume 2518, pp 254–263. DOI: https://doi.org/10.1007/978-3-031-93598-5_19
Link to the paper - Ribeiro, T., Marto, A., Gonçalves, A., Santos, L., Rabadão, C., & de C. Costa, R. L. (2025). SafeARUnity: Real-Time Image Processing to Enhance Privacy Protection in LBARGs. In International Conference on Videogame Sciences and Arts (VJ 2024). Communications in Computer and Information Science (CCIS), volume 2324. pp. 186-200. DOI: https://doi.org/10.1007/978-3-031-81713-7_13
Link to the paper - Cruz, A. C., Costa, R. L. D. C., Santos, L., Rabadão, C., Marto, A., & Gonçalves, A. (2025). “Assessing User Perceptions and Preferences on Applying Obfuscation Techniques for Privacy Protection in Augmented Reality”. Future Internet, 17(2), 55. DOI: https://doi.org/10.3390/fi17020055
Link to the paper - Pereira, C., Marto, A., Ribeiro, R., Gonçalves, A., Rodrigues, N., Rabadão, C., Costa, RLC & Santos, L. (2025). Security and Privacy in Physical–Digital Environments: Trends and Opportunities. Future Internet, 17(2), 83. DOI: https://doi.org/10.3390/fi17020083
Link to the paper - Ribeiro, T., Marto, A., Gonçalves, A., Santos, L., Rabadão, C., & de C. Costa, R. L. (2024). “Image Processing for Privacy Enhancement in Location-Based AR Games”. In International Conference on Graphics and Interaction (ICGI 2024). Best poster award.
Link to the paper
Source code available:
- https://github.com/ipleiria-ciic/safe-ar – A repository containing the source code of SafeAR implemented as a service to run on a server. Users may use YOLOv5, YOLOv8, and YOLOv9. Instructions for installations in conda and docker.
- https://github.com/ipleiria-ciic/safe-ar-unity – A repository containing the source code of SafeAR implemented as library in Unity. It uses Sentis and YOLOv8.
- https://github.com/ipleiria-ciic/loot-ar – LootAR is a Unity-based AR game that combines geolocation with augmented reality. Explore a real-world map, find virtual items, and collect them in AR mode. Features include item collection and obfuscation functionalities.
- https://github.com/ipleiria-ciic/in-pain-tor – Real-time Semantic-aware Inpainting Model. Instead of obuscating objects, they are removed from the image.
- https://github.com/ipleiria-ciic/safe-ar-questionnaire – Source-code used to process the answers of the questionnaire created to assess users’ preferences, experiences, and concerns related to visual obfuscation techniques, namely masking, pixelation, and blurring (used in https://doi.org/10.3390/fi17020055).
- https://github.com/ipleiria-ciic/hololens-training-app – HoloLens application for training on the assembling of computer desktops using real-time privacy protection.
The support presentation in the seminar “Machine Learning and Privacy-Maintenance in Augmented Reality” by Tiago Ribeiro on May 15, 2024.
Contacts
For additional information, please send us an email to:
rogerio.l.costa AT ipleiria DOT pt
Acknowledgements
This work is funded by FCT – Fundação para a Ciência e a Tecnologia, I.P.,.
Reference: 2022.09235.PTDC
Funding: 49 888,90 € (Polytechnic of Leiria)
