Exploration of Social Engagement During Interaction in Metaverse with an Enhanced Avatar amidst Self-Driving

Henry Campbell¹*, Emily Davidson²

[1] Quantum Tech Institute – Australia, henrycampbell782@gmail.com,

²Northern Plains University – Canada, emilydavidson081983@gmail.com,

https://orcid.org/0009-0004-4271-4244

* Correspondence: henrycampbell782@gmail.com

Received:

03 February, 2023

Accepted for publication:

17 March, 2023

ABSTRACT

This research explores the impact of utilizing Microsoft HoloLens 2 in a collaborative mixed reality setting in the Metaverse, focusing on the driver's social engagement with an autonomous driving system. In (semi-) autonomous vehicles, the driver acts as the system monitor, with driving relegated to a secondary task. The study leverages Microsoft Mesh XR technology, enabling immersion in shared mixed reality environments. A comparative user study assessed social engagement in two scenarios: baseline (communication via Skype/Meet on a mobile tablet) and mixed reality collaboration (using Microsoft Mesh on HoloLens 2 with augmented 3D avatars). Participants performed social interaction tasks and a remote tic-tac-toe game while monitoring the vehicle. Social engagement was measured using the Harms and Biocca questionnaire, revealing significant disparities in Co-presence, Perceived Emotional Interdependence, and Perceived Behavioral Interdependence. Participants found ease in interacting with avatars in the mixed reality scenario. The proposed methodology could extend to evaluate driver performance during handover procedures, especially in critical situations faced by autonomous driving systems.

Keywords: social engagement; HoloLens 2; driver involvement; mixed reality; human-machine interaction

Exploración del Compromiso Social durante la Interacción en el Metaverso con un Avatar Mejorado en Medio de la Conducción Autónoma

RESUMEN

Esta investigación explora el impacto de utilizar Microsoft HoloLens 2 en un entorno colaborativo de realidad mixta en el Metaverso, centrándose en el compromiso social del conductor con un sistema de conducción autónoma. En vehículos (semi)autónomos, el conductor actúa como monitor del sistema, relegando la conducción a una tarea secundaria. El estudio aprovecha la tecnología Microsoft Mesh XR, permitiendo la inmersión en entornos compartidos de realidad mixta. Un estudio comparativo evaluó el compromiso social en dos escenarios: uno de referencia (comunicación mediante Skype/Meet en una tableta móvil) y colaboración en realidad mixta (utilizando Microsoft Mesh en HoloLens 2 con avatares 3D aumentados). Los participantes realizaron tareas de interacción social y un juego remoto de tres en línea mientras monitoreaban el vehículo. El compromiso social se midió mediante el cuestionario de Harms y Biocca, revelando disparidades significativas en Copresencia, Interdependencia Emocional Percibida e Interdependencia Conductual Percibida. Los participantes encontraron facilidad al interactuar con avatares en el escenario de realidad mixta. La metodología propuesta podría extenderse para evaluar el rendimiento del conductor durante procedimientos de transferencia, especialmente en situaciones críticas enfrentadas por sistemas de conducción autónoma.

Palabras clave: compromiso social; HoloLens 2; participación del conductor; realidad mixta; interacción humano-máquina

INTRODUCTION

The Metaverse represents a dynamic and participatory space where human connections unfold in both virtual and social dimensions. This evolving concept capitalizes on four key trends. Firstly, the merging of augmented reality (AR) with virtual reality (VR) has made it a pervasive technology, providing users with a tangible sense of physical presence. Secondly, the expansion of the Internet into human-device and human-to-human interfaces facilitates embodied social interactions. Thirdly, the accessibility and popularity of the Metaverse are growing due to the development of immersive mobile technologies, such as smartphones and head-mounted devices. Fourthly, the convergence of pervasive reality with a digital world, driven by artificial intelligence (AI) and blockchain, enables real-time ubiquitous social interactions [1].

In the context of driving, the primary responsibility of the driver is to ensure the vehicle remains on the road and adheres to traffic regulations for a safe arrival at its destination [2]. Additionally, drivers engage in secondary tasks, such as interacting with navigation systems, adjusting radio stations, or responding to phone calls and text messages. Distracted driving, characterized by taking eyes off the road or engaging in these secondary tasks, poses significant safety risks to the driver and other road participants. Cognitive distractions [3] have been identified to negatively impact hazard anticipation, as discussed in a literature review on driver behavior [4].

The driver, as the decision-maker, combines two types of behavior: automatic, which is fast and effortless, and slower, more deliberate actions. Driving, being a safety-critical task, requires the driver to manage risks [5]. In the decision-making process, information flow activates mental models, internal representations of the current state, which undergo stages such as perception, prediction, evaluation, and action [6].

The Society of Automotive Engineers (SAE) outlines six levels of driving automation in their J3016 standard, ranging from SAE level 0 (no automation) to SAE level 5 (full driving autonomy). In (semi-)autonomous driving systems, the driving process becomes a secondary task, with the human operator transitioning to supervising the process in SAE level 3, or being completely out-of-the-loop in SAE level 4 and 5 [7]. The time previously dedicated to driving becomes monitoring time, allowing engagement in non-driving tasks [8]. One such activity is digital communication and collaboration, traditionally done through 2D video conferencing systems (Zoom, Google Meet, Microsoft Skype, Meta Messenger, WhatsApp). Nissan's Invisible-to-Visible (I2V) concept employs mixed reality to connect drivers and other participants as 3D augmented reality (AR) avatars inside the vehicle [9]. While the I2V concept aligns with the Metaverse's progress, it faces technological and social challenges. Acceptance of the new technology can be addressed through user studies, and the system should facilitate social presence, defined as the "sense of being together with another" [10], a crucial factor in 3D remote collaboration systems [11,12]. This work explores the guidelines proposed in [10] to evaluate social presence.

In [13], the introduction of a social robot named AIDA is detailed, utilizing the driver's mobile device as its interface to function as a friendly in-car companion. The outcomes of an experimental study conducted in a simulated in-car environment indicate that AIDA users experienced reduced stress levels. Another study [14] investigated the impact of various types of intelligent virtual agents (IVIAs) on driver perceptions in autonomous driving scenarios, revealing heightened driver engagement in the presence of conversational agents. A recent exploration [15] emphasized the potential enhancement of productivity through the use of AR headsets, offering passengers expansive virtual workspaces during travel in vehicles like cars, trains, subways, and planes. Consequently, a hypothesis emerges regarding the significance of future in-car companions embodying interactive features.

This paper examines how individuals utilizing HoloLens 2 perceive social interaction while engaging in two non-driving activities within a Metaverse-based mixed reality environment during autonomous driving. The study presents findings from participant involvement in interpersonal discussions and a simple yet competitive tic-tac-toe game. Notably, there is a dearth of studies assessing social presence in a collaborative, immersive mixed reality environment during autonomous driving.

Related Work

Collaboration in Metaverse

The concept of collaboration within mixed reality applications has a history spanning three decades, with early articles addressing the seamless integration of activities and reality enhancement in extended environments [16]. Initially, technological constraints led researchers to focus on creating system architectures supporting real-time collaboration, fostering interaction, and enhancing immersion—a goal shared by most virtual and augmented reality scenarios [17]. Technological advancements and increased acceptance have resulted in the availability of commercial mixed reality equipment, including Microsoft HoloLens 2, Vuzix Blade 2 [18], Magic Leap 2 [19], among others, although mixed reality headsets tend to be pricier than VR alternatives such as Oculus Quest 2.

Extended reality technologies [20] can enhance remote collaboration by creating virtual environments where users can access data from multiple sources, including 3D virtual avatars, facilitating real-time interaction [21]. Mixed reality has demonstrated its value in diverse sectors, such as architecture, engineering, construction, and operation (AECO) [25], mining operations [26], food production, consumption [27], healthcare [28], and recently, socializing and entertainment. The blending of VR objects with the real environment is an appealing concept, as evidenced by Meta's rebranding of Facebook and its long-term goal to create the technology necessary to actualize the Metaverse [29].

Representation of Avatars

The primary advantage of incorporating augmented reality (AR) in collaborative settings lies in its minimal computational cost for representation compared to the resource-intensive processes of 3D scanning and reconstruction. Researchers have discerned that comparable levels of immersion to real-life imagery in communication experiments can be achieved by utilizing low-poly animations, thus circumventing the need for continuous retrieval of precise data from involved parties. For instance, despite Meta encountering challenges in realizing the Metaverse experience outlined by Mark Zuckerberg in late 2021, the use of virtual avatars for co-locating users remains attractive. In Meta's new venture, Horizon Worlds, virtual reality takes precedence, featuring avatars with a cartoon-like aesthetic, while other services offer more lifelike avatar options [31].

Avatars have been represented through various methods, including live video streams [32], 3D representations derived from depth camera captures [33], or volumetrically using the established concept of holoportation [34].

Metaverse for Autonomous Driving

The integration of autonomous vehicles into the Metaverse holds significant potential and has garnered attention from major players in the automotive industry, such as Nissan with its Invisible to Visible (I2V) concept [9] and Hyundai with its Metamobility concept [35]. The I2V vision revolves around establishing an augmented reality channel bridging real-world and virtual information to provide an unparalleled in-vehicle experience for drivers and passengers (Figure 1). Nissan's Omni-Sensing technology comprises a virtual hub collecting both exterior (e.g., road status, signage, weather conditions, nearby vehicles, and pedestrians) and interior information (driver's alertness, facial expressions, and body tracking). The real-world data can be utilized in the Metaverse through a digital twin, creating a link between the virtual space and reality. Furthermore, the digital twin facilitates information transfer to an augmented or mixed reality interface from the physical world. BMW's Omniverse [36] also relies on digital twins, analytics, and AI, aiming to establish new benchmarks in virtual factory planning by simulating every aspect of their manufacturing processes.

Figure 1. Application of the Metaverse in autonomous driving systems

Hyundai aims to pioneer a connection between smart devices and the Metaverse, introducing virtual reality (VR) into mobility to help individuals overcome physical limitations in both time and space. An example of this initiative is the Hyundai Mobility Adventure, a Metaverse space on Roblox where users can socialize as digital characters, experience avatars, and explore the company's popular vehicles and future mobility solutions [37].

The development of artificial intelligence algorithms for autonomous driving demands extensive data representing all possible driving scenarios. Tesla's Autopilot, recognized as highly advanced, benefits from training neural networks with billions of miles. Despite Tesla's strides in computer vision, prediction, and path planning, there remains an insufficiency of data for fully autonomous vehicles to comprehend and respond to road events as humans do. The Metaverse emerges as a highly efficient and safe platform for gathering driving data in virtual environments, allowing AI agents to be tested in well-designed scenarios while reducing the carbon footprint. The recent introduction of the Oxbotica Driver offers low-energy, high-performance tools with features such as virtual world simulation, real-time data expansion, and automated discovery of challenging scenarios [38].

In a Metaverse, where users engage in a virtual world, the need for transportation or mobility is ever-present. The Oslo study [39] proposes a new mobility concept based on shared mobility as a service (MaaS). Despite several global pilot projects, creating cross-platform collaborations proves challenging. Connected cars, akin to smartphones in the past, must address issues like surveillance, privacy, and accountability. Simulating various MaaS strategies in a Metaverse could accelerate the adoption of shared mobility on a large scale in the real world. Metaverse users could travel with virtual autonomous vehicles simulating real driver behavior, boosting confidence in using autonomous vehicles in reality. Additionally, the Metaverse offers opportunities for in-vehicle entertainment experiences based on augmented reality, gamification, creative, and localized advertisements.

The role of urban mobility and autonomous driving in a smart city is discussed in [40], suggesting that adopting autonomous vehicles (AV) may not be the smartest choice. Instead, micromobility or other non-AV transportation modes are proposed. Integrating digital twins of real vehicles into digital twins

of smart cities could predict real-time traffic flow, simulate various urban conditions, and enhance road safety.

Connected cars, communicating with other vehicles and infrastructure, align with the future smart city, meeting the needs of AV and the Metaverse. The synergy between the Metaverse and the smart city has the potential to elevate the quality of life and significantly contribute to decarbonization [41,42].

2.4. Presence in Social Interactions Human beings possess an inherent need for social connection, and social interactions are recognized to influence internal states, shaping the capacity to perceive and comprehend others' thoughts, intentions, and associated behaviors. This influence, in turn, alters the decision-making process [43]. The concept of social presence holds particular significance in computer-mediated environments such as online learning, online assisted shopping, teleconferences, gaming, and assisted driving [44].

In contrast to telepresence and self-presence, social presence necessitates the actual experience within a virtual environment (VE) mediated by a co-present entity. It characterizes an individual's state in an environment—the consciousness in a VE and the feeling of being present in a place [45]. "The minimum level of social presence occurs when users feel that a form, behavior, or sensory experience indicates the presence of another intelligence. The amount of social presence is the degree to which a user feels access to the intelligence, intentions, and sensory impressions of another" [46]. Additional definitions on social presence can be found in [47].

Presence, as a concept, is intricate to describe and measure. A standardized instrument for evaluating presence was suggested in [48], identifying six dimensions: social richness, realism, transportation, immersion, social actor within medium, and medium as social actor.

A review of theories on social presence reveals three dimensions concerning interaction and behavior during VE experiences: co-presence, psychological involvement, and behavioral engagement [10]. Short et al. introduced the concept of social presence, encompassing participants' feelings of connectedness and perceived psychological distance during interaction [49]. Social presence involves interpersonal relations, where verbal and non-verbal language plays a pivotal role in VE interactions [50]. It proves vital in a shared VE interaction, fostering a sense of belonging beyond a mere group affiliation and immersing individuals fully in the virtual environment [45]. Immersion, the measurable technological capability of a medium to provide an inclusive, surrounding, and vivid illusion of reality, contributes to the participants' perception of the VE, making the environment more intuitive, realistic, and appealing.

In their extensive review, Oh et al. [50] demonstrate that social presence is influenced by multiple factors, collectively impacting participants' experiences. Studies indicate a preference for face-to-face interactions over computer-mediated communication. The quality of a partner's representation in a VR interaction significantly influences one's "sense of social presence." Visual representation availability and the visual realism of the virtual environment directly impact the level of social presence, with greater impact observed when virtual representations closely resemble human beings in communication and behavior. Behavioral patterns observed in VE, correlating with responses in normal life circumstances, contribute to aspects of immersion [45]. Direct interaction with a partner, especially when both manipulate a virtual object together with haptic feedback, leads to higher perceived levels of social presence [51].

MATERIALS AND METHODS

To assess how participants perceive social presence during interactions, a user study was conducted utilizing both a traditional 2D mobile tablet and a mixed reality HoloLens 2 headset. The baseline scenario involved interaction through the mobile tablet using the familiar 2D platform Skype/Meet. In contrast, the cross-platform Microsoft Mesh application on the HoloLens 2 facilitated collaboration in a mixed reality environment. Participants were tasked with engaging in two activities: the tell-a-lie game [52] and a tic-tac-toe game. This method, while relatively straightforward, aims to shed light on the impact of a novel interaction paradigm involving mixed reality and an animated 3D avatar.

Experimental Setup

The custom simulator employed in this study (depicted in Figure 2) incorporates a Stewart motion platform with six degrees of freedom (MOOG 6 DOF 2000E) [53,54] and a Logitech steering wheel and pedals on a driving seat. The motion platform provides realistic force feedback, enhancing immersion in the virtual environment. The simulator's dynamic model is based on the Motion Cueing Algorithm [55], and implementation details were previously outlined in a separate study [56].

The autonomous vehicle driving behavior was implemented utilizing the tools available in the open-source software platform, CARLA [57]. The study utilized the 0.9.13 release of CARLA on a VR-ready desktop computer equipped with an RTX 3090 graphics card, an AMD Ryzen 9 5950X processor, and 32 GB of RAM. The basic autopilot function integrated into CARLA served the purpose of this study adequately.

Participants in this study experienced the immersive mixed reality environment through a Microsoft HoloLens 2 headset. The mixed reality capabilities of HoloLens 2 are made possible by its see-through holographic lenses featuring a 2k resolution, a second-generation holographic processing unit (HPU), a Qualcomm Snapdragon 850 processor, advanced cameras (light camera, infrared cameras, and depth sensor), a 5-channel microphone array, and built-in spatial sounds. Key features of HoloLens 2 include hand tracking, real-time eye tracking, iris recognition, voice command, and control. The Microsoft Mesh [58] preview application facilitated virtual gatherings and collaborative mixed reality experiences. Mesh includes technologies and tools designed to enable rich, immersive 3D collaboration experiences, allowing users to interact with 3D objects, move around in digital space, and engage in spatial audio experiences. Microsoft Mesh offers out-of-the-box cartoon avatars that users can personalize, representing each participant as a custom cartoon avatar in the 3D digital space.

In terms of participants and procedure, the study involved 24 subjects, comprising 18 men and 6 women, with ages ranging from 23 to 64 years (M = 36.62, SD = 13.09). The participants, master students and professors from the university, had prior experience with 2D video conference systems (Microsoft Skype, Google Meet) but lacked experience with immersive AR systems. The majority (70.83%) had previous AR experience, primarily from playing AR games on mobile devices. All participants volunteered for the study.

The chosen experimental tasks included the social interaction tell-a-lie [52] and a tic-tac-toe game. The participants were familiarized with the research objectives and instructed on the use of HoloLens glasses at the start of the experiment. One participant drove on the simulator using one HoloLens 2, while a researcher played the role of a remote collaborator using another HoloLens 2. The entire process, including participant adaptation to the mixed reality environment and hand tracking capabilities of the HoloLens 2, took approximately 20 minutes.

Figure 3. Images captured from HoloLens 2: (a) collaborative tic-tac-toe game utilizing the Microsoft Mesh application; (b) the 3D avatar greeting the participant.

Given that the driving task held a secondary role, participants were explicitly instructed to direct their attention towards the collaborative interaction with the avatar. The assessment of social presence utilized Harms and Biocca's questionnaire [59], designed to emphasize the sense of togetherness and evaluate attention and behavior during interactions with the AR avatar. The questionnaire employs a 7-point Likert scale rating (1: Strongly disagree–7: Strongly agree).

Results

Responses from Harms and Biocca's social presence questionnaire were centralized using Excel and subsequently analyzed with SPSS. The questionnaire assesses six constructs: Co-presence (CoP), Attentional Allocation (Att), Perceived Message Understanding (Msg), Perceived Affective Understanding (Aff), Perceived Emotional Interdependence (Emo), and Perceived Behavioral Interdependence (Behv). The initial statistical analysis aimed to validate the scales by calculating Cronbach’s alpha [60]. Following that, mean values and standard deviations of the responses were obtained (Table 1 and Figure 4, displayed as box plots). Finally, a t-test analysis was conducted to ascertain if there were significant statistical differences between the baseline and augmented reality scenarios.

Table 1. Average and standard deviation (SD) values for constructs and each assessment.

*Values representing the average and standard deviation

Figure 4. Box plots displaying the outcomes for the Likert scale rating, covering Co-presence (CoP), Attentional Allocation (Att), Perceived Message Understanding (Msg), Perceived Affective Understanding (Aff), Perceived Emotional Interdependence (Emo), and Perceived Behavioral Interdependence (Behv).

Standard Scenario

All constructs demonstrated reliability, with Cronbach's alpha values exceeding 0.69. The results for each construct are as follows: Co-presence (M = 6.103, SD = 0.683), Attentional Allocation (M = 5.062, SD = 0.587), Perceived Message Understanding (M = 5.89, SD = 0.568), Perceived Affective Understanding (M = 4.68, SD = 0.68), Perceived Emotional Interdependence (M = 4.993, SD = 0.68), Perceived Behavioral Interdependence (M = 5.022, SD = 0.709).

Augmented Reality Scenario

Cronbach's alpha indicated good internal consistency for each set of questions (over 0.7). The results for Co-presence (M = 6.563, SD = 0.370) suggest that participants felt the presence of others, likely due to the dynamic and three-dimensional nature of the tasks. Attentional Allocation (M = 4.897, SD = 0.599) received a moderate score as participants divided their attention between the collaborative interaction and the driving scene. Perceived Message Understanding (M = 6.189, SD = 0.511) demonstrated an increased value, benefitting from spatial audio and the movement of the partner's 3D avatar mouth during communication. Smaller values and wider variation in responses were observed for Perceived Affective Understanding (M = 4.862, SD = 0.679) and Perceived Emotional Interdependence (M = 4.430, SD = 0.820), possibly due to the cartoon-like avatar representation and the relatively short study duration. Assessing partner feelings can be challenging, especially with non-familiar individuals, as in this study. Participants gave high ratings for Perceived Behavioral Interdependence (M = 5.917, SD = 0.727), aligning with the interactive nature of playing tic-tac-toe in the same virtual space.

Statistical Analysis

A paired t-test analysis revealed statistically significant differences for three constructs: CoP (t(23) = −2.991 p = 0.007), Emo (t(23) = 2.330, p = 0.29), and Behv (t(23) = −4.393, p = 0.000). No statistically significant differences were found for Att (t(23) = 0.982, p = 0.336), Msg (t(23) = −1.726, p = 0.098), and Aff (t(23) = −0.932, p = 0.361) constructs.

DISCUSSION

The integration of mixed reality devices, bridging digital content with the real world, is gradually gaining traction in our society. While XR technology isn't yet widely accessible, it exhibits significant promise in industrial applications and remote collaboration. Existing studies explore automated driving and mixed reality experiences independently, but this study uniquely addresses both subjects concurrently. In the ongoing transition toward highly and fully autonomous vehicles, this research seeks to contribute insights into interacting with a 3D augmented avatar while engaged in an autonomous driving system.

Given the increasing time spent commuting, averaging 30 to 60 minutes one way [61–63], the ability to manage our time effectively during car rides becomes crucial. Consequently, tasks that can be performed in the car, or perhaps should not be performed, assume paramount importance. Social presence emerges as a critical factor in evaluating the potential acceptance of this new driving paradigm. To foster the adoption of the Metaverse, applications must offer high levels of social presence alongside well-designed user interfaces. Thus, our user study explores social presence in two scenarios: the baseline and mixed reality collaboration. Subjective responses indicate a preference for interacting with the virtual avatar. Significant statistical differences were identified in three of the six constructs: Co-presence, where dynamic interaction heightened the sense of presence; Perceived Emotional Interdependence, influenced by avatar representation and study duration; and Perceived Behavioral Interdependence, increased by interactive tasks.

A noteworthy observation concerns the enhanced value for Perceived Message Understanding, influenced by superior audio and video quality in the interaction. This underscores the pivotal role of paraverbal (audio) and non-verbal (visual) language in augmenting the sense of social presence in AR interactions as part of interpersonal relationships. Novelty in mixed reality interaction with an animated 3D avatar may have also impacted the results.

Results further indicate that participants were more inclined to interact with the avatar in the AR environment than with video conferencing systems. The use of personal electronic devices like smartphones during autonomous driving may induce a feeling of social isolation, as it doesn't facilitate effective collaboration. AR headsets offer a more immersive experience, allowing users to feel like they are genuinely interacting with a remote collaborator rather than merely observing them, as is the case with video conferencing. Immersive virtual avatars significantly enhance social presence, creating a sense of direct communication with the remote person. Additionally, they contribute to a more realistic and engaging experience, crucial for effective collaboration. AR Metaverse environments can potentially transform the travel time of passengers in future autonomous vehicles, providing opportunities for new and productive activities by integrating their real environment with digital content for interactive experiences. However, it's important to emphasize the environmental conditions of the interaction, as the Harms and Biocca questionnaire appears to overlook this aspect, concentrating solely on the interaction itself, a concern validated by another study [64].

CONCLUSIONS

This study delved into the evaluation of social presence during interaction with a 3D augmented avatar within a mixed reality setting, particularly during autonomous driving. The motivation behind this research stems from the anticipation that individuals will engage in non-driving activities in autonomous vehicles and the increasingly crucial role the Metaverse is expected to play. Utilizing the Harms and Biocca questionnaire, we gauged participants' perceptions of collaboration in a mixed reality environment facilitated by the HoloLens 2. Preliminary findings from this study revealed that participants assigned higher ratings to three out of the six constructs in the mixed reality scenario.

Subsequent research endeavors could concentrate on appraising the driver's performance during handover maneuvers while collaborating within a mixed reality environment featuring a 3D animated avatar. Additionally, enhancing the qualitative analysis could involve integrating interviews with participants to gain deeper insights into their experiences. Outcomes from such studies may prove instrumental in designing driver training programs, ensuring that drivers comprehend the system's limitations and are prepared to intervene when the autonomous driving system confronts critical situations.

BIBLIOGRAPHICAL REFERENCES

Wang, Y.; Su, Z.; Zhang, N.; Xing, R.; Liu, D.; Luan, T.H.; Shen, X. A survey on metaverse: Fundamentals, security, and privacy.

IEEE Commun. Surv. Tutor. 2022. [CrossRef]

Jin, L.; Guo, B.; Jiang, Y.; Wang, F.; Xie, X.; Gao, M. Study on the impact degrees of several driving behaviors when driving while performing secondary tasks. IEEE Access 2018, 6, 65772–65782. [CrossRef]

Ebadi, Y.; Fisher, D.L.; Roberts, S.C. Impact of cognitive distractions on drivers’ hazard anticipation behavior in complex scenarios.

Transp. Res. Rec. 2019, 2673, 440–451. [CrossRef]

Ranney, T.A. Models of driving behavior: A review of their evolution. Accid. Anal. Prev. 1994, 26, 733–750. [CrossRef]

Luke, R.; Heyns, G.J. Reducing risky driver behaviour through the implementation of a driver risk management system. J. Transp.

Supply Chain. Manag. 2014, 8, 1–10. [CrossRef]

Boer, E.R.; Hoedemaeker, M. Modeling driver behavior with different degrees of automation: A hierarchical decision framework of interacting mental models. In Proceedings of the 17th European Annual Conference on Human Decision Making and Manual

Control, Valenciennes, France, 14–16 December 1998; pp. 63–72.

International, S. Available online: https://www.sae.org/standards/content/j3016_202104/ (accessed on 21 October 2022).

Naujoks, F.; Forster, Y.; Wiedemann, K.; Neukum, A. Improving usefulness of automated driving by lowering primary task interference through HMI design. J. Adv. Transp. 2017, 2017, 6105087. [CrossRef]

Technology, N. Available online:

https://www.nissan-global.com/EN/TECHNOLOGY/OVERVIEW/i2v.html (accessed on 21

October 2022)

Biocca, F.; Harms, C.; Burgoon, J.K. Toward a more robust theory and measure of social presence: Review and suggested criteria.

Presence Teleoper. Virtual Environ. 2003, 12, 456–480. [CrossRef]

Jo, D.; Kim, K.-H.; Kim, G.J. Effects of avatar and background types on users’ co-presence and trust for mixed reality-based teleconference systems. In Proceedings of the 30th Conference on Computer Animation and Social Agents, Seoul, Republic of

Korea, 22–24 May 2017; pp. 27–36.

Yoon, B.; Kim, H.-i.; Lee, G.A.; Billinghurst, M.; Woo, W. The effect of avatar appearance on social presence in an augmented reality remote collaboration. In Proceedings of the 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Osaka,

Japan, 23–27 March 2019; pp. 547–556.

Williams, K.J.; Peters, J.C.; Breazeal, C.L. Towards leveraging the driver’s mobile device for an intelligent, sociable in-car robotic assistant. In Proceedings of the 2013 IEEE intelligent vehicles symposium (IV), Gold Coast, Australia, 23–26 June 2013; pp.

369–376.

Wang, M.; Lee, S.C.; Kamalesh Sanghavi, H.; Eskew, M.; Zhou, B.; Jeon, M. In-vehicle intelligent agents in fully autonomous driving: The effects of speech style and embodiment together and separately. In Proceedings of the 13th International Conference

on Automotive User Interfaces and Interactive Vehicular Applications, Leeds, UK, 9–14 September 2021; pp. 247–254.

Medeiros, D.; McGill, M.; Ng, A.; McDermid, R.; Pantidi, N.; Williamson, J.; Brewster, S. From Shielding to Avoidance: Passenger

Augmented Reality and the Layout of Virtual Displays for Productivity in Shared Transit. IEEE Trans. Vis. Comput. Graph. 2022,

28, 3640–3650. [CrossRef]

Billinghurst, M.; Kato, H. Collaborative mixed reality. In Proceedings of the First International Symposium on Mixed Reality, Yokohama, Japan, 9–11 March 1999; pp. 261–284.

Cadet, L.B.; Chainay, H. Memory of virtual experiences: Role of immersion, emotion and sense of presence. Int. J. Hum.-Comput.

Stud. 2020, 144, 102506. [CrossRef]

Blade, V. Available online: https://www.vuzix.com/products/vuzix-blade-2-smart-glasses (accessed on 21 October 2022).

Leap, M. Available online: https://www.magicleap.com/magic-leap-2 (accessed on 21 October 2022).

Xi, N.; Chen, J.; Gama, F.; Riar, M.; Hamari, J. The challenges of entering the metaverse: An experiment on the effect of extended reality on workload. Inf. Syst. Front. 2022, 1–22. [CrossRef]

Popescu, G.H.; Ciurlău, C.F.; Stan, C.I. Virtual Workplaces in the Metaverse: Immersive Remote Collaboration Tools, Behavioral Predictive Analytics, and Extended Reality Technologies. Psychosociol. Issues Hum. Resour. Manag. 2022, 10, 21–34.

Cárdenas-Robledo, L.A.; Hernández-Uribe, Ó.; Reta, C.; Cantoral-Ceballos, J.A. Extended reality applications in industry 4.0.—A

systematic literature review. Telemat. Inform. 2022, 73, 101863. [CrossRef]

Vasarainen, M.; Paavola, S.; Vetoshkina, L. A systematic literature review on extended reality: Virtual, augmented and mixed

reality in working life. Int. J. Virtual Real. 2021, 21, 1–28. [CrossRef]

Dai, F.; Olorunfemi, A.; Peng, W.; Cao, D.; Luo, X. Can mixed reality enhance safety communication on construction sites? An industry perspective. Saf. Sci. 2021, 133, 105009. [CrossRef]

Cheng, J.C.; Chen, K.; Chen, W. State-of-the-art review on mixed reality applications in the AECO industry. J. Constr. Eng. Manag.

2020, 146, 03119009. [CrossRef]

Stothard, P.; Squelch, A.; Stone, R.; Van Wyk, E. Towards sustainable mixed reality simulation for the mining industry. Min.

Technol. 2019, 128, 246–254. [CrossRef]

Chai, J.J.; O’Sullivan, C.; Gowen, A.A.; Rooney, B.; Xu, J.-L. Augmented/mixed reality technologies for food: A review. Trends Food Sci. Technol. 2022, 124, 182–194. [CrossRef]

Gsaxner, C.; Li, J.; Pepe, A.; Jin, Y.; Kleesiek, J.; Schmalstieg, D.; Egger, J. The HoloLens in Medicine: A systematic Review and Taxonomy. arXiv 2022, arXiv:2209.03245.

Fernandez, P. Facebook, Meta, the metaverse and libraries. Libr. Hi Tech News 2022, 39, 1–5. [CrossRef]

Kolesnichenko, A.; McVeigh-Schultz, J.; Isbister, K. Understanding emerging design practices for avatar systems in the commercial social vr ecology. In Proceedings of the 2019 on Designing Interactive Systems Conference, San Diego, CA, USA, 23–28 June 2019;

pp. 241–252.

Avatars, U. Available online: https://unionavatars.com/ (accessed on 9 November 2022).

Kim, J.I.; Ha, T.; Woo, W.; Shi, C.-K. Enhancing social presence in augmented reality-based telecommunication system. In Proceedings of the International Conference on Virtual, Augmented and Mixed Reality, Las Vegas, NV, USA, 24 July 2013; pp.

359–367.

Pejsa, T.; Kantor, J.; Benko, H.; Ofek, E.; Wilson, A. Room2room: Enabling life-size telepresence in a projected augmented reality environment. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing, San

Francisco, CA, USA, 27 February–2 March 2016; pp. 1716–1725.

Orts-Escolano, S.; Rhemann, C.; Fanello, S.; Chang, W.; Kowdle, A.; Degtyarev, Y.; Kim, D.; Davidson, P.L.; Khamis, S.; Dou, M.

Holoportation: Virtual 3d teleportation in real-time. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology, Tokyo, Japan, 16–19 October 2016; pp. 741–754.

Concept, H.M. Available online: https://www.hyundai.news/eu/articles/press-releases/hyundai-shares-vision-of-newmetamobility-concept-through-robotics-and-Metaverse-at-CES-2022.html (accessed on 21 October 2022).

Group, B. Available online:

https://www.press.bmwgroup.com/global/article/detail/T0329569EN/bmw-group-and-nvidiatake-virtual-factory-planning-to-the-next-level?language=en (accessed on 21 October 2022).

Adventure, H.M. Available online:

https://www.hyundai.com/worldwide/en/company/newsroom/hyundai-motor-vitalizesfuture-mobility-in-roblox-Metaverse-space%252C-hyundai-mobility-adventure-0000016713 (accessed on 21 October 2022).

Driver, O. Available online: https://www.oxbotica.com/insight/oxbotica-partners-with-nevs-to-reshape-the-future-of-urbanmobility-with-fleet-of-shared-self-driving-all-electric-vehicles/ (accessed on 21 October 2022).

COWI. The Oslo Study—How Autonomous Cars May Change Transport in Cities; COWI: Lyngby, Denmark, 2019; Volume 79.

Richter, M.A.; Hagenmaier, M.; Bandte, O.; Parida, V.; Wincent, J. Smart cities, urban mobility and autonomous vehicles: How different cities needs different sustainable investment strategies. Technol. Forecast. Soc. Chang. 2022, 184, 121857. [CrossRef]

Allam, Z.; Sharifi, A.; Bibri, S.E.; Jones, D.S.; Krogstie, J. The metaverse as a virtual form of smart cities: Opportunities and challenges for environmental, economic, and social sustainability in urban futures. Smart Cities 2022, 5, 771–801. [CrossRef]

Pamucar, D.; Deveci, M.; Gokasar, I.; Tavana, M.; Köppen, M. A metaverse assessment model for sustainable transportation using ordinal priority approach and Aczel-Alsina norms. Technol. Forecast. Soc. Chang. 2022, 182, 121778. [CrossRef]

Soiné, A.; Flöck, A.N.; Walla, P. Electroencephalography (EEG) Reveals Increased Frontal Activity in Social Presence. Brain Sci.

2021, 11, 731. [CrossRef] [PubMed]

Cui, G.; Lockee, B.B.; Meng, C. Building modern online social presence: A review of social presence theory and its instructional design implications for future trends. Educ. Inf. Technol. 2012, 18, 661–685. [CrossRef]

Slater, M.; Wilbur, S. A Framework for Immersive Virtual Environments (FIVE): Speculations on the Role of Presence in Virtual Environments. Presence Teleoper. Virtual Environ. 1997, 6, 603–616. [CrossRef]

Biocca, F. The Cyborg’s Dilemma: Progressive Embodiment in Virtual Environments [1]. J. Comput.-Mediat. Commun. 1997, 3, JCMC324. [CrossRef]

Kreijns, K.; Xu, K.; Weidlich, J. Social presence: Conceptualization and measurement. Educ. Psychol. Rev. 2021, 34, 139–170.

[CrossRef] [PubMed]

Lombard, M.; Ditton, T. At the heart of it all: The concept of presence. J. Comput.-Mediat. Commun. 1997, 3, JCMC321. [CrossRef]

Short, J.; Williams, E.; Christie, B.A. The Social Psychology of Telecommunications; Wiley & Sons: Toronto, ON, Canada, 1976.

Oh, C.S.; Bailenson, J.N.; Welch, G.F. A Systematic Review of Social Presence: Definition, Antecedents, and Implications. Front.

Robot. AI 2018, 5, 114. [CrossRef]

Skalski, P.; Tamborini, R. The role of social presence in interactive agent-based persuasion. Media Psychol. 2007, 10, 385–413.

[CrossRef]

Zuckerman, M.; DePaulo, B.M.; Rosenthal, R. Verbal and nonverbal communication of deception. In Advances in Experimental Social Psychology; Elsevier: Amsterdam, The Netherlands, 1981; Volume 14, pp. 1–59.

Hulme, K.; Kasprzak, E.; English, K.; Moore-Russo, D.; Lewis, K. Experiential learning in vehicle dynamics education via motion simulation and interactive gaming. Int. J. Comput. Games Technol. 2009, 2009, 952524. [CrossRef]

Stewart, D. A platform with six degrees of freedom. Proc. Inst. Mech. Eng. 1965, 180, 371–386. [CrossRef]

Reymond, G.; Kemeny, A. Motion cueing in the Renault driving simulator. Veh. Syst. Dyn. 2000, 34, 249–259. [CrossRef]

Antonya, C.; Irimia, C.; Grovu, M.; Husar, C.; Ruba, M. Co-simulation environment for the analysis of the driving simulator’s actuation. In Proceedings of the 2019 7th International Conference on Control, Mechatronics and Automation (ICCMA), Delft,

The Netherlands, 6–8 November 2019; pp. 315–321.

Dosovitskiy, A.; Ros, G.; Codevilla, F.; Lopez, A.; Koltun, V. CARLA: An open urban driving simulator. In Proceedings of the Conference on Robot Learning, Mountain View, CA, USA, 13–15 November 2017; pp. 1–16.

Mesh, M. Available online: https://learn.microsoft.com/en-us/mesh/overview (accessed on 21 October 2022).

Harms, C.; Biocca, F. Internal consistency and reliability of the networked minds measure of social presence. In Proceedings of the Seventh Annual International Workshop: Presence, Valencia, Spain, 13–15 October 2004.

Hair, J.F.; Anderson, R.; Tatham, R.; Black, W. Factor analysis. Multivariate data analysis. NJ Prentice-Hall 1998, 3, 98–99.

Eurostat. Available online: https://ec.europa.eu/eurostat/web/products-eurostat-news/-/ddn-20201021-2 (accessed on 21

October 2022).

Giménez-Nadal, J.I.; Molina, J.A.; Velilla, J. Trends in commuting time of European workers: A cross-country analysis. Transp.

Policy 2022, 116, 327–342. [CrossRef]

Teodorovicz, T.; Kun, A.L.; Sadun, R.; Shaer, O. Multitasking while driving: A time use study of commuting knowledge workers to assess current and future uses. Int. J. Hum.-Comput. Stud. 2022, 162, 102789. [CrossRef]

Kim, K.; Schubert, R.; Hochreiter, J.; Bruder, G.; Welch, G. Blowing in the wind: Increasing social presence with a virtual human via environmental airflow interaction in mixed reality. Comput. Graph. 2019, 83, 23–32. [CrossRef]