Immersive technologies create highly interactive and engaging digital experiences, blurring the boundaries between the real and virtual. They provide users with a sense of presence and immersion and allow them to interact with computer-generated content in a realistic way. Immersive technologies have integrated many fields, including gaming, entertainment, education, training, healthcare, architecture and design. They have the potential to change how we communicate and learn by offering new opportunities for engagement, creativity and problem-solving.

The basics: What are immersive technologies?

Immersive technologies reside along a continuous scale ranging between the completely real and the completely virtual world, which encompasses all possible variations and compositions of real and virtual objects. This concept was first introduced in 1994 by Paul Milgram.

A line with arrows at both ends showing the range of immersive technologies. From left to right: real environment, augmented reality, mixed reality, augmented virtuality, virtual environment.

At one end, the real environment refers to the actual physical space, objects, and people that exist in the tangible world around us.

At the other end, the virtual environment represents a completely computer-generated and immersive digital space, distinct from the physical reality. Users can perceive the virtual environment using visualization technology such as computer screens, projectors or head-mounted displays to create an immersive experience where they feel as if they are present within the environment and can interact with virtual objects, characters and environments.

The space in the middle is called mixed reality, which is a blend of real and virtual environments, where digital and physical elements coexist and interact in real time. The real environment is an essential component of the mixed reality experience as it grounds the user in the physical world, providing a sense of presence and anchoring the digital elements within a real-world context. Mixed reality experiences can range from simple overlays of virtual objects onto the real environment to more sophisticated interactions. For example, in augmented reality, users might see and interact with virtual objects that appear to be in their real space, while in augmented virtuality they might manipulate real-world objects within a simulated environment.

What are the most common forms of immersive technologies?

Augmented reality

In their simplest form, immersive technologies consist of adding virtual objects to the real world. This is what we call augmented reality. Augmented reality is an interactive experience of a real-world environment where the objects that reside in the real world are enhanced by computer-generated perceptual information, such as information that is seen, heard and felt by the users. In other words, it is a system that provides:

a combination of real and virtual worlds
real-time interaction
accurate spatial registration of virtual and real objects

The overlaid sensory information can be constructive (meaning, added to the real environment), or destructive (meaning, masking the real environment). In this way, augmented reality alters one's perception of a real-world environment. The primary value of augmented reality is the way in which components of the digital world blend into a person's perception of the real world, not as a simple display of data, but through the integration of virtual objects which are perceived as natural parts of an environment.

In augmented reality, the user typically experiences the real world through a device such as a smartphone, tablet, smart glasses, or head-mounted display. The device captures the real-time video feed or uses sensors to understand the user's environment and then superimposes digital content onto the real-world view. It enhances user experiences by providing additional information, visualizations, or interactions that improve how users understand their real-world environment and engage with it.

Virtual reality

In their most complex form, immersive technologies engage users in a fully computer-generated, simulated environment, disconnecting them from the physical world. This virtual reality makes users feel as if they are present within the virtual environment although, physically, they are in a completely different environment.

A person using virtual reality can look around an artificial world, move around it, and interact with virtual features or items. This effect is commonly created by virtual reality headsets but can also be created through specially designed rooms with multiple large screens or projections. Virtual reality typically incorporates video and auditory feedback but can also allow other types of sensory feedback through haptic (touch) technology.

Virtual reality can also be desktop-based, displaying a 3D virtual environment on a regular computer display without use of any specialized positional tracking equipment. For example, modern first-person video games use triggers, responsive objects, and other such interactive features to make the user feel as though they are in a virtual world. This form of virtual reality significantly reduces the sense of immersion since the user can be distracted by what is happening in the real world.

Person sitting at a desk playing a first-person shooting video game. — Person playing a first-person video game.

Head-mounted displays immerse the user in a virtual environment. They typically include small high-resolution monitors which provide separate images for each eye to create stereoscopic graphics rendering the virtual environment, a stereo audio system, and real-time head and displacement tracking. Motion controls with haptic feedback allow users to interact with the virtual environment.

The use of immersive technologies

Immersive technologies have been integrated to video games. First-person shooting games use virtual environments to immerse the player in the game and allow interaction with its content. Other games, such as Pokémon Go!, use augmented reality to place game elements over real world elements.

Pokémon Go! game on a smartphone; augmented reality superimposes virtual in-game elements over real world elements. — A person playing Pokémon Go! on a smartphone.

Immersive technologies offer new possibilities in education and training. They create immersive learning environments that simulate real-world scenarios for training purposes, enhance visualization and understanding of complex concepts, and provide interactive and hands-on experiences.
Immersive technologies are making a significant impact on healthcare where virtual reality is used for therapy and rehabilitation. Surgeons can also use immersive technologies for preparing surgeries and training. Augmented reality can be used for teaching, consulting patient data and precision surgery.
Architects and designers use these technologies to create virtual walkthroughs, 3D visualizations, and interactive models of buildings and spaces. Virtual reality and augmented reality can improve design reviews, client presentations, spatial planning, virtual prototyping and product design.
Tourism and hospitality leverage immersive technologies to provide virtual tours, immersive experiences of travel destinations, and interactive information about landmarks and attractions. Augmented reality can overlay digital content on real-world points of interest, enriching the visitor experience.
Immersive technologies in retail and e-commerce allow for virtual changing rooms, virtual showrooms, and enhanced shopping experiences. Customers can visualize how products would look or fit in their real environment using augmented reality applications.

These are some of the many applications of immersive technologies which provide unique experiences and create new possibilities for innovation and engagement.

How can governments benefit from immersive technologies?

Immersive technologies can be used in a variety of fields, such as:

Training and simulation: Immersive technologies can train government personnel by offering realistic and immersive simulations that help learners practise scenarios, develop skills, and enhance decision-making abilities in a safe and controlled environment. For example, an employee could practise maintaining a heating plant without disrupting the normal operation of the actual heating plant. An employee could also learn about Indigenous culture by immersing themselves into a virtual Indigenous community.
Infrastructure design: Immersive technologies help government planners visualize and assess proposed infrastructure projects, such as a new bridge, allowing stakeholders to explore 3D models to simulate scenarios and understand the potential impact on the environment and communities before implementation.
Emergency management: Immersive technologies can contribute to a disaster preparedness and emergency response by enabling training for crises, such as natural disasters or terrorist attacks, thereby allowing emergency personnel to practise coordination, communication, and decision-making in realistic virtual simulations.
Cultural heritage preservation: Immersive technologies can preserve and showcase cultural heritage sites and artifacts by allowing users to virtually explore historical sites, museums and archaeological sites, providing access to cultural experiences and artifacts that may be physically inaccessible or at risk.

Are there any disadvantages to immersive technologies?

Immersive technologies are expensive to implement since the cost of both procurement and maintenance of sophisticated devices and applications become prohibitive as the immersive technologies become more popular. As with any information technology, newer technologies rapidly make older systems obsolete and increase their life-cycle costs. In many cases, the cost of immersive technologies may be too high to convince management to adopt it, at least as long as a positive impact on performance cannot be demonstrated.

The use of immersive technologies can also bring side effects to the user. The most common being nausea, similar to motion sickness, as the brain cannot correlate the motion observed by the eyes with that of the body standing still in a room. Physical discomfort and ocular fatigue may also occur from prolonged use of immersive devices.

Why do I need to know about this?

Immersive technologies have the potential to impact the way we work, learn and interact with one another. They continue to prove useful in new ways, to evolve and to be integrated into government services. As public servants, we have a responsibility to learn about those technologies and develop the understanding required to use them to deliver better products and services whenever feasible.

Would your work environment benefit from immersive technologies?

Guy Boulet

Immersive Technologies Explained