A New Tracking Technology in 3D space - Using Ultrasound

1. Expansion of Interactions in Space such as Spatial Computing and User Interfaces

Spatial computing is revolutionizing the interaction between the digital world and the physical world, creating immersive experiences where virtual elements blend seamlessly with reality. The technology spans augmented reality (AR), virtual reality (VR) and mixed reality (MR) and enhances fields such as productivity, education, gaming and industrial design. The expansion of spatial computing, driven by continued advances in hardware, software and artificial intelligence, is pushing the boundaries of interactive immersive environments.

Position tracking of hands and controllers is important to realize the interaction between the digital world and the physical world. Position tracking and interaction can also be used in other applications.

Application example

• Touch sensitive displays (3D displays, laptops)
• AR related accurately tracked peripheral devices (bracelets, rings, watches)
• General purpose computer peripheral devices (mouse, trackpads)
• Tools tracking (training tool, surgical scalpel, stylus pen)
• Full-body motion capture
• and more

2. How Position Tracking is Used for Interactions in Space

The position tracking is fundamental to interaction in space and allows you to accurately track position and orientation of objects in 3D space. Accurate position tracking ensures that input from a user is correctly recognized on devices, that virtual elements are correctly positioned in the physical world, and that a user has a seamless and reliable experience.
Without it, the virtual objects cannot properly interact with the real world, reducing immersion and engaging experiences.

2-1 Traditional Position Tracking Technologies

Several technologies are used to achieve position tracking for interactions in space.
The following is a basic overview of the technologies.

Optical hand tracking uses a camera to track form of a hand. This is only done by cameras in devices and does not require additional hardware. However, accuracy of depth and position is poor because the acquired information is 2D images. Moreover, it is heavily dependent on context: environmental light such as sunlight, background textures, contrasts.

Optical marker tracking uses a camera to track markers such as LEDs or features in environments. This is the modern technology and has relatively better accuracy compared to optical hand tracking. However, many track markers are needed, which leads to hardware design limitations. And it is also heavily dependent on the contexts same as optical hand tracking (environmental light such as sunlight, background textures, contrasts).

Inertial tracking uses an accelerometer and a gyroscope (and sometimes a magnetometer) to measure movements. With this technology, the devices can be small and low power. However, inertial sensors cannot estimate the position by themselves and require calibration of the original position. Furthermore, inertial sensors suffer from drift and offset, resulting in poor accuracy.

There are other technologies such as optical tracking and magnetic tracking with base stations. It is difficult for them to spread to the general uses because they are relatively expensive and large.

2-2 Next Generation Position Tracking Technology

There is a new position tracking technology combining ultrasound, optical, and inertial technologies. This new method addresses the limitations of traditional tracking technologies with the features of small size, high accuracy, affordability, and adaptability. This technology is compatible with any devices which include cameras, microphones, and a communication link such as XR devices. Additionally, this technology contributes to a wide range of tracked devices from small form factors to traditional controllers, which is ideal for spatial computing and user interfaces. In the near future, professional verticals such as simulated reality, training, design, creativity, gaming, virtual production, etc. will grow by new devices.

2-3 Technology Demonstrated by Sensoryx's Maliang Magic Pencil

The Maliang Magic Pencil represents the smart fusion technology of the ultrasound, optical, and Inertial technologies, realizing the features such as small size and high accuracy. This technology invites it's users to dive into spatial computing with confidence and precision.

3. Advantages of Position Tracking Technology using Ultrasound

The position tracking is realized by using 1D information (distance and depth).
Ultrasound technology uses sound waves to determine the distance and depth of objects by the principle of time of flight (ToF).

The ultrasound method has several key advantages.

• High accuracy: It has precise accuracy in determining the depth and the position of objects.
• Compact size: Transducers can be integrated into small devices.
• Low power consumption: It is ideal for battery powered devices, ensuring longer operational times.
• Robustness: It performs consistently well in all lighting conditions, including direct sunlight.
• Light information: It realizes fast processing and response of sensing.
• Wide field of view: It contributes to wider area of sensing.

By utilizing multiple ultrasound transducers and their 1D information, the position of objects can be calculated. With this technology, 3D location sensing can be done by only ultrasound, so that the cameras on devices can be replaced.

This ultrasound technology brings a lot of benefits to realize better position tracking.
For future devices, miniaturization is needed. This ultrasound tracking technology offers adoptability with the current devices as the existing microphones can receive low frequency ultrasound, meaning there is no need of additional installation of receivers while other technologies need additional receiver design and optimization.

In peripheral devices or other tracked devices, the modern technologies cannot offer the size of form desired by the user or designers. This ultrasound tracking technology can offer very small form factor. No other technology can or will be able to deliver the highly accurate position tracking at the same time being able to be embedded in intuitive and small tools or devices, such as: watches, pens, rings, etc.

For the future devices where compute performance is limited, this ultrasound tracking technology offers currently the only feasible technology to perform high accuracy tracking. Established technologies like optical technologies are not useable on devices with low compute and/or battery life.
Sensoryx tracking technology is currently the only 6DoF tracking technology.

4. Future Example of Position Tracking Devices using Ultrasound

A future example of the spatial computing and user Interfaces of utilizing new ultrasound technology is smart glasses. These devices are equipped with some microphones which are essential for capturing precise audio and environmental data.
The advanced ultrasound technology is aligned with these devices, leveraging their multiple microphones to provide highly accurate position tracking even under intense light condition and interaction capabilities without the need for any optical reference.

Ultrasound only technology is instrumental for developing their accessory ecosystem and ensuring seamless integration with not limited to the smart glasses but including the next generation of tracking devices. As such, it expands the possibility of realization of better spatial computing and user interfaces.

5. Murata Manufacturing's Developing Ultra-Small Ultrasound Sensor

Murata Manufacturing is now developing piezoelectric MEMS ultrasonic transducer (pMUT) integrating its MEMS technology and ultrasonic technology.

This pMUT brings many advantages with the following features.

• Compact size: It is designed for integration into small devices, enhancing portability.
• Energy efficient: Its low power consumption makes it suitable for portable and battery powered devices.
• Low frequency: Ultrasound transmitted by the pMUT can be received by MEMS microphones.
• Wide field of view: Wide coverage of sensing can be realized.

With these features, Murata Manufacturing's pMUT is suitable for ultrasound tracking technology, and it contributes to the realization of more innovative devices for spatial computing and user interfaces in the future.