Temperature variation could help new touchscreen technology simulate virtual shapes

High-fidelity touch has the potential to significantly expand the scope of what we expect from computing devices, making new remote sensory experiences possible. The research on these advancements, led by a pair of researchers from the J. Mike Walker '66 Department of Mechanical Engineering at Texas A&M University, could help touchscreens simulate virtual shapes. 

Dr. Cynthia Hipwell is studying friction at the finger-device level, while Dr. Jonathan Felts is researching friction in the interaction between single skin cells and the glass of the touchscreen interface. The two are bringing together their respective areas of expertise to apply friction principles at the microscopic level to finger-device interaction mechanics. 

Hipwell highlighted the significance of the pursuit by comparing it to the technologies currently available for conveying immersive and accurate information through high-fidelity audio and video. 

"We can view digitally recorded or remotely transmitted audio and video on a screen with great detail," said Hipwell, Oscar S. Wyatt, Jr. '45 Chair II professor. "We do not yet have that same capability with touch on a touchscreen. Imagine you could feel the skin of a snake that lives on another continent or the fabric of clothes you want to buy online."

Researchers are applying friction principles at the microscopic level as they seek to help touchscreens simulate virtual shapes. | Video: Texas A&M Engineering 

Another application of this technology, which has received high levels of interest recently, is the augmentation of immersive virtual environments, such as the proposed metaverse.

“The touch sensations that would be required to really immerse yourself into a reality that is fully digital requires huge advancements in touch perception,” said Felts, associate professor and Steve Brauer, Jr. '02 Faculty Fellow. “What we’ve done is essentially created an entirely new way to modulate the perception of touch that hasn’t existed before."

The team is working to show that it is possible to mimic the unique mechanical and thermal sensations associated with different surface textures and shapes. Their recent publication in the journal Science Robotics demonstrates the potential for translating these sensations on a touchscreen by using temperature variation alone, rather than expressing them through ultrasonic vibrations or electroadhesion methods. 

"We were actually surprised by the magnitude of the friction increase we were able to achieve," Hipwell said. "Its magnitude is competitive with current surface haptic devices, meaning that there is another option for friction modulation in surface-haptic device rendering."

Researchers have shown that temperature variation can help simulate virtual shapes on touchscreen devices. | Image: Texas A&M Engineering 

Another exciting development, Hipwell said, is that their research has shown that it is possible to localize the friction to the outer layer of the skin and, at least at swipe speeds, control friction without making the device feel hot.

As the research continues, Felts said many of the questions remaining involve how readily the approach can be incorporated into consumer devices and commercialized.

"Can it be scaled down? Can it respond quickly enough? Can it mimic a wide range of surfaces? Can it be affordable? We think these are fair criticisms, yet we look forward to using this phenomenon to improve our basic understanding of haptic feedback and pursuing miniaturization and commercialization avenues," he said. 

The team is continuing their work to address challenges facing the approach by further exploring the complexities of the finger-device interface and variations that occur due to environmental and skin-property differences. They also hope to look at design improvements for miniaturization and integration into touchscreens.