<p><strong>Researchers in the emerging field of spatial computing have developed a prototype augmented reality headset that uses holographic imaging to overlay full-color, 3D moving images on the lenses of what would appear to be an ordinary pair of glasses. Unlike the bulky headsets of present-day augmented reality systems, the new approach delivers a visually satisfying 3D viewing experience in a compact, comfortable, and attractive form factor suitable for all-day wear.</strong></p><p>“Our headset appears to the outside world just like an everyday pair of glasses, but what the wearer sees through the lenses is an enriched world overlaid with vibrant, full-color 3D computed imagery,” said <a href="https://web.stanford.edu/~gordonwz/" data-extlink="" target="_blank">Gordon Wetzstein</a>, an associate professor of electrical engineering and an expert in the fast-emerging field of spatial computing. Wetzstein and a team of engineers introduce their device in a new paper in the journal <a href="https://www.nature.com/articles/s41586-024-07386-0" data-extlink="" target="_blank"><em>Nature</em></a>. Additional information about this advance is available at <a href="https://www.computationalimaging.org/publications/holographicAR/" data-extlink="" target="_blank">this website</a>, created by the research team.</p><p>Though only a prototype now, such a technology, they say, could transform fields stretching from gaming and entertainment to training and education – anywhere computed imagery might enhance or inform the wearer’s understanding of the world around them.</p><p>“One could imagine a surgeon wearing such glasses to plan a delicate or complex surgery or airplane mechanic using them to learn to work on the latest jet engine,” <a href="https://manugopa.github.io/" data-extlink="" target="_blank">Manu Gopakumar</a>, a doctoral student in the Wetzstein-led <a href="https://www.computationalimaging.org/" data-extlink="" target="_blank">Stanford Computational Imaging lab</a> and co-first author of the paper said.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 790px;"><span class="fr-img-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1715819863917-2_20240430_holographic_ar_displays_n6a0965_0.png.webp" style="width: 100%px;" class="fr-fic fr-dib"><span class="fr-inner">New holographic augmented reality system that enables more compact 3D displays | Image courtesy of Andrew Brodhead</span></span></span></p><h2 id="isPasted">Barriers overcome</h2><p>The new approach is the first to thread a complex maze of engineering requirements that have so far produced either ungainly headsets or less-than-satisfying 3D visual experiences that can leave the wearer visually fatigued, or even a bit nauseous at times.</p><p>“There is no other augmented reality system out there now with comparable compact form factor or that matches our 3D image quality,” said <a href="https://gunyeal.github.io/" data-extlink="" target="_blank">Gun-Yeal Lee</a>, a postdoctoral researcher in the Stanford Computational Imaging lab and co-first author of the paper.</p><p>To succeed, the researchers have overcome technical barriers through a combination of AI-enhanced holographic imaging and new nanophotonic device approaches. The first hurdle was that the techniques for displaying augmented reality imagery often require the use of complex optical systems. In these systems, the user does not actually see the real world through the lenses of the headset. Instead, cameras mounted on the exterior of the headset capture the world in real time and combine that imagery with computed imagery. The resulting blended image is then projected to the user’s eye stereoscopically.</p><p>“The user sees a digitized approximation of the real world with computed imagery overlaid. It’s sort of augmented virtual reality, not true augmented reality,” explained Lee.</p><p>These systems, Wetzstein explains, are necessarily bulky because they use magnifying lenses between the wearer’s eye and the projection screens that require a minimum distance between the eye, the lenses, and the screens, leading to additional size.</p><p>“Beyond bulkiness, these limitations can also lead to unsatisfactory perceptual realism and, often, visual discomfort,” said <a href="https://choisuyeon.github.io/" data-extlink="" target="_blank">Suyeon Choi</a>, a doctoral student in the Stanford Computational Imaging lab and co-author of the paper.</p><h2>Killer app</h2><p>To produce more visually satisfying 3D images, Wetzstein leapfrogged traditional stereoscopic approaches in favor of holography, a Nobel-winning visual technique developed in the late-1940s. Despite great promise in 3D imaging, more widespread adoption of holography has been limited by an inability to portray accurate 3D depth cues, leading to an underwhelming, sometimes nausea-inducing, visual experience.</p><p>The Wetzstein team used AI to improve the depth cues in the holographic images. Then, using advances in nanophotonics and waveguide display technologies, the researchers were able to project computed holograms onto the lenses of the glasses without relying on bulky additional optics.</p><p>A waveguide is constructed by etching nanometer-scale patterns onto the lens surface. Small holographic displays mounted at each temple project the computed imagery through the etched patterns which bounce the light within the lens before it is delivered directly to the viewer’s eye. Looking through the glasses’ lenses, the user sees both the real world and the full-color, 3D computed images displayed on top.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 790px;"><span class="fr-img-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1715819889720-3_20240430_holographic_ar_displays_n6a1169_0.png.webp" style="width: 100%px;" class="fr-fic fr-dib"><span class="fr-inner">Using nanophotonic technologies called metasurface optics, the researchers designed and fabricated a novel waveguide design that can relay 3D hologram information of RGB visible light into a single compact device with high transparency. These nanophotonic waveguide samples were fabricated in-house at Stanford Nanofabrication Facility and Stanford Nano Shared Facilities. | Image courtesy of Andrew Brodhead</span></span></span></p><h2 id="isPasted">Life-like quality</h2><p>The 3D effect is enhanced because it is created both stereoscopically, in the sense that each eye gets to see a slightly different image as they would in traditional 3D imaging, and holographically.</p><p>“With holography, you also get the full 3D volume in front of each eye increasing the life-like 3D image quality,” said <a href="https://bchao1.github.io/" data-extlink="" target="_blank">Brian Chao</a>, a doctoral student in the Stanford Computational Imaging lab and also co-author of the paper.</p><p>The ultimate outcome of the new waveguide display techniques and the improvement in holographic imaging is a true-to-life 3D visual experience that is both visually satisfying to the user without the fatigue that has challenged earlier approaches.</p><p>“Holographic displays have long been considered the ultimate 3D technique, but it’s never quite achieved that big commercial breakthrough,” Wetzstein said. “Maybe now they have the killer app they’ve been waiting for all these years.”</p><p><em>Additional authors are from The University of Hong Kong and NVIDIA. Wetzstein is also member of&nbsp;</em><a href="http://biox.stanford.edu/" data-extlink="" target="_blank"><em>Stanford Bio-X</em></a><em>, the&nbsp;</em><a href="https://humanperformance.stanford.edu/" data-extlink="" target="_blank"><em>Wu Tsai Human Performance Alliance</em></a><em>, and the&nbsp;</em><a href="https://neuroinstitute.stanford.edu/" data-extlink="" target="_blank"><em>Wu Tsai Neurosciences Institute</em></a><em>.</em></p>

Prototype of the compact augmented reality glasses. Through holography and AI, these glasses can display full-color, 3D moving images over an otherwise direct view of the real world. | Image courtesy of Andrew Brodhead

Combining advances in display technologies, holographic imaging, and artificial intelligence, engineers at Stanford say they have produced a leap forward for augmented reality.

AI and holography bring 3D augmented reality to regular glasses

<h2 id="isPasted"><strong>A training center at the cutting edge of progress</strong></h2><p>The aeronautics industry is without question a dynamic sector of activity where progress takes place at breakneck pace and where the players in the field have to deal with remarkable technological advances that follow each other at increasingly shorter intervals. This industry would therefore very likely find itself lagging behind considerably without the support of continually updated training programs and lecturers who keep a close eye on market developments.</p><p>And this is exactly what happens at the&nbsp;<a href="https://www.polyaero05.fr/le-centre/" target="_blank">POLYAÉRO® Hautes-Alpes</a> Center of Excellence, which is managed by FORMASUP MÉDITERRANÉE and where a research laboratory is used to ensure that new technologies and the latest aeronautical practices are included in the learners’ educational and technological pathway.</p><p>Harnessing virtual reality (VR) and augmented reality (AR) tools, this immersive laboratory makes it possible for students to examine and handle aeronautical parts and even entire aircraft interactively. However, making digital twins and 3D models available in VR requires reliable scan data to be collected using devices of proven metrological quality, and then post-processed in an intuitive and user-friendly software ecosystem.</p><p>In the wake of these numerous improvements, POLYAÉRO® Center of Excellence thus found itself in need of reliable means for digitizing parts and models that could be moving or static, large or small.</p><h2><strong>Versatile equipment, lasting results</strong></h2><p id="isPasted">One of the aircraft that was scanned for 3D imaging at POLYAÉRO® is a PC-6 Porter, a paratrooper transport aircraft. With a length of eleven meters and a wingspan of sixteen meters, this aircraft from Swiss aircraft manufacturer PILATUS Aircraft presented specific challenges that this scanner was able to handle thanks to its shape, size and mobility.</p><p>The industrial project carried out by the students required studying the conversion of a thermal-powered aircraft into an electric aircraft. This involved creating a digital twin of the entire PC-6 Porter.</p><p>It is important to note that before POLYAÉRO® acquired Creaform’s solutions, scanning was done using a measuring arm equipped with a scanner head, which made data acquisition more complex. In fact, unlike the various CREAFORM scanners, this solution was not suitable to meet their need for flexibility.</p><p><strong><u>Scanning</u></strong></p><p>The scanning device that was chosen had to meet the following indispensable conditions:</p><ul><li>Ergonomic</li><li>Compatible with the digitization of moving and very large objects and parts.</li></ul><p>This led POLYAÉRO® to choose&nbsp;<a href="https://www.creaform3d.com/en" target="_blank">Creaform</a> scanners and photogrammetry systems:</p><ul><li>The&nbsp;<a href="https://www.creaform3d.com/en/optical-3d-scanner-metrascan" target="_blank">MetraSCAN 3D</a> for scanning the exterior of the aircraft</li><li>The&nbsp;<a href="https://www.creaform3d.com/en/handheld-portable-3d-scanner-goscan-3d" target="_blank">Go!SCAN 3D</a> and the&nbsp;<a href="https://www.creaform3d.com/en/handyscan-3d-silver-series-professional-3d" target="_blank">HandySCAN 3D|SILVER</a> for the interior</li><li>The&nbsp;<a href="https://www.creaform3d.com/en/metrology-solutions/optical-measuring-systems-maxshot-3d" target="_blank">MaxSHOT 3D&nbsp;</a>for creating a common marker inside and outside the aircraft</li></ul><p>The scan data was collected within a very short time and then served as the basis for the rest of the project, including retrofitting the aircraft with an electric motor and batteries, all in 3DEXPERIENCE (formerly CATIA V5, from the software publisher Dassault Systèmes).</p><p>Animated in VR in an appropriate setting, the aircraft’s digital twin was given various modifications which, once validated, were adapted in augmented reality.</p><h3><strong>A simple but powerful software platform</strong></h3><p><strong><u>Reverse engineering</u></strong></p><p>After a 3D scan of the aircraft was rendered in real time using Creaform’s various measurement solutions, reverse engineering could start. The geometries thus captured were processed in Creaform’s software module <a href="https://www.creaform3d.com/en/metrology-solutions/3d-applications-software-platforms/vxmodel-scan-cad-software-module" target="_blank">VXmodel</a> by a group of students in training. The reverse engineered components were then reconstructed using 3DEXPERIENCE software.</p><p><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1714726741746-Polyaero_image_3d.webp" style="width: 100%px;" class="fr-fic fr-dib"></p><p id="isPasted">With its intuitive interface, ease of use and fast processing times, the&nbsp;<a href="https://www.creaform3d.com/en/vxelements-viewer" target="_blank">VXelements</a> platform has become an essential tool for setting up a library of 3D models that are derived from scans or formed through reverse engineering. This constantly evolving database fuels the technological projects as well as the digital libraries of the immersive VR and AR laboratory.</p><p>The POLYAÉRO® Hautes Alpes Center of Excellence is at the forefront of educational innovation, offering its students all the hardware and software resources they need to build solid skills for the professions of the future in the aeronautical industry.</p><p><a href="https://www.creaform3d.com/blog/pun5th75ef_wp/wp-content/uploads/Polyaero_equipe31.jpg" target="_blank"><br></a></p><p><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1714726755539-Polyaero_equipe31.webp" style="width: 100%px;" class="fr-fic fr-dib"></p><p id="isPasted">“The ease with which we can use Creaform’s 3D scanners makes it easier for us to train our students in reverse engineering. Direct meshing is a distinct advantage in post-processing, as is the ability to scan an aircraft in the same repository using several scanners simultaneously,” sums up Killian Marandel, operational manager at POLYAÉRO®.</p><p><strong><u>The future of the industry and training</u></strong></p><p><strong>&nbsp;</strong>At the POLYAERO® Hautes Alpes Center of Excellence, students choose their own technological projects and use 3D scanners to study discrepancies between design and production, deformations and reverse engineered models.</p><p>The team of design engineers and lecturers need a large number of 3D models of parts in order to enrich the library and integrate them into virtual or augmented reality teaching materials.</p><p>To do this, the center scans parts of all sizes and aircrafts to complete existing models or create new ones. It is thanks to this updating of databases and knowledge that POLYAÉRO® and the industry in general is evolving rapidly and almost organically. Creaform is thrilled to be part of this adventure<strong>.</strong><strong><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1714726768301-Polyaero_equipe2-1536x1152.webp" style="width: 100%px;" class="fr-fic fr-dib"></strong><br></p>

The aeronautics industry is without question a dynamic sector of activity where progress takes place at breakneck pace and where the players in the field have to deal with remarkable technological advances that follow each other at increasingly shorter intervals.

Electrifying an Aircraft Using 3D Scanning and Augmented Reality

<p id="isPasted">Holographic images have real depth because they are three dimensional, whereas monitors merely simulate depth on a 2D screen. Because we see in three dimensions, holographic images could be integrated seamlessly into our normal view of the everyday world.</p><p>The result is a virtual and augmented reality display that has the potential to be truly immersive, the kind where you can move your head normally and never lose the holographic images from view. “To get a similar experience using a monitor, you would need to sit right in front of a cinema screen,” said <a href="https://engineering.princeton.edu/faculty/felix-heide" target="_blank">Felix Heide</a>, assistant professor of <a href="http://cs.princeton.edu/" target="_blank">computer science</a> and senior author on <a href="https://www.nature.com/articles/s41467-024-46915-3" target="_blank">a paper</a> published April 22 in Nature Communications.</p><p>And you wouldn’t need to wear a screen in front of your eyes to get this immersive experience. Optical elements required to create these images are tiny and could potentially fit on a regular pair of glasses. Virtual reality displays that use a monitor, as current displays do, require a full headset. And they tend to be bulky because they need to accommodate a screen and the hardware necessary to operate it.</p><p>“Holography could make virtual and augmented reality displays easily usable, wearable, and ultrathin,” said Heide. They could transform how we interact with our environments, everything from getting directions while driving, to monitoring a patient during surgery, to accessing plumbing instructions while doing a home repair.</p><p>One of the most important challenges is quality. Holographic images are created by a small chip-like device called a spatial light modulator. Until now, these modulators could only create images that are either small and clear or large and fuzzy. This tradeoff between image size and clarity results in a narrow field of view, too narrow to give the user an immersive experience. “If you look towards the corners of the display, the whole image may disappear,” said Nathan Matsuda, research scientist at Meta and co-author on the paper.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 790px;"><span class="fr-img-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1713912832170-Neural_Etendue_Expander_3_v2-1-2048x1152.png" style="width: 100%px;" class="fr-fic fr-dib"><span class="fr-inner">At left, an image produced using current holographic display technology. At right, the same image created using the new optical element invented by researchers at Princeton and Meta.</span></span></span></p><p id="isPasted">Heide, Matsuda and&nbsp;<a href="https://ethan-tseng.github.io/" target="_blank">Ethan Tseng</a>, doctoral student in computer science, have&nbsp;<a href="https://light.princeton.edu/publication/neural-etendue-expander/" target="_blank">created a device</a> to improve image quality and potentially solve this problem. Along with their collaborators, they built a second optical element to work in tandem with the spatial light modulator. Their device filters the light from the spatial light modulator to expand the field of view while preserving the stability and fidelity of the image. It creates a larger image with only a minimal drop in quality.</p><p>Image quality has been a core challenge preventing the practical applications of holographic displays, said Matsuda. “The research brings us one step closer to resolving this challenge,” he said.</p><p>The new optical element is like a very small custom-built piece of frosted glass, said Heide. The pattern etched into the frosted glass is the key. Designed using AI and optical techniques, the etched surface scatters light created by the spatial light modulator in a very precise way, pushing some elements of an image into frequency bands that are not easily perceived by the human eye. This improves the quality of the holographic image and expands the field of view.</p><p>Still, hurdles to making a working holographic display remain. The image quality isn’t yet perfect, said Heide, and the fabrication process for the optical elements needs to be improved. “A lot of technology has to come together to make this feasible,” said Heide. “But this research shows a path forward.”</p><p><em>The paper, “Neural Etendue Expander for Ultra-Wide-Angle High-Fidelity Holographic Display” was published April 22 in Nature Communications. In addition to Heide and Tseng, co-authors from Princeton include Seung-Hwan Baek and Praneeth Chakravarthula. In addition to Matsuda, co-authors from Meta Research are Grace Kuo, Andrew Maimone, Florian Schiffers, and Douglas Lanman. Qiang Fu and Wolfgang Heidrich from the Visual Computing Center at King Abdullah University of Science and Technology in Saudi Arabia also contributed. The work was supported by Princeton University’s&nbsp;<a href="https://iac.princeton.edu/home.html" target="_blank">Imaging and Analysis Center</a> and the King Abdullah University of Science and Technology’s Nanofabrication Core Lab.</em></p>

Researchers at Princeton and Meta have created a tiny optical device that makes holographic images larger and clearer. Small enough to fit on a pair of eyeglasses, the device could enable a new kind of immersive virtual reality display. Illustration by Liz Sabol, photo by Nathan Matsuda

Setting the stage for a new era of immersive displays, researchers are one step closer to mixing the real and virtual worlds in an ordinary pair of eyeglasses using high-definition 3D holographic images, according to a study led by Princeton University researchers.

Holographic displays offer a glimpse into an immersive future

<p id="isPasted">As augmented and mixed reality experiences become more popular through various devices, advancements in battery technologies are needed to ensure that these crucial components are made smaller, yet maintain their functionality. One <a href="https://www.mse.engineering.cmu.edu/index.html" rel="noopener" target="_blank">Department of Materials Science and Engineering</a> alumnus is involved with developing the technologies needed to bring these products to market at Meta.</p><p>As a battery system engineer, Ankur Gupta, BS’10, MS’10, takes an interdisciplinary approach rooted in his studies at CMU to working with the multiple teams of engineers involved in the development of devices such as the Quest 3 and Ray-Ban | Meta smart glasses.</p><p><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzA4MDk3Njk5MzE2LTAyMDktZW0tZnV0dXJlLXZpc2lvbi5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"></p><p id="isPasted">At one end of the spectrum, he works with cell engineers, often chemical and materials science engineers, who design the chemistry inside the cell. &nbsp;At the other end, Gupta coordinates with the product design team, typically mechanical engineers, who are creating the physical design of what the actual active material has to fit into. Gupta bridges the work of these two groups, drawing on the experiences he had as a student, taking their outputs and designing the battery management unit (BMU). The BMU in the Ray-Ban | Meta smart glasses is now the world’s smallest and has all the circuitry for protection, gauging, and charging.</p><p>Gupta’s interest in electronic materials was nurtured through his work with <a href="https://engineering.cmu.edu/directory/bios/whitacre-jay.html" target="_blank">Jay Whitacre</a>, professor of materials science and engineering, on batteries and energy technologies. When Gupta first enrolled as a student in the College of Engineering at Carnegie Mellon, he had no idea what materials science was. His initial interest was in electrical engineering, though he was also curious about nanotechnology. In his first year, he took the introduction to materials science and engineering course, and he was immediately drawn in. Later in his studies, Whitacre encouraged Gupta to think through bringing concepts to mass production, taking into account the economic influences and incremental changes in materials technologies.</p><p>“The idea of taking a concept and actually bringing it to mass production has stuck with me, both in my past role at Apple and now at Meta,” says Gupta, as he reflects on projects involving batteries for devices such as iPhones, Airpods, Apple Watches, and now the Quest 3 and Ray-Ban | Meta smart glasses.</p><p>While he did eventually go on to pursue a graduate degree in electrical engineering, he attributes his continued education to his background in materials science.</p><p>“The reason why I went back to electric engineering was because of my foundation in materials science, especially in batteries,” says Gupta. “I knew how batteries were made and how they worked, but I wanted to go beyond that.”</p><p>Gupta credits Carnegie Mellon’s emphasis on interdisciplinary studies focused on next generation problems to fostering his interest in batteries, as well as his approach to his work.</p><p>“Batteries in particular are extremely interdisciplinary. There’s the chemistry side with materials science, the mechanical side of physically making the cell, the electrical side of what to do with the energy that comes out of the cell, and the economic side of building a supply chain.”</p><blockquote><p>The idea of taking a concept and actually bringing it to mass production has stuck with me.</p><cite><strong>Ankur Gupta<cite id="isPasted">, MSE'10</cite></strong><br></cite></blockquote><p>Gupta draws on his materials science foundation still in his career today, recalling insights from Professor <a href="https://engineering.cmu.edu/directory/bios/heard-robert.html" target="_blank">Robert Heard</a>, who encouraged students to dive deep into a problem and think about it “at an atomic scale.”</p><p>“For battery packs, we have to really dive into the printed circuit board (PCB) materials. Even though it’s electrical engineering, it’s also materials science getting into what’s going on at the atomic level.”</p><p>Gupta draws on his own personal experiences of using the smart glasses to capture a photo, rather than trying to find his phone, to illustrate the ability that AR could have for users to experience the world, rather than looking down at phones. As Meta moves forward with developments for the Quest line of products with the goal of eventually functioning as a replacement to a traditional laptop, refining batteries to meet product design and functionality needs will be imperative.</p><p>“Some of the key steps involve being able to have a battery that can power all that a laptop can do, but in the size of a phone,” he says. “It takes a lot of engineering and innovation to compact all these features into a very small product.’</p><p>The pathway to a career in industry was not always clear, as the role of materials science in similar companies is not always immediately recognized, but Gupta encourages current students to think outside traditional career pathways.</p><p>“If you think that there’s a way for materials to impact a certain industry, there probably is,” he says. “Make connections, and you can find a way to show there is an avenue.”</p>

Ankur Gupta (MSE’10) is working to develop the battery technologies needed to bring augmented and mixed reality products to market at Meta.

Powering a vision for the future

<p id="isPasted">The researchers, from the University of Cambridge, used machine learning to develop ‘HotGestures’ – analogous to the hot keys used in many desktop applications.</p><p><span contenteditable="false" draggable="true" class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable"><iframe width="640" height="360" src="https://www.youtube.com/embed/3kNFvhU5ntU?&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span><em>Modelling a sailboat in virtual reality Credit: University of Cambridge</em></p><p>HotGestures give users the ability to build figures and shapes in virtual reality without ever having to interact with a menu, helping them stay focused on a task without breaking their train of thought.</p><p>The idea of being able to open and control tools in virtual reality has been a movie trope for decades, but the researchers say that this is the first time such a ‘superhuman’ ability has been made possible. The <a href="https://ieeexplore.ieee.org/document/10269004" target="_blank">results</a> are reported in the journal <em>IEEE Transactions on Visualization and Computer Graphics</em>.</p><p>Virtual reality (VR) and related applications have been touted as game-changers for years, but outside of gaming, their promise has not fully materialised. “Users gain some qualities when using VR, but very few people want to use it for an extended period of time,” said <a href="https://pokristensson.com/" target="_blank">Professor Per Ola Kristensson</a> from Cambridge’s Department of Engineering, who led the research. “Beyond the visual fatigue and ergonomic issues, VR isn’t really offering anything you can’t get in the real world.”</p><p>Most users of desktop software will be familiar with the concept of hot keys – command shortcuts such as ctrl-c to copy and ctrl-v to paste. While these shortcuts omit the need to open a menu to find the right tool or command, they rely on the user having the correct command memorised.</p><p>“We wanted to take the concept of hot keys and turn it into something more meaningful for virtual reality – something that wouldn’t rely on the user having a shortcut in their head already,” said Kristensson, who is also co-Director of the <a href="https://www.chia.cam.ac.uk/" target="_blank">Centre for Human-Inspired Artificial Intelligence</a>.</p><p>Instead of hot keys, Kristensson and his colleagues developed ‘HotGestures’, where users perform a gesture with their hand to open and control the tool they need in 3D virtual reality environments.</p><p>For example, performing a cutting motion opens the scissor tool, and the spray motion opens the spray can tool. There is no need for the user to open a menu to find the tool they need, or to remember a specific shortcut. Users can seamlessly switch between different tools by performing different gestures during a task, without having to pause their work to browse a menu or to press a button on a controller or keyboard.</p><p>“We all communicate using our hands in the real world, so it made sense to extend this form of communication to the virtual world,” said Kristensson.</p><p>For the study, the researchers built a neural network gesture recognition system that can recognise gestures by performing predictions on an incoming hand joint data stream. The system was built to recognise ten different gestures associated with building 3D models: pen, cube, cylinder, sphere, palette, spray, cut, scale, duplicate and delete.</p><p>The team carried out two small studies where participants used HotGestures, menu commands or a combination. The gesture-based technique provided fast and effective shortcuts for tool selection and usage. Participants found HotGestures to be distinctive, fast, and easy to use while also complementing conventional menu-based interaction. The researchers designed the system so that there were no false activations – the gesture-based system was able to correctly recognise what was a command and what was normal hand movement. Overall, the gesture-based system was faster than a menu-based system.</p><p>“There is no VR system currently available that can do this,” said Kristensson. “If using VR is just like using a keyboard and a mouse, then what’s the point of using it? It needs to give you almost superhuman powers that you can’t get elsewhere.”</p><p>The researchers have made the source code and dataset publicly available so that designers of VR applications can incorporate it into their products.</p><p>“We want this to be a standard way of interacting with VR,” said Kristensson. “We’ve had the tired old metaphor of the filing cabinet for decades. We need new ways of interacting with technology, and we think this is a step in that direction. When done right, VR can be like magic.”</p><p>The research was supported in part by the Engineering and Physical Sciences Research Council (EPSRC), part of UK Research and Innovation (UKRI).</p><hr><p><em><strong>Reference:</strong><br>Zhaomou Song; John J Dudley; Per Ola Kristensson. ‘<a href="https://ieeexplore.ieee.org/document/10269004" target="_blank">HotGestures: Complementing Command Selection and Use with Delimiter-Free Gesture-Based Shortcuts in Virtual Reality</a>.’ IEEE Transactions on Visualization and Computer Graphics (2023). DOI: 10.1109/TVCG.2023.3320257</em></p><p><strong>This article was originally published on the <a href="https://www.cam.ac.uk/research/news/machine-learning-gives-users-superhuman-ability-to-open-and-control-tools-in-virtual-reality" target="_blank">University of Cambridge website</a></strong></p>

Researchers have developed a virtual reality application where a range of 3D modelling tools can be opened and controlled using just the movement of a user’s hand. 

Machine learning gives users 'superhuman' ability to open and control tools in virtual reality

<h2 dir="ltr" id="isPasted">Introduction</h2><p dir="ltr">In this blog post, we delve into the dynamic and innovative realm of Immersal's Visual Positioning System (VPS), a groundbreaking technology reshaping various industries, with a special focus on robotics, automation, and the burgeoning field of autonomous mobility and smart spaces. Our exploration will highlight how Immersal's VPS is revolutionizing robotics, particularly for those robots lacking conventional navigation systems like LiDAR, enabling them to interact with and navigate their environments with unprecedented precision and efficiency.</p><p dir="ltr">We will also delve into the transformative potential of VPS in the domain of autonomous vehicles, exploring how this technology is not just enhancing existing systems but also pioneering new pathways in automation and mobility. Additionally, we will examine the significant collaboration between Immersal and Unikie, a union that epitomizes the synergy between advanced spatial computing and edge-driven autonomous solutions. This partnership is a beacon in the international field of autonomous mobility and smart spaces, showcasing how integrated technologies can lead to smarter, more efficient urban landscapes and transportation systems.</p><p dir="ltr">Join us as we navigate through these exciting developments, offering insights into a future where Immersal's VPS technology is at the heart of innovative solutions across multiple sectors, driving progress and redefining the boundaries of what is possible in our increasingly connected world.</p><h2 dir="ltr">The Role of VPS in Robotics</h2><p dir="ltr">In the realm of robotics, the ability to accurately perceive and interact with the environment is paramount. Traditionally, robots have relied on technologies like LiDAR (Light Detection and Ranging) for navigation and mapping. However, LiDAR systems can be costly and complex, limiting their accessibility and versatility in various applications. This is where Immersal's VPS technology comes into play, offering a more cost-effective and flexible solution for spatial awareness and navigation.</p><p dir="ltr">VPS technology enables robots to navigate and operate in complex environments without the need for expensive hardware. By using cameras, robots equipped with Immersal's VPS can understand their surroundings with a high degree of accuracy. This capability is particularly beneficial in industries where environmental conditions are constantly changing or where installing additional infrastructure for navigation is impractical.&nbsp;</p><p dir="ltr">In manufacturing, robots with VPS navigate autonomously, swiftly adapting to changing environments, thus boosting efficiency and reducing reprogramming needs. These robots enhance inventory management in logistics and warehousing by precisely navigating and efficiently handling materials. Additionally, in healthcare and hospitality, VPS-enabled robots improve interactions in dynamic settings, aiding in patient care and customer service, demonstrating their versatility and responsiveness in diverse roles.</p><hr><h2 id="isPasted">What will you build?</h2><p>Whether you're building an AR experience for a large commercial space, a factory, or an industrial setting, or even if you'd just like to create an AR app for fun, Immersal is the way to do it.</p><p>Immersal has released its SDK 2.0 and new Mapper App.&nbsp;</p><p><a href="https://developers.immersal.com/?WEVOLVER=" target="_blank" rel="noopener noreferrer" class="button-main-action">SIGN UP FOR YOUR FREE ACCOUNT HERE.</a></p><hr><h2>Boston Dynamics' Spot Robot and Immersal VPS</h2><p dir="ltr">The integration of Immersal's Visual Positioning System (VPS) with Boston Dynamics' Spot robot represents a fascinating convergence of advanced robotics and cutting-edge positioning technology. Spot, known for its mobility, agility, and stability, can achieve even greater operational capabilities when combined with the precision and spatial awareness provided by Immersal's VPS.</p><p dir="ltr">Immersal's VPS can significantly enhance <a href="https://www.youtube.com/watch?v=aq5SHrw6WSk" target="_blank">Spot's navigation abilities, especially in environments where GPS is unreliable or unavailable</a>. By utilizing visual cues from the environment, Spot can navigate complex indoor spaces or densely packed urban areas with greater accuracy. This integration allows Spot to perform tasks with higher precision, making it an even more valuable tool in various applications.</p><p dir="ltr"><span contenteditable="false" draggable="true" class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable"><iframe width="640" height="360" src="https://www.youtube.com/embed/aq5SHrw6WSk?&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"><span class="fr-mk" style="display: none;">&nbsp;</span></iframe></span></p><h2 dir="ltr">Potential Applications Across Industries</h2><p dir="ltr"><strong>Industrial Inspection and Maintenance:</strong> In industrial settings, a Spot robot can autonomously navigate facilities, performing routine inspections and identifying maintenance needs with minimal human oversight. This leads to increased safety, as Spot can access hazardous or hard-to-reach areas, and improved efficiency in monitoring and maintenance tasks.</p><p dir="ltr"><strong>Emergency Response and Public Safety:</strong>&nbsp;Spot can be deployed to navigate through debris or unsafe structures in emergency situations, such as natural disasters or fire outbreaks. With Immersal's VPS, it can accurately map and assess the environment, providing valuable information to first responders and aiding in search and rescue operations.</p><p dir="ltr"><strong>Healthcare and Assisted Living: </strong>Spot can navigate autonomously in healthcare facilities, delivering supplies or assisting in patient care. The precise navigation enabled by Immersal's VPS ensures safety and efficiency, especially in crowded or dynamically changing environments like hospitals.</p><h2>Pioneering a New Era in Autonomous Mobility and Smart Spaces</h2><p>The international landscape of autonomous mobility and smart spaces is rapidly evolving, with Immersal's Visual Positioning System (VPS) playing a pivotal role. This technology's integration into the realm of autonomous systems and intelligent environments is not just an advancement; it's a transformation. The partnership between Immersal and Unikie, a leader in edge computing and autonomous driving solutions, marks a significant milestone in this journey.</p><p>Immersal's VPS brings unparalleled precision and efficiency to the domain of smart spaces. By accurately mapping and understanding spatial environments, this technology enables a more intuitive and interactive experience in various settings, from commercial complexes to public spaces. The integration of Immersal's VPS in smart spaces lays the groundwork for environments that are not only responsive but also adaptive to the needs and movements of individuals within them.</p><p><a href="https://www.youtube.com/watch?v=-XCjAppPf1I" target="_blank">The collaboration between Unikie and Immersal</a> is a testament to the potential of combining expertise in edge computing and visual positioning. Unikie's focus on autonomous driving solutions complements Immersal's VPS, creating a synergy that enhances the capabilities of autonomous vehicles. This partnership is poised to drive significant advancements in autonomous mobility, offering solutions that are safer, more efficient, and more reliable.</p><p><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1707486614089-car.png" style="width: 300px;" class="fr-fic fr-dib"></p><p dir="ltr">The application of Immersal's VPS in conjunction with Unikie's technology has the potential to transform urban landscapes and transportation systems. In smart cities, autonomous vehicles equipped with this integrated technology can navigate more safely and efficiently, reducing traffic congestion and improving overall urban mobility. This collaboration could lead to the development of smarter, more connected urban environments where autonomous vehicles and smart infrastructure work in harmony.</p><p dir="ltr">Beyond urban mobility, the Immersal-Unikie partnership extends its impact to various sectors. In logistics, for example, autonomous vehicles equipped with Immersal's VPS can optimize route planning and delivery processes. In retail and hospitality, smart spaces enhanced by this technology can offer personalized experiences to customers, revolutionizing service delivery and customer interaction.</p><h2>Conclusion</h2><p>As we reach the end of our exploration into Immersal's VPS, it's evident that this technology is not just an advancement; it's a revolution in the fields of robotics, automation, and autonomous mobility. It represents a pivotal shift towards more efficient, intelligent, and interconnected systems. In robotics, the impact of VPS is transformative, offering a cost-effective and flexible alternative to traditional navigation systems like LiDAR. This innovation enables robots to navigate and interact with dynamic environments with unprecedented precision, opening new possibilities in industries ranging from manufacturing to healthcare. The agility and adaptability provided by VPS technology mean that robots can now operate in complex settings with minimal human intervention, enhancing efficiency and safety.</p><p>The collaboration between Immersal and Unikie exemplifies the fusion of cutting-edge spatial computing and autonomous solutions, marking a significant milestone in the evolution of smart spaces and autonomous mobility. This partnership is not just about enhancing existing systems; it's about pioneering a new era where technology seamlessly integrates with our physical and social environments. In the realm of autonomous vehicles, the integration of Immersal's VPS with edge computing leads to safer, more efficient navigation, reducing urban challenges like traffic congestion and paving the way for smarter, more sustainable cities. Beyond transportation, this technology has the potential to revolutionize public spaces, commercial complexes, and even personal living spaces, making them more responsive and adaptive to human needs. Immersal's VPS is an opportunity to lead in the creation of innovative, efficient, and sustainable solutions.&nbsp;</p><p>This technology is a call to action, urging us to rethink how we interact with our environments, mobilize resources, and automate processes. As we step into this new era, the possibilities are limitless, and the potential for positive change is immense.&nbsp;</p><h2 id="isPasted">What will you build?</h2><p>Whether you're building an AR experience for a large commercial space, a factory, or an industrial setting, or even if you'd just like to create an AR app for fun, Immersal is the way to do it.</p><p>Immersal has released its SDK 2.0 and new Mapper App.&nbsp;</p><p><a href="https://developers.immersal.com/?WEVOLVER=" target="_blank" rel="noopener noreferrer" class="button-main-action">SIGN UP FOR YOUR FREE ACCOUNT HERE.</a></p>

Immersal's Visual Positioning System (VPS) revolutionizes robotics and autonomous mobility, enhancing precision and fostering innovations in smart urban development and transportation through its partnership with Unikie.

Exploring the Future of Navigation and Automation with Immersal's Visual Positioning System

<p dir="ltr" id="isPasted">Imagine a world where the physical and digital realms blend seamlessly, creating extraordinary live experiences. This fusion, driven by Augmented Reality (AR) and enhanced by Visual Positioning Systems (VPS), is revolutionizing the way we interact with entertainment across sports, music, and leisure activities. AR transcends mere visual enhancement, by overlaying digital information onto our real-world environment that can be interacted with and manipulated, allowing for deeper audience engagement and comprehension. VPS plays a crucial role in this transformation, enabling precise and context-aware positioning of augmented elements, thus making the AR experience more realistic and interactive.</p><p dir="ltr">This article dives into the significant impact of AR and VPS in entertainment, focusing on their joint capability to deliver real-time analytics, boost audience involvement, and craft captivating, immersive experiences.&nbsp;</p><p dir="ltr">In a sports stadium, AR could offer spectators live, in-depth analytics overlayed on the field of play, while in a concert setting, it could synchronize elaborate visual effects with live performances.&nbsp;</p><p dir="ltr">The synergy of AR and VPS is reshaping entertainment, offering personalized, enriched, and dynamic experiences that were previously unimaginable. This piece aims to unveil the innovative potential of combining AR with VPS, highlighting how they are collectively setting new benchmarks in interactive and engaging live entertainment experiences.</p><h2 dir="ltr">Exploring the Essence of Augmented Reality</h2><p dir="ltr">Augmented Reality (AR) integrates digital components like images, information, and animations into our physical environment, viewable from devices such as glasses or smartphones, enhancing our perception of reality. Central to its effectiveness is the Visual Positioning System (VPS), utilizing sensors and cameras for accurate real-time interpretation and localization of the environment. To generate impactful AR experiences, careful planning is essential. This includes conducting detailed spatial mapping to ensure the accurate placement of virtual elements within the real world, thus creating a seamless and interactive user experience. This integration of AR and VPS marks a significant step in merging the digital with the physical, opening new dimensions in our interaction with technology.</p><h3 dir="ltr">Elevating the Sports Experience with AR</h3><p dir="ltr">AR is redefining the sports landscape, seamlessly blending the electric ambiance of live events with the analytical depth of broadcast media. For fans in the stands, AR offers an enriched viewing experience by delivering real-time statistics and instant replay capabilities right before their eyes, enhancing their connection with the game. Simultaneously, it provides coaches with a powerful tool for gaining deeper insights into player performances and strategies. This innovative fusion of live action and digital intelligence by AR not only elevates the spectator experience but also revolutionizes coaching techniques, setting a new standard in how sports are played, coached, and enjoyed.</p><h3>Enhanced Case Study: Immersal's Collaboration with Major League Baseball</h3><p>In a groundbreaking initiative in July 2023, Immersal, in collaboration with Nexus Studios, T-Mobile, and Major League Baseball (MLB), unveiled a cutting-edge AR experience during the T-Mobile Home Run Derby and the MLB All-Star Game. This innovative venture brought to life the MLB Next app, powered by Immersal's advanced VPS. The app offered spectators an immersive experience with live vivid 3D visuals and up-to-the-minute data, significantly elevating the traditional sports viewing experience.</p><p>This AR application successfully overcame various environmental hurdles to provide an engaging and interactive platform for fans. The real-time data and striking 3D graphics not only enhanced viewer engagement but also brought a new dimension to the understanding of the game's intricacies.</p><p dir="ltr" id="isPasted">In recognition of this technological marvel, we are thrilled to share that the <a href="https://www.sportsbusinessjournal.com/Articles/2023/12/11/sports-business-awards-tech.aspx" target="_blank">MLB Next App has been shortlisted for the prestigious Sports Business Awards in the “Best in Extended Reality” category.</a> This nomination underscores the app's significant contribution to enhancing sports technology and fan experience, setting a new benchmark in the sports entertainment industry.</p><hr><h2>What will you build?</h2><p>Whether you're building an AR experience for a large commercial space, a factory, or an industrial setting, or even if you'd just like to create an AR app for fun, Immersal is the way to do it.</p><p>Immersal has released its SDK 2.0 and new Mapper App.&nbsp;</p><p><a href="https://developers.immersal.com/?WEVOLVER=" target="_blank" rel="noopener noreferrer" class="button-main-action">Sign up for your free account here.</a></p><hr><h2>Advancements in Athlete Training with AR</h2><p>AR is revolutionizing athlete training by equipping coaches with sophisticated, real-time analytics. This cutting-edge technology facilitates more targeted and effective training approaches. Through AR, coaches and athletes can access a wealth of data, including biomechanical analysis and performance metrics, right at their fingertips. This level of detailed feedback allows for a more nuanced understanding of an athlete's strengths and areas for improvement, leading to tailored training programs that maximize efficiency and effectiveness.</p><p>A prime example of AR's impact in sports training is the case of renowned long-distance runner Eliud Kipchoge. By leveraging AR-driven data, Kipchoge was able to fine-tune his training regimen, focusing on specific aspects of his performance. This personalized approach, enriched with detailed AR analytics, played a crucial role in his outstanding achievement at the 2016 Olympic Games. Kipchoge's success story not only highlights the potential of AR in elevating athletic performance but also serves as an inspiration for how technology can be harnessed to push the boundaries of human potential in sports.</p><h2 dir="ltr" id="isPasted">Transforming Fan Engagement with Gamified AR Experiences</h2><p dir="ltr">In the dynamic realm of sports and entertainment, AR is a game-changer, creating captivating fan experiences through innovations like virtual stadium tours and interactive games. AR allows fans to delve into additional game insights and unique perspectives right from their devices. Significantly, the integration of volumetric video into AR environments is enhancing this experience, bringing a rich, three-dimensional depth to content, making virtual interactions with players and mascots astonishingly realistic and immersive.</p><p dir="ltr">At home, AR transforms the viewing experience, enlivening broadcasts with elements like team mascots and interactive ads. These mascots, once just a part of the field, now come to life in 3D, bridging the gap between the stadium and the living room. This not only adds a fun, interactive twist to home viewing but also leverages the power of volumetric video to make these virtual interactions incredibly lifelike. Furthermore, AR's application in advertising offers interactive, engaging commercial experiences, heightening the overall game-watching experience. These advancements in AR, augmented by the realism of volumetric video, are reshaping fan engagement, turning passive viewing into an immersive, interactive event.</p><h3 dir="ltr">AR in Live Music Events</h3><p dir="ltr">Augmented Reality is swiftly becoming an essential element in live music events, enhancing concerts with immersive, interactive experiences. While AR has been part of concerts for some time, its role is growing more significant and transformative. A notable example is Childish Gambino's innovative use of an AR app during his <a href="https://www.youtube.com/watch?v=kY1Mr0b_I4Y" target="_blank">2019 Coachella performance</a>, which added a new layer of engagement for the audience.</p><p dir="ltr">In a recent landmark event, the popular rock band UVERworld performed at Yokohama Nissan Stadium on July 29-30, 2023, to an enthusiastic crowd of 70,000 fans. In a collaborative effort, Immersal and Realize Mobile Communications crafted a unique AR experience for this concert. Utilizing the technology, around <a href="https://youtu.be/Fgxj19pEaqg" target="_blank">20,000 people sent messages from across the stadium</a>, which were then visually aggregated to form UVERworld's iconic logo. This interactive feature not only fostered a communal atmosphere among the audience but also visually encapsulated their emotions and passion, elevating the collective experience. Remarkably, over 7,000 individuals engaged simultaneously in this large-scale, real-time social interaction, showcasing the immense potential of AR in bringing together and amplifying the concert-going experience.<a href="https://youtu.be/Fgxj19pEaqg?feature=shared" target="_blank"></a></p><h2 dir="ltr"><a href="https://youtu.be/Fgxj19pEaqg?feature=shared" target="_blank"><span contenteditable="false" draggable="true" class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable"><iframe width="640" height="360" src="https://www.youtube.com/embed/Pf2KWpmqgnY?&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span><br></a>Revolutionizing Theme Park Adventures with AR</h2><p>Theme parks are universally adored for their thrills and excitement, but long lines can dampen the enthusiasm. AR emerges as a game-changer in this context, significantly improving the visitor experience. AR is adept at providing real-time estimates of ride wait times, ensuring that guests can manage their time more efficiently. Through strategically placed AR markers and location-based triggers, the physical environment of the park is enriched with interactive virtual content.</p><p>Advanced VPS technologies, like those developed by Immersal, offer a plethora of enhancements to the theme park experience:</p><p><strong>Interactive Maps and Customized Routes:&nbsp;</strong>These features cater to individual preferences, allowing visitors to plot their journey based on their interests and the attractions they wish to see.</p><p><strong>Virtual Walls:&nbsp;</strong>An innovative way for visitors to interact with the park, where they can contribute to a virtual mural at iconic landmarks. A notable example is what Immersal did at Disney Hong Kong in 2022.&nbsp;</p><p dir="ltr"><strong>Digital Signage:</strong> This offers updated information about essential aspects like ride wait times, show schedules, menus, and more, all in a virtual format that's easy to access and understand.</p><p><strong>Wayfinding:&nbsp;</strong>Combining VPS with smartphone sensors and computer vision, systems like Immersal's provide real-time navigation help, guiding visitors with vibrant virtual arrows and cues on the ground.</p><p><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzA3NDg1OTE1MzU4LWdhbWUgaW1tZXJzYWwuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib"></p><p dir="ltr" id="isPasted"><strong>Friend Finding:&nbsp;</strong>This useful feature assists in locating and navigating to friends or family members if you get separated, enhancing the social aspect of the park experience.</p><p dir="ltr">AR in theme parks transcends mere entertainment; it opens doors to personalized and interactive experiences. It encourages visitors to engage more deeply with attractions and characters, enriching their overall experience.</p><p><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzA4MzQ3ODk0OTU0LWhrZGxfd29ybGQgb2YgZnJvemVuXzEwbSBnaWFudCBvLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib"></p><p dir="ltr" id="isPasted">To illustrate, Disneyland Hong Kong's "World of Frozen" campaign celebrated the new attraction's opening by featuring Olaf, the beloved snowman, in various forms across East Asia. These included a massive 10-meter installation, appearances in shopping malls, and <a href="https://www.hongkongdisneyland.com/worldoffrozeninvitation/index.html" target="_blank">innovative augmented reality experiences</a>, showcasing the diverse applications of AR in enhancing theme park adventures.</p><h2>Conclusion</h2><p>Augmented Reality, enhanced by Visual Positioning Systems, is revolutionizing the entertainment landscape, merging digital innovation with real-world experiences. This technology has transformed sports, offering fans immersive, interactive experiences and providing coaches with in-depth performance analytics. In the music industry, AR has elevated concerts into engaging, social events, fostering a deeper connection between artists and audiences. Additionally, theme parks are utilizing AR to personalize visitor experiences and streamline navigation, making every trip more engaging and efficient. As AR continues to evolve, it's redefining entertainment, moving it from passive observation to active, immersive participation, blurring the lines between reality and digital realms in exciting new ways.</p><h2 id="isPasted">What will you build?</h2><p>Whether you're building an AR experience for a large commercial space, a factory, or an industrial setting, or even if you'd just like to create an AR app for fun, Immersal is the way to do it.</p><p>Immersal has released its SDK 2.0 and new Mapper App.&nbsp;</p><p><a href="https://developers.immersal.com/?WEVOLVER=" target="_blank" rel="noopener noreferrer" class="button-main-action">SIGN UP FOR YOUR FREE ACCOUNT HERE.</a></p>

Augmented Reality (AR) and Visual Positioning Systems (VPS) are transforming live entertainment by blending the physical and digital realms to offer immersive, interactive, and personalized experiences across sports, music, and leisure activities.

Augmented Reality: The Transformative Impact of AR in Entertainment

<p dir="ltr" id="isPasted">In the realm of digital immersive experiences, advancements in VR (Virtual Reality) and AR (Augmented Reality) technologies are rapidly evolving. At the heart of this evolution is Finland-based Immersal, a company that's pioneering in the Visual Positioning System (VPS) domain. Thanks to Immersal's cutting-edge spatial mapping and visual positioning technologies, this innovation is revolutionizing how digital and physical realities merge.</p><p dir="ltr">Immersal's VPS was born out of a practical need encountered during an AR project, where the team faced the limitations of existing positioning systems. Their response was to develop a VPS offering centimeter-level accuracy, enabling persistent, shared multiuser AR experiences. They recognized the inadequacies of GPS (Global Positioning Satellites) for AR applications and responded with a VPS that transcends these limitations, enhancing both indoor and outdoor location accuracy and visual mapping. This technological advancement is not just a solution to current challenges but a gateway to unexplored possibilities in augmented reality, paving the way for innovative applications across diverse fields.</p><h2 dir="ltr" id="isPasted">What is VPS?</h2><h3 dir="ltr">Difference Between GPS and VPS Technologies</h3><p dir="ltr" id="isPasted">The main difference between GPS and VPS is the way they determine location. GPS relies on satellite signals and is designed primarily for outdoor use. It provides global coverage but often with limited accuracy. VPS, in contrast, uses computer vision, cameras, and visual data to determine the precise location and orientation of a device in real-time indoors and outdoors, even underground.</p><p dir="ltr"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzA3NDExNTA0MDYzLXZwcy5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"></p><h3 dir="ltr" id="isPasted">What problems does VPS solve?</h3><p>VPS solves critical navigation challenges in areas where GPS is limited, such as indoors or in dense urban environments. It enables precise location tracking and orientation within complex structures like malls or office buildings. VPS significantly enhances AR applications by accurately merging digital and physical worlds. This technology also streamlines asset tracking in challenging environments, offering reliable location data where traditional GPS-based solutions are ineffective.</p><h2 dir="ltr" id="isPasted">When to use VPS for AR applications&nbsp;</h2><p dir="ltr">VPS is a viable option for AR applications that require any of the following attributes:</p><ul><li dir="ltr"><p dir="ltr">both indoor and outdoor positioning</p></li><li dir="ltr"><p dir="ltr">high-precision positioning</p></li><li dir="ltr"><p dir="ltr">device orientation in addition to position and a full 6 degrees of freedom pose</p></li><li dir="ltr"><p dir="ltr">multi-user, shared AR world</p></li><li dir="ltr"><p dir="ltr">persistent virtual content, objects that are tied to the physical world</p></li></ul><p dir="ltr">The versatility and functionality of VPS makes it an excellent choice for applications such as indoor navigation guides; for example, helping a user locate their airport departure gate. Another use case could be virtual content in an art museum, where the story and history behind an art piece could be tied to the art itself, and brought to life with visually pleasing 3D graphics and spatial audio.</p><hr><h1 id="isPasted">What will you build?</h1><p>Whether you're building an AR experience for a large commercial space, a factory, or an industrial setting, or even if you'd just like to create an AR app for fun, Immersal is the way to do it.</p><p>Immersal has released its SDK 2.0 and new Mapper App.&nbsp;</p><p><a href="https://developers.immersal.com/?WEVOLVER=" target="_blank" rel="noopener noreferrer" class="button-main-action">Sign up for your free account here.</a></p><hr><h2>Use Cases of VPS Technology</h2><p dir="ltr">Some common use cases for VPS include augmented reality-based entertainment and guided navigation at theme parks, fan engagement experiences, urban planning, and robot navigation. Sports fans are also looking forward to more immersive experiences and real-time analytics that can easily be displayed in AR when watching a game at the stadium.</p><h3 dir="ltr">Theme parks</h3><p dir="ltr">VPS revolutionizes the theme park experience by enhancing navigation and interaction within these expansive, often crowded environments. VPS aids visitors in pinpointing their exact location and navigating efficiently to attractions, reducing the time spent on orientation. It enriches the park experience through AR features, such as interactive guides and virtual queues, blending the digital with the physical for a more immersive visit. Additionally, VPS technology assists park management in crowd control and operational efficiency, ensuring a smoother, more enjoyable experience for all guests.</p><p dir="ltr">Industrial Settings</p><p dir="ltr">In industrial environments, VPS facilitates accurate indoor navigation, guiding workers through complex facilities like warehouses or manufacturing plants. It enables precise location tracking of machinery, equipment, and inventory, streamlining logistics and inventory management. For maintenance and safety, it provides real-time, site-specific information, helping to identify potential hazards and maintenance needs. Additionally, in AR applications, VPS assists in training and simulations, providing workers with an interactive, detailed view of machinery and processes, thus reducing errors and improving overall operational efficiency.</p><h3 dir="ltr">Wayfinding</h3><p dir="ltr">VPS can be used in complex indoor environments like shopping malls, airports, and museums by displaying easy-to-follow AR navigation guides to customers. VPS accurately locates visitors and offers augmented reality (AR) navigation to guide them effortlessly to their destinations, a task beyond the capabilities of traditional GPS.</p><p dir="ltr"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzA3NDExNzM0ODE3LXdheWZpbmRpbmcuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib"></p><h3 dir="ltr" id="isPasted">Fan engagement at live events</h3><p dir="ltr">VPS can elevate fan engagement at live events by providing immersive, interactive experiences. Fans can navigate sprawling event spaces with ease, using AR for real-time directions to seats, amenities, or points of interest. It also enables personalized content delivery, such as AR-enhanced replays and player stats, directly to fans' devices, creating a more engaging and memorable event experience. Beyond the actual game, AR can also be applied to immersive virtual merchandise stores, allowing fans to preview and purchase products virtually, and enhancing the overall shopping experience. Sponsorships, half-time ads, and bringing mascots to life are a few other opportunities to elevate the in-stadium fan experience with VPS-based AR.</p><h3 dir="ltr" id="isPasted">Urban planning</h3><p dir="ltr">VPS offers transformative capabilities in urban planning by enabling precise mapping and analysis of city landscapes. Planners can use VPS to visualize proposed developments in real-time, overlaying AR models onto existing urban environments. This technology aids in assessing the potential impact of new structures on the surrounding area, enhancing decision-making processes. Additionally, it can assist in efficient space utilization and public infrastructure planning, ensuring more accurate and sustainable urban development.</p><p dir="ltr"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzA3NDExOTI2NjQzLWJ1aWxkLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib"></p><h3 dir="ltr" id="isPasted">Asset tracking</h3><p dir="ltr">VPS greatly enhances asset tracking, especially in complex environments like large warehouses or manufacturing plants. By providing precise, real-time location data, it enables companies to monitor and manage the movement of assets effortlessly. This accuracy ensures more efficient inventory management, reduces the likelihood of misplacement, and speeds up retrieval processes. VPS's detailed spatial awareness also aids in optimizing asset layout and logistics, leading to improved operational efficiency and cost savings.</p><p dir="ltr"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzA3NDExOTc3NTI3LXdhcmVob3VzZS5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"></p><h3 dir="ltr" id="isPasted">WebAR</h3><p dir="ltr">VPS significantly boost the capabilities of WebAR, offering a more immersive and accurate augmented reality experience directly through web browsers. VPS enhances WebAR by providing precise spatial awareness and location tracking, allowing for more realistic and contextually relevant AR overlays on physical environments. The combination of VPS with WebAR paves the way for advanced applications in advertising, education, and entertainment, making these experiences more intuitive and seamlessly integrated into the physical world.</p><h3 dir="ltr">Robot navigation</h3><p dir="ltr">VPS can enhance robot navigation by providing precise location and orientation data, essential for autonomous movement in GPS-challenged environments like indoors or urban areas. This advancement allows robots to navigate efficiently and safely, avoiding obstacles and adapting to changing surroundings which is crucial in industrial warehouses to public spaces and healthcare facilities.</p><h2 dir="ltr">Conclusion</h2><p dir="ltr">The adoption of Visual Positioning Systems (VPS) in augmented reality marks a significant leap in merging digital and physical realms. Spearheaded by companies like Immersal, VPS transcends GPS limitations by offering unprecedented accuracy in spatial mapping and visual positioning, vital for both indoor and outdoor environments. This technology enhances AR applications across various domains, from improving visitor experiences in theme parks and aiding complex indoor navigation to transforming urban planning and streamlining industrial processes. The integration of VPS in AR is not just an enhancement of current capabilities; it's a gateway to unexplored possibilities, paving the way for a future where augmented reality is seamlessly woven into the fabric of daily life.</p><hr><h2>What will you build?</h2><h5>Whether you're building an AR experience for a large commercial space, a factory or industrial setting, or even if you'd just like to create an AR app for fun, Immersal is the way to do it.</h5><p>Immersal has just released their SDK 2.0 and new Mapper App, you can sign up for your free account <a href="https://immersal.com/register?WEVOLVER" target="_blank">here</a>.&nbsp;</p><p><a href="https://developers.immersal.com/?WEVOLVER=" target="_blank" rel="noopener noreferrer" class="button-main-action">Sign up for free here.&nbsp;</a><br><br></p>

Since its inception augmented and virtual reality has vastly evolved, in part thanks to advancements like Immersal's innovation in visual positioning to enhance AR by outpacing GPS limitations.

Unlocking the Potential of VPS in Augmented Reality

In this episode, we delve into the cognitive strategies employed by researchers at EPFL to augment the human body with an additional robotic arm and learn about the profound impact of cognitive enhancements on the integration of advanced robotic limbs

Podcast: Why Robotic Limbs Are Critical To The Human Race

<p id="isPasted">This article was discussed in our <a href="https://www.wevolver.com/profile/the.next.byte" rel="noopener noreferrer" target="_blank">Next Byte podcast</a>.&nbsp;</p><p><span contenteditable="false" draggable="true" class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable"><iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/85f51311-32bd-4a6e-953c-0d6777ee40fc?dark=false" class="fr-draggable" data-gtm-yt-inspected-9="true" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span><br></p><p>The full article will continue below.</p><hr><p id="isPasted">Neuroengineer Silvestro Micera develops advanced technological solutions to help people regain sensory and motor functions that have been lost due to traumatic events or neurological disorders. Until now, he had never before worked on enhancing the human body and cognition with the help of technology.</p><p><span contenteditable="false" draggable="true" class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable"><iframe width="640" height="360" src="https://www.youtube.com/embed/w78rRbaH7Wg?&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-gtm-yt-inspected-8="true" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true" data-gtm-yt-inspected-9="true" id="720627293" title="Cognitive strategies for augmenting the body with a wearable, robotic arm"><span class="fr-mk" style="display: none;">&nbsp;</span></iframe></span></p><p>Now in a study published in Science Robotics, Micera and his team report on how diaphragm movement can be monitored for successful control of an extra arm, essentially augmenting a healthy individual with a third – robotic – arm.</p><p>“This study opens up new and exciting opportunities, showing that extra arms can be extensively controlled and that simultaneous control with both natural arms is possible,” says Micera, Bertarelli Foundation Chair in Translational Neuroengineering at EPFL, and professor of Bioelectronics at Scuola Superiore Sant’Anna.</p><p>The study is part of the <a href="https://nccr-robotics.ch/research/wearable-robotics/third-arm/" target="_blank" rel="noopener">Third-Arm project</a>, previously funded by the Swiss National Science Foundation (NCCR Robotics), that aims to provide a wearable robotic arm to assist in daily tasks or to help in search and rescue. Micera believes that exploring the cognitive limitations of third-arm control may actually provide gateways towards better understanding of the human brain.</p><p>Micera continues, “The main motivation of this third arm control is to understand the nervous system. If you challenge the brain to do something that is completely new, you can learn if the brain has the capacity to do it and if it’s possible to facilitate this learning. We can then transfer this knowledge to develop, for example, assistive devices for people with disabilities, or rehabilitation protocols after stroke.”</p><p>“We want to understand if our brains are hardwired to control what nature has given us, and we’ve shown that the human brain can adapt to coordinate new limbs in tandem with our biological ones,” explains Solaiman Shokur, co-PI of the study and EPFL Senior Scientist at the Neuro-X Institute. “It’s about acquiring new motor functions, enhancement beyond the existing functions of a given user, be it a healthy individual or a disabled one. From a nervous system perspective, it’s a continuum between rehabilitation and augmentation.”</p><p>To explore the cognitive constraints of augmentation, the researchers first built a virtual environment to test a healthy user’s capacity to control a virtual arm using movement of his or her diaphragm. They found that diaphragm control does not interfere with actions like controlling one’s physiological arms, one’s speech or gaze.</p><p>In this virtual reality setup, the user is equipped with a belt that measures diaphragm movement. Wearing a virtual reality headset, the user sees three arms: the right arm and hand, the left arm and hand, and a third arm between the two with a symmetric, six-fingered hand.</p><p>“We made this hand symmetric to avoid any bias towards either the left or the right hand,” explains Giulia Dominijanni, PhD student at EPFL’s Neuro-X Institute.</p><p>In the virtual environment, the user is then prompted to reach out with either the left hand, the right hand, or in the middle with the symmetric hand. In the real environment, the user holds onto an exoskeleton with both arms, which allows for control of the virtual left and right arms. Movement detected by the belt around the diaphragm is used for controlling the virtual middle, symmetric arm. The setup was tested on 61 healthy subjects in over 150 sessions.</p><p>“Diaphragm control of the third arm is actually very intuitive, with participants learning to control the extra limb very quickly,” explains Dominijanni. “Moreover, our control strategy is inherently independent from the biological limbs and we show that diaphragm control does not impact a user’s ability to speak coherently.”</p><p>The researchers also successfully tested diaphragm control with an actual robotic arm, a simplified one that consists of a rod that can be extended out, and back in. When the user contracts the diaphragm, the rod is extended out. In an experiment similar to the VR environment, the user is asked to reach and hover over target circles with her left or right hand, or with the robotic rod.</p><p>Besides the diaphragm, but not reported in the study, vestigial ear muscles have also been tested for feasibility in performing new tasks. In this approach, a user is equipped with ear sensors and trained to use fine ear muscle movement to control the displacement of a computer mouse.</p><p>“Users could potentially use these ear muscles to control an extra limb,” says Shokur, emphasizing that these alternative control strategies may help one day for the development of rehabilitation protocols for people with motor deficiencies.</p><p>Part of the third arm project, previous studies regarding the <a href="https://actu.epfl.ch/news/a-smart-artificial-hand-for-amputees-merges-user-a/" target="_blank">control of robotic arms</a> have been focused on helping amputees. The latest Science Robotics study is a step beyond repairing the human body towards augmentation.</p><p>“Our next step is to explore the use of more complex robotic devices using our various control strategies, to perform real-life tasks, both inside and outside of the laboratory. Only then will we be able to grasp the real potential of this approach,” concludes Micera.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzAyNTY4NDIyNzU0LTgxOTJ4NTQ2NC5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner">Martina Gini controls a simplified robotic arm with breathing. © 2023 EPFL / Alain Herzog, CC-BY-SA</span></span></span></p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzAyNTY4NTQ4Mjg5LWV6Z2lmLTUtNDVjNDI4M2Y2Zi5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner">Silvestro Micera (left) and Leonardo Pollina (right) prepare for the third arm virtual reality experiment. © 2023 EPFL / Alain Herzog, CC-BY-SA</span></span></span></p>

EPFL scientists show that breathing may be used to control a wearable extra robotic arm in healthy individuals, without hindering control of other parts of the body.

Cognitive strategies to augment the body with an extra robotic arm

<div itemprop="description" id="isPasted"><p>Clinical surgery grequires great precision, years of experience, specialised knowledge and the highest level of concentration on the part of the operating surgeons. A current research project aims to provide them with an interactive tool to improve precision and performance. A team from the Embedded Intelligence research department at DFKI is working with doctors at Klinikum Saarbrücken to to develop new methods for AR-based assistance systems to support operations on the pancreas.</p></div><p id="isPasted">The assistance system was developed specifically for the visualisation of vital anatomical structures during the operation on the patient, using a mobile AR system. With the help of special glasses (HoloLens), the information is displayed in the surgeon's real field of vision. Anatomical features are used as reference points to position the 3D model precisely on the patient.</p><h2><strong>Evaluation in clinical practice</strong></h2><p>The DFKI researchers worked closely with the team of Gregor Stavrou, Chief of General and Visceral Surgery, Department of Surgical Oncology at the Saarbrücken Klinikum. Thanks to the cooperation with the hospital, the applications can be evaluated in real surgery and important research data can be collected through observations during the operation and post-hoc interviews after the procedures. "In this ecologically highly valid environment, such insights into practice are invaluable," says project manager Hamraz Javaheri, Researcher at DFKI's Embedded Intelligence Department.</p><p>The prototype was tested in a patient study and proved successful in registering and visualizing the 3D reconstructed model of the patient's vital anatomical structure and thereby supporting surgeons in decision making during surgery. The assistance system has already been&nbsp;<a href="https://www.saarbruecker-zeitung.de/saarland/saarbruecken-winterberg-chirurg-rettet-frau-mit-seltener-op-methode-das-leben-v27_aid-99816935%22%3Evon" title="Stavrou" target="Dr." rel="noreferrer">used successfully by Dr. Gregor Stavrou during an extremely difficult operation</a> to remove a malignant tumor using the so-called "Appleby procedure", in which a main artery had to be rerouted, among other things. It also has great potential for medical training and surgery preparation.</p><h2><strong>HumanE-AI-Net - human-centered AI in the European network</strong></h2><p>The project is being carried out within the framework of the European network of excellence HumanAINet (and is funded by it). The system is currently being successively improved. The valuable findings from studies done by DFKI researchers will be shared with the research community in upcoming scientific articles.</p>

Lead surgeon using application pre-operatively for planning

A team from the Embedded Intelligence research department at DFKI is working with doctors at Klinikum Saarbrücken to to develop new methods for AR-based assistance systems to support operations on the pancreas.

Artificial Intelligence in Surgery: Augmented Reality provides Support in the Operating Room

<p dir="ltr" id="isPasted">As technology continues to advance, Wi-Fi has remained a key component in research and industrial manufacturing. With the introduction of Wi-Fi 7, organizations now have the opportunity to improve their value propositions and provide customers with an enhanced experience. This new wireless communication technology offers substantial benefits for those looking to stay ahead in the game.</p><h2 dir="ltr">The Journey to Wi-Fi 7</h2><p dir="ltr">Wi-Fi, which stands for wireless fidelity, was developed by an organization called the Institute of Electrical and Electronics Engineers (IEEE), who sets standards for the Wi-Fi standard. IEEE 802.11be, dubbed Extremely High Throughput (EHT), is the next amendment of the IEEE 802.11 standard, which will be designated Wi-Fi 7. IEEE 802.11be is still in the development stage, and it is expected that the specifications will be officially released in late 2024 or early 2025.&nbsp;</p><p dir="ltr">The IEEE 802.11be aims to define Media Access Control (MAC) and Physical Layer (PHY) features that are customized to meet the demanding requirements of high-throughput and low-latency applications. The PHY component of IEEE 802.11 is expected to be similar to the one outlined in IEEE 802.11ax standard. For instance, the IEEE 802.11be will incorporate support for technologies like multi-user multiple-input and multiple-output (MU-MIMO), orthogonal frequency division multiple access (OFDMA), and longer symbol durations with smaller subcarrier spacing.&nbsp;</p><h2 dir="ltr">Technical Specifications</h2><p dir="ltr">Recent reports indicate that Wi-Fi 7 has achieved a remarkable increase in wireless data link performance, nearly quadrupling the capabilities of its previous generation, offering a throughput exceeding 36 Gbps. To attain this impressive transfer rate, the IEEE 802.11be standard introduces a range of technological advancements, such as wider channels and higher quadrature amplitude modulation (QAM), that exhibit substantial improvements over its predecessor.&nbsp;</p><p dir="ltr">In Wi-Fi 6, the utilization of OFDMA involved breaking down radio channels into smaller frequency allocations known as Resource Units (RUs). This approach enabled Wi-Fi devices to send smaller data packets to multiple users concurrently, enhancing both throughput and reducing latency. Wi-Fi 7 builds upon this foundational OFDMA concept but introduces the notion of Multiple RUs (MRU), which allows for combinations of different RU sizes.</p><p dir="ltr">The enhancements introduced through MRU have led to a significant reduction in latency, especially when multiple users are attempting to transmit data simultaneously. This improvement is achieved by allocating RUs more efficiently, taking into account the varying data lengths of different users. For example, in a scenario where four users with different data lengths are transmitting data, a Wi-Fi 7 access point (AP) equipped with MRU can achieve the shortest end-to-end latency compared to previous-generation APs, resulting in a 33 percent reduction in latency compared to Wi-Fi 5 and a 25 percent reduction compared to Wi-Fi 6.&nbsp;<iframe id="652e6bda405966a620c0bedf" width="250" class="specs-iframe" height="440" src="https://wevolver.com/iframe/specs/nordic-semiconductor-nrf7002-dk-wi-fi-6-development-kit?iframe=true" frameborder="0" scrolling="false" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe></p><h2 dir="ltr">How Does Wi-Fi 7 Achieve High Throughput?</h2><p dir="ltr">Wi-Fi 6 operated across two frequency bands—2.4 GHz and 5 GHz. In practical usage, only a portion of the available frequencies within each band were utilized, such as 80 MHz in the 2.4 GHz band and 600 MHz in the 5 GHz band. These specific sets of frequencies are referred to as the spectrum, and their actual size can <a href="https://www.lairdconnect.com/support/faqs/what-channels-are-supported-both-24-ghz-and-5ghz-band-most-countries" target="_blank">vary</a> between different countries.</p><h3 dir="ltr">Wider Channels</h3><p dir="ltr">While the previous generation of Wi-Fi could simultaneously handle 160 MHz channels, Wi-Fi 7 introduces a substantial improvement with ultra-wide 320 MHz channels. This advancement translates to a doubled data transmission capacity compared to the previous Wi-Fi 6 and 6E technologies. The broader channel not only minimizes latency but also enhances the overall transmission rate. These 320 MHz channels are situated in the relatively open 6 GHz band, steering clear of the frequently congested wireless transmission protocol bands. Consequently, this configuration reduces interference and ensures a more stable signal.</p><h3 dir="ltr">4K QAM for Higher Data Rate</h3><p dir="ltr">To leverage frequencies for faster computer speeds, Wi-Fi 6 uses a technique called quadrature amplitude modulation (QAM), supporting 1024 QAM. This means it can extract 1024 signals from these frequencies. The greater the number of signals, the more data bits it can carry, resulting in increased throughput. However, a choice must be made regarding the Wi-Fi channel within one of the frequency bands. For instance, in the 2.4 GHz band, a channel width of 20 MHz is typically used, and a broader channel width leads to higher throughput.</p><p dir="ltr">Comparing Wi-Fi 7 to its predecessors reveals significant advancements. Wi-Fi 7 boasts a data rate of 9.6 Gbps, a 39 percent increase compared to the previous generation, primarily due to the jump in QAM from 1024 QAM in Wi-Fi 6. Now, with the advent of Wi-Fi 7, even more enhancements have been introduced. The channel width has doubled, moving from 160 MHz to 320 MHz, effectively doubling the data rate. Additionally, the QAM has seen an improvement, increasing from 1024 QAM in Wi-Fi 6 to 4096 QAM, contributing an additional 20 percent to the data rate.</p><h3 dir="ltr">Improvements in MIMO</h3><p dir="ltr">Wi-Fi achieves data transmission between devices through specific Wi-Fi channels. To further enhance throughput, it utilizes a technique known as spatial streams, also referred to as MIMO. The Wi-Fi 7 standard expands the count of multi-user MIMO streams to 16. This increase in available MIMO streams, coupled with the capability to automatically select the most suitable stream for any given task, will result in a reduction in latency.&nbsp;</p><h3 dir="ltr">MLO for Wider Channels</h3><p dir="ltr">Wi-Fi 7 introduces enhancements to Multi-Link Operation (MLO). MLO enables the simultaneous utilization of multiple bands or channels by Multi-Link Devices. Embracing MLO provides advantages such as load balancing, data aggregation, and reduced latency. When employing the same modulation and coding scheme rate, Wi-Fi 7 with MLO delivers up to three times the effective throughput compared to Wi-Fi 6 with a single link. Furthermore, Wi-Fi 7 featuring MLO brings about an impressive 80 percent reduction in latency compared to the previous generation using a single link.</p><h2 dir="ltr">Why Use Wi-Fi 7 for Next-Gen Applications?</h2><h3 dir="ltr">Speed and Efficiency</h3><p dir="ltr">Wi-Fi 7 can leverage the capabilities of MLO to prioritize efficient data transmission, effectively managing network congestion and interference while optimizing the flow of data. With Wi-Fi 7 routers, dynamic channel selection becomes a key strategy to bypass congestion or interference, ensuring the smooth and reliable movement of data throughout the network.</p><h3 dir="ltr">Reliability</h3><p dir="ltr">In terms of reliability, Wi-Fi 7 elevates the&nbsp;dependability of wireless connections by maintaining a steady connection and mitigating interference, resulting in fewer disruptions and dropouts for users. This technology also addresses limitations associated with the coverage range of specific frequency bands. By intelligently selecting channels and optimizing transmission, it ensures that even in areas with limited signal coverage, connections remain resilient.</p><h3 dir="ltr">Adaptability</h3><p dir="ltr">One important feature of Wi-Fi 7 is its adaptability. By dynamically selecting the most appropriate channel according to current network conditions, it guarantees that the network can adapt to evolving environments and sustain peak performance. This adaptability inherent in Wi-Fi 7 not only ensures resilience but also future-proofs networks. As new devices and technologies come into play, Wi-Fi 7 can accommodate them by adeptly allocating resources and enhancing performance.</p><h2 dir="ltr">Potential Use Cases for Wi-Fi 7</h2><h3 dir="ltr">Real-Time Applications</h3><h4 dir="ltr">4K and 8K Streaming&nbsp;</h4><p dir="ltr">Recognizing Wi-Fi as the cornerstone of advancing digital technologies like edge computing and industrial automation, the introduction of Wi-Fi 7 marks a significant expansion of real-time applications, including the handling of 4K and 8K video streams.</p><h4 dir="ltr">AR and VR&nbsp;</h4><p dir="ltr">Wi-Fi 7 equips users to immerse themselves in technologies like augmented reality (AR) and virtual reality (VR) that offer experiences remarkably close to reality. Thanks to its rapid data transfer speeds and minimal latency, Wi-Fi 7 enhances user experiences by facilitating uninterrupted, real-time interactions within virtual environments.</p><h4 dir="ltr">Cloud Gaming</h4><p dir="ltr">Leveraging MLO technology plays a crucial role in diminishing latency and enhancing stability, a vital factor for cloud gaming enthusiasts. The increased throughput capabilities of Wi-Fi 7 enable real-time communication with latency levels consistently below 5 milliseconds, ensuring a top-tier gaming experience. This translates to high-quality video streaming, elevating the overall quality of the cloud gaming experience.</p><h2 dir="ltr">Industrial Applications</h2><h3 dir="ltr">IoT in Manufacturing</h3><p dir="ltr">Wi-Fi 7 offers the capacity to connect an extensive array of sensors and edge devices essential in the context of industrial automation applications. This capability enables manufacturing companies to automate their operations, resulting in operational efficiency. These sensors can monitor equipment conditions, detect anomalies, and optimize production workflows, resulting in reduced downtime, minimized errors, and enhanced overall productivity.</p><h3 dir="ltr">Automated Systems</h3><p dir="ltr">Manufacturers implementing wireless sensing and control technologies in their factory automation systems gain operational and financial benefits through the adoption of Wi-Fi 7 technology. Wi-Fi 7 enhances the reliability and speed of wireless communication within factory automation systems. This translates into quicker response times, more precise control, and the ability to manage larger volumes of data efficiently.</p><h3 dir="ltr">Public and Business Environments&nbsp;</h3><h3 dir="ltr">Smart Cities</h3><p dir="ltr">Smart cities have an increasing need for high-speed internet connectivity, and Wi-Fi 7 stands out as a solution that significantly enhances connectivity, efficiency, performance, and security for a wide array of smart city applications. These applications include real-time traffic and public transportation system monitoring, as well as the delivery of high-definition video surveillance. By implementing Wi-Fi 7, smart cities can improve their infrastructure, enabling efficient real-time data transfer for critical functions like traffic management and public safety.</p><h3 dir="ltr">Large Venues Like Airports and Stadiums</h3><p dir="ltr">In locations requiring high-density wireless connectivity, such as airports and stadiums, Wi-Fi 7 emerges as a compelling solution due to its advanced features, including the wide 320 MHz channels and enhanced multi-link operation. Wi-Fi 7 has the capacity to accommodate numerous simultaneous connections, allowing travelers and sports enthusiasts to stay connected to the internet without interruptions.</p><h2 dir="ltr">Conclusion</h2><p dir="ltr">Over the past few years, emerging applications with heightened requirements for both high throughput and low latency have fueled the evolution of the IEEE 802.11 standard. Wi-Fi 7, as part of this ongoing development, introduces significant enhancements, primarily centered around the design of high-performance PHY and MAC protocols.</p><p dir="ltr">Furthermore, Wi-Fi 7 incorporates a range of advanced techniques, including Multiple RU utilization, 4096 QAM modulation, multi-link aggregation and operations, MIMO improvements, and coordination techniques across multiple APs. These collective innovations within the standard protocol are geared toward supporting applications such as cloud gaming, IoT in manufacturing, automated systems, and high-quality video streaming.&nbsp;</p><p dir="ltr"><a href="https://www.mouser.com/empowering-innovation/wifi7" target="_blank" rel="noopener noreferrer">Learn more at Mouser.com</a></p><h2 dir="ltr">References</h2><p dir="ltr">[1] Nokia. Wi-Fi 7: a quantum leap in throughput and features. Laszlo Gyalog. July 05, 2023. <a href="https://www.nokia.com/blog/wi-fi-7-a-quantum-leap-in-throughput-and-features/" target="_blank">https://www.nokia.com/blog/wi-fi-7-a-quantum-leap-in-throughput-and-features/</a>. Accessed on September 23, 2023.</p><p dir="ltr">[2] Mediatek. Key Advantages of Wi-Fi 7: Performance, MRU and MLU. White Paper. February 24, 2022. <a href="https://www.mediatek.com/blog/mediatek-wifi-7-whitepaper-detailing-key-advantages-performance-mru-and-mlo" target="_blank">https://www.mediatek.com/blog/mediatek-wifi-7-whitepaper-detailing-key-advantages-performance-mru-and-mlo</a>. Accessed on September 23, 2023.</p><p dir="ltr">[3] Wired. What is Wi-Fi 7? Everything You Need to Know. Simon Hill. December 29, 2022. <a href="https://www.wired.com/story/what-is-wi-fi-7/" target="_blank">https://www.wired.com/story/what-is-wi-fi-7/</a>. Accessed on September 23, 2023.</p><p dir="ltr">[4] Senza Fili. Wi-Fi 7: The next generation in the evolution of Wi-Fi. Monica Paolini. Tech Brief. <a href="https://6ghz.info/wp-content/uploads/2022/07/SenzaFili-IntelTB-Wi-Fi7-The-next-generation-in-the-evolution-of-Wi-Fi.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">https://6ghz.info/wp-content/uploads/2022/07/SenzaFili-IntelTB-Wi-Fi7-The-next-generation-in-the-evolution-of-Wi-Fi.pdf</a>. Accessed on September 23, 2023.</p><p dir="ltr">[5] Intel Corporation. Next-generation Wi-Fi: Wi-Fi 7 and beyond. Carlos Cordeiro. July 2020. <a href="https://www.intel.com/content/dam/www/public/us/en/documents/pdf/wi-fi-7-and-beyond.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">https://www.intel.com/content/dam/www/public/us/en/documents/pdf/wi-fi-7-and-beyond.pdf</a>. Accessed on September 23, 2023.</p><p dir="ltr">[6] Rohde &amp; Schwarz. IEEE 802.11be: Technology Introduction. Lisa Ward and Jörg Köpp. White Paper. <a href="https://scdn.rohde-schwarz.com/ur/pws/dl_downloads/premiumdownloads/premium_dl_brochures_and_datasheets/premium_dl_whitepaper/IEEE-802-11be-technology-introduction_wp_en_3683-4026-52_v0100.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">https://scdn.rohde-schwarz.com/ur/pws/dl_downloads/premiumdownloads/premium_dl_brochures_and_datasheets/premium_dl_whitepaper/IEEE-802-11be-technology-introduction_wp_en_3683-4026-52_v0100.pdf</a>. Accessed on September 23, 2023.</p>

The article discusses the Wi-Fi 7– with improved speed, low latency, and better reliability for your wireless connections.


A short guide to the next generation of wireless connectivity-Wi-Fi 7

<p id="isPasted">For retailers, selling commodities, apparel, home appliances, etc., the amount of consumer information you collect greatly affects sales. Unless you accurately assess what customers want now and stock up on inventory during the optimal period when they are required, in the optimal quantities and at the lowest possible cost, you cannot grow your revenue. To succeed in the retail industry, it is an absolute requirement that you develop the ability to discern customer needs.</p><p>The retail industry has a long history of continuously developing technologies and business methods for handling customers and products, developing distribution strategies, and managing various other types of information. The industry has always adopted the latest up-to-date information processing technology. Here, we explain the movement called “retailtech,” which develops and uses the latest information technologies and has become widely used in the retail industry.</p><h2>Individually supporting diversifying consumer needs with digital technologies</h2><p>Retailtech is a term that combines “retail,” representing the retail industry, and “technology.” It means the incorporation of artificial intelligence (AI), IoT, and other leading-edge IT technologies into the collection and analysis of customer information, the management of distribution and inventory, settlement, etc. to increase their effectiveness and efficiency. Broadly speaking, robots that help with sales, robots that pick products to be delivered at distribution centers, and the shift to cashless settlement, etc. are also included in retailtech.</p><p>Since the start of the 21st century, the environment surrounding the retail industry has dramatically changed.</p><p>First, the sense of values, interests, and preferences of consumers have diversified due to market globalization and the filter bubble effect<sup>*1</sup> due to the expanded use of search engines, social networking services, and other Internet environments. As a result, it is becoming necessary to not only to grasp what is selling in the consumer market overall, but also to discern what items individual customers need and sell such items to them separately.</p><p>*1 The filter bubble effect refers to the phenomenon in which consumers are only able to see the information that they wish to see on the Internet. By continuing to collect and accumulate only information that matches their preferences without coming into contact with information that does not match their sense of values, individual consumers strengthen the tendency to develop radical preferences that are biased in a specific field only, and it is believed that diversification increases as a result.</p><p>Moreover, the information for retail shops to gain insight into customers is being digitized, and the location of commerce is shifting from real sales floors to Internet sites, which is a significant change. The digitization of customer information that started a long time ago from POS systems<sup>*2</sup> and the point cards issued by mass-market retailers is further evolving with the current spread of online shopping. Now, artificial intelligence (AI) analyzes user registration information, their search history on the Internet, and on-site behavior history to discern individual interests and preferences, what products they currently want, and the amount they might be able to pay to purchase a product. Furthermore, retail methods that propose products that individual consumers might want at the optimal time have become commonplace.</p><p>*2 A POS (Point of Sales) system scans the barcode attached to a product when sales are made at the cash register on the sales floor to manage what products were sold when and where. It is used to collect information for effectively managing products, inventory, and customers.</p><p>The use of digital technologies is essential to individually responding to diversified consumer needs. If you are a skilled marketer, changes in market trends can be picked up from market information using your experience and sense. However, a person cannot make decisions about the individual needs and desired products for a massive number of consumers. Therefore, the use of AI and other forms of IT to replace the skills of a marketer and respond to individual consumers as needed becomes essential.</p><h2>Integrating brick-and-mortar stores and online stores in order not to miss a targeted customer</h2><p>In recent years, the movement to integrate online sales and product sales at brick-and-mortar stores by using the latest technologies is becoming apparent in the retail industry. This sales method, which collects customer information through multiple channels and aims to build a mechanism that can sell according to the method desired by each customer, is called “OMO” (Online Merges with Offline) (Figure 1).</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 768px;"><span class="fr-img-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1699267693485-various-tech-8-2en.png" style="width: 766px;" class="fr-fic fr-dib"><span class="fr-inner">Figure 1 Utilize IoT to integrate customer information from brick-and-mortar and online stores</span></span></span></p><p id="isPasted">Since the start of the 21st century, measures have rapidly developed to increase sales at online stores. However, brick-and-mortar stores and online stores each have their advantages and disadvantages. To increase the degree of satisfaction without missing a customer, it is more rational to separately use the characteristics of both types of stores according to customers’ needs. Therefore, the number of retail companies that are advancing attempts to utilize IoT to make it possible to collect the kind of detailed customer information that is collected online at brick-and-mortar stores is increasing in recent years.</p><p>Two technologies that are being utilized in various applications as a means to follow the movements of store customers and products within a brick-and-mortar store and collect customer information are image recognition and RFID<sup>*3</sup> tags (Figure 2).</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 768px;"><span class="fr-img-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1699267714710-various-tech-8-3.jpg" style="width: 766px;" class="fr-fic fr-dib"><span class="fr-inner">Figure 2 Collecting customer information with RFID, etc. and providing a high-quality customer experience with AR, etc. A product with an RFID tag attached (left) and utilizing AR to consider a furniture purchase (right)</span></span></span></p><p id="isPasted">*3 RFID (Radio Frequency Identification) is a technology that uses radio waves to read and write data without contact. By attaching a small semiconductor chip that stores an RFID mechanism and information as a tag to each individual product, it becomes possible to carry out fine-grained management of products by reading and writing information with RFID as needed.</p><p>Retail shops that have installed a large number of cameras, various sensors, microphones, and RFIDs, etc. and introduced IoT systems with AI that analyzes in real time what store customers picked up. The customer information that was collected with previous POS systems and point cards could not be collected unless a customer actually purchased a product. By utilizing such IoT systems, it becomes possible to know what products a store customer was comparing and what they ultimately bought.</p><p>If you utilize information collected in this way, you can take various measures such as sending special deal information to customer smartphone apps to give them a push, providing a reminder several days later about a product that they were interested in but did not purchase, optimizing the types, placement location, and flow lines of products displayed in the store in order not to miss customers. Moreover, if you install sensors on the product shelves and attach RFID tags to the products, you will become able to assess the product display conditions in real time to enable efficient sales that never miss a business opportunity through highly precise inventory management and just-in-time product replenishment.</p><p>In addition, it also became possible to identify the products to purchase and smoothly complete the checkout process just by placing the shopping basket in the designated location at the self-checkout terminal and scanning the RFID tags attached to the products. For store customers who created an account in advance at a cutting-edge store, it is even possible to automatically check out when carrying goods out of the store even if you do not pay at the register. With such mechanisms, the stress of waiting for the register will be eliminated.</p><h2>Providing a high-quality customer experience utilizing AR and VR</h2><p>Some retail stores are introducing systems that utilize augmented reality (AR) technology to provide assistance in selecting a product to purchase or to make accurate suggestions about how to use a product or what product to purchase on that occasion.</p><p>For example, there is a mass furniture retailer that provides an AR app that enables users to check on a smartphone or tablet device display what a piece of furniture they are thinking about purchasing would look like when actually placed in their own room. The impression given off by a piece of furniture changes considerably from the showroom to the location where it is actually placed. Depending on the situation, it may be too large to get into the room. By utilizing an AR app, you can avoid making the wrong choice and shop with a high degree of satisfaction. In the future, it might become possible to utilize virtual reality (VR) with haptic technology, etc. to check how a sofa feels, etc. online.</p><p>There are also supermarkets providing AR apps that can display the basic ingredients, place of production, allergens, etc. of products lined up in the food section. Recently, there are also stores creating systems that present cooking recipes using those ingredients or the seasonings needed to make a dish so that you can purchase them together.</p><p>In the retail industry, entertainment that moves people is required by sensitively perceiving the customer’s emotions and preparing the expected services in advance. All consumers want to enjoy their shopping. Going forward, more advanced forms of retailtech will appear one after another and likely provide new forms of enjoyment to consumers.</p><hr id="isPasted"><p><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1699267752774-1699267752774.png" class="fr-fic fr-dii"></p><p>If you have any questions based on this article that you would like to discuss with an expert at Murata, don't hesitate to reach out. There's an entire team ready to support you with your question.</p><p><a href="https://www.murata.com/en-eu/contactform?&Detail=Wevolver%20request" target="_blank" rel="nofollow" class="button-main-action">GET IN TOUCH</a></p>

Here, we explain the movement called “retailtech,” which develops and uses the latest information technologies and has become widely used in the retail industry.

Revolutionizing Retail: Creating Dynamic Customer Experiences through Advanced RetailTech

<p id="isPasted">To get a sense of what <a href="http://groups.csail.mit.edu/hcie/files/research-projects/structcode/2023-SCF-StructCode-paper.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">StructCode</a> is all about, says Mustafa Doğa Doğan, think of Superman. Not the “faster than a speeding bullet” and “more powerful than a locomotive” version, but a Superman, or Superwoman, who sees the world differently from ordinary mortals — someone who can look around a room and glean all kinds of information about ordinary objects that is not apparent to people with less penetrating faculties.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 768px;"><span class="fr-img-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1698219535969-StructCode5.png" style="width: 766px;" class="fr-fic fr-dib"><span class="fr-inner">“The goal is to change the way we interact with objects,” says MIT PhD student Mustafa Doğa Doğan, “by embedding information into objects in ways that can be readily accessed.” Image courtesy of the researchers/MIT CSAIL.</span></span></span></p><p>That, in a nutshell, is “the high-level idea behind StructCode,” explains Doğan, a PhD student in electrical engineering and computer science at MIT and an affiliate of the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL). “The goal is to change the way we interact with objects” — to make those interactions more meaningful and more meaning-laden — “by embedding information into objects in ways that can be readily accessed.”</p><p>StructCode grew out of an effort called InfraredTags, which Doğan and other colleagues introduced in 2022. That work, as well as the current project, was carried out in the laboratory of MIT Associate Professor Stefanie Mueller — Doğan’s advisor, who has taken part in both projects. In last year’s approach, “invisible” tags — that can only be seen with cameras capable of detecting infrared light — were used to reveal information about physical objects. The drawback there was that many cameras cannot perceive infrared light. Moreover, the method for fabricating these objects and affixing the tags to their surfaces relied on 3D printers, which tend to be very slow and often can only make objects that are small.</p><p><span contenteditable="false" draggable="true" class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable"><iframe width="640" height="360" src="https://www.youtube.com/embed/uZo7SyZV-gM?&amp;wmode=opaque&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span></p><p id="isPasted">StructCode, at least in its original version, relies on objects produced with laser-cutting techniques that can be manufactured within minutes, rather than the hours it might take on a 3D printer. Information can be extracted from these objects, moreover, with the RGB cameras that are commonly found in smartphones; the ability to operate in the infrared range of the spectrum is not required.</p><p>In their initial demonstrations of the idea, the MIT-led team decided to construct their objects out of wood, making pieces such as furniture, picture frames, flowerpots, or toys that are well suited to laser-cut fabrication. A key question that had to be resolved was this: How can information be stored in a way that is unobtrusive and durable, as compared to externally-attached bar codes and QR codes, and also will not undermine an object’s structural integrity?</p><p>The solution that the team has come up with, for now, is to rely on joints, which are ubiquitous in wooden objects made out of more than one component. Perhaps the most familiar is the finger joint, which has a kind of zigzag pattern whereby two wooden pieces adjoin at right angles such that every protruding “finger” along the joint of the first piece fits into a corresponding “gap” in the joint of the second piece and, similarly, every gap in the joint of the first piece is filled with a finger from the second.</p><p>“Joints have these repeating features, which are like repeating bits,” Dogan says. To create a code, the researchers slightly vary the length of the gaps or fingers. A standard size length is accorded a 1. A slightly shorter length is assigned a 0, and a slightly longer length is assigned a 2. The encoding scheme is based on the sequence of these numbers, or bits, that can be observed along a joint. For every string of four bits, there are 81 (3<sup>4</sup>) possible variations.</p><p>The team also demonstrated ways of encoding messages in “living hinges” — a kind of joint that is made by taking a flat, rigid piece of material and making it bendable by cutting a series of parallel, vertical lines. As with the finger joints, the distance between these lines can be varied: 1 being the standard length, 0 being a slightly shorter length, and 2 being slightly longer. And in this way, a code can be assembled from an object that contains a living hinge.</p><p>The idea is described in a paper, “<a href="http://groups.csail.mit.edu/hcie/files/research-projects/structcode/2023-SCF-StructCode-paper.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">StructCode: Leveraging Fabrication Artifacts to Store Data in Laser-Cut Objects</a>,” that was presented this month at the 2023 ACM Symposium on Computational Fabrication in New York City. Doğan, the paper’s first author, is joined by Mueller and four coauthors — recent MIT alumna Grace Tang ’23, MNG ’23; MIT undergraduate Richard Qi; University of California at Berkeley graduate student Vivian Hsinyueh Chan; and Cornell University Assistant Professor Thijs Roumen.</p><p>“In the realm of materials and design, there is often an inclination to associate novelty and innovation with entirely new materials or manufacturing techniques,” notes Elvin Karana, a professor of materials innovation and design at the Delft University of Technology. One of the things that impresses Karana most about StructCode is that it provides a novel means of storing data by “applying a commonly used technique like laser cutting and a material as ubiquitous as wood.”</p><p>The idea for StructCode, adds University of Colorado computer scientist Ellen Yi-Luen Do, “is “simple, elegant, and totally makes sense. It’s like having the Rosetta Stone to help decipher Egyptian hieroglyphs.”</p><p>Patrick Baudisch, a computer scientist at the Hasso Plattner Institute in Germany, views StructCode as “a great step forward for personal fabrication.&nbsp;It takes a key piece of functionality&nbsp;that's only offered today for mass-produced goods and brings it to custom objects."</p><p>Here, in brief, is how it works: First, a laser cutter — guided by a model created via StructCode — fabricates an object into which encoded information has been embedded. After downloading a StructCode app, an user can decode the hidden message by pointing a cellphone camera at the object, which can (aided by StructCode software) detect subtle variations in length found in an object’s outward-facing joints or living hinges.</p><p>The process is even easier if the user is equipped with augmented reality glasses, Doğan says. “In that case, you don’t need to point a camera. The information comes up automatically.” And that can give people more of the “superpowers” that the designers of StructCode hope to confer.</p><p>“The object doesn’t need to contain a lot of information,” Doğan adds. “Just enough — in the form of, say, URLs — to direct people to places they can find out what they need to know.”</p><p>Users might be sent to a website where they can obtain information about the object — how to care for it, and perhaps eventually how to disassemble it and recycle (or safely dispose of) its contents. A flowerpot that was made with living hinges might inform a user, based on records that are maintained online, as to when the plant inside the pot was last watered and when it needs to be watered again. Children examining a toy crocodile could, through StructCode, learn scientific details about various parts of the animal’s anatomy. A picture frame made with finger joints modified by StructCode could help people find out about the painting inside the frame and about the person (or persons) who created the artwork — perhaps linking to a video of an artist talking about this work directly.</p><p>“This technique could pave the way for new applications, such as interactive museum exhibits,” says Raf Ramakers, a computer scientist at Hasselt University in Belgium. “It holds the potential for broadening the scope of how we perceive and interact with everyday objects” — which is precisely the goal that motivates the work of Doğan and his colleagues.</p><p>But StructCode is not the end of the line, as far as Doğan and his collaborators are concerned. The same general approach could be adapted to other manufacturing techniques besides laser cutting, and information storage doesn't have to be confined to the joints of wooden objects. Data could be represented, for instance, in the texture of leather, within the pattern of woven or knitted pieces, or concealed by other means within an image. Doğan is excited by the breadth of available options and by the fact that their “explorations into this new realm of possibilities, designed to make objects and our world more interactive, are just beginning.”</p>

StructCode modifies fabrication features like finger joints and living hinges in laser-cut objects, allowing for the addition of information such as labels and instructions without adding extra parts or materials. It also includes a robust tag decoding system and ensures the structural integrity of the laser-cut objects is maintained. Image courtesy of the researchers/MIT CSAIL.

StructCode, developed by MIT CSAIL researchers, encodes machine-readable data in laser-cut objects by modifying their fabrication features.

A new way to integrate data with physical objects

Mastering PCB Reverse Engineering: Techniques, Tools, and Best Practices for Success in Electronics

VR/AR

Profiles