<p id="isPasted"><em>The utilization of advanced 3D vision technology is critical in pushing the boundaries of robotic capabilities, especially in challenging environments such as space. Extend Robotics, dedicated to making robotic operations more accessible through extended reality technology, embarked on a comparative evaluation of 3D vision solutions to enhance their robotic interface software. This comprehensive test aimed to identify the most suitable 3D camera for demanding applications, including those required in extra-terrestrial conditions.</em></p><p>&nbsp;</p><p><strong>Partner:</strong></p><p><a href="https://www.extendrobotics.com/" target="_blank">Extend Robotics</a></p><p>&nbsp;</p><p><strong>Challenge:</strong></p><p>Extend Robotics aims to address the global skill mismatch caused by automation by making robotic arm programming more accessible. Their mission is to enable non-robotic experts to operate robots remotely using extended reality technology (MR/VR/AR). In their latest exploration project, they sought to demonstrate their technology’s capability to remove and replace satellite modules in a 27U CubeSat at the Catapult test facility for In-Orbit Service and Manufacturing. This project was part of their incubation with the European Space Agency’s Business Incubation Centre (ESA BIC UK).</p><p>To achieve their goals, Extend Robotics needed a 3D vision solution that could handle extreme and challenging environments, including those found in space. They conducted a comparative test between three different 3D cameras to select the optimal hardware for these demanding applications.</p><p>&nbsp;</p><p><strong>Solution:</strong></p><p><a href="https://www.photoneo.com/motioncam-3d/" target="_blank">MotionCam-3D Color</a></p><p>Extend Robotics conducted extensive testing to evaluate the performance of Photoneo’s MotionCam-3D Color against other 3D cameras. The MotionCam-3D Color was tested under extreme light conditions, including a 5,000 watt light simulating the brightness of the sun in space. The camera’s ability to deliver high-resolution 3D scans in both extremely bright and pitch black conditions set it apart from its competitors.</p><p>The MotionCam-3D Color utilizes Photoneo’s advanced Parallel Structured Light technology, which ensures unparalleled accuracy regardless of environmental light levels. This capability is crucial for space applications where lighting conditions can vary dramatically.</p><p>By leveraging the MotionCam-3D Color, Extend Robotics can create precise and accurate 3D models necessary for their remote robotic operations. The model is projected in the virtual headset of the operator who can interact with it. This solution not only enhances the safety and efficiency of their technology but also enables the execution of physical tasks remotely, overcoming geographical limitations.</p><p><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1720524677538-1720524677538.png" alt="" width="400" height="180" srcset="https://www.photoneo.com/wp-content/uploads/2024/06/Extend_BLOG2-400x180.jpg 400w, https://www.photoneo.com/wp-content/uploads/2024/06/Extend_BLOG2-1024x461.jpg 1024w, https://www.photoneo.com/wp-content/uploads/2024/06/Extend_BLOG2-768x346.jpg 768w, https://www.photoneo.com/wp-content/uploads/2024/06/Extend_BLOG2-1536x692.jpg 1536w, https://www.photoneo.com/wp-content/uploads/2024/06/Extend_BLOG2-2048x922.jpg 2048w, https://www.photoneo.com/wp-content/uploads/2024/06/Extend_BLOG2-480x216.jpg 480w" sizes="(max-width: 400px) 100vw, 400px" class="fr-fic fr-dii"></p><p><strong>Discussion:</strong></p><p>The integration of Photoneo’s MotionCam-3D Color into Extend Robotics’ system marks a significant advancement in remote robotic operations. The high-precision 3D vision capabilities of the MotionCam-3D Color allow for detailed scanning and modeling, which are essential for complex tasks such as satellite servicing.</p><p>This collaboration highlights the importance of selecting the right 3D vision technology for demanding applications. Photoneo’s MotionCam-3D Color has proven to be the optimal solution for Extend Robotics, enabling them to push the boundaries of what’s possible in automation and robotics.</p><p>Would you like to discuss how Photoneo technology can benefit your applications? <a href="https://www.photoneo.com/contact-us/?utm_source=Wevolver&utm_medium=Case%20Study&utm_campaign=Extending%20Robotics%20Capabilities%20in%20Space%20with%20Photoneo's%20MotionCam-3D%20Color" target="_blank" rel="noopener noreferrer">Contact us</a> to explore how our advanced 3D vision solutions can enhance your projects.</p>

Extending Robotics Capabilities in Space with Photoneo's MotionCam-3D Color

Extend Robotics leveraged Photoneo’s MotionCam-3D Color for its exceptional 3D vision technology, enabling precise remote operations in challenging environments like space. This innovative solution enhances robotic capabilities, allowing for high-resolution scans under extreme conditions.

Sonar scan overlay with camera image in background.

Discover how sonar makes search and recovery missions safer and more effective, and explore the benefits of integrating Deep Trekker ROVs with sonar.

Sonar for Search and Recovery ROVs

<p>MTi-680 motion tracking sensor from Movella enables continuous levelling and stabilisation of SmartPano's images of road scenes</p><p>Mobile mapping is big business. Amongst other, the automotive industry relies on continually updated visual and geospatial data to enable consumer-friendly features such as satellite navigation and location-aware speed limit information. New driving assistance and automated driving functions require even more detailed data about roads and traffic. Search and mapping services for computers and mobile phones also draw on a huge global data store of geolocated images.</p><p>Until now, this mapping data has been gathered and owned by giant technology companies that deploy specialised 3D cameras mounted on dedicated vehicles. Because of the huge cost of owning and operating the cameras, mapping has been a business for those with very deep pockets.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 768px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzExMDEzNzQ2NzI1LWdlbiAyLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 100%px;" class="fr-fic fr-dib"><span class="fr-inner"><em>The SmartPano 360° camera from Smart Delta</em></span></span></span></p><p>The introduction of the portable and affordable SmartPano 360° camera from SmartDelta breaks down the barriers to the mapping market. The high quality and stability of the SmartPano's images owe much to the 3D motion data produced by the Xsens MTi-680 sensor from Movella.</p><p><strong>A new alternative to expensive, fixed 360° camera systems</strong></p><p>The barrier to participation in the mobile mapping market has been the huge cost of the equipment. Internet search companies and other large tech companies have developed high-performance but high-cost fixed camera/sensor systems that are permanently attached to a host vehicle. Highly complex, these proprietary systems require extensive driver training before use.</p><p>Smart Delta's development of the SmartPano camera turns the existing approaches to mobile mapping on their head. Smart Delta has created a product which is small and light enough to be easily transported anywhere in the world. The IP67 system is delivered in a hand luggage size transport case and weighs only 5.6 KG. The system has four powerful magnetic fixing cups, so it can be easily set on- and removed from the roof of any car. Wi-Fi and Bluetooth connections and 2 interconnected integrated power banks mean that the system does not need any external cables and can be operated for a full day of mapping without the need to recharge.</p><p><br></p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 768px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzExMDEzNzU5ODk3LXNtYXJ0IHBhbm8gMS5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 100%px;" class="fr-fic fr-dib"><span class="fr-inner"><em id="isPasted">The SmartPano camera's base has convenient magnetic cups for fixing it to the roof of a car</em></span></span></span></p><p>Crucially, the SmartPano camera system costs a fraction of the proprietary camera systems typically used for mobile mapping today.</p><p>Yet despite its size and affordability, the SmartPano camera offers geospatial video data and accuracy as good as that of an expensive proprietary system. Four Sony image sensors behind F2.28 aspherical fisheye lenses capture a sharp 360° view at a rate of 7 frames per second in 8KK (UHD) mode. The images are geolocated by a Ublox GNSS receiver.</p><p>Continuous stitching of video captured by the four image sensors requires the images to be levelled and stabilised – as though the entire camera system were mounted on a virtual gimbal. This is achieved by sophisticated SmartDelta software operating on accurate pitch and roll data produced by the Xsens MTi-680 motion tracking sensor. Tightly synchronised to the GNSS receiver and the image sensors, the MTi-680 allows the SmartPano to compensate for every bump, slope and camber in the road. The MTi-680 also provides orientation and 3D acceleration data that enables the SmartPano to maintain accurate geolocation information when it is denied a GNSS signal, such as when driving through a short tunnel.</p><p><br></p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 768px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzExMDEzMzg1MjA0LU1UaS5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 100%px;" class="fr-fic fr-dib"><span class="fr-inner"><em id="isPasted">The MTi-680 motion tracker is small enough to fit in the Smart Pano's compact enclosure</em></span></span></span></p><p>Jan Wierenga, CCOof Smart Delta, says that the combination of high-resolution image sensors, the MTi-680 motion data, and SmartDelta software is the reason the SmartPano can match the industry's highest standards for data quality despite being available at a fraction of the cost of Big Tech systems. "Our SmartPano system has undergone rigorous testing by one of the foremost mobile mapping data providers in the world, emerging triumphantly with exceptional quality and reliability. Positioned as an invaluable asset for mapping and location platform companies, SmartPano presents a compelling opportunity to significantly reduce the expenses associated with consistently updating high-quality road maps – meeting and exceeding the stringent demands of their discerning clientele."</p><p><br></p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 768px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzExMDEzMzk5OTQzLW1haW4gcGljLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 100%px;" class="fr-fic fr-dib"><span class="fr-inner"><em id="isPasted">SmartPano images are seamlessly stitched to form a 360° image</em></span></span></span></p><p>The Smartpano is not only affordable to buy, but it is very scalable to operate as well.</p><p>Jan Wierenga explains: The intuitive user interface ensures any driver can operate the system without the need of any Mobile Mapping knowledge. Specific software settings to meet the customer's requirements can be (remotely) preset and protected so that the driver can focus on his/her task.</p><p>Lidar:</p><p>When the SmartPano system itself is not sufficient, connecting a LiDAR system to the SmartPano Mobile Mapping System is a simple and intuitive action. The system is designed with user-friendliness in mind and provides seamless connectivity options for various pre integrated LiDAR devices.</p><p>This new democratisation of mobile mapping opens up many new opportunities to use the Smart- Pano’s 360° video outputs. Smart Delta sees many different applications where the Smartpano can prove its added value and welcomes potential customers to share their business case.</p><p>More information about the <a href="https://smartdelta.nl/system/smart-pano/#" target="_blank" rel="noopener noreferrer">SmartPano 360°</a> camera.&nbsp;</p><p>More information about the <a href="http://More information about the MTi-680 motion tracker." target="_blank" rel="noopener noreferrer">MTi-680</a> motion tracker.&nbsp;</p>

SmartDelta opens up the market for ultra-HD mobile mapping data with new SmartPano portable, low-cost/ high accuracy 360° mobile mapping system

Smart pano makes mobile mapping easy

Deep Trekker PIVOT ROV investigating a shipwreck.

Discover how creating digital twins using ROVs enables efficient monitoring, and maintenance of underwater assets by exploring the capabilities of advanced modeling techniques.

Improving Underwater Asset Management with ROVs

Stantec 3D map model of a dam structure from the 1800's.

Discover how Deep Trekker ROVs are capable of conducting quality photogrammetry missions to create 3D models with unparalleled accuracy. 

Making a 3D Model with Deep Trekker ROVs

<h2 dir="ltr" id="isPasted">Introduction</h2><p dir="ltr">In this blog post, we delve into the dynamic and innovative realm of Immersal's Visual Positioning System (VPS), a groundbreaking technology reshaping various industries, with a special focus on robotics, automation, and the burgeoning field of autonomous mobility and smart spaces. Our exploration will highlight how Immersal's VPS is revolutionizing robotics, particularly for those robots lacking conventional navigation systems like LiDAR, enabling them to interact with and navigate their environments with unprecedented precision and efficiency.</p><p dir="ltr">We will also delve into the transformative potential of VPS in the domain of autonomous vehicles, exploring how this technology is not just enhancing existing systems but also pioneering new pathways in automation and mobility. Additionally, we will examine the significant collaboration between Immersal and Unikie, a union that epitomizes the synergy between advanced spatial computing and edge-driven autonomous solutions. This partnership is a beacon in the international field of autonomous mobility and smart spaces, showcasing how integrated technologies can lead to smarter, more efficient urban landscapes and transportation systems.</p><p dir="ltr">Join us as we navigate through these exciting developments, offering insights into a future where Immersal's VPS technology is at the heart of innovative solutions across multiple sectors, driving progress and redefining the boundaries of what is possible in our increasingly connected world.</p><h2 dir="ltr">The Role of VPS in Robotics</h2><p dir="ltr">In the realm of robotics, the ability to accurately perceive and interact with the environment is paramount. Traditionally, robots have relied on technologies like LiDAR (Light Detection and Ranging) for navigation and mapping. However, LiDAR systems can be costly and complex, limiting their accessibility and versatility in various applications. This is where Immersal's VPS technology comes into play, offering a more cost-effective and flexible solution for spatial awareness and navigation.</p><p dir="ltr">VPS technology enables robots to navigate and operate in complex environments without the need for expensive hardware. By using cameras, robots equipped with Immersal's VPS can understand their surroundings with a high degree of accuracy. This capability is particularly beneficial in industries where environmental conditions are constantly changing or where installing additional infrastructure for navigation is impractical.&nbsp;</p><p dir="ltr">In manufacturing, robots with VPS navigate autonomously, swiftly adapting to changing environments, thus boosting efficiency and reducing reprogramming needs. These robots enhance inventory management in logistics and warehousing by precisely navigating and efficiently handling materials. Additionally, in healthcare and hospitality, VPS-enabled robots improve interactions in dynamic settings, aiding in patient care and customer service, demonstrating their versatility and responsiveness in diverse roles.</p><hr><h2 id="isPasted">What will you build?</h2><p>Whether you're building an AR experience for a large commercial space, a factory, or an industrial setting, or even if you'd just like to create an AR app for fun, Immersal is the way to do it.</p><p>Immersal has released its SDK 2.0 and new Mapper App.&nbsp;</p><p><a href="https://developers.immersal.com/?WEVOLVER=" target="_blank" rel="noopener noreferrer" class="button-main-action">SIGN UP FOR YOUR FREE ACCOUNT HERE.</a></p><hr><h2>Boston Dynamics' Spot Robot and Immersal VPS</h2><p dir="ltr">The integration of Immersal's Visual Positioning System (VPS) with Boston Dynamics' Spot robot represents a fascinating convergence of advanced robotics and cutting-edge positioning technology. Spot, known for its mobility, agility, and stability, can achieve even greater operational capabilities when combined with the precision and spatial awareness provided by Immersal's VPS.</p><p dir="ltr">Immersal's VPS can significantly enhance <a href="https://www.youtube.com/watch?v=aq5SHrw6WSk" target="_blank">Spot's navigation abilities, especially in environments where GPS is unreliable or unavailable</a>. By utilizing visual cues from the environment, Spot can navigate complex indoor spaces or densely packed urban areas with greater accuracy. This integration allows Spot to perform tasks with higher precision, making it an even more valuable tool in various applications.</p><p dir="ltr"><span contenteditable="false" draggable="true" class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable"><iframe width="640" height="360" src="https://www.youtube.com/embed/aq5SHrw6WSk?&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"><span class="fr-mk" style="display: none;">&nbsp;</span></iframe></span></p><h2 dir="ltr">Potential Applications Across Industries</h2><p dir="ltr"><strong>Industrial Inspection and Maintenance:</strong> In industrial settings, a Spot robot can autonomously navigate facilities, performing routine inspections and identifying maintenance needs with minimal human oversight. This leads to increased safety, as Spot can access hazardous or hard-to-reach areas, and improved efficiency in monitoring and maintenance tasks.</p><p dir="ltr"><strong>Emergency Response and Public Safety:</strong>&nbsp;Spot can be deployed to navigate through debris or unsafe structures in emergency situations, such as natural disasters or fire outbreaks. With Immersal's VPS, it can accurately map and assess the environment, providing valuable information to first responders and aiding in search and rescue operations.</p><p dir="ltr"><strong>Healthcare and Assisted Living: </strong>Spot can navigate autonomously in healthcare facilities, delivering supplies or assisting in patient care. The precise navigation enabled by Immersal's VPS ensures safety and efficiency, especially in crowded or dynamically changing environments like hospitals.</p><h2>Pioneering a New Era in Autonomous Mobility and Smart Spaces</h2><p>The international landscape of autonomous mobility and smart spaces is rapidly evolving, with Immersal's Visual Positioning System (VPS) playing a pivotal role. This technology's integration into the realm of autonomous systems and intelligent environments is not just an advancement; it's a transformation. The partnership between Immersal and Unikie, a leader in edge computing and autonomous driving solutions, marks a significant milestone in this journey.</p><p>Immersal's VPS brings unparalleled precision and efficiency to the domain of smart spaces. By accurately mapping and understanding spatial environments, this technology enables a more intuitive and interactive experience in various settings, from commercial complexes to public spaces. The integration of Immersal's VPS in smart spaces lays the groundwork for environments that are not only responsive but also adaptive to the needs and movements of individuals within them.</p><p><a href="https://www.youtube.com/watch?v=-XCjAppPf1I" target="_blank">The collaboration between Unikie and Immersal</a> is a testament to the potential of combining expertise in edge computing and visual positioning. Unikie's focus on autonomous driving solutions complements Immersal's VPS, creating a synergy that enhances the capabilities of autonomous vehicles. This partnership is poised to drive significant advancements in autonomous mobility, offering solutions that are safer, more efficient, and more reliable.</p><p><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1707486614089-car.png" style="width: 300px;" class="fr-fic fr-dib"></p><p dir="ltr">The application of Immersal's VPS in conjunction with Unikie's technology has the potential to transform urban landscapes and transportation systems. In smart cities, autonomous vehicles equipped with this integrated technology can navigate more safely and efficiently, reducing traffic congestion and improving overall urban mobility. This collaboration could lead to the development of smarter, more connected urban environments where autonomous vehicles and smart infrastructure work in harmony.</p><p dir="ltr">Beyond urban mobility, the Immersal-Unikie partnership extends its impact to various sectors. In logistics, for example, autonomous vehicles equipped with Immersal's VPS can optimize route planning and delivery processes. In retail and hospitality, smart spaces enhanced by this technology can offer personalized experiences to customers, revolutionizing service delivery and customer interaction.</p><h2>Conclusion</h2><p>As we reach the end of our exploration into Immersal's VPS, it's evident that this technology is not just an advancement; it's a revolution in the fields of robotics, automation, and autonomous mobility. It represents a pivotal shift towards more efficient, intelligent, and interconnected systems. In robotics, the impact of VPS is transformative, offering a cost-effective and flexible alternative to traditional navigation systems like LiDAR. This innovation enables robots to navigate and interact with dynamic environments with unprecedented precision, opening new possibilities in industries ranging from manufacturing to healthcare. The agility and adaptability provided by VPS technology mean that robots can now operate in complex settings with minimal human intervention, enhancing efficiency and safety.</p><p>The collaboration between Immersal and Unikie exemplifies the fusion of cutting-edge spatial computing and autonomous solutions, marking a significant milestone in the evolution of smart spaces and autonomous mobility. This partnership is not just about enhancing existing systems; it's about pioneering a new era where technology seamlessly integrates with our physical and social environments. In the realm of autonomous vehicles, the integration of Immersal's VPS with edge computing leads to safer, more efficient navigation, reducing urban challenges like traffic congestion and paving the way for smarter, more sustainable cities. Beyond transportation, this technology has the potential to revolutionize public spaces, commercial complexes, and even personal living spaces, making them more responsive and adaptive to human needs. Immersal's VPS is an opportunity to lead in the creation of innovative, efficient, and sustainable solutions.&nbsp;</p><p>This technology is a call to action, urging us to rethink how we interact with our environments, mobilize resources, and automate processes. As we step into this new era, the possibilities are limitless, and the potential for positive change is immense.&nbsp;</p><h2 id="isPasted">What will you build?</h2><p>Whether you're building an AR experience for a large commercial space, a factory, or an industrial setting, or even if you'd just like to create an AR app for fun, Immersal is the way to do it.</p><p>Immersal has released its SDK 2.0 and new Mapper App.&nbsp;</p><p><a href="https://developers.immersal.com/?WEVOLVER=" target="_blank" rel="noopener noreferrer" class="button-main-action">SIGN UP FOR YOUR FREE ACCOUNT HERE.</a></p>

Immersal's Visual Positioning System (VPS) revolutionizes robotics and autonomous mobility, enhancing precision and fostering innovations in smart urban development and transportation through its partnership with Unikie.

Exploring the Future of Navigation and Automation with Immersal's Visual Positioning System

<p dir="ltr" id="isPasted">Imagine a world where the physical and digital realms blend seamlessly, creating extraordinary live experiences. This fusion, driven by Augmented Reality (AR) and enhanced by Visual Positioning Systems (VPS), is revolutionizing the way we interact with entertainment across sports, music, and leisure activities. AR transcends mere visual enhancement, by overlaying digital information onto our real-world environment that can be interacted with and manipulated, allowing for deeper audience engagement and comprehension. VPS plays a crucial role in this transformation, enabling precise and context-aware positioning of augmented elements, thus making the AR experience more realistic and interactive.</p><p dir="ltr">This article dives into the significant impact of AR and VPS in entertainment, focusing on their joint capability to deliver real-time analytics, boost audience involvement, and craft captivating, immersive experiences.&nbsp;</p><p dir="ltr">In a sports stadium, AR could offer spectators live, in-depth analytics overlayed on the field of play, while in a concert setting, it could synchronize elaborate visual effects with live performances.&nbsp;</p><p dir="ltr">The synergy of AR and VPS is reshaping entertainment, offering personalized, enriched, and dynamic experiences that were previously unimaginable. This piece aims to unveil the innovative potential of combining AR with VPS, highlighting how they are collectively setting new benchmarks in interactive and engaging live entertainment experiences.</p><h2 dir="ltr">Exploring the Essence of Augmented Reality</h2><p dir="ltr">Augmented Reality (AR) integrates digital components like images, information, and animations into our physical environment, viewable from devices such as glasses or smartphones, enhancing our perception of reality. Central to its effectiveness is the Visual Positioning System (VPS), utilizing sensors and cameras for accurate real-time interpretation and localization of the environment. To generate impactful AR experiences, careful planning is essential. This includes conducting detailed spatial mapping to ensure the accurate placement of virtual elements within the real world, thus creating a seamless and interactive user experience. This integration of AR and VPS marks a significant step in merging the digital with the physical, opening new dimensions in our interaction with technology.</p><h3 dir="ltr">Elevating the Sports Experience with AR</h3><p dir="ltr">AR is redefining the sports landscape, seamlessly blending the electric ambiance of live events with the analytical depth of broadcast media. For fans in the stands, AR offers an enriched viewing experience by delivering real-time statistics and instant replay capabilities right before their eyes, enhancing their connection with the game. Simultaneously, it provides coaches with a powerful tool for gaining deeper insights into player performances and strategies. This innovative fusion of live action and digital intelligence by AR not only elevates the spectator experience but also revolutionizes coaching techniques, setting a new standard in how sports are played, coached, and enjoyed.</p><h3>Enhanced Case Study: Immersal's Collaboration with Major League Baseball</h3><p>In a groundbreaking initiative in July 2023, Immersal, in collaboration with Nexus Studios, T-Mobile, and Major League Baseball (MLB), unveiled a cutting-edge AR experience during the T-Mobile Home Run Derby and the MLB All-Star Game. This innovative venture brought to life the MLB Next app, powered by Immersal's advanced VPS. The app offered spectators an immersive experience with live vivid 3D visuals and up-to-the-minute data, significantly elevating the traditional sports viewing experience.</p><p>This AR application successfully overcame various environmental hurdles to provide an engaging and interactive platform for fans. The real-time data and striking 3D graphics not only enhanced viewer engagement but also brought a new dimension to the understanding of the game's intricacies.</p><p dir="ltr" id="isPasted">In recognition of this technological marvel, we are thrilled to share that the <a href="https://www.sportsbusinessjournal.com/Articles/2023/12/11/sports-business-awards-tech.aspx" target="_blank">MLB Next App has been shortlisted for the prestigious Sports Business Awards in the “Best in Extended Reality” category.</a> This nomination underscores the app's significant contribution to enhancing sports technology and fan experience, setting a new benchmark in the sports entertainment industry.</p><hr><h2>What will you build?</h2><p>Whether you're building an AR experience for a large commercial space, a factory, or an industrial setting, or even if you'd just like to create an AR app for fun, Immersal is the way to do it.</p><p>Immersal has released its SDK 2.0 and new Mapper App.&nbsp;</p><p><a href="https://developers.immersal.com/?WEVOLVER=" target="_blank" rel="noopener noreferrer" class="button-main-action">Sign up for your free account here.</a></p><hr><h2>Advancements in Athlete Training with AR</h2><p>AR is revolutionizing athlete training by equipping coaches with sophisticated, real-time analytics. This cutting-edge technology facilitates more targeted and effective training approaches. Through AR, coaches and athletes can access a wealth of data, including biomechanical analysis and performance metrics, right at their fingertips. This level of detailed feedback allows for a more nuanced understanding of an athlete's strengths and areas for improvement, leading to tailored training programs that maximize efficiency and effectiveness.</p><p>A prime example of AR's impact in sports training is the case of renowned long-distance runner Eliud Kipchoge. By leveraging AR-driven data, Kipchoge was able to fine-tune his training regimen, focusing on specific aspects of his performance. This personalized approach, enriched with detailed AR analytics, played a crucial role in his outstanding achievement at the 2016 Olympic Games. Kipchoge's success story not only highlights the potential of AR in elevating athletic performance but also serves as an inspiration for how technology can be harnessed to push the boundaries of human potential in sports.</p><h2 dir="ltr" id="isPasted">Transforming Fan Engagement with Gamified AR Experiences</h2><p dir="ltr">In the dynamic realm of sports and entertainment, AR is a game-changer, creating captivating fan experiences through innovations like virtual stadium tours and interactive games. AR allows fans to delve into additional game insights and unique perspectives right from their devices. Significantly, the integration of volumetric video into AR environments is enhancing this experience, bringing a rich, three-dimensional depth to content, making virtual interactions with players and mascots astonishingly realistic and immersive.</p><p dir="ltr">At home, AR transforms the viewing experience, enlivening broadcasts with elements like team mascots and interactive ads. These mascots, once just a part of the field, now come to life in 3D, bridging the gap between the stadium and the living room. This not only adds a fun, interactive twist to home viewing but also leverages the power of volumetric video to make these virtual interactions incredibly lifelike. Furthermore, AR's application in advertising offers interactive, engaging commercial experiences, heightening the overall game-watching experience. These advancements in AR, augmented by the realism of volumetric video, are reshaping fan engagement, turning passive viewing into an immersive, interactive event.</p><h3 dir="ltr">AR in Live Music Events</h3><p dir="ltr">Augmented Reality is swiftly becoming an essential element in live music events, enhancing concerts with immersive, interactive experiences. While AR has been part of concerts for some time, its role is growing more significant and transformative. A notable example is Childish Gambino's innovative use of an AR app during his <a href="https://www.youtube.com/watch?v=kY1Mr0b_I4Y" target="_blank">2019 Coachella performance</a>, which added a new layer of engagement for the audience.</p><p dir="ltr">In a recent landmark event, the popular rock band UVERworld performed at Yokohama Nissan Stadium on July 29-30, 2023, to an enthusiastic crowd of 70,000 fans. In a collaborative effort, Immersal and Realize Mobile Communications crafted a unique AR experience for this concert. Utilizing the technology, around <a href="https://youtu.be/Fgxj19pEaqg" target="_blank">20,000 people sent messages from across the stadium</a>, which were then visually aggregated to form UVERworld's iconic logo. This interactive feature not only fostered a communal atmosphere among the audience but also visually encapsulated their emotions and passion, elevating the collective experience. Remarkably, over 7,000 individuals engaged simultaneously in this large-scale, real-time social interaction, showcasing the immense potential of AR in bringing together and amplifying the concert-going experience.<a href="https://youtu.be/Fgxj19pEaqg?feature=shared" target="_blank"></a></p><h2 dir="ltr"><a href="https://youtu.be/Fgxj19pEaqg?feature=shared" target="_blank"><span contenteditable="false" draggable="true" class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable"><iframe width="640" height="360" src="https://www.youtube.com/embed/Pf2KWpmqgnY?&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span><br></a>Revolutionizing Theme Park Adventures with AR</h2><p>Theme parks are universally adored for their thrills and excitement, but long lines can dampen the enthusiasm. AR emerges as a game-changer in this context, significantly improving the visitor experience. AR is adept at providing real-time estimates of ride wait times, ensuring that guests can manage their time more efficiently. Through strategically placed AR markers and location-based triggers, the physical environment of the park is enriched with interactive virtual content.</p><p>Advanced VPS technologies, like those developed by Immersal, offer a plethora of enhancements to the theme park experience:</p><p><strong>Interactive Maps and Customized Routes:&nbsp;</strong>These features cater to individual preferences, allowing visitors to plot their journey based on their interests and the attractions they wish to see.</p><p><strong>Virtual Walls:&nbsp;</strong>An innovative way for visitors to interact with the park, where they can contribute to a virtual mural at iconic landmarks. A notable example is what Immersal did at Disney Hong Kong in 2022.&nbsp;</p><p dir="ltr"><strong>Digital Signage:</strong> This offers updated information about essential aspects like ride wait times, show schedules, menus, and more, all in a virtual format that's easy to access and understand.</p><p><strong>Wayfinding:&nbsp;</strong>Combining VPS with smartphone sensors and computer vision, systems like Immersal's provide real-time navigation help, guiding visitors with vibrant virtual arrows and cues on the ground.</p><p><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzA3NDg1OTE1MzU4LWdhbWUgaW1tZXJzYWwuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib"></p><p dir="ltr" id="isPasted"><strong>Friend Finding:&nbsp;</strong>This useful feature assists in locating and navigating to friends or family members if you get separated, enhancing the social aspect of the park experience.</p><p dir="ltr">AR in theme parks transcends mere entertainment; it opens doors to personalized and interactive experiences. It encourages visitors to engage more deeply with attractions and characters, enriching their overall experience.</p><p><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzA4MzQ3ODk0OTU0LWhrZGxfd29ybGQgb2YgZnJvemVuXzEwbSBnaWFudCBvLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib"></p><p dir="ltr" id="isPasted">To illustrate, Disneyland Hong Kong's "World of Frozen" campaign celebrated the new attraction's opening by featuring Olaf, the beloved snowman, in various forms across East Asia. These included a massive 10-meter installation, appearances in shopping malls, and <a href="https://www.hongkongdisneyland.com/worldoffrozeninvitation/index.html" target="_blank">innovative augmented reality experiences</a>, showcasing the diverse applications of AR in enhancing theme park adventures.</p><h2>Conclusion</h2><p>Augmented Reality, enhanced by Visual Positioning Systems, is revolutionizing the entertainment landscape, merging digital innovation with real-world experiences. This technology has transformed sports, offering fans immersive, interactive experiences and providing coaches with in-depth performance analytics. In the music industry, AR has elevated concerts into engaging, social events, fostering a deeper connection between artists and audiences. Additionally, theme parks are utilizing AR to personalize visitor experiences and streamline navigation, making every trip more engaging and efficient. As AR continues to evolve, it's redefining entertainment, moving it from passive observation to active, immersive participation, blurring the lines between reality and digital realms in exciting new ways.</p><h2 id="isPasted">What will you build?</h2><p>Whether you're building an AR experience for a large commercial space, a factory, or an industrial setting, or even if you'd just like to create an AR app for fun, Immersal is the way to do it.</p><p>Immersal has released its SDK 2.0 and new Mapper App.&nbsp;</p><p><a href="https://developers.immersal.com/?WEVOLVER=" target="_blank" rel="noopener noreferrer" class="button-main-action">SIGN UP FOR YOUR FREE ACCOUNT HERE.</a></p>

Augmented Reality (AR) and Visual Positioning Systems (VPS) are transforming live entertainment by blending the physical and digital realms to offer immersive, interactive, and personalized experiences across sports, music, and leisure activities.

Augmented Reality: The Transformative Impact of AR in Entertainment

<p dir="ltr" id="isPasted">In the realm of digital immersive experiences, advancements in VR (Virtual Reality) and AR (Augmented Reality) technologies are rapidly evolving. At the heart of this evolution is Finland-based Immersal, a company that's pioneering in the Visual Positioning System (VPS) domain. Thanks to Immersal's cutting-edge spatial mapping and visual positioning technologies, this innovation is revolutionizing how digital and physical realities merge.</p><p dir="ltr">Immersal's VPS was born out of a practical need encountered during an AR project, where the team faced the limitations of existing positioning systems. Their response was to develop a VPS offering centimeter-level accuracy, enabling persistent, shared multiuser AR experiences. They recognized the inadequacies of GPS (Global Positioning Satellites) for AR applications and responded with a VPS that transcends these limitations, enhancing both indoor and outdoor location accuracy and visual mapping. This technological advancement is not just a solution to current challenges but a gateway to unexplored possibilities in augmented reality, paving the way for innovative applications across diverse fields.</p><h2 dir="ltr" id="isPasted">What is VPS?</h2><h3 dir="ltr">Difference Between GPS and VPS Technologies</h3><p dir="ltr" id="isPasted">The main difference between GPS and VPS is the way they determine location. GPS relies on satellite signals and is designed primarily for outdoor use. It provides global coverage but often with limited accuracy. VPS, in contrast, uses computer vision, cameras, and visual data to determine the precise location and orientation of a device in real-time indoors and outdoors, even underground.</p><p dir="ltr"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzA3NDExNTA0MDYzLXZwcy5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"></p><h3 dir="ltr" id="isPasted">What problems does VPS solve?</h3><p>VPS solves critical navigation challenges in areas where GPS is limited, such as indoors or in dense urban environments. It enables precise location tracking and orientation within complex structures like malls or office buildings. VPS significantly enhances AR applications by accurately merging digital and physical worlds. This technology also streamlines asset tracking in challenging environments, offering reliable location data where traditional GPS-based solutions are ineffective.</p><h2 dir="ltr" id="isPasted">When to use VPS for AR applications&nbsp;</h2><p dir="ltr">VPS is a viable option for AR applications that require any of the following attributes:</p><ul><li dir="ltr"><p dir="ltr">both indoor and outdoor positioning</p></li><li dir="ltr"><p dir="ltr">high-precision positioning</p></li><li dir="ltr"><p dir="ltr">device orientation in addition to position and a full 6 degrees of freedom pose</p></li><li dir="ltr"><p dir="ltr">multi-user, shared AR world</p></li><li dir="ltr"><p dir="ltr">persistent virtual content, objects that are tied to the physical world</p></li></ul><p dir="ltr">The versatility and functionality of VPS makes it an excellent choice for applications such as indoor navigation guides; for example, helping a user locate their airport departure gate. Another use case could be virtual content in an art museum, where the story and history behind an art piece could be tied to the art itself, and brought to life with visually pleasing 3D graphics and spatial audio.</p><hr><h1 id="isPasted">What will you build?</h1><p>Whether you're building an AR experience for a large commercial space, a factory, or an industrial setting, or even if you'd just like to create an AR app for fun, Immersal is the way to do it.</p><p>Immersal has released its SDK 2.0 and new Mapper App.&nbsp;</p><p><a href="https://developers.immersal.com/?WEVOLVER=" target="_blank" rel="noopener noreferrer" class="button-main-action">Sign up for your free account here.</a></p><hr><h2>Use Cases of VPS Technology</h2><p dir="ltr">Some common use cases for VPS include augmented reality-based entertainment and guided navigation at theme parks, fan engagement experiences, urban planning, and robot navigation. Sports fans are also looking forward to more immersive experiences and real-time analytics that can easily be displayed in AR when watching a game at the stadium.</p><h3 dir="ltr">Theme parks</h3><p dir="ltr">VPS revolutionizes the theme park experience by enhancing navigation and interaction within these expansive, often crowded environments. VPS aids visitors in pinpointing their exact location and navigating efficiently to attractions, reducing the time spent on orientation. It enriches the park experience through AR features, such as interactive guides and virtual queues, blending the digital with the physical for a more immersive visit. Additionally, VPS technology assists park management in crowd control and operational efficiency, ensuring a smoother, more enjoyable experience for all guests.</p><p dir="ltr">Industrial Settings</p><p dir="ltr">In industrial environments, VPS facilitates accurate indoor navigation, guiding workers through complex facilities like warehouses or manufacturing plants. It enables precise location tracking of machinery, equipment, and inventory, streamlining logistics and inventory management. For maintenance and safety, it provides real-time, site-specific information, helping to identify potential hazards and maintenance needs. Additionally, in AR applications, VPS assists in training and simulations, providing workers with an interactive, detailed view of machinery and processes, thus reducing errors and improving overall operational efficiency.</p><h3 dir="ltr">Wayfinding</h3><p dir="ltr">VPS can be used in complex indoor environments like shopping malls, airports, and museums by displaying easy-to-follow AR navigation guides to customers. VPS accurately locates visitors and offers augmented reality (AR) navigation to guide them effortlessly to their destinations, a task beyond the capabilities of traditional GPS.</p><p dir="ltr"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzA3NDExNzM0ODE3LXdheWZpbmRpbmcuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib"></p><h3 dir="ltr" id="isPasted">Fan engagement at live events</h3><p dir="ltr">VPS can elevate fan engagement at live events by providing immersive, interactive experiences. Fans can navigate sprawling event spaces with ease, using AR for real-time directions to seats, amenities, or points of interest. It also enables personalized content delivery, such as AR-enhanced replays and player stats, directly to fans' devices, creating a more engaging and memorable event experience. Beyond the actual game, AR can also be applied to immersive virtual merchandise stores, allowing fans to preview and purchase products virtually, and enhancing the overall shopping experience. Sponsorships, half-time ads, and bringing mascots to life are a few other opportunities to elevate the in-stadium fan experience with VPS-based AR.</p><h3 dir="ltr" id="isPasted">Urban planning</h3><p dir="ltr">VPS offers transformative capabilities in urban planning by enabling precise mapping and analysis of city landscapes. Planners can use VPS to visualize proposed developments in real-time, overlaying AR models onto existing urban environments. This technology aids in assessing the potential impact of new structures on the surrounding area, enhancing decision-making processes. Additionally, it can assist in efficient space utilization and public infrastructure planning, ensuring more accurate and sustainable urban development.</p><p dir="ltr"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzA3NDExOTI2NjQzLWJ1aWxkLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib"></p><h3 dir="ltr" id="isPasted">Asset tracking</h3><p dir="ltr">VPS greatly enhances asset tracking, especially in complex environments like large warehouses or manufacturing plants. By providing precise, real-time location data, it enables companies to monitor and manage the movement of assets effortlessly. This accuracy ensures more efficient inventory management, reduces the likelihood of misplacement, and speeds up retrieval processes. VPS's detailed spatial awareness also aids in optimizing asset layout and logistics, leading to improved operational efficiency and cost savings.</p><p dir="ltr"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzA3NDExOTc3NTI3LXdhcmVob3VzZS5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"></p><h3 dir="ltr" id="isPasted">WebAR</h3><p dir="ltr">VPS significantly boost the capabilities of WebAR, offering a more immersive and accurate augmented reality experience directly through web browsers. VPS enhances WebAR by providing precise spatial awareness and location tracking, allowing for more realistic and contextually relevant AR overlays on physical environments. The combination of VPS with WebAR paves the way for advanced applications in advertising, education, and entertainment, making these experiences more intuitive and seamlessly integrated into the physical world.</p><h3 dir="ltr">Robot navigation</h3><p dir="ltr">VPS can enhance robot navigation by providing precise location and orientation data, essential for autonomous movement in GPS-challenged environments like indoors or urban areas. This advancement allows robots to navigate efficiently and safely, avoiding obstacles and adapting to changing surroundings which is crucial in industrial warehouses to public spaces and healthcare facilities.</p><h2 dir="ltr">Conclusion</h2><p dir="ltr">The adoption of Visual Positioning Systems (VPS) in augmented reality marks a significant leap in merging digital and physical realms. Spearheaded by companies like Immersal, VPS transcends GPS limitations by offering unprecedented accuracy in spatial mapping and visual positioning, vital for both indoor and outdoor environments. This technology enhances AR applications across various domains, from improving visitor experiences in theme parks and aiding complex indoor navigation to transforming urban planning and streamlining industrial processes. The integration of VPS in AR is not just an enhancement of current capabilities; it's a gateway to unexplored possibilities, paving the way for a future where augmented reality is seamlessly woven into the fabric of daily life.</p><hr><h2>What will you build?</h2><h5>Whether you're building an AR experience for a large commercial space, a factory or industrial setting, or even if you'd just like to create an AR app for fun, Immersal is the way to do it.</h5><p>Immersal has just released their SDK 2.0 and new Mapper App, you can sign up for your free account <a href="https://immersal.com/register?WEVOLVER" target="_blank">here</a>.&nbsp;</p><p><a href="https://developers.immersal.com/?WEVOLVER=" target="_blank" rel="noopener noreferrer" class="button-main-action">Sign up for free here.&nbsp;</a><br><br></p>

Since its inception augmented and virtual reality has vastly evolved, in part thanks to advancements like Immersal's innovation in visual positioning to enhance AR by outpacing GPS limitations.

Unlocking the Potential of VPS in Augmented Reality

<p dir="ltr" id="isPasted">Environments where robots are being applied are becoming more complex and dynamix, which requires the development of more flexible and advanced vision and localization functions. An essential capability of robots operating in uncontrolled environments is identifying their surrounding. One solution to achieve this is the use of Time of Flight (ToF) imaging sensors with spot and flood illumination options.&nbsp;</p><p dir="ltr">ToF sensors enable robotic applications to image their environment and create a 3D map of their surrounding. Letting the robot know what objects are close as well as gain detailed information about their size, shape and edges. For example, a robotic vacuum cleaner could use this information to understand whether it can drive underneath a sofa or cabinet and clean there as well.&nbsp;</p><p dir="ltr">This article explores how ToF imaging sensors work, their applications and benefits for robotics.</p><h1 dir="ltr">Introducing ToF 3D Image Sensors and Working Principles</h1><p dir="ltr">ToF is an active measurement, meaning that modulated light is emitted by a laser source next to the ToF imager, and the ToF sensor receives the light reflected from the objects in the scene. Infrared (IR) light, e.g at 940nm, is usually used in these applications since it is invisible to humans. ToF imaging sensors currently available on the market are based on two measurement principles: direct and indirect ToF.</p><h3 dir="ltr">Direct ToF measurement</h3><p dir="ltr">For direct ToF measurements, the laser source emits short laser pulses, and the ToF sensor measures the delay between the emitted and the reflected laser pulse. The time delay between the pulse emission and the detection in the sensor (time of flight) is converted into a distance measurement to rebuild a 3D scene. The imaging sensors for direct ToF measurements is more complex as fast detectors (SPADs) are needed and on-chip histograms have to be calculated to enable the necessary peak finding correlating with object surfaces.</p><p dir="ltr"><span class="fr-img-caption fr-fic fr-dib" style="width: 304px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjkyMTIyMTgwNDEwLXRvZiAxLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 302px;" class="fr-fic fr-dib"><span class="fr-inner" spellcheck="false">The simple mechanism of direct ToF sensor. Image credit: [1]</span></span></span></p><h3 dir="ltr" id="isPasted">Indirect ToF measurement</h3><p>For indirect ToF measurements, the laser source, usually vertical-cavity surface-emitting lasers (VCSEL), emits modulated infrared (IR) light with a wavelength between 850 and 940 nm. VCSEL is used over LEDs because they present better results when operating with modulation frequencies of 80 to 100 MHz and have a narrow optical spectrum<a href="https://www.rutronik.com/article/vcsels-for-tof-applications-how-vcsels-can-be-best-put-to-use" target="_blank">2</a>. &nbsp;The ToF sensor captures the reflected signal through its lens and each pixel compares the emitted modulated signal with the captured from received from the scene. The fast global shutter ToF pixel collects the amount of light overlapping with the emitted signal and the amount arriving later. From this information the phase shift between both signals is then determined. For indirect ToF so called phase images are the output of the sensor. Depending on sensor and application 2, 3 and 4 phase-based approaches are used. The advantage of indirect ToF is that no on-chip processing is required.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjkyMTIyMjQyMzQzLTAxIGltYWdlLTAyICgyKSAoMSkuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner" spellcheck="false">Detectors collect light alternately from the object of interest. Image credit: [1]</span></span></span></p><h3>ToF sensors output data</h3><p dir="ltr">In both measurement methods, the measured data is converted on-chip into a digital representation, phase images for indirect ToF and histograms followed by peak finding for direct ToF. Then, the representation is transferred to an application processor (AP) as raw 3D data either via MIPI-CSI-2 or parallel (PIF) interfaces. By processing the raw data, the AP provides two images, a depth map and an amplitude image. The depth map contains the depth information for each pixel that is the distance at this pixel between the object and the camera. The amplitude image contains the amplitude information, representing the brightnessof the object at that position. Sophisticated systems can also provide additional images, such as a confidence image indicating each pixel&#39;s quality measures.</p><p dir="ltr"><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjkyMTIyMzU0MTI3LXRvZiAzLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner" spellcheck="false">Time-of-Flight block diagram. Image credit: Infineon.</span></span></span></p><p dir="ltr" id="isPasted">Depending on the application, a ToF sensor can be combined with an RGB image. In this case, algorithms can combine the RGB and depth images to obtain 3D representations of the scene and its objects containing both depth and color information. This allows advanced application functions such as object recognition to e.g. let the robot decide if the object is e.g. a toy that should not be touched by driving with a safety distance around it or a cabinet that should cleaned closeby. Or face authentication identifying persons based on their facial biometrics in a reliable and secure way.</p><h1 dir="ltr">Benefits of ToF Sensors</h1><p dir="ltr">ToF sensors replace contact and proximity sensors, featuring a high accuracy that allows the detection of small objects thanks to the high number of pixels available. Another advantage of ToF sensors over contact sensors is the absence of mechanically moving parts, which reduces maintenance requirements. In particular for robotics existing LDS (laser distance scanning) sensors are bulky, expensive and have mechanically rotating mirror. ToF imaging sensors are much smaller, more cost effective and do not need mechanically moving parts.</p><p dir="ltr">ToF sensors can also be used in darkness and strong sunlight conditions due to their ambient light robustness. Furthermore, these sensors present a wide distance range, being capable of detecting objects up to 10m away. This feature enables their use for indoor and outdoor applications. Finally, the small camera dimensions enable integration into smaller robots such as consumer robot vacuum cleaners or drones.</p><h1 dir="ltr" id="isPasted">ToF Sensors Role in Robotics Application</h1><p dir="ltr">ToF sensors benefit different robotic applications, including: &nbsp;</p><ul><li dir="ltr"><p dir="ltr">Smart obstacle avoidance</p></li><li dir="ltr"><p dir="ltr">Navigation and path planning</p></li><li dir="ltr"><p dir="ltr">3D mapping and localization (SLAM)</p></li><li dir="ltr"><p dir="ltr">Wall following</p></li><li dir="ltr"><p dir="ltr">Hand and finger gesture recognition</p></li><li dir="ltr"><p dir="ltr">Object or person tracking</p></li><li dir="ltr"><p dir="ltr">Object picking&nbsp;</p></li><li dir="ltr"><p dir="ltr">Cliff detection</p></li></ul><p dir="ltr">Each application has its own complexity and specific focus, and ToF sensors can cover them excellently.</p><h2 dir="ltr">Smart obstacle avoidance</h2><p dir="ltr">Examples of smart obstacle-avoidance applications include vacuum cleaners, transport/logistic robots, autonomous mobile robots (AMRs), robot forklifts, and warehouse systems. By using ToF sensors, robots can precisely detect the distance to objects in their surroundings. Unlike other technologies, ToF sensors do not require specific training to detect particular obstacles and object borders can be easily extracted from the 3D ToF images. Furthermore, ToF systems with wide field-of-view cameras make it feasible to detect even small obstacles.</p><p dir="ltr"><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1692122629739-tof+bear.jpg" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner" spellcheck="false">Obstacle segmentation using ToF sensor. Image credit: Infineon.</span></span></span></p><h2 dir="ltr" id="isPasted">Localization and mapping for robots/drones (SLAM)</h2><p dir="ltr">ToF sensors are utilized in drone or robot-based 3D mapping solutions to provide precise long-range depth information and high-resolution 3D maps due to their high frame rate. This allows the generating of a 3D point cloud, where each point represents a unique point on the surface ofan object with its own set of Cartesian coordinates.The point cloud is used by SLAM algorithms to generate a precise map of the robot&rsquo;s environment and localize the robot itself in this map.</p><h2 dir="ltr">Human interaction robots</h2><p dir="ltr">The depth and amplitude images generated based on ToF data can be used that robots know the human presence, identify persons and determine their location. Depending on its application, the robots can identify intruders or use the 3D point cloud data to understand and respond to human gestures, even detecting subtle movements of the fingers of a person. Moreover, in advanced applications, the robot could identify family members based on biometrics.</p><h2 dir="ltr">Other applications</h2><p dir="ltr">The depth information provided by ToF measurements improves the success rate of robots used to manipulate, pick up and deliver objects. A ToF sensor provides more robust data than traditional sensors, thus also improving the so called cliff detection of robots avoiding them to fall off stairs or other edges.</p><h1 dir="ltr">ToF Imaging Sensors from Infineon</h1><p dir="ltr">An attractive option in the market for ToF sensors is the <a href="https://www.infineon.com/cms/en/product/sensor/tof-3d-image-sensors/" target="_blank">REAL3&trade; image sensor family</a>. It is developed by Infineon Technologies in collaboration with <a href="https://pmdtec.com/en/" target="_blank">pmdtechnologies</a>, a premium partner company from Infineon. These sensors use infrared light for ToF-based depth sensing and facilitate a sophisticated ToF pixel with SBI and microlenses for each pixel for a better sensitivity.. The sensors allow a fast readout via MIPI CSI-2. In addition, the power consumption is low, extending the operation time for battery driven devices.</p><p dir="ltr">The REAL3 sensors provide high image quality with up to VGA resolution and an enhanced sensing range of up to 10 m, making it possible to build 3D maps in real time. The sensor uses SBI (suppression of background light) technology to boost the ambient light robustness, being capable of working even in bright sunlight conditions. Thus, the features and parameter configuration allow the REAL3 to adapt fast to new operating environments.</p><p dir="ltr"><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1692122726052-tof+final.png" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner" spellcheck="false">Principles of work and usage of REAL3&trade; 3D image sensors. Image credit: Infineon.</span></span></span></p><p dir="ltr" id="isPasted">The Infineon portfolio of ToF sensor products is designed to cater to diverse markets. The REAL3 sensor family is commercially available for consumer and automotive applications, with ASIL B/ISO26262 qualification. Each REAL3 model has different resolutions optimized for different applications, covering the requirements of several <a href="https://www.infineon.com/cms/en/product/sensor/tof-3d-image-sensors/tof-3d-image-sensors-for-consumer/" target="_blank">consumer</a>, <a href="https://www.infineon.com/cms/en/product/sensor/tof-3d-image-sensors/tof-3d-image-sensors-for-automotive/" target="_blank">automotive</a>, and <a href="https://www.infineon.com/cms/en/product/sensor/tof-3d-image-sensors/tof-3d-image-sensors-for-consumer/" target="_blank">industrial</a> applications. To make it easier for you to start your project quickly, Infineon has an extensive <a href="https://www.infineon.com/cms/en/product/sensor/tof-3d-image-sensors/#!partners" target="_blank">list of partners</a> specialized in software and hardware worldwide.&nbsp;</p><p dir="ltr">Developers can also benefit from the solutions offered by the partnership between pmdtechnologies and Infineon. The <a href="https://3d.pmdtec.com/en/3d-cameras/flexx2/" target="_blank">flexx2 family</a> is a development kit for prototyping, research, and mass production, featuring Infineon&#39;s latest ToF image sensor for precise depth-sensing. Flexx2 can be connected to Linux or Windows operating systems with USB 3.0. From the industrial perspective, the <a href="https://3d.pmdtec.com/en/ecosystem/" target="_blank">O3R platform</a> is a starting point for developing robotics solutions, such as autonomous vehicles, for the industrial environment. It can support up to 6 3D ToF cameras with Gigabit Ethernet, USB 3.0 and CAN connections for other sensors.</p><h1 dir="ltr" id="isPasted">Conclusion</h1><p dir="ltr">Precisely detecting the surrounding environment and safely performing tasks have been crucial capabilities to be developed in robots. Sensors using ToF measurements are a very efficient solution for such applications.&nbsp;</p><p dir="ltr">The REAL3&trade; image sensor family is an excellent example of high-quality ToF-based sensors. They are manufactured with cutting-edge technologies that improve detection capabilities for different conditions and applications. As the use of robots is growing, exploring and taking advantage of these sensors can benefit the development of future products.</p><h1 dir="ltr">References</h1><ol><li dir="ltr"><p dir="ltr">Fatima Khalid, 2022. &ldquo;<a href="https://www.wevolver.com/article/tof-3d-image-sensors-are-changing-the-way-we-engage-with-photography-and-mixed-reality" target="_blank">ToF 3D image sensors are changing the way we engage with photography and mixed reality</a>&rdquo;</p></li><li dir="ltr"><p dir="ltr">Rutronik, 2022. &ldquo;<a href="https://www.rutronik.com/article/vcsels-for-tof-applications-how-vcsels-can-be-best-put-to-use" target="_blank">VCSELs for ToF Applications - How VCSELs can be best put to use</a>&rdquo;</p></li><li dir="ltr"><p dir="ltr">Tobias Rothermel, Walter Bell, 2022. &ldquo;<a href="https://event.on24.com/eventRegistration/console/apollox/mainEvent?&eventid=4026408&sessionid=1&username=&partnerref=&format=fhvideo1&mobile=&flashsupportedmobiledevice=&helpcenter=&key=4798DA12B2C2875E862D7C41274D3149&newConsole=true&nxChe=true&newTabCon=true&consoleEarEventConsole=false&text_language_id=en&playerwidth=748&playerheight=526&eventuserid=598343268&contenttype=A&mediametricsessionid=521386407&mediametricid=5673041&usercd=598343268&mode=launch" target="_blank">Building robots is hard! How to build a smarter robot faster with Time-of-Flight</a>&rdquo;.</p></li><li dir="ltr"><p dir="ltr">Infineon, &ldquo;<a href="https://www.infineon.com/cms/en/product/sensor/tof-3d-image-sensors/tof-3d-image-sensors-for-consumer/" target="_blank">ToF 3D image sensors for consumer and industrial</a>&rdquo;</p></li><li dir="ltr"><p dir="ltr">Infineon, 2023. &ldquo;<a href="https://www.infineon.com/cms/en/about-infineon/press/market-news/2023/INFPSS202302-067.html" target="_blank">New i-ToF imager enables smallest 3D camera systems with improved quantum efficiency at optimized cost</a>&rdquo;</p></li></ol>

Exploring the working principles, applications, and benefits of ToF sensors in robotics.


Revolutionizing Robotics with ToF Sensors for High-Accuracy 3D Imaging and SLAM

A LiDAR sensor is more compact mountable on a vehicle for driver assistance

Explore the key differences, advantages, and applications of lidar and radar technologies in modern engineering projects.

Lidar vs. Radar: Comprehensive Comparison and Analysis

<p dir="ltr" id="isPasted">Point clouds are an important concept in 3D scanning and <a href="https://www.wevolver.com/article/how-to-make-a-3d-model-for-printing" target="_blank" rel="noopener noreferrer">3D modeling</a>. Sometimes consisting of tens of millions of individual data points, point clouds can represent anything from a tiny piece of jewelry to an entire football stadium.</p><p dir="ltr">So what exactly are point clouds? Where do they come from and how are they used? This article goes over the essential facts, looking at the basics of 3D scanning, point cloud formats, and point cloud software.</p><h1 dir="ltr">What is 3D scanning?</h1><p dir="ltr">Before we get into the nitty gritty of point clouds, we should take a moment to familiarize ourselves with 3D scanning.</p><p dir="ltr">3D scanning is the process of taking a physical, real-world object — an industrial part, a piece of pottery, a human being… you name it — and collecting data about its 3D shape in order to create a digital 3D model of it.</p><p dir="ltr">There are a few variations on the 3D scanning process, all of which use slightly different methods to collect data about an object’s shape. Some of the most common 3D scanning technologies include:</p><ul><li dir="ltr"><p dir="ltr"><strong>Laser scanning</strong>: A laser is shined at the object, and a sensor measures the distance between the light source and the surface of the object, either by timing how long it takes for the light to return (“time of flight” or Lidar) or by triangulation.</p></li><li dir="ltr"><p dir="ltr"><strong>Structured light scanning</strong>: Patterned lines of light are projected onto the object, and cameras are used to analyze how the lines deform over the surface of the object. These deformations indicate the shape of the object.</p></li><li dir="ltr"><p dir="ltr"><strong>Photogrammetry</strong>: Photographs are taken of the object from many different angles, and software stitches these images together to figure out the 3D shape of the object.</p></li></ul><p dir="ltr">The use cases for 3D scanning are nearly endless. In various industrial disciplines, scanners can be used for reverse engineering: a part can be scanned, turned into a digital 3D model, then reproduced in physical form. Other applications include surveying (in areas like construction and civil engineering) using photogrammetry, building information modeling (BIM), crime scene analysis using handheld 3D laser scanners, capture of real-world objects for animation and video games, and scanning body parts in medicine and dentistry.</p><p dir="ltr">3D scanning is also widely used in tandem with digital manufacturing techniques like 3D printing and CNC machining, as scan data can be converted into a CAD model or into a mesh format such as STL, which can then be turned into <a href="https://www.wevolver.com/article/your-guide-to-3d-printing-g-code" target="_blank" rel="noopener noreferrer">machine-readable G-code</a>.</p><p dir="ltr"><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjY3OTgwMDkxMjQ5LTNkX3NjYW5uZXIuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib" alt="structured light"><span class="fr-inner">Structured light scanning for reverse engineering</span></span></span></p><h1 dir="ltr">What is a point cloud?</h1><p dir="ltr">A point cloud is the immediate digital output of the 3D scanning process. When you use a 3D scanner to capture the shape of a physical object, the resulting file format is a point cloud.</p><p dir="ltr">In short, a point cloud is a collection of spatial data points, with each individual point representing a tiny spot on the surface of the scanned object (collected using one of the technologies described above). The point is expressed as a set of X, Y, Z coordinates. Because there are so many of these individual data points, the point cloud as a whole can be used to understand the 3D shape of the scanned object.</p><p dir="ltr">However, not all point clouds are equal: more data points and a higher-density point cloud means more detail, whereas a point cloud with just a few data points will only give an approximate representation of the real-world object. That being said, high-density point clouds require more computer memory and processing power.</p><p dir="ltr">So how many individual points does a 3D point cloud contain? This varies by the size of the scanned object and the chosen resolution of the scan, but the number is typically in the millions. This makes the processing of point cloud data fairly CPU-intensive.</p><p dir="ltr">Depending on the scanning technology, a data point in a point cloud may also contain RGB color (and brightness) information.</p><h1 dir="ltr">Point cloud file formats</h1><p dir="ltr">There are a few different file formats for point clouds, which can make the handling of the raw data confusing. However, most formats fall into two categories, and many are similar to one another and therefore easy to convert. Some formats are universal, while others are proprietary and have been developed by 3D scanning hardware manufacturers.</p><h2 dir="ltr">ASCII</h2><p dir="ltr">The first major type of point cloud format is ASCII (American Standard Code for Information Interchange), which conveys information using text. Each line of text within an ASCII point cloud represents X, Y, Z spatial coordinates.</p><p dir="ltr"><em>Common ASCII extensions: XYZ, OBJ, ASC, PTX</em></p><h2 dir="ltr">Binary</h2><p dir="ltr">The other major type of point cloud format is binary, which retains all information in binary code. Although binary data is harder to access and interpret, the files themselves are much smaller than ASCII files.</p><p dir="ltr"><em>Common binary extensions: LAS, PCD, FLS</em></p><h1 dir="ltr">Turning a point cloud into a 3D model</h1><p dir="ltr">Point clouds are a highly effective way of arranging information about the physical world, but they are not particularly useful for further activity once stored on a computer. Although it is possible to use point cloud datasets directly, many users will ultimately convert the point cloud into one of the following file types:</p><ul><li dir="ltr"><p dir="ltr">CAD model, allowing the 3D scan data to be modified (reverse engineering etc.)</p></li><li dir="ltr"><p dir="ltr">NURBS surface model, turning the 3D scan data into a digital 3D object with curved surfaces (on-screen visualization, video game rendering etc.)</p></li><li dir="ltr"><p dir="ltr">Mesh, turning the 3D scan data into a polygon or triangle mesh (3D printing etc.)</p></li></ul><p dir="ltr">The process of turning a point cloud into a 3D model is known as surface reconstruction. This is because a point cloud itself does not contain “surfaces,” only a multitude of discrete coordinates.</p><p dir="ltr">Of course, the conversion of point cloud data into 3D models is not a simple process. Point cloud processing software must use algorithms to figure out how to represent a collection of points as lines or shapes while determining which data points are genuine and which are outliers. The best software uses deep learning to contextualize point clouds and understand what the data is supposed to represent.</p><p dir="ltr"><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjY3OTgwMTU1MzQ1LXBvbHlnb25hbF9tZXNoLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib" alt="mesh"><span class="fr-inner">A point cloud can be transformed into a polygonal mesh</span></span></span></p><h1 dir="ltr">Point cloud processing software</h1><p dir="ltr">Users may need various types of software to create a smooth 3D scanning workflow. However, one of the most important tools is point cloud processing software. This software is used to manually edit point clouds (to remove outlying data points, for example) and render them visually. Different algorithms can be deployed to automatically filter outlying data points.[1]</p><p dir="ltr">Functions of point cloud processing software may include:</p><ul><li dir="ltr"><p dir="ltr">Manual editing of the point cloud</p></li><li dir="ltr"><p dir="ltr">Automatic cleanup and filtering</p></li><li dir="ltr"><p dir="ltr">Offloading tasks to the cloud</p></li><li dir="ltr"><p dir="ltr">Rendering</p></li><li dir="ltr"><p dir="ltr">Point distance calculation</p></li><li dir="ltr"><p dir="ltr">Registration</p></li><li dir="ltr"><p dir="ltr">Conversion to mesh</p></li><li dir="ltr"><p dir="ltr">Conversation to NURBS</p></li></ul><p dir="ltr">Popular point cloud processing software includes CloudCompare (free and open source), Point Cloud Library (free and open source), Leica Cyclone, Faro Scene, Autodesk Recap, and Trimble RealWorks.</p><h3 dir="ltr">Point cloud registration</h3><p dir="ltr">For 3D scanning applications like product quality control and <a href="https://www.wevolver.com/article/3d-scanning-for-inspection-are-3d-scanners-a-smart-choice" target="_blank">inspection</a>, it may be necessary to perform a process known as point cloud registration, otherwise known as point-set registration or scan matching. The process also has numerous uses in computer vision applications.[2]</p><p dir="ltr">This is when a point cloud is aligned with another point cloud (or a 3D model) that represents the same physical object. Registration serves a few different purposes, including the merging of two or more data points from separate scans to create a more accurate data set (when using two different scanning technologies to capture a single object, for example), and the comparison of one cloud with another in order to note any discrepancies.&nbsp;</p><p dir="ltr">One application of point cloud registration is in quality control, where a point cloud obtained from a 3D scan of a manufactured part can be compared with a point cloud from the original CAD model for the part. This can show whether the manufactured part is faithful to the CAD model, or whether the manufacturing process has produced defects.</p><h1 dir="ltr">Making point clouds 3D printable</h1><p dir="ltr">3D scanning can be used in combination with 3D printing. This combination is useful for applications like fabricating spare and replacement parts: a 3D scanner can be used to capture a physical part; a 3D model can then be created (and modified) before being 3D printed, resulting in a duplicate of the original part.</p><p dir="ltr">To make point cloud data 3D printable, it needs to be converted into a triangle mesh using mesh processing software such as MeshLab (free and open source). Such software can fix holes in the point cloud and turn it into a format such as STL, the standard format for additive manufacturing.</p><p dir="ltr">Unlike a point cloud, an STL file describes the surface geometry of a 3D object through interconnected triangles. Such a file can be “sliced” by 3D printer slicing software to turn it into G-code, which instructs the 3D printer how to make the part.</p><p dir="ltr"><strong>Recommended reading: <a href="https://www.wevolver.com/article/creating-spare-parts-with-3d-scanning-and-3d-printing" target="_blank"><em>Creating Spare Parts with 3D Scanning and 3D Printing</em></a></strong></p><h1 dir="ltr">Key takeaways</h1><p dir="ltr">All 3D scanning technologies produce point clouds. Whether you use a handheld 3D laser scanner, a desktop structured light scanner, or a photogrammetry setup with digital cameras, the digital output of your scans will be a point cloud.</p><p dir="ltr">Processing the point cloud is not always straightforward, as dense clouds can be very large and CPU-intensive. However, there are good software options, including those packaged with 3D scanning hardware, as well as free applications like CloudCompare. Additionally, most users will need to perform surface reconstruction on their point cloud in order to create a 3D model that isn’t simply a collection of spatial data points.</p><p dir="ltr">Once you have become familiar with point clouds and how to handle them, the applications are diverse, from 3D city mapping to animation to 3D printing.</p><h1 dir="ltr">References</h1><p dir="ltr">[1] Han XF, Jin JS, Wang MJ, Jiang W, Gao L, Xiao L. A review of algorithms for filtering the 3D point cloud. Signal Processing: Image Communication. 2017 Sep 1;57:103-12.</p><p dir="ltr">[2] Huang X, Mei G, Zhang J, Abbas R. A comprehensive survey on point cloud registration. arXiv preprint arXiv:2103.02690. 2021 Mar 3.</p>

3D scanners produce point clouds of varying densities

Beginners might be surprised to find that their 3D scans turn out not as solid 3D models, but as clusters of spatial data points. So what exactly is a point cloud, and what can you do with it?

Point clouds explained: scanning, processing, 3D models

<p id="isPasted">On the 24<sup>th</sup> May 2022 in Uvalde, Texas, a gunman killed 19 children and two adults at an elementary school. This is part of the events <a href="https://www.texastribune.org/2022/05/27/uvalde-texas-school-shooting-timeline/" target="_blank">timeline</a>, focused on the police functioning during this event:&nbsp;</p><ul><li><u>11:28 a.m.</u> - Shooter arrives at school</li><li><u>11:30 a.m.</u> - A teacher calls 911</li><li><u>11:35 a.m.</u> - &ldquo;&hellip;Pete Arredondo, the chief of the school district&rsquo;s police department, arrives at the scene around this time. He does not use his radio. <span dir="RTL">)</span><strong>Arredondo wanted both hands on his gun if he encountered the shooter and believed the radio would have slowed him down</strong>, his attorney said.&rdquo;<span dir="RTL">(</span></li><li><u>12:03 p.m.</u> - A student calls 911 from room 112 for a minute and 23 seconds and identifies herself in a whisper. Meanwhile, as many as 19 officers <strong>are positioned in the school&rsquo;s hallway</strong>.</li><li><u>12:46 p.m.</u> - Breach command is given. Arredondo gives his approval to enter the classroom. &ldquo;If y&rsquo;all are ready to do it, you do it,&rdquo; he says, according to a transcript of police body-camera footage.</li><li><u>12:50 p.m.</u> - Forces kill gunman&nbsp;</li></ul><p><strong>It took one hour and twenty minutes from the first 911 call until the gunman was neutralized!</strong></p><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe src="https://www.youtube.com/embed/1NqlAcD-OqE?&ab_channel=WFAA&wmode=opaque&rel=0&enablejsapi=1&origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" style="width: 720px; height: 480px;" data-gtm-yt-inspected-11="true" id="559664948" title="Uvalde shooting: Police bodycam footage shows moments when law enforcement entered Robb Elementary"></iframe></span></p><h2><strong>The problem&nbsp;</strong></h2><p>Emergency responders all over the world need an immediate understanding of their surroundings. This critical information makes the difference between lives saved and unfortunately, horrible disasters.<br>All forces involved share the same desire for precise information ASAP in a clear and visual way that will<span dir="RTL">&nbsp;</span>enable them to make smart decisions in real-time.</p><p>From investigating the horrible attack in Texas, we can specify a few points that were handled poorly:</p><h3><strong>Non-unified data management in control rooms led to inefficient management of the forces.&nbsp;</strong></h3><p>Control rooms and dispatches are constructed of multiple sub-systems and&nbsp;multiple&nbsp;screens. These subsystems rely<span dir="RTL">&nbsp;</span>on:</p><ul><li>Extensive operator training</li><li>The misassumption that operators are not suffering from cognitive load&nbsp;</li><li>The ability to communicate with each other simultaneously</li></ul><p>These factors can cause fragmented data, which can easily lead to the loss of important information and in some cases can cost human lives.<span dir="RTL">&nbsp;</span></p><p><a href="https://www.statista.com/statistics/1183457/iot-connected-devices-worldwide/" target="_blank">In a six-year period, the amount of IoT devices in the world is expected to double</a>, which will make the cognitive load even more challenging and almost impossible to process without data fusion systems.</p><p><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjU4MTc1NjY4NTQ3LXJpb19jb250cm9sX3Jvb21fZ2VvcmdlX3NvYXJlcy5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 720px; height: 480px;" class="fr-fic fr-dib"></p><h3><strong>Unclear visual picture and lack of critical elements geolocation in the school.</strong></h3><p>Emergency responders that arrived at the scene were missing relevant visual information and understanding of the current status at the elementary school - in particular, regarding the attacker&rsquo;s and civilians&#39; locations.&nbsp;</p><h3><strong>Operators&rsquo; limitation to free his hands for radio &amp; tech devices (tablets, etc.).</strong></h3><p>A major fact is that Chief Arredondo was only focused on his gun, claiming to be unable to free his hands for the radio, even at the price of being cut off from communication with others.</p><h3><strong>The flow of information from control rooms to the field.</strong></h3><p>It is clear from this timeline that the information <strong>was</strong> <strong>available</strong> to the control room streaming constantly to the 911 dispatch via oral information.&nbsp;</p><p>Moreover, the ability to exploit accessible data such as smartphone devices of the students or even the gunman was possible, yet, unfortunately, unavailable to the emergency responders as a result of poor distribution to the field.&nbsp;</p><h2>The Solution</h2><p>The ultimate solution for this type of scenario will involve taking all the data that was streamed in real-time to the 911 dispatch, combined with any other accessible data such as the students&rsquo; smartphone information, static cameras deployed in the school and other friendly forces&rsquo; locations that were on-site in order to bring to any relevant emergency responder a <strong>comprehensive image</strong> of what&#39;s going on in a way that will <strong>improve efficiency</strong> and drastically increase situational awareness.</p><p>Oversight develops <em><strong>Absorber</strong></em><strong>&nbsp;- A visual, 3D, Command-and-Control (C&amp;C) solution based on augmented and virtual reality tools</strong> enabling information sharing and management on top of Geographic Information System (GIS) between many users simultaneously. The system does not require a physical location to run in and can be managed remotely.</p><h3><strong>Complete data fusion - one platform for all information stitched together in real-time</strong></h3><p>This product has a novel unified and distributed data management system.<strong>&nbsp;</strong>All devices that are map-locatable and can produce valuable data to the system, (hereinafter, data collection agents) - <u>Drones and other unmanned aerial vehicles, robots, IoT sensors, smartphones, XR headsets (extended reality), PTZ cameras or other stationery cameras&nbsp;</u>and other sources that produce localization information or other visual information.</p><h3><strong>XR-based display overlaid on reality image</strong></h3><p>Another breakthrough is connecting information to the actual world by displaying data collection agents in real-time on the user reality image. Seeing things without any physical limitations will open a whole new world for managing data.</p><h3><strong>Easy to operate</strong></h3><p>Controlling information on such a system is done using the operator&rsquo;s <strong>hands, head and eye gestures</strong> in a common and natural way. This will reduce the number of operators and the level of training required. Logics can be in-built in order to eliminate the operator needing to deal with other devices. For example, he will be able to talk to a partner only by focusing on them with his eyes.</p><h3><strong>Decentralized multiple-player system</strong></h3><p>This method for controlling agents is especially suitable for a rear control room operation while the user in the field can also participate by managing data. All data can be shared on one system, making the system mobile and not so dependent on hardware-based devices.</p><h3><strong>Bottom line</strong></h3><p><em>Absorber&nbsp;</em>is a breakthrough all-in-one system that can independently operate throughout a developing situation and communicate information in real-time. <strong>It enables an immediate flow of information and smart decision making</strong></p><h3><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe src="https://www.youtube.com/embed/7WTHrbVi7r0?&t=7s&ab_channel=Oversight&wmode=opaque&rel=0&enablejsapi=1&origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" style="width: 727px; height: 454px;" data-gtm-yt-inspected-11="true" id="283478871" title="Absorber by Oversight"></iframe></span></h3><h2><em>Absorber&nbsp;</em>and Its Components</h2><p>The different data collection agents <em>Absorber&nbsp;</em>will support have the common ability to continuously convey their current location. This is a basic requirement of the system to allow accurate determining of each agent&rsquo;s location, their spatial relation to others and displaying them on a hologram-based 3D digital table-top view board using XR headsets or smartphones.&nbsp;</p><p><em>Absorber&nbsp;</em>will serve as an add-on or replacement for agent management and can enhance operators&rsquo; situational awareness in real-time.&nbsp;</p><h3>Collectors&nbsp;</h3><p><em>Absorber&nbsp;</em>consists of &ldquo;collectors&rdquo; - software utilities for connecting devices to the system. Collectors convert the protocol of the connected device into <em>Absorber&rsquo;s&nbsp;</em>proprietary protocol - that&rsquo;s what makes the system unified, since each device can have different operating systems, SDK, units of measurement or even global datum reference.</p><p>XR devices serve as the system display and also act as data collection agents since most XR devices can report their relative location, produce a real-time map and report device status. By connecting them to a GPS device they can also be a map-locatable device.</p><p>The very use of immersive technology and holograms to represent information will make it possible to build a multi-user system that can share information even from remote locations.</p><h3>Core &nbsp;</h3><p><em>Absorber&rsquo;s&nbsp;</em>core components are media server for managing videos from multiple devices, map server in order to store relevant geocell information for a comprehensive on-premises solution and an interactive simulation engine in order to create events and scenarios for an offline training solution. Data fusion logics for managing multiple agents will be built on top of that collaborative information.</p><h3>Users and SDK</h3><p>On the other side, we have the end-users who can be control room operators or even emergency responders. The system will also deliver SDK to let other users build solutions on top of this collaborative information.</p><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe src="https://player.vimeo.com/video/731725270?autoplay=1&loop=1&autopause=0&background=1&muted=1&quality=1080p" width="560" height="315" frameborder="0" allowfullscreen="" class="fr-draggable" data-gtm-yt-inspected-11="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span><br></p>

Using AR/VR technology, Oversight gives emergency responders the ability to literally SEE THROUGH WALLS and strengthens the connectivity between control rooms and field operators.

Enhancing Emergency Responders Situational Awareness Using XR Tools

<p id="isPasted">Three-dimensional (3D) mapping is a very useful tool, such as for monitoring construction sites, tracking the effects of climate change on ecosystems and verifying the safety of roads and bridges. However, the technology currently used to automate the mapping process is limited, making it a long and costly endeavor.</p><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe src="https://www.youtube.com/embed/ZIDHoDInfTg?&wmode=opaque&rel=0&enablejsapi=1&origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" style="width: 767px; height: 435px;"></iframe></span></p><p>&ldquo;Switzerland is currently mapping its entire landscape using airborne laser scanners &ndash; the first time since 2000. But the process will take four to five years since the scanners have to fly at an altitude below one kilometer if they are to collect data with sufficient detail and accuracy,&rdquo; says Jan Skaloud, a senior scientist at the Geodetic Engineering Laboratory (Topo) within EPFL&#39;s School of Architecture, Civil and Environmental Engineering (ENAC). &ldquo;With our method, surveyors can send laser scanners as high as five kilometers and still maintain accuracy. Our lasers are more sensitive and can beam light over a much wider area, making the process five times faster.&rdquo;</p><p>The method is described in a paper published in ISPRS Journal of Photogrammetry and Remote Sensing by Davide Cucci, a senior research associate at the Research Center for Statistics of the Geneva School of Economics and Management of the University of Geneva, who works with Topo regularly, Jan Skaloud, and Aur&eacute;lien Brun, lead author, a recent Master&rsquo;s graduate from EPFL and winner of an award from the Western Switzerland Association of Surveyor Engineers (IGSO).</p><h2><strong>Missing the point</strong></h2><p>LiDAR laser scanners beam millions of pulses of light on surfaces to create high-resolution digital twins &ndash; computer-based replicas of objects or landscapes &ndash; that can be used in architecture, road systems and manufacturing, for example. Lasers are particularly effective at collecting spatial data since they don&rsquo;t depend on ambient light, can collect accurate data at large distances and can essentially &ldquo;see through&rdquo; vegetation. But lasers&rsquo; accuracy is often lost when they&rsquo;re mounted on drones or other moving vehicles, especially in areas with numerous obstacles like dense cities, underground infrastructure sites, and places where GPS signals are interrupted. This results in gaps and misalignments in the datapoints used to generate 3D maps (also known as laser-point clouds), and can lead to double vision of scanned objects. These errors must be corrected manually before a map can be used.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 768px;"><span class="fr-img-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1653639402509-8084ff1b.jpg" style="width: 766px;" class="fr-fic fr-dib"><span class="fr-inner">ln Leysin, a LiDAR mounted on a drone maps the landscape, 28 March 2022. &copy; Topo/EPFL&nbsp;</span></span></span></p><p>&ldquo;For now, there&rsquo;s no way to generate perfectly aligned 3D maps without a manual data-correction step,&rdquo; says Cucci. &ldquo;A lot of semi-automatic methods are being explored to overcome this problem, but ours has the advantage of resolving the issue directly at the scanner level, where measurements are taken, eliminating the need to subsequently make corrections. It&rsquo;s also fully software-driven, meaning it can be implemented quickly and seamlessly by end users.&rdquo;</p><h2><strong>On the road to automation</strong></h2><p>The Topo method leverages recent advancements in artificial intelligence to detect when a given object has been scanned several times from different angles. The method involves selecting correspondences and inserting them into what&rsquo;s called a Dynamic Network, in order to correct gaps and misalignments in the laser-point cloud.</p><p>&ldquo;We&rsquo;re bringing more automation to 3D mapping technology, which will go a long way towards improving its efficiency and productivity and allow for a much wider range of applications,&rdquo; says Skaloud.</p><hr><h3><strong>References</strong></h3><p><a href="https://doi.org/10.1016/j.isprsjprs.2022.04.027" rel="noopener" target="_blank">Aur&eacute;lien Brun, Davide A. Cucci and Jan Skaloud. &ldquo;LiDAR Point-to-point Correspondences for Rigorous Registration of Kinematic Scanning in Dynamic Networks.&rdquo; ISPRS Journal of Photogrammetry and Remote Sensing., 19 May 2022</a></p>

The authors Davide A. Cucci, Aurélien Brun and Jan Skaloud. © Alain Herzog/EPFL

Engineers at EPFL and the University of Geneva believe they hold the key to automated drone mapping. By combining artificial intelligence with a new algorithm, their method promises to considerably reduce the time and resources needed to accurately scan complex landscapes.

The missing piece to faster, cheaper and more accurate 3D mapping

<p>When rescue service personnel go to the aid of people trapped in a burning or unsafe building, they are entering the unknown: they do not know what danger they will face, or where the obstacles stand between them and the people they have come to save. And in a fire or after an earthquake, first responders are often quickly disorientated inside a building.</p><p>Emergency services personnel, soldiers, and others could operate more quickly and safely if they had aid to navigation inside an unknown building. But indoors, GPS signals are unavailable or patchy. So how can people&rsquo;s position be tracked indoors, in a tunnel or underground, in the absence of satellite navigation?</p><p>This was the challenge that the Canadian government asked a team of experts in the <a href="https://carleton.ca/embedded-and-multisensor-systems/" rel="noopener noreferrer" target="_blank">Embedded and Multi-sensor Systems Lab (EMSLab)</a> at Ottawa&rsquo;s <a href="https://carleton.ca/" rel="noopener noreferrer" target="_blank">Carleton University</a> to solve. The system that they developed has a special motion tracker at its heart. Small in size but big in importance, the <a href="https://www.xsens.com/mti-300" rel="noopener noreferrer" target="_blank">MTi-300 Attitude and Heading Reference System (AHRS)</a> from Xsens provides a secure foundation for the operation of a sophisticated array of radio and optical sensors which build a 3D map of any building as the operator moves through it.</p><p><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1653017477909-Mohamed+with+MTi-300+2-png.png" style="width: 300px;" class="fr-fic fr-dib"></p><p><em>The EMSLab system&rsquo;s camera, LiDAR sensor, and UWB receiver each use an MTi-300 AHRS to provide a consistent 3D frame of reference</em></p><h3>How to navigate without a map</h3><p>The Canadian government&rsquo;s specification for the Carleton University research project was as hard as it gets in the world of SLAM (Simultaneous Location and Mapping) technology: to precisely track a person&rsquo;s motion through 3D space, in any indoor location, in real-time, with no map or plan available before entering the building.</p><p>To do this, the Carleton University research group developed portable apparatus to be carried by each person who is to be tracked. This apparatus consists of an array of radio, optical, and motion sensors working in concert. Project leader Mohamed Atia, Associate Professor in the Systems and Computer Engineering faculty at Carleton University, says, &lsquo;To track the location of individuals in 3D space, we made a single, multi-sensor system which can be carried or worn by a person. The system we developed is comprised of a vision camera, a LiDAR laser sensor for ranging and object detection, and an ultra-wideband (UWB) receiver, which locates the individual in 3D space relative to UWB beacons dropped at intervals as the operator moves through the building.&rsquo;</p><p>In combination, these technologies are capable of creating a 3D, multi-story map showing obstacles such as walls, windows, doors, and floors, and of positioning the carrier within the map. The way the system creates a 3D map in real-time can be seen in the video below.&nbsp;</p><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe width="640" height="360" src="https://www.youtube.com/embed/8Hb-5xGYwRw?&wmode=opaque&rel=0&enablejsapi=1&origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span></p><p>The biggest technical difficulty in implementing this system is to combine the multiple sensor inputs &ndash; the process of sensor fusion. Since the Carleton University system is worn or carried, it is continually rising, falling, rotating, and tilting in 3D space as the operator walks, crawls, or runs. Uncorrected, this pitch, roll, and yaw movements make the LiDAR, camera, and UWB inputs inconsistent with each other, and inconsistent with their own outputs over time.</p><p>The solution: a stable, accurate 3D frame of reference. And this is provided by three MTi-300 AHRS units. An MTi-300 AHRS affixed to each of the LiDAR sensor, camera, and UWB receiver measures in millimetre-level detail every roll, pitch, and yaw displacement at each sensor, enabling real-time continuous compensation of the sensor&rsquo;s outputs. This means that they can be fed into a map that is always flat relative to the earth&rsquo;s surface, no matter what position the carrier&rsquo;s body is in when each sensor measurement is taken.</p><p>Mohamed Atia says, &lsquo;The crucial requirements for the AHRS were accuracy and stability. Even tiny errors in the 3D frame of reference provided by the AHRS are rapidly amplified over the course of a mission, so we needed our chosen AHRS to be accurate, and to maintain its accuracy over time and temperature with almost zero drift. To fit the portable nature of the apparatus, it also needed to be small and light.</p><p>&lsquo;We studied the market for commercial IMUs carefully, and our evaluation showed that the MTi-300 sensor from Xsens offered the best combination of high performance and small size.&rsquo;</p><p>The Carleton University system has one other clever trick up its sleeve: users wear the&nbsp;<a href="https://www.xsens.com/products/mtw-awinda" rel="noopener noreferrer" target="_blank">MTw Awinda</a> wireless motion capture system, also from Xsens, which allows the system to recognize types of motion, such as motion and crawling, and to perform dead reckoning in the temporary absence of UWB signals. Mohamed Atia says that wireless technology is an important feature of the Awinda product because it makes the SLAM system more easily wearable. He adds, &lsquo;The MTw Awinda system also performs synchronized data sampling accurate to &plusmn;10&micro;s, which gives us a time reference for the fusion of all the other sensors in the system.&rsquo;</p><h3>A life-saving innovation</h3><p>Early field trials of the portable Carleton University system prove in principle that accurate, reliable SLAM can be implemented in any indoor 3D space without satellite positioning signals. Mohamed Atia&rsquo;s hope is that, after further refinement, this technology will one day help save lives. &lsquo;In the military, industrial or emergency services settings, the ability to know exactly where personnel is inside a building could be a game-changer,&rsquo; he says. &lsquo;When a first responder is injured or calls for assistance, colleagues will know exactly where to go, and so personnel can be deployed precisely where and when they are needed. In stressful environments such as a building fire or the battlefield, speed saves lives. Our technology, with the MTi-300 at its heart, will enable people to be found and helped more quickly.&rsquo;</p>

with Xsens' AHRS

A life-saving innovation: Locating first responders indoors

<p data-v-8e5709e8="" id="isPasted">On GitHub we provide a sample project. There you can download the <a data-v-8e5709e8="" href="https://github.com/PCB-Arts/stylized-blender-setup/releases/download/v1.1/stylized-blender-setup-v1.1.zip" rel="nofollow noopener noreferrer" target="_blank">blender file</a>, which contains a complete setup with models, corresponding textures and shader settings.</p><h2 data-v-8e5709e8=""><a data-v-8e5709e8="" href="https://blog.pcb-arts.com/en/blog/blender-tutorial-3#bevel" tabindex="-1" target="_blank"></a>Bevel</h2><p data-v-8e5709e8="">Most SMD components have a very simple geometry: they are straight boxes. Due to the perfect shape, renders often look unrealistic. For example, the JST connector of the Zynq-Feather has many sharp edges. In reality, the edges of connectors, IC packages or SMD components are more or less rounded because of inaccuracies and imperfect conditions during manufacturing.</p><p data-v-8e5709e8="">With the help of a <code data-v-8e5709e8="">Bevel</code> node, such rounded corners can easily be added to a material without having to change the geometry of a single or multiple objects. For this purpose, the node can be connected to the <code data-v-8e5709e8="">Normal</code> input of a <code data-v-8e5709e8="">Shader</code> node. The <code data-v-8e5709e8="">Radius</code> can be used to influence the curvature. Generally small <code data-v-8e5709e8="">Radius</code> values should be chosen for IC packages, because they usually have sharp corners. On the other hand, the solder contacts of SMD resistors, for example, are more rounded. In this case, larger values should be chosen. The following figure shows how a <code data-v-8e5709e8="">Bevel</code> node can be configured in the <code data-v-8e5709e8="">Shader Editor</code>.</p><p data-v-8e5709e8=""><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjUxNDc5NDkxODIxLUJldmVsX05vZGUucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner">The Bevel node.</span></span></span></p><p>At first glance, a difference is hardly noticeable. To get a better impression of the effect, take a look at the following illustration. There, two renders are laid on top of each other, one with and one without bevel. The image segments can be moved using the slider on our <a href="https://blog.pcb-arts.com/en/blog/blender-tutorial-3" rel="noopener noreferrer" target="_blank">blog</a>.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjUxNDgwMjUzNzkzLVdpdGggYW5kIHdpdGhvdXQgQmV2ZWwucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner">Rendered PCB with and without bevel.</span></span></span></p><p data-v-8e5709e8="" id="isPasted">Using Bevel nodes was suggested by one of our Twitter followers. He mentioned it in one of his videos: <a data-v-8e5709e8="" href="https://www.youtube.com/watch?v=7ratnf0rIVU" rel="nofollow noopener noreferrer" target="_blank">youtube.com/watch?v=7ratnf0rIVU</a></p><h2 data-v-8e5709e8=""><a data-v-8e5709e8="" href="https://blog.pcb-arts.com/en/blog/blender-tutorial-3#labels" tabindex="-1" target="_blank"></a>Labels</h2><p data-v-8e5709e8="">IC packages often have a label. It shows a product number, the name of the manufacturer or something similar. The next step is to see how such a label on a component can be implemented in Blender. For this, a texture is needed that contains the labeling as a black and white image. The labels are often shown in the data sheets of the components. They can be extracted from the PDF file of the data sheet as a screenshot or with the help of an image editing program. A <code data-v-8e5709e8="">UV Map</code> must then be set up to project the texture onto the top of the IC package. All other faces should be placed outside of the image. To ensure that the label is not present on the other faces, the <code data-v-8e5709e8="">Extension</code> attribute of the <code data-v-8e5709e8="">Image Texture</code> node in the <code data-v-8e5709e8="">Shader Editor</code> must be set to <code data-v-8e5709e8="">Extended</code> (the default value is <code data-v-8e5709e8="">Repeat</code>).</p><p data-v-8e5709e8="">In the example of the Zynq-Feather a label is to be added to a RAM IC. The following screenshot shows how the <code data-v-8e5709e8="">UV Map</code> was configured.</p><p data-v-8e5709e8=""><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjUxNDgwNDAwMDU2LVJBTV9JQ19VVi5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner">Cconfiguration of the UV Map.</span></span></span></p><p data-v-8e5709e8="" id="isPasted">After the UV map has been set up correctly, the material of the component must be adjusted. Since ICs are often assigned the same materials during export, a copy of the material must first be created. This way, the subsequent changes will only apply to the selected part.</p><p data-v-8e5709e8="">In the production of ICs, the markings are often applied to the packaging surface by a laser. By using a laser, the lettering is slightly brighter than the otherwise black packaging and less rough. This can be taken into account by controlling the &#39;Base Color&#39; and the &#39;Roughness&#39; of the material with the previously set up texture. For the <code data-v-8e5709e8="">Base Color</code> a <code data-v-8e5709e8="">Mix</code> node can be used, which mixes the black color of the case with the dark brown color of the marking. The <code data-v-8e5709e8="">Roughness</code> can be controlled using the <code data-v-8e5709e8="">Map Range</code> node. This maps the values of the texture from <code data-v-8e5709e8="">0.0</code> (black) to <code data-v-8e5709e8="">1.0</code> (white) to the range of values from <code data-v-8e5709e8="">0.4</code> (rather smooth) to <code data-v-8e5709e8="">0.8</code> (rough).&nbsp;</p><p data-v-8e5709e8=""><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjUxNDgwNDcxNDAyLVJBTV9JQ19NYXRlcmlhbC5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner">The settings for the material of the RAM IC.</span></span></span></p><p data-v-8e5709e8="">For close-ups it is also worth to add a <code data-v-8e5709e8="" id="isPasted">Displacement</code> to the ICs material. To create bumps in the surface, a <code data-v-8e5709e8="">Noise Texture</code> can be used. The necessary nodes are shown in the following figure.&nbsp;</p><p data-v-8e5709e8=""><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1651480572544-RAM_IC_Displacement.png" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner">Necessary nodes for Noise Texture.</span></span></span></p><h2 data-v-8e5709e8="" id="isPasted">Denoising</h2><p data-v-8e5709e8="">The last thing that should be considered is the final result: the rendered image. In the <code data-v-8e5709e8="">Cycles</code> mode, the rendered image is quite noisy. Depending on the number of samples , the noise is stronger or weaker. With a large number of samples the noise decreases, but the time for rendering increases. Even with a very high amount of samples, the noise is still noticeable. Hence, denoising should be applied to the render.</p><p data-v-8e5709e8="">To set up denoising, the checkbox <code data-v-8e5709e8="">Denoising Data</code> must be selected in the <code data-v-8e5709e8="">View Layer Properties</code> in the <code data-v-8e5709e8="">Properties</code> window under <code data-v-8e5709e8="">Passes</code>. Another visual improvement can be achieved by using <code data-v-8e5709e8="">Ambient Occlusion</code>, which can be activated in the same process (see screenshot).</p><p data-v-8e5709e8=""><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1651480646573-Enable_Denoising_and_AO.png" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner">Enable Denoising and AO.</span></span></span></p><p data-v-8e5709e8="" id="isPasted">Finally, the activated render layers need to be considered in the <code data-v-8e5709e8="">Compositor</code> in order to apply Denoising and Ambient Occlusion to the render result. The settings in the <code data-v-8e5709e8="">Compositor</code> are shown in the screenshot below.</p><div data-v-8e5709e8="" src="https://static.blog.pcb-arts.com/blender-tutorial-3/Compositor_Denoising.png" title="Configuration of Denoising and Ambient Occlusion in the Compositor"><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1651480715257-Compositor_Denoising.png" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner">Compositor for Denoising.</span></span></span></div><h2 data-v-8e5709e8="" id="isPasted">Conclusion</h2><p data-v-8e5709e8="">The PCB model has now been enhanced with additional details by adding labels to ICs and rounding materials to make them look more realistic. Also, the noise in the final render has been fixed. This is the end of the tutorial series about rendering PCBs in Blender.</p><p data-v-8e5709e8="">&nbsp;The team of PCB Arts hopes you will have a lot of fun creating your own renders!</p>

In this tutorial series it is all about the procedure of creating realistic renders from 3D models of PCBs in Blender. 
In the third (and last) part of this tutorial series, we will take a look at different steps that will lead to a better render result.

3d mapping

Profiles