In a world where automation dominates the manufacturing sector, one question persists: why aren't robots omnipresent in our daily lives? The answer lies in the intricate challenges of bridging the gap between humans and machines. While robots thrive in structured environments, the unpredictability and complexity of human behavior pose significant hurdles to seamless integration.&nbsp;In Robotican's recent work, we have developed a new way for humans to interact with service robots. A person is talking to a robot and command it to perform series of tasks a spoken language. Such a command can be: "Go forward up until the end of the hallway, enter the room in front of you, ask the doctor to give you the examination files, and come back here". The robot uses a Speech-To-Text engine, to write the command in text, then it applies a specially trained LLM (Large Language Model) to divide the command into sub-tasks. In our example the sub-tasks will be: "move forward, enter the room, ask the doctor, go back".&nbsp; The robot start performing the sub-tasks according to the required order. &nbsp;While moving the robot uses it cameras to "see" the environment in front of it. The frame from this camera goes into an LVM (Large Visual Model) which receives a photo and output text. When the way is clear this LVM will output "move forward", and an action will be: send the robot a navigation way point 10 meters ahead". When say, in our example, a group of people are standing in front of the robot blocking the way. Then the LVM will output "People are blocking the way", and the action will be "ask them to let me pass through". &nbsp;Then a Text-To-Speech engine will say "Excuse me, I'm a robot with an important task. May I please pass through? Thank you.” Then the robot pass and continues its task. This process is repeated until the robot complete its task, or is unable to do it. <img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1731928237835-%D7%AA%D7%9E%D7%95%D7%A0%D7%94+2+%D7%9C%D7%9E%D7%90%D7%9E%D7%A8+2+%D7%95%D7%95%D7%99%D7%91%D7%95%D7%9C%D7%91%D7%A8.jpg" style="width: 100%px;" class="fr-fic fr-dib">"The answer lies in the intricate challenges of bridging the gap between humans and machines" This is a unique approach of combining TTS, STT, LLM and LVM to allow robot interact with human, and human interact with a robot in a spoken language, while understanding tasks, seeing and understanding the environment around it, and asking people to help the robot conducting its task successfully. Enter Robotican, a pioneering force at the forefront of autonomous robotics, spearheading the charge to revolutionize Human-Robot Interfaces. As a key player in the Israel Innovation Authority's groundbreaking consortium, Robotican harnesses its extensive expertise and recent advancement in AI and Deep Learning to develop innovative solutions that allows robots to navigate amongst people while interacting with them similarly to the way other peoples do.&nbsp;Championing the quest to teach robots to navigate unstructured environments effortlessly, Robotican embarks on a transformative journey empowered by cutting-edge Large Language and Vision Models (LLM &amp; LVM). Leveraging the revolutionary capabilities of these models, Robotican engineers have unlocked the potential to communicate with robots in a language they understand.&nbsp;Omri Eitan, Robotican's AI expert, sheds light on this groundbreaking advancement, stating, "The transformative power of the latest Language Model technology has ushered in a new era of complexity in decision-making and deep contextual reasoning. Tasks once deemed intensive engineering endeavors now undergo fine-tuning, and interactions once artificial now feel natural." &nbsp; With Robotican's innovative approach, robots decode textual or verbal instructions, process visual data using LVM to navigate crowded environments, and seamlessly engage in dialogue with individuals, courtesy of LLM. This paradigm shift not only enhances precision in task execution but also fosters meaningful human-like interactions, paving the way for unprecedented advancements in automation and artificial intelligence.&nbsp;Witness the dawn of a new era as Robotican redefines the landscape of human-robot interaction, propelling automation into realms previously deemed unimaginable. Join us as we embark on this transformative journey, shaping a future where robots seamlessly integrate into every facet of our lives. #RoboticanRevolution #HumanRobotInteraction #AIInnovation #AutomationAdvancements

Ghostrobotics V60 with Robotican's ARK (autonomous robotic kit)  

Pioneering AI-Driven Collaboration: Transforming Social Navigation into daily routine

Unveiling the Future: Human-Robot Interaction Revolutionizes Social Navigation Using AI and Deep Learning

As you might recall from <a href="https://cypherrobotics.com/2024/10/17/cypher-robotics-puts-captis-on-the-global-stage-at-gitex-global-2024-in-dubai/?et_fb=1&PageSpeed=off" target="_blank">this post</a>, Cypher Robotics and its <a href="https://indrorobotics.ca/cypher-robotics-announces-captis-a-new-solution-for-industrial-scanning/" target="_blank">Captis</a> cycle-counting/inventory management solution recently attended the huge <a href="https://www.gitex.com/" target="_blank">GITEX GLOBAL 2024</a> event in Dubai. It’s the world’s largest technology and AI exhibition, with some 200,000 attendees. It was a *huge* show.But a lot of the work – and the opportunities – for both Cypher Robotics and InDro Robotics (which incubated Cypher and has a technology agreement with the company) took place away from the show floor. Cypher Robotics CEO and InDro Vice President Peter King spent much of his time in high-level meetings with executives from five of the largest companies in the United Arab Emirates.“These were C-Suite level meetings, where we were able to learn more about what these companies do – and discuss how both InDro and Cypher can offer solutions that could benefit them,” says King.These aren’t companies where you can simply call and ask for a meeting with high-level executives. There needs to be a catalyst to facilitate such discussions.And there was: The Government of Canada; specifically, the Canadian Consulate in Dubai.Below: Cypher Robotics CEO Peter King on the floor at GITEX GLOBAL 2024. Much of his time was spent off the floor, meeting with executives from the largest companies in the UAE<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1729798081995-GITEX-Peter.jpg-2.webp" style="width: 100%px;" class="fr-fic fr-dib" alt="Cypher Robotics Booth at GITEX"><h3 id="isPasted">Trade Mission</h3>Among the many responsibilities of the Federal Government is promoting trade between Canadian firms and international clients. Sometimes, there are large “Team Canada Trade Missions” which are led by a Minister and often covered by media. With these missions, there’s a specific push on the Indo-Pacific region. On other occasions, however, the government pulls together smaller groups with a very specific focus. Months before GITEX GLOBAL 2024 was to take the world stage, planning began for a mission in Dubai that would take place the same week.Government officials identified five Canadian firms in the Canadian high-tech sector it felt might be a fit for the UAE market. InDro Robotics was invited to participate – and was the only company among those five from the robotics sector.“They identified that our solutions could be highlighted in Dubai – not only for trade reasons, but also to help solve some really hard problems,” says Peter King. “We were obviously really pleased to be one of a small handful of technology companies to be on the federal government’s radar.”Canada’s Consul General in Dubai, H.E. Tracy Reynolds, was at the helm of this program and coordinated a series of meetings with “the UAE’s most influential business and technology leaders,” reads a Government of Canada document outlining the program.“Consul General Reynolds will lead a two-day outreach program that will allow selected Trade Commissioner Service (TCS) clients to pitch their products and solutions to Dubai’s conglomerates, which are considered to be major buyers of ICT products and solutions. The meetings will also allow the delegation to learn about the latest technologies being adopted by these organizations,” it adds.<h3> </h3><h3 id="isPasted">Why UAE?</h3>Though historically an oil-driven economy, the United Arab Emirates has diversified greatly in recent years. It has evolved, according to the CIA’s World FactBook, “into a trade-oriented logistics and supply chain leader (with) strong foreign direct investment orientation; building trade and investment ties through partnership agreements…” The UAE Gross Domestic Product is the fourth highest in the Middle East (after Turkey, Saudi Arabia and Israel), with an estimated USD 719.733 billion GDP in 2023.In Dubai, the skyline has been utterly transformed over the past few decades. It’s home to the world’s tallest building, the Burj Khalifa, and other ultra-modern architecture. Known also for its luxury shopping and high-end autos, Dubai has also embraced technology in recent years. In fact, the local police count Tesla Cybertrucks among their fleet.Dubai has never been as dependent on oil as the other six Emirates that comprise the UAE – and Dubai has led the way in the UAE in terms of economic diversification. According to Wikipedia, “Oil production, which once accounted for 50% of Dubai’s gross domestic product, contributes less than 1% today. In 2018, wholesale and retail trade represented 26% of the total GDP; transport and logistics, 12%; banking, insurance activities and capital markets, 10%; manufacturing, 9%; real estate, 7%; construction, 6%; tourism, 5%. The International Herald Tribune described it as ‘centrally-planned free-market capitalism’.”In other words, Dubai – and the wider UAE – are a significant and growing global marketplace.Below: The Dubai Skyline at night. Photo by Ivan Siarbolin – https://www.pexels.com/photo/city-skyline-during-night-time-3787839/, CC0, https://commons.wikimedia.org/w/index.php?curid=95959711<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1729798138316-Dubai_Skylines_at_night_Pexels_3787839-scaled.jpg.webp" style="width: 100%px;" class="fr-fic fr-dib" alt="Dubai Skyline"> <h3 id="isPasted">High-Level Agenda</h3>With that context, it’s clear why the Canadian Consulate in Dubai sees opportunity. Being on the ground, Canadian&nbsp;Consul General&nbsp;Tracy Reynolds,&nbsp;Deputy Consul General Anthony Finch and Trade Commissioner&nbsp;Arun Basandai have an insider’s vantage point into the key economic players in Dubai and the UAE.&nbsp;And so, over the course of two days, they accompanied representatives of the five invited Canadian companies to five different high-level meetings.The first was with one of the largest real estate developers in the entire UAE. Remember the earlier reference to the Burj Khalifa? This company owns it. In addition to real estate, its diverse portfolio includes&nbsp;retail, hospitality, and&nbsp;leisure. The firm’s Executive Director and its Head of Information Technology attended the meeting.We don’t want to get into the details, but it was an excellent discussion – which included how solutions from both InDro Robotics and Cypher Robotics might be useful to that firm.&nbsp;From there, it was off to a massive global investment company that has been a major driver of Dubai’s spectacular growth. It’s involved with 10 sectors,&nbsp;including real estate, hospitality, leisure&nbsp;&amp; entertainment, media, ICT, design, education, retail, manufacturing, and&nbsp;logistics and science. It owns hotels, parks, resorts, a huge arena, multiple large retail outlets – and is also involved in multiple projects to accelerate Dubai into a fully Smart City. There were fruitful discussions there as well.Meeting three was with one of the largest retailers in Dubai and the entire UAE with an emphasis on the fashion and lifestyle industry. On the food and beverage side, it runs multiple name brand franchises throughout the UAE and is the distributor/retailer of major fashion brands. It’s a huge company with a massive rolling inventory across several sectors. As with the previous meetings, all five Canadian technology companies had a chance to discuss their offerings.&nbsp;<h3>And There Was More…</h3>Once again, it was C-suite meetings with the full support of senior Consulate staff. The group met with the CEO and Chief Strategy and Technology Officer (CSTO) of the leading shopping mall, retail and leisure company across the Middle East, Africa and Asia. It owns and operates 27 major shopping malls, multiple hotels, cinemas, etc. and has assets in excess of USD 18 billion and 44,000 employees.The final meeting was with “a multinational retail franchise operator of 70 brands in 20 countries.” Those brands include Starbucks, Chipotle and Cheesecake Factory. The company runs hotels and major retail outlets with names you’d recognize. The company’s Chief Strategy and Digital Officer attended this meeting, and was able to learn about solutions offered by all five Canadian technology companies.“These meetings were a tremendous opportunity to learn not only about what these leading UAE companies do, but also explore some of the challenges they face with operations at that scale,” says King. “There was significant interest in solutions from both InDro and Cypher – and I’m confident these were just the first of many conversations to come.”Below: The Cypher Robotics cycle-counting and RFID scanning Captis, which can operate autonomously for five hours and also capture precision digital twins. Below that is the InDro Robotics Sentinel, designed for remote asset inspection, security and surveillance and digital twins<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1729798194505-Captis-1-scaled.jpg.webp" style="width: 100%px;" class="fr-fic fr-dib" alt="Captis Robot"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1729798212185-Sentinel-in-substation-1.jpg.webp" style="width: 100%px;" class="fr-fic fr-dib" alt="Sentinel Inspection Robot"><h3 id="isPasted">InDro's Take</h3>We are pleased to have been selected to take part in this trade mission in the United Arab Emirates – and are exceedingly grateful to the senior staff at the Canadian Consul General in Dubai. These were exceptional meetings and, potentially, the beginning of new business relationships.“These meetings were a significant step along Cypher’s long-term roadmap, which includes markets beyond North America,” says King. “Every business relationship begins with a discussion, and these were very productive introductory meetings for all of the Canadian firms on this trip. I’d like to extend our deepest thanks for Consul General Tracy Reynolds, Trade Commissioner Arun Basandani and Deputy Consul General Anthony Finch.”InDro Founder and CEO Philip Reece is also pleased.“These meetings were a remarkable opportunity for not only InDro and Cypher, but for four other innovative Canadian tech companies,” he says. “The Government of Canada recognizes the global shift toward Industry 4.0 and the role Canadian technology companies can play in that. We are pleased that InDro had this opportunity and extend our thanks to all those involved.”If you’d like to learn more about InDro Robotics solutions, contact us <a href="https://indrorobotics.ca/connectwithanexpert/" target="_blank">here</a>. For Cypher Robotics and Captis, reach out <a href="https://cypherrobotics.com/contact-us/" target="_blank">here</a>.Cover image of Dubai at top of story via Wikimedia Commons by Tim Reckmann, CC BY-SA 3.0

Exploring strategic partnerships and showcasing cutting-edge robotic technologies, InDro Robotics and Cypher Robotics attend pivotal trade discussions to expand their global impact.

InDro Robotics and Cypher Robotics attend High-Level Trade Meetings in Dubai

InDro has taken the wraps off its latest innovation – the all-new Sentinel Uncrewed Ground Vehicle.Sentinel has been built for inspection, remote teleoperation and surveillance over 5G and 4G networks. Its robust build has been designed to take demanding weather and ground conditions in stride, while capturing virtually any kind of data across a broad range of use-cases.“That’s what really excites me about Sentinel,” says Head of R&amp;D Sales <a href="mailto:lcorbeth@indrorobotics.com" target="_blank">Luke Corbeth</a>. “It can be used across multiple verticals, ranging from remotely inspecting assets like electrical sub-stations, through to agriculture, mining and more. It’s a workhorse.”We could list all the sensors and processors onboard (and we will), but it’s the synergy here that counts.“Sentinel has been very purposely designed and integrated so that virtually anyone can carry out highly complex missions with ease,” explains InDro Vice President Peter King. “And with our InDro Autonomy stack onboard, Sentinel can carry out tasks completely on its own for up to four hours before returning to wirelessly recharge at its base.”That workhorse is the result of a lot of R&amp;D by our Area X.O engineering team – and also the logical step forward from our original Sentinel V1. That was our original design for remote teleoperated inspection.But, as you likely know by now, at InDro the innovation never stops.Below: The new InDro Sentinel, carrying out an autonomous mission<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzI0MDkyNDY1Mjc4LURTQ0Y3NzcwLXNjYWxlZC5qcGcud2VicCIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 100%px;" class="fr-fic fr-dib" alt="Sentinel Inspection and Monitoring Robot"><h3 id="isPasted">InDro Commander: Smart and Versatile</h3>What truly sets Sentinel apart from the competition is the synergy of two innovations we’re quite proud of: <a href="https://indrorobotics.ca/indro-commander-module-streamlines-robotics-rd/" target="_blank">InDro Commander</a> and <a href="https://indrorobotics.ca/new-indro-controller-a-simple-solution-to-complex-robotics-missions/" target="_blank">InDro Controller</a>. And while Sentinel takes full advantage of these products, they are both robot-agnostic.InDro Commander is a module that houses Sentinel’s brain – and allows for the seamless integration and power management of other sensors. With multiple USB slots and ROS1 and ROS2 libraries onboard, it makes modifying Sentinel a pleasure rather than a pain. You can, quite simply, plug in a new sensor and it will be detected; no complex coding or wiring required.“Having Commander on board allows clients to easily modify their robots with different sensors for different applications. This is particularly appealing to those in the R&amp;D community,” says Engineering Lead Arron Griffiths.Commander also connects with 5G at near-zero latency for remote teleoperations or monitoring dense data during autonomous missions.“Whether it’s 4G, 5G or WiFi, it does it all,” he adds.With two state-of-the-art EDGE processors onboard, Commander also has the compute power and AI to take on anything. For autonomy, we’ve integrated the <a href="https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-orin/" target="_blank">NVIDIA Jetson AGX Orin</a> – a powerful 64GB developer kit. It’s capable of AI calculations at the astounding speed of 275 trillion operations per second (TOPS). Our InDro Autonomy Stack is embedded in Commander for precision autonomous missions. We’ve also outfitted Sentinel with a Realtime Kinematic (RTK) sensor for centimetre-level positioning accuracy.The other EDGE processor is the <a href="https://www.stereolabs.com/en-ca/products/zed-box" target="_blank">ZED Box</a>, which contains an NVIDIA Orin NX (100 TOPS). This carries out multiple tasks, but is specially built to work with depth-perception ZED cameras. There are two of these cameras on board Sentinel, used for AI obstacle detection and avoidance.Below: A LiDAR-equipped InDro Commander module<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzI0MDkyNTEwNzU4LUluRHJvLUNvbW1hbmRlci0xLjYtY29weS5qcGcud2VicCIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 100%px;" class="fr-fic fr-dib" alt="InDro Commander"><h3 id="isPasted">InDro Controller At The Helm</h3>InDro Controller is our custom-built dashboard, and it’s been designed to match perfectly with InDro Commander for an exceedingly powerful but user-friendly means of programming and monitoring missions, robot health, sensor data and more. Operators can easily customize to get precisely the layout they want.“The dashboard, the cameras, the heads-up display on the autonomous missions – those all can be customised,” says Front End Developer RJ Bundy. “We’re also adding other&nbsp;personal user customisations, like a light and dark mode, metric conversion, schedule missions – all the kinds of features you could want.”The software resides in Commander and is accessed remotely via an encrypted login system. No one will ever see your missions and data except you – and those you grant the same secure access. (Yes, multiple users can simultaneously monitor a remote operation from locations across the country or across the globe.)“We’ve created an interface that makes it very hard for a third party to intercept any of those commands or the data coming from the robot to you,” says Bundy.And while it’s a perfect fit with Sentinel, InDro Controller can be used with any robot.“It’s an all-in-one data visualization, robot management and robot control software,” he adds. “Whether&nbsp;you’re a student first learning how to use a robot or you’re a commercial giant, you’d be able to manage and maintain all of your robots.”He’s not kidding. You can intuitively control Sentinel – or any other robot – using an Xbox controller. We’ve done demonstrations more than 4,000 km away from Area X.O where we’ve simply put a controller in someone’s hands. With imperceptible lag, they’re able to operate the robot. (Plus, you can customise the buttons for specific sensor features like pan, tilt, zoom, etc.)Autonomous missions – including complex actions like stopping at a precise location, tilting a camera to a gauge and then zooming in and capturing the readout – are easily plotted using Controller. Once your mission has been set with multiple waypoints and actions, it’s a single click to repeat it. (And, of course, you can save as many different missions as required.)The InDro Autonomy stacks are pre-loaded, meaning location doesn’t really matter.“We have a GPS-based autonomy – which is better for outdoors – and then we have a SLAM- (Simultaneous Localisation and Mapping) based autonomy which is ideal for indoors,” says Head of R&amp;D Sales Corbeth.There are actually two versions of InDro Controller available; a simpler version is still full-featured but does not ship with InDro Autonomy, meaning it’s ideal for those in the R&amp;D space who want to test their own autonomy software. Several academic institutions have already been using this version, and feedback has been excellent.And the best news? Both versions will only become even more sophisticated over time.“We’re heavily invested in continuously improving the software,” says Corbeth. “So regardless of which version you’re shipped, know that this is something that InDro Robotics is constantly developing and improving with client feedback in mind to ultimately provide the best mission planning, teleoperations and development software tool in the robotics industry.”Below: A screenshot from InDro Controller, showing the path of an autonomous mission; the second image shows realtime data flowing through its customizable dashboard<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1724093227785-Autonomy-Ghosts.png-copy.jpg.webp" style="width: 100%px;" class="fr-fic fr-dib" alt="InDro Controller Dashboard"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1724093240664-Dashboard.png-copy.jpg.webp" style="width: 100%px;" class="fr-fic fr-dib" alt="InDro Controller Dashboard"><h3 id="isPasted">Long Duration, Repeatable Missions</h3>It’s already clear Sentinel has the brains. But it also has the brawn to match. Whether the task is inspection, surveillance or other forms of data acquisition, Sentinel has been designed from the ground up to consistently deliver. Featuring a 1500-watt dual drive system and tracks for locomotion, it’s ready for any terrain – from industrial environments through to farm fields and even snow and ice.With a 120kg load capacity and 30° climbing angle, Sentinel can power its way over pretty much anything. We’ve even designed and manufactured custom metal guards to prevent stones from getting their way between the drive gears and the treads – a common issue with tracked vehicles.That strength is supplemented by a comprehensive array of sensors for obstacle detection, Simultaneous Localisation and Mapping (SLAM), high-resolution photos and video – and more.&nbsp;<h3>Zoom, Zoom, Zoom</h3>For starters, there’s a 30x optical Point-Tilt-Zoom camera that can be manually operated via the InDro Controller UI. Every time the pilot stops and captures data, all of that information – robot position, camera angle, zoom settings etc. are stored as a Point of Interest. All of those POIs from that initial manual mission are saved, meaning the next time Sentinel is deployed the operator can simply recall the mission and launch Sentinel for an autonomous run.In fact, we recently did a demo of Sentinel for a government official. We had previously captured photos of five different objects on a path at Area X.O and had saved that mission. During the demo, the robot deployed and autonomously stopped at all five locations, replicating the original shots. This is the kind of feature that’s exceedingly helpful for spotting anomalies, change detection, etc. Of course, the built-in AI can also identify people, cars, trucks, etc. (In fact, we could set Sentinel to trigger an alert if a human is spotted somewhere they’re not supposed to be, a useful feature for both safety and security applications.)For the operator, whether they’re manually piloting or simply monitoring an autonomous mission, four pinhole cameras provide a 360° view of surroundings for situational awareness. Two ZED 3D depth cameras (one at the front, one at the rear) are on the constant lookout for obstacles. As an additional layer of safety, engineers added two 2D safety-rated LiDAR sensors specifically for detecting people or other objects that might unexpectedly move into Sentinel’s path. If that happens, the robot will automatically stop, then seek an alternate path around the person or object. (It can also be set to simply go to a dead stop.)Sentinel is also outfitted with two powerful Robosense 3D LiDAR units, enabling it to operate in GPS-denied environments, capture precision scans – and basically understand even a completely unfamiliar environment. Robosense has become our LiDAR of choice, for the reasons outlined in <a href="https://indrorobotics.ca/robosense-sets-new-bar-for-affordable-powerful-lidar-sensors/" target="_blank">this recent post</a>.Below: InDro recently built and shipped this fleet for swarm robotics research; all are equipped with Robosense LiDAR units<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1724093295010-Image-1-3-scaled.jpeg.webp" style="width: 100%px;" class="fr-fic fr-dib" alt="Swarm Robotics Research Robots"> <h3 id="isPasted">Wireless Self-Charging</h3>Using Sentinel for remote ops? Many of our clients will be. And there’s not much point in deploying a robot at a remote location if someone has to go and plug it in to charge or run diagnostics.Sentinel returns autonomously to its home base, where a wireless charger awaits. The robot understands where it needs to position itself for optimal charging, snugging up close to (but not touching) the induction charging system. Many remotely operated robots rely on making a mechanical connection with charging equipment, which introduces wear-and-tear and additional points of failure.“Mechanical charging systems fail a lot, and we wanted to avoid that,” explains InDro VP Peter King.<a href="https://indrorobotics.ca/new-indro-controller-a-simple-solution-to-complex-robotics-missions/" target="_blank">InDro Controller</a> also carries out diagnostics remotely, reporting back on parameters such as latency, battery strength, sensor connection and other indicators of overall robot health. The operator can literally be hundreds (or even thousands) of kilometres from the robot and see its condition with a quick glance at the UI.“Sentinel can be customised for any client directly by us,” says Engineering Lead Arron Griffiths. “But having Commander on board allows clients to easily modify their robots with different sensors for different applications. This is particularly appealing to those in the R&amp;D community.”Below: On the shoulders of giants – the original Sentinel V 1.0. InDro has incorporated new AI compute power, sensors and other learnings to take a massive leap forward<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1724093341924-DSCF3838-jpg.webp" style="width: 100%px;" class="fr-fic fr-dib" alt="Original Sentinel"> <h3 id="isPasted">InDro's Take</h3>The new Sentinel is more powerful, and more suitable to broader use-cases, than our first version of this robot. Don’t get us wrong; the original Sentinel was (and still is) a great machine. But this version is equipped with more powerful EDGE processing, and AI with greater capabilities. It’s also laden with additional and newer sensors, RTK positioning, and a more powerful base platform.“The new Sentinel is a perfect fusion of industry-leading hardware and software – including InDro Commander, and the new InDro Controller and InDro Autonomy stacks,” says Indro Robotics Founder and CEO Philip Reece. “Our engineering team has put a lot of thought into making this robot capable of highly complex missions – yet easy to use and modify. Sentinel will be a perfect fit for multiple verticals and use-cases, and we look forward to our first deployments.”Want to learn more about Sentinel – or even arrange to take it for a remote test drive? You can contact Head of R&amp;D Sales Luke Corbeth<a href="https://indrorobotics.ca/connectwithanexpert/" target="_blank">&nbsp;here</a>.

Robust robot built for demanding weather and ground conditions, while capturing virtually any kind of data across a broad range of use-cases.

Remote Inspection, Teleoperation and Surveillance over 5G and 4G Networks

<h1 id="isPasted">Wired for Autonomy: Confronting PCBA Complexities in Mobile Robotics</h1><section>The field of untethered robotics is booming, and across industries, autonomous robots are rapidly assuming responsibility for an increasing number of tasks. In their expansion into increasingly diverse industries, these robots place additional demands on their internal workings, specifically the PCBAs that help them function.However, developing PCBAs for autonomous robot systems presents several unique challenges that electrical engineers must overcome to succeed. We take a deep dive into these challenges by exploring design considerations beyond those associated with conventional PCBAs.<h3>Beyond the Basics: Design Considerations for Autonomous Robots</h3>While core electrical principles remain a top priority, autonomous robotics introduces additional complexity to PCBA design that electrical engineers must consider from day one. To function properly, these machines must anticipate and plan for the unexpected in harsh, changing conditions.Intuitive Machines’ Odysseus moon lander recently brought this to light, when the private spacecraft <a href="https://www.reuters.com/technology/space/moon-lander-described-alive-well-day-after-white-knuckle-lunar-touchdown-2024-02-23/" target="_blank">landed on the moon’s surface</a>. As much as the sideways landing attracted media attention, few reports focused on last-minute troubleshooting that kept the mission afloat, a testament to the quick-thinking nature of the engineers involved.Here are some of the important factors to consider when designing PCBAs for untethered robotics:<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1713979758192-1713979758192.png" alt="Autonomous robot design considerations" class="fr-fic fr-dii"><ul><li>PCBAs Must Prioritize Efficiency: Untethered robots almost entirely rely on batteries, so PCBAs must focus on efficiency. The key to this is meticulous component selection, such as using low-power microcontrollers and efficient voltage regulators, as well as excellent circuit design that minimizes standby currents. Using clock gating also maximizes battery life and reduces heat generation.</li><li>Maintain Signal Integrity Amid Noise: Robots operate in dynamic environments with <a href="https://www.macrofab.com/blog/bypass-caps-decouple-your-way-to-cleaner-power/" target="_blank">potential electrical noise</a> from motors, sensors, and external sources. EMI can wreak havoc on robotic sensors, causing glitches, calibration issues, or even complete sensor failure. Consider layout strategies in detail to cut crosstalk and ground loops and maintain clear signal transmission.</li><li>Build for Endurance: Many robots are subjected to extreme temperatures, temperature fluctuations, vibrations, and shocks. Use ruggedized PCBA boards that have components with adequate vibration and temperature ratings. To prevent overheating, soldering must be properly performed to handle thermal expansion and contraction, mechanical fatigue, and dust and moisture infiltration. Enclosures need to protect against dust and moisture egress.</li><li>Size Matters: Robots must be mobile, so PCBAs must also be compact and lightweight to minimize robot weight. Choose wisely when choosing mechanisms within your robot, like vacuum track-based models or electromagnetic ones. They may increase the mass of your robot and reduce maneuverability. High-density interconnect (HDI) techniques and minimizing wasted space become crucial.</li><li>Don’t Compromise Reliability: Autonomous robots, operating alone, demand ultra-reliable PCBAs to minimize failure. Filtering tackles this by reducing sensor noise for cleaner data, smoothing data for fewer errors, extracting key features for better navigation, and even lowering processing load for faster decision-making. Redundancy and fail-safe features further enhance reliability.</li><li>Play by the Rules: Robots must comply with <a href="https://www.macrofab.com/blog/safeguarding-pcba-design-from-electromagnetic-interference/" target="_blank">Electromagnetic Compatibility (EMC) regulations</a> to ensure proper operation and avoid interfering with other electronic devices. Shielding techniques and filtering components become necessary.</li><li>Ensure Maintainability: After assembling, it’s crucial to diagnose faults and perform maintenance. Make sure test points are conveniently accessible after the final assembly. Consider implementing Built-in Test (BIT) features on the PCBA for self-diagnostic features or options for wireless troubleshooting.</li></ul><h2> </h2><h3>The Unexpected Factors: When Biology Interferes</h3>Biological influences are often overlooked when designing PCBAs for autonomous robotics but are an important factor. Autonomous robotics may work in many different environments, encountering biological challenges that can undermine even the most robust and reliable designs.<h3>Biofouling</h3>Biofouling, or the accumulation of microorganisms on surfaces, happens most frequently in aquatic environments. However, this issue can also occur within humid industrial settings or food processing facilities. Biofouling causes issues in many ways including:<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1713979758511-1713979758511.png" alt="Biofouling" class="fr-fic fr-dii"><ul><li>Reduced Performance: Fouling on sensors, thrusters, and hulls can decrease efficiency and accuracy. As a result, sensors become less sensitive, propellers drag more, and overall power consumption increases. Special coatings applied to surfaces can deter organism attachment.</li><li>Problems with Navigation: Fouling on navigational equipment leads to positioning and localization errors, which make it difficult for the robot to map its surroundings, keep its trajectory, and reach its destination.</li><li>Maintenance Issues: Regular cleaning to remove biofouling is essential, particularly in remote locations. As a result, downtime occurs and reduces operational efficiency. Prevent biofouling with brushes or wipers, as well as scheduled cleaning protocols.</li><li>Damage and Disruption: Biofouling can cause physical damage by promoting barnacle growth and corroding structures. Choosing materials that discourage organism growth is a passive defense, and monitoring systems can detect biofouling early and trigger automatic cleaning.</li></ul>Especially within industrial and food processing industries, biofouling can create a surprising but challenging issue for logistics robotics in order fulfillment, for example, or other autonomous vehicles. Make sure to consider this possibility when designing for specific applications.<h3> </h3><h3>Biological Challenges Across Environments</h3>While biofouling dominates aquatic environments, other biological factors impact robots in other settings:<ul><li>On Land: Dust, dirt, and debris can accumulate on sensors and cameras in terrestrial robots, hindering performance. Agricultural robots may encounter challenges with mud and plant debris.</li><li>Airborne: Airborne robots face issues with dust, pollen, insects, and other airborne particles that can increase drag, reduce lift, and affect sensor accuracy.</li></ul>To address animal-related challenges, EEs can employ design strategies like:1. Physical Resilence:&nbsp;<ul><li>Challenge: Animals may chew on cables or antennas, causing malfunctions.</li><li>Solution: Employ ruggedized components, reinforced substrates, and protective enclosures to enhance PCBA durability against physical damage from animal interactions. Additionally, utilize encapsulation and conformal coatings to safeguard critical components from moisture, dust, and possible damage by animals.</li></ul>2. Sensor Integrity:<ul><li>Challenge: Animals brushing against sensors can distort sensor inputs, leading to navigation errors.</li><li>Solution: Design sensors with robust shielding, noise filtering capabilities, and redundancy features to ensure reliable operation and accurate data despite potential interference from animals. Develop strategies for preventing animals from brushing against sensors to keep distortions and navigational errors from happening. For example, anti-fouling paints or cages around the robot could prevent fouling, or sensors might be placed where animals cannot access them.</li></ul>3. Mobility Optimization:<ul><li>Challenge: Animals blocking pathways or obstructing movement can impede the robot’s forward progress.</li><li>Solution: Develop strategies to address mobility constraints caused by animals blocking pathways or obstructing movement, ensuring that robots can navigate effectively and complete their objectives. This could include low-power LEDs or subtle sounds that encourage animals to move away. Optimize power management systems to maximize operational autonomy and extend battery life in resource-constrained environments where robots may interact with animals.</li></ul><h3>Security: The Next Level in AMR PCBA Design</h3><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1713979758575-1713979758575.png" alt="AMR PCBA security" class="fr-fic fr-dii">AMR PCBA securityPCB design traditionally focuses on protecting against electrical malfunctions and physical damage. However, for AMRs (autonomous mobile robots) connected to the internet, cybersecurity emerges as a key concern. Hacking hazards present significant security risks. A compromised robot can open the door to major security breaches. Thus, incorporating encryption and secure boot mechanisms in PCBA design is essential to block unauthorized access.Hardware Trojan Horses also pose a silent yet profound threat. These sneaky modifications, made during manufacturing, can go almost unnoticed. To counteract this, it’s critical to partner with trustworthy suppliers and employ hardware authentication methods.Consider working with a partner with a connected technology platform that keeps you in touch with your manufacturing process from start to finish. Moreover, work with a partner that employs robust security practices and can provide proof of such, such as ISO and ITAR certifications, or membership in the ERAI, an organization established to <a href="https://www.macrofab.com/blog/electronics-industry-counterfeit-parts-problem/" target="_blank">combat counterfeit electronics</a> and high-risk components in the electronics industry. MacroFab holds these standards, and it is a member of ERAI.Another challenge is the theft of PCBAs after robot deployment, especially those containing valuable intellectual property. Consider using tamper-evident seals or protective enclosures if you will be installing robotics in an unsupervised area.Want to read more about designing PCBAs for robotics? Read <a href="https://www.macrofab.com/documents/reshaping-robotics-how-pcba-advances-drive-tomorrows-autonomous-solutions/" target="_blank">Reshaping Robotics: How PCBA Advances Drive Tomorrow's Autonomous Soutions</a> now.<h2> </h2><h3>Conclusion: Embracing the Challenge</h3>The realm of PCBA design for autonomous robots transcends the traditional electrical engineer’s domain. While electrical fundamentals remain the cornerstone, navigating the complexities of this field necessitates venturing beyond the confines of the schematic. It’s a world where understanding environmental stresses, mitigating biological influences, and safeguarding against security vulnerabilities are equally critical.By embracing these unique challenges and fostering a spirit of relentless innovation, electrical engineers will become the architects of a new era in robotics. The future holds immense potential for these autonomous machines, with robust, adaptable PCBAs serving as their foundation.Need PCBAs for your robotics project?&nbsp;</section><a href="https://factory.macrofab.com/quick_quote?mbsy_source=c2dfbfd3-8c8b-4098-87b7-c7e0e06119ec&campaignid=62990&mbsy=6DBV3m" target="_blank">GET AN INSTANT QUOTE NOW</a><section><h5> </h5></section>

Autonomous robotics must anticipate and plan for the unexpected, often in harsh, changing environments. We take a deep dive into these challenges by exploring design considerations beyond those associated with conventional PCBAs.

Wired for Autonomy: Confronting PCBA Complexities in Mobile Robotics

In this episode, we discuss research coming out of George Mason University where the RobotiXX lab has been training robots to drive more like humans

Podcast: Robot That Drives Better Than The Average American

This article was discussed in our <a href="https://www.wevolver.com/profile/the.next.byte" rel="noopener noreferrer" target="_blank">Next Byte podcast</a>.&nbsp;<iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/2b4595aa-c087-4a18-9ed9-6c05655462a7?dark=false" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe> The full article will continue below.<hr>Picture this: You’re out for a drive and in a hurry to reach your destination.At first, the road is clear and dry. You’ve got great traction and things are going smoothly.&nbsp;But then the road turns to gravel, with twists and turns along the way. You know your vehicle well, and have navigated such terrain before.And so, instinctually, you slow the vehicle to navigate the more challenging conditions. By doing so, you avoid slipping on the gravel. Your experience with driving, and in detecting the conditions, has saved you from a potential mishap. Yes, you slowed down a bit. But you’ll speed up again when the conditions improve. The same scenario could apply to driving on grass, ice – or even just a hairpin corner on a dry paved road.For human beings, especially those with years of driving experience, such adjustments are second-nature. We have learned from experience, and we know the limitations of our vehicles. We see and instantly recognize potentially hazardous conditions – and we react.But what about if you’re a robot? Particularly, a robot that wants to reach a destination at the maximum safe speed?That’s the crux of fascinating research taking place at <a href="https://www.gmu.edu/" target="_blank">George Mason University</a>: Building robots that are taught – and can subsequently teach themselves – how to adapt to changing terrain to ensure stable travel at the maximum safe speed.It’s very cool research, with really positive implications.Below: You don’t want this happening on a critical mission…<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzEyNTk4MjUwMzM0LVBpY3R1cmUyLnBuZy53ZWJwIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 328px;" class="fr-fic fr-dib" alt="research platforms"> <h3 id="isPasted">“XX”</h3>Those are the initials of Dr. Xuesu Xiao, an Assistant Professor at George Mason University. He holds a PhD in Computer Science, and runs a lab that plays off his initials, called the <a href="https://cs.gmu.edu/~xiao/RobotiXX/lab.html" target="_blank">RobotiXX Lab</a>. Here’s a snippet of the description from his website:“At RobotiXX lab, researchers (XX-Men) and robots (XX-Bots) perform robotics research at the intersection of motion planning and machine learning with a specific focus on robustly deployable field robotics. Our research goal is to develop highly capable and intelligent mobile robots that are robustly deployable in the real world with minimal human supervision.”We spoke with Dr. Xiao about this work.It turns out he’s particularly interested in making robots that are particularly useful to First Responders, and carrying out those dull, dirty and dangerous tasks. Speed in such situations can be critical, but comes with its own set of challenges. A robot that makes too sharp a turn at speed on a high friction surface can easily roll over – effectively becoming useless in its task. Plus, there are the difficulties previously flagged with other terrains.This area of “motion planning” fascinates Dr. Xiao. Specifically, how to take robots beyond traditional motion planning and enable them to identify and adapt to changing conditions. And that involves machine vision and machine learning.“Most motion planners used in existing robots are classical methods,” he says. “What we want to do is embed machine learning techniques to make those classical motion planners more intelligent. That means I want the robots to not only plan their own motion, but also learn from their own past experiences.”In other words, he and his students have been focussing on pushing robots to develop capabilities that surpass the instructions and algorithms a roboticist might traditionally program.“So they’re not just executing what has been programmed by their designers, right? I want them to &nbsp;improve on their own, utilising all the different sources of information they can get while working in the field.”&nbsp;<h3>The Platform</h3>The RobotiXX Lab has chosen the <a href="https://indrorobotics.ca/hunter-se/" target="_blank">Hunter SE</a> as its core platform for this work. That platform was supplied by InDro Robotics, and modified with the <a href="https://indrorobotics.ca/commander/" target="_blank" rel="noopener noreferrer"></a><a href="https://indrorobotics.ca/commander/" target="_blank" rel="noopener noreferrer">InDro Commander module</a>. That module enables communication over 5G (and 4G) networks, enabling high speed data throughput. It comes complete with multiple USB slots and the Robot Operating System (ROS) library onboard, enabling the easy addition (or removal) of multiple sensors and other modifications. It also has a remote dashboard for controlling missions, plotting waypoints, etc.Dr. Xiao was interested in this platform for a specific reason.“The main reason is because it’s high speed, with a top speed of 4.8m per second. For a one-fifth/one-sixth scale vehicle that is a very, very high speed. And we want to study what will happen when you are executing a turn, for example, while driving very quickly.”As noted previously, people with driving experience instinctively get it. They know how to react.“Humans have a pretty good grasp on what terrain means,” he says. “Rocky terrain means things will get bumpy, grass can impede a motion, and if you’re driving on a high-friction surface you can’t turn sharply at speed. We understand these phenomenon. The problem is, robots don’t.”So how can we teach robots to be more human in their ability to navigate and adjust to such terrains – and to learn from their mistakes?As you’ll see in the diagram below, it gets *very* technical. But we’ll do our best to explain.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzEyNTk4NDM4NTk4LUdlb3JnZS1NYXNvbi1qcGcud2VicCIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 100%px;" class="fr-fic fr-dib"><h3 id="isPasted">The Approach</h3>The basics here are pretty clear, says Dr. Xiao.“We want to teach the robots to know the consequences of taking some aggressive maneuvers at different speeds on different terrains.&nbsp;If you drive very quickly while the friction between your tires and the ground is high, taking a very sharp turn will actually cause the vehicle to roll over – and there’s no way the robot by itself will be able to recover from it, right?&nbsp;So the whole idea of the paper is trying to enable robots to understand all these consequences; to make them ‘competence aware.'”<a href="https://cs.gmu.edu/~xiao/papers/cahsor.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">The paper</a> Dr. Xiao is referring to has been submitted for scientific publication. It’s pretty meaty, and is intended for engineers/roboticists. It’s authored by Dr. Xiao and researchers Anuj Pokhrel, Mohammad Nazeri, and Aniket Datar. It’s entitled: CAHSOR: Competence-Aware High-Speed Off-Road Ground Navigation in SE(3).That SE(3) term is used to describe&nbsp;how objects can move and rotate in 3D space. Technically, it stands for Special Euclidean group in three dimensions. It refers to keeping track of an object in 3D space – including position and orientation.We’ll get to more of the paper in a minute, but we asked Dr. Xiao to give us some help understanding what the team did to achieve these results. Was it just coding? Or were there some hardware adjustments as well?Turns out, there were both. Yes, there was plenty of complex coding. There was also the addition of an RTK GPS unit so that the robot’s position in space could be measured as accurately as possible. Because the team soon discovered that intense vibration over rough surfaces could loosen components, threadlock was used to keep things tightly in place.But, as you might have guessed, machine vision and machine learning are a big part of this whole process. The robot needs to identify the terrain in order to know how to react.We asked Dr. Xiao if an external data library was used and imported for the project. The answer? “No.”“There’s no dataset out there that includes&nbsp;all these different basic catastrophic consequences when you’re doing aggressive maneuvers.&nbsp;So all the data we used to train the robot and to train our machine learning algorithms were all collected by ourselves.”&nbsp;<h3>Slips, Slides and Rollovers</h3>As part of the training process, the Hunter SE was driven over all manner of demanding terrain.“We actually bumped it through very large rocks many times and also slid it all over the place,” he says. “We actually rolled the vehicle over entirely many times. This was all very important for us to collect some data so that it learns to not do that in the future, right?”And while the cameras and machine vision were instrumental in determining what terrain was coming up, the role of the robot’s Inertial Measurement Unit was also key.“It’s actually multi-modal perception, and vision is just part of it. So we are looking at the terrain using camera images and we are also using our IMU. Those inertial measurement unit readings sense the acceleration and the angular velocities of the robot so that it can better respond,” he says.“Because ultimately it’s not only about the visual appearance of the terrain, it is also about how you drive on it, how you feel it.”&nbsp;<h3>The Results</h3>Well, they’re impressive.The full details are outlined in <a href="https://cs.gmu.edu/~xiao/papers/cahsor.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">this paper</a>, but here’s the headline: Regardless of whether the robot was operating autonomously heading to defined waypoints, or whether a human was controlling it, there was a significant reduction in incidents (slips, slides, rollovers etc.) with only a small reduction in overall speed.Specifically, “CAHSOR (Competence-Aware High-Speed Off-Road Ground Navigation)&nbsp;can efficiently reduce vehicle instability by 62% while only compromising 8.6% average speed with the help of TRON (visual and inertial Terrain Representation for Off-road Navigation).”That’s a tremendous reduction in instability – meaning the likelihood that these robots will reach their destination without incident is greatly improved. Think of the implications for a First Responder application, where without this system a critical vehicle rushing to a scene carrying medical supplies – or even simply for situational awareness – might roll over and be rendered useless. The slight reduction in speed is a small price to pay for greatly enhancing the odds of an incident-free mission.“Without using our method, a robot will just blindly go very aggressively over every single terrain – while risking rolling over, bumps and vibrations on rocks, maybe even sliding and rolling off a cliff.”What’s more, these robots continue to learn with each and every mission. They can also share data with each other, so that the experience of one machine can be shared with many. Dr. Xiao also says the learnings from this project, which began in January of 2023, can also be applied to marine and even aerial robots.For the moment, though, the emphasis has been fully on the ground. And there can be no question this research has profound and positive implications for First Responders (and others) using robots in mission-critical situations.Below: The Hunter SE gets put through its paces. (All images courtesy of Dr. Xiao.)<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzEyNTk4NTAwNzQ4LWh1bnRlcjItanBnLndlYnAiLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 100%px;" class="fr-fic fr-dib"><h3 id="isPasted">InDro's Take</h3>We’re tremendously impressed with the work being carried out by Dr. Xiao and his team at George Mason University. We’re also honoured to have played a small role in supplying the Hunter SE, InDro Commander, as well as occasional support as the project progressed.“The use of robotics by First Responders is growing rapidly,” says InDro Robotics CEO Philip Reece. “Improving their ability to reach destinations safely on mission-critical deployments is extremely important work – and the data results are truly impressive.“We are hopeful the work of Dr. Xiao and his team are adopted in future beyond research and into real-world applications. There’s clearly a need for this solution.”If your institution or R&amp;D facility is interested in learning more about InDro’s stable of robots (and there are many), please reach out to us <a href="https://indrorobotics.ca/connectwithanexpert/" target="_blank">here</a>.

Empowering Robots with Adaptive Terrain Navigation

George Mason University Researchers Enable Robots to Intelligently Navigate Challenging Terrain

What could small robots on earth have to do with exploration on the moon?Quite a lot, as it turns out. Professors and engineering students at Polytechnique Montréal have been busy writing algorithms and running experiments with robots and drones with one goal in mind: To enable them to explore unfamiliar and even hostile surroundings far beyond the reach of GPS or other forms of precision location technology.“What we want to do is to explore environments including caves and surfaces on other planets or satellites using robotics,” explains Dr. Giovanni Beltrame (Ph.D.), a full professor at Polytechnique’s Departments of Computer Engineering and Software Engineering.Before we get to the how, let’s address the why.“Caves and lava tubes can be ideal places for settlement: They can be&nbsp;sealed and provide radiation shielding. There’s also a chance of finding&nbsp;water ice in them,” says Dr. Beltrame.Of course, it’s also less risky – and less expensive –&nbsp;to send robots to other planets and moons rather than human beings. They don’t require life support, don’t get tired (with the exception of having to recharge), and they can gather and process data quickly.Just think of all the data that’s been acquired on Mars by the twin Rovers and the Mars helicopter.Below: A selfie taken by NASA’s Perseverance rover November 1, 2023, during the the 960th Martian day of its mission. The rover was built with a focus on astrobiology, searching for signs of ancient microbial life on the red planet. Image courtesy of NASA.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzA2NTYxNzQ0OTYwLVBJQTI2MjAyLWpwZy53ZWJwIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib" alt="NASA's Perseverance"><h3 id="isPasted">Prepare on Earth, Deploy in Space</h3>It’s a pretty ambitious vision. But for Beltrame and his team, it’s also very real. And it requires a lot of work and research here on earth.“So to get there (space) and do this with&nbsp;multiple robots, we’ve developed all sorts of technologies –&nbsp;navigation, perception, communication, coordination between the robots, and human-robot interfaces,” he says.“We’re doing all these things,&nbsp;because our goal is to use a swarm of robots to do planetary exploration. There’s more, but that’s it in a nutshell.”When you go to the moon, there’s no equivalent of GPS. And environments like caves can be really tricky – both in terms of robots understanding where they are, and also communicating with other robots beyond line of sight.With the right technologies and algorithms, that communication is possible. And much of Beltrame’s research has involved testing this on earth. In particular, he’s focusing on how groups of robots could take on such tasks collaboratively.“So our primary activities focus on swarm robotics,” he says.Generally that starts with simulation models. But there are limits to simulations – and real-world testing is a big part of what’s going on at Polytechnique.“So we do have this deployment philosophy that we try our technologies in simulation, but then we want to go to deploying robots. You can have the best simulation in the world, but there’s still a reality gap and it’s very extremely important to try things on the real robots,” he says.“We have a saying in the lab, which is: ‘Everything works in simulation’.&nbsp;You can always make your algorithm work in simulation, and then you get out in the field and things go wrong. So&nbsp;one thing we do in the lab is we always do the full stack. That’s why we need to have real robots.&nbsp;And we don’t only do experiments with real robots in the lab, we do them in the field.” <h3>MIST</h3>The lab he’s referring to is known as Polytechnique’s MIST, which stands for Making Innovative Space Technology. Dr. Beltrame is the director of the lab, which focuses on computer engineering targeted towards space technologies. In addition to the researchers, the lab is home to a *lot* of robots. There are big ones, small ones, wheeled ones, flying ones (drones) – literally “hundreds” of robots at the lab.But as Dr. Beltrame emphasised, proving that something will truly work requires testing in environments that are similar to what might be found on the moon or elsewhere. Locations where he’s carried out fieldwork include:<ul><li>Lava Beds National Monument in California (with <a href="https://www.jpl.nasa.gov/" target="_blank">NASA&nbsp;</a><a href="https://www.jpl.nasa.gov/" target="_blank">JPL</a>)</li><li>The Kentucky mega-cave with the <a href="https://costar.jpl.nasa.gov/" target="_blank">CoSTAR</a> team</li><li>Tequixtepec&nbsp;in Mexico with SpéléoQuébec</li></ul>Just check out the images below of field work, courtesy of Dr. Beltrame:<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzA2NTYyNDU0NzUyLUlNRy0yMDIzMDMyMy1XQTAwMDAuanBnLndlYnAiLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib" alt="Tight Space Exploration">Tight Spaces<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzA2NTYyNDYyOTQyLWx1bmFyc2V0dGluZy1qcGcud2VicCIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib">ESA Photo<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzA2NTYyNDg0ODU3LWltYWdlLXNjYWxlZC53ZWJwIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib" alt="Cave Drone">Cave Drone<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzA2NTYyNTAzNzAwLWRyb25ldGVhbS1qcGcud2VicCIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib" alt="ESA Team">ESA Photo<h3 id="isPasted">The InDro Connection&nbsp;</h3>Some of the robots used in the MIST lab – and perhaps eventually on the moon – arrived via InDro Robotics, a North American distributor for AgileX. In fact, Polytechnique has purchased a number of AgileX products, including platforms that InDro has modified to help speed the R&amp;D process. These include:<ul><li>24 LIMOs and simulation table</li><li>AgileX Scout Mini</li><li>AgileX Scout 2.0</li><li>Two AgileX Bunker Mini platforms, with custom builds by InDro</li></ul>We’ve <a href="https://indrorobotics.ca/new-limo-pro-ros2-models-bring-advanced-abilities-to-rd/" target="_blank">written about the LIMO before</a> – a small, affordable and versatile robot capable of perceiving its environment and even Simultaneous Localization and Mapping out of the box. It’s also an ideal size, particuarly when doing multi-agent/swarm robotics, for use in the lab. (You’d run out of space pretty fast with something much larger).“The LIMOs are a very good platform for Simultaneous Localization and Mapping &nbsp;– and perception in general,” says Beltrame.He says they’re a good choice “because they have a 3D camera, they’re lighter, agile, and are sufficiently low in cost. So we can use them in large numbers.&nbsp;Another good thing about the LIMOs is that once you have a lot of similar robots that are reasonably agile, you can actually make a full deployment of software (across all robots).”That makes them an ideal platform for multi-agent research and development.“For example, we developed this tool called Swarm SLAM where many robots collaborate to have a better perception of the environment. We’re currently testing it with the full fleet of LIMOs. That’s&nbsp;something we would have believed impossible with larger robots for logistical reasons.”Though the focus is firmly on space, the Polytechnique Montréal research has applications on earth. Swarms of robots could aid in disaster response, Search &amp; Rescue, and more.<h3>&nbsp;</h3><h3>Favourite Robot</h3>The LIMO isn’t the only AgileX product in Polytechnique’s stable. And while Beltram likes all of them, he has a soft spot for one in particular.“I would say that my favorite robot is the <a href="https://indrorobotics.ca/scout-mini-2/" target="_blank">Scout Mini</a>,” he says. “It’s fast, it’s agile and the control is extremely precise.”In fact, Beltrame often takes the Scout Mini with him when doing school presentations. It’s small enough to be carried in the trunk of his car and hand-carried to classrooms. His team has also used the platform to test a new code for path planning and sophisticated energy calculations. It’s capable of tracking the additional energy required for climbing inclines, for example, then calculating when the robot needs to return home to wirelessly recharge.As always, InDro works with clients to deliver precisely what they need. This saves time for those institutions and corporations on builds, allowing them to get on with the business of R&amp;D.“We’ve done quite a bit of integration for them,” says <a href="mailto:lcorbeth@indrorobotics.com" target="_blank">Luke Corbeth</a>, InDro’s Head of R&amp;D Sales.“For example (see picture below), we provide a top plate with all required hardware mounted and integrated. They then add their own sensors, protective structure, etc. So this is a great example of how we work with clients on a case-by-case basis depending on their needs as robotics isn’t one-size-fits-all.”<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzA2NTYyODUzMDA0LWltYWdlLnBuZy53ZWJwIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib" alt="Tracker Robots"> <h3 id="isPasted">One Small Step...</h3>With all of this research, what comes next? Will the work being done today at Polytechnique eventually find its way off this planet?“The answer is it’s going to happen very soon,” says Beltrame.&nbsp;Sometime later this year, a rocket will head toward the moon carrying three small robots. It’s called the Cadre mission.A trio of small rovers will work as a team to explore the moon autonomously, mapping the subsurface in 3D, collecting distributed measurements, and showing the potential of multirobot missions,” says <a href="https://www.jpl.nasa.gov/missions/cadre" target="_blank">NASA’s JPL website.&nbsp;</a>One of Beltrame’s students is working on that mission with JPL.“This is one example of how the work that we’ve been doing in this lab, in the end – through students that were here – become real missions,” says Beltrame.And that’s not all. As early as 2026, a Canadian-built rover could land on the moon in Canada’s first moon mission.Its task? To explore the moon’s south polar region in search of “water ice.” This ice is critical to long-term human habitation on the moon – and can also be converted to fuel, both for energy on the moon and potentially to refuel other spacecraft with destinations further afield.“I have an engineer from the Canadian Space Agency that’s a student of mine that’s developed the Mission Planner. So the idea is that we – our lab – developed the Mission Planner for the Canada rover that’s going to the moon.”Here’s a look at that planned mission, from the CSA:<iframe width="640" height="360" src="https://www.youtube.com/embed/Ti4a0oiWsMk??si=gdkI-fUB2n3fT89r&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true">&amp;lt;span class="fr-mk" style="display: none;"&amp;gt;&amp;amp;nbsp;&amp;lt;/span&amp;gt;&amp;amp;lt;span class="fr-mk" style="display: none;"&amp;amp;gt;&amp;amp;amp;nbsp;&amp;amp;lt;/span&amp;amp;gt;&amp;amp;amp;amp;lt;span class="fr-mk" style="display: none;"&amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;lt;/span&amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;lt;span class="fr-mk" style="display: none;"&amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;amp;lt;/span&amp;amp;amp;amp;amp;gt;</iframe><h3 id="isPasted">And There's More</h3>There was some big news this week from Polytechnique Montréal. On January 24 it announced the formation of ASTROLITH, a body for “research in space resource and infrastructure engineering.”It’s the first Canadian group dedicated to lunar engineering, according to a news release.“Comprising experts from all seven Polytechnique departments, ASTROLITH will pursue the mission of helping to develop next-generation technologies and training the engineers of tomorrow to ensure Canada’s presence in space and lunar exploration, as well as addressing critical needs on our planet within the context of climate change, resource management and sustainable development,” reads the release.So while the emphasis is on the moon, ASTROLITH will also result in some very practical – and urgent – use-cases on our home planet.“As encapsulated in its Latin motto Ad Lunam pro Terra, ASTROLITH is dedicated to developing technologies with direct impacts here on Earth: enabling development of infrastructure in the Far North or facilitating the energy transition, for example,” says the release.“Indeed, the research unit’s founding members are already involved in developing technologies in various areas related to space and extreme environments, from design of resilient habitats and infrastructures for remote regions to deployment of cislunar communications technologies to development of advanced robotics systems for prospecting and mining, among many others. Their work is bolstered by contributions from specialists in life-cycle analysis, sustainable development and space-related policy development.”The team is composed of academics and researchers that span all seven Polytechnique departments. Beltrame, not surprisingly, is on the team – which is pictured below. (He’s in the back row, centre.)<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1706563288421-Screenshot-2024-01-26-at-9.03.42-AM.png.webp" style="width: 300px;" class="fr-fic fr-dib" alt="Research Team"><h3 id="isPasted">INDRO’S TAKE</h3>We find the work being carried out at Polytechnique Montréal, the MIST lab – and now ASTROLITH – both fascinating and important. It’s also a terrific example of how dedicated researchers and students can develop and test projects in the lab that eventually have real-world (and off-world) applications.“I’m incredibly impressed with the work being carried out here, and the fact it can be put to positive use-cases both on earth and in space,” says InDro Robotics CEO Philip Reece.“We wish Dr. Beltrame and his colleagues well, and we’ll certainly be watching these lunar missions with great interest. It’s always a pleasure when InDro can support teams doing important work like this.”You can find more about the MIST lab <a href="http://mistlab.ca/" target="_blank">here</a>. And if you’d like to talk about AgileX robots (or any other robotic solution), connect with the InDro experts <a href="https://indrorobotics.ca/connectwithanexpert/" target="_blank" rel="noopener noreferrer">here</a>.&nbsp;

Bridging Earth and Lunar Exploration with Advanced Research

Robots on Earth Help Prepare for Research on the Moon

<h1 id="isPasted">2023 Autonomous Vehicle Report</h1>In 2020, Wevolver launched its first <a href="https://www.wevolver.com/article/2020.autonomous.vehicle.technology.report" target="_blank" rel="noopener noreferrer">Autonomous Vehicle Report</a> that provided a comprehensive knowledge foundation about the technologies enabling autonomous cars. We are now pleased to launch a new report that surveys the advances in this arena over the last three years. Further, we provide a snapshot comparison of four leading AV companies, their tech stacks, and approaches to understand how these technologies are applied in actual use cases. The report is augmented with interviews with the report sponsors, who provide deeper insights into the current state of autonomous vehicles, highlighting their priorities, challenges, and leadership objectives.&nbsp;The report examines autonomy from the perspective of passenger vehicles, following the approach of the previous report. However, many of the technologies mentioned are also relevant for other autonomous vehicle types that are making a significant impact in industries and applications such as last-mile delivery, warehouse and logistics, agriculture, mining, search and rescue, and healthcare.&nbsp;To create this report, we interviewed dozens of industry experts and collaborated with technical researchers and writers from around the world. We have attempted to cover all relevant technologies; however, due to space constraints, we had to limit some areas.&nbsp;This report was made possible by the generous support of its sponsors, the tireless effort of the Wevolver team, the expertise and generosity of our consulting experts, and the attention to detail of our writers, researchers, and designers.We hope you find value in this report, and we look forward to continuing to make this critical knowledge available for all.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzAwODIwOTIwMDE1LU1pdHN1YmlzaGkgQ2hlbWljYWwgR3JvdXAgIEdyb3d0aCBHYXJhZ2UgIE9wZW4gSW5ub3ZhdGlvbiBQcm9ncmFtICg2OTQgeCA0MzIgcHgpICg5MDEgeCA1NjEgcHgpLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib"><iframe width="540" height="450" src="https://c8056756.sibforms.com/serve/MUIFAMBWtqlAziu5EwFUIjLCWmRdzCiwOJRx3R3LyYkfG1dfYIvOKLn1dfBY8M58jYUZY8sQm7DnIOmrBjZ93ycsAv4ICzMbRD95I3S4FIWFM6EU4i3C7yzFNYn-Kfm0vJN96NgNO2_w4BQcrnOEwEVShkElbpBLjOT3ARKFBFADGHd-PDjMwJZI5avI3TUfr-xTC_fVunqO5MKo" frameborder="0" scrolling="auto" allowfullscreen="" style="display: block;margin-left: auto;margin-right: auto;max-width: 100%;" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true">&nbsp;&nbsp;&nbsp;</iframe><h2>Report Contributors</h2>Cassiano Ferro MoraesFlorianópolis, BrazilTechnical writer with an electrical engineering background and over five years of experience writing articles about electronics, renewable energy, and electric vehicles. Published multiple articles in IEEE explore, including publications in some of the most recognized conferences and journals of the power electronics industry. Vast experience writing articles in the field of electronics, renewable energy, power converters, hardware-in-the-loop, and electric vehicles at Write Choice.&nbsp;Dr. Miroslav MilovanovicNiš, SerbiaAssistant professor at the Faculty of Electronic Engineering, holding a PhD in Computer Science and Electrotechnics. Leader of the Laboratory for Intelligent Control within the Control Systems Department. Author of over 45 scientific publications, centered on Data Science and Deep Learning applications.Gustavo BruismaPato Branco, BrazilElectrical Engineer graduated at the Federal Technological University of Paraná (UTFPR) with MSc. degree in the same field with a deep focus on control systems for electric vehicles. Vast experience writing articles in the field of electronics, renewable energy, power converters, hardware-in-the-loop, and electric vehicles at Write Choice.&nbsp;Ana Carla SorgatoFlorianópolis, BrazilEnvironmental and Sanitary Engineer graduated at the Federal University of Santa Maria (UFSM) with a MSc. in Environmental Engineering from UFSC. Currently pursuing a Ph.D. in Environmental Engineering at the Federal University of Santa Catarina (UFSC) and writing articles about renewable energy, electric vehicles, and engineering at Write Choice.Ian DicksonLondon, United KingdomFreelance automotive journalist and editor with more than two decades’ experience in consumer media and branded content. Cutting his teeth as a road tester on What Car? – where he learnt the art of turning the complex into the simple – he quickly moved through the ranks and ended up editor of MSN Cars, at the time the UK’s largest motoring website. For the past 10 years, he’s been creating and developing content and strategies for brands like Porsche, Ferrari, Volvo Cars and HERE Technologies.Samir JaberLeipzig, GermanyContent Specialist with a background in engineering, nanotechnology, and scientific research. Samir has comprehensive experience working with major engineering and technology companies as a writer, editor, and digital marketing consultant. Featured author in 30+ industrial magazines with a focus on IoT, nanotechnology, materials science, engineering, and sustainability. Samir is also an award-winning engineering researcher in the fields of nanofabrication and microfluidics. Editor-in-Chief of the Wevovler 2023 Edge AI Report.Danny ShapiroRedwood City, United StatesDanny Shapiro is NVIDIA’s Vice President of Automotive, focusing on solutions that enable faster and better design of automobiles, as well as in-vehicle solutions for infotainment, navigation and driver assistance. He's a 25-year veteran of the computer graphics and semiconductor industries, and has been with NVIDIA since 2009. Prior to NVIDIA, Danny served in marketing, business development and engineering roles at ATI, 3Dlabs, Silicon Graphics and Digital Equipment. He holds a BSE in electrical engineering and computer science from Princeton University and an MBA from the Hass School of Business at UC Berkeley. He lives in Northern California, where his home solar panel system charges his electric car.Alexander WischnewskiMunich, GermanyManaging Director and Co-Founder of, driveblocks, a modular, scalable, robust and safe platform for autonomous driving with a focus on commercial vehicle applications. Former PhD lead for the TUM Autonomous Motorsport team.&nbsp;Matteo BaraleMilan, ItalyCo-CEO of autonomous mobility startup, PIX MOVING. Design strategy expert with experience in robotics, industrial design, transportation, architecture, titled Technology Pioneer by World Economic Forum.&nbsp;David WebbLondon, United KingdomHead of Innovation at Centre for Connected and Autonomous Vehicles. David Webb is Head of Innovation for the UK Government’s Centre for Connected &amp; Autonomous Vehicles. CCAV works across government to support the UK’s developing connected and automated vehicle market. CCAV believes that CAVS could change the way we travel, making road transport safer, smoother and more accessible to all. David has a Masters in Aerospace Engineering from Queen Mary, University of London and has spent the last 10 years working for the Ministry of Defence in a variety of technical, analytical and engagement roles.&nbsp;John SoldatosAthens, GreeceHonorary Research Fellow at the University of GlasgowJohn Soldatos holds a Ph.D. in Electrical &amp; Computer Engineering from the National Technical University of Athens (2000) and is currently an Honorary Research Fellow at the University of Glasgow, UK (2014-present). He was Associate Professor and Head of the Internet of Things (IoT) Group at the Athens Information Technology (AIT), Greece (2006–2019), and Adjunct Professor at the Carnegie Mellon University, Pittsburgh, PA (2007–2010).&nbsp;He has significant experience working closely with large multi-national industries (e.g., IBM, INTRACOM, INTRASOFT International) as an R&amp;D consultant and delivery specialist while being a scientific advisor to various high-tech startup enterprises. Dr. Soldatos is an expert in Internet-of-Things (IoT) and Artificial Intelligence (AI) technologies and applications, including IoT/AI applications in smart cities, finance (Finance 4.0), and industry (Industry 4.0).&nbsp;Jess MileyBerlin, Germany&nbsp;Content Director at WevolverContent specialist with a background in architecture and design. Jess has experience working with neuro-tech startups, animated video studios, and news sites as a writer, editor, and business innovation consultant.&nbsp;The text was further contributed to by our report sponsors and partner.&nbsp;<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzAxMTE4ODczNjgzLXNwb25zb3JzIGxvZ29zLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib"><h3 dir="ltr" id="isPasted">Click through to read each of the report's chapters.</h3>2023 Autonomous Vehicle ReportAutonomous Vehicle Introduction<a href="https://www.wevolver.com/article/2023-autonomous-vehicle-report-sensing-technologies/draft" target="_blank">Chapter 1: Sensing Technologies</a><a href="https://www.wevolver.com/article/2023-autonomous-vehicle-report-thinking-and-learning/draft" target="_blank">Chapter 2: Thinking and Learning</a><a href="https://www.wevolver.com/article/2023-autonomous-vehicle-report-edge-computing-and-real-time-operating-systems-for-autonomous-vehicles/draft" target="_blank" rel="noopener noreferrer">Chapter 3: Edge Computing and Real-time Operating Systems for Autonomous Vehicles</a><a href="https://www.wevolver.com/article/2023-autonomous-vehicle-report-communication-and-connectivity/draft" target="_blank" rel="noopener noreferrer">Chapter 4: Communication and Connectivity</a><a href="https://www.wevolver.com/article/2023-autonomous-vehicle-report-security/draft" target="_blank" rel="noopener noreferrer">Chapter 5: Security</a><a href="https://www.wevolver.com/article/2023-autonomous-vehicle-report-tech-stack-review/draft" target="_blank" rel="noopener noreferrer">Chapter 6: Autonomous Vehicle Tech Stack Review</a>Leadership Interviews<a href="https://www.wevolver.com/article/2023-autonomous-vehicle-report-interview-accelerating-av-development-through-customer-collaboration/draft" target="_blank">Interview with Mouser&nbsp;</a><a href="https://www.wevolver.com/article/2023-autonomous-vehicle-report-interview-building-trust-in-autonomous-driving-navigating-future-reliability-and-milestone-achievements/draft" target="_blank">Interview with Murata&nbsp;</a><a href="https://www.wevolver.com/article/2023-autonomous-vehicle-report-interview-the-role-of-pcb-in-shaping-autonomous-vehicle-development/draft" target="_blank">Interview with MacroFab</a><a href="https://www.wevolver.com/article/2023-autonomous-vehicle-report-interview-exploring-the-future-of-high-performance-computing-in-autonomous-vehicles/draft" target="_blank">Interview with Nexperia</a><a href="https://www.wevolver.com/article/2023-autonomous-vehicle-report-interview-driving-global-standards/draft" target="_blank">Interview with SAE International</a><a href="https://www.wevolver.com/article/2023-autonomous-vehicle-report-interview-advancing-open-source-development-in-the-automotive-industry/draft" target="_blank">Interview with Autoware Foundation</a><a href="https://www.wevolver.com/article/2023-autonomous-vehicle-report-interview-insights-from-nvidia-about-artifical-intelligence-in-avs/draft" target="_blank">Interview with NVIDIA</a>

<h1 id="isPasted">Autonomous Vehicle Report</h1>In 2020, Wevolver launched its first <a href="https://www.wevolver.com/article/2020.autonomous.vehicle.technology.report" target="_blank" rel="noopener noreferrer">Autonomous Vehicle Report</a> that provided a comprehensive knowledge foundation about the technologies enabling autonomous cars. We are now pleased to launch a new report that surveys the advances in this arena over the last three years. Further, we provide a snapshot comparison of four leading AV companies, their tech stacks, and approaches to understand how these technologies are applied in actual use cases. The report is augmented with interviews with the report sponsors, who provide deeper insights into the current state of autonomous vehicles, highlighting their priorities, challenges, and leadership objectives.&nbsp;The report examines autonomy from the perspective of passenger vehicles, following the approach of the previous report. However, many of the technologies mentioned are also relevant for other autonomous vehicle types that are making a significant impact in industries and applications such as last-mile delivery, warehouse and logistics, agriculture, mining, search and rescue, and healthcare.&nbsp;To create this report, we interviewed dozens of industry experts and collaborated with technical researchers and writers from around the world. We have attempted to cover all relevant technologies; however, due to space constraints, we had to limit some areas.&nbsp;This report was made possible by the generous support of its sponsors, the tireless effort of the Wevolver team, the expertise and generosity of our consulting experts, and the attention to detail of our writers, researchers, and designers.We hope you find value in this report, and we look forward to continuing to make this critical knowledge available for all.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzAwODIwOTIwMDE1LU1pdHN1YmlzaGkgQ2hlbWljYWwgR3JvdXAgIEdyb3d0aCBHYXJhZ2UgIE9wZW4gSW5ub3ZhdGlvbiBQcm9ncmFtICg2OTQgeCA0MzIgcHgpICg5MDEgeCA1NjEgcHgpLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib"><iframe width="540" height="450" src="https://c8056756.sibforms.com/serve/MUIFAMBWtqlAziu5EwFUIjLCWmRdzCiwOJRx3R3LyYkfG1dfYIvOKLn1dfBY8M58jYUZY8sQm7DnIOmrBjZ93ycsAv4ICzMbRD95I3S4FIWFM6EU4i3C7yzFNYn-Kfm0vJN96NgNO2_w4BQcrnOEwEVShkElbpBLjOT3ARKFBFADGHd-PDjMwJZI5avI3TUfr-xTC_fVunqO5MKo" frameborder="0" scrolling="auto" allowfullscreen="" style="display: block;margin-left: auto;margin-right: auto;max-width: 100%;" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</iframe><h2>Report Contributors</h2>Cassiano Ferro MoraesFlorianópolis, BrazilTechnical writer with an electrical engineering background and over five years of experience writing articles about electronics, renewable energy, and electric vehicles. Published multiple articles in IEEE explore, including publications in some of the most recognized conferences and journals of the power electronics industry. Vast experience writing articles in the field of electronics, renewable energy, power converters, hardware-in-the-loop, and electric vehicles at Write Choice.&nbsp;Dr. Miroslav MilovanovicNiš, SerbiaAssistant professor at the Faculty of Electronic Engineering, holding a PhD in Computer Science and Electrotechnics. Leader of the Laboratory for Intelligent Control within the Control Systems Department. Author of over 45 scientific publications, centered on Data Science and Deep Learning applications.Gustavo BruismaPato Branco, BrazilElectrical Engineer graduated at the Federal Technological University of Paraná (UTFPR) with MSc. degree in the same field with a deep focus on control systems for electric vehicles. Vast experience writing articles in the field of electronics, renewable energy, power converters, hardware-in-the-loop, and electric vehicles at Write Choice.&nbsp;Ana Carla SorgatoFlorianópolis, BrazilEnvironmental and Sanitary Engineer graduated at the Federal University of Santa Maria (UFSM) with a MSc. in Environmental Engineering from UFSC. Currently pursuing a Ph.D. in Environmental Engineering at the Federal University of Santa Catarina (UFSC) and writing articles about renewable energy, electric vehicles, and engineering at Write Choice.Ian DicksonLondon, United KingdomFreelance automotive journalist and editor with more than two decades’ experience in consumer media and branded content. Cutting his teeth as a road tester on What Car? – where he learnt the art of turning the complex into the simple – he quickly moved through the ranks and ended up editor of MSN Cars, at the time the UK’s largest motoring website. For the past 10 years, he’s been creating and developing content and strategies for brands like Porsche, Ferrari, Volvo Cars and HERE Technologies.Samir JaberLeipzig, GermanyContent Specialist with a background in engineering, nanotechnology, and scientific research. Samir has comprehensive experience working with major engineering and technology companies as a writer, editor, and digital marketing consultant. Featured author in 30+ industrial magazines with a focus on IoT, nanotechnology, materials science, engineering, and sustainability. Samir is also an award-winning engineering researcher in the fields of nanofabrication and microfluidics. Editor-in-Chief of the Wevovler 2023 Edge AI Report.Danny ShapiroRedwood City, United StatesDanny Shapiro is NVIDIA’s Vice President of Automotive, focusing on solutions that enable faster and better design of automobiles, as well as in-vehicle solutions for infotainment, navigation and driver assistance. He's a 25-year veteran of the computer graphics and semiconductor industries, and has been with NVIDIA since 2009. Prior to NVIDIA, Danny served in marketing, business development and engineering roles at ATI, 3Dlabs, Silicon Graphics and Digital Equipment. He holds a BSE in electrical engineering and computer science from Princeton University and an MBA from the Hass School of Business at UC Berkeley. He lives in Northern California, where his home solar panel system charges his electric car.Alexander WischnewskiMunich, GermanyManaging Director and Co-Founder of, driveblocks, a modular, scalable, robust and safe platform for autonomous driving with a focus on commercial vehicle applications. Former PhD lead for the TUM Autonomous Motorsport team.&nbsp;Matteo BaraleMilan, ItalyCo-CEO of autonomous mobility startup, PIX MOVING. Design strategy expert with experience in robotics, industrial design, transportation, architecture, titled Technology Pioneer by World Economic Forum.&nbsp;David WebbLondon, United KingdomHead of Innovation at Centre for Connected and Autonomous Vehicles. David Webb is Head of Innovation for the UK Government’s Centre for Connected &amp; Autonomous Vehicles. CCAV works across government to support the UK’s developing connected and automated vehicle market. CCAV believes that CAVS could change the way we travel, making road transport safer, smoother and more accessible to all. David has a Masters in Aerospace Engineering from Queen Mary, University of London and has spent the last 10 years working for the Ministry of Defence in a variety of technical, analytical and engagement roles.&nbsp;John SoldatosAthens, GreeceHonorary Research Fellow at the University of GlasgowJohn Soldatos holds a Ph.D. in Electrical &amp; Computer Engineering from the National Technical University of Athens (2000) and is currently an Honorary Research Fellow at the University of Glasgow, UK (2014-present). He was Associate Professor and Head of the Internet of Things (IoT) Group at the Athens Information Technology (AIT), Greece (2006–2019), and Adjunct Professor at the Carnegie Mellon University, Pittsburgh, PA (2007–2010).&nbsp;He has significant experience working closely with large multi-national industries (e.g., IBM, INTRACOM, INTRASOFT International) as an R&amp;D consultant and delivery specialist while being a scientific advisor to various high-tech startup enterprises. Dr. Soldatos is an expert in Internet-of-Things (IoT) and Artificial Intelligence (AI) technologies and applications, including IoT/AI applications in smart cities, finance (Finance 4.0), and industry (Industry 4.0).&nbsp;Jess MileyBerlin, Germany&nbsp;Content Director at WevolverContent specialist with a background in architecture and design. Jess has experience working with neuro-tech startups, animated video studios, and news sites as a writer, editor, and business innovation consultant.&nbsp;The text was further contributed to by our report sponsors and partner.&nbsp;<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1701276054993-Untitled+%28600+x+200+px%29+%281%29.png" style="width: 300px;" class="fr-fic fr-dib"> <h2 id="isPasted">Leadership Interviews</h2><a href="https://www.wevolver.com/article/2023-autonomous-vehicle-report-interview-accelerating-av-development-through-customer-collaboration" target="_blank" rel="noopener noreferrer">Interview with Mouser</a><a href="https://www.wevolver.com/article/2023-autonomous-vehicle-report-interview-building-trust-in-autonomous-driving-navigating-future-reliability-and-milestone-achievements" target="_blank" rel="noopener noreferrer">Interview with Murata</a><a href="https://www.wevolver.com/article/2023-autonomous-vehicle-report-interview-the-role-of-pcb-in-shaping-autonomous-vehicle-development" target="_blank" rel="noopener noreferrer">Interview with MacroFab</a><a href="https://www.wevolver.com/article/2023-autonomous-vehicle-report-interview-exploring-the-future-of-high-performance-computing-in-autonomous-vehicles" target="_blank" rel="noopener noreferrer">Interview with Nexperia</a><a href="https://www.wevolver.com/article/2023-autonomous-vehicle-report-interview-driving-global-standards" target="_blank" rel="noopener noreferrer">Interview with SAE International</a><a href="https://www.wevolver.com/article/2023-autonomous-vehicle-report-interview-advancing-open-source-development-in-the-automotive-industry" target="_blank" rel="noopener noreferrer">Interview with Autoware Foundation</a><a href="https://www.wevolver.com/article/2023-autonomous-vehicle-report-interview-insights-from-nvidia-about-artifical-intelligence-in-avs" target="_blank" rel="noopener noreferrer">Interview with NVIDIA</a>

Introduction

Sensing Technologies

Thinking and Learning

EDGE and RTOS

Communication and Connectivity

Security

Tech Stack

Examining the latest developments in self-driving vehicles.

Autonomous Vehicle Technology Report

<a href="https://precisiebeurs.nl/home-en/" target="_blank" rel="noopener noreferrer">The Precision Fair 2024</a>, held on November 13 and 14, 2024, at the Brabanthallen in 's-Hertogenbosch, the Netherlands, was a significant event in the field of precision technology. This fair served as a crucial platform for the entire precision technology value chain, emphasizing the importance of this technology as a stable foundation for many devices and components in our daily lives.&nbsp;Precision technology&nbsp;involves the development, production, and assembly of complex products requiring very high accuracy. Think, for example, of products such as microchips, microscopes, measuring equipment, and robots.During its 22nd edition, the Precision Fair facilitated meetings, collaborations, and knowledge sharing, which are essential to keep pace with rapid technological advancements.&nbsp;The event featured six main topics: Mechatronic Engineering Systems, Metrology, Vacuum &amp; Clean, Micro Processing, Motion &amp; Laser Photonics, and Production for High Precision. These topics allowed visitors to engage with experts and explore solutions in these cutting-edge areas.The Precision Fair has consistently played a pivotal role in bringing together various stakeholders in the precision technology sector to collaborate on the next generation of technological innovations. The event was organized by Mikrocentrum and supported by various media partners, underlining its significance in the industry.<iframe width="640" height="360" src="https://www.youtube.com/embed/2NDTXiOq_C4?&amp;feature=youtu.be&amp;wmode=opaque&amp;rel=0" frameborder="0" allowfullscreen="" class="fr-draggable"></iframe> <h2 dir="ltr">Highlights from the Precision Fair 2023</h2>Highlights from the 2-day event included:<h3 dir="ltr">Innovative Deburring Machines&nbsp;</h3>Dieter Munz, Director of Ultratec Innovation GmbH, presented deburring machines equipped with a robotic system for 24/7 workpiece loading. This innovation highlighted the strides in automation and manufacturing efficiency.<h3 dir="ltr">European Semiconductor Market Insights</h3>Dr. Theresa Spaan-Burke from VDMA EMINT provided valuable insights into the European semiconductor market, discussing its business climate and future outlook, which is crucial for understanding industry trends.<h3 dir="ltr">Digital Transformation in Precision Machines</h3>Mauro Fusco, Senior Application Engineer at The MathWorks BV, discussed the integration of digital technologies in precision machines, emphasizing the importance of digital transformation in enhancing machine performance.<h3 dir="ltr">The Power of Precision</h3>Frank Loobeek from <a href="https://mifa.eu/en" target="_blank" rel="noopener noreferrer">Mifa Aluminium BV</a> highlighted the significance of precision in manufacturing, focusing on how precision impacts product quality and performance.<h3 dir="ltr">Advancements in Metal Additive Manufacturing</h3>Rob van Loon from <a href="https://www.kmwe.com/" target="_blank" rel="noopener noreferrer">KMWE Precision BV</a> explored the challenges and solutions in metal additive manufacturing, particularly focusing on the impact of tolerances on manufacturability and cost.<h3 dir="ltr">Data-Driven Design Improvement</h3>Dr. Willem Dijkstra from VDL ETG presented on the use of data analytics in improving the design and reliability of precision components, showcasing the potential of data-driven approaches in precision engineering.<h1 dir="ltr">Young Talent Award Winners</h1>New innovations will be driven by the next generation, which is why Mikrocentrum offers young talent and their budding technology projects a stage there through its Young Talent Program.&nbsp;Start-ups and student teams send in a 30-second pitch about their technology or product and hold a three-minute pitch on the day itself. A jury composed of Bert-Jan Woertman, the Director of Mikrocentrum, and Richard Hulskes, Co-founder of Wevolver, among others, decide the winners based on factors such as the design and substantiation underpinning the presented innovation.<h3 dir="ltr">2024 Young Talent Award Winners&nbsp;</h3><a href="https://www.linkedin.com/company/mikrocentrum/" target="_blank">Mikrocentrum</a> presented the Young Talent Award to <a href="https://www.linkedin.com/company/project-march/" target="_blank">Project MARCH</a> and <a href="https://www.linkedin.com/company/wevolver/" target="_blank">Wevolver</a> to <a href="https://www.linkedin.com/company/team-polar/" target="_blank">Team POLAR</a>. Both teams make real social impact with their solutions.Project March creates exoskeletons that enable people with spinal cord injuries to have a greater range of motion and movement. &nbsp;help people with spinal cord injuries.&nbsp;<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzAwNDc1NjU0OTQxLTNELU1vZGVsLW9mLU1hcmNoLUlWLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib">Concept of the exoskeleton. Image: Project March.&nbsp;Team Polar aims to venture into the unexplored territories of the Antarctic with an autonomous and sustainable vehicle. They want to enable global accessibility of Antarctic research via a modular research platform, detecting new routes, and doing this all in an economical, safe and environmentally-friendly way.&nbsp;Team Polar won $5000 worth of media support from Wevolver. Stay tuned for more articles about this innovative group soon!

The Precision Fair 2023 served as a platform for professionals to share knowledge, explore new ideas, and witness the latest technological advancements, underlining the sector's vital role in various industries and scientific research.

Precision Fair Trade Show showcases young talent enabling autonomous Antarctic research

The intralogistics specialist STILL, a subsidiary of the KION Group, leads the German project consortium of the Intelligent Motion Control -IMOCO4.E- research project.The goal of the IMOCO4.E project is to enable intelligent trucks to navigate autonomously in a factory or warehouse.In numerous warehousing and production settings, semi-autonomous trucks are already in use and have already proven to be profitable assets. However, they remain constrained by certain limitations.&nbsp;While these trucks can presently detect obstacles and apply brakes independently, achieving complete self-driving capabilities requires further development. Intelligent trucks should be able to navigate warehouse environments fully autonomously. They should be able to understand their surroundings, avoid obstacles, and find the most efficient routes.The IMOCO4.E project focuses on applying solutions based on Artificial Intelligence (AI) to address challenges in four main scenarios: navigation, collection, and movement of goods, and delivery to their final position.In this context, STILL's iGo neo platform, which already comes very close to the idea of this autonomously driving vehicle due to its 'intelligent' equipment and the resulting capabilities, &nbsp;was equipped with an Alphasense Autonomy unit from Sevensense Robotics.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk5NDQ4MDQ2ODU4LUFwbGhhc2Vuc2UgQXV0b25vbXkgaW5zdGFsbGVkIG9uIGlHTyBOZW8ucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib" alt="Alphasense Autonomy unit installed on a STILL iGo neo platform - Image credits STILL">Alphasense Autonomy leverages Visual AI technology to provide precise navigation and obstacle avoidance capabilities in complex and dynamic warehouse environments -with racking, inbound and outbound areas, and even when transitioning from indoors to outdoors.The Visual AI system maps out the environment and provides precise positioning without using any type of infrastructure.In addition, Sevensense is prototyping the implementation of AI to perform Semantic Scene Understanding, which allows the vehicles to extract information about the objects in their environment, to detect, classify, determine the position, -and when applicable, predict the movement of obstacles e.g. people and vehicles, and other objects of interest such as pallets, docking or charging stations.With this information, the vehicles are able to adapt their navigation behavior -stopping, slowing down, or overtaking obstacles-, based on real-time perception, allowing them to interact with the environment, for example, docking, picking, or charging, while safely sharing the floor with moving elements like people and other vehicles.“In the IMOCO4.E4.E project, STILL is partly refining and extending existing approaches with regard to general interactions of his iGo neo platform in an unknown environment. The high precision and reliability of the localization provided by Sevensense's Visual AI technology is of great benefit here",&nbsp;says Ansgar Bergmann, responsible for the IMOCO4.E project at STILL.The collaboration between Sevensense Robotics and STILL within the IMOCO4.E research project marks a significant stride toward the realization of fully autonomous trucks in warehouse and factory environments.The project is scheduled to be completed in the fourth quarter of 2024.<a href="https://www.sevensense.ai/" target="_blank" rel="noopener noreferrer">Learn more about Sevensense Robotics and its Visual AI technology</a>

STILL's iGo neo platform operating in a warehouse setting - image credits STILL

Sevensense’s Visual AI implemented into STILL's iGo neo platform for precise Autonomous Navigation and Pallet Handling

Still Incorporates Visual AI Technology on IMOCO4.E Research Project

<h2 dir="ltr" id="isPasted">FREE REPORT: Reducing Human Driver Error and Setting Realistic Expectations with Advanced Driver Assistance Systems</h2>Thousands die or are injured each year in automobile crashes. Bringing automated driving systems technologies into the advanced driver assist systems (ADAS) and connected vehicle space will help humans drive more safely and better prepare us for automated vehicles (AVs).Learn more about ADAS and ADS implementation with the goal of reaching zero deaths and serious injuries by downloading this FREE report from SAE International (valued at $50).&nbsp;<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzAwNzU0MjEyOTk3LVJlZHVjaW5nIEh1bWFuIERyaXZlciBFcnJvciBhbmQgU2V0dGluZyBSZWFsaXN0aWMgRXhwZWN0YXRpb25zIHdpdGggQWR2YW5jZWQgRHJpdmVyIEFzc2lzdGFuY2UgU3lzdGVtcy5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"><h2 dir="ltr"><iframe width="540" height="740" src="https://c8056756.sibforms.com/serve/MUIFAIm7QnvQQ9ZyzkX--H5jNS-TSb9GWr5SQBXi_SUjRPpt67CBveq_yevZYMSeiJ-O1dwYw05f2kl-Q7sfNDRem1Y-03GXYZV4qkTbJeR_K_VpJzgMk_8Ry8fjyfgJ7fAMrCmcRHc1KXsSTuhIU5p5pBa3F1-TGDgOdQgcF-pU8cSTrl8JF2j2b9gl6-5e66Ro-rltcIrGxsRG" frameborder="0" scrolling="auto" allowfullscreen="" style="display: block;margin-left: auto;margin-right: auto;max-width: 100%;" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true">&nbsp;</iframe>Moving toward vehicles equipped with Automated Driving System (ADS) technologies</h2>Over the last 100 years, the transportation industry has been developed primarily around human operators. For example, roadway signs were designed to be easily seen, read and interpreted by humans. However, as vehicle-automation technologies are evolving, driving responsibilities are beginning to shift from the human operator to the automobile. The industry is rapidly moving toward vehicles equipped with Automated Driving System (ADS) technologies&nbsp;— a but a key concern for this inevitable mass deployment of ADS-equipped vehicles is earning public confidence that ADS technology can safely and reliably operate in a mixed environment of automated and human drivers.There also is the expectation that the ADS-equipped vehicles perform equal to or better than their human counterparts.In this vein, ADS developers and Infrastructure Owner-Operators (IOOs) that manage the roadways aspire to safe and efficient operation of automated vehicles. The U.S. Department of Transportation’s (DOT’s) Federal Highway Administration’s (FWHA) Testing and Pilot Design, Development and Evaluation Framework (i.e. “the Framework”) project for ADS-developers and IOOs was launched to assist in fostering collaboration. The goal was to develop a collaborative framework to use in ADS test and pilot programs — the outcomes of which can benefit both ADS and infrastructure entities.Collaboration, not regulation The successful operation of ADSs on U.S. roadways is heavily dependent upon the ADS being able to reliably interpret and navigate its surrounding environment. However, there remain many challenges: ADSs navigating complex traffic scenarios, adapting in adverse weather and identifying infrastructure that facilitates the ADS’ navigation, to name a few. Given that ADS developers and roadway stakeholders seem to own diverse but complementary visions of the testing and evaluation needs, the likelihood of safe and efficient deployment is necessarily contingent on productive industry-wide collaboration.The FHWA Framework for successful testing and evaluation of ADS and roadway features emphasizes collaboration among the ADS and infrastructure organizations (see more <a href="https://ops.fhwa.dot.gov/publications/fhwahop21012/fhwahop21012.pdf" target="_blank" rel="noopener noreferrer" data-lf-fd-inspected-kn9eq4roawkarlvp="true">here</a>). The Framework is technology-independent and consists of a broad set of processes and examples that advise without prescribing regulations or policy. Instead, the Framework adopts the notion that validation and testing of ADS technologies and various infrastructure features is essential to paving the way for safe deployment of ADS-equipped vehicles into the road network.Collaborative testing promises to provide resolution to these issues, resulting in a safe and efficient deployment of ADS. Overall, these efforts will further advance deployment of ADS-equipped vehicles onto roadways throughout the U.S. — with desired outcomes benefiting both ADS and infrastructure entities.Consider a typical State Department of Transportation (DOT) operations scenario where a busy highway is being resurfaced or expanded. Presently, the procedures and standards outlined by the DOT are designed to maximize the safety of the crew within the work zone, while still ensuring mobility for drivers around the work zone. For instance, one direction of travel is shifted from the left lane over the centerline into a lane typically used for the opposite direction of travel (see Figure 1).<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk5MzY0NTQ3Mzc1LTE2OTkzNjQ1NDczNzUucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" id="isPasted" class="fr-fic fr-dii">Figure 1: Example of typical work-zone traffic-flow arrangement. The FHWA's framework for ADS developers seeks to assure that all ADSs would recognize this scenario. (U.S. DOT)Can anyone be certain that the automated vehicles of tomorrow will be able to operate safely through a work zone? The answer lies in the quality and scope of testing. Both the developers of the ADS and the workers within the DOT have a keen interest in how this testing is administered and evaluated.Framework overview The Framework was developed with extensive engagement and input from both ADS and roadway stakeholders, including automotive OEMs, suppliers, technology companies and state, federal and regional government entities. It is far more likely that their combined perspectives can accomplish these goals more rapidly and with fewer errors than could the respective individual entities.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk5MzY0NjYxODkwLTE2OTkzNjQ2NjE4OTAucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib">Figure 2: The framework describes nine overarching themes that support its four key test phases. (U.S. DOT) As shown in Figure 2, the Framework addresses nine overarching themes (described below) that support the four key test phases. The Framework embodies the contextual examples, real-world lessons learned, and various considerations, as they will vary across different tests and disciplines (e.g., ADS developers, IOOs, first responders, fleet operators). Each party has a responsibility to engage in the testing and evaluation process for automated driving systems, and this Framework enables collaborative support toward the shared goals.Collaboration Collaboration between ADS and IOO stakeholders is critical for successful testing and evaluation. Stakeholder collaboration allows for early detection and resolution of ADS issues related to technical, organizational, and strategic test implementations. Open and frequent interactions lead to improved test outcomes.As an example, the Pennsylvania DOT (PennDOT) recognized the need to prepare for the mass deployment of ADS across the state and assembled nine partners (Blazina, 2019), and (Paez, 2019) to work on changing infrastructure to support ADS in work zones by using distinctive coatings for lane markings and barrels to aid in detection by the ADS. In addition, the project developed advanced mapping and communications systems for safe ADS navigation at and around 17 different work zones. Prior to testing in active work zones, the partnership conducted validation in virtual environments, then at a test track. PennDOT will have a better understanding of how ADS systems operate in certain conditions, and thus will be better-equipped to fulfill its mandate for keeping drivers safe and informed.Common Ground Common Ground refers to creating a common or shared working environment so that ADS and IOO stakeholders fully understand each other. When executing ADS/roadway tests, all parties will have clearly-defined expectations, outcomes and success criteria; these include: common goals and benefits, common terminology and common metrics and measures.Test Logistics Test Logistics refers to what to test, how to test, and where to test. This includes development of test scenarios, testing methodologies and the test environment.Institutional / Organization Issues Having organizational experts from both the ADS and IOO organizations participate early and throughout the test phases will greatly aid in navigating any difficulties. The Framework adopts the premise that for ADS/roadway testing and evaluation, safety of all road users is the greatest priority. Within this premise, state and local regulatory policies must be developed, requiring that the policymakers be well-informed on relevant topics and kept up to date on developments.Roles and Responsibilities In the process of ADS/roadway testing and evaluation, it is important to identify whom from the various organizations needs to participate, what roles within the organizations are needed and when (i.e., in which test phases) they need to participate.Plans For a collaborative environment to exist, stakeholders’ participation as part of the test-design plan, data-collection plan and evaluation plan can be beneficial.One example is Waymo’s First Responder Engagement Plan. The objective of this plan is to, in an emergency scenario, provide first responders with the knowledge they need to safely identify, approach and interact with a vehicle equipped with an ADS.Sharing Opportunities Data is a key issue that requires thorough discussions from IOO and ADS stakeholders to avoid challenges (e.g., proprietary data/information, use of data). Resource sharing includes sharing of skills and expertise in addition to sharing of information and existing data.An example is the Arizona IAM Consortium Collaborative Data Sharing. Arizona’s IAM consortium is leveraging existing infrastructure to collect performance data on public roads.A New Driver The new driver of tomorrow will be the vehicle. This Framework is an instrument that provides examples and scenarios to those conducting tests, pilots to prepare for a safe and functional road network with, at first, both human and ADS drivers sharing the road — and then aid in transitioning the network into the ‘new normal.’Success Factors The most critical success factors include enhancement of technical maturity, comprehension of ADS, roadway test elements, the process, stakeholder engagement and collaboration and ongoing public communications. The Framework aids in assisting the ADS and IOO participants in defining test success factors within each test phase.FRAMEWORK APPLICATION The activities for successful collaborative testing and evaluation can be categorized into four test phases, shown in Figure 3.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk5MzY0NzczMTM2LTE2OTkzNjQ3NzMxMzYucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib">Figure 3: The FHWA Framework's four key test phases. (U.S. DOT) Pre-Test Phase The output of the Pre-Test phase is a clearly-identified problem statement that is based both on the internal needs of the ADS developers and the industry. Program risks – including operational, technical, data, legal and financial — should be identified during this phase.Test Definition Phase The objective of the Test Definition phase is to conduct activities which help define the technical and data facets of a collaborative test program. Having clearly-defined success criteria ensures an overall higher test quality.Test Execution Phase In the Test Execution phase, both technical and data facets of the collaborative ADS and roadway testing proceed as defined in the Test Plan. The focus is on efficient collection of performance data. After completion, ADS performance data is gathered and reviewed to determine if the system is ready to advance to the next phase. For example, if the performance failure at night is considered critical by the stakeholders, then it is likely new tests will be scheduled after updates are made.Post-Test Phase In the Post-Test phase, stakeholders review data insights, store data and discuss lessons learned and evaluate the processes used, which drives future programs. This collaborative effort directly leads to a higher level of confidence among the stakeholders — and by extension the public — that the ADS-equipped vehicles will be safely deployed among existing human drivers.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk5MzY0ODM3MzM2LTE2OTkzNjQ4MzczMzYucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib">Figure 4: Driverless robotaxis such as Cruise's AV are a high-profile example of the need for collaboration between AV developers and infrastructure entities. (Cruise) Advancing Successful Outcomes ADS technologies traditionally have been developed and tested with little or no discussion with roadway stakeholders.However, the Collaborative Research Framework for Automated Driving System Developers and Infrastructure Owners and Operators addresses collaboration from multiple perspectives throughout the test lifespan. It provides examples of where collaboration has yielded successful outcomes, the benefits of collaboration and information on how and when to collaborate.Successful collaboration among stakeholders will allow for early detection and resolution of ADS and infrastructure issues related to technical, organization and strategic test implementations. Ultimately, this helps focus the efforts of both the ADS development teams and IOO stakeholders working toward the common goal of safe and efficient deployment of autonomous vehicles.This article was written by the U.S. Dept. of Transportation’s Federal Highway Administration (FHWA).

The Mercedes-Benz Drive Pilot automated-driving system engages distinctively-colored marker lights to inform other road users that the vehicle's ADS is engaged. (Mercedes-Benz)

The Mercedes-Benz EQS sedan equipped with the company’s Drive Pilot automated driving system (ADS), which the company said is the first vehicle with SAE Level 3 automated-driving capability approved by two U.S. states – California and Nevada – for operation on public roads. (Mercedes-Benz).


The 'Framework' for AV development

The safety of pedestrians and cyclists is important in any major city.Bike lanes and crosswalks are the most obvious signs that we create infrastructure for this purpose. But there are limits to what that infrastructure can do.Winter, for example, creates additional threats. Ice and snow are the most obvious problems, but potholes and deep puddles can also cause havoc – particularly for cyclists. Snow can accumulate quickly in a storm – and it’s not uncommon for snow plowed from the roadway to sometimes spill over into bike lanes (and even on sidewalks).That’s part of the reason why Indro Robotics has built an innovative solution that will be put to the test this winter. It’s called the Street Smart Robot, or SSR – and it’s been designed from the ground up to help ensure safe winter cycling.Below: A Wikimedia Commons image, taken in Whitehorse by <a href="https://www.flickr.com/people/71902268@N00" rel="nofollow" target="_blank">Anthony DeLorenzo</a>. Cyclists in Canada are a hardy bunch.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk4Njg1MjA5MDA1LTQwX2JlbG93X2NvbW11dGVfNjczNDE1NzExOS1qcGcud2VicCIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 745px;" class="fr-fic fr-dib"><h3> </h3><h3 id="isPasted">Street Smart Robot</h3>The catalyst for the SSR is a research and development partnership fund called the <a href="https://www.ovinhub.ca/programs/wintertech-av-development/" target="_blank" rel="noopener noreferrer">Wintertech Development Program</a>. &nbsp;Wintertech is run by <a href="https://www.oc-innovation.ca/programs/ontario-vehicle-innovation-network-ovin/" target="_blank" rel="noopener noreferrer">OVIN, the Ontario Vehicle Innovation Network</a>. That’s a province of Ontario initiative which “capitalizes on the economic potential of advanced automotive technologies and smart mobility solutions such as connected and autonomous vehicles (CAVs), and electric and low-carbon vehicle technologies, while enabling the province’s transportation and infrastructure networks to plan for and adapt to this evolution.”And while bicycles aren’t connected and autonomous vehicles yet, we felt robotics could play a role in helping to ensure the safety of cyclists in the Smart City of the future. Specifically, we thought an autonomous robot equipped with the right sensors and processing might help to detect safety issues in bike lanes in the winter.“The idea behind the robot is we want to prolong the use of bike lanes in Ottawa, but also ensure the safety of bike lanes,” explains Indro Robotics Account Executive Luke Corbeth.“There’s really two parts to this: The&nbsp;first is a machine vision element to see if conditions are good enough for biking – no ice, not too many leaves, etc. On the safety side, the Street Smart Robot is more concerned with detecting things like potholes and cracks. And the idea is if you’re able to identify those things, the right resources can be deployed faster and more efficiently to solve the problem in a timely manner.”So that’s the idea behind the SSR. Now, let’s take a look at the technology.Below: A look at the specs of the AgileX&nbsp;Bunker Pro platform, used in SSR. We’ve upgraded the battery for a four-hour run time:<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk4Njk0NDA5MjE1LVNTUiBQb3N0IC0gQnVua2VyIFBybyBTcGVjcy5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 766px;" class="fr-fic fr-dib" alt="Bunker Pro Specs"><h3 id="isPasted">Built Tough</h3>If you’re going to be out and about in an Ottawa winter – where temperatures have reached as low as -33°C – you need a platform suitable for the environment. InDro selected the AgileX <a href="https://indrorobotics.ca/bunker-pro/" target="_blank" rel="noopener noreferrer">Bunker Pro</a> as the starting point.With an IP67 Ingress Protection rating, the Bunker Pro is highly impervious to water and particulate matter – which it will encounter in abundance on Ottawa streets. The treads give an edge over wheels when it comes to navigating over ice, snow and potholes. With a full charge, the platform can run for some four hours – even when it’s cold out.But the platform is just the start. InDro engineers had to put considerable thought into the kinds of sensors that would help the Street Smart Robot carry out its task while safely avoiding cyclists, pedestrians, or other obstacles it might encounter. And that began with four wide-angle pinhole cameras. “The pinholes are just to give the operator a better understanding of their environment and situational awareness,” explains Corbeth. And while we say “operator” here, it might be more appropriate to say “observer.” Though initial deployments will involve teleoperation, the SSR has been built to carry out its tasks autonomously. We anticipate that down the road (or bike path), a person will simply be involved in monitoring missions in case human intervention is required.That autonomy will be carried out by a combination of computing power, our InDro Autonomy software stack, and a plethora of sensors.&nbsp;<h3>Sensors, Sensors, Sensors</h3>The Street Smart Robot is equipped with front- and rear-facing depth cameras. These cameras have two visual sensors each, placed roughly the same distance apart as human eyes. That separation – as in human vision – allows the SSR to perceive the world in depth. Onboard computation (we’ll get to that) calculates how far obstacles might be in the robot’s immediate surroundings, and whether it needs to slow down, stop or change course to avoid obstacles.The depth cameras are supplemented by two 270° 2D LiDAR sensors – which are devoted almost exclusively to obstacle avoidance and safety. There are two additional 3D LiDARS, placed to assist with Simultaneous Localization and Mapping, as well as a GPS and IMU (Inertial Measurement Unit). There’s even a Range Finder that can detect how far an object is off the ground.“If you have an overhanging tree branch, for example, you could see where it is and see if it obscures the bike lane,” says Corbeth.&nbsp;In addition, there’s a thermal/30X optical Pan-Tilt-Zoom camera, complete with a windshield wiper for those sloppy days.Below: Check out what the SSR has on board.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk4NjkzMTY5NTcwLVNTUiBQb3N0IC0gU1NSIFNwZWNzLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 766px;" class="fr-fic fr-dib"> <h3 id="isPasted">Sensing, and Making Sense of its Environment</h3>All of those sensors are great. But they don’t do a lot on their own. They require software and computational power to give their data meaning. Here, the SSR has a Jetson ORIN AGX for EDGE computing. The ROS2 operating system software lives onboard in <a href="https://indrorobotics.ca/commander/" target="_blank" rel="noopener noreferrer">InDro Commander</a> – a module we’ve created for rapid integration of sensors and 5G remote operations. Commander can be thought of as the glue that holds everything together.But still, there’s the issue of making sense of the data. How will the Street Smart Robot identify ice, water, leaves, snow, potholes, debris etc? And how will it determine whether they are significant enough to require a City of Ottawa maintenance crew to take action?Here, the SSR will be going to school. And by that, we mean a combination of Machine Vision and Machine Learning will teach it what’s worth reporting and what isn’t. One way is to take actual images you expect to see on the bike path and identify which ones are significant and which can be ignored.“For example, you could mount a camera on an actual bicycle and then use that dataset to train the robot,” says Corbeth. “But there are a couple of approaches. There’s also obviously the simulation element. You can use specific engines that have high-fidelity graphics to simulate possible different environments in high resolution. And you can use that simulated data for training.”&nbsp;<h3>Then what?&nbsp;</h3>In the early stages, there will be a human operator at the helm. They’ll likely see some of those obstacles with the pinhole cameras, but from the start we anticipate that the onboard AI will trigger an alert for the operator. Once the operator confirms that the image is worthy of the attention of a maintenance crew, they will contact the city directly.That’s how things will start. Once we’ve established a high confidence level that the Machine Vision is correctly identifying anomalies, that process can be automated. The SSR will be programmed to automatically send an alert to the city, complete with a captured image of the potential problem and GPS coordinates.“It’s the idea of anomaly detection – detecting and flagging anomalies. The SSR will also likely be able to identify the severity of the anomaly on a one-to-five scale. And because you have the GPS onboard, the exact location of the problem can be relayed to the city.”Below: The SSR, during its public unveiling at the TCXpo exhibition in June. Front and rear view, to show off all its bits…<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk4NjkzNjI1Nzk5LVNTUiBQb3N0IC0gRnJvbnQgYW5kIEJhY2sgVmlldy5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 766px;" class="fr-fic fr-dib"> <h3 id="isPasted">On The Shoulders of Sentinel</h3>Every technical achievement owes something to its predecessors. And that’s no different with the Street Smart Robot.InDro Robotics previously built <a href="https://indrorobotics.ca/indro-robotics-sentinel/" target="_blank">Sentinel</a>, a workhorse for remote asset inspection. That robot incorporated an AgileX <a href="https://indrorobotics.ca/bunker-4/" target="_blank" rel="noopener noreferrer">Bunker platform</a>, an InDro <a href="https://indrorobotics.ca/commander/" target="_blank" rel="noopener noreferrer">Commander</a> integration module for sensors and remote teleoperations, and several other sensors. It’s a great robot that has proven itself in the most demanding conditions.The Street Smart Robot builds on the success and learnings of Sentinel with additional compute power, many more sensors, a more robust design and extended operational range. And that opens the SSR up to many other tasks beyond bicycle paths.“That’s what really excites me about this project – the idea that this technology is transferable across other inspection verticals,” says Corbeth. “This particular robot could be used in agriculture, and the track platform is really good in mining. I could envision this exact same robot in those two verticals, and others.”&nbsp;<h3>The 5G Connection</h3>When dealing with multiple sensors in real-time, a solid data bandwidth is crucial. Here, SSR is outfitted with a high-speed modem and special antennaes for optimizing 5G and 4G signals. And while the Street Smart Robot can be operated over 4G, it’s 5G that offers the pipeline necessary for the data to really flow. &nbsp;InDro's continuing partnership with Rogers will ensure that the SSR has access to Canada’s largest and most reliable 5G network.<span data-ccp-props='{"134233117":false,"134233118":false,"134245417":true,"134245418":false,"134245529":false,"201341983":0,"335551550":1,"335551620":1,"335559685":0,"335559731":0,"335559737":0,"335559740":240}'>Below: Innovations in the InDro Sentinel paved the way for the Street Smart Robot. This image was taken at the opening of the Drone and Advanced Robot Training and Testing facility (DARTT) in June of 2023. That’s Luke Corbeth holding the microphone.<span data-ccp-props='{"134233117":false,"134233118":false,"134245417":true,"134245418":false,"134245529":false,"201341983":0,"335551550":1,"335551620":1,"335559685":0,"335559731":0,"335559737":0,"335559740":240}'><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk4NjkzODgwNzA5LVNlbnRpbmVsLVdhdGVyLU9uZS1zY2FsZWQuanBnLndlYnAiLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 766px;" class="fr-fic fr-dib"><h3> </h3><h3 id="isPasted">InDro's Take</h3>We’re pretty excited about the SSR. As Luke Corbeth says: “It’s a bad-ass robot.”And it is. But we’re also excited about the project itself.The Smart City of the future will have connected, embedded sensors (including on robots) helping with everything from traffic flow to updated weather and road conditions. Cyclists are a big part of any major urban centre, and the prospect of helping to ensure their safety appeals to us. It’s also an opportunity to really hone in on Machine Vision and Machine Learning as InDro trains the SSR for its task.“Smart Cities are on their way, and it’s great to be involved in yet another project as we build the capacity for the future,” says InDro Robotics CEO Philip Reece.“The Street Smart Robot has an important role to play, and could well be the template for anomaly detection and road maintenance at scale in the future. We’re grateful to OVIN and its Wintertech program for selecting this project for development; we’re excited to see what the SSR can do.”InDro plans to start testing the SSR during the winter, with the project stretching to June of 2024. We’ll be writing more once the robot hits the bike paths.Interested in learning more about how SSR could work for your inspections? &nbsp;Contact InDro <a href="https://indrorobotics.ca/contact-us/" target="_blank" rel="noopener noreferrer">HERE</a>.

For Safer Cycling in the Winter

InDro Robotics Builds Street Smart Inspection Robot

Exploring Cutting-Edge Research with the LIMO Robot

Boston University Uses LIMO Robot For Research

<h2 id="isPasted">Its impact on Visual SLAM and beyond</h2>It is clear that Artificial Intelligence (AI) is here to stay and there is a need for an open discussion, to bring some light into the AI black box.At Sevensense Robotics we strongly believe that transparency builds trust, this is why we decided to tackle the elephant in the room and dissect the role that AI plays in our offering.Let’s start by exploring what AI is and the difference between Machine Learning (ML) and Deep Learning (DL).The original definition of AI, coined in 1955 by John McCarthy -one of the founding fathers of the discipline- was really broad: “The science and engineering of making intelligent machines.”Nowadays, we conceive AI as a subset of computer science focused on creating systems that can replicate human intelligence and problem-solving abilities. It involves any technique that enables machines to mimic human intelligence, using logic, if-then rules, decision trees, and machine learning -including deep learning.Machine Learning (ML) is a subset of AI that intersects with statistics. It involves the development of algorithms and statistical models that enable computers to learn and make predictions or decisions without being explicitly programmed; instead, they recognize patterns in the data and make predictions once new data arrives.Deep Learning (DL) is a specialized subset of Machine Learning that analyzes data with a logical structure similar to how a human would draw conclusions.&nbsp;To achieve this, DL applications use a layered structure of algorithms called an Artificial Neural Network (ANN). It is characterized by its deep layers of interconnected nodes or neurons, which allows it to automatically learn hierarchical representations of data, making it well-suited for tasks like image and speech recognition.The three technologies are nested one within the next:<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk3MDIxMTQzOTQ1LURpZmZlcmVuY2VzIGJldHdlZW4gQUksIE1hY2hpbmUgTGVhcm5pbmcgYW5kIERlZXAgTGVhcm5pbmcgKDIpLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib" alt="Differences between AI, ML and DL">Visual AI refers to Artificial Intelligence applications that focus on enabling machines to interpret, understand, and process visual information, such as images and videos. Visual AI technologies employ advanced Machine Learning algorithms, Deep Learning models, and computer vision techniques to analyze and extract meaningful insights from visual data.AI-Powered Robot AutonomySevensense uses Visual AI to allow robots to identify, understand, and act on imagery and visual data -captured by cameras- just like humans so that they can navigate autonomously.By processing camera images with AI, the process known as Visual Simultaneous Localization and Mapping (SLAM) builds a rich 3D map of the environment and precisely localizes within it.Sevensense’s Visual SLAM technology comprises proprietary and lightweight AI sensor fusion algorithms to enable robots to precisely estimate their position. This is achieved by extracting distinctive elements from the images, for example, a corner of a window, and using these to build a representative yet compact 3D model of the environment.&nbsp;To deliver robust localization under all conditions, including lighting and environmental changes, Sevensense employs AI -more concretely Deep Convolutional Neural Networks- for feature extraction. Incorporating AI for this purpose makes it possible to overcome a limiting constraint of legacy SLAM technology, which until now could not cope with significant illumination and other environmental changes, leading to failures during autonomous navigation in difficult settings.Sevensense also uses machine learning to maintain the visual maps of dynamic spaces to ensure robust localization and reusability of the initial map of the environment. The solution known as Lifelong Visual SLAM is basically a multi-session map that uses the initial map as the basis and later on, the changes in the environment are added to it to enrich it. All of this, without the need to run Visual SLAM from scratch on each session, increases operational efficiency.The Lifelong Visual SLAM solution enables building and maintaining always-up-to-date maps. These maps incorporate data from multiple conditions of the environment, such as during cloudy- and sunny weather, bright and dark lighting situations, and changes in the elements present in the area, e.g. changes in factory floor layouts, or simply in the warehouse or shop inventory. All of these changes are detected automatically and incorporated into the lifelong map.This allows the maps to be used for visual positioning in environments that change considerably over time.To enable autonomous navigation even in complex and ambiguous environments, a Convolutional Neural Network is employed to perform Semantic Scene Understanding making it possible to detect, classify, determine the position, -and when applicable, predict the movement of obstacles e.g. people and vehicles, and other objects of interest such as pallets, trolleys, floor markings, docking or charging stations.After the robots have gained insights into their environment and the expected behavior of the elements around them, it is time for them to plan their next move. Reinforcement Learning makes it possible for robots to adapt their navigation behavior -stopping, slowing down, or overtaking obstacles, based on real-time perception, allowing them to interact with the environment, for example, docking, picking, or charging, while safely sharing the floor with moving elements like people and other vehicles.All of these processes happen on the Edge, on the multi-camera Alphasense Edge AI device -a GPU-accelerated compute unit- embedded in the robot. By bringing computational power closer to the data source, Sevensense minimizes latency challenges -enabling nearly simultaneous, real-time perception and decision-making- and addresses security and privacy issues typically present in solutions purely based on cloud computing, where visual data needs to be recorded and transmitted to remote servers for processing.&nbsp;AI-Powered Continuous Performance ImprovementTo exceed the ever-increasing performance requirements of the market, both in terms of accuracy and robustness, Sevensense leverages ongoing cutting-edge developments in the fields of AI, computer vision, and robotics.&nbsp;Sevensense uses AI models to enrich its simulation, training, and testing infrastructure.Combining Generative AI and Digital Twins, Sevensense is able to create the most challenging training and testing environments to continuously validate its software stack.Visual AI AnalyticsThe visual data captured by the cameras, apart from providing inputs for best-in-class localization and autonomy performance, also unlocks the potential for providing ground-breaking additional services.The rich visual data captured by the cameras allows Sevensense to offer further Visual AI analytic services focused on providing organized data and insights -beyond Visual SLAM- such as asset tracking, stocktaking, surveillance, and inspection, as an additional safety or reporting data layer.

Visual AI enables autonomous navigation and an array of object-recognition-based applications

Its impact on Visual SLAM and beyond

AI in Robotics Perception and Autonomous Navigation

Building robots is hard.Even if you start with a manufactured platform for locomotion, the work ahead can be challenging and time-consuming. How many sensors will require a power supply and data routing? What EDGE processing is needed? How will a remote operator interface with the machine? What coding will allow everything to work in unison and ensure the best data and performance possible? How will data be transmitted or stored?That’s the hard stuff, which inevitably requires a fair bit of time and effort. &nbsp;Typical robotics development is lengthy, it can take anywhere from 3 to 12 months from idea to a functional robot rolling around. The more complex the robotic solution, the more time it takes. <a href="https://indrorobotics.ca/commander/" target="_blank" rel="noopener noreferrer">InDro Commander</a> can reduce this to 1 to 3 months, depending on the complexity, plus InDro Robotics provides clients with software tutorials and one-on-one support.It’s that hurdle – one faced by pretty much everyone in the robotics R&amp;D world – that led to the creation of InDro Commander.<h2><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk1Njc2MjEwMDg5LUNvbW1hbmRlciBvbiBCdW5rZXIuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 766px;" class="fr-fic fr-dib"></h2><h2>What Does InDro Commander Do?</h2><a href="https://indrorobotics.ca/commander/" target="_blank" rel="noopener noreferrer">InDro Commander</a> is a platform-agnostic module that can bolt on to pretty much any means of locomotion. In the photo above, it’s the box mounted on top of the AgileX bunker (just above the InDro logo). &nbsp;Commander is a single box with critical software and hardware designed to simplify payload integration and enable turn-key tele-operations. &nbsp;Whether you’re adding LiDAR, thermal sensors, RTK, Pan-Tilt-Zoom cameras – or pretty much any other kind of sensor – Commander takes the pain out of integration. &nbsp;With its modular system, reduced development time, software tutorials and one-on-one support, clients can get their robot solution to market faster and with less hassle.The module offers multiple USB inputs for sensors, allowing developers to decide on a mounting location and then simply plug them in. A powerful Jetson EDGE computer handles onboard compute functions. The complete Robot Operating System software libraries (ROS1 and ROS2) are bundled in, allowing developers to quickly access the code needed for various sensors and functions.“Our engineering team came up with the concept of the InDro Commander after integrating and customizing our own robots,” says Philip Reece, CEO of InDro Robotics. “We realized there were hurdles common to all of them – so we designed and produced a solution. Commander vastly simplifies turning a platform into a fully functioning robot.”Account Executive, Luke Corbeth takes it further:“The Commander serves as a “brain-box” for any UGV,” he says. “It safely houses the compute, connectivity, cameras, sensors and other hardware in an IP54 enclosure.”It also comes in several options, depending on the client’s requirements.“There are three ‘standard versions’ which are bundles to either be Compute Ready, Teleoperations Ready or Autonomy Ready,” adds Corbeth.“I’ve realized over time that the value of Commander is our ability to customize it to include, or more importantly, not include specific components depending on the needs of the project and what the client already has available. In reality, most Commanders I sell include some, but not usually all, of what’s in the Commander Navigate. We’re also able to customize to specific needs or payloads.”Below: Commander comes in multiple configurations<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk1Njc0MDk1MDA2LVNjcmVlbnNob3QtMjAyMy0wOS0xOS1hdC0yLjMzLjQ2LVBNLnBuZy53ZWJwIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 766px;" class="fr-fic fr-dib"><h2>Commander Does The Work</h2>With InDro Commander, developers can spend more time on their actual project or research – and far less time on the build.“For end-users wanting a fully customized robot, Commander saves a huge amount of time and hassle,” says InDro Engineering Lead, Arron Griffiths. “Customers using this module see immediate benefits for sensor integration, and the web-based console for remote operations provides streaming, real-time data. Commander also supports wireless charging, which is a huge bonus for remote operations.”Commander serves as the brains for several InDro ground robots, including Sentinel. This machine was recently put through its paces over 5G in a <a href="https://indrorobotics.ca/putting-sentinel-through-its-paces-at-epri/" target="_blank">test for EPRI</a>, the Electric Power Research Institute.Below: Commander is mounted on Sentinel, a site based autonomous robot, capable of complete continuous monitoring of infrastructure, with the ability to carry out autonomous inspection rounds with thermal, RGB and zoom sensors, for recognition of change detection.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk1NjczODM5MTAwLVNlbnRpbmVsIEVQUkkuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" class="fr-fic fr-dii"><h2>5G Operations</h2>Depending on the model, Commander can also serve as a plug &amp; play device for operations over 4G or 5G networks. In fact, InDro was invited by US carrier T-Mobile to a 2022 event in Washington State to demonstrate&nbsp;the live, remote tele-operation of a Sentinel inspection robot.Using a simple Xbox controller plugged into a laptop at T-Mobile HQ in Bellevue WA, we operated a Sentinel in Ottawa – more than 4,000 kilometres away. There was no perceptible lag, and even untrained operators were able to easily control remote operations and cycle between the Pan Tilt Zoom camera, a thermal sensor, and a wide-angle camera used for situational awareness by the operator. Data from all sensors was displayed on the dashboard, with the ability for the operator to easily cycle between them.Below: T-Mobile’s John Saw, Executive Vice President, Advanced &amp; Emerging Technologies, talks about InDro Commander-enabled robots teleoperating over 5G networks&nbsp;<iframe width="640" height="360" src="https://www.youtube.com/embed/-C88iEnsh7E?&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true">&amp;lt;span class="fr-mk" style="display: none;"&amp;gt;&amp;amp;nbsp;&amp;lt;/span&amp;gt;&amp;amp;lt;span class="fr-mk" style="display: none;"&amp;amp;gt;&amp;amp;amp;nbsp;&amp;amp;lt;/span&amp;amp;gt;&amp;amp;amp;lt;span class="fr-mk" style="display: none;"&amp;amp;amp;gt;&amp;amp;amp;amp;nbsp;&amp;amp;amp;lt;/span&amp;amp;amp;gt;&amp;amp;amp;amp;lt;span class="fr-mk" style="display: none;"&amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;lt;/span&amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;lt;span class="fr-mk" style="display: none;"&amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;amp;lt;/span&amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;span class="fr-mk" style="display: none;"&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;/span&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;span class="fr-mk" style="display: none;"&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;/span&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;span class="fr-mk" style="display: none;"&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;/span&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;span class="fr-mk" style="display: none;"&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;/span&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;span class="fr-mk" style="display: none;"&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;/span&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;span class="fr-mk" style="display: none;"&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;/span&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;span class="fr-mk" style="display: none;"&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;/span&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;span class="fr-mk" style="display: none;"&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;/span&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;</iframe><h2 id="isPasted">Future-Proof</h2>Platforms change. Needs evolve. New sensors hit the market.With Commander on board, developers don’t need to start from scratch. The modular design enables end-users to seamlessly upgrade platforms down the road by simply unbolting Commander and affixing it to the new set of wheels (or treads).Any sensor, including LiDAR, can be quickly integrated with InDro Commander.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk1Njc1MDA4NjA2LVNjcmVlbnNob3QgMjAyMy0wOS0yNSBhdCA0LjQ5LjIzIFBNLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 766px;" class="fr-fic fr-dib"> You likely know the saying: “Necessity is the mother of invention.”InDro developed this product because we could see its utility – both for our own R&amp;D, and for clients. We’ve put Commander to use on multiple custom InDro robots, with many more to come. (We have even created a version of this for Enterprise drones.)“On the commercial side, our clients have really benefited from the inherent modularity that the Commander provides,” says Luke Corbeth.“Since the ‘brains’ are separate from the ‘body,’ this simplifies their ability to make the inevitable repairs or upgrades they’ll require. These clients generally care about having a high functioning robot reliably completing a repetitive task, and Commander allows us to operate and program our robots to do this.”It can also save developers money.“On the R&amp;D side, the customizable nature of the Commander means they only purchase what they don’t already have,” adds Corbeth.“For instance, many clients are fortunate enough to have some hardware already available to them whether it’s a special camera, LiDAR or a Jetson so we can support the integration of their existing systems remotely or they can send this hardware directly to us. This cuts down lead times and helps us work within our clients’ budgets as we build towards the dream robot for their project.”Still have questions or want to learn more about Commander?&nbsp;<a href="https://indrorobotics.ca/commander/" target="_blank" rel="noopener noreferrer">Click HERE.</a>

Autonomous Ground Vehicles (AGVs)

Profiles