In order to promote the development of the global embodied AI industry, the Unitree G1 robot operation data set is open sourced, adapted to a variety of open source solutions, and continuously updated:&nbsp;Open source data collection:&nbsp;<a tabindex="0" href="https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqa3VhODBhSF8tQzQtdFlpaUk2blczM2t1WFdkd3xBQ3Jtc0ttTkF6bzFrSzVTbkk5elFNaXl4ZmZyQTE5VGFfbWpESlhxVS1VRklKVXozX2JrZ2ItekNReGx3Vm5QOHY2OGxER0diUUhhTWlBdmotSU9LUV9EcWV0dC00Y3pjTWpyQXlQUTU5RmZCYjhoRl9KeXhadw&q=https%3A%2F%2Fgithub.com%2Funitreerobotics%2Favp_teleoperate&v=OTWHXTu09wE" rel="nofollow" target="_blank" data-immersive-translate-walked="dc553b62-3d51-4d74-bb21-9b5c19549e6f">https://github.com/unitreerobotics/av...</a>Open source learning algorithms:&nbsp;<a tabindex="0" href="https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqbFVfRV9DS25fNFpSMXVpaEJicUFBV29QY2NYd3xBQ3Jtc0tuejRtek9OWDF5Qk9Fc0FRNUhzWkl0SEdDSFM4UFRHLWk2MHdOUjR3LWxwN0JlUmxGRkJ5TjhlbE83S0hlZEhDY0IxQmFiOGJaM0MxRFVtdmtta3ZMMWVCOXlQMS1WR2hJa0hla0RtVHpvZWlQc3pPZw&q=https%3A%2F%2Fgithub.com%2Funitreerobotics%2Funitree_IL_lerobot&v=OTWHXTu09wE" rel="nofollow" target="_blank" data-immersive-translate-walked="dc553b62-3d51-4d74-bb21-9b5c19549e6f">https://github.com/unitreerobotics/un...</a>Open source datasets and models:&nbsp;<a tabindex="0" href="https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqbGh0clVCMHBNbkpPWTczdTFjSGtWNGFfLVowUXxBQ3Jtc0trV2VfWFBrLUVNNS1oM3JNR2ZEU0d0MnlYZXhBNDVQVjNfYkN1aGd1b3BMc3JHUGtNTnFBN0JSMi1CQUJHSnRUTzZMTWNIZnJNQ29IUmNtT2t1blZkRXJ3ZEtfZmRKYVdqUGk1OWNZSUl4d1g5Qk16MA&q=https%3A%2F%2Fhuggingface.co%2FUnitreeRobotics&v=OTWHXTu09wE" rel="nofollow" target="_blank" data-immersive-translate-walked="dc553b62-3d51-4d74-bb21-9b5c19549e6f">https://huggingface.co/UnitreeRobotics</a>

In order to promote the development of the global embodied AI industry, the Unitree G1 robot operation data set is open sourced, adapted to a variety of open source solutions, and continuously updated.

Unitree G1 Open Source Dataset

In recent years, the field of robotics has seen remarkable advancements in gripper technology that pushes the boundaries of dexterity and manipulation. Shadow Robot is at the forefront of this evolution as a pioneer in innovative robotics development. Shadow’s approach to the innovative solutions has caught the attention of industry leaders, including Google DeepMind, a leading AI research lab focused on developing artificial intelligence capable of performing a wide range of tasks. This collaboration has led to the creation of DEX-EE, a groundbreaking gripper that may revolutionize robotic manipulation and machine learning research.<h2 dir="ltr">The Challenge from Google DeepMind</h2>In 2017, Google DeepMind approached Shadow Robot with a big challenge. The DeepMind project was conducting research on generalized intelligence and the role of embodiment in learning—how physical interaction with the world plays a part in learning, especially for tasks that involve understanding and manipulating objects. The study required a robot with high sensitivity and a robust design.For their research, the DeepMind team needed a design that could reliably run long-term and perform physically demanding experiments without constraints within its workspace. The study required a robot that could strike its environment repeatedly to learn how to manipulate objects. This meant that the robot needed to be resilient, sensor-rich and provide large volumes of high-quality data, while operating for prolonged periods without interruption for repairs and without the need to restart experiments.The challenge pushed the boundaries of existing robotic capabilities because it required a solution that was both robust and highly precise. Google DeepMind approached Shadow Robot for this challenge. Shadow Robot has partnered with numerous research projects and the company’s flagship Shadow Dexterous Hand has contributed to major advancements in the field of robotics.&nbsp;Read more about the Shadow Dexterous Hand and its contributions in our <a href="https://www.wevolver.com/article/dexterous-robotic-hands-part-1-unraveling-the-history-technology-and-applications-of-dexterous-robotic-hands">three-part Shadow Dexterous Hand series</a>.<h3 dir="ltr">DEX-EE: Shadow Robot’s Innovative Solution</h3><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1730204504132-Untitled+design+%285%29.png" style="width: 100%px;" class="fr-fic fr-dib">In response to the challenge from Google DeepMind, Shadow Robot developed DEX-EE, a groundbreaking dexterous robot hand, as well as a major advancement in the field of machine learning robots. DEX-EE is designed to perform with dynamic and controlled motion in a robust and reliable package. The hand enables long-running experiments without interruptions due to hardware failure.One of DEX-EE’s most innovative features is its high-speed network sensors, which provide rich data including position, force, and inertial measurement. Each finger has hundreds of channels of tactile sensing, and Shadow’s advanced fingertip sensors have hundreds of taxels (short for tactile pixel—the individual sensing elements that make up a tactile sensor), each with a massive dynamic range. With this wealth of sensory data, researchers can gather huge amounts of data from dexterous robotic tasks.DEX-EE was developed with an intensive focus on reliability and robustness. Through iterative design and testing over five years, Shadow created a new dexterous hand that could withstand the physical demands of Google DeepMind’s research. As a result of this focused development, the DEX-EE is a functional, reliable, and data-rich research tool. DEX-EE’s capabilities position it as an ideal hardware platform for dexterous robotic research into cognition and manipulation.<h2 dir="ltr">DEX-EE Technical Specifications and Advancements</h2>DEX-EE is a significant leap in dexterous robotic hand technology. The robot has high-bandwidth torque and position control loops, which enable delicate and precise fingertip dexterity, while torque and inertial measurement throughout the hand make it sensitive to interactions with its environment.The hand also features tactile fingertip sensors based on stereo-camera technology, which gives the hand abundant 3D data about the environment in a robust package. Additional, multi-taxel, 3-DOF sensors on the middle and proximal fingers provide crucial information during grasping and manipulation tasks.Compared to earlier models of the Shadow Hand, DEX-EE offers greater reliability and repairability. It was designed for easy maintenance with minimal training, incorporates fail-safes and a shutdown routing that reduces instances of failure and downtime. The hand’s advanced sensor networks provide rich data, including position, force, and inertial measurement, with hundreds of channels of tactile sensing per finger. Shadow’s innovative fingertip sensors have hundreds of taxels each, with a dynamic range that significantly surpasses the capabilities of other dexterous hands.DEX-EE is also integrated with ROS (Robot Operating System), making it versatile as a research tool. This combination of robustness, advanced sensing capabilities, and ease of integration positions DEX-EE as an ideal platform for dexterous manipulation research that is capable of withstanding the demands of long-term machine learning experiments.<h2 dir="ltr" id="isPasted">Applications in Machine Learning and AI Research</h2><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1730197463018-shadowrobot-grip.png" style="width: 100%px;" class="fr-fic fr-dib">Image: DEX-EE performs machine learning experiments with high sensitivity and robustness | Source:&nbsp;<a href="https://www.shadowrobot.com/dex-ee/" target="_blank" rel="noopener noreferrer">shadowrobot.com</a>DEX-EE represents the next major advancement in dexterous manipulation research because it provides a robust platform for complex machine learning experiments like Google DeepMind. The advanced sensor networks and tactile capabilities of DEX-EE enable researchers to gather enormous amounts of data during experiments and push the boundaries of what’s possible in robotic learning.Google DeepMind has leveraged DEX-EE’s capabilities in their latest AI system DemoStart, which helps robots learn to perform complex tasks that require dexterous movement. With these systems, Google DeepMind has achieved remarkable results, like inserting a plug into a socket, tightening a bolt, and other tasks that were previously challenging for robots to master.DEX-EE’s durability allows for long-running experiments without interruptions, making it ideal for reinforcement learning protocols that involve repeated physical interactions with the environment. This has already helped researchers explore new frontiers in embodied AI, investigating how physical interactions contribute to learning and the development of generalized intelligence.The hand’s integration with ROS further enhances its versatility as a research tool because it allows for easier implementation of new and existing learning algorithms and control strategies. With this combination of robustness, advanced sensing, and flexibility, DEX-EE is positioned to be a transformative platform for exploring the future of dexterous manipulation in AI and robotics.<h2 dir="ltr">Shadow Robot's Unique Approach to Robotics Development</h2>Shadow Robot continues to distinguish itself as a robotics company through problem-solving approaches and focus on custom solutions. Rather than offering off-the-shelf products that don’t meet a particular need, Shadow engages deeply with clients to understand their specific issues and develop tailored solutions.This collaborative approach has been exemplified in the development of DEX-EE, where Shadow worked closely with Google DeepMind over five years. Shadow’s willingness to rethink fundamental design principles demonstrates their commitment to innovation and their ability to work at the cutting edge of technology.Shadow has positioned themselves as a research and hardware company that is capable of tackling any robotics challenge. They have established a unique niche in the industry, bridging academic research and practical, real-world applications.<h2 dir="ltr">Leading Innovation in Custom Robotics Solutions</h2>Shadow Robot stands out as a leader in collaborative robotics and demonstrates a commitment to addressing practical needs through innovative design and partnership. Shadow’s work on DEX-EE, developed alongside Google DeepMind, emphasizes the company’s ability to meet the changing demands of cutting-edge research. Shadow continues to push the boundaries of dexterity and manipulation by fostering collaboration and by focusing on tailored solutions that meet practical needs. In the evolving field of robotics, Shadow’s dedication to understanding and responding to the unique challenges positions the company as an important player in robotics research.<blockquote>DEX-EE marks a significant milestone for Shadow Robot. While designed for our core research market, it embodies our vision of reaching beyond this to real-world application research. We remain committed to our fundamental research customers, but DEX-EE represents our expanding horizons. It's a step towards a future where our technology not only advances scientific understanding but also addresses practical challenges across industries, allowing us to make a broader impact while continuing to innovate within the research community.</blockquote>Gavin Cassidy, Head of Product Development at Shadow Robot.For more information about Shadow’s collaborative problem solving and innovative robotics, check out the <a href="https://www.shadowrobot.com/">Shadow Robot website</a>. The <a href="https://www.wevolver.com/article/dexterous-robotic-hands-part-1-unraveling-the-history-technology-and-applications-of-dexterous-robotic-hands">Shadow Robot Dexterous Hand series</a> also provides an in-depth look at Shadow’s commitment to advancing the field of robotics.<h3 dir="ltr">References</h3><ol><li>&nbsp;Shadow Robot. “Collaborative Working: A Case Study”. June 13, 2024. Accessed from <a href="https://www.shadowrobot.com/blog/collaboration-case-study/">https://www.shadowrobot.com/blog/collaboration-case-study/</a>&nbsp;</li><li>&nbsp;Shadow Robot. “DEX-EE Overview” The most robust dexterous robot hand on the market”. June 6, 2024. Accessed from <a href="https://www.shadowrobot.com/blog/shadow-robot-hand-overview/">https://www.shadowrobot.com/blog/shadow-robot-hand-overview/</a>&nbsp;</li><li>&nbsp;Shadow Robot. “Introducing DEX-EE the most robust dexterous robot hand on the market’. Accessed from <a href="https://www.shadowrobot.com/dex-ee/">https://www.shadowrobot.com/dex-ee/</a>&nbsp;</li><li>&nbsp;Shadow Robot. “DEX-EE Overview: The most dexterous robot hand on the market”. <a href="https://www.shadowrobot.com/blog/shadow-robot-hand-overview/">https://www.shadowrobot.com/blog/shadow-robot-hand-overview/</a>&nbsp;</li><li>&nbsp;Google DeepMind. “Our latest advances in robot dexterity”. 12 September 2024. Accessed from <a href="https://deepmind.google/discover/blog/advances-in-robot-dexterity/">https://deepmind.google/discover/blog/advances-in-robot-dexterity/</a>&nbsp;</li></ol>

Discover how Shadow Robot’s DEX-EE gripper is advancing robotic dexterity with AI-driven precision.

Gripping the Future: Shadow Robot's DEX-EE and the Next Generation of Dexterous Manipulation

<section id="isPasted">Texas A&amp;M University is joining a new National Science Foundation (NSF) Engineering Research Center (ERC), led by Northwestern University, seeking to develop robots capable of enhancing human labor.&nbsp;A central goal of the Human AugmentatioN via Dexterity (HAND) center is to make robotic assistance accessible and applicable to a wide range of physical actions through an engineered system of dexterous robotic hands, AI-powered fine motor skills, human interface, as well as developing the workforce and training for the future.The five-year, $26 million grant also includes Carnegie-Mellon University and Florida A&amp;M University, with faculty support from Syracuse University, the University of Wisconsin-Madison and the Massachusetts Institute of Technology. The center’s NSF grant has the potential to be renewed for another $26 million for an additional five years. Founded in 1985, the NSF ERC program supports U.S. universities conducting convergent research, education and technology translation aimed at creating substantial societal impacts.&nbsp;“We are excited to be a part of this groundbreaking initiative that will redefine the future of work and push the boundaries of technology to make a tangible impact,” said Dr. Robert H. Bishop, vice chancellor and dean for Texas A&amp;M Engineering. “At Texas A&amp;M, we believe in harnessing innovation to solve pressing problems and improve lives.”</section><blockquote>We are excited to be a part of this groundbreaking initiative that will redefine the future of work and push the boundaries of technology to make a tangible impact. At Texas A&amp;M, we believe in harnessing innovation to solve pressing problems and improve lives.<cite>- Dr. Robert H. Bishop</cite></blockquote><section>Dr. Cynthia Hipwell, Oscar S. Wyatt, Jr. ’45 Chair II Professor in the J. Mike Walker ’66 Department of Mechanical Engineering, will serve as a deputy director for the HAND center.&nbsp;Hipwell said that current robotic tools typically require specialized expertise and an expensive integration process and thus are limited to high-volume, highly repeatable operations. The center’s approach in creating robotics that adaptably perform many different tasks and are easy for workers to use could provide affordable support to address labor shortages in manufacturing and caregiving, as well as food processing, handling precious or dangerous materials and more.“Our vision is that it will be like the transition from mainframe computers used by specialized programmers to the arrival of the Apple Macintosh or PC [personal computer] as a tool that everybody could use,” said Hipwell. “We have many small and medium enterprises around our country that haven't been able to benefit from automation technology. We want to create something that workers can use as a tool to complete their work. That's really our vision — democratizing access to robotics.”To achieve the goal of providing a practical and easily accessible tool, researchers have divided their work into three main areas:<ul><li>Hands: sensing, actuation, and design</li><li>Intelligent dexterity: simulation, representation, and control</li><li>Human interface: multimodal interface, no-code programming, and social/legal/industrial studies</li></ul>The center will also develop workforce training materials and an intuitive training interface enabling human operators of all education levels to teach the robots how to perform their necessary tasks.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1725498968571-News-COE-ActualRobotHands-23August2024.jpg" style="width: 100%px;" class="fr-fic fr-dib">An early prototype of robotic hands soldering. | Image:&nbsp;<cite id="isPasted">Courtesy of Dr. Cynthia Hipwell</cite>“The idea is that these tools could be something that any of us could use,” said Hipwell. “This could be something that could help a worker with physical work, or it could help someone stay in their home longer because they have physical assistance with some of their tasks. It could even be brought into areas where the work is dangerous and offer a way to help someone do physical work less dangerously.”The HAND center brings together experts in various fields, including materials, manufacturing, manipulation, soft robotics, artificial intelligence, machine perception, modeling, haptics, human-robot interaction, participatory design and research, team science, education, law and the social sciences.“To make a breakthrough in this area, you can't just move one aspect of it forward — just actuators or just control algorithms or just interface mechanisms. I think making a breakthrough requires this coordinated system approach,” said Hipwell. “We have the opportunity with this system-focused effort to make a much larger impact than we would as individual researchers working alone.”<hr>In addition to Hipwell, several Texas A&amp;M researchers will contribute to the center’s focus areas. <a href="https://cte.tamu.edu/who-we-are/our-team/debra-fowler" rel="noopener" target="_blank">Dr. Debra Fowler</a>, executive director of the Center for Teaching Excellence, will lead overall workforce development programs for the center, leading the creation of programs of study for future leaders in the field and programs for technicians, workers, and K-12 and community college outreach. <a href="https://engineering.tamu.edu/mechanical/profiles/ambrose-robert.html" rel="noopener" target="_blank">Dr. Robert Ambrose</a> will lead the High Consequence Materials Handling Testbed, which will help the center partner with collaborators such as NASA and Los Alamos National Laboratory on specific applications of HAND technology. Drs. <a href="https://engineering.tamu.edu/mechanical/profiles/friesen-rebecca.html" rel="noopener" target="_blank">Rebecca Friesen</a> and <a href="https://engineering.tamu.edu/biomedical/profiles/ware-taylor.html" rel="noopener" target="_blank">Taylor Ware</a> will contribute to the human interface and hands research areas, and<a href="https://engineering.tamu.edu/etid/profiles/gharib-mohamed.html" rel="noopener" target="_blank">&nbsp;Dr. Mohamed Gharib</a> will lead unique K-12 and community college outreach programs.</section>

Image: Rachel Barton/Texas A&M Engineering

Texas A&M University joins a Northwestern University-led Engineering Research Center to bring highly skilled, affordable robotic assistance to the workforce and beyond.

Researchers Team Up to Advance Robotic Dexterity

In this episode, we discuss how MIT researchers are making great strides in developing better robotic hands by focusing on an often overlooked component: the palm.

Podcast: Talk to The Hand: Meet The Robots With a Human Touch

The research team, from the University of Cambridge, used machine learning algorithms to teach a robotic sensor to quickly slide over lines of braille text. The robot was able to read the braille at 315 words per minute at close to 90% accuracy.Although the robot braille reader was not developed as an assistive technology, the researchers say the high sensitivity required to read braille makes it an ideal test in the development of robot hands or prosthetics with comparable sensitivity to human fingertips. The&nbsp;<a href="https://ieeexplore.ieee.org/document/10410896" target="_blank">results</a> are reported in the journal&nbsp;IEEE Robotics and Automation Letters.Human fingertips are remarkably sensitive and help us gather information about the world around us. Our fingertips can detect tiny changes in the texture of a material or help us know how much force to use when grasping an object: for example, picking up an egg without breaking it or a bowling ball without dropping it.Reproducing that level of sensitivity in a robotic hand, in an energy-efficient way, is a big engineering challenge. In <a href="https://birlab.org/" target="_blank">Professor Fumiya Iida’s lab</a> in Cambridge’s Department of Engineering, researchers are developing solutions to this and other skills that humans find easy, but robots find difficult.<iframe width="640" height="360" src="https://www.youtube.com/embed/xqtA2Z668Ic?&amp;wmode=opaque&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe>“The softness of human fingertips is one of the reasons we’re able to grip things with the right amount of pressure,” said Parth Potdar from Cambridge’s Department of Engineering and an undergraduate at Pembroke College, the paper’s first author. “For robotics, softness is a useful characteristic, but you also need lots of sensor information, and it’s tricky to have both at once, especially when dealing with flexible or deformable surfaces.”Braille is an ideal test for a robot ‘fingertip’ as reading it requires high sensitivity, since the dots in each representative letter pattern are so close together. The researchers used an off-the-shelf sensor to develop a robotic braille reader that more accurately replicates human reading behaviour.“There are existing robotic braille readers, but they only read one letter at a time, which is not how humans read,” said co-author PhD student <a href="http://www.eng.cam.ac.uk/profiles/dsh46" target="_blank">David Hardman</a>, also from the Department of Engineering. “Existing robotic braille readers work in a static way: they touch one letter pattern, read it, pull up from the surface, move over, lower onto the next letter pattern, and so on. We want something that’s more realistic and far more efficient.”The robotic sensor the researchers used has a camera in its ‘fingertip’, and reads by using a combination of the information from the camera and the sensors. “This is a hard problem for roboticists as there’s a lot of image processing that needs to be done to remove motion blur, which is time and energy-consuming,” said Parth Potdar.The team developed machine learning algorithms so the robotic reader would be able to ‘deblur’ the images before the sensor attempted to recognise the letters. They trained the algorithm on a set of sharp images of braille with fake blur applied. After the algorithm had learned to deblur the letters, they used a computer vision model to detect and classify each character.Once the algorithms were incorporated, the researchers tested their reader by sliding it quickly along rows of braille characters. The robotic braille reader could read at 315 words per minute at 87% accuracy, which is twice as fast and about as accurate as a human Braille reader.“Considering that we used fake blur the train the algorithm, it was surprising how accurate it was at reading braille,” said David Hardman. “We found a nice trade-off between speed and accuracy, which is also the case with human readers.”“Braille reading speed is a great way to measure the dynamic performance of tactile sensing systems, so our findings could be applicable beyond braille, for applications like detecting surface textures or slippage in robotic manipulation,” said Parth Potdar.In future, the researchers are hoping to scale the technology to the size of a humanoid hand or skin. The research was supported in part by the Samsung Global Research Outreach Program.<hr>Reference: Parth Potdar et al. ‘<a href="https://ieeexplore.ieee.org/document/10410896" target="_blank">High-Speed Tactile Braille Reading via Biomimetic Sliding Interactions</a>.’ IEEE Robotics and Automation Letters (2024). DOI: 10.1109/LRA.2024.3356978

Robot braille reader  Credit: Parth Potdar

Researchers have developed a robotic sensor that incorporates artificial intelligence techniques to read braille at speeds roughly double that of most human readers.

Robot trained to read braille at twice the speed of humans

Robotics has been pushing technology and innovation and inspiring ambitious research for ages, since the Greek mathematician and philosopher Archytas of Tarentum postulated the existence of a steam-powered bird in the 4th century BC. Among the various aspects of robotics, robot grasping remains one of the most challenging yet crucial capabilities in fully functional robotic systems. Grasping is not just about having a robot hold an object; it's about how the robot perceives, plans, and executes this interaction with precision and adaptability.&nbsp;This complexity makes robot grasping a fascinating area for researchers and engineers alike. The editorial "Current Challenges and Future Developments in Robot Grasping" by Antonio Morales, Beatriz León, Eris Chinellato, and Raúl Suárez, published in Frontiers in Robotics and AI, dives deep into these challenges and the strides being made in this field.&nbsp;In this article, we take a look at the significance of robotic grasping and why researchers continue to pursue this major challenge in robotics. We also discuss the research itself, the advances already being made, and the challenges outlined in the editorial by Morales et al.&nbsp;<h2 dir="ltr">The Significance of Grasping in Robotics</h2>Robot grasping is more than a mechanical action; it's a symphony of intricate processes that must work in harmony. When a robot performs a grasp, it &nbsp;must first understand &nbsp;the object, plan the grasp, and then execute it in a way that mimics the dexterity and adaptability of the human hand. This capability has wide-ranging applications, from manufacturing and logistics to healthcare and service industries, and has the potential to transform these fields by performing precise work while keeping humans safe. However, the complexity of grasping, which involves handling diverse objects in varied environments, makes it a particularly tough nut to crack in robotics.<h3 dir="ltr">Recent Advances in Robot Grasping</h3>In recent years, the field of robot grasping has seen a significant shift towards integrating advanced computational techniques, such as deep learning, to enhance perception, planning, and control. Deep learning applications in robot grasping are not confined to just object recognition; they extend to understanding the nuances of different types of grasps, like a power grip with the whole hand and palm, or a precise grip between fingertips, as well as the dynamics of object manipulation.&nbsp;<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzA4MDgxMjgzOTcwLWhhbmRzLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib">Illustration showing some of the different types of grasps a robotic hand might make, from power grasps to precision grasps. Source: researchgate.netAnother area of advancement is in the design and development of novel grippers and robotic hands. The focus of this research is on soft mechanism principles, which aim to create more adaptable and versatile gripping solutions. For example, the use of soft materials for fingertips improves the hold a robotic hand has on an object and allows for finer manipulation of objects. These innovative grippers are designed to handle a variety of objects with different shapes, sizes, and textures, much like the human hand.<h2 dir="ltr">A hand for deep research</h2>Shadow Robot is the developer of a highly dexterous robotic hand, and is a company that works closely with researchers to improve the computational techniques related to grip and the future of robotic applications. By providing researchers with the most cutting-edge hardware, teams that use the Shadow Hand can focus on their specific goals rather than being sidelined by their own hardware development issues.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzA4MDgxOTIxNzk0LWhhbmRzIHNoLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 300px;" class="fr-fic fr-dib">Researchers at Google Brain taught the Shadow Hand how to manipulate multiple objects using minimal data. Source: shadowrobot<h2 dir="ltr" id="isPasted">Highlighted Research and Innovations: A Closer Look</h2>One of the notable aspects of the editorial "Current Challenges and Future Developments in Robot Grasping" is the involvement of Beatriz León from Shadow Robot Company, a leading firm in robotic hands and grasping technology. This association is significant because it reflects the blend of academic research and practical industry applications in the field of robotics.<h3 dir="ltr">Shadow Robot Company's Contributions</h3>Shadow Robot Company, known for its advanced robotic hand solutions, represents the forefront of applying practical, innovative solutions to the challenges of robot grasping. The integration of the company’s expertise in this research editorial underscores the essential collaboration between academia and industry in pushing the boundaries of robotic technology.<h3 dir="ltr">Innovative Approaches in Robot Grasping</h3>Overcoming the challenges of robotic grasping has the potential to result in huge benefits to both industry and the academic world. Prioritising this challenge has led to some innovative approaches to grasping in robotics research. Here are some examples.&nbsp;1. Experience-Based Robotic GraspingThe paper titled “<a href="https://www.frontiersin.org/articles/10.3389/frobt.2020.00120/full" target="_blank" rel="noopener noreferrer">DGCM-Net: Dense Geometrical Correspondence Matching Network for Incremental Experience-Based Robotic Grasping</a>,” highlights a method where robots learn from past successful grasping experiences. This approach is crucial for handling novel objects, enabling robots to adapt previous grasping strategies to new situations.2. Human Perception in Robot Grasping<a href="https://www.frontiersin.org/articles/10.3389/frobt.2021.652760/full" target="_blank" rel="noopener noreferrer">“Leveraging Human Perception in Robot Grasping and Manipulation Through Crowdsourcing and Gamiﬁcation”&nbsp;</a>showcases a unique combination of crowdsourcing and gamification. This framework aims to enhance object recognition by tapping into human intelligence, a creative solution that blurs the lines between human and machine learning.3. Heuristic-Based and Data-Driven Active VisionThe paper “<a href="https://www.frontiersin.org/articles/10.3389/frobt.2021.696587/full" target="_blank" rel="noopener noreferrer">Aiding Grasp Synthesis for Novel Objects Using Heuristic-Based and Data-Driven Active Vision Methods,</a>” offers an innovative perspective on optimising data collection for grasp synthesis. Using a two-finger parallel gripper and a depth camera, this research presents a new angle on active vision in robotics.4. Combining Depth Vision and Haptic Bayesian Exploration&nbsp;Another groundbreaking study, “<a href="https://www.frontiersin.org/articles/10.3389/frobt.2021.703869/full" target="_blank" rel="noopener noreferrer">Grasp Stability Prediction for a Dexterous Robotic Hand Combining Depth Vision and Haptic Bayesian Exploration,”</a> addresses the challenge of stable grasp selection. This research uses a multimodal approach, combining visual and tactile explorations, to improve the reliability of grasping actions.<h2 dir="ltr" id="isPasted">The Role of Shadow Robot in Advancing Grasping Technology</h2>The involvement of Shadow Robot Company, particularly through Beatriz León's contributions, brings an element of real-world application to these research findings. The company's expertise in developing advanced robotic hands plays a pivotal role in translating these academic insights into tangible, practical solutions for various industries.&nbsp;The Shadow Hand itself is innovative, and provides highly dexterous manipulation with multiple degrees of freedom and tactile sensing. The Shadow Hand can be integrated with other systems, meaning that researchers can apply artificial intelligence to its control systems to improve how it can learn and adapt to new tasks through machine learning. The Shadow Glove, a teleoperated controller for the Shadow Hand is also highly customizable and allows for the integration of custom feedback mechanisms and tactile sensor, offering even more precise control of the hand in real-world settings.<h3 dir="ltr">The Future of Robot Grasping</h3>As the field of robot grasping evolves, the synergy between academic research and industry expertise becomes increasingly vital. Companies like Shadow Robot are not just implementing these findings but are also instrumental in shaping the direction of future research. Shadow Robot’s practical experience in the field of robotic grasping provides valuable knowledge and data to the academic community by helping refine theories and models for better applicability in real-world scenarios.<h2 dir="ltr">Conclusion: Bridging Academia and Industry</h2>Robotic grasping remains a significant challenge in the field of robotics, but it is one worth pursuing. The potential for this technology to shape industries and how humans work means that researchers are constantly seeking new grasping technologies, using sensors, machine learning, and soft mechanics to reach human-like precision and dexterity.&nbsp;The editorial by Beatriz León, et al. explores current challenges and future developments in robot grasping, enriched by the involvement of <a href="https://www.shadowrobot.com/robotics-research/" target="_blank" rel="noopener noreferrer">Shadow Robot Company</a>. The article is a testament to the dynamic nature of robotics research. It highlights the importance of collaboration between academia and industry in driving innovation, ensuring that advancements in robot grasping continue to be both groundbreaking and applicable in practical settings. As we move forward, this collaborative approach will be crucial in navigating the complex landscape of robotics and automation.<h2>References</h2>Akhtar Khurshid, Abdul Ghafoor, and M. Afzaal Malik “Robotic Grasping and Fine Manipulation Using Soft Fingertip” in Advances in Mechatronics. Accessed from <a href="https://www.intechopen.com/chapters/18895" target="_blank" id="isPasted">https://www.intechopen.com/chapters/18895</a>

A glimpse into the latest research advances in robotics grasping, showcasing the ongoing evolution and intricate challenges in this field.

Navigating the Future of Robotics: Innovations and Challenges in Robot Grasping

A primary objective in the field of robotic manipulation is to equip robots with dexterity and compliance akin to human abilities, particularly for a variety of everyday tasks. Recent years have seen numerous efforts directed towards this aim. However, several challenges remain unresolved, notably in the areas of grasping and manipulating objects using robots with multiple fingers. A significant development, as presented in the paper "<a href="https://pubmed.ncbi.nlm.nih.gov/35767493/" target="_blank">Multifingered Robot Hand Compliant Manipulation Based on Vision-Based Demonstration and Adaptive Force Control,</a>" marks a significant leap in this pursuit. The research was conducted by researchers from the University of West England using the Shadow Robot, Shadow Hand.&nbsp;<h2 dir="ltr">Learning-control mechanism integrating human input</h2>The researchers present a novel approach, merging human interaction with robotic learning and control. This innovative method blends a vision-based teleoperation system with adaptive force control, enabling the conversion of image inputs into precise force commands for a robotic hand.&nbsp;For the learning aspect, the team developed a unique end-to-end neural network, specifically designed to understand the relationship between human hand positions and the joint movements of a robotic hand. During operation, a camera captures a depth image of the hand, and this network, referred to as "TeachNet," calculates the optimal joint angles for the robot based on this image.&nbsp;To address the dynamic challenges of hand grasping and manipulation, the team engineered an adaptive force control strategy. This strategy is capable of predicting subsequent control commands based on the desired joint angles and the current state of the robot. The foundation of their force control system is a computational model inspired by human motor learning principles. The controller adjusts control variables like impedance and feedforward terms in real-time, combining them to generate force and torque commands that are then relayed to the robot hand in its joint space.The research team's work stands out due to several key innovations and contributions. First, they developed a human-guided, markerless, vision-based teleoperation system specifically for robotic hand demonstrations. In conjunction with this system, they trained an end-to-end neural network to map the poses from a human hand to a robotic hand. Secondly, they introduced an impedance-based force control strategy, seamlessly integrated into the teleoperation system. This strategy allows for the generation of desired force control commands by analyzing the differences in current hand poses. This approach facilitates more compliant manipulation behaviors compared to the commonly used position control methods in this field.Thirdly, the researchers rigorously tested their approach in both simulated and real-world settings across various task types. It is noteworthy that their work is among the few that have successfully implemented adaptive force control in a tendon-driven, multi-fingered robotic hand, such as the Shadow Hand, for the learning of compliant skills. This achievement marks a significant advancement in the realm of robotic manipulation.<h2 dir="ltr" id="isPasted">Real world tests and applications</h2>The application of this research is not confined to the lab. Its effectiveness has been proven in both simulation and real-world scenarios using the Shadow Hand. This is not just about surpassing traditional position control methods; it's about redefining what is possible in compliant grasping and manipulation behaviors.As engineers and technologists, this research offers a glimpse into a future where robots are not just tools, but extensions of our human capabilities. The precision, adaptability, and sensitivity of the multi-fingered robot hand open new horizons in various fields, from medical robotics to industrial automation.This breakthrough is a clarion call to engineers and roboticists: the future of robotics is here, and it is more dexterous, compliant, and human-like than ever before. The marriage of human intuition and robotic precision in this research paves the way for innovations that we have only begun to imagine. The multifingered robot hand is not just a technological achievement; it's a beacon of the limitless potential of human-robot collaboration.Read the full paper at <a href="https://pubmed.ncbi.nlm.nih.gov/35767493/" target="_blank">https://pubmed.ncbi.nlm.nih.gov/35767493/</a><h2 dir="ltr" id="isPasted">Shadow Robot: Empowering cutting-edge robotic research</h2>For over twenty years, Shadow Robot has been deeply invested in research, forming the core of its mission. The company has engaged in collaborations with both the academic community and research-focused corporations, addressing practical challenges in various industry sectors. Their work has spanned the development and application of artificial intelligence, machine learning, and mobile manipulation technologies.Their research partner shop program offers universities and labs the chance to use their robots to make a significant impact. Shadow Robot supports researchers by offering discounted technology, In-depth market knowledge and industry insights, a strong network of thought leaders, smart robotics engineering (from mechanical to software and a passion for robots that solve a solution or provide a meaningful purpose.&nbsp;<a href="https://www.shadowrobot.com/contact/" target="_blank" rel="noopener noreferrer">Get in touch with Shadow Robot</a> to learn more about the ways they support the work of researchers.&nbsp;


A look at the latest research advancing robotic dexterity with human-guided learning control for multi-fingered robot hands.


Revolutionizing Robotics: The Future of Dexterous Manipulation with Multifingered Robot Hands

At Voltera, we love our test engineering interns. They bring fresh new ideas into our workspace and have a ton of fun exploring with our machines. It’s a privilege to provide mentorship, equipment, and resources to nurture brilliant young minds, and see what they dream up.&nbsp;At the beginning of his work term, Dawson Martian — a second-year undergrad studying Mechatronics, Robotics, and Automation Engineering at the University of Waterloo — came up with a project idea that would put both the <a href="http://voltera.io/v-one" target="_blank">V-One</a> and <a href="http://voltera.io/nova" target="_blank">NOVA</a> to the test: Designing and building a robot hand that would mimic the movements of an operator wearing a control glove.Dawson had been working on a different project at the time, designing a light-up glove using stretchable ink and thermoplastic polyurethane&nbsp;(TPU). “I noticed when the glove was put on and powered, the brightness of the LEDs would vary depending on whether the user’s hand was opened or closed,” he recalled. “This got me thinking about how the lights were dimming due to the resistance of the traces increasing whenever they were stretched, essentially the traces were acting as strain gauges,” he said. “From there I realized I could take advantage of this property and use this varying resistance to control a machine or program…what better way to show this unique property than by having a robotic hand mimic my movements?”So, Dawson got to work. He started by building a robotic hand with movable fingers. The hand components were 3D-printed using non-flexible filament, the joints were made with TPU, and then assembled. He ran the wire up the fingers and connected these to servos. These would operate as tendons, so the hand could move. He connected the servos to an Arduino™ with a custom breakout board — which he made using the V-One.&nbsp;For the control glove, Dawson printed strain gauges onto TPU using NOVA and heat-pressed the strain gauges onto a fabric glove using a t-shirt press. Dawson wired a second Arduino and custom breakout board to the traces on the control glove using a conductive adhesive and mounted the board to a wrist strap.&nbsp;The two boards communicate via Bluetooth® transceivers, so if all worked as it should, the board on the control glove would measure resistance in the strain gauges, then transmit movement instructions to the second board. This information would control the actuators of the robotic hand.&nbsp;Of course, nothing goes exactly&nbsp;as planned, and Dawson hit a bit of a snag when he was designing the strain gauges. But, let’s rewind for a minute.&nbsp;<h2>What is a strain gauge, anyway?</h2>“A strange gauge is essentially just a resistor whose resistance value varies with changes in strain — being stretched, bent, or compressed,” Dawson explained. “You can think of it as a stretchable pipe with water running through it. As you stretch the pipe, it becomes longer but also narrower, restricting the flow”.&nbsp;Now, apply this same principle to the traces in the control glove, which are made with stretchable conductive ink. As the traces stretch, they become longer and narrower, restricting the flow of electricity — it changes the resistance. The accompanying transmitter board interprets this change in resistance, and that information can then be used to control other devices — like the robotic hand.&nbsp;Dawson’s biggest hurdle was designing the strain gauges for the control glove — finding the optimal procedure that would allow the gauges to fully stretch without losing conductivity completely, but still have a measurable change in resistance. In the end, he found that thicker traces resulted in more consistent continuity while stretching.&nbsp; <img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1704362901155-6373badbe9c8bd5af2c5faf5_Robot+Glove_1_optimized.gif" style="width: 300px;" class="fr-fic fr-dib"> <h2 id="isPasted">Imagine the possibilities</h2>Stretchable conductive ink gives you the ability to create a strain gauge on almost any stretchable material, and that has incredible potential to drive innovation. Industries across the board, from manufacturing to healthcare to wearable technology, are on the threshold of major change, thanks to the emerging field of&nbsp;<a href="https://www.idtechex.com/en/research-article/flexible-hybrid-electronics-future-of-flexible-printed-circuit-boards/20603" target="_blank">flexible hybrid electronics</a>.&nbsp;Over the course of just&nbsp;two months, Dawson was able to successfully design and build a fully-functional robotic hand that mimics the movements of an operator wearing a control glove. Imagine what could be accomplished by seasoned researchers and engineers.When asked about the potential applications for this kind of technology, Dawson lit up. “Why stop at just a hand? Why not use this technology to have a whole arm that mimics my own? Possibly even a full robot that mirrors my every move, and the majority of that could be achieved using similar techniques that were used for just the hand,” Dawson exclaimed. “The most exciting thing for me was the realization of the true possibilities that NOVA brings to the world of stretchable and flexible electronics”.This technology could have life-altering and transformative potential in industries across the board — from biomedical applications and wearables to prosthetics. “I could see this technology being used in workplaces that present a high risk of injury, but still require the delicate handling of a human. For instance, if some work needs to be done on a machine but the area is radioactive or unsafe, the worker could remotely control this hand to perform the task, while staying in a safe location.” Dawson said. “This project could also be scaled up or down in size to meet specific requirements, for instance, you could have a small robotic hand do work inside a machine, in spaces where a human could not fit, being controlled by an operator outside the machine”.&nbsp;You can read the finer details of Dawson’s work with&nbsp;<a href="https://uploads-ssl.webflow.com/6334888a34bfac03b46aa715/6362829f808f8037c5b5e9d9_NOVA-0094%20-%20Robot%20Glove%20Case%20Study_FINAL%20(4).pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">NOVA</a> and the&nbsp;<a href="https://uploads-ssl.webflow.com/6334888a34bfac03b46aa715/636282bb3c1c464ccd273de1_VONE-0001%20-%20Robot%20Glove%20Case%20Study_FINAL%20(3).pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">V-One</a> by checking out&nbsp;<a href="https://uploads-ssl.webflow.com/6334888a34bfac03b46aa715/6554cb7c0b836d9be99ef8fa_2022_NOVA%20robot%20glove%20case%20study.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">case studies</a> on this project.

Dawson Martian — a second-year undergrad at the University of Waterloo — came up with a project idea that would put both the V-One and NOVA to the test: Designing and building a robot hand that would mimic the movements of an operator wearing a control glove.

Building a Remote Controlled Robotic Hand with Strain Gauges

The field of robotic dexterous hands is rapidly evolving with technological advancements and innovative research. These agile devices have a potent potential to change industries and day-to-day life and so are the focus of research within the robotics industry.&nbsp;However, the engineering challenges posed by dexterous robotic hands are numerous. Complexities like miniaturization, grip and sensing, accurate simulation testing as well as the application of AI and machine learning are hurdles that researchers must overcome to achieve robotic hands that are as capable as human hands.In this article, we will discuss the complexities of dexterous hand development and offer some insights into how they are being solved using examples from <a href="https://www.wevolver.com/profile/shadow.robot" target="_blank" rel="noopener noreferrer">Shadow Robot</a>, a company known for creating one of the most advanced dexterous robotic hands in the world.&nbsp;<h1 dir="ltr">Complexities of Dexterous Robotic Hands</h1>Key areas of robotics development like miniaturization, sensing, and control are particularly complex when integrated with dexterous hands. Miniaturization is a challenge because dexterous hands must be capable of human-like precision and control, which is difficult to achieve in a compact space. Sensing is also a challenge because dexterous hands require a high degree of sensitivity in order to detect and respond to elements in their environment. Control poses yet another challenge, because dexterous hands require sophisticated algorithms and engineering to coordinate the movements of multiple fingers and joints[1].Addressing these challenges is important for broader adoption of dexterous hands in both research and industry.&nbsp;Improving sensing and grasping methods opens new applications for robotic manipulation. When combined with AI and machine learning, dexterous robotic systems have the potential to apply human-like adaptation and movement to tasks that are repetitive, hazardous, or that require more speed or precision than a human can provide[2].<h2 dir="ltr">Multi-force grip and sensing</h2><h3 dir="ltr">The challenge</h3>Dexterous robotic hands must be able to perform intricate tasks and manipulate different types of objects in variable environments. This means that dexterous hands need complex sensor and control systems and must incorporate multi-force gripping that can adapt according to the object and the task. Human grip force is controlled by biological systems, and to replicate this in robotic hands, researchers must develop sensors and control systems that can accurately measure and respond to features of the object and environment, like pressure, interaction forces, and vibrations[3].<h3 dir="ltr">The solution</h3>Sensing capabilities play an important role in dexterous robotic hands. Visual and tactile feedback enables robots to quantify physical properties and is used increasingly in robotic systems. The growing availability of affordable tactile sensors has encouraged exploration of real-world tactile applications, such as the field of agriculture, where grip force is critical in applications like automated harvesting[4].Researchers have developed innovative sensors and control systems to enable dexterous manipulation of objects by robotic hands. Examples include:<ul><li dir="ltr">MIT’s GelSight Svelte, a finger-shaped camera-based sensor that provides high-resolution tactile sensing over a large area, which enables the hand to perform different types of grasps, including pinch grasps. The sensor has a flexible backbone and uses mirrors to reflect light into a camera, which can see along the length of each finger.[5]</li><li dir="ltr">The ILDA (integrated linkage-driven dexterous anthropomorphic robotic hand), developed by researchers at Ajou University, with six-axis force and torque sensors that can measure the various forces acting on the robotic fingertips. Researchers integrated algorithms that can determine the contact force parameters, including the contact, location, and the triaxial force applied to the fingertips[6].</li><li dir="ltr">Shadow Robot’s Dexterous Hand, which has over 100 sensors that operate at up to 1 KHz, which give the hand multiple tactile sensing options and fast response. The hand can be integrated with tactile fingertips, which enable it to perform multiple grasps, like pinch grasps, lateral pinch grasps, and a power grasp that uses the entire sensing area of the fingers[7].</li></ul><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzAwNzMzNjQxMzgxLWltYWdlMS5qcGVnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib">The Shadow Dexterous Hand gripping a lightbulb. Source: shadowrobot.com<h2 dir="ltr" id="isPasted">AI and machine learning</h2><h3 dir="ltr">The challenge</h3>Artificial intelligence and machine learning play an important role in the development of dexterous robotic hands. When these technologies are applied to sensor data, they can improve perception, object recognition, grasping, and manipulation capabilities. However, deploying AI and ML in robotic hands is a challenge because it can be difficult to produce enough data for effective learning[8].<h3 dir="ltr">The solution</h3>Shadow Robot has integrated AI and ML into their dexterous hand technology to manipulate physical objects and complete tasks in a way that is intuitive for the operator[9]. The company has worked with companies like OpenAI to advance research in this area. The Dexterous Hand provides abundant sensor and positional data that researchers can apply to machine learning algorithms and develop innovative control systems[10].Deep reinforcement learning has also emerged as a promising approach to control dexterous robotic hand systems. Deep reinforcement is model-agnostic, meaning that controls do not rely on a specific model of the system. To overcome the problem of the small sample size for learning data, researchers have combined deep reinforcement learning with human demonstration, which enables learning using sample sizes of just a few hours of robot learning[11].<h1 dir="ltr">The role of simulation testing to overcome challenges</h1>The development of dexterous robotic hands is a complex and challenging task that requires substantial testing and iteration to achieve successful mimicry of the human hand. However, it can quickly become costly, and sometimes hazardous, to perform field testing with physical robots. Instead, simulation testing provides data points on the capabilities of robotic systems using a computer-based simulation of the hardware[12].Simulation testing is a practical, safer, and more scalable alternative to field testing and is critical to developing human-like functionality in dexterous hands[13]. Using simulation testing, developers can evaluate and compare the performance of robotic hands while adjusting design characteristics and environmental factors. Researchers can perform stress tests on the system in less time and without damaging physical hardware or putting the safety of humans at risk[14].However, creating realistic simulations for dexterous robotic hands poses another set of challenges. One of the biggest challenges of simulation testing is to create a simulation that is realistic enough to be useful. If the simulation is unable to replicate the kind of environment that the robot will operate in and adapt to, then the data from the simulation cannot reliably be used to improve the robot[15].Another challenge of simulation testing is the lack of reproducibility and the resource costs associated with simulators[16]. Despite these challenges, simulation testing is critical to the ongoing development of dexterous robotic hands. It allows developers to test and evaluate new algorithms and models in a controlled environment before building physical systems[17].<h2 dir="ltr">Shadow Robot’s vision and simulation testing</h2>Simulation testing is a key part of how Shadow Robot is overcoming major challenges in dexterous hand development. Shadow Robot is renowned for pioneering work in advanced dexterous hand development. The company is driven by a passion to create advanced dexterous robots that solve real-world problems and inspire innovation.Shadow Robot uses several simulators to test innovations in dexterous hand technology:<ul><li dir="ltr">The NVIDIA Isaac robotics simulator, which allows robots to be trained efficiently in a simulated universe. This simulation environment provides data that is diverse enough to effectively train the hand to perform challenging tasks.</li><li dir="ltr">The MuJoCo (Multi-Joint dynamics with Contact) physics engine&nbsp;for simulating rigid-body systems and soft-body deformation. This engine provides fast and accurate simulation of how physical bodies interact, with sophisticated replication of kinematics and dynamics.</li><li dir="ltr">Gazebo, an open-source simulator with capabilities to test complex scenarios and algorithms in a 3D environment. Gazebo supports a wide range of sensors and provides real-time data for visualization and analysis.</li></ul>The company’s Dexterous Hand is the most advanced 5-fingered robotic hand in the world. It can manipulate tools and objects with unprecedented accuracy, opening new avenues for research in robotics and artificial intelligence. The Shadow Hand is designed to replicate the human hand as closely as possible and brings the field of robotics closer to overcoming the challenges of dexterous hand development. Shadow Robot’s advancements in dexterous hand technology have the potential to transform industries and daily life.<h1 dir="ltr">Conclusion</h1>The field of dexterous robotic hands is rapidly evolving. Progress in this field is driven by advancements in technology and research, as well as the potential to reshape how humans live and work. In this article, we have shown that the complexities of robot hand development are numerous, but that promising technologies are emerging to overcome these problems. Researchers have found amazing solutions to solve the challenges of simulation testing, miniaturization, gripping and sensing, as well as the integration of AI and ML.Shadow Robot’s vision encompasses the potential for these advancements with their groundbreaking work in dexterous hand technology. The company is positioned to transform industry and daily life. The journey is challenging, but the potential rewards are immense in a future where robotic hands are integrated into our daily life.For more information about Shadow Robot, check out their website at <a href="http://www.shadowrobot.com" target="_blank">www.shadowrobot.com</a>.You can also read our three-part series on <a href="https://www.wevolver.com/article/dexterous-robotic-hands-part-1-unraveling-the-history-technology-and-applications-of-dexterous-robotic-hands" target="_blank">Dexterous Robotic Hands</a>.<h2 dir="ltr">References</h2>[1] “Highly dexterous robot hands can operate in the dark — just like humans”. National Science Foundation (June 9, 2023). Accessed from <a href="https://new.nsf.gov/news/highly-dexterous-robot-hand-can-operate-dark-just" target="_blank">https://new.nsf.gov/news/highly-dexterous-robot-hand-can-operate-dark-just</a>&nbsp;[2] Shuang Li, et al. “A hand-arm teleoperation system for robotic dexterous manipulation”. Tactile Sensing, Skill Learning, and Robotic Dexterous Manipulation (2022). Accessed from <a href="https://www.sciencedirect.com/topics/computer-science/robotic-hand" target="_blank">https://www.sciencedirect.com/topics/computer-science/robotic-hand</a>&nbsp;[3] W. Mandil, et. al. “Tactile-Sensing Technologies: Trends, Challenges and Outlook in Agri-Food Manipulation”. Sensors (Basel) (2023). Accessed from <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10490130/" target="_blank">https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10490130/</a>&nbsp;[4] W. Mandil (2023).[5] A. Zewe. “Finger-shaped sensor enables more dexterous robots”. MIT News. (4 Oct. 2023). Accessed from <a href="https://news.mit.edu/2023/finger-shaped-sensor-enables-more-dexterous-robots-1004" target="_blank">https://news.mit.edu/2023/finger-shaped-sensor-enables-more-dexterous-robots-1004</a>&nbsp;[6] U. Kim et al. “Integrated linkage-driven dexterous anthropomorphic robotic hand”. Nature Communications, Vol 12. (14 Dec. 2021). Accessed from <a href="https://www.nature.com/articles/s41467-021-27261-0" target="_blank">https://www.nature.com/articles/s41467-021-27261-0</a>&nbsp;[7] Shadow Robot. Dexterous Hand Series. Accessed from <a href="https://www.shadowrobot.com/dexterous-hand-series/" target="_blank">https://www.shadowrobot.com/dexterous-hand-series/</a>&nbsp;[8] A. Rajeswaran, et. al. “Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations”. Arxiv (2018). Accessed from <a href="https://arxiv.org/abs/1709.10087" target="_blank">https://arxiv.org/abs/1709.10087</a>&nbsp;[9] Shadow Robot. Dexterous Hand Series. Accessed from <a href="https://www.shadowrobot.com/dexterous-hand-series/" target="_blank">https://www.shadowrobot.com/dexterous-hand-series/</a>&nbsp;&nbsp;[10]Shadow Robot. “The benefits of the Shadow Hand for AI and Machine Learning”. Accessed from <a href="https://www.shadowrobot.com/blog/the-benefits-of-the-shadow-hand-for-ai-and-machine-learning/" target="_blank">https://www.shadowrobot.com/blog/the-benefits-of-the-shadow-hand-for-ai-and-machine-learning/</a>&nbsp;[11] A. Rajeswaran, et. al. “Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations”. Arxiv (2018). Accessed from <a href="https://arxiv.org/abs/1709.10087" target="_blank">https://arxiv.org/abs/1709.10087</a>&nbsp;&nbsp;[12]S. Crowe. “10 challenges of using simulators for testing robots”. The Robot Report (May 25, 2020). Accessed from <a href="https://www.therobotreport.com/10-challenges-simulators-robotics-testing/" target="_blank">https://www.therobotreport.com/10-challenges-simulators-robotics-testing/</a>&nbsp;[13] S. Crowe (2020).[14] N. Elangovan, et. al. “An Accessible, Open-Source Dexterity Test: Evaluating the Grasping and Dexterous Manipulation Capabilities of Humans and Robots”. Front Robot AI (2022). Accessed from <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9081435/" target="_blank">https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9081435/</a>&nbsp;[15] S. Crowe (2020).[16] S. Crowe (2020).[17] S. Andersson. “Simulation challenges in robotic grasping”. Umeå University (Spring, 2021). Accessed from <a href="https://www.diva-portal.org/smash/get/diva2:1583322/FULLTEXT01.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">https://www.diva-portal.org/smash/get/diva2:1583322/FULLTEXT01.pdf</a>&nbsp;[18] Shadow Robot. “Simulating the Shadow Hand”. (1 Aug. 2023). Accessed from <a href="https://www.shadowrobot.com/news/press-releases/simulating-the-shadow-hand/" target="_blank">https://www.shadowrobot.com/news/press-releases/simulating-the-shadow-hand/</a>&nbsp;

Simulation is key to the efficient development of dexterous robot hands with human-like abilities. 


Navigating Technological Challenges for Dexterous Hands

3D printing is advancing rapidly, and the range of materials that can be used has expanded considerably. While the technology was previously limited to fast-curing plastics, it has now been made suitable for slow-curing plastics as well. These have decisive advantages as they have enhanced elastic properties and are more durable and robust.The use of such polymers is made possible by a new technology developed by researchers at ETH Zurich and a US start-up. As a result, researchers can now 3D print complex, more durable robots from a variety of high-quality materials in one go. This new technology also makes it easy to combine soft, elastic, and rigid materials. The researchers can also use it to create delicate structures and parts with cavities as desired.<h2>Materials that return to their original state</h2>Using the new technology, researchers at ETH Zurich have succeeded for the first time in printing a robotic hand with bones, ligaments and tendons made of different polymers in one go. “We wouldn’t have been able to make this hand with the fast-curing polyacrylates we’ve been using in 3D printing so far,” explains Thomas Buchner, a doctoral student in the group of ETH Zurich robotics professor Robert Katzschmann and first author of the study. “We’re now using slow-curing thiolene polymers. These have very good elastic properties and return to their original state much faster after bending than polyacrylates.” This makes thiolene polymers ideal for producing the elastic ligaments of the robotic hand.<iframe width="640" height="360" src="https://www.youtube.com/embed/54jg4uduIt0?&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true">&nbsp;&nbsp;</iframe>(Video: ETH Zurich / Nicole Davidson)In addition, the stiffness of thiolenes can be fine-tuned very well to meet the requirements of soft robots. “Robots made of soft materials, such as the hand we developed, have advantages over conventional robots made of metal. Because they’re soft, there is less risk of injury when they work with humans, and they are better suited to handling fragile goods,” Katzschmann explains.<h2>Scanning instead of scraping</h2>3D printers typically produce objects layer by layer: nozzles deposit a given material in viscous form at each point; a UV lamp then cures each layer immediately. Previous methods involved a device that scraped off surface irregularities after each curing step. This works only with fast-curing polyacrylates. Slow-curing polymers such as thiolenes and epoxies would gum up the scraper.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1700105394264-image.imageformat.1286.1632000253.jpg" style="width: 766px;" class="fr-fic fr-dib">The robotic hand is printed layer by layer using polymers of varying elasticity (left: schematic, right: computer graphics). (Visualisations: Buchner TJK et al., Nature 2023)To accommodate the use of slow-curing polymers, the researchers developed 3D printing further by adding a 3D laser scanner that immediately checks each printed layer for any surface irregularities. “A feedback mechanism compensates for these irregularities when printing the next layer by calculating any necessary adjustments to the amount of material to be printed in real time and with pinpoint accuracy,” explains Wojciech Matusik, a professor at the Massachusetts Institute of Technology (MIT) in the US and co-author of the study. This means that instead of smoothing out uneven layers, the new technology simply takes the unevenness into account when printing the next layer.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1700105413274-image.imageformat.1286.1443377098.jpg" style="width: 766px;" class="fr-fic fr-dib">Other examples of 3D-printing include a legged robot and metamaterials. The latter could be used to absorb vibrations. (Photographs: ETH Zurich / Thomas Buchner)Inkbit, an MIT spin-off, was responsible for developing the new printing technology. The ETH Zurich researchers developed several robotic applications and helped optimise the printing technology for use with slow-curing polymers. The researchers from Switzerland and the US have now jointly published the technology and their sample applications in the journal Nature.At ETH Zurich, Katzschmann’s group will use the technology to explore further possibilities and to design even more sophisticated structures and develop additional applications. Inkbit is planning to use the new technology to offer a 3D printing service to its customers and to sell the new printers.<hr><h2 id="isPasted">Reference</h2>Buchner TJK, Rogler S, Weirich S, Armati Y, Cangan BG, Ramos J, Twiddy S, Marini D, Weber A, Chen D, Ellson G, Jacob J, Zengerle W, Katalichenko D, Keny C, Matusik W, Katzschmann RK: Vision-Controlled Jetting for Composite Systems and Robots, Nature, 15. November 2023, doi: <a href="https://doi.org/10.1038/s41586-023-06684-3" target="_blank">external page10.1038/s41586-023-06684-3</a>

3D printed in one go: A robotic hand made of varyingly rigid and elastic polymers. (Photograph: ETH Zurich/Thomas Buchner)

For the first time, researchers have succeeded in printing a robotic hand with bones, ligaments and tendons made of different polymers using a new laser scanning technique.

Printed robots with bones, ligaments, and tendons

Autonomous Mobile Robots employed for material handling; Credits: Teradyne

This article offers an in-depth view of AMR robots, their underlying technology, and applications crucial in today's rapidly evolving technological landscape.

Understanding AMR Robots: A Comprehensive Guide

<h3 id="isPasted">Let’s explore some methods on how to achieve this.</h3>First and foremost, there exists the “Straightforward Programming” approach. This means: getting a skilled person to meticulously describe a task, to such an extent that one can write either a simple script or an intricate algorithm that can guide the robot to do the task. This method is great and highly effective when the task is well-defined and specific, something often done in industrial automation. However, the complexity can escalate rapidly, making it challenging. Thus, if the approach is successful, that’s excellent – but if not, we must ponder – what alternative strategies can we employ?Secondly, an alternative approach to consider is “Programming by Demonstration“. This method involves guiding the robot in performing the task. By equipping the robot with advanced force sensing capabilities and vision algorithms, this technique can prove to be quite effective. However, it’s crucial to note that its effectiveness is reliant on the robot’s abilities. Usually, robots are equipped with rudimentary grippers rather than adept, dexterous hands, which can limit the complexity of tasks they can perform.&nbsp;Okay, so how can we effectively record human actions for a robot to mimic?Ideally, we’d guide a human through the task, document it in detail, and then replay it to the robot. We might even consider feeding this information into a machine-learning system, enabling it to learn from our human-guided demonstration. Alternatively, at the very least, we can analyze these human activities, deconstruct them into simpler parts, and use this information to develop more efficient algorithms in the way that a song is deconstructed in a recording studio.&nbsp;OK, so how about, filming the person doing the task and using “machine vision” to track what they are doing?It’s certainly a good starting point. This approach provides valuable insights into the nuances of human behaviour – such as hand-switching moments, object-holding techniques, and tool usage – But our experience is that it’s really hard to track human movement&nbsp;using cameras. You need clear lines of sight, and you’ll have lots of trouble tracking the fine movements of the fingers and hands.&nbsp;Even the most advanced image-based tracking systems, while capable of monitoring hands with minimal occlusion, fail to consistently track all joint movements. Consequently, the resultant data tends to be somewhat inconsistent and noisy.&nbsp;Alright, what about using motion capture?Motion capture systems, commonly employed in film production and sports science, operate by attaching easily visible markers to the individual, such as passive elements like white dots or active elements like pulsed LEDs. This method certainly helps with the accuracy of tracking. However, it doesn’t entirely overcome the previously mentioned issues related to occlusion. Additionally, calibrating the optical systems to an error margin of less than a few millimetres is quite challenging – which means that if the task involves intricate movements, such as rolling a pen between fingers, it’s likely it’ll miss a lot.&nbsp;&nbsp;Okay. I need it to be accurate, what’s my best option then?For precise human movement tracking, your best bet is to use a tight-fitting movement-tracking&nbsp;<a href="https://www.shadowrobot.com/shadow-hand-glove/" target="_blank">glove</a>. This glove can incorporate a multitude of techniques to achieve such a feat, some even providing additional features to assist the operator (and some that make it more difficult for the operator). For instance, some gloves can relay a sense of weight, touch, or stiffness, all functionalities grouped under the term ‘haptics.’ The realm of haptics is broad, including diverse technologies from ultrasound devices transmitting signals to fingers, electrical skin stimulation, vibrating piezoelectric elements, to inflating air balloons, miniature motors, or linear actuators, among many others.Gloves that measure human hand movement can do it by utilising:<ul><li dir="ltr">Sensors that fit around the joints and track the specific movement of the joint</li><li dir="ltr">Mechanical linkages that go from one end of the finger to the other and track the whole movement of the finger</li><li dir="ltr">Inertial sensors (that clever combination of accelerometer, gyroscope and possibly magnetometer) - worn at various points on the hand to follow the movement</li><li dir="ltr">Wireless measuring sensors - detecting the location of the fingers relative to a source worn on the hand somewhere.</li><li dir="ltr">Cameras that look away from the hand and track the movement of the world. (I’m not sure I’ve seen this one used in anger but I’m sure it’s possible!)</li></ul> Even then several variations of these technologies exist. For instance, some sensors measure the flex of a glove, correlating it to joint movement. Other types of sensors stretch and track their own deformation to gauge distance. Occasionally, mechanical linkages are placed over individual joints for precise motion tracking.&nbsp;Each of these methods presents a unique approach to capturing and interpreting human movement<h3 id="isPasted">Our experience:</h3>From our extensive experience, we have found that accurately measuring the mechanical properties of the human hand can be challenging, as it often leads to significant inaccuracies. The placement of the measurement sensor is crucial, and unfortunately, the potential for slippage is high – gloves and similar apparatus tend to shift excessively, complicating the data collection. Relating the measurements to the actual rotation of a finger joint’s axis, especially the thumb, is another daunting task!<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk3NTM2Nzk3ODE3LWxvbmcgLSBQb2xoZW11cyAmIFNoYWRvdyAoMTkyMCB4IDEwODAgcHgpLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 766px;" class="fr-fic fr-dib">Mechanical linkages, although valuable, can quickly become cumbersome. There is also a risk that they could discomfort the operator to the point where task completion becomes impossible (too heavy, too rigid etc). Inertial sensors present a viable option; however, (once again) preventing them from drifting is generally difficult, and the precision level they offer often falls short of what we needWe have found that wireless measurement sensors, particularly those from&nbsp;<a href="https://www.shadowrobot.com/shadow-robot-polhemus/" target="_blank">Polhemus</a>, give us excellent results. These sensors entail a transmitting source worn on the palm and receivers located at the fingertips, providing us with high-quality data for hand movement tracking. However, there remains the challenge of locating the hand within the wider three-dimensional space. For this, we have found that the HTC Vive is an excellent solution and so you will often see the Vive tracker featured in our demonstrations.Now – what we want is to accurately know where to put the robot hand – which isn’t quite the same thing as knowing accurately where the human hand is. We have to take the data from the human hand and map it to the robot hand. This is a kinematic challenge – the movements of the human and the movements of the robot don’t agree entirely, and we need to convert one to the other. The best method for mapping human data to a robot hand will depend on the specific application. For example, if you need to control a robot’s hand in real-time, then direct mapping may be the best option. If you are only interested in controlling a robot’s hand for offline analysis, then inverse kinematics or hybrid mapping may be more feasible options. Needless to say, the more complex the robot the more challenging this becomes. Lucky for us complex robots are a speciality at Shadow!&nbsp;So, the human wears a set of gloves, and they control the robot to do the task. After completing a few rudimentary training tasks the operator gets the hang of the process and they are doing it pretty well. What do we do to learn from this?We can precisely monitor the instructions sent to the robot and reproduce them. This is ideal when operating in a controlled environment where all variables remain consistent – a scenario commonly found in an automation setting within a quality assurance laboratory. In the past, we employed this technique to build a functional robotic kitchen. While it allows you to execute repetitive tasks, it lacks the ability to cope with variation.We can capture and analyze data from numerous tests, assessing them for consistency. If the results are largely similar, they can be directly applied. However, if they vary a lot, we can delve into our algorithm toolbox to try and understand these discrepancies. This is where it helps to have a <a href="https://www.shadowrobot.com/artificial-intelligence-machine-learning/" target="_blank">wide range of data</a> from <a href="https://www.shadowrobot.com/dexterous-hand-series/" target="_blank">the robot</a>, from force, <a href="https://www.shadowrobot.com/shadow-robot-11-lessons-about-tactile-sensors/" target="_blank">touch</a>, and joint sensors. The question arises: Can we employ data from vision, such as cameras aimed at the work area, to discern what’s different? To be effective, this vision data needs to be synchronized with the robot sensor data, necessitating a shared framework for all data streams.We can gather data from multiple tests and use it by “Feeding it to the AI” – meaning use it to instruct a machine learning system on how to replicate similar movements from similar data.This can be done in several ways.We might train a traditional learning model on the datasets so that similar inputs generate similar outputs. Alternatively, if we have enough data, we could use a transformer model to generate movements. We can even break down the data into smaller segments and train a network to assemble these fragments into the correct sequences.Given the impressive capabilities of modern machine learning systems in generating text, images, video, and even code, this approach appears highly promising – and it’s only the beginning!So, does this sound like it could help with a problem that you face? If it does, then let’s have&nbsp;<a href="https://www.shadowrobot.com/blog/data-humans-to-robots/hand@shadowrobot.com" target="_blank">a chat to</a> see if the Shadow Robot team can help!

Humans excel at manual tasks, and ideally, robots should surpass us in these functions. We have the advantage due to our superior dexterity, and we can perform tasks that robots should, in theory, be capable of.
As we advance robotics our goal is to effectively transfer our human skills to robots.

Getting Data from Humans to Robots

This article originally appeared in <a href="https://news.stanford.edu/2023/10/03/robot-provides-personalized-room-cleanup/" target="_blank" rel="noopener noreferrer">Stanford University</a> and has been republished with permission.<hr><iframe width="640" height="360" src="https://www.youtube.com/embed/dYdi0Aek664?&amp;wmode=opaque&amp;rel=0&amp;enablejsapi=1&amp;origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-gtm-yt-inspected-8="true" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true">&nbsp;</iframe>The one-armed robot TidyBot can encounter new objects and determine – with about 85% accuracy – where they should be put away, based on the personalized preferences it’s been given previously.Whenever <a href="https://profiles.stanford.edu/shuran-song" target="_blank">Shuran Song</a> teaches robotics, she asks her students the same question: What daily tasks would you most like a robot to do?“The two requests that always come up are ‘washing dishes’ and ‘cleaning my room,’” said Song, assistant professor of electrical engineering at Stanford.So far, the first task is best left to a dishwasher. But for the second, Song and an interinstitutional team of roboticists may have a solution: <a href="https://tidybot.cs.princeton.edu/" target="_blank">TidyBot</a>, a one-armed helper that cleans according to personal preferences. The group, which includes researchers from Princeton University, The Nueva School, Columbia University, and Google, presented a paper on the robot and its performance at the International Conference on Intelligent Robots and Systems on October 2nd. An extended journal version of <a href="https://arxiv.org/abs/2305.05658" target="_blank">the paper</a> will appear soon in Autonomous Robots in the special issue on robots and large language models.“We all experience it: our home or lab or office gets messy,” said <a href="https://profiles.stanford.edu/jeannette-bohg" target="_blank">Jeannette Bohg</a>, assistant professor of computer science at Stanford and another of the paper’s authors. “Now imagine you just tell a robot, ‘In my place, this is where that thing goes, and that is where this thing goes.’ Then you leave and let the robot tidy up.”<h2>For personal pick-up, no one size fits all</h2>To navigate your space, TidyBot rolls around on powered wheels hidden beneath a box-shaped platform. A seven-jointed arm sprouting from the top ends in a two-fingered gripper for manipulating objects, from opening a drawer and gently placing a toy inside to precisely tossing an empty can into a bin. Cameras on the ceiling allow TidyBot to pinpoint objects and obstacles across a room, while others on its body provide close-up views to analyze items in front of it.But it’s TidyBot’s adaptability, not just its physical form, that could make it so useful around the house.“Household organization is so personal,” Bohg said. Recognizing this, the group worked to make TidyBot customizable to different preferences.TidyBot uses a large language model (a type of AI similar to the one used in ChatGPT) trained on massive amounts of information from the internet to identify broad categories of objects like clothes, food, and toys. This means you can give TidyBot just a few examples of where certain belongings go and it will understand to clean up similar items in the same ways – even if it’s never seen those items before.Suppose, for instance, that you want your kid’s loose toys back on their shelf. “You could say, ‘I want the teddy bear on the shelf,’ and TidyBot is able to generalize that concept to other toys in your home,” Song explained. TidyBot also recognizes qualities like color, allowing it to do things like sort light and dark laundry. In real-world tests, the robot can properly put away 85% of objects.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk2NTA2ODYwNzEyLXRpZHlib3Rfc3JfMi53ZWJwIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 766px;" class="fr-fic fr-dib">TidyBot is trained within a specific space so it can learn how different types of objects are put away in that area. For example, the researchers were able to make it put clothes on the couch. (Image credit: Kurt Hickman)<h2 id="isPasted">An endearingly imperfect helper</h2>That 85% success rate means TidyBot still makes mistakes, of course – sometimes to comical effect. Bohg recalls a visit from a former adviser during a TidyBot demo.“He was visible in TidyBot’s ‘playing field,’ and TidyBot tried to pick him up,” she laughed.Luckily, it didn’t succeed: the team designed the robot mainly for handheld-sized objects or soft, deformable items like clothing. TidyBot also struggles with difficult-to-grasp shapes, like a credit card lying flat on a table. Bohg hopes to improve robots’ abilities to replicate human strategies in scenarios like this – such as dragging the card to the table’s edge to pick it up. And she’d like to investigate whether multiple TidyBots could cooperate to accomplish bigger tasks like moving furniture or making a bed.Like most humans, TidyBot also has more trouble with new spaces than with new objects: an unfamiliar kind of drawer in a strange location might trip it up, just as it might take time to learn your way around a new kitchen.“The challenge is having TidyBot work in all sorts of environments – in your home and my home, in this lab and the other lab,” Bohg said.Getting TidyBot to generalize across environments as well as objects will take time and effort. But as large language models and vision models improve, so will TidyBot’s ability to process information and make correct decisions in new scenarios. In theory, Song said, there may be no limit to how many combinations of items and environments TidyBot could eventually work with. As long as an object exists on the internet, TidyBot could learn to recognize and categorize it.“Maybe in the long term we can think about a more general-purpose robot that’s also able to cook my meal, make my bed, do my laundry,” Song mused. But for now, she said, “If I have a robot that’s able to reliably put my things away, that’s a pretty good goal to achieve already.”

TidyBot is trained within a specific space so it can learn how different types of objects are put away in that area. For example, the researchers were able to make it put clothes on the couch. (Image credit: Kurt Hickman)

The one-armed “TidyBot” needs only a few starter examples to figure out how you like your stuff put away.

This robot learns to clean your space just the way you like it

Part 2 of our dexterous robotic hand series. 

Dexterous Robotic Hands Part 2: How the Shadow Dexterous Hand is Revolutionizing Robotics

There are many robot arms on the market, and you&rsquo;re probably familiar with a few. Your modern cobot ranges in cost from &euro;25k to &euro;200k &ndash; and that is for something with 6 or 7 degrees of freedom (DoF). So you might say &ldquo;&euro;4k-&euro;14k per DoF&rdquo; and use that as a &ldquo;rule of thumb&rdquo; to price off-the-shelf commercial robots.There are plenty of people with very low-cost robot hands and arms out there &ndash; if you&rsquo;re a hobbyist or you love to prototype, then that&rsquo;s where you should go! A quick look online finds 3d printed structures for a few tens or hundreds of Euro to print, although you still have to supply valves and compressors and electronics and maybe sensing. A fun project for a hardware developer! Then there are some cool hobbyist complete systems &ndash; I can find a 6DoF robot arm on Amazon for &euro;120 today, which would be great for teaching beginners the fundamentals of robot operation.But if you&rsquo;re planning to do work with the robot, then you want a complete system. And you want something that will lift a few kilos, has the sensors and motors to move with precision and speed across a decent set of kinematics, has safety circuits and controls built in, and supports the &ldquo;mechanical engineering&rdquo; interfaces allowing you to build it into your lab properly. Suddenly you&rsquo;re back to &ldquo;it&rsquo;s going to be a 5 figure sum&rdquo;. And there are good reasons for that &ndash; you&rsquo;re getting something with mechanical strength, highly engineered sensors, motors and gearboxes, and sophisticated control systems to make it work.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjk0NzcwMTg4NzE4LVVudGl0bGVkICgyOTcgw5cgMjEwbW0pICgyMDAgw5cgMjAwbW0pICgzMDAgw5cgMTAwbW0pLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 766px;" class="fr-fic fr-dib"> And then you&rsquo;re going to want documentation, software interfaces, and support when it doesn&rsquo;t do what you want.Don&rsquo;t get me wrong &ndash; that isn&rsquo;t because you bought something that wasn&rsquo;t up to the job. Most support tickets in robotics come with one of two sentences at the start: &ldquo;I was doing &hellip; and it broke&rdquo; or &ldquo;How can I get it to do &hellip;&rdquo;. And we&rsquo;ve seen enough robot arms slam things into tables to know that breaking things is part of the game here &ndash; honestly, if you&rsquo;re not breaking the odd robot you&rsquo;re probably not doing research at the frontiers!So that&rsquo;s a real, commercial robot arm. Assume &euro;5k per DoF for a generic cobot, and maybe &euro;8-10k if you&rsquo;re getting something really amazing. Even the dinky 6DoF arm we use in one of our production processes was &euro;17k for 6DoF (to be fair to it, the precision is really impressive, but it&rsquo;s still shoebox-size).However, <a href="https://www.shadowrobot.com/" target="_blank" rel="noopener noreferrer">Shadow Robot</a> isn&#39;t in the robot arm business. We make what the marketing people politely call &ldquo;end of arm tooling&rdquo;. This space ranges from welding guns to lasers, from two jaw grippers to vacuum suckers, and right the way over there &mdash;&gt; robot hands. And according to the good people at the International Federation of Robotics, adding EoAT and the other parts needed to make it work can triple the price of a robot arm!A simple gripper that opens and closes without any fine movement control, without force sensing and without position sensing might well be included in the price of the cobot. Lots of companies do that and it works well for their customers. But we&rsquo;re here for data. We want precise control. We want to be able to move each fingertip independently and position it accurately in space with respect to the other fingertips.In short, we want a robot hand where each finger behaves like a small robot arm &ndash; and we want all that complexity and performance in something 10x smaller! What would that cost? Well, &euro;5k per DoF gets us to &euro;100k for 20DoF &hellip; and the price for the full Shadow Hand Plus today is &hellip; &euro;110k including shipping, installation, training and support!Of course, we can make simpler Hands with fewer degrees of freedom &ndash; and fewer fingers &nbsp;&ndash; and these go right down to a 2-finger 7DoF system which is more like &euro;45k including shipping, installation, training and support. You can see the full range <a href="https://www.shadowrobot.com/dexterous-hand-series/" target="_blank">here</a>, and if you want to talk to us about getting one, then <a href="https://www.shadowrobot.com/contact/" target="_blank" rel="noopener noreferrer">get in touch</a>!&nbsp;

Spending your hard-earned budget on a robot is a step you don’t want to take lightly, Any worthwhile robot is likely to represent a significant piece of capital equipment.

So, let’s take a look at some hardware and what you might expect to pay for.

How Much does a Robot Hand Cost?

Soft Robotics Podcast-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

Soft Robotics Podcast-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Soft Robotics Podcast-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Soft Robotics Podcast-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

Soft Robotics Podcast-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

NEURA Robotics-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

NEURA Robotics-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

NEURA Robotics-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

NEURA Robotics-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

NEURA Robotics-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Rozum Robotics-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

Rozum Robotics-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Rozum Robotics-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Rozum Robotics-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

Rozum Robotics-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

READY Robotics-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

READY Robotics-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

READY Robotics-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

READY Robotics-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

READY Robotics-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

YellowFIN Robotic Solutions-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

YellowFIN Robotic Solutions-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

YellowFIN Robotic Solutions-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

YellowFIN Robotic Solutions-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

YellowFIN Robotic Solutions-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Adapta Robotics-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

Adapta Robotics-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Adapta Robotics-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Adapta Robotics-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

Adapta Robotics-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Inovo Robotics Ltd-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

Inovo Robotics Ltd-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Inovo Robotics Ltd-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

Inovo Robotics Ltd-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

Inovo Robotics Ltd-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

United Robotics Group-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

United Robotics Group-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

United Robotics Group-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

United Robotics Group-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

United Robotics Group-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

NODE Robotics-Fri Jun 30 2023 23:59:59 GMT+0200 (Central European Summer Time)

NODE Robotics-Mon Jul 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

NODE Robotics-Thu Aug 31 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

NODE Robotics-Thu Aug 31 2023 23:59:59 GMT+0200 (Central European Summer Time)

NODE Robotics-Sat Sep 30 2023 23:59:59 GMT+0300 (Eastern European Summer Time)

robotic hands

robotic hands

Profiles