RT/ A technique for more effective multipurpose robots
Robotics & AI biweekly vol.96, 28th May — 11th June
TL;DR
- MIT researchers developed a technique to combine robotics training data across domains, modalities, and tasks using generative AI models. They create a combined strategy from several different datasets that enables a robot to learn to perform new tasks in unseen environments.
- Researchers have shown that members of the public have little trouble in learning very quickly how to use a third thumb — a controllable, prosthetic extra thumb — to pick up and manipulate objects. The team tested the robotic device on a diverse range of participants, which they say is essential for ensuring new technologies are inclusive and can work for everyone.
- A new study shows how AI-controlled charging stations, through smart algorithms, can offer electric vehicle users personalized prices, and thus minimize both price and waiting time for customers. But the researchers point to the importance of taking the ethical issues seriously, as there is a risk that the artificial intelligence exploits information from motorists.
- A novel, human-inspired approach to training AI systems to identify objects and navigate their surroundings could set the stage for the development of more advanced AI systems to explore extreme environments or distant worlds, according to new research.
- An autonomous robot created a shock-absorbing shape no human ever could — and what it means for designing safer helmets, packaging, car bumpers, and more.
- Using more robots to close labor gaps in the hospitality industry may backfire and cause more human workers to quit, according to a Washington State University study.
- Led by researchers at the University of Massachusetts Amherst, a new study identifying how to develop robot guide dogs with insights from guide dog users and trainers won a Best Paper Award at CHI 2024: Conference on Human Factors in Computing Systems (CHI).
- Physicists revealed a microscopic phenomenon that could greatly improve the performance of soft devices, such as agile flexible robots or microscopic capsules for drug delivery.
- A new advanced AI algorithm more accurately model how genes associated with specific autoimmune diseases are expressed and regulated and to identify additional genes of risk. The method outperforms existing methodologies and identified 26% more novel gene and trait associations.
- Researchers have determined that a protein prediction technology can yield accurate results in the hunt to efficiently find the best possible drug candidates for many conditions.
- And more!
Robotics market
The global market for robots is expected to grow at a compound annual growth rate (CAGR) of around 26 percent to reach just under 210 billion U.S. dollars by 2025.
Size of the global market for industrial and non-industrial robots between 2018 and 2025 (in billion U.S. dollars):
Latest News & Research
PoCo: Policy Composition from and for Heterogeneous Robot Learning
by Lirui Wang, Jialiang Zhao, Yilun Du, Edward H. Adelson, Russ Tedrake in arXiv
Let’s say you want to train a robot so it understands how to use tools and can then quickly learn to make repairs around your house with a hammer, wrench, and screwdriver. To do that, you would need an enormous amount of data demonstrating tool use.
Existing robotic datasets vary widely in modality — some include color images while others are composed of tactile imprints, for instance. Data could also be collected in different domains, like simulation or human demos. And each dataset may capture a unique task and environment. It is difficult to efficiently incorporate data from so many sources in one machine-learning model, so many methods use just one type of data to train a robot. But robots trained this way, with a relatively small amount of task-specific data, are often unable to perform new tasks in unfamiliar environments.
In an effort to train better multipurpose robots, MIT researchers developed a technique to combine multiple sources of data across domains, modalities, and tasks using a type of generative AI known as diffusion models. They train a separate diffusion model to learn a strategy, or policy, for completing one task using one specific dataset. Then they combine the policies learned by the diffusion models into a general policy that enables a robot to perform multiple tasks in various settings.
In simulations and real-world experiments, this training approach enabled a robot to perform multiple tool-use tasks and adapt to new tasks it did not see during training. The method, known as Policy Composition (PoCo), led to a 20 percent improvement in task performance when compared to baseline techniques.
“Addressing heterogeneity in robotic datasets is like a chicken-egg problem. If we want to use a lot of data to train general robot policies, then we first need deployable robots to get all this data. I think that leveraging all the heterogeneous data available, similar to what researchers have done with ChatGPT, is an important step for the robotics field,” says Lirui Wang, an electrical engineering and computer science (EECS) graduate student and lead author of a paper on PoCo.
Wang’s coauthors include Jialiang Zhao, a mechanical engineering graduate student; Yilun Du, an EECS graduate student; Edward Adelson, the John and Dorothy Wilson Professor of Vision Science in the Department of Brain and Cognitive Sciences and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL); and senior author Russ Tedrake, the Toyota Professor of EECS, Aeronautics and Astronautics, and Mechanical Engineering, and a member of CSAIL. The research will be presented at the Robotics: Science and Systems Conference.
A robotic policy is a machine-learning model that takes inputs and uses them to perform an action. One way to think about a policy is as a strategy. In the case of a robotic arm, that strategy might be a trajectory, or a series of poses that move the arm so it picks up a hammer and uses it to pound a nail. Datasets used to learn robotic policies are typically small and focused on one particular task and environment, like packing items into boxes in a warehouse.
“Every single robotic warehouse is generating terabytes of data, but it only belongs to that specific robot installation working on those packages. It is not ideal if you want to use all of these data to train a general machine,” Wang says.
The MIT researchers developed a technique that can take a series of smaller datasets, like those gathered from many robotic warehouses, learn separate policies from each one, and combine the policies in a way that enables a robot to generalize to many tasks. They represent each policy using a type of generative AI model known as a diffusion model. Diffusion models, often used for image generation, learn to create new data samples that resemble samples in a training dataset by iteratively refining their output.
But rather than teaching a diffusion model to generate images, the researchers teach it to generate a trajectory for a robot. They do this by adding noise to the trajectories in a training dataset. The diffusion model gradually removes the noise and refines its output into a trajectory.
This technique, known as Diffusion Policy, was previously introduced by researchers at MIT, Columbia University, and the Toyota Research Institute. PoCo builds off this Diffusion Policy work. The team trains each diffusion model with a different type of dataset, such as one with human video demonstrations and another gleaned from teleoperation of a robotic arm. Then the researchers perform a weighted combination of the individual policies learned by all the diffusion models, iteratively refining the output so the combined policy satisfies the objectives of each individual policy.
“One of the benefits of this approach is that we can combine policies to get the best of both worlds. For instance, a policy trained on real-world data might be able to achieve more dexterity, while a policy trained on simulation might be able to achieve more generalization,” Wang says.
Because the policies are trained separately, one could mix and match diffusion policies to achieve better results for a certain task. A user could also add data in a new modality or domain by training an additional Diffusion Policy with that dataset, rather than starting the entire process from scratch.
The researchers tested PoCo in simulation and on real robotic arms that performed a variety of tools tasks, such as using a hammer to pound a nail and flipping an object with a spatula. PoCo led to a 20 percent improvement in task performance compared to baseline methods.
“The striking thing was that when we finished tuning and visualized it, we can clearly see that the composed trajectory looks much better than either one of them individually,” Wang says.
In the future, the researchers want to apply this technique to long-horizon tasks where a robot would pick up one tool, use it, then switch to another tool. They also want to incorporate larger robotics datasets to improve performance.
“We will need all three kinds of data to succeed for robotics: internet data, simulation data, and real robot data. How to combine them effectively will be the million-dollar question. PoCo is a solid step on the right track,” says Jim Fan, senior research scientist at NVIDIA and leader of the AI Agents Initiative, who was not involved with this work.
Evaluating initial usability of a hand augmentation device across a large and diverse sample
by Dani Clode, Lucy Dowdall, Edmund da Silva, Klara Selén, Dorothy Cowie, Giulia Dominijanni, Tamar R. Makin in Science Robotics
Cambridge researchers have shown that members of the public have little trouble in learning very quickly how to use a third thumb — a controllable, prosthetic extra thumb — to pick up and manipulate objects.
The team tested the robotic device on a diverse range of participants, which they say is essential for ensuring new technologies are inclusive and can work for everyone. An emerging area of future technology is motor augmentation — using motorised wearable devices such as exoskeletons or extra robotic body parts to advance our motor capabilities beyond current biological limitations.
While such devices could improve the quality of life for healthy individuals who want to enhance their productivity, the same technologies can also provide people with disabilities new ways to interact with their environment. Professor Tamar Makin from the Medical Research Council (MRC) Cognition and Brain Sciences Unit at the University of Cambridge said: “Technology is changing our very definition of what it means to be human, with machines increasingly becoming a part of our everyday lives, and even our minds and bodies.
“These technologies open up exciting new opportunities that can benefit society, but it’s vital that we consider how they can help all people equally, especially marginalised communities who are often excluded from innovation research and development. To ensure everyone will have the opportunity to participate and benefit from these exciting advances, we need to explicitly integrate and measure inclusivity during the earliest possible stages of the research and development process.”
Dani Clode, a collaborator within Professor Makin’s lab, has developed the Third Thumb, an extra robotic thumb aimed at increasing the wearer’s range of movement, enhancing their grasping capability and expanding the carrying capacity of the hand. This allows the user to perform tasks that might be otherwise challenging or impossible to complete with one hand or to perform complex multi-handed tasks without having to coordinate with other people.
The Third Thumb is worn on the opposite side of the palm to the biological thumb and controlled by a pressure sensor placed under each big toe or foot. Pressure from the right toe pulls the Thumb across the hand, while the pressure exerted with the left toe pulls the Thumb up toward the fingers. The extent of the Thumb’s movement is proportional to the pressure applied, and releasing pressure moves it back to its original position.
In 2022, the team had the opportunity to test the Third Thumb at the annual Royal Society Summer Science Exhibition, where members of the public of all ages were able to use the device during different tasks. Over the course of five days, the team tested 596 participants, ranging in age from three to 96 years old and from a wide range of demographic backgrounds. Of these, only four were unable to use the Third Thumb, either because it did not fit their hand securely, or because they were unable to control it with their feet (the pressure sensors developed specifically for the exhibition were not suitable for very lightweight children).
Participants were given up to a minute to familiarise themselves with the device, during which time the team explained how to perform one of two tasks. The first task involved picking up pegs from a pegboard one at a time with just the Third Thumb and placing them in a basket. Participants were asked to move as many pegs as possible in 60 seconds. 333 participants completed this task. The second task involved using the Third Thumb together with the wearer’s biological hand to manipulate and move five or six different foam objects. The objects were of various shapes that required different manipulations to be used, increasing the dexterity of the task. Again, participants were asked to move as many objects as they could into the basket within a maximum of 60 seconds. 246 participants completed this task.
Almost everyone was able to use the device straightaway. 98% of participants were able to successfully manipulate objects using the Third Thumb during the first minute of use, with only 13 participants unable to perform the task.
Ability levels between participants were varied, but there were no differences in performance between genders, nor did handedness change performance — despite the Thumb always being worn on the right hand. There was no definitive evidence that people who might be considered ‘good with their hands’ — for example, they were learning to play a musical instrument, or their jobs involved manual dexterity — were any better at the tasks.
Older and younger adults had a similar level of ability when using the new technology, though further investigation just within the older adults age bracket revealed a decline in performance with increasing age. The researchers say this effect could be due to the general degradation in sensorimotor and cognitive abilities that are associated with ageing and may also reflect a generational relationship to technology.
Performance was generally poorer among younger children. Six out of the 13 participants that could not complete the task were below the age of 10 years old, and of those that did complete the task, the youngest children tended to perform worse compared to older children. But even older children (aged 12–16 years) struggled more than young adults.
Dani said: “Augmentation is about designing a new relationship with technology — creating something that extends beyond being merely a tool to becoming an extension of the body itself. Given the diversity of bodies, it’s crucial that the design stage of wearable technology is as inclusive as possible. It’s equally important that these devices are accessible and functional for a wide range of users. Additionally, they should be easy for people to learn and use quickly.”
Co-author Lucy Dowdall, also from the MRC Cognition and Brain Science Unit, added: “If motor augmentation — and even broader human-machine interactions — are to be successful, they’ll need to integrate seamlessly with the user’s motor and cognitive abilities. We’ll need to factor in different ages, genders, weight, lifestyles, disabilities — as well as people’s cultural, financial backgrounds, and even likes or dislikes of technology. Physical testing of large and diverse groups of individuals is essential to achieve this goal.”
There are countless examples of where a lack of inclusive design considerations has led to technological failure:
- Automated speech recognition systems that convert spoken language to text have been shown to perform better listening to white voices over Black voices.
- Some augmented reality technologies have been found to be less effective for users with darker skin tones.
- Women face a higher health risk from car accidents, due to car seats and seatbelts being primarily designed to accommodate ‘average’ male-sized dummies during crash testing.
- Hazardous power and industrial tools designed for a right-hand dominant use or grip have resulted in more accidents when operated by left-handers forced to use their non-dominant hand.
Personalized dynamic pricing policy for electric vehicles: Reinforcement learning approach
by Sangjun Bae, Balázs Kulcsár, Sébastien Gros in Transportation Research Part C: Emerging Technologies
As more and more people drive electric cars, congestion and queues can occur when many people need to charge at the same time. A new study from Chalmers University of Technology in Sweden shows how AI-controlled charging stations, through smart algorithms, can offer electric vehicle users personalised prices, and thus minimise both price and waiting time for customers. But the researchers point to the importance of taking the ethical issues seriously, as there is a risk that the artificial intelligence exploits information from motorists.
Today’s commercial charging infrastructure can be a jungle. The market is dynamic and complex with a variety of subscriptions and free competition between providers. At some fast charging stations, congestion and long queues may even occur. In a new study, researchers at Chalmers have created a mathematical model to investigate how fast charging stations controlled by artificial intelligence, AI, can help by offering electric car drivers personalised prices, which the drivers can choose to accept or refuse. The AI uses algorithms that can adjust prices based on individual factors, such as battery level and the car’s geographic location.
“The electric car drivers can choose to share information with the charging station providers and receive a personal price proposal from a smart charging station. In our study, we could show how rational and self-serving drivers react by only accepting offers that are beneficial to themselves. This leads to both price and waiting times being minimized,” says Balázs Kulcsár, professor at the department of electrical engineering at Chalmers.
In the study, the drivers always had the option to refuse the personal price, and choose a conventional charging station with a fixed price instead. The personal prices received by the drivers could differ significantly from each other, but were almost always lower than the market prices. For the providers of charging stations, the iterative AI algorithm can find out which individual prices are accepted by the buyer, and under which conditions. However, during the course of the study, the researchers noted that on some occasions the algorithm raised the price significantly when the electric car’s batteries were almost completely empty, and the driver consequently had no choice but to accept the offer.
“Smart charging stations can solve complex pricing in a competitive market, but our study shows that they need to be developed and introduced with privacy protection for consumers, well in line with responsible-ethical AI paradigms,” says Balázs Kulcsár.
Incorporating simulated spatial context information improves the effectiveness of contrastive learning models
by Lizhen Zhu, James Z. Wang, Wonseuk Lee, Brad Wyble in Patterns
A novel, human-inspired approach to training artificial intelligence systems to identify objects and navigate their surroundings could set the stage for the development of more advanced AI systems to explore extreme environments or distant worlds, according to research from an interdisciplinary team at Penn State.
In the first two years of life, children experience a somewhat narrow set of objects and faces, but with many different viewpoints and under varying lighting conditions. Inspired by this developmental insight, the researchers introduced a new machine learning approach that uses information about spatial position to train AI visual systems more efficiently. They found that AI models trained on the new method outperformed base models by up to 14.99%. They reported their findings in the May issue of the journal Patterns.
“Current approaches in AI use massive sets of randomly shuffled photographs from the internet for training. In contrast, our strategy is informed by developmental psychology, which studies how children perceive the world,” said Lizhen Zhu, the lead author and doctoral candidate in the College of Information Sciences and Technology at Penn State.
The researchers developed a new contrastive learning algorithm, which is a type of self-supervised learning method in which an AI system learns to detect visual patterns to identify when two images are derivations of the same base image, resulting in a positive pair. These algorithms, however, often treat images of the same object taken from different perspectives as separate entities rather than as positive pairs. Taking into account environmental data, including location, allows the AI system to overcome these challenges and detect positive pairs regardless of changes in camera position or rotation, lighting angle or condition and focal length, or zoom, according to the researchers.
“We hypothesize that infants’ visual learning depends on location perception. In order to generate an egocentric dataset with spatiotemporal information, we set up virtual environments in the ThreeDWorld platform, which is a high-fidelity, interactive, 3D physical simulation environment. This allowed us to manipulate and measure the location of viewing cameras as if a child was walking through a house,” Zhu added.
The scientists created three simulation environments — House14K, House100K and Apartment14K, with ‘14K’ and ‘100K’ referring to the approximate number of sample images taken in each environment. Then they ran base contrastive learning models and models with the new algorithm through the simulations three times to see how well each classified images. The team found that models trained on their algorithm outperformed the base models on a variety of tasks. For example, on a task of recognizing the room in the virtual apartment, the augmented model performed on average at 99.35%, a 14.99% improvement over the base model. These new datasets are available for other scientists to use in training through www.child-view.com.
“It’s always hard for models to learn in a new environment with a small amount of data. Our work represents one of the first attempts at more energy-efficient and flexible AI training using visual content,” said James Wang, distinguished professor of information sciences and technology and advisor of Zhu.
The research has implications for the future development of advanced AI systems meant to navigate and learn from new environments, according to the scientists.
“This approach would be particularly beneficial in situations where a team of autonomous robots with limited resources needs to learn how to navigate in a completely unfamiliar environment,” Wang said. “To pave the way for future applications, we plan to refine our model to better leverage spatial information and incorporate more diverse environments.”
Superlative mechanical energy absorbing efficiency discovered through self-driving lab-human partnership
by Kelsey L. Snapp, Benjamin Verdier, Aldair E. Gongora, Samuel Silverman, Adedire D. Adesiji, Elise F. Morgan, Timothy J. Lawton, Emily Whiting, Keith A. Brown in Nature Communications
Inside a lab in Boston University’s College of Engineering, a robot arm drops small, plastic objects into a box placed perfectly on the floor to catch them as they fall. One by one, these tiny structures — feather-light, cylindrical pieces, no bigger than an inch tall — fill the box. Some are red, others blue, purple, green, or black.
Each object is the result of an experiment in robot autonomy. On its own, learning as it goes, the robot is searching for, and trying to make, an object with the most efficient energy-absorbing shape to ever exist.
To do this, the robot creates a small plastic structure with a 3D printer, records its shape and size, moves it to a flat metal surface — and then crushes it with a pressure equivalent to an adult Arabian horse standing on a quarter. The robot then measures how much energy the structure absorbed, how its shape changed after being squashed, and records every detail in a vast database. Then, it drops the crushed object into the box and wipes the metal plate clean, ready to print and test the next piece. It will be ever-so-slightly different from its predecessor, its design and dimensions tweaked by the robot’s computer algorithm based on all past experiments — the basis of what’s called Bayesian optimization. Experiment after experiment, the 3D structures get better at absorbing the impact of getting crushed.
These experiments are possible because of the work of Keith Brown, an ENG associate professor of mechanical engineering, and his team in the KABlab. The robot, named MAMA BEAR — short for its lengthy full title, Mechanics of Additively Manufactured Architectures Bayesian Experimental Autonomous Researcher — has evolved since it was first conceptualized by Brown and his lab in 2018. By 2021, the lab had set the machine on its quest to make a shape that absorbs the most energy, a property known as its mechanical energy absorption efficiency. This current iteration has run continuously for over three years, filling dozens of boxes with more than 25,000 3D-printed structures.
Why so many shapes? There are countless uses for something that can efficiently absorb energy — say, cushioning for delicate electronics being shipped across the world or for knee pads and wrist guards for athletes. “You could draw from this library of data to make better bumpers in a car, or packaging equipment, for example,” Brown says.
To work ideally, the structures have to strike the perfect balance: they can’t be so strong that they cause damage to whatever they’re supposed to protect, but should be strong enough to absorb impact. Before MAMA BEAR, the best structure anyone ever observed was about 71 percent efficient at absorbing energy, says Brown. But on a chilly January afternoon in 2023, Brown’s lab watched their robot hit 75 percent efficiency, breaking the known record.
“When we started out, we didn’t know if there was going to be this record-breaking shape,” says Kelsey Snapp (ENG’25), a PhD student in Brown’s lab who oversees MAMA BEAR. “Slowly but surely we kept inching up, and broke through.”
The record-breaking structure looks like nothing the researchers would have expected: it has four points, shaped like thin flower petals, and is taller and narrower than the early designs.
“We’re excited that there’s so much mechanical data here, that we’re using this to learn lessons about design more generally,” Brown says.
Their extensive data is already getting its first real-life application, helping to inform the design of new helmet padding for US Army soldiers. Brown, Snapp, and project collaborator Emily Whiting, a BU College of Arts & Sciences associate professor of computer science, worked with the US Army and went through field testing to ensure helmets using their patent-pending padding are comfortable and provide sufficient protection from impact. The 3D structure used for the padding is different from the record-breaking piece — with a softer center and shorter stature to help with comfort.
MAMA BEAR is not Brown’s only autonomous research robot. His lab has other “BEAR” robots performing different tasks — like the nano BEAR, which studies the way materials behave at the molecular scale using a technology called atomic force microscopy. Brown has also been working with Jörg Werner, an ENG assistant professor of mechanical engineering, to develop another system, known as the PANDA — short for Polymer Analysis and Discovery Array — BEAR to test thousands of thin polymer materials to find one that works best in a battery.
“They’re all robots that do research,” Brown says. “The philosophy is that they’re using machine learning together with automation to help us do research much faster.”
“Not just faster,” adds Snapp. “You can do things you couldn’t normally do. We can reach a structure or goal that we wouldn’t have been able to achieve otherwise, because it would have been too expensive and time-consuming.” He has worked closely with MAMA BEAR since the experiments began in 2021, and gave the robot its ability to see — known as machine vision — and clean its own test plate.
The KABlab is hoping to further demonstrate the importance of autonomous research. Brown wants to keep collaborating with scientists in various fields who need to test incredibly large numbers of structures and solutions. Even though they already broke a record, “we have no ability to know if we’ve reached the maximum efficiency,” Brown says, meaning they could possibly break it again. So, MAMA BEAR will keep on running, pushing boundaries further, while Brown and his team see what other applications the database can be useful for. They’re also exploring how the more than 25,000 crushed pieces can be unwound and reloaded into the 3D printers so the material can be recycled for more experiments.
“We’re going to keep studying this system, because mechanical efficiency, like so many other material properties, is only accurately measured by experiment,” Brown says, “and using self-driving labs helps us pick the best experiments and perform them as fast as possible.”
Are robots stealing our jobs? Examining robot-phobia as a job stressor in the hospitality workplace
by Chun-Chu (Bamboo) Chen, Ruiying Cai in International Journal of Contemporary Hospitality Management
Using more robots to close labor gaps in the hospitality industry may backfire and cause more human workers to quit, according to a Washington State University study.
The study, involving more than 620 lodging and food service employees, found that “robot-phobia” — specifically the fear that robots and technology will take human jobs — increased workers’ job insecurity and stress, leading to greater intentions to leave their jobs. The impact was more pronounced with employees who had real experience working with robotic technology. It also affected managers in addition to frontline workers.
“The turnover rate in the hospitality industry ranks among the highest across all non-farm sectors, so this is an issue that companies need to take seriously,” said lead author Bamboo Chen, a hospitality researcher in WSU’s Carson College of Business. “The findings seem to be consistent across sectors and across both frontline employees and managers. For everyone, regardless of their position or sector, robot-phobia has a real impact.”
Food service and lodging industries were hit particularly hard by the pandemic lockdowns, and many businesses are still struggling to find enough workers. For example, the accommodation workforce in April 2024 was still 9.2% below what it was in February 2020, according to U.S. Bureau of Labor Statistics. The ongoing labor shortage has inspired some employers to turn to robotic technology to fill the gap.
While other studies have focused on customers’ comfort with robots, this study focuses on how the technology impacted hospitality workers. Chen and WSU colleague Ruying Cai surveyed 321 lodging and 308 food service employees from across the U.S., asking a range of questions about their jobs and attitudes toward robots. The survey defined “robots” broadly to include a range of robotic and automation technologies, such as human-like robot servers and automated robotic arms as well as self-service kiosks and tabletop devices.
Analyzing the survey data, the researchers found that having a higher degree of robot-phobia was connected to greater feelings of job insecurity and stress — which were then correlated with “turnover intention” or workers’ plans to leave their jobs. Those fears did not decrease with familiarity: employees who had more actual engagement with robotic technology in their daily jobs had higher fears that it would make human workers obsolete.
Perception also played a role. The employees who viewed robots as being more capable and efficient also ranked higher in turnover intention. Robots and automation can be good ways to help augment service, Chen said, as they can handle tedious tasks humans typically do not like doing such as washing dishes or handling loads of hotel laundry. But the danger comes if the robotic additions cause more human workers to quit. The authors point out this can create a “negative feedback loop” that can make the hospitality labor shortage worse. Chen recommended that employers communicate not only the benefits but the limitations of the technology — and place a particular emphasis on the role human workers play.
“When you’re introducing a new technology, make sure not to focus just on how good or efficient it will be. Instead, focus on how people and the technology can work together,” he said.
Towards Robotic Companions: Understanding Handler-Guide Dog Interactions for Informed Guide Dog Robot Design
by Hochul Hwang, Hee-Tae Jung, Nicholas A Giudice, Joydeep Biswas, Sunghoon Ivan Lee, Donghyun Kim in CHI 2024: Conference on Human Factors in Computing Systems
What features does a robotic guide dog need? Ask the blind, say the authors of an award-winning paper. Led by researchers at the University of Massachusetts Amherst, a study identifying how to develop robot guide dogs with insights from guide dog users and trainers won a Best Paper Award at CHI 2024: Conference on Human Factors in Computing Systems (CHI).
Guide dogs enable remarkable autonomy and mobility for their handlers. However, only a fraction of people with visual impairments have one of these companions. The barriers include the scarcity of trained dogs, cost (which is $40,000 for training alone), allergies and other physical limitations that preclude caring for a dog. Robots have the potential to step in where canines can’t and address a truly gaping need — if designers can get the features right.
“We’re not the first ones to develop guide-dog robots,” says Donghyun Kim, assistant professor in the UMass Amherst Manning College of Information and Computer Science (CICS) and one of the corresponding authors of the award-winning paper. “There are 40 years of study there, and none of these robots are actually used by end users. We tried to tackle that problem first so that, before we develop the technology, we understand how they use the animal guide dog and what technology they are waiting for.”
The research team conducted semistructured interviews and observation sessions with 23 visually impaired dog-guide handlers and five trainers. Through thematic analysis, they distilled the current limitations of canine guide dogs, the traits handlers are looking for in an effective guide and considerations to make for future robotic guide dogs.
One of the more nuanced themes that came from these interviews was the delicate balance between robot autonomy and human control. “Originally, we thought that we were developing an autonomous driving car,” says Kim. They envisioned that the user would tell the robot where they want to go and the robot would navigate autonomously to that location with the user in tow. This is not the case.
The interviews revealed that handlers do not use their dog as a global navigation system. Instead, the handler controls the overall route while the dog is responsible for local obstacle avoidance. However, even this isn’t a hard-and-fast rule. Dogs can also learn routes by habit and may eventually navigate a person to regular destinations without directional commands from the handler.
“When the handler trusts the dog and gives more autonomy to the dog, it’s a bit delicate,” says Kim. “We cannot just make a robot that is fully passive, just following the handler, or just fully autonomous, because then [the handler] feels unsafe.”
The researchers hope this paper will serve as a guide, not only in Kim’s lab, but for other robot developers as well. “In this paper, we also give directions on how we should develop these robots to make them actually deployable in the real world,” says Hochul Hwang, first author on the paper and a doctoral candidate in Kim’s robotics lab. For instance, he says that a two-hour battery life is an important feature for commuting, which can be an hour on its own. “About 90% of the people mentioned the battery life,” he says. “This is a critical part when designing hardware because the current quadruped robots don’t last for two hours.”
These are just a few of the findings in the paper. Others include: adding more camera orientations to help address overhead obstacles; adding audio sensors for hazards approaching from the occluded regions; understanding ‘sidewalk’ to convey the cue, “go straight,” which means follow the street (not travel in a perfectly straight line); and helping users get on the right bus (and then find a seat as well).
“The most exciting aspect of winning this award is that the research community recognizes and values our direction,” says Kim. “Since we don’t believe that guide dog robots will be available to individuals with visual impairments within a year, nor that we’ll solve every problem, we hope this paper inspires a broad range of robotics and human-robot interaction researchers, helping our vision come to fruition sooner.”
Diffusiophoretic Fast Swelling of Chemically Responsive Hydrogels
by Chinmay Katke, Peter A. Korevaar, C. Nadir Kaplan in Physical Review Letters
In paper, Virginia Tech physicists revealed a microscopic phenomenon that could greatly improve the performance of soft devices, such as agile flexible robots or microscopic capsules for drug delivery.
The paper, written by doctoral candidate Chinmay Katke, assistant professor C. Nadir Kaplan, and co-author Peter A. Korevaar from Radboud University in the Netherlands, proposes a new physical mechanism that could speed up the expansion and contraction of hydrogels. For one thing, this opens up the possibility for hydrogels to replace rubber-based materials used to make flexible robots — enabling these fabricated materials to perhaps move with a speed and dexterity close to that of human hands.
Soft robots are already being used in manufacturing, where a hand-like device is programmed to grab an item from a conveyer belt — picture a hot dog or piece of soap — and place it in a container to be packaged. But the ones in use now lean on hydraulics or pneumatics to change the shape of the “hand” to pick up the item.
Akin to our own body, hydrogels mostly contain water and are everywhere around us, e.g., food jelly and shaving gel. Katke, Korevaar, and Kaplan’s research appears to have found a method that allows hydrogels to swell and contract much more quickly, which would improve their flexibility and capability to function in different settings.
Living organisms use osmosis for such activities as bursting seed dispersing fruits in plants or absorbing water in the intestine. Normally, we think of osmosis as a flow of water moving through a membrane, with bigger molecules like polymers unable to move through. Such membranes are called semi-permeable membranes and were thought to be necessary to trigger osmosis.
Previously, Korevaar and Kaplan had done experiments by using a thin layer of hydrogel film comprised of polyacrylic acid. They had observed that even though the hydrogel film allows both water and ions to pass through and is not selective, the hydrogel rapidly swells due to osmosis when ions are released inside the hydrogel and shrinks back again.
Katke, Korevaar, and Kaplan developed a new theory to explain the above observation. This theory tells that microscopic interactions between ions and polyacrylic acid can make hydrogel swell when the released ions inside the hydrogel are unevenly spread out. They called this “diffusio-phoretic swelling of the hydrogels.” Furthermore, this newly discovered mechanism allows hydrogels to swell much faster than what has been previously possible.
Kaplan explained: Soft agile robots are currently made with rubber, which “does the job but their shapes are changed hydraulically or pneumatically. This is not desired because it is difficult to imprint a network of tubes into these robots to deliver air or fluid into them.”
Imagine, Kaplan said, how many different things you can do with your hand and how fast you can do them owing to your neural network and the motion of ions under your skin. Because the rubber and hydraulics are not as versatile as your biological tissues, which is a hydrogel, state-of-the-art soft robots can only do a limited number of movements.
Katke explained that the process they have researched allows the hydrogels to change shape then change back to their original form “significantly faster this way” in soft robots that are larger than ever before.
At present, only microscopic-sized hydrogel robots can respond to a chemical signal quickly enough to be useful and larger ones require hours to change shape, Katke said. By using the new diffusio-phoresis method, soft robots as large as a centimeter may be able to transform in just a few seconds, which is subject to further studies.
Larger agile soft robots that could respond quickly could improve assistive devices in healthcare, “pick-and-place” functions in manufacturing, search and rescue operations, cosmetics used for skincare, and contact lenses.
Integrating single cell expression quantitative trait loci summary statistics to understand complex trait risk genes
by Lida Wang, Chachrit Khunsriraksakul, Havell Markus, Dieyi Chen, Fan Zhang, Fang Chen, Xiaowei Zhan, Laura Carrel, Dajiang. J. Liu, Bibo Jiang in Nature Communications
A new advanced artificial intelligence algorithm may lead to better — and earlier — predictions and novel therapies for autoimmune diseases, which involve the immune system mistakenly attacking their body’s own healthy cells and tissues. The algorithm digs into the genetic code underlying the conditions to more accurately model how genes associated with specific autoimmune diseases are expressed and regulated and to identify additional genes of risk.
The work, developed by a team led by Penn State College of Medicine researchers, outperforms existing methodologies and identified 26% more novel gene and trait associations, the researchers said.
“We all carry some DNA mutations, and we need to figure out how any one of these mutations may influence gene expression linked to disease so we can predict disease risk early. This is especially important for autoimmune disease,” said Dajiang Liu, distinguished professor, vice chair for research, and director of artificial intelligence and biomedical informatics at the Penn State College of Medicine and co-senior author of the study. “If an AI algorithm can more accurately predict disease risk, it means we can carry out interventions earlier.”
Genetics often underpin disease development. Variations in DNA can influence gene expression, or the process by which the information in DNA is converted into functional products like a protein. How much or how little a gene is expressed can influence disease risk.
Genome-wide association studies (GWAS), a popular approach in human genetics research, can home in on regions of the genome associated with a particular disease or trait but can’t pinpoint the specific genes that affect disease risks. It’s like sharing your location with a friend with the precise location setting turned off on your smartphone — the city might be obvious, but the address is obscured. Existing methods are also limited in the granularity of its analysis. Gene expression can be specific to certain types of cells. If the analysis doesn’t distinguish between distinct cell types, the results may overlook real causal relationships between genetic variants and gene expression.
The research team’s method, dubbed EXPRESSO for EXpression PREdiction with Summary Statistics Only, applies a more advanced artificial intelligence algorithm and analyzes data from single-cell expression quantitative trait loci, a type of data that links genetic variants to the genes they regulate. It also integrates 3D genomic data and epigenetics — which measures how genes may be modified by environment to influence disease — into its modeling. The team applied EXPRESSO to GWAS datasets for 14 autoimmune diseases, including lupus, Crohn’s disease, ulcerative colitis and rheumatoid arthritis.
“With this new method, we were able to identify many more risk genes for autoimmune disease that actually have cell-type specific effects, meaning that they only have effects in a particular cell type and not others,” said Bibo Jiang, assistant professor at the Penn State College of Medicine and senior author of the study.
The team then used this information to identify potential therapeutics for autoimmune disease. Currently, there aren’t good long-term treatment options, they said.
“Most treatments are designed to mitigate symptoms, not cure the disease. It’s a dilemma knowing that autoimmune disease needs long-term treatment, but the existing treatments often have such bad side effects that they can’t be used for long. Yet, genomics and AI offer a promising route to develop novel therapeutics,” said Laura Carrel, professor of biochemistry and molecular biology at the Penn State College of Medicine and co-senior author of the study.
The team’s work pointed to drug compounds that could reverse gene expression in cell types associated with an autoimmune disease, such as vitamin K for ulcerative colitis and metformin, which is typically prescribed for type 2 diabetes, for type 1 diabetes. These drugs, already approved by the Food and Drug Administration as safe and effective for treating other diseases, could potentially be repurposed.
AlphaFold2 structures guide prospective ligand discovery
by Jiankun Lyu, Nicholas Kapolka, Ryan Gumpper, Assaf Alon, Liang Wang, et al in Science
Artificial intelligence has numerous applications in healthcare, from analyzing medical imaging to optimizing the execution of clinical trials, and even facilitating drug discovery.
AlphaFold2, an artificial intelligence system that predicts protein structures, has made it possible for scientists to identify and conjure an almost infinite number of drug candidates for the treatment of neuropsychiatric disorders. However recent studies have sown doubt about the accuracy of AlphaFold2 in modeling ligand binding sites, the areas on proteins where drugs attach and begin signaling inside cells to cause a therapeutic effect, as well as possible side effects.
In a new paper, Bryan Roth, MD, PhD, the Michael Hooker Distinguished Professor of Pharmacology and director of the NIMH Psychoactive Drug Screening Program at the University of North Carolina School of Medicine, and colleagues at UCSF, Stanford and Harvard determined that AlphaFold2 can yield accurate results for ligand binding structures, even when the technology has nothing to go off of.
“Our results suggest that AF2 structures can be useful for drug discovery,” said Roth, senior author who holds a joint appointment at the UNC Eshelman School of Pharmacy. “With a nearly infinite number of possibilities to create drugs that hit their intended target to treat a disease, this sort of AI tool can be invaluable.”
Much like weather forecasting or stock market prediction, AlphaFold2 works by pulling from a massive database of known proteins to create models of protein structures. Then, it can simulate how different molecular compounds (like drug candidates) fit into the protein’s binding sites and produce wanted effects. Researchers can use the resulting combinations to better understand protein interactions and create new drug candidates.
To determine the accuracy of AlphaFold2, researchers had to compare the results of a retrospective study against that of a prospective study. A retrospective study involves researchers feeding the prediction software compounds they already know bind to the receptor. Whereas, a prospective study requires researchers to use the technology as a fresh slate, and then feed the AI platform information about compounds that may or may not interact with the receptor.
Researchers used two proteins, sigma-2 and 5-HT2A, for the study. These proteins, which belong to two different protein families, are important in cell communication and have been implicated in neuropsychiatric conditions such as Alzheimer’s disease and schizophrenia. The 5-HT2A serotonin receptor is also the main target for psychedelic drugs which show promise for treating a large number of neuropsychiatric disorders.
Roth and colleagues selected these proteins because AlphaFold2 had no prior information about sigma-2 and 5-HT2A or the compounds that might bind to them. Essentially, the technology was given two proteins for which it wasn’t trained on — essentially giving the researchers a “blank slate.”
First, researchers fed the AlphaFold system the protein structures for sigma-2 and 5-HT2A, creating a prediction model. Researchers then accessed physical models of the two proteins that were produced using complex microscopy and x-ray crystallography techniques. With a press of a button, as many as 1.6 billion potential drugs were targeted to the experimental models and AlphaFold2 models. Interestingly, every model had a different drug candidate outcome.
Despite the models having differing results, they show great promise for drug discovery. Researchers determined that the proportion of compounds that actually altered protein activity for each of the models were around 50% and 20% for the sigma-2 receptor and 5-HT2A receptors, respectively. A result greater than 5% is exceptional. Out of the hundreds of millions of potential combinations, 54% of the drug-protein interactions using the sigma-2 AlphaFold2 protein models were successfully activated through a bound drug candidate. The experimental model for sigma-2 produced similar results with a success rate of 51%.
“This work would be impossible without collaborations among several leading experts at UCSF, Stanford, Harvard, and UNC-Chapel Hill,” Roth said. “Going forward we will test whether these results might be applicable to other therapeutic targets and target classes.”
Subscribe to Paradigm!
Medium. Twitter. Telegram. Telegram Chat. Reddit. LinkedIn.
Main sources
Research articles
