Recent papers:
- Constrained Stein Variational Trajectory Optimization
Thomas Power and Dmitry Berenson
IEEE Transactions on Robotics (T-RO), June 2024. (Details | PDF | video)
- Subgoal Diffuser: Coarse-To-Fine Subgoal Generation to Guide Model Predictive Control for Robot Manipulation
Zixuan Huang, Yating Lin, Fan Yang, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2024. (Details | video)
- Subgoal Diffuser: Coarse-To-Fine Subgoal Generation to Guide Model Predictive Control for Robot Manipulation
Zixuan Huang, Yating Lin, Fan Yang, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2024. (Details | video)
- The Grasp Loop Signature: A Topological Representation for Manipulation Planning with Ropes and Cables
Peter Mitrano and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2024. (Details | video)
- Improving Out-Of-Distribution Generalization of Learned Dynamics by Learning Pseudometrics and Constraint Manifolds
Yating Lin, Glen Chou, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2024. (Details | video)
- Online Adaptation of Sampling-Based Motion Planning with Inaccurate Models
Marco Faroni and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2024. (Details | video)
- Learning a Generalizable Trajectory Sampling Distribution for Model Predictive Control
Thomas Power and Dmitry Berenson
IEEE Transactions on Robotics (T-RO), vol. 40, pp. 2111-2127, 2024. (Details | video)
- Motion Planning as Online Learning: A Multi-Armed Bandit Approach to Kinodynamic Sampling-Based Planning
Marco Faroni and Dmitry Berenson
IEEE Robotics and Automation Letters (RA-L), online September 2023. (Details | PDF | video)
- CHSEL: Producing Diverse Plausible Pose Estimates from Contact and Free Space Data
Sheng Zhong, Nima Fazeli, and Dmitry Berenson
Robotics: Science and Systems (RSS), July 2023. (Details | PDF | video)
- Integrated Object Deformation and Contact Patch Estimation from Visuo-Tactile Feedback
Mark Van der Merwe, Youngsun Wi, Dmitry Berenson, Nima Fazeli
Robotics: Science and Systems (RSS), July 2023. (Details | PDF)
- Focused Adaptation of Dynamics Models for Deformable Object Manipulation
Peter Mitrano, Alex LaGrassa, Oliver Kroemer, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May 2023. (Details | PDF | video)
- Data-Efficient Learning of Natural Language to Linear Temporal Logic Translators for Robot Task Specification
Jiayi Pan, Glen Chou, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May 2023. (PDF | video)
Research Interests
Our research focuses on creating algorithms that allow robots to interact with the world. These general-purpose motion planning, machine learning, and manipulation algorithms can be applied to robots that work in homes, factories, and operating rooms. We are interested in all aspects of algorithm development; including creating efficient algorithms, proving their theoretical properties, validating them on real-world robots and problems, integrating them with sensing and higher-level reasoning, and distributing them to open-source communities. Our lab draws on ideas in search, optimization, machine learning, motion planning, control theory, and topology to develop these algorithms and to prove their properties. We also seek to develop algorithms which can generalize to many types of practical tasks and application areas.
Press
|
|
The goal of bridging learning and planning is to create algorithms whose performance improves with experience while maintaining generality and reliability. Learning algorithms are good at quickly producing solutions to problems that have been encountered before, but they have difficulty generalizing to new situations. Planning algorithms are good at generalizing to new situations but at significant (and sometimes prohibitive) computational cost. Our work seeks to combine the best of both approaches to create a method whose performance improves as it gains more experience while retaining the ability to generalize to new situations. |
Dmitry Berenson, Pieter Abbeel, and Ken Goldberg
IEEE International Conference on Robotics and Automation (ICRA), May, 2012.
Details | PDF | Video
Calder Phillips-Grafflin and Dmitry Berenson
International Symposium on Robotics Research (ISRR), September, 2015.
Changshuo Li and Dmitry Berenson
International Symposium on Experimental Robotics (ISER), October, 2016.
PDF | Video
Yu-Chi Lin and Dmitry Berenson
IEEE-RAS International Conference on Humanoid Robots (Humanoids), November, 2016.
PDF | Video
Yu-Chi Lin and Dmitry Berenson
IEEE-RAS International Conference on Humanoid Robots (Humanoids), November, 2017.
PDF | Video
Anahita Mohseni-Kabir, Changshuo Li, Victoria Wu, Daniel Miller, Benjamin Hylak, Sonia Chernova, Dmitry Berenson, Candace Sidner, Charles Rich
Autonomous Robots (AuRo), Vol. 43, No. 4, pp. 859-874, April 2019.
Details
Yu-Chi Lin, Brahayam Ponton, Ludovic Righetti, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2019.
PDF | Video
Andrew Price*, Linyi Jin*, and Dmitry Berenson
International Symposium on Robotics Research (ISRR), October 2019.
Details | PDF | Video
Glen Chou, Necmiye Ozay, and Dmitry Berenson
Conference on Robot Learning (CoRL), October, 2019.
Details | PDF
Dale McConachie, Thomas Power, Peter Mitrano, and Dmitry Berenson
IEEE Robotics and Automation Letters (RA-L) (Presented at ICRA 2020), vol. 5, no. 2, pp. 3540-3547, April 2020.
Details | PDF | Video
Glen Chou, Necmiye Ozay, and Dmitry Berenson
IEEE Robotics and Automation Letters (RA-L) (presented at ICRA 2020), vol. 5, no. 2, pp. 3682-3690, April 2020.
Details | PDF | Video
Yu-Chi Lin
Ph.D. dissertation, Robotics Institute, University of Michigan, April 2020.
Dale McConachie
Ph.D. dissertation, Robotics Institute, University of Michigan, April 2020.
Glen Chou, Necmiye Ozay, and Dmitry Berenson
Robotics: Science and Systems (RSS), July 2020.
Details | PDF | Video
Glen Chou, Necmiye Ozay, and Dmitry Berenson
Conference on Robot Learning (CoRL), November 2020.
Details | PDF | Video
Thomas Power and Dmitry Berenson
IEEE Robotics and Automation Letters (RA-L) (Presented at ICRA 2021), vol. 6, no. 2, pp. 1184-1191, April 2021.
Details | PDF | Video
Sheng Zhong, Zhenyuan Zhang, Nima Fazeli, and Dmitry Berenson
IEEE Robotics and Automation Letters (RA-L) (Presented at ICRA 2021), vol. 6, no. 2, pp. 1447-1454, April 2021.
Details | PDF | Video
Craig Knuth*, Glen Chou*, Necmiye Ozay, and Dmitry Berenson
IEEE Robotics and Automation Letters (RA-L) (Presented at ICRA 2021), vol. 6, no. 3, pp. 5129 - 5136, March 2021.
Details | PDF | Video
Peter Mitrano, Dale McConachie, and Dmitry Berenson
Science Robotics, vol. 6, no. 54, May 2021.
Details | Video
Yu-Chi Lin and Dmitry Berenson
Autonomous Robots (AuRo), vol. 45, no. 6, pp. 937-956, June 2021.
Details | PDF | Video
Glen Chou, Dmitry Berenson, and Necmiye Ozay
International Journal of Robotics Research (IJRR), vol. 40, no. 10-11, pp. 1255-1283, September 2021.
Details | PDF
Brad Saund
Ph.D. dissertation, Robotics Institute, University of Michigan, July 2021.
Glen Chou, Necmiye Ozay, and Dmitry Berenson
Autonomous Robots (AuRo), vol. 46, pp. 149-174, January 2022.
Details | PDF | Video
Glen Chou, Necmiye Ozay, and Dmitry Berenson
Conference on Decision and Control (CDC), December 2021.
PDF | Video
Peter Mitrano and Dmitry Berenson
Robotics Science and Systems (RSS), June 2022.
Details | Video
Thomas Power and Dmitry Berenson
Robotics Science and Systems (RSS), June 2022.
Details | Video
|
Our work in deformable object manipulation has explored planning and control methods for elastic objects as well as objects like cloth and rope. These objects have infinite-dimensional configuration spaces and are difficult to model and simulate. The key to our approach is to find appropriate model reductions that make planning and control tractable despite these challenges. We also use machine learning methods to approximate the dynamics of deformable objects and determine where our approximations are valid. The approximate dynamics models are then used in motion planning and control to accomplish manipulation tasks. |
Dmitry Berenson
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), November, 2013
PDF | Video
Calder Phillips-Grafflin and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2014.
PDF | Video
Weifu Wang, Dmitry Berenson, and Devin Balkcom
IEEE International Conference on Robotics and Automation (ICRA), May, 2015.
PDF | Video
Calder Phillips-Grafflin and Dmitry Berenson
International Symposium on Robotics Research (ISRR), September, 2015.
Dale McConachie and Dmitry Berenson
Workshop on the Algorithmic Foundations of Robotics (WAFR), December, 2016.
PDF | Video
Dale McConachie, Mengyao Ruan, and Dmitry Berenson
International Symposium on Robotics Research (ISRR), December, 2017.
PDF | Video
Mengyao Ruan, Dale McConachie, and Dmitry Berenson
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October, 2018.
PDF | Video
Cheng Chi and Dmitry Berenson
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), November 2019.
Details | PDF | Video
Dale McConachie, Thomas Power, Peter Mitrano, and Dmitry Berenson
IEEE Robotics and Automation Letters (RA-L) (Presented at ICRA 2020), vol. 5, no. 2, pp. 3540-3547, April 2020.
Details | PDF | Video
Dale McConachie, Andrew Dobson, Mengyao Ruan, and Dmitry Berenson
International Journal of Robotics Research (IJRR), vol. 39, no. 8, pp. 957-982, July 2020.
Details | Video
Dale McConachie
Ph.D. dissertation, Robotics Institute, University of Michigan, April 2020.
Thomas Power and Dmitry Berenson
IEEE Robotics and Automation Letters (RA-L) (Presented at ICRA 2021), vol. 6, no. 2, pp. 1184-1191, April 2021.
Details | PDF | Video
Yixuan Wang, Dale McConachie, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2021.
Details | PDF | Video
Peter Mitrano, Dale McConachie, and Dmitry Berenson
Science Robotics, vol. 6, no. 54, May 2021.
Details | Video
Glen Chou, Necmiye Ozay, and Dmitry Berenson
Conference on Decision and Control (CDC), December 2021.
PDF | Video
Peter Mitrano and Dmitry Berenson
Robotics Science and Systems (RSS), June 2022.
Details | Video
|
This project seeks to develop perception and planning methods that allow robots to manipulate in very cluttered environments. A key challenge for perception is the extreme occlusion and the lack of knowledge about how objects will move when manipulated, which makes it difficult to estimate/track them. We investigate shape completion and tracking methods to mitigate the effects of occlusion. For planning it is difficult to reason about occlusion without being too conservative or too reckless. We investigate ways to respond to unanticipated contacts online and to plan in the presence of uncertainty about the occupancy of the environment. |
Brad Saund and Dmitry Berenson
International Symposium on Experimental Robotics (ISER), November 2018.
PDF | Video
Andrew Price*, Linyi Jin*, and Dmitry Berenson
International Symposium on Robotics Research (ISRR), October 2019.
Details | PDF | Video
Brad Saund, Sanjiban Choudhury, Siddhartha Srinivasa, and Dmitry Berenson
International Symposium on Robotics Research (ISRR), October, 2019.
PDF | Video
Brad Saund and Dmitry Berenson
Conference on Robot Learning (CoRL), November 2020.
Details | PDF | Video
Sheng Zhong, Zhenyuan Zhang, Nima Fazeli, and Dmitry Berenson
IEEE Robotics and Automation Letters (RA-L) (Presented at ICRA 2021), vol. 6, no. 2, pp. 1447-1454, April 2021.
Details | PDF | Video
Andrew Price*, Kun Huang*, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2021.
Details | PDF | Video
Brad Saund and Dmitry Berenson
Conference on Robot Learning (CoRL), November 2021.
Details | PDF | Video
Sheng Zhong, Nima Fazeli, and Dmitry Berenson
IEEE Robotics and Automation Letters (RA-L) (Presented at ICRA 2022), vol. 7, no. 2, pp. 3507-3514, April 2022.
Details | PDF | Video
This project investigates methods to track deformable objects despite partial (self-)occlusions. Our methods are based on Coherent Point Drift (CPD) and use RGBD data to reason about the location of the deformable object in 3D. A key ability is ensuring that the tracking estimate complies with the geometric constraints of the object and environment.
Cheng Chi and Dmitry Berenson
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), November 2019.
Details | PDF | Video
Yixuan Wang, Dale McConachie, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2021.
Details | PDF | Video
|
Because of their human-like morphology and capabilities, humanoid robots offer the promise of taking the place of people operating in shipboard environments and disaster sites. For instance, a shipboard maintenance robot requires the ability to autonomously traverse environments with complex geometry and respond to significant disturbances; i.e., the robot will need to pass through narrow corridors, open doors, climb and descend ladders and stairs, and navigate through clutter. The robot must also be able to operate despite significant disturbance forces arising from e.g. ocean waves. We are developing new locomotion planning algorithms that use previous experience and integrate with perception to quickly compute effective plans for these scenarios. This project is in collaboration with the ExtReMe Lab at Virginia Tech. |
Yu-Chi Lin and Dmitry Berenson
IEEE-RAS International Conference on Humanoid Robots (Humanoids), November, 2016.
PDF | Video
Will Pryor, Yu-Chi Lin, and Dmitry Berenson
IEEE-RAS International Conference on Humanoid Robots (Humanoids), November, 2016.
PDF | Video
Yu-Chi Lin and Dmitry Berenson
IEEE-RAS International Conference on Humanoid Robots (Humanoids), November, 2017.
PDF | Video
Yu-Chi Lin and Dmitry Berenson
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October, 2018
PDF | Video
Yu-Chi Lin, Brahayam Ponton, Ludovic Righetti, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2019.
PDF | Video
Yu-Chi Lin, Ludovic Righetti, and Dmitry Berenson
IEEE Robotics and Automation Letters (RA-L) (Presented at ICRA 2020), vol. 5, no. 2, pp. 2451-2458, April 2020.
Details | PDF | Video
Yu-Chi Lin
Ph.D. dissertation, Robotics Institute, University of Michigan, April 2020.
|
This project addresses a large space of tasks that cannot be fully automated because of either the limitations of current algorithms or prohibitive cost and set-up time. Such tasks generally require humans to collaborate in close proximity and adapt to each other's decisions and motions. This project explores accomplishing these tasks through human-robot collaboration. Recent hardware developments in robotics have made human-robot collaboration physically possible, but robots still require new algorithms to ensure safety, efficiency, and fluency when working with people. Creating such algorithms is difficult because there can be high uncertainty in what a person is going to do and how they are going to do it. This project explores the integration of reasoning about how a person moves and how he or she makes decisions into a robot motion planning and decision-making framework. The research centers on the development of new algorithmic frameworks for modeling, simulating, and planning for human-robot collaboration, which requires advances in robot training, task modeling, human motion understanding, high-dimensional motion planning with uncertainty, and metrics to assess human-robot joint action. |
Jim Mainprice and Dmitry Berenson
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), November, 2013
PDF | Video
Jim Mainprice, Rafi Hayne, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2015.
PDF | Video
Ruikun Luo and Dmitry Berenson
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), September, 2015
Rafi Hayne, Ruikun Luo, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2016.
PDF | Video
Jim Mainprice, Rafi Hayne, and Dmitry Berenson
IEEE Transactions on Robotics (T-RO), Vol. 32, No. 4, pp. 897-908, August 2016.
Details | PDF
Antonio Umali and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2017.
PDF | Video
Ruikun Luo, Rafi Hayne, and Dmitry Berenson
Autonomous Robots (AuRo), Vol. 42, No. 3, pp 631-648, March 2018.
Details | PDF | Video
|
Most robotic systems today are rigid, or composed of parts that are hard and unable to flex, morph, or change other physical characteristics. This can be an issue when a rigid robotic hand is trying to grasp something irregularly shaped. These rigid designs need precise control over position, force, and other senses to successfully pick up objects. Our soft hand design builds on the work of Raphael Deimel and Oliver Brock at TU Berlin in soft pneumatic robot fingers. The idea behind a soft robotic hand is to use soft and flexible fingers that can adapt to the objects being picked up. A soft hand is far more robust in regards to the uncertainty of the objects’ placement, size, and shape. Although some certainty is still required, less attention needs to be given to the placement of the object in 3D space. A soft robotic hand can simply morph to the object's shape, and then get a firm grip on it. The purpose of this project is to research the development, effectiveness, and practicality of a soft robotic hand that can be used on robots that manipulate complex objects. The initial goal established was to construct a hand prototype that could be pneumatically controlled to pick up objects of different sizes, shapes, and weights. This is an ongoing project, however, as there are many areas for expansion. These include sensor integration, stronger fingers, and a more sophisticated control system. Now that a proof of concept has been completed, the doors have been opened for a vast range of ways to improve the quality of the hand. Source code available on GitHub. CAD for the molds and hand. Documentation for the hand.
|
|
John Morrow, Hee-Sup Shin, Calder Phillips-Grafflin, Sung-Hwan Jang, Jacob Torrey, Riley Larkins, Steven Dang, Yong-Lae Park, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2016.
PDF | Video
|
We seek to create a user-guided manipulation framework for High Degree-of-Freedom robots operating in environments with limited communication to apply to the DARPA Robotics Challenge. Our approach consists of three elements: (1) a user-guided perception interface which assists the user to provide task level commands to the robot, (2) planning algorithms that autonomously generate robot motion while obeying relevant constraints, and (3) a trajectory execution and monitoring system. We are prototyping our work on the PR2 but are applying it for the DRC on the Hubo humanoid robot. We are part of a multi-university Track A DARPA Robotics Challenge Team led by Drexel University. See a description of the challenge here. This project is joint work with the Interaction Lab (Prof. Sonia Chernova) and the HIVE lab (Prof. Rob Lindeman) at WPI. |
Nicholas Alunni, Calder Phillips-Grafflin, Halit Bener Suay, Daniel Lofaro, Dmitry Berenson, Sonia Chernova, Robert W. Lindeman, Paul Oh
IEEE International Conference on Technologies for Practical Robot Applications (TePRA), April, 2013
PDF | Video
Jim Mainprice, Calder Phillips-Grafflin, Halit Bener Suay, Nicholas Alunni, Daniel Lofaro, Dmitry Berenson, Sonia Chernova, Robert W. Lindeman, and Paul Oh
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), November, 2014
PDF | Video
Calder Phillips-Grafflin, Nicholas Alunni, Halit Bener Suay, Jim Mainprice, Daniel Lofaro, Dmitry Berenson, Sonia Chernova, Robert W. Lindeman, and Paul Oh
Journal of Intelligent Service Robotics, Vol. 7, No. 3, pp. 121-131, July 2014
Details | PDF
|
Our everyday lives are full of tasks that constrain our movement. Carrying a coffee mug, lifting a heavy object, or sliding a milk jug out of a refrigerator are examples of tasks that involve constraints imposed on our bodies as well as on the manipulated objects. Creating algorithms for general-purpose robots to perform these kinds of tasks also involves computing motions that are subject to multiple simultaneous task constraints. |
|
Task Space Regions
Some of the most common constraints in manipulation planning are constraints on the pose of a robot's end-effector. They arise in tasks such as reaching to grasp an object, carrying a cup of coffee, or opening a door. These constraints can act on the entire path, such as not spilling a cup of coffee, and/or on the goal of a path, such as reaching to grasp an object. In general a set of constraints for a manipulator's end-effector can consist of an arbitrary number of poses spread in an arbitrary way throughout the task space. However, such a broad representation lacks three fundamental properties that are necessary for sampling-based planning:
- The set of poses must be easy to specify.
- Sampling from the set of poses must be efficient.
- The distance to the set must be fast to compute.
TSRs describe end-effector pose constraints as volumes in SE(3), the space of rigid spatial transformations. These volumes are particularly useful for manipulation tasks such as reaching to grasp an object, manipulating an object with pose constraints, such as a glass of water, or placing an object onto a surface, such as table. TSRs are intuitive to specify, they can be efficiently sampled, and the distance to a TSR can be evaluated very quickly, making them ideal for sampling-based planners such as CBiRRT.
Task Space Region Chains for Whole-Body Manipulation
|
While TSRs can represent many useful constraints, a single TSR, or even a finite set of TSRs, is sometimes insufficient to capture the pose constraints of a given task. To describe more complex constraints such as closed chain kinematics and manipulating articulated objects, we use TSR Chains, which are defined by linking a series of TSRs. A TSR Chain can be thought of as a virtual manipulation whose reachability is the set of allowable poses. Though the sampling for TSR Chains follows clearly from that of TSRs, the distance metric for TSR Chains is radically different. TSR Chains allow high degree-of-freedom systems such as humanoid robots to perform constrained whole-body manipulation tasks while exploiting the redundancy of these systems. |
Planning with Pose Uncertainty
A common assumption when planning for robotic manipulation tasks is that the robot has perfect knowledge of the geometry and pose of objects in the environment. For a robot operating in a home environment it may be reasonable to have geometric models of the objects the robot manipulates frequently and/or the robot's work area. However, these objects and the robot frequently move around the environment, introducing uncertainty into the pose of the objects relative to the robot. Laser-scanners, cameras, and sonar sensors can all be used to help resolve the poses of objects in the environment, but these sensors are never perfect and usually localize the objects to be within some hypothetical probability distribution of pose estimates. If this set of probable poses is large, planning with the best hypothesis alone can be unreliable because the robot may fail to complete the task and unsafe because the robot could collide with a poorly-localized object.
Since we would like to compute a plan that is guaranteed to meet task specifications for all hypotheses of object pose, we must modify the TSRs of a given task to account for pose uncertainty and introduce virtual obstacles into the simulation to avoid potential collisions. We start by duplicating the TSRs for each pose hypothesis and computing the intersection of all duplicates. We can then sample from the 6-dimensional polytope of intersection to obtain poses that are guaranteed to obey task constraints despite pose uncertainty. A key advantage of this approach is that if the pose uncertainty is too great to accomplish a certain task, we can quickly reject that task without invoking a planner. This approach can also eliminate grasping strategies which are not robust to uncertainty.
Motion Planning Theory
We were the first to present a proof for the probabilistic completeness of RRT-based algorithms when planning with constraints on end-effector pose. If the manifold of valid configurations corresponding to the pose constraint has non-zero measure in the C-space, it is straightforward to show that algorithms like CBiRRT are probabilistically complete. This is because random sampling in the C-space will eventually place samples inside of the manifold.Â
However, pose constraints can also induce lower-dimensional constraint manifolds in the configuration space, making rejection sampling techniques infeasible. Sampling-based algorithms such as CBiRRT can overcome this problem by using the sample-project method: sampling coupled with a projection operator to move configuration space samples onto the constraint manifold. Before our work, it was not known whether the sample-project method produces adequate coverage of the constraint manifold to guarantee probabilistic completeness. Our proof guarantees probabilistic completeness for a class of RRT-based algorithms given an appropriate projection operator. This proof is valid for constraint manifolds of any fixed dimensionality.
Base Placement for Mobile Manipulators
Path planning for a mobile manipulator involves multiple levels of planning which are often divided into sub-problems to manage complexity. For a pick-and place operation the break-down might be: 1) move the robot to a configuration near the object, 2) grasp the object, 3) move the robot (holding the object) to some configuration which places the object into its goal configuration. Breaking the problem into the above sub-problems reduces the complexity of the overall task by allowing sub-plans to be generated in series. However, ignoring the coupling between sub-problems can turn feasible problems into infeasible ones and introduce unnecessary difficulty for the path planning algorithm. We show how to couple sub-problems through choosing optimal grasps and base placements for transition configurations, i.e. configurations where the robot first grasps or releases the object. We find optimal configurations (in terms of grasp quality, manipulability, and clutter) by searching a constrained space of base placements and grasps using a co-evolutionary algorithm. This algorithm significantly out-performs random sampling and is able to generate high-quality configurations in extremely cluttered environments.
Dmitry Berenson
Ph.D. dissertation, Robotics Institute, School of Computer Science, Carnegie Mellon University, June, 2011.
Details | PDF
The goal of a grasping algorithm is to find a pose and configuration of a robot's hand which grasps a given object in a desirable way. Development of grasp quality metrics, such as force-closure, has been intensely studied in robotics for many years; however these metrics neglect a key component of grasp selection: an object's environment. Our research in grasping has focused on extracting information about an object's environment to make decisions about how that object should be grasped. Considering the object's environment allows the robot to quickly eliminate hand poses that place the hand in areas where there are obstacles and to focus on areas where the hand is collision-free and where the fingers make good contact with the object. Our grasping algorithms are able to find grasps in environments where state-of-the-art algorithms cannot, while requiring only several seconds of computation. We have also extended these algorithms to consider two-handed grasps with similar results.
Dmitry Berenson and Siddhartha Srinivasa
IEEE-RAS International Conference on Humanoid Robots (Humanoids), December, 2008
Details | PDF | Video
Dmitry Berenson, Rosen Diankov, Koichi Nishiwaki, Satoshi Kagami, and James Kuffner
IEEE-RAS International Conference on Humanoid Robots (Humanoids), December, 2007
Details | PDF | Video
Faculty
PhD Students
Undergraduate Students
Alumni
- Obeying Constraints During Motion Planning
Dmitry Berenson
Humanoid Robotics: A Reference, A. Goswami and P. Vadakkepat (Eds.), Springer, 2019.
Details | PDF - Multi-finger Manipulation via Trajectory Optimization with Differentiable Rolling and Geometric Constraints
Fan Yang, Thomas Power, Sergio Aguilera, Soshi Iba, Rana Soltani Zarrin, and Dmitry Berenson
IEEE Robotics and Automation Letters (RA-L), in press. - Constraining Gaussian Process Implicit Surfaces for Robot Manipulation via Dataset Refinement
Abhinav Kumar, Peter Mitrano, and Dmitry Berenson
IEEE Robotics and Automation Letters (RA-L), vol. 9, no. 12, pp. 10994-11001, December 2024.
Details | Video - Constrained Stein Variational Trajectory Optimization
Thomas Power and Dmitry Berenson
IEEE Transactions on Robotics (T-RO), vol. 40, pp. 3602-3619, 2024.
Details | PDF | Video - Learning a Generalizable Trajectory Sampling Distribution for Model Predictive Control
Thomas Power and Dmitry Berenson
IEEE Transactions on Robotics (T-RO), vol. 40, pp. 2111-2127, 2024.
Details | Video - Motion Planning as Online Learning: A Multi-Armed Bandit Approach to Kinodynamic Sampling-Based Planning
Marco Faroni and Dmitry Berenson
IEEE Robotics and Automation Letters (RA-L), vol. 8, no. 10, pp. 6651-6658, October 2023.
Details | PDF | Video - The Blindfolded Traveler's Problem: A Search Framework for Motion Planning with Contact Estimates
Brad Saund, Sanjiban Choudhury, Siddhartha Srinivasa, and Dmitry Berenson
International Journal of Robotics Research (IJRR), Vol. 42, No. 4-5, pp. 289-309, May 2023.
Details | PDF - Soft Tracking Using Contacts for Cluttered Objects to Perform Blind Object Retrieval
Sheng Zhong, Nima Fazeli, and Dmitry Berenson
IEEE Robotics and Automation Letters (RA-L) (Presented at ICRA 2022), vol. 7, no. 2, pp. 3507-3514, April 2022.
Details | PDF | Video - Gaussian Process Constraint Learning for Scalable Chance-Constrained Motion Planning from Demonstrations
Glen Chou*, Hao Wang*, and Dmitry Berenson
IEEE Robotics and Automation Letters (RA-L) (Presented at ICRA 2022), vol. 7, no. 2, pp. 3827-3834, April 2022.
Details | PDF | Video - Challenges and Outlook in Robotic Manipulation of Deformable Objects
Jihong Zhu, Andrea Cherubini, Claire Dune, David Navarro-Alarcon, Farshid Alambeigi, Dmitry Berenson, Fanny Ficuciello, Kensuke Harada, Jens Kober, Xiang Li, Jia Pan, Wenzhen Yuan, and Michael Gienger
IEEE Robotics and Automation Magazine (RAM), vol. 29, no. 3, pp. 67-77, Sept. 2022.
Details | PDF - Learning temporal logic formulas from suboptimal demonstrations: theory and experiments
Glen Chou, Necmiye Ozay, and Dmitry Berenson
Autonomous Robots (AuRo), vol. 46, pp. 149-174, January 2022.
Details | PDF | Video - Learning where to trust unreliable models in an unstructured world for deformable object manipulation
Peter Mitrano, Dale McConachie, and Dmitry Berenson
Science Robotics, vol. 6, no. 54, May 2021.
Details | Video - Long-horizon humanoid navigation planning using traversability estimates and previous experience
Yu-Chi Lin and Dmitry Berenson
Autonomous Robots (AuRo), vol. 45, no. 6, pp. 937-956, June 2021.
Details | PDF | Video - Learning Constraints from Demonstrations with Grid and Parametric Representations
Glen Chou, Dmitry Berenson, and Necmiye Ozay
International Journal of Robotics Research (IJRR), vol. 40, no. 10-11, pp. 1255-1283, September 2021.
Details | PDF - Planning with Learned Dynamics: Probabilistic Guarantees on Safety and Reachability via Lipschitz Constants
Craig Knuth*, Glen Chou*, Necmiye Ozay, and Dmitry Berenson
IEEE Robotics and Automation Letters (RA-L) (Presented at ICRA 2021), vol. 6, no. 3, pp. 5129 - 5136, March 2021.
Details | PDF | Video - Keep it Simple: Data-efficient Learning for Controlling Complex Systems with Simple Models
Thomas Power and Dmitry Berenson
IEEE Robotics and Automation Letters (RA-L) (Presented at ICRA 2021), vol. 6, no. 2, pp. 1184-1191, April 2021.
Details | PDF | Video - TAMPC: A Controller for Escaping Traps in Novel Environments
Sheng Zhong, Zhenyuan Zhang, Nima Fazeli, and Dmitry Berenson
IEEE Robotics and Automation Letters (RA-L) (Presented at ICRA 2021), vol. 6, no. 2, pp. 1447-1454, April 2021.
Details | PDF | Video - Manipulating Deformable Objects by Interleaving Prediction, Planning, and Control
Dale McConachie, Andrew Dobson, Mengyao Ruan, and Dmitry Berenson
International Journal of Robotics Research (IJRR), vol. 39, no. 8, pp. 957-982, July 2020.
Details | Video - Learning When to Trust a Dynamics Model for Planning in Reduced State Spaces
Dale McConachie, Thomas Power, Peter Mitrano, and Dmitry Berenson
IEEE Robotics and Automation Letters (RA-L) (Presented at ICRA 2020), vol. 5, no. 2, pp. 3540-3547, April 2020.
Details | PDF | Video - Learning Constraints from Locally-Optimal Demonstrations under Cost Function Uncertainty
Glen Chou, Necmiye Ozay, and Dmitry Berenson
IEEE Robotics and Automation Letters (RA-L) (presented at ICRA 2020), vol. 5, no. 2, pp. 3682-3690, April 2020.
Details | PDF | Video - Robust Humanoid Contact Planning with Learned Zero- and One-Step Capturability Prediction
Yu-Chi Lin, Ludovic Righetti, and Dmitry Berenson
IEEE Robotics and Automation Letters (RA-L) (Presented at ICRA 2020), vol. 5, no. 2, pp. 2451-2458, April 2020.
Details | PDF | Video - Fast Planning Over Roadmaps via Selective Densification
Brad Saund and Dmitry Berenson
IEEE Robotics and Automation Letters (RA-L) (Presented at ICRA 2020), vol. 5, no. 2, pp. 2873-2880, April 2020.
Details | PDF | Video - Asymptotically Near-Optimal Methods for Kinodynamic Planning with Initial State Uncertainty
Kaiwen Liu, Yang Zhang, Andrew Dobson, and Dmitry Berenson
IEEE Robotics and Automation Letters (RA-L), Vol. 4, No. 2, pp. 2124-2131, April 2019.
Details | PDF - Simultaneous learning of hierarchy and primitives for complex robot tasks
Anahita Mohseni-Kabir, Changshuo Li, Victoria Wu, Daniel Miller, Benjamin Hylak, Sonia Chernova, Dmitry Berenson, Candace Sidner, Charles Rich
Autonomous Robots (AuRo), Vol. 43, No. 4, pp. 859-874, April 2019.
Details - Estimating Model Utility for Deformable Object Manipulation Using Multi-Armed Bandit Methods
Dale McConachie and Dmitry Berenson
IEEE Transactions on Automation Science and Engineering (T-ASE), Vol. 15, No. 3, pp. 967-979, July 2018.
Details - Unsupervised Early Prediction of Human Reaching for Human-robot Collaboration in Shared Workspaces
Ruikun Luo, Rafi Hayne, and Dmitry Berenson
Autonomous Robots (AuRo), Vol. 42, No. 3, pp 631-648, March 2018.
Details | PDF | Video - Analysis and Observations From the First Amazon Picking Challenge
Nikolaus Correll, Kostas E. Bekris, Dmitry Berenson, Oliver Brock, Albert Causo, Kris Hauser, Kei Okada, Alberto Rodriguez, Joseph M. Romano, and Peter R. Wurman
IEEE Transactions on Automation Science and Engineering (T-ASE), Vol. 15, No. 1, pp. 172 - 188, January 2018.
Details - Team WPI-CMU: Achieving Reliable Humanoid Behavior in the DARPA Robotics Challenge
Mathew DeDonato, Felipe Polido, Kevin Knoedler, Benzun P. W. Babu, Nandan Banerjee, Christoper P. Bove, Xiongyi Cui, Ruixiang Du, Perry Franklin, Joshua P. Graff, Peng He, Aaron Jaeger, Lening Li, Dmitry Berenson, Michael A. Gennert, Siyuan Feng, Chenggang Liu, X Xinjilefu, Joohyung Kim, Christopher G. Atkeson, Xianchao Long and Taskin Padir
Journal of Field Robotics (JFR), Vol. 34, No. 2, pp. 381-399, March 2017.
Details - Goal Set Inverse Optimal Control and Iterative Re-planning for Predicting Human Reaching Motions in Shared Workspaces
Jim Mainprice, Rafi Hayne, and Dmitry Berenson
IEEE Transactions on Robotics (T-RO), Vol. 32, No. 4, pp. 897-908, August 2016.
Details | PDF - From Autonomy to Cooperative Traded Control of Humanoid Manipulation Tasks with Unreliable Communication
Calder Phillips-Grafflin, Halit Bener Suay, Jim Mainprice, Nicholas Alunni, Daniel Lofaro, Dmitry Berenson, Sonia Chernova, Robert W. Lindeman, and Paul Oh
Journal of Intelligent and Robotic Systems (JINT), Vol. 82, No. 3, pp. 341-361, June 2016.
Details | PDF - Toward a user-guided manipulation framework for high-DOF robots with limited communication
Calder Phillips-Grafflin, Nicholas Alunni, Halit Bener Suay, Jim Mainprice, Daniel Lofaro, Dmitry Berenson, Sonia Chernova, Robert W. Lindeman, and Paul Oh
Journal of Intelligent Service Robotics, Vol. 7, No. 3, pp. 121-131, July 2014
Details | PDF - Robot-Guided Open-Loop Insertion of Skew-Line Needle Arrangements for High Dose Rate Brachytherapy
Animesh Garg, Timmy Siauw, Dmitry Berenson, J. Adam M. Cunha, I-Chow Hsu, Jean Pouliot, Dan Stoianovici, and Ken Goldberg
IEEE Transactions on Automation Science and Engineering (T-ASE), Vol. 10, No. 4, October 2013
Details | PDF - NPIP: A Skew Line Needle Configuration Optimization System for HDR Brachytherapy
Timmy Siauw, Adam Cunha, Dmitry Berenson, Alper Atamturk, I-Chow Hsu, Ken Goldberg, and Jean Pouliot
Medical Physics, Volume 39, Number 7, pp. 4339 - 4346, July, 2012.
Details | PDF - HERB 2.0: Lessons Learned from Developing a Mobile Manipulator for the Home
Siddhartha S. Srinivasa, Dmitry Berenson, Maya Cakmakz, Alvaro Collet, Mehmet R. Dogar, Anca D. Dragan, Ross A. Knepper, Tim Niemueller, Kyle Strabala, Mike Vande Weghe, Julius Ziegler
Proceedings of the IEEE, Vol. 100, No. 8, pp. 2410 - 2428, July, 2012.
Details | PDF - Task Space Regions: A Framework for Pose-Constrained Manipulation Planning
Dmitry Berenson, Siddhartha Srinivasa, and James Kuffner
International Journal of Robotics Research (IJRR), Volume 30, Number 12, pp. 1435 - 1460, October, 2011.
Details | PDF - HERB: A Home Exploring Robotic Butler
Siddhartha Srinivasa, David Ferguson, Casey Helfrich, Dmitry Berenson, Alvaro Collet Romea, Rosen Diankov, Garratt Gallagher, Geoffrey Hollinger, James Kuffner, and J Michael Vandeweghe
Autonomous Robots (AuRo), Vol. 28, No. 1, pp. 5-20, January, 2010.
Details | PDF - Implicit Contact Diffuser: Sequential Contact Reasoning with Latent Point Cloud Diffusion
Zixuan Huang, Yinong He, Yating Lin, Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2025. - Language-Guided Object Search in Agricultural Environments
Advaith Balaji, Saket Pradhan, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2025. - Diffusion-Informed Probabilistic Contact Search for Multi-Finger Manipulation
Abhinav Kumar, Thomas Power, Fan Yang, Sergio Aguilera, Soshi Iba, Rana Soltani Zarrin, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2025. - Quantifying Aleatoric and Epistemic Dynamics Uncertainty via Local Conformal Calibration
Luís Marques and Dmitry Berenson
Workshop on the Algorithmic Foundations of Robotics (WAFR), October 2024.
Details - Subgoal Diffuser: Coarse-To-Fine Subgoal Generation to Guide Model Predictive Control for Robot Manipulation
Zixuan Huang, Yating Lin, Fan Yang, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2024.
Details | Video - The Grasp Loop Signature: A Topological Representation for Manipulation Planning with Ropes and Cables
Peter Mitrano and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2024.
Details | Video - Improving Out-Of-Distribution Generalization of Learned Dynamics by Learning Pseudometrics and Constraint Manifolds
Yating Lin, Glen Chou, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2024.
Details | Video - Online Adaptation of Sampling-Based Motion Planning with Inaccurate Models
Marco Faroni and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2024.
Details | Video - TactileVAD: Geometric Aliasing-Aware Dynamics for High-Resolution Tactile Control
Miquel Oller, Dmitry Berenson, and Nima Fazeli
Conference on Robot Learning (CoRL), November 2023.
Details | PDF | Video - CHSEL: Producing Diverse Plausible Pose Estimates from Contact and Free Space Data
Sheng Zhong, Nima Fazeli, and Dmitry Berenson
Robotics: Science and Systems (RSS), July 2023.
Details | PDF | Video - Integrated Object Deformation and Contact Patch Estimation from Visuo-Tactile Feedback
Mark Van der Merwe, Youngsun Wi, Dmitry Berenson, Nima Fazeli
Robotics: Science and Systems (RSS), July 2023.
Details | PDF - Focused Adaptation of Dynamics Models for Deformable Object Manipulation
Peter Mitrano, Alex LaGrassa, Oliver Kroemer, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2023.
Details | PDF | Video - Data-Efficient Learning of Natural Language to Linear Temporal Logic Translators for Robot Task Specification
Jiayi Pan, Glen Chou, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2023.
Details | PDF | Video - Learning the Dynamics of Compliant Tool-Environment Interaction for Visuo-Tactile Contact Servoing
Mark Van der Merwe, Dmitry Berenson, and Nima Fazeli
Conference on Robot Learning (CoRL), December 2022.
Details | Video - Manipulation via Membranes: High-Resolution and Highly Deformable Tactile Sensing and Control
Miquel Oller, Mireia Planas, Dmitry Berenson, and Nima Fazeli
Conference on Robot Learning (CoRL), December 2022.
Details | Video - Data Augmentation for Manipulation
Peter Mitrano and Dmitry Berenson
Robotics Science and Systems (RSS), June 2022.
Details | Video - Variational Inference MPC using Normalizing Flows and Out-of-Distribution Projection
Thomas Power and Dmitry Berenson
Robotics Science and Systems (RSS), June 2022.
Details | Video - Safe Output Feedback Motion Planning from Images via Learned Perception Modules and Contraction Theory
Glen Chou, Necmiye Ozay, and Dmitry Berenson
Workshop on the Algorithmic Foundations of Robotics (WAFR), June 2022.
Details | Video - Model Error Propagation via Learned Contraction Metrics for Safe Feedback Motion Planning of Unknown Systems
Glen Chou, Necmiye Ozay, and Dmitry Berenson
Conference on Decision and Control (CDC), December 2021.
PDF | Video - CLASP: Constrained Latent Shape Projection for Refining Object Shape from Robot Contact
Brad Saund and Dmitry Berenson
Conference on Robot Learning (CoRL), November 2021.
Details | PDF | Video - Tracking Partially-Occluded Deformable Objects While Enforcing Geometric Constraints
Yixuan Wang, Dale McConachie, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2021.
Details | PDF | Video - Fusing RGBD Tracking and Segmentation Tree Sampling for Multi-Hypothesis Volumetric Segmentation
Andrew Price*, Kun Huang*, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2021.
Details | PDF | Video - Diverse Plausible Shape Completions from Ambiguous Depth Images
Brad Saund and Dmitry Berenson
Conference on Robot Learning (CoRL), November 2020.
Details | PDF | Video - Uncertainty-Aware Constraint Learning for Adaptive Safe Motion Planning from Demonstrations
Glen Chou, Necmiye Ozay, and Dmitry Berenson
Conference on Robot Learning (CoRL), November 2020.
Details | PDF | Video - Explaining Multi-stage Tasks by Learning Temporal Logic Formulas from Suboptimal Demonstrations
Glen Chou, Necmiye Ozay, and Dmitry Berenson
Robotics: Science and Systems (RSS), July 2020.
Details | PDF | Video - Inferring Obstacles and Path Validity from Visibility-Constrained Demonstrations
Craig Knuth, Glen Chou, Necmiye Ozay, and Dmitry Berenson
Workshop on the Algorithmic Foundations of Robotics (WAFR), 2020.
Details - Occlusion-robust Deformable Object Tracking without Physics Simulation
Cheng Chi and Dmitry Berenson
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), November 2019.
Details | PDF | Video - Learning Parametric Constraints in High Dimensions from Demonstrations
Glen Chou, Necmiye Ozay, and Dmitry Berenson
Conference on Robot Learning (CoRL), October, 2019.
Details | PDF - Inferring Occluded Geometry Improves Performance when Retrieving an Object from Dense Clutter
Andrew Price*, Linyi Jin*, and Dmitry Berenson
International Symposium on Robotics Research (ISRR), October 2019.
Details | PDF | Video - The Blindfolded Robot: A Bayesian Approach to Planning with Contact Feedback
Brad Saund, Sanjiban Choudhury, Siddhartha Srinivasa, and Dmitry Berenson
International Symposium on Robotics Research (ISRR), October, 2019.
PDF | Video - Efficient Humanoid Contact Planning using Learned Centroidal Dynamics Prediction
Yu-Chi Lin, Brahayam Ponton, Ludovic Righetti, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2019.
PDF | Video - Learning Constraints from Demonstrations
Glen Chou, Dmitry Berenson, Necmiye Ozay
Workshop on the Algorithmic Foundations of Robotics (WAFR), December, 2018.
PDF - Motion Planning for Manipulators in Unknown Environments with Contact Sensing Uncertainty
Brad Saund and Dmitry Berenson
International Symposium on Experimental Robotics (ISER), November 2018.
PDF | Video - Accounting for Directional Rigidity and Constraints in Control for Manipulation of Deformable Objects without Physical Simulation
Mengyao Ruan, Dale McConachie, and Dmitry Berenson
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October, 2018.
PDF | Video - Humanoid Navigation Planning in Large Unstructured Environments Using Traversability-Based Segmentation
Yu-Chi Lin and Dmitry Berenson
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October, 2018
PDF | Video - Incremental Segmentation of ARX Models
Glen Chou, Necmiye Ozay, and Dmitry Berenson
IFAC Symposium on System Identification (SYSID), July, 2018.
PDF - Interleaving Planning and Control for Deformable Object Manipulation
Dale McConachie, Mengyao Ruan, and Dmitry Berenson
International Symposium on Robotics Research (ISRR), December, 2017.
PDF | Video - Humanoid Navigation in Uneven Terrain using Learned Estimates of Traversability
Yu-Chi Lin and Dmitry Berenson
IEEE-RAS International Conference on Humanoid Robots (Humanoids), November, 2017.
PDF | Video - A Framework For Robot-Assisted Doffing of Personal Protective Equipment
Antonio Umali and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2017.
PDF | Video - Bandit-Based Model Selection for Deformable Object Manipulation
Dale McConachie and Dmitry Berenson
Workshop on the Algorithmic Foundations of Robotics (WAFR), December, 2016.
PDF | Video - Planning and Resilient Execution of Policies for Manipulation in Contact with Actuation Uncertainty
Calder Phillips-Grafflin and Dmitry Berenson
Workshop on the Algorithmic Foundations of Robotics (WAFR), December, 2016.
PDF | Video - Using Previous Experience for Humanoid Navigation Planning
Yu-Chi Lin and Dmitry Berenson
IEEE-RAS International Conference on Humanoid Robots (Humanoids), November, 2016.
PDF | Video - Integrated Affordance Detection and Humanoid Locomotion Planning
Will Pryor, Yu-Chi Lin, and Dmitry Berenson
IEEE-RAS International Conference on Humanoid Robots (Humanoids), November, 2016.
PDF | Video - Learning Object Orientation Constraints and Guiding Constraints for Narrow Passages from One Demonstration
Changshuo Li and Dmitry Berenson
International Symposium on Experimental Robotics (ISER), October, 2016.
PDF | Video - Improving Soft Pneumatic Actuator Fingers through Integration of Soft Sensors, Position and Force Control, and Rigid Fingernails
John Morrow, Hee-Sup Shin, Calder Phillips-Grafflin, Sung-Hwan Jang, Jacob Torrey, Riley Larkins, Steven Dang, Yong-Lae Park, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2016.
PDF | Video - Considering Avoidance and Consistency in Motion Planning for Human-Robot Manipulation in a Shared Workspace
Rafi Hayne, Ruikun Luo, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2016.
PDF | Video - Reproducing Expert-Like Motion in Deformable Environments Using Active Learning and IOC
Calder Phillips-Grafflin and Dmitry Berenson
International Symposium on Robotics Research (ISRR), September, 2015.
PDF - A Framework for Unsupervised Online Human Reaching Motion Recognition and Early Prediction
Ruikun Luo and Dmitry Berenson
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), September, 2015
PDF - Predicting Human Reaching Motion in Collaborative Tasks Using Inverse Optimal Control and Iterative Re-planning
Jim Mainprice, Rafi Hayne, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2015.
PDF | Video - An Online Method for Tight-tolerance Insertion Tasks for String and Rope
Weifu Wang, Dmitry Berenson, and Devin Balkcom
IEEE International Conference on Robotics and Automation (ICRA), May, 2015.
PDF | Video - From Autonomy to Cooperative Traded Control of Humanoid Manipulation Tasks with Unreliable Communication: System Design and Lessons Learned
Jim Mainprice, Calder Phillips-Grafflin, Halit Bener Suay, Nicholas Alunni, Daniel Lofaro, Dmitry Berenson, Sonia Chernova, Robert W. Lindeman, and Paul Oh
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), November, 2014
PDF | Video - A Representation Of Deformable Objects For Motion Planning With No Physical Simulation
Calder Phillips-Grafflin and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2014.
PDF | Video - Human-Robot Collaborative Manipulation Planning Using Early Prediction of Human Motion
Jim Mainprice and Dmitry Berenson
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), November, 2013
PDF | Video - Manipulation of Deformable Objects Without Modeling and Simulating Deformation
Dmitry Berenson
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), November, 2013
PDF | Video - Toward A User-Guided Manipulation Framework for High-DOF Robots with Limited Communication
Nicholas Alunni, Calder Phillips-Grafflin, Halit Bener Suay, Daniel Lofaro, Dmitry Berenson, Sonia Chernova, Robert W. Lindeman, Paul Oh
IEEE International Conference on Technologies for Practical Robot Applications (TePRA), April, 2013
PDF | Video - Initial Experiments toward Automated Robotic Implantation of Skew-Line Needle Arrangements for HDR Brachytherapy
Animesh Garg, Timmy Siauw, Dmitry Berenson, Adam Cunha, I-Chow Hsu, Jean Pouliot, Dan Stoianovici, Ken Goldberg
IEEE International Conference on Automation Science and Engineering (CASE), August, 2012. Best Application Paper Award
PDF - Estimating Part Tolerance Bounds Based on Adaptive Cloud-Based Grasp Planning with Slip
Ben Kehoe, Dmitry Berenson, and Ken Goldberg
IEEE International Conference on Automation Science and Engineering (CASE), August, 2012.
PDF - A Constraint-Aware Motion Planning Algorithm for Robotic Folding of Clothes
Karthik Lakshmanan, Apoorva Sachdev, Ziang Xie, Dmitry Berenson, Ken Goldberg, and Pieter Abbeel
International Symposium on Experiment Robotics (ISER), June, 2012.
PDF | Video - A Robot Path Planning Framework that Learns from Experience
Dmitry Berenson, Pieter Abbeel, and Ken Goldberg
IEEE International Conference on Robotics and Automation (ICRA), May, 2012.
Details | PDF | Video - Constellation - An Algorithm for Finding Robot Configurations that Satisfy Multiple Constraints
Peter Kaiser, Dmitry Berenson, Nikolaus Vahrenkamp, Tamim Asfour, Rudiger Dillmann, Siddhartha Srinivasa
IEEE International Conference on Robotics and Automation (ICRA), May, 2012.
PDF - Toward Cloud-Based Grasping with Uncertainty in Shape: Estimating Lower Bounds on Achieving Force Closure with Zero-Slip Push Grasps
Ben Kehoe, Dmitry Berenson, and Ken Goldberg
IEEE International Conference on Robotics and Automation (ICRA), May, 2012.
PDF - Addressing Cost-Space Chasms for Manipulation Planning
Dmitry Berenson, Thierry Simeon, Siddhartha Srinivasa
IEEE International Conference on Robotics and Automation (ICRA), May, 2011.
Details | PDF | Video - People Helping Robots Helping People: Crowdsourcing for Grasping Novel Objects
Alexander Sorokin, Dmitry Berenson, Siddhartha Srinivasa, and Martial Hebert
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October, 2010
Details | PDF | Video - Probabilistically Complete Planning with End-Effector Pose Constraints
Dmitry Berenson and Siddhartha Srinivasa
IEEE International Conference on Robotics and Automation (ICRA), May, 2010
Details | PDF - Pose-Constrained Whole-Body Planning using Task Space Region Chains
Dmitry Berenson, Joel Chestnutt, Siddhartha Srinivasa, James Kuffner, and Satoshi Kagami
IEEE-RAS International Conference on Humanoid Robots (Humanoids), December, 2009
Details | PDF | Video - Addressing Pose Uncertainty in Manipulation Planning Using Task Space Regions
Dmitry Berenson, Siddhartha Srinivasa, and James Kuffner
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October, 2009
Details | PDF - Humanoid Motion Planning for Dual-Arm Manipulation and Re-Grasping Tasks
Nikolaus Vahrenkamp, Dmitry Berenson, Tamim Asfour, James Kuffner, and Rudiger Dillmann
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October, 2009
Details | PDF - Manipulation Planning on Constraint Manifolds
Dmitry Berenson, Siddhartha Srinivasa, David Ferguson, and James Kuffner
IEEE International Conference on Robotics and Automation (ICRA), May, 2009
Details | PDF | Video - Manipulation Planning with Workspace Goal Regions
Dmitry Berenson, Siddhartha Srinivasa, David Ferguson, Alvaro Collet Romea, and James Kuffner
IEEE International Conference on Robotics and Automation (ICRA), May, 2009
Details | PDF - Object Recognition and Full Pose Registration from a Single Image for Robotic Manipulation
Alvaro Collet Romea, Dmitry Berenson, Siddhartha Srinivasa, and David Ferguson
IEEE International Conference on Robotics and Automation (ICRA), May, 2009
Details | PDF - Grasp Synthesis in Cluttered Environments for Dexterous Hands
Dmitry Berenson and Siddhartha Srinivasa
IEEE-RAS International Conference on Humanoid Robots (Humanoids), December, 2008
Details | PDF | Video - The Robotic Busboy: Steps Towards Developing a Mobile Robotic Home Assistant
Siddhartha Srinivasa, David Ferguson, J Michael Vandeweghe, Rosen Diankov, Dmitry Berenson, Casey Helfrich, and Hauke Strasdat
International Conference on Intelligent Autonomous Systems (IAS), July, 2008
Details | PDF - An Optimization Approach to Planning for Mobile Manipulation
Dmitry Berenson, Howie Choset, and James Kuffner
IEEE International Conference on Robotics and Automation (ICRA), May, 2008
Details | PDF - Grasp Planning in Complex Scenes
Dmitry Berenson, Rosen Diankov, Koichi Nishiwaki, Satoshi Kagami, and James Kuffner
IEEE-RAS International Conference on Humanoid Robots (Humanoids), December, 2007
Details | PDF | Video - Hardware Evolution of Analog Circuits for In-situ Robotic Fault-Recovery
Dmitry Berenson, Nicholas Esteves, and Hod Lipson
NASA/DoD Conference on Evolvable Hardware, June, 2005
Details | PDF | Video - A Demonstration of Planar Dragging of a Hose with Obstacles
Peter Mitrano, Alison Ryckman, and Dmitry Berenson
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Demonstration Sessions, October 2023.
PDF | Video - SLHAP: Simultaneous Learning of Hierarchy and Primitives
Anahita Mohseni-Kabir, Changshuo Li, Victoria Wu, Daniel Miller, Benjamin Hylak, Sonia Chernova, Dmitry Berenson, Candace Sidner, and Charles Rich
ACM/IEEE International Conference on Human-Robot Interaction (HRI) Videos, March, 2017.
PDF | Video - DARPA Robotics Challenge: Towards a User-Guided Manipulation Framework for High-DOF Robots
Nicholas Alunni, Halit Bener Suay, Calder Phillips-Grafflin, Jim Mainprice, Dmitry Berenson, Sonia Chernova, Robert W. Lindeman, Daniel Lofaro, and Paul Oh
IEEE International Conference on Robotics and Automation (ICRA) Video Proceedings, May, 2014.
PDF | Video - Improving Path Execution in Deformable Environments Using Reactive Cost-space Control
Calder Phillips-Grafflin and Dmitry Berenson
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Workshop on Robot Manipulation: What Has Been Achieved and What Remains to Be Done?, Chicago, September 2014.
PDF - Using Task Symmetry for Human-Robot Collaborative Manipulation of Deformable Objects Without Modeling Deformation
Dmitry Berenson
EEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Workshop on Cognitive Surgical Robotics: From Virtual Fixtures to Advanced Cooperative Control, November, 2013. Best Poster Award
PDF - Grasp Synthesis in Cluttered Environments for Dexterous Hands
Dmitry Berenson and Siddhartha Srinivasa
Robotics Science and Systems (RSS) Workshop on Robot Manipulation: Intelligence in Human Environments, June, 2008
Details | PDF - Contact-based Perception and Planning for Robotic Manipulation in Novel Environments
Sheng Zhong
Ph.D. dissertation, Robotics Department, University of Michigan, July 2024.
PDF - Learning and Inference for Adaptable Manipulation Planning
Thomas Power
Ph.D. dissertation, Robotics Department, University of Michigan, June 2024.
PDF - Data-Efficient Robotic Manipulation of Deformable One-dimensional Objects with Unreliable Dynamics
Peter Mitrano
Ph.D. dissertation, Robotics Department, University of Michigan, December 2023.
PDF - Safe End-to-end Learning-based Robot Autonomy via Integrated Perception, Planning, and Control
Glen Chou
Ph.D. dissertation, EECS Department, University of Michigan, August 2022.
Details | PDF - Belief Representations for Planning with Contact Uncertainty
Brad Saund
Ph.D. dissertation, Robotics Institute, University of Michigan, July 2021.
PDF - Deformable Object Manipulation: Learning While Doing
Dale McConachie
Ph.D. dissertation, Robotics Institute, University of Michigan, April 2020.
PDF - Learning for Humanoid Multi-Contact Navigation Planning
Yu-Chi Lin
Ph.D. dissertation, Robotics Institute, University of Michigan, April 2020.
PDF - Enabling Motion Planning and Execution for Tasks Involving Deformation and Uncertainty
Calder Phillips-Grafflin
Ph.D. dissertation, Robotics Engineering, Worcester Polytechnic Institute, June, 2017.
PDF - Constrained Manipulation Planning
Dmitry Berenson
Ph.D. dissertation, Robotics Institute, School of Computer Science, Carnegie Mellon University, June, 2011.
Details | PDF
Book Chapters
Refereed Journal Papers
Refereed Conference Papers
Refereed Demonstrations and Videos
Refereed Workshop Papers
Theses
Visit usWe are happy to arrange visits to the lab for K-12 student groups during the school year or summer. Please contact Prof. Dmitry Berenson (dmitryb [at] umich.edu) to arrange a visit. Where is the ARM lab?Our lab is located at 2140 Ford Motor Company Robotics Building (FMCRB) on North Campus, University of Michigan, Ann Arbor, MI. Please see this map for the location of the building. |
 |
See photos from recent outreach events here.
A kinematics library that uses PyTorch to compute forward kinematics for multiple configurations in parallel. The computation is differentiable and the library loads robot descriptions from the URDF, SDF, and MJCF formats.
This repository implements Model Predictive Path Integral (MPPI) with approximate dynamics in pytorch.
The latest version of our deformable object tracker, CDCPD2, is now available here:
https://github.com/UM-ARM-Lab/cdcpd/tree/CDCPD2
it implements the method from this paper:
Tracking Partially-Occluded Deformable Objects while Enforcing Geometric Constraints
Yixuan Wang, Dale McConachie, and Dmitry Berenson
IEEE International Conference on Robotics and Automation (ICRA), May, 2021.
Details | PDF | Video
The original CDCPD is an implementation of
Occlusion-robust Deformable Object Tracking without Physics Simulation
Cheng Chi and Dmitry Berenson
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), November 2019.
This library includes object-oriented API for tracker, visualization utilities and a ROS node that subscribes and publishes PointCloud2.
Given an environment specified as a set of polygonal surface and a robot model, the code generates contact sequences for a humanoid robot using both arms and legs. We provide an example with the Escher humanoid robot model. The code is written in Python 2.7 and tested in Ubuntu 14.04 with ROS Indigo
https://github.com/UM-ARM-Lab/Traversability-Based-Contact-Space-Planner
Our open source CAD repository is here: https://github.com/UM-ARM-Lab/CAD/
Visit our Github page to get the latest code developed in the lab. Watch out, it's a jungle in there!
The Datalink Toolkit is a ROS package designed for remote operation of a robot over a high-latency and low-bandwidth datalink. The package allows the user to easily set up relays and compression methods for a single-master system. These relays avoid duplicating data sent over the datalink while compressing common datatypes (i.e. point-clouds and images) to minimize bandwidth usage. The package was developed and extensively tested as part of the DARPA robotics challenge, though it is not specific to a type of robot.
The package allows the user to easily set up relays and compression methods for a single-master system. These relays avoid duplicating data sent over the datalink while compressing common datatypes (i.e. point-clouds and images) to minimize bandwidth usage.
The toolkit includes both message-based and service-based relays so that data can be sent on-demand or at a specified frequency. The service-based relays are more robust in low-bandwidth conditions, guaranteeing the synchronization of camera images and camera info messages, and allow more reconfiguration while running.
The key features of the package are:
- Generic relays with integrated rate throttling for all message types
- Dedicated relays with rate throttling for images and pointclouds
- Generic service-based relays with integrated rate throttling for all message types
- Dedicated service-based relays with integrated rate throttling for images and pointclouds
- Image resizing and compression using methods from OpenCV and image_transport
- Pointcloud voxel filtering and compression using methods from PCL, Zlib, and other algorithms. (Note: pointcloud compression is provided in a separate library that can be easily integrated with other projects)
- Launch files for easy use of the datalink software with RGBD cameras
- Works with ROS Hydro
For more information, please see the wiki.
Get the package from our git repository.
CoMPS is implemented in C++ and compiles in linux only. There are also several examples in python and matlab that show how to interface with openrave to use the plugins in CoMPS.
This package is available on SourceForge.
LightningROS is a ROS package implementing the Lightning Path Planning Framework. This approach uses a path library to store previous experience while allowing generality by also planning from scratch. Please see this paper for more details.
This package uses OMPL planners to implement each component in Lightning and can be called the same way as any other OMPL planner.
This package is available for ROS Feurte here. For later versions of ROS, see here.
Github archive of projects at WPI 2012-2016.