Earlier this month NVIDIA opened its new robotics research lab in Seattle near the University of Washington campus and has released a few videos teasing some of the projects currently under development within the lab. The lab has been created to “drive breakthrough robotics research to enable the next generation of robots that perform complex manipulation tasks to safely work alongside humans and transform industries such as manufacturing, logistics, healthcare, and more.” Around 50 research scientists, faculty visitors, and student interns will perform research within the robotics lab. check out the videos below to learn more about the NVIDIA Robotics lab.
“What makes the system unique is that it integrates a suite of cutting-edge technologies developed by the lab researchers. These technologies working together enable detection of objects, tracking the position of doors and drawers, and generate control commands so that the robot can grasp and move objects from one place to another.”
The system is comprised and built on the following technologies:
– Dense Articulated Real-Time Tracking (DART): DART, which was first developed in Fox’s UW robotics lab, uses depth cameras to keep track of a robot’s environment. It is a general framework for tracking rigid objects, such as coffee mugs and cereal boxes, and articulated objects often encountered in indoor environments, like furniture and tools, as well as human and robot bodies including hands and manipulators.
– Pose-CNN: 6D Object Pose Estimation: Detecting the 6D pose and orientation of known objects is a crucial capability for robots that pick up and move objects in an environment. This problem is challenging due to changing lighting conditions and complex scenes caused by clutter and occlusions between objects. Pose-CNN is a deep neural network trained to detect objects using regular cameras.
– Riemannian Motion Policies (RMPs) for Reactive Manipulator Control: RMPs are a new mathematical framework that consistently combines a library of simple actions into complex behavior. RMPs allow the team to efficiently program fast, reactive controllers that use the detection and tracking information from Pose-CNN and DART to safely interact with objects and humans in dynamic environments.
– Physics-based Photorealistic Simulation: NVIDIA’s Isaac Sim tool enables the generation of realistic simulation environments that model the visual properties of objects as well as the forces and contacts between objects and manipulators. A simulated version of the kitchen is used to test the manipulation system and train the object detection network underlying Pose-CNN. If done on a real robot this training and development process would be an expensive and time-consuming process. Once simulation models of objects and the environment are available, training and testing can be done far more efficiently, saving precious development time.
Source: Adafruit
