If you have trouble reading the article because of paywall, please use the link from my personal blog here

This article contains details on sim2real in robotic manipulation for following tasks:

  • Perception for manipulation (DOPE / SD-MaskRCNN).
  • Grasping (Dex-Net 3.0 / 6DOF GraspNet).
  • End-to-end policies. (Contact rich manipulation tasks & In hand manipulation of rubik’s cube)
  • Guided domain randomization techniques (ADR / Sim-Opt).

The reality gap:

An increasingly impressive skills have been mastered by DeepRL algorithms over the years in simulation (DQN / AlphaGo / OpenAI Five). Both Deep learning and RL algorithms require super huge amounts of data. Moreover, RL algorithms there…


If you have trouble reading the article because of paywall, please use the link from my personal blog here

Bonus: Also contains details on how to generate labels and learn semantic instance segmentation.

This article contains descriptions of:

  • How to efficiently generate labels for 3D key-points and instance segmentation ?
  • How to learn 3D key-points and instance segmentation ?
  • How to use 3D key-points in pick and place / contact rich manipulation tasks ?

Why learn 3D key points ?

Consider the problem of hanging any mug by handle on a rack. How can we go about solving this task using robotic manipulator ?

Task of hanging any mug on a rack. Illustration of specifying this task using semantic 3D key points. Notice how yellow key point is at center of handle for different cup variations. PC: k-PAM[5]

There are…


Picture Credit: https://samchivers.com/Nature-Magazine-Robot-Hands
Picture Credit: https://samchivers.com/Nature-Magazine-Robot-Hands
Picture Credit: https://samchivers.com/Nature-Magazine-Robot-Hands

If you have trouble reading the article because of paywall, please use the link from my personal blog here

This article contains heuristics for following:

  • Parallel jaw grasps.
  • Suction grasps.
  • Linear push policies for improving parallel jaw grasps.
  • Toppling policies for improving suction grasps.

Grasping is one of the fundamental subtask of a robotic manipulation pipeline. Both learning based and physics / geometry based grasping methods can benefit from grasp sampling heuristics in this article. Even if you are using a large arm farm to teach your robots the skills of grasping, you can save your robots quite a lot…

Darshan Hegde

Robot Learning Engineer working on Robotic Manipulation. For more: https://darshanhegde.github.io/

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store