Project Summary
The initial goal of this project was to program a 4 degree of freedom robotic arm to pick up and sorts balls by color. Additionally, my group and I developed the image processing pipeline to recognize when the workspace is being obstructed, and wait for any motion to subside before moving to the next ball. Even further, we allowed to recognize and remove stationary objects in the workspace.
Motion
In order to plan the motions of the robotic arm, we need to perform inverse and forward kinematics to find what joint angles are necessary to get to any given location in 3D space. Using the DH convention, we assigned reference frames for each joint. Using transformations between reference frames, we could calculate the forward kinematics of the arm. We calculated the formula for the inverse kinematics of the arm by hand, which can be used to solve for required joint angles given a set of world coordinates, and a target angle of the end effector.
Perception
In order to recognize balls and other objects, our image processing pipeline consists of many steps. We initially undistort and then crop the image. Then we apply a binary mask to recognize a specific color, or the difference between the current image and an empty workspace. Finally we apply a median filter to the result. In order to extract information, we use blob detection, and perform calculations to take pixel coordinates back to world coordinates.
