Vision-Language Autonomous Agent for Robotic Tabletop Sorting
Open-vocabulary object detection and natural language command parsing guiding a robotic arm to pick and sort novel items on camera.
Project Overview
Combines Grounding DINO zero-shot object detection with Segment Anything Model (SAM). Users can speak or type commands like 'Pick up the green screwdriver and place it in the red bin'; the system segments the object mask and computes 3D grasp coordinates for the robotic arm.
Combines Grounding DINO zero-shot object detection with Segment Anything Model (SAM). Users can speak or type commands like 'Pick up the green screwdriver and place it in the red bin'; the system segments the object mask and computes 3D grasp coordinates for the robotic arm.
Open-vocabulary object detection and natural language command parsing guiding a robotic arm to pick and sort novel items on camera.