Robot Arm — Automated Poker Chip Sorting

Overview

In this Design-Based Learning project (4GB20, Q4 2024) at TU Eindhoven, our team of 7 programmed and controlled a 3-DOF robot arm to autonomously sort and stack colored poker chips from a moving conveyor belt. The system integrates computer vision for chip detection, inverse kinematics for motion planning, feedback/feedforward control for precision, and a Stateflow state machine for task coordination.

System Architecture

The robot arm system consists of four main subsystems:

1. Computer Vision

A camera detects poker chips on the conveyor belt using color masks (red, yellow, blue). The vision system outputs XY coordinates and a color classification integer to the control system. Detection rate is constrained to under 5 Hz.

2. Inverse Kinematics

Given desired XY coordinates from the camera and a Z height for pickup/placement, the inverse kinematics block computes the required joint angles for the 3 motors (R, X, Z axes).

3. Controller Design

Each axis has its own feedback + feedforward controller, designed using frequency-domain methods:

• System identification via FRF measurement
• Loop shaping in ShapeIt (MATLAB toolbox)
• Stability margins: Phase Margin > 30°, Gain Margin > 6 dB, Modulus Margin < 6 dB

4. Stateflow — Task Coordination

The core logic runs in a Simulink Stateflow state machine that coordinates the full pick-sort-stack cycle:

1. Wait for camera detection of a chip
2. Predict chip position accounting for conveyor speed and camera delay: $Y_{in} = Y_{in} + v_{conveyor} \cdot t_{delay}$
3. Move to intercept position
4. Lower arm and activate vacuum gripper
5. Pick up chip while conveyor is moving
6. Move to destination stack based on color
7. Adjust Z height for stacking (increments per chip count)
8. Release and return to home position

Key Technical Challenges

• Picking from a moving conveyor — Required predicting the chip's future position using conveyor speed and camera delay compensation
• Z-axis coupling — The inverse kinematics caused unintended Z-axis motion during XY moves, requiring careful investigation of the IK block
• Stacking accuracy — Each subsequent chip needed a precise Z-offset; errors accumulated across the stack
• Controller tuning — The X-axis controller that looked unstable in ShapeIt actually performed well on hardware, highlighting the gap between model and reality

Results

The robot arm could:

• Consistently pick up poker chips from a moving conveyor belt
• Detect and classify 3 colors (red, yellow, blue)
• Sort chips into 3 separate stacks by color
• Stack multiple chips with sufficient accuracy
• Move at an average speed of 1.8 m/s

Video Demos

Results & Discussion

This project was my first experience with full closed-loop control on a physical robot. The most valuable lesson was that frequency-domain controller design (FRF → loop shaping → stability margins) is the real-world workflow — far more practical than the pole-placement methods taught in theory courses. Debugging the camera delay compensation also taught me that timing issues in real-time systems are often harder than the control design itself.

Technologies Used

MATLAB, Simulink, Stateflow, ShapeIt (FRF-based controller design), computer vision (color masking), inverse kinematics, vacuum gripper, conveyor belt system