Shared Human-Robot Control of Soccer Robots

Overview

For my BSc thesis at Tech United Eindhoven, I designed and implemented a shared human-robot control framework for the TURTLEs — Tech United's autonomous soccer robots competing in the RoboCup Middle Size League. The core challenge: enable a single operator to effectively control up to 5 robots simultaneously using a PS4 controller, while the system handles low-level execution, aim correction, and intelligent player switching.

The framework was implemented as a 2,100+ line Python module (fifa.py) that interfaces with the existing Tech United software stack through pyRTDB, a UDP multicast real-time database. It runs three PD controllers with adaptive gain scheduling, a sliding autonomy system that adapts assistance levels to operator skill, and automatic player switching based on ball possession.

Problem & Motivation

Fully autonomous multi-robot soccer suffers in adversarial, unpredictable environments — robots lack the tactical intuition of a human player. Pure teleoperation doesn't scale: one operator cannot individually command 5 robots in real time. Shared control offers a middle ground where the human provides high-level intent and the system handles low-level execution.

The key design question was: how much autonomy should the system inject, and when? Too little assistance and the operator is overwhelmed. Too much and the human becomes a passive observer, defeating the purpose of shared control and its potential as a learning-by-demonstration testbed.

System Architecture

The shared control module sits between operator input and the existing Tech United software stack. Each TURTLE runs four main processes — Vision, WorldModel, Strategy, and Motion — communicating via pyRTDB over UDP multicast. The shared control script (fifa.py) replaces Strategy's role by injecting operator commands through dedicated COACH_FOR1–COACH_FOR6 channels:

Operator (PS4 DualShock 4 / Keyboard)
        │
        ▼
┌──────────────────────────────┐
│  fifa.py — Shared Control    │
│  ┌────────────────────────┐  │
│  │ Input Processing       │  │  Parse joystick axes/buttons
│  │ Player Selection       │  │  Auto/manual robot switching
│  │ PD Controllers (×3)    │  │  Rotation, aiming, passing
│  │ Sliding Autonomy       │  │  Adapt assistance to skill
│  │ Kick Power System      │  │  Variable hold-to-charge
│  └────────────────────────┘  │
└──────────┬───────────────────┘
           │ RefboxCommands via pyRTDB
           │ (UDP multicast)
           ▼
┌──────────────────────────────┐
│  TURTLE Robots (1–6)         │
│  Vision → WorldModel → Motion│
│  (Simulink low-level control)│
└──────────────────────────────┘

pyRTDB Communication

Each control command is packaged into a manual_mode_control_t struct and broadcast through pyRTDB:

# Control data sent to each robot at ~10-20 Hz
class manual_mode_control_t:
    v_x: int          # Forward/backward velocity (-100 to 100)
    v_y: int          # Left/right velocity (-100 to 100)
    v_phi: int        # Rotation velocity (-100 to 100)
    command: int      # JS_MOVE(1), JS_FLAT_SHOT(2), JS_LOB_SHOT(3)
    kick_effort: int  # Kick power (15–100)
 
# Broadcast to specific robot via dedicated channel
coach_channels = {
    1: pyrtdb.Id.COACH_FOR1,
    2: pyrtdb.Id.COACH_FOR2,
    # ... up to 6
}
db.put(coach_channels[active_robot], refbox_cmd, agent=dev_pc_number)

Robot state flows back through STRATEGY_SHARED, providing position (current_xyo in mm and milliradians), velocity, ball possession status (CPBrobot), and role information. Ball position comes through BALL_SHARED with merged multi-robot observations.

Control Design

PD Controller with Adaptive Gain Scheduling

The core of the assisted control modes is a PD controller with three-level adaptive gain scheduling. Gains are selected based on angular error magnitude — aggressive tracking for small corrections, damped response for large turns to prevent overshoot:

where $e(t)$ is the angular error and $\dot{e}_f$ is the low-pass filtered derivative. The gain schedule:

The derivative term uses a first-order low-pass filter ($\alpha = 0.7$) to suppress noise from discrete differentiation:

# Derivative filtering — suppress sensor noise
if dt > 0 and dt < 0.1:
    raw_derivative = (angle_difference - last_angle_error) / dt
    derivative_filter = 0.7 * derivative_filter + 0.3 * raw_derivative
    derivative = derivative_filter * kd

Output velocity is clamped to prevent actuator saturation, with the ceiling scaling proportionally to error magnitude:

This ensures the robot doesn't spin at full speed for tiny corrections but can respond quickly to large reorientations.

Three Controller Instances

This PD architecture is instantiated three times with task-specific parameters:

1. Joystick Rotation Controller — Maps the PS4 right stick to smooth heading control. The operator points the stick in the desired direction; the PD controller tracks the target angle rather than applying raw angular velocity. This gives precise heading control even with imprecise stick input.

2. Assisted Aiming Controller — Automatically rotates the robot to face the goal center when the operator initiates a lob shot. Executes the shot when alignment error drops below 1.0°. Max rotation speed: 50.

3. Assisted Passing Controller — Rotates the robot to face the nearest teammate. Uses a tighter tolerance of 0.5° before auto-executing the pass, and a lower max speed of 45 for precision.

Shared Control Implementation

Player Switching

The system supports two selection modes, toggled with the T key or PS4 right arrow:

Auto mode tracks ball possession across the team. When a robot gains possession (CPBrobot == 1), the operator's input automatically transfers to that robot. If no robot has the ball, control falls back to whichever robot is closest:

# Auto-switch to ball possessor
if strategy_shared.CPBrobot == 1:
    turtle_with_ball = robot_id
    last_ball_possessor = turtle_with_ball
    last_possession_time = current_time

Manual mode keeps the current selection unless the operator explicitly switches with number keys (1–6) or cycles with R1. A pass-in-progress detection system prevents premature switching during passes by tracking a 3-second timeout window after the last possession event.

Assisted Passing

When the operator presses F (or PS4 X), the system evaluates pass quality using perpendicular distance — how far the nearest teammate is from the robot's current aiming line:

# Perpendicular distance: how far off is the operator's aim?
perp_angle_rad = math.radians(passing_robot_aim_angle + 90)
perp_unit_x = math.cos(perp_angle_rad)
perp_unit_y = math.sin(perp_angle_rad)
distance_error = abs(dx * perp_unit_x + dy * perp_unit_y)

Based on this distance, three outcomes:

The 2000 mm threshold isn't fixed — it adapts through sliding autonomy based on operator performance:

where $f_{\text{perf}} = 1.5 - 0.5 \cdot p$ (a beginner with $p = 0$ gets 50% more tolerance than an expert with $p = 1$). Performance $p$ is a rolling average over the last 10 passes:

PASS_HISTORY_LENGTH = 10
 
def sliding_pass_tolerance(operator_performance=0.0, base_tolerance=1500):
    performance_factor = 1.5 - 0.5 * operator_performance
    return base_tolerance * performance_factor

Assisted Shooting

Lob shots (Space / PS4 Circle) use a hold-to-charge power system. While the button is held, kick effort increases every 300 ms from a base of 15 up to a maximum of 100:

If the robot isn't directly facing the goal when the operator releases the button, the assisted aiming PD controller activates. It rotates the robot toward goal center and automatically fires when aligned within 1.0°, using the stored kick power from the charge phase.

Goal Detection

On-target detection projects the robot's aiming line to the goal plane and checks whether the intersection falls between the posts. The field uses MSL-standard dimensions:

Speed Control

The PS4 triggers provide three-tier speed modulation:

Video Demos

The following videos demonstrate each shared control feature running on the physical TURTLE robots:

Field Boundary Impedance

Assisted Ball Intercept

Assisted Shooting

Assisted Passing

Player Switching

Results & Discussion

The framework was validated on physical TURTLE robots during lab sessions at Tech United. The adaptive gain scheduling proved essential — early iterations with fixed gains caused oscillatory behavior during large reorientations and sluggish response during fine adjustments. The three-level schedule was the minimum complexity that eliminated both failure modes.

The sliding autonomy for assisted passing was the most interesting design decision. Rather than a binary assist-on/assist-off, the system continuously adjusts how much aim error it will tolerate based on operator track record. In practice, this meant new operators got generous assistance that gradually tightened as their pass accuracy improved — a natural difficulty curve without explicit difficulty settings.

The biggest limitation was the communication latency through pyRTDB. At ~10–20 Hz update rate, the PD controllers needed conservative gains to avoid instability, which meant the assisted modes felt slightly sluggish compared to what the mechanical platform could achieve with a tighter control loop.

Presentation

Technologies Used

Python (2,100+ lines), pyRTDB (UDP multicast real-time database), PD control with adaptive gain scheduling, PS4 DualShock 4 integration, Simulink (low-level motion control), RoboCup MSL platform (Tech United TURTLEs)