Documentation

Everything you need to reproduce the ball-on-arc benchmark: hardware specs, simulation environment, training pipeline, evaluation protocols, 3D-printable parts, and the open dataset.

13

Controllers

2

Experiments

1M

Calibrated Transitions

20Hz

Control Rate

Hardware

Bill of Materials

Control & Drive (Beckhoff)

Part	Description	Role
C6015-0010	Ultra-compact IPC (Atom E3845, 4-core)	TwinCAT 3 runtime
EK1100	EtherCAT Coupler	Terminal stack head
EL7201-0010	Servo Terminal (OCT, 48V, 2.8A rms)	Drives AM8121
EL9576	Brake Chopper Terminal	Regen energy dissipation
ZB8110	Braking Resistor (100W, 10Ω)	EL9576 accessory
EL6002	2-port RS-232 Serial Terminal	Host PC comms (ch 2)
EL1859	8 DI + 8 DO combo	Limit switches, status
AM8121-0FH0	Servo Motor (0.50 Nm, OCT)	Cart actuation
PS2001-2410	24V DC PSU (10A / 240W)	Powers terminal stack

Actuator & Sensors

Part	Description	Notes
LEFB32NXS-1000	SMC Linear Actuator (1000mm stroke)	Belt-driven, cart rides on this
VL53L0X × 2	Time-of-Flight distance sensor	I²C, one per arc edge, 20ms budget
Adafruit #2167	IR Beam Break (508 thru-beam)	Ball-at-center detection
Arduino Uno	Sensor acquisition MCU	Reads ToF + IR, USB to host

Cables & 3D Parts

Item	Qty	Notes
OCT motor cable (ZK4000-series)	1	Power + encoder, AM8121 → EL7201
EtherCAT patch (RJ-45, CAT5e+)	1	C6015 → EK1100
USB-to-RS232 adapter (FTDI/CH340)	1	Host ↔ EL6002 X2
USB-A to USB-B	1	Host ↔ Arduino
3D-printed arc (PLA, 0.2mm, 30%)	1	R=2.101m, Gaussian dip
Acrylic sheets (320×90mm)	2	Side walls for ball containment
Steel ball (24 g, 9 mm radius / 18 mm Ø)	1	Standard bearing ball; matches `BALL_MASS_KG=0.024`, `BALL_RADIUS_M=0.009`

Physical constants in balancer/core/params.py · Serial config in balancer/hardware/serial_config.py · Full BOM and Assembly ↗

3D-Printable Part

The arc is the only custom 3D-printed component. Drag to orbit, scroll to zoom.

Arc (Cart Mount)

3D-printed arc with Gaussian dip profile at center. Mounts directly on linear cart.

↓ Download STL

Environment

Gymnasium Environment

The simulation is a Gymnasium-compatible environment implementing the full nonlinear dynamics of the ball-on-arc system including Coulomb/viscous friction, motor first-order lag, and a Gaussian dip profile.

Key Features

• RK4 integration with configurable substeps (default 10/step)
• Domain randomization: randomizes friction, motor lag, arc radius, ball mass at episode reset
• Blind-zone modeling: masks ball observation when outside sensor FoV (matches hardware ±90mm limit)
• Observation noise: Gaussian noise on ball position (σ≈1.7mm, matching VL53L0X)
• Action space: continuous velocity command ∈ [-1, 1] (scaled to ±0.9 m/s)
• Observation: [cart_pos, cart_vel, ball_angle, ball_angular_vel] (4D)

Dynamics Model

\ddot{\theta} = \frac{-R\cos\theta \, \ddot{x} - \tfrac{1}{2}C_{\text{eff}}'\dot{\theta}^2 - g(-R\sin\theta + \alpha) - Q_{\text{fric}}}{C_{\text{eff}}(\theta)}

Source: balancer/core/dynamics.py · Env: balancer/envs/sim.py

Training

RL Training Pipeline

Built on Stable-Baselines3 + Hydra for config management + Weights and Biases for experiment tracking. All RL training runs on CPU (Appendix G: CPU is faster than GPU for this small LSTM policy).

Supported Algorithms

PPO

SAC

TD3

TQC

TRPO

+ IQL, MPPI, PPO-WM

Key Training Details

• Vectorized envs: 16 parallel environments (DummyVecEnv)
• Timesteps: 3M total (PPO ~40 min on CPU; see Appendix B Table B.4 for full times)
• Domain randomization (--dr): friction ±30%, motor lag ±50%, mass ±20% per episode via env.fixed_param=False
• Blind zone (--blind-zone): env.sensor_blind_zone=True; clamps ball observation near arc edges (matches VL53L0X min-range saturation, θ ≈ 0.071 rad)
• Reward (balanced): dual-Gaussian position + velocity gate + cart-wall penalty + boundary penalty + effort/jerk penalties
• Checkpointing: best model saved by eval callback every 50k steps

# Train PPO with domain randomization + blind zone (the BZ+DR configuration)
cd training
python scripts/train_all_cont.py \
  --algos ppo --rewards balanced --dr --blind-zone \
  --timesteps 3000000 --tag ppo_balanced_dr_bz

Training Times (CPU, Appendix B Table B.4)

Algorithm	Time (CPU)
PPO	~40 min
TRPO	~58 min
SAC / TD3	~5.7 h
TQC	~8.5 h

Source: training/scripts/train.py · Configs: training/configs/ · Training README ↗

Experiments

Experimental Protocols

Experiment 1: Baseline Performance

50 hardware trials per controller with difficulty-stratified initial cart positions (cart in [±0.10, ±0.40, ±0.65, ±0.7765 m]; ball angle arising from the cart jolt is uncontrolled, θ₀ ∈ [−4.8°, 4.3°]). Success = ball within ±0.01 rad (±2 mm at arc centre) held for 1.0 s continuous, within a 30 s trial timeout.

Experiment 2: Frequency Robustness

Control frequency swept 5 / 10 / 20 / 50 Hz. Tests graceful degradation under computational budget constraints. 50 simulation trials per frequency per controller.

Metrics

• Success Rate (SR): % of trials achieving settling criterion
• Settling Time: time from start to first continuous 1s in band
• Median Settling: median settling time across successful trials

# Sim evaluation (Exp 1 = baseline benchmark; Exp 3 = frequency robustness)
python scripts/eval_sim.py --exp 1 --action-type cont

# Hardware evaluation (Exp 1, stratified 50 trials)
python scripts/eval.py --exp 1 --action-type cont --trials 50

Source: evaluation/scripts/ · CMA-ES tuner: evaluation/tuning/ · Evaluation README ↗

Reproduce

Reproduce the Results

The benchmark pipeline has 7 stages. Each card below gives the goal and the key command; click through for the full details. The canonical end-to-end reference is the Reproducibility Guide ↗.

1 Environment Setup

Clone the repo and install all Python dependencies in a single conda environment.

git clone https://github.com/arc-benchmark-review/ball-balancing-on-arc
cd ball-balancing-on-arc
make install-full    # creates "gym" env, registers balancer package
make check           # verify 1,050 trials (21 configs x 50) present

Full instructions ↗ Repo README ↗

2 Hardware Assembly (optional)

Build the physical ball-on-arc platform: SMC linear actuator, Beckhoff servo drive, VL53L0X sensors, Arduino acquisition MCU.

Skip this stage if you only need simulation results and figures; everything except hardware evaluation runs without the platform.

BOM and Assembly ↗

3 Sensor Calibration

Calibrate the two VL53L0X time-of-flight sensors for ball-on-arc position measurement (sequential mode, 34.5 ms budget, acquiring 20 Hz state).

Calibration notes ↗ CALIBRATION_NOTES.md ↗

4 Data Collection (or use released data)

Collect random-exploration data for dynamics modelling, or use the released 1M-transition dataset. For offline RL, build the post-recalibration demonstration dataset from hardware evaluation logs.

# Random exploration (requires hardware)
cd data
python collect_data.py --steps 3000000 --output dataset/my_data.h5 --hz 20

# Build IQL dataset from evaluation logs
python build_post_recalib_dataset.py

Full instructions ↗ Data README ↗

5 Train Controllers

Train RL agents (PPO, world model, IQL) and tune classical controller parameters via CMA-ES.

# PPO (BZ+DR, ~40 min on CPU)
make train-ppo

# World model (LSTM dynamics, stateless training)
make train-worldmodel

# IQL (offline RL from hardware data, separate conda env)
conda activate offrl && make train-iql

# Tune classical controllers
cd evaluation
python -m tuning.tune_controllers --controller all

Full instructions ↗ Training README ↗

6 Evaluate in Simulation

Run all 13 controllers through the 50-trial stratified protocol in simulation (~30 min, no hardware). Should reproduce supplementary Table B.3 (Sim SR near 100% for RL agents).

make eval-sim

Full instructions ↗ Evaluation README ↗

7 Run on Hardware + Generate Figures

Run the 50-trial stratified protocol on the physical platform. Then regenerate all paper figures and compile the PDFs.

# Hardware evaluation (requires platform)
make eval-hw

# Regenerate all paper figures (~30 s, no GPU)
make figures

# Compile main paper
cd paper/ram
pdflatex main && bibtex main && pdflatex main && pdflatex main

# Compile supplementary
cd paper/ram/supplementary
pdflatex supplementary && bibtex supplementary && pdflatex supplementary && pdflatex supplementary

Full instructions ↗ Figure generation ↗

Full end-to-end one-shot reproduction: Reproducibility Guide §11 ↗

Controllers

Available Controllers

class BaseController(ABC):
    def step(self, state, cart_limit) -> float: ...
    def reset(self) -> None: ...

ClassicalPDProportional-Integral-Derivative with wall override

ClassicalLQRLinear Quadratic Regulator + Luenberger observer

ClassicalMPCLinear Model Predictive Control (20-step horizon)

ClassicalNMPCNonlinear MPC (CasADi/IPOPT, N=10)

ClassicalSMCSliding Mode Control (equivalent-control + boundary-layer sat)

RLPPOProximal Policy Optimization + DR + blind zone

RLTD3Twin Delayed DDPG + domain randomization

RLSACSoft Actor-Critic (entropy-regularized)

RLTQCTruncated Quantile Critics + domain randomization

RLTRPOTrust Region Policy Optimization + DR

RLIQLImplicit Q-Learning (offline, from logged hardware data)

RLMPPIModel Predictive Path Integral via learned LSTM world model

RLPPO-WMPPO trained inside the learned world model

All controllers share the BaseController interface.

State vector: [cart_pos, cart_vel, ball_angle, ball_ang_vel]. Output: normalized velocity ∈ [-1, 1].

Controllers source code ↗

Datasets

Open Datasets

Two complementary datasets from the physical system, both stored as HDF5.

1. Real Hardware Demonstrations

Episodic trajectories from all controller families operating on real hardware. Structured for imitation learning, offline RL, and benchmarking.

Metric	Value
Total episodes	3,862
Successful / Failed	2,620 (67.8%) / 1,242 (32.2%)
Total transitions	~1.23 M
Controller families	RL, IQL (offline RL), NMPC, MPC, PID, LQR, SMC, MPPI
File size	~79 MB

arcball_real_demonstrations.h5
  /episodes/N/observations   float32 [T, 4]  # state per timestep
  /episodes/N/actions        float32 [T]     # velocity command
  /episodes/N/timestamps     float32 [T]     # seconds from start
  /flat/observations            float32 [T_total, 4]  # concatenated
  /flat/next_observations       float32 [T_total, 4]
  /flat/actions                 float32 [T_total]
  /flat/terminals               bool    [T_total]
  /episode_metadata/            # controller_name, success, settling_time, ...

2. Random Exploration

Continuous stream of transitions under a random walk policy, used to train the LSTM world model for MPPI and PPO-WM. Best for dynamics modelling and system identification.

Metric	Value
Total transitions	1,000,000
Policy	Random walk (uniform ±0.9 m/s, p_stick = 0.5)
Structure	Continuous stream, no episode boundaries
Sampling	20 Hz
File size	~60 MB

arcball_cont_1M_calib196_148.h5
  /dataset  (1,000,000, 15) float32  : raw transitions
    cols: [prev_state(4), prev_raw_tof(2), action(1),
           cur_state(4), cur_raw_tof(2), reward(1), dip_reward(1)]

# Also released (earlier pre-calibration run, 11-col):
#   arcball_cont_1M_pre_neg.h5  # 999,999 rows; same data but omits the 4 raw ToF
#                                  # columns (2 sensors x 2 timesteps: prev+cur),
#                                  # keeping only the calibrated state vector

Source: data/ License: CC-BY-4.0 Dataset docs ↗ Data README ↗