Speed-Limit RL Reward from Trajectory Samples

Implement a reward function that penalizes speed-limit violations from raw trajectory samples shaped `[batch, num_waypoint, 2]` sampled at 10 Hz, then reason about state-dependent speed limits. Requirements Input: raw trajectory tensor shaped [batch, num_waypoint, 2]. Sampling frequency: 10 Hz. Implement a reward that encourages staying under the speed limit. Two viable reward definitions were discussed: total time spent over the limit, and penalty proportional to how far above the limit each step is. Follow-up: the speed limit changes over time, such as 50 mph for the first two seconds and 30 mph for the next three seconds. Explain the failure mode of a reward that assumes a fixed limit. Notes Compute per-step speed from coordinate deltas and sampling interval before comparing against the limit. At 10 Hz, each interval is 0.1 seconds, so unit conversion must be explicit. For a time-over-limit reward, aggregate an indicator over steps. For magnitude penalty, aggregate max(0, speed - limit) or a squared version. The state-dependent follow-up is the real test: a fixed-limit reward can reinforce behavior that is legal in one segment and illegal in another. The reward needs access to route context or a per-timestep / per-state limit signal. Reward shaping should be tested on simplified trajectories before training; bad broadcasting, unnormalized terms, and over-weighted dense rewards can silently train the wrong behavior. Conceptual discussion may include PPO vs. GRPO and heuristic reward vs. learned reward trade-offs. Preparation Build a toy trajectory batch with known distances and verify speed in mph or m/s by hand before writing the reward aggregation. Implement both reward definitions and a per-timestep speed-limit vector; then test the first-2-seconds / next-3-seconds limit change explicitly. Run random-action or random-trajectory sanity checks and inspect the reward distribution so the penalty scale is not dominated by one term.