A custom Deep Q-Network estimates the future value of seven actions: walk, turn left, turn right, roll, brake, fly, and land. Its 18-value state combines three directional sensors with goal displacement, momentum, altitude, heading, and flight mode.
Forward, left, and right rays intersect obstacle bounding volumes. Each returns normalized distance, obstacle type, and the reward assigned to the intersected face. Contacts use the same north/south/east/west convention; landings and ground traps use the top reward.
An 18 → 32 (ReLU) → 7 (linear) network learns from randomly sampled batches of 24 transitions in a 10,000-entry replay buffer. Huber-loss gradients are clipped and updated with SGD. A target network synchronizes every 100 updates. The Bellman target is r + 0.97 max Q(s', a'); terminal states omit the future term. Rewards are scaled by 100 for training.
Exploration begins at 100%, decays by 0.992 after each episode, and bottoms out at 5%. Every step costs -1, flight activation costs another -0.5, movement toward the goal earns 2 times distance progress, and the finish awards +100. Traps never terminate an episode. Only the finish or the configured step timeout does.
Walking uses fixed-speed steps. Rolling accumulates momentum, turning retains it, and braking reduces it to 20%. Flying raises the agent above low obstacles and traps; tall walls still block it. Learning is stochastic and does not guarantee convergence.