Autonomous Snake Training (Deep Q-Network)

The model used is Deep Q-Network (DQN), which combines traditional Q-Learning with deep neural networks. The agent learns the function Q(s,a), which estimates the expected future reward for each possible action in each state.

State Representation: Each state is encoded as 8 binary values — 4 indicate the presence of hazards (wall or body) in the four directions surrounding the head, and 4 specify the direction of the food relative to the head.

Bellman Equation: The network is trained to update Q-values using Q(s,a) = r + γ × max Q(s',a'), where r is the immediate reward, γ = 0.95 is the discount factor that controls the importance of future rewards, and max Q(s',a') is the highest expected value in the next state.

Experience Replay: Instead of learning from a single experience per step, the agent stores the last 10,000 transitions in a buffer and samples 64 random experiences from it for training each step. This breaks the temporal correlation between samples and prevents catastrophic forgetting.

Epsilon-Greedy: Initially, the agent picks random actions 100% of the time (exploration). After each episode, this rate decays by a factor of 0.995 until it reaches just 1% (exploiting learned knowledge). The result: the network gradually learns to associate states with the actions that yield the highest cumulative reward.