Machine Learning for Flappy Bird using Neural Network and Genetic Algorithm

Here is the source code for a HTML5 project that implements a machine learning algorithm in the Flappy Bird video game using neural networks and a genetic algorithm. The program teaches a little bird how to flap optimally in order to fly safely through barriers as long as possible.

The complete tutorial with much more details and demo you can find here:
http://www.askforgametask.com/tutorial/machine-learning-algorithm-flappy-bird

Here you can also watch a short video with a simple presentation of the algorithm:
https://www.youtube.com/watch?v=aeWmdojEJf0

All code is written in HTML5 using Phaser framework and Synaptic Neural Network library for neural network implementation.

Neural Network Architecture

To play the game, each unit (bird) has its own neural network consisted of the next 3 layers:

1. an input layer with 2 neurons presenting what a bird sees:

   1) horizontal distance between the bird and the closest gap
   2) height difference between the bird and the closest gap
   

2. a hidden layer with 6 neurons

3. an output layer with 1 neuron used to provide an action as follows:

    if output > 0.5 then flap else do nothing
   

There is used Synaptic Neural Network library to implement entire artificial neural network instead of making a new one from the scratch.

The Main Concept of Machine Learning

The main concept of machine learning implemented in this program is based on the neuro-evolution form. It uses evolutionary algorithms such as a genetic algorithm to train artificial neural networks. Here are the main steps:

1. create a new population of 10 units (birds) with a random neural network

2. let all units play the game simultaneously by using their own neural networks

3. for each unit calculate its fitness function to measure its quality as:

   fitness = total travelled distance - distance to the closest gap
   

   

4. when all units are killed, evaluate the current population to the next one using genetic algorithm operators (selection, crossover and mutation) as follows:

   1. sort the units of the current population in decreasing order by their fitness ranking
   2. select the top 4 units and mark them as the winners of the current population
   3. the 4 winners are directly passed on to the next population
   4. to fill the rest of the next population, create 6 offsprings as follows:
       - 1 offspring is made by a crossover of two best winners
       - 3 offsprings are made by a crossover of two random winners
       - 2 offsprings are direct copy of two random winners
   5. to add some variations, apply random mutations on each offspring.
   

5. go back to the step 2

Implementation

Requirements

Since the program is written in HTML5 using Phaser framework and Synaptic Neural Network library you need these files:

• phaser.min.js
• synaptic.min.js

gameplay.js

The entire game logic is implemented in gameplay.js file. It consists of the following classes:

• App.Main, the main routine with the following essential functions:

  • preload() to preload all assets
  • create() to create all objects and initialize a new genetic algorithm object
  • update() to run the main loop in which the Flappy Bird game is played by using AI neural networks and the population is evolved by using genetic algorithm
  • drawStatus() to display information of all units

• TreeGroup Class, extended Phaser Group class to represent a moving barrier. This group contains a top and a bottom Tree sprite.

• Tree Class, extended Phaser Sprite class to represent a Tree sprite.

• Bird Class, extended Phaser Sprite class to represent a Bird sprite.

• Text Class, extended Phaser BitmapText class used for drawing text.

genetic.js

The genetic algorithm is implemented in genetic.js file which consists of the following class:

• GeneticAlgorithm Class, the main class to handle all genetic algorithm operations. It needs two parameters: max_units to set a total number of units in population and top_units to set a number of top units (winners) used for evolving population. Here are its essential functions:

  • reset() to reset genetic algorithm parameters
  • createPopulation() to create a new population
  • activateBrain() to activate the AI neural network of an unit and get its output action according to the inputs
  • evolvePopulation() to evolve the population by using genetic operators (selection, crossover and mutations)
  • selection() to select the best units from the current population
  • crossOver() to perform a single point crossover between two parents
  • mutation() to perform random mutations on an offspring