Learning to Play Pinball (in simulation)

Write a learning system that learns to play pinball well on a simple pinball simulator that we will provide you with.

Here is a suggestion for an interesting scenario:

Modify the table so that the ball gets easily lost when it ends up in the wrong part of the table. Now rearrange the bumpers so that the system can score high in that part of the table. The task of the learning system is to get as high a score as possible with, say, 50 balls. It can observe the coordinates of the ball, but has no prior knowledge of the layout of the table. Now the system has to trade-off between shooting the ball where it can learn something new, shooting the ball where it can score high, and shooting the ball where it is safe. How does your learning system handle this trade-off? For example, does it reason about the trade-off explicitly or implicitly?

There isn't much work on learning to play pinball out there but here are three pointers:

Some Explorations in Reinforcement Learning Techniques Applied to the Problem of Learning to Play Pinball. Nathaniel S. Winstead, AAAI Press, Menlo Park, CA, 1996.

Pinball: Planning and Learning in a Dynamic Real-Time Environment. Nathaniel S. Winstead and Alan D. Christiansen, AAAI Press, Menlo Park, CA, 1994.

Algorithms for Pinball Simulation, Ball Tracking and Learning Flipper Control. Michael Patrick Johnson, bachelor's thesis.

Use these to get an idea of what has been done but be creative and come up with ideas on your own (either something completely different or something that extends the previous ideas).