Procedural Content Generation

No Man's Sky

CS 4803 PCG / CS 8803 PCG
Semester: Spring 2022

Instructor: Greg Turk
E-mail: turk@cc.gatech.edu

Time: MWF 12:30 - 1:20pm
Location: Van Leer E283

Course Description

Procedural Content Generation (PCG) refers to the creation of content (geometric models, images) by writing computer code (procedures). There are a wide variety of geometric models that can be created using procedural methods, including terrain, plants, animals, streets, dungeons, buildings, vehicles and so on. The most common applications of PCG are for video games, feature film special effects, virtual reality and computer-generated art. Some video games that make use of PCG include Minecraft, Spore, and No Man's Sky. This course will cover a large array of algorithms that are used to create geometric models and images. There are two broad categories of algorithms that are most commonly used for content generation: 1) random number driven geometric algorithms, and 2) deep neural network methods. We will study both kinds of algorithms for content creation in this course.

This course will cover many of the methods used in content generation for games and from academic research. Course topics include: creature anatomy, terrain, street layout, buildings, dungeons, mazes, plants, textures, convolutional neural networks, generative adversarial networks, Bezier surfaces, implicit surfaces, pseudo-random numbers, and band-limited noise.

Prerequisites and Media Thread Credit

Prerequisites for CS 4803 PCG: CS 1332, CS 2110 or 2261, CS 2340
Prerequisites for CS 8803 PCG: graduate-level standing

No prior experience in computer graphics or machine learning is required.
You also do not require any prior experience with Unity.

CS 4803 PCG counts as a Pick for the Media thread.

Programming Projects

Content creation methods are best learned by writing code. Each of you will complete several medium/large programming projects, most of which will be written in C# scripts in the Unity game development environment. The project on Generative Adversarial Networks (GANs) will be carried out in Python using Jupyter notebooks. There will be some smaller warm-up projects in addition to the larger projects. You will carry out all of the projects individually. Note that it is impossible to get a good grade in this course without completing the programming assignments.

Late Policy

The grade on a late assignment will drop 5% for each day beyond the due date. A day ends at 11:59 pm. Assignments turned in up to three days late will be accepted (with penalty). No assignments will be accepted more than three days late. For example, if a project is due on March 10, you can turn in the assignment up to March 13 (with a 15% late penalty), but an assignment turned in on March 14 or later will not be accepted.

Academic Integrity

You are expected to follow Georgia Tech’s code of academic conduct. It is fine for you to discuss high level concepts about the projects with other students. However, the details of turning these concepts into working code is for you to carry out on your own. You must write all of the code for each assignment yourself without any form of code sharing by electronic, written, verbal or any other means. The only code from others that may be used in these assignments are those that are provided by the instructor. Do not share your code with other members of the class. You should not show your code to other students, nor is it alright for you to view the code that other students have written. Do not post your code on Github or any other publicly available web site.

Attendance

Any course exams will most likely take place in-person in the classroom. No formal attendance will be taken for class lectures, and missing a lecture will not affect your overall grade in the course. However, you will be expected to attend or to watch a recorded video of each class lecture. Most of what you need to know about the projects will be explained in the lectures, so attending and watching the lectures is important for you to do well on these projects. In addition, the midterm and final exams will draw heavily upon lecture materials, and thus you will be responsible for knowing the lecture materials.

Accommodations for Students with Disabilities

If needed, we will make classroom accommodations for students with documented disabilities. These accommodations must be arranged in advance and in accordance with the office of disability services.

Reading

There will be no required textbook for the class. Most of the materials will draw upon published work from the video game, computer graphics and computer vision communities. For the neural network portions of this course, here are some excellent resources that are recommended:

Learning Objectives

Upon completion of this course, students should be able to:

Describe the basic algorithms used to create both natural and human-made object geometry, typically guided by random number distributions.
Program various geometry creation algorithms to create 3D content with the help of tools such as the Unity game.
Perform data collection to create a library of images that can be used to train and test neural networks algorithms such as GANs for image generation.
Understand the basic components of neural network image classification and image generation methods, and be able to modify, train and test neural networks using a library such as PyTorch.

Expanded List of Course Topics

Games that use PCG

Minecraft

Spore

No Man's Sky

Roguelikes and Dungeon Crawlers

Spore

Natural Phenomena

Terrain generation

Creature anatomy

Plants

Texture synthesis

Flocking

Terrain Synthesis - Austin Passmore

Human-Made Content

Street layout

Dungeons and Mazes

Buildings

Traffic

Buildings - Da In Ryoo

Neural Networks

Multi-layer perceptrons

Backpropagation

Autoencoders

Convolutional neural networks

Generative adversarial networks (GANs)

StyleGAN2

VQGAN-CLIP

Synthetic Portraits - SyleGAN2

Images Collections for Training

Sources of images

Downloading images (APIs)

Image formatting and curation

Forest Scene - VQGAN+CLIP

Random Numbers

Random number generators

Band-limited (Perlin) noise

Modifying numerical distributions