CS 4803 EPR

Performance Engineering
Enterprise Computing Performance Engineering
Spring 2005

Instructor: Brian Wilson (brian.wilson@cc.gatech.edu) 
Office hours: Email for Appointment. 
Project Assistant: Mike Hawkins

Classes: Tu/Th, 6:00 - 7:30pm
Instructional Center 102

Description | Announcements | Tentative Course Schedule | Grading | Project
Course Resources | Student Expectations | Instructor Expectations | Required Reading
Student Presentations | CS4803EPR Email List

Description

This course is a fundamental study of distributed computing performance, scalability and reliability simulation theories, methods and practices. The course will cover:


The course material consists primarily of white papers and case studies (and student presentations thereof), lecture notes, required and recommended reading, instructor led or guided lectures, guest lectures, state-of-the-art tools and discussions.  There will be a project that applies the concepts and techniques discussed in the class to enterprise computing performance validation.


The early part of the course will be largely theoretical. The majority of the required reading will take place during that period. The remainder of the course will become progressively technical in nature.

Announcements

1/14: I posted a new version of the Project SOW, including modifications to the Project Grading and Resources sections
1/18: I posted the presentation schedule
1/18: More details on written presentation requirements
1/31: Updated course schedule; LR CBT due 2/15
1/31: Project performance test plan due 2/22

Course Schedule

Reading assignments and student presentations are expected to be prepared/complete by the beginning of the associated lecture.

Week

Date

Description

Time

Lecture Slides

Reading / Deliverables

1

1/11

Course Introduction

Enterprise Computing Lab visit

1:00

:30

Lecture Slides

1/13

Course Project Overview

1:30

Project SOW

Load Testing to Predict Performance

 Load Testing to Predict Performance

2

1/18

SPE Overview - 1

  • Early Integration (prediction)
  • Late Integration (evaluation)

1:30

Lecture Slides

Smith: Chapter 1

Experience with Performance Testing of Software Systems

1/20

 SPE Overview - 2

  • Queuing Theory: Solving Models

  • Review of Markov, Little, Poisson theories
  • Modeling processes and tools

1:30

 

Lecture Slides

Smith: Chapter 2-3,6

A Model of Web Server Performance

Fortier: Chapter 6-7

3

1/25

Performance Test Planning  Part 1

1:30

Lecture Slides

Performance Requirements 

Smith: Chapter 9.2

1/27

Performance Test Planning - Part 2

1:30

Lecture Slides

 

4

2/1

Case Studies and White Papers

  • Student Presentations 1,2,3

1:30

 

 

 

2/3

Performance Test Planning - Part 3

Case Studies and White Papers

  • Student Presentations 4

1:10

:20

Lecture Slides

Fortier: Chapter 3 

5

2/8

Performance Test Planning - Part 4

  • Test Data
  • Test Script Development
  • Project Planning

1:30

Lecture Slides

Fortier: Chapter 11 (tool selection)

2/10

Test Execution

  • Types of Performance Tests
  • Performance Measurement

1:30

Lecture Slides

Fortier: Chapter 10 (monitoring)

6

2/15

Results Analysis

  • Collating results
  • Bottleneck isolation
  • Reporting and presentation

1:30

Lecture Slides

Fortier: Chapter 8

LoadRunner CBT Due

2/17

Mid-Term Exam Review

1:30

N/A

7

2/22

Mid-Term Exam

  • Performance Modeling
  • Queuing Theory
  • Performance Test Planning
  • Test Execution
  • Test Results Analysis

1:30

N/A

  

2/24

Performance Simulation - Part 1

  • Simulation environment
  • Performance simulation tools

Project Performance Test Plan student presentation (for extra credit)

:45

 

 

 

:45

Lecture Slides

Project Performance Test Plan Due

8

3/1

Performance Simulation - Part 2

  • Simulation Scripting Fundamentals
    • Recording
    • Creating actions
    • Runtime settings
    • Correlation fundamentals
    • Parameters
  • Exercises

1:00

:30

3/3

Performance Simulation - Part 3

  • Simulation Scripting Fundamentals
    • Playback
    • Script debugging
    • Data parameterization and maintenance
  • Exercises

1:00


 

:30

Fortier: Chapter 3

9

3/8

Performance Simulation - Part 4

  • Advanced Simulation Scripting
    • Manual parameterization [runtime / design time]
    • Automatic correlation
    • Exercises

:30


 

1:00

3/10

Performance Simulation - Part 5

  • Advanced Simulation Scripting
    • Checkpoints
    • Logging
    • Error trapping
    • Script organization
    • Code commenting and standards
    • Transactions
  • Exercises

1:30

10

3/15

J2EE Deep Diagnostics Overview

  • Running experiments
  • Capturing application metrics
  • Building experiments
  • Using reports
  • Using call graphs and profiles
  • Exporting and importing project data
  • Instrumenting applications
  • Capturing custom data
  • Diagnosing defects

1:30

3/17

Guest Lecturer: Bill Hayden

  • Outline TBD

1:30

11

SPRING BREAK

12

3/29

Performance Simulation - Part 7

  • Advanced Simulation Scripting
    • File I/O
    • Multi-protocol introduction
    • LR Internals
  • Exercises

1:00

:30

3/31

Performance Simulation - Part 8

  • Scenario Development
    • Intro to scenarios
    • Scheduling
    • Remote agents
    • IP Spoofing
    • Monitors
  • Bottleneck identification tool review

1:00

:30

13

4/5

Guest Lecturer: Caleb Billingsley

  • Outline TBD

1:30

Lecture Slides

4/7

 Student Presentations 2

1:30

14

4/12

 Project Script Review

1:30

4/14

Performance Analysis and Reporting - Part 1

  • Analysis
    • Transaction response times
    • System resource metrics
    • Component metrics
    • Other metrics
    • Analyzing trends
    • Developing theories

1:30

15

4/19

Performance Analysis and Reporting - Part 2

  • Reporting Results
    • Organization
    • Publishing
    • Final presentation
  • Example Reports

1:30

Smith: Chapter 11

4/21

Performance Management - SiteScope

  • SiteScope installation and setup of monitoring
  • Alerting strategies
  • Setup and management of groups
  • Planning and using alerts
  • Using diagnostics tools in SiteScope
  • Generating and viewing reports

1:30

16

4/26

Final Project Presentations

1:30

4/28

Final Exam Review

1:30

17

5/3

FINAL EXAM

2:00


Required Reading

In addition to specific reading assignments found throughout the Course Schedule, this reading should be completed prior to the final exam, or in accordance with the class schedule.

Title Author(s) ISBN Chapters
Performance Solutions [a Practical Guide to Creating responsive, Scalable Software] Connie U. Smith
Lloyd G. Williams		 0201722291 1-3,6,15
Computer Systems Performance Evaluation and Prediction Paul J. Fortier
Howard E. Michel		 1-55558-260-5 3,6-7,8.1-8.3,10-11

 

Grading
 

•
Class Participation
 		 5%
 
Student Presentations
 		 10%
 
Midterm Exam
 		 15%
 
Final Exam
 		 30%
 
Project 40%

 


 

Student Presentations
of Case Studies

Students are required to choose one case study from the list below for his/her presentation. Each student will give one 15-20 minute presentation summarizing the analysis of one of these case studies. Each case study may be presented by one student only. Case study selections must be made, via email, to the Instructor no later than the beginning of the second lecture (1/13); the instructor will schedule each student's presentation at that time. The presentations must be presented to the Instructor, via email, by midnight prior to the assigned presentation date. The presentation schedule is posted here.

 

Written Presentation: The paper will be in Microsoft Word format and will be of no less than 3 and no more than 4 single-spaced pages in length. It will contain an executive summary, problem statement, findings and a conclusion. There is no specific form or format in the format of commentary as long as you cover the required outline given below:

Name of the paper

 

Oral Presentation: Oral presentations will use a PowerPoint slideshow for enhancement. Students should use oral and written presentation techniques acceptable in the technical business community. Other useful tips on oral and written presentations may be found here. There is an opportunity for one student to present his/her team's project test plan on 2/24; the opportunity is first come first served. Contact the Instructor immediately if interested. An additional 1-5% of the student's overall course grade will be awarded.
 

ScalableEcommerceSolutions HP_Scalability
EJB_Performance Performance Analysis In the Age of the Internet
  TSEperformance