Guy Lebanon, PhD
Assistant Professor
College of Computing (CSE)
Georgia Institute of Technology

Technical Notes

  The notes below are written in a concise form that is less readable than a textbook. They may be useful as condensed summaries but are not meant to be a replacement for a more complete explanation.

Some of the notes were written as part of course material. As such they are not written in full generality and sometimes are not completely accurate. For example, measure-theoretic arguments are omitted and some assumptions such as differentiability are implicitly made.

Please notify me if you find any mistakes or typos.

   

Probability

 

[P1] Sample space, events, event space, probability as a function and its axioms. Discrete and continuous sample spaces
[P2] Probability distributions on finite sample spaces, classical model for finite and continuous sample spaces
[P3] Conditional probability, independence of events, Bayes theorem
[P4] Random variables: basic definitions, discrete and continuous
[P5] Important random variables: Bernoulli, geometric, binomial, Poisson, uniform, exponential, normal.
[P6] Functions of Random Variable
[P7] Expectations and Variances
[P8] Vector of Random Variables - basic definitions
[P9] Functions of vector of random variables
[P10] Conditional probability and vector random variables
[P11] Expectation and vector random variables
[P12] The multinomial and the multivariate Gaussian distributions
[P13] The weak law of large numbers and central limit theorem
[P14] The moment generating function
   

Statistics

 

[S1] Sampling distributions
[S2] The bias, variance and MSE of estimators
[S3] Confidence intervals
[S4] Relative efficiency, efficiency and the Fisher information
[S5] Consistency of estimators
[S6] Sufficient statistics
[S7] The Rao-Blackwell theorem and the UMVUE
[S8] The method of moments
[S9] Maximum likelihood estimation
[S10] Hypothesis tests
[S11] p-Values, power and the Neyman Pearson lemma
[S13] Likelihood ratio test
[S14] Pearson's Chi-square
[S15] Linear regression
[S16] Consistency of the maximum likelihood estimator
[S17] Asymptotic Efficiency of the maximum likelihood estimator
[S18] Basic sampling methods
[S19] Markov chain Monte Carlo and Metropolis-Hastings
[S20] Missing data and the EM algorithm
[S21] Hidden Markov models
[S22] Support vector machines
[S23] M-estimators and Z-estimators
   
   

Random Processes and LTI filtering

 

[R1] Random processes - basic definitions
[R2] Discrete-time discrete-valued random processes (the iid process and counting processes)
[R3] Continuous-time discrete-valued random processes (Poisson process)
[R4] Continuous-time continuous-valued random processes (Gaussian processes and the Wiener process)
[R5] Stationary and wide sense stationary (WSS) processes
[R6] Response of LTI systems to WSS processes
   
   

Computer Science

 

[C1] Turing Machines
[C2] Complexity and Algorithms
[C3] Operating Systems and Useful Linux Commands
   

Miscellaneous

 

[M1] EM for means of mixture of Gaussians
[M2] Convex Sets
[M3] Convex Functions
[M4] Connections and covariant derivatives