Course Syllabus

The items in blue are illustrative topics in statistics, machine learning, and statistical signal processing. 

1. Introduction and review (1 lecture)

Course introduction
Some math: Sets, sums, functions and counting

2. Probability (2 lectures)

What's probability
Sample space: discrete, continuous
Events
Axioms of probability
Basic properties of probability; union of events bound
Discrete probability models; computing probability using counting
Continuous probability models

3. Conditional Probability (1.5 lectures)

Conditional probability and the chain rule
Sequential calculation of probability
Law of total probability and Bayes rule
Independence

4. Random variables I (1.5 lectures)

What's a random variable
Discrete random variables: probability mass function (pmf)
Popular discrete r.v.s
Mean and variance of discrete r.v.s

5. Random variables II (2 lectures)

Continuous random variables: probability density function (pdf)
Popular continuous r.v.s
Mean and variance
General random variables: cumulative distribution function (cdf)
Functions of a random variable

6. Two random variables  (2.5 lectures)

Two discrete r.v.s:  joint, marginal, and conditional pmf
Two continuous r.v.s: joint, marginal, and conditional pdf
One discrete and one continuous r.v.s
Signal detection
Joint cdf
Functions of two random variables

7. Expectation  (1.5 lectures)

Definition and properties
Correlation and covariance 
Linear MSE estimation

8. Conditional expectation (2 lectures)

Law of total expectation
Conditional expectation as a r.v.
Iterated expectation
MSE estimation
Quantization

9. Multiple random variables (2.5 lectures)

Joint and conditional pmf, pdf, cdf
Independence and conditional independence
Parameter estimation: ML estimation
Classification: MAP,  naive Bayes
Mean and variance of sum of r.v.s
Evaluating the quality of estimators

10. Inequalities and limit theorems (1.5 lectures)

Markov and Chebychev inequalities
Weak law of large numbers
Central Limit Theorem
Confidence Intervals
Appendix: Moment generating function
Appendix: Proof of the CLT

11. Epilogue (0.5 lecture)