Information Criterion

This post will be my summary about the Akaike Information Criterion(AIC)  and the Takeuchi Information Criterion(TIC).  In particular, a derivation of AIC and TIC is shown. And if I can understand more about the Generalized Information Criterion, I will cover it too.

Handbook of Markov Chain Monte Carlo

Today I received my copy of “Handbook of Markov Chain Monte Carlo“. Up to now I have consulted “Monte Carlo Strategies in Scientific Computing“ of Jun S.Liu. In this post I will give my first thought on the new book.

The book of Liu is indeed awesome. The author himself has made many original contributions to sampling algorithms, and many of his ideas was explained in the book. The chapters on Gibbs Sampler (chapter 6),General Conditional Sampling (chapter 7) and multi-chain MCMC (chapter 10, 11) were excellent. But given that the book was written 10 years ago, many recent developments is missing. Some algorithms were not given enough spaces (in particular, Reversible Jump MCMC was given only  2 pages!).

Formulating the PCA

Today I have thought about how one can formulate the Principal Component Analysis (PCA) method. In particular I want to reformulate PCA as a solution for a regression problem. The idea of reformulation PCA as a solution for some regression problem is useful in Sparse PCA , in which a regularization term is inserted into a ridge regression formula to enforce spareness of the coefficients (i.e. elastic net). There are at least two equivalent ways to motivate PCA. In this post I will first give a formulation of PCA based on orthogonal projection, and then discuss a regression-type reformulation of PCA.

RJMCMC in clustering

Slide from a 30-minute presentation. There are some mistakes in the slide.

Nonparametric Bayesian Seminar 1 : Notes

(mục đích chính là viết ra để khỏi quên nên sẽ  lộn xộn. Không có hình vẽ)

Paper: Introduction to Nonparametric Bayesian Models (Naonori Ueda, Takeshi Yamada)

Flows: the psychology of optimal experience

Ising model

Boltzman distribution:

Q

(These are some thoughts I got while reading the inspiring book by James D.Watson “Avoid boring people and other lessons from a life in science.” Maybe I will give a full summary on the lessons from the book later.)

Mathematics and the Unexpected

(Just some thoughts while I read Mathematics and the Unexpected and Innumeracy: Mathematical Illiteracy and its consequence. Maybe some more detailed post later.)

Probability and Computing: Chapter 7 Exercises

Exercise 7.12: Let be the sum of independent rolls of a fair dice. Show that, for any , .