2013 Sang Han
A collection of notes and useful articles on numerical algorithms.
Markov Chains: A special type of stochastic process. The standard definition of a stochastic process is an ordered collection of random variables:
Self-Organized Criticality (SOC): “Self-Organized” means that from any initial condition, the system tends to move toward a critical state, and stay there, without external control. A system is “critical” if it is in transition between two phases.
Kernel Density Estimation: Because fuck histograms.
- Least Squares
- Ridge Regression
- Last Angle Regression
- Elastic Net
- Kernel Ridge Regression
- Support Vector Machines (SVR)
- Partial Least Squares (PLS)
- Linear Discriminant Analysis (LDA)
- Basic Perceptron,
- Elastic Net
- Logistic Regression (Kernel)
- Support Vector Machines (SVM)
- Diagonal Linear
- Discriminant Analysis (DLDA),
- Golub Classifier
- Fisher Discriminant Classifier
- Classification Tree
- Maximum Likelihood Classifier
- Hierarchical Clustering
- Hierarchical Clustering
- Fisher Discriminant (FDA),
- Spectral Regression Discriminant Analysis (SRDA)
- Principal Component Analysis (PCA)
Critical systems demonstrate common behaviors
- Long-tailed distributions of some physical quantities.
- Fractal geometries:
- Variations in time that exhibit pink noise or a time series with many frequency components.
- In white noise, all of the components have equal power.
- In “pink” noise, low-frequency components have more power than high-frequency components.
- A reductionist model describes a system by describing its parts and their interactions.
- When a reductionist model is used as an explanation, it depends on an analogy between the components of the model and the components of the system.
- Holistic models are more focused on similarities between systems and less interested in analogous parts.
- Usually fits the simplest model that demonstrates that behavior Optimization
- Agents that model intelligent behavior, usually with a simple set of rules.
- Useful for modeling the dynamics of systems that are not in equilibrium.
- Agent-based models are useful for modeling the dynamics of systems that are not in equilibrium.
- Particularly useful for understanding relationships between individual decisions and system behavior.
Stochastic Volatility Model:
- Models the latent volatility variable, modeled as a stochastic process.
A naiive but simple optimization algorithm and performance analysis.
Introduces detection of change, using electromyography to model the usage of a threshold potential to signify a change in the enviornment.
Introduces the Cumulative sum (CUSUM) algorithm.
Uses one-dimensional linear interpolation in temporal alignment of cyclic data.
The DTW algorithm finds the optimum alignment between two sequences of observations by warping the time dimension with certain constraints.
DTW is good for classifying sequences that have different frequences or that are out of phase.
When it comes to timeseries classification, 1 Nearest Neighbor (K=1) and Dynamic Timewarping is very difficult to beat.
Introduces using Gaussians to represent beliefs in the Bayesian sense. Gaussians allow us to implement the algorithms used in the Discrete Bayes Filter to work in continuous domains.
Implements a Kalman filter by modifying the Discrete Bayesian Filter to use Gaussians. This is a full featured Kalman filter, albeit only useful for 1D problems.
Discusses the ensemble Kalman Filter, which uses a Monte Carlo approach to deal with very large Kalman filter states in nonlinear systems.
Thorough explanation of MCMC(Markov Chain Monte Carlo), operates and diagnostic tools.
Introduces the PyMCMC library which implements the MCMC exploring algorithm and exends a public
The Model class has very simple internals: just a list of unobserved variables and a list of factors which go into computing the posterior density.
Digital Signal Processing
A brief description about the basic properties of signals. Introduces Nyquist-Shannon Theorem for sampling, and discusses linear systems, issues with quantization and RNG’s.
The original fourier series implemented in Python.
FFT and it’s application to PSD. Also implements Short-Time Fourier Transform when you just want an estimate.
A thorough introduction to data filtering and the most basic filters typically used in signal processing.