Pattern Recognition

https://www.youtube.com/watch?v=-yQb_ZJnFXw - for ANOVA (check these before exam)

https://sphweb.bumc.bu.edu/otlt/mph-modules/bs/bs704_hypothesistesting-
anova/bs704_hypothesistesting-anova_print.html

https://www.youtube.com/watch?v=U6NwbQ1V6KU - Chi-Square Test

https://www.bisd303.org/cms/lib3/wa01001636/centricity/domain/587/chi-squarepractice.pdf
- sums(see before tes)

K-Means, GMM, EM
https://www.youtube.com/watch?v=LmpkKwsyQj4&list=PLoROMvodv4rNH7qL6-
efu_q2_bPuy0adh&index=16

https://www.youtube.com/watch?v=LMZZPneTcP4
gmm, em, knn , andrew ng- https://www.youtube.com/watch?v=rVfZHWTwXSA

DBScan

Nested clusters in high dimension

Clusters are the high density region
Outliers in low density region
DBSCAN - https://www.youtube.com/watch?v=-p354tQsKrs
https://www.youtube.com/watch?v=RDZUdRSDOok - Statquest

https://www.youtube.com/watch?
v=MrLPzBxG95I&list=PLl8OlHZGYOQ7bkVbuRthEsaLr7bONzbXS - Cornell

https://www.cs.cornell.edu/courses/cs4780/2018fa/lectures/ - all lectures listed for cornell

https://cs229.stanford.edu/lectures-spring2022/main_notes.pdf - Stanford

https://www.youtube.com/watch?
v=ZMxfDWPXmjc&list=PLoROMvodv4rNyWOpJg_Yh4NSqI4Z4vOYy&index=8 - Stanford
videos

https://www.youtube.com/watch?v=ZAfarappAO0 - naive bayes

https://www.youtube.com/watch?v=kqeksuj0d8w
https://www.youtube.com/watch?app=desktop&v=XzSlEA4ck2I
https://www.kdnuggets.com/2020/06/naive-bayes-algorithm-everything.html - Naive bayes
see for theory

https://www.youtube.com/watch?v=_L39rN6gz7Y - decision tree

https://www.youtube.com/watch?app=desktop&v=ZOiBe-nrmc4 - mit open courseware
decision tree
https://www.geeksforgeeks.org/gini-impurity-and-entropy-in-decision-tree-ml/

https://www.youtube.com/watch?v=coOTEc-0OGw - decision tree sum - ID3

https://www.youtube.com/watch?v=JO2wiZif2OM
https://www.youtube.com/watch?v=2A8AGfxs0D8
https://www.youtube.com/watch?v=zNYdkpAcP-g - Gini-Index
https://www.youtube.com/watch?v=_L39rN6gz7Y - Gini

https://towardsdatascience.com/decision-trees-for-classification-id3-algorithm-explained-
89df76e72df1 - ID3

SVD, PCA

https://www.youtube.com/playlist?list=PLMrJAkhIeNNSVjnsviglFoY2nXildDCcv
https://www.youtube.com/watch?v=FgakZw6K1QQ&vl=en
SVD sum
https://www.d.umn.edu/~mhampton/m4326svd_example.pdf
https://medium.com/intuition/singular-value-decomposition-svd-working-example-
c2b6135673b5 (this is the best do this)
https://www.youtube.com/watch?v=cOUTpqlX-Xs - MIT open courseware
https://www.youtube.com/watch?v=uOzMM13iElw (check this)
PCA sum
https://www.geeksforgeeks.org/mathematical-approach-to-pca/
https://www.youtube.com/watch?v=ZtS6sQUAh0c&list=RDCMUCPi23Ql765_5smMj2-
r0X4g&start_radio=1&rv=ZtS6sQUAh0c&t=0 (check this especially no.6)
https://www.youtube.com/watch?v=TS_9u_NAc24 - this one

LDA

https://sebastianraschka.com/Articles/2014_python_lda.html
https://medium.com/machine-learning-researcher/dimensionality-reduction-pca-and-
lda-6be91734f567 (PCA and LDA)
https://www.youtube.com/watch?v=txgqfG4rfos (see this - mahesh huddar)

Practise problem to construct/ draw Dendogram using Agglomerative Hierarchical

Clustering on a given distance matrix - https://www.youtube.com/watch?v=YH0r47m0kFM
https://www.youtube.com/watch?v=CsLZnCeYRMg

Practise problem to perform divisive hierarchical clustering using minimum spanning

tree on a given distance matrix - https://www.youtube.com/watch?v=vQEXvV5W7s0

The plan for day before the exam -

practice the decision tree (first) (watch again) - done Gini, left ID3, concept theory left
 SVM - done
PCA - done ( https://www.youtube.com/watch?v=TS_9u_NAc24)
LDA - done (https://www.youtube.com/watch?v=txgqfG4rfos)
ANOVA - done
Chi-Square - done
Naive Bayes - done (https://www.kdnuggets.com/2020/06/naive-bayes-algorithm-
everything.html)
Clustering - k-means done, GMM and EM left, Agglomerative and hierarchical
clustering, Linkage - Single, Complete, Average (https://www.youtube.com/watch?
v=YH0r47m0kFM&list=PL4gu8xQu0_5JBO1FKRO5p20wc8DprlOgn&index=117)
Fuzzy C-Means
Regularization
DBScan - done
ICA - not done

Parametric vs. Non-parametric algorithms

So far we have introduced a variety of algorithms. One can categorize these into different
families,
such as generative vs. discriminative, or probabilistic vs. non-probabilistic. Here we will
introduce
another one, parametric vs. non-parametric.
A parametric algorithm is one that has a constant set of parameters, which is independent of
the
number of training samples. You can think of it as the amount of much space you need to
store the
trained classifier. An examples for a parametric algorithm is the Perceptron algorithm, or
logistic
regression. Their parameters consist of , which define the separating hyperplane. The
dimension
of depends of the dimension of the training data, but not on how many training samples you
use
for training.
In contrast, the number of parameters of a non-parametric algorithm scales as a function of
the
training samples. An example of a non-parametric algorithm is the -Nearest Neighbors
classifier.
Here, during "training" we store the entire training data -- so the parameters that we learn
are
identical to the training set and the number of parameters (/the storage we require) grows
linearly
with the training set size.

w, b
w

An interesting edge case is kernel-SVM. Here it depends very much which kernel we are
using. E.g.
linear SVMs are parametric (for the same reason as the Perceptron or logistic regression).
So if the
kernel is linear the algorithm is clearly parametric. However, if we use an RBF kernel then
we cannot
represent the classifier of a hyper-plane of finite dimensions. Instead we have to store the
support
vectors and their corresponding dual variables -- the number of which is a function of the
data
set size (and complexity). Hence, the kernel-SVM with an RBF kernel is non-parametric. A
strange
in-between case is the polynomial kernel. It represents a hyper-plane in an extremely high
but still
finite-dimensional space. So technically one could represent any solution of an SVM with a
polynomial kernel as a hyperplane in an extremely high dimensional space with a fixed
number of
parameters, and the algorithm is therefore (technically) parametric. However, in practice this
is not
practical. Instead, it is almost always more economical to store the support vectors and their
corresponding dual variables (just like with the RBF kernel). It therefore is technically
parametric
but for all means and purposes behaves like a non-parametric algorithm.
Decision Trees are also an interesting case. If they are trained to full depth they are non-
parametric,
as the depth of a decision tree scales as a function of the training data (in practice ). If
we however limit the tree depth by a maximum value they become parametric (as an upper
bound
of the model size is now known prior to observing the training data).

Sums -
https://www.youtube.com/watch?
v=35VgJ84sqqI&list=PL4gu8xQu0_5JBO1FKRO5p20wc8DprlOgn&index=2 - Average
Linkage Hierarchical Agglomerative Clustering Algorithm draw dendrogram in ML

EM - Later (https://www.youtube.com/watch?v=3oefV-
AoP0E&list=PL4gu8xQu0_5JBO1FKRO5p20wc8DprlOgn&index=12)

SVD - https://www.youtube.com/watch?v=uOzMM13iElw

PCA - https://www.youtube.com/watch?v=TS_9u_NAc24

LDA - https://www.youtube.com/watch?
v=txgqfG4rfos&list=PL4gu8xQu0_5JBO1FKRO5p20wc8DprlOgn&index=21

Decision Tree - https://www.youtube.com/watch?v=coOTEc-0OGw - decision tree sum - ID3

https://www.youtube.com/watch?v=JO2wiZif2OM
https://www.youtube.com/watch?v=2A8AGfxs0D8
https://www.youtube.com/watch?v=zNYdkpAcP-g - Gini-Index
https://www.youtube.com/watch?v=_L39rN6gz7Y - Gini
K-means

https://www.youtube.com/watch?
v=KzJORp8bgqs&list=PL4gu8xQu0_5JBO1FKRO5p20wc8DprlOgn&index=112

DBScan - https://www.youtube.com/watch?
v=ZOLYaa9Jex0&list=PL4gu8xQu0_5JBO1FKRO5p20wc8DprlOgn&index=118 (Similarity
index and MinPts)
https://www.youtube.com/watch?v=-
p354tQsKrs&list=PL4gu8xQu0_5JBO1FKRO5p20wc8DprlOgn&index=119 (Eps and MinPts)

Cross Validation - https://www.youtube.com/watch?

v=PF2wLKv2lsI&list=PL4gu8xQu0_5JBO1FKRO5p20wc8DprlOgn&index=214&pp=iAQB

Normalization - https://www.youtube.com/watch?
v=jMvlyoegui4&list=PL4gu8xQu0_5JBO1FKRO5p20wc8DprlOgn&index=216