Classification: The Techniques

Here I desire to accumulate all the classification techniques learnt so far.

Motive is to keep a repository for me to remember and others to benefit from.

Use cases of Classification(googled as per my interest):

• Indian Crop classification of Temporal Multi- Spectral Satellite Image Link
• Classification of whether to sell, purchase or hold stock for a particular stock in Indian Stock Market Link
• Classification for diabetes and cardiovascular diseases
• Traffic congestion classification Link

Types of Classification Algorithms most widely used:

• Logistic Regression
• Naive Bayes
• Perceptron
• XG Boost Classifier
• Decision Tree
• SVM Classifier
• Random Forest Classifier
• K- Nearest Neighbours

Let’s see what metrics can be used to Choose the Best Algorithm for Classification:

• Classification Accuracy = Ratio of correct predictions to total number of input samples. If label is 98% A and 2% B, then accuracy of 98% can be achieved, which is false represented of high accuracy.
• Logarithmic Loss/ Cross Entropy(works for multi class classifications) No upper cap on loss, we try to minimise this loss.

y_ij whether sample i belongs to class j or not: 0 or 1, p_ij is probability of it

• Confusion Matrix where True Positive(Predict 1 Actual 1), True Negative(Predict 0 Actual 0), False Positive(Predict 1 Actual 0), False Negative(Predict 0 Actual 1)

• Area under Curve of plot False Positive Rate vs True Positive Rate. For Binary classification, AUC classifier is equal to the probability that the classifier will rank a randomly chosen positive example higher than a randomly chosen negative example. AUC belongs between [0,1]. The greater the value, the better is the performance of model.

• F1 Score Harmonic mean between precision and recall. The range for F1 is [0,1]. It tells how precise your classifier is(how many instances it classifies correctly), as well how robust it is (it doesn’t miss a significant number of instances)