What Makes Convex Functions so special for Machine Learning??

Every Machine learning Problem is an Optimization Problem.

Now bring on your toolkit and let’s mend our ways to get things rolling.

Characteristics of Convex Functions which makes them so special and dear to Data Scientists:

1. Every Convex function has a global Minima…so no problem of fake local minimas disturbing your results.
2. This Global Minima is Unique too..so bingo as you could now recover ground truth argmin from your cost/loss functions.
3. Convexity is preserved over addition and multiplication for convex functions….this allows you to use regularization and weights to your loss functions without a problem.
4. There are tons of techniques developed on Convex Optimization specially for Linear and Quadratic functions.

Examples of convex optimization problems in machine learning:

• linear regression/ Ridge regression, with Tikhonov regularization, etc
• sparse linear regression with L1 regularization, such as lasso
• support vector machines
• parameter estimation in linear-Gaussian time series (Kalman filter and friends)

Typical examples of non-convex optimization in ML are

• neural networks
• maximum likelihood mixtures of Gaussians

MY SOCIAL SPACE

Instagram https://www.instagram.com/codatalicious/

LinkedIn https://www.linkedin.com/in/shaily-jain-6a991a143/

Medium https://codatalicious.medium.com/

YouTube https://www.youtube.com/channel/UCKowKGUpXPxarbEHAA7G4MA

Resources:

1. https://www.princeton.edu/~aaa/Public/Teaching/ORF523/S16/ORF523_S16_Lec7_gh.pdf
2. https://www.quora.com/Why-is-Convex-Optimization-such-a-big-deal-in-Machine-Learning