Skip to main content

Posts

Showing posts from February, 2021

What is Regularization and Why regularization reduces overfitting?

Regularization If you suspect your neural network is overfitting your data i.e. you have high variance problem, one of the first thing you should try is regularization. The other way to address high variance is to get more training data that's also quite relatable. But you can't always get more training data or it could be expensive to get more training data. But adding regularization will often helps to prevent overfitting or to reduce the errors in your network. How regularization works? Let's say I'm using logistic regression, so my cost function is defined as: `J(w,b)= \frac{1}{m}\sum_{i=1}^{m}L(\hat{y}^{(i)},y^{(i)})` To add regularization to the logistic regression, you add \Lambda  which is called the regularization paramter. `J(w,b)= \frac{1}{m}\sum_{i=1}^{m}L(\hat{y}^{(i)},y^{(i)})+\frac{\lambda}{2m}\sum ||w||_{2}^{2}` `||w||_{2}^{2}=\sum_{j=1}^{n_{x}}w_{j}^{2}=w^{T}w` Now why do we regularize just the parameter w? why don't you add something here abou...
1 comment
Read more

What is Regularization and Why regularization reduces overfitting?

Regularization If you suspect your neural network is overfitting your data i.e. you have high variance problem, one of the first thing you should try is regularization. The other way to address high variance is to get more training data that's also quite relatable. But you can't always get more training data or it could be expensive to get more training data. But adding regularization will often helps to prevent overfitting or to reduce the errors in your network. How regularization works? Let's say I'm using logistic regression, so my cost function is defined as: `J(w,b)= \frac{1}{m}\sum_{i=1}^{m}L(\hat{y}^{(i)},y^{(i)})` To add regularization to the logistic regression, you add \Lambda  which is called the regularization paramter. `J(w,b)= \frac{1}{m}\sum_{i=1}^{m}L(\hat{y}^{(i)},y^{(i)})+\frac{\lambda}{2m}\sum ||w||_{2}^{2}` `||w||_{2}^{2}=\sum_{j=1}^{n_{x}}w_{j}^{2}=w^{T}w` Now why do we regularize just the parameter w? why don't you add something here abou...
1 comment
Read more

Why feature scaling is performed in certain Algorithms of ML and DL?

What is Feature Scaling? In simple words, it is a way of transforming your data into common range of values. Now there are two ways of performing feature scaling. 1) Standardizing 2) Normalizing Standardizing Standardizing is completed by taking each value of your column, subtracting the mean of the column, and then dividing by the standard deviation of the column. In Python, let's say you have a column in your dataframe called weight You could create a standardized weight as: `df["weight_standard"]= \frac{(df['weight'] - df['weight'].mean())}{df['weight'].std()}` This will create a new "standardized" column where each value is a comparison to the mean of the column, and a new, standardized value can be interpreted as the number of standard deviations the original weight was from the mean. This type of feature scaling is by far the most common of all techniques Normalizing A second type of feature scaling that is very popular is know...
3 comments
Read more

Popular posts from this blog

Why feature scaling is performed in certain Algorithms of ML and DL?

What is Feature Scaling? In simple words, it is a way of transforming your data into common range of values. Now there are two ways of performing feature scaling. 1) Standardizing 2) Normalizing Standardizing Standardizing is completed by taking each value of your column, subtracting the mean of the column, and then dividing by the standard deviation of the column. In Python, let's say you have a column in your dataframe called weight You could create a standardized weight as: `df["weight_standard"]= \frac{(df['weight'] - df['weight'].mean())}{df['weight'].std()}` This will create a new "standardized" column where each value is a comparison to the mean of the column, and a new, standardized value can be interpreted as the number of standard deviations the original weight was from the mean. This type of feature scaling is by far the most common of all techniques Normalizing A second type of feature scaling that is very popular is know...
3 comments
Read more

What is Regularization and Why regularization reduces overfitting?

Regularization If you suspect your neural network is overfitting your data i.e. you have high variance problem, one of the first thing you should try is regularization. The other way to address high variance is to get more training data that's also quite relatable. But you can't always get more training data or it could be expensive to get more training data. But adding regularization will often helps to prevent overfitting or to reduce the errors in your network. How regularization works? Let's say I'm using logistic regression, so my cost function is defined as: `J(w,b)= \frac{1}{m}\sum_{i=1}^{m}L(\hat{y}^{(i)},y^{(i)})` To add regularization to the logistic regression, you add \Lambda  which is called the regularization paramter. `J(w,b)= \frac{1}{m}\sum_{i=1}^{m}L(\hat{y}^{(i)},y^{(i)})+\frac{\lambda}{2m}\sum ||w||_{2}^{2}` `||w||_{2}^{2}=\sum_{j=1}^{n_{x}}w_{j}^{2}=w^{T}w` Now why do we regularize just the parameter w? why don't you add something here abou...
1 comment
Read more