ML Interview question: Overfitting and ways to prevent it?
Answers
-
What Is Overfitting?
Overfitting is a modeling error in statistics that occurs when a function is too closely aligned to a limited set of data points. As a result, the model is useful in reference only to its initial data set, and not to any other data sets.Overfitting the model generally takes the form of making an overly complex model to explain idiosyncrasies in the data under study. In reality, the data often studied has some degree of error or random noise within it. Thus, attempting to make the model conform too closely to slightly inaccurate data can infect the model with substantial errors and reduce its predictive power.
How to Prevent Overfitting
Ways to prevent overfitting include cross-validation, in which the data being used for training the model is chopped into folds or partitions and the model is run for each fold. Then, the overall error estimate is averaged. Other methods include ensembling: predictions are combined from at least two separate models, data augmentation, in which the available data set is made to look diverse, and data simplification, in which the model is streamlined to avoid overfitting.
-
Overfitting :- when model performs very well in predicting value of training datasets , but its accuracy fell in test dataset.
Ways to prevent Overfitting problems are:-
*Cross-validation :-In standard k-fold cross-validation, we partition the data into k subsets, called folds. Then, we iteratively train the algorithm on k-1 folds while using the remaining fold as the test set .
*** Remove features:-** by removing irrelevant input features.Early stopping;-Up until a certain number of iterations, new iterations improve the model. After that point, however, the model’s ability to generalize can weaken as it begins to overfit the training data.
Early stopping refers stopping the training process before the learner passes that point.*Regularization
- Ensembling
