14/11/2023, 14:47 Automatic Hyperparameter Tuning with Sklearn Using Grid and Random Search | by Bex T.

ch | by Bex T. | Towards Data Science

Open in app

Search Write

Member-only story

Automatic Hyperparameter Tuning

with Sklearn Using Grid and
Random Search
Deep dive into GridSearch and RandomSearch classes of Scikit-learn

Bex T. · Follow
Published in Towards Data Science · 6 min read · Mar 5, 2021

157 1

https://medium.com/towards-data-science/automatic-hyperparameter-tuning-with-sklearn-gridsearchcv-and-randomizedsearchcv-e94f53a518ee 1/8
14/11/2023, 14:47 Automatic Hyperparameter Tuning with Sklearn Using Grid and Random Search | by Bex T. | Towards Data Science

Photo by Helena Lopes on Pexels

What is a hyperparameter?
Today, algorithms that hide a world of math under the hood can be trained
with only a few lines of code. Their success depends first on the data trained
and then, on what hyperparameters were used by the user. So, what are
these hyperparameters?

Hyperparameters are user-defined values like k in kNN and alpha in Ridge

and Lasso regression. They strictly control the fit of the model and this
means, for each dataset, there is a unique set of optimal hyperparameters to
be found. The most basic way of finding this perfect set would be randomly
trying out different values based on gut feeling. However, as you might
guess, this method quickly becomes useless when there are many
hyperparameters to tune.

https://medium.com/towards-data-science/automatic-hyperparameter-tuning-with-sklearn-gridsearchcv-and-randomizedsearchcv-e94f53a518ee 2/8
14/11/2023, 14:47 Automatic Hyperparameter Tuning with Sklearn Using Grid and Random Search | by Bex T. | Towards Data Science

Instead, today you will learn about two methods for automatic
hyperparameter tuning: Random search and Grid search. Given a set of
possible values for all hyperparameters of a model, a Grid search fits a
model using every single combination of these hyperparameters. What is
more, in each fit, the Grid search uses cross-validation to account for
overfitting. After all combinations are tried, the search retains the
parameters that resulted in the best score so that you can use them to build
your final model.

Random search takes a bit different approach than Grid. Instead of

exhaustively trying out every single combination of hyperparameters, which
can be computationally expensive and time-consuming, it randomly samples
hyperparameters and tries to get closer to the best set.

Fortunately, Scikit-learn provides GridSearchCV and RandomizedSearchCV

classes that make this process a breeze. Today, you will learn all about them!

Join Medium with my referral link - BEXGBoost

Get exclusive access to all my premium ⚡ ⚡
content and all over
Medium without limits. Support my work by buying me a…
ibexorigin.medium.com

Get the best and latest ML and AI papers chosen and summarized by a
powerful AI — Alpha Signal:

Read the best stories about data science from

industry leaders on Medium.
https://medium.com/towards-data-science/automatic-hyperparameter-tuning-with-sklearn-gridsearchcv-and-randomizedsearchcv-e94f53a518ee 3/8
14/11/2023, 14:47 Automatic Hyperparameter Tuning with Sklearn Using Grid and Random Search | by Bex T. | Towards Data Science

The author made this story available to Medium members only. Upgrade to instantly
unlock this story plus other member-only benefits.

Access all member-only stories on Medium

Become an expert in your areas of interest

Get in-depth answers to thousands of questions about data science

Grow your career or build a new one

Marc-André Giroux Carlos Arguelles Tony Yiu Brandeis Marshall

Sr. Software Developer Sr. Staff Engineer Director CEO
Netflix Google Nasdaq DataedX

Cassie Kozyrkov Memo Akten Vitali Zaidman Camille Fournier

Chief Decision Scientist Asst. Professor Software Architect Head of Engineering
Google UCSD Meta JPMorgan Chase

Upgrade

More from the list: "Predictive Modeling w/ Python"

Curated by Ben Putney

https://medium.com/towards-data-science/automatic-hyperparameter-tuning-with-sklearn-gridsearchcv-and-randomizedsearchcv-e94f53a518ee 4/8
14/11/2023, 14:47 Automatic Hyperparameter Tuning with Sklearn Using Grid and Random Search | by Bex T. | Towards Data Science

Luís Fernando … in LatinX… Bex T. in Towards AI Alparslan M… in MLearnin

How I Deployed a 19 Most Elegant Sklearn Data Science Project:

Machine Learning Mode… Tricks I Found After 3… Sales Forecasting with…
13 min read · Jul 19 · 12 min read · Jun 16 4 min read · Feb 5, 2022

View list

Written by Bex T. Follow

38K Followers · Writer for Towards Data Science

🥇
BEXGBoost | DataCamp Instructor | Top 10 AI/ML Writer on Medium | Kaggle
Master | https://www.linkedin.com/in/bextuychiev/

More from Bex T. and Towards Data Science

Bex T. in Towards AI Marco Peixeiro in Towards Data Science

6 Pandas Mistakes That Silently TimeGPT: The First Foundation

Tell You Are a Rookie Model for Time Series Forecasting
https://medium.com/towards-data-science/automatic-hyperparameter-tuning-with-sklearn-gridsearchcv-and-randomizedsearchcv-e94f53a518ee 5/8
14/11/2023, 14:47 Automatic Hyperparameter Tuning with Sklearn Using Grid and Random Search | by Bex T. | Towards Data Science

No error messages — that’s what makes them Explore the first generative pre-trained
subtle forecasting model and apply it in a project…

· 7 min read · Sep 4 · 12 min read · Oct 24

1.2K 9 2.1K 18

Adrian H. Raudaschl in Towards Data Science Bex T. in Towards AI

Forget RAG, the Future is RAG- Forget PIP, Conda,

Fusion requirements.txt! Use Poetry…
The Next Frontier of Search: Retrieval Pain-free dependency management is finally
Augmented Generation meets Reciprocal… here

· 10 min read · Oct 6 · 8 min read · Jun 25

2.3K 24 2K 24

See all from Bex T. See all from Towards Data Science

Recommended from Medium

https://medium.com/towards-data-science/automatic-hyperparameter-tuning-with-sklearn-gridsearchcv-and-randomizedsearchcv-e94f53a518ee 6/8
14/11/2023, 14:47 Automatic Hyperparameter Tuning with Sklearn Using Grid and Random Search | by Bex T. | Towards Data Science

Roushanak Rahmat, PhD in GoPenAI Marco Peixeiro in Towards Data Science

Feature Importance Analysis with A Practical Guide to Feature

Python Tutorial Selection Using Sklearn
Table of Contents: A hands-on tutorial on how to select the best
features for your model using Python and…

· 6 min read · Sep 14 · 10 min read · Sep 27, 2022

14 2 101 1

Lists

Predictive Modeling w/ ChatGPT prompts

Python 30 stories · 649 saves
20 stories · 590 saves

ChatGPT Practical Guides to Machine

22 stories · 253 saves Learning
10 stories · 670 saves

https://medium.com/towards-data-science/automatic-hyperparameter-tuning-with-sklearn-gridsearchcv-and-randomizedsearchcv-e94f53a518ee 7/8
14/11/2023, 14:47 Automatic Hyperparameter Tuning with Sklearn Using Grid and Random Search | by Bex T. | Towards Data Science

Hiroaki NATSUME in Optuna Mustafa Germec, PhD in AI Mind

Black-box optimization of Hyperparameter optimization of

geometry and functionality by… random forest model using Optun…
This article introduces Tunny, Optuna’s 3D
CAD wrapper. We will start with an…

16 min read · Sep 20 · 9 min read · Jul 21

21 144

Simon Benavides Pinjosovsky, PhD Rukshan Pramoditha

Normalize data before or after split Recursive Feature Elimination

of training and testing data? (RFE) in Regression and…
When working with machine learning models, Using the RFECV() function in Scikit-learn and
it is important to preprocess the data before… Yellowbrick

3 min read · Jul 5 · 6 min read · Jun 19

118 1 18

See more recommendations

https://medium.com/towards-data-science/automatic-hyperparameter-tuning-with-sklearn-gridsearchcv-and-randomizedsearchcv-e94f53a518ee 8/8