Name: Data Preprocessing With Python
Author: International Online University

<h3><strong>What is data preprocessing?</strong></h3><p>Data preprocessing is a process of preparing the raw data and making it suitable for a machine learning model. It is the first and crucial step in creating a machine-learning model.</p><p>When creating a machine learning project, it is not always the case that we come across clean and formatted data. And while doing any operation with data, it is mandatory to clean it and put it in a formatted way. So for this, we use the data preprocessing task.</p><h6>&nbsp;<br><h3><strong>Why do we need data preprocessing?</strong></h3></h6><p>Real-world data generally contains noises, and missing values, and may be in an unusable format that cannot be directly used for machine learning models. Data preprocessing is a required task for cleaning the data and making it suitable for a machine learning model which also increases the accuracy and efficiency of a machine learning model.</p><p>&nbsp;</p><h3><strong>Important steps of data preprocessing:</strong></h3><p>There are 4 main important steps for the preprocessing of data.</p><ul><li>Splitting of the data set in Training and Validation sets</li><li>Taking care of Missing values</li><li>Taking care of Categorical Features</li><li>Normalization of the data set</li></ul><p>&nbsp;</p><h3><strong>Train Test Split:</strong></h3><p>Train Test Split is one of the important steps in Machine Learning. It is very important because your model needs to be evaluated before it has been deployed. And that evaluation needs to be done on unseen data because when it is deployed, all incoming data is unseen.</p><p>The main idea behind the train test split is to convert the original data set into 2 parts</p><ul><li>train</li><li>test</li></ul><p>where the train consists of training data and training labels and the test consists of testing data and testing labels.</p><p>&nbsp;</p><h3><strong>Taking Care of Missing Values:</strong></h3><p>There is a famous Machine Learning phrase that you might have heard that is</p><blockquote><em>Garbage in Garbage out</em></blockquote><p>If your data set is full of NaNs and garbage values, then surely your model will perform garbage too. So taking care of such missing values is important. Let’s take a dummy data set to see how we can tackle this problem of taking care of garbage values.</p><p>Taking Care of Categorical Features:</p><p>We can take care of categorical features by converting them to integers. There are 2 common ways to do so.</p><ol><li>Label Encoding</li><li>One Hot Encoding</li></ol><p>In<strong>&nbsp;Label Encoder,</strong>&nbsp;we can convert the Categorical values into numerical labels.</p><p>In&nbsp;<strong>OneHotEncoder&nbsp;</strong>we make a new column for each unique categorical value, and the value is 1 for that column, if in an actual data frame that value is there, it is 0.</p><h3><strong>Normalizing the Dataset:&nbsp;</strong></h3><p>This brings us to the last part of data preprocessing, which is the normalization of the dataset. It is proven from certain experimentation that Machine Learning and Deep Learning Models perform way better on a normalized data set as compared to a data set that is not normalized.</p><p>The goal of normalization is to change values to a common scale without distorting the difference between the range of values.</p><p><br></p><h3><strong>Importance of Data Preprocessing:</strong></h3><ol><li>Most of Machine Learning performance gets slowed down if features (data ) are not scaled. Let’s understand suppose you have two features one is on the scale between ( 0-2 ) and the other ( 0-1000000). Now if you are performing regression on top of it. There will be many iterations of adjustment in the value of the regression coefficient in order to achieve accurate prediction. This phenomenon will increase the time in the training data set. But if you scale them uniformly it would be performance oriented.</li><li>We should always remove unexpected values from the data set. For example, random forest algorithms do not support null values. So replacing such values with some significant sort of values is also under data preprocessing.</li><li>The data set should be in a condition where we can easily change the underlying machine learning algorithm over it. Here preprocessing principle converts them into an incompatible format.</li><li>We have to convert the categorical data into numeric ones. As you know, all machine learning underlines work on numeric data.</li></ol><p>&nbsp;</p>

Master data preprocessing with Python: clean, transform, and prepare data for analysis efficiently. Enroll and Optimize your data workflows now!

Data Preprocessing With Python

What you will learn

Shareable Certificate

100% online courses

Flexible Schedule

Language

Description

This Course includes:

Access on mobile and TV

Full Lifetime Access

7 video lessons

1 Quiz

0 Assignment

Skills that are tagged

Data Preprocessing

Data Cleaning

Programming

+1 more

Course Content

Student Reviews

4.4

8156 Learners

229 Courses

Try some of our courses