Research work · 2023
Data Science in the Database
Encyclopedia of Data Science and Machine Learning
Abstract
An important part of the data lifecycle is preparing the data for analysis. This includes exploratory data analysis to find problems, data cleaning to solve problems related to 'dirty' data, and data transformation to achieve the format required by data analytics tools. Much data resides in databases, and it is beneficial for the data scientist to be able to carry out as much of this work as possible inside the database. This can be achieved using SQL. This article overviews some of the basic approaches to data exploration, cleaning and wrangling using SQL, including methods to deal with missing data, outliers, and bad values.