Data science for everybody

In this article, I’ll unpack the what, why, and how of data science. By the end of this article, you’ll have a better grasp of how data is being used around you and how you can use data.

If we Google “What is data science?”, we’ll see a huge amount of confusing information.

But data science is actually simple. It’s a set of methodologies for taking in thousands of forms of data that are available to us today and using them to draw meaningful conclusions. Data is being collected all around us. Every like, click, email, credit card swipe, or tweet on Twitter is a new piece of data that can be used to better describe the present or better predict the future.

So what can data do? It can help detect anomalous events, such as fraudulent purchases. If we have data on what has happened previously, we can increase efficiency by automatically detecting a new event that is unexpected or abnormal. Data can also diagnose the causes of observed events and behaviors, for instance, your activity on Spotify or Netflix. Rather than determining correlations between small numbers of events, data science techniques help us understand complex systems with many possible causes. Finally, data can predict future events, such as forecasting population size. We can use new techniques to take various causes into account and predict potential outcomes. Further, we can evaluate the probability of our prediction mathematically to clarify our level of uncertainty.

Workflow of Data science

Data science workflow are as follows: Data collection and storage -Data preparation- exploration and visualization-experiment and prediction.

So, how do we start to use data? In data science, we generally have four steps to any project as mentioned above. First, we collect data from many sources, such as surveys, web traffic results, geo-tagged social media posts, and financial transactions. Once collected, we store that data in a safe and accessible way. At this point, data is in its raw form, so the next step is to prepare data. This includes “cleaning data”, for instance finding missing or duplicate values, and converting data into a more organized format. Then, we explore and visualize the cleaned data. This could involve building dashboards to track how the data changes over time or performing comparisons between two sets of data. Finally, we run experiments and predictions on the data. For example, this could involve building a system that forecasts temperature changes or performing a test to find which web page acquires more customers.

APPLICATION OF DATA SCIENCE

Since we already know the workflow of data science, let’s take a step further by knowing the real-world application of data science. but wait, Let’s quickly take a deep dive into three exciting areas of data science: traditional machine learning, the Internet of Things, and deep learning. to get a better understanding of things let’s use fraud detection as a case study:

Suppose you work in fraud detection at a large bank. You’d like to use data to determine the probability that the transaction is fake.

To answer this question, you might start by gathering information about each purchase, such as the amount, date, location, purchase type, and card-holder’s address. You’ll need many examples of transactions, including this information, as well as a label that tells you whether each transaction is valid or fraudulent. Luckily, you probably have this information in a database. These records are called “training data”, and are used to build an algorithm. Each time a new transaction occurs, you’ll give your algorithm information, like amount and date, and it will answer the original question: What is the probability that this transaction is fraudulent?

What do we need for machine learning?

Before we can answer that question, First, a data science problem begins with a well-defined question. Our question was “What is the probability that this transaction is fraudulent?” Next, we need some data to analyze. We have months of old credit card transactions and associated metadata, like date and location, that have already been identified as either fraudulent or valid. Finally, we need additional data every time we want to make a new prediction. We need to have the same type of information on every new purchase so that we could label it as “fraudulent” or “valid”.

Internet of Things (IoT)

Your smart-watch is part of a fast-growing field called “the Internet of Things”, also known as IoT, which is often combined with Data Science. IoT refers to gadgets that are not standard computers, but still, have the ability to transmit data. This includes smart-watches, internet-connected home security systems, electronic toll collection systems, building energy management systems, and much, much more. IoT data is a great resource for data science projects!

Deep learning

We need more advanced algorithms from a subfield of machine learning called deep learning. In deep learning, multiple layers of mini-algorithms, called “neurons”, work together to draw complex conclusions. Deep learning takes much, much more training data than a traditional machine learning model, but is also able to learn relationships that traditional models cannot. Deep learning is used to solve data-intensive problems, such as image classification or language understanding.

DATA SCIENCE ROLES AND TOOLS.

Generally, there are four jobs: Data Engineer, Data Analyst, Data Scientist, and Machine Learning Scientist. Let’s explore each one.

Data engineer

Data engineers control the flow of data: they build custom data pipelines and storage systems. They design infrastructure so that data is not only collected, but easy to obtain and process. Within the data science workflow, they focus on the first stage: data collection and storage.

Data engineering tools

Data engineers are proficient in SQL, which they use to store and organize data. They also use one of the following programming languages like Java, Scala, or Python to process data. They use Shell on the command line to automate and run tasks. Finally, data engineers, now more than ever, need to be comfortable with cloud computing to ingest and store large amounts of data.

Data analyst

Data analysts describe the present via data. They do this by exploring the data and creating visualizations and dashboards. To do these tasks, they often have to clean data first. Analysts have less programming and stats experience than the other roles. Within the workflow, they focus on the middle two stages: data preparation and exploration and visualization.

Data analyst tools

Data analysts use SQL, the same language used by data engineers, to query data. While data engineers build and configure SQL storage solutions, analysts use existing databases to retrieve and aggregate data relevant to their analysis. Data analysts use spreadsheets to perform simple analyses on small quantities of data. Analysts also use Business Intelligence, or BI tools, such as Tableau, Power BI, or Looker, to create dashboards and share their analyses. More advanced data analysts may be comfortable with Python or R for cleaning and analyzing data.

Data scientist

Data Scientists have a strong background in statistics, enabling them to find new insights from data, rather than solely describing data. They also use traditional machine learning for prediction and forecasting. Within the workflow, they focus on the last three stages: data preparation and exploration and visualization, and experimentation and prediction.

Data scientist tools

Similar to analysts, data scientists have strong skills in SQL. Data scientists must be proficient in at least Python on R. Within these languages, they use popular data science libraries, such as pandas or tidyverse. These libraries contain reusable code for common data science tasks.

Machine learning scientist

Machine learning scientists are similar to data scientists, but with a machine learning specialization. Machine learning is perhaps the buzziest part of Data Science; it’s used to extrapolate what’s likely to be true from what we already know. These scientists use training data to classify larger, unrulier data, whether it’s to classify images that contain a car or create a chatbot. They go beyond traditional machine learning with deep learning. Within the workflow, they do the last three stages with a strong focus on prediction.

Machine learning tools

Machine learning scientists use either Python or R to create their predictive models. Within these languages, they use popular machine learning libraries, such as TensorFlow, to run powerful deep learning algorithms.

It may be intimidating to see all these tools and languages, but they aren’t as difficult to learn as spoken languages. If you know English, it may take you years to learn French. Programming languages are more similar to power tools. If you know how to use a power drill, you don’t necessarily know how to use an electric saw, but you can learn with a little training!

you might be wondering where to start your data career journey as you might be interested in one of the careers mentioned above or want to have more knowledge on a particular term, you can easily click here to get started.

This is the end of the article, Thanks for reading.