Sudipta Deb
Founder of Technical Potpourri, Co-Founder of Shrey Tech, Enterprise Cloud Architect
A Z-score, or standard score, indicates the distance of a data point from the mean. More formally, it quantifies the number of standard deviations a raw score deviates from the population mean.
A Z-score can be positioned on a normal distribution curve. To calculate a Z-score, it is essential to know the mean μ and the population standard deviation σ. These scores range from -3 standard deviations (which would fall to the far left of the normal distribution curve) up to +3 standard deviations (which would fall to the far right of the normal distribution curve).
Z-scores are a way to compare results to a “normal” population. Results from tests or surveys have thousands of possible results and units; those results can often seem meaningless. For example, knowing that someone’s earning is 100 K dollar might be good information, but if you want to compare it to the “average” person’s earning, looking at a vast table of data can be overwhelming. The scores can tell you where that person’s earning is compared to the average population’s mean earning.
I have published a dedicated post covering the topic Z-Score. If you want to know in details about Z-Score, I would highly recommend going through that post first. Link is given below.
The form you have selected does not exist.
Sample Code
I normally put all my codes in my personal Github repository, so that I can easily share with others. Here is the Github Code Repo where you can find the code samples used in this blog post.
Z-Score And Outliers
In a Normal Distribution, it is estimated that 68% of the data points lie between +/- 1 standard deviation, 95% of the data points lie between +/- 2 standard deviation, 99.7% of the data points lie between +/-3 standard deviation.
Now if the Z-Score of a data point is more than 3, it indicates that the data point is quite different from rest of the data points. Such a data points are called Outliers.
Calculate Outliers Using Z-Score
import numpy as np
data = [1,1,2,2,1188,2,3,4,1,2,2]
print(data)
mean = np.mean(data)
standard_deviation = np.std(data)
print('Mean: ',mean)
print("Standard Deviation: ", standard_deviation)Output:
[1, 1, 2, 2, 1188, 2, 3, 4, 1, 2, 2]
Mean: 109.81818181818181
Standard Deviation: 340.95209425740796
# Find the Outliers
threshold = 3
outliers = []
for index in data:
z_score = (index - mean) / standard_deviation
if(z_score >= threshold):
outliers.append(index)
print(outliers)Output:
[1188]
So, we can easilly see that number 1188 is the outlier in the dataset.
Disclaimer
This article is not endorsed by Salesforce, Google, or any other company in any way. I shared my knowledge on this topic in this blog post. Please always refer to Official Documentation for the latest information.
0 Comments