Python for Data Science: NumPy, pandas, Statistics and Machine Learning (2026 Guide)

By Skillancy Editorial Team · 20 October 2026 · 13 min read

A code window with pandas and scikit-learn code above a small bar chart

Python for data science means using Python, and a small set of libraries built on it, to load, clean, analyze, visualize and model data. You do not need to master the whole language to start. Most real data science work in Python uses a core of NumPy for numerical arrays, pandas for tables, a plotting library, SciPy for statistics and scikit-learn for machine learning. This guide teaches that core in the order you will actually use it, with one realistic dataset and the real output of every snippet, then gives you a learning roadmap, project ideas and the mistakes to avoid.

Why Python for data science?

One language from cleaning to modelling. You can load a CSV, clean it, test a hypothesis, train a model and automate the report without switching tools.

The libraries are the reason. NumPy, pandas, SciPy, scikit-learn, matplotlib and seaborn are open source, well documented and widely used in industry.

It is readable. Python code reads close to plain English, which matters when you share analysis with teammates.

It connects to everything else. Databases, spreadsheets, APIs, cloud storage and AI tools all have Python libraries, which is why Python is also the language of most generative AI and agent frameworks.

SQL is still the first tool for pulling data out of a database, and it remains central to analyst interviews. Python takes over when the work needs more flexibility than a query: complex cleaning, statistics, automation and machine learning.

How to run the code in this guide

You need nothing installed for the first half. The free Skillancy Python compiler runs Python with NumPy and pandas in your browser. For SciPy, scikit-learn and charts, use Google Colab or a local Jupyter notebook, which come with those libraries. The outputs below were produced with pandas 3; if you are on pandas 2, text columns show their type as object instead of str, and everything else is the same.

Step 1: Python basics you actually need

Before pandas, get comfortable with a handful of building blocks: numbers and strings, lists, dictionaries, loops, list comprehensions and functions.

orders = [1200, 800, 450, 2000, 650]
print(sum(orders) / len(orders))

big = [x for x in orders if x > 700]
print(big)

order = {"order_id": 101, "city": "Bengaluru", "amount": 1200}
print(order["city"])

def gst(amount, rate=0.18):
    return round(amount * rate, 2)

print(gst(1200))

Output:

1020.0
[1200, 800, 2000]
Bengaluru
216.0

That is enough Python to start data work. Classes, decorators and advanced syntax can wait until you need them.

Step 2: NumPy for fast numerical work

NumPy arrays let you apply an operation to every value at once (vectorization) instead of writing loops. pandas is built on top of NumPy, so these ideas carry over directly.

import numpy as np

amounts = np.array([1200, 800, 450, 2000, 650])
print(amounts * 1.18)
print(amounts.mean(), np.median(amounts), amounts.std().round(1))
print(amounts[amounts > 700])

Output:

[1416.  944.  531. 2360.  767.]
1020.0 800.0 548.3
[1200  800 2000]

The last line is a boolean mask: amounts > 700 produces True or False for every value, and indexing with it keeps only the True ones. You will use the same idea constantly in pandas.

One detail worth knowing: NumPy's std() calculates the population standard deviation by default, while pandas' .std() calculates the sample standard deviation. Pass ddof=1 to NumPy if you want them to match.

Step 3: pandas, the heart of Python for data science

pandas gives you the DataFrame: a table with named columns, like a spreadsheet or a SQL table, that you manipulate with code. Here is a small orders dataset with the kind of problems real data has: inconsistent city names, a missing amount, dates stored as text and a duplicated order.

import pandas as pd

orders = pd.DataFrame({
    "order_id":    [101, 102, 103, 104, 105, 105, 106, 107],
    "customer_id": [1, 1, 2, 3, 3, 3, 4, 2],
    "city":        ["Bengaluru", "bengaluru ", "Delhi", "Mumbai", "Mumbai", "Mumbai", "Pune", "delhi"],
    "order_date":  ["2026-01-05", "2026-02-11", "2026-01-20", "2026-02-02",
                    "2026-03-15", "2026-03-15", "2026-03-01", "2026-03-20"],
    "amount":      [1200, 800, None, 2000, 650, 650, 300, 900],
})

print(orders.shape)
print(orders.head(3))
print(orders.isna().sum())

Output:

(8, 5)
   order_id  customer_id        city  order_date  amount
0       101            1   Bengaluru  2026-01-05  1200.0
1       102            1  bengaluru   2026-02-11   800.0
2       103            2       Delhi  2026-01-20     NaN
order_id       0
customer_id    0
city           0
order_date     0
amount         1
dtype: int64

In real projects you will usually load data with pd.read_csv("orders.csv") or pd.read_excel(...), or pull it from a database with SQL, and then run exactly these first checks: shape, a preview, and missing values per column.

Cleaning the data

Cleaning is where most of the time goes, and where interviewers look for judgment.

clean = orders.drop_duplicates(subset="order_id").copy()
clean["city"] = clean["city"].str.strip().str.title()
clean["order_date"] = pd.to_datetime(clean["order_date"])
clean["amount"] = clean["amount"].fillna(clean["amount"].median())

print(clean.shape)
print(clean["city"].value_counts())
print(clean.dtypes)

Output:

(7, 5)
city
Bengaluru    2
Delhi        2
Mumbai       2
Pune         1
Name: count, dtype: int64
order_id                int64
customer_id             int64
city                      str
order_date     datetime64[us]
amount                float64
dtype: object

Four fixes, each a decision you should be able to explain:

Duplicates: order 105 appeared twice. Decide what defines a duplicate (here, the same order ID) before removing anything.

Inconsistent text: "bengaluru " with a trailing space and "delhi" in lowercase would otherwise be counted as separate cities.

Dates as text: converting to a real datetime type enables date filtering, sorting and grouping by month.

Missing values: the missing amount was filled with the median (850). That is a choice, not a default. Sometimes it is better to drop the row, or to keep it and flag it, depending on why the value is missing.

Filtering, sorting and new columns

clean["order_value"] = np.where(clean["amount"] >= 1000, "High", "Standard")
clean["month"] = clean["order_date"].dt.to_period("M").astype(str)

print(
    clean.loc[clean["amount"] >= 800, ["order_id", "city", "amount", "order_value"]]
    .sort_values("amount", ascending=False)
)

Output:

   order_id       city  amount order_value
3       104     Mumbai  2000.0        High
0       101  Bengaluru  1200.0        High
7       107      Delhi   900.0    Standard
2       103      Delhi   850.0    Standard
1       102  Bengaluru   800.0    Standard

.loc[rows, columns] is the standard way to select: a boolean condition for rows and a list for columns.

Grouping and aggregating

groupby is the pandas equivalent of SQL's GROUP BY. Named aggregation keeps the output readable:

summary = (
    clean.groupby("city", as_index=False)
    .agg(orders=("order_id", "count"), revenue=("amount", "sum"), avg_order=("amount", "mean"))
    .sort_values("revenue", ascending=False)
)
print(summary)

Output:

        city  orders  revenue  avg_order
2     Mumbai       2   2650.0     1325.0
0  Bengaluru       2   2000.0     1000.0
1      Delhi       2   1750.0      875.0
3       Pune       1    300.0      300.0

Merging tables

merge is the pandas version of a SQL join. A left merge keeps every customer, even those with no orders:

customers = pd.DataFrame({
    "customer_id": [1, 2, 3, 4, 5],
    "segment": ["Retail", "Retail", "Corporate", "Retail", "Corporate"],
})
merged = customers.merge(clean, on="customer_id", how="left")

print(merged.groupby("segment", as_index=False)["amount"].sum())
print(merged.loc[merged["order_id"].isna(), "customer_id"].tolist())

Output:

     segment  amount
0  Corporate  2650.0
1     Retail  4050.0
[5]

Customer 5 has no orders. If the join logic is new to you, our guide to joins in SQL explains inner, left and outer joins with the same kind of example. The concepts are identical in pandas.

Reshaping with pivot tables

pivot = clean.pivot_table(index="city", columns="month", values="amount",
                          aggfunc="sum", fill_value=0)
print(pivot)

Output:

month      2026-01  2026-02  2026-03
city                                
Bengaluru   1200.0    800.0      0.0
Delhi        850.0      0.0    900.0
Mumbai         0.0   2000.0    650.0
Pune           0.0      0.0    300.0

This is the same summary an Excel pivot table produces, but reproducible: rerun the script next month and the report rebuilds itself.

Step 4: Visualization

Charts are how most analysis reaches decision-makers. matplotlib is the foundation; seaborn adds attractive statistical charts with less code; pandas can plot directly from a DataFrame.

import matplotlib.pyplot as plt

summary.plot(kind="bar", x="city", y="revenue", legend=False, title="Revenue by city")
plt.ylabel("Revenue (₹)")
plt.tight_layout()
plt.show()

Run this in Colab or Jupyter to see the chart. A few rules make charts far more useful: one message per chart, a title that states the finding, labelled axes with units, and bar charts that start at zero.

Step 5: Statistics with pandas and SciPy

Start with descriptive statistics, which pandas gives you in one line:

print(clean["amount"].describe().round(1))
print(round(clean["amount"].mean(), 1), clean["amount"].median())

Output:

count       7.0
mean      957.1
std       534.2
min       300.0
25%       725.0
50%       850.0
75%      1050.0
max      2000.0
Name: amount, dtype: float64
957.1 850.0

The mean is above the median because one large order (₹2,000) pulls it up. With skewed values like order amounts or salaries, report the median as the typical value.

A simple A/B test

Suppose a checkout redesign was shown to 1,000 users (variant B) and the old design to another 1,000 (variant A). A converted 120 users (12%) and B converted 150 (15%). Is the difference real or noise? A chi-square test on the 2×2 table answers that:

from scipy.stats import chi2_contingency

table = [[120, 880],   # variant A: converted, not converted
         [150, 850]]   # variant B
chi2, p, dof, expected = chi2_contingency(table)
print(round(p, 4))

Output:

0.0577

A p-value of about 0.058 is above the common 0.05 threshold, so this test alone does not show a statistically significant difference, even though 15% looks clearly better than 12%. The honest conclusion is "promising, not proven". Collect more data or run the test longer before rolling out, and always report the size of the effect alongside the p-value.

Step 6: Your first machine learning model with scikit-learn

Machine learning in Python follows the same pattern for almost every model in scikit-learn: split the data, fit on training data, predict on unseen test data, evaluate. Here is a customer churn model on a synthetic dataset of 500 customers (generated with a fixed random seed, so the numbers are reproducible):

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

rng = np.random.default_rng(42)
n = 500
customers = pd.DataFrame({
    "tenure_months": rng.integers(1, 60, n),
    "monthly_spend": rng.normal(800, 250, n).round(0),
    "support_tickets": rng.poisson(1.5, n),
})
logit = -1.0 - 0.06 * customers["tenure_months"] + 0.7 * customers["support_tickets"]
customers["churned"] = (rng.random(n) < 1 / (1 + np.exp(-logit))).astype(int)
print(round(customers["churned"].mean(), 3))

X = customers[["tenure_months", "monthly_spend", "support_tickets"]]
y = customers["churned"]
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42, stratify=y
)

model = LogisticRegression(max_iter=1000)
model.fit(X_train, y_train)
pred = model.predict(X_test)

print(round(accuracy_score(y_test, pred), 3))
print(round(1 - y_test.mean(), 3))
print(pd.Series(model.coef_[0], index=X.columns).round(3))

Output:

0.21
0.808
0.792
tenure_months     -0.074
monthly_spend      0.001
support_tickets    0.705
dtype: float64

This output teaches the most important lesson in beginner machine learning:

Compare against a baseline. Only 21% of customers churn, so a "model" that always predicts "no churn" is already 79.2% accurate. Our model's 80.8% is barely better. Accuracy alone is misleading on imbalanced data.

Use metrics that match the business question. For churn, the team cares about catching customers who will leave, so look at recall, precision and ROC AUC (all in sklearn.metrics), not just accuracy.

Read the coefficients. Longer tenure lowers the churn risk and more support tickets raise it, which matches how the data was generated. Spend has almost no effect. In real projects, this kind of interpretation is often more valuable to the business than the prediction itself.

From here, the natural next steps are feature scaling, cross-validation, tree-based models such as random forests and gradient boosting, and clustering for segmentation.

A practical roadmap for learning Python for data science

Weeks	Focus	You can now
1-2	Python basics: types, lists, dictionaries, loops, functions	Write small scripts and read other people's code
3-5	pandas: load, inspect, clean, filter, group, merge, pivot	Turn a messy CSV into a clean summary table
6	NumPy and visualization	Vectorize calculations and present findings in charts
7-8	Statistics: distributions, sampling, hypothesis tests, A/B tests	Say whether a difference is likely real
9-11	Machine learning: regression, classification, evaluation, clustering	Build and honestly evaluate a first model
12	One end-to-end project	Explain a full analysis in an interview

Timelines vary with your starting point and weekly hours. The order matters more than the speed: most people who struggle with machine learning skipped the pandas and statistics stages.

If you prefer to learn this sequence live, with mentors and graded projects, the Data Science with AI program follows the same path, from Python and statistics through machine learning, deep learning and agentic AI, over 12 weeks.

Project ideas to practice

Sales trend analysis: clean a retail sales dataset, build monthly and category summaries, and explain what is driving growth or decline.

A/B test write-up: analyze an experiment, test significance, and write a one-page recommendation that includes effect size and caveats.

Customer segmentation: use RFM (recency, frequency, monetary) features and K-means clustering, then describe each segment in business terms.

Churn prediction: build a classification model, compare it to a baseline, and explain which factors drive churn.

For more ideas with step-by-step briefs, see our list of data analysis projects, and our data science roadmap for the wider skill path.

Common mistakes when learning Python for data science

Spending months on Python syntax before touching data. Start pandas early; learn the remaining syntax as problems require it.

Practicing only on clean datasets. Real work is mostly cleaning. Seek out messy data.

Writing loops over DataFrame rows. Use vectorized operations, groupby and merge; they are clearer and much faster.

Filling missing values without thinking. Ask why values are missing before choosing to drop, fill or flag them.

Judging a model by accuracy alone. Always compare to a baseline and pick metrics that fit the decision.

Not being able to explain the code. In interviews, the explanation of why you chose each step matters as much as the result.

Python for data science interview questions to prepare

1. What is the difference between a list and a NumPy array?

2. How do loc and iloc differ in pandas?

3. How would you find and handle missing values and duplicates?

4. How do merge, join and concat differ?

5. What does groupby return, and how do you aggregate several columns at once?

6. When would you report the median rather than the mean?

7. What is overfitting, and how does a train/test split help detect it?

8. Why can accuracy be misleading, and what would you use instead?

Our free Python interview questions let you practice pandas and NumPy problems in a live editor with instant checking, and our guide to data analyst interview questions covers the rest of the interview loop.

Frequently asked questions

How long does it take to learn Python for data science?

With consistent weekly practice, many learners become comfortable with pandas-based analysis in two to three months, and with basic machine learning a few months after that. Your starting point and weekly time matter more than any fixed timeline.

Do I need to be good at maths to learn data science with Python?

You need comfort with basic statistics (averages, spread, distributions, probability) to start. Linear algebra and calculus help for deeper machine learning, but you can learn them as you go.

Should I learn SQL or Python first?

For analyst roles, SQL first, then Python. For data scientist roles, learn both: SQL to get the data and Python to analyze and model it.

Which Python libraries are most important for data science?

pandas and NumPy first, then matplotlib or seaborn for charts, SciPy for statistics and scikit-learn for machine learning. Deep learning frameworks come later, if your role needs them.

Can I learn Python for data science without a coding background?

Yes. Many data scientists started from commerce, economics, engineering and other non-computer-science backgrounds. Start with the basics in this guide and practice on real datasets every week.

Is Python enough to get a data science job?

Python is necessary but not sufficient. You also need SQL, statistics, a portfolio of projects you can explain, and the ability to communicate findings to non-technical colleagues.

Sources and publishing references

pandas documentation: User guide: https://pandas.pydata.org/docs/user_guide/index.html

NumPy documentation: Absolute basics for beginners: https://numpy.org/doc/stable/user/absolute_beginners.html

scikit-learn documentation: Getting started: https://scikit-learn.org/stable/getting_started.html

SciPy documentation: chi2_contingency: https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.chi2_contingency.html

All code outputs on this page were produced by running the snippets with Python, NumPy, pandas 3, SciPy and scikit-learn.

Python for Data Science: NumPy, pandas, Statistics and Machine Learning (2026 Guide)

Why Python for data science?

How to run the code in this guide

Step 1: Python basics you actually need

Step 2: NumPy for fast numerical work

Step 3: pandas, the heart of Python for data science

Cleaning the data

Filtering, sorting and new columns

Grouping and aggregating

Merging tables

Reshaping with pivot tables

Step 4: Visualization

Step 5: Statistics with pandas and SciPy

A simple A/B test

Step 6: Your first machine learning model with scikit-learn

A practical roadmap for learning Python for data science

Project ideas to practice

Common mistakes when learning Python for data science

Python for data science interview questions to prepare

Frequently asked questions

Recommended internal links

Sources and publishing references