Pandas Series and DataFrames: Your Table Toolkit

Everything you have learned so far leads here. Pandas is the library that turns Python into a serious data analysis tool. It gives you the DataFrame, a table with labelled rows and columns, much like a spreadsheet or a SQL table, but programmable, repeatable and able to handle millions of rows. If you have used SQL, think of a DataFrame as a result set that you can keep slicing, grouping and reshaping in memory. If you have used Excel, think of it as a sheet where every formula is applied to the whole column in one line.

In this lesson we meet the two core objects, the Series (one column) and the DataFrame (a table of columns), load the FreshBasket files, and learn the routine every analyst performs on every new dataset: look at the first rows, check the shape, check the data types, and summarise. We will also do our first calculations. The next lessons build on this: selecting and filtering, cleaning, grouping, joining and dates.

Worked examples

Example 1: A Series

Question: What is a Series, and how does it differ from a list or an array?

import pandas as pd
import numpy as np

prices = pd.Series([68, 45, 96, 62], index=["Milk", "Curd", "Eggs", "Bread"], name="price")
print(prices)
print(prices["Eggs"], prices.iloc[0])
print(prices * 2)
print(prices.mean(), prices.idxmax(), prices.dtype)
print(prices[prices > 60])

Milk     68
Curd     45
Eggs     96
Bread    62
Name: price, dtype: int64
96 68
Milk     136
Curd      90
Eggs     192
Bread    124
Name: price, dtype: int64
67.75 Eggs int64
Milk     68
Eggs     96
Bread    62
Name: price, dtype: int64

How it works: A Series is a one-dimensional array with an index (labels) and an optional name. It behaves like a NumPy array (vectorised arithmetic, boolean masks) but you can look values up by label as well as by position. Operations keep the labels attached, which is what makes pandas so convenient.

Reading the result: The most expensive item is Eggs and the average price is ₹67.75. The boolean filter kept only the items above 60, which are ['Milk', 'Eggs', 'Bread'].

Example 2: A DataFrame from scratch

Question: Build a small table from a dictionary.

df = pd.DataFrame({
    "store_id": ["S01", "S02", "S03"],
    "city": ["Bengaluru", "Bengaluru", "Mumbai"],
    "orders": [1301, 905, 945],
    "avg_minutes": [23.4, 24.1, 22.8],
})

print(df)
print(df.shape, list(df.columns), df.index.tolist())
print(df.dtypes)
print(type(df["orders"]).__name__, type(df).__name__)

  store_id       city  orders  avg_minutes
0      S01  Bengaluru    1301         23.4
1      S02  Bengaluru     905         24.1
2      S03     Mumbai     945         22.8
(3, 4) ['store_id', 'city', 'orders', 'avg_minutes'] [0, 1, 2]
store_id           str
city               str
orders           int64
avg_minutes    float64
dtype: object
Series DataFrame

How it works: A dictionary of equal-length lists becomes a DataFrame: keys are column names and lists are the columns. Each column is a Series, all sharing one index (here the default 0, 1, 2). Every column has its own dtype, which is the key difference from a NumPy 2-D array.

Reading the result: The table has 3 rows and 4 columns; orders is stored as int64 and city as str. A DataFrame is basically a dictionary of Series.

Example 3: Loading a CSV

Question: Load the orders file and look at it.

orders = pd.read_csv("orders.csv")

print(orders.head())
print(orders.shape)
print(orders.tail(3))

  order_id customer_id store_id  order_date  delivery_slot payment_method order_status  delivery_minutes  discount_pct
0   O02299       C0078      S03  2025-01-01  Evening (4-8)           Card    Delivered              34.0            15
1   O04533       C0230      S07  2025-01-01  Midday (10-2)            UPI    Delivered              24.0            15
2   O00778       C0328      S09  2025-01-02   Night (8-11)           Card    Delivered              37.0            15
3   O03607       C0310      S02  2025-01-02  Evening (4-8)            UPI    Delivered              29.0             0
4   O04377       C0280      S09  2025-01-02  Evening (4-8)            COD    Delivered              20.0             0
(6000, 9)
     order_id customer_id store_id  order_date  delivery_slot payment_method order_status  delivery_minutes  \
5997   O05315       C0561      S04  2026-06-30   Night (8-11)            UPI    Delivered              25.0   
5998   O05942       C0409      S01  2026-06-30  Evening (4-8)            UPI    Delivered              35.0   
5999   O05950       C0313      S06  2026-06-30  Evening (4-8)            UPI    Delivered              26.0   

      discount_pct  
5997            10  
5998             0  
5999             0  

How it works: pd.read_csv reads the whole file, uses the first row as column names, and guesses a dtype for each column. head() shows the first five rows (give it a number for more or fewer) and tail() shows the last ones. This is always the first command you run on a new file, because it reveals the column names and what the values look like.

Reading the result: The orders table has 6,000 rows and 9 columns. Sketch your questions from these columns: what is in order_status? Is delivery_minutes ever blank?

Example 4: Structure and types

Question: Check column types and missing values.

orders.info()
print()
print(orders.dtypes)
print(orders.isna().sum())
print(orders["order_date"].head(2).tolist())

<class 'pandas.DataFrame'>
RangeIndex: 6000 entries, 0 to 5999
Data columns (total 9 columns):
 #   Column            Non-Null Count  Dtype  
---  ------            --------------  -----  
 0   order_id          6000 non-null   str    
 1   customer_id       6000 non-null   str    
 2   store_id          6000 non-null   str    
 3   order_date        6000 non-null   str    
 4   delivery_slot     6000 non-null   str    
 5   payment_method    6000 non-null   str    
 6   order_status      6000 non-null   str    
 7   delivery_minutes  5630 non-null   float64
 8   discount_pct      6000 non-null   int64  
dtypes: float64(1), int64(1), str(7)
memory usage: 422.0 KB

order_id                str
customer_id             str
store_id                str
order_date              str
delivery_slot           str
payment_method          str
order_status            str
delivery_minutes    float64
discount_pct          int64
dtype: object
order_id              0
customer_id           0
store_id              0
order_date            0
delivery_slot         0
payment_method        0
order_status          0
delivery_minutes    370
discount_pct          0
dtype: int64
['2025-01-01', '2025-01-01']

How it works: info() prints the number of non-null values and the dtype of every column plus the memory use. isna().sum() counts missing values per column (True counts as 1). Note that order_date is currently text (shown as str in pandas 3), not a date, which the dates lesson will fix. The numeric column delivery_minutes is a float because blanks are stored as NaN.

Reading the result: Only delivery_minutes has missing values: 370 of them, exactly the number of cancelled orders. Knowing this matters when you take averages.

Example 5: Summary statistics

Question: Get quick statistics for the numeric columns, and for the text columns.

print(orders.describe())
print(orders.describe(include="str"))
print(orders["delivery_minutes"].describe().round(2))

       delivery_minutes  discount_pct
count       5630.000000   6000.000000
mean          23.669627      5.853333
std            6.929812      6.871173
min            8.000000      0.000000
25%           19.000000      0.000000
50%           24.000000      5.000000
75%           28.000000     10.000000
max           54.000000     20.000000
       order_id customer_id store_id  order_date  delivery_slot payment_method order_status
count      6000        6000     6000        6000           6000           6000         6000
unique     6000         588       11         540              4              4            3
top      O02299       C0406      S07  2026-04-12  Evening (4-8)            UPI    Delivered
freq          1         232      689          34           2220           3330         5391
count    5630.00
mean       23.67
std         6.93
min         8.00
25%        19.00
50%        24.00
75%        28.00
max        54.00
Name: delivery_minutes, dtype: float64

How it works: describe() reports count, mean, standard deviation, minimum, quartiles and maximum for numeric columns. With include="str" it summarises text columns instead: count, number of unique values, the most frequent value and its frequency. Always read the count row: it shows the non-missing numbers, so a column with fewer entries than the others has blanks.

Reading the result: The mean delivery time is 23.7 minutes and the slowest delivery took 54 minutes. The most common payment method is UPI.

Example 6: Selecting columns and adding new ones

Question: Pick some columns and add a calculated one.

print(orders["order_status"].head(3).tolist())
print(orders[["order_id", "order_status"]].head(3))

orders["is_fast"] = orders["delivery_minutes"] <= 20
orders["slot_short"] = orders["delivery_slot"].str.split(" ").str[0]

print(orders[["order_id", "delivery_minutes", "is_fast", "slot_short"]].head(4))
print(orders["is_fast"].sum(), orders["is_fast"].mean().round(3))
print(orders["slot_short"].unique())

['Delivered', 'Delivered', 'Delivered']
  order_id order_status
0   O02299    Delivered
1   O04533    Delivered
2   O00778    Delivered
  order_id  delivery_minutes  is_fast slot_short
0   O02299              34.0    False    Evening
1   O04533              24.0    False     Midday
2   O00778              37.0    False      Night
3   O03607              29.0    False    Evening
1836 0.306
['Evening' 'Midday' 'Night' 'Morning']

How it works: One pair of brackets with a name returns a Series; a list of names inside brackets returns a DataFrame. Assigning to a new column name creates it. .str unlocks vectorised string methods, here splitting Evening (4-8) and keeping the first word. A boolean column can be summed (count of True) or averaged (the share).

Reading the result: 1,836 orders arrived in 20 minutes or less, which is 30.6% of all orders (cancelled orders count as not fast because NaN <= 20 is false). The delivery slot names are ['Evening', 'Midday', 'Night', 'Morning'].

Example 7: Column arithmetic with order items

Question: Compute the line value of every order item and the total.

items = pd.read_csv("order_items.csv")
products = pd.read_csv("products.csv")

items["line_value"] = items["quantity"] * items["unit_price"]
print(items.head(3))
print(items["line_value"].sum().round(2))
print(items["line_value"].describe().round(2))
print(items["quantity"].value_counts().sort_index())

  order_id product_id  quantity  unit_price  line_value
0   O00001       P046         1       340.0       340.0
1   O00001       P015         1        55.0        55.0
2   O00001       P005         2       180.0       360.0
3955409.0
count    19962.00
mean       198.15
std        224.94
min         15.00
25%         62.00
50%        112.00
75%        240.00
max       2560.00
Name: line_value, dtype: float64
quantity
1    10981
2     5682
3     2311
4      988
Name: count, dtype: int64

How it works: Multiplying two columns multiplies row by row, exactly like NumPy. No loop is needed. value_counts() produces a frequency table (a Series indexed by the distinct values), and sort_index() orders it by the values rather than by frequency.

Reading the result: The 19,962 order lines add up to ₹3,955,409 (all statuses). Most lines have quantity 1, and the biggest single line is worth ₹2,560.

Example 8: value_counts and nunique

Question: How are orders spread across statuses, payment methods and stores?

print(orders["order_status"].value_counts())
print(orders["order_status"].value_counts(normalize=True).round(3))
print(orders["payment_method"].value_counts().head(3))
print(orders["store_id"].nunique(), orders["customer_id"].nunique())
print(orders["customer_id"].value_counts().head(3))

order_status
Delivered    5391
Cancelled     370
Returned      239
Name: count, dtype: int64
order_status
Delivered    0.898
Cancelled    0.062
Returned     0.040
Name: proportion, dtype: float64
payment_method
UPI     3330
Card    1225
COD      976
Name: count, dtype: int64
11 588
customer_id
C0406    232
C0532    202
C0122    125
Name: count, dtype: int64

How it works: value_counts() is the pandas equivalent of GROUP BY ... COUNT(*) ORDER BY count DESC. Pass normalize=True to get proportions. nunique() is COUNT(DISTINCT ...). Counting customer ids shows who orders most.

Reading the result: 89.8% of orders were delivered. The business has 588 distinct customers ordering from 11 stores, and the most active customer placed 232 orders.

Example 9: Sorting

Question: Find the five slowest deliveries and the five fastest.

slowest = orders.sort_values("delivery_minutes", ascending=False).head(5)
print(slowest[["order_id", "store_id", "delivery_minutes"]])

fastest = orders.nsmallest(3, "delivery_minutes")[["order_id", "delivery_minutes"]]
print(fastest)

two_keys = orders.sort_values(["store_id", "delivery_minutes"], ascending=[True, False]).head(3)
print(two_keys[["store_id", "order_id", "delivery_minutes"]])

     order_id store_id  delivery_minutes
5209   O03914      S05              54.0
5593   O01116      S11              53.0
5833   O05843      S04              53.0
3153   O00750      S04              52.0
4263   O05255      S07              48.0
    order_id  delivery_minutes
32    O05089               8.0
107   O02068               8.0
145   O02175               8.0
     store_id order_id  delivery_minutes
362       S01   O00231              45.0
3915      S01   O02003              44.0
4419      S01   O02698              42.0

How it works: sort_values takes a column (or list of columns) and an ascending flag (or a list of flags, one per column), similar to ORDER BY. nlargest and nsmallest are shortcuts for the top-N. Rows keep their original index labels, so you see numbers jump around; use reset_index(drop=True) for a fresh 0, 1, 2 sequence.

Reading the result: The slowest delivery was 54 minutes. Missing values are placed last by default, so cancelled orders do not pollute the top of the list.

Example 10: Saving results

Question: Write a summary table to a CSV file and read it back.

summary = (
    orders.groupby("store_id")
    .agg(orders=("order_id", "count"), avg_minutes=("delivery_minutes", "mean"))
    .round(2)
    .reset_index()
)
print(summary.head(3))

summary.to_csv("store_summary.csv", index=False)
back = pd.read_csv("store_summary.csv")
print(back.shape, back.equals(summary))

  store_id  orders  avg_minutes
0      S01     682        23.57
1      S02     619        23.98
2      S03     457        23.51
(11, 3) True

How it works: A first taste of groupby (the grouping lesson will cover it in depth) builds a per-store summary. to_csv(..., index=False) writes it without the row index, which you almost always want; otherwise you get an unnamed extra column when reading back. equals confirms a perfect round trip.

Reading the result: The store summary has 11 rows, and the busiest store is S07 with 689 orders.

Common mistakes

1. Forgetting index=False in to_csv.

2. Expecting df.shape to be a method. It is an attribute: no brackets.

3. Using a Python loop over rows when a vectorised column expression exists.

4. Selecting one column with double brackets and expecting a Series. df[["a"]] is a DataFrame, df["a"] is a Series.

5. Misspelling a column name.

orders["Order_Status"]

Traceback (most recent call last):
  File "example.py", line 3641, in get_loc
    return self._engine.get_loc(casted_key)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
  File "example.py", line 168, in pandas._libs.index.IndexEngine.get_loc
  File "example.py", line 197, in pandas._libs.index.IndexEngine.get_loc
  File "example.py", line 7668, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "example.py", line 7676, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'Order_Status'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "example.py", line 1, in <module>
    orders["Order_Status"]
    ~~~~~~^^^^^^^^^^^^^^^^
  File "example.py", line 4378, in __getitem__
    indexer = self.columns.get_loc(key)
  File "example.py", line 3648, in get_loc
    raise KeyError(key) from err
KeyError: 'Order_Status'

Practice questions

Practice 1 (Easy): First look

Load customers.csv and print its shape, the column names and the number of Prime customers.

customers = pd.read_csv("customers.csv")
print(customers.shape)
print(list(customers.columns))
print(customers["is_prime"].sum())

(600, 7)
['customer_id', 'name', 'email', 'city', 'signup_date', 'acquisition_channel', 'is_prime']
161

Practice 2 (Medium): Margin per product

Load products.csv, compute a margin_pct column as (unit_price - cost_price) / unit_price * 100, and show the three highest-margin products.

products["margin_pct"] = (products["unit_price"] - products["cost_price"]) / products["unit_price"] * 100
top = products.nlargest(3, "margin_pct")[["product_name", "category", "margin_pct"]].round(1)
print(top)

               product_name       category  margin_pct
41          Face Wash 100ml  Personal Care        34.5
26  Digestive Biscuits 400g         Snacks        31.5
39          Toothpaste 150g  Personal Care        30.8

Practice 3 (Hard): Orders per customer distribution

Count the orders per customer, then show how many customers placed 1, 2, 3, ... orders (a frequency of frequencies), sorted by the number of orders.

per_customer = orders["customer_id"].value_counts()
distribution = per_customer.value_counts().sort_index()
print(distribution.head(8))
print(per_customer.describe().round(2))

count
1    28
2    43
3    65
4    60
5    60
6    57
7    49
8    33
Name: count, dtype: int64
count    588.00
mean      10.20
std       16.92
min        1.00
25%        4.00
50%        6.00
75%       10.00
max      232.00
Name: count, dtype: float64

Key takeaways

• A DataFrame is a table of labelled columns; each column is a Series.
• The first-look routine is head, shape, info, dtypes, isna().sum() and describe.
• Column arithmetic is vectorised: no loops.
• value_counts and nunique answer "how many of each?" and "how many different?".
• sort_values and nlargest rank rows; missing values go last.
• Save with to_csv(index=False) and verify by reading it back.