Series, DataFrames and the index

A Series is one column of data: a sequence of values of a single type (its dtype), each paired with a label. The labels are the Series’ index. A DataFrame is a table built from Series: every column is one, the columns can have different dtypes, and all of them share one index, so a row is simply the values of every column that carry the same label. That shared index is the part people coming from spreadsheets tend to miss. When pandas combines two Series it pairs values by label, not by position, so a flat’s rent meets that same flat’s maintenance even when the two lists are in different orders, and a label that only one side has gives a missing value instead of a wrong sum. This lesson builds both structures, shows alignment helping and surprising, and ends with a sensible default: keep the numbered index pandas gives you, and leave keys in ordinary columns until a task needs labels.

Every example is a complete script, and the output under it is what it printed on Python 3.14.8 with pandas 3.0.6. To run them yourself, set up a project with pandas as the lesson on your analysis environment shows.

A Series: values with labels

Build a Series from a list and pandas numbers the values for you, 0, 1, 2 and so on. That default index is a RangeIndex. Build one from a dict and the keys become the labels:

import pandas as pd

# From a list: pandas labels the values 0, 1, 2, ... (a RangeIndex).
marks = pd.Series([72, 85, 91], name="marks")
print(marks)
print(marks.index)

# From a dict: the keys become the labels.
rent = pd.Series({"A-101": 18000, "A-102": 21000, "B-201": 16500}, name="rent")
print(rent)
print(rent.index)
print(rent.dtype, rent.name, rent.size)
print(rent["A-102"])  # one value, looked up by its label
print(rent.to_numpy())  # the values alone, as a NumPy array

0    72
1    85
2    91
Name: marks, dtype: int64
RangeIndex(start=0, stop=3, step=1)
A-101    18000
A-102    21000
B-201    16500
Name: rent, dtype: int64
Index(['A-101', 'A-102', 'B-201'], dtype='str')
int64 rent 3
21000
[18000 21000 16500]

A few things in that output are worth naming:

• RangeIndex(start=0, stop=3, step=1) is how pandas stores the numbered labels: as a start, a stop and a step, not as a list of numbers. The last section of this lesson shows what that saves.
• The labels of rent are flat numbers, so its index has the dtype str, the text type of pandas 3 that the next lesson explains. The values are int64, whole numbers.
• rent["A-102"] looks a value up by its label, and to_numpy() drops the labels and hands back a plain NumPy array, which is what you pass to code that knows nothing about pandas.
• The name travels with the Series. Put the Series into a DataFrame and the name becomes its column name.

A DataFrame: columns that share one index

Tables reach pandas in a handful of shapes, and the constructor accepts each of them. Here is one small table of orders per city, built four ways:

import io

import pandas as pd

# 1. A dict of lists: one list per column.
by_columns = pd.DataFrame({"city": ["Pune", "Delhi", "Kochi"], "orders": [120, 95, 64]})

# 2. A list of dicts: one dict per row, the shape of JSON records.
by_rows = pd.DataFrame([
    {"city": "Pune", "orders": 120},
    {"city": "Delhi", "orders": 95},
    {"city": "Kochi", "orders": 64},
])

# 3. Records: tuples in column order, with the column names given once.
by_records = pd.DataFrame.from_records(
    [("Pune", 120), ("Delhi", 95), ("Kochi", 64)], columns=["city", "orders"]
)

# 4. CSV text, read exactly as a file would be.
by_csv = pd.read_csv(io.StringIO("city,orders\nPune,120\nDelhi,95\nKochi,64\n"))

print(by_columns)
print(by_columns.dtypes)
print(by_columns.shape, by_columns.columns.tolist())
for name, df in [("rows", by_rows), ("records", by_records), ("csv", by_csv)]:
    print(name, "equals the dict of lists:", df.equals(by_columns))

orders = by_columns["orders"]  # one column is a Series with the frame's index
print(type(orders).__name__, orders.dtype, orders.index.equals(by_columns.index))

    city  orders
0   Pune     120
1  Delhi      95
2  Kochi      64
city        str
orders    int64
dtype: object
(3, 2) ['city', 'orders']
rows equals the dict of lists: True
records equals the dict of lists: True
csv equals the dict of lists: True
Series int64 True

• A dict of lists gives one list per column. It is the most direct way to type a small table into code.
• A list of dicts gives one dict per row, which is exactly the shape of JSON records that APIs return. A key that is missing from one dict becomes a missing value in that row.
• Records are tuples in column order, with the column names given once. Rows fetched from a database driver usually arrive like this.
• CSV text goes through read_csv, the same reader you use for files. The reader works out each column’s dtype from the text, which is why orders came out as int64 and city as str, the same as in the dict version.

equals checks that two frames have the same shape, the same values, the same column dtypes and equal row and column labels, and it is True for all four, so they really are the same table. shape is the number of rows and columns, and the last line confirms what the diagram shows: one column, by_columns["orders"], is a Series with the same index as the frame. A DataFrame is a collection of Series lined up on a shared set of row labels.

Arithmetic matches labels, not positions

Two Series from different sources rarely list the same items in the same order. One holds this month’s rent per flat, the other the maintenance bill, and the maintenance list starts at a different flat and includes one the rent list does not have:

import pandas as pd

rent = pd.Series({"A-101": 18000, "A-102": 21000, "B-201": 16500}, name="rent")
maintenance = pd.Series({"A-102": 2500, "B-201": 2200, "C-301": 2400}, name="maintenance")

# + matches labels: each flat's rent meets that flat's maintenance.
monthly = rent + maintenance
print(monthly)
print(monthly.isna().sum(), "flats have no total")

# Adding by position pairs A-101's rent with A-102's maintenance.
print(rent.to_numpy() + maintenance.to_numpy())

# Decide what a missing value means. Here: nothing is owed.
print(rent.add(maintenance, fill_value=0))

A-101        NaN
A-102    23500.0
B-201    18700.0
C-301        NaN
dtype: float64
2 flats have no total
[20500 23200 18900]
A-101    18000.0
A-102    23500.0
B-201    18700.0
C-301     2400.0
dtype: float64

Flats A-102 and B-201 appear in both Series, at different positions, and their totals are right: 21,000 plus 2,500, and 16,500 plus 2,200. A-101 has no maintenance bill and C-301 has no rent, so pandas has nothing to add for them and marks them as missing (NaN, “not a number”). The result has one row for every label found on either side, the union of the two indexes, here in sorted order.

Notice that the whole numbers became float64. NaN is a floating-point value, so a column of integers that gains a missing value is turned into floats to hold it. The next lesson introduces the nullable Int64 type, which keeps the integers and marks the gap as <NA> instead.

The NumPy line of the output shows what alignment protects you from. Adding the two bare arrays pairs the first rent with the first maintenance bill, A-101’s rent with A-102’s bill, and produces three plausible numbers that are all wrong. That is exactly what happens in a spreadsheet when two columns are sorted differently and added row by row.

The final add call makes a decision explicit: add(..., fill_value=0) treats a value missing on one side as 0 before adding, so A-101 owes its rent and C-301 its maintenance. Whether “no bill” means “nothing owed” is a fact about your data, not something pandas can know, which is why + leaves it to you. Every arithmetic operator has a method twin with fill_value (add, sub, mul, div and the rest); a label that is missing on both sides stays missing.

Assigning a column aligns too

Alignment is not limited to arithmetic. Assigning a Series to a DataFrame column also matches labels, which is a classic source of “my sort did nothing”:

import pandas as pd

scores = pd.DataFrame({"student": ["Asha", "Ravi", "Meena", "John"], "score": [62, 88, 75, 91]})

ranked = scores["score"].sort_values(ascending=False)
print(ranked)

# Assigning a Series aligns it on the labels again, so nothing moves.
scores["sorted_score"] = ranked
# A NumPy array has no labels: its values are placed by position.
scores["by_position"] = ranked.to_numpy()
print(scores)

# Labels the frame does not have are dropped; rows with no match get NaN.
bonus = pd.Series([5, 10, 15], index=[1, 3, 7])
scores["bonus"] = bonus
print(scores[["student", "bonus"]])

3    91
1    88
2    75
0    62
Name: score, dtype: int64
  student  score  sorted_score  by_position
0    Asha     62            62           91
1    Ravi     88            88           88
2   Meena     75            75           75
3    John     91            91           62
  student  bonus
0    Asha    NaN
1    Ravi    5.0
2   Meena    NaN
3    John   10.0

sort_values reordered the scores but kept each score’s label (3, 1, 2, 0). Assigning that sorted Series to sorted_score put every value back at its own label, so the new column is identical to score. A NumPy array has no labels, so ranked.to_numpy() is placed by position, and Asha, who scored 62, now sits next to 91. Positional assignment is right only when the order itself carries the meaning.

The bonus Series shows the remaining two rules. Rows 1 and 3 found their labels and got 5 and 10, rows 0 and 2 found nothing and got NaN, and the value labelled 7 was dropped without a word, because the frame has no row 7.

Moving columns into and out of the index

set_index moves a column into the index, and reset_index moves the labels back out into a column:

import pandas as pd

orders = pd.DataFrame({
    "city": ["Pune", "Delhi", "Pune", "Kochi"],
    "month": ["Jan", "Jan", "Feb", "Feb"],
    "orders": [120, 95, 131, 64],
})

by_city = orders.set_index("city")  # the city column becomes the row labels
print(by_city)
print("unique labels:", by_city.index.is_unique)

print(type(by_city.loc["Delhi"]).__name__)  # one row has the label: a Series
print(by_city.loc["Delhi"])
print(type(by_city.loc["Pune"]).__name__)  # two rows share it: a DataFrame
print(by_city.loc["Pune"])

print(by_city.reset_index())  # the labels become a column again

      month  orders
city               
Pune    Jan     120
Delhi   Jan      95
Pune    Feb     131
Kochi   Feb      64
unique labels: False
Series
month     Jan
orders     95
Name: Delhi, dtype: object
DataFrame
     month  orders
city              
Pune   Jan     120
Pune   Feb     131
    city month  orders
0   Pune   Jan     120
1  Delhi   Jan      95
2   Pune   Feb     131
3  Kochi   Feb      64

The labels of an index do not have to be unique. Pune appears twice, and is_unique says so. That has a consequence that catches people out: loc (label lookup, covered in the next lessons) returns a Series when a label belongs to one row and a DataFrame when it belongs to several. Code that reads by_city.loc[city]["orders"] as one number works for Delhi and breaks for Pune, and it can start breaking the day a new file brings a repeated label. A list of labels, by_city.loc[["Delhi"]], always returns a DataFrame, whatever the data.

reset_index turned city back into an ordinary column and gave the frame a fresh numbered index. In pandas 3 the verify_integrity=True option of set_index, which used to refuse repeated labels, is deprecated; check result.index.is_unique yourself when uniqueness matters.

Keep the default index until you need labels

A new DataFrame gets a RangeIndex. Storing a million numbered labels one by one would cost 8 bytes each; the RangeIndex stores three numbers:

import numpy as np
import pandas as pd

n = 1_000_000
default = pd.RangeIndex(n)  # what a new DataFrame gets: start, stop and step
stored = pd.Index(np.arange(n))  # the same labels, stored one by one

print(default.equals(stored))
print(type(default).__name__, default.memory_usage(), "bytes")
print(type(stored).__name__, stored.dtype, stored.memory_usage(), "bytes")

True
RangeIndex 132 bytes
Index int64 8000000 bytes

Both indexes hold exactly the same labels (equals is True), but the RangeIndex takes 132 bytes and the stored one 8,000,000. The space matters less than the habit it suggests: keep your keys in ordinary columns, and move one into the index only when you want label behaviour.

When you do move a key into the index, check index.is_unique straight away, and remember that from then on every arithmetic step aligns on it. Time series are the main exception to “keep it a column”: dates in the index are what resampling and time-based rolling windows use by default, as the time-series module shows.

Practise: combine sales from two systems

The online shop reports sales per city, and the physical stores report their own; the two city lists only partly overlap. Produce one Series of total sales per city that treats a city missing from one system as 0, sorted from the largest total to the smallest, and turn the totals back into whole numbers.

Try it with the online figures Pune 420, Delhi 515 and Kochi 130, and the store figures Delhi 300, Jaipur 210 and Pune 95. One method call does the alignment and the zero together; sort_values(ascending=False) does the order.

import pandas as pd

online = pd.Series({"Pune": 420, "Delhi": 515, "Kochi": 130}, name="online")
shop = pd.Series({"Delhi": 300, "Jaipur": 210, "Pune": 95}, name="shop")

totals = online.add(shop, fill_value=0).sort_values(ascending=False)
print(totals)
print(totals.astype("int64").to_dict())

Delhi     815.0
Pune      515.0
Jaipur    210.0
Kochi     130.0
dtype: float64
{'Delhi': 815, 'Pune': 515, 'Jaipur': 210, 'Kochi': 130}

add with fill_value=0 keeps Kochi (online only) and Jaipur (stores only), and Delhi comes first with 815. The totals are float64 because the alignment produced missing values before the fill replaced them; once you know none are left, astype("int64") turns them back into whole numbers.

Interview questions

Fresher · What is the difference between a pandas Series and a DataFrame, and what is the index for?

A Series is one column of data: values of a single dtype, each with a label. A DataFrame is a table of such columns; each column is a Series with its own dtype, and all of them share one index of row labels. The index is what pandas uses to line data up. Arithmetic and assignment between Series match values by label, not position, so two Series in different orders still combine correctly, and a label present on only one side produces NaN. It also supports lookups by label with loc. By default the index is a RangeIndex, 0 to n−1, which costs almost no memory. You move a key column into it with set_index and back out with reset_index. Labels need not be unique, so a lookup can return several rows.

Key takeaways

• A Series is values of one dtype with a label for each; a DataFrame is a set of Series that share one index.
• Arithmetic and assignment match labels, not positions. The result covers every label on either side, and a label found on one side only becomes NaN.
• Say what a missing partner means with fill_value (add, sub, mul …), and use to_numpy() only when you really mean positions.
• Index labels can repeat, and loc then returns a DataFrame instead of a Series. Check index.is_unique after set_index.
• The default RangeIndex costs a few bytes. Keep keys as ordinary columns until you need label lookups, deliberate alignment or a time series.