Converting Between Numpy Arrays Pandas Dataframes

You just spent an hour cleaning a CSV file in Pandas. Your DataFrame looks perfect — labeled columns, no missing values, all the right data types. Now you need to feed it into a scikit-learn model. And scikit-learn? It wants a NumPy array.

So you try model.fit(df). Error.

You try df.values. It works, but now you’ve lost your column names. You try .tolist() and get a nested mess of brackets. You’re not sure if your data is still integers, or if it silently became floats. The error messages are cryptic: “setting an array element with a sequence” or “could not convert string to float.”

Sound familiar?

Here’s the thing: Python data science has three core data structures — lists, NumPy arrays, and Pandas DataFrames — and each speaks a different native language. Lists are for generic storage. Arrays are for fast math. DataFrames are for labeled analysis. Moving data between them is a skill you must master, or you’ll keep hitting these walls.

By the end of this article, you’ll have a mental map of every conversion direction, the gotchas that trip up beginners, and a cheat sheet you can bookmark. Let’s start.

First, a Quick Refresher: What Each Data Structure Is Good At

Before we dive into conversions, let’s get clear on what each structure is. Think of them as tools with distinct personalities.

• Python list: The Swiss Army knife. It holds anything — numbers, strings, other lists, even functions. But it’s slow for math. If you try to add 5 to every element in a list, you need a loop. Lists are for storage, not computation.
• NumPy array: The race car. All elements must be the same type (homogeneous). The payoff? Blazing-fast vectorized operations. array * 2 doubles every element in one go. Arrays are for math.
• Pandas DataFrame: The spreadsheet. It has labeled rows and columns, handles mixed types (integers in one column, strings in another), and comes with powerful analysis tools. DataFrames are for analysis.

In plain English: Lists are for storage, arrays are for math, DataFrames are for analysis.

Let’s see each one in action:

import numpy as np
import pandas as pd

# A Python list
my_list = [1, 2, 3, 4, 5]
print(f"List type: {type(my_list)}")

# A NumPy array
my_array = np.array([1, 2, 3, 4, 5])
print(f"Array type: {type(my_array)}")
print(f"Array shape: {my_array.shape}")

# A Pandas DataFrame
my_df = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]})
print(f"DataFrame type: {type(my_df)}")
print(my_df)

Notice how each one prints differently. The list just shows values. The array shows shape. The DataFrame shows a table with column names. They’re all storing similar data, but they present it — and behave — very differently.

Conversion 1: Python List ↔ NumPy Array (The Easiest Round Trip)

This is the simplest conversion pair. If you understand this, the rest gets easier.

List → Array

Use np.array(your_list). It’s that straightforward for 1D data.

import numpy as np

# 1D list to 1D array
my_list = [10, 20, 30, 40, 50]
my_array = np.array(my_list)
print(f"Original list: {my_list}")
print(f"Array: {my_array}")
print(f"Array shape: {my_array.shape}")
print(f"Array dtype: {my_array.dtype}")

For 2D data (a list of lists), the shape is preserved:

import numpy as np

# 2D list to 2D array
my_2d_list = [[1, 2], [3, 4], [5, 6]]
my_2d_array = np.array(my_2d_list)
print(f"Original 2D list: {my_2d_list}")
print(f"2D array:\n{my_2d_array}")
print(f"Array shape: {my_2d_array.shape}")

Memory tip: If your input is already a NumPy array, np.asarray() is faster because it doesn’t copy the data. Use np.array() when you’re sure you want a new copy.

Array → List

Use .tolist(). This is the cleanest way.

import numpy as np

# 1D array to list
my_array = np.array([100, 200, 300])
my_list = my_array.tolist()
print(f"Array: {my_array}")
print(f"List: {my_list}")
print(f"List type: {type(my_list)}")

Here’s the catch: For 2D arrays, .tolist() returns a nested list. This surprises many beginners.

import numpy as np

# 2D array to nested list
my_2d_array = np.array([[1, 2], [3, 4]])
my_2d_list = my_2d_array.tolist()
print(f"2D array:\n{my_2d_array}")
print(f"Nested list: {my_2d_list}")
print(f"First row: {my_2d_list[0]}")

Gotcha: Using list(array) on a 2D array gives you a list of arrays, not a list of lists. Only .tolist() gives you plain Python lists all the way down.

import numpy as np

# The wrong way for 2D
my_2d_array = np.array([[1, 2], [3, 4]])
wrong_list = list(my_2d_array)
print(f"list() on 2D array: {wrong_list}")
print(f"Type of first element: {type(wrong_list[0])}")  # It's a numpy array, not a list!

Edge case: NumPy 1.24+ raises a ValueError if your list has uneven dimensions (e.g., [[1, 2], [3]]). If you must handle ragged data, use dtype=object:

import numpy as np

# Ragged list - use dtype=object to avoid error
ragged_list = [[1, 2], [3]]
ragged_array = np.array(ragged_list, dtype=object)
print(f"Ragged array: {ragged_array}")
print(f"Shape: {ragged_array.shape}")

Conversion 2: NumPy Array → Pandas DataFrame (Adding Labels)

This is where beginners lose data. The array has no column names, and the DataFrame constructor needs help. If you don’t provide names, you get integer labels.

Basic conversion

import numpy as np
import pandas as pd

# Create a 2D array (3 rows, 4 columns)
array_data = np.array([[1, 2, 3, 4],
                       [5, 6, 7, 8],
                       [9, 10, 11, 12]])

# Convert to DataFrame WITHOUT column names
df_no_names = pd.DataFrame(array_data)
print("DataFrame without column names:")
print(df_no_names)

See the integer column labels (0, 1, 2, 3)? That’s the default. Usually you want meaningful names.

With named columns

import numpy as np
import pandas as pd

array_data = np.array([[1, 2, 3, 4],
                       [5, 6, 7, 8],
                       [9, 10, 11, 12]])

# Convert with column names - this is the 'aha' moment
df_named = pd.DataFrame(array_data, columns=['A', 'B', 'C', 'D'])
print("DataFrame with named columns:")
print(df_named)

Alternative: from_records

Useful when your array is a list of records (each row is a record).

import numpy as np
import pandas as pd

array_data = np.array([[1, 2], [3, 4], [5, 6]])
df_records = pd.DataFrame.from_records(array_data, columns=['X', 'Y'])
print("DataFrame from records:")
print(df_records)

Adding an array as a new column

You can assign an array directly to a new column, but only if the length matches.

import numpy as np
import pandas as pd

# Start with a DataFrame
df = pd.DataFrame({'A': [1, 2, 3]})

# Create an array to add as a new column
new_column_array = np.array([10, 20, 30])

# Assign it
df['B'] = new_column_array
print("DataFrame with array as new column:")
print(df)

Mixed-type gotcha

If your array has strings and numbers, everything becomes object dtype. You’ll need to convert columns after creation.

import numpy as np
import pandas as pd

# Mixed-type array (strings and numbers)
mixed_array = np.array([['Alice', 25], ['Bob', 30], ['Charlie', 35]])

# Convert to DataFrame - everything becomes object
df_mixed = pd.DataFrame(mixed_array, columns=['Name', 'Age'])
print("Mixed-type DataFrame:")
print(df_mixed)
print(f"Dtypes:\n{df_mixed.dtypes}")

# Convert 'Age' back to integer
df_mixed['Age'] = df_mixed['Age'].astype(int)
print(f"\nAfter conversion:\n{df_mixed.dtypes}")

Conversion 3: Pandas DataFrame → NumPy Array (Stripping Labels)

This is the reverse: you have a beautiful DataFrame with column names, but your ML model just wants numbers.

The modern way: .to_numpy()

Introduced in Pandas 0.24, this is cleaner and more explicit than the older .values.

import pandas as pd
import numpy as np

# Create a DataFrame
df = pd.DataFrame({'A': [1, 2, 3], 'B': [4.0, 5.0, 6.0], 'C': [7, 8, 9]})
print("Original DataFrame:")
print(df)
print(f"Dtypes:\n{df.dtypes}")

# Convert to NumPy array
array_from_df = df.to_numpy()
print(f"\nArray from to_numpy():")
print(array_from_df)
print(f"Array dtype: {array_from_df.dtype}")
print(f"Array shape: {array_from_df.shape}")

Notice the dtype became float64. That’s because column B was float, and NumPy arrays are homogeneous — everything gets upcast to the most general type.

Selecting specific columns first

Often you only want numeric columns. Select them before converting.

import pandas as pd
import numpy as np

# DataFrame with mixed types
df_mixed = pd.DataFrame({
    'Name': ['Alice', 'Bob', 'Charlie'],
    'Age': [25, 30, 35],
    'Salary': [50000, 60000, 70000]
})

# Select only numeric columns
numeric_cols = ['Age', 'Salary']
array_numeric = df_mixed[numeric_cols].to_numpy()
print("Numeric columns as array:")
print(array_numeric)
print(f"Dtype: {array_numeric.dtype}")

The .values legacy method

You’ll still see df.values in old code. It works, but .to_numpy() is preferred because it’s more explicit and gives you control over dtype and copying.

import pandas as pd
import numpy as np

df = pd.DataFrame({'A': [1, 2, 3]})

# Legacy method
array_legacy = df.values
print(f"Using .values: {array_legacy}")
print(f"Type: {type(array_legacy)}")

# Modern method
array_modern = df.to_numpy()
print(f"Using .to_numpy(): {array_modern}")
print(f"Type: {type(array_modern)}")

Mixed-type gotcha (again)

When your DataFrame has strings and numbers, .to_numpy() upcasts to object or string dtype. This breaks vectorized operations.

import pandas as pd
import numpy as np

df_mixed = pd.DataFrame({
    'Name': ['Alice', 'Bob'],
    'Age': [25, 30]
})

# Convert the whole DataFrame
array_mixed = df_mixed.to_numpy()
print(f"Mixed DataFrame as array:\n{array_mixed}")
print(f"Dtype: {array_mixed.dtype}")  # object!

# Convert only numeric columns
array_numeric = df_mixed[['Age']].to_numpy()
print(f"\nNumeric only:\n{array_numeric}")
print(f"Dtype: {array_numeric.dtype}")  # int64

Conversion 4: Python List ↔ Pandas DataFrame (The Middleman)

Sometimes you go directly from list to DataFrame without NumPy. This is great for quick prototyping.

List of lists → DataFrame

import pandas as pd

# List of lists
list_of_lists = [
    ['Alice', 25, 'Engineer'],
    ['Bob', 30, 'Designer'],
    ['Charlie', 35, 'Manager']
]

# Convert to DataFrame with column names
df_from_lists = pd.DataFrame(list_of_lists, columns=['Name', 'Age', 'Job'])
print("DataFrame from list of lists:")
print(df_from_lists)

List of dicts → DataFrame

This is the most natural way if you’re scraping data. Keys become column names automatically.

import pandas as pd

# List of dictionaries (like scraped data)
scraped_data = [
    {'city': 'New York', 'temp': 72},
    {'city': 'Los Angeles', 'temp': 85},
    {'city': 'Chicago', 'temp': 60}
]

# Convert to DataFrame - keys become column names
df_scraped = pd.DataFrame(scraped_data)
print("DataFrame from list of dicts:")
print(df_scraped)

DataFrame → list

You need a two-step conversion: DataFrame → array → list.

import pandas as pd
import numpy as np

df = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]})

# Two-step: DataFrame -> array -> list
array_step = df.to_numpy()
list_final = array_step.tolist()
print(f"DataFrame:\n{df}")
print(f"As list: {list_final}")

# Or in one line
list_one_liner = df.to_numpy().tolist()
print(f"One-liner: {list_one_liner}")

Common mistake: list(df) gives you column names, not data!

import pandas as pd

df = pd.DataFrame({'A': [1, 2], 'B': [3, 4]})
wrong_list = list(df)
print(f"list(df) gives: {wrong_list}")  # ['A', 'B'] - just column names!

Performance note

For large data, going through NumPy is faster than direct list→DataFrame. If you have a million rows, use np.array() first, then pd.DataFrame().

The Hard Part: Keeping Your Data Type Intact

This is where most people lose their minds — the dtype changes silently. Let’s name the problem explicitly.

NumPy arrays are homogeneous: if you have integers and floats in the same array, everything becomes float. If you have strings and numbers, everything becomes object.

Pandas DataFrames are column-wise heterogeneous: each column can have its own dtype. But when you convert to an array, that heterogeneity collapses.

Let’s see the surprise:

import pandas as pd
import numpy as np

# DataFrame with int, float, and string columns
df_types = pd.DataFrame({
    'ints': [1, 2, 3],
    'floats': [1.5, 2.5, 3.5],
    'strings': ['a', 'b', 'c']
})

print("Original dtypes:")
print(df_types.dtypes)

# Convert entire DataFrame to array
full_array = df_types.to_numpy()
print(f"\nFull array dtype: {full_array.dtype}")  # object!
print(f"Full array:\n{full_array}")

# Convert only numeric columns
numeric_array = df_types[['ints', 'floats']].to_numpy()
print(f"\nNumeric array dtype: {numeric_array.dtype}")  # float64
print(f"Numeric array:\n{numeric_array}")

See what happened? The full array became object because of the string column. The numeric-only array became float64 because of the float column.

Solution: Always check the dtype after conversion. If you need integers, force the dtype (but beware of data loss).

import pandas as pd
import numpy as np

df = pd.DataFrame({'A': [1, 2, 3], 'B': [4.0, 5.0, 6.0]})

# Force integer dtype (will truncate floats!)
int_array = df.to_numpy(dtype=int)
print(f"Forced int array: {int_array}")
print(f"Dtype: {int_array.dtype}")

Putting It All Together: A Real-World Pipeline

Let’s walk through a complete workflow: scrape data as a list of dicts → clean in a DataFrame → convert to NumPy for ML → add predictions back.

import pandas as pd
import numpy as np
from sklearn.preprocessing import StandardScaler

# Step 1: Simulate scraped data (list of dicts)
scraped_data = [
    {'city': 'NYC', 'population': 8.4, 'area': 302.6, 'density': 27750},
    {'city': 'LA', 'population': 3.9, 'area': 468.7, 'density': 8320},
    {'city': 'Chicago', 'population': 2.7, 'area': 227.6, 'density': 11860},
    {'city': 'Houston', 'population': 2.3, 'area': 669.2, 'density': 3440},
    {'city': 'Phoenix', 'population': 1.7, 'area': 517.9, 'density': 3280}
]

# Step 2: Convert to DataFrame for cleaning
df = pd.DataFrame(scraped_data)
print("Step 2 - Raw DataFrame:")
print(df)
print(f"Shape: {df.shape}")

# Step 3: Select numeric columns for ML (drop 'city')
feature_cols = ['population', 'area', 'density']
X = df[feature_cols].to_numpy()
print(f"\nStep 3 - Feature array shape: {X.shape}")
print(f"Feature array dtype: {X.dtype}")

# Step 4: Scale the features (ML preprocessing)
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
print(f"\nStep 4 - Scaled array:\n{X_scaled[:2]}")

# Step 5: Simulate predictions (dummy: just sum of scaled features)
predictions = X_scaled.sum(axis=1)
print(f"\nStep 5 - Predictions: {predictions}")

# Step 6: Add predictions back to DataFrame as a new column
df['prediction_score'] = predictions.tolist()
print(f"\nStep 6 - Final DataFrame:")
print(df)
print(f"\nFinal shape: {df.shape}")

Notice how the shape changed from (5, 4) to (5, 3) when we selected only numeric columns, then back to (5, 5) after adding predictions. Each conversion was intentional and checked.

Quick Reference: The Conversion Cheat Sheet

Here’s a table to bookmark. It covers all six directions plus the bonus two-step path.

From	To	Method	Notes
List	Array	np.array(list)	Preserves shape; use dtype=object for ragged data
Array	List	array.tolist()	Returns nested list for 2D+; list(array) only works for 1D
Array	DataFrame	pd.DataFrame(array, columns=[...])	Always provide column names to avoid integer labels
DataFrame	Array	df.to_numpy()	Prefer over .values; select columns first for mixed types
List	DataFrame	pd.DataFrame(list_of_dicts) or pd.DataFrame(list_of_lists, columns=[...])	List of dicts auto-creates column names
DataFrame	List	df.to_numpy().tolist()	Two-step conversion; list(df) gives column names, not data
List → Array → DataFrame	DataFrame	pd.DataFrame(np.array(list), columns=[...])	Faster for large data than direct list→DataFrame

What You Learned (and What’s Next)

You now know:

1. Six conversion directions between lists, arrays, and DataFrames.
2. The dtype gotchas — how types change silently, especially with mixed data.
3. A real-world pipeline that moves data through all three structures.
4. The key takeaway: always check the dtype and shape after conversion. Don’t assume.

What’s next? Now that you can move data between structures, you’re ready to handle missing values without breaking everything. NaN in arrays behaves differently than None in lists or NaT in DataFrames. That’s the next article in this series.

Check Your Understanding

Test yourself with these questions, ordered from basic recall to creative application.

Remember: What method converts a NumPy array to a Python list? What happens with 2D arrays?

Understand: Why does pd.DataFrame(array) give integer column names by default? When would that be a problem?

Apply: You have a DataFrame with columns ['name', 'age', 'salary']. Write code to convert only the numeric columns to a NumPy array.

Analyze: A colleague uses list(df) to convert a DataFrame to a list. What will they actually get? Why is this a common mistake?

Evaluate: You have a list of 10 million numbers. Should you convert it directly to a DataFrame, or go through NumPy first? Justify your answer.

Create: Design a function that takes a list of dictionaries (scraped data), converts it to a DataFrame, selects numeric columns, scales them, and returns both the scaled array and the original DataFrame with a new column for the scaled values.

Related articles

• Difference Between Array, List, Series, and DataFrame — A deeper dive into when to use each structure, building on the refresher in this article.
• From Arrays to Lists: Converting Numpy Arrays with Python — Focuses on the .tolist() method and nested list behavior we covered in Conversion 1.
• How to Convert a NumPy Array to Pandas DataFrame — Explores the three methods (constructor, from_records, column assignment) we demonstrated in Conversion 2.