The cheat sheet is clean and presents you with all of the different methods for getting what you want out of Python! Pro Tip: This is one you should definitely keep laminated and tapped to the desk! The short hands are amazing and come in handy when you know you need one but do not remember quite how to write it out. Df‘cleanzip’ = df‘zip code’.apply(fixzipcode) Explore Our Python Course This entry was posted in Coding, Data and tagged data frame, data science, python 3, python cheat sheet pdf, python cheatsheet, python syntax on May 26, 2020 by Joannie Anderson. Pandas Cheat Sheet for Python For working with data in python, Pandas is an essential tool you must use. This is a fast, powerful, flexible and easy to use open source data analysis and manipulation tool, built on top of the Python programming language.
Pandas is arguably the most important Python package for data science. Not only does it give you lots of methods and functions that make working with data easier, but it has been optimized for speed which gives you a significant advantage compared with working with numeric data using Python’s built-in functions.
- The Pandas cheat sheet will guide you through some more advanced indexing techniques, DataFrame iteration, handling missing values or duplicate data, grouping and combining data, data functionality, and data visualization. In short, everything that you need to complete your data manipulation with Python!
- Data Cleaning: Use these commands to perform a variety of data cleaning tasks. 1 thought on “Pandas Cheat Sheet For Data Science In Python” Ryan.
It’s common when first learning pandas to have trouble remembering all the functions and methods that you need, and while at Dataquest we advocate getting used to consulting the pandas documentation, sometimes it’s nice to have a handy reference, so we’ve put together this cheat sheet to help you out!
If you’re interested in learning pandas, you can consult our two-part pandas tutorial blog post, or you can signup for free and start learning pandas through our interactive pandas for data science course.
Key and Imports
In this cheat sheet, we use the following shorthand:
df | Any pandas DataFrame object |
s | Any pandas Series object |
You’ll also need to perform the following imports to get started:
Importing Data
pd.read_csv(filename) | From a CSV file |
pd.read_table(filename) | From a delimited text file (like TSV) |
pd.read_excel(filename) | From an Excel file |
pd.read_sql(query, connection_object) | Read from a SQL table/database |
pd.read_json(json_string) | Read from a JSON formatted string, URL or file. |
pd.read_html(url) | Parses an html URL, string or file and extracts tables to a list of dataframes |
pd.read_clipboard() | Takes the contents of your clipboard and passes it to read_table() |
pd.DataFrame(dict) | From a dict, keys for columns names, values for data as lists |
Exporting Data
df.to_csv(filename) | Write to a CSV file |
df.to_excel(filename) | Write to an Excel file |
df.to_sql(table_name, connection_object) | Write to a SQL table |
df.to_json(filename) | Write to a file in JSON format |
Create Test Objects
Useful for testing code segements
pd.DataFrame(np.random.rand(20,5)) | 5 columns and 20 rows of random floats |
pd.Series(my_list) | Create a series from an iterable my_list |
df.index = pd.date_range('1900/1/30', periods=df.shape[0]) | Add a date index |
Viewing/Inspecting Data
df.head(n) | First n rows of the DataFrame |
df.tail(n) | Last n rows of the DataFrame |
df.shape() | Number of rows and columns |
df.info() | Index, Datatype and Memory information |
df.describe() | Summary statistics for numerical columns |
s.value_counts(dropna=False) | View unique values and counts |
df.apply(pd.Series.value_counts) | Unique values and counts for all columns |
Python Cheat Sheet Pdf
Selection
df[col] | Return column with label col as Series |
df[[col1, col2]] | Return Columns as a new DataFrame |
s.iloc[0] | Selection by position |
s.loc['index_one'] | Selection by index |
df.iloc[0,:] | First row |
df.iloc[0,0] | First element of first column |
Data Cleaning
df.columns = ['a','b','c'] | Rename columns |
pd.isnull() | Checks for null Values, Returns Boolean Arrray |
pd.notnull() | Opposite of pd.isnull() |
df.dropna() | Drop all rows that contain null values |
df.dropna(axis=1) | Drop all columns that contain null values |
df.dropna(axis=1,thresh=n) | Drop all rows have have less than n non null values |
df.fillna(x) | Replace all null values with x |
s.fillna(s.mean()) | Replace all null values with the mean (mean can be replaced with almost any function from the statistics section) |
s.astype(float) | Convert the datatype of the series to float |
s.replace(1,'one') | Replace all values equal to 1 with 'one' |
s.replace([1,3],['one','three']) | Replace all 1 with 'one' and 3 with 'three' |
df.rename(columns=lambda x: x + 1) | Mass renaming of columns |
df.rename(columns={'old_name': 'new_ name'}) | Selective renaming |
df.set_index('column_one') | Change the index |
df.rename(index=lambda x: x + 1) | Mass renaming of index |
Filter, Sort & Groupby
df[df[col] > 0.5] | Rows where the col column is greater than 0.5 |
df[(df[col] > 0.5) & (1.7)] | Rows where 0.7 > col > 0.5 |
df.sort_values(col1) | Sort values by col1 in ascending order |
df.sort_values(col2,ascending=False) | Sort values by col2 in descending order |
df.sort_values([col1,ascending=[True,False]) | Sort values by col1 in ascending order then col2 in descending order |
df.groupby(col) | Return a groupby object for values from one column |
df.groupby([col1,col2]) | Return groupby object for values from multiple columns |
df.groupby(col1)[col2] | Return the mean of the values in col2, grouped by the values in col1 (mean can be replaced with almost any function from the statistics section) |
df.pivot_table(index=col1,values=[col2,col3],aggfunc=max) | Create a pivot table that groups by col1 and calculates the mean of col2 and col3 |
df.groupby(col1).agg(np.mean) | Find the average across all columns for every unique col1 group |
data.apply(np.mean) | Apply a function across each column |
data.apply(np.max,axis=1) | Apply a function across each row |
Join/Comine
df1.append(df2) | Add the rows in df1 to the end of df2 (columns should be identical) |
df.concat([df1, df2],axis=1) | Add the columns in df1 to the end of df2 (rows should be identical) |
df1.join(df2,on=col1,how='inner') | SQL-style join the columns in df1 with the columns on df2 where the rows for col have identical values. how can be one of 'left', 'right', 'outer', 'inner' |
Statistics
These can all be applied to a series as well.
df.describe() | Summary statistics for numerical columns |
df.mean() | Return the mean of all columns |
df.corr() | Finds the correlation between columns in a DataFrame. |
df.count() | Counts the number of non-null values in each DataFrame column. |
df.max() | Finds the highest value in each column. |
df.min() | Finds the lowest value in each column. |
df.median() | Finds the median of each column. |
df.std() | Finds the standard deviation of each column. |
Download a printable version of this cheat sheet
If you’d like to download a printable version of this cheat sheet you can do so below.
Data Science is rapidly becoming a vital discipline for all types of businesses. An ability to extract insight and meaning from a large pile of data is a skill set worth its weight in gold. Due to its versatility and ease of use, Python has become the programming language of choice for data scientists.
In this Python cheat sheet, we will walk you through a couple of examples using two of the most used data types: the list and the Pandas DataFrame. The list is self-explanatory; it’s a collection of values set in a one-dimensional array. A Pandas DataFrame is just like a tabular spreadsheet, it has data laid out in columns and rows.
Let’s take a look at a few neat things we can do with lists and DataFrames in Python!
Get the pdf here.
Python Cheat Sheet
Lists
Creating Lists
Create an empty list and use a for loop to append new values.
#add two to each value
my_list = []
for x in range(1,11):
my_list.append(x+2)
We can also do this in one step using list comprehensions:
my_list = [x + 2 for x in range(1,11)]
Creating Lists with Conditionals
As above, we will create a list, but now we will only add 2 to the value if it is even.
#add two, but only if x is even
my_list = []
for x in range(1,11):
if x % 2 0:
my_list.append(x+2)
else:
my_list.append(x)
Using a list comp:
my_list = [x+2 if x % 2 0 else x
for x in range(1,11)]
Selecting Elements and Basic Stats
Select elements by index.
#get the first/last element
first_ele = my_list[0]
last_ele = my_list[-1]
Some basic stats on lists:
#get max/min/mean value
biggest_val = max(my_list)
smallest_val = min(my_list)avg_val = sum(my_list) / len(my_list)
DataFrames
Reading in Data to a DataFrame
We first need to import the pandas module.
import pandas as pd
Then we can read in data from csv or xlsx files:
df_from_csv = pd.read_csv(‘path/to/my_file.csv’,
sep=’,’,
nrows=10)
xlsx = pd.ExcelFile(‘path/to/excel_file.xlsx’)
df_from_xlsx = pd.read_excel(xlsx, ‘Sheet1’)
Slicing DataFrames
Python Cheat Sheet
We can slice our DataFrame using conditionals.
df_filter = df[df[‘population’] > 1000000]
df_france = df[df[‘country’] ‘France’]
Sorting values by a column:
df.sort_values(by=’population’,
ascending=False)
Filling Missing Values
Let’s fill in any missing values with that column’s average value.
df[‘population’] = df[‘population’].fillna(
value=df[‘population’].mean()
)
Applying Functions to Columns
Data Structures Cheat Sheet Python
Apply a custom function to every value in one of the DataFrame’s columns.
Basic Python Cheat Sheet
def fix_zipcode(x):
”’
make sure that zipcodes all have leading zeros
”’
return str(x).zfill(5)
df[‘clean_zip’] = df[‘zip code’].apply(fix_zipcode)