Pandas Cheat Sheet

by Justin1209 (Justin1209) via [Link]/101982/cs/21202/

Import the Pandas Module Loading and Saving CSVs (cont) Converting Datatypes

import pandas as pd > # Get the first DataFrame chunk: # Convert argument to numeric type
df_urb_pop [Link]​_nu​mer​ic(arg, errors​="ra​‐
Create a DataFrame df_urb_pop = next(u​rb_​pop​_re​ader) ise​")

# Method 1 errors:
Inspect a DataFrame "​rai​se" -> raise an exception
df1 = [Link]​aFr​ame({
[Link](5) First 5 rows "​coe​rce​" -> invalid parsing will be set as NaN
​ ​ 'name':
​ ['John Smith',
'Jane Doe'], [Link]() Statistics of columns (row
DataFrame for Select Columns / Rows
​ ​ 'address':
​ ['13 Main St.', count, null values, datatype)
'46 Maple Ave.'], df = [Link]([

​ ​ 'age':
​ [34, 28] Reshape (for Scikit) ​ ​['J​anu​ary', 100, 100, 23,

}) 100],
nums = [Link](range(1, 11))
# Method 2 ​ ​['F​ebr​uary', 51, 45, 145,
-> [ 1 2 3 4 5 6 7 8 9 10]
df2 = [Link]​aFr​ame([ 45],
nums = nums.r​esh​ape(-1, 1)
​ ​ ​ ​['John Smith', '123 Main ​ ​['M​arch', 81, 96, 65, 96],
-> [ [1],
St.', 34], ​ ​['A​pril', 80, 80, 54, 180],
[2],
​ ​ ​ ​['Jane Doe', '456 Maple ​ ​['May', 51, 54, 54, 154],
[3],
Ave.', 28], ​ ​['J​une', 112, 109, 79,
[4],
​ ​ ​ ​['Joe Schmo', '9 129]],
[5],
Broadway', 51] ​ ​col​umn​s=[​'mo​nth',
[6],
​ ​ ​ ], 'east', 'north', 'south',
[7],
​ ​ columns=['name',
​ 'address', 'west']
[8],
'age']) )
[9],
[10]]
Loading and Saving CSVs Select Columns
You can think of reshape() as rotating this
# Load a CSV File in to a # Select one Column
array. Rather than one big row of numbers,
clinic​_north = [Link]
DataFrame nums is now a big column of numbers -
df = [Link]​d_c​sv(​'my​-cs​v- there’s one number in each row. --> Reshape values for Scikit

f​[Link]') learn: clinic​_north.[Link]​‐

# Saving DataFrame to a CSV File sha​pe(-1, 1)

df.to_​csv​('n​ew-​csv​-fi​‐ # Select multiple Columns

le.c​sv') clinic​_no​rth​_south
= df[['n​‐

# Load DataFrame in Chunks (For orth', 'south']]

large Datasets) Make sure that you have a double set of

# Initialize reader object: brackets [[ ]], or this command won’t work!
urb_po​p_r​eader
urb_po​p_r​eader = [Link]​d_c​‐
sv(​'in​d_p​op_​dat​[Link]',
chunks​ize​=1000)

By Justin1209 (Justin1209) Published 23rd November, 2019. Sponsored by [Link]

[Link]/justin1209/ Last updated 31st January, 2020. Measure your website readability!
Page 1 of 4. [Link]
 Pandas Cheat Sheet
by Justin1209 (Justin1209) via [Link]/101982/cs/21202/

Select Rows Adding a Column Performing Column Operation (cont)

# Select one Row df = [Link]([ > -> lower, upper

march = [Link][2] ​ [1, '3 inch screw', 0.5, # Perform a lambda Operation on a Column
# Select multiple Rows 0.75], get_la​st_name = lambda x: [Link]​t(" "​)[-1]
jan_fe​b_march = [Link]​c[:3] ​ [2, '2 inch nail', 0.10, df['la​st_​name'] = [Link](​get​_la​st_​‐
feb_ma​rch​_april = [Link]​‐ 0.25], name)

c[1:4] ​ [3, 'hammer', 3.00, 5.50],

Performing a Operation on Multiple
may_june = [Link]​c[-2:] ​ [4, 'screw​dri​ver', 2.50,
Columns
# Select Rows with Logic 3.00]
january = df[df.m​onth == ], df = [Link]([
'January'] ​ ​col​umn​s=[​'Pr​oduct ID', ​ ​["Ap​ple​", 1.00, "​No"],
-> <, >, <=, >=, !=, == 'Descr​ipt​ion', 'Cost to ​ ​["Mi​lk", 4.20, "​No"],
march_​april = df[([Link] == Manufa​cture', 'Price'] ​ ​["Paper Towels​", 5.00, "​‐
'March') | ([Link] == ) Yes​"],
'April')] # Add a Column with specified ​ ​["Light Bulbs", 3.75, "​‐
-> &, | row-values Yes​"],
januar​y_f​ebr​uar​y_march = df['Sold in Bulk?'] = ['Yes', ],
df[[Link](['Jan​uary', 'Yes', 'No', 'No'] ​ ​col​umn​s=[​"​Ite​m", "​‐
'Febru​ary', 'March'])] # Add a Column with same value Pri​ce", "Is taxed?​"])
-> column​_na​me.i​si​n([​" ", " in every row # Lambda Function
"]) df['Is taxed?'] = 'Yes' df['Price with Tax'] = [Link]​‐
# Add a Column with calcul​ation ly(​lambda row:
Selecting a Subset of a Dataframe often
df['Re​venue'] = df['Pr​ice'] - ​ ​ ​ ​ ​row​['P​rice'] * 1.075
results in non-co​nse​cutive indices.
df['Cost to Manufa​cture'] ​ ​ ​ ​ if row['Is taxed?'] ==

Using .reset​_in​dex() will create a new 'Yes'

DataFrame move the old indices into a new Performing Column Operation ​ ​ ​ ​ else row['P​rice'],
colum called index. df = [Link]([
​ ​ ​axis=1
​
)
​ ​['JOHN SMITH', 'john.s​mi​‐
Use .reset​_in​dex​(dr​op=​True) if you dont th@​gma​il.c​om'], We apply a lambda to rows, as opposed to
need the index column. columns, when we want to perform functi​‐
​ ​['Jane Doe', 'jdoe@​yah​‐
Use .reset​_in​dex​(in​pla​ce=​True) to prevent a onality that needs to access more than one
oo.c​om'],
new DataFrame from brein created. column at a time.
​ ​['joe schmo', 'joesc​hmo​‐
@ho​tma​il.c​om']
],
column​s=[​'Name', 'Email'])
# Changing a column with an
Operation
df['Name'] = [Link]. apply(​‐
lower)

By Justin1209 (Justin1209) Published 23rd November, 2019. Sponsored by [Link]

[Link]/justin1209/ Last updated 31st January, 2020. Measure your website readability!
Page 2 of 4. [Link]
 Pandas Cheat Sheet
by Justin1209 (Justin1209) via [Link]/101982/cs/21202/

Rename Columns Column Statistics Pivot Tables

# Method 1 Mean = Average [Link]() orders =

[Link] = ['NewN​ame_1', Median [Link]() pd.read_csv('[Link]')
'NewNa​me_2, 'NewNa​me_3', shoe_c​ounts = orders.
Minimal Value [Link]()
'...'] groupb​y([​'sh​oe_​type',
Maximum Value [Link]()
# Method 2 'shoe_​col​or']).
Number of Values [Link]()
[Link](columns={ [Link]​nt(​).r​ese​t_i​ndex()
​ ​ ​ ​'Ol​dNa​me_1': 'NewNa​‐ Unique Values [Link]() shoe_c​oun​ts_​pivot = shoe_c​‐
me_1', Standard Deviation [Link]() oun​ts.p​ivot(
​ ​ ​ ​'Ol​dNa​me_2': 'NewNa​‐ List of Unique Values [Link]() index = 'shoe_​type',
me_2' columns = 'shoe_​color',
Dont't forget reset_​index() at the end of a
}, inplac​e=True
) values = 'id').r​es​et_​index()
groupby operation
Using inplac​e=True lets us edit the original We have to build a temporary table where
DataFrame. Calcul​ating Aggregate Functions we group by the columns we want to

# Group By include in the pivot table

Series vs. Dataframes
grouped = df. groupb​y([​'col1',
Merge (Same Column Name)
# Dataframe and Series 'col2']).col3
print(​typ​e(c​lin​ic_​nor​th)): .measur​ement()
. reset_​index() sales = pd.read_csv('[Link]')
# <class 'panda​s.c​[Link]​rie​s.S​eri​es'> # -> group by column1 and targets = [Link]​d_c​sv(​'ta​‐
print(​typ​e(df)): column2, calculate values of rge​ts.c​sv')
# <class 'panda​s.c​[Link]​[Link]​taF​ram​e'> column3 men_women = [Link]​d_c​sv(​'me​‐
print(​typ​e(c​lin​ic_​nor​th_​south)) n_w​ome​n_s​ale​[Link]')
# Percentile
# <class 'panda​s.c​[Link]​[Link]​taF​ram​e'> # Method 1
high_e​arners = [Link]​upb​y('​‐
In Pandas cat​ego​ry'​).wage sales_​targets = [Link](sales,
- a series is a one-di​men​sional object that apply(​lambda
​ ​ ​ . x: [Link]​‐ targets, how=" ")
contains any type of data. cen​tile(x, 75)) # how: "​inn​er"(​def​ault), "​‐
reset_​index()
​ ​ ​ . out​er", "​lef​t", "​rig​ht"
- a dataframe is a two-di​men​sional object #Method 2 (Method Chaining)
# [Link]​centile can calculate
that can hold multiple columns of different all_data =
any percentile over an array of
types of data.
values [Link](targets).merge(men_w
omen)
A single column of a dataframe is a series, Don't forget reset.i​ndex()
and a dataframe is a container of two or
more series objects.

By Justin1209 (Justin1209) Published 23rd November, 2019. Sponsored by [Link]

[Link]/justin1209/ Last updated 31st January, 2020. Measure your website readability!
Page 3 of 4. [Link]
 Pandas Cheat Sheet
by Justin1209 (Justin1209) via [Link]/101982/cs/21202/

Inner Merge (Different Column Name) Melt

orders = [Link]​lt(​Dat​aFrame, id_vars, value_​vars, var_name, value_​nam​e='​‐

pd.read_csv('[Link]') value')
products = [Link]​d_c​sv(​'pr​‐ id_vars: Column(s) to use as identifier variables.
odu​[Link]') value_​vars: Column(s) to unpivot. If not specified, uses all columns that are not set as id_vars.
# Method 1: Rename Columns var_name: Name to use for the ‘variable’ column.
orders​_pr​oducts = value_​name: Name to use for the ‘value’ column.
[Link](orders, Unpivot a DataFrame from wide to long
[Link](colum​ns=​{'i​‐ format, optionally leaving identi​fiers set.
d':​'pr​odu​ct_​id'}), how=" ")
.reset​_in​dex() Assert Statements
# how: "​inn​er"(​def​ault), "​‐ # Test if country is of type
out​er", "​lef​t", "​rig​ht" object
# Method 2: assert gapmin​[Link]​unt​ry.d​‐
orders​_pr​oducts = types == [Link]
[Link](orders, products, # Test if year is of type int64
​ ​ ​ ​ ​ ​ ​ ​ left_on="pr​‐
​ ​ ​ ​ ​ ​ ​ assert gapmin​[Link]​ar.d​types
odu​ct_​id", == np.int64
​ ​ ​ ​ ​ ​ ​ ​ right_on="id​",
​ ​ ​ ​ ​ ​ ​ # Test if life_e​xpe​ctancy is
​ ​ ​ ​ ​ ​ ​ ​ suffixes=["_​‐
​ ​ ​ ​ ​ ​ ​ of type float64
ord​ers​"​,"_p​rod​uct​s"]) assert gapmin​[Link]​fe_​exp​‐
Method 2: ect​anc​y.d​types == np.float64
If we use this syntax, we’ll end up with two # Assert that country does not
columns called id. contain any missing values
Pandas won’t let you have two columns assert [Link]​nul​l(g​apm​ind​‐
with the same name, so it will change them er.c​ou​ntr​y).a​ll()
to id_x and id_y. # Assert that year does not
We can help make them more useful by contain any missing values
using the keyword suffixes. assert [Link]​nul​l(g​apm​ind​‐
er.y​ea​r).a​ll()
Concat​enate

bakery =
pd.read_csv('[Link]')
ice_cream = [Link]​d_c​sv(​'ic​‐
e_c​rea​[Link]')
menu = [Link]([bakery,
ice_cr​eam])

By Justin1209 (Justin1209) Published 23rd November, 2019. Sponsored by [Link]

[Link]/justin1209/ Last updated 31st January, 2020. Measure your website readability!
Page 4 of 4. [Link]