Profound Python Data Science

Profound

Python

Data Science

Önder Teker

Godoro Publishing

GODORO PUBLISHING

Publisher Certificate No: 40946

The Name Of The Book:

Profound Python Data Science

Copyright © 2023 Godoro Publishing

The Author Of The Book:

Önder Teker

First Edition, August 2023, Istanbul

ISBN:

978-605-71725-9-4

Cover Design & Makeup:

Önder Teker

Printing & Binding:

Godoro

Special Printing Solutions

Atikali Mah. Fatih Cad. No: 81 D: 2

Fatih / İstanbul

Telefon : (533) 561-2435

http://www.godoro.com

GODORO PUBLISHING

Atikali Mah. Fatih Cad. No: 81 D: 2

Fatih / Istanbul

Phone : (533) 561-2435

http://www.godoro.com

Numpy Basics

The Numpy library can be used to work on numbers or do mathematical or statistical operations. In order to use function in the library, an import statement as below must be added:

import numpy as np

The name np is a short name for the Numpy library. Although another name can be chosen, using np is right because it is widespread and conventional.

In the sections below, Python topics such as array or type are not explained if it is used in the same way as in Python. The subjects related to Numpy are focussed.

Array

The Numpy library includes an array class named  ndarray (n-dimensional array). In addition to the normal array operations, it contains features such as working with arrays of different dimensions, changing dimension count or size, supporting lots of utility functions. These topics are explained when the occassion arises.

One-Dimensional Array

The array() function can be used to create an array. This function takes a Python array as a parameter. An empty array is returned if no element is given as below:

myvector=np.array( [] )

If there ara values, the parameter is given as an array as below:

myvector=np.array( [3,6,1,5] )

In order to write elements of an array, it can be given to the print() function as below:

print(Initialized array:\n,myvector)

The console output will be as below:

[3 6 1 5]

A for loop can be use to print elements as below:

for i in range(0,len(myvector)):

   print((i+1),Element: ,myvector[i])

Example

An example of creating arrays can be written as below:

import numpy as np

vector_empty=np.array([])

print(Empty Vector:\n,vector_empty)

vector_initialized=np.array([3,6,1,5])

print(Initialized Vector:\n,vector_initialized)

print(Vector Elements:)

for i in range(0,len(vector_initialized)):

 print(Element ,(i+1),:,vector_initialized[i])

Yukarıda iki dizi bildirimi yapılmaktadır. İlkinde boş (empty) bir dizi oluşturulmaktadır. İkincisinde ise bir Python dizisi, içinde öğeler olacak biçimde verilerek ilklendirme (initialization) yapılmaktadır.

Ekran çıktısı aşağıdaki gibi oluşur:

Empty Vector:

[]

Initialized Vector:

[3 6 1 5]

Vector Elements:

Element 1 :  3

Element 2 :  6

Element 3 :  1

Element 4 :  5

Elements of Array

Many operations can be done by arrays. Some of these are append, insert and delete.

Append

The append() function can be used to add an element to an array. Example:

vector= np.append( vector,11 )

This call adds the element of 11 to the end of the array.

Insert

Instead of adding to the end, the insert() function can be used to put an element in any place. Example:

vector= np.insert( vector, 2, 12 )

The call puts the value 12 to the element with index value of 2, which is the 3rd element. The elements in the position and the ones after that will be shifted forward one by one.

Delete

In order to delete an element by index, the delete() function can be used. Example:

vector= np.delete(vector,3)

After this operation, the elements after the one with index 3 will be shifted backward.

Example

The example below contains operations of append, insert and delete

Aşağıda iliştirme (append), sokma (insert) ve silme (delete) işlemlerini içeren bir örnek bulunmaktadır:

import numpy as np

vector=np.array([3,6,1,5])

print(Vector Initialized:\n,vector)

vector= np.append(vector,11)

print(Vector Appended:\n,vector)

vector= np.insert(vector,2,12)

print(Vector Inserted:\n,vector)

vector= np.delete(vector,3)

print(Vector Deleted:\n,vector)

Ekran çıktısı aşağıdaki gibi olur:

Array Initialized:

[3 6 1 5]

Array Initialized:

[ 3  6  1  5 11]

Array Initialized:

[ 3  6 12  1  5 11]

Array Initialized:

[ 3  6 12  5 11]

The Element Type of An Array

Unless specified otherwise, the type of elements in an array is determined by the values given as parameters. However, it can be defined as a specific type. The sections below contains some information about types of elements in an array.

Getting Element Type

If an array is defined as below:

vector=np.array( [3, 6, 1, 5] )

the type of elements in the array is defined as int32. The number 32 here denotes 32-bit for the integer number.

To learn the type of elements in an array, the dtype (data type) field in the array object can be used. For example, the code below

print(Array Type:,vector.dtype)

will print the result as below:

Array Type: int32

If there is a difference between types of elements in an array, the widest type is chosen. For instance, if an array contains elements of integer and float, the type of elements is chosen as float. For example, the code below

vector_mixed=np.array( [3,6.2,1,5] )

the value 6.2 is a real number, the type of the array will be float64. This is a 64-bit float number, which is called double in some languages.

Setting Element Type

In order to define the type of elements in an array, dtype (data type) property is given as a parameter in initializing. For example, the code

vector_long=np.array([3,6,1,5],dtype=np.int64)

defines int64 as the type. It is an integer number of 64-bit, which is called long in some languages.

Rounding

The function around() can be used to remove the fractional part of a float (real) number. For example, the code below

vector_round = np.around( vector_real )

turns an array of float numbers to integer values.

Instead of removing all the fractional parts, the function in question can be used to reduce the digits of the fraction. For this purpose, the property named decimals can be used. For example the code below

vector_rounder=np.around( vector_real, decimals=2 )

round values to the 2 digits in the fractional part.

Example

A complete example containing operations about types of elements in arrays can be written as below:

import numpy as np

vector=np.array([3,6,1,5])

print(Array Type:,vector.dtype)

vector_mixed=np.array([3,6.2,1,5])

print(Array Type Mixed:,vector_mixed.dtype)

vector_long=np.array([3,6,1,5],dtype=np.int64)

print(Array Type Long:,vector_long.dtype)

vector_real=np.array([3.14159, 2.71828,1.61803,2.41421])

vector_round=np.around(vector_real)

print(Round Array:)

print(vector_round)

vector_rounder=np.around(vector_real, decimals=2 )

print(Rounder Array:)

print(vector_rounder)

Ekran çıktısı:

Array Type: int32

Array Type Mixed: float64

Array Type Long: int64

Round Array:

[3. 3. 2. 2.]

Rounder Array:

[3.14 2.72 1.62 2.41]

Array Arithmetics

The Numpy library contains many operations of mathematics about arrays. Among them, there are many advanced topics such as linear algebra and statistics. Those used frequently is explained here.

Basic Operations

There are some operations such as add(), subtract(), multiply() and divide() to work on corresponding elements of two arrays. These ara called array arithmetics. For example, if there are two array as below:

vec1=np.array([2.1,2.2,2.3])

vec2=np.array([1.1,1.2,1.3])

some arithmetical operations can be used on them as below:

vec_added = np.add(vec1,vec2)

vec_subtracted = np.subtract(vec1,vec2)

vec_multiplied = np.multiply(vec1,vec2)

vec_divided = np.divide(vec1,vec2)

Mathematical Functions

In addition to basic operations, there are many functions for arrays. For example sqrt() (square root ) funcion can be used as below:

vec_rooted=np.sqrt(vec1)

The result returned above is an array the elements of which are square roots of the elements of array given as parameter.

Instead of returning another array, there are some functions returning a single, scalar value. Fr example, sum() function gets the summation of elements in an array. It can be used as below:

vec_summed = np.sum( vec1 )

The function named dot() finds the dot product of two vectors, as in vectors of linear algebra.

Clip

In order to keep elements of an array in a range of certain values, clip() function can be used. The minimum and maximum values are given as parameters to this function. For example, to make all the values of in array between -10 and 10, a call can be made as below:

vec_clipped = np.clip( vec_untamed, -10, 10 )

In the result array, all the elements below -10 will be equal to -10, all the elements above 10 will be equal to 10.

If it is required, clipping can be done in one direction. For example, minimum() function clips values above a certain value. According to this, it can be secured that the minimum value will be 10 as below:

vec_minimized = np.minimum( vec_untamed, 10 )

Just the opposite of this can be done by the maximum() function. For example, in order to keep all the values up to -10, the statement as below can be written:

vec_maximized = np.maximum(vec_untamed, -10 )

The names minimum and maximum can be confused and used in a reverse direction. The word minimum does not contain the meaning of letting the values of array to a minimum value. Instead, the values above the parameter will be minimized to it. That is, the values above the parameter will be equal to that value. The same thing is true for maximum  function. All the other values below the parameter will be maximized to it.

Example

An examples containing various mathematical operations can be written as below:

import numpy as np

vec1=np.array([2.1,2.2,2.3])

vec2=np.array([1.1,1.2,1.3])

vec_added = np.add(vec1,vec2)

print(Added: ,vec_added)

vec_subtracted = np.subtract(vec1,vec2)

print(Subtracted: ,vec_subtracted)

vec_multiplied = np.multiply(vec1,vec2)

print(Multiplied: ,vec_multiplied)

vec_divided = np.divide(vec1,vec2)

print(Divided: ,vec_divided)

vec_rooted = np.sqrt(vec1)

print(Square Rooted: ,vec_rooted)

vec_summed = np.sum(vec1)

print(Summated: ,vec_summed)

vec_dot = np.dot(vec1,vec2)

print(Dot Product: ,vec_dot)

vec_untamed=[ -16, 3, -1, 34, 28, -11, 18, 7, 5 ]

vec_clipped=np.clip( vec_untamed, -10, 10 )

print(Clipped:)

print(vec_clipped)

vec_minimized=np.minimum( vec_untamed, 10 )

print(Minimized:)

print(vec_minimized)

vec_maximized=np.maximum(vec_untamed, -10 )

print(Maximized:)

print(vec_maximized)

Ekran çıktısı:

Added:  [3.2 3.4 3.6]

Subtracted:  [1. 1. 1.]

Multiplied:  [2.31 2.64 2.99]

Divided:  [1.90909091 1.83333333 1.76923077]

Square Rooted:  [1.44913767 1.4832397  1.51657509]

Summated:  6.6000000000000005

Dot Product:  7.940000000000001

Clipped:

[-10   3  -1  10  10 -10  10   7   5]

Minimized:

[-16   3  -1  10  10 -11  10   7   5]

Maximized:

[-10   3  -1  34  28 -10  18   7   5]

Array Creation

The Numpy library contains some useful functions about array creation.

Empty

The empty() function creates an array without any initialization. The function takes a shape value, which contains the size of an array. No value is assigned to elements of the array. The values will be whatever it is in the memory.

For example, the code below:

array_empty = np.empty( 4 )

will create an array of 4 elements. If the array is printed on the console, they may be equal to any value.

Zeros

In order to create an array, elements of which will be equal to 0, the function zeros() can be used. For example, the use as below

array_zeros = np.zeros(4)

will create an array as below:

[0. 0. 0. 0.]

Ones

If all values in an array is required to be 1, the function ones() can be used. Example:

array_ones=np.ones(4)

The result will be as below:

[1. 1. 1. 1.]

Full

In order to set all elements of an array to a certain value other than 0 or 1, the full() function can be used. For example, the code below

array_full=np.full(4,3.14)

will create an array of 4 elements with the same value:

[3.14 3.14 3.14 3.14]

Gibi (Like)

In some cases, an array must be initialized similar to another one. Even if values of the array are not the same, their size must be equal. For this purpose, functions ending with _like are used. For instance, there is a version of ones() named ones_like(), which creates a new array with the one given as parameter and contains only the value 1.

For example, let's create an array like this:

array_sample = np.array( [9.1, 9.2, 9.3, 9.4 ] )

An array of the same size can be obtained as below:

array_like = np.ones_like( array_zeros )

[1. 1. 1. 1.]

As it can be seen, the shape (size) of the arrays are the same.

Example

There is a complete example of array creation below:

import numpy as np

array_empty=np.empty(4)

print(Array Empty: ,array_empty)

array_zeros=np.zeros(4)

print(Array Zeros: ,array_zeros)

array_ones=np.ones(4)

print(Array Ones: ,array_ones)

array_full=np.full(4,3.14)

print(Array Full: ,array_full)

array_sample=np.array([9.1,9.2,9.3,9.4])

array_like=np.ones_like(array_zeros)

print(Array Like: ,array_like)

When the script runs, the result as below will be produced:

Array Empty:  

[0.00000000e+000 4.24399158e-314 1.11086480e-310 3.46724292e+125]

Array Zeros:  [0. 0. 0. 0.]

Array Ones:  [1. 1. 1. 1.]

Array Full:  [3.14 3.14 3.14 3.14]

Array Like:  [1. 1. 1. 1.]

Random

In some cases, it may be necessary to create an array of random values for reasons such as testing. For this purpose, an interval or size can be given.  Or, a minimum or maximum values are supplied to create values.

Uniform Distribution

There are two ways to generate values randomly. The most basic way is to create them in equal weights in the given interval. It is called uniform distribution. For this, the uniform() function of the numpy.random package can be used.

For example, in order to produce random values between 100 and 200, a piece of code can be written as below:

values=np.random.uniform(100,200,1000)

Normal Distribution

In some cases, using an uniform distribution, which gives equal probability to every value is not correct. For example,  if it is needed to generate heights of human beings between 140cm and 200cm, equal distribution would give approximately the same number of people of average heights as the long or short people. Actually most people are close to the average weight.

In situations like this, a normal distribution is needed. The function normal() of the random package can be used. This function takes  mean (average) value with a standard deviation. For example, in order to generate 1000 number with mean (average) value of 10 and standard deviation of 2, a piece of code can be written as below:

values=np.random.normal( 10.0, 2.0, 1000 )

Shape

Generation of random values can be done for multi dimensional values. For this, a shape value is given as the size parameter. For example, the code below

randoms_table=np.random.uniform(

   low=1, high=6, size=(10,4))

generates a table of 10 rows and 4 columns.

Unscaled Uniform

The function uniform() is a scaled version of a function  generating random values. Actually, it returns a scaled version of unscaled values between 0 and 1. When it is necessary, the raw function named rand() can be used to generate values in the range between 0 and 1. 0 is inclusive, 1 is exclusive.

For example, the code below generates 10 values between 0 and