Advanced Programming -

Python -
HANAFI M. YASSINE
2

PREFACE
n today’s technology-driven world, computer science plays a pivotal

I role in shaping innovation across industries. Python, a powerful and

versatile programming language, has become a staple in both aca-
demic and industrial applications. Its simplicity, readability, and vast
ecosystem of libraries make it an ideal language for learning program-
ming fundamentals, which is crucial for third-year engineering students
in computer science.
This course book, Fundamentals of Python, is designed specifically
to provide a strong foundation in Python programming. Whether you
are new to the language or looking to deepen your understanding, this
book offers a comprehensive introduction to the core concepts and prin-
ciples that underpin Python’s effectiveness. From basic syntax to con-
trol structures, and from data types to functions, we aim to cover all
the fundamental elements necessary for developing a solid programming
skill set.
The book emphasizes problem-solving techniques and algorithmic
thinking, which are at the heart of engineering. Each concept is ex-
plained clearly, with practical examples and step-by-step guidance to
ensure that you can translate theory into working code. We also in-
corporate exercises and coding challenges throughout the chapters to
reinforce your learning and encourage hands-on practice.
Key topics include:

• Python syntax and semantics

• Variables and data types

• Control flow (if statements, loops)

• Functions and modularity

• Lists, dictionaries, and other data structures

• File handling

• Basic error handling and debugging techniques

By focusing on these fundamental concepts, this course will prepare you
for more advanced topics in future studies while also giving you the con-
fidence to write clean, efficient, and effective Python code. Throughout
the course, you will not only gain practical skills but also develop the
mindset of an engineer, where logic, precision, and problem-solving are
paramount.
We hope this book serves as a stepping stone on your journey to
becoming a proficient Python programmer and software engineer. With
the tools and concepts covered in this course, you will be well-equipped
to tackle more complex projects and challenges in your academic and
professional career.
 CONTENTS
PART 1 BEGINNING PYTHON . . . . . . . . . . . . 1
1 LEXICAL MATTERS . . . . . . . . . . . . . . 2
1.1 Lines . . . . . . . . . . . . . . . . . . . . 2
1.2 Comments . . . . . . . . . . . . . . . . . 2
1.3 Names and tokens . . . . . . . . . . . . 2
1.4 Blocks and indentation . . . . . . . . . . 3
1.5 Doc strings . . . . . . . . . . . . . . . . 4
1.6 Program structure . . . . . . . . . . . . 4
1.7 Operators . . . . . . . . . . . . . . . . . 5
1.8 Code evaluation . . . . . . . . . . . . . 6
2 STATEMENTS AND INSPECTION - PRELIM-
INARIES . . . . . . . . . . . . . . . . . . . . . 8
3 BUILT-IN DATA-TYPES . . . . . . . . . . . . 9
3.1 Numeric types . . . . . . . . . . . . . . 9
3.2 Tuples and lists . . . . . . . . . . . . . . 9
3.3 Strings . . . . . . . . . . . . . . . . . . . 11
3.4 Dictionaries . . . . . . . . . . . . . . . . 12
3.5 Files . . . . . . . . . . . . . . . . . . . . 14
4 Functions and Classes: A Preview . . . . . . . . 17
4.1 Classes . . . . . . . . . . . . . . . . . . 18
5 STATEMENTS . . . . . . . . . . . . . . . . . . 23
5.1 if: elif: else: . . . . . . . . . . . . . . . 23
5.2 for, while: . . . . . . . . . . . . . . . . 23
6 Functions, Modules, Packages, and Debugging . 26
6.1 Functions . . . . . . . . . . . . . . . . . 26
6.2 Modules . . . . . . . . . . . . . . . . . . 30
6.3 Packages . . . . . . . . . . . . . . . . . 32
PART 2 Advanced Python . . . . . . . . . . . . . . . . . 33
7 Regular Expressions . . . . . . . . . . . . . . . 33
7.1 Regex Module in Python . . . . . . . . . 34
7.2 Metacharacters . . . . . . . . . . . . . . 35
7.3 Special Sequences . . . . . . . . . . . . . 42
7.4 RegEx Functions . . . . . . . . . . . . . 42
7.5 Match Object . . . . . . . . . . . . . . . 48
8 Iterator Objects . . . . . . . . . . . . . . . . . . 50
9 Unit Tests . . . . . . . . . . . . . . . . . . . . . 51
9.1 Why Choose Unittest in Python? . . . . 52
9.2 Assert Methods in Python Unittest Frame-
work . . . . . . . . . . . . . . . . . . . . 52
 9.3 OOP Concepts Supported by Unittest Frame-
work . . . . . . . . . . . . . . . . . . . . 52
9.4 Python Unittest Example . . . . . . . . 53
9.5 Outcomes Possible in Unit Testing . . . 54
10 Extending and embedding Python . . . . . . . . 57
10.1 Extending Python with C or C++ . . . 57
10.2 Embedding Python in Another Application 61
11 Parsing . . . . . . . . . . . . . . . . . . . . . . . 66
11.1 Introduction to XML . . . . . . . . . . . 66
11.2 Key Features of XML . . . . . . . . . . 66
11.3 Common Use Cases of XML . . . . . . . 67
11.4 Overview of xml.etree.ElementTree . 67
11.5 Reading and Parsing XML . . . . . . . . 67
11.6 Navigating and Extracting Data . . . . 67
11.7 Modifying XML . . . . . . . . . . . . . 67
11.8 Key Functions in ElementTree . . . . . . 68
11.9 Advanced Features in XML Parsing . . . 68
12 Python GUI Applications . . . . . . . . . . . . 72
12.1 Use Cases of Tkinter . . . . . . . . . . . 72
12.2 Getting Started with Tkinter . . . . . . 73
12.3 Tkinter Widgets . . . . . . . . . . . . . 75
12.4 Geometry Management . . . . . . . . . 78
 1

PART 1
BEGINNING PYTHON
ython has emerged as one of the most popular programming

P languages for beginners and professionals alike, thanks to its sim-

plicity, readability, and broad range of applications. In this sec-
tion, Beginning Python, we aim to introduce you to the basic building
blocks of the language, laying a strong foundation for your journey into
programming.
As you embark on this chapter, you will learn how to navigate Python’s
straightforward syntax and structure, making it an ideal starting point
for understanding programming concepts. We begin by exploring the
essential elements of Python, including:
• Basic syntax: Understanding how Python is written and executed.
• Variables and data types: learning how to store and manipulate
data using integers, strings, lists, and more.
• Input and output: Interacting with users and displaying results.
• Control structures: Utilizing if statements, loops, and logical op-
erations to control the flow of your programs.
Throughout this section, the focus will be on breaking down these
concepts into manageable pieces, providing practical examples that illus-
trate how each component works in real-world applications. Each topic
is designed to build upon the previous one, ensuring that you develop a
solid, step-by-step understanding of Python’s fundamentals.
By the end of this section, you will have the skills necessary to write
your first Python programs. You’ll also begin to develop a problem-
solving mindset, using Python to break down tasks into logical steps.
This knowledge will not only help you grasp more advanced programming
concepts but will also provide the confidence to apply Python in various
domains, from data analysis to web development.
Whether you are new to programming or transitioning from another
language, this section will serve as your gateway into the Python world,
equipping you with the core skills that every programmer needs.
2 BEGINNING PYTHON

1
LEXICAL MATTERS
Lines
• Python does what you want it to do most of the time so that you
only have to add extra characters some of the time.
• Statement separator is a semicolon but is only needed when there
is more than one statement on a line. And writing more than one
statement on the same line is considered bad form.
• Continuation lines A backslash as last character of the line makes
the following line a continuation of the current line. But, note that
an opening "context" (parenthesis, square bracket, or curly bracket)
makes the backslash unnecessary.

Comments
Everything after "#" on a line is ignored. No block comments, but doc
strings are a comment in quotes at the beginning of a module, class,
method or function. Also, editors with support for Python often provide
the ability to comment out selected blocks of code, usually with " ””” ".

Names and tokens

• Allowed characters: "az", "AZ", "09" underscore, and must begin
with a letter or underscore "_".
• Names and identifiers are case sensitive.
• Identifiers can be of unlimited length.
• Special names, customizing, etc. Usually begin and end in double
underscores.
• Special name classes: Single and double underscores.
– Single leading single underscore: Suggests a "private" method or
variable name. Not imported by "from module import *"
LEXICAL MATTERS 3

– Single trailing underscore: Use it to avoid conflicts with Python

keywords.
– Double leading underscores: Used in a class definition to cause
name mangling (weak hiding). But, not
• Naming conventions: Not rigid, but:
– Modules and packages: all lower case.
– Globals and constants: Upper case.
– Classes: Bumpy caps with initial upper.
– Methods and functions: All lower case with words separated by
underscores.
– Local variables/Lower case (with underscore between words) or
bumpy caps with initial lower or your choice.
– Good advice Follow the conventions used in the code on which
you are working.
• Names/variables in Python do not have a type. Values have types.

Blocks and indentation

Python represents block structure and nested block structure with in-
dentation, not with begin and end brackets.
The empty block: Use the pass no-op statement.
Benefits of the use of indentation to indicate structure:
• Reduces the need for a coding standard. Only need to specify that
indentation is 4 spaces and no hard tabs.
• Reduces inconsistency. Code from different sources follow the same
indentation style. It has to.
• Reduces work. Only need to get the indentation correct, not both
indentation and brackets.
• Reduces clutter. Eliminates all the curly brackets.
• If it looks correct, it is correct. Indentation cannot fool the reader.
4 BEGINNING PYTHON

Editor considerations: The standard is 4 spaces (no hard tabs) for

each indentation level. You will need a text editor that helps you respect
that.

Doc strings
Doc strings are like comments, but they are carried with executing code.
Doc strings can be viewed with several tools, e.g. help(), obj.__doc__,
and, in IPython, a question mark (?) after a name will produce help.
A doc string is written as a quoted string that is at the top of a
module or the first lines after the header line of a function or class.
We can use triplequoting to create doc strings that span multiple
lines.
There are also tools that extract and format doc strings, for example:

• pydoc: Documentation generator and online help system http://

docs.python.org/lib/modulepydoc.html.

• epydoc: Epydoc: Automatic API Documentation Generation for

Python http://epydoc.sourceforge.net/index.html.

• Sphinx Can also extract documentation from Python doc strings.

See http://sphinxdoc.org/index.html.

See the following for suggestions and more information on doc strings:
Docstring conventions http://www.python.org/dev/peps/pep0257/.

Program structure
• Execution def, class, etc are executable statements that add some-
thing to the current namespace. Modules can be both executable
and importable.

• Statements, data structures, functions, classes, modules, packages.

• Functions

• Classes
LEXICAL MATTERS 5

• Modules correspond to files with a "*.py" extension. Packages corre-

spond to a directory (or folder) in the file system; a package contains
a file named "__init__.py". Both modules and packages can be im-
ported (see section import statement).

• Packages A directory containing a file named "__init__.py". Can

provide additional initialization when the package or a module in it
is loaded (imported).

Operators
Python supports various operators, including arithmetic, comparison,
logical, and assignment operators.

• See: http://docs.python.org/ref/operators.html. Python defines

the following operators in Fig 1.

Fig. 1 The operations list

The comparison operators <> and != are alternate spellings of the

same operator. != is the preferred spelling; <> is obsolescent.

• Logical operators:

Fig. 2 Logical Operations

• There are also (1) the dot operator, (2) the subscript operator [ ],
and the function/method call operator ().

• For information on the precedences of operators, see the table at

http://docs.python.org/2/reference/expressions.html#operatorprecedence,
which is reproduced below.
6 BEGINNING PYTHON

• For information on what the different operators do, the section in the
"Python Language Reference" titled "Special method names" may be
of help:
http://docs.python.org/2/reference/datamodel.html#specialmethodnames
The following table summarizes the operator precedence in Python,
from lowest precedence (least binding) to highest precedence (most
binding). Operators on the same line have the same precedence.
Unless the syntax is explicitly given, operators are binary. Opera-
tors on the same line group left to right (except for comparisons,
including tests, which all have the same precedence and chain from
left to right and exponentiation, which groups from right to left):

Fig. 3 Operation Descriptions

• Note that most operators result in calls to methods with special

names, for example __add__, __sub__, __mul__, etc. See Spe-
cial method names:
http://docs.python.org/2/reference/datamodel.html#specialmethodnames
Later, we will see how these operators can be emulated in classes that
you define yourself, through the use of these special names.

Code evaluation

Understanding the Python execution model: How Python evaluates

and executes your code.

– Evaluating expressions.
LEXICAL MATTERS 7

– Creating names/variables (Binding): The following all create

names (variables) and bind values (objects) to them: (1) assign-
ment, (2) function definition, (3) class definition, (4) function
and method call, (5) importing a module, ...
– First class objects: Almost all objects in Python are first class.
Definition: An object is first class if: (1) we can put it in a
structured object; (2) we can pass it to a function; (3) we can
return it from a function.
– References: Objects (or references to them) can be shared. What
does this mean?
∗ The object(s) satisfy the identity test operator is.
∗ The built-in function id() returns the same value.
∗ The consequences for mutable objects are different from those
for immutable objects.
∗ Changing (updating) a mutable object referenced through
one variable or container also changes that object referenced
through other variables or containers because it is the same
object.
∗ del(): The built-in function del() removes a reference, not
(necessarily) the object itself.

Fig. 4 References 1
8 BEGINNING PYTHON

Fig. 5 Result 1

Fig. 6 References 2

Fig. 7 Result 2

2
STATEMENTS AND INSPECTION -
PRELIMINARIES
• print:

Fig. 8 Print function

• for:
• Learn what the type of an object is
BUILT-IN DATA-TYPES 9

Fig. 9 For statement

Fig. 10 Type statement

3
BUILT-IN DATA-TYPES
For information on built-in data types, see section Built-in Types: http:
//docs.python.org/lib/types.html in the Python standard documen-
tation.

Numeric types
The numeric types are:

• Plain integers: Same precision as a C long, usually a 32-bit binary

number.

• Long integers: Define with 100L. But, plain integers are automati-
cally promoted when needed.

• Floats: Implemented as a C double. Precision depends on your

machine.

• Complex numbers: Define with, for example, 3j or complex(3.0,

2.0).

Tuples and lists

List: A list is a dynamic array/sequence. It is ordered and indexable. A
list is mutable.
List constructors: [ ], list(), range() and xrange():
10 BEGINNING PYTHON

• range(n) creates a list of n integers. Optional arguments are the

starting integer and a stride.

• xrange() is like range(), except that it creates an iterator that pro-

duces the items in the list of integers instead of the list itself.

Tuples: A tuple is a sequence. A tuple is immutable. Tuple constructors:

( ), but really a comma; also tuple( ). Tuples are like lists, but are not
mutable.
Python lists are (1) heterogeneous (2) indexable, and (3) dynamic.
For example, we can add to a list and make it longer.
Notes on sequence constructors:

• To construct a tuple with a single element, use (x,); a tuple with a

single element requires a comma.

• You can spread elements across multiple lines (and no need for back-
slash continuation character "¨).

• A comma can follow the last element.

The length of a tuple or list (or other container): len(mylist).

Operators for lists:

• Try: list1 + list2, list1 * n, list1 += list2, etc.

• Comparison operators: <, ==, >=, etc.

• Test for membership with the in operator

Subscription:

• Indexing into a sequence.

• Negative indexe: Effectively, length of sequence plus (minus) index.

• Slicing: Example: data[ 2 : 5 ]. Default values: beginning and end

of list.

• Slicing with strides: Example: data[ : : 2].

BUILT-IN DATA-TYPES 11

Operations on tuples: No operations that change the tuple, since tuples

are immutable. We can do iteration and subscription. We can do "con-
tains" (the in operator) and get the length (the len() operator). We can
use certain boolean operators.
Operations on lists: Operations similar to tuples plus:

• Append: mylist.append(newitem).

• Insert : mylist.insert(index, newitem). Note on efficiency: The insert

method is not as fast as the append method. If you find that you
need to do a large number of mylist.insert(0, obj) (that is, inserting
at the beginning of the list) consider using a deque instead

• Extend: mylist.extend(anotherlist). Also can use + and +=

• Remove: mylist.remove(item) and mylist.pop(). Note that append()

together with pop() implements a stack.

• Delete: del mylist[index].

• Pop: Get last (right-most) item and remove from list: mylist.pop().
List operators: +, *, etc.

Strings
A String is a data structure in Python Programming that represents a
sequence of characters. It is an immutable data type, meaning that once
you have created a string, you cannot change it. Python String are used
widely in many different applications, such as storing and manipulating
text data, representing names, addresses, and other types of data that
can be represented as text.
Strings in Python can be created using single quotes or double quotes
or even triple quotes.
Accessing characters in Python String: In Python Program-
ming tutorials, individual characters of a String can be accessed by using
the method of Indexing. Indexing allows negative address references to
access characters from the back of the String, e.g. -1 refers to the last
character, -2 refers to the second last character, and so on.
12 BEGINNING PYTHON

Fig. 11 String data structure

String Slicing Python: Slicing in a String is done by using a Slicing

operator, i.e., a colon ( : ). One thing to keep in mind while using this
method is that the string returned after slicing includes the character at
the start index but not the character at the last index.

Dictionaries

A Python dictionary is a data structure that stores the value in key :

value pairs.
BUILT-IN DATA-TYPES 13

Fig. 12 Python Output

Fig. 13 Dictionaries

Fig. 14 Dict Results

Python dictionaries are essential for efficient data mapping and ma-
nipulation in programming. To deepen your understanding of dictionar-
ies and explore advanced techniques in data handling, consider enrolling
in our Complete Machine Learning & Data Science Program.
Python Dictionary Syntax:

Fig. 15 Dict Syntax

Dictionary Methods:
14 BEGINNING PYTHON

Table 1 Dictionary Methods

Method Description
dict() This function is used to create a new dictionary or convert other iterable objects into a dictionary.
dict.clear() Remove all the elements from the dictionary
dict.copy() Returns a copy of the dictionary
dict.get(key, default = “None”) Returns the value of specified key
dict.items() Returns a list containing a tuple for each key value pair
dict.keys() Returns a list containing dictionary’s keys
dict.update(dict2) Updates dictionary with specified key-value pairs
dict.values() Returns a list of all the values of dictionary
pop() Remove the element with specified key
popItem() Removes the last inserted key-value pair
dict.setdefault(key,default= “None”) set the key to the default value if the key is not specified in the dictionary
dict.has_key(key) returns true if the dictionary contains the specified key.

Files
File handling in Python is a powerful and versatile tool that can be
used to perform a wide range of operations. However, it is important to
carefully consider the advantages and disadvantages of file handling when
writing Python programs, to ensure that the code is secure, reliable, and
performs well.
Python supports file handling and allows users to handle files i.e.,
to read and write files, along with many other file handling options, to
operate on files.
Python treats files differently as text or binary and this is important.
Each line of code includes a sequence of characters, and they form a text
file. Each line of a file is terminated with a special character, called the
EOL or End of Line characters like comma , or newline character. It
ends the current line and tells the interpreter a new one has begun.
Advantages of File Handling in Python:

• Versatility : File handling in Python allows you to perform a wide

range of operations, such as creating, reading, writing, appending,
renaming, and deleting files.
• Flexibility : File handling in Python is highly flexible, as it allows
you to work with different file types (e.g. text files, binary files, CSV
files , etc.), and to perform different operations on files (e.g. read,
write, append, etc.).
• User – friendly : Python provides a user-friendly interface for file
handling, making it easy to create, read, and manipulate files.
BUILT-IN DATA-TYPES 15

• Cross-platform : Python file-handling functions work across dif-

ferent platforms (e.g. Windows, Mac, Linux), allowing for seamless
integration and compatibility.
Disadvantages of File Handling in Python:
• Error-prone: File handling operations in Python can be prone to
errors, especially if the code is not carefully written or if there are
issues with the file system (e.g. file permissions, file locks, etc.).
• Security risks : File handling in Python can also pose security
risks, especially if the program accepts user input that can be used
to access or modify sensitive files on the system.
• Complexity : File handling in Python can be complex, especially
when working with more advanced file formats or operations. Careful
attention must be paid to the code to ensure that files are handled
properly and securely.
• Performance : File handling operations in Python can be slower
than other programming languages, especially when dealing with
large files or performing complex operations.
Python File Open: Before performing any operation on the file like
reading or writing, first, we have to open that file. For this, we should
use Python’s inbuilt function open() but at the time of opening, we have
to specify the mode, which represents the purpose of the opening file.
Where the following mode is supported:
• r: open an existing file for a read operation.
• w: open an existing file for a write operation. If the file already
contains some data, then it will be overridden but if the file is not
present then it creates the file as well.
• a: open an existing file for append operation. It won’t override
existing data.
• r+: To read and write data into the file. This mode does not override
the existing data, but you can modify the data starting from the
beginning of the file.
16 BEGINNING PYTHON

• w+: To write and read data. It overwrites the previous file if one
exists, it will truncate the file to zero length or create a file if it does
not exist.
• a+: To append and read data from the file. It won’t override existing
data.

The four primary functions used for file handling in Python are:

• open() : Opens a file and returns a file object.

• read() : Reads data from a file.
• write() : Writes data to a file.
• close() : Closes the file, releasing its resources.
FUNCTIONS AND CLASSES: A PREVIEW 17

4
FUNCTIONS AND CLASSES: A PREVIEW

• Structured code: Python programmes are made up of expressions,

statements, functions, classes, modules, and packages.

• Python objects are first class objects.

Fig. 16 First class object

• Expressions are evaluated.

• Statements are executed.

• Functions are objects and callable.

18 BEGINNING PYTHON

Classes
A class is a user-defined blueprint or prototype from which objects are
created. Classes provide a means of bundling data and functionality
together. Creating a new class creates a new type of object, allowing
new instances of that type to be made. Each class instance can have
attributes attached to it to maintain its state. Class instances can also
have methods (defined by their class) for modifying their state.
To understand the need for creating a class and object in Python let’s
consider an example, let’s say you wanted to track the number of dogs
that may have different attributes like breed and age. If a list is used,
the first element could be the dog’s breed while the second element could
represent its age. Let’s suppose there are 100 different dogs, then how
would you know which element is supposed to be which? What if you
wanted to add other properties to these dogs? This lacks organization
and it’s the exact need for classes.

Fig. 17 Class Definition

Fig. 18 Object Definition

The class creates a user-defined data structure, which holds its own
data members and member functions, which can be accessed and used
by creating an instance of that class. A class is like a blueprint for an
object.
Some points on Python class:

• Classes are created by keyword class.

• Attributes are the variables that belong to a class.

FUNCTIONS AND CLASSES: A PREVIEW 19

• Attributes are always public and can be accessed using the dot (.)
operator. Eg.: My class.Myattribute

Creating a Python Class:

Fig. 19 Class Example

Object of Python Class:

In Python programming an Object is an instance of a Class. A class
is like a blueprint while an instance is a copy of the class with actual
values. It’s not an idea anymore, it’s an actual dog, like a dog of breed
pug who’s seven years old. You can have many dogs to create many
different instances, but without the class as a guide, you would be lost,
not knowing what information is required.
An object consists of:

• State: It is represented by the attributes of an object. It also reflects

the properties of an object.

• Behavior: It is represented by the methods of an object. It also

reflects the response of an object to other objects.

• Identity: It gives a unique name to an object and enables one object

to interact with other objects.

Fig. 20 Class Object

Declaring Class Objects (Also called instantiating a class):

20 BEGINNING PYTHON

When an object of a class is created, the class is said to be instan-

tiated. All the instances share the attributes and the behavior of the
class. But the values of those attributes, i.e. the state are unique for
each object. A single class may have any number of instances.

Fig. 21 Class Declaration

Self Parameter:
When we call a method of this object as myobject.method(arg1,
arg2), this is automatically converted by Python into MyClass.method(myobject,
arg1, arg2) – this is all the special self is about.

Fig. 22 Self Parameter example

FUNCTIONS AND CLASSES: A PREVIEW 21

The Self Parameter does not call it self.You can use any other name
instead of it. Here we change the self to the word someone and the
output will be the same.

Fig. 23 Self Parameter example 2

__init__() method:
The __init__ method is similar to constructors in C++ and Java.
Constructors are used for initialising the object’s state. Like methods,
a constructor also contains a collection of statements(i.e. instructions)
that are executed at the time of Object creation. It runs as soon as
an object of a class is instantiated. The method is useful to do any
initialization you want to do with your object.

Fig. 24 Init Example

22 BEGINNING PYTHON

__str__() method:
Python has a particular method called __str__(). that is used to
define how a class object should be represented as a string. It is often
used to give an object a human-readable textual representation, which
is helpful for logging, debugging, or showing users object information.
When a class object is used to create a string using the built-in functions
print() and str(), the __str__() function is automatically used. You
can alter how objects of a class are represented in strings by defining the
__str__() method.

Fig. 25 str method example

STATEMENTS 23

5
STATEMENTS
In Python, control statements are used to alter the flow of execution
based on specific conditions or looping requirements. The main types of
control statements are:

• Conditional statements: if, else, elif

• Looping statements: for, while

• Control flow statements: break, continue, pass, return

if: elif: else:

The if-else statement in Python is used to control the flow of the program
based on a condition. It has the following syntax:

Fig. 26 if condition 1

A single if condition can have at most one else statement. How-

ever, you can have multiple elif (else if) statements to check additional
conditions if needed:

for, while:
The For Loops in Python are a special type of loop statement that is
used for sequential traversal.
Python For loop is used for iterating over an iterable like a String,
Tuple, List, Set, or Dictionary.
24 BEGINNING PYTHON

Fig. 27 if condition 2

Fig. 28 If Else condition

Fig. 29 If Else condition

Python While Loop is used to execute a block of statements repeat-

edly until a given condition is satisfied. When the condition becomes
false, the line immediately after the loop in the program is executed.
STATEMENTS 25

Fig. 30 elif

Fig. 31 For loop

Fig. 32 While loop

26 BEGINNING PYTHON

6
FUNCTIONS, MODULES, PACKAGES, AND
DEBUGGING
Functions
Python Functions is a block of statements that return the specific task.
The idea is to put some commonly or repeatedly done tasks together and
make a function so that instead of writing the same code again and again
for different inputs, we can do the function calls to reuse code contained
in it over and over again.

Fig. 33 Function

Python Function with Parameters:

If you have experience in C/C++ or Java then you must be thinking
about the return type of the function and data type of arguments. That
is possible in Python as well (specifically for Python 3.5 and above).
Default Arguments: A default argument is a parameter that as-
sumes a default value if a value is not provided in the function call for
that argument.
Keyword Arguments: The idea is to allow the caller to specify the
argument name with values so that the caller does not need to remember
the order of parameters.
Function within Functions:
FUNCTIONS, MODULES, PACKAGES, AND DEBUGGING 27

Fig. 34 Python Function Syntax with Parameters

Fig. 35 Python Function Syntax with Parameters

Fig. 36 Default Arguments

A function that is defined inside another function is known as the

inner function or nested function. Nested functions can access variables
of the enclosing scope. Inner functions are used so that they can be
protected from everything happening outside the function.
28 BEGINNING PYTHON

Fig. 37 Keyword Arguments

Fig. 38 Function within Functions

Anonymous Functions: In Python, an anonymous function means

that a function is without a name. As we already know the def keyword
is used to define the normal functions and the lambda keyword is used
to create anonymous functions.

Fig. 39 anonymous functions

FUNCTIONS, MODULES, PACKAGES, AND DEBUGGING 29

Recursive Functions:
Recursion in Python refers to when a function calls itself. There
are many instances when you have to build a recursive function to solve
Mathematical and Recursive Problems.
Using a recursive function should be done with caution, as a recursive
function can become like a non-terminating loop. It is better to check
your exit statement while creating a recursive function.

Fig. 40 Recursive Functions

Pass by Reference and Pass by Value: One important thing to

note is, in Python every variable name is a reference. When we pass a
variable to a function Python, a new reference to the object is created.
Parameter passing in Python is the same as reference passing in Java.

Fig. 41 Pass function example 1

When we pass a reference and change the received reference to some-

thing else, the connection between the passed and received parameters
is broken.
30 BEGINNING PYTHON

Fig. 42 Pass function example 2

Modules

In Python, a module is a file containing Python code that can define

functions, classes, and variables, and can also include runnable code.
These modules are used to organize Python code logically, making the
code easier to understand and use. Modules can be imported into other
modules or into the main part of the program, enabling code reuse and
namespace management.
Create a Python Module:
To create a Python module, write the desired code and save that in
a file with .py extension.

Fig. 43 Modules example

Import module in Python:

FUNCTIONS, MODULES, PACKAGES, AND DEBUGGING 31

We can import the functions, and classes defined in a module to

another module using the import statement in some other Python
source file.
When the interpreter encounters an import statement, it imports the
module if the module is present in the search path.

Fig. 44 Import module

Python Import From Module:

Python’s from statement lets you import specific attributes from a
module without importing the module as a whole.

Fig. 45 Import from modules

Renaming the Python Module:

We can rename the module while importing it using the keyword as.

Fig. 46 Rename modules

32 BEGINNING PYTHON

Packages

Python Packages are a way to organize and structure your Python code
into reusable components. Think of it like a folder that contains related
Python files (modules) that work together to provide certain functional-
ity. Packages help keep your code organized, make it easier to manage
and maintain, and allow you to share your code with others. They’re
like a toolbox where you can store and organize your tools (functions
and classes) for easy access and reuse in different projects.
How to Create Package in Python?
Creating packages in Python allows you to organize your code into
reusable and manageable modules. Here’s a brief overview of how to
create packages:

• Create a Directory: Start by creating a directory (folder) for

your package. This directory will serve as the root of your package
structure.

• Add Modules: Within the package directory, you can add Python
files (modules) containing your code. Each module should represent
a distinct functionality or component of your package.

• Init File: Include an __init__.py file in the package directory.

This file can be empty or can contain an initialization code for your
package. It signals to Python that the directory should be treated
as a package.

• Subpackages: You can create sub-packages within your package by

adding additional directories containing modules, along with their
own __init__.py files.

• Importing: To use modules from your package, import them into

your Python scripts using dot notation. For example, if you have a
module named module1.py inside a package named mypackage, you
would import its function like this: from mypackage.module1 import
greet.
REGULAR EXPRESSIONS 33

• Distribution: If you want to distribute your package for others to

use, you can create a setup.py file using Python’s setuptools library.
This file defines metadata about your package and specifies how it
should be installed.

Fig. 47 Create packages

Here’s a basic code sample demonstrating how to create a simple

Python package:
1. Create a directory named mypackage.
2. Inside mypackage, create two Python files: module1.py and mod-
ule2.py.
3. Create an __init__.py file inside mypackage (it can be empty).
4. Add some code to the modules.
5. Finally, demonstrate how to import and use the modules from the
package.

PART 2
Advanced Python
7
REGULAR EXPRESSIONS
A Regular Expression or RegEx is a special sequence of characters that
uses a search pattern to find a string or set of strings.
34 Advanced Python

It can detect the presence or absence of a text by matching it with

a particular pattern and also can split a pattern into one or more sub-
patterns.

Regex Module in Python

Regular Expression (RegEx) in Python with Examples Last Updated :
05 Sep, 2024 A Regular Expression or RegEx is a special sequence of
characters that uses a search pattern to find a string or set of strings.
It can detect the presence or absence of a text by matching it with
a particular pattern and also can split a pattern into one or more sub-
patterns.
Regex Module in Python Python has a built-in module named “re”
that is used for regular expressions in Python. We can import this mod-
ule by using the import statement.

Fig. 48 Python RegEx

Fig. 49 RegEx Result

Note: Here r character (r’Python’) stands for raw, not regex. The
raw string is slightly different from a regular string, it won’t interpret the
\ character as an escape character. This is because the regular expression
engine uses \ character for its own escaping purpose.
REGULAR EXPRESSIONS 35

Before starting with the Python regex module let’s see how to actu-
ally write regex using metacharacters or special sequences.

Metacharacters

Metacharacters are the characters with special meaning.

To understand the RE analogy, Metacharacters are useful and im-
portant. They will be used in functions of module re. Below is the list
of metacharacters.

Table 2 Metacharacters list

MetaCharacters Description
Used to drop the special meaning of character
\
following it
[] Represent a character class
^ Matches the beginning
$ Matches the end
. Matches any character except newline
Means OR (Matches with any of the characters
|
separated by it).
? Matches zero or one occurrence
Any number of occurrences (including 0 occur-
*
rences)
+ One or more occurrences
Indicate the number of occurrences of a preced-
{}
ing regex to match.
() Enclose a group of Regex

1). \ – Backslash: The backslash ( \ ) makes sure that the character

is not treated in a special way. This can be considered a way of
escaping metacharacters.
36 Advanced Python

For example, if you want to search for the dot(.) in the string then
you will find that dot(.) will be treated as a special character as
is one of the metacharacters (as shown in the above table). So for
this case, we will use the backslash( \ ) just before the dot(.) so
that it will lose its specialty. See the below example for a better
understanding.

Fig. 50 Backslash example

2). [] – Square Brackets: Square Brackets ([]) represent a character class

consisting of a set of characters that we wish to match. For example,
the character class [abc] will match any single a, b, or c.
We can also specify a range of characters using – inside the square
brackets. For example,

• [0, 3] is sample as [0123]

• [a − c] is same as [abc]

We can also invert the character class using the caret(ˆ) symbol. For
example,

• [ˆ0 − 3] means any character except 0, 1, 2, or 3

• [ˆa − c] means any character except a, b, or c
REGULAR EXPRESSIONS 37

Example: In this code, you’re using regular expressions to find all

the characters in the string that fall within the range of ‘a’ to ‘m’.
The re.findall() function returns a list of all such characters. In the
given string, the characters that match this pattern are: ‘c’, ‘k’, ‘b’,
‘f’, ‘j’, ‘e’, ‘h’, ‘l’, ‘d’, ‘g’.

Fig. 51 Square Brackets

Fig. 52 Square Brackets Result

3). ˆ – Caret: Caret (ˆ) symbol matches the beginning of the string i.e.
checks whether the string starts with the given character(s) or not.
For example –

• ˆg will check if the string starts with g such as geeks, globe, girl,
g, etc.
• ˆge will check if the string starts with ge such as gee etc. Exam-
ple: This code uses regular expressions to check if a list of strings
starts with “The”. If a string begins with “The,” it’s marked as
“Matched” otherwise, it’s labeled as “Not matched”.

4). $ – Dollar: Dollar($) symbol matches the end of the string i.e checks
whether the string ends with the given character(s) or not. For
example-
38 Advanced Python

Fig. 53 Caret example

• s$ will check for the string that ends with a such as Tests, ends,
s, etc.
• ts$ will check for the string that ends with ks such as Tests ks,
etc. Example: This code uses a regular expression to check if the
string ends with “World!”. If a match is found, it prints “Match
found!” otherwise, it prints “Match not found”.

Fig. 54 Dollar Example

5). . – Dot: Dot(.) symbol matches only a single character except for
the newline character (\n). For example –
REGULAR EXPRESSIONS 39

• a.b will check for the string that contains any character at the
place of the dot such as acb, acbd, abbb, etc
• .. will check if the string contains at least 2 characters
Example: This code uses a regular expression to search for the pat-
tern “brown.fox” within the string. The dot (.) in the pattern rep-
resents any character. If a match is found, it prints “Match found!”
otherwise, it prints “Match not found”.

Fig. 55 Dot example

6). | - OR: Or symbol works as the or operator meaning it checks whether

the pattern before or after the or symbol is present in the string or
not. For example –
a|b will match any string that contains a or b such as acd, bcd, abcd,
etc.

Fig. 56 Or Example
40 Advanced Python

7). ? – Question Mark: The question mark (?) is a quantifier in regu-

lar expressions that indicates that the preceding element should be
matched zero or one time. It allows you to specify that the element
is optional, meaning it may occur once or not at all. For example,
ab?c will be matched for the string ac, acb, dabc but will not be
matched for abbc because there are two b. Similarly, it will not be
matched for abdc because b is not followed by c.

Fig. 57 Question mark Example

8). * – Star: Star (*) symbol matches zero or more occurrences of the
regex preceding the * symbol. For example –
ab*c will be matched for the string ac, abc, abbbc, dabc, etc. but
will not be matched for abdc because b is not followed by c.

Fig. 58 Star Example

REGULAR EXPRESSIONS 41

9). + – Plus: Plus (+) symbol matches one or more occurrences of the
regex preceding the + symbol. For example –
ab+c will be matched for the string abc, abbc, dabc, but will not
be matched for ac, abdc, because there is no b in ac and b, is not
followed by c in abdc.

Fig. 59 Plus Example

10). {m, n} – Braces: Braces match any repetitions preceding regex from
m to n both inclusive. For example –
a{2, 4} will be matched for the string aaab, baaaac, gaad, but will
not be matched for strings like abc, bc because there is only one a
or no a in both the cases.

Fig. 60 Braces Example

11). (<regex>) – Group: Group symbol is used to group sub-patterns.

For example –
(a|b)cd will match for strings like acd, abcd, gacd, etc.
42 Advanced Python

Fig. 61 RegEx Group

Special Sequences
Special sequences do not match for the actual character in the string
instead it tells the specific location in the search string where the match
must occur. It makes it easier to write commonly used patterns.

RegEx Functions
re module contains many functions that help us to search a string for a
match.
Let’s see various functions provided by this module to work with
regex in Python.

1. re.findall(): Return all non-overlapping matches of pattern in string,

as a list of strings. The string is scanned left-to-right, and matches
are returned in the order found.

Fig. 62 Finadall Example

REGULAR EXPRESSIONS 43

Table 3 Special Sequences

Characters Description Example
Returns a match if the specified characters are
\A "\AThe"
at the beginning of the string
Returns a match where the specified characters
are at the beginning or at the end of a word
\b r"\bain" r"ain\b"
(the "r" in the beginning is making sure that the
string is being treated as a "raw string")
Returns a match where the specified characters
are present, but NOT at the beginning (or at
\B the end) of a word (the "r" in the beginning is r"\Bain" r"ain\B"
making sure that the string is being treated as
a "raw string")
Returns a match where the string contains digits
\d "\d"
(numbers from 0-9)
Returns a match where the string DOES NOT
\D "\D"
contain digits
Returns a match where the string contains a
\s "\s"
white space character
Returns a match where the string DOES NOT
\S "\S"
contain a white space character
Returns a match where the string contains any
\w word characters (characters from a to Z, digits "\w"
from 0-9, and the underscore _ character)
Returns a match where the string DOES NOT
\W "\W"
contain any word characters
Returns a match if the specified characters are
\Z "Hello\Z"
at the end of the string

2. re.compile(): Regular expressions are compiled into pattern ob-

jects, which have methods for various operations such as searching
for pattern matches or performing string substitutions.

3. re.split():Split string by the occurrences of a character or a pattern;

upon finding that pattern, the remaining characters from the string
are returned as part of the resulting list.

Syntax : re.split(pattern, string, maxsplit=0, flags=0)

44 Advanced Python

Table 4 RegEx Functions

Function Description
finds and returns all matching occurrences in a
re.findall()
list
Regular expressions are compiled into pattern
re.compile()
objects
Split string by the occurrences of a character or
re.split()
a pattern.
Replaces all occurrences of a character or patter
re.sub()
with a replacement string.
re.escape() Escapes special character
Searches for first occurrence of character or pat-
re.search()
tern

Fig. 63 Compile Example 1

Fig. 64 Compile Example 2

REGULAR EXPRESSIONS 45

Fig. 65 Compile Example 3

The first parameter, pattern, denotes the regular expression; string

is the given string in which pattern will be searched for and in which
splitting occurs; maxsplit, if not provided, is considered to be zero
‘0’, and if any nonzero value is provided, then at most that many
splits occur. If maxsplit = 1, then the string will split once only,
resulting in a list of length 2. The flags are very useful and can
help to shorten code; they are not necessary parameters, eg: flags =
re.IGNORECASE, In this split, the case, i.e. the lowercase or the
uppercase, will be ignored.

Fig. 66 Split Example

46 Advanced Python

Fig. 67 Split Example 2

4. re.sub(): The’sub’ in the function stands for SubString; a certain

regular expression pattern is searched in the given string (3rd pa-
rameter), and upon finding the substring pattern is replaced by repl
(2nd parameter), which checks and maintains the number of times
this occurs.

Syntax : re.sub(pattern, repl, string, count=0, flags=0)

Fig. 68 Sub Example

5. re.escape():Returns a string with all non-alphanumerics backslashed;

this is useful if you want to match an arbitrary literal string that may
have regular expression metacharacters in it.
REGULAR EXPRESSIONS 47

Fig. 69 Escape Example

6. re.search(): This method either returns None (if the pattern doesn’t
match) or a re.MatchObject contains information about the match-
ing part of the string. This method stops after the first match, so this
is best suited for testing a regular expression more than extracting
data.

Fig. 70 Search Example

48 Advanced Python

Match Object
A Match object contains all the information about the search and the
result, and if there is no match found, then None will be returned. Let’s
see some of the commonly used methods and attributes of the match
object. The Match object has properties and methods used to retrieve
information about the search and the result.

• string returns the string passed into the function.

• span() returns a tuple containing the start- and end positions of the
match.

• group() returns the part of the string where there was a match

1. Getting the string and the regex :

match.re attribute returns the regular expression passed, and match.string
attribute returns the string passed.

Fig. 71 Getting the string and the regex

2. Getting index of matched object :

• start() method returns the starting index of the matched sub-

string.
• end() method returns the ending index of the matched substring.
REGULAR EXPRESSIONS 49

Fig. 72 Getting index of matched object

Fig. 73 Getting matched substring

• span() method returns a tuple containing the starting and end-

ing index of the matched substring.

3. Getting matched substring :

group() method returns the part of the string for which the patterns
match. See the below example for a better understanding.
50 Advanced Python

8
ITERATOR OBJECTS
An iterator in Python is an object that is used to iterate over iterable
objects like lists, tuples, dicts, and sets. The Python iterators object
is initialized using the iter() method. It uses the next() method for
iteration.

• __iter__(): The iter() method is called for the initialization of

an iterator. This returns an iterator object
• __next__(): The next method returns the next value for the
iterable. When we use a for loop to traverse any iterable object,
internally it uses the iter() method to get an iterator object, which
further uses the next() method to iterate over.

Fig. 74 Create an Iterator

UNIT TESTS 51

Fig. 75 Iterator Example

9
UNIT TESTS
Unit Testing is the first level of software testing where the smallest
testable parts of software are tested. This is used to validate that each
software unit performs as designed.
Python Unittest is a built-in testing framework that provides a set
of tools for testing our code’s functionality in a more systematic and
organized manner. With unittest framework, we can create test cases,
fixtures, and suites to verify if our code behaves as expected. It allows
52 Advanced Python

us to write test methods within classes that check different aspects of

our code such as input, output, and edge cases. It also supports test
discovery making it easy for us to automate test execution across our
project.

Why Choose Unittest in Python?

Developers often choose Python’s unittest framework for its built-in na-
ture, widespread familiarity, comprehensive features, seamless integra-
tion with other tools, and proven reliability. As part of Python’s stan-
dard library, unittest requires no additional installations, making it eas-
ily accessible. Unit tests promote consistency in testing practices, aiding
collaboration and maintenance. While other frameworks like Pytest
and nose offer alternatives, unittest remains a popular choice for its
stability and versatility.

Assert Methods in Python Unittest Framework

unittest has lots of methods to assert on the values, types, and existence
of variables. Below are some of the methods that are commonly used to
write assertions.

OOP Concepts Supported by Unittest Framework

The White Box Testing method is used for Unit tests. Below are some
of supported oops concept by Unitttest framework:

• test fixture: A test fixture is used as a baseline for running tests

to ensure that there is a fixed environment in which tests are run so
that results are repeatable. Examples :creating temporary databases
or starting a server process.

• test case: A test case is a set of conditions which is used to deter-

mine whether a system under test works correctly.

• test suite: Test suite is a collection of testcases that are used to

test a software program to show that it has some specified set of
behaviours by executing the aggregated tests together.
UNIT TESTS 53

Table 5
Method Description
Checks if a is equal to b, similar to the expres-
.assertEqual(a, b)
sion a == b.
Asserts that the boolean value of x is True,
.assertTrue(x)
equivalent to bool(x) is True.
Asserts that a is an instance of class b, similar
.assertIsInstance(a, b)
to the expression isinstance(a, b).
Ensures that x is None, similar to the expression
.assertIsNone(x)
x is None.
Asserts that the boolean value of x is False, sim-
.assertFalse(x)
ilar to bool(x) is False.
Verifies if a is identical to b, akin to the expres-
.assertIs(a, b)
sion a is b.
Checks if a is a member of b, akin to the expres-
.assertIn(a, b)
sion a in b.

• test runner: A test runner is a component which set up the execu-

tion of tests and provides the outcome to the user.

Python Unittest Example

Example 1: Test.py
Here, we will write a a simple function named add() that takes two
numbers and returns their sum. Additionally, it includes a test case class
TestAddFunction that inherits from unittest.TestCase. Within this
class, three test methods are defined to verify different scenarios of the
add() function.
This is the basic test code using unittest framework, which is having
a single test. This test() method will fail if TRUE is ever FALSE.
For instance, if we change the expected value from 3 to, let’s say,
2 in the test_add_positive_numbers, the test would fail and we’d
have this output.
54 Advanced Python

Fig. 76 UnitTest Example 1

Outcomes Possible in Unit Testing

There are three types of possible test outcomes :

• OK – This means that all the tests are passed.

• FAIL – This means that the test did not pass and an Assertion-
Error exception is raised.

• ERROR – This means that the test raises an exception other than
AssertionError.

Description of Tests: from UnitTest Example 3 we can describe

the following:
unittest.TestCase is used to create test cases by subclassing it.
The last block of the code at the bottom allows us to run all the tests
just by running the file. Basic terms used in the code :
UNIT TESTS 55

Fig. 77 UnitTest Example2

• assertEqual() – This statement is used to check if the result ob-

tained is equal to the expected result.
• assertTrue() / assertFalse() – This statement is used to verify
if a given statement is true or false.
• assertRaises() – This statement is used to raise a specific excep-
tion.

test_strings_a(): This test is used to test the property of string

in which a character say ‘a’ multiplied by a number say ‘x’ gives the
output as x times ‘a’. The assertEqual() statement returns true in this
case if the result matches the given output.
56 Advanced Python

Fig. 78 UnitTest Example 3

test_upper(): This test is used to check if the given string is con-

verted to uppercase or not. The assertEqual() statement returns true
if the string returned is in uppercase.
test_isupper(): This test is used to test the property of string
which returns TRUE if the string is in uppercase else returns False. The
assertTrue() / assertFalse() statement is used for this verification.
test_strip(): This test is used to check if all chars passed in the
function have been stripped from the string. The assertEqual() state-
ment returns true if the string is stripped and matches the given output.
test_split(): This test is used to check the split function of the
string which splits the string through the argument passed in the function
and returns the result as list. The assertEqual() statement returns true
in this case if the result matches the given output.
EXTENDING AND EMBEDDING PYTHON 57

10
EXTENDING AND EMBEDDING PYTHON
Python’s built-in objects—such as numbers, sequences, dictionaries, sets,
and files—are coded in C, as are several modules in Python’s standard
library. Modern platforms support dynamic-load libraries, with file ex-
tensions such as .dll on Windows, .so on Linux, and .dylib on Mac:
building Python produces such binary files. You can code your own
extension modules for Python in C, using the Python C API covered
in this chapter, to produce and deploy dynamic libraries that Python
scripts and interactive sessions can later use with the import statement,
covered in “The import Statement”.
Extending Python means building modules that Python code can
import to access the features the modules supply. Embedding Python
means executing Python code from an application coded in another lan-
guage. For such execution to be useful, Python code must in turn be able
to access some of your application’s functionality. In practice, therefore,
embedding implies some extending, as well as a few embedding-specific
operations. The three main reasons for wishing to extend Python can
be summarized as follows:

• Re-implementing some functionality (originally coded in Python) in

a lower-level language, hoping to get better performance

• Letting Python code access some existing functionality supplied by

libraries coded in (or, at any rate, callable from) lower-level lan-
guages

• Letting Python code access System-Level Features

Extending Python with C or C++

It is quite easy to add new built-in modules to Python, if you know how
to program in C. Such extension modules can do two things that can’t be
done directly in Python: they can implement new built-in object types,
and they can call C library functions and system calls.
58 Advanced Python

To support extensions, the Python API (Application Programmers

Interface) defines a set of functions, macros and variables that provide
access to most aspects of the Python run-time system. The Python API
is incorporated in a C source file by including the header "Python.h".
Note: The C extension interface is specific to CPython, and exten-
sion modules do not work on other Python implementations. In many
cases, it is possible to avoid writing C extensions and preserve portabil-
ity to other implementations. For example, if your use case is calling
C library functions or system calls, you should consider using the ctypes
module or the cffi library rather than writing custom C code. These mod-
ules let you write Python code to interface with C code and are more
portable between implementations of Python than writing and compiling
a C extension module.
A Simple Example:
Let’s create an extension module called spam and let’s say we want
to create a Python interface to the C library function system(). This
function takes a null-terminated character string as argument and returns
an integer. We want this function to be callable from Python as follows:

import spam
s t a t u s = spam . system ( " l s ␣− l " )

Begin by creating a file spammodule.c. (Historically, if a module

is called spam, the C file containing its implementation is called spam-
module.c; if the module name is very long, like spammify, the module
name can be just spammify.c.)
The first line of our file can be:

#i n c l u d e <Python . h>

which pulls in the Python API (you can add a comment describing the
purpose of the module and a copyright notice if you like).
EXTENDING AND EMBEDDING PYTHON 59

All user-visible symbols defined by Python.h have a prefix of Py or

PY, except those defined in standard header files. For convenience, and
since they are used extensively by the Python interpreter, "Python.h"
includes a few standard header files: <stdio.h>,<string.h>, <er-
rno.h> and <stdlib.h>. If the latter header file does not exist on
your system, it declares the functions malloc(), free() and realloc()
directly.
The next thing we add to our module file is the C function that will be
called when the Python expression spam.system(string) is evaluated
(we’ll see shortly how it ends up being called):

s t a t i c PyObject ∗
spam_system ( PyObject ∗ s e l f , PyObject ∗ a r g s )
{
const char ∗command ;
int s t s ;
i f ( ! PyArg_ParseTuple ( a rg s , " s " , &command ) )
return NULL;
s t s = system (command ) ;
return PyLong_FromLong ( s t s ) ;
}

There is a straightforward translation from the argument list in Python

(for example, the single expression "ls -l") to the arguments passed to the
C function. The C function always has two arguments, conventionally
named self and args.
The self argument points to the module object for module-level func-
tions; for a method it would point to the object instance.
The args argument will be a pointer to a Python tuple object contain-
ing the arguments. Each item of the tuple corresponds to an argument
in the call’s argument list. The arguments are Python objects — in order
to do anything with them in our C function we have to convert them to
C values. The function PyArg_ParseTuple() in the Python API checks
the argument types and converts them to C values. It uses a template
string to determine the required types of the arguments as well as the
types of the C variables into which to store the converted values.
60 Advanced Python

PyArg_ParseTuple() returns true (nonzero) if all arguments have the

right type and its components have been stored in the variables whose
addresses are passed. It returns false (zero) if an invalid argument list
was passed. In the latter case it also raises an appropriate exception so
the calling function can return NULL immediately.
It returns NULL (the error indicator for functions returning object
pointers) if an error is detected in the argument list, relying on the
exception set by PyArg_ParseTuple(). Otherwise the string value of
the argument has been copied to the local variable command. This is
a pointer assignment and you are not supposed to modify the string to
which it points (so in Standard C, the variable command should properly
be declared as const char *command).
The next statement is a call to the Unix function system(), passing
it the string we just got from PyArg_ParseTuple():

s t s = system (command ) ;

Our spam.system() function must return the value of sts as a Python

object. This is done using the function PyLong_FromLong().

return PyLong_FromLong ( s t s ) ;

In this case, it will return an integer object. (Yes, even integers are
objects on the heap in Python!)
If you have a C function that returns no useful argument (a func-
tion returning void), the corresponding Python function must return
None. You need this idiom to do so (which is implemented by the
Py_RETURN_NONE macro):

Py_INCREF( Py_None ) ;
return Py_None ;

Py_None is the C name for the special Python object None. It

is a genuine Python object rather than a NULL pointer, which means
“error” in most contexts, as we have seen.
EXTENDING AND EMBEDDING PYTHON 61

Embedding Python in Another Application

The previous section discussed how to extend Python, that is, how to
extend the functionality of Python by attaching a library of C functions
to it. It is also possible to do it the other way around: enrich your
C/C++ application by embedding Python in it. Embedding provides
your application with the ability to implement some of the functionality
of your application in Python rather than C or C++. This can be used
for many purposes; one example would be to allow users to tailor the
application to their needs by writing some scripts in Python. You can
also use it yourself if some of the functionality can be written in Python
more easily.
Embedding Python is similar to extending it, but not quite. The
difference is that when you extend Python, the main program of the
application is still the Python interpreter, while if you embed Python,
the main program may have nothing to do with Python — instead, some
parts of the application occasionally call the Python interpreter to run
some Python code.
So if you are embedding Python, you are providing your own main
program. One of the things this main program has to do is initialize
the Python interpreter. At the very least, you have to call the function
Py_Initialize(). There are optional calls to pass command line argu-
ments to Python. Then later you can call the interpreter from any part
of the application.
There are several different ways to call the interpreter: you can pass
a string containing Python statements to PyRun_SimpleString(), or
you can pass a stdio file pointer and a file name (for identification in
error messages only) to PyRun_SimpleFile(). You can also call the
lower-level operations described in the previous chapters to construct
and use Python objects.
Very High Level Embedding:
The simplest form of embedding Python is the use of the very high
level interface. This interface is intended to execute a Python script
without needing to interact with the application directly. This can for
example be used to perform some operation on a file.
#include <Python . h>
62 Advanced Python

int
main ( int argc , char ∗ argv [ ] )
{
wchar_t ∗ program = Py_DecodeLocale ( argv [ 0 ] , NULL ) ;
i f ( program == NULL) {
f p r i n t f ( s t d e r r , " F a t a l ␣ e r r o r : ␣ cannot ␣ decode ␣ argv [ 0 ] \ n "
exit (1);
}
Py_SetProgramName ( program ) ; /∗ o p t i o n a l b u t recommended ∗/
Py_Initialize ();
PyRun_SimpleString ( " from ␣ time ␣ import ␣ time , ctime \n "
" p r i n t ( ’ Today␣ i s ’ , ␣ ctime ( time ( ) ) ) \ n " ) ;
i f ( Py_FinalizeEx ( ) < 0) {
exit (120);
}
PyMem_RawFree( program ) ;
return 0 ;
}

The Py_SetProgramName() function should be called before Py_Initialize()

to inform the interpreter about paths to Python run-time libraries. Next,
the Python interpreter is initialized with Py_Initialize(), followed by
the execution of a hard-coded Python script that prints the date and
time. Afterwards, the Py_FinalizeEx() call shuts the interpreter down,
followed by the end of the program. In a real program, you may want to
get the Python script from another source, perhaps a text-editor routine,
a file, or a database. Getting the Python code from a file can better be
done by using the PyRun_SimpleFile() function, which saves you the
trouble of allocating memory space and loading the file contents.
Pure Embedding:
The first program aims to execute a function in a Python script. Like
in the section about the very high level interface, the Python interpreter
does not directly interact with the application.
The code to run a function defined in a Python script is:
#include <Python . h>
int main ( int argc , char ∗ argv [ ] )
EXTENDING AND EMBEDDING PYTHON 63

{
PyObject ∗pName , ∗pModule , ∗ pDict , ∗pFunc ;
PyObject ∗ pArgs , ∗ pValue ;
int i ;
i f ( a r g c < 3) {
f p r i n t f ( s t d e r r , " Usage : ␣ c a l l ␣ p y t h o n f i l e ␣ funcname ␣ [ a r g s ] \ n " )
return 1 ;
}
Py_Initialize ();
pName = PyUnicode_DecodeFSDefault ( argv [ 1 ] ) ;
/∗ Error c h e c k i n g o f pName l e f t o u t ∗/
pModule = PyImport_Import (pName ) ;
Py_DECREF(pName ) ;
i f ( pModule != NULL) {
pFunc = PyObject_GetAttrString ( pModule , argv [ 2 ] ) ;
/∗ pFunc i s a new r e f e r e n c e ∗/
i f ( pFunc && PyCallable_Check ( pFunc ) ) {
pArgs = PyTuple_New ( a r g c − 3 ) ;
for ( i = 0 ; i < a r g c − 3 ; ++i ) {
pValue = PyLong_FromLong ( a t o i ( argv [ i + 3 ] ) ) ;
i f ( ! pValue ) {
Py_DECREF( pArgs ) ;
Py_DECREF( pModule ) ;
f p r i n t f ( s t d e r r , " Cannot ␣ c o n v e r t ␣ argument \n " ) ;
return 1 ;
}
/∗ pValue r e f e r e n c e s t o l e n h e r e : ∗/
PyTuple_SetItem ( pArgs , i , pValue ) ;
}
pValue = PyObject_CallObject ( pFunc , pArgs ) ;
Py_DECREF( pArgs ) ;
i f ( pValue != NULL) {
p r i n t f ( " R e s u l t ␣ o f ␣ c a l l : ␣%l d \n " , PyLong_AsLong ( pVal
Py_DECREF( pValue ) ;
}
else {
64 Advanced Python

Py_DECREF( pFunc ) ;
Py_DECREF( pModule ) ;
PyErr_Print ( ) ;
f p r i n t f ( s t d e r r , " C a l l ␣ f a i l e d \n " ) ;
return 1 ;
}
}
else {
i f ( PyErr_Occurred ( ) )
PyErr_Print ( ) ;
f p r i n t f ( s t d e r r , " Cannot ␣ f i n d ␣ f u n c t i o n ␣\"% s \ " \ n " , a
}
Py_XDECREF( pFunc ) ;
Py_DECREF( pModule ) ;
}
else {
PyErr_Print ( ) ;
f p r i n t f ( s t d e r r , " F a i l e d ␣ t o ␣ l o a d ␣\"% s \ " \ n " , argv [ 1 ] ) ;
return 1 ;
}
i f ( Py_FinalizeEx ( ) < 0) {
return 1 2 0 ;
}
return 0 ;
}

This code loads a Python script using argv[1], and calls the function
named in argv[2]. Its integer arguments are the other values of the
argv array. If you compile and link this program (let’s call the finished
executable call), and use it to execute a Python script, such as:
def m u l t i p l y ( a , b ) :
print ( " W i l l ␣ compute " , a , " t i m e s " , b )
c = 0
for i in range ( 0 , a ) :
c = c + b
return c
EXTENDING AND EMBEDDING PYTHON 65

then the result should be:

$ c a l l multiply multiply 3 2
W i l l compute 3 t i m e s 2
Result of c a l l : 6
66 Advanced Python

11
PARSING
Introduction to XML
What is XML?
XML (eXtensible Markup Language) is a markup language designed
to store and transport data. It is:

• Human-readable: The structure is text-based and easy to under-

stand.

• Machine-readable: Designed for software to parse and process

efficiently.

Key Features of XML

1. Hierarchical Structure: Data is represented as a tree with nested
elements.

Fig. 79 XML File

2. Customizable Tags: Tags are user-defined, making XML flexible

for various use cases.

3. Attributes: Tags can have attributes to add more information:

4. Self-contained: An XML document is standalone and doesn’t rely

on external references.
PARSING 67

Fig. 80 XML File

Common Use Cases of XML

• Configuration files: Defining application settings (e.g., config.xml).
• Data exchange: Between systems (e.g., SOAP APIs, RSS feeds).
• Document structure: Representing structured documents (e.g.,
sitemaps, invoices).

Overview of xml.etree.ElementTree
ElementTree is a built-in Python library for XML parsing. It provides
methods to:
• Parse XML files or strings.
• Navigate and extract elements and attributes.
• Modify and write XML data.

Reading and Parsing XML

Use ElementTree to load and parse XML data.

Navigating and Extracting Data

You can navigate through elements and extract data using loops and tag
names.

Modifying XML
Add, edit, or remove elements.
68 Advanced Python

Fig. 81 Book XML File

Fig. 82 Python code XML example

Key Functions in ElementTree

Advanced Features in XML Parsing

1. Advanced Navigation with XPath: XPath (XML Path Lan-

guage) is a syntax for navigating XML documents. It allows you to
locate specific nodes or attributes based on conditions.
PARSING 69

Fig. 83 Extracting data from XML example

Fig. 84 XML File manipulating

2. Using XPath with xml.etree.ElementTree: While xml.etree.ElementTree

has limited XPath support, it includes basic functionality for com-
mon use cases.

3. Error Handling in XML Parsing: XML parsing might fail if

the document is malformed. Use try...except to handle such errors
gracefully.
70 Advanced Python

Table 6 Key Functions in ElementTree

Function Purpose
ET.parse(file) Parses an XML file and returns a tree.
tree.getroot() Returns the root element.
element.find(tag) Finds the first sub-element with the tag.
element.findall(tag) Finds all sub-elements with the tag.
ET.Element(tag) Creates a new XML element.

Fig. 85 Xpath example

Fig. 86 Error Handling Example

4. Validating XML: Validating XML against an XSD (XML Schema

Definition) ensures the document adheres to a predefined structure.
For this, you can use external libraries like lxml.
5. Converting XML to Other Formats: You can transform XML
into formats like JSON or CSV for easier consumption.
PARSING 71

Fig. 87 XML validation

Fig. 88 XML converting to JSON

72 Advanced Python

12
PYTHON GUI APPLICATIONS
Tkinter is a Python Package for creating GUI applications. Python has
a lot of GUI frameworks, but Tkinter is the only framework that’s built
into the Python standard library.
Tkinter has several strengths; it’s cross-platform, so the same code
works on Windows, macOS, and Linux.
Tkinter is lightweight and relatively painless to use compared to other
frameworks. This makes it a compelling choice for building GUI appli-
cations in Python, especially for applications where a modern shine is
unnecessary, and the top priority is to build something functional and
cross-platform quickly.

Use Cases of Tkinter

1. Creating windows and dialog boxes: Tkinter can be used to

create windows and dialog boxes that allow users to interact with your
program. These can be used to display information, gather input, or
present options to the user.
To create a window or dialog box, you can use the Tk() function to
create a root window, and then use functions like Label, Button, and
Entry to add widgets to the window.
2. Building a GUI for a desktop application: Tkinter can be
used to create the interface for a desktop application, including buttons,
menus, and other interactive elements.
To build a GUI for a desktop application, you can use functions like
Menu, Checkbutton, and RadioButton to create menus and interactive
elements and use layout managers like pack and grid to arrange the
widgets on the window.
3. Adding a GUI to a command-line program: Tkinter can
be used to add a GUI to a command-line program, making it easier for
users to interact with the program and input arguments.
To add a GUI to a command-line program, you can use functions
like Entry and Button to create input fields and buttons, and use event
handlers like command and bind to handle user input.
PYTHON GUI APPLICATIONS 73

4. Creating custom widgets: Tkinter includes a variety of built-

in widgets, such as buttons, labels, and text boxes, but it also allows you
to create your own custom widgets.
To create a custom widget, you can define a class that inherits from
the Widget class and overrides its methods to define the behavior and
appearance of the widget.
5. Prototyping a GUI: Tkinter can be used to quickly prototype
a GUI, allowing you to test and iterate on different design ideas before
committing to a final implementation.
To prototype a GUI with Tkinter, you can use the Tk() function
to create a root window, and then use functions like Label, Button, and
Entry to add widgets to the window and test different layouts and design
ideas.

Getting Started with Tkinter

• Import tkinter package and all of its modules (from tkinter im-
port *).

• Create a root window using Tk(). Give the root window a title(using
title()) and dimension(using geometry()). All other widgets will
be inside the root window.

• Use mainloop() to call the endless loop of the window. If you forget
to call this nothing will appear to the user. The window will wait
for any user interaction till we close it.

• We’ll add a label using the Label Class and change its text configu-
ration as desired. The grid() function is a geometry manager which
keeps the label in the desired location inside the window. If no pa-
rameters are mentioned by default it will place it in the empty cell;
that is 0,0 as that is the first location.

• Now add a button to the root window. Changing the button config-
urations gives us a lot of options. In this example we will make the
button display a text once it is clicked and also change the color of
the text inside the button.
74 Advanced Python

Fig. 89 Tkinter first app

Fig. 90 Tkinter first app window

• Using the Entry() class we will create a text box for user input.
To display the user input text, we’ll make changes to the function
clicked(). We can get the user entered text using the get() func-
tion. When the Button after entering of the text, a default text
concatenated with the user text. Also change button grid location
to column 2 as Entry() will be column 1.
PYTHON GUI APPLICATIONS 75

Fig. 91 Tkinter button

Fig. 92 Tkinter button window

Tkinter Widgets
Tkinter is the GUI library of Python, it provides various controls, such as
buttons, labels and text boxes used in a GUI application. These controls
are commonly called Widgets.
• Label: The Label widget is used to provide a single-line caption for
other widgets. It can also contain images.
76 Advanced Python

Fig. 93 Tkinter button window 2

Fig. 94 Tkinter Function

• Button: The Button widget is used to display buttons in your

application. Entry The Entry widget is used to display a single-line
text field for accepting values from a user.
PYTHON GUI APPLICATIONS 77

Fig. 95 Tkinter Function window

• Menu: The Menu widget is used to provide various commands to a

user. These commands are contained inside Menubutton.

• Canvas: The Canvas widget is used to draw shapes, such as lines,

ovals, polygons and rectangles, in your application.

• Checkbutton: The Checkbutton widget is used to display a number

of options as checkboxes. The user can select multiple options at a
time.

• Frame: The Frame widget is used as a container widget to organize

other widgets.

• Listbox: The Listbox widget is used to provide a list of options to

a user.

• Menubutton: The Menubutton widget is used to display menus in

your application.

• Message: The Message widget is used to display multiline text

fields for accepting values from a user.

• Radiobutton: The Radiobutton widget is used to display a number

of options as radio buttons. The user can select only one option at
a time.

• Scale: The Scale widget is used to provide a slider widget.

78 Advanced Python

• Scrollbar: The Scrollbar widget is used to add scrolling capability

to various widgets, such as list boxes.
• Text: The Text widget is used to display text in multiple lines.
• Toplevel: The Toplevel widget is used to provide a separate window
container.
• LabelFrame: A labelframe is a simple container widget. Its pri-
mary purpose is to act as a spacer or container for complex window
layouts.
• tkMessageBox: This module is used to display message boxes in
your applications.
• Spinbox: The Spinbox widget is a variant of the standard Tkinter
Entry widget, which can be used to select from a fixed number of
values.
• PanedWindow: A PanedWindow is a container widget that may
contain any number of panes, arranged horizontally or vertically.

Geometry Management
All Tkinter widgets have access to specific geometry management meth-
ods, which have the purpose of organizing widgets throughout the parent
widget area. Tkinter exposes the following geometry manager classes:
pack, grid, and place.

• pack(): This geometry manager organizes widgets in blocks before

placing them in the parent widget.
• grid(): This geometry manager organizes widgets in a table-like
structure in the parent widget.
• place(): This geometry manager organizes widgets by placing them
in a specific position in the parent widget.