SQLAlchemy

SQLAlchemy

About the Tutorial

SQLAlchemy is a popular SQL toolkit and Object Relational Mapper. It is written in
Python and gives full power and flexibility of SQL to an application developer. It is an
open source and cross-platform software released under MIT license.

SQLAlchemy is famous for its object-relational mapper (ORM), using which classes can be
mapped to the database, thereby allowing the object model and database schema to
develop in a cleanly decoupled way from the beginning.

Audience
This tutorial is designed for all those Python programmers who would like to understand
the ORM framework with SQLAlchemy and its API.

Prerequisites
Before you start proceeding with this tutorial, we assume you have a good understanding
of the Python programming language. A basic understanding of relational databases, DB-
API, and SQL is desired to understand this tutorial.

Copyright & Disclaimer

 Copyright 2018 by Tutorials Point (I) Pvt. Ltd.

All the content and graphics published in this e-book are the property of Tutorials Point (I)
Pvt. Ltd. The user of this e-book is prohibited to reuse, retain, copy, distribute or republish
any contents or a part of contents of this e-book in any manner without written consent
of the publisher.

We strive to update the contents of our website and tutorials as timely and as precisely as
possible, however, the contents may contain inaccuracies or errors. Tutorials Point (I) Pvt.
Ltd. provides no guarantee regarding the accuracy, timeliness or completeness of our
website or its contents including this tutorial. If you discover any errors on our website or
in this tutorial, please notify us at [email protected]

i
 SQLAlchemy

Table of Contents
About the Tutorial ............................................................................................................................................ i

Audience ........................................................................................................................................................... i

Prerequisites ..................................................................................................................................................... i

Copyright & Disclaimer ..................................................................................................................................... i

Table of Contents ............................................................................................................................................ ii

1. SQLAlchemy – Introduction....................................................................................................................... 1

What is ORM? .................................................................................................................................................. 1

SQLAlchemy - Environment setup ................................................................................................................... 1

SQLALCHEMY CORE .................................................................................................................... 3

2. SQLAlchemy Core – Expression Language ................................................................................................. 4

3. SQLAlchemy Core – Connecting to Database............................................................................................. 5

4. SQLAlchemy Core – Creating Table ........................................................................................................... 7

5. SQLAlchemy Core – SQL Expressions ....................................................................................................... 10

6. SQLAlchemy Core – Executing Expression ............................................................................................... 11

7. SQLAlchemy Core – Selecting Rows......................................................................................................... 14

8. SQLAlchemy Core – Using Textual SQL .................................................................................................... 16

9. SQLAlchemy Core – Using Aliases ........................................................................................................... 18

10. SQLAlchemy Core – Using UPDATE Expression ........................................................................................ 19

11. SQLalchemy Core – Using DELETE Expression .......................................................................................... 21

12. SQLAlchemy Core – Using Multiple Tables .............................................................................................. 22

13. SQLAlchemy Core – Using Multiple Table Updates .................................................................................. 26

14. SQLAlchemy Core – Parameter-Ordered Updates ................................................................................... 27

15. SQLAlchemy Core – Multiple Table Deletes............................................................................................. 28

16. SQLAlchemy Core – Using Joins ............................................................................................................... 29

17. SQLAlchemy Core – Using Conjunctions .................................................................................................. 31

and_() function .............................................................................................................................................. 31

ii
 SQLAlchemy

or_() function ................................................................................................................................................. 32

asc() function ................................................................................................................................................. 33

desc() function ............................................................................................................................................... 34

between() function ........................................................................................................................................ 34

18. SQLAlchemy Core – Using Functions ....................................................................................................... 35

19. SQLAlchemy Core – Using Set Operations ............................................................................................... 37

union() ........................................................................................................................................................... 37

union_all() ..................................................................................................................................................... 38

except_() ........................................................................................................................................................ 38

intersect() ...................................................................................................................................................... 39

SQLALCHEMY ORM ................................................................................................................... 40

20. SQLAlchemy ORM – Declaring Mapping .................................................................................................. 41

Declare Mapping ........................................................................................................................................... 41

21. SQLAlchemy ORM – Creating Session ...................................................................................................... 44

22. SQLAlchemy ORM – Adding Objects........................................................................................................ 45

23. SQLAlchemy ORM – Using Query ............................................................................................................ 47

24. SQLAlchemy ORM – Updating Objects .................................................................................................... 49

25. SQLAlchemy ORM – Applying Filter ........................................................................................................ 51

26. SQLAlchemy ORM – Filter Operators ...................................................................................................... 53

27. SQLAlchemy ORM – Returning List and Scalars ....................................................................................... 57

28. SQLAlchemy ORM – Textual SQL ............................................................................................................. 59

29. SQLAlchemy ORM – Building Relationship .............................................................................................. 61

30. SQLAlchemy ORM – Working with Related Objects ................................................................................ 64

31. SQLAlchemy ORM – Working with Joins ................................................................................................. 67

32. SQLAlchemy ORM – Common Relationship Operators ............................................................................ 70

33. SQLAlchemy ORM – Eager Loading ......................................................................................................... 72

Subquery Load ............................................................................................................................................... 72

iii
 SQLAlchemy

Joined Load .................................................................................................................................................... 73

34. SQLAlchemy ORM – Deleting Related Objects ........................................................................................ 74

35. SQLAlchemy ORM – Many to Many Relationships .................................................................................. 78

36. SQLAlchemy – Dialects ............................................................................................................................ 85

PostgreSQL .................................................................................................................................................... 85

MySQL ........................................................................................................................................................... 86

Oracle ............................................................................................................................................................ 86

Microsoft SQL Server ..................................................................................................................................... 86

SQLite ............................................................................................................................................................ 86

Conclusion ..................................................................................................................................................... 87

iv
 1. SQLAlchemy – Introduction SQLAlchemy

SQLAlchemy is a popular SQL toolkit and Object Relational Mapper. It is written in

Python and gives full power and flexibility of SQL to an application developer. It is an
open source and cross-platform software released under MIT license.

SQLAlchemy is famous for its object-relational mapper (ORM), using which, classes can be
mapped to the database, thereby allowing the object model and database schema to
develop in a cleanly decoupled way from the beginning.

As size and performance of SQL databases start to matter, they behave less like object
collections. On the other hand, as abstraction in object collections starts to matter, they
behave less like tables and rows. SQLAlchemy aims to accommodate both of these
principles.

For this reason, it has adopted the data mapper pattern (like Hibernate) rather than
the active record pattern used by a number of other ORMs. Databases and SQL will
be viewed in a different perspective using SQLAlchemy.

Michael Bayer is the original author of SQLAlchemy. Its initial version was released in
February 2006. Latest version is numbered as 1.2.7, released as recently as in April 2018.

What is ORM?
ORM (Object Relational Mapping) is a programming technique for converting data between
incompatible type systems in object-oriented programming languages. Usually, the type
system used in an Object Oriented (OO) language like Python contains non-scalar types.
These cannot be expressed as primitive types such as integers and strings. Hence, the OO
programmer has to convert objects in scalar data to interact with backend database.
However, data types in most of the database products such as Oracle, MySQL, etc., are
primary.

In an ORM system, each class maps to a table in the underlying database. Instead of
writing tedious database interfacing code yourself, an ORM takes care of these issues for
you while you can focus on programming the logics of the system.

SQLAlchemy - Environment setup

Let us discuss the environmental setup required to use SQLAlchemy.

Any version of Python higher than 2.7 is necessary to install SQLAlchemy. The easiest way
to install is by using Python Package Manager, pip. This utility is bundled with standard
distribution of Python.

pip install sqlalchemy

Using the above command, we can download the latest released version of SQLAlchemy
from http://pypi.python.org/pypi/SQLAlchemy and install it to your system.

In case of anaconda distribution of Python, SQLAlchemy can be installed from conda

terminal using the below command:

1
 SQLAlchemy

conda install -c anaconda sqlalchemy

It is also possible to install SQLAlchemy from below source code:

python setup.py install

SQLAlchemy is designed to operate with a DBAPI implementation built for a particular

database. It uses dialect system to communicate with various types of DBAPI
implementations and databases. All dialects require that an appropriate DBAPI driver is
installed.

The following are the dialects included:

 Firebird
 Microsoft SQL Server
 MySQL
 Oracle
 PostgreSQL
 SQLite
 Sybase

To check if SQLAlchemy is properly installed and to know its version, enter the following
command in the Python prompt:

>>> import sqlalchemy

>>>sqlalchemy.__version__
'1.2.7'

2
 SQLAlchemy

SQLAlchemy Core

3
 2. SQLAlchemy Core – Expression Language
SQLAlchemy

SQLAlchemy core includes SQL rendering engine, DBAPI integration, transaction

integration, and schema description services. SQLAlchemy core uses SQL Expression
Language that provides a schema-centric usage paradigm whereas SQLAlchemy ORM
is a domain-centric mode of usage.

The SQL Expression Language presents a system of representing relational database

structures and expressions using Python constructs. It presents a system of representing
the primitive constructs of the relational database directly without opinion, which is in
contrast to ORM that presents a high level and abstracted pattern of usage, which itself is
an example of applied usage of the Expression Language.

Expression Language is one of the core components of SQLAlchemy. It allows the

programmer to specify SQL statements in Python code and use it directly in more complex
queries. Expression language is independent of backend and comprehensively covers
every aspect of raw SQL. It is closer to raw SQL than any other component in SQLAlchemy.

Expression Language represents the primitive constructs of the relational database

directly. Because the ORM is based on top of Expression language, a typical Python
database application may have overlapped use of both. The application may use
expression language alone, though it has to define its own system of translating application
concepts into individual database queries.

Statements of Expression language will be translated into corresponding raw SQL queries
by SQLAlchemy engine. We shall now learn how to create the engine and execute various
SQL queries with its help.

4
 3. SQLAlchemy Core – Connecting to Database
SQLAlchemy

In the previous chapter, we have discussed about expression Language in SQLAlchemy.

Now let us proceed towards the steps involved in connecting to a database.

Engine class connects a Pool and Dialect together to provide a source of database
connectivity and behavior. An object of Engine class is instantiated using the
create_engine() function.

The create_engine() function takes the database as one argument. The database is not
needed to be defined anywhere. The standard calling form has to send the URL as the first
positional argument, usually a string that indicates database dialect and connection
arguments. Using the code given below, we can create a database.

>>> from sqlalchemy import create_engine

>>> engine = create_engine('sqlite:///college.db', echo=True)

For a MySQL database, use the below command:

engine = create_engine("mysql://user:pwd@localhost/college",echo=True)

To specifically mention DB-API to be used for connection, the URL string takes the form
as follows:

dialect[+driver]://user:password@host/dbname

For example, if you are using PyMySQL driver with MySQL, use the following command:

mysql+pymysql://<username>:<password>@<host>/<dbname>

The echo flag is a shortcut to set up SQLAlchemy logging, which is accomplished via
Python’s standard logging module. In the subsequent chapters, we will learn all the
generated SQLs. To hide the verbose output, set echo attribute to None. Other arguments
to create_engine() function may be dialect specific.

5
 SQLAlchemy

The create_engine() function returns an Engine object. Some important methods of

Engine class are:

connect() Returns connection object

execute() Executes a SQL statement construct

begin() Returns a context manager delivering a Connection with a

Transaction established. Upon successful operation, the
Transaction is committed, else it is rolled back

dispose() Disposes of the connection pool used by the Engine

driver() Driver name of the Dialect in use by the Engine

table_names() Returns a list of all table names available in the database

transaction() Executes the given function within a transaction boundary

6
 4. SQLAlchemy Core – Creating TableSQLAlchemy

Let us now discuss how to use the create table function.

The SQL Expression Language constructs its expressions against table columns.
SQLAlchemy Column object represents a column in a database table which is in turn
represented by a Tableobject. Metadata contains definitions of tables and associated
objects such as index, view, triggers, etc.

Hence an object of MetaData class from SQLAlchemy Metadata is a collection of Table

objects and their associated schema constructs. It holds a collection of Table objects as
well as an optional binding to an Engine or Connection.

from sqlalchemy import MetaData

meta=MetaData()

Constructor of MetaData class can have bind and schema parameters which are by default
None.

Next, we define our tables all within above metadata catalog, using the Table construct,
which resembles regular SQL CREATE TABLE statement.

An object of Table class represents corresponding table in a database. The constructor

takes the following parameters:

Name Name of the table

Metadata MetaData object that will hold this table

column(s) One or more objects of column class

Column object represents a column in a database table. Constructor takes name, type
and other parameters such as primary_key, autoincrement and other constraints.

SQLAlchemy matches Python data to the best possible generic column data types defined
in it. Some of the generic data types are:

 BigInteger

 Boolean

 Date

 DateTime

 Float

 Integer

 Numeric

 SmallInteger

7
 SQLAlchemy

 String

 Text

 Time

To create a students table in college database, use the following snippet:

from sqlalchemy import Table, Column, Integer, String, MetaData

meta = MetaData()
students = Table('students', meta, Column('id', Integer, primary_key=True),
Column('name', String), Column('lastname', String), )

The create_all() function uses the engine object to create all the defined table objects and
stores the information in metadata.

meta.create_all(engine)

Complete code is given below which will create a SQLite database college.db with a
students table in it.

from sqlalchemy import create_engine, MetaData, Table, Column, Integer, String

engine = create_engine('sqlite:///college.db', echo=True)
meta=MetaData()
students = Table('students', meta, Column('id', Integer, primary_key=True),
Column('name', String), Column('lastname', String), )
meta.create_all(engine)

Because echo attribute of create_engine() function is set to True, the console will display
the actual SQL query for table creation as follows:

CREATE TABLE students (

id INTEGER NOT NULL,
name VARCHAR,
lastname VARCHAR,
PRIMARY KEY (id)
)

The college.db will be created in current working directory. To check if the students table
is created, you can open the database using any SQLite GUI tool such as SQLiteStudio.

The below image shows the students table that is created in the database:

8
SQLAlchemy

9
 5. SQLAlchemy Core – SQL Expressions
SQLAlchemy

In this chapter, we will briefly focus on the SQL Expressions and their functions.

SQL expressions are constructed using corresponding methods relative to target table
object. For example, the INSERT statement is created by executing insert() method as
follows:

ins=students.insert()

The result of above method is an insert object that can be verified by using str() function.
The below code inserts details like student id, name, lastname.

'INSERT INTO students (id, name, lastname) VALUES (:id, :name, :lastname)'

It is possible to insert value in a specific field by values() method to insert object. The
code for the same is given below:

>>> ins = users.insert().values(name='Karan')

>>> str(ins)
'INSERT INTO users (name) VALUES (:name)'

The SQL echoed on Python console doesn’t show the actual value (‘Karan’ in this case).
Instead, SQLALchemy generates a bind parameter which is visible in compiled form of the
statement.

ins.compile().params
{'name': 'Karan'}

Similarly, methods like update(), delete() and select() create UPDATE, DELETE and
SELECT expressions respectively. We shall learn about them in later chapters.

10
 6. SQLAlchemy Core – Executing Expression
SQLAlchemy

In the previous chapter, we have learnt SQL Expressions. In this chapter, we shall look
into the execution of these expressions.

In order to execute the resulting SQL expressions, we have to obtain a connection

object representing an actively checked out DBAPI connection resource and then
feed the expression object as shown in the code below.

conn = engine.connect()

The following insert() object can be used for execute() method:

ins=students.insert().values(name='Ravi', lastname='Kapoor')
result = conn.execute(ins)

The console shows the result of execution of SQL expression as below:

INSERT INTO students (name, lastname) VALUES (?, ?)

('Ravi', 'Kapoor')
COMMIT

Following is the entire snippet that shows the execution of INSERT query using
SQLAlchemy’s core technique:

from sqlalchemy import create_engine, MetaData, Table, Column, Integer, String

engine = create_engine('sqlite:///college.db', echo=True)
meta=MetaData()
students = Table('students', meta, Column('id', Integer, primary_key=True),
Column('name', String), Column('lastname', String), )
ins=students.insert()
ins=students.insert().values(name='Ravi', lastname='Kapoor')
conn = engine.connect()
result = conn.execute(ins)

The result can be verified by opening the database using SQLite Studio as shown in the
below screenshot:

11
 SQLAlchemy

The result variable is known as a ResultProxy object. It is analogous to the DBAPI cursor
object. We can acquire information about the primary key values which were generated
from our statement using ResultProxy.inserted_primary_key as shown below:

result.inserted_primary_key
[1]

To issue many inserts using DBAPI’s execute many() method, we can send in a list of
dictionaries each containing a distinct set of parameters to be inserted.

conn.execute(students.insert(), [
{'name':'Rajiv', 'lastname' : 'Khanna'},
{'name':'Komal','lastname' : 'Bhandari'},
{'name':'Abdul','lastname' : 'Sattar'},
{'name':'Priya','lastname' : 'Rajhans'},
])

12
 SQLAlchemy

This is reflected in the data view of the table as shown in the following figure:

13
 7. SQLAlchemy Core – Selecting RowsSQLAlchemy

In this chapter, we will discuss about the concept of selecting rows in the table object.

The select() method of table object enables us to construct SELECT expression.

s=students.select()

The select object translates to SELECT query by str(s) function as shown below:

'SELECT students.id, students.name, students.lastname FROM students'

We can use this select object as a parameter to execute() method of connection object as
shown in the code below:

result=conn.execute(s)

When the above statement is executed, Python shell echoes following equivalent SQL
expression:

SELECT students.id, students.name, students.lastname

FROM students

The resultant variable is an equivalent of cursor in DBAPI. We can now fetch records using
fetchone() method.

row=result.fetchone()

All selected rows in the table can be printed by a for loop as given below:

for row in result:

print (row)

The complete code to print all rows from students table is shown below:

from sqlalchemy import create_engine, MetaData, Table, Column, Integer, String

engine = create_engine('sqlite:///college.db', echo=True)
meta=MetaData()
students = Table('students', meta, Column('id', Integer, primary_key=True),
Column('name', String), Column('lastname', String), )
s=students.select()

conn = engine.connect()

14
 SQLAlchemy

result=conn.execute(s)
for row in result:
print (row)

The output shown in Python shell is as follows:

(1, 'Ravi', 'Kapoor')

(2, 'Rajiv', 'Khanna')
(3, 'Komal', 'Bhandari')
(4, 'Abdul', 'Sattar')
(5, 'Priya', 'Rajhans')

The WHERE clause of SELECT query can be applied by using Select.where(). For
example, if we want to display rows with id >2

s=students.select().where(students.c.id>2)
result=conn.execute(s)
for row in result:
print (row)

Here c attribute is an alias for column. Following output will be displayed on the shell:

(3, 'Komal', 'Bhandari')

(4, 'Abdul', 'Sattar')
(5, 'Priya', 'Rajhans')

Here, we have to note that select object can also be obtained by select() function in
sqlalchemy.sql module. The select() function requires the table object as argument.

from sqlalchemy.sql import select

s = select([users])
result = conn.execute(s)

15
 8. SQLAlchemy Core – Using Textual SQL
SQLAlchemy

SQLAlchemy lets you just use strings, for those cases when the SQL is already known and
there isn’t a strong need for the statement to support dynamic features.
The text() construct is used to compose a textual statement that is passed to the database
mostly unchanged.

It constructs a new TextClause, representing a textual SQL string directly as shown in

the below code:

from sqlalchemy import text

t = text("SELECT * FROM students")
result = connection.execute(t)

The advantages text() provides over a plain string are:

 backend-neutral support for bind parameters

 per-statement execution options
 result-column typing behaviour

The text()function requires Bound parameters in the named colon format. They are
consistent regardless of database backend. To send values in for the parameters, we pass
them into the execute() method as additional arguments.

The following example uses bound parameters in textual SQL:

from sqlalchemy.sql import text

s=text("select students.name, students.lastname from students where
students.name between :x and :y")
conn.execute(s, x='A', y='L').fetchall()

The text() function constructs SQL expression as follows:

select students.name, students.lastname from students where students.name

between ? and ?

The values of x=’A’ and y=’L’ are passed as parameters. Result is a list of rows with names
between ‘A’ and ‘L’:

[('Komal', 'Bhandari'), ('Abdul', 'Sattar')]

The text() construct supports pre-established bound values using the

TextClause.bindparams() method. The parameters can also be explicitly typed as follows:

16
 SQLAlchemy

stmt = text("SELECT * FROM students WHERE students.name BETWEEN :x AND :y")

stmt = stmt.bindparams(bindparam("x", type_=String), bindparam("y",
type_=String))
result = conn.execute(stmt, {"x": "A", "y": "L"})
The text() function also be produces fragments of SQL within a select() object
that accepts text() objects as an arguments. The “geometry” of the statement is
provided by select() construct , and the textual content by text() construct.
We can build a statement without the need to refer to any pre-established Table
metadata. from sqlalchemy.sql import select
s=select([text("students.name, students.lastname from
students")]).where(text("students.name between :x and :y"))
conn.execute(s, x='A', y='L').fetchall()

You can also use and_() function to combine multiple conditions in WHERE clause created
with the help of text() function.

from sqlalchemy import and_

from sqlalchemy.sql import select
s=select([text("* from students")]) \
.where(
and_(
text("students.name between :x and :y"),
text("students.id>2")
)
)
conn.execute(s, x='A', y='L').fetchall()

Above code fetches rows with names between “A” and “L” with id greater than 2. The
output of the code is given below:

[(3, 'Komal', 'Bhandari'), (4, 'Abdul', 'Sattar')]

17
 9. SQLAlchemy Core – Using AliasesSQLAlchemy

The alias in SQL corresponds to a “renamed” version of a table or SELECT statement,

which occurs anytime you say “SELECT * FROM table1 AS a”. The AS creates a new name
for the table. Aliases allow any table or subquery to be referenced by a unique name.

In case of a table, this allows the same table to be named in the FROM clause multiple
times. It provides a parent name for the columns represented by the statement, allowing
them to be referenced relative to this name.

In SQLAlchemy, any Table, select() construct, or other selectable object can be turned
into an alias using the From Clause.alias() method, which produces an Alias construct.
The alias() function in sqlalchemy.sql module represents an alias, as typically applied to
any table or sub-select within a SQL statement using the AS keyword.

from sqlalchemy.sql import alias

st=students.alias("a")

This alias can now be used in select() construct to refer to students table:

s=select([st]).where(st.c.id>2)

This translates to SQL expression as follows:

SELECT a.id, a.name, a.lastname FROM students AS a WHERE a.id > 2

We can now execute this SQL query with the execute() method of connection object. The
complete code is as follows:

from sqlalchemy.sql import alias, select

st=students.alias("a")
s=select([st]).where(st.c.id>2)
conn.execute(s).fetchall()

When above line of code is executed, it generates the following output:

[(3, 'Komal', 'Bhandari'), (4, 'Abdul', 'Sattar'), (5, 'Priya', 'Rajhans')]

18
10. SQLAlchemy Core – Using UPDATE Expression
SQLAlchemy

The update() method on target table object constructs equivalent UPDATE SQL
expression.

table.update().where(conditions).values(SET expressions)

The values() method on the resultant update object is used to specify the SET conditions
of the UPDATE. If left as None, the SET conditions are determined from those parameters
passed to the statement during the execution and/or compilation of the statement.

The where clause is an Optional expression describing the WHERE condition of

the UPDATE statement.

Following code snippet changes value of ‘lastname’ column from ‘Khanna’ to ‘Kapoor’ in
students table:

stmt=students.update().where(students.c.lastname=='Khanna').values(lastname='Ka
poor')

The stmt object is an update object that translates to:

'UPDATE students SET lastname=:lastname WHERE students.lastname = :lastname_1'

The bound parameter lastname_1 will be substituted when execute() method is

invoked. The complete update code is given below:

from sqlalchemy import create_engine, MetaData, Table, Column, Integer, String

engine = create_engine('sqlite:///college.db', echo=True)
meta=MetaData()
students = Table('students', meta, Column('id', Integer, primary_key=True),
Column('name', String), Column('lastname', String), )
conn = engine.connect()
stmt=students.update().where(students.c.lastname=='Khanna').values(lastname='Ka
poor')
conn.execute(stmt)
s=students.select()
conn.execute(s).fetchall()

19
 SQLAlchemy

The above code displays following output with second row showing effect of update
operation as in the screenshot given:

[(1, 'Ravi', 'Kapoor'),

(2, 'Rajiv', 'Kapoor'),
(3, 'Komal', 'Bhandari'),
(4, 'Abdul', 'Sattar'),
(5, 'Priya', 'Rajhans')]

Note that similar functionality can also be achieved by using update() function in
sqlalchemy.sql.expression module as shown below:

from sqlalchemy.sql.expression import update

stmt=update(students).where(students.c.lastname=='Khanna').values(lastname='Kap
oor')

20
11. SQLalchemy Core – Using DELETE Expression
SQLAlchemy

In the previous chapter, we have understood what an Update expression does. The next
expression that we are going to learn is Delete.

The delete operation can be achieved by running delete() method on target table object
as given in the following statement:

stmt=students.delete()

In case of students table, the above line of code constructs a SQL expression as following:

'DELETE FROM students'

However, this will delete all rows in students table. Usually DELETE query is associated
with a logical expression specified by WHERE clause. The following statement shows where
parameter:

stmt=students.delete().where(students.c.id>2)

The resultant SQL expression will have a bound parameter which will be substituted at
runtime when the statement is executed.

'DELETE FROM students WHERE students.id > :id_1'

Following code example will delete those rows from students table having lastname as
‘Khanna’:

from sqlalchemy.sql.expression import update

from sqlalchemy import create_engine, MetaData, Table, Column, Integer, String
engine = create_engine('sqlite:///college.db', echo=True)
meta=MetaData()
students = Table('students', meta, Column('id', Integer, primary_key=True),
Column('name', String), Column('lastname', String), )
conn = engine.connect()
stmt=students.delete().where(students.c.lastname=='Khanna')
conn.execute(stmt)
s=students.select()
conn.execute(s).fetchall()

To verify the result, refresh the data view of students table in SQLiteStudio.

21
 12. SQLAlchemy Core – Using Multiple Tables
SQLAlchemy

One of the important features of RDBMS is establishing relation between tables. SQL
operations like SELECT, UPDATE and DELETE can be performed on related tables. This
section describes these operations using SQLAlchemy.

For this purpose, two tables are created in our SQLite database (college.db). The students
table has the same structure as given in the previous section; whereas the addresses table
has st_id column which is mapped to id column in students table using foreign key
constraint.

The following code will create two tables in college.db:

from sqlalchemy import create_engine, MetaData, Table, Column, Integer, String,

ForeignKey
engine = create_engine('sqlite:///college.db', echo=True)
meta=MetaData()
students = Table('students', meta, Column('id', Integer, primary_key=True),
Column('name', String), Column('lastname', String), )
addresses=Table('addresses', meta, Column('id', Integer, primary_key=True),
Column('st_id', Integer, ForeignKey('students.id')), Column('postal_add',
String), Column('email_add', String))
meta.create_all(engine)

Above code will translate to CREATE TABLE queries for students and addresses table as
below:

CREATE TABLE students (

id INTEGER NOT NULL,
name VARCHAR,
lastname VARCHAR,
PRIMARY KEY (id)
)
CREATE TABLE addresses (
id INTEGER NOT NULL,
st_id INTEGER,
postal_add VARCHAR,
email_add VARCHAR,
PRIMARY KEY (id),
FOREIGN KEY(st_id) REFERENCES students (id)
)

22
 SQLAlchemy

The following screenshots present the above code very clearly:

These tables are populated with data by executing insert() method of table objects. To
insert 5 rows in students table, you can use the code given below:

from sqlalchemy import create_engine, MetaData, Table, Column, Integer, String

engine = create_engine('sqlite:///college.db', echo=True)
meta=MetaData()
conn = engine.connect()
students = Table('students', meta, Column('id', Integer, primary_key=True),
Column('name', String), Column('lastname', String), )

23
 SQLAlchemy

conn.execute(students.insert(), [
{'name':'Ravi', 'lastname':'Kapoor'},
{'name':'Rajiv', 'lastname' : 'Khanna'},
{'name':'Komal','lastname' : 'Bhandari'},
{'name':'Abdul','lastname' : 'Sattar'},
{'name':'Priya','lastname' : 'Rajhans'},
])

Rows are added in addresses table with the help of the following code:

from sqlalchemy import create_engine, MetaData, Table, Column, Integer, String

engine = create_engine('sqlite:///college.db', echo=True)
meta=MetaData()
conn = engine.connect()
addresses=Table('addresses', meta, Column('id', Integer, primary_key=True),
Column('st_id', Integer), Column('postal_add', String), Column('email_add',
String))
conn.execute(addresses.insert(), [
{'st_id':1, 'postal_add':'Shivajinagar Pune',
'email_add':'[email protected]'},
{'st_id':1, 'postal_add':'ChurchGate Mumbai',
'email_add':'[email protected]'},
{'st_id':3, 'postal_add':'Jubilee Hills Hyderabad',
'email_add':'[email protected]'},
{'st_id':5, 'postal_add':'MG Road Bangaluru',
'email_add':'[email protected]'},
{'st_id':2, 'postal_add':'Cannought Place new Delhi',
'email_add':'[email protected]'},
])

Note that the st_id column in addresses table refers to id column in students table. We
can now use this relation to fetch data from both the tables. We want to fetch name and
lastname from students table corresponding to st_id in the addresses table.

from sqlalchemy.sql import select

s = select([students, addresses]).where(students.c.id == addresses.c.st_id)
result=conn.execute(s)
for row in result:
print (row)

The select objects will effectively translate into following SQL expression joining two tables
on common relation:

24
 SQLAlchemy

SELECT students.id, students.name, students.lastname, addresses.id,

addresses.st_id, addresses.postal_add, addresses.email_add
FROM students, addresses
WHERE students.id = addresses.st_id

This will produce output extracting corresponding data from both tables as follows:

(1, 'Ravi', 'Kapoor', 1, 1, 'Shivajinagar Pune', '[email protected]')

(1, 'Ravi', 'Kapoor', 2, 1, 'ChurchGate Mumbai', '[email protected]')
(3, 'Komal', 'Bhandari', 3, 3, 'Jubilee Hills Hyderabad', '[email protected]')
(5, 'Priya', 'Rajhans', 4, 5, 'MG Road Bangaluru', '[email protected]')
(2, 'Rajiv', 'Khanna', 5, 2, 'Cannought Place new Delhi', '[email protected]')

25
13. SQLAlchemy Core – Using Multiple TableSQLAlchemy
Updates

In the previous chapter, we have discussed about how to use multiple tables. So we
proceed a step further and learn multiple table updates in this chapter.

Using SQLAlchemy’s table object, more than one table can be specified in WHERE clause
of update() method. The PostgreSQL and Microsoft SQL Server support UPDATE
statements that refer to multiple tables. This implements “UPDATE FROM” syntax, which
updates one table at a time. However, additional tables can be referenced in an additional
“FROM” clause in the WHERE clause directly. The following lines of codes explain the
concept of multiple table updates clearly.

stmt = students.update().\
values({
students.c.name:'xyz',
addresses.c.email_add:'[email protected]'
}).\
where(students.c.id == addresses.c.id)

The update object is equivalent to the following UPDATE query:

UPDATE students SET email_add=:addresses_email_add, name=:name FROM addresses

WHERE students.id = addresses.id

As far as MySQL dialect is concerned, multiple tables can be embedded into a single
UPDATE statement separated by a comma as given below:

stmt = students.update().\
values(name='xyz').\
where(students.c.id == addresses.c.id)

The following code depicts the resulting UPDATE query:

'UPDATE students SET name=:name FROM addresses WHERE students.id =

addresses.id'

SQLite dialect however doesn’t support multiple-table criteria within UPDATE and shows
following error:

NotImplementedError: This backend does not support multiple-table criteria

within UPDATE

26
14. SQLAlchemy Core – Parameter-Ordered SQLAlchemy
Updates

The UPDATE query of raw SQL has SET clause. It is rendered by the update() construct
using the column ordering given in the originating Table object. Therefore, a particular
UPDATE statement with particular columns will be rendered the same each time. Since the
parameters themselves are passed to the Update.values() method as Python dictionary
keys, there is no other fixed ordering available.

In some cases, the order of parameters rendered in the SET clause are significant. In
MySQL, providing updates to column values is based on that of other column values.

Following statement’s result:

UPDATE table1 SET x = y + 10, y = 20

will have a different result than:

UPDATE table1 SET y = 20, x = y + 10

SET clause in MySQL is evaluated on a per-value basis and not on per-row basis. For this
purpose, the preserve_parameter_order is used. Python list of 2-tuples is given as
argument to the Update.values() method:

stmt = table1.update(preserve_parameter_order=True).\
values([(table1.c.y, 20), (table1.c.x, table1.c.y + 10)])

The List object is similar to dictionary except that it is ordered. This ensures that the “y”
column’s SET clause will render first, then the “x” column’s SET clause.

27
 15. SQLAlchemy Core – Multiple Table Deletes
SQLAlchemy

In this chapter, we will look into the Multiple Table Deletes expression which is similar to
using Multiple Table Updates function.

More than one table can be referred in WHERE clause of DELETE statement in many DBMS
dialects. For PG and MySQL, “DELETE USING” syntax is used; and for SQL Server, using
“DELETE FROM” expression refers to more than one table. The SQLAlchemy delete()
construct supports both of these modes implicitly, by specifying multiple tables in the
WHERE clause as follows:

stmt = users.delete().\
where(users.c.id == addresses.c.id).\
where(addresses.c.email_address.startswith('xyz%'))
conn.execute(stmt)

On a PostgreSQL backend, the resulting SQL from the above statement would render as:

DELETE FROM users USING addresses

WHERE users.id = addresses.id
AND (addresses.email_address LIKE %(email_address_1)s || '%%')

If this method is used with a database that doesn’t support this behaviour, the compiler
will raise NotImplementedError.

28
 16. SQLAlchemy Core – Using Joins SQLAlchemy

In this chapter, we will learn how to use Joins in SQLAlchemy.

Effect of joining is achieved by just placing two tables in either the columns clause or
the where clause of the select() construct. Now we use the join() and outerjoin()
methods.

The join() method returns a join object from one table object to another.

join(right, onclause=None, isouter=False, full=False)

The functions of the parameters mentioned in the above code are as follows:

 right – the right side of the join; this is any Table object

 onclause – a SQL expression representing the ON clause of the join. If left at None,
it attempts to join the two tables based on a foreign key relationship

 isouter – if True, renders a LEFT OUTER JOIN, instead of JOIN

 full – if True, renders a FULL OUTER JOIN, instead of LEFT OUTER JOIN

For example, following use of join() method will automatically result in join based on the
foreign key.

>>> print(students.join(addresses))

This is equivalent to following SQL expression:

students JOIN addresses ON students.id = addresses.st_id

You can explicitly mention joining criteria as follows:

j = students.join(addresses, students.c.id == addresses.c.st_id)

If we now build the below select construct using this join as:

stmt = select([students]).select_from(j)

This will result in following SQL expression:

SELECT students.id, students.name, students.lastname

FROM students JOIN addresses ON students.id = addresses.st_id

If this statement is executed using the connection representing engine, data belonging to
selected columns will be displayed. The complete code is as follows:

29
 SQLAlchemy

from sqlalchemy import create_engine, MetaData, Table, Column, Integer, String,

ForeignKey
engine = create_engine('sqlite:///college.db', echo=True)
meta=MetaData()
conn = engine.connect()
students = Table('students', meta, Column('id', Integer, primary_key=True),
Column('name', String), Column('lastname', String), )
addresses=Table('addresses', meta, Column('id', Integer, primary_key=True),
Column('st_id', Integer,ForeignKey('students.id')), Column('postal_add',
String), Column('email_add', String))
from sqlalchemy import join
from sqlalchemy.sql import select
j = students.join(addresses, students.c.id == addresses.c.st_id)
stmt = select([students]).select_from(j)
result=conn.execute(stmt)
result.fetchall()

The following is the output of the above code:

[(1, 'Ravi', 'Kapoor'),

(1, 'Ravi', 'Kapoor'),
(3, 'Komal', 'Bhandari'),
(5, 'Priya', 'Rajhans'),
(2, 'Rajiv', 'Khanna')]

30
 17. SQLAlchemy Core – Using Conjunctions
SQLAlchemy

Conjunctions are functions in SQLAlchemy module that implement relational operators

used in WHERE clause of SQL expressions. The operators AND, OR, NOT, etc., are used to
form a compound expression combining two individual logical expressions. A simple
example of using AND in SELECT statement is as follows:

SELECT * from EMPLOYEE WHERE salary>10000 AND age>30

SQLAlchemy functions and_(), or_() and not_() respectively implement AND, OR and NOT
operators.

and_() function
It produces a conjunction of expressions joined by AND. An example is given below for
better understanding:

from sqlalchemy import and_

print(and_(
students.c.name=='Ravi',
students.c.id <3
)
)

This translates to:

students.name = :name_1 AND students.id < :id_1

To use and_() in a select() construct on a students table, use the following line of code:

stmt=select([students]).where(and_(students.c.name=='Ravi', students.c.id <3))

SELECT statement of the following nature will be constructed:

SELECT students.id, students.name, students.lastname

FROM students
WHERE students.name = :name_1 AND students.id < :id_1

31
 SQLAlchemy

The complete code that displays output of the above SELECT query is as follows:

from sqlalchemy import create_engine, MetaData, Table, Column, Integer, String,

ForeignKey, select
engine = create_engine('sqlite:///college.db', echo=True)
meta=MetaData()
conn = engine.connect()
students = Table('students', meta, Column('id', Integer, primary_key=True),
Column('name', String), Column('lastname', String), )

from sqlalchemy import and_, or_

stmt=select([students]).where(and_(students.c.name=='Ravi', students.c.id <3))

result=conn.execute(stmt)
print (result.fetchall())

Following row will be selected assuming that students table is populated with data used in
previous example:

[(1, 'Ravi', 'Kapoor')]

or_() function
It produces conjunction of expressions joined by OR. We shall replace the stmt object in
the above example with the following one using or_()

stmt=select([students]).where(or_(students.c.name=='Ravi', students.c.id <3))

Which will be effectively equivalent to following SELECT query:

SELECT students.id, students.name, students.lastname

FROM students
WHERE students.name = :name_1 OR students.id < :id_1

Once you make the substitution and run the above code, the result will be two rows falling
in the OR condition:

[(1, 'Ravi', 'Kapoor'),

(2, 'Rajiv', 'Khanna')]

32
 SQLAlchemy

asc() function
It produces an ascending ORDER BY clause. The function takes the column to apply the
function as a parameter.

from sqlalchemy import asc

stmt = select([students]).order_by(asc(students.c.name))

The statement implements following SQL expression:

SELECT students.id, students.name, students.lastname

FROM students ORDER BY students.name ASC

Following code lists out all records in students table in ascending order of name column:

from sqlalchemy import create_engine, MetaData, Table, Column, Integer, String,

ForeignKey, select
engine = create_engine('sqlite:///college.db', echo=True)
meta=MetaData()
conn = engine.connect()
students = Table('students', meta, Column('id', Integer, primary_key=True),
Column('name', String), Column('lastname', String), )
from sqlalchemy import asc
stmt = select([students]).order_by(asc(students.c.name))
result=conn.execute(stmt)
for row in result:
print (row)

Above code produces following output:

(4, 'Abdul', 'Sattar')

(3, 'Komal', 'Bhandari')
(5, 'Priya', 'Rajhans')
(2, 'Rajiv', 'Khanna')
(1, 'Ravi', 'Kapoor')

33
 SQLAlchemy

desc() function
Similarly desc() function produces descending ORDER BY clause as follows:

from sqlalchemy import desc

stmt = select([students]).order_by(desc(students.c.lastname))

The equivalent SQL expression is:

SELECT students.id, students.name, students.lastname

FROM students ORDER BY students.lastname DESC

And the output for the above lines of code is:

(4, 'Abdul', 'Sattar')

(5, 'Priya', 'Rajhans')
(2, 'Rajiv', 'Khanna')
(1, 'Ravi', 'Kapoor')
(3, 'Komal', 'Bhandari')

between() function
It produces a BETWEEN predicate clause. This is generally used to validate if value of a
certain column falls between a range. For example, following code selects rows for which
id column is between 2 and 4:

from sqlalchemy import between

stmt = select([students]).where(between(students.c.id,2,4))
print (stmt)

The resulting SQL expression resembles:

SELECT students.id, students.name, students.lastname

FROM students
WHERE students.id BETWEEN :id_1 AND :id_2

and the result is as follows:

(2, 'Rajiv', 'Khanna')

(3, 'Komal', 'Bhandari')
(4, 'Abdul', 'Sattar')

34
 18. SQLAlchemy Core – Using Functions
SQLAlchemy

Some of the important functions used in SQLAlchemy are discussed in this chapter.

Standard SQL has recommended many functions which are implemented by most dialects.
They return a single value based on the arguments passed to it. Some SQL functions take
columns as arguments whereas some are generic. Thefunc keyword in SQLAlchemy
API is used to generate these functions.

In SQL, now() is a generic function. Following statements renders the now() function using
func:

from sqlalchemy.sql import func

result=conn.execute(select([func.now()]))
print (result.fetchone())

Sample result of above code may be as shown below:

(datetime.datetime(2018, 6, 16, 6, 4, 40),)

On the other hand, count() function which returns number of rows selected from a table,
is rendered by following usage of func:

from sqlalchemy.sql import func

result=conn.execute(select([func.count(students.c.id)]))
print (result.fetchone())

From the above code, count of number of rows in students table will be fetched.

Some built-in SQL functions are demonstrated using Employee table with following data:

ID Name Marks

1 Kamal 56

2 Fernandez 85

3 Sunil 62

4 Bhaskar 76

35
 SQLAlchemy

The max() function is implemented by following usage of func from SQLAlchemy which will
result in 85, the total maximum marks obtained:

from sqlalchemy.sql import func

result=conn.execute(select([func.max(employee.c.marks)]))
print (result.fetchone())

Similarly, min() function that will return 56, minimum marks, will be rendered by following
code:

from sqlalchemy.sql import func

result=conn.execute(select([func.min(employee.c.marks)]))
print (result.fetchone())

So, the AVG() function can also be implemented by using the below code:

from sqlalchemy.sql import func

result=conn.execute(select([func.avg(employee.c.marks)]))
print (result.fetchone())
Functions are normally used in the columns clause of a select statement. They
can also be given label as well as a type. A label to function allows the
result to be targeted in a result row based on a string name, and a type is
required when you need result-set processing to occur.from sqlalchemy.sql
import func
result=conn.execute(select([func.max(students.c.lastname).label('Name')]))
print (result.fetchone())

36
 19. SQLAlchemy Core – Using Set Operations
SQLAlchemy

In the last chapter, we have learnt about various functions such as max(), min(), count(),
etc., here, we will learn about set operations and their uses.

Set operations such as UNION and INTERSECT are supported by standard SQL and most
of its dialect. SQLAlchemy implements them with the help of following functions:

union()
While combining results of two or more SELECT statements, UNION eliminates duplicates
from the resultset. The number of columns and datatype must be same in both the tables.

The union() function returns a CompoundSelect object from multiple tables. Following
example demonstrates its use:

from sqlalchemy import create_engine, MetaData, Table, Column, Integer, String,

union
engine = create_engine('sqlite:///college.db', echo=True)
meta=MetaData()
conn = engine.connect()
addresses=Table('addresses', meta, Column('id', Integer, primary_key=True),
Column('st_id', Integer), Column('postal_add', String), Column('email_add',
String))
u=union(addresses.select().where(addresses.c.email_add.like('%@gmail.com')),
addresses.select().where(addresses.c.email_add.like('%@yahoo.com')))
result=conn.execute(u)
result.fetchall()

The union construct translates to following SQL expression:

SELECT addresses.id, addresses.st_id, addresses.postal_add, addresses.email_add

FROM addresses
WHERE addresses.email_add LIKE ? UNION SELECT addresses.id, addresses.st_id,
addresses.postal_add, addresses.email_add
FROM addresses
WHERE addresses.email_add LIKE ?

From our addresses table, following rows represent the union operation:

37
 SQLAlchemy

[(1, 1, 'Shivajinagar Pune', '[email protected]'),

(2, 1, 'ChurchGate Mumbai', '[email protected]'),
(3, 3, 'Jubilee Hills Hyderabad', '[email protected]'),
(4, 5, 'MG Road Bangaluru', '[email protected]')]

union_all()
UNION ALL operation cannot remove the duplicates and cannot sort the data in the
resultset. For example, in above query, UNION is replaced by UNION ALL to see the effect.

u=union_all(addresses.select().where(addresses.c.email_add.like('%@gmail.com'))
, addresses.select().where(addresses.c.email_add.like('%@yahoo.com')))

The corresponding SQL expression is as follows:

SELECT addresses.id, addresses.st_id, addresses.postal_add, addresses.email_add

FROM addresses
WHERE addresses.email_add LIKE ? UNION ALL SELECT addresses.id,
addresses.st_id, addresses.postal_add, addresses.email_add
FROM addresses
WHERE addresses.email_add LIKE ?

except_()
The SQL EXCEPT clause/operator is used to combine two SELECT statements and return
rows from the first SELECT statement that are not returned by the second SELECT
statement. The except_() function generates a SELECT expression with EXCEPT clause.

In the following example, the except_() function returns only those records from addresses
table that have ‘gmail.com’ in email_add field but excludes those which have ‘Pune’ as
part of postal_add field.

u=except_(addresses.select().where(addresses.c.email_add.like('%@gmail.com')),
addresses.select().where(addresses.c.postal_add.like('%Pune')))

Result of the above code is the following SQL expression:

SELECT addresses.id, addresses.st_id, addresses.postal_add, addresses.email_add

FROM addresses
WHERE addresses.email_add LIKE ? EXCEPT SELECT addresses.id, addresses.st_id,
addresses.postal_add, addresses.email_add
FROM addresses
WHERE addresses.postal_add LIKE ?

38
 SQLAlchemy

Assuming that addresses table contains data used in earlier examples, it will display
following output:

[(2, 1, 'ChurchGate Mumbai', '[email protected]'),

(3, 3, 'Jubilee Hills Hyderabad', '[email protected]')]

intersect()
Using INTERSECT operator, SQL displays common rows from both the SELECT statements.
The intersect() function implements this behaviour.

In following examples, two SELECT constructs are parameters to intersect() function. One
returns rows containing ‘gmail.com’ as part of email_add column, and other returns rows
having ‘Pune’ as part of postal_add column. The result will be common rows from both
resultsets.

u=intersect(addresses.select().where(addresses.c.email_add.like('%@gmail.com'))
, addresses.select().where(addresses.c.postal_add.like('%Pune')))

In effect, this is equivalent to following SQL statement:

SELECT addresses.id, addresses.st_id, addresses.postal_add, addresses.email_add

FROM addresses
WHERE addresses.email_add LIKE ? INTERSECT SELECT addresses.id,
addresses.st_id, addresses.postal_add, addresses.email_add
FROM addresses
WHERE addresses.postal_add LIKE ?

The two bound parameters ‘%gmail.com’ and ‘%Pune’ generate a single row from original
data in addresses table as shown below:

[(1, 1, 'Shivajinagar Pune', '[email protected]')]

39
 SQLAlchemy

SQLAlchemy ORM

40
 20. SQLAlchemy ORM – Declaring Mapping
SQLAlchemy

The main objective of the Object Relational Mapper API of SQLAlchemy is to facilitate
associating user-defined Python classes with database tables, and objects of those classes
with rows in their corresponding tables. Changes in states of objects and rows are
synchronously matched with each other. SQLAlchemy enables expressing database
queries in terms of user defined classes and their defined relationships.

The ORM is constructed on top of the SQL Expression Language. It is a high level and
abstracted pattern of usage. In fact, ORM is an applied usage of the Expression Language.

Although a successful application may be constructed using the Object Relational Mapper
exclusively, sometimes an application constructed with the ORM may use the Expression
Language directly where specific database interactions are required.

Declare Mapping
First of all, create_engine() function is called to set up an engine object which is
subsequently used to perform SQL operations. The function has two arguments, one is the
name of database and other is an echo parameter when set to True will generate the
activity log. If it doesn’t exist, the database will be created. In the following example, a
SQLite database is created.

from sqlalchemy import create_engine

engine = create_engine('sqlite:///sales.db', echo=True)

The Engine establishes a real DBAPI connection to the database when a method like
Engine.execute() or Engine.connect() is called. It is then used to emit the SQLORM which
does not use the Engine directly; instead, it is used behind the scenes by the ORM.

In case of ORM, the configurational process starts by describing the database tables and
then by defining classes which will be mapped to those tables. In SQLAlchemy, these two
tasks are performed together. This is done by using Declarative system; the classes
created include directives to describe the actual database table they are mapped to.

A base class stores a catlog of classes and mapped tables in the Declarative system. This
is called as the declarative base class. There will be usually just one instance of this base
in a commonly imported module. The declarative_base() function is used to create base
class. This function is defined in sqlalchemy.ext.declarative module.

from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()

Once base classis declared, any number of mapped classes can be defined in terms of it.
Following code defines a Customer’s class. It contains the table to be mapped to, and
names and datatypes of columns in it.

41
 SQLAlchemy

class Customers(Base):
__tablename__ = 'customers'

id = Column(Integer, primary_key=True)
name = Column(String)
address = Column(String)
email = Column(String)

A class in Declarative must have a __tablename__ attribute, and at least one Column
which is part of a primary key. Declarative replaces all the Column objects with special
Python accessors known as descriptors. This process is known as instrumentation which
provides the means to refer to the table in a SQL context and enables persisting and
loading the values of columns from the database.

This mapped class like a normal Python class has attributes and methods as per the
requirement.

The information about class in Declarative system, is called as table metadata.

SQLAlchemy uses Table object to represent this information for a specific table created by
Declarative. The Table object is created according to the specifications, and is associated
with the class by constructing a Mapper object. This mapper object is not directly used but
is used internally as interface between mapped class and table.

Each Table object is a member of larger collection known as MetaData and this object is
available using the .metadata attribute of declarative base class. The
MetaData.create_all() method is, passing in our Engine as a source of database
connectivity. For all tables that haven’t been created yet, it issues CREATE TABLE
statements to the database.

Base.metadata.create_all(engine)

The complete script to create a database and a table, and to map Python class is given
below:

from sqlalchemy import Column, Integer, String

from sqlalchemy import create_engine
engine = create_engine('sqlite:///sales.db', echo=True)
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Customers(Base):
__tablename__ = 'customers'
id = Column(Integer, primary_key=True)

name = Column(String)
address = Column(String)

42
 SQLAlchemy

email = Column(String)
Base.metadata.create_all(engine)

When executed, Python console will echo following SQL expression being executed:

CREATE TABLE customers (

id INTEGER NOT NULL,
name VARCHAR,
address VARCHAR,
email VARCHAR,
PRIMARY KEY (id)
)

If we open the Sales.db using SQLiteStudio graphic tool, it shows customers table inside
it with above mentioned structure.

43
 21. SQLAlchemy ORM – Creating Session
SQLAlchemy

In order to interact with the database, we need to obtain its handle. A session object is
the handle to database. Session class is defined using sessionmaker() – a configurable
session factory method which is bound to the engine object created earlier.

from sqlalchemy.orm import sessionmaker

Session = sessionmaker(bind=engine)

The session object is then set up using its default constructor as follows:

session = Session()

Some of the frequently required methods of session class are listed below:

begin() begins a transaction on this session

add() places an object in the session. Its state is
persisted in the database on next flush
operation
add_all() adds a collection of objects to the session
commit() flushes all items and any transaction in
progress
delete() marks a transaction as deleted
execute() executes a SQL expression
expire() marks attributes of an instance as out of
date
flush() flushes all object changes to the database
invalidate() closes the session using connection
invalidation
rollback() rolls back the current transaction in
progress
close() Closes current session by clearing all items
and ending any transaction in progress

44
 22. SQLAlchemy ORM – Adding Objects
SQLAlchemy

In the previous chapters of SQLAlchemy ORM, we have learnt how to declare mapping and
create sessions. In this chapter, we will learn how to add objects to the table.

We have declared Customer class that has been mapped to customers table. We have to
declare an object of this class and persistently add it to the table by add() method of
session object.

c1 = Sales(name='Ravi Kumar', address='Station Road Nanded',

email='[email protected]')
session.add(c1)

Note that this transaction is pending until the same is flushed using commit() method.

session.commit()

Following is the complete script to add a record in customers table:

from sqlalchemy import Column, Integer, String

from sqlalchemy import create_engine
engine = create_engine('sqlite:///sales.db', echo=True)
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Customers(Base):
__tablename__ = 'customers'

id = Column(Integer, primary_key=True)
name = Column(String)
address = Column(String)
email = Column(String)

from sqlalchemy.orm import sessionmaker

Session = sessionmaker(bind=engine)
session = Session()
c1 = Customers(name='Ravi Kumar', address='Station Road Nanded',
email='[email protected]')
session.add(c1)
session.commit()

45
 SQLAlchemy

To add multiple records, we can use add_all() method of the session class.

session.add_all([Customers(name='Komal Pande', address='Koti, Hyderabad',

email='[email protected]'), Customers(name='Rajender Nath', address='Sector 40,
Gurgaon', email='[email protected]'), Customers(name='S.M.Krishna',
address='Budhwar Peth, Pune', email='[email protected]')])
session.commit()

Table view of SQLiteStudio shows that the records are persistently added in customers
table. The following image shows the result:

46
 23. SQLAlchemy ORM – Using QuerySQLAlchemy

All SELECT statements generated by SQLAlchemy ORM are constructed by Query object.
It provides a generative interface, hence successive calls return a new Query object, a
copy of the former with additional criteria and options associated with it.

Query objects are initially generated using the query() method of the Session as follows:

q=session.query(mapped class)

Following statement is also equivalent to the above given statement:

q=Query(mappedClass, session)

The query object has all() method which returns a resultset in the form of list of objects.
If we execute it on our customers table:

result=session.query(Customers).all()

This statement is effectively equivalent to following SQL expression:

SELECT customers.id AS customers_id, customers.name AS customers_name,

customers.address AS customers_address, customers.email AS customers_email
FROM customers

The result object can be traversed using For loop as below to obtain all records in
underlying customers table. Here is the complete code to display all records in Customers
table:

from sqlalchemy import Column, Integer, String

from sqlalchemy import create_engine
engine = create_engine('sqlite:///sales.db', echo=True)
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Customers(Base):
__tablename__ = 'customers'
id = Column(Integer, primary_key=True)
name = Column(String)

address = Column(String)
email = Column(String)
from sqlalchemy.orm import sessionmaker

47
 SQLAlchemy

Session = sessionmaker(bind=engine)
session = Session()
result=session.query(Customers).all()
for row in result:
print ("Name: ",row.name, "Address:",row.address, "Email:",row.email)

Python console shows list of records as below:

Name: Ravi Kumar Address: Station Road Nanded Email: [email protected]

Name: Komal Pande Address: Koti, Hyderabad Email: [email protected]
Name: Rajender Nath Address: Sector 40, Gurgaon Email: [email protected]
Name: S.M.Krishna Address: Budhwar Peth, Pune Email: [email protected]

The Query object also has following useful methods:

add_columns() It adds one or more column expressions to the list of result

columns to be returned.
add_entity() It adds a mapped entity to the list of result columns to be
returned.
count() It returns a count of rows this Query would return.
delete() It performs a bulk delete query. Deletes rows matched by
this query from the database.
distinct() It applies a DISTINCT clause to the query and return the
newly resulting Query.
filter() It applies the given filtering criterion to a copy of this
Query, using SQL expressions.
first() It returns the first result of this Query or None if the result
doesn’t contain any row.
get() It returns an instance based on the given primary key
identifier providing direct access to the identity map of the
owning Session.
group_by() It applies one or more GROUP BY criterion to the query and
return the newly resulting Query
join() It creates a SQL JOIN against this Query object’s criterion
and apply generatively, returning the newly resulting
Query.
one() It returns exactly one result or raise an exception.
order_by() It applies one or more ORDER BY criterion to the query and
returns the newly resulting Query.
update() It performs a bulk update query and updates rows matched
by this query in the database.

48
 24. SQLAlchemy ORM – Updating Objects
SQLAlchemy

In this chapter, we will see how to modify or update the table with desired values.

To modify data of a certain attribute of any object, we have to assign new value to it and
commit the changes to make the change persistent.

Let us fetch an object from the table whose primary key identifier, in our Customers table
with ID=2. We can use get() method of session as follows:

x=session.query(Customers).get(2)

We can display contents of the selected object with the below given code:

print ("Name: ",x.name, "Address:",x.address, "Email:",x.email)

From our customers table, following output should be displayed:

Name: Komal Pande Address: Koti, Hyderabad Email: [email protected]

Now we need to update the Address field by assigning new value as given below:

x.address='Banjara Hills Secunderabad'

session.commit()

The change will be persistently reflected in the database. Now we fetch object
corresponding to first row in the table by using first() method as follows:

x=session.query(Customers).first()

This will execute following SQL expression:

SELECT customers.id AS customers_id, customers.name AS customers_name,

customers.address AS customers_address, customers.email AS customers_email
FROM customers
LIMIT ? OFFSET ?

The bound parameters will be LIMIT=1 and OFFSET=0 respectively which means first row
will be selected.

print ("Name: ",x.name, "Address:",x.address, "Email:",x.email)

49
 SQLAlchemy

Now, the output for the above code displaying the first row is as follows:

Name: Ravi Kumar Address: Station Road Nanded Email: [email protected]

Now change name attribute and display the contents using the below code:

x.name='Ravi Shrivastava'
print ("Name: ",x.name, "Address:",x.address, "Email:",x.email)

The output of the above code is:

Name: Ravi Shrivastava Address: Station Road Nanded Email: [email protected]

Even though the change is displayed, it is not committed. You can retain the earlier
persistent position by using rollback() method with the code below.

session.rollback()
print ("Name: ",x.name, "Address:",x.address, "Email:",x.email)

Original contents of first record will be displayed.

For bulk updates, we shall use update() method of the Query object. Let us try and give a
prefix, ‘Mr.’ to name in each row (except ID=2). The corresponding update() statement is
as follows:

session.query(Customers).filter(Customers.id!=2).update({Customers.name:"Mr."+C
ustomers.name}, synchronize_session=False)

The update() method requires two parameters as follows:

 A dictionary of key-values with key being the attribute to be updated, and value
being the new contents of attribute.

 synchronize_session attribute mentioning the strategy to update attributes in the

session. Valid values are false: for not synchronizing the session, fetch: performs
a select query before the update to find objects that are matched by the update
query; and evaluate: evaluate criteria on objects in the session.

Three out of 4 rows in the table will have name prefixed with ‘Mr.’ However, the changes
are not committed and hence will not be reflected in the table view of SQLiteStudio. It will
be refreshed only when we commit the session.

50
 25. SQLAlchemy ORM – Applying Filter
SQLAlchemy

In this chapter, we will discuss how to apply filter and also certain filter operations along with
their codes.

Resultset represented by Query object can be subjected to certain criteria by using filter()
method. The general usage of filter method is as follows:

session.query(class).filter(criteria)

In the following example, resultset obtained by SELECT query on Customers table is

filtered by a condition, (ID>2):

result=session.query(Customers).filter(Customers.id>2)

This statement will translate into following SQL expression:

SELECT customers.id AS customers_id, customers.name AS customers_name,

customers.address AS customers_address, customers.email AS customers_email
FROM customers
WHERE customers.id > ?

Since the bound parameter (?) is given as 2, only those rows with ID column>2 will be
displayed. The complete code is given below:

from sqlalchemy import Column, Integer, String

from sqlalchemy import create_engine
engine = create_engine('sqlite:///sales.db', echo=True)
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Customers(Base):
__tablename__ = 'customers'

id = Column(Integer, primary_key=True)
name = Column(String)

address = Column(String)
email = Column(String)
from sqlalchemy.orm import sessionmaker
Session = sessionmaker(bind=engine)

51
 SQLAlchemy

session = Session()
result=session.query(Customers).filter(Customers.id>2)
for row in result:
print ("ID:", row.id, "Name: ",row.name, "Address:",row.address,
"Email:",row.email)

The output displayed in the Python console is as follows:

ID: 3 Name: Rajender Nath Address: Sector 40, Gurgaon Email: [email protected]
ID: 4 Name: S.M.Krishna Address: Budhwar Peth, Pune Email: [email protected]

52
 26. SQLAlchemy ORM – Filter Operators
SQLAlchemy

Now, we will learn the filter operations with their respective codes and output.

Equals
The usual operator used is == and it applies the criteria to check equality.

result=session.query(Customers).filter(Customers.id==2)
for row in result:
print ("ID:", row.id, "Name: ",row.name, "Address:",row.address,
"Email:",row.email)

SQLAlchemy will send following SQL expression:

SELECT customers.id AS customers_id, customers.name AS customers_name,

customers.address AS customers_address, customers.email AS customers_email
FROM customers
WHERE customers.id = ?

The output for the above code is as follows:

ID: 2 Name: Komal Pande Address: Banjara Hills Secunderabad Email:

[email protected]

Not Equals
The operator used for not equals is != and it provides not equals criteria.

result=session.query(Customers).filter(Customers.id!=2)
for row in result:
print ("ID:", row.id, "Name: ",row.name, "Address:",row.address,
"Email:",row.email)

The resulting SQL expression is:

SELECT customers.id AS customers_id, customers.name AS customers_name,

customers.address AS customers_address, customers.email AS customers_email
FROM customers
WHERE customers.id != ?

The output for the above lines of code is as follows:

ID: 1 Name: Ravi Kumar Address: Station Road Nanded Email: [email protected]

53
 SQLAlchemy

ID: 3 Name: Rajender Nath Address: Sector 40, Gurgaon Email: [email protected]
ID: 4 Name: S.M.Krishna Address: Budhwar Peth, Pune Email: [email protected]

Like
like() method itself produces the LIKE criteria for WHERE clause in the SELECT expression.

result=session.query(Customers).filter(Customers.name.like('Ra%'))
for row in result:
print ("ID:", row.id, "Name: ",row.name, "Address:",row.address,
"Email:",row.email)

Above SQLAlchemy code is equivalent to following SQL expression:

SELECT customers.id AS customers_id, customers.name AS customers_name,

customers.address AS customers_address, customers.email AS customers_email
FROM customers
WHERE customers.name LIKE ?

And the output for the above code is:

ID: 1 Name: Ravi Kumar Address: Station Road Nanded Email: [email protected]
ID: 3 Name: Rajender Nath Address: Sector 40, Gurgaon Email: [email protected]

IN
This operator checks whether the column value belongs to a collection of items in a list.
It is provided by in_() method.

result=session.query(Customers).filter(Customers.id.in_([1,3]))
for row in result:
print ("ID:", row.id, "Name: ",row.name, "Address:",row.address,
"Email:",row.email)

Here, the SQL expression evaluated by SQLite engine will be as follows:

SELECT customers.id AS customers_id, customers.name AS customers_name,

customers.address AS customers_address, customers.email AS customers_email
FROM customers
WHERE customers.id IN (?, ?)

The output for the above code is as follows:

ID: 1 Name: Ravi Kumar Address: Station Road Nanded Email: [email protected]
ID: 3 Name: Rajender Nath Address: Sector 40, Gurgaon Email: [email protected]

54
 SQLAlchemy

AND
This conjunction is generated by either putting multiple commas separated criteria
in the filter or using and_() method as given below:

result=session.query(Customers).filter(Customers.id>2,
Customers.name.like('Ra%'))
for row in result:
print ("ID:", row.id, "Name: ",row.name, "Address:",row.address,
"Email:",row.email)

from sqlalchemy import and_

result=session.query(Customers).filter(and_(Customers.id>2,
Customers.name.like('Ra%')))
for row in result:
print ("ID:", row.id, "Name: ",row.name, "Address:",row.address,
"Email:",row.email)

Both the above approaches result in similar SQL expression:

SELECT customers.id AS customers_id, customers.name AS customers_name,

customers.address AS customers_address, customers.email AS customers_email
FROM customers
WHERE customers.id > ? AND customers.name LIKE ?

The output for the above lines of code is:

ID: 3 Name: Rajender Nath Address: Sector 40, Gurgaon Email: [email protected]

OR
This conjunction is implemented by or_() method.

from sqlalchemy import or_

result=session.query(Customers).filter(or_(Customers.id>2,
Customers.name.like('Ra%')))
for row in result:
print ("ID:", row.id, "Name: ",row.name, "Address:",row.address,
"Email:",row.email)

As a result, SQLite engine gets following equivalent SQL expression:

SELECT customers.id AS customers_id, customers.name AS customers_name,

customers.address AS customers_address, customers.email AS customers_email
FROM customers

55
 SQLAlchemy

WHERE customers.id > ? OR customers.name LIKE ?

The output for the above code is as follows:

ID: 1 Name: Ravi Kumar Address: Station Road Nanded Email: [email protected]
ID: 3 Name: Rajender Nath Address: Sector 40, Gurgaon Email: [email protected]
ID: 4 Name: S.M.Krishna Address: Budhwar Peth, Pune Email: [email protected]

56
27. SQLAlchemy ORM – Returning List andSQLAlchemy
Scalars

There are a number of methods of Query object that immediately issue SQL and return a
value containing loaded database results.

Here’s a brief rundown of returning list and scalars:

all()
It returns a list. Given below is the line of code for all() function.

session.query(Customers).all()

Python console displays following SQL expression emitted:

SELECT customers.id AS customers_id, customers.name AS customers_name,

customers.address AS customers_address, customers.email AS customers_email
FROM customers

first()
It applies a limit of one and returns the first result as a scalar.

SELECT customers.id AS customers_id, customers.name AS customers_name,

customers.address AS customers_address, customers.email AS customers_email
FROM customers
LIMIT ? OFFSET ?

The bound parameters for LIMIT is 1 and for OFFSET is 0.

one()
This command fully fetches all rows, and if there is not exactly one object identity or
composite row present in the result, it raises an error.

session.query(Customers).one()

With multiple rows found:

MultipleResultsFound: Multiple rows were found for one()

With no rows found:

NoResultFound: No row was found for one()

The one() method is useful for systems that expect to handle “no items found” versus
“multiple items found” differently.

57
 SQLAlchemy

scalar()
It invokes the one() method, and upon success returns the first column of the row as
follows:

session.query(Customers).filter(Customers.id==3).scalar()

This generates following SQL statement:

SELECT customers.id AS customers_id, customers.name AS customers_name,

customers.address AS customers_address, customers.email AS customers_email
FROM customers
WHERE customers.id = ?

58
 28. SQLAlchemy ORM – Textual SQLSQLAlchemy

Earlier, textual SQL using text() function has been explained from the perspective of core
expression language of SQLAlchemy. Now we shall discuss it from ORM point of view.

Literal strings can be used flexibly with Query object by specifying their use with the text()
construct. Most applicable methods accept it. For example, filter() and order_by().

In the example given below, the filter() method translates the string “id<3” to the WHERE
id<3

from sqlalchemy import text

for cust in session.query(Customers).filter(text("id<3")):
print(cust.name)

The raw SQL expression generated shows conversion of filter to WHERE clause with the
code illustrated below:

SELECT customers.id AS customers_id, customers.name AS customers_name,

customers.address AS customers_address, customers.email AS customers_email
FROM customers
WHERE id<3

From our sample data in Customers table, two rows will be selected and name column will
be printed as follows:

Ravi Kumar
Komal Pande

To specify bind parameters with string-based SQL, use a colon,and to specify the values,
use the params() method.

cust=session.query(Customers).filter(text("id=:value")).params(value=1).one()

The effective SQL displayed on Python console will be as given below:

SELECT customers.id AS customers_id, customers.name AS customers_name,

customers.address AS customers_address, customers.email AS customers_email
FROM customers
WHERE id=?

To use an entirely string-based statement, a text() construct representing a complete

statement can be passed to from_statement().

session.query(Customers).from_statement(text("SELECT * FROM customers")).all()

59
 SQLAlchemy

The result of above code will be a basic SELECT statement as given below:

SELECT * FROM customers

Obviously, all records in customers table will be selected.

The text() construct allows us to link its textual SQL to Core or ORM-mapped column
expressions positionally. We can achieve this by passing column expressions as positional
arguments to the TextClause.columns() method.

stmt = text("SELECT name, id, name, address, email FROM customers")

stmt=stmt.columns(Customers.id, Customers.name)
session.query(Customers.id, Customers.name).from_statement(stmt).all()

The id and name columns of all rows will be selected even though the SQLite engine
executes following expression generated by above code shows all columns in text()
method:

SELECT name, id, name, address, email FROM customers

60
 29. SQLAlchemy ORM – Building Relationship
SQLAlchemy

This session describes creation of another table which is related to already existing one in
our database. The customers table contains master data of customers. We now need to
create invoices table which may have any number of invoices belonging to a customer.
This is a case of one to many relationships.

Using declarative, we define this table along with its mapped class, Invoices as given
below:

from sqlalchemy import create_engine, ForeignKey, Column, Integer, String

engine = create_engine('sqlite:///sales.db', echo=True)
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
from sqlalchemy.orm import relationship
class Customer(Base):
__tablename__ = 'customers'

id = Column(Integer, primary_key=True)
name = Column(String)
address = Column(String)
email = Column(String)

class Invoice(Base):
__tablename__ = 'invoices'

id=Column(Integer, primary_key=True)
custid=Column(Integer, ForeignKey('customers.id'))
invno=Column(Integer)
amount=Column(Integer)
customer = relationship("Customer", back_populates="invoices")

Customer.invoices = relationship("Invoice", order_by=Invoice.id,

back_populates="customer")
Base.metadata.create_all(engine)

61
 SQLAlchemy

This will send a CREATE TABLE query to SQLite engine as below:

CREATE TABLE invoices (

id INTEGER NOT NULL,
custid INTEGER,
invno INTEGER,
amount INTEGER,
PRIMARY KEY (id),
FOREIGN KEY(custid) REFERENCES customers (id)
)

We can check that new table is created in sales.db with the help of SQLiteStudio tool.

Invoices class applies ForeignKey construct on custid attribute. This directive indicates that
values in this column should be constrained to be values present in id column in customers
table. This is a core feature of relational databases, and is the “glue” that transforms
unconnected collection of tables to have rich overlapping relationships.

A second directive, known as relationship(), tells the ORM that the Invoice class should be
linked to the Customer class using the attribute Invoice.customer. The relationship() uses
the foreign key relationships between the two tables to determine the nature of this
linkage, determining that it is many to one.

An additional relationship() directive is placed on the Customer mapped class under the
attribute Customer.invoices. The parameter relationship.back_populates is assigned to
refer to the complementary attribute names, so that each relationship() can make
intelligent decision about the same relationship as expressed in reverse. On one side,
Invoices.customer refers to Invoices instance, and on the other side, Customer.invoices
refers to a list of Customers instances.

62
 SQLAlchemy

The relationship function is a part of Relationship API of SQLAlchemy ORM package. It

provides a relationship between two mapped classes. This corresponds to a parent-child
or associative table relationship.

Following are the basic Relationship Patterns found:

One To Many: A One to Many relationship refers to parent with the help of a foreign key
on the child table. relationship() is then specified on the parent, as referencing a collection
of items represented by the child. The relationship.back_populates parameter is used to
establish a bidirectional relationship in one-to-many, where the “reverse” side is a many
to one.

Many To One: On the other hand, Many to One relationship places a foreign key in the
parent table to refer to the child. relationship() is declared on the parent, where a new
scalar-holding attribute will be created. Here again the relationship.back_populates
parameter is used for Bidirectionalbehaviour.

One To One: One To One relationship is essentially a bidirectional relationship in nature.

The uselist flag indicates the placement of a scalar attribute instead of a collection on the
“many” side of the relationship. To convert one-to-many into one-to-one type of relation,
set uselist parameter to false.

Many To Many: Many to Many relationship is established by adding an association table

related to two classes by defining attributes with their foreign keys. It is indicated by the
secondary argument to relationship(). Usually, the Table uses the MetaData object
associated with the declarative base class, so that the ForeignKey directives can locate the
remote tables with which to link. The relationship.back_populates parameter for each
relationship() establishes a bidirectional relationship. Both sides of the relationship contain
a collection.

63
30. SQLAlchemy ORM – Working with Related Objects
SQLAlchemy

In this chapter, we will focus on the related objects in SQLAlchemy ORM.

Now when we create a Customer object, a blank invoice collection will be present in the
form of Python List.

c1=Customer(name="Gopal Krishna", address="Bank Street Hydarebad",

email="[email protected]")

The invoices attribute of c1.invoices will be an empty list. We can assign items in the list
as:

c1.invoices=[Invoice(invno=10, amount=15000), Invoice(invno=14, amount=3850)]

Let us commit this object to the database using Session object as follows:

from sqlalchemy.orm import sessionmaker

Session = sessionmaker(bind=engine)
session = Session()
session.add(c1)
session.commit()

This will automatically generate INSERT queries for customers and invoices tables:

INSERT INTO customers (name, address, email) VALUES (?, ?, ?)

('Gopal Krishna', 'Bank Street Hydarebad', '[email protected]')
INSERT INTO invoices (custid, invno, amount) VALUES (?, ?, ?)
(2, 10, 15000)
INSERT INTO invoices (custid, invno, amount) VALUES (?, ?, ?)
(2, 14, 3850)

Let us now look at contents of customers table and invoices table in the table view of
SQLiteStudio:

64
 SQLAlchemy

You can construct Customer object by providing mapped attribute of invoices in the
constructor itself by using the below command:

65
 SQLAlchemy

c2=[Customer(name="Govind Pant", address="Gulmandi Aurangabad",

email="[email protected]",
invoices=[Invoice(invno=3, amount=10000), Invoice(invno=4,
amount=5000)])]

Or a list of objects to be added using add_all() function of session object as shown below:

rows=[Customer(name="Govind Kala", address="Gulmandi Aurangabad",

email="[email protected]",
invoices=[Invoice(invno=7, amount=12000), Invoice(invno=8,
amount=18500)]),
Customer(name="Abdul Rahman", address="Rohtak", email="[email protected]",
invoices=[Invoice(invno=9, amount=15000), Invoice(invno=11,
amount=6000)])
]
session.add_all(rows)
session.commit()

66
 31. SQLAlchemy ORM – Working with Joins
SQLAlchemy

Now that we have two tables, we will see how to create queries on both tables at the same
time. To construct a simple implicit join between Customer and Invoice, we can use
Query.filter() to equate their related columns together. Below, we load the Customer and
Invoice entities at once using this method:

from sqlalchemy.orm import sessionmaker

Session = sessionmaker(bind=engine)
session = Session()
for c, i in session.query(Customer,
Invoice).filter(Customer.id==Invoice.custid).all():
print ("ID: {} Name: {} Invoice No: {} Amount: {}".format(c.id,c.name,
i.invno, i.amount))

The SQL expression emitted by SQLAlchemy is as follows:

SELECT customers.id AS customers_id, customers.name AS customers_name,

customers.address AS customers_address, customers.email AS customers_email,
invoices.id AS invoices_id, invoices.custid AS invoices_custid, invoices.invno
AS invoices_invno, invoices.amount AS invoices_amount
FROM customers, invoices
WHERE customers.id = invoices.custid

And the result of the above lines of code is as follows:

ID: 2 Name: Gopal Krishna Invoice No: 10 Amount: 15000

ID: 2 Name: Gopal Krishna Invoice No: 14 Amount: 3850
ID: 3 Name: Govind Pant Invoice No: 3 Amount: 10000
ID: 3 Name: Govind Pant Invoice No: 4 Amount: 5000
ID: 4 Name: Govind Kala Invoice No: 7 Amount: 12000
ID: 4 Name: Govind Kala Invoice No: 8 Amount: 8500
ID: 5 Name: Abdul Rahman Invoice No: 9 Amount: 15000
ID: 5 Name: Abdul Rahman Invoice No: 11 Amount: 6000

The actual SQL JOIN syntax is easily achieved using the Query.join() method as follows:

session.query(Customer).join(Invoice).filter(Invoice.amount==8500).all()

The SQL expression for join will be displayed on the console:

SELECT customers.id AS customers_id, customers.name AS customers_name,

customers.address AS customers_address, customers.email AS customers_email

67
 SQLAlchemy

FROM customers JOIN invoices ON customers.id = invoices.custid

WHERE invoices.amount = ?

We can iterate through the result using for loop:

result=session.query(Customer).join(Invoice).filter(Invoice.amount==8500)
for row in result:
for inv in row.invoices:
print (row.id, row.name, inv.invno, inv.amount)

With 8500 as the bind parameter, following output is displayed:

4 Govind Kala 8 8500

Query.join() knows how to join between these tables because there’s only one foreign key
between them. If there were no foreign keys, or more foreign keys, Query.join() works
better when one of the following forms are used:

query.join(Invoice, id==Address.custid) explicit condition

query.join(Customer.invoices) specify relationship from left to right
query.join(Invoice, Customer.invoices) same, with explicit target
query.join('invoices') same, using a string

Similarly outerjoin() function is available to achieve left outer join.

query.outerjoin(Customer.invoices)

The subquery() method produces a SQL expression representing SELECT statement

embedded within an alias.

from sqlalchemy.sql import func

stmt = session.query(Invoice.custid,
func.count('*').label('invoice_count')).group_by(Invoice.custid).subquery()

The stmt object will contain a SQL statement as below:

SELECT invoices.custid, count(:count_1) AS invoice_count FROM invoices GROUP BY

invoices.custid

Once we have our statement, it behaves like a Table construct. The columns on the
statement are accessible through an attribute called c as shown in the below code:

for u, count in session.query(Customer, stmt.c.invoice_count).outerjoin(stmt,

Customer.id==stmt.c.custid).order_by(Customer.id):
print(u.name, count)

The above for loop displays name-wise count of invoices as follows:

Arjun Pandit None

68
 SQLAlchemy

Gopal Krishna 2
Govind Pant 2
Govind Kala 2
Abdul Rahman 2

69
 32. SQLAlchemy ORM – Common Relationship
SQLAlchemy

Operators

In this chapter, we will discuss about the operators which build on relationships.

__eq__()
The above operator is a many-to-one “equals” comparison. The line of code for this
operator is as shown below:

s=session.query(Customer).filter(Invoice.invno.__eq__(12))

The equivalent SQL query for the above line of code is:

SELECT customers.id AS customers_id, customers.name AS customers_name,

customers.address AS customers_address, customers.email AS customers_email
FROM customers, invoices
WHERE invoices.invno = ?

__ne__()
This operator is a many-to-one “not equals” comparison. The line of code for this operator
is as shown below:

s=session.query(Customer).filter(Invoice.custid.__ne__(2))

The equivalent SQL query for the above line of code is given below:

SELECT customers.id AS customers_id, customers.name AS customers_name,

customers.address AS customers_address, customers.email AS customers_email
FROM customers, invoices
WHERE invoices.custid != ?

contains()
This operator is used for one-to-many collections and given below is the code for
contains():

s=session.query(Invoice).filter(Invoice.invno.contains([3,4,5]))

The equivalent SQL query for the above line of code is:

SELECT invoices.id AS invoices_id, invoices.custid AS invoices_custid,

invoices.invno AS invoices_invno, invoices.amount AS invoices_amount
FROM invoices

70
 SQLAlchemy

WHERE (invoices.invno LIKE '%' + ? || '%')

any()
any() operator is used for collections as shown below:

s=session.query(Customer).filter(Customer.invoices.any(Invoice.invno==11))

The equivalent SQL query for the above line of code is shown below:

SELECT customers.id AS customers_id, customers.name AS customers_name,

customers.address AS customers_address, customers.email AS customers_email
FROM customers
WHERE EXISTS (SELECT 1
FROM invoices
WHERE customers.id = invoices.custid AND invoices.invno = ?)

has()
This operator is used for scalar references as follows:

s=session.query(Invoice).filter(Invoice.customer.has(name='Arjun Pandit'))

The equivalent SQL query for the above line of code is:

SELECT invoices.id AS invoices_id, invoices.custid AS invoices_custid,

invoices.invno AS invoices_invno, invoices.amount AS invoices_amount
FROM invoices
WHERE EXISTS (SELECT 1
FROM customers
WHERE customers.id = invoices.custid AND customers.name = ?)

71
 33. SQLAlchemy ORM – Eager Loading
SQLAlchemy

Eager load reduces the number of queries. SQLAlchemy offers eager loading functions
invoked via query options which give additional instructions to the Query. These options
determine how to load various attributes via the Query.options() method.

Subquery Load
We want that Customer.invoices should load eagerly. The orm.subqueryload() option gives
a second SELECT statement that fully loads the collections associated with the results just
loaded. The name “subquery” causes the SELECT statement to be constructed directly via
the Query re-used and embedded as a subquery into a SELECT against the related table.

from sqlalchemy.orm import subqueryload

c1 =
session.query(Customer).options(subqueryload(Customer.invoices)).filter_by(name
='Govind Pant').one()

This results in the following two SQL expressions:

SELECT customers.id AS customers_id, customers.name AS customers_name,

customers.address AS customers_address, customers.email AS customers_email
FROM customers
WHERE customers.name = ?
('Govind Pant',)

SELECT invoices.id AS invoices_id, invoices.custid AS invoices_custid,

invoices.invno AS invoices_invno, invoices.amount AS invoices_amount,
anon_1.customers_id AS anon_1_customers_id
FROM (SELECT customers.id AS customers_id
FROM customers
WHERE customers.name = ?) AS anon_1 JOIN invoices ON anon_1.customers_id =
invoices.custid ORDER BY anon_1.customers_id, invoices.id
2018-06-25 18:24:47,479 INFO sqlalchemy.engine.base.Engine ('Govind Pant',)

To access the data from two tables, we can use the below program:

print (c1.name, c1.address, c1.email)

for x in c1.invoices:
print ("Invoice no : {}, Amount : {}".format(x.invno, x.amount))

72
 SQLAlchemy

The output of the above program is as follows:

Govind Pant Gulmandi Aurangabad [email protected]

Invoice no : 3, Amount : 10000
Invoice no : 4, Amount : 5000

Joined Load
The other function is called orm.joinedload(). This emits a LEFT OUTER JOIN. Lead object
as well as the related object or collection is loaded in one step.

from sqlalchemy.orm import joinedload

c1 =
session.query(Customer).options(joinedload(Customer.invoices)).filter_by(name='
Govind Pant').one()

This emits following expression giving same output as above:

SELECT customers.id AS customers_id, customers.name AS customers_name,

customers.address AS customers_address, customers.email AS customers_email,
invoices_1.id AS invoices_1_id, invoices_1.custid AS invoices_1_custid,
invoices_1.invno AS invoices_1_invno, invoices_1.amount AS invoices_1_amount
FROM customers LEFT OUTER JOIN invoices AS invoices_1 ON customers.id =
invoices_1.custid
WHERE customers.name = ? ORDER BY invoices_1.id
('Govind Pant',)

The OUTER JOIN resulted in two rows, but it gives one instance of Customer back. This is
because Query applies a “uniquing” strategy, based on object identity, to the returned
entities. Joined eager loading can be applied without affecting the query results.

The subqueryload() is more appropriate for loading related collections while joinedload()
is better suited for many-to-one relationship.

73
34. SQLAlchemy ORM – Deleting Related Objects
SQLAlchemy

It is easy to perform delete operation on a single table. All you have to do is to delete an
object of the mapped class from a session and commit the action. However, delete
operation on multiple related tables is little tricky.

In our sales.db database, Customer and Invoice classes are mapped to customer and
invoice table with one to many type of relationship. We will try to delete Customer object
and see the result.

As a quick reference, below are the definitions of Customer and Invoice classes:

from sqlalchemy import create_engine, ForeignKey, Column, Integer, String

engine = create_engine('sqlite:///sales.db', echo=True)
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
from sqlalchemy.orm import relationship
class Customer(Base):
__tablename__ = 'customers'

id = Column(Integer, primary_key=True)
name = Column(String)
address = Column(String)
email = Column(String)

class Invoice(Base):
__tablename__ = 'invoices'

id=Column(Integer, primary_key=True)
custid=Column(Integer, ForeignKey('customers.id'))
invno=Column(Integer)
amount=Column(Integer)
customer = relationship("Customer", back_populates="invoices")

Customer.invoices = relationship("Invoice", order_by=Invoice.id,

back_populates="customer")

74
 SQLAlchemy

We setup a session and obtain a Customer object by querying it with primary ID using the
below program:

from sqlalchemy.orm import sessionmaker

Session = sessionmaker(bind=engine)
session = Session()
x=session.query(Customer).get(2)

In our sample table, x.name happens to be 'Gopal Krishna'. Let us delete this x from the
session and count the occurrence of this name.

session.delete(x)
session.query(Customer).filter_by(name='Gopal Krishna').count()

The resulting SQL expression will return 0.

SELECT count(*) AS count_1

FROM (SELECT customers.id AS customers_id, customers.name AS customers_name,
customers.address AS customers_address, customers.email AS customers_email
FROM customers
WHERE customers.name = ?) AS anon_1
('Gopal Krishna',)
0

However, the related Invoice objects of x are still there. It can be verified by the following
code:

session.query(Invoice).filter(Invoice.invno.in_([10,14])).count()

Here, 10 and 14 are invoice numbers belonging to customer Gopal Krishna. Result of the
above query is 2, which means the related objects have not been deleted.

SELECT count(*) AS count_1

FROM (SELECT invoices.id AS invoices_id, invoices.custid AS invoices_custid,
invoices.invno AS invoices_invno, invoices.amount AS invoices_amount
FROM invoices
WHERE invoices.invno IN (?, ?)) AS anon_1
(10, 14)
2

This is because SQLAlchemy doesn’t assume the deletion of cascade; we have to give a
command to delete it.

To change the behavior, we configure cascade options on the User.addresses relationship.

Let us close the ongoing session, use new declarative_base() and redeclare the User class,
adding in the addresses relationship including the cascade configuration.

75
 SQLAlchemy

The cascade attribute in relationship function is a comma-separated list of cascade rules

which determines how Session operations should be “cascaded” from parent to child. By
default, it is False, which means that it is "save-update, merge".

The available cascades are as follows:

 save-update
 merge
 expunge
 delete
 delete-orphan
 refresh-expire

Often used option is "all, delete-orphan" to indicate that related objects should follow along
with the parent object in all cases, and be deleted when de-associated.

Hence redeclared Customer class is shown below:

class Customer(Base):
__tablename__ = 'customers'

id = Column(Integer, primary_key=True)
name = Column(String)
address = Column(String)
email = Column(String)
invoices = relationship("Invoice", order_by=Invoice.id,
back_populates="customer",cascade="all, delete, delete-orphan" )

Let us delete the Customer with Gopal Krishna name using the below program and see the
count of its related Invoice objects:

from sqlalchemy.orm import sessionmaker

Session = sessionmaker(bind=engine)
session = Session()
x=session.query(Customer).get(2)
session.delete(x)
session.query(Customer).filter_by(name='Gopal Krishna').count()
session.query(Invoice).filter(Invoice.invno.in_([10,14])).count()

The count is now 0 with following SQL emitted by above script:

SELECT customers.id AS customers_id, customers.name AS customers_name,

customers.address AS customers_address, customers.email AS customers_email
FROM customers
WHERE customers.id = ?

76
 SQLAlchemy

(2,)
SELECT invoices.id AS invoices_id, invoices.custid AS invoices_custid,
invoices.invno AS invoices_invno, invoices.amount AS invoices_amount
FROM invoices
WHERE ? = invoices.custid ORDER BY invoices.id
(2,)
DELETE FROM invoices WHERE invoices.id = ?
((1,), (2,))
DELETE FROM customers WHERE customers.id = ?
(2,)
SELECT count(*) AS count_1
FROM (SELECT customers.id AS customers_id, customers.name AS customers_name,
customers.address AS customers_address, customers.email AS customers_email
FROM customers
WHERE customers.name = ?) AS anon_1
('Gopal Krishna',)
SELECT count(*) AS count_1
FROM (SELECT invoices.id AS invoices_id, invoices.custid AS invoices_custid,
invoices.invno AS invoices_invno, invoices.amount AS invoices_amount
FROM invoices
WHERE invoices.invno IN (?, ?)) AS anon_1
(10, 14)
0

77
35. SQLAlchemy ORM – Many to Many Relationships
SQLAlchemy

Many to Many relationship between two tables is achieved by adding an association

table such that it has two foreign keys — one from each table’s primary key. Moreover,
classes mapping to the two tables have an attribute with a collection of objects of other
association tables assigned as secondary attribute of relationship() function.

For this purpose, we shall create a SQLite database (mycollege.db) with two tables —
department and employee. Here, we assume that an employee is a part of more than one
department, and a department has more than one employee. This constitutes many-to-
many relationship.

Definition of Employee and Department classes mapped to department and employee table
is as follows:

from sqlalchemy import create_engine, ForeignKey, Column, Integer, String

engine = create_engine('sqlite:///mycollege.db', echo=True)
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
from sqlalchemy.orm import relationship
class Department(Base):
__tablename__ = 'department'
id = Column(Integer, primary_key=True)
name = Column(String)
employees = relationship('Employee', secondary='link')

class Employee(Base):
__tablename__ = 'employee'
id = Column(Integer, primary_key=True)
name = Column(String)
departments = relationship(Department,secondary='link')

We now define a Link class. It is linked to link table and contains department_id and
employee_id attributes respectively referencing to primary keys of department and
employee table.

class Link(Base):
__tablename__ = 'link'

department_id = Column(Integer, ForeignKey('department.id'),

primary_key=True)

78
 SQLAlchemy

employee_id = Column(Integer, ForeignKey('employee.id'), primary_key=True)

Here, we have to make a note that Department class has employees attribute related to
Employee class. The relationship function’s secondary attribute is assigned a link as its
value.

Similarly, Employee class has departments attribute related to Department class. The
relationship function’s secondary attribute is assigned a link as its value.

All these three tables are created when the following statement is executed:

Base.metadata.create_all(engine)

The Python console emits following CREATE TABLE queries:

CREATE TABLE department (

id INTEGER NOT NULL,
name VARCHAR,
PRIMARY KEY (id)
)

CREATE TABLE employee (

id INTEGER NOT NULL,
name VARCHAR,
PRIMARY KEY (id)
)

CREATE TABLE link (

department_id INTEGER NOT NULL,
employee_id INTEGER NOT NULL,
PRIMARY KEY (department_id, employee_id),
FOREIGN KEY(department_id) REFERENCES department (id),
FOREIGN KEY(employee_id) REFERENCES employee (id)
)

We can check this by opening mycollege.db using SQLiteStudio as shown in the

screenshots given below:

79
SQLAlchemy

80
 SQLAlchemy

Next we create three objects of Department class and three objects of Employee class as
shown below:

d1=Department(name="Accounts")
d2=Department(name="Sales")
d3=Department(name="Marketing")

e1=Employee(name="John")
e2=Employee(name="Tony")
e3=Employee(name="Graham")

Each table has a collection attribute having append() method. We can add Employee
objects to Employees collection of Department object. Similarly, we can add Department
objects to departments collection attribute of Employee objects.

e1.departments.append(d1)
e2.departments.append(d3)
d1.employees.append(e3)
d2.employees.append(e2)
d3.employees.append(e1)
e3.departments.append(d2)

81
 SQLAlchemy

All we have to do now is to set up a session object, add all objects to it and commit the
changes as shown below:

from sqlalchemy.orm import sessionmaker

Session = sessionmaker(bind=engine)
session = Session()
session.add(e1)
session.add(e2)
session.add(d1)
session.add(d2)
session.add(d3)
session.add(e3)
session.commit()

Following SQL statements will be emitted on Python console:

INSERT INTO department (name) VALUES (?)

('Accounts',)
INSERT INTO department (name) VALUES (?)
('Sales',)
INSERT INTO department (name) VALUES (?)
('Marketing',)
INSERT INTO employee (name) VALUES (?)
('John',)
INSERT INTO employee (name) VALUES (?)
('Graham',)
INSERT INTO employee (name) VALUES (?)
('Tony',)
INSERT INTO link (department_id, employee_id) VALUES (?, ?)
((1, 2), (3, 1), (2, 3))
INSERT INTO link (department_id, employee_id) VALUES (?, ?)
((1, 1), (2, 2), (3, 3))

82
 SQLAlchemy

To check the effect of above operations, use SQLiteStudio and view data in department,
employee and link tables:

83
 SQLAlchemy

To display the data, run the following query statement:

from sqlalchemy.orm import sessionmaker

Session = sessionmaker(bind=engine)
session = Session()
for x in session.query( Department, Employee).filter(Link.department_id ==
Department.id,
Link.employee_id==Employee.id).order_by(Link.department_id).all():
print ("Department: {} Name: {}".format(x.Department.name,
x.Employee.name))

As per the data populated in our example, output will be displayed as below:

Department: Accounts Name: John

Department: Accounts Name: Graham
Department: Sales Name: Graham
Department: Sales Name: Tony
Department: Marketing Name: John
Department: Marketing Name: Tony

84
 36. SQLAlchemy – Dialects SQLAlchemy

SQLAlchemy uses system of dialects to communicate with various types of databases. Each
database has a corresponding DBAPI wrapper. All dialects require that an appropriate DBAPI
driver is installed.

Following dialects are included in SQLAlchemy API:

 Firebird
 Microsoft SQL Server
 MySQL
 Oracle
 PostgreSQL
 SQL
 Sybase
An Engine object based on a URL is produced by create_engine() function. These URLs can
include username, password, hostname, and database name. There may be optional keyword
arguments for additional configuration. In some cases, a file path is accepted, and in others,
a “data source name” replaces the “host” and “database” portions. The typical form of a
database URL is as follows:

dialect+driver://username:password@host:port/database

PostgreSQL
The PostgreSQL dialect uses psycopg2 as the default DBAPI. pg8000 is also available as a pure-
Python substitute as shown below:

# default
engine = create_engine('postgresql://scott:tiger@localhost/mydatabase')

# psycopg2
engine =
create_engine('postgresql+psycopg2://scott:tiger@localhost/mydatabase')

# pg8000
engine = create_engine('postgresql+pg8000://scott:tiger@localhost/mydatabase')

85
 SQLAlchemy

MySQL
The MySQL dialect uses mysql-python as the default DBAPI. There are many MySQL DBAPIs
available, such as MySQL-connector-python as follows:

# default
engine = create_engine('mysql://scott:tiger@localhost/foo')

# mysql-python
engine = create_engine('mysql+mysqldb://scott:tiger@localhost/foo')

# MySQL-connector-python
engine = create_engine('mysql+mysqlconnector://scott:tiger@localhost/foo')

Oracle
The Oracle dialect uses cx_oracle as the default DBAPI as follows:

engine = create_engine('oracle://scott:[email protected]:1521/sidname')

engine = create_engine('oracle+cx_oracle://scott:tiger@tnsname')

Microsoft SQL Server

The SQL Server dialect uses pyodbc as the default DBAPI. pymssql is also available.

# pyodbc
engine = create_engine('mssql+pyodbc://scott:tiger@mydsn')

# pymssql
engine = create_engine('mssql+pymssql://scott:tiger@hostname:port/dbname')

SQLite
SQLite connects to file-based databases, using the Python built-in module sqlite3 by default.
As SQLite connects to local files, the URL format is slightly different. The “file” portion of the
URL is the filename of the database. For a relative file path, this requires three slashes as
shown below:

engine = create_engine('sqlite:///foo.db')

And for an absolute file path, the three slashes are followed by the absolute path as given
below:

86
 SQLAlchemy

engine = create_engine('sqlite:///C:\\path\\to\\foo.db')

To use a SQLite:memory:database, specify an empty URL as given below:

engine = create_engine('sqlite://')

Conclusion
In the first part of this tutorial, we have learnt how to use the Expression Language to
execute SQL statements. Expression language embeds SQL constructs in Python code. In
the second part, we have discussed object relation mapping capability of SQLAlchemy. The
ORM API maps the SQL tables with Python classes.

87