The Information Schema - MySQL 8 Query Performance Tuning - A Systematic Method For Improving Execution Speeds
The Information Schema - MySQL 8 Query Performance Tuning - A Systematic Method For Improving Execution Speeds
The Information Schema - MySQL 8 Query Performance Tuning: A Systematic Method for Improving Execution Speeds
MySQL 8 Query Performance Tuning: A Systematic Method for Improving Execution Speeds
PREV NEXT
⏮ ⏭
6. The sys Schema 8. SHOW Statements
🔎
7. The Information
Schema
1
Jesper Wisborg Krogh
When you need to optimize a query, it is common that you need information
about the schema, indexes, and the like. In that case, the Information Schema
is a good resource of data. This chapter introduces the Information Schema
together with an overview of the views it contains. The Information Schema
is used on several occasions in the rest of the book.
After the introduction of the Performance Schema in MySQL 5.5, the aim is to
make relatively static data such as schema information available through the
Information Schema and more volatile data belonging to the Performance
Schema. That said, it is not always clear-cut what belongs where, for
example, index statistics are relatively volatile, but are also part of the
schema information. There is also some information such as the InnoDB
metrics that for historical reasons still reside in the Information Schema.
Caution If you are still using MySQL 5.7 or earlier, be careful with
queries against views such as the TABLES and COLUMNS views in the
Information Schema. They can take a long time if the tables they contain
data for are not yet in the table definition cache or if the cache is not large
enough to hold all tables. An example of the performance difference of
the Information Schema between MySQL 5.7 and 8 is discussed in a blog
by the MySQL Server team: https://mysqlserverteam.com/mysql-
8-0-scaling-and-performance-of-information_schema/.
Privileges
The Information Schema is a virtual database, and the access to the views
works a little different from other tables. All users will see that the
information_schema schema exists, and they will see all views. However,
the result of querying the views depends on the privileges assigned to the
account. For example, an account that has no other privileges than the global
USAGE privilege will only see the Information Schema views when querying
the information_schema.TABLES view.
mysql> SELECT *
FROM information_schema.INNODB_METRICS;
ERROR: 1227: Access denied; you need (at least one of)
the PROCESS privilege(s) for this operation
Now, it is time to look at what kind of information you can find in the
Information Schema views.
Views
The data that is available in the Information Schema ranges from high-level
information about the system to low-level InnoDB metrics. This section
provides an overview of the views but will not go into detail as the most
important of the views from a performance tuning perspective are discussed
in the relevant parts of later chapters.
Find answers on the fly, or master something new. Subscribe today. See pricing options.
https://learning.oreilly.com/library/view/mysql-8-query/9781484255841/html/484666_1_En_7_Chapter.xhtml 1/12
4/15/2020 7. The Information Schema - MySQL 8 Query Performance Tuning: A Systematic Method for Improving Execution Speeds
Note Some plugins add their own views to the Information Schema.
The extra plugin views are not considered here.
SYSTEM INFORMATION
The highest level of information that is available in the Information Schema
concerns the whole MySQL instance. This includes such information as
which character sets are available and which plugins are installed.
The charact
CHARACTER_SETS
sets availab
The collatio
available fo
each charac
set. This
includes the
of the
collation
which in so
COLLATIONS
cases (e.g., i
the binary l
is used to
uniquely
specify both
the collation
and charact
set.
The mappin
of collation
character se
COLLATION_CHARACTER_SET_APPLICABILITY (the same a
the first two
columns of
COLLATION
The storage
engines tha
ENGINES are known a
whether the
are loaded.
A list of the
default
stopwords
that are use
INNODB_FT_DEFAULT_STOPWORD
when creati
a full text
index on an
InnoDB tab
A list of the
keywords in
MySQL and
KEYWORDS
whether the
keyword is
reserved.
The plugins
known to
PLUGINS MySQL
including th
status.
The resourc
groups that
are used by
threads to d
their work.
resource
RESOURCE_GROUPS
group
specifies th
priority of a
thread and
which CPUs
can use.
A list of the
spatial
reference
systems
including th
SRS_ID
ST_SPATIAL_REFERENCE_SYSTEMS column whi
contains the
used to spe
a reference
system for
spatial
columns.
The KEYWORDS view is, for example, useful when testing an upgrade as you
Find answers on the fly,can
orusemaster something new. Subscribe today. See pricing options.
it to verify whether any of your schema, table, column, routine, or
https://learning.oreilly.com/library/view/mysql-8-query/9781484255841/html/484666_1_En_7_Chapter.xhtml 2/12
4/15/2020 7. The Information Schema - MySQL 8 Query Performance Tuning: A Systematic Method for Improving Execution Speeds
parameter names matches a keyword in the new version. If that is the case,
you will need to update the application to quote the identifier, if that is not
already the case. To find all column names matching a keyword:
The query uses the COLUMNS view to find all column names except for the
system schemas (you can choose to include those, if you use those in your
application or in scripts). The COLUMNS view is one of several views
describing the schema objects.
SCHEMA INFORMATION
The views with information about the schema objects are among the most
useful views in the Information Schema. These are also the source for several
of the SHOW statements . You can use the views to find information from
everything from parameters for a stored routine to database names. The
views with schema information are summarized in Table 7-2.
The definition of
histograms including the
COLUMN_STATISTICS statistics. This is a very
useful view for query
performance tuning.
Information about
FILES
InnoDB tablespace files.
A snapshot of the
INNODB_FT_DELETED
view during an
OPTIMIZE TABLE
INNODB_FT_BEING_DELETED
statement for the InnoDB
table specified in the
innodb_ft_aux_table
option .
Configuration
information for full text
indexes on the InnoDB
INNODB_FT_CONFIG
table specified in the
innodb_ft_aux_table
option.
https://learning.oreilly.com/library/view/mysql-8-query/9781484255841/html/484666_1_En_7_Chapter.xhtml 3/12
4/15/2020 7. The Information Schema - MySQL 8 Query Performance Tuning: A Systematic Method for Improving Execution Speeds
Information about
indexes on InnoDB
tables. This includes
INNODB_INDEXES internal information
such as the page number
of the root page and the
merge threshold.
Internal metadata
information about virtual
INNODB_VIRTUAL
generated columns on
InnoDB tables.
Information about
REFERENTIAL_CONSTRAINTS
foreign keys.
Information about
ST_GEOMETRY_COLUMNS columns with a spatial
data type.
Find answers on the fly, or master something new. Subscribe today. See pricing options.
https://learning.oreilly.com/library/view/mysql-8-query/9781484255841/html/484666_1_En_7_Chapter.xhtml 4/12
4/15/2020 7. The Information Schema - MySQL 8 Query Performance Tuning: A Systematic Method for Improving Execution Speeds
Several of the views are closely related, for example, the columns are in
tables which are in schemas and constraints refer to tables and columns.
This means that some of the column names are present in several of the
views. The most commonly used column names that relate to these views
are
TABLE_NAME: Used in the views not specific to InnoDB for the table name.
TABLE_SCHEMA: Used in the views not specific to InnoDB for the schema
name.
COLUMN_NAME: Used in the views not specific to InnoDB for the column
name.
NAME: The InnoDB-specific views use a column called NAME to give the
name of the object irrespective of the object type.
In addition to the use of the names as in this list, there are also examples
where these column names are slightly modified like in the view
KEY_COLUMN_USAGE where you find the columns
REFERENCED_TABLE_SCHEMA, REFERENCED_TABLE_NAME, and
REFERENCED_COLUMN_NAME that are used in the description of foreign
keys. As an example, if you want to use the KEY_COLUMN_USAGE view to
find the tables with foreign keys referencing the sakila.film table, you can use
a query like this:
This shows that the film_actor, film_category, and inventory tables all have
foreign keys where the film table is the parent table. For example, if you look
at the table definition for film_actor:
Find answers on the fly, or master something new. Subscribe today. See pricing options.
https://learning.oreilly.com/library/view/mysql-8-query/9781484255841/html/484666_1_En_7_Chapter.xhtml 5/12
4/15/2020 7. The Information Schema - MySQL 8 Query Performance Tuning: A Systematic Method for Improving Execution Speeds
The views with information specific to InnoDB use the SPACE and
TABLE_ID to identify the tablespace and table. Each tablespace has a unique
id with ranges reserved for different tablespace types. For example, the data
dictionary tablespace file (<datadir>/mysql.ibd) has space id
4294967294, the temporary tablespace has id 4294967293, undo log
tablespaces start with 4294967279 and decrement, and user tablespaces
start at 1.
The views with information about InnoDB full text indexes are special as
they require you to set the innodb_ft_aux_table global variable with the
name of the table you want to get information for. For example, to get the full
text index configuration of the sakila.film_text table :
mysql> SELECT *
FROM information_schema.INNODB_FT_CONFIG;
+---------------------------+-------+
| KEY | VALUE |
+---------------------------+-------+
| optimize_checkpoint_limit | 180 |
| synced_doc_id | 1002 |
| stopword_table_name | |
| use_stopword | 1 |
+---------------------------+-------+
4 rows in set (0.0009 sec)
PERFORMANCE INFORMATION
The group of views that relate to performance are those that you will likely
use the most in your performance tuning work together with the
COLUMN_STATISTICS and STATISTICS views from the previous group of
views. The views with performance-related information are listed in Table 7-
3.
A list of the
pages in the
InnoDB buffer
pool which can
be used to
determine
which tables
and indexes
are currently
cached.
INNODB_BUFFER_PAGE Warning:
There is a high
overhead of
querying this
table
particularly for
large buffer
pools and
many tables
and indexes. It
is best used on
test systems.
Find answers on the fly, or master something new. Subscribe today. See pricing options.
https://learning.oreilly.com/library/view/mysql-8-query/9781484255841/html/484666_1_En_7_Chapter.xhtml 6/12
4/15/2020 7. The Information Schema - MySQL 8 Query Performance Tuning: A Systematic Method for Improving Execution Speeds
Information
about the
pages in the
InnoDB buffer
pool and how
they are
ordered in the
least recently
used (LRU) list.
Warning:
INNODB_BUFFER_PAGE_LRU
There is a high
overhead of
querying this
table
particularly for
large buffer
pools and
many tables
and indexes. It
is best used on
test systems.
Statistics about
the usage of
the InnoDB
buffer pool.
The
information is
similar to what
can be found in
the SHOW
INNODB_BUFFER_POOL_STATS ENGINE
INNODB
STATUS output
in the BUFFER
POOL AND
MEMORY
section. This is
one of the
most useful
views.
A summary of
the number of
index pages
INNODB_CACHED_INDEXES cached in the
InnoDB buffer
pool for each
index.
Statistics about
INNODB_CMP operations
related to
INNODB_CMP_RESET compressed
InnoDB tables.
The same as
INNODB_CMP_PER_INDEX
INNODB_CMP
but grouped by
INNODB_CMP_PER_INDEX_RESET
the index.
Statistics about
INNODB_CMPMEM compressed
pages in the
INNODB_CMPMEM_RESET InnoDB buffer
pool.
Similar to the
global status
INNODB_METRICS variables but
specific to
InnoDB.
Metadata
including the
connection id,
file path, and
size for InnoDB
temporary
tablespace files
(each session
gets its own
file in MySQL
8.0.13 and
INNODB_SESSION_TEMP_TABLESPACES
later). It can be
used to link a
session to a
tablespace file
which is very
useful if you
notice one file
becoming
large. The view
was added in
8.0.13.
Information
INNODB_TRX about InnoDB
transactions.
Find answers on the fly, or master something new. Subscribe today. See pricing options.
https://learning.oreilly.com/library/view/mysql-8-query/9781484255841/html/484666_1_En_7_Chapter.xhtml 7/12
4/15/2020 7. The Information Schema - MySQL 8 Query Performance Tuning: A Systematic Method for Improving Execution Speeds
When the
optimizer trace
is enabled, the
OPTIMIZER_TRACE
trace can be
queried from
this view.
The same as
PROCESSLIST SHOW
PROCESSLIST.
When profiling
is enabled, the
profiling
statistics can
be queried
from this view.
This is
PROFILING
deprecated,
and it is
recommended
to use the
Performance
Schema
instead.
For the views with information about InnoDB compressed tables, the table
with _RESET as the suffix returns the operation and timing statistics as
deltas since the last time the view was queried.
The metrics can be turned on and off as needed with the current status
found in the STATUS column. You specify the name of the metric as the value
to the innodb_monitor_enable or innodb_monitor_disable variable,
and you can use % as a wildcard. The value all works as a special value to
affect all metrics. Listing 7-1 shows an example of enabling and using all the
metrics matching %cpu% (which happens to be the metrics in the cpu
subsystem). The counter values depend on the workload you have at the
time of the query.
https://learning.oreilly.com/library/view/mysql-8-query/9781484255841/html/484666_1_En_7_Chapter.xhtml 8/12
4/15/2020 7. The Information Schema - MySQL 8 Query Performance Tuning: A Systematic Method for Improving Execution Speeds
STATUS: enabled
COMMENT: Number of cpus
6 rows in set (0.0011 sec)
mysql> SET GLOBAL innodb_monitor_disable = '%cpu%';
Query OK, 0 rows affected (0.0004 sec)
The InnoDB metrics are also included in the sys.metrics view together
with the global status variables and a few other metrics and when the
metrics are retrieved.
PRIVILEGE INFORMATION
MySQL uses privileges assigned to the accounts to determine which accounts
can access which schemas, tables, and columns. The common way to
determine the privileges for a given account is to use the SHOW GRANTS
statement, but the Information Schema also includes views that allow you to
query the privileges.
The Information Schema privilege views are summarized in Table 7-4. The
views are ordered from global privileges to column privileges.
mysql> SELECT *
FROM information_schema.USER_PRIVILEGES
WHERE GRANTEE =
'''mysql.sys''@''localhost'''\G
*************************** 1. row
***************************
GRANTEE: 'mysql.sys'@'localhost'
TABLE_CATALOG: def
PRIVILEGE_TYPE: USAGE
IS_GRANTABLE: NO
1 row in set (0.0006 sec)
mysql> SELECT *
FROM information_schema.SCHEMA_PRIVILEGES
WHERE GRANTEE =
'''mysql.sys''@''localhost'''\G
*************************** 1. row
***************************
GRANTEE: 'mysql.sys'@'localhost'
TABLE_CATALOG: def
TABLE_SCHEMA: sys
PRIVILEGE_TYPE: TRIGGER
IS_GRANTABLE: NO
1 row in set (0.0005 sec)
mysql> SELECT *
FROM information_schema.TABLE_PRIVILEGES
WHERE GRANTEE =
'''mysql.sys''@''localhost'''\G
*************************** 1. row
***************************
GRANTEE: 'mysql.sys'@'localhost'
TABLE_CATALOG: def
TABLE_SCHEMA: sys
TABLE_NAME: sys_config
PRIVILEGE_TYPE: SELECT
IS_GRANTABLE: NO
1 row in set (0.0005 sec)
mysql> SELECT *
FROM information_schema.COLUMN_PRIVILEGES
WHERE GRANTEE =
'''mysql.sys''@''localhost'''\G
Empty set (0.0005 sec)
https://learning.oreilly.com/library/view/mysql-8-query/9781484255841/html/484666_1_En_7_Chapter.xhtml 9/12
4/15/2020 7. The Information Schema - MySQL 8 Query Performance Tuning: A Systematic Method for Improving Execution Speeds
Notice how the single quotes around the username and hostname are
escaped by doubling the quotes.
While the views with the privilege information are not directly usable for
performance tuning, they are very useful for maintaining a stable system as
you can use them to easily identify whether any accounts have privileges
that they do not need.
Tip It is best practice to limit accounts to have just the privileges they
need and no more. That is one of the steps to keep the system secure.
The last topic to consider about the Information Schema is how data related
to index statistics are cached.
The index statistics themselves are however still originating from the storage
engine layer, so it is relatively expensive to query those. To improve the
performance, the statistics are cached in the data dictionary. You can control
how old the statistics are allowed to be before MySQL refreshes the cache.
This is done with the information_schema_stats_expiry variable
which defaults to 86400 seconds (one day). If you set the value to 0, you will
always get the latest values available from the storage engine; this is the
equivalent of the MySQL 5.7 behavior. The variable can be set both at the
global and session scopes, so you can set it to 0 for the session, if you are
investigating an issue where it is important to see the current statistics, for
example, if the optimizer is not using the index you expect.
The caching affects the columns listed in Table 7-5. The SHOW statements
displaying the same data are also affected.
The estimate
for the number
of unique
values for the
STATISTICS CARDINALITY part of the
index up and
including to the
column in the
same row.
The estimated
data length
divided with
AVG_ROW_LENGTH
the estimated
number of
rows.
The table
checksum. It is
CHECKSUM not used by
InnoDB, so the
value is NULL.
An estimate of
the amount of
free space in
the tablespace
the table
DATA_FREE belongs to. For
InnoDB, this is
the size of
completely free
extents minus a
safety margin.
Find answers on the fly, or master something new. Subscribe today. See pricing options.
https://learning.oreilly.com/library/view/mysql-8-query/9781484255841/html/484666_1_En_7_Chapter.xhtml 10/12
4/15/2020 7. The Information Schema - MySQL 8 Query Performance Tuning: A Systematic Method for Improving Execution Speeds
The estimated
size of the row
data. For
InnoDB, it is the
size of the
clustered index,
DATA_LENGTH
which is found
as the number
of pages in the
clustered index
multiplied with
the page size.
The estimated
size of
secondary
indexes. For
InnoDB, this is
INDEX_LENGTH
the sum of
pages in non-
clustered
indexes times
the page size.
The maximum
allowed size of
the data length.
MAX_DATA_LENGTH
It is not used by
InnoDB, so the
value is NULL.
The estimated
number of
rows. For
InnoDB tables,
TABLE_ROWS this comes from
the cardinality
of the primary
key or clustered
index.
When the
tablespace file
was last
updated. For
tables in the
InnoDB system
tablespace, the
value is NULL.
As data is
UPDATE_TIME written to the
tablespace
asynchronously,
the time will
not in general
reflect the time
of the last
statement
changing the
data.
You can force an update of this data for a given table by executing ANALYZE
TABLE for the table.
There are times when querying the data does not update the cached data:
When the cached data has not yet expired, that is, it was refreshed less
than information_schema_stats_expiry seconds ago
When MySQL or InnoDB is running in a read-only mode, that is, when one
of the modes, read_only, super_read_only,
transaction_read_only, or innodb_read_only, is enabled.
When the query also includes data from the Performance Schema
Summary
This chapter introduced the Information Schema by first discussing what the
Information Schema is and how the user privileges work. The remainder of
the chapter walked through the standard views and how caching works. The
Information Schema views can be grouped by the type of information they
contain: system, schema, performance, and privilege information.
The system information includes the character sets and collations, resource
groups, keywords, and information related to spatial data. This is useful as
an alternative to using the reference manual.
The schema information is the largest group of views and includes all the
information available from schema data down to columns, indexes, and
constraints. These views together with the performance views that have
information such as metrics and InnoDB buffer pool statistics are the most
commonly used views in performance tuning. The privilege-related views
are not so often used for performance tuning, but they are very useful to help
maintain a stable system.
Settings / Support / Sign Out Find answers on the fly, or master something new. Subscribe today. See pricing options.
https://learning.oreilly.com/library/view/mysql-8-query/9781484255841/html/484666_1_En_7_Chapter.xhtml 11/12
4/15/2020 7. The Information Schema - MySQL 8 Query Performance Tuning: A Systematic Method for Improving Execution Speeds
© 2020
PREVO'Reilly Media, Inc. Terms of Service / Privacy Policy NEXT
⏮ ⏭
6. The sys Schema 8. SHOW Statements
Find answers on the fly, or master something new. Subscribe today. See pricing options.
https://learning.oreilly.com/library/view/mysql-8-query/9781484255841/html/484666_1_En_7_Chapter.xhtml 12/12