</sect2>
</sect1>
+ <sect1 id="hot-standby">
+ <title>Hot Standby</title>
+
+ <para>
+ Hot Standby is the term used to describe the ability to connect to
+ the server and run queries while the server is in archive recovery. This
+ is useful for both log shipping replication and for restoring a backup
+ to an exact state with great precision.
+ The term Hot Standby also refers to the ability of the server to move
+ from recovery through to normal running while users continue running
+ queries and/or continue their connections.
+ </para>
+
+ <para>
+ Running queries in recovery is in many ways the same as normal running
+ though there are a large number of usage and administrative points
+ to note.
+ </para>
+
+ <sect2 id="hot-standby-users">
+ <title>User's Overview</title>
+
+ <para>
+ Users can connect to the database while the server is in recovery
+ and perform read-only queries. Read-only access to catalogs and views
+ will also occur as normal.
+ </para>
+
+ <para>
+ The data on the standby takes some time to arrive from the primary server
+ so there will be a measurable delay between primary and standby.
+ Queries executed on the standby will be correct as of the data that had
+ been recovered at the start of the query (or start of first statement,
+ in the case of Serializable transactions). Running the same query nearly
+ simultaneously on both primary and standby might therefore return
+ differing results. We say that data on the standby is eventually
+ consistent with the primary.
+ </para>
+
+ <para>
+ When a connection is made in recovery, the parameter
+ default_transaction_read_only will be forced to be true, whatever its
+ setting in postgresql.conf. As a result, all transactions started during
+ this time will be limited to read-only actions only. In all other ways,
+ connected sessions will appear identical to sessions initiated during
+ normal processing mode. There are no special commands required to
+ initiate a connection at this time, so all interfaces will work
+ normally without change.
+ </para>
+
+ <para>
+ Read-only here means "no writes to the permanent database tables". So
+ there are no problems with queries that make use of temporary sort and
+ work files will be used. Temporary tables cannot be created and
+ therefore cannot be used at all in recovery mode.
+ </para>
+
+ <para>
+ The following actions are allowed
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ Query access - SELECT, COPY TO including views and SELECT RULEs
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Cursor commands - DECLARE, FETCH, CLOSE,
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Parameters - SHOW, SET, RESET
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Transaction management commands
+ <itemizedlist>
+ <listitem>
+ <para>
+ BEGIN, END, ABORT, START TRANSACTION
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ SAVEPOINT, RELEASE, ROLLBACK TO SAVEPOINT
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ EXCEPTION blocks and other internal subtransactions
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ LOCK, with restrictions, see later
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Plans and resources - PREPARE, EXECUTE, DEALLOCATE, DISCARD
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Plugins and extensions - LOAD
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ These actions will produce error messages
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ DML - Insert, Update, Delete, COPY FROM, Truncate which all write data.
+ Any RULE which generates DML will throw error messages as a result.
+ Note that there is no action possible that can result in a trigger
+ being executed.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ DDL - Create, Drop, Alter, Comment (even for temporary tables because
+ currently these cause writes to catalog tables)
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ SELECT ... FOR SHARE | UPDATE which cause row locks to be written
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Transaction management commands that explicitly set non-read only state
+ <itemizedlist>
+ <listitem>
+ <para>
+ BEGIN READ WRITE,
+ START TRANSACTION READ WRITE
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ SET TRANSACTION READ WRITE,
+ SET SESSION CHARACTERISTICS AS TRANSACTION READ WRITE
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ SET transaction_read_only = off; or
+ SET default_transaction_read_only = off;
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Two-phase commit commands - PREPARE TRANSACTION, COMMIT PREPARED,
+ ROLLBACK PREPARED because even read-only transactions need to write
+ WAL in the prepare phase (the first phase of two phase commit).
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ sequence update - nextval()
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ LISTEN, UNLISTEN, NOTIFY since they currently write to system tables
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ Note that current behaviour of read only transactions when not in
+ recovery is to allow the last two actions, so there is a small and
+ subtle difference in behaviour between standby read-only transactions
+ and read only transactions during normal running.
+ It is possible that the restrictions on LISTEN, UNLISTEN, NOTIFY and
+ temporary tables may be lifted in a future release, if their internal
+ implementation is altered to make this possible.
+ </para>
+
+ <para>
+ If failover or switchover occurs the database will switch to normal
+ processing mode. Sessions will remain connected while the server
+ changes mode. Current transactions will continue, though will remain
+ read-only. After this, it will be possible to initiate read-write
+ transactions, though users must *manually* reset their
+ default_transaction_read_only setting first, if they want that
+ behaviour.
+ </para>
+
+ <para>
+ Users will be able to tell whether their session is read-only by
+ issuing SHOW default_transaction_read_only. In addition a set of
+ functions <xref linkend="functions-recovery-info-table"> allow users to
+ access information about Hot Standby. These allow you to write
+ functions that are aware of the current state of the database. These
+ can be used to monitor the progress of recovery, or to allow you to
+ write complex programs that restore the database to particular states.
+ </para>
+
+ <para>
+ In recovery, transactions will not be permitted to take any lock higher
+ other than AccessShareLock or AccessExclusiveLock. In addition,
+ transactions may never assign a TransactionId and may never write WAL.
+ The LOCK TABLE command by default applies an AccessExclusiveLock.
+ Any LOCK TABLE command that runs on the standby and requests a specific
+ lock type other than AccessShareLock will be rejected.
+ </para>
+
+ <para>
+ During recovery database changes are applied using full MVCC rules.
+ In general this means that queries will not experience lock conflicts
+ with writes, just like normal Postgres concurrency control (MVCC).
+ </para>
+ </sect2>
+
+ <sect2 id="hot-standby-conflict">
+ <title>Handling query conflicts</title>
+
+ <para>
+ There is some potential for conflict between standby queries
+ and WAL redo from the primary node. The user is provided with a number
+ of optional ways to handle these conflicts, though we must first
+ understand the possible reasons behind a conflict.
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ Access Exclusive Locks from primary node, including both explicit
+ LOCK commands and various kinds of DDL action
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Early cleanup of data still visible to the current query's snapshot
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Dropping tablespaces on the primary while standby queries are using
+ those tablespace for temporary work files (work_mem overflow)
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Dropping databases on the primary while that role is connected on standby.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Waiting to acquire buffer cleanup locks (for which there is no time out)
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ Some WAL redo actions will be for DDL actions. These DDL actions are
+ repeating actions that have already committed on the primary node, so
+ they must not fail on the standby node. These DDL locks take priority
+ and will automatically *cancel* any read-only transactions that get in
+ their way, after a grace period. This is similar to the possibility of
+ being canceled by the deadlock detector, but in this case the standby
+ process always wins, since the replayed actions must not fail. This
+ also ensures that replication doesn't fall behind while we wait for a
+ query to complete. Again, we assume that the standby is there for high
+ availability purposes primarily.
+ </para>
+
+ <para>
+ An example of the above would be an Administrator on Primary server
+ runs a DROP TABLE command that refers to a table currently in use by
+ a User query on the standby server.
+ </para>
+
+ <para>
+ Clearly the query cannot continue if we let the DROP TABLE proceed. If
+ this situation occurred on the primary, the DROP TABLE would wait until
+ the query has finished. When the query is on the standby and the
+ DROP TABLE is on the primary, the primary doesn't have information about
+ what the standby is running and so does not wait on the primary. The
+ WAL change records come through to the standby while the query is still
+ running, causing a conflict.
+ </para>
+
+ <para>
+ The second reason for conflict between standby queries and WAL redo is
+ "early cleanup". Normally, PostgreSQL allows cleanup of old row versions
+ when there are no users who may need to see them to ensure correct
+ visibility of data (known as MVCC). If there is a standby query that has
+ been running for longer than any query on the primary then it is possible
+ for old row versions to be removed by either VACUUM or HOT. This will
+ then generate WAL records that, if applied, would remove data on the
+ standby that might *potentially* be required by the standby query.
+ In more technical language, the Primary's xmin horizon is later than
+ the Standby's xmin horizon, allowing dead rows to be removed.
+ </para>
+
+ <para>
+ We have a number of choices for resolving query conflicts. The default
+ is that we wait and hope the query completes. If the recovery is not paused,
+ then the server will wait automatically until the server the lag between
+ primary and standby is at most max_standby_delay seconds. Once that grace
+ period expires, we then take one of the following actions:
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ If the conflict is caused by a lock, we cancel the standby transaction
+ immediately, even if it is idle-in-transaction.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ If the conflict is caused by cleanup records we tell the standby query
+ that a conflict has occurred and that it must cancel itself to avoid the
+ risk that it attempts to silently fails to read relevant data because
+ that data has been removed. (This is very similar to the much feared
+ error message "snapshot too old").
+ </para>
+
+ <para>
+ Note also that this means that idle-in-transaction sessions are never
+ canceled except by locks. Users should be clear that tables that are
+ regularly and heavily updated on primary server will quickly cause
+ cancellation of any longer running queries made against those tables.
+ </para>
+
+ <para>
+ If cancellation does occur, the query and/or transaction can always
+ be re-executed. The error is dynamic and will not necessarily occur
+ the same way if the query is executed again.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ Other remdial actions exist if the number of cancelations is unacceptable.
+ The first option is to connect to primary server and keep a query active
+ for as long as we need to run queries on the standby. This guarantees that
+ a WAL cleanup record is never generated and we don't ever get query
+ conflicts as described above. This could be done using contrib/dblink
+ and pg_sleep(), or via other mechanisms.
+ </para>
+
+ <para>
+ A second option is to pause recovery using recovery control functions.
+ These can pause WAL apply completely and allows queries to proceed to
+ completion. We can issue pg_recovery_continue() at any time, so the pause
+ can be held for long or short periods, as the administrator allows. This
+ method of conflict resolution may mean that there is a build up of WAL
+ records waiting to be applied and this will progressively increase the
+ failover delay. If there is regular arrival of WAL records this would
+ quickly prevent the use of the standby as a high availability failover
+ target. Some users may wish to use multiple standby servers for various
+ purposes. Pauses in recovery stay until explicitly released, so that
+ pauses override the setting of max_standby_delay.
+ </para>
+
+ <para>
+ Note that max_standby_delay is set in recovery.conf. It applies to the
+ server as a whole, so once used it may not be available for other users.
+ They will have to wait for the server to catch up again before the grace
+ period is available again. So max_standby_delay is a configuration
+ parameter set by the administrator which controls the maximum acceptable
+ failover delay and is not a user-settable parameter to specify how long
+ their query needs to run in.
+ </para>
+
+ <para>
+ Waits for buffer cleanup locks do not currently result in query
+ cancelation. Long waits are uncommon, though can happen in some cases
+ with long running nested loop joins.
+ </para>
+
+ <para>
+ Dropping tablespaces or databases is discussed in the administrator's
+ section since they are not typical user situations.
+ </para>
+ </sect2>
+
+ <sect2 id="hot-standby-admin">
+ <title>Administrator's Overview</title>
+
+ <para>
+ If there is a recovery.conf file present then the will start in Hot Standby
+ mode by default, though this can be disabled by setting
+ "recovery_connections = off" in recovery.conf. The server may take some
+ time to enable recovery connections since the server must first complete
+ sufficient recovery to provide a consistent state against which queries
+ can run before enabling read only connections. Look for these messages
+ in the server logs
+
+<programlisting>
+LOG: consistent recovery state reached
+LOG: database system is ready to accept read only connections
+</programlisting>
+
+ If you are running file-based log shipping ("warm standby"), you may need
+ to wait until the next WAL file arrives, which could be as long as the
+ archive_timeout setting on the primary. This is because consistency
+ information is recorded once per checkpoint on the primary. The
+ consistent state can also be delayed in the presence of both transactions
+ that contain large numbers of subtransactions and long-lived transactions.
+ </para>
+
+ <para>
+ The setting of max_connections on the standby should be equal to or
+ greater than the setting of max_connections on the primary. This is to
+ ensure that standby has sufficient resources to manage incoming
+ transactions.
+ </para>
+
+ <para>
+ It is important that the administrator consider the appropriate setting
+ of "max_standby_delay", set in recovery,conf. The default is 60 seconds,
+ though there is no optimal setting and it should be set according to
+ business priorities. For example if the server is primarily tasked as a
+ High Availability server, then you may wish to lower max_standby_delay
+ or even set it to zero. If the standby server is tasked as an additional
+ server for decision support queries then it may be acceptable to set this
+ to a value of many hours, e.g. max_standby_delay = 43200 (12 hours). It
+ is also possible to set max_standby_delay to -1 which means "always wait"
+ if there are conflicts, which will be useful when performing an archive
+ recovery from a backup.
+ </para>
+
+ <para>
+ A set of functions allow superusers to control the flow of recovery
+ are described in <xref linkend="functions-recovery-control-table">.
+ These functions allow you to pause and continue recovery, as well
+ as dynamically set new recovery targets wile recovery progresses.
+ Note that when a server is paused the apparent delay between primary
+ and standby will continue to increase.
+ </para>
+
+ <para>
+ Transaction status "hint bits" written on primary are not WAL-logged,
+ so data on standby will likely re-write the hints again on the standby.
+ Thus the main database blocks will produce write I/Os even though
+ all users are read-only; no changes have occurred to the data values
+ themselves. Users will be able to write large sort temp files and
+ re-generate relcache info files, so there is no part of the database
+ that is truly read-only during hot standby mode. There is no restriction
+ on use of set returning functions, or other users of tuplestore/tuplesort
+ code. Note also that writes to remote databases will still be possible,
+ even though the transaction is read-only locally.
+ </para>
+
+ <para>
+ Failover can be initiated at any time by allowing the startup process to
+ reach the end of WAL, or by issuing the function pg_recovery_stop()
+ as superuser.
+ </para>
+
+ <para>
+ The following types of administrator command will not be accepted
+ during recovery mode
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ Data Definition Language (DDL) - e.g. CREATE INDEX
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Privilege and Ownership - GRANT, REVOKE, REASSIGN
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Maintenance commands - ANALYZE, VACUUM, CLUSTER, REINDEX
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ Note again that some of these commands are actually allowed during
+ "read only" mode transactions on the primary.
+ </para>
+
+ <para>
+ As a result, you cannot create additional indexes that exist solely
+ on the standby, nor can statistics that exist solely on the standby.
+ </para>
+
+ <para>
+ pg_cancel_backend() will work on user backends, but not the Startup
+ process, which performs recovery. pg_locks will show locks held by
+ backends as normal. pg_locks also shows a virtual transaction
+ managed by the Startup process that owns all AccessExclusiveLocks held
+ by transactions being replayed by recovery. pg_stat_activity does not
+ show an entry for the Startup process, nor do recovering transactions
+ show as active.
+ </para>
+
+ <para>
+ check_pgsql will work, but it is very simple. check_postgres will also
+ work, though many some actions could give different or confusing results.
+ e.g. last vacuum time will not be maintained for example, since no
+ vacuum occurs on the standby (though vacuums running on the primary do
+ send their changes to the standby).
+ </para>
+
+ <para>
+ WAL file control commands will not work during recovery
+ e.g. pg_start_backup(), pg_switch_xlog() etc..
+ </para>
+
+ <para>
+ Dynamically loadable modules work, including the pg_stat_statements.
+ </para>
+
+ <para>
+ Advisory locks work normally in recovery, including deadlock detection.
+ Note that advisory locks are never WAL logged, so it is not possible for
+ an advisory lock on either the primary or the standby to conflict with WAL
+ replay. Nor is it possible to acquire an advisory lock on the primary
+ and have it initiate a similar advisory lock on the standby. Advisory
+ locks relate only to a single server on which they are acquired.
+ </para>
+
+ <para>
+ Trigger-based replication systems (Slony, Londiste, Bucardo etc) won't
+ run on the standby at all, though they will run happily on the primary
+ server. WAL replay is not trigger-based so you cannot relay from the
+ standby to any system that requires additional database writes or
+ relies on the use of triggers.
+ </para>
+
+ <para>
+ New oids cannot be assigned, though some UUID generators may still
+ work as long as they do not rely on writing new status to the database.
+ </para>
+
+ <para>
+ Currently, creating temp tables is not allowed during read only
+ transactions, so in some cases existing scripts will not run correctly.
+ It is possible we may relax that restriction in a later release. This is
+ both a SQL Standard compliance issue and a technical issue, so will not
+ be resolved in this release.
+ </para>
+
+ <para>
+ DROP TABLESPACE can only succeed if the tablespace is empty. Some standby
+ users may be actively using the tablespace via their temp_tablespaces
+ parameter. If there are temp files in the tablespace we currently
+ cancel all active queries to ensure that temp files are removed, so
+ that we can remove the tablespace and continue with WAL replay.
+ </para>
+
+ <para>
+ Running DROP DATABASE, ALTER DATABASE SET TABLESPACE, or ALTER DATABASE
+ RENAME on primary will cause all users connected to that database on the
+ standby to be forcibly disconnected, once max_standby_delay has been
+ reached.
+ </para>
+
+ <para>
+ In normal running, if you issue DROP USER or DROP ROLE for a role with login
+ capability while that user is still connected then nothing happens to the
+ connected user - they remain connected. The user cannot reconnect however.
+ This behaviour applies in recovery also, so a DROP USER on the primary does
+ not disconnect that user on the standby.
+ </para>
+
+ <para>
+ Stats collector is active during recovery. All scans, reads, blocks,
+ index usage etc will all be recorded normally on the standby. Replayed
+ actions will not duplicate their effects on primary, so replaying an
+ insert will not increment the Inserts column of pg_stat_user_tables.
+ The stats file is deleted at start of recovery, so stats from primary
+ and standby will differ; this is considered a feature not a bug.
+ </para>
+
+ <para>
+ Autovacuum is not active during recovery, though will start normally
+ at the end of recovery.
+ </para>
+
+ <para>
+ Background writer is active during recovery and will perform
+ restartpoints (similar to checkpoints on primary) and normal block
+ cleaning activities. The CHECKPOINT command is accepted during recovery,
+ though performs a restartpoint rather than a new checkpoint.
+ </para>
+ </sect2>
+
+ <sect2 id="hot-standby-parameters">
+ <title>Hot Standby Parameter Reference</title>
+
+ <para>
+ The following additional parameters are supported/provided within the
+ <filename>recovery.conf</>.
+
+ <variablelist>
+
+ <varlistentry id="recovery-connections" xreflabel="recovery_connections">
+ <term><varname>recovery_connections</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Specifies whether you would like to connect during recovery, or not.
+ The default is on, though you may wish to disable it to avoid
+ software problems, should they occur. Parameter can only be changed
+ be stopping and restarting the server.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="recovery-starts-paused" xreflabel="recovery_starts_paused">
+ <term><varname>recovery_starts_paused</varname> (<type>boolean</type>)</term>
+ <listitem>
+ <para>
+ Allows the Administrator to start recovery in paused mode. The default
+ is to start recovery so that it will continue processing all available
+ records.
+ </para>
+ </listitem>
+ </varlistentry>
+
+
+
+ <varlistentry id="max-standby-delay" xreflabel="max_standby_delay">
+ <term><varname>max_standby_delay</varname> (<type>string</type>)</term>
+ <listitem>
+ <para>
+ This parameter allows the Administrator to set a wait policy for
+ queries that conflict with incoming data changes. Valid settings
+ are -1, meaning wait forever, or a wait time of 0 or more seconds.
+ If a conflict should occur the server will delay up to this
+ amount before it begins trying to resolve things less amicably,
+ described in <xref linkend="hot-standby-conflict">. The
+ <varname>max_standby_delay</varname> may be set at server start
+ or it may be dynamically adjusted using <function>pg_recovery_max_standby_delay</>
+ described in <xref linkend="functions-recovery-control-table">.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </para>
+ </sect2>
+
+ <sect2 id="hot-standby-caveats">
+ <title>Caveats</title>
+
+ <para>
+ At this writing, there are several limitations of Hot Standby.
+ These can and probably will be fixed in future releases:
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ Operations on hash indexes are not presently WAL-logged, so
+ replay will not update these indexes. Hash indexes will not be
+ available for use when running queries during recovery.
+ </para>
+ </listitem>
+ </itemizedlist>
+
+ </para>
+ </sect2>
+
+ </sect1>
+
<sect1 id="migration">
<title>Migration Between Releases</title>
allows. See <xref linkend="sysvipc"> for information on how to
adjust those parameters, if necessary.
</para>
+
+ <para>
+ When running a standby server, it is strongly recommended that you
+ set this parameter to be the same or higher than the master server.
+ Otherwise, queries on the standby server may fail.
+ </para>
</listitem>
</varlistentry>
</listitem>
</varlistentry>
+ <varlistentry id="guc-trace-recovery-messages" xreflabel="trace_recovery_messages">
+ <term><varname>trace_recovery_messages</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>trace_recovery_messages</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Controls which message levels are written to the server log
+ for system modules needed for recovery processing. This allows
+ the user to override the normal setting of log_min_messages,
+ but only for specific messages. This is intended for use in
+ debugging Hot Standby.
+ Valid values are <literal>DEBUG5</>, <literal>DEBUG4</>,
+ <literal>DEBUG3</>, <literal>DEBUG2</>, <literal>DEBUG1</>,
+ <literal>INFO</>, <literal>NOTICE</>, <literal>WARNING</>,
+ <literal>ERROR</>, <literal>LOG</>, <literal>FATAL</>, and
+ <literal>PANIC</>. Each level includes all the levels that
+ follow it. The later the level, the fewer messages are sent
+ to the log. The default is <literal>WARNING</>. Note that
+ <literal>LOG</> has a different rank here than in
+ <varname>client_min_messages</>.
+ Parameter should be set in the postgresql.conf only.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-zero-damaged-pages" xreflabel="zero_damaged_pages">
<term><varname>zero_damaged_pages</varname> (<type>boolean</type>)</term>
<indexterm>
<xref linkend="continuous-archiving">.
</para>
+ <indexterm>
+ <primary>pg_is_in_recovery</primary>
+ </indexterm>
+ <indexterm>
+ <primary>pg_last_recovered_xid</primary>
+ </indexterm>
+ <indexterm>
+ <primary>pg_last_recovered_xact_timestamp</primary>
+ </indexterm>
+ <indexterm>
+ <primary>pg_last_recovered_xlog_location</primary>
+ </indexterm>
+ <indexterm>
+ <primary>pg_recovery_pause</primary>
+ </indexterm>
+ <indexterm>
+ <primary>pg_recovery_continue</primary>
+ </indexterm>
+ <indexterm>
+ <primary>pg_recovery_pause_xid</primary>
+ </indexterm>
+ <indexterm>
+ <primary>pg_recovery_pause_timestamp</primary>
+ </indexterm>
+ <indexterm>
+ <primary>pg_recovery_pause_location</primary>
+ </indexterm>
+ <indexterm>
+ <primary>pg_recovery_stop</primary>
+ </indexterm>
+ <indexterm>
+ <primary>pg_recovery_max_standby_delay</primary>
+ </indexterm>
+
+ <para>
+ The functions shown in <xref
+ linkend="functions-recovery-info-table"> provide information
+ about the current status of Hot Standby.
+ These functions may be executed during both recovery and in normal running.
+ </para>
+
+ <table id="functions-recovery-info-table">
+ <title>Recovery Information Functions</title>
+ <tgroup cols="3">
+ <thead>
+ <row><entry>Name</entry> <entry>Return Type</entry> <entry>Description</entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry>
+ <literal><function>pg_is_in_recovery</function>()</literal>
+ </entry>
+ <entry><type>bool</type></entry>
+ <entry>True if recovery is still in progress. If you wish to
+ know more detailed status information use <function>pg_current_recovery_target</>.
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <literal><function>pg_last_recovered_xid</function>()</literal>
+ </entry>
+ <entry><type>integer</type></entry>
+ <entry>Returns the transaction id (32-bit) of the last completed transaction
+ in the current recovery. Later numbered transaction ids may already have
+ completed, so the value could in some cases be lower than the last time
+ this function executed. If recovery has completed then the return value will
+ remain static at the value of the last transaction applied during that
+ recovery. When the server has been started normally without a recovery
+ then the return value will be InvalidXid (zero).
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <literal><function>pg_last_recovered_xact_timestamp</function>()</literal>
+ </entry>
+ <entry><type>timestamp with time zone</type></entry>
+ <entry>Returns the original completion timestamp with timezone of the
+ last recovered transaction. If recovery is still in progress this
+ will increase monotonically, while if recovery has completed then this
+ value will remain static at the value of the last transaction applied
+ during that recovery. When the server has been started normally without
+ a recovery then the return value will be a default value.
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <literal><function>pg_last_recovered_xlog_location</function>()</literal>
+ </entry>
+ <entry><type>text</type></entry>
+ <entry>Returns the transaction log location of the last recovered
+ transaction in the current recovery. This value is updated only
+ when transaction completion records (commit or abort) arrive, so
+ WAL records beyond this value may also have been recovered.
+ If recovery is still in progress this will increase monotonically.
+ If recovery has completed then this value will remain static at the
+ value of the last WAL record applied during that recovery. When the
+ server has been started normally without a recovery then the return
+ value will be InvalidXLogRecPtr (0/0).
+ </entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ The functions shown in <xref
+ linkend="functions-recovery-control-table"> can be used to control archive recovery
+ when executed in Hot Standby mode.
+ These functions can only be executed during recovery. Their use is
+ restricted to superusers only.
+ </para>
+
+ <table id="functions-recovery-control-table">
+ <title>Recovery Control Functions</title>
+ <tgroup cols="3">
+ <thead>
+ <row><entry>Name</entry> <entry>Return Type</entry> <entry>Description</entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry>
+ <literal><function>pg_recovery_pause</function>()</literal>
+ </entry>
+ <entry><type>void</type></entry>
+ <entry>Pause recovery processing, unconditionally.</entry>
+ </row>
+ <row>
+ <entry>
+ <literal><function>pg_recovery_continue</function>()</literal>
+ </entry>
+ <entry><type>void</type></entry>
+ <entry>If recovery is paused, continue processing.</entry>
+ </row>
+ <row>
+ <entry>
+ <literal><function>pg_recovery_stop</function>()</literal>
+ </entry>
+ <entry><type>void</type></entry>
+ <entry>End recovery and begin normal processing.</entry>
+ </row>
+ <row>
+ <entry>
+ <literal><function>pg_recovery_pause_xid</function>(xid integer)</literal>
+ </entry>
+ <entry><type>void</type></entry>
+ <entry>Continue recovery until specified xid completes, if it is ever
+ seen, then pause recovery.
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <literal><function>pg_recovery_pause_timestamp</function>(endtime timestamp)</literal>
+ </entry>
+ <entry><type>void</type></entry>
+ <entry>Continue recovery until a transaction with specified timestamp
+ completes, if one is ever seen, then pause recovery.
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <literal><function>pg_recovery_pause_location</function>(location text)</literal>
+ </entry>
+ <entry><type>void</type></entry>
+ <entry>Continue recovery until a transaction with an LSN higher than
+ the specified WAL location completes, if one is ever seen,
+ then pause recovery. The location is specified as a string of the
+ same form output by <function>pg_current_xlog_location()</function>,
+ e.g. pg_recovery_pause_location('0/D4445B8')
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <literal><function>pg_recovery_advance</function>(num_records integer)</literal>
+ </entry>
+ <entry><type>void</type></entry>
+ <entry>Advance recovery specified number of records then pause.</entry>
+ </row>
+ <row>
+ <entry>
+ <literal><function>pg_current_recovery_target</function>()</literal>
+ </entry>
+ <entry><type>text</type></entry>
+ <entry>Returns details of the server's current recovery target, if any.
+ If recovery is paused then the return value is 'Recovery paused'.
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <literal><function>pg_recovery_max_standby_delay</function>(delay integer)</literal>
+ </entry>
+ <entry><type>void</type></entry>
+ <entry>Set the max_standby_delay for recovery conflict processing (in seconds).</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ <function>pg_recovery_pause</> and <function>pg_recovery_continue</> allow
+ a superuser to control the progress of recovery on the database server.
+ Once recovery is paused it will stay paused until you release it, even if
+ the server falls further behind than max_standby_delay.
+ Recovery can be paused, continued, paused, continued, etc. as many times
+ as required. If the superuser wishes recovery to complete and normal
+ processing mode to start, execute <function>pg_recovery_stop</>.
+ </para>
+
+ <para>
+ The paused state provides a stable, unchanging database that can be
+ queried to determine how far forwards recovery has progressed. Recovery
+ can never go backwards because previous data may have been overwritten,
+ so some care must be taken to recover to a specific point.
+ <function>pg_recovery_pause_xid</> and
+ <function>pg_recovery_pause_timestamp</>, allow the specification of a trial
+ recovery target, similarly to <xref linkend="recovery-config-settings">.
+ Recovery will then progress to the specified point and then pause. This
+ allows the superuser to assess whether this is a desirable stopping point for
+ recovery, or a good place to copy data that is known to be deleted
+ later in the recovery. <function>pg_recovery_pause_location</>
+ can also be used to pause recovery after a transaction completion record
+ arrives that has a higher LSN.
+ </para>
+
+ <para>
+ <function>pg_recovery_advance</> allows recovery to progress record by
+ record, for very careful analysis or debugging. Step size can be 1 or
+ more records. If recovery is not yet paused then
+ <function>pg_recovery_advance</> will process the specified number of
+ records then pause. If recovery is already paused, recovery will continue
+ for another N records before pausing again.
+ </para>
+
+ <para>
+ If you pause recovery while the server is waiting for a WAL file when
+ operating in standby mode it will have apparently no effect until the
+ file arrives. Once the server begins processing WAL records again it
+ will notice the pause request and will act upon it. This is not a bug.
+ </para>
+
+ <para>
+ You can see if recovery is paused by checking the process title, or
+ by using <function>pg_current_recovery_target</>.
+ </para>
+
<para>
The functions shown in <xref linkend="functions-admin-dbsize"> calculate
the disk space usage of database objects.
<xref linkend="wal"> for more information about the WAL system.
</para>
+ <para>
+ If executed during recovery, the <command>CHECKPOINT</command> command
+ will force a restartpoint rather than writing a new checkpoint.
+ </para>
+
<para>
Only superusers can call <command>CHECKPOINT</command>. The command is
not intended for use during normal operation.
{
uint8 info = record->xl_info & ~XLR_INFO_MASK;
+ /*
+ * GIN indexes do not require any conflict processing. XXX really?
+ */
+ if (InHotStandby)
+ RecordKnownAssignedTransactionIds(record->xl_xid);
+
RestoreBkpBlocks(lsn, record, false);
topCtx = MemoryContextSwitchTo(opCtx);
uint8 info = record->xl_info & ~XLR_INFO_MASK;
MemoryContext oldCxt;
+ /*
+ * GIST indexes do not require any conflict processing. XXX really?
+ */
+ if (InHotStandby)
+ RecordKnownAssignedTransactionIds(record->xl_xid);
+
RestoreBkpBlocks(lsn, record, false);
oldCxt = MemoryContextSwitchTo(opCtx);
IndexScanDesc scan;
HashScanOpaque so;
+ /*
+ * Hash indexes are not recoverable, so cannot ever be used
+ * during recovery mode. We try to avoid this by tweaking the
+ * cost of hash index scans during recovery (see selfuncs.c),
+ * but we may still get called, so specifically prevent scans here.
+ * XXX We expect at some point to be able to exclude index scans on
+ * non-recoverable index types at the index AM level.
+ */
+ if (RecoveryInProgress())
+ elog(ERROR, "Cannot use hash indexes during recovery");
+
scan = RelationGetIndexScan(rel, keysz, scankey);
so = (HashScanOpaque) palloc(sizeof(HashScanOpaqueData));
so->hashso_bucket_valid = false;
}
}
+/*
+ * Update the latestRemovedXid for the current VACUUM. This gets called
+ * only rarely, since we probably already removed rows earlier.
+ * see comments for vacuum_log_cleanup_info().
+ */
+void
+HeapTupleHeaderAdvanceLatestRemovedXid(HeapTupleHeader tuple,
+ TransactionId *latestRemovedXid)
+{
+ TransactionId xmin = HeapTupleHeaderGetXmin(tuple);
+ TransactionId xmax = HeapTupleHeaderGetXmax(tuple);
+ TransactionId xvac = HeapTupleHeaderGetXvac(tuple);
+
+ if (tuple->t_infomask & HEAP_MOVED_OFF ||
+ tuple->t_infomask & HEAP_MOVED_IN)
+ {
+ if (TransactionIdPrecedes(*latestRemovedXid, xvac))
+ *latestRemovedXid = xvac;
+ }
+
+ if (TransactionIdPrecedes(*latestRemovedXid, xmax))
+ *latestRemovedXid = xmax;
+
+ if (TransactionIdPrecedes(*latestRemovedXid, xmin))
+ *latestRemovedXid = xmin;
+
+ Assert(TransactionIdIsValid(*latestRemovedXid));
+}
+
+/*
+ * Perform XLogInsert to register a heap cleanup info message. These
+ * messages are sent once per VACUUM and are required because
+ * of the phasing of removal operations during a lazy VACUUM.
+ * see comments for vacuum_log_cleanup_info().
+ */
+XLogRecPtr
+log_heap_cleanup_info(RelFileNode rnode, TransactionId latestRemovedXid)
+{
+ xl_heap_cleanup_info xlrec;
+ XLogRecPtr recptr;
+ XLogRecData rdata;
+
+ xlrec.node = rnode;
+ xlrec.latestRemovedXid = latestRemovedXid;
+
+ rdata.data = (char *) &xlrec;
+ rdata.len = SizeOfHeapCleanupInfo;
+ rdata.buffer = InvalidBuffer;
+ rdata.next = NULL;
+
+ recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_CLEANUP_INFO, &rdata);
+
+ return recptr;
+}
+
/*
* Perform XLogInsert for a heap-clean operation. Caller must already
* have modified the buffer and marked it dirty.
* Note: prior to Postgres 8.3, the entries in the nowunused[] array were
* zero-based tuple indexes. Now they are one-based like other uses
* of OffsetNumber.
+ *
+ * For 8.5 we also include the latestRemovedXid which allows recovery
+ * processing to cancel long standby queries that would be have their
+ * results changed if we applied these changes.
*/
XLogRecPtr
log_heap_clean(Relation reln, Buffer buffer,
OffsetNumber *redirected, int nredirected,
OffsetNumber *nowdead, int ndead,
OffsetNumber *nowunused, int nunused,
- bool redirect_move)
+ TransactionId latestRemovedXid, bool redirect_move)
{
xl_heap_clean xlrec;
uint8 info;
xlrec.node = reln->rd_node;
xlrec.block = BufferGetBlockNumber(buffer);
+ xlrec.latestRemovedXid = latestRemovedXid;
xlrec.nredirected = nredirected;
xlrec.ndead = ndead;
return recptr;
}
+/*
+ * Handles CLEANUP_INFO
+ */
+static void
+heap_xlog_cleanup_info(XLogRecPtr lsn, XLogRecord *record)
+{
+ xl_heap_cleanup_info *xlrec = (xl_heap_cleanup_info *) XLogRecGetData(record);
+
+ if (InHotStandby)
+ {
+ VirtualTransactionId *backends;
+
+ RecordKnownAssignedTransactionIds(record->xl_xid);
+ backends = GetConflictingVirtualXIDs(xlrec->latestRemovedXid,
+ InvalidOid,
+ true);
+ ResolveRecoveryConflictWithVirtualXIDs(backends,
+ "VACUUM index cleanup",
+ CONFLICT_MODE_ERROR_DEFERRABLE,
+ lsn);
+ }
+
+ /*
+ * Actual operation is a no-op. Record type exists to provide a means
+ * for conflict processing to occur before we begin index vacuum actions.
+ * see vacuumlazy.c
+ */
+}
+
/*
* Handles CLEAN and CLEAN_MOVE record types
*/
int nunused;
Size freespace;
+ if (InHotStandby)
+ {
+ VirtualTransactionId *backends;
+
+ RecordKnownAssignedTransactionIds(record->xl_xid);
+ backends = GetConflictingVirtualXIDs(xlrec->latestRemovedXid,
+ InvalidOid,
+ true);
+ ResolveRecoveryConflictWithVirtualXIDs(backends,
+ "VACUUM heap cleanup",
+ CONFLICT_MODE_ERROR_DEFERRABLE,
+ lsn);
+ }
+
+ RestoreBkpBlocks(lsn, record, true);
+
if (record->xl_info & XLR_BKP_BLOCK_1)
return;
- buffer = XLogReadBuffer(xlrec->node, xlrec->block, false);
+ buffer = XLogReadBufferExtended(xlrec->node, MAIN_FORKNUM, xlrec->block, RBM_NORMAL);
if (!BufferIsValid(buffer))
return;
+ LockBufferForCleanup(buffer);
page = (Page) BufferGetPage(buffer);
if (XLByteLE(lsn, PageGetLSN(page)))
Buffer buffer;
Page page;
+ /*
+ * Freezing tuples does not require conflict processing
+ */
+ if (InHotStandby)
+ RecordKnownAssignedTransactionIds(record->xl_xid);
+
+ RestoreBkpBlocks(lsn, record, false);
+
if (record->xl_info & XLR_BKP_BLOCK_1)
return;
- buffer = XLogReadBuffer(xlrec->node, xlrec->block, false);
+ buffer = XLogReadBufferExtended(xlrec->node, MAIN_FORKNUM, xlrec->block, RBM_NORMAL);
if (!BufferIsValid(buffer))
return;
+ LockBufferForCleanup(buffer);
page = (Page) BufferGetPage(buffer);
if (XLByteLE(lsn, PageGetLSN(page)))
{
uint8 info = record->xl_info & ~XLR_INFO_MASK;
+ /*
+ * Heap operations don't overwrite MVCC data so no conflict
+ * processing is required.
+ */
+ if (InHotStandby)
+ RecordKnownAssignedTransactionIds(record->xl_xid);
+
RestoreBkpBlocks(lsn, record, false);
switch (info & XLOG_HEAP_OPMASK)
switch (info & XLOG_HEAP_OPMASK)
{
case XLOG_HEAP2_FREEZE:
- RestoreBkpBlocks(lsn, record, false);
heap_xlog_freeze(lsn, record);
break;
case XLOG_HEAP2_CLEAN:
- RestoreBkpBlocks(lsn, record, true);
heap_xlog_clean(lsn, record, false);
break;
case XLOG_HEAP2_CLEAN_MOVE:
- RestoreBkpBlocks(lsn, record, true);
heap_xlog_clean(lsn, record, true);
break;
+ case XLOG_HEAP2_CLEANUP_INFO:
+ heap_xlog_cleanup_info(lsn, record);
+ break;
default:
elog(PANIC, "heap2_redo: unknown op code %u", info);
}
{
xl_heap_clean *xlrec = (xl_heap_clean *) rec;
- appendStringInfo(buf, "clean: rel %u/%u/%u; blk %u",
+ appendStringInfo(buf, "clean: rel %u/%u/%u; blk %u remxid %u",
xlrec->node.spcNode, xlrec->node.dbNode,
- xlrec->node.relNode, xlrec->block);
+ xlrec->node.relNode, xlrec->block,
+ xlrec->latestRemovedXid);
}
else if (info == XLOG_HEAP2_CLEAN_MOVE)
{
xl_heap_clean *xlrec = (xl_heap_clean *) rec;
- appendStringInfo(buf, "clean_move: rel %u/%u/%u; blk %u",
+ appendStringInfo(buf, "clean_move: rel %u/%u/%u; blk %u remxid %u",
xlrec->node.spcNode, xlrec->node.dbNode,
- xlrec->node.relNode, xlrec->block);
+ xlrec->node.relNode, xlrec->block,
+ xlrec->latestRemovedXid);
+ }
+ else if (info == XLOG_HEAP2_CLEANUP_INFO)
+ {
+ xl_heap_cleanup_info *xlrec = (xl_heap_cleanup_info *) rec;
+
+ appendStringInfo(buf, "cleanup info: remxid %u",
+ xlrec->latestRemovedXid);
}
else
appendStringInfo(buf, "UNKNOWN");
typedef struct
{
TransactionId new_prune_xid; /* new prune hint value for page */
- int nredirected; /* numbers of entries in arrays below */
+ TransactionId latestRemovedXid; /* latest xid to be removed by this prune */
+ int nredirected; /* numbers of entries in arrays below */
int ndead;
int nunused;
/* arrays that accumulate indexes of items to be changed */
if (!PageIsPrunable(page, OldestXmin))
return;
+ /*
+ * We can't write WAL in recovery mode, so there's no point trying to
+ * clean the page. The master will likely issue a cleaning WAL record
+ * soon anyway, so this is no particular loss.
+ */
+ if (RecoveryInProgress())
+ return;
+
/*
* We prune when a previous UPDATE failed to find enough space on the page
* for a new tuple version, or when free space falls below the relation's
* of our working state.
*/
prstate.new_prune_xid = InvalidTransactionId;
+ prstate.latestRemovedXid = InvalidTransactionId;
prstate.nredirected = prstate.ndead = prstate.nunused = 0;
memset(prstate.marked, 0, sizeof(prstate.marked));
prstate.redirected, prstate.nredirected,
prstate.nowdead, prstate.ndead,
prstate.nowunused, prstate.nunused,
- redirect_move);
+ prstate.latestRemovedXid, redirect_move);
PageSetLSN(BufferGetPage(buffer), recptr);
PageSetTLI(BufferGetPage(buffer), ThisTimeLineID);
== HEAPTUPLE_DEAD && !HeapTupleHeaderIsHotUpdated(htup))
{
heap_prune_record_unused(prstate, rootoffnum);
+ HeapTupleHeaderAdvanceLatestRemovedXid(htup,
+ &prstate->latestRemovedXid);
ndeleted++;
}
* find another DEAD tuple is a fairly unusual corner case.)
*/
if (tupdead)
+ {
latestdead = offnum;
+ HeapTupleHeaderAdvanceLatestRemovedXid(htup,
+ &prstate->latestRemovedXid);
+ }
else if (!recent_dead)
break;
else
scan->keyData = NULL;
+ /*
+ * During recovery we ignore killed tuples and don't bother to kill them
+ * either. We do this because the xmin on the primary node could easily
+ * be later than the xmin on the standby node, so that what the primary
+ * thinks is killed is supposed to be visible on standby. So for correct
+ * MVCC for queries during recovery we must ignore these hints and check
+ * all tuples. Do *not* set ignore_killed_tuples to true when running
+ * in a transaction that was started during recovery. AMs can set it to
+ * false at any time. xactStartedInRecovery should not be touched by AMs.
+ */
scan->kill_prior_tuple = false;
- scan->ignore_killed_tuples = true; /* default setting */
+ scan->xactStartedInRecovery = TransactionStartedDuringRecovery();
+ scan->ignore_killed_tuples = !scan->xactStartedInRecovery;
scan->opaque = NULL;
/*
* If we scanned a whole HOT chain and found only dead tuples,
- * tell index AM to kill its entry for that TID.
+ * tell index AM to kill its entry for that TID. We do not do
+ * this when in recovery because it may violate MVCC to do so.
+ * see comments in RelationGetIndexScan().
*/
- scan->kill_prior_tuple = scan->xs_hot_dead;
+ if (!scan->xactStartedInRecovery)
+ scan->kill_prior_tuple = scan->xs_hot_dead;
/*
* The AM's gettuple proc finds the next index entry matching the
immediately deleted due to a subsequent crash, there is no loss of
consistency, and the empty page will be picked up by the next VACUUM.
+Scans during Recovery
+---------------------
+
+The btree index type can be safely used during recovery. During recovery
+we have at most one writer and potentially many readers. In that
+situation the locking requirements can be relaxed and we do not need
+double locking during block splits. Each WAL record makes changes to a
+single level of the btree using the correct locking sequence and so
+is safe for concurrent readers. Some readers may observe a block split
+in progress as they descend the tree, but they will simple move right
+onto the correct page.
+
+During recovery all index scans start with ignore_killed_tuples = false
+and we never set kill_prior_tuple. We do this because the oldest xmin
+on the standby server can be older than the oldest xmin on the master
+server, which means tuples can be marked as killed even when they are
+still visible on the standby. We don't WAL log tuple killed bits, but
+they can still appear in the standby because of full page writes. So
+we must always ignore them and that means it's not worth setting them
+either.
+
Other Things That Are Handy to Know
-----------------------------------
}
if (ndeletable > 0)
- _bt_delitems(rel, buffer, deletable, ndeletable);
+ _bt_delitems(rel, buffer, deletable, ndeletable, false, 0);
/*
* Note: if we didn't find any LP_DEAD items, then the page's
*
* This routine assumes that the caller has pinned and locked the buffer.
* Also, the given itemnos *must* appear in increasing order in the array.
+ *
+ * We record VACUUMs and b-tree deletes differently in WAL. InHotStandby
+ * we need to be able to pin all of the blocks in the btree in physical
+ * order when replaying the effects of a VACUUM, just as we do for the
+ * original VACUUM itself. lastBlockVacuumed allows us to tell whether an
+ * intermediate range of blocks has had no changes at all by VACUUM,
+ * and so must be scanned anyway during replay.
*/
void
_bt_delitems(Relation rel, Buffer buf,
- OffsetNumber *itemnos, int nitems)
+ OffsetNumber *itemnos, int nitems, bool isVacuum,
+ BlockNumber lastBlockVacuumed)
{
Page page = BufferGetPage(buf);
BTPageOpaque opaque;
+ Assert(isVacuum || lastBlockVacuumed == 0);
+
/* No ereport(ERROR) until changes are logged */
START_CRIT_SECTION();
/* XLOG stuff */
if (!rel->rd_istemp)
{
- xl_btree_delete xlrec;
XLogRecPtr recptr;
XLogRecData rdata[2];
- xlrec.node = rel->rd_node;
- xlrec.block = BufferGetBlockNumber(buf);
+ if (isVacuum)
+ {
+ xl_btree_vacuum xlrec_vacuum;
+ xlrec_vacuum.node = rel->rd_node;
+ xlrec_vacuum.block = BufferGetBlockNumber(buf);
+
+ xlrec_vacuum.lastBlockVacuumed = lastBlockVacuumed;
+ rdata[0].data = (char *) &xlrec_vacuum;
+ rdata[0].len = SizeOfBtreeVacuum;
+ }
+ else
+ {
+ xl_btree_delete xlrec_delete;
+ xlrec_delete.node = rel->rd_node;
+ xlrec_delete.block = BufferGetBlockNumber(buf);
+
+ /*
+ * We would like to set an accurate latestRemovedXid, but there
+ * is no easy way of obtaining a useful value. So we use the
+ * probably far too conservative value of RecentGlobalXmin instead.
+ */
+ xlrec_delete.latestRemovedXid = InvalidTransactionId;
+ rdata[0].data = (char *) &xlrec_delete;
+ rdata[0].len = SizeOfBtreeDelete;
+ }
- rdata[0].data = (char *) &xlrec;
- rdata[0].len = SizeOfBtreeDelete;
rdata[0].buffer = InvalidBuffer;
rdata[0].next = &(rdata[1]);
rdata[1].buffer_std = true;
rdata[1].next = NULL;
- recptr = XLogInsert(RM_BTREE_ID, XLOG_BTREE_DELETE, rdata);
+ if (isVacuum)
+ recptr = XLogInsert(RM_BTREE_ID, XLOG_BTREE_VACUUM, rdata);
+ else
+ recptr = XLogInsert(RM_BTREE_ID, XLOG_BTREE_DELETE, rdata);
PageSetLSN(page, recptr);
PageSetTLI(page, ThisTimeLineID);
IndexBulkDeleteCallback callback;
void *callback_state;
BTCycleId cycleid;
- BlockNumber lastUsedPage;
+ BlockNumber lastBlockVacuumed; /* last blkno reached by Vacuum scan */
+ BlockNumber lastUsedPage; /* blkno of last non-recyclable page */
BlockNumber totFreePages; /* true total # of free pages */
MemoryContext pagedelcontext;
} BTVacState;
vstate.callback = callback;
vstate.callback_state = callback_state;
vstate.cycleid = cycleid;
+ vstate.lastBlockVacuumed = BTREE_METAPAGE; /* Initialise at first block */
vstate.lastUsedPage = BTREE_METAPAGE;
vstate.totFreePages = 0;
*/
if (ndeletable > 0)
{
- _bt_delitems(rel, buf, deletable, ndeletable);
+ BlockNumber lastBlockVacuumed = BufferGetBlockNumber(buf);
+
+ _bt_delitems(rel, buf, deletable, ndeletable, true, vstate->lastBlockVacuumed);
+
+ /*
+ * Keep track of the block number of the lastBlockVacuumed, so
+ * we can scan those blocks as well during WAL replay. This then
+ * provides concurrency protection and allows btrees to be used
+ * while in recovery.
+ */
+ if (lastBlockVacuumed > vstate->lastBlockVacuumed)
+ vstate->lastBlockVacuumed = lastBlockVacuumed;
+
stats->tuples_removed += ndeletable;
/* must recompute maxoff */
maxoff = PageGetMaxOffsetNumber(page);
#include "access/nbtree.h"
#include "access/transam.h"
+#include "access/xact.h"
#include "storage/bufmgr.h"
+#include "storage/procarray.h"
+#include "utils/inval.h"
+#include "miscadmin.h"
/*
- * We must keep track of expected insertions due to page splits, and apply
+ * We must keep track of expected insertions due to page spl its, and apply
* them manually if they are not seen in the WAL log during replay. This
* makes it safe for page insertion to be a multiple-WAL-action process.
*
xlrec->leftsib, xlrec->rightsib, isroot);
}
+static void
+btree_xlog_vacuum(XLogRecPtr lsn, XLogRecord *record)
+{
+ xl_btree_vacuum *xlrec;
+ Buffer buffer;
+ Page page;
+ BTPageOpaque opaque;
+
+ if (record->xl_info & XLR_BKP_BLOCK_1)
+ return;
+
+ xlrec = (xl_btree_vacuum *) XLogRecGetData(record);
+
+ /*
+ * We need to ensure every block is unpinned between the
+ * lastBlockVacuumed and the current block, if there are any.
+ * This ensures that every block in the index is touched during
+ * VACUUM as required to ensure scans work correctly.
+ */
+ if ((xlrec->lastBlockVacuumed + 1) != xlrec->block)
+ {
+ BlockNumber blkno = xlrec->lastBlockVacuumed + 1;
+
+ for (; blkno < xlrec->block; blkno++)
+ {
+ /*
+ * XXX we don't actually need to read the block, we
+ * just need to confirm it is unpinned. If we had a special call
+ * into the buffer manager we could optimise this so that
+ * if the block is not in shared_buffers we confirm it as unpinned.
+ */
+ buffer = XLogReadBufferExtended(xlrec->node, MAIN_FORKNUM, blkno, RBM_NORMAL);
+ if (BufferIsValid(buffer))
+ {
+ LockBufferForCleanup(buffer);
+ UnlockReleaseBuffer(buffer);
+ }
+ }
+ }
+
+ /*
+ * We need to take a cleanup lock to apply these changes.
+ * See nbtree/README for details.
+ */
+ buffer = XLogReadBufferExtended(xlrec->node, MAIN_FORKNUM, xlrec->block, RBM_NORMAL);
+ if (!BufferIsValid(buffer))
+ return;
+ LockBufferForCleanup(buffer);
+ page = (Page) BufferGetPage(buffer);
+
+ if (XLByteLE(lsn, PageGetLSN(page)))
+ {
+ UnlockReleaseBuffer(buffer);
+ return;
+ }
+
+ if (record->xl_len > SizeOfBtreeVacuum)
+ {
+ OffsetNumber *unused;
+ OffsetNumber *unend;
+
+ unused = (OffsetNumber *) ((char *) xlrec + SizeOfBtreeVacuum);
+ unend = (OffsetNumber *) ((char *) xlrec + record->xl_len);
+
+ PageIndexMultiDelete(page, unused, unend - unused);
+ }
+
+ /*
+ * Mark the page as not containing any LP_DEAD items --- see comments in
+ * _bt_delitems().
+ */
+ opaque = (BTPageOpaque) PageGetSpecialPointer(page);
+ opaque->btpo_flags &= ~BTP_HAS_GARBAGE;
+
+ PageSetLSN(page, lsn);
+ PageSetTLI(page, ThisTimeLineID);
+ MarkBufferDirty(buffer);
+ UnlockReleaseBuffer(buffer);
+}
+
static void
btree_xlog_delete(XLogRecPtr lsn, XLogRecord *record)
{
return;
xlrec = (xl_btree_delete *) XLogRecGetData(record);
+
+ /*
+ * We don't need to take a cleanup lock to apply these changes.
+ * See nbtree/README for details.
+ */
buffer = XLogReadBuffer(xlrec->node, xlrec->block, false);
if (!BufferIsValid(buffer))
return;
{
uint8 info = record->xl_info & ~XLR_INFO_MASK;
+ /*
+ * Btree delete records can conflict with standby queries. You might
+ * think that vacuum records would conflict as well, but they don't.
+ * XLOG_HEAP2_CLEANUP_INFO records provide the highest xid cleaned
+ * by the vacuum of the heap and so we can resolve any conflicts just
+ * once when that arrives. After that any we know that no conflicts exist
+ * from individual btree vacuum records on that index.
+ */
+ if (InHotStandby)
+ {
+ RecordKnownAssignedTransactionIds(record->xl_xid);
+ if (info == XLOG_BTREE_DELETE)
+ {
+ xl_btree_delete *xlrec = (xl_btree_delete *) XLogRecGetData(record);
+ VirtualTransactionId *backends;
+
+ /*
+ * XXX Currently we put everybody on death row, because
+ * currently _bt_delitems() supplies InvalidTransactionId.
+ * This can be fairly painful, so providing a better value
+ * here is worth some thought and possibly some effort to
+ * improve.
+ */
+ backends = GetConflictingVirtualXIDs(xlrec->latestRemovedXid,
+ InvalidOid,
+ true);
+
+ ResolveRecoveryConflictWithVirtualXIDs(backends,
+ "drop tablespace",
+ CONFLICT_MODE_ERROR_DEFERRABLE,
+ lsn);
+ }
+ }
+
+ /*
+ * Exclusive lock on a btree block is as good as a Cleanup lock,
+ * so need to special case btree delete and vacuum.
+ */
RestoreBkpBlocks(lsn, record, false);
switch (info)
case XLOG_BTREE_SPLIT_R_ROOT:
btree_xlog_split(false, true, lsn, record);
break;
+ case XLOG_BTREE_VACUUM:
+ btree_xlog_vacuum(lsn, record);
+ break;
case XLOG_BTREE_DELETE:
btree_xlog_delete(lsn, record);
break;
xlrec->level, xlrec->firstright);
break;
}
+ case XLOG_BTREE_VACUUM:
+ {
+ xl_btree_vacuum *xlrec = (xl_btree_vacuum *) rec;
+
+ appendStringInfo(buf, "vacuum: rel %u/%u/%u; blk %u, lastBlockVacuumed %u",
+ xlrec->node.spcNode, xlrec->node.dbNode,
+ xlrec->node.relNode, xlrec->block,
+ xlrec->lastBlockVacuumed);
+ break;
+ }
case XLOG_BTREE_DELETE:
{
xl_btree_delete *xlrec = (xl_btree_delete *) rec;
- appendStringInfo(buf, "delete: rel %u/%u/%u; blk %u",
+ appendStringInfo(buf, "delete: rel %u/%u/%u; blk %u, latestRemovedXid %u",
xlrec->node.spcNode, xlrec->node.dbNode,
- xlrec->node.relNode, xlrec->block);
+ xlrec->node.relNode, xlrec->block,
+ xlrec->latestRemovedXid);
break;
}
case XLOG_BTREE_DELETE_PAGE:
a tuple, though there are a few other places that need an XID assigned.
If a subtransaction requires an XID, we always first assign one to its
parent. This maintains the invariant that child transactions have XIDs later
-than their parents, which is assumed in a number of places.
+than their parents, which is assumed in a number of places. In 8.5 onwards,
+some corner cases exist that require XID assignment to be WAL logged.
The subsidiary actions of obtaining a lock on the XID and and entering it into
pg_subtrans and PG_PROC are done at the time it is assigned.
the bulk update. However, all these paths are designed to write data that
no other transaction can see until after T1 commits. The situation is thus
not different from ordinary WAL-logged updates.
+
+Transaction Emulation during Recovery
+-------------------------------------
+
+During Recovery we replay transaction changes in the order they occurred.
+As part of this replay we emulate some transactional behaviour, so that
+read only backends can take MVCC snapshots. We do this by maintaining a
+list of XIDs belonging to transactions that are being replayed, so that
+each transaction that has recorded WAL records for database writes exist
+in the array until it commits. Further details are given in comments in
+procarray.c.
+
+Many actions write no WAL records at all, for example read only transactions.
+These have no effect on MVCC in recovery and we can pretend they never
+occurred at all. Subtransaction commit does not write a WAL record either
+and has very little effect, since lock waiters need to wait for the
+parent transaction to complete.
+
+Not all transactional behaviour is emulated, for example we do not insert
+a transaction entry into the lock table, nor do we maintain the transaction
+stack in memory. Clog entries are made normally. Multitrans is not maintained
+because its purpose is to record tuple level locks that an application has
+requested to prevent write locks. Since write locks cannot be obtained at all,
+there is never any conflict and so there is no reason to update multitrans.
+Subtrans is maintained during recovery but the details of the transaction
+tree are ignored and all subtransactions reference the top-level TransactionId
+directly. Since commit is atomic this provides correct lock wait behaviour
+yet simplifies emulation of subtransactions considerably.
+
+Further details on locking mechanics in recovery are given in comments
+with the Lock rmgr code.
#include "access/clog.h"
#include "access/slru.h"
#include "access/transam.h"
+#include "access/xact.h"
+#include "access/xlogutils.h"
#include "pg_trace.h"
#include "postmaster/bgwriter.h"
/* Backup blocks are not used in clog records */
Assert(!(record->xl_info & XLR_BKP_BLOCK_MASK));
+ if (InHotStandby)
+ RecordKnownAssignedTransactionIds(record->xl_xid);
+
if (info == CLOG_ZEROPAGE)
{
int pageno;
* MultiXactSetNextMXact and/or MultiXactAdvanceNextMXact. Note that we
* may already have replayed WAL data into the SLRU files.
*
- * We don't need any locks here, really; the SLRU locks are taken
- * only because slru.c expects to be called with locks held.
+ * We want this operation to be atomic to ensure that other processes can
+ * use MultiXact while we complete recovery. We access one page only from the
+ * offset and members buffers, so once locks are acquired they will not be
+ * dropped and re-acquired by SLRU code. So we take both locks at start, then
+ * hold them all the way to the end.
*/
void
StartupMultiXact(void)
/* Clean up offsets state */
LWLockAcquire(MultiXactOffsetControlLock, LW_EXCLUSIVE);
+ LWLockAcquire(MultiXactMemberControlLock, LW_EXCLUSIVE);
/*
* Initialize our idea of the latest page number.
MultiXactOffsetCtl->shared->page_dirty[slotno] = true;
}
- LWLockRelease(MultiXactOffsetControlLock);
-
/* And the same for members */
- LWLockAcquire(MultiXactMemberControlLock, LW_EXCLUSIVE);
/*
* Initialize our idea of the latest page number.
}
LWLockRelease(MultiXactMemberControlLock);
+ LWLockRelease(MultiXactOffsetControlLock);
/*
* Initialize lastTruncationPoint to invalid, ensuring that the first
* isn't valid (because StartupMultiXact hasn't been called yet) and so
* SimpleLruTruncate would get confused. It seems best not to risk
* removing any data during recovery anyway, so don't truncate.
+ * We are executing in the bgwriter, so we must access shared status.
*/
if (!RecoveryInProgress())
TruncateMultiXact();
/* Backup blocks are not used in multixact records */
Assert(!(record->xl_info & XLR_BKP_BLOCK_MASK));
+ if (InHotStandby)
+ RecordKnownAssignedTransactionIds(record->xl_xid);
+
if (info == XLOG_MULTIXACT_ZERO_OFF_PAGE)
{
int pageno;
#include "commands/dbcommands.h"
#include "commands/sequence.h"
#include "commands/tablespace.h"
+#include "storage/sinval.h"
#include "storage/freespace.h"
{"Tablespace", tblspc_redo, tblspc_desc, NULL, NULL, NULL},
{"MultiXact", multixact_redo, multixact_desc, NULL, NULL, NULL},
{"Reserved 7", NULL, NULL, NULL, NULL, NULL},
- {"Reserved 8", NULL, NULL, NULL, NULL, NULL},
+ {"Relation", relation_redo, relation_desc, NULL, NULL, NULL},
{"Heap2", heap2_redo, heap2_desc, NULL, NULL, NULL},
{"Heap", heap_redo, heap_desc, NULL, NULL, NULL},
{"Btree", btree_redo, btree_desc, btree_xlog_startup, btree_xlog_cleanup, btree_safe_restartpoint},
* commands to set the commit status of transactions whose bits are in
* already-truncated segments of the commit log (see notes in
* SlruPhysicalWritePage). Hence, if we are InRecovery, allow the case
- * where the file doesn't exist, and return zeroes instead.
+ * where the file doesn't exist, and return zeroes instead. We also
+ * return a zeroed page when seek and read fails.
*/
fd = BasicOpenFile(path, O_RDWR | PG_BINARY, S_IRUSR | S_IWUSR);
if (fd < 0)
if (lseek(fd, (off_t) offset, SEEK_SET) < 0)
{
+ if (InRecovery)
+ {
+ ereport(LOG,
+ (errmsg("file \"%s\" doesn't exist, reading as zeroes",
+ path)));
+ MemSet(shared->page_buffer[slotno], 0, BLCKSZ);
+ return true;
+ }
slru_errcause = SLRU_SEEK_FAILED;
slru_errno = errno;
close(fd);
errno = 0;
if (read(fd, shared->page_buffer[slotno], BLCKSZ) != BLCKSZ)
{
+ if (InRecovery)
+ {
+ ereport(LOG,
+ (errmsg("file \"%s\" doesn't exist, reading as zeroes",
+ path)));
+ MemSet(shared->page_buffer[slotno], 0, BLCKSZ);
+ return true;
+ }
slru_errcause = SLRU_READ_FAILED;
slru_errno = errno;
close(fd);
#include "access/slru.h"
#include "access/subtrans.h"
#include "access/transam.h"
+#include "miscadmin.h"
#include "pg_trace.h"
#include "utils/snapmgr.h"
* 0xFFFFFFFF/SUBTRANS_XACTS_PER_PAGE, and segment numbering at
* 0xFFFFFFFF/SUBTRANS_XACTS_PER_PAGE/SLRU_SEGMENTS_PER_PAGE. We need take no
* explicit notice of that fact in this module, except when comparing segment
- * and page numbers in TruncateSUBTRANS (see SubTransPagePrecedes).
+ * and page numbers in TruncateSUBTRANS (see SubTransPagePrecedes)
+ * and in recovery when we do ExtendSUBTRANS.
*/
/* We need four bytes per xact */
ptr = (TransactionId *) SubTransCtl->shared->page_buffer[slotno];
ptr += entryno;
- /* Current state should be 0 */
- Assert(*ptr == InvalidTransactionId);
+ /*
+ * Current state should be 0, except in recovery where we may
+ * need to reset the value multiple times
+ */
+ Assert(*ptr == InvalidTransactionId ||
+ (InRecovery && *ptr == parent));
*ptr = parent;
/*
* This must be called ONCE during postmaster or standalone-backend startup,
* after StartupXLOG has initialized ShmemVariableCache->nextXid.
- *
- * oldestActiveXID is the oldest XID of any prepared transaction, or nextXid
- * if there are none.
*/
void
StartupSUBTRANS(TransactionId oldestActiveXID)
{
- int startPage;
- int endPage;
+ TransactionId xid = ShmemVariableCache->nextXid;
+ int pageno = TransactionIdToPage(xid);
- /*
- * Since we don't expect pg_subtrans to be valid across crashes, we
- * initialize the currently-active page(s) to zeroes during startup.
- * Whenever we advance into a new page, ExtendSUBTRANS will likewise zero
- * the new page without regard to whatever was previously on disk.
- */
LWLockAcquire(SubtransControlLock, LW_EXCLUSIVE);
- startPage = TransactionIdToPage(oldestActiveXID);
- endPage = TransactionIdToPage(ShmemVariableCache->nextXid);
-
- while (startPage != endPage)
- {
- (void) ZeroSUBTRANSPage(startPage);
- startPage++;
- }
- (void) ZeroSUBTRANSPage(startPage);
+ /*
+ * Initialize our idea of the latest page number.
+ */
+ SubTransCtl->shared->latest_page_number = pageno;
LWLockRelease(SubtransControlLock);
}
ExtendSUBTRANS(TransactionId newestXact)
{
int pageno;
+ static int last_pageno = 0;
- /*
- * No work except at first XID of a page. But beware: just after
- * wraparound, the first XID of page zero is FirstNormalTransactionId.
- */
- if (TransactionIdToEntry(newestXact) != 0 &&
- !TransactionIdEquals(newestXact, FirstNormalTransactionId))
- return;
+ Assert(TransactionIdIsNormal(newestXact));
- pageno = TransactionIdToPage(newestXact);
+ if (!InRecovery)
+ {
+ /*
+ * No work except at first XID of a page. But beware: just after
+ * wraparound, the first XID of page zero is FirstNormalTransactionId.
+ */
+ if (TransactionIdToEntry(newestXact) != 0 &&
+ !TransactionIdEquals(newestXact, FirstNormalTransactionId))
+ return;
+
+ pageno = TransactionIdToPage(newestXact);
+ }
+ else
+ {
+ /*
+ * InRecovery we keep track of the last page we extended, so
+ * we can compare that against incoming XIDs. This will only
+ * ever be run by startup process, so keep it as a static variable
+ * rather than hiding behind the SubtransControlLock.
+ */
+ pageno = TransactionIdToPage(newestXact);
+
+ if (pageno == last_pageno ||
+ SubTransPagePrecedes(pageno, last_pageno))
+ return;
+
+ ereport(trace_recovery(DEBUG1),
+ (errmsg("extend subtrans xid %u page %d last_page %d",
+ newestXact, pageno, last_pageno)));
+
+ last_pageno = pageno;
+ }
LWLockAcquire(SubtransControlLock, LW_EXCLUSIVE);
static XidStatus cachedFetchXidStatus;
static XLogRecPtr cachedCommitLSN;
-/* Handy constant for an invalid xlog recptr */
-static const XLogRecPtr InvalidXLogRecPtr = {0, 0};
-
/* Local functions */
static XidStatus TransactionLogFetch(TransactionId transactionId);
FreeDir(cldir);
}
+void
+ProcessTwoPhaseStandbyRecords(TransactionId xid)
+{
+ char *buf;
+ char *bufptr;
+ TwoPhaseFileHeader *hdr;
+
+ /* Read and validate file, if possible */
+ buf = ReadTwoPhaseFile(xid);
+ if (buf != NULL)
+ {
+ /* Deconstruct header */
+ hdr = (TwoPhaseFileHeader *) buf;
+ Assert(TransactionIdEquals(hdr->xid, xid));
+ bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
+ bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
+ bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
+ bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
+
+ /*
+ * Recover other state using resource managers
+ */
+ ProcessRecords(bufptr, xid, twophase_postcommit_standby_callbacks);
+
+ pfree(buf);
+ }
+}
+
/*
* RecordTransactionCommitPrepared
*
/* Emit the XLOG commit record */
xlrec.xid = xid;
xlrec.crec.xact_time = GetCurrentTimestamp();
+ xlrec.crec.xinfo = 0;
+ xlrec.crec.nmsgs = 0;
xlrec.crec.nrels = nrels;
xlrec.crec.nsubxacts = nchildren;
+
rdata[0].data = (char *) (&xlrec);
rdata[0].len = MinSizeOfXactCommitPrepared;
rdata[0].buffer = InvalidBuffer;
#include "storage/lock.h"
#include "utils/inval.h"
+const TwoPhaseCallback twophase_postcommit_standby_callbacks[TWOPHASE_RM_MAX_ID + 1] =
+{
+ NULL, /* END ID */
+ NULL, /* Lock - see notes in xact_redo_commit() */
+ inval_twophase_postcommit, /* Inval - see notes in xact_redo_commit() */
+ NULL, /* notify/listen doesn't work in recovery */
+ NULL /* pgstat doesn't record recovery work */
+};
const TwoPhaseCallback twophase_recover_callbacks[TWOPHASE_RM_MAX_ID + 1] =
{
#include "storage/fd.h"
#include "storage/lmgr.h"
#include "storage/procarray.h"
+#include "storage/sinval.h"
#include "storage/sinvaladt.h"
#include "storage/smgr.h"
#include "utils/combocid.h"
ResourceOwner curTransactionOwner; /* my query resources */
TransactionId *childXids; /* subcommitted child XIDs, in XID order */
int nChildXids; /* # of subcommitted child XIDs */
+ int nReportedChildXids;
int maxChildXids; /* allocated size of childXids[] */
Oid prevUser; /* previous CurrentUserId setting */
bool prevSecDefCxt; /* previous SecurityDefinerContext setting */
bool prevXactReadOnly; /* entry-time xact r/o state */
+ bool startedInRecovery; /* did we start in recovery? */
+ bool reportedXid;
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
NULL, /* cur transaction resource owner */
NULL, /* subcommitted child Xids */
0, /* # of subcommitted child Xids */
+ 0, /* # of reported child Xids */
0, /* allocated size of childXids[] */
InvalidOid, /* previous CurrentUserId setting */
false, /* previous SecurityDefinerContext setting */
false, /* entry-time xact r/o state */
+ false, /* startedInRecovery */
+ false, /* reportedXid */
NULL /* link to parent state block */
};
*/
static MemoryContext TransactionAbortContext = NULL;
+/*
+ * Local state to optimise recovery conflict resolution
+ */
+static TransactionId latestRemovedXid = InvalidTransactionId;
+
/*
* List of add-on start- and end-of-xact callbacks
*/
static const char *TransStateAsString(TransState state);
+static TransactionId *xactGetUnreportedChildren(int threshold, int *nxids);
+static TransactionId *xactCollectUnreportedChildren(TransactionState s, TransactionId *xids);
+
/* ----------------------------------------------------------------
* transaction state accessors
* ----------------------------------------------------------------
bool isSubXact = (s->parent != NULL);
ResourceOwner currentOwner;
+ if (RecoveryInProgress())
+ elog(ERROR, "cannot assign TransactionIds during recovery");
+
/* Assert that caller didn't screw up */
Assert(!TransactionIdIsValid(s->transactionId));
Assert(s->state == TRANS_INPROGRESS);
}
PG_END_TRY();
CurrentResourceOwner = currentOwner;
-}
+ /*
+ * Every PGPROC_MAX_CACHED_SUBXIDS assigned transaction ids within each
+ * top-level transaction we issue a WAL record for the assignment. We
+ * include the top-level xid and all the subxids that have not yet been
+ * reported using XLOG_XACT_ASSIGNMENT records.
+ *
+ * This required to limit ensure snaphots taken during recovery do not
+ * overflow. See notes for RecordKnownAssignedTransactionIds().
+ *
+ * We don't actually keep track of the immediate parent of each subxid,
+ * only the top-level transaction that each subxact belongs to. This
+ * is correct in recovery only because aborted subtransactions are
+ * separately WAL logged.
+ */
+ if (isSubXact && XLogArchivingActive())
+ {
+ int nchildren;
+ TransactionId *children;
+ children = xactGetUnreportedChildren(PGPROC_MAX_CACHED_SUBXIDS, &nchildren);
+
+ if (children != NULL)
+ {
+ XLogRecData rdata[2];
+ xl_xact_assignment xlrec;
+
+ /*
+ * We say "IfAny" to avoid recursion again here.
+ */
+ xlrec.xtop = GetTopTransactionIdIfAny();
+ Assert(TransactionIdIsValid(xlrec.xtop));
+
+ xlrec.nsubxacts = nchildren;
+
+ rdata[0].data = (char *) (&xlrec);
+ rdata[0].len = MinSizeOfXactAssignment;
+ rdata[0].buffer = InvalidBuffer;
+ rdata[0].next = &rdata[1];
+
+ rdata[1].data = (char *) children;
+ rdata[1].len = sizeof(TransactionId) * nchildren;
+ rdata[1].buffer = InvalidBuffer;
+ rdata[1].next = NULL;
+
+ (void) XLogInsert(RM_XACT_ID, XLOG_XACT_ASSIGNMENT, rdata);
+ }
+ }
+}
/*
* GetCurrentSubTransactionId
return false;
}
+/*
+ * TransactionStartedDuringRecovery, used during index scans
+ */
+bool
+TransactionStartedDuringRecovery(void)
+{
+ TransactionState s = CurrentTransactionState;
+
+ return s->startedInRecovery;
+}
/*
* CommandCounterIncrement
* This is exported only to support an ugly hack in VACUUM FULL.
*/
TransactionId
-RecordTransactionCommit(void)
+RecordTransactionCommit(bool isVacuumFull)
{
TransactionId xid = GetTopTransactionIdIfAny();
bool markXidCommitted = TransactionIdIsValid(xid);
bool haveNonTemp;
int nchildren;
TransactionId *children;
+ int nmsgs;
+ SharedInvalidationMessage *invalMessages = NULL;
+ bool RelcacheInitFileInval;
/* Get data needed for commit record */
nrels = smgrGetPendingDeletes(true, &rels, &haveNonTemp);
nchildren = xactGetCommittedChildren(&children);
-
+ nmsgs = xactGetCommittedInvalidationMessages(&invalMessages,
+ &RelcacheInitFileInval);
/*
* If we haven't been assigned an XID yet, we neither can, nor do we want
* to write a COMMIT record.
/*
* Begin commit critical section and insert the commit XLOG record.
*/
- XLogRecData rdata[3];
+ XLogRecData rdata[4];
int lastrdata = 0;
xl_xact_commit xlrec;
/* Tell bufmgr and smgr to prepare for commit */
BufmgrCommit();
+ /*
+ * Set flags required for recovery processing of commits.
+ * Nothing too important here that we would want to include this
+ * within the critical section following.
+ */
+ xlrec.xinfo = 0;
+ if (RelcacheInitFileInval)
+ xlrec.xinfo |= XACT_COMPLETION_UPDATE_RELCACHE_FILE;
+ if (isVacuumFull)
+ xlrec.xinfo |= XACT_COMPLETION_VACUUM_FULL;
+
/*
* Mark ourselves as within our "commit critical section". This
* forces any concurrent checkpoint to wait until we've updated
xlrec.xact_time = xactStopTimestamp;
xlrec.nrels = nrels;
xlrec.nsubxacts = nchildren;
+ xlrec.nmsgs = nmsgs;
+
rdata[0].data = (char *) (&xlrec);
rdata[0].len = MinSizeOfXactCommit;
rdata[0].buffer = InvalidBuffer;
rdata[2].buffer = InvalidBuffer;
lastrdata = 2;
}
+ /* dump shared cache invalidation messages */
+ if (nmsgs > 0)
+ {
+ rdata[lastrdata].next = &(rdata[3]);
+ rdata[3].data = (char *) invalMessages;
+ rdata[3].len = nmsgs * sizeof(SharedInvalidationMessage);
+ rdata[3].buffer = InvalidBuffer;
+ lastrdata = 3;
+ }
rdata[lastrdata].next = NULL;
(void) XLogInsert(RM_XACT_ID, XLOG_XACT_COMMIT, rdata);
s->childXids = NULL;
s->nChildXids = 0;
s->maxChildXids = 0;
+ s->nReportedChildXids = 0;
+ s->reportedXid = false;
}
/* ----------------------------------------------------------------
s->childXids = NULL;
s->nChildXids = 0;
s->maxChildXids = 0;
+ s->nReportedChildXids = 0;
+ s->reportedXid = false;
}
/* ----------------------------------------------------------------
s->gucNestLevel = 1;
s->childXids = NULL;
s->nChildXids = 0;
+ s->nReportedChildXids = 0;
+ s->reportedXid = false;
s->maxChildXids = 0;
+ s->startedInRecovery = RecoveryInProgress();
GetUserIdAndContext(&s->prevUser, &s->prevSecDefCxt);
/* SecurityDefinerContext should never be set outside a transaction */
Assert(!s->prevSecDefCxt);
/*
* Here is where we really truly commit.
*/
- latestXid = RecordTransactionCommit();
+ latestXid = RecordTransactionCommit(false);
TRACE_POSTGRESQL_TRANSACTION_COMMIT(MyProc->lxid);
s->childXids = NULL;
s->nChildXids = 0;
s->maxChildXids = 0;
+ s->nReportedChildXids = 0;
+ s->reportedXid = false;
/*
* done with commit processing, set current transaction state back to
s->childXids = NULL;
s->nChildXids = 0;
s->maxChildXids = 0;
+ s->nReportedChildXids = 0;
+ s->reportedXid = false;
/*
* done with 1st phase commit processing, set current transaction state
s->childXids = NULL;
s->nChildXids = 0;
s->maxChildXids = 0;
+ s->nReportedChildXids = 0;
+ s->reportedXid = false;
/*
* done with abort processing, set current transaction state back to
return s->nChildXids;
}
+static TransactionId *
+xactGetUnreportedChildren(int threshold, int *nxids)
+{
+ TransactionState s;
+ int nTotalUnreportedXids = 0;
+ TransactionId *xids;
+
+ /* Count unreported xids in the tree */
+ for (s = CurrentTransactionState; s != NULL; s = s->parent)
+ {
+ if (!s->reportedXid)
+ nTotalUnreportedXids++;
+ nTotalUnreportedXids += s->nChildXids - s->nReportedChildXids;
+ if (s->reportedXid)
+ break;
+ }
+
+ *nxids = nTotalUnreportedXids;
+
+ if (nTotalUnreportedXids < threshold)
+ return NULL;
+
+ xids = (TransactionId *) palloc(sizeof(TransactionId) * nTotalUnreportedXids);
+ xactCollectUnreportedChildren(CurrentTransactionState, xids);
+ return xids;
+}
+
+/* Helper function for xactGetUnreportedChildren */
+static TransactionId *
+xactCollectUnreportedChildren(TransactionState s, TransactionId *xids)
+{
+ int nUnreportedChildXids;
+
+ if (s->parent != NULL)
+ {
+ xids = xactCollectUnreportedChildren(s->parent, xids);
+ if (!s->reportedXid)
+ {
+ s->reportedXid = true;
+ *(xids++) = s->transactionId;
+ }
+ }
+
+ nUnreportedChildXids = s->nChildXids - s->nReportedChildXids;
+ memcpy(xids, &s->childXids[s->nReportedChildXids],
+ nUnreportedChildXids * sizeof(TransactionId));
+ xids += nUnreportedChildXids;
+
+ s->nReportedChildXids = s->nChildXids;
+
+ return xids;
+}
+
+/*
+ * Record an enhanced snapshot of running transactions into WAL.
+ *
+ * The definitions of RunningTransactionData and xl_xact_running_xacts
+ * are similar. We keep them separate because xl_xact_running_xacts
+ * is a contiguous chunk of memory and never exists fully until it is
+ * assembled in WAL.
+ */
+XLogRecPtr
+LogCurrentRunningXacts(RunningTransactions CurrRunningXacts)
+{
+ xl_xact_running_xacts xlrec;
+ XLogRecData rdata[4];
+ int lastrdata = 0;
+
+ xlrec.xcnt = CurrRunningXacts->xcnt;
+ xlrec.subxcnt = CurrRunningXacts->subxcnt;
+ xlrec.numLocks = CurrRunningXacts->numLocks;
+ xlrec.lock_overflow = CurrRunningXacts->lock_overflow;
+ xlrec.subxid_overflow = CurrRunningXacts->subxid_overflow;
+ xlrec.latestRunningXid = CurrRunningXacts->latestRunningXid;
+ xlrec.oldestRunningXid = CurrRunningXacts->oldestRunningXid;
+ xlrec.latestCompletedXid = CurrRunningXacts->latestCompletedXid;
+
+ /* Header */
+ rdata[0].data = (char *) (&xlrec);
+ rdata[0].len = MinSizeOfXactRunningXacts;
+ rdata[0].buffer = InvalidBuffer;
+
+ /* array of RunningXact */
+ if (xlrec.xcnt > 0)
+ {
+ rdata[0].next = &(rdata[1]);
+ rdata[1].data = (char *) CurrRunningXacts->xrun;
+ rdata[1].len = xlrec.xcnt * sizeof(RunningXact);
+ rdata[1].buffer = InvalidBuffer;
+ lastrdata = 1;
+ }
+
+ /* array of TransactionIds */
+ if (xlrec.subxcnt > 0)
+ {
+ rdata[lastrdata].next = &(rdata[2]);
+ rdata[2].data = (char *) CurrRunningXacts->subxip;
+ rdata[2].len = xlrec.subxcnt * sizeof(TransactionId);
+ rdata[2].buffer = InvalidBuffer;
+ lastrdata = 2;
+ }
+
+ /* array of Locks */
+ if (xlrec.numLocks > 0)
+ {
+ rdata[lastrdata].next = &(rdata[3]);
+ rdata[3].data = (char *) CurrRunningXacts->loggableLocks;
+ rdata[3].len = xlrec.numLocks * sizeof(xl_rel_lock);
+ rdata[3].buffer = InvalidBuffer;
+ lastrdata = 3;
+ }
+
+ rdata[lastrdata].next = NULL;
+
+ return XLogInsert(RM_XACT_ID, XLOG_XACT_RUNNING_XACTS, rdata);
+}
+
+/*
+ * We need to issue shared invalidations and hold locks. Holding locks
+ * means others may want to wait on us, so we need to make lock table
+ * inserts to appear like a transaction. We could create and delete
+ * lock table entries for each transaction but its simpler just to create
+ * one permanent entry and leave it there all the time. Locks are then
+ * acquired and released as needed. Yes, this means you can see the
+ * Startup process in pg_locks once we have run this.
+ */
+void
+InitRecoveryTransactionEnvironment(void)
+{
+ VirtualTransactionId vxid;
+
+ /*
+ * Initialise shared invalidation management for Startup process,
+ * being careful to register ourselves as a sendOnly process so
+ * we don't need to read messages, nor will we get signalled
+ * when the queue starts filling up.
+ */
+ SharedInvalBackendInit(true);
+
+ /*
+ * Record the PID and PGPROC structure of the startup process.
+ */
+ PublishStartupProcessInformation();
+
+ /*
+ * Lock a virtual transaction id for Startup process.
+ *
+ * We need to do GetNextLocalTransactionId() because
+ * SharedInvalBackendInit() leaves localTransactionid invalid and
+ * the lock manager doesn't like that at all.
+ *
+ * Note that we don't need to run XactLockTableInsert() because nobody
+ * needs to wait on xids. That sounds a little strange, but table locks
+ * are held by vxids and row level locks are held by xids. All queries
+ * hold AccessShareLocks so never block while we write or lock new rows.
+ */
+ vxid.backendId = MyBackendId;
+ vxid.localTransactionId = GetNextLocalTransactionId();
+ VirtualXactLockTableInsert(vxid);
+}
+
+void
+XactClearRecoveryTransactions(void)
+{
+ /*
+ * Remove entries from shared data structures
+ */
+ ExpireAllKnownAssignedTransactionIds();
+ RelationReleaseAllRecoveryLocks();
+}
+
+/*
+ * LatestRemovedXidAdvances - returns true if latestRemovedXid is moved
+ * forwards by the latest provided value
+ */
+bool
+LatestRemovedXidAdvances(TransactionId latestXid)
+{
+ /*
+ * Don't bother checking for conflicts for cleanup records earlier than
+ * we have already tested for.
+ */
+ if (!TransactionIdIsValid(latestRemovedXid) ||
+ (TransactionIdIsValid(latestRemovedXid) &&
+ TransactionIdPrecedes(latestRemovedXid, latestXid)))
+ {
+ latestRemovedXid = latestXid;
+ return true;
+ }
+
+ return false;
+}
+
/*
* XLOG support routines
*/
+/*
+ * Before 8.5 this was a fairly short function, but now it performs many
+ * actions for which the order of execution is critical.
+ */
static void
-xact_redo_commit(xl_xact_commit *xlrec, TransactionId xid)
+xact_redo_commit(xl_xact_commit *xlrec, TransactionId xid,
+ XLogRecPtr lsn, bool preparedXact)
{
TransactionId *sub_xids;
TransactionId max_xid;
int i;
- /* Mark the transaction committed in pg_clog */
sub_xids = (TransactionId *) &(xlrec->xnodes[xlrec->nrels]);
- TransactionIdCommitTree(xid, xlrec->nsubxacts, sub_xids);
- /* Make sure nextXid is beyond any XID mentioned in the record */
- max_xid = xid;
- for (i = 0; i < xlrec->nsubxacts; i++)
+ max_xid = TransactionIdLatest(xid, xlrec->nsubxacts, sub_xids);
+
+ /*
+ * Mark the transaction committed in pg_clog.
+ */
+ if (!InHotStandby)
+ TransactionIdCommitTree(xid, xlrec->nsubxacts, sub_xids);
+ else
{
- if (TransactionIdPrecedes(max_xid, sub_xids[i]))
- max_xid = sub_xids[i];
+ /*
+ * Just when you thought it was safe to go swimming again,
+ * along comes a nasty hack with bells on. Half way through
+ * VACUUM FULL it emits a false commit record, so it ends up
+ * emitting two commit records with the same xid. Oh, and it
+ * musn't release locks at the first commit either. So we
+ * have to specially mark the commit record "ignore me".
+ * On primary it actually marks clog committed yet stays
+ * visible in procarray. Cthulhu fhtagn. Run away screaming.
+ */
+ if (XactCompletionVacuumFull(xlrec))
+ {
+ elog(trace_recovery(DEBUG4), "skipping VACUUM FULL pseudo-commit %u", xid);
+ return;
+ }
+
+ /*
+ * Record any newly known assigned transactions. This looks
+ * strange to add xids and then immediately remove them, but
+ * we do other important processing here also do don't remove
+ * them (again).
+ */
+ RecordKnownAssignedTransactionIds(max_xid);
+
+ /*
+ * Mark the transaction committed in pg_clog. We use async commit
+ * protocol during recovery to provide information on database
+ * consistency for when users try to set hint bits. It is important
+ * that we do not set hint bits until the minRecoveryPoint is past
+ * this commit record. This ensures that if we crash we don't see
+ * hint bits set on changes made by transactions that haven't yet
+ * recovered. It's unlikely but it's good to be safe.
+ */
+ TransactionIdAsyncCommitTree(xid, xlrec->nsubxacts, sub_xids, lsn);
+
+ /*
+ * We must mark clog before we update the ProcArray.
+ */
+ ExpireTreeKnownAssignedTransactionIds(xid, xlrec->nsubxacts, sub_xids,
+ max_xid, false);
+
+ if (preparedXact)
+ {
+ /*
+ * Commit prepared xlog records do not carry invalidation data,
+ * since this is already held within the two phase state file.
+ * So we read it from there instead, with much the same effects.
+ */
+ ProcessTwoPhaseStandbyRecords(xid);
+ }
+ else
+ {
+ /*
+ * Send any cache invalidations attached to the commit. We must
+ * maintain the same order of invalidation then release locks
+ * as occurs in RecordTransactionCommit.
+ */
+ if (xlrec->nmsgs > 0)
+ {
+ int offset = OffsetSharedInvalInXactCommit();
+ SharedInvalidationMessage *msgs = (SharedInvalidationMessage *)
+ (((char *) xlrec) + offset);
+
+ /*
+ * Relcache init file invalidation requires processing both
+ * before and after we send the SI messages. See AtEOXact_Inval()
+ */
+ if (XactCompletionRelcacheInitFileInval(xlrec))
+ RelationCacheInitFileInvalidate(true);
+
+ SendSharedInvalidMessages(msgs, xlrec->nmsgs);
+
+ if (XactCompletionRelcacheInitFileInval(xlrec))
+ RelationCacheInitFileInvalidate(false);
+ }
+ }
+
+ /*
+ * Release locks, if any. We do this for both two phase and normal
+ * one phase transactions. In effect we are ignoring the prepare
+ * phase and just going straight to lock release. This explains
+ * why the twophase_postcommit_standby_callbacks[] do not invoke
+ * a special routine to handle locks - that is performed here
+ * instead.
+ */
+ RelationReleaseRecoveryLockTree(xid, xlrec->nsubxacts, sub_xids);
}
+
+ /* Make sure nextXid is beyond any XID mentioned in the record */
+ /* We don't expect anyone else to modify nextXid, hence we
+ * don't need to hold a lock while checking this. We still acquire
+ * the lock to modify it, though.
+ */
if (TransactionIdFollowsOrEquals(max_xid,
ShmemVariableCache->nextXid))
{
+ LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
ShmemVariableCache->nextXid = max_xid;
TransactionIdAdvance(ShmemVariableCache->nextXid);
+ LWLockRelease(XidGenLock);
+ }
+
+ /* Same here, don't use lock to test, but need one to modify */
+ if (TransactionIdFollowsOrEquals(max_xid,
+ ShmemVariableCache->latestCompletedXid))
+ {
+ LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE);
+ ShmemVariableCache->latestCompletedXid = max_xid;
+ LWLockRelease(ProcArrayLock);
}
/* Make sure files supposed to be dropped are dropped */
}
}
+/*
+ * Be careful with the order of execution, as with xact_redo_commit().
+ * The two functions are similar but differ in key places.
+ *
+ * Note also that an abort can be for a subtransaction and its children,
+ * not just for a top level abort. That means we have to consider
+ * topxid != xid, whereas in commit we would find topxid == xid always
+ * because subtransaction commit is never WAL logged.
+ */
static void
xact_redo_abort(xl_xact_abort *xlrec, TransactionId xid)
{
TransactionId max_xid;
int i;
- /* Mark the transaction aborted in pg_clog */
sub_xids = (TransactionId *) &(xlrec->xnodes[xlrec->nrels]);
+ max_xid = TransactionIdLatest(xid, xlrec->nsubxacts, sub_xids);
+
+ /* Mark the transaction aborted in pg_clog, no need for async stuff */
TransactionIdAbortTree(xid, xlrec->nsubxacts, sub_xids);
- /* Make sure nextXid is beyond any XID mentioned in the record */
- max_xid = xid;
- for (i = 0; i < xlrec->nsubxacts; i++)
+ if (InHotStandby)
{
- if (TransactionIdPrecedes(max_xid, sub_xids[i]))
- max_xid = sub_xids[i];
+ /*
+ * Record any newly known assigned transactions. This looks
+ * strange to add xids and then immediately remove them, but
+ * we do other important processing here also do don't remove
+ * them (again).
+ */
+ RecordKnownAssignedTransactionIds(max_xid);
+
+ /*
+ * We must mark clog before we update the ProcArray.
+ */
+ ExpireTreeKnownAssignedTransactionIds(xid, xlrec->nsubxacts, sub_xids,
+ max_xid, false);
+
+ /*
+ * There are no flat files that need updating, nor invalidation
+ * messages to send or undo.
+ */
+
+ /*
+ * Release locks, if any. There are no invalidations to send.
+ */
+ RelationReleaseRecoveryLockTree(xid, xlrec->nsubxacts, sub_xids);
}
+
+ /* Make sure nextXid is beyond any XID mentioned in the record */
if (TransactionIdFollowsOrEquals(max_xid,
ShmemVariableCache->nextXid))
{
ShmemVariableCache->nextXid = max_xid;
+ ShmemVariableCache->latestCompletedXid = ShmemVariableCache->nextXid;
TransactionIdAdvance(ShmemVariableCache->nextXid);
}
}
}
+static void
+xact_redo_assignment(XLogRecPtr lsn, xl_xact_assignment *xlrec)
+{
+ TransactionId max_xid;
+ int i;
+
+ Assert(InHotStandby);
+
+ /*
+ * Notice that we update pg_subtrans with the top-level xid, rather
+ * than the parent xid. This is a difference between normal
+ * processing and recovery, yet is still correct in all cases. The
+ * reason is that subtransaction commit is not marked in clog until
+ * commit processing, so all aborted subtransactions have already been
+ * clearly marked in clog. As a result we are able to refer directly
+ * to the top-level transaction's state rather than skipping through
+ * all the intermediate states in the subtransaction tree.
+ */
+ for (i = 0; i < xlrec->nsubxacts; i++)
+ {
+ TransactionId subxid = xlrec->xsub[i];
+
+ ExtendSUBTRANS(subxid);
+ SubTransSetParent(subxid, xlrec->xtop);
+ }
+
+ max_xid = TransactionIdLatest(xlrec->xtop, xlrec->nsubxacts, xlrec->xsub);
+
+ /*
+ * Remove the subxids from the array, which must occur after we have
+ * set their parents correctly in subtrans. Record overflowed state.
+ */
+ ExpireTreeKnownAssignedTransactionIds(InvalidTransactionId,
+ xlrec->nsubxacts, xlrec->xsub,
+ max_xid, true);
+}
+
void
xact_redo(XLogRecPtr lsn, XLogRecord *record)
{
{
xl_xact_commit *xlrec = (xl_xact_commit *) XLogRecGetData(record);
- xact_redo_commit(xlrec, record->xl_xid);
+ xact_redo_commit(xlrec, record->xl_xid, lsn, false);
}
else if (info == XLOG_XACT_ABORT)
{
}
else if (info == XLOG_XACT_PREPARE)
{
+ if (InHotStandby)
+ RecordKnownAssignedTransactionIds(record->xl_xid);
+
/* the record contents are exactly the 2PC file */
RecreateTwoPhaseFile(record->xl_xid,
XLogRecGetData(record), record->xl_len);
{
xl_xact_commit_prepared *xlrec = (xl_xact_commit_prepared *) XLogRecGetData(record);
- xact_redo_commit(&xlrec->crec, xlrec->xid);
+ xact_redo_commit(&xlrec->crec, xlrec->xid, lsn, true);
RemoveTwoPhaseFile(xlrec->xid, false);
}
else if (info == XLOG_XACT_ABORT_PREPARED)
xact_redo_abort(&xlrec->arec, xlrec->xid);
RemoveTwoPhaseFile(xlrec->xid, false);
}
+ else if (info == XLOG_XACT_ASSIGNMENT)
+ {
+ xl_xact_assignment *xlrec = (xl_xact_assignment *) XLogRecGetData(record);
+
+ if (InHotStandby)
+ xact_redo_assignment(lsn, xlrec);
+ }
+ else if (info == XLOG_XACT_RUNNING_XACTS)
+ {
+ xl_xact_running_xacts *xlrec = (xl_xact_running_xacts *) XLogRecGetData(record);
+
+ if (InHotStandby)
+ ProcArrayApplyRecoveryInfo(lsn, xlrec);
+ }
else
elog(PANIC, "xact_redo: unknown op code %u", info);
}
{
int i;
+ if (XactCompletionRelcacheInitFileInval(xlrec))
+ appendStringInfo(buf, "; relcache init file inval");
+
appendStringInfoString(buf, timestamptz_to_str(xlrec->xact_time));
if (xlrec->nrels > 0)
{
- appendStringInfo(buf, "; rels:");
+ appendStringInfo(buf, "; %d rels:", xlrec->nrels);
for (i = 0; i < xlrec->nrels; i++)
{
char *path = relpath(xlrec->xnodes[i], MAIN_FORKNUM);
if (xlrec->nsubxacts > 0)
{
TransactionId *xacts = (TransactionId *)
- &xlrec->xnodes[xlrec->nrels];
-
- appendStringInfo(buf, "; subxacts:");
+ &xlrec->xnodes[xlrec->nrels];
+ appendStringInfo(buf, "; %d subxacts:", xlrec->nsubxacts);
for (i = 0; i < xlrec->nsubxacts; i++)
appendStringInfo(buf, " %u", xacts[i]);
}
+ if (xlrec->nmsgs > 0)
+ {
+ /*
+ * The invalidation messages are the third variable length array
+ * from the start of the record. The record header has everything
+ * we need to calculate where that starts.
+ */
+ int offset = OffsetSharedInvalInXactCommit();
+ SharedInvalidationMessage *msgs = (SharedInvalidationMessage *)
+ (((char *) xlrec) + offset);
+ appendStringInfo(buf, "; %d inval msgs:", xlrec->nmsgs);
+ for (i = 0; i < xlrec->nmsgs; i++)
+ {
+ SharedInvalidationMessage *msg = msgs + i;
+
+ if (msg->id >= 0)
+ appendStringInfo(buf, "catcache id%d ", msg->id);
+ else if (msg->id == SHAREDINVALRELCACHE_ID)
+ appendStringInfo(buf, "relcache ");
+ else if (msg->id == SHAREDINVALSMGR_ID)
+ appendStringInfo(buf, "smgr ");
+ }
+ }
}
static void
if (xlrec->nsubxacts > 0)
{
TransactionId *xacts = (TransactionId *)
- &xlrec->xnodes[xlrec->nrels];
+ &xlrec->xnodes[xlrec->nrels];
- appendStringInfo(buf, "; subxacts:");
+ appendStringInfo(buf, "; %d subxacts:", xlrec->nsubxacts);
for (i = 0; i < xlrec->nsubxacts; i++)
appendStringInfo(buf, " %u", xacts[i]);
}
}
+static void
+xact_desc_assignment(StringInfo buf, xl_xact_assignment *xlrec)
+{
+ int i;
+
+ appendStringInfo(buf, " nsubxids %u;", xlrec->nsubxacts);
+
+ for (i = 0; i < xlrec->nsubxacts; i++)
+ appendStringInfo(buf, " %u", xlrec->xsub[i]);
+}
+
+static void
+xact_desc_running_xacts(StringInfo buf, xl_xact_running_xacts *xlrec)
+{
+ int xid_index,
+ subxid_index;
+ TransactionId *subxip = (TransactionId *) &(xlrec->xrun[xlrec->xcnt]);
+
+ appendStringInfo(buf, "nxids %u nsubxids %u latestRunningXid %u",
+ xlrec->xcnt,
+ xlrec->subxcnt,
+ xlrec->latestRunningXid);
+
+ appendStringInfo(buf, " oldestRunningXid %u latestCompletedXid %u",
+ xlrec->oldestRunningXid,
+ xlrec->latestCompletedXid);
+
+ for (xid_index = 0; xid_index < xlrec->xcnt; xid_index++)
+ {
+ RunningXact *rxact = (RunningXact *) xlrec->xrun;
+
+ appendStringInfo(buf, " %u", rxact[xid_index].xid);
+
+ if (rxact[xid_index].nsubxids > 0)
+ {
+ appendStringInfo(buf, "; nsubxids %u offset %u ovflow? %s",
+ rxact[xid_index].nsubxids,
+ rxact[xid_index].subx_offset,
+ (rxact[xid_index].overflowed ? "t" : "f"));
+
+ appendStringInfo(buf, "; subxacts: ");
+ for (subxid_index = 0; subxid_index < rxact[xid_index].nsubxids; subxid_index++)
+ appendStringInfo(buf, " %u",
+ subxip[subxid_index + rxact[xid_index].subx_offset]);
+ }
+ }
+}
+
void
xact_desc(StringInfo buf, uint8 xl_info, char *rec)
{
{
xl_xact_commit_prepared *xlrec = (xl_xact_commit_prepared *) rec;
- appendStringInfo(buf, "commit %u: ", xlrec->xid);
+ appendStringInfo(buf, "commit prepared %u: ", xlrec->xid);
xact_desc_commit(buf, &xlrec->crec);
}
else if (info == XLOG_XACT_ABORT_PREPARED)
{
xl_xact_abort_prepared *xlrec = (xl_xact_abort_prepared *) rec;
- appendStringInfo(buf, "abort %u: ", xlrec->xid);
+ appendStringInfo(buf, "abort prepared %u: ", xlrec->xid);
xact_desc_abort(buf, &xlrec->arec);
}
+ else if (info == XLOG_XACT_ASSIGNMENT)
+ {
+ xl_xact_assignment *xlrec = (xl_xact_assignment *) rec;
+
+ /*
+ * Note that we ignore the WAL record's xid, since we're more
+ * interested in the top-level xid that issued the record
+ * and which xids are being reported here.
+ */
+ appendStringInfo(buf, "xid assignment xtop %u", xlrec->xtop);
+ xact_desc_assignment(buf, xlrec);
+ }
+ else if (info == XLOG_XACT_RUNNING_XACTS)
+ {
+ xl_xact_running_xacts *xlrec = (xl_xact_running_xacts *) rec;
+
+ appendStringInfo(buf, "running xacts: ");
+ xact_desc_running_xacts(buf, xlrec);
+ }
else
appendStringInfo(buf, "UNKNOWN");
}
#include "access/clog.h"
#include "access/multixact.h"
+#include "access/nbtree.h"
#include "access/subtrans.h"
#include "access/transam.h"
#include "access/tuptoaster.h"
#include "storage/ipc.h"
#include "storage/pmsignal.h"
#include "storage/procarray.h"
+#include "storage/sinval.h"
#include "storage/smgr.h"
#include "storage/spin.h"
#include "utils/builtins.h"
#include "utils/ps_status.h"
#include "pg_trace.h"
-
/* File path names (all relative to $PGDATA) */
#define BACKUP_LABEL_FILE "backup_label"
#define BACKUP_LABEL_OLD "backup_label.old"
#define RECOVERY_COMMAND_FILE "recovery.conf"
#define RECOVERY_COMMAND_DONE "recovery.done"
+/* copied from tcopprot.h rather than include whole file */
+extern int PostAuthDelay;
/* User-settable parameters */
int CheckPointSegments = 3;
*/
bool InRecovery = false;
+static XLogRecPtr LastRec;
+
/*
* Local copy of SharedRecoveryInProgress variable. True actually means "not
* known, need to check the shared state".
static int LocalXLogInsertAllowed = -1;
/* Are we recovering using offline XLOG archives? */
-static bool InArchiveRecovery = false;
+bool InArchiveRecovery = false;
/* Was the last xlog file restored from archive, or local? */
static bool restoredFromArchive = false;
/* options taken from recovery.conf */
static char *recoveryRestoreCommand = NULL;
static char *recoveryEndCommand = NULL;
-static bool recoveryTarget = false;
static bool recoveryTargetExact = false;
static bool recoveryTargetInclusive = true;
static TransactionId recoveryTargetXid;
static TimestampTz recoveryTargetTime;
+static XLogRecPtr recoveryTargetLSN;
+static int recoveryTargetAdvance = 0;
+
+/*
+ * InHotStandby is an optional sub-state of InArchiveRecovery
+ * so is only ever set in the Startup process. Recovery connections
+ * will not be enabled until a valid RunningXact record has arrived.
+ */
+bool InHotStandby = true;
+
+/* recovery target modes */
+#define RECOVERY_TARGET_NONE 0
+#define RECOVERY_TARGET_PAUSE_ALL 1
+#define RECOVERY_TARGET_PAUSE_XID 2
+#define RECOVERY_TARGET_PAUSE_TIME 3
+#define RECOVERY_TARGET_PAUSE_LSN 4
+#define RECOVERY_TARGET_ADVANCE 5
+#define RECOVERY_TARGET_STOP_IMMEDIATE 6
+#define RECOVERY_TARGET_STOP_XID 7
+#define RECOVERY_TARGET_STOP_TIME 8
+static int recoveryTargetMode = RECOVERY_TARGET_NONE;
+static bool recoveryStartsPaused = false;
+
+#define DEFAULT_MAX_STANDBY_DELAY 30
+int maxStandbyDelay = DEFAULT_MAX_STANDBY_DELAY; /* initial setting, seconds */
+
static TimestampTz recoveryLastXTime = 0;
+static TransactionId recoveryLastXid = InvalidTransactionId;
/* if recoveryStopsHere returns true, it saves actual stop xid/time here */
static TransactionId recoveryStopXid;
/* end+1 of the last record replayed (or being replayed) */
XLogRecPtr replayEndRecPtr;
+ int recoveryTargetMode;
+ TransactionId recoveryTargetXid;
+ TimestampTz recoveryTargetTime;
+ int recoveryTargetAdvance;
+ XLogRecPtr recoveryTargetLSN;
+
+ TimestampTz recoveryLastXTime;
+ TransactionId recoveryLastXid;
+ XLogRecPtr recoveryLastRecPtr;
+ int maxStandbyDelay;
+
slock_t info_lck; /* locks shared variables shown above */
} XLogCtlData;
static void exitArchiveRecovery(TimeLineID endTLI,
uint32 endLogId, uint32 endLogSeg);
static bool recoveryStopsHere(XLogRecord *record, bool *includeThis);
+static void recoveryPausesAfterLSN(void);
+static void RequiresWALControlPermissions(void);
+static void SetRecoveryTargetMode(int mode, TransactionId xid, TimestampTz ts,
+ XLogRecPtr lsn, int advance);
+static void SetMaxStandbyDelay(int delay);
+static void CheckMaxConnections(int maxcon);
static void LocalSetXLogInsertAllowed(void);
static void CheckPointGuts(XLogRecPtr checkPointRedo, int flags);
FIN_CRC32(rdata_crc);
record->xl_crc = rdata_crc;
-#ifdef WAL_DEBUG
- if (XLOG_DEBUG)
- {
- StringInfoData buf;
-
- initStringInfo(&buf);
- appendStringInfo(&buf, "INSERT @ %X/%X: ",
- RecPtr.xlogid, RecPtr.xrecoff);
- xlog_outrec(&buf, record);
- if (rdata->data != NULL)
- {
- appendStringInfo(&buf, " - ");
- RmgrTable[record->xl_rmid].rm_desc(&buf, record->xl_info, rdata->data);
- }
- elog(LOG, "%s", buf.data);
- pfree(buf.data);
- }
-#endif
-
/* Record begin of record in appropriate places */
ProcLastRecPtr = RecPtr;
Insert->PrevRecord = RecPtr;
return true;
}
+/*
+ * Test whether database is consistent up to the LSN requested, specifically
+ * to decide whether it is safe to SetHintBits().
+ *
+ * Works whether we are in recovery or not.
+ */
+bool
+DBConsistentUpToLSN(XLogRecPtr lsn)
+{
+ /*
+ * If we are operating normally, just check is WAL needs flushing.
+ */
+ if (!RecoveryInProgress())
+ return !XLogNeedsFlush(lsn);
+
+ /* Quick exit if already known flushed */
+ if (XLByteLE(lsn, minRecoveryPoint))
+ return true;
+
+ /*
+ * Don't try too hard to check consistency, it's not updated
+ * that often during recovery so this could easily become a
+ * contention hotspot with many users and many CPUs.
+ */
+ if (LWLockConditionalAcquire(ControlFileLock, LW_EXCLUSIVE))
+ {
+ /* update local copy */
+ minRecoveryPoint = ControlFile->minRecoveryPoint;
+
+ LWLockRelease(ControlFileLock);
+
+ /* check again */
+ if (XLByteLE(lsn, minRecoveryPoint))
+ return true;
+ }
+
+ return false;
+}
+
/*
* Create a new XLOG file segment, or open a pre-existing one.
*
ereport(LOG,
(errmsg("recovery_target_xid = %u",
recoveryTargetXid)));
- recoveryTarget = true;
+ recoveryTargetMode = RECOVERY_TARGET_STOP_XID;
recoveryTargetExact = true;
}
else if (strcmp(tok1, "recovery_target_time") == 0)
*/
if (recoveryTargetExact)
continue;
- recoveryTarget = true;
+ recoveryTargetMode = RECOVERY_TARGET_STOP_TIME;
recoveryTargetExact = false;
/*
ereport(LOG,
(errmsg("recovery_target_inclusive = %s", tok2)));
}
+ else if (strcmp(tok1, "recovery_connections") == 0)
+ {
+ /*
+ * enables/disables snapshot processing and user connections
+ */
+ if (!parse_bool(tok2, &InHotStandby))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("parameter \"recovery_connections\" requires a Boolean value")));
+ ereport(LOG,
+ (errmsg("recovery_connections = %s", tok2)));
+ }
+ else if (strcmp(tok1, "recovery_starts_paused") == 0)
+ &nb
projects / users / simon / postgres.git / commitdiff