Tomas Vondra [Thu, 19 Jan 2017 21:04:06 +0000 (22:04 +0100)]
make output of 'select' test suite stable by adding ORDER BY
In XL the order of rows may change depending on timing of responses
from remote nodes, unless the query has an explicit ORDER BY (on
PostgreSQL the regression tests often rely on reading the data from
disk in a single process).
Tomas Vondra [Thu, 19 Jan 2017 20:51:05 +0000 (21:51 +0100)]
remove 'missing redistribute_path' error from recurse_set_operations
I came to the conclusion that we probably don't need to redistribute
the path at this place. We're producing a generic subquery path with
particular distribution - if if a path with different distribution
is needed, it should be responsibility of the caller to do that.
The assumption is that there might be multiple possible plans, each
requiring a path with a different distribution.
Tomas Vondra [Thu, 19 Jan 2017 16:18:20 +0000 (17:18 +0100)]
obvious plan changes in inherit, aggregates, select and select_distinct
Update expected regression results in case of obvious plan changes,
as for example converting 2-phase aggregation to Final/Partial,
or wrapping the whole query into Remote Fast Query.
Tomas Vondra [Thu, 19 Jan 2017 14:47:01 +0000 (15:47 +0100)]
resolve most differences in insert_conflict regression test
The plan changes are either due to parallel query, or due to adding
a RemoteSubplan node at the top of the plan.
Tomas Vondra [Thu, 19 Jan 2017 14:30:06 +0000 (15:30 +0100)]
resolve most differences in create_index test suite
Most of the differences were trivial consequence of using partial
aggregates, instead of the XL-specific 2-phase aggregate. The
main difference is that that EXPLAIN now prefixes the Aggregate
nodes with either "Finalize" or "Partial" (while before it used
just Aggregate).
There's one remaining plan change, but that also switches from
plain Index Scan to Bitmap Index Scan. That probably deserves
a bit more attention and further investigation.
Tomas Vondra [Thu, 19 Jan 2017 13:44:02 +0000 (14:44 +0100)]
resolve failures in box regression tests (missing SP-GiST part)
The expected output was missing the new section, testing SP-GiST
on a box column. Make sure the table with test data is replicated
to get stable output.
Tomas Vondra [Thu, 19 Jan 2017 13:41:17 +0000 (14:41 +0100)]
resolve failures in timestamp/timestamptz regression tests
Update the SQL and expected output for XL, and remove unnecessary
whitespace (mostly at the end of line) to make diffing easier.
Tomas Vondra [Thu, 19 Jan 2017 13:08:37 +0000 (14:08 +0100)]
use text argument instead of cstring in pg_msgmodule_set()
opr_sanity checks that functions with cstring arguments are either
type input or conversion functions. Tweaking the function to accept
text seems like a better idea than relaxing opr_sanity tests.
Tomas Vondra [Thu, 19 Jan 2017 12:18:45 +0000 (13:18 +0100)]
mark pg_msgmodule_disable_all() as PROVOLATILE_IMMUTABLE
The pg_proc entry had provolatile=0, which is not a legal value,
so it was triggering an error in opr_sanity regression. Mark the
function as PROVOLATILE_IMMUTABLE.
Tomas Vondra [Thu, 19 Jan 2017 09:40:01 +0000 (10:40 +0100)]
skip RenameSequenceGTM() for SET SCHEMA with the same schema
Commit
bc4996e6 in upstream changed how CheckSetNamespace() handles
moving to the same namespace, e.g. when executing
ALTER SEQUENCE s1.s SET SCHEMA s1;
On PostgreSQL 9.5 this fails with an error, but since
bc4996e6 the
command is silently ignored (which was already the case for other
object types - ALTER EXTENSION etc).
XL however relied on the ERROR interrupting the program flow before
the call to RenameSequenceGTM(), so after
bc4996e6 it was executed
anyway.
GTM however does not expect unnecessary sequence renames, and treats
this as an error in seq_add_seqinfo.
Tomas Vondra [Thu, 19 Jan 2017 07:56:07 +0000 (08:56 +0100)]
add name of the originating node to application_name
Until now the application_name was set to 'pgxc' for all connections.
Set it to 'pgxc:NODE_NAME' instead, to make it easier to understand
data from pg_stat_activity.
Tomas Vondra [Wed, 18 Jan 2017 15:56:12 +0000 (16:56 +0100)]
fix reading of :mergeCollations in _readMergeJoin()
Fixes error observed in many regression tests
ERROR: did not find '}' at end of input node
Tomas Vondra [Wed, 18 Jan 2017 15:29:15 +0000 (16:29 +0100)]
properly decide when Grouping Sets do not require redistribution
When adding a GroupingSetsPath, an explicit redistribution is not
needed when either (distribution=NULL) or for replicated tables.
This resolves the crash in 'groupingsets' regression set, due to
triggering this assert in make_remotesubplan():
Assert(!equal(resultDistribution, execDistribution))
Tomas Vondra [Wed, 18 Jan 2017 14:59:29 +0000 (15:59 +0100)]
do not use Append with redistributed childrels as a partial path
Currently the subpaths are redistributed in create_append_path,
but even though that sets path.parallel_safe=false, it's too late
for set_append_rel_pathlist() to realize that.
It simply grabs the first path from childrel->partial_pathlist and
builds an AppendPath on top of that, assuming the resulting Append
is also parallel_safe.
But if the child relations had to be redistributed by adding a
RemoteSubplan on each of them, then the Append is parallel unsafe
and the attempt to add Gather on top of that fails because of
tripping on Assert(subpath->parallel_safe).
Make sure the result of create_append_path is also parallel safe
before adding it as a partial path.
I'd be surprised if there were no other places relying on the same
assumption. Ultimately, we probably want to build the redistributed
paths before calling create_append_path().
Tomas Vondra [Wed, 18 Jan 2017 11:32:42 +0000 (12:32 +0100)]
remove an extra lappend(subroots, subroot) from inheritance_planner
This resolves Assert() failure in create_modifytable_path(), causing
failures in regression tests (inherit, updatable_views, alter_table,
returning).
Tomas Vondra [Tue, 17 Jan 2017 20:06:37 +0000 (21:06 +0100)]
resolve crashes in queries involving merge-sort in a RemoteSubplan
Tomas Vondra [Mon, 16 Jan 2017 18:18:05 +0000 (19:18 +0100)]
temporarily merge all tuplesort changes from REL9_6_STABLE
Tomas Vondra [Mon, 16 Jan 2017 00:09:22 +0000 (01:09 +0100)]
add T_GroupingFunc to pgxc_shippability_walker
Planning grouping sets queries with grouping() function failed with
ERROR: XX000: unrecognized node type: 307
LOCATION: pgxc_shippability_walker, pgxcship.c:1199
STATEMENT: select grouping(a, b), count(*) from t1
group by grouping sets ((a), (b));
We simply inspect arguments of the grouping() function, although
that probably is not necessary - it can only reference grouping
expressions, which are checked as part of other nodes.
Tomas Vondra [Sun, 15 Jan 2017 23:07:47 +0000 (00:07 +0100)]
fix grouping sets planning by removing XL changes from make_sort()
Commit
44fd89e213 enabled grouping sets, but planning of some queries
failed in fix_upper_expr. This was happening when the grouping sets
need multiple sort orders - in that case create_groupingsets_plan
builds additional Sort nodes (using make_sort_from_groupcols), but
the XL code in make_sort() got confused by this and ended up with
a RemoteSubplan with an empty targetlist.
Removing the XL optimization from make_sort() resolve the planning
issue, producing plans like this:
QUERY PLAN
---------------------------------------------
GroupAggregate
Group Key: a
Sort Key: b
Group Key: b
-> Remote Subquery Scan on all (dn1,dn2)
-> Sort
Sort Key: a
-> Seq Scan on t1
(8 rows)
Which seems correct.
Tomas Vondra [Sun, 15 Jan 2017 03:53:51 +0000 (04:53 +0100)]
set pathtarget and parallel_aware/safe flags in redistribute_path
This fixes segfault in 'boolean' regression test suite.
Tomas Vondra [Sun, 15 Jan 2017 03:07:11 +0000 (04:07 +0100)]
if the whole grouping can be pushed down, don't construct XL paths
When the whole aggregate can be pushed to remote nodes, don't bother
constructing distributed 2-phase aggregate paths, because we can't
possibly beat the fully pushed-down path.
Tomas Vondra [Sun, 15 Jan 2017 02:59:49 +0000 (03:59 +0100)]
specify correct transtype for json_agg_transfn/json_agg_finalfn
Mistake in merge conflict resolution, which removed the custom XL
partial aggregate types, but failed to fix the aggregate itself.
Tomas Vondra [Sun, 15 Jan 2017 02:52:13 +0000 (03:52 +0100)]
fix 'cache lookup failed for function 0' errors during initdb
Commit
0e882bd2e02 broke planning for aggregates not supporting partial
aggregation (e.g. array_agg, which lacks the serialize, deserialize and
combine functions). The code attempted to initialize the costing info
anyway, which failed when attempting to fetch info about those functions
from syscache.
Introduce a new flag 'try_distributed_aggregation' akin to the existing
one for simple parallel case. This is a good idea anyway, because it'll
allow us to disable distributed paths if we happen to find out the whole
aggregate can be pushed down.
Tomas Vondra [Sat, 14 Jan 2017 23:22:32 +0000 (00:22 +0100)]
mark RemoteSubplan paths with parallel_safe=false and parallel_aware=false
We don't want to run parallel queries on the coordinators, particularly
when those queries query data nodes.
Tomas Vondra [Sat, 14 Jan 2017 23:04:06 +0000 (00:04 +0100)]
generate distributed grouping paths with a combine phase
Generate grouping paths with an extra aggregate step before the
remote subplan, looking like this:
Finalize GroupAggregate
-> Remote Subquery Scan
-> Combine Aggregate
-> Gather
-> Partial GroupAggregate
-> ...
The idea is that the combine phase may significantly reduce the
number of rows transferred to the coordinators (and processed on
them), possibly up to 1/N where N is the number of workers.
The plans don't seem to be really chosen at this point, most likely
because of incorrect costing that makes them look more expensive
than the paths without the combine step.
Tomas Vondra [Sat, 14 Jan 2017 19:51:55 +0000 (20:51 +0100)]
comment about generating grouping paths with extra combine phase
Tomas Vondra [Sat, 14 Jan 2017 18:56:35 +0000 (19:56 +0100)]
comment about cardinality estimates for 2-phase distributed aggregates
Tomas Vondra [Sat, 14 Jan 2017 16:24:19 +0000 (17:24 +0100)]
WIP: allow GROUPING SETS, ROLLUP and CUBE (without push-down)
This enables groupin sets path by removing the elog(ERROR, ...) from
transformGroupClause(). The paths are currently considered as not
eligible for push-down, irrespectedly of the distribution.
This could be improved by enabling pushdown if all the grouping sets
are compatible with the distribution (as if each grouping set was
a separate GROUP BY clause). So for example assuming a table 't'
distributed by column 'a', we could push down these grouping sets
GROUP BY GROUPING SETS ((a), (a, b), (a, c))
but not
GROUP BY GROUPING SETS ((b), (a, b), (a, c))
because the first grouping set does not contain the distribution key.
The grouping sets require sorted paths, so we get paths like this:
SELECT a, b, COUNT(c) FROM t GROUP BY GROUPING SETS ((a), (a, b));
QUERY PLAN
---------------------------------------------
GroupAggregate
Group Key: a, b
Group Key: a
-> Remote Subquery Scan on all (dn1,dn2)
-> Sort
Sort Key: a, b
-> Seq Scan on t2
(7 rows)
There's however a bug somewhere (likely in fix_upper_expr or elsewhere
in setrefs.c), resulting in failures like this:
SELECT a, b, COUNT(c) FROM t GROUP BY GROUPING SETS ((a), (b));
ERROR: variable not found in subplan target list
The only difference between the queries is removal of the distribution
key from the second grouping set.
Tomas Vondra [Sat, 14 Jan 2017 04:57:22 +0000 (05:57 +0100)]
generate distributed 2-phase grouping paths
Tomas Vondra [Sat, 14 Jan 2017 02:19:55 +0000 (03:19 +0100)]
fix output/read plans in _outAggref(), _readAggref() and _readAgg()
PostgreSQL 9.6 added new fields to Aggref for to partial aggregation,
and the functions were not reading them correctly.
Tomas Vondra [Sat, 14 Jan 2017 02:17:31 +0000 (03:17 +0100)]
make sure the distribution is propagated through grouping paths
When creating grouping paths (agg, group, gather, ...) the create_*_path
functions need to inherit the distribution from the subpath. Otherwise
we lose the information about the distribution and fail to create the
remote subplans at the top of the plan.
Tomas Vondra [Fri, 13 Jan 2017 23:37:07 +0000 (00:37 +0100)]
make grouping paths work again, without XL optimizations
Commit
d31927431b26af0c14f7a2abe6f2ee0af33f7b61 resolved some of the
conflicts in planner.c, caused by the pathification, by removing some
of the XL code. While this made the code compilable, it effectively
broken the aggregation on XL. This patch makes grouping paths work
again, in the sense that the generated paths should be correct and
return correct results.
The planner however does not generate advanced XL paths yet, in
particular it does not paths with 2-phase distributed aggregates
like this one:
Aggregate (2nd phase: combine and finalize)
-> Remote Subquery
-> Aggregate (1st phase: partial)
Thanks to the parallel aggregate, implemented in 9.6, the planner
however can generate queries like this:
Aggregate (2nd phase: combine and finalize)
-> Remote Subquery
-> Gather
-> Aggregate (1st phase: partial)
Tomas Vondra [Fri, 13 Jan 2017 07:33:30 +0000 (08:33 +0100)]
fix tuplesort_begin_merge() broken by rework of tuplesort in 9.6
The tuplesort got reworked quite a bit in 9.6, and this function
was subtly broken by the merge. In particular, it
* failed to set 'movetup' to valid function
* beginmerge() was always called with (finalMergeBatch=true)
* multiple arrays were left un-allocated
In total, this was causing segfaults when execRemote used the
merge sort.
Tomas Vondra [Fri, 13 Jan 2017 04:22:32 +0000 (05:22 +0100)]
introduce adjust_path_distribution(), adding RemoteSubplan as needed
This moves a huge chunk of code as a function, mostly for convenience.
Tomas Vondra [Fri, 13 Jan 2017 04:19:12 +0000 (05:19 +0100)]
rework grouping_distribution_match() to accept list of clauses
Accepting (Query*) is not sufficient, because we need to know which
of the clause lists to inspect (DISTINCT, GROUP BY, ...).
Note: Perhaps we should use PathKeys instead of those clause lists?
Tomas Vondra [Fri, 13 Jan 2017 04:15:31 +0000 (05:15 +0100)]
initialize state->getlen field in tuplesort_begin_cluster()
Not sure if this is an issue, but all other tuplesort_begin methods
do that, so presumably this one should do that too.
Tomas Vondra [Fri, 13 Jan 2017 03:59:50 +0000 (04:59 +0100)]
reintroduce XL logic into _readSort() to make Sort pushdown work
During merge, the XL portable_input infrastructure got removed from
_readSort(), resulting in errors like this
ERROR: did not find '}' at end of input node
when running queries with Sort push-down. Fixed by reintroducing
the coding from 9.5 branch, however this is hardly the last bug
in node functions.
Tomas Vondra [Thu, 12 Jan 2017 19:03:51 +0000 (20:03 +0100)]
grouping_distribution_match() and equal_distributions() comments
Add proper comments for the functions, used in planner.c.
Tomas Vondra [Thu, 12 Jan 2017 19:01:25 +0000 (20:01 +0100)]
remove grouping_distribution(), obsoleted by the pathification
The function was injecting Plan node (RemoteSubplan), but that
is incompatible with the pathified planner approach. The new
code uses grouping_distribution_match(), which only checks that
the grouping and distributions are compatible.
Tomas Vondra [Thu, 12 Jan 2017 11:25:38 +0000 (12:25 +0100)]
remove the extra & from start_postmaster() command in pg_ctl
The start_postmaster() function in pg_ctl constructed
snprintf(cmd, MAXPGPATH, "exec \"%s\" %s%s < \"%s\" 2>&1 &",
exec_path, pgdata_opt, post_opts, DEVNULL);
which resulted in extra fork thanks to the extra & at the end. Due to
this the 'pmpid' and 'pm_pid' in test_postmaster_connection() did not
match, resulting in failures like this:
pg_ctl: pg_ctl: could not start server
Tomas Vondra [Thu, 12 Jan 2017 09:47:38 +0000 (10:47 +0100)]
make pgxc_ctl connect to 'postgres' database when pinging a node
pingNode() was not specifying database in the connection string,
so the OS username was used implicitly. When such database did
not exist (e.g. when initializing new nodes, under OS account
different from 'postgres'), this resulted in FATAL failures
in the server log:
FATAL: database "user" does not exist
Adding 'database=postgres' to the connection string fixes the
issue. It might be useful to allow using custom database names,
but 'postgres' is pretty much guaranteed to exist.
Bug report by me, patch by Pavan Deolasee.
Pavan Deolasee [Thu, 12 Jan 2017 07:25:24 +0000 (12:55 +0530)]
Initialise a CachedPlan member correctly to avoid incorrect invalidation of a
RemoteSubplan
Tomas Vondra [Wed, 11 Jan 2017 03:56:01 +0000 (04:56 +0100)]
generate correct plan for INSERT .. SELECT .. FROM generate_series()
The planner now generates the correct plan
QUERY PLAN
---------------------------------------------------------------------------------------------
Remote Subquery Scan on all (dn1,dn2) (cost=100.00..119.00 rows=1000 width=12)
-> Insert on t1 (cost=100.00..119.00 rows=1000 width=12)
-> Remote Subquery Scan on all (dn2) (cost=100.00..119.00 rows=1000 width=12)
Distribute results by H: i
-> Function Scan on generate_series s (cost=0.00..10.00 rows=1000 width=12)
(5 rows)
for queries like
INSERT INTO t1 SELECT i, i, i FROM generate_series(1,1000) s(i);
Sadly no rows are inserted, for some reason. But the plan looks fine
(at least it seems similar to 9.5).
Tomas Vondra [Wed, 11 Jan 2017 02:46:04 +0000 (03:46 +0100)]
READ_PLAN_FIELDS has to read parallel_aware and plan_node_id fields
This XL-specific macro should be probably replaced by ReadCommonPlan,
added in PostgreSQL 9.6.
Tomas Vondra [Tue, 10 Jan 2017 23:19:42 +0000 (00:19 +0100)]
move LIMIT push-down to planner.c (when constructing the path)
When constructing a path, try to push-down the LIMIT to the remote
node, at least partially. Originally this was implemented as part
of make_limit() in createplan.c, but after the pathification it's
easy to move to planner.c, which seems like a better place.
Tomas Vondra [Tue, 10 Jan 2017 20:27:10 +0000 (21:27 +0100)]
make DISTINCT ON plans work again (WIP)
Insert RemoteSubplan paths at appropriate places, to generate plans
like this:
QUERY PLAN
--------------------------------------------------------------------------------------
HashAggregate (cost=153.86..155.86 rows=200 width=4)
Group Key: b
-> Remote Subquery Scan on all (dn1,dn2) (cost=100.00..148.76 rows=2040 width=4)
-> Seq Scan on t1 (cost=0.00..30.40 rows=2040 width=4)
or
QUERY PLAN
--------------------------------------------------------------------------------------
Unique (cost=242.54..262.94 rows=200 width=4)
-> Remote Subquery Scan on all (dn1,dn2) (cost=242.54..257.84 rows=2040 width=4)
-> Sort (cost=142.54..147.64 rows=2040 width=4)
Sort Key: b
-> Seq Scan on t1 (cost=0.00..30.40 rows=2040 width=4)
These changes may not be entirely correct yet - the "Unique" one seems
to be missing a "Sort" node on top of "Remote Subquery Scan", although
that may not be necessary for some distributions (matching the grouping
column).
For the "HashAggregate" plans, it might be possible to use the 2-phase
aggregation, pushing some of the computation to remote nodes (reducing
the amount of data transferred).
Tomas Vondra [Tue, 10 Jan 2017 19:23:19 +0000 (20:23 +0100)]
make LIMIT/OFFSET plans work again
Add RemoteSubPath below LimitPath, when needed. This reintroduces
the necessary logic into planner.c, using paths instead of plans
(to match the pathification in 9.6).
This adds a simple create_remotesubplan_path() implementation, but
still keeps additional code in make_limit() - this should move to
pathnode.c too, eventually.
This produces the same plans as XL 9.5, i.e.
QUERY PLAN
---------------------------------------------------------------------------------------
Limit (cost=100.00..100.32 rows=10 width=12)
-> Remote Subquery Scan on all (dn1,dn2) (cost=100.00..165.08 rows=2040 width=12)
-> Limit (cost=0.00..30.40 rows=2040 width=12)
-> Seq Scan on t1 (cost=0.00..30.40 rows=2040 width=12)
There's some difference in costing (likely due to using paths and
not plans), but most importantly the the results are empry for some
reason. So more work is needed, I guess.
Tomas Vondra [Tue, 10 Jan 2017 19:06:32 +0000 (20:06 +0100)]
make pg_dump work on 9.6 by adding columns to getTables()
The 9.6 if-branch in getTables() was missing XL-specific columns,
namely pgxclocatortype, pgxcattnum and pgxc_node_names, resulting
in strange errors like this:
pg_dump: column number -1 is out of range 0..33
pg_dump: column number -1 is out of range 0..33
cannot duplicate null pointer (internal error)
After adding the columns, everything seems to be working fine.
Tomas Vondra [Tue, 10 Jan 2017 17:56:39 +0000 (18:56 +0100)]
remove unnecessary 'distribution' variable from grouping_planner()
The compiler complains about this being unused, and we can pass this
in the Path anyway I believe, so get rid of it.
Tomas Vondra [Tue, 10 Jan 2017 00:04:27 +0000 (01:04 +0100)]
add missing REMOTESUBPLAN and REMOTESTMT to readfuncs.c
Tomas Vondra [Mon, 9 Jan 2017 21:45:34 +0000 (22:45 +0100)]
fix various minor bugs in initdb, to make it work
1) setup_nodeself() was starting it's own postgres process, which
then conflicted with the already running one. Instead just pass
the 'cmdfd' as for the other setup_* methods.
2) make_template0() included VACUUM command with a single '\n'
at the end of the line, resulting in this error:
FATAL: VACUUM cannot be executed from a function or multi-command string
Apparently, there need to be '\n\n' as in the non-XL code.
Tomas Vondra [Mon, 9 Jan 2017 21:10:56 +0000 (22:10 +0100)]
fix segfault in subquery_planner() when accessing parent_root
The check was moved at the very beginning, not realizing the pointer
may be NULL for outer-most queries. So evaluate the level explicitly.
Tomas Vondra [Mon, 9 Jan 2017 20:52:56 +0000 (21:52 +0100)]
fix SharedQueuesInit to register the tranche properly
The code was mixing code for named and regular tranches (identified
by just ID, in this case LWTRANCHE_SHARED_QUEUES). So switch to
the LWLockRegisterTranche() approach.
There were a two more bugs though:
1) The tranche struct was local in the block, which does not work as
it's referenced in the global array without creating a copy. So just
make it static and global.
2) The locks were not properly initialized. Add a LWLockInitialize
call to the loop.
Tomas Vondra [Mon, 9 Jan 2017 20:51:25 +0000 (21:51 +0100)]
fix order of LWLock tranches (LWTRANCHE_FIRST_USER_DEFINED last)
Clearly LWTRANCHE_FIRST_USER_DEFINED is a special name, meant to be
the last one in the array of IDs. Asserts in RegisterLWLockTranches
rely on this, for example.
Tomas Vondra [Mon, 9 Jan 2017 20:18:09 +0000 (21:18 +0100)]
remove XL-specific aggregate states (numeric, numeric_poly, json)
There were still three aggregate states defined in pg_type.h:
- numeric_agg_state
- numeric_poly_agg_state
- json_agg_state
After reverting to in-core parallel aggregate infrastructure, this
is no longer necessary. Also remove associated functions from pg_proc.h
(revert to functions as defined in core).
Tomas Vondra [Sun, 8 Jan 2017 21:13:16 +0000 (22:13 +0100)]
make find_push_down_plan/find_delete_push_down_plan static in createplan.c
After upper-planner pathification, optimizer/plan/planner.c no longer
needs to tweak plan directly, so restrict those functions lo createplan.c.
Eventually, we'll probably pathify XL too, i.e. move all the logic into
path construction (i.e. pathnode.c or it's callers).
Tomas Vondra [Fri, 6 Jan 2017 21:14:23 +0000 (22:14 +0100)]
add comment about recursiveOk handling in subquery_planner()
The variable is likely unnecessary, the code pattern is confusing.
Tomas Vondra [Fri, 6 Jan 2017 21:12:15 +0000 (22:12 +0100)]
fix detection of DML (INSERT/UPDATE/DELETE) in subquery_planner
After the merge, there was no condition on commandType, so it was
probably triggered even on SELECT subqueries. Also, move it at the
very beginning of subquery_planner().
Tomas Vondra [Thu, 5 Jan 2017 21:37:23 +0000 (22:37 +0100)]
remove pgxc_make_modifytable() prototype, not implemented anywhere
Tomas Vondra [Fri, 11 Nov 2016 00:13:52 +0000 (01:13 +0100)]
fix producerReceiveSlot() to return bool, as expected (was void before)
Tomas Vondra [Thu, 10 Nov 2016 22:34:40 +0000 (23:34 +0100)]
remove AggDistribution and Agg->aggdistribution
As the plan is to rework the aggregate to use the infrastructure
from 9.6, the best idea seems to be to use the partial aggregate
stuff. Which means we can use the aggsplit field in Agg plan node,
and we don't need the custom aggdistribution.
Due to the upper-planner pathification, this should be at AggPath
anyway (but we already have the aggsplit there).
Furthermore, remove the custom plan construction from make_agg().
This was appropriate when the upper planner was not using paths,
but that's not true anymore. The idea is to construct the usual
Finalize+Partial aggregate nodes, possibly with a RemoteSubPath
node in between them (when pushdown is not possible).
I wonder how this will interact with the parallel query, as that
is also using the Partial node, but I suppose that should work
just fine. In the worst case we'll need to introduce something
like GatherCombine, collecting (and combining) partial results
on the data node before sending them to the coordinator.
Thanks to using Finalize/Partial Agg nodes, we can also remove
the changes from fix_upper_expr, or at least that's how it looks
to me.
At some point, bits of the removed code probably will need to be
restored, although most likely different place. That applies in
particular to:
(a) find_referenced_cols_walker(), which checks what part of the
aggregate we can push to the remote node (everything, partial
or nothing)
(b) the removed bits of make_agg() constructing the remote plan
(although most of this will hopefully be handled by the partial
aggregate infrastructure)
Tomas Vondra [Thu, 10 Nov 2016 20:46:47 +0000 (21:46 +0100)]
sync nodeAgg.c to upstream as of
b5bce6c1ec
This pretty much undoes the rest of XL-specific bits, although the
differences were not particularly significant.
Tomas Vondra [Thu, 10 Nov 2016 20:41:49 +0000 (21:41 +0100)]
remove the collect function/type from pg_aggregate
This also removes quite a bit of code from nodeAgg/nodeWindowAgg and
related bits (e.g. the collect functions themselves).
Tomas Vondra [Thu, 10 Nov 2016 19:02:00 +0000 (20:02 +0100)]
remove forgotten reference to planUserId (hidden in assert)
Tomas Vondra [Thu, 10 Nov 2016 18:59:35 +0000 (19:59 +0100)]
add missing return value in printtup()
Tomas Vondra [Wed, 9 Nov 2016 17:56:11 +0000 (18:56 +0100)]
fix pg_regresss (temp_config was replaced with temp_configs)
Tomas Vondra [Wed, 9 Nov 2016 17:46:39 +0000 (18:46 +0100)]
get rid of the ifdefs in tab-completion.c entirely
This rather confused the merge, resulting in unbalanced ifdef blocks
and mysterious compile-time errors.
Tomas Vondra [Wed, 9 Nov 2016 17:20:09 +0000 (18:20 +0100)]
fix definition of sql_commands[] in tab-complete.c
Fix the whitespace and get rid of the ifdef guarding the keywords.
The comment was already out of sync with the contents, as the merges
matched the non-XL block when adding two new commands:
- IMPORT
- REFRESH MATERIALIZED VIEW
Tomas Vondra [Wed, 9 Nov 2016 16:56:45 +0000 (17:56 +0100)]
remove unnecessary include of storage/proc.h from rel.h
This was causing #error failures in s_lock.h and atomics.h because
rel.h is included also from frontend code.
Tomas Vondra [Wed, 9 Nov 2016 16:49:25 +0000 (17:49 +0100)]
fix pg_proc - set proparallel in XL functions where it was omitted
PROPARALLEL_UNSAFE
- pgxc_pool_check
- pgxc_pool_reload
- pgxc_node_str
- pgxc_is_committed
- pgxc_is_inprogress
- pgxc_lock_for_backup
PROPARALLEL_SAFE
- numeric_agg_*
- numeric_poly_*
- json_agg_*
Tomas Vondra [Wed, 9 Nov 2016 15:19:59 +0000 (16:19 +0100)]
allocate SQueue LWLocks using a named tranche (dynamically)
LWLockAssign() got removed so this is the only way to do this.
In any case, once we switch to the built-in shared queues, this should
not be needed at all I believe.
Tomas Vondra [Wed, 9 Nov 2016 14:41:52 +0000 (15:41 +0100)]
fix LWLock management in storm_stats extension
Tomas Vondra [Wed, 9 Nov 2016 14:38:31 +0000 (15:38 +0100)]
fix calls to build_aggregate_fnexprs in nodeWindowAgg
But it's as broken as nodeAgg.c.
Tomas Vondra [Wed, 9 Nov 2016 14:29:35 +0000 (15:29 +0100)]
fix beginmerge() call by passing finalMergeBatch=true
Not sure this is the right value, perhaps it should be false.
Tomas Vondra [Wed, 9 Nov 2016 14:22:47 +0000 (15:22 +0100)]
switch handling of prepared statements in resowner to ResourceArray API
Tomas Vondra [Wed, 9 Nov 2016 14:12:52 +0000 (15:12 +0100)]
fix declarations in mctx.c
- MemoryContextCallResetCallbacks was missing
- two (conflicting) MemoryContextStatsInternal declarations
Tomas Vondra [Wed, 9 Nov 2016 14:05:57 +0000 (15:05 +0100)]
fix AtEOXact_GUC - add missing_ok=true to GetConfigOptionByName()
Maybe this should be 'false' though, not sure.
Also moved the variable into the block, to minimize difference
with respect to upstream.
Tomas Vondra [Wed, 9 Nov 2016 13:58:45 +0000 (14:58 +0100)]
fix plancache.c (remaining bits of
45639a0525)
The commit removed two fields from plansource:
plansource->row_security_env
plansource->planUserId
but the code was still setting them on a few places.
Tomas Vondra [Wed, 9 Nov 2016 13:46:40 +0000 (14:46 +0100)]
fix compile errors in ruleutils.c
The errors were due to
a3c7a993d, which removed the last flag from
processIndirection().
The ruleutils code however seem significantly different from master,
for no apparent reason. Perhaps we should look into that later.
Tomas Vondra [Wed, 9 Nov 2016 13:39:37 +0000 (14:39 +0100)]
fix broken call to SetUserIdAndSecContext() in ri_triggers.c
Probably due to incorrect merge or something like that.
Tomas Vondra [Wed, 9 Nov 2016 04:51:18 +0000 (05:51 +0100)]
remove redefinition numericvar_to_int128 in numeric.c
Tomas Vondra [Wed, 9 Nov 2016 04:48:49 +0000 (05:48 +0100)]
fix calls to ProcessUtilitySlow in utility.c
A new call was missing sentToRemote parameter. Also cleanup some
of the #ifdefs in ProcessUtilitySlow() calls.
Tomas Vondra [Wed, 9 Nov 2016 04:43:21 +0000 (05:43 +0100)]
fix minor issues in pquery.c (posOverflow was removed by
23a27b039d)
Tomas Vondra [Wed, 9 Nov 2016 04:36:43 +0000 (05:36 +0100)]
missing variable in LockCheckConflicts()
Tomas Vondra [Tue, 8 Nov 2016 22:26:40 +0000 (23:26 +0100)]
resolve (remove) a lot of conflicts in planner to make it compile
This correctly resolves as many conflicts in planner, to make it pass
compilation. It however removes a lot of XL specific chunks, which were
broken by the upper-planner pathification - mostly from the grouping
planner. It's impossible to fix those reasonably until the whole code
base compiles, at which point we'll have to "pathificate" the matching
bits in XL and reintroduce them into grouping_planner() et al.
Tomas Vondra [Mon, 7 Nov 2016 22:01:44 +0000 (23:01 +0100)]
fix some of the compilation errors in planner (pathification etc.)
there's still quite a few errors and unresolved conflicts in planner.c
Tomas Vondra [Mon, 7 Nov 2016 21:16:57 +0000 (22:16 +0100)]
fix set_subquery_pathlist()
Thanks to upper-planner pathification (
3fc6e2d7f), subquery_planner
now produces a list of paths instead of a single plan. So tweak the
code to process all the paths.
XXX All the paths share the distribution, apparently. So maybe this
does not make much sense and we should pick one of the paths?
Tomas Vondra [Mon, 7 Nov 2016 20:35:33 +0000 (21:35 +0100)]
fix trivial merge error in pathnode.h (missing comma)
Tomas Vondra [Mon, 7 Nov 2016 20:31:46 +0000 (21:31 +0100)]
fix merge conflicts in readfuncs.c and outfuncs.c
There's a lot of overlap with new nodes for parallel query, so
hopefully this was merged correctly. Also, a lot of differences
w.r.t. upstream are due to portable_input all over the code, but
that could easily be minimized by moving the check into the
macros at the beginning (as we only need to do that for OIDs).
Tomas Vondra [Mon, 7 Nov 2016 19:58:04 +0000 (20:58 +0100)]
get rid of HAVE_SIGPROCMASK
The compilation was failing in poolmgr.c because of incorrect types
when tweaking signals. After inspection it seems HAVE_SIGPROCMASK
is not actually needed (it's not defined anywhere, and upstream is
not using it either) so just get rid of it entirely.
Tomas Vondra [Mon, 7 Nov 2016 19:47:23 +0000 (20:47 +0100)]
fix includes in poolmgr.c because of s_lock.h failures
The frontend libraries need to be included last, because libpq-int.h
defines FRONTEND, which triggers #error in s_lock.h. Including the
libraries at the end is not particularly pretty, but it works.
Also reorder the includes a bit into the usual lexicographic order.
Tomas Vondra [Mon, 7 Nov 2016 19:38:31 +0000 (20:38 +0100)]
fix calls to tuplesort_gettupleslot() from execRemote.c
There's a fourth argument (abbrev), so just pass NULL.
Tomas Vondra [Mon, 7 Nov 2016 19:34:15 +0000 (20:34 +0100)]
fix list of LWLocks (add XL-specific locks)
Originally the locks were only added conditionally using #ifdef, but
as the list was moved to a text file (from a header), that's no
longer possible. Which is probably for the better anyway.
Tomas Vondra [Mon, 7 Nov 2016 19:26:09 +0000 (20:26 +0100)]
fix compilation errors in secure_read/secure_write
There was undefined variable 'w' causing the compilation to fail.
There were also other differences, but there's little reason why
this should differ from core, so just revert to upstream code.
Tomas Vondra [Mon, 7 Nov 2016 18:59:22 +0000 (19:59 +0100)]
fix compilation errors in parse_agg.c and nodeAgg.c
This is almost certainly horribly broken, and those changes merely
make the code compilable. But the code is a mix of the original
XL partial-aggregate code, and new partial aggregates from core.
There's a bunch of FIXMEs on various places, demarcating the
particularly ugly hacks.
Tomas Vondra [Mon, 7 Nov 2016 18:12:30 +0000 (19:12 +0100)]
resolve merge conflict in RelationCreateStorage()
cae1c788 changed how backend ID is determined for temporary tables
because of parallel query (calling BackendIdForTempRelations()
instead of using MyBackendId directly)
Tomas Vondra [Mon, 7 Nov 2016 18:03:15 +0000 (19:03 +0100)]
add missing object_classes and includes to dependency.c
OCLASS_PGXC_NODE/OCLASS_PGXC_GROUP were missing in object_classes[]
Tomas Vondra [Mon, 7 Nov 2016 17:47:30 +0000 (18:47 +0100)]
add missing include of storage/s_lock.h into clustermon.h
Tomas Vondra [Mon, 7 Nov 2016 17:42:51 +0000 (18:42 +0100)]
remove redefinition of heap_rescan_set_params() in heapam.c
Tomas Vondra [Mon, 7 Nov 2016 17:35:50 +0000 (18:35 +0100)]
remove unnecessary include of lwlock.h from pgxc.h
As pgxc.h is included from src/common/relpath.c, this caused a circular
dependency failure, because it requires lwlocknames.h, generated in
src/backend/storage (which in turn requires src/common already built).