Tomas Vondra [Mon, 9 Jan 2017 21:45:34 +0000 (22:45 +0100)]
fix various minor bugs in initdb, to make it work
1) setup_nodeself() was starting it's own postgres process, which
then conflicted with the already running one. Instead just pass
the 'cmdfd' as for the other setup_* methods.
2) make_template0() included VACUUM command with a single '\n'
at the end of the line, resulting in this error:
FATAL: VACUUM cannot be executed from a function or multi-command string
Apparently, there need to be '\n\n' as in the non-XL code.
Tomas Vondra [Mon, 9 Jan 2017 21:10:56 +0000 (22:10 +0100)]
fix segfault in subquery_planner() when accessing parent_root
The check was moved at the very beginning, not realizing the pointer
may be NULL for outer-most queries. So evaluate the level explicitly.
Tomas Vondra [Mon, 9 Jan 2017 20:52:56 +0000 (21:52 +0100)]
fix SharedQueuesInit to register the tranche properly
The code was mixing code for named and regular tranches (identified
by just ID, in this case LWTRANCHE_SHARED_QUEUES). So switch to
the LWLockRegisterTranche() approach.
There were a two more bugs though:
1) The tranche struct was local in the block, which does not work as
it's referenced in the global array without creating a copy. So just
make it static and global.
2) The locks were not properly initialized. Add a LWLockInitialize
call to the loop.
Tomas Vondra [Mon, 9 Jan 2017 20:51:25 +0000 (21:51 +0100)]
fix order of LWLock tranches (LWTRANCHE_FIRST_USER_DEFINED last)
Clearly LWTRANCHE_FIRST_USER_DEFINED is a special name, meant to be
the last one in the array of IDs. Asserts in RegisterLWLockTranches
rely on this, for example.
Tomas Vondra [Mon, 9 Jan 2017 20:18:09 +0000 (21:18 +0100)]
remove XL-specific aggregate states (numeric, numeric_poly, json)
There were still three aggregate states defined in pg_type.h:
- numeric_agg_state
- numeric_poly_agg_state
- json_agg_state
After reverting to in-core parallel aggregate infrastructure, this
is no longer necessary. Also remove associated functions from pg_proc.h
(revert to functions as defined in core).
Tomas Vondra [Sun, 8 Jan 2017 21:13:16 +0000 (22:13 +0100)]
make find_push_down_plan/find_delete_push_down_plan static in createplan.c
After upper-planner pathification, optimizer/plan/planner.c no longer
needs to tweak plan directly, so restrict those functions lo createplan.c.
Eventually, we'll probably pathify XL too, i.e. move all the logic into
path construction (i.e. pathnode.c or it's callers).
Tomas Vondra [Fri, 6 Jan 2017 21:14:23 +0000 (22:14 +0100)]
add comment about recursiveOk handling in subquery_planner()
The variable is likely unnecessary, the code pattern is confusing.
Tomas Vondra [Fri, 6 Jan 2017 21:12:15 +0000 (22:12 +0100)]
fix detection of DML (INSERT/UPDATE/DELETE) in subquery_planner
After the merge, there was no condition on commandType, so it was
probably triggered even on SELECT subqueries. Also, move it at the
very beginning of subquery_planner().
Tomas Vondra [Thu, 5 Jan 2017 21:37:23 +0000 (22:37 +0100)]
remove pgxc_make_modifytable() prototype, not implemented anywhere
Tomas Vondra [Fri, 11 Nov 2016 00:13:52 +0000 (01:13 +0100)]
fix producerReceiveSlot() to return bool, as expected (was void before)
Tomas Vondra [Thu, 10 Nov 2016 22:34:40 +0000 (23:34 +0100)]
remove AggDistribution and Agg->aggdistribution
As the plan is to rework the aggregate to use the infrastructure
from 9.6, the best idea seems to be to use the partial aggregate
stuff. Which means we can use the aggsplit field in Agg plan node,
and we don't need the custom aggdistribution.
Due to the upper-planner pathification, this should be at AggPath
anyway (but we already have the aggsplit there).
Furthermore, remove the custom plan construction from make_agg().
This was appropriate when the upper planner was not using paths,
but that's not true anymore. The idea is to construct the usual
Finalize+Partial aggregate nodes, possibly with a RemoteSubPath
node in between them (when pushdown is not possible).
I wonder how this will interact with the parallel query, as that
is also using the Partial node, but I suppose that should work
just fine. In the worst case we'll need to introduce something
like GatherCombine, collecting (and combining) partial results
on the data node before sending them to the coordinator.
Thanks to using Finalize/Partial Agg nodes, we can also remove
the changes from fix_upper_expr, or at least that's how it looks
to me.
At some point, bits of the removed code probably will need to be
restored, although most likely different place. That applies in
particular to:
(a) find_referenced_cols_walker(), which checks what part of the
aggregate we can push to the remote node (everything, partial
or nothing)
(b) the removed bits of make_agg() constructing the remote plan
(although most of this will hopefully be handled by the partial
aggregate infrastructure)
Tomas Vondra [Thu, 10 Nov 2016 20:46:47 +0000 (21:46 +0100)]
sync nodeAgg.c to upstream as of
b5bce6c1ec
This pretty much undoes the rest of XL-specific bits, although the
differences were not particularly significant.
Tomas Vondra [Thu, 10 Nov 2016 20:41:49 +0000 (21:41 +0100)]
remove the collect function/type from pg_aggregate
This also removes quite a bit of code from nodeAgg/nodeWindowAgg and
related bits (e.g. the collect functions themselves).
Tomas Vondra [Thu, 10 Nov 2016 19:02:00 +0000 (20:02 +0100)]
remove forgotten reference to planUserId (hidden in assert)
Tomas Vondra [Thu, 10 Nov 2016 18:59:35 +0000 (19:59 +0100)]
add missing return value in printtup()
Tomas Vondra [Wed, 9 Nov 2016 17:56:11 +0000 (18:56 +0100)]
fix pg_regresss (temp_config was replaced with temp_configs)
Tomas Vondra [Wed, 9 Nov 2016 17:46:39 +0000 (18:46 +0100)]
get rid of the ifdefs in tab-completion.c entirely
This rather confused the merge, resulting in unbalanced ifdef blocks
and mysterious compile-time errors.
Tomas Vondra [Wed, 9 Nov 2016 17:20:09 +0000 (18:20 +0100)]
fix definition of sql_commands[] in tab-complete.c
Fix the whitespace and get rid of the ifdef guarding the keywords.
The comment was already out of sync with the contents, as the merges
matched the non-XL block when adding two new commands:
- IMPORT
- REFRESH MATERIALIZED VIEW
Tomas Vondra [Wed, 9 Nov 2016 16:56:45 +0000 (17:56 +0100)]
remove unnecessary include of storage/proc.h from rel.h
This was causing #error failures in s_lock.h and atomics.h because
rel.h is included also from frontend code.
Tomas Vondra [Wed, 9 Nov 2016 16:49:25 +0000 (17:49 +0100)]
fix pg_proc - set proparallel in XL functions where it was omitted
PROPARALLEL_UNSAFE
- pgxc_pool_check
- pgxc_pool_reload
- pgxc_node_str
- pgxc_is_committed
- pgxc_is_inprogress
- pgxc_lock_for_backup
PROPARALLEL_SAFE
- numeric_agg_*
- numeric_poly_*
- json_agg_*
Tomas Vondra [Wed, 9 Nov 2016 15:19:59 +0000 (16:19 +0100)]
allocate SQueue LWLocks using a named tranche (dynamically)
LWLockAssign() got removed so this is the only way to do this.
In any case, once we switch to the built-in shared queues, this should
not be needed at all I believe.
Tomas Vondra [Wed, 9 Nov 2016 14:41:52 +0000 (15:41 +0100)]
fix LWLock management in storm_stats extension
Tomas Vondra [Wed, 9 Nov 2016 14:38:31 +0000 (15:38 +0100)]
fix calls to build_aggregate_fnexprs in nodeWindowAgg
But it's as broken as nodeAgg.c.
Tomas Vondra [Wed, 9 Nov 2016 14:29:35 +0000 (15:29 +0100)]
fix beginmerge() call by passing finalMergeBatch=true
Not sure this is the right value, perhaps it should be false.
Tomas Vondra [Wed, 9 Nov 2016 14:22:47 +0000 (15:22 +0100)]
switch handling of prepared statements in resowner to ResourceArray API
Tomas Vondra [Wed, 9 Nov 2016 14:12:52 +0000 (15:12 +0100)]
fix declarations in mctx.c
- MemoryContextCallResetCallbacks was missing
- two (conflicting) MemoryContextStatsInternal declarations
Tomas Vondra [Wed, 9 Nov 2016 14:05:57 +0000 (15:05 +0100)]
fix AtEOXact_GUC - add missing_ok=true to GetConfigOptionByName()
Maybe this should be 'false' though, not sure.
Also moved the variable into the block, to minimize difference
with respect to upstream.
Tomas Vondra [Wed, 9 Nov 2016 13:58:45 +0000 (14:58 +0100)]
fix plancache.c (remaining bits of
45639a0525)
The commit removed two fields from plansource:
plansource->row_security_env
plansource->planUserId
but the code was still setting them on a few places.
Tomas Vondra [Wed, 9 Nov 2016 13:46:40 +0000 (14:46 +0100)]
fix compile errors in ruleutils.c
The errors were due to
a3c7a993d, which removed the last flag from
processIndirection().
The ruleutils code however seem significantly different from master,
for no apparent reason. Perhaps we should look into that later.
Tomas Vondra [Wed, 9 Nov 2016 13:39:37 +0000 (14:39 +0100)]
fix broken call to SetUserIdAndSecContext() in ri_triggers.c
Probably due to incorrect merge or something like that.
Tomas Vondra [Wed, 9 Nov 2016 04:51:18 +0000 (05:51 +0100)]
remove redefinition numericvar_to_int128 in numeric.c
Tomas Vondra [Wed, 9 Nov 2016 04:48:49 +0000 (05:48 +0100)]
fix calls to ProcessUtilitySlow in utility.c
A new call was missing sentToRemote parameter. Also cleanup some
of the #ifdefs in ProcessUtilitySlow() calls.
Tomas Vondra [Wed, 9 Nov 2016 04:43:21 +0000 (05:43 +0100)]
fix minor issues in pquery.c (posOverflow was removed by
23a27b039d)
Tomas Vondra [Wed, 9 Nov 2016 04:36:43 +0000 (05:36 +0100)]
missing variable in LockCheckConflicts()
Tomas Vondra [Tue, 8 Nov 2016 22:26:40 +0000 (23:26 +0100)]
resolve (remove) a lot of conflicts in planner to make it compile
This correctly resolves as many conflicts in planner, to make it pass
compilation. It however removes a lot of XL specific chunks, which were
broken by the upper-planner pathification - mostly from the grouping
planner. It's impossible to fix those reasonably until the whole code
base compiles, at which point we'll have to "pathificate" the matching
bits in XL and reintroduce them into grouping_planner() et al.
Tomas Vondra [Mon, 7 Nov 2016 22:01:44 +0000 (23:01 +0100)]
fix some of the compilation errors in planner (pathification etc.)
there's still quite a few errors and unresolved conflicts in planner.c
Tomas Vondra [Mon, 7 Nov 2016 21:16:57 +0000 (22:16 +0100)]
fix set_subquery_pathlist()
Thanks to upper-planner pathification (
3fc6e2d7f), subquery_planner
now produces a list of paths instead of a single plan. So tweak the
code to process all the paths.
XXX All the paths share the distribution, apparently. So maybe this
does not make much sense and we should pick one of the paths?
Tomas Vondra [Mon, 7 Nov 2016 20:35:33 +0000 (21:35 +0100)]
fix trivial merge error in pathnode.h (missing comma)
Tomas Vondra [Mon, 7 Nov 2016 20:31:46 +0000 (21:31 +0100)]
fix merge conflicts in readfuncs.c and outfuncs.c
There's a lot of overlap with new nodes for parallel query, so
hopefully this was merged correctly. Also, a lot of differences
w.r.t. upstream are due to portable_input all over the code, but
that could easily be minimized by moving the check into the
macros at the beginning (as we only need to do that for OIDs).
Tomas Vondra [Mon, 7 Nov 2016 19:58:04 +0000 (20:58 +0100)]
get rid of HAVE_SIGPROCMASK
The compilation was failing in poolmgr.c because of incorrect types
when tweaking signals. After inspection it seems HAVE_SIGPROCMASK
is not actually needed (it's not defined anywhere, and upstream is
not using it either) so just get rid of it entirely.
Tomas Vondra [Mon, 7 Nov 2016 19:47:23 +0000 (20:47 +0100)]
fix includes in poolmgr.c because of s_lock.h failures
The frontend libraries need to be included last, because libpq-int.h
defines FRONTEND, which triggers #error in s_lock.h. Including the
libraries at the end is not particularly pretty, but it works.
Also reorder the includes a bit into the usual lexicographic order.
Tomas Vondra [Mon, 7 Nov 2016 19:38:31 +0000 (20:38 +0100)]
fix calls to tuplesort_gettupleslot() from execRemote.c
There's a fourth argument (abbrev), so just pass NULL.
Tomas Vondra [Mon, 7 Nov 2016 19:34:15 +0000 (20:34 +0100)]
fix list of LWLocks (add XL-specific locks)
Originally the locks were only added conditionally using #ifdef, but
as the list was moved to a text file (from a header), that's no
longer possible. Which is probably for the better anyway.
Tomas Vondra [Mon, 7 Nov 2016 19:26:09 +0000 (20:26 +0100)]
fix compilation errors in secure_read/secure_write
There was undefined variable 'w' causing the compilation to fail.
There were also other differences, but there's little reason why
this should differ from core, so just revert to upstream code.
Tomas Vondra [Mon, 7 Nov 2016 18:59:22 +0000 (19:59 +0100)]
fix compilation errors in parse_agg.c and nodeAgg.c
This is almost certainly horribly broken, and those changes merely
make the code compilable. But the code is a mix of the original
XL partial-aggregate code, and new partial aggregates from core.
There's a bunch of FIXMEs on various places, demarcating the
particularly ugly hacks.
Tomas Vondra [Mon, 7 Nov 2016 18:12:30 +0000 (19:12 +0100)]
resolve merge conflict in RelationCreateStorage()
cae1c788 changed how backend ID is determined for temporary tables
because of parallel query (calling BackendIdForTempRelations()
instead of using MyBackendId directly)
Tomas Vondra [Mon, 7 Nov 2016 18:03:15 +0000 (19:03 +0100)]
add missing object_classes and includes to dependency.c
OCLASS_PGXC_NODE/OCLASS_PGXC_GROUP were missing in object_classes[]
Tomas Vondra [Mon, 7 Nov 2016 17:47:30 +0000 (18:47 +0100)]
add missing include of storage/s_lock.h into clustermon.h
Tomas Vondra [Mon, 7 Nov 2016 17:42:51 +0000 (18:42 +0100)]
remove redefinition of heap_rescan_set_params() in heapam.c
Tomas Vondra [Mon, 7 Nov 2016 17:35:50 +0000 (18:35 +0100)]
remove unnecessary include of lwlock.h from pgxc.h
As pgxc.h is included from src/common/relpath.c, this caused a circular
dependency failure, because it requires lwlocknames.h, generated in
src/backend/storage (which in turn requires src/common already built).
Pavan Deolasee [Thu, 27 Oct 2016 15:02:55 +0000 (20:32 +0530)]
Pavan Deolasee [Wed, 26 Oct 2016 08:48:16 +0000 (14:18 +0530)]
Some more regression fixes.
Pavan Deolasee [Wed, 26 Oct 2016 08:09:30 +0000 (13:39 +0530)]
Fix a left-over reference from RLS related cherry picks
Noah Misch [Mon, 21 Sep 2015 00:45:41 +0000 (20:45 -0400)]
Remove the row_security=force GUC value.
Every query of a single ENABLE ROW SECURITY table has two meanings, with
the row_security GUC selecting between them. With row_security=force
available, every function author would have been advised to either set
the GUC locally or test both meanings. Non-compliance would have
threatened reliability and, for SECURITY DEFINER functions, security.
Authors already face an obligation to account for search_path, and we
should not mimic that example. With this change, only BYPASSRLS roles
need exercise the aforementioned care. Back-patch to 9.5, where the
row_security GUC was introduced.
Since this narrows the domain of pg_db_role_setting.setconfig and
pg_proc.proconfig, one might bump catversion. A row_security=force
setting in one of those columns will elicit a clear message, so don't.
Noah Misch [Mon, 21 Sep 2015 00:47:17 +0000 (20:47 -0400)]
Remove the SECURITY_ROW_LEVEL_DISABLED security context bit.
This commit's parent made superfluous the bit's sole usage. Referential
integrity checks have long run as the subject table's owner, and that
now implies RLS bypass. Safe use of the bit was tricky, requiring
strict control over the SQL expressions evaluating therein. Back-patch
to 9.5, where the bit was introduced.
Based on a patch by Stephen Frost.
Stephen Frost [Wed, 30 Sep 2015 11:39:24 +0000 (07:39 -0400)]
Include policies based on ACLs needed
When considering which policies should be included, rather than look at
individual bits of the query (eg: if a RETURNING clause exists, or if a
WHERE clause exists which is referencing the table, or if it's a
FOR SHARE/UPDATE query), consider any case where we've determined
the user needs SELECT rights on the relation while doing an UPDATE or
DELETE to be a case where we apply SELECT policies, and any case where
we've deteremind that the user needs UPDATE rights on the relation while
doing a SELECT to be a case where we apply UPDATE policies.
This simplifies the logic and addresses concerns that a user could use
UPDATE or DELETE with a WHERE clauses to determine if rows exist, or
they could use SELECT .. FOR UPDATE to lock rows which they are not
actually allowed to modify through UPDATE policies.
Use list_append_unique() to avoid adding the same quals multiple times,
as, on balance, the cost of checking when adding the quals will almost
always be cheaper than keeping them and doing busywork for each tuple
during execution.
Back-patch to 9.5 where RLS was added.
Stephen Frost [Mon, 28 Sep 2015 19:48:36 +0000 (15:48 -0400)]
Ensure a few policies remain for pg_upgrade
To make sure that pg_dump/pg_restore function properly with RLS
policies, arrange to have a few of them left around at the end of the
regression tests.
Back-patch to 9.5 where RLS was added.
Stephen Frost [Tue, 15 Sep 2015 19:49:31 +0000 (15:49 -0400)]
Enforce ALL/SELECT policies in RETURNING for RLS
For the UPDATE/DELETE RETURNING case, filter the records which are not
visible to the user through ALL or SELECT policies from those considered
for the UPDATE or DELETE. This is similar to how the GRANT system
works, which prevents RETURNING unless the caller has SELECT rights on
the relation.
Per discussion with Robert, Dean, Tom, and Kevin.
Back-patch to 9.5 where RLS was introduced.
Stephen Frost [Tue, 15 Sep 2015 19:49:31 +0000 (15:49 -0400)]
RLS refactoring
This refactors rewrite/rowsecurity.c to simplify the handling of the
default deny case (reducing the number of places where we check for and
add the default deny policy from three to one) by splitting up the
retrival of the policies from the application of them.
This also allowed us to do away with the policy_id field. A policy_name
field was added for WithCheckOption policies and is used in error
reporting, when available.
Patch by Dean Rasheed, with various mostly cosmetic changes by me.
Back-patch to 9.5 where RLS was introduced to avoid unnecessary
differences, since we're still in alpha, per discussion with Robert.
Stephen Frost [Tue, 8 Sep 2015 21:02:49 +0000 (17:02 -0400)]
Lock all relations referred to in updatable views
Even views considered "simple" enough to be automatically updatable may
have mulitple relations involved (eg: in a where clause). We need to
make sure and lock those relations when rewriting the query.
Back-patch to 9.3 where updatable views were added.
Pointed out by Andres, patch thanks to Dean Rasheed.
Stephen Frost [Fri, 28 Aug 2015 15:39:37 +0000 (11:39 -0400)]
Ensure locks are acquired on RLS-added relations
During fireRIRrules(), get_row_security_policies can add to
securityQuals and withCheckOptions. Make sure to lock any relations
added at that point and before firing RIR rules on those expressions.
Back-patch to 9.5 where RLS was added.
Stephen Frost [Fri, 21 Aug 2015 12:22:22 +0000 (08:22 -0400)]
In AlterRole, make bypassrls an int
When reworking bypassrls in AlterRole to operate the same way the other
attribute handling is done, I missed that the variable was incorrectly a
bool rather than an int. This meant that on platforms with an unsigned
char, we could end up with incorrect behavior during ALTER ROLE.
Pointed out by Andres thanks to tests he did changing our bool to be the
one from stdbool.h which showed this and a number of other issues.
Add regression tests to test CREATE/ALTER role for the various role
attributes. Arrange to leave roles behind for testing pg_dumpall, but
none which have the LOGIN attribute.
Back-patch to 9.5 where the AlterRole bug exists.
Stephen Frost [Mon, 3 Aug 2015 19:32:49 +0000 (15:32 -0400)]
RLS: Keep deny policy when only restrictive exist
Only remove the default deny policy when a permissive policy exists
(either from the hook or defined by the user). If only restrictive
policies exist then no rows will be visible, as restrictive policies
shouldn't make rows visible. To address this requirement, a single
"USING (true)" permissive policy can be created.
Update the test_rls_hooks regression tests to create the necessary
"USING (true)" permissive policy.
Back-patch to 9.5 where RLS was added.
Per discussion with Dean.
Stephen Frost [Mon, 27 Jul 2015 20:48:26 +0000 (16:48 -0400)]
Improve RLS handling in copy.c
To avoid a race condition where the relation being COPY'd could be
changed into a view or otherwise modified, keep the original lock
on the relation. Further, fully qualify the relation when building
the query up.
Also remove the poorly thought-out Assert() and check the entire
relationOids list as, post-RLS, there can certainly be multiple
relations involved and the planner does not guarantee their ordering.
Per discussion with Noah and Andres.
Back-patch to 9.5 where RLS was introduced.
Stephen Frost [Mon, 28 Mar 2016 13:03:20 +0000 (09:03 -0400)]
Reset plan->row_security_env and planUserId
In the plancache, we check if the environment we planned the query under
has changed in a way which requires us to re-plan, such as when the user
for whom the plan was prepared changes and RLS is being used (and,
therefore, there may be different policies to apply).
Unfortunately, while those values were set and checked, they were not
being reset when the query was re-planned and therefore, in cases where
we change role, re-plan, and then change role again, we weren't
re-planning again. This leads to potentially incorrect policies being
applied in cases where role-specific policies are used and a given query
is planned under one role and then executed under other roles, which
could happen under security definer functions or when a common user and
query is planned initially and then re-used across multiple SET ROLEs.
Further, extensions which made use of CopyCachedPlan() may suffer from
similar issues as the RLS-related fields were not properly copied as
part of the plan and therefore RevalidateCachedQuery() would copy in the
current settings without invalidating the query.
Fix by using the same approach used for 'search_path', where we set the
correct values in CompleteCachedPlan(), check them early on in
RevalidateCachedQuery() and then properly reset them if re-planning.
Also, copy through the values during CopyCachedPlan().
Pointed out by Ashutosh Bapat. Reviewed by Michael Paquier.
Back-patch to 9.5 where RLS was introduced.
Security: CVE-2016-2193
Stephen Frost [Fri, 9 Oct 2015 14:49:10 +0000 (10:49 -0400)]
Handle append_rel_list in expand_security_qual
During expand_security_quals, we take the security barrier quals on an
RTE and create a subquery which evaluates the quals. During this, we
have to replace any variables in the outer query which refer to the
original RTE with references to the columns from the subquery.
We need to also perform that replacement for any Vars in the
append_rel_list.
Only backpatching to 9.5 as we only go through this process in 9.4 for
auto-updatable security barrier views, which UNION ALL queries aren't.
Discovered by Haribabu Kommi
Patch by Dean Rasheed
Pavan Deolasee [Wed, 26 Oct 2016 05:58:42 +0000 (11:28 +0530)]
Fix tablesample handling which was broken during the merge
Pavan Deolasee [Tue, 25 Oct 2016 08:44:12 +0000 (14:14 +0530)]
Fix a few regression test failures post cherry picking of patches.
Tom Lane [Sun, 26 Jul 2015 20:19:08 +0000 (16:19 -0400)]
Make entirely-dummy appendrels get marked as such in set_append_rel_size.
The planner generally expects that the estimated rowcount of any relation
is at least one row, *unless* it has been proven empty by constraint
exclusion or similar mechanisms, which is marked by installing a dummy path
as the rel's cheapest path (cf. IS_DUMMY_REL). When I split up
allpaths.c's processing of base rels into separate set_base_rel_sizes and
set_base_rel_pathlists steps, the intention was that dummy rels would get
marked as such during the "set size" step; this is what justifies an Assert
in indxpath.c's get_loop_count that other relations should either be dummy
or have positive rowcount. Unfortunately I didn't get that quite right
for append relations: if all the child rels have been proven empty then
set_append_rel_size would come up with a rowcount of zero, which is
correct, but it didn't then do set_dummy_rel_pathlist. (We would have
ended up with the right state after set_append_rel_pathlist, but that's
too late, if we generate indexpaths for some other rel first.)
In addition to fixing the actual bug, I installed an Assert enforcing this
convention in set_rel_size; that then allows simplification of a couple
of now-redundant tests for zero rowcount in set_append_rel_size.
Also, to cover the possibility that third-party FDWs have been careless
about not returning a zero rowcount estimate, apply clamp_row_est to
whatever an FDW comes up with as the rows estimate.
Per report from Andreas Seltenreich. Back-patch to 9.2. Earlier branches
did not have the separation between set_base_rel_sizes and
set_base_rel_pathlists steps, so there was no intermediate state where an
appendrel would have had inconsistent rowcount and pathlist. It's possible
that adding the Assert to set_rel_size would be a good idea in older
branches too; but since they're not under development any more, it's likely
not worth the trouble.
Pavan Deolasee [Tue, 18 Oct 2016 11:37:50 +0000 (17:07 +0530)]
Fix regression framework so that "make check" runs
Pavan Deolasee [Tue, 18 Oct 2016 10:57:13 +0000 (16:27 +0530)]
Fix compilation issues
Tom Lane [Sat, 25 Jul 2015 18:39:00 +0000 (14:39 -0400)]
Redesign tablesample method API, and do extensive code review.
The original implementation of TABLESAMPLE modeled the tablesample method
API on index access methods, which wasn't a good choice because, without
specialized DDL commands, there's no way to build an extension that can
implement a TSM. (Raw inserts into system catalogs are not an acceptable
thing to do, because we can't undo them during DROP EXTENSION, nor will
pg_upgrade behave sanely.) Instead adopt an API more like procedural
language handlers or foreign data wrappers, wherein the only SQL-level
support object needed is a single handler function identified by having
a special return type. This lets us get rid of the supporting catalog
altogether, so that no custom DDL support is needed for the feature.
Adjust the API so that it can support non-constant tablesample arguments
(the original coding assumed we could evaluate the argument expressions at
ExecInitSampleScan time, which is undesirable even if it weren't outright
unsafe), and discourage sampling methods from looking at invisible tuples.
Make sure that the BERNOULLI and SYSTEM methods are genuinely repeatable
within and across queries, as required by the SQL standard, and deal more
honestly with methods that can't support that requirement.
Make a full code-review pass over the tablesample additions, and fix
assorted bugs, omissions, infelicities, and cosmetic issues (such as
failure to put the added code stanzas in a consistent ordering).
Improve EXPLAIN's output of tablesample plans, too.
Back-patch to 9.5 so that we don't have to support the original API
in production.
Pavan Deolasee [Tue, 18 Oct 2016 06:30:41 +0000 (12:00 +0530)]
Ensure we're not going beyond the configured limits of max_coordinators or
max_datanodes
The user can still break things by first setting a high value for max_datanodes
or max_coordinators and then lowering it again after adding nodes. So this
basic check is not enough, but better than what we've currently.
Pavan Deolasee [Tue, 18 Oct 2016 05:18:39 +0000 (10:48 +0530)]
Use OS timestamp for computing if the current statement has taken more than
log_min_statement_duration for execution
Since we compare the statement start time with the one obtained by
GetCurrentTimestamp(), which returns OS time without any adjustments for clock
skew, it seems fair to use statement start time obtained by the same clock.
Pavan Deolasee [Fri, 14 Oct 2016 10:19:49 +0000 (15:49 +0530)]
Use version string from the server for pgxc_ctl
Pavan Deolasee [Tue, 4 Oct 2016 07:50:45 +0000 (13:20 +0530)]
Fix an oversight in
a5b53b0dda52 where we failed to track whether parameter
value has been computed or not at all places
Report and test case by Agoston David
Pavan Deolasee [Tue, 4 Oct 2016 05:27:00 +0000 (10:57 +0530)]
Correctly initialise coordMaxWALSenders and datanodeMaxWALSenders while adding
new nodes via pgxc_ctl
Pavan Deolasee [Fri, 30 Sep 2016 10:17:36 +0000 (15:47 +0530)]
Allow COMMIT/ROLLBACK PREPARED xact to succeed on the local coordinator even if
the GTM no longer has state information for the transaction when
xc_maintenance_mode is active.
Information on the GTM is important for explicit prepared transactions because
GTM tracks the participants nodes in that case and the COMMIT/ROLLBACK PREPARED
commands can then to forwarded to those participants. So when
xc_maintenance_mode is active, we only cleanup prepared transaction on the
local node (or remote when used with EXECUTE DIRECT). So it seems ok not to
guarantee the GTM to have the state information.
Pavan Deolasee [Thu, 29 Sep 2016 05:27:19 +0000 (10:57 +0530)]
Handle remote parameters correctly by sending then down to the remote nodes in
the correct order and correction position
Per report and test-case by Daniel Bunford-Jones
Pavan Deolasee [Fri, 16 Sep 2016 07:07:04 +0000 (12:37 +0530)]
Make another attempt to fix vpath build for pgxc_ctl contrib module
Pavan Deolasee [Thu, 15 Sep 2016 09:09:57 +0000 (14:39 +0530)]
Fix VPATH build for contrib/pgxc_ctl
Pavan Deolasee [Wed, 7 Sep 2016 04:55:52 +0000 (10:25 +0530)]
Properly quote database name while dropping it on remote node
Per bug report by Răzvan-Sebastian Botez
Pavan Deolasee [Tue, 6 Sep 2016 05:29:55 +0000 (10:59 +0530)]
Revert a change accidentally got checked in during merge
bbf2fec2. Regressions
show no additional failures.
Thanks Tomas Vondra for spotting this.
Tomas Vondra [Sun, 28 Aug 2016 11:16:17 +0000 (13:16 +0200)]
rename adjustSubplanDistribution() to adjust_subplan_distribution()
A cosmetic change to so that the whole file uses the same naming style.
Tomas Vondra [Sun, 28 Aug 2016 03:10:55 +0000 (05:10 +0200)]
remove enable_remotejoin and enable_remotegroup C variables
None of those variables was referenced in C code, and there are
no GUC variables matching them, so it's a dead code.
Those variables are still referenced in three sets of regression
tests - rangefuncs, xc_limit and xc_sort.
For rangefuncs the reference is however only in one of multiple
possible outputs (and apparently we pick another one), and the
other two regression tests are not part of any test schedule.
Tomas Vondra [Sun, 28 Aug 2016 02:53:49 +0000 (04:53 +0200)]
remove unused functions from pgxcpath.c
The following three functions defined in pgxcpath.c were not referenced
from anywhere, so let's remove them
- create_plainrel_rqpath
- create_joinrel_rqpath
- pgxc_find_remotequery_path
Both create_* functions were external, but none of them was listed in
any header file, so there's no chance they migh be called from some
external tool (also, if we have no callers, we can't test the code).
Tomas Vondra [Sun, 28 Aug 2016 02:48:11 +0000 (04:48 +0200)]
add an assert to create_remotequery_path for RELOPT_BASEREL case
Make sure that for base relations, both the paths (left and right)
and join restrictlist are NULL (or NIL for the list). I'm not sure
if this is true for RELOPT_OTHER_MEMBER_REL also, so only do this
for RELOPT_BASEREL.
Perhaps we should also convert the inverse condition (for join
relations) to an assert - currently it's an if condition:
if (rel->reloptkind == RELOPT_JOINREL && (!leftpath || !rightpath))
...
but I've left that alone for now.
Tomas Vondra [Sun, 28 Aug 2016 02:39:32 +0000 (04:39 +0200)]
minor create_remotequery_path improvement, rename RemoteQueryPath fields
The RemoteQueryPath structure includes three flags tracking features of
the path - originally, those were called
- rqhas_unshippable_qual
- rqhas_temp_rel
- rqhas_unshippable_tlist
which is a bit awkward because we already know it's part of RemoteQuery,
so the 'rq' prefix is redundant. So I've renamed them to
- has_unshippable_qual
- has_temp_rel
- has_unshippable_tlist
Also, reformat/reword the comment a bit.
Tomas Vondra [Sun, 28 Aug 2016 02:27:04 +0000 (04:27 +0200)]
fix pgxcpath.c header comment
Tomas Vondra [Sat, 27 Aug 2016 23:32:44 +0000 (01:32 +0200)]
change tts_tupleDescriptor/tts_datarow checks into asserts
The following check at the beginning of slot_deform_datarow
if (slot->tts_tupleDescriptor == NULL || slot->tts_datarow == NULL)
return;
was replaced with two asserts, enforcing the same conditions (except
that instead of silently returning it'll cause failure).
Silently ignoring such condition seems quite suspicious, and likely
only masks other errors - for example why should it be OK to call
this function without a data row?
This change seems safe, because all current callers already access
tts_tupleDescriptor - for example in the fastpath checks - and so
this field can't ever be NULL here. Also, we only call this function
after explicitly checking tts_datarow, so that can't be NULL either.
Tomas Vondra [Sat, 27 Aug 2016 23:13:27 +0000 (01:13 +0200)]
minor fixes in slot_deform_datarow - comment and attnum/natts rename
Added a comment explaining possible optimization when deforming
a DataRow message. Also renamed 'attnum' to 'natts' to more clearly
show the intent (attribute number vs. number of attributes).
Tomas Vondra [Sun, 28 Aug 2016 00:26:14 +0000 (02:26 +0200)]
remove genbki.sh shell script
It was replaced with genbki.pl for all catalogs.
Tomas Vondra [Sun, 28 Aug 2016 00:04:41 +0000 (02:04 +0200)]
remove unnecessary header changes from indexam.c
Tomas Vondra [Sun, 28 Aug 2016 20:30:27 +0000 (22:30 +0200)]
remove get_parsed_hba() - not needed after removing contrib/pghba
contrib/pghba was the sole used of the function. After removing it,
both hba.c and hba.h are clean copies from upstream, so also remove
the copyright notice from the .c file.
Tomas Vondra [Wed, 24 Aug 2016 10:12:24 +0000 (12:12 +0200)]
add pgxc_monitor to Makefile
The pgxc_monitor contrib module was not included in the Makefile, thus
vulnerable to undetected compile breakage. That's not desiable, so add
it to the Makefile.
Tomas Vondra [Wed, 24 Aug 2016 10:05:24 +0000 (12:05 +0200)]
remove pghba contrib module
The purpose of the module is unknown, and it does not even compile
for quite some time, so apparently no one uses it. Instead of fixing
it, let's remove it - if someone realizes it's useful, we can get
it from history.
Tomas Vondra [Wed, 24 Aug 2016 12:08:34 +0000 (14:08 +0200)]
remove unused copy of gtm_opt_handler.c in src/gtm/config
There's another copy of the file(s) in src/gtm/common.
Tomas Vondra [Wed, 24 Aug 2016 11:12:40 +0000 (13:12 +0200)]
remove pgxc/pool/postgresql_fdw.c as is_immutable_func() is unused
There was only a single function in this module, and it was not
referenced from anywhere. So remove the module and all references
to it (e.g. includes of the header).
In createplan.c, this is a bit more difficult, because it got
getpid() through this file, so include the pgxcnode.h instead,
as it uncludes unistd.h.
Tomas Vondra [Wed, 24 Aug 2016 10:40:05 +0000 (12:40 +0200)]
remove stormutils, containing only stormdb_promote_standby() function
Although stormdb_promote_standby() is a function visible from SQL,
it does not seem mentioned anywhere (e.g. in the docs etc.). So
let's get rid of it, as it's apparently not needed.
Tomas Vondra [Tue, 23 Aug 2016 16:49:33 +0000 (18:49 +0200)]
add barrierdesc.c to gitignore in pg_xlogdump