Commit Graph

10426 Commits

Author SHA1 Message Date
ca1e85513e Remove configure probe for dlopen, and refactor.
dlopen() is in SUSv2 and all targeted Unix systems have it.  We still
need replacement functions for Windows, but we don't need a configure
probe for that.

Since it's no longer needed by other operating systems, rename dlopen.c
to win32dlopen.c and move the declarations into win32_port.h.

Likewise, the macros RTLD_NOW and RTLD_GLOBAL now only need to be
defined on Windows, since all targeted Unix systems have 'em.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA+hUKGJ3LHeP9w5Fgzdr4G8AnEtJ=z=p6hGDEm4qYGEUX5B6fQ@mail.gmail.com
2022-08-05 09:12:45 +12:00
bcabbfc6a9 Fix formatting and comment typos
Justin Pryzby

Discussion: https://www.postgresql.org/message-id/20220801181136.GJ15006%40telsasoft.com
2022-08-04 16:41:29 +07:00
245e14e28b Fix inconsistent comments for some function declarations in headers
Some of the headers list a couple of function prototypes located in a
different file than what is referred to.  This fixes a couple of
places where this inconsistency exists.

Author: Richard Guo
Discussion: https://postgr.es/m/CAMbWs4__RdcSNXPa7L62Ozvo_Q4LvT60o3Bnp8yrQ_m9y5CKvg@mail.gmail.com
2022-08-04 17:36:21 +09:00
56f2c7b58b Support SSE2 intrinsics where available
SSE2 vector instructions are part of the spec for the 64-bit x86
architecture. Until now we have relied on the compiler to autovectorize
in some limited situations, but some useful coding idioms can only be
expressed explicitly via compiler intrinsics. To this end, add a header
that defines USE_SSE2 where available. While x86-only for now, we can
add other architectures in the future. This will also be the intended
place for helper functions that use vector operations.

Reviewed by Nathan Bossart and Masahiko Sawada

Discussion: https://www.postgresql.org/message-id/CAFBsxsE2G_H_5Wbw%2BNOPm70-BK4xxKf86-mRzY%3DL2sLoQqM%2B-Q%40mail.gmail.com
2022-08-04 13:49:18 +07:00
1aa8dad41f Fix incorrect tests for SRFs in relation_can_be_sorted_early().
Commit fac1b470a thought we could check for set-returning functions
by testing only the top-level node in an expression tree.  This is
wrong in itself, and to make matters worse it encouraged others
to make the same mistake, by exporting tlist.c's special-purpose
IS_SRF_CALL() as a widely-visible macro.  I can't find any evidence
that anyone's taken the bait, but it was only a matter of time.

Use expression_returns_set() instead, and stuff the IS_SRF_CALL()
genie back in its bottle, this time with a warning label.  I also
added a couple of cross-reference comments.

After a fair amount of fooling around, I've despaired of making
a robust test case that exposes the bug reliably, so no test case
here.  (Note that the test case added by fac1b470a is itself
broken, in that it doesn't notice if you remove the code change.
The repro given by the bug submitter currently doesn't fail either
in v15 or HEAD, though I suspect that may indicate an unrelated bug.)

Per bug #17564 from Martijn van Oosterhout.  Back-patch to v13,
as the faulty patch was.

Discussion: https://postgr.es/m/17564-c7472c2f90ef2da3@postgresql.org
2022-08-03 17:33:42 -04:00
9fc1776dda Remove unused fields from ExprEvalStep
These were added recently by 1349d2790.

Reported-by: Zhihong Yu
Discussion: https://postgr.es/m/CALNJ-vTi+YDuAWKp4Z_Dv=mrz=aq81qTg0D7wzc8y7rS_+i_cw@mail.gmail.com
2022-08-03 09:46:02 +12:00
1349d2790b Improve performance of ORDER BY / DISTINCT aggregates
ORDER BY / DISTINCT aggreagtes have, since implemented in Postgres, been
executed by always performing a sort in nodeAgg.c to sort the tuples in
the current group into the correct order before calling the transition
function on the sorted tuples.  This was not great as often there might be
an index that could have provided pre-sorted input and allowed the
transition functions to be called as the rows come in, rather than having
to store them in a tuplestore in order to sort them once all the tuples
for the group have arrived.

Here we change the planner so it requests a path with a sort order which
supports the most amount of ORDER BY / DISTINCT aggregate functions and
add new code to the executor to allow it to support the processing of
ORDER BY / DISTINCT aggregates where the tuples are already sorted in the
correct order.

Since there can be many ORDER BY / DISTINCT aggregates in any given query
level, it's very possible that we can't find an order that suits all of
these aggregates.  The sort order that the planner chooses is simply the
one that suits the most aggregate functions.  We take the most strictly
sorted variation of each order and see how many aggregate functions can
use that, then we try again with the order of the remaining aggregates to
see if another order would suit more aggregate functions.  For example:

SELECT agg(a ORDER BY a),agg2(a ORDER BY a,b) ...

would request the sort order to be {a, b} because {a} is a subset of the
sort order of {a,b}, but;

SELECT agg(a ORDER BY a),agg2(a ORDER BY c) ...

would just pick a plan ordered by {a} (we give precedence to aggregates
which are earlier in the targetlist).

SELECT agg(a ORDER BY a),agg2(a ORDER BY b),agg3(a ORDER BY b) ...

would choose to order by {b} since two aggregates suit that vs just one
that requires input ordered by {a}.

Author: David Rowley
Reviewed-by: Ronan Dunklau, James Coleman, Ranier Vilela, Richard Guo, Tom Lane
Discussion: https://postgr.es/m/CAApHDvpHzfo92%3DR4W0%2BxVua3BUYCKMckWAmo-2t_KiXN-wYH%3Dw%40mail.gmail.com
2022-08-02 23:11:45 +12:00
6b24d3f9cc Move common catalog cache access routines to lsyscache.c
In passing, move pg_relation_is_publishable next to similar functions.

Suggested-by: Alvaro Herrera
Author: Amit Kapila
Reviewed-by: Hou Zhijie
Discussion: https://postgr.es/m/CAHut+PupQ5UW9A9ut0Yjt21J9tHhx958z5L0k8-9hTYf_NYqxA@mail.gmail.com
2022-08-02 10:47:22 +05:30
c689baa158 Fix comment in pg_db_role_setting.h
Noted by Japin Li

Discussion: https://www.postgresql.org/message-id/MEYP282MB16691ACEDBC94161CF4BA1CCB69A9%40MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM
2022-08-02 11:49:37 +07:00
3592e0ff98 Have ExecFindPartition cache the last found partition
Here we add code which detects when ExecFindPartition() continually finds
the same partition and add a caching layer to improve partition lookup
performance for such cases.

Both RANGE and LIST partitioned tables traditionally require a binary
search for the set of Datums that a partition needs to be found for. This
binary search is commonly visible in profiles when bulk loading into a
partitioned table.  Here we aim to reduce the overhead of bulk-loading
into partitioned tables for cases where many consecutive tuples belong to
the same partition and make the performance of this operation closer to
what it is with a traditional non-partitioned table.

When we find the same partition 16 times in a row, the next search will
result in us simply just checking if the current set of values belongs to
the last found partition.  For LIST partitioning we record the index into
the PartitionBoundInfo's datum array.  This allows us to check if the
current Datum is the same as the Datum that was last looked up.  This
means if any given LIST partition supports storing multiple different
Datum values, then the caching only works when we find the same value as
we did the last time.  For RANGE partitioning we simply check if the given
Datums are in the same range as the previously found partition.

We store the details of the cached partition in PartitionDesc (i.e.
relcache) so that the cached values are maintained over multiple
statements.

No caching is done for HASH partitions.  The majority of the cost in HASH
partition lookups are in the hashing function(s), which would also have to
be executed if we were to try to do caching for HASH partitioned tables.
Since most of the cost is already incurred, we just don't bother.  We also
don't do any caching for LIST partitions when we continually find the
values being looked up belong to the DEFAULT partition.  We've no
corresponding index in the PartitionBoundInfo's datum array for this case.
We also don't cache when we find the given values match to a LIST
partitioned table's NULL partition.  This is so cheap that there's no
point in doing any caching for this.  We also don't cache for a RANGE
partitioned table's DEFAULT partition.

There have been a number of different patches submitted to improve
partition lookups. Hou, Zhijie submitted a patch to detect when the value
belonging to the partition key column(s) were constant and added code to
cache the partition in that case.  Amit Langote then implemented an idea
suggested by me to remember the last found partition and start to check if
the current values work for that partition.  The final patch here was
written by me and was done by taking many of the ideas I liked from the
patches in the thread and redesigning other aspects.

Discussion: https://postgr.es/m/OS0PR01MB571649B27E912EA6CC4EEF03942D9%40OS0PR01MB5716.jpnprd01.prod.outlook.com
Author: Amit Langote, Hou Zhijie, David Rowley
Reviewed-by: Amit Langote, Hou Zhijie
2022-08-02 09:55:27 +12:00
283129e325 Support pg_read_[binary_]file (filename, missing_ok).
There wasn't an especially nice way to read all of a file while
passing missing_ok = true.  Add an additional overloaded variant
to support that use-case.

While here, refactor the C code to avoid a rats-nest of PG_NARGS
checks, instead handling the argument collection in the outer
wrapper functions.  It's a bit longer this way, but far more
straightforward.

(Upon looking at the code coverage report for genfile.c, I was
impelled to also add a test case for pg_stat_file() -- tgl)

Kyotaro Horiguchi

Discussion: https://postgr.es/m/20220607.160520.1984541900138970018.horikyota.ntt@gmail.com
2022-07-29 15:38:49 -04:00
851f4cc75c Clean up some residual confusion between OIDs and RelFileNumbers.
Commit b0a55e43299c4ea2a9a8c757f9c26352407d0ccc missed a few places
where we are referring to the number used as a part of the relation
filename as an "OID". We now want to call that a "RelFileNumber".

Some of these places actually made it sound like the OID in question
is pg_class.oid rather than pg_class.relfilenode, which is especially
good to clean up.

Dilip Kumar with some editing by me.
2022-07-28 10:20:29 -04:00
5f858dd3be Bump catversion for commit d8cd0c6c95c0120168df93aae095df4e0682a08a.
The catalog contents haven't changed, but it's good to make clear
that initdb is required. Changing RELMAPPER_FILEMAGIC would be more
appropriate, but that doesn't actually produce a useful diagnostic,
so cheat by doing this instead.

Discussion: http://postgr.es/m/20220727171939.6ixixqcjt5riil2o@alvherre.pgsql
2022-07-27 16:18:21 -04:00
3ac88fddd9 Convert macros to static inline functions (buf_internals.h)
Dilip Kumar, reviewed by Vignesh C, Ashutosh Sharma, and me.

Discussion: http://postgr.es/m/CAFiTN-tYbM7D+2UGiNc2kAFMSQTa5FTeYvmg-Vj2HvPdVw2Gvg@mail.gmail.com
2022-07-27 13:54:37 -04:00
ce3049b021 Refactor code in charge of grabbing the relations of a subscription
GetSubscriptionRelations() and GetSubscriptionNotReadyRelations() share
mostly the same code, which scans pg_subscription_rel and fetches all
the relations of a given subscription.  The only difference is that the
second routine looks for all the relations not in a ready state.  This
commit refactors the code to use a single routine, shaving a bit of
code.

Author: Vignesh C
Reviewed-By: Kyotaro Horiguchi, Amit Kapila, Michael Paquier, Peter
Smith
Discussion: https://postgr.es/m/CALDaNm0eW-9g4G_EzHebnFT5zZoasWCS_EzZQ5BgnLZny9S=pg@mail.gmail.com
2022-07-27 19:50:06 +09:00
d0b193c0fa Split tuplesortvariants.c from tuplesort.c
This commit puts the implementation of Tuple sort variants into the separate
file tuplesortvariants.c.  That gives better separation of the code and
serves well as the demonstration that Tuple sort variant can be defined outside
of tuplesort.c.

Discussion: https://postgr.es/m/CAPpHfdvjix0Ahx-H3Jp1M2R%2B_74P-zKnGGygx4OWr%3DbUQ8BNdw%40mail.gmail.com
Author: Alexander Korotkov
Reviewed-by: Pavel Borisov, Maxim Orlov, Matthias van de Meent
Reviewed-by: Andres Freund, John Naylor
2022-07-27 08:28:26 +03:00
ffd1b6bb6f Add overflow protection for block-related data in WAL records
XLogRecordBlockHeader, the header holding the information for the data
related to a block, tracks the length of the data appended to the WAL
record with data_length (uint16).  This limitation in size was not
enforced by the public routine in charge of registering the data
assembled later to form the WAL record inserted, XLogRegisterBufData().
Incorrectly used, it could lead to the generation of records with some
of its data overflowed.  This commit adds some safeguards to prevent
that for the block data, complaining immediately if attempting to add to
a record block information with a size larger than UINT16_MAX, which is
the limit implied by the internal logic.

Note that this also adjusts XLogRegisterData() and XLogRegisterBufData()
so as the length of the WAL record data given by the caller is unsigned,
matching with what gets stored in XLogRecData->len.

Extracted from a larger patch by the same author.  The original patch
includes more protections when assembling a record in full that will be
looked at separately later.

Author: Matthias van de Meent
Reviewed-by: Andres Freund, Heikki Linnakangas, Michael Paquier, David
Zhang
Discussion: https://postgr.es/m/CAEze2WgGiw+LZt+vHf8tWqB_6VxeLsMeoAuod0N=ij1q17n5pw@mail.gmail.com
2022-07-27 13:35:40 +09:00
d8cd0c6c95 Remove the restriction that the relmap must be 512 bytes.
Instead of relying on the ability to atomically overwrite the
entire relmap file in one shot, write a new one and durably
rename it into place. Removing the struct padding and the
calculation showing why the map is exactly 512 bytes, and change
the maximum number of entries to a nearby round number.

Patch by me, reviewed by Andres Freund and Dilip Kumar.

Discussion: http://postgr.es/m/CA+TgmoZq5%3DLWDK7kHaUbmWXxcaTuw_QwafgG9dr-BaPym_U8WQ%40mail.gmail.com
Discussion: http://postgr.es/m/CAFiTN-ttOXLX75k_WzRo9ar=VvxFhrHi+rJxns997F+yvkm==A@mail.gmail.com
2022-07-26 14:56:25 -04:00
f92944137c Force immediate commit after CREATE DATABASE etc in extended protocol.
We have a few commands that "can't run in a transaction block",
meaning that if they complete their processing but then we fail
to COMMIT, we'll be left with inconsistent on-disk state.
However, the existing defenses for this are only watertight for
simple query protocol.  In extended protocol, we didn't commit
until receiving a Sync message.  Since the client is allowed to
issue another command instead of Sync, we're in trouble if that
command fails or is an explicit ROLLBACK.  In any case, sitting
in an inconsistent state while waiting for a client message
that might not come seems pretty risky.

This case wasn't reachable via libpq before we introduced pipeline
mode, but it's always been an intended aspect of extended query
protocol, and likely there are other clients that could reach it
before.

To fix, set a flag in PreventInTransactionBlock that tells
exec_execute_message to force an immediate commit.  This seems
to be the approach that does least damage to existing working
cases while still preventing the undesirable outcomes.

While here, add some documentation to protocol.sgml that explicitly
says how to use pipelining.  That's latent in the existing docs if
you know what to look for, but it's better to spell it out; and it
provides a place to document this new behavior.

Per bug #17434 from Yugo Nagata.  It's been wrong for ages,
so back-patch to all supported branches.

Discussion: https://postgr.es/m/17434-d9f7a064ce2a88a3@postgresql.org
2022-07-26 13:07:03 -04:00
a45388d6e0 Add xheader_width pset option to psql
The setting controls tha maximum length of the header line in expanded
format output. Possible settings are full, column, page, or an integer.
the default is full, the current behaviour, and in this case the header
line is the length of the widest line of output. column causes the
header to be truncated to the width of the first column, page causes it
to be truncated to the width of the terminal page, and an integer causes
it to be truncated to that value. If the full value is less than the
page or integer value no truncation occurs. If given without an argument
this option prints its current setting.

Platon Pronko, somewhat modified by me.

Discussion: https://postgr.es/m/f03d38a3-db96-a56e-d1bc-dbbc80bbde4d@gmail.com
2022-07-25 14:25:02 -04:00
b35617de37 Process session_preload_libraries within InitPostgres's transaction.
Previously we did this after InitPostgres, at a somewhat randomly chosen
place within PostgresMain.  However, since commit a0ffa885e doing this
outside a transaction can cause a crash, if we need to check permissions
while replacing a placeholder GUC.  (Besides which, a preloaded library
could itself want to do database access within _PG_init.)

To avoid needing an additional transaction start/end in every session,
move the process_session_preload_libraries call to within InitPostgres's
transaction.  That requires teaching the code not to call it when
InitPostgres is called from somewhere other than PostgresMain, since
we don't want session_preload_libraries to affect background workers.
The most future-proof solution here seems to be to add an additional
flag parameter to InitPostgres; fortunately, we're not yet very worried
about API stability for v15.

Doing this also exposed the fact that we're currently honoring
session_preload_libraries in walsenders, even those not connected to
any database.  This seems, at minimum, a POLA violation: walsenders
are not interactive sessions.  Let's stop doing that.

(All these comments also apply to local_preload_libraries, of course.)

Per report from Gurjeet Singh (thanks also to Nathan Bossart and Kyotaro
Horiguchi for review).  Backpatch to v15 where a0ffa885e came in.

Discussion: https://postgr.es/m/CABwTF4VEpwTHhRQ+q5MiC5ucngN-whN-PdcKeufX7eLSoAfbZA@mail.gmail.com
2022-07-25 10:27:43 -04:00
e757cdd6ad Remove dead getpwuid_r replacement code.
getpwuid_r is in SUSv2 and all targeted Unix systems have it.  We don't
use it for Windows.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Greg Stark <stark@mit.edu>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Discussion: https://postgr.es/m/CA+hUKGJ3LHeP9w5Fgzdr4G8AnEtJ=z=p6hGDEm4qYGEUX5B6fQ@mail.gmail.com
2022-07-24 09:44:29 +12:00
fb12becdfb Remove dead handling for pre-POSIX sigwait().
sigwait() is in SUSv2 and all targeted Unix systems have it.  An earlier
pre-standard function prototype existed on some older systems, but we
no longer need a workaround for that.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Greg Stark <stark@mit.edu>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Discussion: https://postgr.es/m/CA+hUKGJ3LHeP9w5Fgzdr4G8AnEtJ=z=p6hGDEm4qYGEUX5B6fQ@mail.gmail.com
2022-07-24 09:32:34 +12:00
86e5eb4f58 Remove dead getrusage replacement code.
getrusage() is in SUSv2 and all targeted Unix systems have it.

Note that POSIX only covers ru_utime and ru_stime and we rely on many
more fields without any kind of configure probe, but that predates this
commit.

The only supported system we need replacement code for now is Windows,
and that can be done without a configure probe.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Greg Stark <stark@mit.edu>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Discussion: https://postgr.es/m/CA+hUKGJ3LHeP9w5Fgzdr4G8AnEtJ=z=p6hGDEm4qYGEUX5B6fQ@mail.gmail.com
2022-07-24 09:29:48 +12:00
634a89c708 Remove configure probe for wctype.h.
This header is present in SUSv2 and Windows.

Also remove the inclusion of <wchar.h>, following clues that it was only
included for the benefit of historical systems that didn't have
<wctype.h>.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CA%2BhUKGKAmTgbg_hMiGG5T7pkpzOnY1cWFAHYtZXHCpqeC_hCkA%40mail.gmail.com
2022-07-23 16:54:00 +12:00
a3b8d2a997 Remove configure probe for sys/tas.h.
The last reference to HAVE_SYS_TAS_H disappeared with commit 718aa43a.
2022-07-23 14:59:11 +12:00
a1b56090eb Remove O_FSYNC and associated macros.
O_FSYNC was a pre-POSIX way of spelling O_SYNC, supported since commit
9d645fd84c3 for non-conforming operating systems of the time.  It's not
needed on any modern system.  We can just use standard O_SYNC directly
if it exists (= all targeted systems except Windows), and get rid of our
OPEN_SYNC_FLAG macro.

Similarly for standard O_DSYNC, we can just use that directly if it
exists (= all targeted systems except DragonFlyBSD), and get rid of our
OPEN_DATASYNC_FLAG macro.

We still avoid choosing open_datasync as a default value for
wal_sync_method if O_DSYNC has the same value as O_SYNC (= only
OpenBSD), so there is no change in default behavior.

Discussion: https://postgr.es/m/CA%2BhUKGJE7y92NY7FG2ftUbZUaqohBU65_Ys_7xF5mUHo4wirTQ%40mail.gmail.com
2022-07-22 12:41:17 +12:00
4f1f5a7f85 Remove fls(), use pg_leftmost_one_pos32() instead.
Commit 4f658dc8 provided the traditional BSD fls() function in
src/port/fls.c so it could be used in several places.  Later we added a
bunch of similar facilities in pg_bitutils.h, based on compiler
builtins that map to hardware instructions.  It's a bit confusing to
have both 1-based and 0-based variants of this operation in use in
different parts of the tree, and neither is blessed by a standard.
Let's drop fls.c and the configure probe, and reuse the newer code.

Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CA%2BhUKG%2B7dSX1XF8yFGmYk-%3D48dbjH2kmzZj16XvhbrWP-9BzRg%40mail.gmail.com
2022-07-22 10:41:50 +12:00
3225399021 Extend size_t support in pg_bitutils.h.
Use a more compact notation that allows us to add more size_t variants
as required.  This will be used by a later commit.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CA%2BhUKG%2B7dSX1XF8yFGmYk-%3D48dbjH2kmzZj16XvhbrWP-9BzRg%40mail.gmail.com
2022-07-22 09:40:39 +12:00
2a51a066dd Add missing space in comments.
Author: Junwang Zhao
Discussion: https://postgr.es/m/CAEG8a3++YQ6A-y5-w6KxP8QH6qxDJDk4dEtZw0cLcW9bsQFydg@mail.gmail.com
2022-07-21 16:25:07 +05:30
366283961a Allow users to skip logical replication of data having origin.
This patch adds a new SUBSCRIPTION parameter "origin". It specifies
whether the subscription will request the publisher to only send changes
that don't have an origin or send changes regardless of origin. Setting it
to "none" means that the subscription will request the publisher to only
send changes that have no origin associated. Setting it to "any" means
that the publisher sends changes regardless of their origin. The default
is "any".
Usage:
CREATE SUBSCRIPTION sub1 CONNECTION 'dbname=postgres port=9999'
PUBLICATION pub1 WITH (origin = none);

This can be used to avoid loops (infinite replication of the same data)
among replication nodes.

This feature allows filtering only the replication data originating from
WAL but for initial sync (initial copy of table data) we don't have such a
facility as we can only distinguish the data based on origin from WAL. As
a follow-up patch, we are planning to forbid the initial sync if the
origin is specified as none and we notice that the publication tables were
also replicated from other publishers to avoid duplicate data or loops.

We forbid to allow creating origin with names 'none' and 'any' to avoid
confusion with the same name options.

Author: Vignesh C, Amit Kapila
Reviewed-By: Peter Smith, Amit Kapila, Dilip Kumar, Shi yu, Ashutosh Bapat, Hayato Kuroda
Discussion: https://postgr.es/m/CALDaNm0gwjY_4HFxvvty01BOT01q_fJLKQ3pWP9=9orqubhjcQ@mail.gmail.com
2022-07-21 08:47:38 +05:30
af119e08fd Dump more fields when dumping planner internal data structures.
Commit 964d01ae9 marked a lot of fields as read_write_ignore
to stay consistent with what was dumped by the manually-maintained
outfuncs.c code.  However, it seems that a pretty fair number
of those omissions were either flat-out oversights, or a shortcut
taken because hand-written code seemed like it'd be too much trouble.
Let's upgrade things where it seems to make sense to dump.

To do this, we need to add support to gen_node_support.pl and
outfuncs.c for variable-length arrays of Node pointers.  That's
pretty straightforward given the model of the existing code
for arrays of scalars, but I found I needed to tighten the
type-recognizing regexes in gen_node_support.pl.  (As they
stood, they mistook "foo **" for "foo *".  Make sure they're
all fully anchored to prevent additional problems.)

The main thing left un-done here is that a lot of partitioning-related
structs are still not dumped, because they are bare structs not Nodes.
I'm not sure about the wisdom of that choice ... but changing it would
be fairly invasive, so it probably requires more justification than
just making planner node dumps more complete.

Discussion: https://postgr.es/m/1295668.1658258637@sss.pgh.pa.us
2022-07-20 13:54:30 -04:00
2d04277121 Make serialization of Nodes' scalar-array fields more robust.
When the ability to print variable-length-array fields was first
added to outfuncs.c, there was no corresponding read capability,
as it was used only for debug dumps of planner-internal Nodes.
Not a lot of thought seems to have been put into the output format:
it's just the space-separated array elements and nothing else.
Later such fields appeared in Plan nodes, and still later we grew
read support so that Plans could be transferred to parallel workers,
but the original text format wasn't rethought.  It seems inadequate
to me because (a) no cross-check is possible that we got the right
number of array entries, (b) we can't tell the difference between
a NULL pointer and a zero-length array, and (c) except for
WRITE_INDEX_ARRAY, we'd crash if a non-zero length is specified
when the pointer is NULL, a situation that can arise in some fields
that we currently conveniently avoid printing.

Since we're currently in a campaign to make the Node infrastructure
generally more it-just-works-without-thinking-about-it, now seems
like a good time to improve this.

Let's adopt a format similar to that used for Lists, that is "<>"
for a NULL pointer or "(item item item)" otherwise.  Also retool
the code to not have so many copies of the identical logic.

I bumped catversion out of an abundance of caution, although I think
that we don't use any such array fields in Nodes that can get into
the catalogs.

Discussion: https://postgr.es/m/1528424.1658272135@sss.pgh.pa.us
2022-07-20 13:04:33 -04:00
bcedd8f5fc Make subquery aliases optional in the FROM clause.
This allows aliases for sub-SELECTs and VALUES clauses in the FROM
clause to be omitted.

This is an extension of the SQL standard, supported by some other
database systems, and so eases the transition from such systems, as
well as removing the minor inconvenience caused by requiring these
aliases.

Patch by me, reviewed by Tom Lane.

Discussion: https://postgr.es/m/CAEZATCUCGCf82=hxd9N5n6xGHPyYpQnxW8HneeH+uP7yNALkWA@mail.gmail.com
2022-07-20 09:29:42 +01:00
bde60daa0e Fix warnings on Windows.
Avoid macro redefinition warnings.

Reported-by: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/CAApHDvocHp4SXcPeMTwFiCQGaf9JypjTJ3Bh90jcPuGwxyDjjQ%40mail.gmail.com
2022-07-20 16:09:50 +12:00
9430fb407b Add wal_sync_method=fdatasync for Windows.
Windows 10 gained support for flushing NTFS files with fdatasync()
semantics.  The main advantage over open_datasync (in Windows API terms
FILE_FLAG_WRITE_THROUGH) is that the latter does not flush SATA drive
caches.  The default setting is not changed, so users have to opt in to
this.

Discussion: https://postgr.es/m/CA%2BhUKGJZJVO%3DiX%2Beb-PXi2_XS9ZRqnn_4URh0NUQOwt6-_51xQ%40mail.gmail.com
2022-07-20 13:55:52 +12:00
13d8388151 Fix missed corner cases for grantable permissions on GUCs.
We allow users to set the values of not-yet-loaded extension GUCs,
remembering those values in "placeholder" GUC entries.  When/if
the extension is loaded later in the session, we need to verify that
the user had permissions to set the GUC.  That was done correctly
before commit a0ffa885e, but as of that commit, we'd check the
permissions of the active role when the LOAD happens, not the role
that had set the value.  (This'd be a security bug if it had made it
into a released version.)

In principle this is simple enough to fix: we just need to remember
the exact role OID that set each GUC value, and use that not
GetUserID() when verifying permissions.  Maintaining that data in
the guc.c data structures is slightly tedious, but fortunately it's
all basically just copy-n-paste of the logic for tracking the
GucSource of each setting, as we were already doing.

Another oversight is that validate_option_array_item() hadn't
been taught to check for granted GUC privileges.  This appears
to manifest only in that ALTER ROLE/DATABASE RESET ALL will
fail to reset settings that the user should be allowed to reset.

Patch by myself and Nathan Bossart, per report from Nathan Bossart.
Back-patch to v15 where the faulty code came in.

Discussion: https://postgr.es/m/20220706224727.GA2158260@nathanxps13
2022-07-19 17:21:55 -04:00
d6a3aeb9a3 Convert planner's AggInfo and AggTransInfo structs to proper Nodes.
This is mostly just to get outfuncs.c support for them, so that
the agginfos and aggtransinfos lists can be dumped when dumping
the contents of PlannerInfo.

While here, improve some related comments; notably, clean up
obsolete comments left over from when preprocess_minmax_aggregates
had to make its own scan of the query tree.

Discussion: https://postgr.es/m/742479.1658160504@sss.pgh.pa.us
2022-07-19 12:29:37 -04:00
e2f6c307c0 Estimate cost of elided SubqueryScan, Append, MergeAppend nodes better.
setrefs.c contains logic to discard no-op SubqueryScan nodes, that is,
ones that have no qual to check and copy the input targetlist unchanged.
(Formally it's not very nice to be applying such optimizations so late
in the planner, but there are practical reasons for it; mostly that we
can't unify relids between the subquery and the parent query until we
flatten the rangetable during setrefs.c.)  This behavior falsifies our
previous cost estimates, since we would've charged cpu_tuple_cost per
row just to pass data through the node.  Most of the time that's little
enough to not matter, but there are cases where this effect visibly
changes the plan compared to what you would've gotten with no
sub-select.

To improve the situation, make the callers of cost_subqueryscan tell
it whether they think the targetlist is trivial.  cost_subqueryscan
already has the qual list, so it can check the other half of the
condition easily.  It could make its own determination of tlist
triviality too, but doing so would be repetitive (for callers that
may call it several times) or unnecessarily expensive (for callers
that can determine this more cheaply than a general test would do).

This isn't a 100% solution, because createplan.c also does things
that can falsify any earlier estimate of whether the tlist is
trivial.  However, it fixes nearly all cases in practice, if results
for the regression tests are anything to go by.

setrefs.c also contains logic to discard no-op Append and MergeAppend
nodes.  We did have knowledge of that behavior at costing time, but
somebody failed to update it when a check on parallel-awareness was
added to the setrefs.c logic.  Fix that while we're here.

These changes result in two minor changes in query plans shown in
our regression tests.  Neither is relevant to the purposes of its
test case AFAICT.

Patch by me; thanks to Richard Guo for review.

Discussion: https://postgr.es/m/2581077.1651703520@sss.pgh.pa.us
2022-07-19 11:18:19 -04:00
14a8bd9827 Convert macros to static inline functions (itup.h)
Reviewed-by: Amul Sul <sulamul@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/5b558da8-99fb-0a99-83dd-f72f05388517%40enterprisedb.com
2022-07-19 07:24:55 +02:00
c290e79cf0 windows: msvc: Define STDIN/OUT/ERR_FILENO.
Because they are not available we've used _fileno(stdin) in some places, but
that doesn't reliably work, because stdin might be closed. This is the
prerequisite of the subsequent commit, fixing a failure introduced in
76e38b37a5.

Author: Andres Freund <andres@anarazel.de>
Reported-By: Sandeep Thakkar <sandeep.thakkar@enterprisedb.com>
Message-Id: 20220520164558.ozb7lm6unakqzezi@alap3.anarazel.de (on pgsql-packagers)
Backpatch: 15-, where 76e38b37a5 came in
2022-07-18 17:21:54 -07:00
f58d7073b7 Convert macros to static inline functions (tupmacs.h)
Reviewed-by: Amul Sul <sulamul@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/5b558da8-99fb-0a99-83dd-f72f05388517%40enterprisedb.com
2022-07-18 08:01:27 +02:00
8cf64d35ea Mark all symbols exported from extension libraries PGDLLEXPORT.
This is in preparation for defaulting to -fvisibility=hidden in extensions,
instead of relying on all symbols in extensions to be exported.

This should have been committed before 089480c0770, but something in my commit
scripts went wrong.

Author: Andres Freund <andres@anarazel.de>
Reviewed-By: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/20211101020311.av6hphdl6xbjbuif@alap3.anarazel.de
2022-07-17 18:50:14 -07:00
089480c077 Default to hidden visibility for extension libraries where possible
Until now postgres built extension libraries with global visibility, i.e.
exporting all symbols.  On the one platform where that behavior is not
natively available, namely windows, we emulate it by analyzing the input files
to the shared library and exporting all the symbols therein.

Not exporting all symbols is actually desirable, as it can improve loading
speed, reduces the likelihood of symbol conflicts and can improve intra
extension library function call performance. It also makes the non-windows
builds more similar to windows builds.

Additionally, with meson implementing the export-all-symbols behavior for
windows, turns out to be more verbose than desirable.

This patch adds support for hiding symbols by default and, to counteract that,
explicit symbol visibility annotation for compilers that support
__attribute__((visibility("default"))) and -fvisibility=hidden. That is
expected to be most, if not all, compilers except msvc (for which we already
support explicit symbol export annotations).

Now that extension library symbols are explicitly exported, we don't need to
export all symbols on windows anymore, hence remove that behavior from
src/tools/msvc. The supporting code can't be removed, as we still need to
export all symbols from the main postgres binary.

Author: Andres Freund <andres@anarazel.de>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/20211101020311.av6hphdl6xbjbuif@alap3.anarazel.de
2022-07-17 17:49:51 -07:00
f2b73c8d75 Add central declarations for dlsym()ed symbols
This is in preparation for defaulting to -fvisibility=hidden in extensions,
instead of exporting all symbols. For that symbols intended to be exported
need to be tagged with PGDLLEXPORT. Most extensions only need to do so for
_PG_init() and functions defined with PG_FUNCTION_INFO_V1. Adding central
declarations avoids each extension having to add PGDLLEXPORT. Any existing
declarations in extensions will continue to work if fmgr.h is included before
them, otherwise compilation for Windows will fail.

Author: Andres Freund <andres@anarazel.de>
Reviewed-By: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/20211101020311.av6hphdl6xbjbuif@alap3.anarazel.de
2022-07-17 17:23:42 -07:00
c94ae9d827 Emulate sigprocmask(), not sigsetmask(), on Windows.
Since commit a65e0864, we've required Unix systems to have
sigprocmask().  As noted in that commit's message, we were still
emulating the historical pre-standard sigsetmask() function in our
Windows support code.  Emulate standard sigprocmask() instead, for
consistency.

The PG_SETMASK() abstraction is now redundant and all calls could in
theory be replaced by plain sigprocmask() calls, but that isn't done by
this commit.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/3153247.1657834482%40sss.pgh.pa.us
2022-07-16 17:03:38 +12:00
507ba16b28 Convert macros to static inline functions (xlog_internal.h)
Reviewed-by: Amul Sul <sulamul@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/5b558da8-99fb-0a99-83dd-f72f05388517%40enterprisedb.com
2022-07-15 14:56:00 +02:00
3e9ca52601 Support gcc -fkeep-inline-functions
For some systems, we need to avoid unsatisfied-external-reference
errors in static inlines.  See
27d2693187d1bcf2563ee7142ba37d4788c8d52b for example.  In order to
test that on other systems, the gcc option -fkeep-inline-functions can
be used.  But it actually is a bit stricter than what we currently
have in place, so fix up a few more places along the lines of the
above commit.  (This undoes part of commit
2cd2569c72b8920048e35c31c9be30a6170e1410.)

(Note, this does not add that gcc option anywhere to the build system,
it just makes it possible to use it successfully manually.)

Discussion: https://www.postgresql.org/message-id/flat/E1oBgIW-002ehP-VJ%40gemulon.postgresql.org
2022-07-15 12:12:30 +02:00
80ad91ea8c Fix inconsistent parameter names between prototype and declaration
Noticed while working in this area.  This code was introduced in PG15,
which is still in beta, so backpatch to there for consistency.

Backpatch-through: 15
2022-07-15 15:26:34 +12:00
7c0eb3c622 Tighten up parsing logic in gen_node_support.pl.
Teach this script to handle function pointer fields honestly.
Previously they were just silently ignored, but that's not likely to
be a behavior we can accept indefinitely.  This mostly entails fixing
it so that a field declaration spanning multiple lines can be parsed,
because we have a bunch of such fields that're laid out that way.
But that's a good improvement in its own right.

With that change and a minor regex adjustment, the only struct it
fails to parse in the node-defining headers is A_Const, because
of the embedded union.  The path of least resistance is to move
that union declaration outside the struct.

Having done those things, we can make it error out if it finds
any within-struct syntax it doesn't understand, which seems like
a pretty important property for robustness.

This commit doesn't change the output files at all; it's just in
the way of future-proofing.

Discussion: https://postgr.es/m/2593369.1657759779@sss.pgh.pa.us
2022-07-14 09:04:23 -04:00