Commit Graph

415 Commits

Author SHA1 Message Date
ebfc56d3fb Handle impending sinval queue overflow by means of a separate signal
(SIGUSR1, which we have not been using recently) instead of piggybacking
on SIGUSR2-driven NOTIFY processing.  This has several good results:
the processing needed to drain the sinval queue is a lot less than the
processing needed to answer a NOTIFY; there's less contention since we
don't have a bunch of backends all trying to acquire exclusive lock on
pg_listener; backends that are sitting inside a transaction block can
still drain the queue, whereas NOTIFY processing can't run if there's
an open transaction block.  (This last is a fairly serious issue that
I don't think we ever recognized before --- with clients like JDBC that
tend to sit with open transaction blocks, the sinval queue draining
mechanism never really worked as intended, probably resulting in a lot
of useless cache-reset overhead.)  This is the last of several proposed
changes in response to Philip Warner's recent report of sinval-induced
performance problems.
2004-05-23 03:50:45 +00:00
4af3421161 Get rid of rd_nblocks field in relcache entries. Turns out this was
costing us lots more to maintain than it was worth.  On shared tables
it was of exactly zero benefit because we couldn't trust it to be
up to date.  On temp tables it sometimes saved an lseek, but not often
enough to be worth getting excited about.  And the real problem was that
we forced an lseek on every relcache flush in order to update the field.
So all in all it seems best to lose the complexity.
2004-05-08 19:09:25 +00:00
0370951347 Tiny assorted fixes: correct a typo in a comment in vacuumlazy.c, remove
some unused #include directives from bufmgr.c, and clarify comments in
bufmgr.h and buf.h
2004-04-25 23:50:58 +00:00
139abc2896 Make LocalRefCount and PrivateRefCount arrays of int32, rather than long.
This saves a small amount of per-backend memory for LP64 machines.
2004-04-22 07:21:55 +00:00
95a03e9cdf Another round of code cleanup on bufmgr. Use BM_VALID flag to keep track
of whether we have successfully read data into a buffer; this makes the
error behavior a bit more transparent (IMHO anyway), and also makes it
work correctly for local buffers which don't use Start/TerminateBufferIO.
Collapse three separate functions for writing a shared buffer into one.
This overlaps a bit with cleanups that Neil proposed awhile back, but
seems not to have committed yet.
2004-04-21 18:06:30 +00:00
011c3e62e7 Code review for ARC patch. Eliminate static variables, improve handling
of VACUUM cases so that VACUUM requests don't affect the ARC state at all,
avoid corner case where BufferSync would uselessly rewrite a buffer that
no longer contains the page that was to be flushed.  Make some minor
other cleanups in and around the bufmgr as well, such as moving PinBuffer
and UnpinBuffer into bufmgr.c where they really belong.
2004-04-19 23:27:17 +00:00
31338352bd * Most changes are to fix warnings issued when compiling win32
* removed a few redundant defines
* get_user_name safe under win32
* rationalized pipe read EOF for win32 (UPDATED PATCH USED)
* changed all backend instances of sleep() to pg_usleep

    - except for the SLEEP_ON_ASSERT in assert.c, as it would exceed a
32-bit long [Note to patcher: If a SLEEP_ON_ASSERT of 2000 seconds is
acceptable, please replace with pg_usleep(2000000000L)]

I added a comment to that part of the code:

    /*
     *  It would be nice to use pg_usleep() here, but only does 2000 sec
     *  or 33 minutes, which seems too short.
     */
    sleep(1000000);

Claudio Natoli
2004-04-19 17:42:59 +00:00
c672aa823b For application to HEAD, following community review.
* Changes incorrect CYGWIN defines to __CYGWIN__

* Some localtime returns NULL checks (when unchecked cause SEGVs under
Win32
regression tests)

* Rationalized CreateSharedMemoryAndSemaphores and
AttachSharedMemoryAndSemaphores (Bruce, I finally remembered to do it);
requires attention.

Claudio Natoli
2004-02-25 19:41:23 +00:00
7a57a67278 Replace opendir/closedir calls throughout the backend with AllocateDir
and FreeDir routines modeled on the existing AllocateFile/FreeFile.
Like the latter, these routines will avoid failing on EMFILE/ENFILE
conditions whenever possible, and will prevent leakage of directory
descriptors if an elog() occurs while one is open.
Also, reduce PANIC to ERROR in MoveOfflineLogs() --- this is not
critical code and there is no reason to force a DB restart on failure.
All per recent trouble report from Olivier Hubaut.
2004-02-23 23:03:10 +00:00
f83356c7f5 Do a direct probe during postmaster startup to determine the maximum
number of openable files and the number already opened.  This eliminates
depending on sysconf(_SC_OPEN_MAX), and allows much saner behavior on
platforms where open-file slots are used up by semaphores.
2004-02-23 20:45:59 +00:00
fc65a3e1fd Fixed bug where FlushRelationBuffers() did call StrategyInvalidateBuffer()
for already empty buffers because their buffer tag was not cleard out
when the buffers have been invalidated before.

Also removed the misnamed BM_FREE bufhdr flag and replaced the checks,
which effectively ask if the buffer is unpinned, with checks against the
refcount field.

Jan
2004-02-12 15:06:56 +00:00
c3c09be34b Commit the reasonably uncontroversial parts of J.R. Nield's PITR patch, to
wit: Add a header record to each WAL segment file so that it can be reliably
identified.  Avoid splitting WAL records across segment files (this is not
strictly necessary, but makes it simpler to incorporate the header records).
Make WAL entries for file creation, deletion, and truncation (as foreseen but
never implemented by Vadim).  Also, add support for making XLOG_SEG_SIZE
configurable at compile time, similarly to BLCKSZ.  Fix a couple bugs I
introduced in WAL replay during recent smgr API changes.  initdb is forced
due to changes in pg_control contents.
2004-02-11 22:55:26 +00:00
87bd956385 Restructure smgr API as per recent proposal. smgr no longer depends on
the relcache, and so the notion of 'blind write' is gone.  This should
improve efficiency in bgwriter and background checkpoint processes.
Internal restructuring in md.c to remove the not-very-useful array of
MdfdVec objects --- might as well just use pointers.
Also remove the long-dead 'persistent main memory' storage manager (mm.c),
since it seems quite unlikely to ever get resurrected.
2004-02-10 01:55:27 +00:00
8d09e25693 Backing out the background writer sync() option.
Jan
2004-02-04 01:24:53 +00:00
c77f363384 Ensure that close() and fclose() are checked for errors, at least in
cases involving writes.  Per recent discussion about the possibility
of close-time failures on some filesystems.  There is a TODO item for
this, too.
2004-01-26 22:35:32 +00:00
d77b63b17c Added GUC variable bgwriter_flush_method controlling the action
done by the background writer between writing dirty blocks and
napping.

    none (default)   no action
	sync             bgwriter calls smgrsync() causing a sync(2)

A global sync() is only good on dedicated database servers, so
more flush methods should be added in the future.

Jan
2004-01-24 20:00:46 +00:00
dfdd59e918 Adjusted calculation of shared memory requirements to new
ARC buffer replacement strategy.

Jan
2004-01-15 16:14:26 +00:00
037e2fcf8f Must test for __hppa__ as well as __hppa to make linux-hppa happy. 2004-01-03 05:47:44 +00:00
f8eed65dfb Improve spinlock code for recent x86 processors: insert a PAUSE
instruction in the s_lock() wait loop, and use test before test-and-set
in TAS() macro to avoid unnecessary bus traffic.  Patch from Manfred
Spraul, reworked a bit by Tom.
2003-12-27 20:58:58 +00:00
afb09b5a31 Use inlined TAS() on PA-RISC, if we are compiling with gcc.
Patch inspired by original submission from ViSolve.
2003-12-23 22:15:07 +00:00
9adaf64da3 Mop-up for HAS_TEST_AND_SET refactoring. Un-break two or three platforms
that were broken, try to make layout of s_lock.h entries consistent,
use HAVE_SPINLOCKS in preference to HAS_TEST_AND_SET everywhere outside
s_lock.h itself.
2003-12-23 18:13:17 +00:00
caf6e9d2dd Have configure --without-spinlocks actually not use spinlock code, even
if supported by the cpu.
2003-12-23 03:52:10 +00:00
69f2e9b0fc Move slock_t typdefs into s_lock.h from include/port files for
centralization and easier maintanence.
2003-12-23 03:31:30 +00:00
887b5a7be0 Remove NEED_I386_TAS_ASM and just test for compiler defines. 2003-12-23 00:32:06 +00:00
b731d04101 Test for __alpha and __alpha__. 2003-12-22 23:36:38 +00:00
d75b2ec4eb This patch is the next step towards (re)allowing fork/exec.
Claudio Natoli
2003-12-20 17:31:21 +00:00
fef0c8345a I posted some bufmgr cleanup a few weeks ago, but it conflicted with
some concurrent changes Jan was making to the bufmgr. Here's an
updated version of the patch -- it should apply cleanly to CVS
HEAD and passes the regression tests.

This patch makes the following changes:

     - remove the UnlockAndReleaseBuffer() and UnlockAndWriteBuffer()
       macros, and replace uses of them with calls to the appropriate
       functions.

     - remove a bunch of #ifdef BMTRACE code: it is ugly & broken
       (i.e. it doesn't compile)

     - make BufferReplace() return a bool, not an int

     - cleanup some logic in bufmgr.c; should be functionality
       equivalent to the previous code, just cleaner now

     - remove the BM_PRIVATE flag as it is unused

     - improve a few comments, etc.
2003-12-14 00:34:47 +00:00
2afacfc403 This patch properly sets the prototype for the on_shmem_exit and
on_proc_exit functions, and adjust all other related code to use
the proper types too.

by Kurt Roeckx
2003-12-12 18:45:10 +00:00
f288877f10 Fix thinko in comment. 2003-12-11 21:21:55 +00:00
5e2b99db95 Avoid assuming that type key_t is 32 bits, since it reportedly isn't
on 64-bit Solaris.  Use a non-system-dependent datatype for UsedShmemSegID,
namely unsigned long (which we were already assuming could hold a shmem
key anyway, cf RecordSharedMemoryInLockFile).
2003-12-01 22:15:38 +00:00
e7ca867485 Try to reduce confusion about what is a lock method identifier, a lock
method control structure, or a table of control structures.

. Use type LOCKMASK where an int is not a counter.

. Get rid of INVALID_TABLEID, use INVALID_LOCKMETHOD instead.

. Use INVALID_LOCKMETHOD instead of (LOCKMETHOD) NULL, because
  LOCKMETHOD is not a pointer.

. Define and use macro LockMethodIsValid.

. Rename LOCKMETHOD to LOCKMETHODID.

. Remove global variable LongTermTableId in lmgr.c, because it is
  never used.

. Make LockTableId static in lmgr.c, because it is used nowhere else.
  Why not remove it and use DEFAULT_LOCKMETHOD?

. Rename the lock method control structure from LOCKMETHODTABLE to
  LockMethodData.  Introduce a pointer type named LockMethod.

. Remove elog(FATAL) after InitLockTable() call in
  CreateSharedMemoryAndSemaphores(), because if something goes wrong,
  there is elog(FATAL) in LockMethodTableInit(), and if this doesn't
  help, an elog(ERROR) in InitLockTable() is promoted to FATAL.

. Make InitLockTable() void, because its only caller does not use its
  return value any more.

. Rename variables in lock.c to avoid statements like
        LockMethodTable[NumLockMethods] = lockMethodTable;
        lockMethodTable = LockMethodTable[lockmethod];

. Change LOCKMETHODID type to uint16 to fit into struct LOCKTAG.

. Remove static variables BITS_OFF and BITS_ON from lock.c, because
  I agree to this doubt:
 * XXX is a fetch from a static array really faster than a shift?

. Define and use macros LOCKBIT_ON/OFF.


Manfred Koizar
2003-12-01 21:59:25 +00:00
55b113257c make sure the $Id tags are converted to $PostgreSQL as well ... 2003-11-29 22:41:33 +00:00
cfeca62148 Background writer process
This first part of the background writer does no syncing at all.
It's only purpose is to keep the LRU heads clean so that regular
backends seldom to never have to call write().

Jan
2003-11-19 15:55:08 +00:00
6b86d62b00 2nd try for the ARC strategy.
I added a couple more Assertions while tracking down the exact
cause of the former bug.

All 93 regression tests pass now.

Jan
2003-11-13 14:57:15 +00:00
923e994d79 ARC strategy backed out ... sorry
Jan
2003-11-13 05:34:58 +00:00
48adc0b34b Replacement of the buffer replacement strategy with an ARC
algorithm adopted for PostgreSQL.

Jan
2003-11-13 00:40:02 +00:00
c1d62bfd00 Add operator strategy and comparison-value datatype fields to ScanKey.
Remove the 'strategy map' code, which was a large amount of mechanism
that no longer had any use except reverse-mapping from procedure OID to
strategy number.  Passing the strategy number to the index AM in the
first place is simpler and faster.
This is a preliminary step in planned support for cross-datatype index
operations.  I'm committing it now since the ScanKeyEntryInitialize()
API change touches quite a lot of files, and I want to commit those
changes before the tree drifts under me.
2003-11-09 21:30:38 +00:00
f8a769b47a Cause stats processes to detach from shared memory when started, so that
they do not prevent the postmaster from deleting the shmem segment during
a post-backend-crash restart cycle.  Per recent discussion.
2003-11-07 21:55:50 +00:00
c119c554ed Improve message wording for spinlocks-missing compilation error. 2003-11-04 09:53:36 +00:00
9821455425 Rename __arm__/__arm__ to __arm__/__arm, found by Neil Conway 2003-10-10 03:58:57 +00:00
f7fca96366 Fix #error message to mention renamed option --disable-spinlocks. 2003-09-29 04:20:22 +00:00
06e3ec7a54 Implement compiler #error if spinlock code not found, add configure flag
to bypass the error, --without-spinlocks.
2003-09-12 16:10:27 +00:00
7a3693716d Reimplement hash index locking algorithms, per my recent proposal to
pghackers.  This fixes the problem recently reported by Markus KrÌutner
(hash bucket split corrupts the state of scans being done concurrently),
and I believe it also fixes all the known problems with deadlocks in
hash index operations.  Hash indexes are still not really ready for prime
time (since they aren't WAL-logged), but this is a step forward.
2003-09-04 22:06:27 +00:00
ffafacc1f6 Repair potential deadlock created by recent changes to recycle btree
index pages: when _bt_getbuf asks the FSM for a free index page, it is
possible (and, in some cases, even moderately likely) that the answer
will be the same page that _bt_split is trying to split.  _bt_getbuf
already knew that the returned page might not be free, but it wasn't
prepared for the possibility that even trying to lock the page could
be problematic.  Fix by doing a conditional rather than unconditional
grab of the page lock.
2003-08-10 19:48:08 +00:00
46785776c4 Another pgindent run with updated typedefs. 2003-08-08 21:42:59 +00:00
f3c3deb7d0 Update copyrights to 2003. 2003-08-04 02:40:20 +00:00
089003fb46 pgindent run. 2003-08-04 00:43:34 +00:00
13ac54d1ca Since HPUX now exists for Itanium, we should decouple the assumption
that OS=hpux is the same as CPU=hppa.  First steps at doing this.
With these patches, we still work on hppa with either gcc or HP's cc.
We might work on hpux/itanium with gcc, but I can't test it.  Definitely
will not work on hpux/itanium with non-gcc compiler, for lack of spinlock
code.
2003-08-01 19:12:52 +00:00
e8db9b26d0 elog mop-up. 2003-07-27 17:10:07 +00:00
9132506477 Add Opteron/Itanium comment. 2003-07-20 04:31:32 +00:00