mirror of
https://git.postgresql.org/git/postgresql.git
synced 2026-02-18 20:36:58 +08:00
b7467ab71c
Supply a new memory manager for RuntimeDyld, to avoid crashes in generated code caused by memory placement that can overflow a 32 bit data type. This is a drop-in replacement for the llvm::SectionMemoryManager class in the LLVM library, with Michael Smith's proposed fix from https://www.github.com/llvm/llvm-project/pull/71968. We hereby slurp it into our own source tree, after moving into a new namespace llvm::backport and making some minor adjustments so that it can be compiled with older LLVM versions as far back as 12. It's harder to make it work on even older LLVM versions, but it doesn't seem likely that people are really using them so that is not investigated for now. The problem could also be addressed by switching to JITLink instead of RuntimeDyld, and that is the LLVM project's recommended solution as the latter is about to be deprecated. We'll have to do that soon enough anyway, and then when the LLVM version support window advances far enough in a few years we'll be able to delete this code. Unfortunately that wouldn't be enough for PostgreSQL today: in most relevant versions of LLVM, JITLink is missing or incomplete. Several other projects have already back-ported this fix into their fork of LLVM, which is a vote of confidence despite the lack of commit into LLVM as of today. We don't have our own copy of LLVM so we can't do exactly what they've done; instead we have a copy of the whole patched class so we can pass an instance of it to RuntimeDyld. The LLVM project hasn't chosen to commit the fix yet, and even if it did, it wouldn't be back-ported into the releases of LLVM that most of our users care about, so there is not much point in waiting any longer for that. If they make further changes and commit it to LLVM 19 or 20, we'll still need this for older versions, but we may want to resynchronize our copy and update some comments. The changes that we've had to make to our copy can be seen by diffing our SectionMemoryManager.{h,cpp} files against the ones in the tree of the pull request. Per the LLVM project's license requirements, a copy is in SectionMemoryManager.LICENSE. This should fix the spate of crash reports we've been receiving lately from users on large memory ARM systems. Back-patch to all supported releases. Co-authored-by: Thomas Munro <thomas.munro@gmail.com> Co-authored-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com> Reviewed-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com> Reviewed-by: Daniel Gustafsson <daniel@yesql.se> (license aspects) Reported-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com> Discussion: https://postgr.es/m/CAO6_Xqr63qj%3DSx7HY6ZiiQ6R_JbX%2B-p6sTPwDYwTWZjUmjsYBg%40mail.gmail.com
src/tools/pginclude/README NOTE: headerscheck and headerscheck --cplusplus are in current use, and any problems they find should generally get fixed. The other scripts in this directory have not been used in some time, and have issues. pgrminclude in particular has a history of creating more problems than it fixes. Be very wary of applying their results blindly. pginclude ========= These utilities help clean up #include file usage. They should be run in this order so that the include files have the proper includes before the C files are tested. pgfixinclude change #include's to <> or "" pgcompinclude [-v] report which #include files can not compile on their own pgrminclude [-v] remove extra #include's pgcheckdefines check for #ifdef tests on symbols defined in files that weren't included --- this is a necessary sanity check on pgrminclude pgdefine create macro calls for all defines in the file (used by the above routines) It is also a good idea to sort the pg-specific include files in alphabetic order. This is best done with a text editor. Typical usage order would be: pgfixinclude sort include references run multiple times: pgcompinclude pgrminclude /src/include pgrminclude / pgcheckdefines There is a complexity when modifying /src/include. If include file 1 includes file 2, and file 2 includes file 3, then when file 1 is processed, it needs only file 2, not file 3. However, if later, include file 2 is processed, and file 3 is not needed by file 2 and is removed, file 1 might then need to include file 3. For this reason, the pgcompinclude and pgrminclude /src/include steps must be run several times until all includes compile cleanly. Also, tests should be done with configure settings of --enable-cassert and EXEC_BACKEND on and off. It is also wise to test a WIN32 compile. Another tools that does a similar task is at: http://code.google.com/p/include-what-you-use/ An include file visualizer script is available at: http://archives.postgresql.org/pgsql-hackers/2011-09/msg00311.php headerscheck ============ This script can be run to verify that all Postgres include files meet the project convention that they will compile "standalone", that is with no prerequisite headers other than postgres.h (or postgres_fe.h or c.h, as appropriate). A small number of header files are exempted from this requirement, and are skipped by the headerscheck script. The easy way to run the script is to say "make -s headerscheck" in the top-level build directory after completing a build. You should have included "--with-perl --with-python" in your configure options, else you're likely to get errors about related headers not being found. A limitation of the current script is that it doesn't know exactly which headers are for frontend or backend; when in doubt it uses postgres.h as prerequisite, even if postgres_fe.h or c.h would be more appropriate. Also note that the contents of macros are not checked; this is intentional. headerscheck --cplusplus ======================== The headerscheck in --cplusplus mode can be run to verify that all Postgres include files meet the project convention that they will compile as C++ code. Although the project's coding language is C, some people write extensions in C++, so it's helpful for include files to be C++-clean. A small number of header files are exempted from this requirement, and are skipped by the script in the --cplusplus mode. The easy way to run the script is to say "make -s cpluspluscheck" in the top-level build directory after completing a build. You should have included "--with-perl --with-python" in your configure options, else you're likely to get errors about related headers not being found. If you are using a non-g++-compatible C++ compiler, you may need to override the script's CXXFLAGS setting by setting a suitable environment value. A limitation of the current script is that it doesn't know exactly which headers are for frontend or backend; when in doubt it uses postgres.h as prerequisite, even if postgres_fe.h or c.h would be more appropriate. Also note that the contents of macros are not checked; this is intentional.