If more than one place in the code submits tasks to the shared
pool, then waitTillDone() becomes unreliable.
Add a tagging mechanism, so different callsites can wait
on different sets of tasks.
Also try to protect our worker threads against exceptions from
the thread tasks code.
Change-Id: Idde664ab50008d31a2dd73910bb22f50e62ae22f
Reviewed-on: https://gerrit.libreoffice.org/27042
Tested-by: Jenkins <ci@libreoffice.org>
Reviewed-by: Noel Grandin <noelgrandin@gmail.com>
For no good or obvious reason, SfxMedium::StorageCommit_Impl() swallows
embed::UseBackupException if there is a pTempFile, which (as the comment
claims) is "always now". This results in the temp file actually being
copied to the user-visible file and the SaveAs "succeeding", when it
clearly did not.
Also move the exception throwing to the end of ZipOutputStream::finish()
to avoid more memory leaks.
Change-Id: I448cc43291754ef20adfa6b65916282fcc365a11
In the bugdoc of tdf#91807 there are at least 49 corrupt zip streams
that raise exceptions in the DeflateThreads. Because the maximum
allowed number of threads happens to be 48, this results in an infinite
loop in ZipOutputStream::reduceScheduledThreadsToGivenNumberOrLess().
(regression from 7e2ea27e5d56f5cf767a6718a0f5edc28e24af14)
In case an exception is thrown, don't re-throw it immediately, which
might cause trouble such as leaking all of the ZipOutputEntry instances
in m_aEntries.
Change-Id: Ia74ab8e46fa1349c049d05dbec3454bfbe7d61d9
A new static member getPreferredConcurrency added to
comphelper::ThreadPool to return a configurable max
number of threads.
By default the new function returns the hardware_concurrency
value provided by std::thread. When MAX_CONCURRENCY envar is
defined, the return value is limited to whatever is set there.
Three call-sites that used std:🧵:hardware_concurrency
have been replaced with getPreferredConcurrency.
Unittests added to cover the functionality of the new member.
Unittests are capped to 4 threads.
Change-Id: I3332e393a88a5ed436316fa712ed920a4b37f4af
Reviewed-on: https://gerrit.libreoffice.org/26254
Tested-by: Jenkins <ci@libreoffice.org>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Searched source for using declarations.
Checked if those symbols reappear in the source file,
even in comments or dead code but not in #include statements.
If they don't reappear, remove the declaration.
Remove includes whose symbol got removed.
Change-Id: Ibb77163f63c1120070e9518e3dc0a78c6c59fab0
Reviewed-on: https://gerrit.libreoffice.org/24148
Reviewed-by: Jochen Nitschke <j.nitschke+logerrit@ok.de>
Reviewed-by: Noel Grandin <noelgrandin@gmail.com>
Tested-by: Noel Grandin <noelgrandin@gmail.com>
At ODT export time writing and zipping comtained data packages is nicely
parallelized, but not limited to an upper bounds of threads to use.
Together with memory consumption this makes ressource usage and runtime
behaviour bad to crashing (mostly on 32bit).
I have now limited the processing dependent on the number of available
cores to get a good processing/ressource ratio. The result uses much less
memory, is faster and runs on 32bit systems.
Change-Id: I8bd516a9a0cefd644f5d7001214bc717f29770ab
Reviewed-on: https://gerrit.libreoffice.org/23305
Tested-by: Jenkins <ci@libreoffice.org>
Reviewed-by: Noel Grandin <noelgrandin@gmail.com>
(as some tests derive from the latter only for the Directories part, not for the
setUp/tearDown overrides: those tests will be cleaned up next)
Change-Id: Ib6b78eea868b8bc21d4cc6e8fd9e1d025deca05f
Reviewed-on: https://gerrit.libreoffice.org/23078
Tested-by: Jenkins <ci@libreoffice.org>
Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
A ridiculously fast way of doing this is:
for i in $(pcregrep -l -M -r --include='.*[hc]xx$' \
--exclude-dir=workdir --exclude-dir=instdir '^
{3,}' .)
do
perl -0777 -i -pe 's/^
{3,}/
/gm' $i
done
Change-Id: Iebb93eccbee9e4fc5c4380474ba595858a27ac2c
Reviewed-on: https://gerrit.libreoffice.org/22224
Tested-by: Jenkins <ci@libreoffice.org>
Reviewed-by: Chris Sherlock <chris.sherlock79@gmail.com>
create an InterfaceContainer2 class to replace InterfaceContainer.
It uses a std::vector instead of a Sequence for the mutable listener
list, which provides far better performance.
Switch all our internal use-sites to the new class.
Change-Id: I6b56cfa511ded2395faa22e68fab3b2f16c3cb88
Somewhat arbitrarily prefer public over private derivation; the former is what
is used by default across the code base.
Change-Id: I936c1b0d3231ac97015863f396a5af49f6c9647b
I see "warn:legacy.osl:22439:1:package/source/zipapi/ZipFile.cxx:583:
Can't detect password correctness without digest!"
when opening file saved with password.
Obviously css::xml::crypto::XDigestContext used in ZipOutputEntry does not
work properly when encrypting files in parallel, so don't do that.
Change-Id: I4b354535240a4f31a6bc6855cf7f9af527634e7e
Reviewed-on: https://gerrit.libreoffice.org/21238
Tested-by: Jenkins <ci@libreoffice.org>
Reviewed-by: Matúš Kukan <matus@libreoffice.org>
Ported update_pch.sh to Python with improved performance
and features. The new script is invoked from the same
update_pch.sh which calls it for each library in
parallel, although it can be invoked directly.
The ported script (update_pch) updates all PCH files
in ~15 seconds where the old script took ~4500 seconds.
In addition, the new script supports 3-tiered headers
(system, module, and local) and is very flexible to
support other improvement. It has a per-library
optimal configuration settings that can be updated
using another new scripts (update_pch_autotune.sh)
which finds optimal per-PCH settings.
PCH files have been generated using the new scripts
which builds significantly faster (2-3x, depending
on module and configuration) and the intermediate
binaries are noticably smaller (by several GBs).
The new script stamps each generated PCH file with
the command that generated it to make it trivial
for users to update them, and also adds the command
to invoke another script (update_pch_bisect) that
helps find missing headers or conflicting headers
that may break the build after updating the PCH.
Finally update_pch has built-in unit-tests for
makefile parsing and other core functionality.
Change-Id: Ib933b50e50374d7e2e7e3e95ba8799b0cc8a27fa
Reviewed-on: https://gerrit.libreoffice.org/19965
Tested-by: Jenkins <ci@libreoffice.org>
Reviewed-by: Caolán McNamara <caolanm@redhat.com>
Tested-by: Caolán McNamara <caolanm@redhat.com>
Our current Maven based Java toolchain produces JARs, which
have a different "version needed to extract" in the ZIP local
and central directory header.
I had a look at 7zip and unzip and they already ignore the version
but compare other data LO already ignores - sig. The "standard"
document from PKWARE doesn't help.
So just compare the file path and calculate the data offset and
otherwise ignore all (duplicated) information from the local index
and rely on a correct central directory entry. Various programs
produce(d) "broken" ZIP files; even LO at some point (see git log).
Change-Id: I8d63abb0d49a1087c7654f401b62355c147c3118
Reviewed-on: https://gerrit.libreoffice.org/19779
Tested-by: Jenkins <ci@libreoffice.org>
Reviewed-by: Thorsten Behrens <Thorsten.Behrens@CIB.de>
Reviewed-by: Jan-Marek Glogowski <glogow@fbihome.de>
