[Scribus] Scribus hangs for 40 mins, then disappears

[Craig Ringer]

ols6000 at sbcglobal.net wrote:
> At 01:33 PM 4/30/2007, avox wrote:

>> That's kind of difficult, since memory is consumed in a lot of places
>> which are not necessarily under Scribus's control (most notably Qt).
> 
> Surely it is possible to get error returns from Qt.

If you build Qt with exceptions enabled, it will throw std::bad_alloc
when a memory allocation fails. We can catch this and report a sensible
error to the user in most cases.

Some distributors unfortunately build Qt without exceptions. Since Qt
its self does not rely on exceptions, this is acceptable, but really
annoying when it comes to handling OOM situations and similar problems.
If exceptions are off, a `new' will return 0 if the allocation failed.
Either Qt will detect this and return an error value (where possible) or
it'll access the null pointer and crash with a SIGSEGV that we can catch
and report to the user.

In all three cases, the application goes splat, but it should do so
after telling the user. However, we don't currently ensure that the
error reporting code makes absolutely no memory allocations, so there
are some circumstances where there is insufficient memory to report an
out of memory condition. The application will then terminate.

> However, judging 
> from the number of problems I've seen attributed to its use, I 
> suggest eliminating Qt from Scribus.

Even if we wanted to, that's not realistic. For historical and practical
reasons Scribus uses Qt's STL-analogue, the QTL, in most of its internal
data structures. Not only that, but the application is designed around
the run-time dynamic signals-and-slots mechanism, and libsigc++ (being a
compile time signal library) doesn't fit there.

In any case, it'd basically be a total rewrite. While some parts of
Scribus would benefit strongly from exactly that, it'd take so long that
it'd simply be unrealistic. The Netscape -> Mozilla saga should serve as
a lesson about clean slate rewrites.

On top of all that, if I was to pick a toolkit for any new C++ project
Qt would be my preferred choice. We've had some problems with the way
distros package Qt (butchery by bad patches and building without
exceptions, mostly) but it's still a fantastic library. You're only
seeing the problem reports, not just how much more efficient it makes
much of the day-to-day development work.

>> What are the sizes of the images?
> 
> An example image is 6495 x 3952 pixels, 4800 dpi. This image can be 
> displayed (on a Windows XP machine) as a thumbnail by Windows 
> Explorer, and at any size by PicaJet (an image organizer), so I know 
> the image file is OK.

Yep. Unfortunately, Scribus generates image previews in an extremely
inefficient manner. It uncompresses the whole image into memory, copies
it, and then resizes it for preview before discarding the full size
uncompressed version. Last time I checked, it was still doing something
horrible with string buffers that could double or quadruple the required
memory for no purpose, as well ... though I think the worst of that has
been cleaned out in 1.3.x .

Scribus should not be keeping the large copy in RAM ... but it's not
impossible that just one of those images, plus the previews for the rest
of them and its other memory overheads, is bringing it to its knees.

Large image handling really needs a thumbnailer/preview generator that
does not load the entire image uncompressed into RAM. We also need some
image conversion code that can work on images in small chunks. I'm not
aware of any multi-format libraries for anything like that, so it's
going to be some interesting work when it gets to the top of the pile.
That said, memory mapped files may well be the way to achieve this while
retaining some degree of sanity, since on all modern platforms the OS's
virtual memory manager is capable of such mapping with decent performance.

In any case, there are some significant known issues with the handling
of large images. See the "excessive memory use" metabug in the tracker
for a collection of related issues.

--
Craig Ringer