[Scribus] Scripter

[Craig Ringer]

On Wed, 2004-12-29 at 11:36, Kobus Wolvaardt wrote:
> Hi,
> 
> The documentation says there are no thread support for the python plugins.  

Short answer: it still doesn't work, and is unlikely to for the
foreseeable future, HOWEVER some cool stuff can still be done without
threading (see below).

Long answer (I'm very tired, so this probably rambles a lot and makes
much less sense than it could):

I haven't done super-extensive testing, but it looks like there are
issues with the GIL (global interpreter lock) and/or Qt event loop that
cause the Python threads not to progress properly. While Python is
threaded, it serializes threads based on the global interpreter lock.
Yes, it's pretty awful - see comp.lang.python for more discussion of
this than any normal human being can stand :S

Even if it was possible to spawn Python threads and have them actually
run, you would quickly run into another problem: you can't make calls to
Qt GUI operations (or emit signals to slots that perform GUI operations)
from threads other than the main thread. Since any code in Scribus may
trigger GUI calls, it's not safe to call anything from other threads
(ignoring the issue that Scribus is also not thread safe). Because of
all this, all Python code must run in the main thread. This is why the
sub-interpreter running the user's script runs in the main thread and
blocks execution of Scribus until the script finishes.

To work around this, one would need to have the Python functions post
events asking the main thread to do the work, then wait until the main
thread got around to it. As Scribus is not thread safe, the threads
would also have to be very careful how they operated. If anybody is
volunteering to convert the scripter API so that the Python functions
all post events to the main thread to do the real work, I'm all ears,
but I think it'd be a big and very tedious job right now. Perhaps once
its possible to auto-wrap more things, it might be practical.

Note that even if you can get python threads to run in parallel with
scribus and can somehow make scripter API calls, you're still not safe.
The main thread of your script runs in the main thread of Scribus, so
Scribus doesn't resume processing events for the user etc until after
the script terminates. When your script's main thread terminates, the
interpreter state is cleaned up, so any other threads are left dangling
without the interpreter they belong to and crash Scribus next time they
try to run.

In summary, if you want threading in Python from the embedded
interpreter in Scribus, you'll probably need to:
    - Make all the scripter calls either post events to the main thread
or alter object attributes in a thread-safe manner, as appropriate for
each function.
    - Get Python's threading working by examining the handling of the
GIL and the interaction between the Python interpreter and the Qt event
loop. 
    - Make each user script run in a sub thread and ensure that either
only one script can ever be running at once, or that the scripts
interact in a thread safe manner (including things like selections).

In other words, it's a huge amount of work for IMO not all that much
gain (for normal scripts).



There are, however, a few cheats that let you do a lot without
threading. Perhaps the most important is that you don't actually have to
use sub interpreters - you can run scripts in the main interpreter. This
is in fact what you do when you use the script console. I have an (ugly)
patch to permit users to run scripts from files in the main thread (and
use PyQt!!), but some more work is needed before it can be considered
for inclusion. The other key point is that scripts don't need to run in
parallel with scribus to be useful - if they can request a function be
periodically executed using a Qt timer, or bind themselves to a key/menu
item, they can run on demand and momentarily block the main interpreter
without problems. This works now (from the main thread, with PyQt).

Note that scripts run in the main interpreter share state, so if one
script changes a setting in a module the next script sees the change.
More importantly, top-level functions and variables are not cleaned up
when the script terminates, so scripts run in the main intepreter must
be very careful to clean up after themselves manually. Writing scripts
to extend scribus will be a very different matter to hacking out a quick
Python script to load some text from a database.

Eventually, I'd like to see the main interpreter used to load
'extension' scripts that need to run in parallel with scribus, say for
macros bound to keys and extra menu items. I have a couple of things in
progress to make that practical that I really need to tidy up for
possible inclusion:
    - Tidy up the 'load script' patch, add a disabled-by-default pref to
control access to it, and integrate it into the menus.
    - Implement scripter API calls to register a function for execution
by a Qt timer, access and modify the menus, and register global keyboard
shortcuts. Access to these functions would need to be confined to
scripts run from the main thread using load script, as normal scripts
calling these threads would probably crash scribus when they exited.
    - Convince the folks in charge that including said changes wouldn't
be dangerous, or no more dangerous than including a Python interpreter
in the app is ;-)

I'd be glad to work with you on those goals Kobus, if you know some C++,
as I have less time to work on this than I did until recently. Feel free
to drop me a note on or off list if you're interested.

> Is it still true (scribus tends to change faster than the docs)?

Yes, as described above.

> Are there a function that can tell me how many lines of text are in a  
> textbox?

http://bugs.scribus.net/view.php?id=1406

Feel free to add a note describing exactly what you need if its not
described in the bug.

I was really hoping to get that done for 1.2.1, but couldn't manage it.
I don't expect to try to implement support for those things in the
scripter until/unless either the core code's text handling is collected
up into classes separate from the PageItem code, or string metrics are
implemented as separate functions. I'm not volunteering :S (nor asking
anybody else to, though I do think they'd be very very useful).

I do think string and text metrics are very important to make the
scripter useful, but just can't tackle them with the current Scribus
codebase.

--
Craig Ringer