testplugin_v020.zip

[KevinH]

Hi All,

To help Linux packagers and others who build Sigil from source, we have created a simple testplugin_v020.zip that will test the available Python 3.X interpreter environment setup.

1. Use the attached testplugin_v019.zip here

2. open Sigil.app to the normal nearly blank template epub it generates when opened. It is important to note that the plugin's tests rely on this nearly blank epub during its testing.

3. use Plugins->Manage Plugins menu and make sure the "Use Bundled Python" checkbox is checked on Mac OS X and Windows. On Linux please properly set the path to your system Python 3 interpreter.

4. use the "Add Plugin" button to navigate to and add testplugin.zip and then hit "Okay" to exit the Manage Plugins Dialog

5. use Plugins->Manage Plugins->Edit->testplugin to launch the plugin and hit the "Start" button to run it

6. check the plugin output window for your missing or broken plugin test results

It tests for the full set of python 3 packages available to the Bundled Python as well as Hunspell SpellChecking, using the bs4 sigil gumbo adapter, Sigil's internal python libaries, and the many of the BookContainer basic interface functions.

I have attached it here in case anyone wants to play with it.

[DiapDealer]

Windows users: please note that Sigil 0.9.0's bundled python 3 will fail the cssutils test (missing) when running this plugin. I don't think any plugins are currently using cssutils, but this problem wiil be fixed in the next Sigil release.

[KevinH]

Mac OS X using Sigil-0.9.0 is in the same boat ....

The current Sigil-0.9.0 on Mac OS X will fail for cssutils, and for some things done in PIL. Both of these are fixed for our next release.

So creating the test plugin actually helped us track down and fix our own issues before a plugin developer ran into them!

KevinH

Originally Posted by DiapDealer

Windows users: please note that Sigil 0.9.0's bundled python 3 will fail the cssutils test (missing) when running this plugin. I don't think any plugins are currently using cssutils, but this problem wiil be fixed in the next Sigil release.

[eschwartz]

Cool, sounds great!

And I've verified my AUR package passes the plugin tests with flying colors.


A couple requests:

• Can you summarize the failed tests -- if any -- at the end? Just to make it easier to read.
• Would it be possible to report which components are part of the core functionality, and which are recommended but not crucial? I am thinking of some of the python packages, which plugins may expect but Sigil itself does not use.
  Reason I ask is, because I list those as optdepends ("python-whatever: recommended for plugins"), and debian can list them as suggests or recommends.
• You documented the plugin in docs/Bundling_Python3_With_Sigil_on_MacOSX.txt but not docs/BuildingOnLinux.md
  It isn't very helpful to linux packagers if they don't know it exists so I think it would be nice if you made it more likely they will see it.

[KevinH]

Hi eschwartz,

Originally Posted by eschwartz

Cool, sounds great!
And I've verified my AUR package passes the plugin tests with flying colors.

Nicely done.

[*]Can you summarize the failed tests -- if any -- at the end? Just to make it easier to read.
[*]Would it be possible to report which components are part of the core functionality, and which are recommended but not crucial? I am thinking of some of the python packages, which plugins may expect but Sigil itself does not use.
Reason I ask is, because I list those as optdepends ("python-whatever: recommended for plugins"), and debian can list them as suggests or recommends.

The output itself is only 1 page long in total and it tells you everything. So a quick visual inspection will tell you anything you want.

I can split out the extra Python external packages such as cssutils, cssselect, PIL, html5lib, six, into their own reporting section. That said if you or other linux distributors treat them differently from other python 3 packages, some future plugin may fail generating an unneeded bug report and more inconsistencies across platforms. I would prefer to keep all versions of Sigil with the exact same capabilties just to be safe and consistent and to make support easier.

By the way, without lxml, Sigil will not even start up, making testing the plugins impossible.

[*]You documented the plugin in docs/Bundling_Python3_With_Sigil_on_MacOSX.txt but not docs/BuildingOnLinux.md
It isn't very helpful to linux packagers if they don't know it exists so I think it would be nice if you made it more likely they will see it.

I only just added it to the Mac OSX building docs before I posted it here. The Linux docs will get a similar treatment but there are only so many hours of volunteer time this weekend. Before we release Sigil-0.9.1, we'll make that change.

Thanks,
KevinH

[eschwartz]

Originally Posted by KevinH

Hi eschwartz,



Nicely done.



The output itself is only 1 page long in total and it tells you everything. So a quick visual inspection will tell you anything you want.

I can split out the extra Python external packages such as cssutils, cssselect, PIL, html5lib, six, into their own reporting section. That said if you or other linux distributors treat them differently from other python 3 packages, some future plugin may fail generating an unneeded bug report and more inconsistencies across platforms. I would prefer to keep all versions of Sigil with the exact same capabilties just to be safe and consistent and to make support easier.

Quick visual inspection that requires you to fully read each line.
Maybe instead you could colorize it (red warning) or print "**** FAILED ****" if the plugin runner doesn't support color (probably not...)

It isn't that big a deal, it's just that currently it doesn't stand out very well.


As for optional components, linux distributors like splitting out optdepends, and expect users to know what they're doing.
apt-based systems can use Recommends, which I believe is installed by default -- an advanced setting lets you only install the minimal packages necessary, and advanced users are expected to know what they are doing later on down the line.
ArchLinux is a "user-centric, not user-friendly" distro, and we are expected to make note of the "optional dependencies for ____" list in pacman's output.

Perhaps, like the Failed checks, you can simply add a Recommended status but still group them as you do now. I'm just floating ideas here.

Again, it's your choice, but packagers may appreciate the information since it helps them figure out how to write the dependency list based on their preferred guidelines.

I only just added it to the Mac OSX building docs before I posted it here. The Linux docs will get a similar treatment but there are only so many hours of volunteer time this weekend. Before we release Sigil-0.9.1, we'll make that change.

Thanks,
KevinH

Thanks. Just wanted to make sure you were aware.

[DiapDealer]

I don't think any of the python modules should truly be considered "optional." Sure, Sigil could run without them, but as Kevin mentioned, those are the modules that are being packaged with Sigil (and eventually, all three platforms will have a package). Plugin developers will not be required to "declare" any of those modules as needing to be installed to use their plugin. It will be assumed that any Sigil installation will have access to all of those. At the very least, they should be considered "Most Strenuously Recommended If You Want Help Figuring Out Why A Particular Downloaded Plugin Isn't Working for You."

I can guarantee my first trouble-shooting step will be to require that the Test Plugin passes every single test before proceeding.

[KevinH]

Couldn't agree more. Why on earth would maintainers want to have the same release have different versions with different capabilities flying around. If you want to call it "Sigil", it should be Sigil and have the exact same capabilities as Sigil on all other platforms. Any other way makes for a support nightmare.

So Linux distributors liking to use "optional" or "recommended" packages will simply result in crippled versions of applications existing in the wild causing problems.

KevinH

[eschwartz]

If plugins are optional, then the modules that are only needed for plugins, are optional as well.

The opinion of your average distro packaging standards, is that plugins which aren't necessary for core functionality, are optional extras.


That's why its nice to list them separately, and if a distro prioritizes user-friendliness, they can default to installing them, and if a distro prioritizes giving users the tools to make unusually informed decisions, they can do that too.


And for a standalone binary download, it makes sense to include them as a matter of course.

...

More information makes it more likely that packagers include those modules in any shape or form, rather than having some maintainer somewhere say "huh, this isn't doing anything, let's ax it".

[DiapDealer]

How's this then: we don't consider any of the python modules optional for core functionality. So our list IS prioritized. Distros are, of course, free to prioritize what they consider optional (which they're going to do regardless of what we say anyway). Which is ultimately going to result in us getting a metric crap-ton of support requests/bug reports because people are installing versions of Sigil with crippled/incomplete plugin frameworks. Call me crazy, but I don't really feel compelled to supply the bullets that are going to be used to shoot us with later.

[KevinH]

Updated the first post to testplugin_v012.zip to work with the forthcoming Sigil 0.9.4 release that includes a Nav by default under epub3.

KevinH

[patrik]

Would it be useful to have the testplugin check for python3-tk?

I just ran a couple of plugins for the first time, and they all failed due to not finding the module _tkinter.
Which was easily solved by installing python3-tk.

The plugins are optional, and the testplugin checks the basic installation, but maybe it could save some time for some users?

[DiapDealer]

Originally Posted by patrik

Would it be useful to have the testplugin check for python3-tk?

I just ran a couple of plugins for the first time, and they all failed due to not finding the module _tkinter.
Which was easily solved by installing python3-tk.

The plugins are optional, and the testplugin checks the basic installation, but maybe it could save some time for some users?

The linux build instructions specifically mention python3-tk as one of the minimum requirements in the "Getting Python 3.4" section. But we'll certainly take it under advisement.

[darkbreath]

I tried the plugin test and it failed. Can anyone help me interpret the errors? This was on an Ubuntu computer:

Status: failed

Verify sys.path settings manually
/usr/local/share/sigil/plugin_launchers/python
/usr/lib/python3.4
/usr/lib/python3.4/plat-x86_64-linux-gnu
/usr/lib/python3.4/lib-dynload
/usr/local/lib/python3.4/dist-packages
/usr/lib/python3/dist-packages
/home/kingmidas/.local/share/sigil-ebook/sigil/plugins/testplugin

Verifying proper Python packages are available
Python Package: PIL Found
Python Package: cssselect Found
Python Package: cssutils Found
Python Package: html5lib Found
Python Package: lxml Found
Python Package: regex Found
Python Package: chardet Found
Python Package: six Found

Verifying Sigil Python Libraries can be found/loaded
Sigil Python library: epub_utils Found
Sigil Python library: quickparser Found
Sigil Python library: compatibility_utils Found
Sigil Python library: sigil_bs4 Found

Verifying Hunspell Spell Checking
Hunspell en_US affix file and dictionary Missing
Hunspell shared library Missing
Hunspell spellchecking works False

Verifying Sigil Gumbo Library operation
Sigil Gumbo BS4 Adapter library Traceback (most recent call last):
File "/usr/local/share/sigil/plugin_launchers/python/launcher.py", line 135, in launch
self.exitcode = target_script.run(container)
File "/home/kingmidas/.local/share/sigil-ebook/sigil/plugins/testplugin/plugin.py", line 182, in run
import sigil_gumbo_bs4_adapter as gumbo_bs4
File "/usr/local/share/sigil/plugin_launchers/python/sigil_gumbo_bs4_adapter.py", line 35, in <module>
import sigil_gumboc as gumboc
File "/usr/local/share/sigil/plugin_launchers/python/sigil_gumboc.py", line 161, in <module>
SourcePosition.EMPTY = SourcePosition.in_dll(_dll, 'kGumboEmptySourcePosition')
ValueError: /usr/bin/python3: undefined symbol: kGumboEmptySourcePosition
Error: /usr/bin/python3: undefined symbol: kGumboEmptySourcePosition

[eschwartz]

Not sure what the deal is with the gumboc error, but the hunspell error implies you a) don't have a recognized en_US hunspell dictionary, and b) do not have libhunspell.so which is supposed to be a stable symlink to libhunspell-1.3.so.${sover}