Borkify Plugin - Possible limitation?

[KarlG]

Hello,

I have an epub which causes Sigil to crash when I attempt to restructure it to Sigil Norm.

Normally I would use the borkify plugin on the epub to allow me to post it for further investigation, however the borkify plugin crashes with the following error message

..converting: cover.xhtml with manifest id: cover
Traceback (most recent call last):
File "/usr/local/share/sigil/plugin_launchers/python/launcher.py", line 142, in launch
self.exitcode = target_script.run(container)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/karl/.local/share/sigil-ebook/sigil/plugins/Borkify/plugin.py", line 113, in run
with open(fpath, "wb") as f:
^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmphk94al94/OEBPS/cover.xhtml'
Error: [Errno 2] No such file or directory: '/tmp/tmphk94al94/OEBPS/cover.xhtml'

I've manually unpacked the epub and there isn't an OEBPS directory, so I'm assuming the borkify plugin will only work on standard epubs which already have the correct directory structure.

Any suggestions as to how I could get round this?

Thanks

Karl

[BeckyEbook]

Borkify plugin is 7 years old, actually only accepts the standard folder structure (Sigil Norm) inside the EPUB file.

The solutions are two:
1. restructure first, then Borkify.
2. improve the Borkify plugin to take into account a different folder structure.

Look at post 8.

Edit: Removed bad code.

[KarlG]

Hi Becky,

Thanks for the quick reply. Restructuring is not an option, as it's the restructuring that causes Sigil to crash! :-)

I replaced the code in plugin.py as you suggested. That certainly fixed the crash, but the output file isn't borkified at all.

Last time I had an issue with an epub, I shared it via Proton Drive. If you like I can do this again and then you can let me know after you've grabbed it and I can remove the share.

[Turtle91]

I also get occasional failures of the restructuring algorithm but usually it’s due to malformed coding in the epub and is easily fixable. I would ignore the whole Borkify issue until you get it to pass the restructuring.

When you first open the epub in Sigil hit F7 to run a well-formed check. That should give you a list of problems to fix. When fixed you should be able to restructure, then Borkify (if you still need to).

[KarlG]

Hi Dion,

Already tried that. No problems found, but Sigil still crashes when I attempt a restructure.

Karl

[KevinH]

You could just pm a private link to DiapDealer or myself here on MR. We can then use it to track down and fix the bug without you needing to make it publicly available.

BTW, Alternatively, try using Find and Replace to a selected body tag contents with the same a single paragraph of text. If Restructuring the that works, then you know the issue is with that file's content links of some sort. But depending on the number of files, this may take a long time.

Or try find and replace and turn on the Text Mode flag and replace all (ie outside tags) with nonsense text.

Or, make a Checkpoint, then run Mend on all HTML files and see if the restructure bug still happens. If not compare it to the Checkpoint to see what changed.

Or try running epubcheck to see if it detects anything unusual about that epub that could give us a hint about what is going on.

Please do let us know what you find, as Sigil should never crash.

[KarlG]

Just PM'd you the link Kevin. Don't see anything in my outbox, so please let me know if you got the message or not.

[BeckyEbook]

You are right. My proposal made no sense.
Other adjustments are needed.

        # write out modified file
        destdir = ""
        filename = ""
        if bk.launcher_version() >= 20190927:
            filename = bk.id_to_bookpath(mid)
            fpath = os.path.join(temp_dir, filename)
        else:
            if "/" in href:
                destdir, filename = unquote(filename).split("/")
            fpath = os.path.join(temp_dir, "OEBPS", destdir, filename)

[KevinH]

@KarlG,

Well the problem is NOT in Sigil itself but in Qt. It seems that Qt has enabled the use of its JIT compiler in its version of PCRE2 that is used internally in QRegularExpression.

Unfortunately, by turning on the JIT for PCRE2 that makes any use of QRegularExpression in a multiple threaded environment (such as using QtConcurrent) can potentially cause a serious crash.

Really, in a multithreaded environment Qt should not be making code that is non-reentrant.

The only solution is either to patch Qt to not do that (but that leaves out our Linux users who do not build their own Qt), or force the use of the following environment variable which tells Qt to not use its internal Qt PCRE2 in JIT mode in its Regular Expression code:

QT_ENABLE_REGEXP_JIT=0


So try setting this environment variable before firing up Sigil and Restructure to Sigil Norm will no longer crash at all - but will take a really really long time to run in your test case so be very patient.

FWIW, the resulting renames for duplicates will work but will not make numeric sense. You would be much better off renaming those images making their parent folder become part of their name.

I will create a bug report on Qt about this but I would not hold my breath for a bug fix.

KevinH

[KarlG]

@Becky

Thank you, borkify now working as expected.

@Kevin

Thank you for finding the cause of this.

I've entered 'export QT_ENABLE_REGEXP_JIT=0' in /usr/local/bin/sigil and it does indeed now run to completion without crashing.

Does take a bit longer, but it's not too bad. Took about 45 secs - 1 minute on my system (which is an older system with an Intel Core i5 CPU, so all in all that's not too bad)

At least we now know what the cause was - be interesting to see if Qt actually DO provide a fix for this.

[DiapDealer]

Out of curiosity, what flavor/version of Linux are you using? I ran the test epub on my Arch system (Qt6.7.2) and couldn't get it to crash on Restructure. I might have just gotten lucky.

[KarlG]

Originally Posted by DiapDealer

Out of curiosity, what flavor/version of Linux are you using? I ran the test epub on my Arch system (Qt6.7.2) and couldn't get it to crash on Restructure. I might have just gotten lucky.

Xubuntu 24.04

[DiapDealer]

Originally Posted by KarlG

Xubuntu 24.04

So System Qt6.4.2, then. Thanks. I'm just wondering if they changed the default setting and when.

[KarlG]

Originally Posted by DiapDealer

So System Qt6.4.2, then. Thanks. I'm just wondering if they changed the default setting and when.

I always compile Sigil from source and I'm also using Qt 6.7.2

[KevinH]

The Qt bug was not actually in Qt itself but instead in Qt's bundled PCRE2 code. Qt 6.7.2 just missed getting the updated PCRE2 version with that bug fixed.


Sigil has its own copy of PCRE2 that has that same bug, but we do not use the JIT at all.
I will update our version of Sigil to the fixed version as well.

Thank you for your bug report!