Register Guidelines E-Books Today's Posts Search

Go Back   MobileRead Forums > E-Book Software > Sigil > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 07-19-2024, 07:06 AM   #1
KarlG
Connoisseur
KarlG began at the beginning.
 
Posts: 62
Karma: 10
Join Date: Mar 2024
Device: none
Borkify Plugin - Possible limitation?

Hello,

I have an epub which causes Sigil to crash when I attempt to restructure it to Sigil Norm.

Normally I would use the borkify plugin on the epub to allow me to post it for further investigation, however the borkify plugin crashes with the following error message

..converting: cover.xhtml with manifest id: cover
Traceback (most recent call last):
File "/usr/local/share/sigil/plugin_launchers/python/launcher.py", line 142, in launch
self.exitcode = target_script.run(container)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/karl/.local/share/sigil-ebook/sigil/plugins/Borkify/plugin.py", line 113, in run
with open(fpath, "wb") as f:
^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmphk94al94/OEBPS/cover.xhtml'
Error: [Errno 2] No such file or directory: '/tmp/tmphk94al94/OEBPS/cover.xhtml'

I've manually unpacked the epub and there isn't an OEBPS directory, so I'm assuming the borkify plugin will only work on standard epubs which already have the correct directory structure.

Any suggestions as to how I could get round this?

Thanks

Karl
KarlG is offline   Reply With Quote
Old 07-19-2024, 07:27 AM   #2
BeckyEbook
Guru
BeckyEbook ought to be getting tired of karma fortunes by now.BeckyEbook ought to be getting tired of karma fortunes by now.BeckyEbook ought to be getting tired of karma fortunes by now.BeckyEbook ought to be getting tired of karma fortunes by now.BeckyEbook ought to be getting tired of karma fortunes by now.BeckyEbook ought to be getting tired of karma fortunes by now.BeckyEbook ought to be getting tired of karma fortunes by now.BeckyEbook ought to be getting tired of karma fortunes by now.BeckyEbook ought to be getting tired of karma fortunes by now.BeckyEbook ought to be getting tired of karma fortunes by now.BeckyEbook ought to be getting tired of karma fortunes by now.
 
BeckyEbook's Avatar
 
Posts: 783
Karma: 2298438
Join Date: Jan 2017
Location: Poland
Device: Various
Borkify plugin is 7 years old, actually only accepts the standard folder structure (Sigil Norm) inside the EPUB file.

The solutions are two:
1. restructure first, then Borkify.
2. improve the Borkify plugin to take into account a different folder structure.

Look at post 8.

Edit: Removed bad code.

Last edited by BeckyEbook; 07-19-2024 at 12:58 PM.
BeckyEbook is offline   Reply With Quote
Advert
Old 07-19-2024, 08:03 AM   #3
KarlG
Connoisseur
KarlG began at the beginning.
 
Posts: 62
Karma: 10
Join Date: Mar 2024
Device: none
Hi Becky,

Thanks for the quick reply. Restructuring is not an option, as it's the restructuring that causes Sigil to crash! :-)

I replaced the code in plugin.py as you suggested. That certainly fixed the crash, but the output file isn't borkified at all.

Last time I had an issue with an epub, I shared it via Proton Drive. If you like I can do this again and then you can let me know after you've grabbed it and I can remove the share.
KarlG is offline   Reply With Quote
Old 07-19-2024, 08:27 AM   #4
Turtle91
A Hairy Wizard
Turtle91 ought to be getting tired of karma fortunes by now.Turtle91 ought to be getting tired of karma fortunes by now.Turtle91 ought to be getting tired of karma fortunes by now.Turtle91 ought to be getting tired of karma fortunes by now.Turtle91 ought to be getting tired of karma fortunes by now.Turtle91 ought to be getting tired of karma fortunes by now.Turtle91 ought to be getting tired of karma fortunes by now.Turtle91 ought to be getting tired of karma fortunes by now.Turtle91 ought to be getting tired of karma fortunes by now.Turtle91 ought to be getting tired of karma fortunes by now.Turtle91 ought to be getting tired of karma fortunes by now.
 
Turtle91's Avatar
 
Posts: 3,228
Karma: 19001261
Join Date: Dec 2012
Location: Charleston, SC today
Device: iPhone 15/11/X/6/iPad 1,2,Air & Air Pro/Surface Pro/Kindle PW & Fire
I also get occasional failures of the restructuring algorithm but usually it’s due to malformed coding in the epub and is easily fixable. I would ignore the whole Borkify issue until you get it to pass the restructuring.

When you first open the epub in Sigil hit F7 to run a well-formed check. That should give you a list of problems to fix. When fixed you should be able to restructure, then Borkify (if you still need to).
Turtle91 is offline   Reply With Quote
Old 07-19-2024, 08:33 AM   #5
KarlG
Connoisseur
KarlG began at the beginning.
 
Posts: 62
Karma: 10
Join Date: Mar 2024
Device: none
Hi Dion,

Already tried that. No problems found, but Sigil still crashes when I attempt a restructure.

Karl
KarlG is offline   Reply With Quote
Advert
Old 07-19-2024, 10:07 AM   #6
KevinH
Sigil Developer
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 8,160
Karma: 5450818
Join Date: Nov 2009
Device: many
You could just pm a private link to DiapDealer or myself here on MR. We can then use it to track down and fix the bug without you needing to make it publicly available.

BTW, Alternatively, try using Find and Replace to a selected body tag contents with the same a single paragraph of text. If Restructuring the that works, then you know the issue is with that file's content links of some sort. But depending on the number of files, this may take a long time.

Or try find and replace and turn on the Text Mode flag and replace all (ie outside tags) with nonsense text.

Or, make a Checkpoint, then run Mend on all HTML files and see if the restructure bug still happens. If not compare it to the Checkpoint to see what changed.

Or try running epubcheck to see if it detects anything unusual about that epub that could give us a hint about what is going on.

Please do let us know what you find, as Sigil should never crash.

Last edited by KevinH; 07-19-2024 at 10:21 AM.
KevinH is offline   Reply With Quote
Old 07-19-2024, 10:45 AM   #7
KarlG
Connoisseur
KarlG began at the beginning.
 
Posts: 62
Karma: 10
Join Date: Mar 2024
Device: none
Just PM'd you the link Kevin. Don't see anything in my outbox, so please let me know if you got the message or not.
KarlG is offline   Reply With Quote
Old 07-19-2024, 12:14 PM   #8
BeckyEbook
Guru
BeckyEbook ought to be getting tired of karma fortunes by now.BeckyEbook ought to be getting tired of karma fortunes by now.BeckyEbook ought to be getting tired of karma fortunes by now.BeckyEbook ought to be getting tired of karma fortunes by now.BeckyEbook ought to be getting tired of karma fortunes by now.BeckyEbook ought to be getting tired of karma fortunes by now.BeckyEbook ought to be getting tired of karma fortunes by now.BeckyEbook ought to be getting tired of karma fortunes by now.BeckyEbook ought to be getting tired of karma fortunes by now.BeckyEbook ought to be getting tired of karma fortunes by now.BeckyEbook ought to be getting tired of karma fortunes by now.
 
BeckyEbook's Avatar
 
Posts: 783
Karma: 2298438
Join Date: Jan 2017
Location: Poland
Device: Various
You are right. My proposal made no sense.
Other adjustments are needed.

Code:
        # write out modified file
        destdir = ""
        filename = ""
        if bk.launcher_version() >= 20190927:
            filename = bk.id_to_bookpath(mid)
            fpath = os.path.join(temp_dir, filename)
        else:
            if "/" in href:
                destdir, filename = unquote(filename).split("/")
            fpath = os.path.join(temp_dir, "OEBPS", destdir, filename)

Last edited by BeckyEbook; 07-19-2024 at 12:57 PM.
BeckyEbook is offline   Reply With Quote
Old 07-19-2024, 12:38 PM   #9
KevinH
Sigil Developer
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 8,160
Karma: 5450818
Join Date: Nov 2009
Device: many
@KarlG,

Well the problem is NOT in Sigil itself but in Qt. It seems that Qt has enabled the use of its JIT compiler in its version of PCRE2 that is used internally in QRegularExpression.

Unfortunately, by turning on the JIT for PCRE2 that makes any use of QRegularExpression in a multiple threaded environment (such as using QtConcurrent) can potentially cause a serious crash.

Really, in a multithreaded environment Qt should not be making code that is non-reentrant.

The only solution is either to patch Qt to not do that (but that leaves out our Linux users who do not build their own Qt), or force the use of the following environment variable which tells Qt to not use its internal Qt PCRE2 in JIT mode in its Regular Expression code:

QT_ENABLE_REGEXP_JIT=0


So try setting this environment variable before firing up Sigil and Restructure to Sigil Norm will no longer crash at all - but will take a really really long time to run in your test case so be very patient.

FWIW, the resulting renames for duplicates will work but will not make numeric sense. You would be much better off renaming those images making their parent folder become part of their name.

I will create a bug report on Qt about this but I would not hold my breath for a bug fix.

KevinH

Last edited by KevinH; 07-19-2024 at 01:10 PM.
KevinH is offline   Reply With Quote
Old 07-19-2024, 02:11 PM   #10
KarlG
Connoisseur
KarlG began at the beginning.
 
Posts: 62
Karma: 10
Join Date: Mar 2024
Device: none
@Becky

Thank you, borkify now working as expected.

@Kevin

Thank you for finding the cause of this.

I've entered 'export QT_ENABLE_REGEXP_JIT=0' in /usr/local/bin/sigil and it does indeed now run to completion without crashing.

Does take a bit longer, but it's not too bad. Took about 45 secs - 1 minute on my system (which is an older system with an Intel Core i5 CPU, so all in all that's not too bad)

At least we now know what the cause was - be interesting to see if Qt actually DO provide a fix for this.

Last edited by KarlG; 07-19-2024 at 06:41 PM.
KarlG is offline   Reply With Quote
Old 07-19-2024, 04:18 PM   #11
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 28,045
Karma: 199464182
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
Out of curiosity, what flavor/version of Linux are you using? I ran the test epub on my Arch system (Qt6.7.2) and couldn't get it to crash on Restructure. I might have just gotten lucky.
DiapDealer is offline   Reply With Quote
Old 07-19-2024, 04:21 PM   #12
KarlG
Connoisseur
KarlG began at the beginning.
 
Posts: 62
Karma: 10
Join Date: Mar 2024
Device: none
Quote:
Originally Posted by DiapDealer View Post
Out of curiosity, what flavor/version of Linux are you using? I ran the test epub on my Arch system (Qt6.7.2) and couldn't get it to crash on Restructure. I might have just gotten lucky.
Xubuntu 24.04
KarlG is offline   Reply With Quote
Old 07-19-2024, 04:59 PM   #13
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 28,045
Karma: 199464182
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
Quote:
Originally Posted by KarlG View Post
Xubuntu 24.04
So System Qt6.4.2, then. Thanks. I'm just wondering if they changed the default setting and when.
DiapDealer is offline   Reply With Quote
Old 07-19-2024, 05:26 PM   #14
KarlG
Connoisseur
KarlG began at the beginning.
 
Posts: 62
Karma: 10
Join Date: Mar 2024
Device: none
Quote:
Originally Posted by DiapDealer View Post
So System Qt6.4.2, then. Thanks. I'm just wondering if they changed the default setting and when.
I always compile Sigil from source and I'm also using Qt 6.7.2
KarlG is offline   Reply With Quote
Old 07-19-2024, 08:43 PM   #15
KevinH
Sigil Developer
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 8,160
Karma: 5450818
Join Date: Nov 2009
Device: many
The Qt bug was not actually in Qt itself but instead in Qt's bundled PCRE2 code. Qt 6.7.2 just missed getting the updated PCRE2 version with that bug fixed.


Sigil has its own copy of PCRE2 that has that same bug, but we do not use the JIT at all.
I will update our version of Sigil to the fixed version as well.

Thank you for your bug report!

Last edited by KevinH; 07-19-2024 at 08:46 PM.
KevinH is offline   Reply With Quote
Reply


Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
kepub limitation? CyberPaul Kobo Reader 16 06-08-2022 05:01 PM
Borkify ePUB Toxaris Plugins 18 01-26-2017 07:21 PM
Kepub limitation aluisscp Kobo Reader 7 06-29-2014 05:22 PM
Library Limitation Catlady General Discussions 11 04-21-2014 08:26 PM
Size limitation? Kumabjorn Calibre 2 02-19-2012 12:12 PM


All times are GMT -4. The time now is 01:52 AM.


MobileRead.com is a privately owned, operated and funded community.