[Conversion Plugin] KFX Input

[Patrichor]

Originally Posted by Patrichor

Trying to convert B0BHPS3DK9, I get a bunch of nav_container representation has extra data errors.

Commenting out the check_empty() calls for nav_container and nav_container representation results in a seemingly valid ePub. Obviously that's a hack, but I wanted to confirm there weren't additional problems.

[jhowell]

Originally Posted by Patrichor

Trying to convert B0BHPS3DK9, I get a bunch of nav_container representation has extra data errors.

Thanks for the bug report. That book is unusual in that it has an enhanced table of contents with images and descriptions. (See the attached image from the Kindle for iOS app.) I have only ever seen that before in Kindle magazines. I am curious about how the authors were able to do that.

Originally Posted by Patrichor

Commenting out the check_empty() calls for nav_container and nav_container representation results in a seemingly valid ePub. Obviously that's a hack, but I wanted to confirm there weren't additional problems.

Your hack is fine since there is no EPUB equivalent to the Kindle enhanced TOC. I will update the plugin to ignore this data in the next release.

ADDED: I will update the plugin to create Kindle-specific markup in the NCX file with the descriptive text when producing an EPUB from this book so that nothing is lost. This extra data will be discarded by calibre unless the "From KFX" toolbar action is used.

[Jaws]

When converting a KFX to an Epub, most of the time the cover is marked correctly as the cover (as viewed in Calibre's editor), but sometimes there's a duplicate "cover" that's actually at the bottom of the file list. OK, fine: Publishers are, umm, technologically challenged and don't even handle this consistently in house.

That's only sometimes, though. Whether the table of contents will be "marked" as the table of contents appears to be completely random: I've done multiple KFX books from the same imprint of the same publisher, all released during the same month, and sometimes the table of contents is marked as a table of contents and sometimes it isn't -- regardless of what epub version it's converted to. This wasn't a problem with AZW or mobi inputs so far as I remember. But I'm not sure if this is:

A KFX Input thingy (bug, feature, you choose);

A general KFX thingy;

Random goofiness from the 'zon;

Publisher incompetence, especially when blindly creating e-book files from Adobe;

My computer getting drunk and just doing random stuff to annoy me;

More than one of the above.

Thoughts?

[jhowell]

Originally Posted by Jaws

When converting a KFX to an Epub, most of the time the cover is marked correctly as the cover (as viewed in Calibre's editor), but sometimes there's a duplicate "cover" that's actually at the bottom of the file list. OK, fine: Publishers are, umm, technologically challenged and don't even handle this consistently in house.

That might be caused by a mistake by the publisher or it could be an artifact left over from replacement of the original cover by calibre. If you can call out some of the books that exhibit this problem I can look into this further.

Originally Posted by Jaws

Whether the table of contents will be "marked" as the table of contents appears to be completely random: I've done multiple KFX books from the same imprint of the same publisher, all released during the same month, and sometimes the table of contents is marked as a table of contents and sometimes it isn't -- regardless of what epub version it's converted to. This wasn't a problem with AZW or mobi inputs so far as I remember.

In old versions of MOBI format there was a single table of contents that was part of the text of the book. It was important that the publisher mark where this occurred so that "Go to the TOC" would function properly on Kindles.

These days the important TOC is the logical one that is separate from the book's contents. Some publishers may have gotten sloppy about marking where the TOC that is part of the text of the book occurs since that link is not needed by the Kindle reading software.

Again some examples would be helpful in determining what is occurring in the cases you have experienced.

[matejdro]

Originally Posted by tomsem

If you are set up to download and de-DRM AZW3, you might try Kindle Unpack plugin. I have had some success creating ePub with it even where calibre fails (or at least I gave up after 5 hours on one I tried to convert today, Kindle Unpack took less than about 30 seconds). (and yes, this was in an Amazon fixed layout format)

Interestingly I have had no issues previously converting to ePub even for fixed layout formats with calibre, but had two of these just in last week. I think there are some older ones I need to check (I turned on automatic conversion to ePub on import only 2 or 3 months ago, so there are probably a number of them that remain in AZW3 only).

Kindle Unpack also exports images to a folder so it would be easy to covert to CBZ or PDF if that would work better.

Huge thanks, that helped a lot!

[eddardStark1942]

Hey, I see that in gather_filetype.py, there's an error in the regex for the 'azw8' branch:

The line:

elif orig_ext == ".azw8" and re.match("^{0-9A-F-}{36}$", orig_dir):

should as far as I can tell read

elif orig_ext == ".azw8" and re.match("^[0-9A-F-]{36}$", orig_dir):

[jhowell]

Originally Posted by eddardStark1942

Hey, I see that in gather_filetype.py, there's an error in the regex for the 'azw8' branch

Thanks for pointing out that error. It is in a part of the code intended to handle Kindle books obtained from either the Kindle for iOS app or the new Kindle for Mac app. Those are unsupported by the DRM removal plugin so that part of KFX Input is not actually useful anyway.

I will fix it in the next plugin release. Perhaps someday it may be worth having.

[Fitz Frobozz]

Thanks so much for the plugin! Works great.

Regarding the Scribe notebook conversion feature, would I need to use the KFX Input plugin CLI on a copied version of my .notebook folder? I can access the contents of my Scribe through File Explorer in Windows, but I haven't yet figured out how to navigate to its contents via CLI as there is no associated drive letter. (I'm guessing I somehow missed a step.)

[jhowell]

Originally Posted by Fitz Frobozz

Regarding the Scribe notebook conversion feature, would I need to use the KFX Input plugin CLI on a copied version of my .notebook folder?

Yes, you need to copy it.

When the Scribe was first released it was possible to mount its file system as a drive in Windows and access notebooks directly. With the change over to MTP for file transfer in recent Scribe firmware that no longer works.

Like most software this plugin only sees files accessible on a mounted drive, so you need to copy the notebook file from the Scribe before it can be converted.

[Fitz Frobozz]

Originally Posted by jhowell

Yes, you need to copy it.

When the Scribe was first released it was possible to mount its file system as a drive in Windows and access notebooks directly. With the change over to MTP for file transfer in recent Scribe firmware that no longer works.

Like most software this plugin only sees files accessible on a mounted drive, so you need to copy the notebook file from the Scribe before it can be converted.

Great, thanks so much for the quick reply. Works like a charm. [Additional question redacted, as it turns out I was just making a dumb mistake.]

[sts]

Thank you very much for this great plugin. I mostly use it on command line (the "calibre-debug -r..." way), works flawlessly so far. But recently I got some strange warnings:

"WARNING: KFX font family name serif fixed to multiple names: serif and Serif"

Several thousand times for each book. May I safely ignore this warning (my reading device overrides any set fonts anyway)?

[jhowell]

Originally Posted by sts

Thank you very much for this great plugin. I mostly use it on command line (the "calibre-debug -r..." way), works flawlessly so far. But recently I got some strange warnings:

"WARNING: KFX font family name serif fixed to multiple names: serif and Serif"

Several thousand times for each book. May I safely ignore this warning (my reading device overrides any set fonts anyway)?

I would like to look into this further. Can you let me know some books that result in this warning? The ten character ASIN would be helpful.

[sts]

Originally Posted by jhowell

I would like to look into this further. Can you let me know some books that result in this warning? The ten character ASIN would be helpful.

i.e.: B087G2KJ7B (Philip P. Peterson - Vakuum)

[jhowell]

Version 2.11.0 - 22 May 2024

Handle books containing an enhanced table of contents with chapter descriptions. This will be converted to Kindle-specific markup in the NCX file when producing an EPUB. This extra data will be discarded by calibre unless the "From KFX" toolbar action is used. (Fixes "ERROR: nav_container representation has extra data: {$146:...")

Fix page-progression-direction not being set to rtl in books with vertical right-to-left text.

Fix file gathering code intended for possible future use with books obtained from the Kindle for iOS or the new Kindle for Mac app.

Improve handling of font names that differ only in case. (Fixes "WARNING: KFX font family name serif fixed to multiple names: serif and Serif")

Support for inline interactive audio and video content in reflowable books.

[GreenEyedLady]

I have just installed this plugin. Previous I've used Kindle for PC 1.17 to convert books, but wanted to purchase a newer book so installed this plugin. I also installed Kindle for PC 2.3.5 (70840) and have latest calibre update.

Have been using Tools 10.0.9 since last fall without problem on pre 2023 books.

Something is not working for me. Posted below is a book I tried to import into Calibre. It is a past purchased book that previously was imported without problem as aw3, now it is KFX.

Spoiler:

"calibre, version 7.11.0
ERROR: Loading book failed: Failed to open the book at C:\Users\dlare\Calibre Library\Steven L. Peck\A Short Stay in Hell (8946)\metadata.opf. Click "Show details" for more info.

Failed to convert book: C:\Users\dlare\Calibre Library\Steven L. Peck\A Short Stay in Hell (8946)\metadata.opf with error:
InputFormatPlugin: HTML Input running
on C:\Users\dlare\Calibre Library\Steven L. Peck\A Short Stay in Hell (8946)\metadata.opf
Parsing all content...
Failed to run pipe worker with command: from calibre.srv.render_book import viewer_main; viewer_main()
Traceback (most recent call last):
File "runpy.py", line 198, in _run_module_as_main
File "runpy.py", line 88, in _run_code
File "site.py", line 83, in <module>
File "site.py", line 78, in main
File "site.py", line 50, in run_entry_point
File "calibre\utils\ipc\worker.py", line 196, in main
File "<string>", line 1, in <module>
File "calibre\srv\render_book.py", line 948, in viewer_main
File "calibre\srv\render_book.py", line 939, in render_for_viewer
File "calibre\srv\render_book.py", line 916, in render
File "calibre\ebooks\oeb\iterator\book.py", line 64, in extract_book
File "calibre\customize\conversion.py", line 245, in __call__
File "calibre\ebooks\conversion\plugins\html_input. py", line 113, in convert
File "calibre\ebooks\conversion\plumber.py", line 1336, in create_oebbook
File "calibre\ebooks\oeb\reader.py", line 97, in __call__
File "calibre\ebooks\oeb\reader.py", line 724, in _all_from_opf
File "calibre\ebooks\oeb\reader.py", line 369, in _spine_from_opf
calibre.ebooks.oeb.base.OEBError: Spine is empty"

I'm not sure if this can be addressed here or not, but thank you for your time if you can help.

Deb