[Conversion Plugin] KFX Input

[jhowell]

Originally Posted by dunhill

Ask the KFX Entry plugin how to use it from linux?
I've tried using kindle for pc and Kindle Previewer with wine but it crashes.
Any alternative to use?
I thought that it could contain the kindle for pc or Kindle Previewer 3 (3.91.0) libraries that can be used in linux.

Converting to KFX from another format uses the KFX Output plugin, not KFX Input. It does require that Kindle Previewer be installed and working, but does not need Kindle for PC.

Some people have reported being able to get specific versions to run under Wine. See the Installation section in the first post of the KFX Output thread for details. If you do manage to make it work please provide details of how you did it and I will update the instructions for Linux users. Sorry, but I cannot help beyond that.

[texasnightowl]

Have a kfx file importing as kfx-zip. Says decryption succeeded and then starts processing containers and one line I see says:

Exception: Book is incomplete. All of the KFX container files that make up the book must be combined into a KFX-ZIP file for successful conversion. (Missing containers CR!20YE8Z24YH5Z19FFWJJB25N0G9XE, CR!8Z2PAMAFR97CSCESJ53SF77WK13J)

What is interesting is that those two containers listed are not the 2 containers that are in the folder with the azw file. Is it common for CR! files that are in the folder call other containers that are not in the folder?

edit: forgot to say downloaded using k4pc 2.4.0 (70904)

Spoiler:

ApplicationPaletteChange event ignored
Using calibre Qt style: True
EpubMerge: DEBUG: 2025-02-23 14:12:51,187: calibre_plugins.epubmerge.epubmerge_plugin(156): macmenuhack file_path:C:\Users\*myusername*\AppData\Roaming\ca libre\plugins\fanficfare_macmenuhack.txt
FFF: DEBUG: 2025-02-23 14:12:51,212: calibre_plugins.fanficfare_plugin.fff_plugin(217): Plugin FanFicFare macmenuhack file_path:C:\Users\*myusername*\AppData\Roaming\ca libre\plugins\fanficfare_macmenuhack.txt
calibre Debug log
calibre 7.26 embedded-python: True
Windows-10-10.0.19045-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.19045')
Python 3.11.5
Windows: ('10', '10.0.19045', 'SP0', 'Multiprocessor Free')
Interface language: None
EXE path: C:\Program Files\Calibre2\calibre-debug.exe
Successfully initialized third party plugins: DeACSM (0, 0, 16) && Gather KFX-ZIP (from KFX Input) (2, 20, 0) && DeDRM (10, 0, 9) && Package KFX (from KFX Input) (2, 20, 0) && Amazon.com Multiple Countries (1, 0, 0) && Apple Books covers (0, 2, 0) && Barnes & Noble (1, 5, 6) && Count Pages (1, 13, 6) && EpubMerge (3, 2, 0) && EpubSplit (3, 10, 0) && Extract ISBN (1, 6, 5) && FanFicFare (4, 42, 0) && FictionDB (1, 4, 2) && Find Duplicates (1, 10, 9) && Goodreads (1, 8, 2) && Goodreads Sync (1, 16, 11) && KFX metadata reader (from KFX Input) (2, 20, 0) && From KFX (2, 20, 0) && KFX Input (2, 20, 0) && Set KFX metadata (from KFX Output) (2, 14, 0) && KFX Output (2, 14, 0) && KindleUnpack - The Plugin (0, 83, 7) && Kobo Metadata (1, 10, 0) && Manage Series (1, 5, 2) && Modify ePub (1, 8, 5) && Obok DeDRM (10, 0, 3) && Open With (1, 8, 4) && Quality Check (1, 13, 16)
calibre 7.26 embedded-python: True
Windows-10-10.0.19045-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.19045')
Python 3.11.5
Windows: ('10', '10.0.19045', 'SP0', 'Multiprocessor Free')
Interface language: None
EXE path: C:\Program Files\Calibre2\calibre-debug.exe
Successfully initialized third party plugins: DeACSM (0, 0, 16) && Gather KFX-ZIP (from KFX Input) (2, 20, 0) && DeDRM (10, 0, 9) && Package KFX (from KFX Input) (2, 20, 0) && Amazon.com Multiple Countries (1, 0, 0) && Apple Books covers (0, 2, 0) && Barnes & Noble (1, 5, 6) && Count Pages (1, 13, 6) && EpubMerge (3, 2, 0) && EpubSplit (3, 10, 0) && Extract ISBN (1, 6, 5) && FanFicFare (4, 42, 0) && FictionDB (1, 4, 2) && Find Duplicates (1, 10, 9) && Goodreads (1, 8, 2) && Goodreads Sync (1, 16, 11) && KFX metadata reader (from KFX Input) (2, 20, 0) && From KFX (2, 20, 0) && KFX Input (2, 20, 0) && Set KFX metadata (from KFX Output) (2, 14, 0) && KFX Output (2, 14, 0) && KindleUnpack - The Plugin (0, 83, 7) && Kobo Metadata (1, 10, 0) && Manage Series (1, 5, 2) && Modify ePub (1, 8, 5) && Obok DeDRM (10, 0, 3) && Open With (1, 8, 4) && Quality Check (1, 13, 16)
QPA platform: windows
devicePixelRatio: 1.25
logicalDpi: 96.0 x 96.0
physicalDpi: 87.59498746867168 x 87.34567164179104
[0.00] Starting up...
[0.00] Showing splash screen...
[1.34] splash screen shown
[1.34] Initializing db...
[1.41] db initialized
[1.41] Constructing main UI...
DEBUG: 0.0 obok::utilities.py - loading translations
DEBUG: 0.0 obok::dialogs.py - loading translations
DEBUG: 0.0 obok::config.py - loading translations
DEBUG: 0.0 obok::action_err.py - loading translations
[3.20] GUI main window shown
[3.36] main UI initialized...
[3.36] Hiding splash screen
[3.36] splash screen hidden
[3.36] Started up in 3.36 seconds with 1457 books
Gather KFX-ZIP (from KFX Input) 2.20.0: Importing D:/Ebooks/K4PC2.4.0/My Kindle Content/B007V65E98_EBOK/B007V65E98_EBOK.azw
Gather KFX-ZIP (from KFX Input): Gathered 4 file(s) as C:\Users\*myusername*\AppData\Local\Temp\calibre_i hp2kh2w\hxl3f0vs.kfx-zip
DeDRM v10.0.9: Trying to decrypt hxl3f0vs.kfx-zip
Decrypting KFX-ZIP ebook: hxl3f0vs
Got DSN key from database default_key_1740288937
Found 9 keys to try after 0.3 seconds
Decrypting KFX DRM voucher: amzn1.drm-voucher.v1.126a7db1-bff3-495e-9ba4-8a49ba350939.voucher
Decryption failed, trying next fallback
Decryption succeeded
KFX DRM voucher successfully decrypted
Decrypting KFX DRMION: B007V65E98_EBOK.azw
Decryption succeeded after 0.4 seconds
DeDRM v10.0.9: Finished after 0.4 seconds
Package KFX (from KFX Input) 2.20.0: Packaging C:\Users\*myusername*\AppData\Local\Temp\calibre_i hp2kh2w\40vxxnbc.kfx-zip
Processing container: B007V65E98_EBOK.azw
Processing container: CR!7APPGJDWAN44KDMMAC7AXN53D4ZM.azw.md
Processing container: CR!SA1A0XFJAD2S3BGSPNX6JKT20394.azw.res
asset_id (CR!20YE8Z24YH5Z19FFWJJB25N0G9XE) != main container_id (CR!73WZN8NPYD5D78RW2S3D37687M3V)
Running file type plugin Package KFX (from KFX Input) failed with traceback:
Traceback (most recent call last):
File "calibre\customize\ui.py", line 200, in _run_filetype_plugins
File "calibre_plugins.kfx_input.package_filetype", line 23, in run
File "calibre_plugins.kfx_input.package_filetype", line 34, in package_kfx
File "calibre_plugins.kfx_input.kfxlib.yj_book", line 79, in convert_to_single_kfx
File "calibre_plugins.kfx_input.kfxlib.yj_book", line 224, in decode_book
File "calibre_plugins.kfx_input.kfxlib.yj_structure ", line 276, in check_consistency
Exception: Book is incomplete. All of the KFX container files that make up the book must be combined into a KFX-ZIP file for successful conversion. (Missing containers CR!20YE8Z24YH5Z19FFWJJB25N0G9XE, CR!8Z2PAMAFR97CSCESJ53SF77WK13J)
KFX metadata reader (from KFX Input) activated for C:\Users\*myusername*\AppData\Local\Temp\calibre_i hp2kh2w\g_d55bpv_add_books\0\B007V65E98_EBOK.kfx-zip
Added MoonRush: An Action-Packed Adventure (From the author of 30 Pieces of Silver and Encrypted) to db in: 0.1
Added 1 books in 2.1 seconds
Worker Launch took: 0.05 seconds
Job: 0 Convert book 1 of 1 (MoonRush: An Action-Packed Adventure (From the author of 30 Pieces of Silver and Encrypted)) finished
Convert book 1 of 1 (MoonRush: An Action-Packed Adventure (From the author of 30 Pieces of Silver and Encrypted))

DeDRM v10.0.9: Trying to decrypt aqvhwjf3.kfx-zip
Decrypting KFX-ZIP ebook: aqvhwjf3
Got DSN key from database default_key_1740288937
Found 9 keys to try after 0.3 seconds
The .kfx-zip archive does not contain an encrypted DRMION file
Decryption succeeded after 0.3 seconds
DeDRM v10.0.9: Finished after 0.3 seconds
Conversion options changed from defaults:

verbose: 2

output_profile: 'kindle_oasis'

read_metadata_from_opf: 'C:\\Users\\*myusername*\\AppData\\Local\\Temp\\ca libre_ihp2kh2w\\jnmhji9u.opf'

Resolved conversion options

calibre version: 7.26.0

{'add_alt_text_to_img': False,

'allow_conversion_with_errors': False,

'asciiize': False,

'author_sort': None,

'authors': None,

'base_font_size': 0.0,

'book_producer': None,

'change_justification': 'original',

'chapter': "//*[((name()='h1' or name()='h2') and re:test(., "

"'\\s*((chapter|book|section|part)\\s+)|((prolog|p rologue|epilogue)(\\s+|$))', "

"'i')) or @class = 'chapter']",

'chapter_mark': 'pagebreak',

'comments': None,

'cover': None,

'debug_pipeline': None,

'dehyphenate': True,

'delete_blank_paragraphs': True,

'disable_font_rescaling': False,

'dont_compress': False,

'duplicate_links_in_toc': False,

'embed_all_fonts': False,

'embed_font_family': None,

'enable_heuristics': False,

'expand_css': False,

'extra_css': None,

'extract_to': None,

'filter_css': '',

'fix_indents': True,

'font_size_mapping': None,

'format_scene_breaks': True,

'html_unwrap_factor': 0.4,

'input_encoding': None,

'input_profile': <calibre.customize.profiles.InputProfile object at 0x000001CB93F04050>,

'insert_blank_line': False,

'insert_blank_line_size': 0.5,

'insert_metadata': False,

'isbn': None,

'italicize_common_cases': True,

'keep_ligatures': False,

'language': None,

'level1_toc': None,

'level2_toc': None,

'level3_toc': None,

'line_height': 0.0,

'linearize_tables': False,

'margin_bottom': 5.0,

'margin_left': 5.0,

'margin_right': 5.0,

'margin_top': 5.0,

'markup_chapter_headings': True,

'max_toc_links': 50,

'minimum_line_height': 120.0,

'mobi_toc_at_start': False,

'no_chapters_in_toc': False,

'no_inline_navbars': False,

'no_inline_toc': False,

'output_profile': <calibre.customize.profiles.KindleOasisOutput object at 0x000001CB93EF2C90>,

'page_breaks_before': '/',

'prefer_author_sort': False,

'prefer_metadata_cover': False,

'pretty_print': False,

'pubdate': None,

'publisher': None,

'rating': None,

'read_metadata_from_opf': 'C:\\Users\\*myusername*\\AppData\\Local\\Temp\\ca libre_ihp2kh2w\\jnmhji9u.opf',

'remove_fake_margins': True,

'remove_first_image': False,

'remove_paragraph_spacing': False,

'remove_paragraph_spacing_indent_size': 1.5,

'renumber_headings': True,

'replace_scene_breaks': '',

'search_replace': '[]',

'series': None,

'series_index': None,

'share_not_sync': False,

'smarten_punctuation': False,

'sr1_replace': None,

'sr1_search': None,

'sr2_replace': None,

'sr2_search': None,

'sr3_replace': None,

'sr3_search': None,

'start_reading_at': None,

'subset_embedded_fonts': False,

'tags': None,

'timestamp': None,

'title': None,

'title_sort': None,

'toc_filter': None,

'toc_threshold': 6,

'toc_title': None,

'transform_css_rules': '[]',

'transform_html_rules': '[]',

'unsmarten_punctuation': False,

'unwrap_lines': True,

'use_auto_toc': False,

'verbose': 2}

DeDRM v10.0.9: Trying to decrypt n7oclm80.kfx-zip
Decrypting KFX-ZIP ebook: n7oclm80
Got DSN key from database default_key_1740288937
Found 9 keys to try after 0.0 seconds
The .kfx-zip archive does not contain an encrypted DRMION file
Decryption succeeded after 0.0 seconds
DeDRM v10.0.9: Finished after 0.0 seconds
InputFormatPlugin: KFX Input running

on C:\Users\*myusername*\AppData\Local\Temp\calibre_i hp2kh2w\p2r6myjn.kfx-zip

Software versions: KFX Input 2.20.0, calibre 7.26, Windows-10-10.0.19045-SP0

KFX Input plugin help is available at https://www.mobileread.com/forums/sh...d.php?t=291290

Converting C:\Users\*myusername*\AppData\Local\Temp\calibre_i hp2kh2w\p2r6myjn.kfx-zip

Processing container: B007V65E98_EBOK.azw

Processing container: CR!7APPGJDWAN44KDMMAC7AXN53D4ZM.azw.md

Processing container: CR!SA1A0XFJAD2S3BGSPNX6JKT20394.azw.res

ERROR: asset_id (CR!20YE8Z24YH5Z19FFWJJB25N0G9XE) != main container_id (CR!73WZN8NPYD5D78RW2S3D37687M3V)

Traceback (most recent call last):

File "calibre_plugins.kfx_input.__init__", line 97, in convert

File "calibre_plugins.kfx_input.kfxlib.yj_book", line 224, in decode_book

File "calibre_plugins.kfx_input.kfxlib.yj_structure ", line 276, in check_consistency

Exception: Book is incomplete. All of the KFX container files that make up the book must be combined into a KFX-ZIP file for successful conversion. (Missing containers CR!20YE8Z24YH5Z19FFWJJB25N0G9XE, CR!8Z2PAMAFR97CSCESJ53SF77WK13J)



During handling of the above exception, another exception occurred:



Traceback (most recent call last):

File "runpy.py", line 198, in _run_module_as_main

File "runpy.py", line 88, in _run_code

File "site.py", line 83, in <module>

File "site.py", line 78, in main

File "site.py", line 50, in run_entry_point

File "calibre\utils\ipc\worker.py", line 215, in main

File "calibre\gui2\convert\gui_conversion.py", line 38, in gui_convert_override

File "calibre\gui2\convert\gui_conversion.py", line 25, in gui_convert

File "calibre\ebooks\conversion\plumber.py", line 1128, in run

File "calibre\customize\conversion.py", line 242, in __call__

File "calibre_plugins.kfx_input.__init__", line 120, in convert

calibre.ebooks.conversion.ConversionUserFeedBack: {"msg": "<b>Cannot convert MoonRush: An Action-Packed Adventure (From the author of 30 Pieces of Silver and Encrypted)</b><br><br>Exception('Book is incomplete. All of the KFX container files that make up the book must be combined into a KFX-ZIP file for successful conversion. (Missing containers CR!20YE8Z24YH5Z19FFWJJB25N0G9XE, CR!8Z2PAMAFR97CSCESJ53SF77WK13J)')", "level": "error", "det_msg": "", "title": "KFX conversion failed"}


[108.27] Shutdown starting...
[110.53] Shutdown complete, quitting...

[jhowell]

Originally Posted by texasnightowl

Have a kfx file importing as kfx-zip. Says decryption succeeded and then starts processing containers and one line I see says:

Exception: Book is incomplete. All of the KFX container files that make up the book must be combined into a KFX-ZIP file for successful conversion. (Missing containers CR!20YE8Z24YH5Z19FFWJJB25N0G9XE, CR!8Z2PAMAFR97CSCESJ53SF77WK13J)

Based on the error log it appears that the KFX format for that book was generated incorrectly by Amazon. Possibly one of their early efforts when KFX was just being introduced. Amazon shows that book is no longer for sale.

I will try to alter the plugin to try to work around the problem. If so I will put out a new release.

[texasnightowl]

Thanks! That is the only one, aside from one TPZ book, that "failed" out of 1463 books. That said, this was strictly based on K4PC testing. I do have this book in my main library from a d&t file that is fine. And yes, it is apparently no longer for sale.

I do have 6 other books that also imported as kfx-zip but will convert to other formats. I'm going to go into debug mode and check those out again and see what happens.

edit: So the other 6 books are all fine. I don't know what happened the first time, but I removed the download in K4PC then re-downloaded all of them and re-imported into Calibre and all 6 are absolutely fine and show as kfx and convert fine.

[jhowell]

Originally Posted by texasnightowl

Thanks! That is the only one, aside from one TPZ book, that "failed" out of 1463 books. That said, this was strictly based on K4PC testing. I do have this book in my main library from a d&t file that is fine. And yes, it is apparently no longer for sale.

After more research I find that there are just too many possibilities of what might be wrong with that book for me to develop a workaround. Since you have a working copy from another source and this is likely a one-off problem we can just leave it at that.

However, I am not opposed to finding out what is wrong with that book, purely for research purposes. If you want you can PM me a link to some online storage containing the kfx-zip and the associated DRM key. It is fine with me if you do not want to do that.

Originally Posted by texasnightowl

edit: So the other 6 books are all fine. I don't know what happened the first time, but I removed the download in K4PC then re-downloaded all of them and re-imported into Calibre and all 6 are absolutely fine and show as kfx and convert fine.

I have seen reports in the past of importing too many books at once causing random DRM removal failures, likely a flaw in the DeDRM plugin that only occurs under stress.

[JSWolf]

Originally Posted by jhowell

I have seen reports in the past of importing too many books at once causing random DRM removal failures, likely a flaw in the DeDRM plugin that only occurs under stress.

That should have been fixed.

[DNSB]

Originally Posted by JSWolf

That should have been fixed.

Sadly, it can still occur. A nice random and non-reproducible bug.

I can import 50 books and no problems. I do the same 50 books again and a couple of them show up as azw and were not decrypted.

[texasnightowl]

Originally Posted by jhowell

After more research I find that there are just too many possibilities of what might be wrong with that book for me to develop a workaround. Since you have a working copy from another source and this is likely a one-off problem we can just leave it at that.

Completely understand. No worries. The K4PC method was just for my own testing purposes since I downloaded everything by azw3 as a backup. I wanted to make sure I could do it going forward if needed! (or until Amazon gets it locked down.) And this is an older book that is no longer sold, so there is that too!

However, I am not opposed to finding out what is wrong with that book, purely for research purposes. If you want you can PM me a link to some online storage containing the kfx-zip and the associated DRM key. It is fine with me if you do not want to do that.

Sure...let me just zip them up. I'll zip the original book folder which will have the voucher file and I'll zip the folder from Calibre with the hopefully decrypted kfx-zip.

[texasnightowl]

Originally Posted by jhowell

I have seen reports in the past of importing too many books at once causing random DRM removal failures, likely a flaw in the DeDRM plugin that only occurs under stress.

Yeah...I used the add books from folder method, so it was importing over 1400 at one go! And actually, during the initial run, it thought there 2700 because it thought the res.md files in each folder were also books. (In later runs I added a rule to ignore files unless they were .azw.) So maybe...