[Conversion Plugin] KFX Input

[jhowell]

Originally Posted by luisdent

I have decent luck so far, but one book "inferno" by dan brown won't convert to kfx.

This thread for for the KFX Input plugin that converts from KFX to other formats. Did you mean to post your question in the KFX Output thread?

If so then you will need to edit the book to remove or replace the images that are causing conversion to fail.

[luisdent]

Originally Posted by jhowell

This thread for for the KFX Input plugin that converts from KFX to other formats. Did you mean to post your question in the KFX Output thread?

If so then you will need to edit the book to remove or replace the images that are causing conversion to fail.

oops. yes I did mean to do that. i was looking at all the kfx threads at once.

question has been moved.

[jhowell]

Version 1.1.0 - 31 Jan 2018

Added a built-in File Type Plugin to automatically gather the component files of a book and produce KFX-ZIP format during book import. This eliminates the need for the user to produce KFX-ZIP files manually, simplifying the process of importing KFX books into calibre. (DRM removal must still be accomplished by other means, prior to import.)

Added another built-in File Type Plugin to combine the components of a KFX-ZIP into monolithic KFX format. This process will only succeed if all of the components of the book are present in the KFX-ZIP and do not contain DRM. This makes it easier to determine whether on not the import was successful since a successful import will result in KFX format and an unsuccessful one will result in KFX-ZIP format.

Added a Metadata Reader Plugin to extract KFX book metadata. This allows the title, author, and other metadata to appear in the calibre book entry when importing a KFX book.

Ignore approximate page numbers created by the KFX Output plugin.

Handle changes to KPF format produced by Kindle Create 1.3.30.0 and Kindle Previewer 3.17.1.

Fix KPF format not being accepted for conversion using the GUI interface.

Allow conversion of additional KFX features.

Improve detection of unsupported KFX features.

[jraffa]

Thanks for the plugin

this plugin accepts a ZIP archive with the extension ".kfx-zip" containing multiple KFX files.

this quote in the original post is no longer true, right?

[DoctorOhh]

Originally Posted by jraffa

this quote in the original post is no longer true, right?

Why would you think the info in the first post wasn't accurate? It was last updated only a week ago. Did you try using this format and have it fail? My guess is the info in the 1st post is accurate, but I don't have first hand experience to back up that thought.

This is a quote from the first post from the recent update.

Added another built-in File Type Plugin to combine the components of a KFX-ZIP into monolithic KFX format. This process will only succeed if all of the components of the book are present in the KFX-ZIP and do not contain DRM. This makes it easier to determine whether on not the import was successful since a successful import will result in KFX format and an unsuccessful one will result in KFX-ZIP format.

[jhowell]

Originally Posted by jraffa

this quote in the original post is no longer true, right?

It is still true. The latest version of the plugin continues to accept KFX-ZIP as an input format for conversion. In addition it can accept either monolithic KFX format or KPF format (generated by Kindle Create) for conversion.

As with the previous version of the plugin, users have the option of creating a ZIP file of the book components (with a .kfx-zip extension) and then importing that. The latest plugin version also allows the user to just import the main KFX file of the book without the need to create a ZIP file first. The plugin will gather the other files that make up the book and produce a KFX-ZIP automatically during import.

As a separate step it will then take the KFX-ZIP and combine it into a monolithic KFX file. This step was added so that import results in a format that is suitable for purposes other than just conversion. A monolithic KFX file can be sideloaded to a Kindle device or viewed using the Kindle Previewer.

In any case it is up to the user to remove DRM before importing. I am still hopeful that the DeDRM software will someday be updated to automatically remove DRM from KFX-ZIP format during the import process, but there is no sign that anyone is working on that.

[GuerinZ]

I was getting a errors converting my latest KPF formatted files and I simply changed them from self.log.error to self.log.debug and the conversions went ok. Not a lot of documentation to figure out why this is an issue. Hopefully, you can figure it out

section_position_id_map has 4240 eids, 2 missing: HGd8K2A1Q-2ANB28cOTEWA111/4241, HGd8K2A1Q-2ANB28cOTEWA111/4242
Built location_map with 7191 locations from 4278 eids and 790910 positions
missing numeric id for kfx_id symbols: HGd8K2A1Q-2ANB28cOTEWA111, HGd8K2A1Q-2ANB28cOTEWA112
kfxgen version: kfxlib-20180319/KC-1.5.60.0/716.

[jhowell]

Originally Posted by GuerinZ

I was getting a errors converting my latest KPF formatted files and I simply changed them from self.log.error to self.log.debug and the conversions went ok. Not a lot of documentation to figure out why this is an issue. Hopefully, you can figure it out

Thanks for the error report!

This plugin reflects my understanding of KFX and KPF, reverse engineered by examining books. I intentionally put in checks so that when unexpected data is encountered it causes the conversion to fail so that I would be made aware of areas where my understanding is incorrect. The KPF file you converted contains something strange that I have not been able to replicate with my own test documents.

If you feel comfortable doing so you can email me a copy of the KPF file. My email address can be found in the plugin source files. If you would rather not, I understand.

In the next plugin release I will add a tweak that can be set to allow conversion to proceed even if errors are detected. That will provide a work-around without having to patch the plugin.

[jhowell]

Version 1.2.0 - 02 Apr 2018

Added an option to proceed with conversion even if unknown or incorrect content is detected.

Handle additional features of KPF format produced by Kindle Create.

Fix KFX-ZIP archives not being created for e-book samples.

Fix debug message logging when creating KFX-ZIP archives.

Internal restructuring and performance improvements.

----

A simplified procedure for importing KFX books into calibre is now available. See the revised instructions in the first post of this thread for details.

[spinky]

Originally Posted by jhowell

No workable import method currently exists for Fire tablets or the Kindle for iOS/Android apps.

First: THANK YOU SO MUCH YOU ARE A LIFESAVER!!! I had a near catastrophe today where my Kindle for PC refused to download a book I bought this morning and desperately want to read. (I hate when people only sell their books on Amazon. )

Second: I was able to successfully import and convert a book from the Android app.

First I had to locate the kfx file on my phone, which took a while. (The files were in /storage/emulated/0/Android/data/com.amazon.kindle/files but I don't know if that's universal I found it doing a search for .kfx.)

Then I zipped the whole directory labelled with the B0* book ID.

I emailed it to myself (because it beat messing around trying to set up mtpfs crap or get adb to work so I could transfer files). I also sent along with the .ticr file and the one that was just a string of numbers and characters from the files directory. (I don't know if those two files are necessary, but I was recreating the directory structure.)

On my desktop, where I have Calibre, I created a "KFX" directory. I dropped the .ticr file and the other in the folder root and then extracted the zipped B0* folder.

I followed your setup directions on the desktop, used the extracted B0* directory as the import source, and bam, it worked like a charm. I'll be reading my shiny no-longer-kfx book on my aging Nook tonight. Thanks!

[jhowell]

Originally Posted by spinky

Second: I was able to successfully import and convert a book from the Android app.

I am glad it worked for you. I will update the instructions to include handling KFX files from the Kindle for Android app.

(That will only work if either the generic encryption key was used for that book or if you have managed to extract the DRM keys associated with your Android app installation.)

[misterr0b0t]

Originally Posted by spinky

First: THANK YOU SO MUCH YOU ARE A LIFESAVER!!! I had a near catastrophe today where my Kindle for PC refused to download a book I bought this morning and desperately want to read. (I hate when people only sell their books on Amazon. )

Second: I was able to successfully import and convert a book from the Android app.

First I had to locate the kfx file on my phone, which took a while. (The files were in /storage/emulated/0/Android/data/com.amazon.kindle/files but I don't know if that's universal I found it doing a search for .kfx.)

Then I zipped the whole directory labelled with the B0* book ID.

I emailed it to myself (because it beat messing around trying to set up mtpfs crap or get adb to work so I could transfer files). I also sent along with the .ticr file and the one that was just a string of numbers and characters from the files directory. (I don't know if those two files are necessary, but I was recreating the directory structure.)

On my desktop, where I have Calibre, I created a "KFX" directory. I dropped the .ticr file and the other in the folder root and then extracted the zipped B0* folder.

I followed your setup directions on the desktop, used the extracted B0* directory as the import source, and bam, it worked like a charm. I'll be reading my shiny no-longer-kfx book on my aging Nook tonight. Thanks!

I tried your method, but it didn't seem to work. I was able to zip the entire B0* directory, created a "KFX" directory on my desktop, then extracted the zipped B0* folder. But when I tried adding in the directory to Calibre, it began adding all the files in the extracted folder (45 total). I don't know if it matters, but I either couldn't find the .ticr file and the one that was a string of numbers and characters, or they weren't there.

Any ideas of what I am doing wrong?

[jhowell]

Originally Posted by misterr0b0t

I tried your method, but it didn't seem to work.

The only files actually needed from the Android app are the .kfx files (book content) and the .ast file (DRM voucher). Extra files are ignored.

The simplest way to do this is to place the contents of the B0xxxxxxxx directory in a ZIP file. Then change the extension of the ZIP file from .zip to .kfx-zip and import that into calibre.

[misterr0b0t]

Originally Posted by jhowell

The only files actually needed from the Android app are the .kfx files (book content) and the .ast file (DRM voucher). Extra files are ignored.

The simplest way to do this is to place the contents of the B0xxxxxxxx directory in a ZIP file. Then change the extension of the ZIP file from .zip to .kfx-zip and import that into calibre.

I changed the extension of the zip file from .zip to .kfx-zip and was able to import it into Calibre. However, after the import, the imported file shows the format as KFX-ZIP instead of KFX. I ran Calibre in debug mode and tried importing the book again. This is what the log file shows:

calibre Debug log
calibre 3.21 embedded-python: True is64bit: False
Windows-10-10.0.16299-SP0 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '10', '10.0.16299')
Python 2.7.12+
Windows: ('10', '10.0.16299', 'SP0', u'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: Gather KFX-ZIP File Type (from KFX Input) (1, 2, 0) && DeDRM (6, 6, 0) && Package KFX File Type (from KFX Input) (1, 2, 0) && KFX metadata reader (from KFX Input) (1, 2, 0) && KFX Input (1, 2, 0)
calibre 3.21 embedded-python: True is64bit: False
Windows-10-10.0.16299-SP0 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '10', '10.0.16299')
Python 2.7.12+
Windows: ('10', '10.0.16299', 'SP0', u'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: Gather KFX-ZIP File Type (from KFX Input) (1, 2, 0) && DeDRM (6, 6, 0) && Package KFX File Type (from KFX Input) (1, 2, 0) && KFX metadata reader (from KFX Input) (1, 2, 0) && KFX Input (1, 2, 0)
Turning on automatic hidpi scaling
devicePixelRatio: 1.0
logicalDpi: 96.0 x 96.0
physicalDpi: 92.5388994307 x 92.6756756757
Using calibre Qt style: True
[0.00] Starting up...
[0.00] Showing splash screen...
[0.05] splash screen shown
[0.05] Initializing db...
[0.08] db initialized
[0.08] Constructing main UI...
[1.81] main UI initialized...
[1.81] Hiding splash screen
[16.45] splash screen hidden
[16.45] Started up in 16.45 seconds with 408 books
DeDRM v6.6.0: Trying to decrypt B00FWOKE28.kfx-zip
Using Library AlfCrypto Python
Decrypting KFX-ZIP ebook: B00FWOKE28
Found 2 keys to try after 0.1 seconds
Decrypting KFX DRM voucher: amzn1.drm-voucher.v1.56acb1d8-56cd-427e-b50d-b55aa4c44ad0.ast
DeDRM v6.6.0: Failed to decrypt with error: Failed to decrypt KFX DRM voucher with any key
DeDRM v6.6.0: Looking for new default Kindle Key after 0.8 seconds
searching for kinfoFiles in C:\Users\Kenny Chan\AppData\Local
No K4PC kindle.info/kinf/kinf2011 files have been found.
DeDRM v6.6.0: Ultimately failed to decrypt after 0.8 seconds. Read the FAQs at Harper's repository: https://github.com/apprenticeharper/...master/FAQs.md
Running file type plugin DeDRM failed with traceback:
Traceback (most recent call last):
File "site-packages\calibre\customize\ui.py", line 171, in _run_filetype_plugins
File "calibre_plugins.dedrm.__init__", line 620, in run
File "calibre_plugins.dedrm.__init__", line 570, in KindleMobiDecrypt
DeDRMError: DeDRM v6.6.0: Ultimately failed to decrypt after 0.8 seconds. Read the FAQs at Harper's repository: https://github.com/apprenticeharper/...master/FAQs.md
Package KFX File Type (from KFX Input) 1.2.0: Packaging C:/Users/Kenny Chan/Desktop/KFX/B00FWOKE28.kfx-zip
Processing container: CR!0D05ZRH8RH7YQ7A0RRMZM129PQ3N.kfx
Processing container: CR!1QE2C8QH596MF1NQJF2GQP89X2W9.kfx
Processing container: CR!2BFA0R2HW53FBE921DYXD6W8084H.kfx
Processing container: CR!3XRB5FZ7XD6BD5MS19H2BNEJGA0E.kfx
Processing container: CR!559X20NXAN7X54MJ72MGCYXANKTE.kfx
Processing container: CR!5GW6A1BPDN0HQ8Q74GKA97GHG9QE.kfx
Processing container: CR!5Q3RHC3VYX3TH34K3DMBV6YN68WR.kfx
Processing container: CR!6PNJJD115H53S98Q4CVCWXBRJ66Q.kfx
Processing container: CR!6TZ8PSWBJS7E59D9TTM22M2QKS5F.kfx
Processing container: CR!7CMH2AS2QN62Q2AZEH4XF99M1RD0.kfx
Processing container: CR!7QA6SWY1CN2MH1PV8RQWBHPXDFY7.kfx
Processing container: CR!7WWAHWXZJ51R74PEBJQPX29N47AQ.kfx
Processing container: CR!9E9FFGNK0S4GDE8Y90ZE8AEG6RNA.kfx
Processing container: CR!A20EW5MKM15HQFTWQBG6FRGZSXJH.kfx
Processing container: CR!APDF6D5F890N3957H8KJ3ETXW88S.kfx
Processing container: CR!B7FY9PM4AH4ZZ6ZR4BW83XP5CQJZ.kfx
Processing container: CR!BJHG9FZKC15KVF8GHQ3X85QKX6Q3.kfx
Processing container: CR!BYWS8QDERS6MZ6R797C8FSXGXASJ.kfx
Processing container: CR!C3W7QY764H4HZDB9YMT5WRQKHN4E.kfx
Processing container: CR!FW229VJ33137N43SBPPXPPDMZ3RZ.kfx
Processing container: CR!GMDF52E0T53PS0K6SVE6FQ9MW1G4.kfx
Processing container: CR!JSB70ZYM595A58GCF8MGH0RP1WCN.kfx
Processing container: CR!K932DBYX857SZ4RG4C37J6SG7CBC.kfx
Processing container: CR!M7DW4EXZF90AZ773B168AR1SE3Y4.kfx
Processing container: CR!MSVSWTKZC53D5F0TSSWASE9BCR1B.kfx
Processing container: CR!N1N207CMEH3WH2VFM387NJDS8W3B.kfx
Processing container: CR!N3H2P8S3T93Z5F9XHRDPM9A55SW4.kfx
Processing container: CR!N7GKM2PPP936BD5XWR0P6B23D608.kfx
Processing container: CR!NRPTGRN4Q15D54VKM1CCX55NPBKS.kfx
Processing container: CR!PBEYN6J2RX0HS13ETW7KV6AZTS0G.kfx
Processing container: CR!R01J0MBEWX7FV7M91K6DSDE2ADWD.kfx
Processing container: CR!R5DAPETPJN0WX8RV21600Z70J38P.kfx
Processing container: CR!SK523QM4B93VZAPMKYBK30GMGBB0.kfx
Processing container: CR!SWDSBJCQ2N1TDE4YDRD994V45R8Z.kfx
Processing container: CR!VAQN6RXCWS0RKEMR4C6375477XJA.kfx
Processing container: CR!VDEWKDMMR96Z58TV1RJX0W6Z8MPS.kfx
Processing container: CR!W2TTQHBY9N31Q5PP99EZMM49KX38.kfx
Running file type plugin Package KFX File Type (from KFX Input) failed with traceback:
Traceback (most recent call last):
File "site-packages\calibre\customize\ui.py", line 171, in _run_filetype_plugins
File "calibre_plugins.kfx_input.package_filetype", line 22, in run
File "calibre_plugins.kfx_input.package_filetype", line 35, in run_kfx
File "calibre_plugins.kfx_input.kfxlib.yj_book", line 52, in convert_to_single_kfx
File "calibre_plugins.kfx_input.kfxlib.yj_book", line 118, in decode_book
File "calibre_plugins.kfx_input.kfxlib.yj_book", line 205, in get_container
KFXDRMError: Book container CR!W2TTQHBY9N31Q5PP99EZMM49KX38.kfx has DRM and cannot be converted
KFX metadata reader activated for C:/Users/Kenny Chan/Desktop/KFX/B00FWOKE28.kfx-zip
Added The Essential Calvin and Hobbes: A Calvin and Hobbes Treasury to db in: 1.4
Added 1 books in 5.5 seconds

[jhowell]

Originally Posted by misterr0b0t

...
DeDRM v6.6.0: Failed to decrypt with error: Failed to decrypt KFX DRM voucher with any key
...
DeDRM v6.6.0: Ultimately failed to decrypt after 0.8 seconds. Read the FAQs at Harper's repository: https://github.com/apprenticeharper/...master/FAQs.md
...

The log shows a failure to remove the DRM from the book. If the publisher has chosen to have DRM applied to that book then you will not be able to convert a copy of it obtained from Kindle for Android.

See the link to the FAQ that was given, especially the section titled "Why don't the tools work with Kindle for Android ebooks?".

As I wrote previously:

Originally Posted by jhowell

(That will only work if either the generic encryption key was used for that book or if you have managed to extract the DRM keys associated with your Android app installation.)