Dealing with Kindle ebooks in calibre

[tamhas]

Having been using the download and transfer tool for some time, sticking with Calibre 4.3 and the old DeDRM tools, I am now having to deal with moving to modern Calibre and noDRM. I thought it might be a good idea to go over the steps so that I am sure I know what to do.
1. Backup current Calibre
2. Backup Calibre data
3. Remove DeDRM plugin from Calibre
4. Update Calibre from within Calibre
5. Add KFX Input plugin from Get New Plugins
6. Download DeDRM tools from https://github.com/noDRM/DeDRM_tools...es/tag/v10.0.9
7. Load DeDRM tools with Plugin From File.
8. All set?
Load Calibre from Paperwhite going forward and convert to ePub for wife's use.
Paperwhite is7th generation, firmware 5.16.2.1.1
I am assuming DeDRM Tools, not alpha since I have no need to be on the edge.
What am I not thinking of?

[Lynx-lynx]

Why are you removing the dedrm plugin? I don't recall doing that, though it was some time ago now that I upgraded from that version of calibre

[texasnightowl]

So...K4PC method...my first time messing around with it. I have all my books from Amazon in Calibre already from the download and transfer method, but with it going away, I wanted to make sure I could do the K4PC route...as long as it lasts.

I was 99% successful! But I have a couple questions...

For reference, K4PC 2.4.0 (70904) and updates disabled. Calibre 7.23 (I know, 7.26 is out). nodrm 10.0.0.9 alpha. KFX Input 2.19 (I know, 2.20 just came out, I should update it.)

I have reconciled my content library on amazon against my k4pc content and I have 1463 books. after downloading everything in k4pc I have 1463 folders. so k4pc did apparently download everything.

I created a new library in Calibre. Then under Add books I chose add from selected folder and subfolders. For the "Multi-file books?" popup that said "Assume all e-book files in a single folder are multiple formats of the same book?" I honestly can't remember what I selected. Based on behavior, I think I said no. Then I selected "My Kindle Content" folder.

It started processing and said 2702 books! I let it go through and had a bunch of Unknown author entries and CR!* titles...so I think I should have answered Yes to the multi-file books popup.

Anyway, when it was done I deleted all the "unknown author" CR!* title books to get them out of the way.

I ended up with 1456 books. So, that leaves 7 missing books (1463).

Not bad.

Of those 1456, all but 2 were in correct formats such as azw3, htmlz, kfx, and mobi. 2 were in kfx-zip. My understanding was that this means dedrm didn't work. But...that isn't exactly the case here. I was able to convert one of them to epub no problem. So even though it was kfx-zip it worked. The other gives an error that it cannot convert and 'Book is incomplete. All of the KFX container files that make up the book must be combined into a KFX-ZIP file for successful conversion. (Missing containers *id1*, *id2*)'

OK, wow. This is getting long.

My questions...what can I do to improve my % from 99% to 100%?

1. Is there a better way to add the books? Since I don't want to go through 1463 individual folders to drag the azw files over?

2. How can I create a log to figure out what the missing 7 books are?

3. Any ideas on the one that says the book is incomplete?

[texasnightowl]

Update...doing my own testing.

1. I answered yes to the multi-file popup this time and I also added a rule to ignore any file that doesn't end in .azw. Calibre is currently importing 1464 books.

...a few minutes later...

I have ended with 1464 books. The breakdown is 1 azw, 16 azw3, 91 htmlz, 1233 kfx, 7 kfx-zip, and 116 mobi.

the 1 azw file is apparently an invalid topaz file. when I tried to open it, this is part of the log entry: calibre, version 7.23.0
ERROR: Loading book failed: Failed to open the book at D:\Calibre Libraries\K4PCTesting\Unknown\B004HKIGE6 EBOK (3525)\B004HKIGE6 EBOK - Unknown.azw. Click "Show details" for more info.

Failed to convert book: D:\Calibre Libraries\K4PCTesting\Unknown\B004HKIGE6 EBOK (3525)\B004HKIGE6 EBOK - Unknown.azw with error:

ValueError: 'D:\Calibre Libraries\K4PCTesting\Unknown\B004HKIGE6 EBOK (3525)\B004HKIGE6 EBOK - Unknown.azw': Invalid Topaz format - no metadata record
[a bunch of stuff omitted here] calibre.ebooks.mobi.reader.mobi6.TopazError: This is an Amazon Topaz book. It cannot be processed.

OK...one topaz book. oh well.


Of the 7 kfx-zip files, 6 of them continued to convert to epub just fine. So even though they say kfx-zip they worked.

The last kfx-zip file is the same one that failed the first time around with the book is incomplete message and missing containers. For what it is worth, the book does open in K4PC. So I'm confused on that one.

Anyway...that means of my 1464 books, 1462 appear to have worked just fine. Only 2 failures.

edit: 99.86% success is pretty good...