Calibre thinks there are multiple copies of books on my Kindle running KOReader

[lightmaster]

I have a Kindle Paperwhite and I'm running KOReader on it. When I connect KOReader to WiFi and wirelessly connect to Calibre (Calibre detects it as "Wireless Device: KOReader (KindlePaperWhite4)"), Calibre keeps telling me that there are multiple copies of the same book on my device. I do not use the stock Kindle reader at all, and all formats on it are stored as epub. I can reliably reproduce this issue simply by sending a book that is not currently on my kindle to the kindle twice, then disconnecting and reconnecting to Calibre. Calibre will then tell me that the book exists twice, and when I go to "Remove Books > Remove matching books from device", it tells me that both copies are on "Main Memory" and have identical paths (ie, same exact file detected twice). If I send the same book a third time, and disconnect and reconnect, it will say the single file is now 3 different copies of the book.

[kovidgoyal]

presumably the KOReader firmware is reporting the book twice to calibre. I dont know much about the wireless driver or KOReader, but that's pretty much the only way I know of that calibre will think there are multiple copies of a file.

[lightmaster]

I thought I was creating this post in the "Calibre" section, didn't realize I created in the "Calibre > Device" section. Please move this thread to the appropriate section if this isn't the correct one.


Is it Calibre, the wireless driver, or the KOReader app that creates/modifies the `.metadata.calibre` file that's in the library folder on my device? If I open the metadata file and look for the book that calibre falsely thinks is on the device twice, I see completely identical, duplicate entries for the book in question. I guess whatever part of the process that modifies that file is failing to check if the book already exists in the metadata, so when calibre reads the metadata file on the subsequent connections, it sees the metadata there twice?

{
        "series_index": 1,
        "size": 642262,
        "series": "Kiera Hudson 2",
        "tags": [
            "Werewolves",
            "tim orourke",
            "Fantasy",
            "Vampires",
            "werewolf",
            "potter",
            "Romance",
            "Young Adult",
            "diaries",
            "kiera hudson",
            "Horror",
            "vampire",
            "To Read"
        ],
        "uuid": "22fc4ff1-4c97-4e54-b2ab-92ff1557b04a",
        "last_modified": "2022-10-29T06:48:05+00:00",
        "lpath": "Kiera Hudson 2/1 - Dead Flesh - O'Rourke, Tim.epub",
        "authors": [
            "Tim O'Rourke"
        ],
        "title": "Dead Flesh"
    },
    {
        "series_index": 1,
        "size": 642262,
        "series": "Kiera Hudson 2",
        "tags": [
            "Werewolves",
            "tim orourke",
            "Fantasy",
            "Vampires",
            "werewolf",
            "potter",
            "Romance",
            "Young Adult",
            "diaries",
            "kiera hudson",
            "Horror",
            "vampire",
            "To Read"
        ],
        "uuid": "22fc4ff1-4c97-4e54-b2ab-92ff1557b04a",
        "last_modified": "2022-10-29T06:48:05+00:00",
        "lpath": "Kiera Hudson 2/1 - Dead Flesh - O'Rourke, Tim.epub",
        "authors": [
            "Tim O'Rourke"
        ],
        "title": "Dead Flesh"
    }

[chaley]

Originally Posted by lightmaster

Is it Calibre, the wireless driver, or the KOReader app that creates/modifies the `.metadata.calibre` file that's in the library folder on my device?

The wireless driver doesn't create or use that file, which means it must be koreader creating it.

[JSWolf]

Maybe this should be in the KOReader forum.

[lightmaster]

Yea, I'm just making sure it wasn't some component of Calibre first, since Calibre does say that it's updating metadata on the device each time I send a book to KOReader. Gonna make a post on their GitHub about it when I get a chance.

[chaley]

Originally Posted by lightmaster

Yea, I'm just making sure it wasn't some component of Calibre first, since Calibre does say that it's updating metadata on the device each time I send a book to KOReader.

FWIW: that says that the wireless driver is sending "updated" metadata back to the device for books the device claims to have. The exact meaning is complicated by whether or not the device supports metadata caching in the driver, but is up to the device to update its own "database" with the metadata sent by the driver.

The driver has no access to the device's memory. It requests information from the client and uses that to build a list of books on the device. Later it gives metadata back to the client, and the client does whatever it wants with that metadata. When you send a book it is the client that decides where to store it and whether it is an overwrite or a new format.

[lightmaster]

So if I understand the flow correctly, then this means it's entirely on the device to determine whether the newly added/overridden book is a brand new file that isn't currently stored on the device, versus this book has the exact same folder and filename and therefore is just an updated book. Therefore the bug must be in KOReader's implementation of a wireless Calibre client.

[lightmaster]

The KOReader team found a bug in the wireless Calibre plugin in KOReader. Thanks for pointing me over there.