What does Calibre do when files are imported?

[stefan230]

Good morning,

I have a general question on how calibre works. I am trying to get to the root of a bug.

The general story is the following:
I have an E-book from Thalia (a german retailer for books and E-Books) and when I import the book into calibre it does something to the book. And I am trying to understand why and what it actually does to the book.

When imported my manga gets slighty changed by calibre which in turn leads to displaying problems on my Pocketbook E-Reader (Pocketbook Inkpad Color 3 (PB743K3)) in its PB-Reader software.

The working theory is that the E-Book I got from Thalia is either real EPUB2 or EPUB3. It is listed as an EPUB3 in its file though. I think that while importing calibre sees this and then corrects the file to be "proper" EPUB3. Which in turn makes the PB-Reader break. The Epub3 capabilities of the software are generally bad.

So generally I don't see this as being a bug with calibre but with pocketbooks PB-Reader and its lackluster Epub3 implementation.

I would like to report this bug to pocketbook later. In order to this I would like to understand *what* calibre exactly does to the book and *why* it does that. Is my working theory correct here at all? Is there some sort of Epub-sanitation? Is this something I could turn off?

I know this is not a problem with calibre, but rather the PB-Reader, since the same file works well enough with KOReader (But I still have other problems with that one, which are not related to this query).

I attached two content.opf files to this post. The original one from the E-Book from Thalia and a second one of calibre after importing the E-Book into it. I will also attach a diff screenshot from vscode aswell.

It would be nice if some could chime in and explain what happens here. I could boil my problem down to the first two lines of the content.opf which gets changed. So are these the lines which differentiate the unproper EPUB3 file from the proper one in calibre?

[kovidgoyal]

importing does not touch files they remain byte for byte identical. Unless you have some third party plugins installed in calibre.

[chaley]

Although files aren't changed when importing, sending them to the device does update the metadata (opf). You can turn that off by disabling the "metadata writer" plugin for the format type, probably EPUB, in Preferences / Plugins. Alternatively, use "Save to disk" and copy the book manually to the device, or copy it directly from the folder in the calibre library (use the shortcut O to open the folder for the book).

[stefan230]

Thank you two for your input!

Originally Posted by kovidgoyal

importing does not touch files they remain byte for byte identical. Unless you have some third party plugins installed in calibre.

I was fairly sure it did something, but of course you are the dev you would know :) I looked at it again and the files are indeed identical. It was actually the transfer to the device that changed them. (Interestingly enough not only for the device but also in calibre library itself.

Originally Posted by chaley

Although files aren't changed when importing, sending them to the device does update the metadata (opf). You can turn that off by disabling the "metadata writer" plugin for the format type, probably EPUB, in Preferences / Plugins. Alternatively, use "Save to disk" and copy the book manually to the device, or copy it directly from the folder in the calibre library (use the shortcut O to open the folder for the book).

This was the hint I needed. Deactivating all those plugins comes with the expected result. The Epubs are now byte to byte copies. So thats actually very nice. Do I lose any vital functionality by disabling those plugins though? (I guess the metadata would not be in the opf-file and it would stay the way it is even if I change metadata in calibre? Like adding the author and a tag and stuff)

[JSWolf]

Originally Posted by chaley

Although files aren't changed when importing, sending them to the device does update the metadata (opf). You can turn that off by disabling the "metadata writer" plugin for the format type, probably EPUB, in Preferences / Plugins. Alternatively, use "Save to disk" and copy the book manually to the device, or copy it directly from the folder in the calibre library (use the shortcut O to open the folder for the book).

Is there going to be any problem disabling the metadata writer?

[chaley]

Originally Posted by stefan230

This was the hint I needed. Deactivating all those plugins comes with the expected result. The Epubs are now byte to byte copies. So thats actually very nice. Do I lose any vital functionality by disabling those plugins though? (I guess the metadata would not be in the opf-file and it would stay the way it is even if I change metadata in calibre? Like adding the author and a tag and stuff)

Originally Posted by JSWolf

Is there going to be any problem disabling the metadata writer?

If you disable the metadata writer then calibre cannot update metadata in the book, e.g., changes made in calibre. I know it will affect send to the device, save to disk, downloading via the content server, and polish books. It could affect the editor embedding metadata. It could affect plugins like quality check. And possibly, indeed probably, more.

Personally, I wouldn't do it because it is using a huge hammer to fix a small problem. Instead I would directly copy the file to the device, perhaps using Save to disk with the device's books folder as the target. Save to disk supports turning off updating metadata during the save.

[stefan230]

Originally Posted by JSWolf

Is there going to be any problem disabling the metadata writer?

I did just check for myself real quick. My reader has its own little Library program. Where you can sort and view your collections in different ways. It uses though the metadata from that opf-file to get the metadata. So if you never write that metadata to the eBook stuff like the series of the book a description you added and other stuff will not be displayed on the E-Reader and cannot be used to sort stuff with the filters of said library program.

This might a bigger or lesser problem to you.

Obviously enabling the metadata writer writes the metadata to the book and you can use it to sort the library

I did also generally found koreader to work pretty well with those E-Books as well. So that could a great alternative and circumvents the problem. Since it is indeed a problem PB-Reader being just not as good. If you have a B/W-Reader I would go for that