PDB to PMLZ File type plugin

[pdurrant]

I'm trying to write a filetype plugin that takes as input one file type, but passes a different file type along to Calibre for further processing.

I see that there is one already that takes a PML file (& associated images) and converts into a PMLZ file, so I think it must be possible, but I don't seem to be able to get mine working.

I have been trying to return the new file in a temporary file, as I've done successfully with another File type plugin.

But no joy. Is there anything else I need to be doing in the plugin to tell Calibre that the file type has changed?

Of course, it might just be that I have something else wrong in the plug-in and it isn't getting called. Is there a way to debug a filetype plugin without having to compile up all of Calibre in a debug version?

Thanks for any help.

[Starson17]

Originally Posted by pdurrant

Is there a way to debug a filetype plugin without having to compile up all of Calibre in a debug version?

Can't you just set up the development environment and debug with print statements?

[kovidgoyal]

Use

calibredb add

and stick print statements in your plugin code, these will then be printed out to the command prompt

[pdurrant]

Originally Posted by kovidgoyal

Use

calibredb add

and stick print statements in your plugin code, these will then be printed out to the command prompt

Ah - just the ticket. Thanks.

[pdurrant]

Originally Posted by pdurrant

Ah - just the ticket. Thanks.

OK, I now have it 'working' - in that a book of type PMLZ now gets imported into Calibre.

But by doing it this route, I seem to have bypassed the PMLZ import filter, and so none of the metadata or images work right. I've checked that my PMLZ file is a good file, by re-importing it. Which gets me cover image* and metadata correctly.

Is there any way I can get Calibre to reprocess the file I return from my Filetype plugin, to pass it on to the PMLZ import filter?

Or am I asking the wrong question, and there's a better way to do what I want?




*Well, if I manually change the <pmlname>_img folder name in the archive to "images". I thought the PMLZ import supported the _img folder name, but apparently not.

[kovidgoyal]

IIRC file import plugins are run after metadata reading plugins, so metadata wont be read from the converted file.

May I ask why you need this plugin at all? Why not just write an input format plugin for the conversion system?

[pdurrant]

Originally Posted by kovidgoyal

IIRC file import plugins are run after metadata reading plugins, so metadata wont be read from the converted file.

May I ask why you need this plugin at all? Why not just write an input format plugin for the conversion system?

Well, if you must know, , I was trying to make something that would strip the DRM from an encrypted eReader file on import, rather like the one for encrypted Mobipocket files.

The only script I have that decrypts eReader files converts them into a PML file and a folder of images, and Calibre seems to like to have those as a zip file -- a PMLZ.

It may be that I've misunderstood the way things work, but I thought that for doing things on import a filetype plugin was the way to go.

And I'm surprised to hear that metadata plugins getr called before filetype plugins, as I had reports of errors with earlier versions of the Mobipocket plugin where a mis-understanding of how to handle temporary files could lead to the Meta data plugin reporting that it couldn't read the metadata.

[KevinH]

> The only script I have that decrypts eReader files converts them into a PML file and a folder of images, and Calibre seems to like to have those as a zip file -- a PMLZ.

FWIW - As you know it is trivial to convert that scripts output to create a pmlz archive instead of a pml file and images. In fact, the code to do just that was contributed to the DR list recently.

So if I understand things, you want to use those additions and have that script work as a plugin to Calibre and somehow use its pmlz output as a temporary format internal to the plugin and have that be recognized and properly handled by Calibre.

Sort of a plugin for one format (pdb) that creates a different intermediate format (pmlz) -- which normally another input plugin would recognize and handle -- but all in one go.

Is that right?

[user_none]

Originally Posted by pdurrant

The only script I have that decrypts eReader files converts them into a PML file and a folder of images, and Calibre seems to like to have those as a zip file -- a PMLZ.

There is a PML2PMLZ import plugin already. When you import a PML file it will take it plus the images and turn it into a PMLZ. Is this not working?

[pdurrant]

Originally Posted by KevinH

>

So if I understand things, you want to use those additions and have that script work as a plugin to Calibre and somehow use its pmlz output as a temporary format internal to the plugin and have that be recognized and properly handled by Calibre.

Sort of a plugin for one format (pdb) that creates a different intermediate format (pmlz) -- which normally another input plugin would recognize and handle -- but all in one go.

Is that right?

I have the conversion to PMLZ working fine. And it gets put in the Calibre database OK. But none of the metadata gets read.

I was hoping that when my filetype plugin returned a PLMZ file instead of a PDB file, that the PMLZ metadata importer would be called.

I think it works for Mobipocket because the Mobipocket importer returns a Mobipocket file (just decrypted). I don't have a way to go from encrypted PDB file to decrypted PDB file, only to a PMLZ.

[pdurrant]

Originally Posted by user_none

There is a PML2PMLZ import plugin already. When you import a PML file it will take it plus the images and turn it into a PMLZ. Is this not working?

That's working fine (I think - I haven't double-checked just now). But that requires the user to do the conversion from PDB to PML manually, before trying to import anything into Calibre. I was hoping to get a 'one step' import process going, as has been done for Mobipocket.

[kovidgoyal]

I just looked at the code and metadata is read before import plugins are called. Why is the metadata not being read from the PDB file? IIRC calibre has a PDB metadata reader.

[user_none]

Originally Posted by kovidgoyal

Why is the metadata not being read from the PDB file? IIRC calibre has a PDB metadata reader.

The metadata for DRMed eReader PDBs cannot be read because the formats DRMs the metadata as well as the content. The metadata reading would have to take place after the import plugin strips the DRM for this to work.

[pdurrant]

Originally Posted by kovidgoyal

I just looked at the code and metadata is read before import plugins are called. Why is the metadata not being read from the PDB file? IIRC calibre has a PDB metadata reader.

I've just been looking at the built-in file type plug-ins, and I can see that for most of them reading the metadata from the original file will work.

But what about your Archive Extract Filetype plug-in? Doesn't it have the same problem that my plug-in has - no meta data extracted from the file?

(goes to check)
Yes, it does - If I put a (non-DRMed) Mobipocket file into a zip archive and import it, it imports as a PRC format file, and can be viewed in the viewer, but no metadata has been read for it.

Should I raise an enhancement request for this? Of can you think of a work-around?

[KevinH]

Originally Posted by user_none

The metadata for DRMed eReader PDBs cannot be read because the formats DRMs the metadata as well as the content. The metadata reading would have to take place after the import plugin strips the DRM for this to work.

If I am reading the latest version of erdr2pml.py correctly, the metadata is not actually drm'd, it just looks like it. The metadata has actually just been XOR'd with a table that is actually present before any decrypting or unshuffling.

So, FWIW it may be possible to read the metadata without actually decrypting an an ereader pdb.