Metadata retrieval using series/series index

[geekraver]

Hi all

I am starting to create a new Calibre library for magazines that I have like Hacker Monthly.

My intention with these was to set the title to "Hacker Monthly" for each one, and identify the individual issues using the series index. I wanted to write a metadata download plugin to pull the table of contents off the website and use it to set the comments field (and maybe also tags, which I would generate from the contents with something like https://github.com/apresta/tagger).

The problem I have is it looks like metadata downloader plugins get passed title and author to identify() but not series information. Is my understanding here correct? And what would be the best workaround then? I can't use title and author necessarily to query Calibre itself to get the current metadata if these are the same for all issues. Would it be best then to keep issue numbers in the titles?

Thanks!
Gram

[geekraver]

It's a bit of an ugly hack but another way I thought of doing this was to just have a Python script that modifies the OPF files, and then use a consistency check to get the DB to update.

OTOH it looks like the title and author are necessarily unique per "book", or the issues will get folded together as dups. If that is the case, then I guess I have to leave the issue number in the title and this problem goes away.

[kovidgoyal]

The metadata download plugins also get passed the identifiers, so create an identifier to store the issue number, something like this:

issue:101

or just a url directly,

url:http://whatever

[geekraver]

Thanks Kovid. I'll look into that approach later. Right now I'm using the title, but I'm not getting very far at all. It doesn't seem my plugin is being called at all. It is loaded successfully, and shows up as a metadata source when downloading metadata, but I see no output from the print statements. My __init__.py is below; can you see what I might be doing wrong?

from calibre.ebooks.metadata.sources.base import Source

class Magazines(Source):

    name = 'Magazines'
    description = _('Download metadata for various magazines')
    version = (0, 1, 0)
    author = 'Graham Wheeler'
    minimum_calibre_version = (0, 9, 0)

    capabilities = frozenset(['identify'])
    touched_fields = frozenset ([
        'comments'
        ])
    has_html_comments = True


    def identify(self, log, result_queue, abort, title=None, authors=None,
                 identifiers={}, timeout=30):

        print 'In identify: %s ' % title
        from calibre_plugins.magazines.worker import Worker

        # Search for results in different thread, as searching takes time and blocks...
        worker = Worker(result_queue, log, title, authors, self, cfg.getOption(cfg.KEY_MAX_DOWNLOADS))
        worker.start()

        while not abort.is_set():
            worker.join(0.2)

            if not worker.is_alive():
                break

        print 'Out of worker: %s' % result_queue.qsize()

        return None

[kovidgoyal]

You did enable it in Preferences->Metadata download didn't you?

[geekraver]

Yes, it is enabled. I tried disabling all other sources too. All I see in the output at the end of the debug run is:

Job: 1 Download metadata for 1 books finished
Starting job: Download metadata for 1 books
Download complete, with 1 failures

The order of those messages is a bit weird.

[kovidgoyal]

You want to look at the metadata download log not the debug log. CLick the view log button in the metadtata download dialog.

[geekraver]

Do you mean the 'View Details'? That shows just:

Hacker Monthly #15 (Failed metadata)

[geekraver]

BTW I put this at the start of identify:

print 'In identify: %s ' % title
log.info('In identify: %s' % title)

I don't see it either way. I wasn't sure what to use because in the docs on debugging plugins you say to use print, but in the docs for identify you say the log object is for debugging messages.

[kovidgoyal]

No, I mean the *View log* button on the *metadata download dialog* the one you get when you click download metadata in the edit metadata dialog.

[geekraver]

Ah, I see. I was initiating the metadata download by right clicking and using the context menu. There is no View Log in this case. You have to open the Edit Metadata dialog and do a Download Metadata from within that to get the logs.

[geekraver]

Got it working now. Thanks!

So which is better for diagnostics - using log or print?

[kovidgoyal]

Metadata download is run in a separate process, therefore print will not work for it, you need to use log. And there most definitely is a view log button for bulk metadata downloads. It is on the confirmation dialog that pops up after the download completes.

[geekraver]

I wrote a long response to your earlier post when your first mentioned this, detailing everything I saw, before I tried the metadata download from the edit dialog ; after that I discarded the details. But there was no view log. Maybe if some succeed there would be but if they all fail all you get is an error popup with three buttons: copy to clipboard, view details and OK. And the view details here doesn't show the log, just the one line failure error message.