[Metadata Source Plugin] Goodreads

[kiwidude]

This plugin allows Calibre to read book information from goodreads.com when you choose to download/fetch metadata. Adding this plugin can potentially increase both the success rate and quality of information retrieved for some of your books.

Main Features

• Can retrieve goodreads id, title, author, series, isbn, comments, rating, publisher, publication date, tags, language and covers
• Option to retrieve the precise rating and # votes into identifiers than can be bound to custom columns.
• Option to customise the Goodreads genre -> Calibre tag mappings. A default set of the most popular genre tag mappings is included as a starting point.
• Option to additionally search multiple editions of a book for the best set of metadata excluding audio editions (plugin will run slightly slower with this enabled, disabled by default)
• Option to retrieve all contributors to a book as an author. By default this is turned off, however Goodreads is able to provide illustrators, editors etc should you want these retrieved.
• By retrieving the Goodreads id this plugin offers improved integration with the Goodreads Sync plugin. This works both ways - once you have linked to a specific Goodreads edition then retrieving metadata will obtain it for only that edition.
• The goodreads id will also be displayed in the book details panel to be clicked on and taken directly to the website for that book.

Special Notes:

• Help Documentation
• Change History
• Requires Calibre 2.0.0 or later.

Paypal Donations:

• If you find this or any of my other plugins useful please feel free to show your appreciation. I have spent many hundreds of unpaid hours in their development and support so any encouragement for me to continue is appreciated!
  

[Sassafrass]

works great. Thank you!!

[kiwidude]

Thx for the feedback Sassafrass, great to hear.

[sweetheart51224]

Thank you!! This is awesome

[DoctorOhh]

I am loving the tag mapping.

[kiwidude]

Originally Posted by dwanthny

I am loving the tag mapping.

Excellent

I have found one issue which I will publish a new version for (just waiting to see if anything else comes up). The genre tags are sorted by popularity on their web page, but I have some (legacy) code that sorts the tags after doing the mappings. So if you say set the metadata download to get only 3 tags as a limit, you get the first three alphabetically rather than the first 3 by popularity which is obviously wrong.

[DoctorOhh]

Originally Posted by kiwidude

I have found one issue which I will publish a new version for (just waiting to see if anything else comes up). The genre tags are sorted by popularity on their web page, but I have some (legacy) code that sorts the tags after doing the mappings. So if you say set the metadata download to get only 3 tags as a limit, you get the first three alphabetically rather than the first 3 by popularity which is obviously wrong.

Since you already found it I don't have to mention it. Changing from alphabetical to popular will help a lot.

In the interim I have extended my limit to 10 tags and have been reviewing many books manually to figure out what mapping I want to include and dumping the outliers. If I already have some odd tags listed I do a manual look up on goodreads site itself to find out which tags they have and which have the highest popularity and decide which tags I might want to map out.

Thanks for this plugin.

[kiwidude]

Changes in this release:

• Ensure tags mapped are returned by order of Goodreads genre popularity not alphabetically so applying a tag threshold works better
• Add support for API change upcoming in Calibre 0.7.58 allowing hyperlinked ids in book details panel

The second of the changes has been made in such a way as to be backwards compatible so as soon as you install Calibre 0.7.58 or later the new feature it is intended for will be enabled.

[DoctorOhh]

Originally Posted by kiwidude

Changes in this release:

• Ensure tags mapped are returned by order of Goodreads genre popularity not alphabetically so applying a tag threshold works better

This doesn't seem to be hitting the mark and I can't tell what order they are being prioritized. Here is one example and what I get depending how many tags I limit calibre to.

Based on the info below I would expect the top 5 to be Vampires, Paranormal, Romance, Chick Lit, Urban Fantasy. What I get seems to defy reason.

Title - Real Vampires Have Curves
Author - Gerry Bartlett
IDs - isbn:9780425220962, goodreads:2619878

Top 01 - Humor
Top 02 - Humor, Vampires
Top 03 - Humor, Vampires, Paranormal
Top 04 - Humor, Vampires, Paranormal, Romance
Top 05 - Humor, Vampires, Paranormal, Romance, Fantasy
Top 06 - Humor, Vampires, Paranormal, Romance, Fantasy, Adult
Top 07 - Humor, Vampires, Paranormal, Romance, Fantasy, Adult, Chick Lit
Top 08 - Humor, Vampires, Paranormal, Romance, Fantasy, Adult, Chick Lit, Comedy
Top 09 - Humor, Vampires, Paranormal, Romance, Fantasy, Adult, Chick Lit, Comedy, Urban Fantasy
Top 10 - Humor, Vampires, Paranormal, Romance, Fantasy, Adult, Chick Lit, Comedy, Urban Fantasy, Fantasy

Goodreads Genre listing for this book by popularity.

Dark Fantasy > Vampires --- 54 users
Paranormal --- 54 users
Romance > Paranormal Romance --- 33 users
Womens Fiction > Chick Lit --- 26 users
Romance --- 19 users
Fantasy > Urban Fantasy --- 11 users
Fantasy --- 7 users
Adult Fiction --- 5 users
Humor --- 3 users
Humor > Comedy --- 2 users

My Mappings:
Dark Fantasy > Vampires | Vampires
Paranormal | Paranormal
Romance > Paranormal Romance | Paranormal, Romance
Womens Fiction > Chick Lit | Chick Lit
Romance | Romance
Fantasy > Urban Fantasy | Urban Fantasy
Fantasy | Fantasy
Adult Fiction | Adult
Humor | Humor
Humor > Comedy | Comedy

[kiwidude]

@Walt - thx for the info. I think I can see the problem - I just got "lucky" on the titles that I tested on. It is to do with using some (unordered) Python sets instead of lists which stopped a tag from being added twice but of course in hindsight lost the ordering. I will push a new version.

[kiwidude]

Changes in this release:

• Properly fix the ordering of tags.

Sorry folks for the false start on supposedly having this issued "fixed" last time, it had just coincidentally worked on the couple of titles I tried it.

[capnm]

Is this just a simple bug in the goodreads plugin when dealing with accented characters?

Example:

Starting download
Query: title:The Night Manager; authors:John le Carré
Running identify query with parameters:
{u'authors': [u'John le Carr\xe9'], u'identifiers': {}, u'timeout': 30, u'title': u'The Night Manager'}
Using plugins: Goodreads, Amazon.com, Overdrive
The log from individual plugins is below

****************************** Goodreads ******************************
Request extra headers: [('User-agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.472.63 Safari/534.3')]
Found 0 results
Downloading from Goodreads took 0.0629999637604
Plugin Goodreads failed
Traceback (most recent call last):
File "site-packages\calibre\ebooks\metadata\sources\identify. py", line 40, in run
File "calibre_plugins.goodreads.__init__", line 111, in identify
File "calibre_plugins.goodreads.__init__", line 72, in create_query
File "urllib.py", line 1238, in quote
KeyError: u'\xe9'

[kiwidude]

Hi capnm - yes, that does look the case. I'll figure out a fix shortly.

[kiwidude]

Changes in this release:

• Fix to ensure non ascii author names are parsed correctly.

[DoctorOhh]

I used this plugin and went through my 7300 books to update my tags. I started with around 953 tags and now that I finished I am sitting at 208 tags.

I attached my goodreads.json file with the mappings in place. I wish I had a more extensive mapping when I was doing my update to begin with. For the most part I limited tags to 4 per book, sometimes I would trim back to 1 or 2 and for Non-fiction books I would often allow up to 8.

The only thing I don't have mapped is Contemporary, Modern, Adult Fiction and Adult. If a book pops up as erotica then the adult tag is also added to that book.

All of the mappings in the file work. The good news with this json file is you don't have to hunt around to see what is there and how to make it work and you can easily remove tag mappings, but since it is easy to limit tags to a particular number, leaving everything in place works great without gaining too many tags.

Update: If you choose to use this file you must do the following.
1. Exit calibre
2. Unzip this file, you now have a goodreads.json file
3. Find the plugins directory under your calibre configuration directory.
4. Rename your existing goodreads.json file to goodreads.json.org in case you feel the need to go back to it.
5. Place the new goodreads.json file in the plugins directory.
6. Start calibre

Now you can go to Preferences - Metadata download - select goodreads - click Configure selected source to see what is currently mapped for goodreads metadata downloadable tags.

Update 2: I am now running a modified version of the json file created by Huisie. This json file can be found here. It starts with about 800 goodreads tags mapped. The Goodreads json file attached to this post has about 250 Goodreads tags mapped. Either is a good starting point.