[Metadata Source Plugin] Goodreads

[anthony.burton4]

I'm trying to figure this out - raised a thread elsewhere, but this is probably a better place.

If I use Goodreads Sync, I can link a book to Goodreads - the IDs field is populated with the Goodreads ID.
If I then use Goodreads metadata update (disabling all the other metadata update plugins) it fails to find the book. There are no errors in the calibre log.

An example that fails for me - I think they all do
Title:"You & Me & the Devil Makes Three- Esquire's Fiction for Men"
Author:"Aaron Gwyn; Jess Walter; Luis Alberto Urrea"
IDs:"goodreads:14739608"

[DoctorOhh]

Originally Posted by anthony.burton4

I'm trying to figure this out - raised a thread elsewhere, but this is probably a better place.

If I use Goodreads Sync, I can link a book to Goodreads - the IDs field is populated with the Goodreads ID.
If I then use Goodreads metadata update (disabling all the other metadata update plugins) it fails to find the book. There are no errors in the calibre log.

An example that fails for me - I think they all do
Title:"You & Me & the Devil Makes Three"
Author:"Aaron Gwyn; Jess Walter; Luis Alberto Urrea"
IDs:"goodreads:14739608"

It would appear that the Goodreads metadata plugin doesn't use the Goodreads ID to look up metadata. But if I enter the title (without the extra - Esquire's Fiction for Men) and authors correctly (see spoiler) it fills in fine using only the Goodreads metadata plugin. Why limit the sources? I configure each source I use so I only get tags and series from Goodreads, but let the rest fill in. There are times were I have to get metadata that doesn't use Goodreads, save it, then get metadata again to have the Goodreads tags and series to show up.

Spoiler:

Running identify query with parameters:
{u'title': u'You & Me & the Devil Makes Three', u'identifiers': {u'goodreads': u'14739608'}, u'authors': [u'Aaron Gwyn; Jess Walter; Luis Alberto Urrea'], u'timeout': 30}
Using plugins: Goodreads (1, 1, 12)
The log from individual plugins is below

****************************** Goodreads (1, 1, 12) ******************************
Found 1 results
Downloading from Goodreads took 1.37300014496


---
Title : You and Me and the Devil Makes Three
Author(s) : Luis Alberto Urrea & Jess Walter & Aaron Gwyn
Publisher : Esquire
Series : Esquire's Fiction for Men #1
Languages : eng
Rating : 1.7
Published : 2012-06-05T00:00:00+00:00
Identifiers : goodreads:14739608
Comments : <p>Three new stories for men, by three of the most ambitious and talented names in fiction </p>
<p>A search for an eight-ball of cocaine, turns into a pistol-whipping and executions in Aaron Gwyn’s stunning and disquieting “You and Me and the Devil Makes Three.” Gwyn, who was born in Tulsa and raised on a cattle ranch, is the author of the novel The World Beneath and the story collection Dog on a Cross, about which the Boston Globe wrote, “In Gwyn’s expert hands, nothing, including good or evil, is ever so simple . . . part Flannery O’Connor, part Shirley Jackson, and wholly original—so brilliantly compelling.” This new story is no less shocking and revelatory. </p>
<p>In “Young Man Blues,” Luis Alberto Urrea introduces us to Joey, the son of a motorcycle gangster tasked with caring for his mother while his dad serves time in Pelican Bay. Packed with rottweilers and German Lugers, roaring car engines and doughnut girls, this story is as riveting and funny as it is affecting. Urrea, a 2005 Pulitzer Prize finalist for nonfiction and a member of the Latino Literature Hall of Fame, is the author of The Hummingbird’s Daughter, which was twenty years in the making and named a book of the year by numerous publications. His most recent novel, Queen of America, is a New York Times “editor’s choice” selection for 2011. </p>
<p>Jess Walter’s “Big Man” depicts middle-aged men trying to relive their glory days in their local recreational basketball league. Beer bellies, ex-wives, and middle age come together in a hilarious and absurd riff on mortality that is vintage Walter. Walter’s novels include The Zero, a finalist for the National Book Award, and The Financial Lives of the Poets, which Time called “the funniest way-we-live-now book of the year.” He most recently published Beautiful Ruins, which Richard Russo describes as “an absolute masterpiece.” </p>
<p>Esquire's Fiction for Men is a new ebook series whose mission is to publish the type of original short stories men love to read—plot-driven, immediate, essential, and impossible to put down.</p>
Goodreads book url: u'http://www.goodreads.com/book/show/14739608-aaaa'

************************************************** ******************************
The identify phase took 1.41 seconds
The longest time (1.373000) was taken by: Goodreads
Merging results from different sources and finding earliest publication dates from the worldcat.org service
We have 1 merged results, merging took: 0.00 seconds

[BetterRed]

Moderator Notice
@anthony.burton4

I merged your other thread in Library Management into here.

If you think a thread (particularly if it's yours) should be moved or merged please click the '!' and make a request. The '!' is to the right of the 'karma' button at the bottom of the sidebar. where the poster's nick etc is displayed.

See # 3 in ==>> Guidelines

BR

[anthony.burton4]

"Why limit the sources?"

Well, the metadata source plugin is rubbish at finding the right book unless the ISBN is populated - and even then it often goes and finds a different book. With this example, you managed to "find" the right book by massaging the search data - sometimes you have to try multiple versions before finding a book, and sometimes it comes back with the wrong book and it isn't immediately apparent. When an ID exists, the plugin SHOULD use it - otherwise it's just screwing up your database.

Note, the Goodreads sync plugin is fantastic at finding the right book - you can quickly see all the relevant options for each book and pick the right one.

Looks like I'll have to investigate modifying the Goodreads metadata plugin Dang it, I hate Python!

[anthony.burton4]

In the Goodreads metadata plugin it states the following

"# Unlike the other metadata sources, if we have a goodreads id then we
# do not need to fire a "search" at Goodreads.com. Instead we will be
# able to go straight to the URL for that book."

If the plugin is able to extract a goodreads id it goes straight to:
http://www.goodreads.com/book/show/%ID%
This URL works fine.

The plugin itself appears to be unused though - if I add logging statements to it, they do not show up in the logfile - not sure what's going on here. The plugin is definitely loaded by calibre as I made a couple of typos with my edits, and they DID show in the log as errors.

[DoctorOhh]

Originally Posted by anthony.burton4

In the Goodreads metadata plugin it states the following

"# Unlike the other metadata sources, if we have a goodreads id then we
# do not need to fire a "search" at Goodreads.com. Instead we will be
# able to go straight to the URL for that book."

So it appears that this is currently broken - perhaps Goodreads have changed their website? investigating further...

Looking forward to your results

[davidfor]

Originally Posted by anthony.burton4

"Why limit the sources?"

Well, the metadata source plugin is rubbish at finding the right book unless the ISBN is populated - and even then it often goes and finds a different book. With this example, you managed to "find" the right book by massaging the search data - sometimes you have to try multiple versions before finding a book, and sometimes it comes back with the wrong book and it isn't immediately apparent. When an ID exists, the plugin SHOULD use it - otherwise it's just screwing up your database.

Note, the Goodreads sync plugin is fantastic at finding the right book - you can quickly see all the relevant options for each book and pick the right one.

Looks like I'll have to investigate modifying the Goodreads metadata plugin Dang it, I hate Python!

The Goodreads metadata plugin does use the Goodreads identifier if it is there. It only reverts to a search, or using the ISBN if the Goodreads identifier doesn't return a book.

For the example you gave, I created an empty book and searched for it. This found the book on Goodreads and was able to save the metadata including the identifier. I saved this, then reopened the metadata, changed the title and author to "fred" and saved it again. Then I opened the metadata editor again and clicked the download metadata button again. After a while, the books were displayed. The only book from Goodreads was the one I found above with the identifier.

There must be something else going on. As I stated before, I need to see the log from the metadata download to know. That is obtained by click "View log" button at the bottom of the dialog and copying displayed log.

Edit: You posted while I was testing and writing. But the comments all remain the same.

[anthony.burton4]

My log is empty - nothing in there of any use at all.
So either the logging is broken AND the plugin is broken, or more likely, the Goodreads plugin is not being used for some reason.

calibre Debug log
calibre 2.85.1 [64bit] embedded-python: True is64bit: True
Windows-7-6.1.7601-SP1 Windows ('64bit', 'WindowsPE')
('Windows', '7', '6.1.7601')
Python 2.7.9
Windows: ('7', '6.1.7601', 'SP1', 'Multiprocessor Free')
Successfully initialized third party plugins: Kindle Collections (1, 7, 17) && Modify ePub (1, 3, 13) && Quality Check (1, 9, 11) && Count Pages (1, 7, 0) && Goodreads Sync (1, 10, 1) && Favourites Menu (1, 0, 4) && KindleUnpack - The Plugin (0, 81, 4) && Goodreads (1, 1, 12) && Open With (1, 5, 10) && FictionDB (1, 0, 10) && Extract ISBN (1, 4, 4) && Author Book Count (2, 1, 5)
devicePixelRatio: 1.0
logicalDpi: 96.0 x 96.0
physicalDpi: 72.035450517 x 72.0567375887
Starting up...
DEBUG: 0.0 HttpHelper::__init__: proxy=None
Started up in 4.69 seconds with 1728 books
Metadata sources cache was recently updated not updating again
Job: 1 Download metadata for one book finished
Starting job: Download metadata for one book
Download complete, with 1 failures

[anthony.burton4]

An example of some of the logging I added. I added as "Error" in case there was filtering going on, and info messages weren't being included.

def get_book_url(self, identifiers):
log.error('GetBookURL')
goodreads_id = identifiers.get('goodreads', None)
if goodreads_id:
return ('goodreads', goodreads_id,
'%s/book/show/%s' % (Goodreads.BASE_URL, goodreads_id))

def create_query(self, log, title=None, authors=None, identifiers={}):
log.error('CreateQuery')
isbn = check_isbn(identifiers.get('isbn', None))
q = ''
if isbn is not None:

[anthony.burton4]

OK - exactly the same problem if I select Amazon/Google in the metadata download configuration screen.

The problem is clearly not in the plugin.

[anthony.burton4]

Using Calibre Portable, and the latest Goodreads plugin, it works as expected.
This gives me a workaround, but it would be good to figure out what's broken with my main calibre program.

Calibre portable at 0.9.44
Main Calibre is latest edition: 2.85.1

[davidfor]

Originally Posted by anthony.burton4

My log is empty - nothing in there of any use at all.
So either the logging is broken AND the plugin is broken, or more likely, the Goodreads plugin is not being used for some reason.

Just to check. Are you saying the that after the search has finished, when click the "View log" button? If you do this before the search finishes, nothing useful will be displayed.

The log you posted is the calibre log from running calibre in debug mode. The log entries from the metadata download not appear there. They are only in the above log window.

[davidfor]

Originally Posted by anthony.burton4

Using Calibre Portable, and the latest Goodreads plugin, it works as expected.
This gives me a workaround, but it would be good to figure out what's broken with my main calibre program.

In that case, there is a problem with your calibre installation. I would start by uninstalling it and reinstalling. This will not affect your library or configuration.

[anthony.burton4]

Do you mean the "Show Details" button when the DL has finished?
That just states that the metadata download failed.

I've already uninstalled/reinstalled calibre - no joy.

I tried to double down on that - I completely removed calibre - deleting both the program files folder (it was empty anyway) and the UserData folder.
Then reinstalled the latest 64 bit version.

I ONLY installed the Goodreads plugin, and then attempted to get the metadata as before. Same issue.

[anthony.burton4]

OK - I'd turned the firewall off, but for some reason this wasn't enough. Uninstalling the firewall fixed the problem.