[GUI Plugin] Goodreads Sync

[Rellwood]

Originally Posted by StegBook

I've been getting this error message for the last few days. This error comes up on the first attempt to update a book after launching Calibre. Any ideas?


calibre, version 7.8.0
ERROR: Goodreads Failure: The request contacting Goodreads has failed.
If it reoccurs you may have exceeded a request limit imposed by Goodreads.
In which case wait an additional 5-10 minutes before retrying.

URL: https://www.goodreads.com/shelf/add_to_shelf.xml
Response Code: 500
b'<?xml version="1.0" encoding="UTF-8"?>\n<hash>\n <status>500</status>\n <error>Internal Server Error</error>\n</hash>\n'

FYI...Goodreads has been going through and removing books from their site. These books are either changed by Amazon (new cover/title) or some other reason, and you wont be able to upload them again. If they are currently on your shelves, don't remove them and expect to reupload them. This may be why you are getting the error. I got it too.

Obsolete books from Amazon without ISBN's, Single books that ended up becoming an omnibus, anthology parts, author changes, etc. If the book doesn't have an ISBN, ASIN, or ISBN13 it will be removed from Goodreads. So, if it was an ebook from Amazon, and Amazon deleted it from their system for some reason or another, then it won't be in Goodreads outside of those currently shelved. If you want to know which books can be re-uploaded, just export your current library .csv and then create a new goodreads account and try to reimport it (breaking it in chunks first). The books that don't end up on your shelves are the ones that are causing the errors.

[kiwidude]

Originally Posted by bzxc

I'm using Linux and I've replaced the content of httplib2 folder inside the plugin zip with the latest version from archlinux repos (python-httplib2 0.22.0-5) and it works great.
Python version 3.12.3
Calibre version 7.12
Plugin version 1.16.6

Unfortunately that combination doesn't work on Windows - there are too many errors/issues due to how calibre bundles it's own python that you aren't seeing on linux I assume because of python being external. I'm in the process of reverting that attempt I made to apply that.

I have wasted far too many hours on this already that has me properly grumpy about this whole thing but there is one last attempt I am willing to try with someone trying to help at the moment - if that works out then great, otherwise linux support is getting dropped.

[chaley]

@kiwidude: If I were you I wouldn't do anything with this. Reason: changing calibre's python version possibly, perhaps even probably, breaks stuff in base calibre. TMK there hasn't been any beta testing to check if calibre still reliably works with the new python version.

My view: wait for Kovid to upgrade calibre's python then do whatever work is required.

[kiwidude]

@chaley - that was my original intent of ignoring the linux users who have a disproportionate amount of support required.

Then someone submitted a PR for this so I thought push it out give it a go. Except it turned out they never tested their fix. Then they submitted another PR - I pushed that out, didnt fix the issue either, once again they hadn't tested it.

So then I tried myself just abandoning those changes and taking on the latest python3 version of httplib2. Except internally they made so many refactorings to that with dependencies on additional lib modules (pyparsing) and imports that it won't work as a calibre plugin without further hacks and tweaks.

A second person just now has submitted another PR - but their PR makes no sense at all and when questioned they said their change they tested was with a different version of the codebase of the plugin. I've given them the opportunity to send me the files they have actually tested to see if something can be resurrected from that as the last roll of the dice.

When Kovid moves to 3.12 then this is all going to have to get sorted out as presumably this key_file stuff will impact windows users. So I was willing to do a little effort if it solved the future problem. But people submitting untested PR's has really pissed me off. This is one situation where "hey at least they had a go" is not true and commendable - I've wasted at least 8-10 hours on this nonsense as a result...

[kiwidude]

Release Notes:
https://github.com/kiwidude68/calibr...s_sync-v1.16.8

[Ambassador Dave]

Originally Posted by kiwidude

Ive been told by the person who submitted this that it works on linux. I know that it hasn't broken Windows - so if it still doesn't work for other linux users then tough luck...

That appears to have done the trick, everything went smoothly through a few Goodreads actions.

Thank you for sticking with it, even when us Linux users can be a pain!



Python version 3.12.3
Calibre version 7.12
Plugin version 1.16.8

[kiwidude]

Thanks Dave for the quick feedback and donation (much appreciated) and also to Bernard for the final fix.

[PeterT]

As others have pointed out the true issue here is NOT Linux support; rather it's the insistence by repository maintainers to modify calibre and change its dependencies to using the system supplied python and other components, as opposed to keeping to calibre as designed / tested / supported by Kovid.

Sent from my Pixel 7a using Tapatalk

[Ambassador Dave]

kiwidude, you're quite welcome on both counts. I said it in my first post, and I'll say it again, thank you for this plugin.

[Ambassador Dave]

PeterT, I don't disagree with you, however, I do think it's a bit more nuanced than what you wrote.

Just as an end user, I benefit from the repository maintainers using the source code that Kovid supplies. And yep, they move past his bold, red warning to create the distro specific install of calibre I enjoy. As one benefit, I've got about a dozen programs (calibre, Steam, etc.) that require Python. Python needs about 70MB when installed (not counting its dependencies) which would end up nearing a gig of space if every program included its own copy. That's 'bloat' I'd prefer not to have on my system.

Second, when I first migrated from Windows to Linux, I followed Kovid's recommended Linux install. Every time there was an update, I had to enter the manual command-line command again. Not a huge deal, yet across each program within a system, well, it adds up in time and effort. With the Arch Linux repository almost all of my programs are updated at once.

Sometimes there are troubles, like this one. Kovid may never update Python in his binary packages again. Yet I noticed that kiwidude said version 3.10 is the current Windows packaged version while Kovid lists 3.11.5 in his "up-to-date list of dependencies that the official calibre binaries are bundled with". 3.12 may or may not come soon, however calibre does appear to work fine with it.

I see you rock a Pixel 7a. Have you been using Android long enough to remember when we had to update every app individually and manually? I'd venture an opinion that when the Play Store switched to automatically updating apps everything got easier for users, yet tougher for developers. I feel that the repository maintainers 'modifying' calibre through doing a source code vs. binary installation which builds upon the "system supplied python and other components" is analogous. It makes it easier for users like me, and tougher for developers like kiwidude when updates to those components 'break' the earlier code, just as each new version of Android has done to some of my dear old apps.

Whichever way you think and feel, I am thankful for this community. My copy of calibre & kiwidude's Goodreads plugin work again thanks to efforts directly from this group.

[paulneocube]

Originally Posted by kiwidude

This plugin allows you to synchronise your book collection in Calibre with your shelves on a Goodreads.com account.

I am not the coding type of person but I am missing something (most likely something so obvious) because I am not sure how to get this working. I set names for all my shelves as tags in calibre but when I go to sync I get the error, "Unable to sync, You must specify sync actions or columns for at least one shelf first."

I am not able to understand what I am doing incorrectly or what I may be missing TBH. Please help this n00b. Thanks.

[chaley]

Originally Posted by PeterT

As others have pointed out the true issue here is NOT Linux support; rather it's the insistence by repository maintainers to modify calibre and change its dependencies to using the system supplied python and other components, as opposed to keeping to calibre as designed / tested / supported by Kovid.

Originally Posted by Ambassador Dave

PeterT, I don't disagree with you, however, I do think it's a bit more nuanced than what you wrote.

[...]

Sometimes there are troubles, like this one. Kovid may never update Python in his binary packages again. Yet I noticed that kiwidude said version 3.10 is the current Windows packaged version while Kovid lists 3.11.5 in his "up-to-date list of dependencies that the official calibre binaries are bundled with". 3.12 may or may not come soon, however calibre does appear to work fine with it.

FWIW: Kovid said in this bug report about a problem in Python 3.11 involving SSL certificate storage (emphasis mine):

... or wait till the version of python used in calibre is updated to 3.12 which is the earliest version of python with the fix, but python 3.12 has many other incompatibilities so it wont be anytime soon.

For example, there are at least two usages of "key_file" in calibre 7.12, one in the https section of the content server and one in the code used for metadata downloading. I don't know what the other incompatibilities are.

[BananaTricky]

Originally Posted by paulneocube

I am not the coding type of person but I am missing something (most likely something so obvious) because I am not sure how to get this working. I set names for all my shelves as tags in calibre but when I go to sync I get the error, "Unable to sync, You must specify sync actions or columns for at least one shelf first."

I am not able to understand what I am doing incorrectly or what I may be missing TBH. Please help this n00b. Thanks.

Hi Paul, you need to set some actions to sync. Eg, I sync the date read, review text and the star rating (all of which are tick boxes), I also remove any tags like 'To Be Read' and add tags like '2024 Read'. Go to 'customise the Goodreads plug-in' and edit 'Sync' Actions and edit 'Shelf Add' Actions to obtain the desired effects for each shelf in Goodreads (ie when I sync the 2023 Read shelf in Goodreads it adds '2023 Read' in Calibre etc. Apologies, I am replying on a different computer to the one on which I use Calibre so I can't add screenshots. A

[laxarus]

Hey, just a question, is it possible to use a custom column for the "goodreads edition" corresponding to the synced book? This metadata does not get synced?

[milady133]

It gets kept as an identifier. In the same field where the ISBN or mobi-asin of Kindle ebooks is stored, the Goodreads Id is added