LibraryThing Match plugin - beta testers wanted

[Bradles]

Note: This thread will soon be closed to further comments. Please refer to the main thread at :

[GUI Plugin] LibraryThing Match

Are you a calibre user who also uses LibraryThing? Or a LibraryThing user who also uses calibre? If so, I've got a little job for you...

LibraryThing is a social cataloging web application for storing and sharing book catalogs and various types of book metadata. It is used by authors, individuals, libraries, and publishers.

LibraryThing Match (LTMatch) is a calibre plug-in for comparing your calibre e-book list with your LibraryThing catalog.

If you'd like to test LTMatch, please install it manually by downloading the attached zip file, and installing as described in "How do I install a plugin manually?" in the Introduction to plugins thread.

LTMatch can be run in "non-invasive" mode, i.e. without modifying the calibre database in any way. For best results, it can be used with a custom column as described in the attached user guide.

Of particular interest during testing is the "first run" experience, performance with large libraries, and behaviour on non-windows systems.

Thanks and good luck!

[aborel]

Originally Posted by Bradles

Are you a calibre user who also uses LibraryThing? Or a LibraryThing user who also uses calibre? If so, I've got a little job for you...

I was very excited by the proposition, but unfortunately I am not yet ready to upgrade to Calibre 5. Does your plugin actually rely on Python 3 elements, or Calibre 5-specific features? Or did you simply choose a requirement that is not unreasonable but might not be not be necessary? That would be perfectly understandable, I'm asking in order to decide whether I will try to tinker with the code and make the check more permissive :-)

[JSWolf]

Originally Posted by aborel

I was very excited by the proposition, but unfortunately I am not yet ready to upgrade to Calibre 5. Does your plugin actually rely on Python 3 elements, or Calibre 5-specific features? Or did you simply choose a requirement that is not unreasonable but might not be not be necessary? That would be perfectly understandable, I'm asking in order to decide whether I will try to tinker with the code and make the check more permissive :-)

Why aren't you ready to upgrade to Calibre 5.x? If you run Windows, you can install a portable version of 5.x.

[aborel]

Originally Posted by JSWolf

Why aren't you ready to upgrade to Calibre 5.x?

A certain plugin that may not be discussed in depth here isn't quite reliable enough for my taste in its Calibre 5-compatible version. It's getting there, but I'm not yet ready to jump.

If you run Windows (...)

I do not

[rosalita]

Originally Posted by Bradles

Are you a calibre user who also uses LibraryThing? Or a LibraryThing user who also uses calibre? If so, I've got a little job for you...

I am a longtime user of Library Thing and would love to help test this but I cannot upgrade to 5.x Calibre on my iMac due to plug-in incompatibility issues. If you ever figure out a way to make it work with older versions of Calibre, let me know!

[JSWolf]

Originally Posted by aborel

A certain plugin that may not be discussed in depth here isn't quite reliable enough for my taste in its Calibre 5-compatible version. It's getting there, but I'm not yet ready to jump.

The 7.0.3 version of the DeDRM plugin works for KF8, ePub, and Mobi. It doe snot work for PDF. So for me, it is working. I've not had any trouble with it for those three formats.

So make a backup of your current configuration, save your keys from DeDRM and then upgrade both Calibre and DeDRM.If DeDRM is not working, delete the .jason file and reload all the keys and you'll be good to go. It's that easy.

[aborel]

Originally Posted by JSWolf

The 7.0.3 version of the DeDRM plugin works for KF8, ePub, and Mobi. It doe snot work for PDF. So for me, it is working.

I know. Well, that's not good enough for me, and Calibre 4.X has been more than enough for me until now. I am of course following the progress of DeDRM and at some point I'm sure I will find it mature enough to switch. Until then I don't feel a strong need to switch to Calibre 5 (this should not be understood as a criticism of Calibre 5, by the way: I completely understand the reasons for the plugin incompatibility and I think Kovid was 110% right to move to Python3).

Now here I am seeing a plugin that looks interesting to me, I'm observing that it requires Calibre >= 5.0.0, and I'm just curious on the strength of this requirement. Not disappointed, not frustrated, just curious. I don't mean that it needs to be 4.X-compatible, I'm just interested in testing that possibility. And apparently it will take more than a single code change, as far as I can tell from my first quick test. Not a huge problem, but perhaps a fun challenge.

And now, returning to the original topic: kudos to Bradles for the cool idea, obviously!

[Bradles]

Originally Posted by aborel

<snip>

Now here I am seeing a plugin that looks interesting to me, I'm observing that it requires Calibre >= 5.0.0, and I'm just curious on the strength of this requirement. Not disappointed, not frustrated, just curious. I don't mean that it needs to be 4.X-compatible, I'm just interested in testing that possibility. And apparently it will take more than a single code change, as far as I can tell from my first quick test. Not a huge problem, but perhaps a fun challenge.

And now, returning to the original topic: kudos to Bradles for the cool idea, obviously!

Thanks aborel.

To answer your question, I would say it requires Python 3 / calibre 5. Why? Well, I'm a very inexperienced python developer, so I had to look up nearly EVERYTHING! Where there was a choice how to do something, I usually chose the python 3 version. Examples:

• conditional assignment - if (a = b):
• setting logging level with a string
• ordered dictionaries (more an assumption than requirement)

I suspect it would be quite difficult to unpack. (I did a quick test with 4.23.)

I hesitated to upgrade to 5 as well. I eventually concluded that a certain plugin would NEVER be finished, so the parts that were working were good enough for me.

IMO the suggestion of upgrading to 5 and keeping a portable 4 "for emergencies" is the way to go.

[davidfor]

Having a quick look at the first post, is there any reason you are not using an identifier and a metadata source plugin? This would tie into the metadata download process and set the identifier for the book. You then have the LibraryThing listed in the details pane and can click to view the page.

You would need a separate plugin to update LibraryThing with your books, or to mark somehow in Calibre that you have the book in LibraryThing. This could be a separate column or a tag.

You probably have considered the above, but I felt the need to put them out there. The comments come from maintaining the Goodreads metadata source plugin and the Goodreads Sync plugin. The latter is doing the steps of syncing between the GR and the calibre library. It does things in both directions. Whether the API you are using allows this sort of interaction, I do not know.

[Bradles]

Hi David,

Thanks for your question. The two possible answers are:

1. I was more interested in the simple existence of the book in each source, not so much in syncing the metadata. More a personal preference in how I use LT and calibre.

2. Complexity. This is my first "real" plugin, my first attempt at python programming, and my first experience with web-related development. I wanted to keep it simple.

I have looked at a couple of your metadata source plugins, and I can foresee it wouldn't be TOO difficult to adapt one for LT. I do like the idea of clicking on the identifier to go to the book in LT. I will give this some thought if the current plugin gets any traction.

To touch on your other comments, I do use a "LibraryThing ID" column in calibre to keep track of the matched books. The LT API is one-way, so no facility to update books there.

[davidfor]

Originally Posted by Bradles

Hi David,

Thanks for your question. The two possible answers are:

1. I was more interested in the simple existence of the book in each source, not so much in syncing the metadata. More a personal preference in how I use LT and calibre.

Which is completely sensible thing to do.

2. Complexity. This is my first "real" plugin, my first attempt at python programming, and my first experience with web-related development. I wanted to keep it simple.

I have looked at a couple of your metadata source plugins, and I can foresee it wouldn't be TOO difficult to adapt one for LT. I do like the idea of clicking on the identifier to go to the book in LT. I will give this some thought if the current plugin gets any traction.

In general, the metadata source plugins are easy. The complexity comes from what the search on the site returns. And that they like to change.

You can get the same effect without a metadata source plugin. You can add a rule for identifiers to create the link. This is done in the preferences on the "Look & Feel" page and then the "Book details". Press the "Create rules to convert..." button. I have several of these but an example if for the MobileRead library:

Key: mobileread
Name: MobileRead
Template: https://www.mobileread.com/forums/showthread.php?t={id}

As you have a separate column, you could also add a column using a template to give a link. I haven't done this, but, I know it's been discussed here in the past.

To touch on your other comments, I do use a "LibraryThing ID" column in calibre to keep track of the matched books.

The existence of the identifier could be used for that. Or a tag. I have a "status" column for reading status and couple of other things. It is a tag-like column, so adding a "LibraryTag" to that might work. Or using the Reading List plugin.

The LT API is one-way, so no facility to update books there.

That's a pity. And good to know. As I'm sure I have seen a suggestion to replace GR with LibraryThing as the GR API I use is no longer supported and might stop working at any time. I thought I'd seen LibraryThing suggested as an alternative, but, if there is no API support for updating it, it isn't very attractive.

[Bradles]

Yep, got your gist now.

LT has concepts of "work", which is the generic published work (e.g. Little Women by Louisa May Alcott) and "book", which is the specific instance of the work as it appears in users' catalogs (e.g. Little Women by Louisa May Alcott, ebook, isbn, cover, my review, my rating, etc.)

I'm using the "JSON Books API" to return the list of users books. LT users are very protective of their book metadata, and I suspect any metadata updates should only come from their book metadata. Unfortunately, this API has no way to return a single book based on any identifier.

The "JSON Works API" returns a work based on an ISBN search, but can only be used once per second and up to 1000 times per day. This would be okay for editing metadata one at a time but not in bulk.

For the ID linking, my quick test worked great:

https://www.librarything.com/work/book/{id}

Simple as that. Would there be any concerns over updating that identifier outside of a metadata plugin? e.g. (cribbing from your Smashwords plugin):

mi = Metadata(...)
mi.set_identifier('librarything', librarything_id)
self.lt_id = librarything_id

[davidfor]

Originally Posted by Bradles

Yep, got your gist now.

LT has concepts of "work", which is the generic published work (e.g. Little Women by Louisa May Alcott) and "book", which is the specific instance of the work as it appears in users' catalogs (e.g. Little Women by Louisa May Alcott, ebook, isbn, cover, my review, my rating, etc.)

Goodreads does roughly the same. They call "works" and "edition".

I'm using the "JSON Books API" to return the list of users books. LT users are very protective of their book metadata, and I suspect any metadata updates should only come from their book metadata. Unfortunately, this API has no way to return a single book based on any identifier.

The "JSON Works API" returns a work based on an ISBN search, but can only be used once per second and up to 1000 times per day. This would be okay for editing metadata one at a time but not in bulk.

That's not uncommon. What are they counting this agains? An IP address or the user? Or a developers key?

For the ID linking, my quick test worked great:

https://www.librarything.com/work/book/{id}

Yep.

Simple as that. Would there be any concerns over updating that identifier outside of a metadata plugin? e.g. (cribbing from your Smashwords plugin):

mi = Metadata(...)
mi.set_identifier('librarything', librarything_id)
self.lt_id = librarything_id

I can think of a couple of places this is done, including the Goodreads Sync plugin. Basically, as long as you don't break anything, it will be OK.

[maddz]

I would love to help with testing, but alas I'm stuck with Calibre 3.48 - the highest version I can access with my current Mac OS (High Sierra). So no Calibre 5x for me.

What I've been doing to link my LT record to my Calibre record is to insert the LT Book ID in the IDs field in my Calibre record as ltid: and to add the calibre ID into my LT book entry as an 'other call number' as Calibre ##### (using leading zeros). This is done as part of my manual logging process as I acquire new ebooks and load them into Calibre.

It's not perfect, because where I split omnibuses, the IDs get reused, and when I merge short stories, there's multiple IDs. It's also manual, but I can process a record in a couple of minutes, so it's not too onerous. It would be nice to be able to automate the matching and insertion of IDs.

[Bradles]

Originally Posted by maddz

What I've been doing to link my LT record to my Calibre record is to insert the LT Book ID in the IDs field in my Calibre record as ltid: and to add the calibre ID into my LT book entry as an 'other call number' as Calibre ##### (using leading zeros). This is done as part of my manual logging process as I acquire new ebooks and load them into Calibre.

I suspect a future version of LTMatch will use the calibre identifiers, then leverage that to link directly to users' books in LT.

It's not perfect, because where I split omnibuses, the IDs get reused, and when I merge short stories, there's multiple IDs. It's also manual, but I can process a record in a couple of minutes, so it's not too onerous. It would be nice to be able to automate the matching and insertion of IDs.

LTMatch is one-way (due to limitations of the LT interface), so there's no automated way to update LT with your calibre IDs.

Best of luck.