[GUI Plugin] Library Codes

[davidjoseph1]

Originally Posted by iienderii

Anyone out there have a functioning alternative in the meantime? I'm hopeful that this plugin can be brought back into action, but if not, what else are you using to tie LCC and/or DCC codes to your Calibre library books? If at all possible, I'm hoping to not have to do it manually.

I was playing around with Tellico, which can import and export to calibre, and which can do Z39.50 queries, but it's not as beautifully robust as Library Codes has been. The boolean search language isn't flexible at all. this is Tellico 3.4.4 on Ubuntu 22.04

If I knew what I was doing in python, I would try to code up a decent calibre Z39.50 plugin, but I absolutely do not know what I am doing.

[webtroter]

It seems that Classify2 was discontinued on Jan 31st

https://www.oclc.org/go/en/classify-...tinuation.html

[SidJnsn]

Well that sucks. I was hoping to get back around to this and update the code to handle the last roadblock that arose, but scratch that.
I see that OCLC has added DDC to their Worldcat records, so that will be worth a look. Also, Anna’s Archive screen-scraped Worldcat sometime back and made the data freely available. I’m wondering if they did it before DDc was added or after.

[kokeyama]

Maybe the library codes plugin can be changed to use the LOC API for metadata access. The vast majority of books have good LCC codes available, and at least half in my manual testing have also shown DCC codes (via the web interface, not the API).

https://catalog.loc.gov/vwebv/ui/en_...elp/index.html

[davidjoseph1]

Originally Posted by kokeyama

Maybe the library codes plugin can be changed to use the LOC API for metadata access. The vast majority of books have good LCC codes available, and at least half in my manual testing have also shown DCC codes (via the web interface, not the API).

https://catalog.loc.gov/vwebv/ui/en_...elp/index.html

I tried to update Library Codes to use the modern LC Catalog language, but I found an old metadata source plugin called SRU that did the same thing, and got it working.

https://www.mobileread.com/forums/sh...14&postcount=5

[davidjoseph1]

Originally Posted by davidjoseph1

I tried to update Library Codes to use the modern LC Catalog language, but I found an old metadata source plugin called SRU that did the same thing, and got it working.

https://www.mobileread.com/forums/sh...14&postcount=5

To follow up on this brief post:

Library Codes generated a query based on LCCN, ISBN, or ISSN that returned a page from the OCLC Classify service, stripped the table inside that page, and used BeautifulSoup to scan that table for markup elements denoting the various custom columns (Dewey, LoC classification, etc). I was trying to figure out how to substitute the loc publicly served query page into that beautiful soup protocol that DaltonST coded, but the markup is xml and I didn’t figure out how to do that.

However, DaltonST chose to implement a scraper for a resource that wasn’t optimized for data interchange - namely, whatever html generated by Classify. However, there *are* much more convenient and interpretable data formats that can be called via various services. The Z 39.50 protocol was adapted for service through a Web interface and this protocol is called SRU. The fields are standardized, there are many SRU and Marcxml Services out there can be par by regular expression to extract the data from the labeled fields


Long time ago, more than 11 years, a gentleman In Germany coded the metadata source plugin SRU, Which does exactly that. The parser doesn’t use Beautiful soup, it hits the SRU server with an extremely fast recall time, and the SRU service provides many more data fields than were present on the old class webpage. In addition, SRU Returns, As One of the fields , a static marcxml url that provides the MARCXML extremely efficiently, and I would almost prefer to parse that, But I have no idea how to do that. It does embed a LOC link as an identifier in the calibre book record.

I am trying to do is perform a transplant of the search language and configuration in SRU into Library codes, in order to return the parsed Significant data fields into library codes python routine for updating the custom column.

I’ve never done python before, taught myself everything I needed to learn in order to update SRU for calibre 6.17, and I would welcome any help that people might provide. SRU as written cannot pass custom column information as a metadata source plugin - I tried that already.


Any suggestions?

[teditor]

I have no help to give, but I do have cheers, eagerness, and hope! Thanks for continuing to work on a solution.

[davidjoseph1]

Development efforts update:

DaltonST posted two older versions of Library Codes using the Classify API, and I am dissecting them in order to replace the deprecated Classify lookup with an alternative using SearchRetrieve(over)Url interface language at the Library of Congress SRU server.

I have roughed out two working scripts that return LCCN (card catalog entries) and Dewey and Library of Congress classification codes from fields '010', '050', and '082' of the MARC21 specification implemented in marcxml at the permalink "http://lccn.loc.gov/[LCCNgoeshere]/marcxml" and now I have to process the subject fields found in the '65x' fields. I may end up parsing MODS instead to make my life easier.

I'm trying to recover the VIAF and FAST functionality from the subject and name authorities file at the library of congress name and subject authority databases on the SRU servers.

[davidjoseph1]

I am happy to report I have modified the Classify scraper to query the SRU server at the library of congress for MODS 3.8 xml and from that return DDC and LCC fields. It's not perfect - I haven't been able to recover the FAST subject heading information yet, and there is no VIAF or author data or oclc-owi information yet, but this is finally working again.

[pdurrant]

Originally Posted by davidjoseph1

I am happy to report I have modified the Classify scraper to query the SRU server at the library of congress for MODS 3.8 xml and from that return DDC and LCC fields. It's not perfect - I haven't been able to recover the FAST subject heading information yet, and there is no VIAF or author data or oclc-owi information yet, but this is finally working again.

As the original author hasn't been active on MobileRead for a year now, and this looks like more than a minor modification, perhaps you could start a new plugin thread, with a link back to this one, of course.

That way your new, working, plugin would be available through the calibre plugin interface.

[PeterT]

Originally Posted by pdurrant

As the original author hasn't been active on MobileRead for a year now, and this looks like more than a minor modification, perhaps you could start a new plugin thread, with a link back to this one, of course.



That way your new, working, plugin would be available through the calibre plugin interface.

Wouldn't this also need a new name for the plug-in to differentiate it from the existing one?

Sent from my Pixel 7a using Tapatalk

[JSWolf]

Originally Posted by PeterT

Wouldn't this also need a new name for the plug-in to differentiate it from the existing one?

Sent from my Pixel 7a using Tapatalk

Why not just replace the version in the first post? It's been done in other threads when mods were made that fix something or added new feature(s).

[davidjoseph1]

thanks for the suggestion, I have posted a new thread.

[pdurrant]

Originally Posted by JSWolf

Why not just replace the version in the first post? It's been done in other threads when mods were made that fix something or added new feature(s).

It rather depends on whether the function has changed or not, and whether the maintainer seems keen to continue with tweaks.

New thread here: https://www.mobileread.com/forums/sh...d.php?t=362404

I've also added a link in the first post of this thread. A one-off update I can do.