[GUI Plugin] Clean Metadata

[WS64]

My first plugin...

This plugin edits the metadata of selected books in your library.
You can either just mark the books that will get changed or directly execute the changes.
The main purpose is to mass-update many books at once that have pretty messed up metadata.

Main Features:

• Cleanup German umlauts in title: "Boser Ueberfall und Duell der Daemonen" => "Boser Überfall und Duell der Dämonen"
• Cleanup titles: "This is a book title (German Edition)" / "This is a book title. A novel" / "This_is_a_book_title" => "This is a book title"
• Get series info from titles: "Harry Potter 3: The Prisoner of Azkaban" => title: "The Prisoner of Azkaban", series: "Harry Potter", series_index: 3
• Cleanup authors: "Lastname, A B.C. Firstname; Anotherlastname, Anotherfirstname" => 1: "A. B. C. Firstname Lastname", 2: "Anotherfirstname Anotherlastname"
• Swap author and title if title matches an existing author: title: "Stephen King", author: "Cujo" => title: "Cujo", author: "Stephen King" *if* there is already one othebook in your library with author "Stephen King"

Special Notes:

• Requires Calibre 2.0.0 or later
• Use at your own risk! Due to some false findings you might create more damage to your metadata than you will repair using this plugin!
• So far I completely failed to save the settings of the two checkboxes, the interfacedemo plugin was no great help for this. If someone could help me here that would be great, but beware, I'm a complete newbie to Python and Qt5!

Installation Notes:

1. Download the attached zip file and install the plugin/restart Calibre/add to context menu as described in the Introduction to plugins thread.

Running from command line:

• not supported

Paypal Donations:

• nope. I wrote this mainly for myself and don't want to send money to myself!

Version History:

Spoiler:

Version 0.0.6 - 22 Nov 2014
Renamed from "WS64 Cleanup" to "Clean Metadata"

Version 0.0.5 - 20 Nov 2014
New function: Find Duplicate author candidates

Version 0.0.4 - 09 Nov 2014
Bugfixes and minor improvements

Version 0.0.3 - 07 Nov 2014
Initial release. Since just three days ago I have not seen any Python/Qt5 source code the code is far from perfect and probably still very buggy.

[jhowell]

Originally Posted by WS64

If someone has an idea how I can remove the "libprs500" from the progressbar window please let me know!

As Kovid mentioned in another thread you need to use setWindowTitle. I tried your plugin and saw the libprs title. I modified main.py to add

        progress.setWindowTitle("WS64 CleanUp")

just after

        progress = QProgressDialog("Processing books","Stop" , 0,len(ids) ,self)

and it worked for me.

[WS64]

@jhowell, thanks, worked!
I'm not sure why it didn't when I tried exactly this the first time...

[DaltonST]

Originally Posted by WS64

[*]Cleanup German umlauts in title: "Boser Ueberfall und Duell der Daemonen" => "Boser Überfall und Duell der Dämonen"

@WS64,

If you install the add-on/plug-in QuarantineAndScrub, be sure to add your umlaut word changes to table _title_rules. I would imagine that changing all "ue" and "ae" dipthongs to u's and a's with umlauts would be incorrect, since I noticed that Duell has no umlaut.

The Tag Rules table, _tag_rules, also would need to have entries for Tags that routinely should have an umlaut.


DaltonST

[WS64]

Hi DaltonST, thanks for your feedback.
I guess our plugins have similar goals. ;)
However, the installation of your QuarantineAndScrub looks too complicated for me to try it...

You are right of course that I don't change all "ue"s to "ü"s. I use a blacklist to avoid false changings. Still not perfect, but close...

[dijks005]

Hi, i am using yr plugin since yesterday. Great plugin!

However using it today i encoutered a problem. The title check is stopping at 61%.

This is the fault report:

calibre, version 2.11.0
FOUT: Niet afgehandelde fout: <b>error</b>:unbalanced parenthesis

calibre 2.11 [64bit] isfrozen: True is64bit: True
Windows-7-6.1.7601-SP1 Windows ('64bit', 'WindowsPE')
('Windows', '7', '6.1.7601')
Python 2.7.8
Windows: ('7', '6.1.7601', 'SP1', 'Multiprocessor Free')
Successfully initialized third party plugins: Anobii Fetcher && Fantastic Fiction && Barnes & Noble && Search The Internet && Quality Check && Find Duplicates && Extract ISBN && Goodreads && BOL_NL && Clean Metadata
Traceback (most recent call last):
File "calibre_plugins.CleanMetadata.main", line 125, in <lambda>
File "calibre_plugins.CleanMetadata.main", line 596, in process_books
File "calibre_plugins.CleanMetadata.main", line 379, in process_titles
File "re.py", line 142, in search
File "re.py", line 244, in _compile
error: unbalanced parenthesis

Is there something i can do?

tks

[WS64]

I stumbled upon the same problem the other day. Seems to happen when the author looks something like this: "abc(def" (no closing brackets).
Havn't checked any further so far (no time right now).
But maybe you can check if you find a book like this in your collection and check if the error goes away if you exclude that one.
Of course this is something I have to fix...

[dijks005]

OK, i will try to find such one. The plugin stopped at about 60%. How is yr plugin processing, starting at record 1, so i can skipp the first 60% of the database?

[dijks005]

Hi, I checked all 10k+ titles and not such an example is present in my database.

[a_librarian]

Hi,
The thread is quite old already, but maybe you're still reading here an have the time to reply.

I just had a look at your great plugin and I found it very useful, especially in terms of correcting German titles.
I also like the feature were you can extract the series from the title, but it doesn't work too well for me, probably because it looks for a specific format in the titles ('series' 'series_index': 'title', right?). When the format is not right, for example because it's 'author' - 'series' - 'series_index' - 'title', or just 'series' 'series_index', it does't find any entries.
Is there a way to give the plugin a certain string instead of the default one?
For example: I have a list of books, were the author or title field contain the 'series' 'series_index'. I would like to just bulk-copy that to the series field, instead of having to do it manually.

Thanks very much for the good work!
a librarian

[DaltonST]

https://www.mobileread.com/forums/sho...40048#poststop

[a_librarian]

Thanks very much for the link - that looks very promising! I don't know how I missed this before...

[AnselmD]

Hi,

I am using Ubuntu 16.04 LTS.

"CleanupGerman Umlauts in Title:" -> "Mark Only": the task is done, but the "Clean Metadata" Window is still there and has a corrupt content. I is not possible to close it. Even in in calibre Debug modus I do not see an error.

[jasonreadmobile]

Originally Posted by dijks005

Hi, I checked all 10k+ titles and not such an example is present in my database.

A bit of a necro-thread... I ran into the same problem as others with processing stopping midway. The plugin code needs to be modified to treat the fields it reads from the metadata more carefully.
The use of re.search with raw strings in the python code is interpreting "regex-like" strings from the title and author fields as regexes. Specific cases of "[2003-LABEL]" or something similar are seen as invalid character class ranges.
I modified my local version of the code to preprocess the title and author strings with "re.escape(str)" prior to use to protect against this.
I don't know how to resubmit my changes for the plugin, the owner can probably fix this if he/she cares.

[AnselmD]

Originally Posted by jasonreadmobile

I don't know how to resubmit my changes for the plugin, the owner can probably fix this if he/she cares.

Last Activity of Ian_Stott: 07-29-2012 09:23 PM

You can try to send a personal mail.

For another plugin, i add a version to the thread, see here:

https://www.mobileread.com/forums/sh...9&postcount=24