[GUI Plugin] Quality Check

kiwidude · 03-13-2011, 07:59 AM

This plugin is intended to help you identify covers or book metadata that may need editing in your library that the normal calibre search criteria cannot identify, or is more conveniently accessible. It also offers a battery of ePub quality checks to help you identify ePubs that may warrant further attention to remove cruft files, reconvert etc. You can also search across your ePubs for ad hoc criteria to find text or specifically named items using regular expressions.

Amazon Fire users will find the Mobi checks useful to identify books which will not show in the correct folder due to internal metadata issues with the ASIN or cdetype.

For the OCD types, this plugin can be used to find books that need better quality covers based on physical or file size to be sourced to replace those in your library currently.

You may have noticed when editing individual books in the Edit Metadata screen that your book has a red colored background field indicating a potential problem. This plugin allows you to bulk find all such books in your library so that you can resolve any issues such as by using the bulk metadata edit dialog. A separate "Check missing" menu offers a quick way to serch for common missing data without having to remember calibre's search syntax.

Main Features:

Find ePub formats that have one, multiple, legacy or no jackets. Legacy jackets are from calibre conversions prior to 0.6.50 which will result in duplicated jackets if you convert again.
Find ePub formats that are invalid due to a missing container.xml file
Find ePub formats with invalid namespaces specified in the container or opf manifest
Find ePub formats that have non dc: namespace elements in the manifest
Find ePub formats that have files listed in the .opf manifest that are not in the ePub file
Find ePub formats that have files in the ePub that are not included in the .opf manifest
Find ePub formats that have CSS files in the ePub manifest that are not used from the html content
Find ePub formats that have unused image files that can be safely deleted
Find ePub formats that have iTunes plist or artwork files from viewing the ePub in iTunes
Find ePub formats that have calibre bookmarks files from the calibre ebook viewer
Find ePub formats that have OS artifacts like .DS_Store or Thumbs.db
Find ePub formats that have an NCX TOC that is hierarchical
Find ePub formats that have an NCX TOC with < 3 entries
Find ePub formats that have an NCX TOC with broken link entries
Find ePub formats that have broken <guide> references in the manifest
Find ePub formats that have html files larger than 260KB requiring splitting
Find ePub formats that have DRM
Find ePub formats that have Adobe DRM <meta> tags
Find ePub formats that have or do not have a cover that is replaceable when exporting from calibre
Find ePub formats that have or do not have SVG covers created by calibre
Find ePub formats that have or have not been converted using calibre
Find ePub formats that have <address> smart tags which corrupt readability
Find ePub formats that have embedded fonts
Find ePub formats that have @font-face declarations in CSS or html files
Find ePub formats that have Adobe .xpgt files with margins specified
Find ePub formats that have Adobe inline .xpgt links in the html files
Find ePub formats that have are left or unjustified i.e. no CSS file containing text-align:justify
Find ePub formats that have book margins different to your calibre defaults
Find ePub formats that have no book margins defined
Find ePub formats that have inline margins defined on the content files
Find ePub formats that have javascript files or <script> blocks
Find ePub formats that have unsmartened punctuation
Search the contents of ePub formats for ad hoc regular expressions of your own
Find Mobi/AZW/AZW3 formats that will not show in the documents folder on an Amazon Fire due to cdetype or ASIN missing/incorrect (with a Fix option below).
Find Mobi/AZW/AZW3 formats that have their Facebook/Twitter sharing disable on an Amazon Fire (with a Fix option below)
Find Mobi/AZW/AZW3 formats that have publisher imposed limits on the amount of text that can be placed in clipping notes when read on a Kindle.
Find books that have covers based on a choice of criteria such as file size or dimensions
Find books with an invalid title sort
Find books with an invalid author sort
Find books with an invalid ISBN
Find books with an invalid pubdate (equal to date timestamp)
Find books with a duplicate ISBN
Find books with a duplicate series
Find books with a series gap
Find books with a series numbering that does not match the published date
Find books with more than a specified number of tags
Find books with comments that have HTML style tags embedded like bold or italic
Find books with comments that have no HTML tags at all
Find books with commas in the author (useful if your author is FN LN)
Find books with no commas in the author (useful if your author is LN, FN)
Find books with authors that are all upper or all lower case
Find books with authors that contain non-alphabetic characters like incorrect separators or cruft
Find books with authors that contain non-ascii characters of accents/diatrics
Find books with authors that have initials in a different punctuation to your preference
Find books with titles that have possible series info like hyphens/numerics
Find books with titles that do not appear to be a valid title case
Search for books missing metadata, such as titles, authors, isbn, pubdate, publisher, tags, rating, comments, languages, cover or formats.
Change the search scope to either search your entire library (respecting search restrictions) or only your selected book(s).
Swap author names for the selected book(s) between FN LN and LN, FN order or vice versa
Fix author initials for selected books to your preferred naming format
Rename author names for selected books to replace non-ascii characters
Find and fix the file sizes stored for a book format (useful if you edit books outside of calibre)
Find and fix folder/file paths that are missing commas for inconsistencies within a library for books added after calibre 0.8.35
Cleanup orphaned opf/jpg files/folders resulting from using Save to Disk followed by Remove from device. Do NOT ever run this against a calibre library folder, it is intended for external save to disk/device folders only.
Fix Mobi/AZW/AZW3 files for a Kindle Fire to insert an ASIN and set the correct cdetype in the internal data to enable sharing and appear in the correct folder
Ability to add books to an exclusion list for a particular quality check, if you intentionally want to exempt them. You can view and edit your exclusion lists.
Customise the menus to hide items not of interest

Special Notes:

Help Documentation
Change History
The default scope of all checks is to your entire library if you have no search restrictions in place, or just to the restricted subset of books if you have applied a restriction. You might find applying a restriction useful if you have a very large library to do cover checks and some of the ePub checks which can be time consuming. Or you can use the "Search scope" submenu to switch to searching selected books only.
Requires calibre v3.41.0 or later.

Paypal Donations:

If you find this or any of my other plugins useful please feel free to show your appreciation. I have spent many hundreds of unpaid hours in their development and support so any encouragement for me to continue is appreciated!

theducks · 03-13-2011, 12:47 PM

Once again, you have done it

(FWIW I just gave it a run on my Ubuntu Maverick system)

kiwidude · 03-19-2011, 06:46 PM

I've added a few more checks to this plugin. Most of these replicate what you can do with saved searches, but I just found it convenient to include and saves the user the effort of creating them.

Additions to v1.1:

Find books with comments that have no HTML. Note that this is not the same as inverting the "comments with HTML" search results. This search finds books with no HTML at all be it style tags or paragraph/div tags added by Calibre when you save from the Edit Metadata dialog.
Find authors with commas in name (for people who use FN LN)
Find authors with no commas in name (for people who use LN, FN)
Find titles with possible series info (searches for hyphens or numeric values in the title).

All three of the latter searches can be customised within the search dialog so you may choose to make them more specific to how you require your authors/titles to be stored.

pchrist7 · 03-20-2011, 04:39 AM

Another brilliant Tool !
Why not fix thinks before Catalog fails, etc.
Works fine so far here !
Actually found that 2 new Calibre Catalogs had Author_sort mismatch ;-D
(Author:calibre, Author_sort:date of creation in Danish format/language)

DoctorOhh · 03-20-2011, 05:32 AM

Quote:

Originally Posted by pchrist7

Another brilliant Tool !

...

Quote:

Originally Posted by pchrist7

Why not fix thinks before Catalog fails, etc.

Actually found that 2 new Calibre Catalogs had Author_sort mismatch ;-D

I'm glad you found the Author_sort mismatch, but these mismatches don't mean your catalog will fail. I have a few dozen Author_sort mismatches and catalog creation works fine. I often enter the author_sort field with data that doesn't match the author field. This is a feature, not a bug and I use it often.

GRiker · 03-20-2011, 07:03 AM

Quote:

Originally Posted by dwanthny

I'm glad you found the Author_sort mismatch, but these mismatches don't mean your catalog will fail. I have a few dozen Author_sort mismatches and catalog creation works fine. I often enter the author_sort field with data that doesn't match the author field. This is a feature, not a bug and I use it often.

Catalog generation does fail for MOBI when author_sort mismatches are encountered, but not EPUB. MOBI generation is where the author_sort mismatch is potentially problematic.

G

DoctorOhh · 03-20-2011, 07:39 AM

Quote:

Originally Posted by GRiker

Catalog generation does fail for MOBI when author_sort mismatches are encountered, but not EPUB. MOBI generation is where the author_sort mismatch is potentially problematic.

Thanks for the clarification. If I ever get a Kindle I'll keep this in mind.

theducks · 03-20-2011, 09:10 AM

Quote:

Originally Posted by GRiker

Catalog generation does fail for MOBI when author_sort mismatches are encountered, but not EPUB. MOBI generation is where the author_sort mismatch is potentially problematic.

G

How do you Not have a mis-match when the title is mistakenly treated as having an article and you have to correct it by hand?

Title sample: A is for Apple
(no quotes, anywhere)

Do other work-around's and you have look-up failures

BTW I have not recently tested to see if this triggers a fail

GRiker · 03-20-2011, 09:29 AM

Eh? The catalog generator only looks for author_sort mismatches on the same author (and only during MOBI generation). I'm not sure what failure you're referring to with title articles in reference to catalogs.

G

theducks · 03-20-2011, 09:35 AM

Quote:

Originally Posted by GRiker

Eh? The catalog generator only looks for author_sort mismatches on the same author (and only during MOBI generation). I'm not sure what failure you're referring to with title articles in reference to catalogs.

G

My bad,
The plugin (and the Red Title-sort box) indicated a mis-match and I thought all mis-matches caused problems.

kiwidude · 03-27-2011, 10:55 AM

This adds a single new feature of "Check and fix book size". This will iterate through all of the book formats in your library and update where necessary the internal Calibre file size that is stored.

So if you make a habit of editing your books once they have been added to the library (such as editing an EPUB in Sigil), you can use this function to ensure your file sizes stored in Calibre are up to date without any re-adding of the book format to Calibre.

Perkin · 03-27-2011, 11:51 AM

Lovely-jubbly.
I asked about filesize-update to be incorporated into calibre ages ago.

Thanks.

theducks · 03-27-2011, 02:30 PM

I had a filter in place when I ran a Covers quality check.
Those were the only ones that got checked.
A great time saver

kiwidude · 03-27-2011, 04:42 PM

Indeed, sorry I should have mentioned on the first post that was the behaviour applied. Glad it was a nice surprise for you

theducks · 03-27-2011, 05:44 PM

Quote:

Originally Posted by kiwidude

Indeed, sorry I should have mentioned on the first post that was the behaviour applied. Glad it was a nice surprise for you

Nothing to be sorry about. The PI did everything you said it would

Think of it as an 'Easter Egg' to be discovered

03-19-2011, 06:46 PM	#3
kiwidude Calibre Plugins Developer Posts: 4,682 Karma: 2162246 Join Date: Oct 2010 Location: Australia Device: Kindle Oasis	v1.1 Released I've added a few more checks to this plugin. Most of these replicate what you can do with saved searches, but I just found it convenient to include and saves the user the effort of creating them. Additions to v1.1: Find books with comments that have no HTML. Note that this is not the same as inverting the "comments with HTML" search results. This search finds books with no HTML at all be it style tags or paragraph/div tags added by Calibre when you save from the Edit Metadata dialog. Find authors with commas in name (for people who use FN LN) Find authors with no commas in name (for people who use LN, FN) Find titles with possible series info (searches for hyphens or numeric values in the title). All three of the latter searches can be customised within the search dialog so you may choose to make them more specific to how you require your authors/titles to be stored.

03-27-2011, 10:55 AM	#11
kiwidude Calibre Plugins Developer Posts: 4,682 Karma: 2162246 Join Date: Oct 2010 Location: Australia Device: Kindle Oasis	v1.2 Released This adds a single new feature of "Check and fix book size". This will iterate through all of the book formats in your library and update where necessary the internal Calibre file size that is stored. So if you make a habit of editing your books once they have been added to the library (such as editing an EPUB in Sigil), you can use this function to ensure your file sizes stored in Calibre are up to date without any re-adding of the book format to Calibre.

03-27-2011, 02:30 PM	#13
theducks Well trained by Cats Posts: 30,443 Karma: 58055868 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	Plugin respects filters I had a filter in place when I ran a Covers quality check. Those were the only ones that got checked. A great time saver

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[GUI Plugin] Search the Internet	kiwidude	Plugins	434	09-30-2024 04:04 AM
[GUI Plugin] Clipboard Search	kiwidude	Plugins	29	04-02-2024 11:05 PM
[GUI Plugin] Open With	kiwidude	Plugins	403	04-01-2024 09:39 AM
[GUI Plugin] Kindle Collections (old)	meme	Plugins	2070	08-11-2014 01:02 AM
[GUI Plugin] Book Sync Deprecated	kiwidude	Plugins	111	06-07-2011 08:47 PM

03-13-2011, 12:47 PM	#2
theducks Well trained by Cats Posts: 30,443 Karma: 58055868 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	Once again, you have done it (FWIW I just gave it a run on my Ubuntu Maverick system)

03-20-2011, 04:39 AM	#4
pchrist7 Addict Posts: 385 Karma: 6514 Join Date: Aug 2010 Location: Denmark Device: Kindle 3 3G+Wifi, Oasis	Another brilliant Tool ! Why not fix thinks before Catalog fails, etc. Works fine so far here ! Actually found that 2 new Calibre Catalogs had Author_sort mismatch ;-D (Author:calibre, Author_sort:date of creation in Danish format/language)

03-20-2011, 09:29 AM	#9
GRiker Comparer of the Ephemeris Posts: 1,496 Karma: 424697 Join Date: Mar 2009 Device: iPad	Eh? The catalog generator only looks for author_sort mismatches on the same author (and only during MOBI generation). I'm not sure what failure you're referring to with title articles in reference to catalogs. G

03-27-2011, 11:51 AM	#12
Perkin Guru Posts: 657 Karma: 64171 Join Date: Sep 2010 Location: Kent, England, Sol 3, ZZ9 plural Z Alpha Device: Sony PRS-300, Kobo Aura HD, iPad (Marvin)	Lovely-jubbly. I asked about filesize-update to be incorporated into calibre ages ago. Thanks.

03-27-2011, 04:42 PM	#14
kiwidude Calibre Plugins Developer Posts: 4,682 Karma: 2162246 Join Date: Oct 2010 Location: Australia Device: Kindle Oasis	Indeed, sorry I should have mentioned on the first post that was the behaviour applied. Glad it was a nice surprise for you

Advert

Advert