[Plugin] RemoveUnusedBookmarks - removes all unused bookmarks from epubs

[slowsmile]

Removes all unused bookmarks from epubs

Requirements
Plugin Type: Edit
MIT Licence(OSI)
Minimum Sigil requirement: v0.9.3 or higher
Python Requirements: Python 3.4+ (Bundled or External)
OS Requirements: Windows, Linux or OSX
Current Version: "0.1.8"

Installation
* Select Manage Plugins from the Plugins menu. In the dialog box, select either the Bundled Python or the External Python(Python 3.4+ should be installed on your computer to run this plugin externally).

* Click Add Plugin and select RemoveUnusedBookmarks_vXXX.zip. This will load and install the plugin into Sigil, which you can then select and run the plugin by selecting Plugins > Edit > RemoveUnusedBookmarls

Description
This epub2/epub3 plugin's main task is to remove all unused bookmarks(link destination ids) from the epub. The plugin also does the following tasks:

* Automatically converts all 'name' attributes to 'id' attributes.

* Removes all unused bookmarks/ids from the xhtml files including the toc.ncx and nav.xhtml(epub3 only).

* Automatically removes any anchor tags that contain no attributes.

* Ensures that all anchor start and end tags(with id only and no link) are always correctly formatted.

Caveat
* This plugin will remove all destination bookmarks/ids that are not directly used by internal links in the xhtml, toc.ncx or nav.xhtml(for epub 3). The only exception to this is if an id contains markers such as "toc", "cover", "text", "start", which will not be removed.

* This plugin can now process Epub 2 and Epub 3 files. Pagelists and nav maps are supported.

Plugin Run
Load your epub into Sigil and then just run the plugin.

At the end of the run, the plugin will now display a results message window showing number of files processed, total bookmarks found and total bookmarks removed.(v0.1.1)

Update: This plugin can now process epubs that contain svg images without giving svg errors in Epubcheck.(v0.1.3)

Update: Both Epub 2 and Epub 3 files can now be processed with this plugin.


Changes:

Spoiler:

v0.1.8

-- Fixed a problem with xml header layout spacing.

v0.1.7

-- Fixed a bug whereby if the epub contained SVG wrappers then the SVG code would be corrupted by the plugin, causing epubcheck errors.

v0.1.6

-- Improved html output(layout).

v0.1.5

-- Fixed several bugs which, under certain conditions, would give epubcheck errors.
-- This plugin can now be use with epub 2 and epub 3 files. My thanks to @democrite.

v0.1.4

-- The href ids are now more widely gathered from the following files: xhtml files, toc.ncx, opf manifest and opf guides. These href ids are then compared to existing ids/bookmarks in the xhtml files. The xhtml ids are only removed if they don't exist in the href id list.
-- NCX pagelists and page-map files are now not supported with this plugin. If either are detected in the epub then there will be a warning with bailout.

v0.1.3

-- Fixed a problem causing svg formatting errors in Epubcheck. SVG images can now be used in epubs without problems when using this plugin.

v0.1.2

-- The plugin now removes the meta line if it contains the 'name="calibre:cover"' attribute in the cover file. Thanks to Becky.
-- The plugin will now not change any "name" attribute values or names in any meta tags. Thanks to DiapDealer.

v0.1.1

-- Added a terminal window to display relevant plugin processing information for the user. Thanks to roger64.

v0.1.0

-- Initial release

[roger64]

Thank you for this useful plugin.

I would appreciate to read in the terminal window a comment of sort: "processed x files, found y bookmarks". Now it processes silentlly and I do not know if it really did the job.

[slowsmile]

@roger64...I'll try and add that terminal results window as a final info display when I have some time - I'm a bit busy at the moment.

[slowsmile]

Update: The plugin will now display a terminal message window at the end of processing to show relevant processing information to the user.(v0.1.1)

[roger64]

Thank you

[BeckyEbook]

@slowsmile: Think about it:

<meta name="description" content="Story awarded (first award)."/>
<meta name="keywords" content="award, story, john, doe, horror"/>
<meta name="author" content="John Doe"/>
<meta name="generator" content="BeckyPad 1.07"/>

And by the way about calibre:cover markers:

<meta name="calibre:cover" content="true"/>

Just skip the meta while checking.

[slowsmile]

@Becky...It's now 12:10 am in the Philippines where I live and I'm going to bed now. I'll try and put in a fix for the "calibre:cover" IDErrorCheck problem tomorrow morning.

I would also appreciate it you would keep problems concerning the IDErrorCheck plugin on the correct forum thread. This thread is primarily for RemoveUnusedBookmarks plugin problems.

[BeckyEbook]

This thread IS correct.
Plugin RemoveUnusedBookmarks destroys meta tags.

IDErrorCheck too.

CustomCleanerPlus "only" remove them.

[DiapDealer]

I'm unclear as to what you're referring to as "bookmarks" and what is being used to determine that they're "unused."

From what I can see, this plugin modifies id attributes and anchor tags. Am I missing something?

[slowsmile]

@DiapDealer...By bookmarks we mean ids.

What Becky is referring to is the the odd Calibre id that is inserted in Becky's cover.xhtml meta tag in her scrambled.epub document. The layout of the cover.xhtml file looks like this:

Spoiler:

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
<meta content="true" name="calibre:cover"/>
<title>Scrambled</title>
<style type="text/css">
@page {padding: 0pt; margin:0pt}
body { text-align: center; padding:0pt; margin: 0pt; }
</style>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
</head>

<body>

Becky is insistent that I should fix the above issue, which is actually caused by the Calibre conversion of her original doc to epub. This problem is caused by Calibre's non-standard use of meta ids in other words. I really don't know why that "calibre:cover" id is there in the meta tags of the cover file. It's obviously an irregular id that has been put there by Kovid Goyal for reasons that I am unable to fathom.

I can certainly put in a fix for this problem(by simply removing the offending meta line in the cover file) but it does irk me that such a non-standard and obviously fudged id is being generated by Calibre in all its conversions to epub which cause problems when my plugins are being used.

[DiapDealer]

Originally Posted by slowsmile

@DiapDealer...By bookmarks we mean ids.

OK, thanks. I'd just never heard them referred to as bookmarks before.

[BeckyEbook]

But I do not insist.
I just notice that the use of some plugins damages my metadata in files. The workaround would be to leave name unchanged in the meta tags.

The default cover template in calibre actually contains

<meta content="true" name="calibre:cover"/>

It would be better to remove the whole line than leave such a version (after using the plugin):

<meta content="true"/>

[BetterRed]

Originally Posted by slowsmile

I really don't know why that "calibre:cover" id is there in the meta tags of the cover file. It's obviously an irregular id that has been put there by Kovid Goyal for reasons that I am unable to fathom.

Rather than making snide comments here, why don't you ask Kovid Goyal directly why the "calibre:cover" id exists, he's only a PM away.

BR

[DiapDealer]

Becky's got a point. Your plugin really should be excluding meta tags when it's doing it's processing. Those tags aren't relevant to your plugin's stated workflow after all.

Calibre/Kovid's reasons for what he does are immaterial to this issue. "name" is a valid attribute for meta tags, and "calibre:cover" violates no xhtml value character rules. Your plugin should not mangle them (or any existing meta tag)--especially since they're not in your plugin's bailiwick to begin with.

Your code is changing every "name" attribute it finds (in any tag) to an "id" attribute. That's just a bad idea.

Change:

#ensure all bookmarks have only 'id' attributes
for tag in soup.find_all(True):
    if tag.has_attr('name'):
        idref = tag['name']
        del tag['name']
        tag['id'] = idref

to:

#ensure all bookmarks have only 'id' attributes
for tag in soup.find_all(True):
    if tag.has_attr('name') and tag.name != "meta":
        idref = tag['name']
        del tag['name']
        tag['id'] = idref

and be done.

There's no reason for this plugin to be changing any meta tags.

[slowsmile]

@DiapDealer...I've already fixed the 'calibre:cover' issue by removing the line in the cover file. But I take your point that meta tags with ids should not be checked or changed. Will work on a fix for that now. These fixes will be included in both IDErrorCheck and RemoveUnusedBookmarks plugins. And thanks for the heads up info on meta ids, I didn't know that.