[Plugin] CustomCleanerPlus

[DNSB]

Originally Posted by slowsmile

And, like I've said, I can't fix that problem if the cause is inside python's BeautifulSoup module. So your only option appears to be to manually remove all of those useless single tag span entities in Sigil using search and replace.

As I've mentioned, I've already added the search lines to convert the problem lines to simple spaces to my saved searches. I was just hoping that would be unnecessary but since it is necessary, I can live with it. My search/replace list is getting a bit unwieldy and trying to automate is being a bit of a pain.

The first time was the nasty surprise since if I hadn't run Spellcheck again, I wouldn't have noticed those missing spaces which might have made the client unhappy. Now that I am aware of it, I can avoid the issue.

[BeckyEbook]

@slowsmile: Don't give up! Your plugin has potential.

I know you don't want to improve this plugin anymore, but still, I found a few bugs in it, as well as the one that caused spaces to "disappear".

The options.py file was also missing the default entry

REMOVE_EMPTY_PARAS = False

I didn't check all the features in the plugin.
I was doing a clean test on the example EPUB file attached.

I have shown my modifications in the attached image.

Note that I called this problem a "last file issue" due to the lack of indentation in the cutils2.py file.
This lack of indentation caused the last commands (including space destruction) to appear only in the last (x)HTML file. I think this was the main reason why you could not reproduce the problem described by @DNSB.

The main culprit was a single line that was saving the file and, in the process, cutting out spaces.

You do what you want with it.

Spoiler:

From 037f91289da18d3105605c73205a50fc55b463cf Mon Sep 17 00:00:00 2001
From: BeckyDTP <becky@fr.pl>
Date: Mon, 11 Dec 2023 16:59:40 +0100
Subject: [PATCH] Fix

---
 cutils.py   |  3 ++-
 cutils2.py  | 14 +++++++-------
 dialogs2.py |  6 +++---
 3 files changed, 12 insertions(+), 11 deletions(-)

diff --git a/cutils.py b/cutils.py
index 98c4295..0da707c 100644
--- a/cutils.py
+++ b/cutils.py
@@ -1381,7 +1381,8 @@ def svgAttributes2CamelCase(wdir, file):
         if not svg.has_attr('xmlns:xlink'):
             svg['xmlns:xlink'] = "http://www.w3.org/1999/xlink"        
                
-    outfp.writelines(str(soup.prettyprint_xhtml(indent_level=0, eventual_encoding="utf-8", formatter="minimal", indent_chars="  ")))
+    outfp.write(str(soup))
+    #outfp.writelines(str(soup.prettyprint_xhtml(indent_level=0, eventual_encoding="utf-8", formatter="minimal", indent_chars="  ")))
     outfp.close()
     os.remove(file)
     os.rename(output, file)
diff --git a/cutils2.py b/cutils2.py
index 830ce2b..832a686 100644
--- a/cutils2.py
+++ b/cutils2.py
@@ -1215,13 +1215,13 @@ def removeUnusedBookmarks2(wdir, t_fnames, href_ids):
                 if tag.name == 'a' and tag.attrs == {}:
                     tag.unwrap()
                     
-    outfp.write(str(soup))             
-    #outfp.write(str(soup.prettyprint_xhtml(indent_level=0, eventual_encoding="utf-8", formatter="minimal", indent_chars="  ")))
-    outfp.close()
-    os.remove(file)
-    os.rename(outfile, file)
-    options.TOTAL_REMOVED_BOOKMARKS = bm_removed         
-    svgAttributes2CamelCase(wdir, file)
+        outfp.write(str(soup))             
+        #outfp.write(str(soup.prettyprint_xhtml(indent_level=0, eventual_encoding="utf-8", formatter="minimal", indent_chars="  ")))
+        outfp.close()
+        os.remove(file)
+        os.rename(outfile, file)
+        options.TOTAL_REMOVED_BOOKMARKS = bm_removed         
+        svgAttributes2CamelCase(wdir, file)
     return(0)        
  
 def svgAttributes2CamelCase_New(wdir, file):
diff --git a/dialogs2.py b/dialogs2.py
index 105012e..00c1519 100644
--- a/dialogs2.py
+++ b/dialogs2.py
@@ -423,19 +423,19 @@ class AppDialog2(Tk):
                 options.REMOVE_IDS = False                        
                 
             gui_prefs['remove_divs'] = self.div.get()
-            if self.div == True:
+            if self.div.get() == True:
                 options.REMOVE_DIV_TAGS = True
             else:
                 options.REMOVE_DIV_TAGS = False          
 
             gui_prefs['remove_page_links'] = self.rmpagelinks.get()
-            if self.rmpagelinks == True:
+            if self.rmpagelinks.get() == True:
                 options.REMOVE_PAGE_LINKS = True
             else:
                 options.REMOVE_PAGE_LINKS = False                    
 
             gui_prefs['remove_empty_spans'] = self.rmempty.get()
-            if self.rmLH == True:
+            if self.rmempty.get() == True:
                 options.REMOVE_EMPTY_SPANS = True
             else:
                 options.REMOVE_EMPTY_SPANS = False                               
-- 
2.39.1.windows.1

[DNSB]

@BeckyEbook: Hate to be the bearer of bad news but I added a second XHTML file to my test epub and running the 0.5.6 code, I lost the spaces in both files. Not to mention that 0.5.6 pops up a notification that an update 0.5.5 is available due to the version checker code only checking that the installed version is not equal to the downloadable from message 1 version. If I change the != on line 84 of plugin.py to >=, the message disappears.

    if is_connected: 
        # check for new plugin versions
        latest_version, installed_version = updateCheck(SITE_URL, PLUGIN_PATH)
        if latest_version and latest_version >= installed_version:
            options.NEW_PLUGIN_VERSION = True
            options.MSG_NEW_VERSION_AVAILABLE = "A new version of this plugin is now available from MR - v" + latest_version

I'm tending to agree with slowsmile that this issue is not really worth the effort to fix it and I'll live with my search/replace solution.

[BeckyEbook]

Oh, I would be happy to check your file.
You can send me a piece via PM.


This problem is indeed not a major issue, but other fixes in the plugin are worth improving.

[DNSB]

Originally Posted by BeckyEbook

Oh, I would be happy to check your file.
You can send me a piece via PM.

This problem is indeed not a major issue, but other fixes in the plugin are worth improving.

Probably easier just to attach the file here. Oddly, I forgot to remove my default selections when I ran CCPlus 0.5.6. When I cleared all selections, the spaces weren't removed unlike with 0.5.5. The options that I've found to trigger the issue were Remove ad hoc black text color declarations from html and CSS & Remove ad hoc white bg text color declarations from html and CSS. When I ran CCPlus with the options shown in the attached images, it looked good so it appears to be those two color options that cause my issues as a second test with everything selected on the 2nd page of the options also did not cause the missing space issue.

[BeckyEbook]

Deleted file.

[DNSB]

Originally Posted by BeckyEbook

New test version (0.5.7).

Looks good. Those two options no longer trigger the space removal on my test ePub.

I ran the ePub that originally triggered the issue for me through CCPlus and it also looked good. I still manually removed the <i> tags that surrounded the spaces because they were annoying.

[slowsmile]

@BeckyEbook . . . Although your effort creating a new version of the plugin is appreciated, having two different downloadable plugin versions on the same thread could cause some version confusion as well. After all, the main release(and all plugin updates) for all plugins are always on the first post aren't they? And, currently, my plugin version is at v0.5.5. But now you have posted a new and different plugin version as v0.5.7 elsewhere in this thread -- which also appears to be considerably smaller in file size than my version -- in your post.

So to avoid any plugin version confusion, could you please release your version of the plugin, with your changes, as a separate and new release and under a different plugin name. And after you have released your new plugin version on MR, could you also remove your downloadable version of the plugin from your post above. Hopefully, doing that will help to avoid any future version confusion on this thread. In case your wondering why I'm asking you to release your new version under your own name it's because I don't know what changes you have made to the plugin code.

As to me not fixing @DNSB's problem, I still think that a "clean-up" plugin has no business turning itself into a "fix-bad-code" plugin. That's really why I would prefer keeping my plugin version separate from your plugin version. And don't worry, I'm not upset or anything. I'm just a little worried that there are now two different, downloadable release versions of the plugin by two different people on the same thread. That should never happen. All advertised releases, updates and downloads should always be in the first post to avoid any version confusion.

So whether you decide to use my cleanup plugin in a different way with a new release or not, I would really like you to remove your plugin download link from your post above as quickly as possible to help avoid any future version confusion in this thread. Thanx.

[BeckyEbook]

Sure.
I am not part of the target group for this plugin, so I prefer not to publish it in a renamed version. I simply wanted to check why the tagged spaces were being removed.
I took it purely as a challenge.

[slowsmile]

@BeckyeBook . . . My thanks to you for deleting the file so quickly. And I do understand your urge to help solve @DNSB's <span> problem. But for me I regarded the problem as self-inflicted bad code probably caused by using a poor Epub converter. And, as I keep saying, this plugin is for "clean-up". It isn't a "fix-it" plugin.

[nabsltd]

Originally Posted by slowsmile

And I've already said in several previous posts that surrounding blank text spaces with those tags is wrong

You keep saying that like it's true, but there are valid reasons to wrap whitespace (and only whitespace) in an HTML tag. They are definitely edge cases, but they do exist.

An example would be creating an empty checkbox by using a border around a space. Since most EPUB renderers do not support form elements, you can't use an actual HTML checkbox. Perhaps this would be better done by using a custom font with a checkbox character, but that doesn't mean that using a border around a space is wrong...just unusual.

[slowsmile]

@nabsltd . . , Alright, you've convinced me -- with sincere apologies to @BeckyeBook. I'll implement and fix that problem with another plugin update as quickly as I can. After fixing the selectable "Remove empty spans" option, the plugin user will be able to select that option, which will act to remove or delete all instances of <span> tags that contain no text in the html. I'll get that change done hopefully by tomorrow.

My thanks to both @BeckyeBook and @nabsltd.

[DNSB]

Thank you. As you said, it's lousy code but in my experience, a lot of epub creators would not recognize good coding practices if they bit them on the butt.

[slowsmile]

@DNSB,
@BeckyeBook,
@nabsltd,

I've put in all @BeckyBook's fixes and I've just tested the new fixed version of the plugin. I just want to clarify and confirm how the new plugin version behaves now.

When I run the new plugin with the "Remove empty spans" option selected with this epub code:

<p><span>This is a plugin test run</span></p>

The html is changed to this -- all empty spans(with no attributes) are removed:

<p>This is a plugin test run</p>

Also, if I run the new plugin with no options set and run it on this code:

<p>This <span> </span>is a plugin test <span class="P1"> </span>run</p>

I get this as a result:

<p>This is a plugin test run</p>

I can also confirm that when I ran the same tests using @BeckyeBook's plugin version -- I got the same results.

Is everyone now happy with the new plugin's behaviour? I will be releasing the plugin tomorrow morning, so please let me know your thoughts before then.

[BeckyEbook]

For me, this is OK.