Sigil-2.2.0 Released

[BetterRed]

Originally Posted by DiapDealer

. . .

EDIT: I just noticed you said "every place".

As I recall the qt-style line puts a border on buttons etc, which I really like. Wish it was as easy in every application, it's not even doable in most.

BR

[repilo]

Originally Posted by KevinH

Again this is not a bug. The minimum size in Sigil 2.2.0 is now 6 pixels larger than before to accommodate focus borders.

Sorry if I seem annoying, but it appears to me that the difference from the previous version is more than 6 pixels. I also apologize if the automatic translator sounds rude or something like that.
Img 1: Sigil 2.1.0 minimum width.
Img 2: Idem when closing and reopening.
Img 3: Sigil 2.2.0 minimum width.
Img 4: Idem when closing and reopening.
I understand that there is a minimum when it comes to reducing the width, but once it allows you to choose a width to your liking and lets you work that way for hours, if it is fine during all that time, why isn't it restored to that size the next time?
Please understand that I couldn't be more grateful for all your selfless efforts. My only intention is to bring attention to a possible defect, and if it is not, I apologize.

[BetterRed]

Originally Posted by repilo

Sorry if I seem annoying, but it appears to me that the difference from the previous version is more than 6 pixels. I also apologize if the automatic translator sounds rude or something like that.
Img 1: Sigil 2.1.0 minimum width.
Img 2: Idem when closing and reopening.
Img 3: Sigil 2.2.0 minimum width.
Img 4: Idem when closing and reopening.
I understand that there is a minimum when it comes to reducing the width, but once it allows you to choose a width to your liking and lets you work that way for hours, if it is fine during all that time, why isn't it restored to that size the next time?
Please understand that I couldn't be more grateful for all your selfless efforts. My only intention is to bring attention to a possible defect, and if it is not, I apologize.

Did you read the first line of post #43.

Originally Posted by BetterRed

↑ ↑ ↑ ---- Looks like it expanded to accommodate another clip bar item 'saltoescena'.

When I flick back and forth through the recently posted images, I see once again that the resizing appears to be related to accommodating (making room, creating space) for clip item labels.

BR

[DiapDealer]

The 6 pixel size difference just means that your preferred working width falls probably below that minimum. Not that the difference in width after reopening Sigil is 6 pixels. The 6 pixels must be enough for the clip bar to believe it needs to display just one more item. And that just one more item is what's visibly expanding Sigil's width to noticeably overlap the other application. Change the UI font size (or the font itself) or shorten one of your clips descriptions, and I'll bet you'll find a combination that works to accommodate your preferred working width.

It may be annoying to you, but it's not a Sigil bug or a defect. It's just that your preferred width happens to have slightly different consequences when it comes to restoring/saving Sigil's widget geometry with the new 6 pixel difference in Sigil's overall width.

The number of clips able to be assigned to each clip bar was also increased in Sigil 2.2.0. The new space available might be hinting to Qt that "just one more clip" is worth expanding ever so slightly to accommodate. Temporarily hide your clips bar(s) and see if your preferred width is maintained. Or remove one of your lesser used clips. Or shorten a clips description. Or lengthen one.

[KevinH]

Again, this is a change but NOT a bug. There is no guaranteed way have Sigil exactly restore itself and its dockwidgets back to what you had, once you have shrunk Sigil past its preferred size. It will require manual intervention. This happens with the previous version of Sigil as well, it is just the newer version needs 6 more pixels of horizontal space and allows more clip icons per clip bar.

Have you actually tried any of the ways mentioned previously to help conserve space? ie using the Sigil View menu to hide Table of Contents and Preview, hide Find and Replace, turn off one of the clips bars, or in Preferences change ui font or size, change icon size change, or change clips text, move clips to the second tool bar, etc.

[towhit523]

Originally Posted by Doitsu

I also tested this with Sigil 0.9.8 and it actually renamed the id and idref items in the .opf file. The OP must have previously used a rather old Sigil version.

I also noticed the change in behavior with the ids in the opf and I am not using a very old version. The ids are renamed in version 2.0.0. I have that version running on my mac and specifically check the version since I noticed the change after I upgraded the version on my pc to the latest. I know it doesn't matter but I do like for them to match so I will keep using the 2.0.0 until I have to upgrade for another reason.

[KevinH]

Originally Posted by towhit523

I also noticed the change in behavior with the ids in the opf and I am not using a very old version. The ids are renamed in version 2.0.0. I have that version running on my mac and specifically check the version since I noticed the change after I upgraded the version on my pc to the latest. I know it doesn't matter but I do like for them to match so I will keep using the 2.0.0 until I have to upgrade for another reason.

I actually don't remember exactly when that change happened. I believe it was when we moved to BulkRenameResources to deal with bugs related to how slow things got when working on epubs with thousands of files. Changing the id when bulk renaming really slowed things down. So whenever that bug fix happened is when that change occurred.

I will look to see if there is a faster way for Bulk Rename to handle changing the id as well. Or, more likely, stop a single file rename in BookBrowser from using the Bulk Rename routine and instead use the old slower rename approach.

No promises but I will at least take a look.

[KevinH]

Upon examining the code further, the other reason we moved to always using the BulkRename routine was actually correctness (now that I can read the code). The old version of Rename used to assume that manifest ids are only used in the spine, when in fact under epub3 they can be used in manifest fallback ids for non-primary resource types, and in media-overlay ids for smil files as well as some external namespace metadata types can refer to manifest ids as well.

So the old routine by changing the unique id of a file in the manifest and its idref in the spine could end up breaking smil media-overlays and fallback resource manifest ids.

It is just safer to not change the manifest ids when renaming the file itself as it is technically not needed and really is only a "cosmetic" change seen only by the epub developer, never the end user, that can really break things if you are not careful.

To change just one manifest id, would require walking the entire spine and then walking the entire manifest itself looking at fallback ids, media-overlay ids, and then walking all the metadata and trying to see if any reference that specific manifest id.

When dealing with thousands of files being auto renamed/renumbered, the routine would be end up being quadratic in time parsing and reparsing the opf to fix each tag thousands of times in a loop.

I am not sure going back to the old behaviour makes much sense, even just for single renames.

The old routine was passable for old epub 2 but under epub3 it was both a performance and correctness bug waiting to happen.

Correctness and faster performance are the reasons I moved to the Bulk Resource Rename approach. Adding back an incorrect approach, is probably not the best idea.

[snarkophilus]

Minor formatting glitch in the new features section of the first post?

Originally Posted by DiapDealer

Sigil-2.2.0

New Features in this release

• add new environment variable: SIGIL_PREVIEW_TIMEOUT (in milliseconds) to allow user control of the Preview updating intervals, valid range 100
• 10000. default value: 1000. Set only once at Sigil startup.

That should be ".. range 100 - 10000. Default value: 1000. Set ..." ?

Thanks always to you both for the another Sigil release!

[DiapDealer]

Originally Posted by snarkophilus

Minor formatting glitch in the new features section of the first post?



That should be ".. range 100 - 10000. Default value: 1000. Set ..." ?

Thanks always to you both for the another Sigil release!

Good catch. Thanks! "- " is the list indicator for GitHub's markdown. So the dash got converted to Mobileread's bbcode list indicator. Fixed now.

[KevinH]

For the record ... and so I do not have to spend another set of hours restudying the epub 2 and epub3 specs:

Here are the places in the OPF where idrefs (references to manifest ids) are used (that I can find documented at least), making changing the manifest id for a simple resource file rename more complicated.

- if you are renaming any image file and *change its manifest id*, you would have to walk all of the metadata to make sure you had just not changed the old epub2 cover image idref metadata. Tying it to the manifest properties itself made much more sense (as is done in epub3).

- if you rename the NCX and *change its manifest id*, you would have to change the toc attribute value in the spine tag

- if your opf is using the "bindings" tag (not used much in the wild admittedly) then you must walk all of its mediaType children and compare its "handler" attribute to see if it is a changed idref

- you will need to walk all the manifest item tags looking at every fallback attribute value and updating it if needed ... AND look at every media-overlay attribute value and update it as needed since both are idrefs.

- and of course you will need to walk the spine itemrefs and update each idref as needed

And these are the only ones I can find documented. There may easily be others inside custom metadata and even smil files that would have to be updated.

This is why just changing the unique id of a resource just to match its current name is fraught with issues that all go away if you just leave that unique id alone.

Which is why it was designed in that way - *independent* of the original resource file name. Sigil and others using that expedient of basing it on its original name really was not a good idea when mass renaming/renumbering of files is needed.

So I will look into making BulkRename try to update all of these possible uses of idrefs in the OPF for some FUTURE RELEASE. *If* I can do that and not kill performance with epubs with large numbers of files (read that thousands) then at some point I will add back that rename id "feature" but that is really a big IF.


Hope this explains things.

[KevinH]

In case anyone wants to try their hand at a python3lib script or standalone python script to update just the opf redoing *all* manifest ids to rely on filenames, to form the basis of a new Sigil tool call it "Fixup Manifest IDs" here is where I am as a basic approach:

Approach:

1. Pass 1 - parse the OPF building up a dict of all_ids
- while parsing store away and ncx id and cover id

2. create an empty changed_id dict

3. Pass 2 - parse the opf and walk the manifest
- create a potential new id based on current file name
- if different from current id
- use the dict of all_ids to make sure the new_id it valid and unique
- create the new manifest entry, replacing the old id with the new one
- update all_ids dict to remove the old id and add in the new id
- update dict changed_id[old_id] = new_id

4. Still in Pass 2 - when you reach the spine if ncxid in change_id, parse and update the spine toc attribute

5. Still in Pass 2 - walk the spine entries updating idrefs from change_id

6. Still in Pass 2 - when you reach bindings (it it exists) walk the mediatypes entries in bindings - updating the "handler" attribute as needed

7. save the changed opf.

8. Pass 3 - parse the new opf if cover_id, walk the metadata updating any cover id meta as needed

9. Still in Pass 3 - walk the manifest entries updating the "media-overlay" and "fallback" attribute values from change_id

10. Save the final version of the OPF as it stands

Of course, if you parse to store more state in a changeable format then you can reduce the total number of passes at the expense of more data structures.

And FWIW - I think using the quickparser.py tool on the opf will do what is needed here in 3 passes without any additional state needed at all.

[KevinH]

Okay just in case anyone is interested in a standalone python opf id updater that they can play with and tweak.

I have written one that uses modified version of an earlier opf parser I had lying around and some valid id generation code I borrowed from Sigil and converted to python and a pure python third party module called unidecode (to ascify file names) to make them suitable for id values.

I have put all of this in a zip and it is free for anyone to use in their own plugin or anyplace else.

It follows the rough approach of my previous post, should work for epub2 and epub3.

And since it uses a stateful opf parser, it can function in one pass.

I have attached it. Feel free to tweak it to make it suit your needs.

Comments and Improvements welcome.

[KevinH]

I forgot the instructions ... In case anyone wants to try it:

unzip opf_id_updater.zip
cd opf_id_updater
python3 ./fix_opf_ids.py ./content.opf

It will spit out the updated opf to stdout.

Of course you can replace ./content.opf with the path to any opf you have lying around.
And you can redirect stdout to a file:

python3 ./fix_opf_ids.py PATH_TO_INPUT_OPF_FILE > PATH_TO_OUTPUT_OPF_FILE

I am sure there is an equivalent on Windows as well.

[KevinH]

In case people like this idea enough, we could modify it to remove the file extension, and instead of x prefaced uuids when collisions occur, we could add a 4 digit running count to the base name to make unique values. Or we could convert the file extension and append it after an underscore (after removing its preceding dot.

So Cover.jpg it could generate Cover0007 (assuming th other 6 were already used) or use "Cover_jpg" instead. Or do a combination of both.

Ideas welcome.