2.3.1: Accented character encoding automatically updated?

[Mister L]

Edit: Oops, I checked the update log for 2.3.0 and I think this is probably the answer to my question:
"we are now using NFC unicode normalization form for all content, links, urls, and file paths." Sorry, I didn't make the connection when I read it the first time. Please delete this thread.


Thanks for the latest version, it looks great.

Spoiler:

This is just a request for confirmation.

TLDR:
Questions:
1. does Sigil automatically update all accented characters (çéàùïöÔôÂâ etc.) to proper unicode characters now regardless of source encoding? On open, as well as on paste?

2. If Sigil catches them now on its own I don't have to worry about it, but if some of them can still sneak through, is it possible to restore my backup .ini (with the old encoding) of my saved searches? (how?)

I usually do those searches routinely on all files just in case and don't bother to check the reports, I just want to know if I should start checking again for anything in future.




Context:

Some old Mac / InDesign files add accents as a separate character:
è = e + `

In Notepad++ using the arrow keys to move the cursor it takes 2 clicks to pass the double characters and they used to be listed in the Characters Used report in Sigil. They can (occasionally) display wrong (accent next to the letter instead of above it) so I have saved searches to replace them all with correct unicode characters.

I've just noticed that in Sigil 2.3.1 the ini file has been modified:

111\Name=bases/suppl\xe9mentaires/accents en 2 cars/E aigu 2c
111\Find=E\x301
111\Replace=\xc9

Instead of this:

115\Name=bases/supplémentaires/accents en 2 cars/E aigu 2c
115\Find=É
115\Replace=É

(Find is 2 characters, pretty sure that is not preserved here)

Now those searches return ALL accented characters (including correct unicode ones).

I pasted some of the double characters from Notepad++ into Sigil and only the correct unicode character was listed in the report, but if I copy just the floating accent it will display.

Code view

Report


So it seems like those characters are now automatigally updated when they are pasted into code view, but I want to be sure so I don't accidentally miss any.

Thanks for any information you can give.

[KevinH]

There is an environment variable that disables the unicode C normalization where the spec allows it. It makes no sense to me to use it but ....

Check out the latest Sigil Users Guide for a table of environment vars if interested.

[Mister L]

Originally Posted by KevinH

There is an environment variable that disables the unicode C normalization where the spec allows it. It makes no sense to me to use it but ....

Check out the latest Sigil Users Guide for a table of environment vars if interested.

Thanks Kevin. I don't want to disable the normalisation since it saves me some work, this is a great feature. I'll probably wait a bit before I retire my saved searches to be sure nothing random is sneaking past.

I will take a look at the variables table, thanks for that info.


One other question, my default window size is not retained, any idea why that might happen? When I open Sigil the main window is in the right place but it's too wide on the right side, regardless of the size / position of the window on the last close.

[KevinH]

What OS? What Qt version? (see the About Sigil menu).
Does this happen only with Sigil 2.3.1 or can it happen with earlier versions as well?

When we moved our release builds to Qt6 only we removed a bunch of cruft that was needed for remembering sizes for Qt5 only but that was fixed in Qt 6.7.x series. Perhaps some change there is impacting things?

Are you using 2 or more monitors? Are it (they) using fractional dpi scaling by chance?

[Mister L]

Originally Posted by KevinH

What OS? What Qt version? (see the About Sigil menu).

Windows 10, Sigil 2.3.1, QT 6.7.2. Let me know if you need anything more precise about the windows version.

Originally Posted by KevinH

Does this happen only with Sigil 2.3.1 or can it happen with earlier versions as well?

It didn't happen until I updated to 2.3.1 but I just reinstalled 2.2.1 / Qt 6.6.2, launched Sigil, fixed the window size, opened a file, saved, closed, then re-launched and the window size was not saved. Time stamp says the .ini file is updated each time.

Then I tried with Sigil 1.9.30 / Qt 5.12.9 and the window position / size was properly saved.

Sigil v1 uses a different .ini file to the v2 Sigils: sigil.ini (Sigil 1) and sigil_v6.ini (Sigil v2). I copied the "mainwindow" geometry from the (correct) sigil.ini file into the sigil_v6.ini to test but it didn't change anything.

Originally Posted by KevinH

When we moved our release builds to Qt6 only we removed a bunch of cruft that was needed for remembering sizes for Qt5 only but that was fixed in Qt 6.7.x series. Perhaps some change there is impacting things?

I imagine in that case other people would see the same thing? Otherwise probably something weird in my .ini?

Originally Posted by KevinH

Are you using 2 or more monitors? Are it (they) using fractional dpi scaling by chance?

One monitor, 1920x1080 resolution, 100% scale.

Not sure this is useful at all but in case it helps, here is the mainwindow geometry from the sigil_v6.ini file (used by Sigil v2):

Spoiler:

[mainwindow]
maximized=false
fullscreen=false
geometry=@ByteArray(\x1\xd9\xd0\xcb\0\x3\0\0\0\0\x 2\x5\0\0\0\0\0\0\x5\xfd\0\0\x4\xe\0\0\x2\x5\0\0\0\ x1f\0\0\x5\xfd\0\0\x4\xe\0\0\0\0\0\0\0\0\a\x80\0\0 \x2\x5\0\0\0\x1f\0\0\x5\xfd\0\0\x4\xe)
toolbars=@ByteArray(\0\0\0\xff\0\0\0\0\xfd\0\0\0\x 3\0\0\0\0\0\0\0\x8c\0\0\x3\x1f\xfc\x2\0\0\0\x2\xfb \0\0\0\x16\0\x62\0o\0o\0k\0\x62\0r\0o\0w\0s\0\x65\ 0r\x1\0\0\0\xb5\0\0\x2\x10\0\0\0X\0\xff\xff\xff\xf b\0\0\0\x16\0\x63\0l\0i\0p\0s\0w\0i\0n\0\x64\0o\0w \x1\0\0\x2\xcb\0\0\x1\t\0\0\0X\0\xff\xff\xff\0\0\0 \x1\0\0\0\x82\0\0\x3\x1f\xfc\x2\0\0\0\x2\xfb\0\0\0 \x1a\0p\0r\0\x65\0v\0i\0\x65\0w\0w\0i\0n\0\x64\0o\ 0w\x3\0\0\x5\xa1\0\0\x1\x9e\0\0\x1\xdf\0\0\x2s\xfb \0\0\0\x1e\0t\0\x61\0\x62\0l\0\x65\0o\0\x66\0\x63\ 0o\0n\0t\0\x65\0n\0t\0s\x1\0\0\0\xb5\0\0\x3\x1f\0\ 0\0X\0\xff\xff\xff\0\0\0\x3\0\0\x3\x93\0\0\0\xeb\x fc\x1\0\0\0\x1\xfb\0\0\0*\0v\0\x61\0l\0i\0\x64\0\x 61\0t\0i\0o\0n\0r\0\x65\0s\0u\0l\0t\0s\0n\0\x61\0m \0\x65\0\0\0\0\0\0\0\x3\x93\0\0\0\x46\0\xff\xff\xf f\0\0\x2\xdf\0\0\x3\x1f\0\0\0\x4\0\0\0\x4\0\0\0\b\ 0\0\0\b\xfc\0\0\0\x5\0\0\0\x2\0\0\0\a\0\0\0$\0t\0o \0o\0l\0\x42\0\x61\0r\0\x46\0i\0l\0\x65\0\x41\0\x6 3\0t\0i\0o\0n\0s\x1\0\0\0\0\xff\xff\xff\xff\0\0\0\ 0\0\0\0\0\0\0\0 \0t\0o\0o\0l\0\x42\0\x61\0r\0T\0\x65\0x\0t\0M\0\x6 1\0n\0i\0p\x1\0\0\0Z\0\0\0:\0\0\0\0\0\0\0\0\0\0\0\ x16\0t\0o\0o\0l\0\x42\0\x61\0r\0\x42\0\x61\0\x63\0 k\x1\0\0\0\x94\xff\xff\xff\xff\0\0\0\0\0\0\0\0\0\0 \0\x1a\0t\0o\0o\0l\0\x42\0\x61\0r\0\x44\0o\0n\0\x6 1\0t\0\x65\x1\0\0\0\xee\xff\xff\xff\xff\0\0\0\0\0\ 0\0\0\0\0\0\"\0t\0o\0o\0l\0\x42\0\x61\0r\0I\0n\0s\ 0\x65\0r\0t\0i\0o\0n\0s\x1\0\0\x1#\xff\xff\xff\xff \0\0\0\0\0\0\0\0\0\0\0\x18\0t\0o\0o\0l\0\x42\0\x61 \0r\0T\0o\0o\0l\0s\x1\0\0\x1\xf3\xff\xff\xff\xff\0 \0\0\0\0\0\0\0\0\0\0\x1c\0t\0o\0o\0l\0\x42\0\x61\0 r\0P\0l\0u\0g\0i\0n\0s\x1\0\0\x2\x97\0\0\x1\xb2\0\ 0\0\0\0\0\0\0\0\0\0\x2\0\0\0\b\0\0\0\x1e\0t\0o\0o\ 0l\0\x42\0\x61\0r\0H\0\x65\0\x61\0\x64\0i\0n\0g\0s \x1\0\0\0\0\xff\xff\xff\xff\0\0\0\0\0\0\0\0\0\0\0$ \0t\0o\0o\0l\0\x42\0\x61\0r\0T\0\x65\0x\0t\0\x46\0 o\0r\0m\0\x61\0t\0s\x1\0\0\0\x35\xff\xff\xff\xff\0 \0\0\0\0\0\0\0\0\0\0 \0t\0o\0o\0l\0\x42\0\x61\0r\0T\0\x65\0x\0t\0\x41\0 l\0i\0g\0n\x1\0\0\x1#\xff\xff\xff\xff\0\0\0\0\0\0\ 0\0\0\0\0\x18\0t\0o\0o\0l\0\x42\0\x61\0r\0L\0i\0s\ 0t\0s\x1\0\0\x1\xc7\xff\xff\xff\xff\0\0\0\0\0\0\0\ 0\0\0\0\x1c\0t\0o\0o\0l\0\x42\0\x61\0r\0I\0n\0\x64 \0\x65\0n\0t\0s\x1\0\0\x2!\xff\xff\xff\xff\0\0\0\0 \0\0\0\0\0\0\0\"\0t\0o\0o\0l\0\x42\0\x61\0r\0\x43\ 0h\0\x61\0n\0g\0\x65\0\x43\0\x61\0s\0\x65\x1\0\0\x 2{\xff\xff\xff\xff\0\0\0\0\0\0\0\0\0\0\0(\0t\0o\0o \0l\0\x42\0\x61\0r\0T\0\x65\0x\0t\0\x44\0i\0r\0\x6 5\0\x63\0t\0i\0o\0n\0\0\0\0\0\xff\xff\xff\xff\0\0\ 0\0\0\0\0\0\0\0\0\x1e\0t\0o\0o\0l\0\x42\0\x61\0r\0 P\0l\0u\0g\0i\0n\0s\0\x32\x1\0\0\x2\xb0\0\0\0\xca\ 0\0\0\0\0\0\0\0\0\0\0\x2\0\0\0\x1\0\0\0\x18\0t\0o\ 0o\0l\0\x42\0\x61\0r\0\x43\0l\0i\0p\0s\x1\0\0\0\0\ xff\xff\xff\xff\0\0\0\0\0\0\0\0\0\0\0\x2\0\0\0\0\0 \0\0\x2\0\0\0\t\0\0\0\"\0t\0o\0o\0l\0\x42\0\x61\0r \0N\0\x65\0w\0\x41\0\x63\0t\0i\0o\0n\0s\x1\0\0\0\0 \xff\xff\xff\xff\0\0\0\0\0\0\0\0\0\0\0$\0t\0o\0o\0 l\0\x42\0\x61\0r\0\x41\0\x64\0\x64\0\x45\0x\0i\0s\ 0t\0i\0n\0g\x1\0\0\0Z\xff\xff\xff\xff\0\0\0\0\0\0\ 0\0\0\0\0$\0t\0o\0o\0l\0\x42\0\x61\0r\0R\0\x65\0p\ 0o\0\x41\0\x63\0t\0i\0o\0n\0s\x1\0\0\0\x8f\xff\xff \xff\xff\0\0\0\0\0\0\0\0\0\0\0&\0t\0o\0o\0l\0\x42\ 0\x61\0r\0I\0n\0\x64\0\x65\0x\0\x41\0\x63\0t\0i\0o \0n\0s\x1\0\0\x1\x33\xff\xff\xff\xff\0\0\0\0\0\0\0 \0\0\0\0\x1e\0t\0o\0o\0l\0\x42\0\x61\0r\0U\0n\0\x6 4\0o\0R\0\x65\0\x64\0o\x1\0\0\x1\xd7\xff\xff\xff\x ff\0\0\0\0\0\0\0\0\0\0\0\x1c\0t\0o\0o\0l\0\x42\0\x 61\0r\0X\0\x45\0\x64\0i\0t\0o\0r\x1\0\0\x2\x31\xff \xff\xff\xff\0\0\0\0\0\0\0\0\0\0\0\x16\0t\0o\0o\0l \0\x42\0\x61\0r\0\x46\0i\0n\0\x64\x1\0\0\x2\x66\xf f\xff\xff\xff\0\0\0\0\0\0\0\0\0\0\0\x1a\0t\0o\0o\0 l\0\x42\0\x61\0r\0\x43\0l\0i\0p\0s\0\x32\x1\0\0\x2 \xc0\xff\xff\xff\xff\0\0\0\0\0\0\0\0\0\0\0\x1e\0t\ 0o\0o\0l\0\x42\0\x61\0r\0\x41\0u\0t\0o\0m\0\x61\0t \0\x65\x1\0\0\x2\xde\xff\xff\xff\xff\0\0\0\0\0\0\0 \0)

[DiapDealer]

I've never had any problems with mainwindow retaining it's size on Windows 10. I also wouldn't recommend copying data directly from Sigil.ini to Sigil_v6.ini. I'm suspecting that parts of your .ini may be corrupted.

[Mister L]

Originally Posted by DiapDealer

I've never had any problems with mainwindow retaining it's size on Windows 10. I also wouldn't recommend copying data directly from Sigil.ini to Sigil_v6.ini. I'm suspecting that parts of your .ini may be corrupted.

That seems plausible. How can I fix it? Can I delete the sigil_v6.ini and let it regenerate a new .ini file or will that cause more problems?

[DiapDealer]

Originally Posted by Mister L

That seems plausible. How can I fix it? Can I delete the sigil_v6.ini and let it regenerate a new .ini file or will that cause more problems?

That would be the easiest and safest way, actually. You can back up your existing _v6.ini if you like. If regenerating a new ini fixes things (and I'm betting it will), I'd try reconfiguring things to you liking rather than copying stuff from the backup _v6.ini (there's probably some things you could safely copy, but I certainly wouldn't copy any old geometries over). Cross that bridge after determining if deleting the _v6.ini and letting Sigil recreate it fixes the window size restore issue.

Also keep in mind that when multiple instance of Sigil are running, the last one to close is the geometry that get's saved. So make sure the last instance is set to to the size you want before closing.

[Mister L]

Originally Posted by DiapDealer

That would be the easiest and safest way, actually. You can back up your existing _v6.ini if you like. If regenerating a new ini fixes things (and I'm betting it will), I'd try reconfiguring things to you liking rather than copying stuff from the backup _v6.ini (there's probably some things you could safely copy, but I certainly wouldn't copy any old geometries over). Cross that bridge after determining if deleting the _v6.ini and letting Sigil recreate it fixes the window size restore issue.

Hm, it didn't work. :/ I changed the original _v6.ini to _v6.ini.old then opened Sigil, a new _v6.ini was created, I fixed the window, opened a file, saved it, closed Sigil, opened it again and the window is too large again. It's not the end of the world but it means the preview window is covering part of the ToC so I have to resize it each time.

Originally Posted by DiapDealer

Also keep in mind that when multiple instance of Sigil are running, the last one to close is the geometry that get's saved. So make sure the last instance is set to to the size you want before closing.

Oh, I know. I very frequently have 2 epubs open at the same time in different positions and if I accidentally close the "right" one first I relaunch another instance of Sigil before I close the "wrong" one.

[DiapDealer]

Originally Posted by Mister L

Hm, it didn't work. :/ I changed the original _v6.ini to _v6.ini.old then opened Sigil, a new _v6.ini was created, I fixed the window, opened a file, saved it, closed Sigil, opened it again and the window is too large again. It's not the end of the world but it means the preview window is covering part of the ToC so I have to resize it each time.

What is actually changing size? Is the MainWindow changing, or is one dockwidget (in a stacked configuration with another dockwidget) claiming a little more vertical space within the main Window each time Sigil is launched? If it's the latter, are you running Sigil maximized? The info you posted earlier indicated that you weren't running maximized. But if you are, there was an issue beginning with Sigil 2.3 that could make this happen. It's still there to some extent in Sigil 2.3.1 on Windows. Run "almost-but-not-quite" maximized and see if that stops the problem. We'll keep looking into it.

[Mister L]

Originally Posted by DiapDealer

What is actually changing size? Is the MainWindow changing, or is one dockwidget (in a stacked configuration with another dockwidget) claiming a little more vertical space within the main Window each time Sigil is launched? If it's the latter, are you running Sigil maximized? The info you posted earlier indicated that you weren't running maximized. But if you are, there was an issue beginning with Sigil 2.3 that could make this happen. It's still there to some extent in Sigil 2.3.1 on Windows. Run "almost-but-not-quite" maximized and see if that stops the problem. We'll keep looking into it.

It's the main window which is changing size (with the toolbars, code view, etc. docked), but the width, not the height. I have the preview open in a smaller window to the right of the main window and instead of opening with the two windows lined up next to each other, the small preview window overlaps the main window.

Here are some screenshots:
How I want it

How it is when I open it


I tried reducing the height a little bit and then opening a file / saving / closing / relaunching; the new height was saved but still not the width.

[DiapDealer]

Best guess is that you're shrinking one element below it's (current Qt) recommended width. Qt tries to juggle all of these things but isn't always successful at restoring everything exactly.

Try temporarily closing the F&R Window, and see if that stops the growth.
Or the Clips bar.

Everything looks to be staying the same size to me except for the central widget (which in turn makes MainWindow grow).

[BeckyEbook]

The main window (in my case, Windows 10) has a minimum required width of around 955-960px with the F&R window open. If you shrink the window below this value then the next time you open Sigil it is set to this minimum width.

On the other hand, it's hard to see such a situation as a bug, as Qt simply tries to fit all widgets/controls on the screen.

[Mister L]

Ok, it is indeed a question of minimum window size apparently. The size I want is approximately 927px, the size I get is 1014 pixels (according to my screen cap).

If I close the F&R + several toolbars + the ToC I still get 1014.


If I leave it at 1014 but move it to the left (so the preview is not blocking it) I can keep all the usual toolbars + ToC and F&R and the size / position is restored on open.

Means I have a bit less space for reference documents to the left of Sigil but I will learn to live with it, I guess.

Thanks very much to both of you for helping me figure it out.