Sigil-0.9.13 Released

[BetterRed]

Originally Posted by DiapDealer

Can you exclude Sigil's program directory and the %localappdata%\sigil-ebook folder from all anti-virus, anti-malware scanners?

@icallaci - and is there any chance that an anti-whatever scanner, or an always-on backup service (OneDrive, Dropbox etc) is accessing the directory in which Sigil puts its working files - this can be set in Preferences->General->Set Folder where temporary files should be created. By default they are in %APPDATA%\Local\Temp\Sigil-Xxxxxx - where Xxxxxx is random.

BR

[icallaci]

Excluding Sigil and its program and working directories from the antivirus made no difference. There are no cloud services at all installed. I have two other computers in the house, so I tried those as well. One is a Windows 8.1 laptop and the other is a Windows 7 desktop (both with a different antivirus installed). Both computers show the same behavior, with increasing memory usage and slowdowns. I am beginning to suspect that either I use Sigil for longer sessions than most users, or else something about the regexes I run are using up system resources. At this point, if no one else is having trouble, I am willing to concede that it is something I am doing. Also, it is not really a problem for me to close Sigil and reopen it, because that sets everything back to normal and I can continue working without problems for a while. Thanks for looking into this. If I can help in some other way, I'll be glad to try; otherwise, go have yourself a beer and stop worrying about this.

[KevinH]

Originally Posted by icallaci

. I am beginning to suspect that either I use Sigil for longer sessions than most users, or else something about the regexes I run are using up system resources.

It could be a regex related search and replace that has a memory leak we simply have not touched on. For a leak to show up in Valgrind or macOS "leaks tool" you have to actually exercise the problem code to detect a leak.

So it would really help us if you could please post a sample epub and a list of regex expressions you run typically and the order. That way we can try the exact samething you are doing in a leak detector to see if memory really is leaking. Please try to include an exact set of steps that seems to use the most memory.

If you prefer, you can pm DiapDealer and KevinH here with the link to the epub and list of regexes.

I really would like to get this tracked down and fixed.

Thanks,

KevinH

[KevinH]

Also can you please confirm this same issue does NOT happen with Sigil 0.9.10 or Sigil 0.9.9?

Or could it have existed all along but only now you can recreate it more easily?

That info would help narrow things down, as nothing in the Search and Replace code has changed since those releases, just Python (which is not used here) and Qt have had significant code changes.

Thanks!

KevinH

[icallaci]

I'm 99% sure this didn't happen with 0.9.10 or 0.9.9 because it's only been an obvious problem with the latest releases. I haven't changed my workflow in ages, but it's possible that there was a problem that just didn't surface as quickly with the earlier releases. If and when I get a block of time, I'll uninstall 0.9.13 and go back to 0.9.10 to see what happens. That may take me a few days to get to, though. In the meantime, I'll post my regexes so you can take a look at them.

Originally Posted by KevinH

Also can you please confirm this same issue does NOT happen with Sigil 0.9.10 or Sigil 0.9.9?

Or could it have existed all along but only now you can recreate it more easily?

That info would help narrow things down, as nothing in the Search and Replace code has changed since those releases, just Python (which is not used here) and Qt have had significant code changes.

Thanks!

KevinH

[icallaci]

I've attached an .rtf with all the regexes I run, in order (and notes to myself about what each regex is supposed to do). It's quite tedious, lol, but I'd rather fix things up front than be annoyed by problems in the middle of reading. The memory leak shows up most quickly in non-fiction books with lots of formatting, like footnotes, index entries, bibliographies with a href links, etc. The more complicated the better.

Originally Posted by KevinH

It could be a regex related search and replace that has a memory leak we simply have not touched on. For a leak to show up in Valgrind or macOS "leaks tool" you have to actually exercise the problem code to detect a leak.

So it would really help us if you could please post a sample epub and a list of regex expressions you run typically and the order. That way we can try the exact samething you are doing in a leak detector to see if memory really is leaking. Please try to include an exact set of steps that seems to use the most memory.

If you prefer, you can pm DiapDealer and KevinH here with the link to the epub and list of regexes.

I really would like to get this tracked down and fixed.

Thanks,

KevinH

[KevinH]

Great! Thank you! I will run full memory leak detection using these.

Also, can anyone point me to an epub like the one described by icallaci above in our own MR library or somewhere in the public domain to test with?

Thanks

The memory leak shows up most quickly in non-fiction books with lots of formatting, like footnotes, index entries, bibliographies with a href links, etc. The more complicated the better.

[icallaci]

I'm sorry I don't have an epub for you to run your tests against. Once I fix them, I delete the originals.

Originally Posted by KevinH

Great! Thank you! I will run full memory leak detection using these.

Also, can anyone point me to an epub like the one described by icallaci above in our own MR library or somewhere in the public domain to test with?

Thanks

[Tex2002ans]

Originally Posted by icallaci

I'm still experiencing major slowdowns

Is anyone else getting a really slow-to-finally-popup Spellcheck List?

Compared to Sigil 0.9.10, running Tools > Spellcheck > Spellcheck is noticeably slower. (Same EPUB ~<1 second in 0.9.10 to 6+ seconds in 0.9.13).

Windows 10, 64-bit.
Sigil 0.9.13 64-bit.

I rolled back my Sigil install + sigil.ini to 0.9.10 when the .12 trouble happened, so this was a straight 0.9.10->0.9.13 upgrade.

Note: I'll be sure to test on more books... this was just using the latest EPUB I've been working on for a few months, which is why I found it very noticeable.

[theducks]

Funny you should ask.
I noticed almost every GUI state change seems a bit sluggish (Open a new tab, a dialog open...). The rest of the task runs at speed.
W10 x64 on a Intel Core Duo w/8G RAM (certified to NOT to be a speed demon )

[DiapDealer]

Even if you're using what you believe to be "clean" preferences, please back them up and make sure you can reproduce your symptoms with freshly generated ones.

Also make sure you're using the very latest driver available for your video card.

[BetterRed]

FWIW - I installed Windows 64 bit 9.13 over 9.9 early this morning, been using it on and off all day in one to two hour sessions. If anything, my impression is that it's snappier, including - even especially - the spell checker, and reports

Thanks for the extra Open With slots.

Added, up-to-date Windows Pro 64 bit version 1809, Dell XPS 1890, i7-7700 CPU, 16Gb RAM, NVIDIA GeForce GTX 1060 6GB, with driver version 23.21.13.9125

BR

[icallaci]

KevinH, I was going to PM you an epub that might help, but it appears I can't attach a file to a PM, and I don't think it matters what book you use anyway. Also, after thinking about this during the night (when I SHOULD have been asleep, lol), the problem may not be related to regexes at all. I can actually cause this problem to happen just by opening and closing the Preview window, or doing a spellcheck, or opening and closing individual .html files in the epub. It takes quite a bit longer for the slowdown to occur, but if I watch the memory in Task Manager, it climbs steadily. Honestly, if you can't track this down, it's not a problem for me to simply close Sigil and reopen it, especially since most people are not experiencing this problem. I must be special

[DiapDealer]

My big concern is that it seems to be happening for you on multiple machines (with different OSes). 0.9.13 is actually snappier for me on both Win7 and Win10 than previous versions. Url-filled epubs with many, images and many xhtml files (all open simultaneously) rarely hits the 270mb mark of memory usage. The 32-bit version often runs at under 100Mb for me with "average" epubs.

[KevinH]

Also, please understand the memory footprint will increase and decrease at times as it builds caches and things. With large books I can get the memory footprint to approach 500 meg but never the 1 gig and over that you are seeing.

That said, the issue is not the memory footprint, it is that this memory growth actually impacts your performance of the app in that is causes massive slowdowns and crashes.

Does this Sigil-0.9.13 still cause huge slowdowns and crashes that you reported earlier? Or is it just that the memory footprint seems to grow and not shrink for you.

Thanks,

KevinH