Pocketbook Annotations Viewer and Tools (AVATeR), need testing

[wold]

Mainly an interactive annotation viewer/exporter, with some added tools from (my) 'pocketbooktools' plugin, but these are pretty basic:

- backup DBs and files
- send system (dict, fonts) files to device / restore these from backup (except e-books)
- check database integrity
- merge/fix annotations for an e-book (in case an e-book got re-loaned)



Requirements
- Pocketbook readers, likely only firmware 6.x.
- Windows 8/10/11+ 64-bit. Windows 7 might work.
- Linux versions

Download
Downloads are available below (basic HTML page, no ads, no JS).

https://www.syncoda.nl/software/avater/
https://www.syncoda.nl/software/avat...html#downloads

Changelog

Spoiler:

0.11.1 Dec 7th 2022
- Windows: fixed crash when removing last reader without localmirror
- Windows: fixed rare error in USB related function

0.11 Oct 30th 2022
- New release packaging (Linux .deb and .rpm; Windows .zip and .exe installer)
- Fixed: after viewer update (row addition/removal), re-apply filters
- Fixed: also reload annotations when annotation count remains the same
- Fixed: only rescan unmounted devices upon device monitor changes (Linux mainly)
- Internal: Changed and extended highlight/note location data parsing (WIP)
- Hide empty devicelabel

- Linux (Fedora): output CLI configuration messages using printf instead of as debug messages
- Linux: fixed changelog location
- Linux: added manpage
- Internal: reworked "connected" check logic
- Show update nag screen every 6 months if automatic updates are disabled

0.10 Aug 31th 2022
- Added: Restore search settings for the last used device on startup (if present)
- Added: Markdown annotation export (experimental)
- Added: Show deleted annotations (and export if shown)
- Added: PB profile selection possible from top-menu
- Added: show basic annotation count (current/total)

- Changed: Store Qt5/6 column settings separately
Switching between Qt versions could induce a crash due to unknown causes (not UTF-8 related). As such you may need to re-configure your column sizes.
- Changed: After filtering on title from the context-menu, don't set the title-page sorting mode
- Changed: increased attempts for Windows USB-related loops
- Changed: Rely on devices.conf group serial ID again (Reverse compatibility is maintained)

Viewer related
- Fixed: show missing page numbers as "?"
- Fixed: missing page number for certain bookmarks
- Fixed: toggling case-sensitive sort works now for all sorting types (including custom sorting)
- Fixed: allow custom sorting on columns after having selected a sort mode
- Fixed: improved setting showing bookmarks/deleted annotations from UI and at model creation

- Fixed: Better handling of model refreshes due to device updates
- Fixed: "Reload annotations" menu-option better handles DB changes
- Fixed: Minor Qt model errors

Other
- Fixed: GUI removal for e-reader or localmirror
- Fixed: copy localmirror if enabled but none was present on device arrival
- Fixed: if localmirror was disabled remotely (via devices.conf), remove and toggle UI elements

- Fixed for Fedora: Qt UI fixes; output all debug messages
- Fixed for Windows: issue with closing uploader progressdialog
- Improved: improved profile rescans
- More, including docs...

0.9.9 Jul 22th
- Improved: HTML export (layout, colors)
- Improved: CSV export (added variables)
- Improved: set debugmode from GUI, added logfile option to CLI
- Added: Raw SQL export option

0.9.8.3 Jun 20th
- Fixed: Show creation date for annotations
- Fixed: Correct raw page numbers
- Linux fix: enforce UTF-8 for config files
- Fixed: Debug display UTF-8 issues
- Changed: write logfile to OS temporary location
- Windows: updated zlib and libzip libraries
- Stick to C++17 spec for now
- Minor changes

0.9.8 May 28th https://www.syncoda.nl/posts/2022/avater-release-0-9-8/
- New: show highlight colors
- New: sticky annotation selection option

- Improved: row resizing when scrolling
- Improved: Windows USB device detection
- Improved: Replaced USB/mount related delays with retry loops
- Improved: Local mirror handles removal via sync

- Fixed: Removed deadlock situation with disabled device monitor
- Fixed: URLs for website/updating
- Various minor changes/fixes

0.9.7 Mar 18th
- Faster annotation importing
- Faster row resizing
- Stop stretching the last column
- Various fixes/changes

0.9.6 Mar 13th
- Improved search filter responsiveness
- Added filter presets to date/page filter
- Sorting fixes + added custom sort mode indicator
- Improved saving column sizes
- Various fixes/changes

0.9.5 Mar 1st 2022 (first public release)
- fixed devicelabel issue with unmirrored device
- skip mount delay on startup
- fixed switching to localmirror after enabling

0.9.4 Feb 25th 2022
- fixed changing datadir in settings
- fixed config text encoding issue with serial keys
- improved data sharing in key points
- various fixes

0.9.3 Feb 23th 2022
- improved HTML export formatting
- added help viewer and buttons
- added debug logging options
- added checking/sanitizing of input values (backuplabel, preferences)
- various fixes

0.9.2 Jan 31th 2022
- extended MergeFix with UI dialog
- Added device counter
- various fixes

0.9.1
- Added exporter sorting choice
- Improved USB ID handling
- various fixes

0.9.0
- Improved handling of (dis)connection/disconnected devices
- Implemented archive loader (WIP)
- various fixes

0.8.0
- Improved annotation filter and export routines
- various improvements/fixes

0.7.0
- Reworked filesystem to use STD
- Added context menus
- various improvements/fixes

0.6.0
- First working Windows version with USB detection

0.5.0
- Implemented multiple reader support
- Implemented USB detection (linux)

0.4.0
- Integrated Annotation Viewer in GUI

0.3.0
- Added drive-based device selector (main only)

0.2.0
- Implemented tool-button GUI window (Linux)
- Ported Annotation Merge/Fix tool

0.1.0
- First build
- Ported backup routine and annotation HTML exporter

To install
There's an .exe installer, and a packaged archive. The archived version can run in portable mode as well (stores everything in the app dir).

Privacy
On first start, the program will ask permission to check for updates. The update checks only a version number, downloading is done by visiting the website in your browser.

---

Any comments or requests are welcome.

[wold]

Known issues
- 0.9.7: uploader final dialog always reports an operation was "cancelled", despite being successful. This warning can be ignored. Fixed in 0.9.8

Hashes for archives (SHA-1)
-No download links-

Spoiler:

v0.9.8.3 Intel/AMD 64-bit Debian .deb
.deb.gz SHA256 a0b5dc76308dd9931f959d78001304cf5b6823d6060da33bed ba38dff328217d
.deb SHA256 8f0ecff050e1fe0921157b202e776958e46d5c504804857685 b7abdb7db8cb67
COMPATIBLE .deb.gz SHA256 0327f0a06a5246b362784a125c3c2e50f7421096a407ed0ce8 b196ea157f7de7
COMPATIBLE .deb SHA256 9bcf2fcd5feb939998c5726a98950bcc7f4cb84c32b525280c 38229f037f2146


----


v0.9.8.3 (Windows Intel/AMD 64-bit):
.7z SHA256 1bbbeae3fa4e20ec43eb5cfec4cacb59ba8af0c1346eb52a60 f23f477a392c18
.zip SHA256 e563114b25ef59c3a7e7105d1ab041e1b723674fe384c9e323 85e0e61f5b1810

0.9.7 (Intel/AMD 64-bit) .7z archive (9.4MB)
- SHA256 e9c7e0a7cd3588141ae2af27684a5475b9a43a95c74f623ccd 6f440af7b72f34
- SHA-1 987c06ef963ed5762470e09713c44b44382985d0

0.9.7 (Intel/AMD 64-bit) .zip archive (13.5MB)
- SHA256 dff15dc5005ee6f062d2a32ccb6b5a7057c0c9d0d5df3ef5a0 3c207d77e51915
- SHA-1 b193f8006658a337c0d38c0f0ae9b2883dee3892

0.9.5 (Intel/AMD 64-bit) .7z archive:
- SHA256 647d1f149e02a7f8cf2befd93e13b527d655b90cbb8f0a3767 613a877b5258c8
- SHA1 816be8643b08bcd3856bfe4873ec443b30a49496

0.9.5 (Intel/AMD 64-bit) .zip archive:
- SHA256 89e2b3ef1354ccf3d9a21453e570ca1358c06ae270c80635cd 6408ac3f80f139
- SHA1 5b2c988625f160f14c8cd6445375ac8ab98d5497

[Huwaetzel]

First of all, great, this is a great initiative and the first version is very promising.
What I like:
- Interactivity. Have not fully tested yet, but this allows keep annotations "clean" at the end. Great would be, if one could also extend the own comment given, not the highlighting itself.
- The many different formats to export. Everybody is different and while I am preferring more text/markdown, others may prefer csv
- General stability

Ideas for improvement:
- I am using an InkPad Color. Keeping that color coding I am used to would be great

I am happy to test the Linux version. My systems are all based Debian or Ubuntu. Windows is only used in the few cases I can not prevent using it

[wold]

Originally Posted by Huwaetzel

First of all, great, this is a great initiative and the first version is very promising.
What I like:
- Interactivity. Have not fully tested yet, but this allows keep annotations "clean" at the end. Great would be, if one could also extend the own comment given, not the highlighting itself.
- The many different formats to export. Everybody is different and while I am preferring more text/markdown, others may prefer csv
- General stability

Thanks for testing! Great to know it runs OK elsewhere.

Editing comments is a todo. Deleting will likely be added first.
One hurdle is the local mirror setup. A solution for now is to keep the reader connected for editing, so both DBs can be updated.

A json export option was planned, but it was dropped to finish the first release. Markdown wouldn't be too complex to add.

Ideas for improvement:
- I am using an InkPad Color. Keeping that color coding I am used to would be great

Interesting. Implementing that would require the colour names that PB uses in the DB. I know about 'yellow', and blue being 'cian' [sic].
Might you be willing to check those in your DB?

I am happy to test the Linux version. My systems are all based Debian or Ubuntu. Windows is only used in the few cases I can not prevent using it

Stay tuned. A working .deb installer was made at one point.
Can you use Flathub? It removes the need for distro-specific releases.
Still, for Linux the part that monitors USB devices needs work, the main thing keeping it from release.

[Huwaetzel]

Hi, about the colors, I looked into my db and found the following dolors: green, magenta, cian, yellow, red,blue

about flathub: Yes, for the time being happy to test with a flathub package

[wold]

Originally Posted by Huwaetzel

Hi, about the colors, I looked into my db and found the following dolors: green, magenta, cian, yellow, red,blue

about flathub: Yes, for the time being happy to test with a flathub package

Thanks for the color codes (nice you managed to find them )

An update is in the works, It improves row resizing, the Windows USB monitor and local mirror rechecking.
That will need wrapping up first (these things really tend to snowball...), at 2 weeks max.
Colors & flathub are next on the list.

[wold]

Basic color support has been added in 0.9.8, with other improvements. Still Windows only...

[wold]

@Huwaetzel, I rolled a .deb installer for Debian. Can you try it?
This was the quickest option for now, re-using earlier work.

It will pull in a load of dependencies (below). it will fail if any are not present for your distro.

Depends: libc6 (>= 2.33), libgcc-s1 (>= 3.0), libqt5core5a (>= 5.15.1), libqt5gui5 (>= 5.0.2) | libqt5gui5-gles (>= 5.0.2), libqt5sql5 (>= 5.0.2), libqt5widgets5 (>= 5.15.1), libstdc++6 (>= 11), libudev1 (>= 183), libzip4 (>= 1.0)

Note the device monitor is disabled, so you'll need to F5 to find the reader.

Edit: added a qt5 version, as the previous installer was compiled against qt6...

Edit2: Added new .deb for amd64. Tested OK on ubuntu 22, save for failure to write the logfile.
Sha256: d77482ea1b6035aab19bf7fbdeb36c6a575d3bd35ec402b3b2 548b45f5078587

[wold]

Replaced the .deb installer, see previous post for the new one.

The old one was compiled against qt6 , while generally only 5 is available. Also set it to amd64, which might help.
Tested OK on ubuntu 22, save for a failure to write the logfile... and possibly the config files. I'll look into that.

[Huwaetzel]

Originally Posted by wold

Replaced the .deb installer, see previous post for the new one.

The old one was compiled against qt6 , while generally only 5 is available. Also set it to amd64, which might help.
Tested OK on ubuntu 22, save for a failure to write the logfile... and possibly the config files. I'll look into that.

I tested the latest version on my Kubuntu 22.04 system: Smooth installation and the same functionality as far as tested as on the windows installation. The manual mounting is not an issue for me, in Linux the next step, in case a device is not detected is a manual mount. => for me an issue with lower priority.

I tested also with my Kali Linux installation based on a rolling Debian system. Here the error "... uses unknown compression for member control.tar.zst" comes up. This seems to happen typically when using a file which is packed for ubuntu originally. I looks like the following bug is responsible. To what I could find using another compression does solve the issue.

From the packages you mentioned libqt5gui5-gles would be missing and would need manual installation

Great work and a very promising start. Thanks

[Huwaetzel]

Originally Posted by wold

Thanks for the color codes (nice you managed to find them )

An update is in the works, It improves row resizing, the Windows USB monitor and local mirror rechecking.
That will need wrapping up first (these things really tend to snowball...), at 2 weeks max.
Colors & flathub are next on the list.

Here my test on the Windows version:
First of all, this was fast and it does all I wanted and even more with the flexibility to select how the color is reflected in the table: Minor wish would be to have the color code also in the exported HTML which I use to keep a copy of the annotations in Calibre.

During more detailed testing I found the following error:
My date does seem to be off for the annotations. It always shows the actual date or the date when it was last pulled and not the date of the annotation


Very good progress

[Huwaetzel]

Sorry, I forgot the screenshot.

[wold]

Many thanks Huwaetzel!
I'm very grateful you wanted to test it

Originally Posted by Huwaetzel

I tested also with my Kali Linux installation based on a rolling Debian system. Here the error "... uses unknown compression for member control.tar.zst" comes up. This seems to happen typically when using a file which is packed for ubuntu originally. I looks like the following bug is responsible. To what I could find using another compression does solve the issue.

It's set to gzip from now on, which should be generally supported (see attachment).
Disabling compression is another option, at 1MB extra it's doable. Will read up on that discussion.

From the packages you mentioned libqt5gui5-gles would be missing and would need manual installation

OK, that one didn't show up with the tool that generated the list on the build system (PopOS, debian-based). Perhaps it's interdependent there.

Minor wish would be to have the color code also in the exported HTML which I use to keep a copy of the annotations in Calibre.

No problem, it's very doable to add it for HTML and csv. For text probably also.

During more detailed testing I found the following error:
My date does seem to be off for the annotations. It always shows the actual date or the date when it was last pulled and not the date of the annotation

That's fixed now (attached). The converted date string used to be generated during import. That was dropped, as these are often unneeded anyway. The new return value however wasn't fully implemented yet.

Saw you also circled the page number: seems the increment from 0 is missing. IIRC, incrementing was moved from the import to the display, but still needed to account for a difference between PDFs and epubs... Anyway, I'll look into that and fix it soon.

Again, thanks!

[Huwaetzel]

Originally Posted by wold

Many thanks Huwaetzel!
I'm very grateful you wanted to test it



It's set to gzip from now on, which should be generally supported (see attachment).
Disabling compression is another option, at 1MB extra it's doable. Will read up on that discussion.



OK, that one didn't show up with the tool that generated the list on the build system (PopOS, debian-based). Perhaps it's interdependent there.



No problem, it's very doable to add it for HTML and csv. For text probably also.



That's fixed now (attached). The converted date string used to be generated during import. That was dropped, as these are often unneeded anyway. The new return value however wasn't fully implemented yet.

Saw you also circled the page number: seems the increment from 0 is missing. IIRC, incrementing was moved from the import to the display, but still needed to account for a difference between PDFs and epubs... Anyway, I'll look into that and fix it soon.

Again, thanks!

Hi wold, I did a quick check of your special version:
- Installation on Kubuntu 22.04 still fine
- Installation on Debian possible, so your change tp gzip did do the trick
- during tuntime I get the following error "/lib/x86_64-linux-gnu/lib.so.6: version GLIBC_2.34 not found (required by avater)" it looks like version 2.34 is for Debian still in experimental and therefore not available in the typical conservative sources from Debian. My Kali Linux is based on buster stable.

I may want to try to switch at the weekend to experimental, but there is probably still a lot of the buster stable versions out there

Thanks again for the providing special versions

[wold]

Originally Posted by Huwaetzel

Hi wold, I did a quick check of your special version:
- Installation on Kubuntu 22.04 still fine
- Installation on Debian possible, so your change tp gzip did do the trick
- during tuntime I get the following error "/lib/x86_64-linux-gnu/lib.so.6: version GLIBC_2.34 not found (required by avater)" it looks like version 2.34 is for Debian still in experimental and therefore not available in the typical conservative sources from Debian. My Kali Linux is based on buster stable.

I may want to try to switch at the weekend to experimental, but there is probably still a lot of the buster stable versions out there

Thanks again for the providing special versions

Ah, glibc troubles... read bad things about that, but it seems quite fixable.
glibc 2.28 is the version for buster: https://launchpad.net/debian/+source/glibc
Anything older (<2019) seems unlikely, so let's try use that. Stay tuned.

Ideally a build is made for each distro, but I'm kinda hoping to avoid that if possible.
There's SUSE's OBS, but that's not set up in a day

BTW, better don't update your machine. Never know what breaks. I had a string of bad luck with failing resume from suspend (nvidia-related).