Kute File Monitor: A smarter file-trigger based launcher

[NiLuJe]

The idea of building a better fmon has come up a few times, with the main goal of avoiding potential catch-22 issues by making sure an icon has been processed by nickel before launching anything...

This came up once again in a KOReader enhancement request, this time with a POC.

Since I don't use KSM, and I've always been interested in the trick used by fmon to launch stuff (using inotify, which I'm familiar with, especially how to deal with it in the peculiar environment of an USBMS device), I decided to take the plunge and build a proper, polished product out of that concept.

So here comes KFMon! The goal is to basically do what fmon does, but safer, more user-friendly, and more configurable.

Of particular interest to document readers is the ability to inhibit the launch of *anything* through KFMon while a specific action is running (that's in addition to the general protection against running concurrent instances of the same action). This feature is enabled in the KOReader & Plato configs featured in the repo .

Speaking of KOReader & Plato, if that's what you're trying to run, see the One Click Packages thread .

I won't be rehashing the documentation here, see the GitHub project for the full and up-to-date version .
This documentation includes install instruction (notice how I'm not repeating those here, so you will *have* to read the doc? ;p).

Some quick but important notes: make sure you have full backups of your device beforehand, in particular your Nickel database & settings.
Full disclosure: I never had to resort to something more drastic than a pinhole reboot & a db & settings restore (i.e; no lost content, no factory reset) during my early tests, but those Kobos sure are finicky, so, better be safe than sorry .

It ships with a default config file providing a KOReader launch icon. In order for it not to do anything stupid, it requires a current version of KOReader.

Of course, running both fmon & kfmon in parallel is highly discouraged. Thankfully, unless you're doing something wrong, you actually can't .

If you encounter anything weird, detailed feedback will be key to fix potential issues .
And, as usual, I'm open to all kind of trivial comments. In particular if you feel the documentation is lacking or confusing, don't hesitate to tell me .

And, it bears repeating: carefully read the full documentation from top to bottom before diving blindly into stuff .

NOTE: macOS users, you may have to bypass a few of your OS's most dubious "Quality of Life" features to get things to work properly, c.f., this post for more details.

Enjoy!

[frostschutz]

Very nice! (never could find the source code of the original fmon)

You could also work with visual cues, since nickel is going to display the image after all. If it actually appears on screen there should be no doubt left that it wasn't just a background database update or some other script fiddling with your files...

For example if the PNG image had to contain some bright pink pixels you could check the framebuffer whether or not such pixels are present, similar to how scanline detects sleep/shutdown messages in my screensaver mod.

Taking this idea further, maybe you could also put a menu / touchable buttons in the PNG itself.

eInk is black&white but the framebuffer is actually color (RGB565) so a lot could be done with color codes (and should survive image scaling) that do not necessarily alter the appearance on the eInk screen.

[NiLuJe]

@frostschutz: Thanks! That's certainly an interesting idea...

Afraid it might be a tad above my pay grade (I'm not actually a CS student at all, more of an English lit one, so I'm basically terrible at C ;p), but a neat idea .

If I ever find some time/motivation, one utility I'd like to do is a simple (i.e., no Qt) tool to print text messages to the framebuffer (for those familiar with it, we have eips on Kindles, and it's damn useful).
For instance, right now, hitting a SQL busy timeout can be a bit mystifying, since there's zero actual feedback besides the action not launching :/.

[frostschutz]

Originally Posted by NiLuJe

I'm basically terrible at C ;p

C isn't my primary language either.

Originally Posted by NiLuJe

tool to print text messages to the framebuffer

I wanted to add such things to my pngshow program (which currently only does transparency, I wanted it to do offsets, delays, and other scripting) but well, takes time and motivation to find...

So those mods of mine which do give feedback on screen do this in a very primitive fashion (scanline debug mode drawing a line on screen, autoshelf having a autoshelf icon flashing on and off in lieu of a progress bar, magic memory upgrade even just blanking the screen...).

Not sure how much Kobo development I'll be doing this year or the next for that matter in the end it's a device for just reading and it does that fine

[Lucas Malor]

given the intricacies of how Nickel actually processes books, it might trigger an action before the trigger file has successfully been processed by Nickel [...] KFMon tries to alleviate this issue by doing a number of checks before deeming that launching the action is "safe": we first check if the file in question is even in the Library database, and then we confirm that it has been processed further by checking for the existence of the various thumbnails Nickel creates when first displaying a book in its Library and Homescreen.

Do you mean that Nickel should display the book icon before its processing is finished?

[NiLuJe]

@Lucas Malor: At the very least in terms of thumbnails processing, yes. And since Nickel has to actually open the source file to process the thumbnail, that's a hit for fmon/kfmon.

Note that this is the behavior I noted on current FW versions (i.e., it does that on demand: library thumbnails are generated for the current page you're browsing in the Library, and the "last book opened" tile is generated when it has to be shown, the first time you close a new book). Nickel might have been doing some more aggressive stuff before, which might have triggered more bizarre behavior.
Or not. (After all, a number of people have been using fmon without encountering significant issues).


In any case, not treating those spurious hits as actual user-triggered opens is saner and safer.
As is not launching something that will potentially kill Nickel while Nickel has a hold on the DB, since we have no real way of knowing if Kobo bothered to fully gracefully handle SIGTERMs, since that should allegedly never happen on a vanilla production device.

And as the sickel fiasco showed, they work in mysterious ways...

[davidfor]

Originally Posted by NiLuJe

@Lucas Malor: At the very least in terms of thumbnails processing, yes. And since Nickel has to actually open the source file to process the thumbnail, that's a hit for fmon/kfmon.

Note that this is the behavior I noted on current FW versions (i.e., it does that on demand: library thumbnails are generated for the current page you're browsing in the Library, and the "last book opened" tile is generated when it has to be shown, the first time you close a new book). Nickel might have been doing some more aggressive stuff before, which might have triggered more bizarre behavior.
Or not. (After all, a number of people have been using fmon without encountering significant issues).

A very long time ago, the firmware generated all four cover images when the book was first added to the device. And it also generated and saved cover images for books on the external SD card. For sideloaded books, this was done during the book processing phase after disconnecting from the PC. Then, still a long time ago, but not quite so long, Kobo changed to generate the cover images as needed. And not to save the cover images for books on the external SD card (thought there is at least one firmware version that did). And changed where the cover images were stored. This has been the case for at least two years.

I am pretty sure that fmon was originally written during the first phase. That would have meant that fmon, or whatever called it, was not active at the point that the books were opened to generate the cover images.

Note: When reading this post, make the browser window very narrow, tilt the screen away from you and scroll the text just a little to fast to read easily. If your brain doesn't supply the correct theme music, hand in your nerd card.

[PeterT]

I HEARD THE MUSIC!!!!

But maybe this post should have been made on May 4th...

[NiLuJe]

@davidfor: Easy mark, I recently attended a John Williams tribute concert ;p.

Thanks for the Nickel history lesson though . And for keeping part of it alive in the Calibre Kobo driver, that was obviously massively helpful when working on this

[pazos]

Tested here! Worked really nice even on corner cases (adding some new watches in a row with png triggers/ proccesing huge KoboRoot.tgz files...)

I replaced fmon in my two kobo auras hd, modify the ini file to restart nickel instead of rebooting. So far so good!

I must say that my nickel DB is almost empty, as I use koreader as my main reader and keep the books outside nickel processing.


The only way I found to broke it is running lighttpd & php, with h5ai serving /mnt/onboard. h5ai has preview options for png files and this will trigger kfmon once for each watch. So, in my case this will trigger koreader, coolreader and usbnet-toggle, and of course, this breaks the system.

The solution for me is really simple, modify root of webserver to /mnt/onboard/.ebooks and put the (watched) files on /mnt/onboard.

[NiLuJe]

@pazos: Great, thanks for the feedback (in particular the Nickel restart thing we discussed on GH, thanks .).

As for the web server thing, yeah, I'm afraid there's not much else that could be done. Inotify doesn't provide us with any information about who/what triggered the file access, so, no possibility to single out nickel .

[the_doc]

I don't use KSM at all and I use fmon to start both KOReader and PBChess.
Has anyone tested this kfmon with PBChess?

[Lucas Malor]

Originally Posted by NiLuJe

we have no real way of knowing if Kobo bothered to fully gracefully handle SIGTERMs

Can't you send SIGSTOP?
Sometimes I lost cover files. KOReader should not boot anymore this way.

[NiLuJe]

@Lucas Malor: I *think* there's an issue of Nickel holding exclusive access to the input devices which prevents us from using SIGSTOP, but I *might* be mixing stuff up (since that's used in various fun & interesting ways on Kindles).

EDIT: I was half-right ;p. We get the touch input, but Nickel holds an exclusive lock on the power/light buttons.
That's still an interesting possibility to speed up debugging sessions, though...

----

I'll probably open up the wiki on the GH page to let people add common watch configs, that should make it easier for people to try and migrate.

[frostschutz]

nickel does open touch input exclusively, but you can patch it... it's a string somewhere, something=1 to something=0, i don't remember... however nickel might be confused and crash on resume... also display rotation must be the same... also there is sickel the evil rebooter if nickel does not appear to be working... it got disabled finally but how long will that last