NickelMenu script for fixing Pocket images

[qkqw]

As you may know, the current Pocket implementation in Kobo readers struggles with images other than JPEG. JPEGs will be displayed fine, but all other image types, like PNG, SVG or GIF simply won't be displayed.

As I grew tired of seeing the "image not found" icon - too many articles using png files and the like - I wrote a small script for NickelMenu to convert these images to JPEG on the fly.

Add a line like this one to your NickelMenu config:

menu_item  :library  :Fix Pocket Images  :cmd_spawn  :quiet:/bin/sh /mnt/onboard/.adds/pocket/fix.sh
  chain_success  :dbg_toast  :Done

Then extract pocket.zip to /mnt/onboard/.adds/pocket.

The script currently supports the following formats: GIF, PNG, SVG, WEBP.

Ps. You might need to reboot your Kobo if the article already was cached.

Changelog:

2022-05-28:

• Initial release

2022-06-08:

• Add ImageMagick timeout flags
• Only move files if ImageMagick was successful
• Added toast message to NickelMenu config

2022-06-08:

• Switch to ImageMagick 7
• Add support for GIF, SVG, WEBP
• Add support for SVG with embedded text using a default font

[xyclonei]

Hi there! Thank you for this excellent add on! Any reason why it's png only for now?

Can it be expanded to other file formats?

[JSWolf]

But how does the website code know to pick up the converted images given that they will now have a different file extension?

[xyclonei]

Originally Posted by JSWolf

But how does the website code know to pick up the converted images given that they will now have a different file extension?

This is my assumption on how it works.

If you have a look at .kobo/articles directory, you can see that the images are stored as a UID filename with no extension. The article file (index.html) makes a call to the filename only. If it's JPG encoded, it displays, else it fails.

What this script does is to make an in-place replacement of the PNG file with a JPG one. Since the filename remains, the call to the image should still be valid and will display correctly.

What I want to check though is whether this can be used as a catch-all for any image type.

[qkqw]

Originally Posted by xyclonei

Hi there! Thank you for this excellent add on! Any reason why it's png only for now?

Can it be expanded to other file formats?

Sure, that would be quite easy to do. However I could only find Jpg and Png files so far. If you happen to find any other format, I'd gladly amend the script. Let me know!

Also, @NiLuJe's ImageMagick only seems to support Jpg, Jfif and Png at this time, but I could cross compile a version with support for more file types if needed.

[qkqw]

Originally Posted by xyclonei

This is my assumption on how it works.

If you have a look at .kobo/articles directory, you can see that the images are stored as a UID filename with no extension. The article file (index.html) makes a call to the filename only. If it's JPG encoded, it displays, else it fails.

What this script does is to make an in-place replacement of the PNG file with a JPG one. Since the filename remains, the call to the image should still be valid and will display correctly.

What I want to check though is whether this can be used as a catch-all for any image type.

That is absolutely correct. The heavy lifting is done by Pocket, which already seems to convert images. Unfortunately their API is not public, so I haven't had the chance to see what they do exactly.

To use a catch-all, you'd just need to change this line

  if [ "$FORMAT" == "PNG" ]; then

to this

  if [ "$FORMAT" != "JPG" ]; then

Obviously we'd need an ImageMagick with support for more formats

[xyclonei]

Just thinking out loud, but is there a timeout for this?

Is there a possibility that it might get hung up on an image and keep eating battery?

[qkqw]

Originally Posted by xyclonei

Just thinking out loud, but is there a timeout for this?

Is there a possibility that it might get hung up on an image and keep eating battery?

Currently there is no timeout: ImageMagick will either suceed or fail transforming the images, but it won't get stuck.

[NiLuJe]

IM has a builtin fairly complex limits framework that can be set up to avoid issues in degenerate use-cases (because you can't be 100% sure one weird bug on specific input wouldn't cause a long busy loop).

(at a minima, passing something like -limit time 60 might not be entirely stupid).

[xyclonei]

Thank you, @NiLuJe! That's the fringe case example I was thinking of.

@qkqw, will adding -limit time 60 to the variables CONVERT & IDENTIFY work?

#!/bin/sh

ARTICLES="/mnt/onboard/.kobo/articles"
CONVERT="/mnt/onboard/.adds/pocket/convert -limit time 60"
IDENTIFY="/mnt/onboard/.adds/pocket/identify -quiet -limit time 60"

LD_LIBRARY_PATH="/mnt/onboard/.adds/pocket/lib:${LD_LIBRARY_PATH}"
export LD_LIBRARY_PATH

for i in $(find $ARTICLES -type f -not -iname "*.html"); do
  FORMAT=$($IDENTIFY -format "%m" "$i")
  if [ "$FORMAT" == "PNG" ]; then
    $CONVERT "$i" "$i.jpg"
    mv "$i.jpg" "${i%.jpg}"
  fi
done

[qkqw]

Originally Posted by xyclonei

@qkqw, will adding -limit time 60 to the variables CONVERT & IDENTIFY work?

Yes, I've added this and a few more improvements to the original script.

@NiLuJe thanks for the assist!

[xyclonei]

Originally Posted by qkqw

Yes, I've added this and a few more improvements to the original script.

@NiLuJe thanks for the assist!

Thank you very much for the update!

[xyclonei]

I've been scratching my head over this.

The below NM config displays Done when the script process is spawned and not when the Pocket image conversions are done.

menu_item:library:Fix Pocket Images:cmd_spawn:quiet:/bin/sh /mnt/onboard/.adds/pocket/fix.sh
  chain_success:dbg_toast:Done

I thought using cmd_output instead of cmd_spawn so that the script has a chance to indicate when it is done might sort it out:

menu_item:library:Fix Pocket Images:cmd_output:500:/bin/sh /mnt/onboard/.adds/pocket/fix.sh
  chain_success:dbg_toast:Done

But this would mean that the script would need to provide some sort of output that it was done. So I modified it to the below (apologies for the amateur code):

#!/bin/sh

ARTICLES="/mnt/onboard/.kobo/articles"
CONVERT="/mnt/onboard/.adds/pocket/convert -limit time 60"
IDENTIFY="/mnt/onboard/.adds/pocket/identify -limit time 60 -quiet"

PROCESSED="0"

LD_LIBRARY_PATH="/mnt/onboard/.adds/pocket/lib:${LD_LIBRARY_PATH}"
export LD_LIBRARY_PATH

for i in $(find $ARTICLES -type f -not -iname "*.html"); do
  FORMAT=$($IDENTIFY -format "%m" "$i")
  if [ "$FORMAT" == "PNG" ]; then
    $CONVERT "$i" "$i.jpg"
    if [ $? -eq 0 ]; then
      mv "$i.jpg" "${i%.jpg}"
      (($PROCESSED)) || PROCESSED="1";
    fi
  fi
done

if [ "$PROCESSED" == "1" ]; then
	echo "DONE: Pocket articles optimised";
else echo "NOTE: Pocket articles not/already optimised";
fi

However, cmd_output message box pops up an error stating that there was a timeout.

Any idea what I'm doing wrong?

[sherman]

Originally Posted by xyclonei

However, cmd_output message box pops up an error stating that there was a timeout.

Any idea what I'm doing wrong?

Yeah,

cmd_output

has a configurable timeout setting that you can set from 0-10 seconds (provided in milliseconds). (

:cmd_output         :500:command

). It has such a strict limit because your kobo will essentially be frozen until the command completes.

A better option would be to use a program such as FBInk to output messages to the screen. You may have to hunt for precompiled binaries, though it is available in NiLuJe's stuff pack.

Alternatively, another way to output messages is my own NickelDBus.

Of the two, FBInk isn't as intrusive, as it doesn't hook itself into the Kobo software in any manner.

[xyclonei]

Thank you for the feedback, @sherman!

Seems like this might be more than I can chew with very little pay-off.

I'll look into both FBInk & NickelDBus to see if I can make sense of them.