New "VNC" viewer for recent Kobo devices and firmware

[NiMa]

Hi all,
This evening I just had an idea; I thought I could use NiLuJe's excellent FBInk library to create an image-based network display viewer. I finished it in 3 hours and you'll find it attached in this post.
It is still in very early development stage, and it is *horribly* slow. Like you have to wait 1 second for the screen to update. Maybe if you use it with USBNet it could be a little better. I join as well some photo and video to prove it working.

Here's a video: https://www.youtube.com/watch?v=JBLq...ature=youtu.be

Grosso modo, that's how it runs:

1. It starts by starting (via SSH) a 'watch' process that tells FBInk to refresh the screen every 30 seconds.
2. After that, it starts a neverending loop that uses gnome-screenshot on the host system to take screenshots of the screen live, every +/- 1 second. I tried with scrot and it worked very well, the only fatal flaw was that it wasn't possible to see the mouse cursor in the capture. The screenshot named continuous.png is put on /tmp and is constantly replaced to deliver an up-to-date image.
3. After that, I use sshpass to login in the Kobo without password prompt, and transfer via scp the screenshot that is in /tmp to the /tmp folder of the Kobo so it doesn't wear the SD card on the write cycles.
4. Finally, to show the image on the screen, I use sshpass again with SSH to tell fbink to print an image on the screen, full screen. I recommend using your Kobo in landscape mode, and putting your host screen in 1024x768, as the text will display better in that resolution on the host and orientation on the Kobo.
5. The loop starts over and repeats the process until you decide to stop it via SIGINT (CTRL+C). *NOTE* I didn't find how to stop the 'watch' process when you press CTRL+C. I tried different 'trap' ideas, but none of them worked. Any help or ideas are appreciated, I'm not very experienced in bash programming
For now you'll have to stop 'watch' manually by SSH to the Kobo after ending the image sending from the host and by typing 'killall watch', then 'exit'.

REQUIREMENTS:
1. SSH and FBInk from NiLuJe https://www.mobileread.com/forums/sh...d.php?t=254214 on your Kobo
2. gnome-screenshot and sshpass on your host machine
3. Your host machine *must* be a Linux system, otherwise you probably will experience issues.
4. The Landscape mode hack for supported AND non-supported Kobos (will enable Landscape system-wide), in Kobo eReader.conf:
[DeveloperSettings]
ForceAllowLandscape=true
5. If you've never SSH-ed your Kobo before, SSH on it before you start the script, otherwise you may receive an error like 'Host key verification failed'. In the first SSH you'll do will likely be given a message like this:
The authenticity of host '192.168.0.181 (192.168.0.181)' can't be established.
ECDSA key fingerprint is SHA256:65X9gdR9f71y6Fs9qSNZFCez8mJ8RAQ4Lhq+1c46Ttw .
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '192.168.0.181' (ECDSA) to the list of known hosts.
Just type 'yes' and you're good to go!
6. Python 3 if you use the HTTP version, which is faster but less secure.

******NOTES:
I suggest to open the web browser on your Kobo before launching the script, as you will not see the left corner hour display changing minute after minute and therefore hiding a part of the host screen on your Kobo's screen, and as it will keep the WiFi on as long as you keep it open.
This viewer doesn't support touch at all. (Maybe in the far future?)
GitHub repository: https://github.com/tux-linux/Kobo-ne...display-viewer

So... enjoy it! And you can write your feedback here!

[DevonHess]

Pretty cool experiment. However, as you're probably aware, VNC refers to a specific standard. Kind of misleading to call this VNC.

There is actually a VNC client that supports Kobo.

Keep up the good work, though.

[NiMa]

Originally Posted by DevonHess

Pretty cool experiment. However, as you're probably aware, VNC refers to a specific standard. Kind of misleading to call this VNC.

There is actually a VNC client that supports Kobo.

Keep up the good work, though.

Thanks for your input! Yes, I knew that VNC was a specific standard. That is why I put the word in quote marks in the title. However, if a moderator wants to change it to a more accurate title (like 'Use your Kobo as an external monitor'), I would be entirely okay with that.
Also, I already knew that there was an existing VNC client for Kobos, but it was outdated (2013) and (I tried to run it) wouldn't work with recent hardware and firmware.
Note though that I may implement touch in the viewer in the future.

[JSWolf]

Originally Posted by DevonHess

Pretty cool experiment. However, as you're probably aware, VNC refers to a specific standard. Kind of misleading to call this VNC.

There is actually a VNC client that supports Kobo.

Keep up the good work, though.

But does this VNC still work on the current firmware give how old the code is?

[NiMa]

No, not at all, just look at its thread when users (including me) describe encountering several errors. While it worked perfectly for old devices/firmware, it is incompatible with recent hardware without serious tweaking. (From what I experienced, though)

[NiMa]

Any feedback...? Ideas...? I'd like to improve my software listening to you.
Of course my main concern here is to improve speed. So, any feedback, ideas on how to improve it, are very welcome!

Thanks!

(I may put the code on GitHub (even if it's just an 8-line script) if someone wants to contribute)

[Elektron]

Originally Posted by NiMa

2. After that, it starts a neverending loop that uses gnome-screenshot on the host system to take screenshots of the screen live, every +/- 1 second.

Is it possible to make screenshots more frequent?

[NiMa]

Originally Posted by Elektron

Is it possible to make screenshots more frequent?

The screenshots are no longer in the limitation of one second; it takes a screenshot as fast as it could. The speed depends on the speed of the SCP transfer. You can look in the script, it's only 6 lines of Bash code, not very complicated :

#!/bin/sh
while :; do read -r -p 'IP: ' ip && break; done
while :; do read -r -p "Your Kobo device screen width resolution: " width && break; done
while :; do read -r -p "Your Kobo device screen height resolution: " height && break; done
    sshpass -p test ssh root@$ip 'watch -n 30 "fbink -H -k -f | tee -a output.txt" &>/dev/null &'
    while true; do gnome-screenshot -p --file=/tmp/continuous.png && sshpass -p test scp /tmp/continuous.png root@$ip:/tmp && sshpass -p test ssh root@$ip 'fbink -g file=/tmp/continuous.png,w='$width',h='$height''; done

What I could implement is an automated 'watch' process on the Kobo that displays the image via FBInk every x seconds, regardless if the images were updated or not.
I'm looking in that way right now. I may update the script by this evening.

For now, you can try it with USBNet, it improves the speed a little.

[NiMa]

@elektron : updated zip and Git repo to improve speed. Screenshots are still sent via SCP, but FBInk displays /tmp/continuous.png regardless if it's been updated or not every 0.5 seconds.
You can change the value if you want, I tried 0.1 but I think it's pretty much the same.

However, there's a noticeable difference of speed on USBNet -> old script vs USBNet -> updated script.

[NiMa]

Updated Zip files and Git repo. The script in vnc-http.zip now uses HTTP protocol to transfer screenshots. You'll have to have Python3 installed, which is likely to be included by default on your Linux system.
This release features a speed improvement compared to the version with SCP (vnc.zip).

[Elektron]

Any idea how to get it running on an Aura H2O2v2? (mk.7) Also does it have to be a Linux sytem or can I run it in a Linux sub-environment on my Windows laptop?

[NiMa]

Originally Posted by Elektron

Any idea how to get it running on an Aura H2O2v2? (mk.7) Also does it have to be a Linux sytem or can I run it in a Linux sub-environment on my Windows laptop?

The same for my Libra and Glo HD for the device part: Download NiLuJe's stuff (link in the first post), install it on your Kobo by copying the KoboRoot.tgz in the .kobo folder of the Kobo eReader partition, then ejecting it and it should install the software.
TL;DR : If FBInk already works on your device, don't reinstall it. If FBInk works on your device, then that will 99,99% sure work with your Aura H2Ov2.

I didn't test it with a sub-environment and don't know exactly how it works and how it manages the screen, but I think it may be better if you installed Ubuntu/Linux Mint on a virtual machine with VirtualBox https://www.virtualbox.org/ or if you boot the LiveCD of your USB key on your computer, without needing of installing it.
Contact me for more info if you need it, I'll be happy to help!

[Elektron]

Thanks!

[NiMa]

Originally Posted by Elektron

Thanks!

It works? Glad to have helped you out!

[benjaffe]

I got it working on MacOS with a small tweak. Replace the `while true` line with:

```
while true; do screencapture -x -R0,0,840,632 /tmp/continuous.png && convert /tmp/continuous.png -rotate 270 -colors 255 -depth 8 +profile "*" -strip /tmp/continuous.png && sshpass -p test scp /tmp/continuous.png root@$ip:/tmp ; done
```

This captures a smaller area of the screen so things are bigger on the Kobo (I have a Libra H2O), and uses ImageMagick to rotate the png and compress it for faster transfer. The four numbers are x, y, width, and height.

To install ImageMagick using Brew, run `brew install imagemagick`. If you don't have Brew, follow the instructions on their website to install it.

If you Brew complains about architecture because you're on an Apple Silicon mac, there are a few options. The quick hack is `arch -x86_64 brew install imagemagick` to run it under emulation, but if you use Brew a lot, you may want to search around for whatever the best solution is in the future.