Calibre-Web on RPi4 Connect Cloudflare Tunnel Problems

[Quoth]

You might be better running Calibre and starting the content server.

As a result of looking at your issue I discovered my workstation Apache was broken.

Anyway, this system has had upgrades and the Apache not used since. So why is it not running when it's installed?
1: Trying to load php7.4 instead of php8.1, so fixed that.
2: No server name. Easy fix
3: Something already on port 80. Seems "lighttpd" is installed. No idea why and nothing answers when I connect to port 80. Edited its config to port 8020 in case it's needed rather than uninstall. Restarted it.

Then restarted Apache and port 80 now gives me the default "Welcome page".

The Calibre content web server seems friendly enough and you'll get support on it.

Third party programs using the Calibre Database isn't a good idea.

[Quoth]

Apache2 uses /etc/apache2/ports.conf

# If you just change the port or add more ports here, you will likely also
# have to change the VirtualHost statement in
# /etc/apache2/sites-enabled/000-default.conf

Listen 80

<IfModule ssl_module>
	Listen 443
</IfModule>

<IfModule mod_gnutls.c>
	Listen 443
</IfModule>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

But I think using apache is overkill.

[MontyJ]

You may be right about Apache being overkill, but since calibrre-web (cps) is a 'web app', it is needed. I looked at the calibre content server 12 years ago and it was not what I needed for a small group of avid remote ereaders. I then found this:

http://pypi.org/project/calibreweb/

and haven't looked back since.

Not trying to convince you it is better than anything else...it is just that it is the only program I have found that has a ton of features and internet access to a local db for ebooks.

Anyway, how do I add a new port to that file you show and others if necessary? I know zip about the structure/format of those config files. I was using SSH under NOIP, so at some point want to add it back as well, but initially I will be happy to get plain old http working

Monty

[MontyJ]

I found some examples on the web that said to add these to get Apache2 to listen to additional ports:

"Listen 8083" right below "Listen 80" in the "ports.conf" file. Did that.
It now reads:
Listen 80
Listen 8083

"Then it said add "*:8083" after the "*:80" in the "sites-enabled/000-default.conf" file in this line: <VirtualHost *:80>. Did that.
It now reads:
<VirtualHost *:80 *:8083>

Restarted the Apache service: sudo service apache2 restart

No errors.

Then ran "cps" and received error: Error starting server: [Errno 98] Address already in use

Rebooted the RPi, same error.

I verified with netstat apache is listening on 8083:
tcp6 0 0 :::8083 :::* LISTEN 817/apache2

I don't get it that cps says the address is "in use"? I did check that there were no other "cps" or "copsweb" processes running, so what is using 8083 other than apache listening on it?

I looked in the calibre-web log files, and this seems to confirm it is starting on port 8083:

[2024-03-21 16:15:43,759] INFO {apscheduler.scheduler:885} Added job "end scheduled task" to job store "default"
[2024-03-21 16:15:51,226] INFO {cps.server:231} Starting Tornado server on :8083
[2024-03-21 16:15:51,227] ERROR {cps.server:251} Error starting server: [Errno 98] Address already in use
[2024-03-21 16:15:51,228] INFO {cps.server:284} webserver stop (restart=False)
[2024-03-21 16:15:51,229] INFO {apscheduler.scheduler:202} Scheduler has been shut down

Monty

[rantanplan]

Wouldn't it be easier for you to just use this pre-configured docker-container:
https://hub.docker.com/r/linuxserver/calibre-web

Especially if you want to expose the webserver to the internet, no one would be able to escape the virtualized server in case someone tries to take over the machine.

[MontyJ]

Originally Posted by rantanplan

Wouldn't it be easier for you to just use this pre-configured docker-container:
https://hub.docker.com/r/linuxserver/calibre-web

Especially if you want to expose the webserver to the internet, no one would be able to escape the virtualized server in case someone tries to take over the machine.

That sounds great, tho I know absolutely nothing about dockers. And can I use the existing setup files for calibre-web? I would like to keep all the current users and their setups, passwords, etc.

[MontyJ]

Well, I may have leaped before I looked, but I am so far behind on getting this going it isn't funny.

With zip experience with docker, I went ahead and install it to the RPi.

But I did back up every file/folder I could find related to my current install of CW, as well as the previous NOIP install files/folders plus the certs I used for SSL. All that worked for a 'normal' connection for remote access.

After I got docker installed, I went to the https://hub.docker.com/r/linuxserver/calibre-web
site. There is a button there

docker pull linuxserver/calibre-web

that seemed appropriate to get the applicable CW image.

Applied that from the CLI of the RPi, but received this error:

$ docker pull linuxserver/calibre-web
Using default tag: latest
latest: Pulling from linuxserver/calibre-web
no matching manifest for linux/arm/v8 in the manifest list entries

Am I on the right track or what? How many zillion steps did I leave out, LoL?

Monty

[MontyJ]

Originally Posted by MontyJ

Well, I may have leaped before I looked, but I am so far behind on getting this going it isn't funny.

With zip experience with docker, I went ahead and install it to the RPi.

But I did back up every file/folder I could find related to my current install of CW, as well as the previous NOIP install files/folders plus the certs I used for SSL. All that worked for a 'normal' connection for remote access.

After I got docker installed, I went to the https://hub.docker.com/r/linuxserver/calibre-web
site. There is a button there that seemed appropriate to get the applicable CW image.

Applied that from the CLI of the RPi, but received this error:

Am I on the right track or what? How many zillion steps did I leave out, LoL?

Monty

Well, just found out the raspbian ver I have tries to pull the x32 bit version of the docker image, even tho it says the os is x64. go figure. So, gonna wipe it all and put on Ubuntu on the RPi and be done with those hassles.

So will probably rejoin this thread once I get the new os and cw going again, but adios for now! And thanks for tthe great tips!!

[MontyJ]

Ok, wiped RPi4 and installed the Ubuntu Server.
Installed docker and pulled the LinuxServer.io Calibre-Web image
Created the container and now have access to a new/blank calibre-web setup.

However, when I created the container, I set the config folder as /home/ubuntu/calibre, which has all my original config files. I also set my storage to the enternal USB drive, which contains ~500gb of eBooks with the metadata.db file for the calibre database.

But those locations did not 'take', so all I have is a blank CW at this point.

How do I setup my custom config files in the new container? I did use docker exec to 'edit' the container files, and tried to copy my config files into the default /config folder in the container. But that didn't work, so stuck.

FYI, total noob with docker

Monty

[MontyJ]

Originally Posted by rantanplan

Wouldn't it be easier for you to just use this pre-configured docker-container:
https://hub.docker.com/r/linuxserver/calibre-web

Especially if you want to expose the webserver to the internet, no one would be able to escape the virtualized server in case someone tries to take over the machine.

Took your suggestion

Got a lot to learn about using docker, but at least I now have local access to CW through the 8083 port.

[MontyJ]

Ok, found the correct way to cp my original config files into the container, so now have my normal (so far) CW config...except, can't get the ebook storage location to take.

I have mounted the external USB SSD drive, in the normal /mnt location, and verified the ebooks are in /mnt/eStorage/eBooks/All-eBooks folder.

But when I select that location, I get "db location is invalid. Enter correct path".

But when I go to the folder for the "All-eBooks", it shows empty. I shut the RPi down, removed the SSD drive, attached it to my Linux PC and when I mounted it, all the ebooks were there.

Put it back on the RPi, and the folder is blank inside and outside of the container. There some other folders/files in 'eStorage' as well, but they do not show up either in the Ubuntu /mnt/eStorage folder...only the eBooks folder for some reason.

In normal Ubuntu system, /etc/fstab shows:

LABEL=writable / ext4 discard,errors=remount-ro 0 1
LABEL=system-boot /boot/firmware vfat defaults 0 1

I did mount the external SSD and it shows up in /mnt, but shouldn't it show up in fstab? Do I need to mount it in fstab manually?

Monty

Edit: Ok, answered my own question. Yes I had to manually make an entry to the /mnt configuration would be 'permanent' and mount on boot.

[MontyJ]

Well, now stuck on how to change the ebook storage database location. I can login to CW (yippee!), but using my original config files means it points to a different location.

My original location on the RPi was:

/mnt/eStorage/eBooks/All-eBooks

I did retain that, but for some reason the container was not showing any files in that folder, which is a copy of my Calibre ebook database files.

So looking at some examples on the web, they all said to use the 'media' mount point. So I changed the Ubuntu mount point to /media/... and I can see all the ebooks properly. I then deleted all entries in the /mnt folder so it could not conflict somehow.

Next I removed the calbire-web container and started over, using this command to create a new container:

docker create --name=calibre-web -e TZ=America/Los_Angeles -e PUID=1000 -e GUID=1000 -e DOCKER_MODS=linuxserver/calibre-web:calibre -v /home/ubuntu/calibre -v /media/eStorage/eBooks/All-eBooks -p 8083:8083/tcp --restart unless-stopped linuxserver/calibre-web:0.6.21

That works and creates a new CW container. I then copied my original CW config files into the default /config folder in the container.

I get a normal login screen, and my old logins work, so I know it is looking at the right files.

However, those old config files point to /mnt/...., which didn't work, so I change it to /media/...

But CW won't let me change it! I cannot get past the Database Configuration screen. I can put in the new path manually, or use the browse to go to /media/..., but it simply will not take.

I cannot delete app.db, as that contains all the user profiles, settings, passwords, etc...been there, done that before and it is a major job and disruption for users personal settings which they have to start over on.

So, any tips on how to get by this?

Monty

[mariosipad]

@MontyJ

Why change a winning team? I mean, why change you original calibre-web setup that worked?

Then your only (big) problem would be "how to remotely connect to your e-books?"

Can you revert to your old setup that worked? Without docker!

Then you can focus on how to make cloudflare working for your setup.
I've never done this, but I always wanted to try. So this seems to be a good opportunity to dive in (next week).


For you immediate problem with docker:
1 - kill the container
2 - umount the /media path
3 - mount the /mnt path in fsstab. reboot and check if you can go to your e-books following the path /mnt/.....
4 - make a new container and export (-v) to the original /mnt/..... path
5 - put your config files in the config directory and stop and start the container.

What is the situation now if you connect to calibre-web from the LAN?

[MontyJ]

Originally Posted by mariosipad

@MontyJ

Why change a winning team? I mean, why change you original calibre-web setup that worked?

Then your only (big) problem would be "how to remotely connect to your e-books?"

Can you revert to your old setup that worked? Without docker!

Then you can focus on how to make cloudflare working for your setup.
I've never done this, but I always wanted to try. So this seems to be a good opportunity to dive in (next week).


For you immediate problem with docker:
1 - kill the container
2 - umount the /media path
3 - mount the /mnt path in fsstab. reboot and check if you can go to your e-books following the path /mnt/.....
4 - make a new container and export (-v) to the original /mnt/..... path
5 - put your config files in the config directory and stop and start the container.

What is the situation now if you connect to calibre-web from the LAN?

Well, finally I was able to get local access...yipeeee! I had looked around and saw several negative comments about using Raspbian, so blew it away and installed Ubuntu server. Then I installed docker and loaded the LinuxServer.io image for CW. It took some back and forth stuff on Discord to figure out the proper way to create a container. It wound up being a problem with the way I had to declare my external USB ebook storage device in the container. Anyway, it is now working on a local level, no internet access yet through the cloudflared tunnel.

Yes, the old way worked before I went with T-Mobile Home Internet. But I contacted the folks at NOIP, which was working fine, and they said they had no way to make it work with the double nat and IPV6 of T-Mobile. I didn't contact Pagekite, who has a similar no-port-forwarding approach, but they are in Iceland, and email support takes like forever and begrudging at that. It worked, but only with a ton of technical settings that only a network engineer could love...

Anyway, that's where I am at. I have to reinstall cloudflared on the new ubuntu setup. Then I will know more about what does and doesn't work.

Good to hear from you again It has been about a year since I was here last and got some great assistance from you and other folks to get the RPi setup going with SSL. Cheers!!

Monty

Edit: Your container suggestion might work on a different image of CW, but I had to add a "-v /media/eStorage/eBooks/All-eBooks:/books" setting when creating the container. For some reason, It was the .../books folder that then linked to the contents of the .../All-eBooks folder on the USB SSD drive. That folder probably could be named anything, but I was happy enough to get the dang thing going I didn't want to rock the boat, so to speak. And yes, fstab had to be updated as well. So when I logged into CW, changed the database folder to /books, it could see all the files and that got things going. Also added /opt/appdata in the Ubuntu OS as my config folder for creating the container. This way I can just change/edit the CW config files in Ubuntu w/o having to muck around in the container.

[mariosipad]

@MontyJ

If it works, it works! OK.

Do you have a reasonably fixed IPv6 address for your Pi?

Then your clients could access your books over IPv6 (if your router allows pass trough to your Pi.)