Calibre-Web on RPi4 Connect Cloudflare Tunnel Problems

[MontyJ]

Hola to all,

Been a (long) while since I visited! But, in over my head (again!) and looking for some help.

Switched to the T-Mobile 5G Home Internet service, and of course they don't allow port forwarding in their router and have IPV6 addresses. So went with a Cloudflare (free) tunnel setup. My previous setup was with NOIP, but they said they could not help with this new setup.

Calibre-Web is installed and runs with "cps" to start it. cps and all the support files are in home/pi/.local/bin

I have the 'cloudflared' tunnel setup, and it runs but I get the "bad gateway error code 502" when I connect with my new website name. I am sure that is because I have not connected the 'cps' startup to the apache2 web server with the new name.

So how to I get "mywebsite.net" to connect with "cps"?

Thanks!

Monty

Edit: I should have added these details. I start the tunnel on the RPi with:
cloudflared tunnel run --url 192.168.12.165:8083 mywebsite

The tunnel starts up and runs with no errors. But when I try to connect with a browser on the internet I get this error from the tunnel process on the RPi:

2024-03-19T20:40:53Z ERR error="Unable to reach the origin service. The service may be down or it may not be responding to traffic from cloudflared: dial tcp 192.168.12.165:8083: connect: connection refused" cfRay=867047b5d89808a8-LAX originService=http://192.168.12.165:8083
2024-03-19T20:40:53Z ERR Request failed error="Unable to reach the origin service. The service may be down or it may not be responding to traffic from cloudflared: dial tcp 192.168.12.165:8083: connect: connection refused" connIndex=1 dest=http://mywebsite.net/ ip=198.41.192.77 type=http

The 192.168.12.165 is the local IP assigned by the TMO router. 8083 is the port I have always used (Pagekite and NOIP), so prefer to keep it if possible.

I have no clue what the 198.41.192.77 is all about.

[Quoth]

You may need a different ISP if you want to remotely access your home. Is "T-Mobile 5G Home Internet service" also supporting IPV4? Do they use NAT for IP V4? Is it really a mobile/cellular service? Note "5G" is not broadband, no matter what band it uses or how fast it is.

So forget about direct home access, tunnel or not if you don't have cable, fibre or xDSL.

Hosting anything via mobile (any version of G) is very difficult to nearly impossible. The only way to make 5G work like broadband is this:
1) Have a hosted website.
2) Use rsync or similar over SFTP or similar on home PC to update or read the hosted website. Connections only initiated from home to the hosting, so no port forwarding needed. Works on ANY internet connection.
3) Only access your hosted website when out from home.

No tunnel needed. Even if you have symmetric T1 and and static IP, or fantastic FTTP with 50 MBps or 100 Mbps up from home and 500 or 1000 Mbps down (remote access is limited by home upload), using the hosted solution (about $100 pa) with rsync or similar running from home is better.

[theducks]

for bringing up that T-mobil does not support Port forward. That saved me from making a bad move in the future, when I dump my over priced Cable Internet.
Currently, those are my only choices (The fibre is coiled on the pole at the end of my block for over a year. it may still be dark, even then)

[Quoth]

Originally Posted by theducks

for bringing up that T-mobil does not support Port forward.

The 5G is essentially a cell/mobile system, so even if you had a router/modem that supported "port forwarding" it likely wouldn't work due to how their network is connected to the actual internet. It won't work well with most satellite systems, but for different reasons.

Port forwarding is for connections originated outside your home network from the public internet. That will mostly work for xDSl, Cable, fibre and radio links uing broadband protocols, i.e. always on connections. If you don't have a static IP then a program on your home network has to report the public IP when it changes to the Dynamic DNS server service you sign up to.

All cell/mobile networks (1G, 2G, 3G, 4G, 5G, Wimax) only connect on demand and only regular calls (not IP port access by a remote) incoming 100% enable the connection as the bandwidth is shared by ALL users in the cell (mast area). An outgoing data connection works, otherwise it would be no use. But even that may not actually connect if the mast is fully loaded. All cell/mobile networks (1G, 2G, 3G, 4G, 5G, Wimax etc) are like an instant dialup. You can even get modems controlled by AT commands like the old analog POTS.

Real broadband is always on, or at least mostly always on, and unlike all cell/mobile networks (1G, 2G, 3G, 4G, 5G, Wimax etc) you always get a connection; any severe contention simply reduces the speed.

Most marketing of 3G, 4G and 5G has been dishonest. It's certainly internet access, but not true broadband. Old DSL only giving 1 Mbps is more reliable than 500 Mbps 5G, because it's always there unless broken. The peak speed quoted for 3G/4G/5G is assuming a perfect signal and only one user on that sector of the mast. A mast might have separate systems in different directions to improve capacity.

How do I know? I did most of the R&D for an ISP that had fibre, dsl, two kinds of satellite, 4G, point-to-point wireless and point to many wireless using cable protocol over microwave (DOCSIS 2.0 then DOCSIS 3.0). They also have datacentres. I was researching ways to use cable broadband protocols over wireless links.

[theducks]

FWIW I plug my wired telephone into my cable router, therefore my IP is static (it did not change in 3 years).

Looking at some of the T-Mobile router (backs), they have a Phone jack, but I suspect it behaves more like a Mobile as you say.

BTW this issue is not just about Calibre...It can affect Gamers

[MontyJ]

Ok, thanks for the comments! But having gone this far, I want to go just a little further, hard headed as I am...
I do have a connection to my machine through the internet via a working tunnel. How reliable it will be, due to those cellular quirks may make it unusable, but I need to confirm it, as I have seen comments by others this tunnel approach works.
So, lets assume for the moment I have my old fiber connection. I have at one point, had the Apache default page popup, so I know an internet connection to my machines web page is possible.
I just need to know how to tell Apache to monitor port 8083 and how to get "cps" available to be seen as a webpage.
I realize this is not directly a Calibre-Web issue, but I can't get help from Cloudflare because I have the "free" plan so no technical support there.
Again, If I run "cps", Calibre-Web starts up. When I try a local connection to 192.168.12.165:8083, I get:
The connection was reset
The connection to the server was reset while the page was loading.
I am guessing it is an Apache config file setting, but have no clue where to start.
Thanks.
Monty

[Quoth]

You need to have any connection initiated from the home end for cell/mobile/4G/5G Internet service. That's why I recommend you have hosting, not a tunnel. Then it's trivial for the Pi at home to update the hosted account using rsync or similar.

Nothing else will be reliable.


This is NOTHING to do with Calibre.

[MontyJ]

Originally Posted by Quoth

You need to have any connection initiated from the home end for cell/mobile/4G/5G Internet service. That's why I recommend you have hosting, not a tunnel. Then it's trivial for the Pi at home to update the hosted account using rsync or similar.

Nothing else will be reliable.


This is NOTHING to do with Calibre.

Ok, you made you point, a couple of times. And you advice makes sense for many but I have my reasons to host my data locally for access by others w/o passing through a hosting service.

And there others that say it work. Perhaps they are BS'sing about it, dunno. But since I can initiate a connection to a webpage on my RPi through the internet, using the TMO tunnel, I need to prove it will/won't work for my setup, no disrespect for your knowledge intended.

So if you can't offer help for making the webpage of Calibre-Web show up, ok. Got it.

If anyone else can, I am open to suggestions

Thanks,

Monty

[Quoth]

SMS* or cron job to open a connection from R Pi to Cloudflare or a website to wake the 5G connection. Then your connection to R Pi server might work.

And there others that say it work.

Maybe something in the home is using the internet when they need to connect.

Certainly it can work, but inherently unreliably if it's a mobile/cell connection.

[* I used an old Nokia and serial port adaptor to control PC with SMS. Only "allow listed" numbers and the first word of the message would be the command. Nothing able to be deleted or changed. A reply SMS was a stock level. Data was charged per second in those days and many phones had no email anyway. SMS on a PC is still handy if you have password.

Nowadays with widespread spoofing I'd have a password too. Also there are now cheap Mobile/cell modules for Pi GPIO port. You do need an account and SIM. The modules are a fraction of the USB/WiFi access cell/mobile hotspots and unlike them give Pi access to send and receive SMS, make or answer voice calls etc.]

[MontyJ]

Ok, but I need something much more basic to start with...and for the moment, let's forget all about any 'tunnel'

Linux environment.

When I type "cps", the calibre-web process runs, waiting for a connection. Remember, this has been running years using NOIP and Pagekite. But I have somehow messed up the webpage access, even locally.

When I open my browser and put in "http://localhost:8083", I should get the UI login for calibre-web. Instead I get: "Unable to connect. Firefox can’t establish a connection to the server at 192.168.12.165:8083."

It's the same with Chromium and Chrome browsers.

I am pretty sure it is the basic config files to hook port 8083 to cps. And cps is in "/home/pi/.local/bin", which is in the environment path.

But I don't know which of those basic config files are what need to be changed or what the changes in them should be. Previous NOIP configurations may still be in effect, which I can't use anymore, so want to start with basic webpage access through the Apache2 setup. Once local access is available, i can tackle the web access issues.

So, how to get calibre-web (cps web app) to show up locally?

Thanks!

Monty

[Quoth]

Originally Posted by MontyJ

When I open my browser and put in "http://localhost:8083", I should get the UI login for calibre-web. Instead I get: "Unable to connect. Firefox can’t establish a connection to the server at 192.168.12.165:8083."

OK, I don't use Calibre-web, but I've set up loads of web servers.

When I open my browser and put in "http://localhost:8083"

Presumably a browser on the Pi?
If you are not sure what port calibre web is using I think you can
export CALIBRE_PORT=8083
and that forces it (or any port not used). Some people seem to use 7878.

Which Pi OS?
I just got a Pi4b a few days ago and didn't like the official Pi OS, so downloaded Ubuntu Mate. I'd abandoned Ubuntu years ago for Mint + Mate on laptops & desktops, so spent 1/2 a day Mintifying it a bit.
I have a 128 G SD card after test on a 32G. I used Disks on desktop Mint to dump image of SD card and wrote it to the 128G card.

The alloy case and 3.5A 3rd party PSU came today. Less scary than bare PCB and power via my tablet PSU or Dell shoebox 7050 USB-C port!

So I might set up Calibre Web/Content server on it out of curiosity, though I have a "real" server in the attic (less real than 20 years ago, but probably "better") which has a web server. I have multiple hosted websites too.

So certainly:
1) Get access and operation working on the Pi with browser on the Pi. Make sure it either has an excluded static IP, or that your DHCP server is set to assign same IP always via Mac. Disable IP6 on the Pi. It should ideally be cabled ethernet, though WiFi is possible.

2) Verify access works on a different computer on the LAN.

3) Consider how to access remotely.

I didn't realise (1) was broken.

[Quoth]

I just started the Calibre Library on my workstation.
Went: Connect/share -> Start Content Server.
If I go back to "Connect/share" I see:
"Stop Content Server [192.168.0.130:8080]"
So 8080 is the default.

Connecting with Firefox localhost:8080 is fine.

So I went to Pi and paused the Wizard of Oz video
Didn't work! Ah, but it helps to connect the LAN cable as I never setup WiFi. Now works.

So started Calibre, then content server on Pi (only one ebook rather 7000+ on workstation). localhost:8080 is OK

Tried 192.168.0.170:8080 on Workstation. Works. Though that IP of the Pi isn't fixed.

[MontyJ]

Yes, on the RPi with Chromium, and on my main PC on the same network with Firefox and Chromium.

If I use port 80 on the RPi, I do get the Apache2 "welcome page", so Apache is working. So I need to get 8083 to connect. When I do run "cps", I can verify it is running as a process, so it should be available on 8083, as it was setup before on that port and as far as I know, the cps settings have not changed.

I access the RPi directly (keyboard and monitor, through SSH, and via my main Linux PC to the desktop so have lots of connection options.

EDIT: Here are the files I suspect need updating in some way to get Apache to serve port 8083. Looking at their contents, I can see no reference to that port, but perhaps I am looking in the wrong place. Here are the files:
pi@raspberrypi:/etc/apache2 $ ls -ltr
total 84
-rw-r--r-- 1 root root 320 Jun 8 2022 ports.conf
-rw-r--r-- 1 root root 31063 Jun 8 2022 magic
-rw-r--r-- 1 root root 1782 Jun 8 2022 envvars
-rw-r--r-- 1 root root 7224 Jun 8 2022 apache2.conf
drwxr-xr-x 2 root root 4096 Jan 18 2023 mods-enabled
drwxr-xr-x 2 root root 4096 Jan 18 2023 conf-enabled
drwxr-xr-x 2 root root 4096 Jan 18 2023 sites-enabled
drwxr-xr-x 2 root root 4096 May 10 2023 conf-available
drwxr-xr-x 2 root root 16384 May 10 2023 mods-available
drwxr-xr-x 2 root root 4096 Mar 21 09:12 sites-available

Monty

[Quoth]

Are you running Calibre, then starting the Calibre Content server?

Or are you doing something else?

I could be wrong, but I don't think Calibre uses Apache. I can connect locally or on the LAN on port 8080 on Pi or Workstation when I start Calibre Content server. Apache doesn't seem running on either.

It's not even installed on Pi.
It is installed on the Workstation, but apparently not running as no connection on port 80, which is what is configured (no SSL). I'm not sure why.

I used Synaptic Package manager on both to check.

[MontyJ]

Originally Posted by Quoth

Are you running Calibre, then starting the Calibre Content server?

Or are you doing something else?

I could be wrong, but I don't think Calibre uses Apache. I can connect locally or on the LAN on port 8080 on Pi or Workstation when I start Calibre Content server. Apache doesn't seem running on either.

It's not even installed on Pi.
It is installed on the Workstation, but apparently not running as no connection on port 80, which is what is configured (no SSL). I'm not sure why.

I used Synaptic Package manager on both to check.

CPS is web app fork of Calibre-Web, which uses a calibre database, so calibre is not directly involved in this setup. Calibre is where my master database resides, CPS uses/reads that database, independent of calibre, to present the ebooks in a user friendly way on the web. So CPS stands alone and just reads the calibre db. When I update the master calibre db, I go into CPS and do a 'reindex', and it then rereads the master db and picks up the new entries.

Gotta run for a few hours for some meetings. Thanks for the input!

Monty