Content Server in read-only mode?

Caleb666 · 01-16-2015, 06:43 PM

Hi,

I would like to run the calibre content server on a separate computer and _only_ use it to serve content. Is this possible?

Currently, when I try to launch it with a library that is mounted as read-only I get:

Quote:

Traceback (most recent call last):
File "site.py", line 51, in main
File "site-packages/calibre/library/server/main.py", line 124, in main
File "site-packages/calibre/db/legacy.py", line 70, in __init__
File "site-packages/calibre/db/legacy.py", line 44, in create_backend
File "site-packages/calibre/db/backend.py", line 349, in __init__
File "site-packages/calibre/utils/filenames.py", line 115, in is_case_sensitive
IOError: [Errno 30] Read-only file system: u'/nfs/Calibre_Library/calibre_test_case_sensitivity.txt'

Thanks.

kovidgoyal · 01-16-2015, 10:08 PM

No, the content server needs write access to the db file.

Caleb666 · 01-17-2015, 09:21 AM

I see. I managed to get a hacky "solution" by using aufs.

Basically, I mount the calibre library as read-only and on top of it I stack a read-write temporary folder (say /var/tmp/calibre-server-scratch), so if the calibre server wants to write to a file, aufs will use COW to copy the file and write the changes in the temporary location.
This probably means that I should kill the content server, clear the temp folder and then start it again every once in a while (cron script?) so that it will pick up any changes to the database.

I think that one caveat is to pay attention to the case-sensitivity of the filesystem of the calibre library and the one of the temp folder, in my case they're both on ext4 so I'm probably not going to be seeing any weird problems.

itimpi · 01-17-2015, 09:38 AM

Although the content server mounts the database with read-write permissions, I have a feeling that it never actually writes to the database and the requirement for write access is an artefact of the database interfacing library in use. Maybe you will be able to see for certain with the approach you are using?

kovidgoyal · 01-17-2015, 09:47 AM

The calibre server currently only ever writes to the db if the db schema needs to be upgraded. In the future, that will change, as I plan on making the content server read/write.

Caleb666 · 01-17-2015, 09:57 AM

Quote:

Originally Posted by itimpi

Although the content server mounts the database with read-write permissions, I have a feeling that it never actually writes to the database and the requirement for write access is an artefact of the database interfacing library in use. Maybe you will be able to see for certain with the approach you are using?

You are correct, I don't see a copy of the database file in the temp folder. So I guess that deleting the temp folder might not be strictly necessary, but I still prefer to do it.

I'm working on a script that will wrap all this functionality into a service so I could simply do "sudo service calibre-ro-server start/stop/restart" and have it do all the necessary work. Then I might simply add a cronjob to restart the service (which will clear the temp folder) at 3am or something.

I will post it here once I'm done, if anyone's interested.

chaley · 01-17-2015, 12:53 PM

Instead of all the filesystem hackery, why not just copy the db to a RW filesystem (/tmp?) and set the CALIBRE_OVERRIDE_DATABASE_PATH environment variable appropriately?

Caleb666 · 01-17-2015, 01:11 PM

Quote:

Originally Posted by chaley

Instead of all the filesystem hackery, why not just copy the db to a RW filesystem (/tmp?) and set the CALIBRE_OVERRIDE_DATABASE_PATH environment variable appropriately?

Because I'm mounting the library as read-only, and I don't want any writes to it whatsoever. The error I posted above makes me think that calibre will still attempt to write the the location of the library itself to check for case sensitivity even if the database itself is copied elsewhere.

WaldoPepper · 01-20-2015, 03:38 AM

Quote:

Originally Posted by kovidgoyal

...In the future, that will change, as I plan on making the content server read/write.

Would this create difficulties for the environment I am running Calibre in, as described in this thread?

I would like Calibre to keep some level of "concurrency" in the database access, since I think this would be in line with all the tendencies to locate data in "clouds" or on NAS'es.

A content-server running on a low power "headless" server to serve the content to multiple mobile reader apps (like the CC) and the full-featured Calibre frontend on one or more desktop machines to do all the import/maintenance, all of them accessing one central Calibre db: that, for me, is the perfect setup...

rockpilp · 02-25-2015, 10:11 AM

I too would like to be able to use Calibre-server in read-only mode, as I currently rsync the library from my Mac to a dedicated server. If the server were to write to the DB, it would get overwritten on the next sync anyway…

eschwartz · 02-25-2015, 11:35 AM

Then what is the problem? Let it operate in rw mode.

IIRC the database library calibre uses does not support opening a database ro -- and that isn't compelling enough to change it.

Unless there is a compelling outside reason to mount the underlying filesystem as ro, I don't see the problem.

PeterT · 02-25-2015, 11:44 AM

You might want to read up on how chaley has his content server running on an offsite VPS. See this post from him for his scripts and what he has done. https://www.mobileread.com/forums/sho...23&postcount=2

This includes how he replicates data from his calibre to the VPS, and how he ensures that the content server restarts when changes have been made.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Calibre Content Server and Windows Home Server 2011	HughMcC	Calibre	8	10-01-2014 08:26 AM
is there a tool to open (read) books from the calibre's content server?	tsolignani	Related Tools	2	04-16-2013 08:14 AM
500 Internal Server Error accessing content server Calibre 0.8.8	DaddyO57	Calibre	1	07-20-2012 07:08 PM
Content server not serving all content.	False_Chicken	Calibre	0	10-19-2011 06:56 PM
Wiki server move - read-only mode	Alexander Turcic	Announcements	1	04-24-2007 03:59 PM

01-16-2015, 10:08 PM	#2
kovidgoyal creator of calibre Posts: 44,559 Karma: 24495948 Join Date: Oct 2006 Location: Mumbai, India Device: Various	No, the content server needs write access to the db file.

01-17-2015, 09:21 AM	#3
Caleb666 Zealot Posts: 100 Karma: 400000 Join Date: Jul 2010 Device: iPad 2 64GB	I see. I managed to get a hacky "solution" by using aufs. Basically, I mount the calibre library as read-only and on top of it I stack a read-write temporary folder (say /var/tmp/calibre-server-scratch), so if the calibre server wants to write to a file, aufs will use COW to copy the file and write the changes in the temporary location. This probably means that I should kill the content server, clear the temp folder and then start it again every once in a while (cron script?) so that it will pick up any changes to the database. I think that one caveat is to pay attention to the case-sensitivity of the filesystem of the calibre library and the one of the temp folder, in my case they're both on ext4 so I'm probably not going to be seeing any weird problems.

01-17-2015, 09:38 AM	#4
itimpi Wizard Posts: 4,553 Karma: 950151 Join Date: Nov 2008 Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)	Although the content server mounts the database with read-write permissions, I have a feeling that it never actually writes to the database and the requirement for write access is an artefact of the database interfacing library in use. Maybe you will be able to see for certain with the approach you are using?

01-17-2015, 09:47 AM	#5
kovidgoyal creator of calibre Posts: 44,559 Karma: 24495948 Join Date: Oct 2006 Location: Mumbai, India Device: Various	The calibre server currently only ever writes to the db if the db schema needs to be upgraded. In the future, that will change, as I plan on making the content server read/write.

01-17-2015, 12:53 PM	#7
chaley Grand Sorcerer Posts: 12,032 Karma: 7257323 Join Date: Jan 2010 Location: Notts, England Device: Kobo Libra 2	Instead of all the filesystem hackery, why not just copy the db to a RW filesystem (/tmp?) and set the CALIBRE_OVERRIDE_DATABASE_PATH environment variable appropriately?

02-25-2015, 10:11 AM	#10
rockpilp Junior Member Posts: 1 Karma: 10 Join Date: Feb 2015 Device: Nexus 6	I too would like to be able to use Calibre-server in read-only mode, as I currently rsync the library from my Mac to a dedicated server. If the server were to write to the DB, it would get overwritten on the next sync anyway…

02-25-2015, 11:35 AM	#11
eschwartz Ex-Helpdesk Junkie Posts: 19,421 Karma: 85397180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	Then what is the problem? Let it operate in rw mode. IIRC the database library calibre uses does not support opening a database ro -- and that isn't compelling enough to change it. Unless there is a compelling outside reason to mount the underlying filesystem as ro, I don't see the problem.

02-25-2015, 11:44 AM	#12
PeterT Grand Sorcerer Posts: 12,756 Karma: 75000002 Join Date: Nov 2007 Location: Toronto Device: Libra H2O, Libra Colour	You might want to read up on how chaley has his content server running on an offsite VPS. See this post from him for his scripts and what he has done. https://www.mobileread.com/forums/sho...23&postcount=2 This includes how he replicates data from his calibre to the VPS, and how he ensures that the content server restarts when changes have been made.