Using Calibre as a client for another Calibre instance?

[kovidgoyal]

database_path is a legacy config item, library_path is the important one.

[solomon]

Originally Posted by kovidgoyal

database_path is a legacy config item, library_path is the important one.

Ok, but when calibre-server launches, from where does it populate this value into global.py?

Richard

[toddos]

I think I have it working. I didn't want to have to give rw permissions to Network Service on metadata.db, but that seems to be the only way to make it work.

So, recapping:

1. Move config to a different location using CALIBRE_CONFIG_DIRECTORY, prepopulate it with the existing configuration files, and give Network Service rw access to the folder
2. Give Network Service read access to your library path (file system and share)
3. Give Network Service rw access to the database path. I actually did it at the folder level, but you might be able to do it at the file level. Also, because the database apparently needs rw access but I don't want to give write access to the library itself this is another good reason to separate the database from the library.

I really don't like that calibre-server, ostensibly a read-only process, has to have rw access to the database in order to run. Any thoughts on changing that? Should I file a ticket?

[kovidgoyal]

No point, the db access library calibre uses has no way to open a sqlite db in read only mode.

[jhoff80]

So by the title of it, I thought this thread was what I wanted, but it seems you're talking about something different. Seems like you're talking about the WHS serving books to Calibre clients over the local network. However, it would be even better if I could put my http://<username>.homeserver.com:8080 link into Calibre and have it connect as a client over the internet.

Any chance of that being possible in the near future?

[kovidgoyal]

Not in the near future, but it is in my long term plans.

[JvdW]

Originally Posted by solomon

Perhaps we can get Kovid to weigh in on why the service doesn't run properly as another user? I too don't like the idea of leaving it running as admin!

Richard

That is easy: Administrators have network access and if you run services using the SYSTEM account or the special accounts available on Vista/Win7 then you can't access files on UNC paths or mapped drives.

The trick usually is to make a special account for this kind of operations which allows it to read/write to the network but doesn't have any rights on the rest of the system(s).

Regards,

Joop

[toddos]

Originally Posted by JvdW

That is easy: Administrators have network access and if you run services using the SYSTEM account or the special accounts available on Vista/Win7 then you can't access files on UNC paths or mapped drives.

The trick usually is to make a special account for this kind of operations which allows it to read/write to the network but doesn't have any rights on the rest of the system(s).

Regards,

Joop

The built-in NETWORK SERVICE account has network access and can read UNC paths where it has permissions. In an earlier post I sorted through all of the permissions required to allow network service to run/access Calibre, but that gets blown away on upgrade. Calibre updates so often that I've just sucked it up and switched to running the service as admin. Not ideal, but it works. It'd be great if we could get mysql or postgesql backend support for Calibre some day (sqlite doesn't necessarily migrate easily, but it's still SQL-ish so with enough work it should be possible). I realize that's not high on Kovid's list and I don't have personal time at the moment to take on such a huge project so I'll live with my hack.

Edit: Actually, let me amend that slightly. Something happened during an upgrade that broke my service running as network service. Checking my permissions, everything seems to still be correct, but after an upgrade (maybe it was 0.6 to 0.7?) the service stopped working and I hadn't had time to debug it. It could be a change in code that requires more/different permissions, or that requires permissions somewhere I haven't provided them. Network service has permissions to read/execute calibre-server and read/write the database, but no permissions to read or write the actual book location. I figured that was the safest way to protect my books since I have to give write access to the database due to sqlite. Maybe a change to the catalog/server now requires reading the library as well as the database, I don't know. Eventually I'll get around to debugging it, or not and just live with running as admin.

[DoctorOhh]

Originally Posted by toddos

Edit: Actually, let me amend that slightly. Something happened during an upgrade that broke my service running as network service. Checking my permissions, everything seems to still be correct, but after an upgrade (maybe it was 0.6 to 0.7?) the service stopped working and I hadn't had time to debug it. It could be a change in code that requires more/different permissions, or that requires permissions somewhere I haven't provided them. Network service has permissions to read/execute calibre-server and read/write the database, but no permissions to read or write the actual book location. I figured that was the safest way to protect my books since I have to give write access to the database due to sqlite. Maybe a change to the catalog/server now requires reading the library as well as the database, I don't know. Eventually I'll get around to debugging it, or not and just live with running as admin.

Your info is smack dab in the middle of my ignorance zone.

That said I'll pass on this info.

Calibre never supported UNC paths, but they worked previously. With the change to 0.7.x many folks have reported that UNC paths have stopped working properly with calibre.

[toddos]

Originally Posted by dwanthny

Your info is smack dab in the middle of my ignorance zone.

That said I'll pass on this info.

Calibre never supported UNC paths, but they worked previously. With the change to 0.7.x many folks have reported that UNC paths have stopped working properly with calibre.

I'm not actually using UNC shares for the service (I'm using UNC shares from my client machine and it's working perfectly on 0.7.5, even with the database redirected to a separate UNC share than the library). Technically you're supposed to always use the share paths on WHS, but I've found that using d:\shares is generally safe. So I'm using that and just dealing with local file system permissions, though everything should be setup and ready to go with UNC shares or mapped drives if needed in the future.

[JvdW]

Originally Posted by dwanthny

Calibre never supported UNC paths, but they worked previously. With the change to 0.7.x many folks have reported that UNC paths have stopped working properly with calibre.

I was one of them reporting that and maybe I have found why it stopped working.
Yesterday I did some work on it and it looks like it might be a bug in QT 4.6.2. Atleast I can reproduce it with an example included with the QT I installed to get Sigil to compile. Its the findfiles example. If I let it search for pdf's on a server share and then try to start it it does nothing but if I let it search my local disk for pdf's it will start my pdf viewer just fine. It uses the same syntax to start an external programme as Calibre does --> bug in QT
Now I'm going to install Calibre-0.6.55 to see which QT version it uses.

one moment please ....

Ah, its using 4.6.0, so that would mean, check the sources of 4.6.0 against 4.6.2 and see what changed in the handling of QDesktopServices.openUrl(QUrl.fromLocalFile(name)) . QUrl.fromLocalFile() is producing the expected output but QDesktopServices.openUrl is the culprit

The latest QT, 4.6.3 might have this fixed but didn't see it in the changelog

PS:
Just saw that the qt dlls from calibre-0.7x and 0.6.55 are the same myself for not checking that earlier. So what is different then ???
The call to QDesktopServices.openUrl(QUrl.fromLocalFile(name)) is the same in 0.6.55 and 0.76 and as in findfiles but in 0.6.55 it works and in the others it doesn't


Regards,

Joop

[kovidgoyal]

The Qt version hasn't changed between 0.6.55 and 0.7.x. What has changed (I think) is that earlier calibre was not using QUrl.fromLocalFile it was directly passing a filename to QDesktopServices. While this used to work for UNC paths, it broke in several other more common cases, so I switched it to QUrl.fromLocalFile

If you can figure out the correct way to pass UNC filenames to QUrl I'll be happy to make the change.

You can test by doing

calibre-debug -c "from PyQt4.Qt import *; app = QApplication([]); f = r'\\server\whatever'; url = QUrl.fromLocalFile(f); QDesptopServices.open(url)"

[JvdW]

Originally Posted by kovidgoyal

The Qt version hasn't changed between 0.6.55 and 0.7.x. What has changed (I think) is that earlier calibre was not using QUrl.fromLocalFile it was directly passing a filename to QDesktopServices. While this used to work for UNC paths, it broke in several other more common cases, so I switched it to QUrl.fromLocalFile

Yes, I figured that out too. I even downloaded serveral older version to see which one would allow me to use UNC paths. I had to go back quite far, sofar as that version is using QT 4.5.2. I downloaded the sources of QT 4.6 and 4.5 and compared QDesktopServices to see what changed. They changed another subroutine and think that must be the problem, not you changing your source although I think I'll see what you changed and if it can be changed back without side effects.

You can test by doing

calibre-debug -c "from PyQt4.Qt import *; app = QApplication([]); f = r'\\server\whatever'; url = QUrl.fromLocalFile(f); QDesptopServices.open(url)"

Thanks for the example with typos Adding a print statement to see what url looked like didn't give me any clues yet (file://mcc-srv/apps/e.pdf). According to Wikipedia its a valid URI scheme (RFC1738)

Testing a bit further I might decide to recompile QT 4.6.3 with some debugging print statements added to qdesktopservices_win.cpp to see what it makes of the file://server/file.pdf since I have a feeling that things go very wrong there.
Atleast the version of 4.5.2 does something more sensible to my eyes.

Regards,

Joop