'calibredb add' throws python error when calibre-server is running

[CubGeek]

Raspberry Pi 4 (8gb)
Raspbery Pi OS 64-bit (no, not ArchLinuxARM)
Calibre installed from the repos (yes, I know they're woefully out of date)

$ calibredb --verison
calibredb (Calibre 3.39.1)

$ python -c "import os; print(os.uname())"
('Linux', 'raspberrypi', '5.10.63-v8+', '#1453 SMP PREEMPT Tue Sep 14 15:00:30 BST 2021', 'aarch64')

$ python --version
Python 2.7.16

$ python3 --version
Python 3.7.3

$ cat /etc/default/locale
#  File generated by update-locale
LANG=en_US.UTF-8

I was able to get the server installed, created libraries, and I was then able to import books (mostly epubs from Humble Bundles). I then started the calibre-server and I can see, read, and download books through the web interface. And, once I enabled --with-auth and set up a user, I can now edit metadata through the web interface.

However, I’d like to be able to add books to the server without stopping the server then restarting it afterwards. It seems, according to the messages I’ve seen and the manual that calibredb can do this. So, I’m currently trying to figure out how. Looking at the manual page for calibredb, I saw how to list the library_id for each of the libraries on the content server:

$ calibredb —with-library 'http://localhost:8080/#-' list —username MyUser —password MyPass
cookbooks
fiction
non-fiction
technical

Okay, so now I’ve got a library_id and I attempt to put it into a command:

calibredb add /path-to-books-to-add-to-library/cookbooks/* —with-library 'http://localhost:8080/#cookbooks' —username MyUser —password MyPass

This result in the following

Traceback (most recent call last):
File “/usr/bin/calibredb”, line 20, in <module>
     sys.exit(main())
File “/usr/lib/calibre/calibre/db/cli/main.py”, line 255, in main
     return run_cmd(cmd, opts, args[1:], DBCtx(opts))
File “/usr/lib/calibre/calibre/db/cli/main.py”, line 55, in run_cmd
     ret = m.main(opts, args, dbctx)
File “/usr/lib/calibre/calibre/db/cli/cmd_add.py”, line 366, in main
     identifiers, lcodes, opts.filters
File “/usr/lib/calibre/calibre/db/cli/cmd_add.py”, line 164, in do_add
     oidentifiers, olanguages
File “/usr/lib/calibre/calibre/db/cli/main.py”, line 191, in run
     return self.remote_run(name, m, args)
File “/usr/lib/calibre/calibre/db/cli/main.py”, line 213, in remote_run
     res = self.br.open_novisit(rq)
File “/usr/lib/python2.7/dist-packages/mechanize/_mechanize.py”, line 199, in open_novisit
     return self._mech_open(url, data, visit=False, timeout=timeout)
File “/usr/lib/python2.7/dist-packages/mechanize/_mechanize.py”, line 230, in _mech_open
     response = UserAgentBase.open(self, request, data)
File “/usr/lib/python2.7/dist-packages/mechanize/_opener.py”, line 193, in open
     response = urlopen(self, req, data)
File “/usr/lib/python2.7/dist-packages/mechanize/_urllib2_fork.py”, line 344, in _open
    ’_open’, req)
File “/usr/lib/python2.7/dist-packages/mechanize/_urllib2_fork.py”, line 332, in _call_chain
     result = func(args)
File “/usr/lib/python2.7/dist-packages/mechanize/_urllib2_fork.py”, line 1142, in http_open
     return self.do_open(httplib.HTTPConnection, req)
File “/usr/lib/python2.7/dist-packages/mechanize/_urllib2_fork.py”, line 1115, in do_open
     h.request(req.get_method(), req.get_selector(), req.data, headers)
File “/usr/lib/python2.7/httplib.py”, line 1058, in request
     self._send_request(method, url, body, headers)
File “/usr/lib/python2.7/httplib.py”, line 1098, in _send_request
     self.endheaders(body)
File “/usr/lib/python2.7/httplib.py”, line 1054, in endheaders
     self._send_output(message_body)
File “/usr/lib/python2.7/httplib.py”, line 890, in _send_output
     msg += message_body
UnicodeDecodeError: ‘utf8’ codec can’t decode byte 0x9e in position 0: invalid start byte

However, if I stop the server, add the books (substituting the path to the library for the http-and-library_id path), and then start the server again, it works flawlessly:

$ sudo systemctl stop calibre-server

$ calibredb add /path-to-books-to-add-to-library/cookbooks/* --with-library ~/calibre-library/cookbooks --username MyUser --password MyPass
Added book ids: 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26

$ sudo systemctl start calibre-server

The books are added and I can then see/download/read/edit metadata.

Searching, I saw this post by KovidGoyal that showed an equal sign between the --with-library and the http-and-library_id path and no single quotes like when I got the library_id listing. So, I gave that a shot.

calibredb add /path-to-books-to-add-to-library/cookbooks/* —with-library=http://localhost:8080/#cookbooks —username MyUser —password MyPass

Similar results. I can add books if I stop the server, add them and then start the server again. But using calibredb (which the manual says is used to connect to a server that's running) throws an error. What am I missing or have misconfigured?

[kovidgoyal]

Your version of calibre is too old.

[CubGeek]

Originally Posted by kovidgoyal

Your version of calibre is too old.

Ah. The functionality of calibredb to interface with a currently-running instance of calibre-server was added in versions after the one available in the repos.

Good to know that. Thank you.

While it's unfortunate that the ARM repos are so out-of-date, I do appreciate the software and the effort that's gone into creating it. As well as your respose here. Thanks! While i wait for repos to get updated to something approximating "current" i'll set up a cron job to stop the server, the one to import books, and then one to restart the server.

best wishes!