![]() |
#1 |
Member
![]() Posts: 11
Karma: 10
Join Date: Jul 2024
Device: many
|
Error after import of a lot of books
Hi Mobile Readers.
I recently learned how to create and manage a Calibre Library on an external drive, so I did it and imported the full folder of books I have. It took more than a day, and then once it finished I got the following errors : Code:
calibre, version 6.13.0 AVERTISSEMENT : Erreurs pendant l'ajout: Il y a eu des problèmes lors de l'ajout de certains fichiers, cliquez "Afficher les détails" pour plus d'information ---------------------------------------------------------------------- Échec de la lecture des métadonnées depuis le fichier : /media/lienrag/Elements/Livres/Source/Stephen King/La ballade de la balle elastique & L'h omme qui refusait de serrer la main (8837)/La ballade de la balle elastique & L'homme - Stephen King.epub Avec l'erreur : Traceback (most recent call last): File "/usr/lib/calibre/calibre/utils/ipc/pool.py", line 371, in worker_main result = func(*job.args, **job.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/calibre/calibre/ebooks/metadata/worker.py", line 81, in read_metadata mi, opf, has_cover = serialize_metadata_for(paths, tdir, group_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/calibre/calibre/ebooks/metadata/worker.py", line 27, in serialize_metadata_for f.write(cdata) OSError: [Errno 28] Aucun espace disponible sur le périphérique Code:
---------------------------------------------------------------------- Échec à ajouter le livremetadata /media/lienrag/Elements/Livres/Bibliothèque calibre/John Schember/Guide de demarrage rapide (1)/metadata.opf Avec l'erreur : Traceback (most recent call last): File "/usr/lib/calibre/calibre/gui2/add.py", line 458, in add_book book_id = self.dbref().create_book_entry(mi, cover=cdata) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/calibre/calibre/db/legacy.py", line 341, in create_book_entry ret = self.new_api.create_book_entry(mi, cover=cover, add_duplicates=add_duplicates, force_id=force_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/calibre/calibre/db/cache.py", line 77, in call_func_with_lock return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/calibre/calibre/db/cache.py", line 1965, in create_book_entry self.backend.execute('INSERT INTO books(title, series_index, author_sort) VALUES (?, ?, ?)', File "/usr/lib/calibre/calibre/db/backend.py", line 1022, in execute return self.conn.cursor().execute(sql, bindings) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "src/cursor.c", line 235, in resetcursor apsw.CantOpenError: CantOpenError: unable to open database file What caused that ? And how can I solve it ? There clearly was sufficient space on the external drive (more than 100 Gigabytes). Is there necessary to have some also on my /home ? And/or / ? How much ? |
![]() |
![]() |
![]() |
#2 |
Resident Curmudgeon
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 75,787
Karma: 134321338
Join Date: Nov 2006
Location: Roslindale, Massachusetts
Device: Kobo Libra 2, Kobo Aura H2O, PRS-650, PRS-T1, nook STR, PW3
|
According to the first code block, you've run out of disk space. I don't know if that's the drive that holds the library or the drive that hold the temp files. But you need to free up some space someplace.
|
![]() |
![]() |
Advert | |
|
![]() |
#3 |
creator of calibre
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 44,302
Karma: 23661992
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
You need to have space in /tmp. Some brain dead linux systems put /tmp in RAM. Either fix that or use the calibre env var to change its tmp directory to somewhere you actually have space.
https://manual.calibre-ebook.com/cus...ment-variables |
![]() |
![]() |
![]() |
#4 |
Member
![]() Posts: 11
Karma: 10
Join Date: Jul 2024
Device: many
|
Thanks.
Any idea on how much space I need ? And, is the current situation fixable or do I need to delete the library and re-import it ? |
![]() |
![]() |
![]() |
#5 |
creator of calibre
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 44,302
Karma: 23661992
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
You can use the bulk metadata edit dialog to set metadata from ebook files in all books. But again if you are going to be operating on huge numbers of books dont put /tmp in RAM, preferably use the same disk as your library is on to host /tmp for best performance and robustness.
|
![]() |
![]() |
Advert | |
|
![]() |
#6 | |
Member
![]() Posts: 11
Karma: 10
Join Date: Jul 2024
Device: many
|
Quote:
But how ? Manually, for 4000 books of which I don't know the metadata, that's not realistic. Or is there a way to have Calibre automatically recognize the metadata from the files that are on the disk, considering that I don't know which of the 36 k books are the 4 000 ones without metadata ? |
|
![]() |
![]() |
![]() |
#7 |
Member
![]() Posts: 11
Karma: 10
Join Date: Jul 2024
Device: many
|
Also, how much space do I need ?
I have 12 Gb of RAM (though I'm not sure that all of it was free when I launched the import - maybe 6 GiB was) and I don't have that free space on may current / nor my /home. Is it really a good idea to put the Calibre tmp directory on the external drive ? The external drive has no free space problem, that at least would be solved. |
![]() |
![]() |
![]() |
#8 |
Member
![]() Posts: 11
Karma: 10
Join Date: Jul 2024
Device: many
|
(weird, apparently my first reply is waiting for moderation, but the second passed fine...)
(not a real problem since they addressed different questions, it just makes the "also" of the second reply seems a bit weird, ah too bad) |
![]() |
![]() |
![]() |
#9 | |
Resident Curmudgeon
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 75,787
Karma: 134321338
Join Date: Nov 2006
Location: Roslindale, Massachusetts
Device: Kobo Libra 2, Kobo Aura H2O, PRS-650, PRS-T1, nook STR, PW3
|
Quote:
|
|
![]() |
![]() |
![]() |
#10 |
Well trained by Cats
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 30,334
Karma: 58000000
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
|
Do them in small batches. Groups of <20 allows an easy recover when things go south. If you don't, and have a fail, you lose the whole fetch job.
And WHY do they ALL need to get external metadata? Most books have the basics, and Calibre should have used that when it added (unless you set Add to use the Filename instead) Be more selective. ![]() Now for the downside of BATCH get metadata: Some of it is WRONG ![]() ![]() FWIW it took me almost a year to pound the metadata (Library data) into shape for 900+ paper books I track with Calibre |
![]() |
![]() |
![]() |
#11 | |
Well trained by Cats
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 30,334
Karma: 58000000
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
|
Quote:
Recently, this has also happened to old users that had not posted recent (for whatever the definition of Recent was) |
|
![]() |
![]() |
![]() |
#12 |
creator of calibre
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 44,302
Karma: 23661992
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
Do them all, and yes put /tmp on the same disk as your library. How much space you need will depend on how many books their sizes, etc.
Or redo the import this time with /tmp on the same drive as your library. |
![]() |
![]() |
![]() |
#13 |
Bibliophagist
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 39,196
Karma: 153098470
Join Date: Jul 2010
Location: Vancouver
Device: Kobo Sage, Libra Colour, Lenovo M8 FHD, Paperwhite 4, Tolino epos
|
If you are enthused about bulk fetching metadata, you are liable to find your IP address blocked. When I last played with this and , I found that Amazon allowed me to do 3-5 metadata fetches and then I started getting timeouts. After 15 or so minutes, I could fetch another block.
|
![]() |
![]() |
![]() |
#14 |
Member
![]() Posts: 11
Karma: 10
Join Date: Jul 2024
Device: many
|
Now Calibre won't start..
Even withe calibre --verbose, I only get back to the bash prompt, no information is given. calibre-debug -g gives : Code:
calibre-debug -g calibre 6.13 embedded-python: False Linux-6.1.0-23-amd64-x86_64-with-glibc2.36 Linux ('64bit', 'ELF') ('Linux', '6.1.0-23-amd64', '#1 SMP PREEMPT_DYNAMIC Debian 6.1.99-1 (2024-07-15)') Python 3.11.2 Interface language: fr QPA platform: xcb devicePixelRatio: 1.0 logicalDpi: 96.0 x 96.0 physicalDpi: 112.28608414239481 x 112.1103448275862 Using calibre Qt style: True |
![]() |
![]() |
![]() |
#15 |
the rook, bossing Never.
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 12,247
Karma: 89531599
Join Date: Jun 2017
Location: Ireland
Device: All 4 Kinds: epub eink, Kindle, android eink, NxtPaper11
|
Mint 21.3 with Mate Desktop
Code:
calibre-debug -g calibre 7.16 embedded-python: True Linux-5.15.0-117-generic-x86_64-with-glibc2.35 Linux ('64bit', 'ELF') ('Linux', '5.15.0-117-generic', '#127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024') Python 3.11.5 Interface language: None EXE path: /opt/calibre/bin/calibre-debug Successfully initialized third party plugins: DeACSM (0, 0, 16) && DeDRM (10, 0, 3) && Amazon.com Multiple Countries (1, 0, 0) && Annotations (1, 17, 13) && Author Book Count (2, 2, 3) && Barnes & Noble (1, 5, 6) && Comments Cleaner (1, 14, 0) && Copy Cover To Device (2, 0, 4) && Diaps Editing Toolbag (0, 4, 0) && EpubCheck (0, 2, 5) && EpubMerge (3, 1, 0) && EpubSplit (3, 8, 0) && Find Duplicates (1, 10, 9) && Generate Cover (2, 3, 4) && Goodreads (1, 8, 2) && Hyphenate This! (0, 1, 3) && KePub Input (3, 6, 6) && KePub Metadata Reader (3, 4, 3) && KePub Metadata Writer (3, 4, 3) && KePub Output (3, 6, 9) && KindleUnpack - The Plugin (0, 83, 1) && Kobo Books (1, 9, 2) && Kobo Utilities (2, 17, 1) && KoboTouchExtended (3, 6, 9) && Library Splitter (1, 0, 7) && Manage Series (1, 5, 2) && Modify ePub (1, 8, 4) && Obok DeDRM (10, 0, 3) && Open With (1, 8, 3) && Quality Check (1, 13, 14) && Quick Preferences (1, 7, 1) && Reading List (1, 15, 4) && Resize Cover (1, 2, 1) && ScrambleEbook (0, 6, 0) && Skoob Books (1, 5, 5) && Smashwords Metadata (1, 0, 2) QPA platform: xcb devicePixelRatio: 1.0 logicalDpi: 145.0 x 145.0 physicalDpi: 162.56 x 161.36470588235292 ApplicationPaletteChange event ignored Using calibre Qt style: True [0.00] Starting up... [0.00] Initializing db... [1.38] db initialized [1.38] Constructing main UI... DEBUG: 0.0 Annotations plugin: Error loading libiMobileDevice. This hasn't worked for a while, and is blacklisted in calibre v3. DEBUG: 0.0 Annotations plugin: Error is: No module named 'calibre.devices.idevice' DEBUG: 0.0 Annotations::reader_app_support.py - loading translations DEBUG: 0.0 Annotations::common_utils.py - loading translations DEBUG: 0.0 Annotations::appearance.py - loading translations DEBUG: 0.0 Annotations::config.py - loading translations DEBUG: 0.0 Annotations::annotated_books.py - loading translations DEBUG: 0.1 Annotations::action.py - loading translations DEBUG: 0.0 Kobo Utilities Configuration - CreateNewCustomColumn is supported DEBUG: 0.0 KoboUtilites::action.py - loading translations DEBUG: 0.0 KoboUtilites::dialogs.py - loading translations DEBUG: 0.0 KoboUtilites::action.py - loading translations DEBUG: 0.0 obok::utilities.py - loading translations DEBUG: 0.0 obok::dialogs.py - loading translations DEBUG: 0.0 obok::config.py - loading translations DEBUG: 0.0 obok::action_err.py - loading translations qt.webengine.QWebEngineView OK qt.gui.imageio: libpng warning: iCCP: known incorrect sRGB profile qt.gui.imageio: libpng warning: iCCP: cHRM chunk does not match sRGB qt.gui.imageio: libpng warning: iCCP: known incorrect sRGB profile [3.87] GUI main window shown Key 'Ctrl+Shift+N' for shortcut Browse notes is already used by Fetch news, ignoring EpubMerge: DEBUG: 2024-08-08 17:41:05,318: calibre_plugins.epubmerge.epubmerge_plugin(156): macmenuhack file_path:/home/mikew/.config/calibre/plugins/fanficfare_macmenuhack.txt Key 'Ctrl+Shift+N' for shortcut Browse notes is already used by Fetch news, ignoring DEBUG: 1.9 No Kobo Touch, Glo or Mini appears to be connected DEBUG: 1.9 rebuild_menus - self.supports_ratings=None, self.supports_tiles=None DEBUG: 1.9 KoboUtilities:set_toolbar_button_tooltip - start: text='None' DEBUG: 1.9 KoboUtilities:set_toolbar_button_tooltip - setting to text='Utilities to use with Kobo ereaders Driver: KoboTouch' Key 'Ctrl+Shift+N' for shortcut Browse notes is already used by Fetch news, ignoring Key 'Ctrl+Shift+N' for shortcut Browse notes is already used by Fetch news, ignoring Key 'Ctrl+Shift+N' for shortcut Browse notes is already used by Fetch news, ignoring [3.97] main UI initialized... [3.97] Started up in 3.97 seconds with 7924 books Code:
Key 'Ctrl+Shift+N' for shortcut Browse notes is already used by Fetch news, ignoring |
![]() |
![]() |
![]() |
|
![]() |
||||
Thread | Thread Starter | Forum | Replies | Last Post |
That's a lot of books! | wodin | Lounge | 0 | 03-29-2017 07:19 PM |
Export/Import all Calibre data - import error | IanKBrown | Calibre | 5 | 02-15-2017 08:32 PM |
Import on Filename changes mid import due to book error | Vortex | Calibre | 3 | 12-05-2014 11:49 AM |
Best way to automatically add a lot of books | foxbooks | Library Management | 3 | 05-31-2012 10:41 AM |
Import failed Error:404 when attempting to import from Calibre to Stanza | dvond | Apple Devices | 0 | 05-13-2011 03:00 PM |