backup

Moogwi · 08-03-2024, 05:38 PM

Hi,

My Calibre library size like 100 Go (75k of .epub)

but it takes too long to backup:

my config:

OS: Debian GNU/Linux 12 (bookworm) x86_64
Host: Mini IT13
Kernel: 6.9.12-1-liquorix-amd64
CPU: 13th Gen Intel i9-13900H (20) @ 3.001GHz
GPU: Intel Raptor Lake-P [Iris Xe Graphics]
Memory: 3346MiB / 31858MiB

I'm trying to backup on SSD NTFS

Sirtel · 08-03-2024, 05:57 PM

Export/Import is meant for data transfer between two computers, not for backup. For backup just copy both the library folder and the configuration folder, and paste them to your backup drive. Going forward, you can make incremental backups (updating only the files that have been changed) via FreeFileSync, using its mirror (one-way) option.

theducks · 08-03-2024, 06:02 PM

Calibre Export creates a custom Spanned Zip
100G is a huge amount of data to process and compress.

The old way was to manually backup the ENTIRE CONFIGURATION FOLDER and Library folder(s)
Export just keeps all the pieces together. Including multiple Libraries if Calibre is configured that way

Another common way is to use something like FreeFileSync IN MIRROR MODE ONLY
After the initial run, it does an incremental (just the changed pieces)

Sirtel · 08-03-2024, 06:04 PM

Given how many posts we've had here complaining about not being able to restore their libraries from the export/import files, I would not use that method for backup.

Moogwi · 08-03-2024, 06:27 PM

Quote:

Originally Posted by Sirtel

Export/Import is meant for data transfer between two computers, not for backup. For backup just copy both the library folder and the configuration folder, and paste them to your backup drive. Going forward, you can make incremental backups (updating only the files that have been changed) via FreeFileSync, using its mirror (one-way) option.

Thx !

Moogwi · 08-03-2024, 06:28 PM

Quote:

Originally Posted by Sirtel

Export/Import is meant for data transfer between two computers, not for backup. For backup just copy both the library folder and the configuration folder, and paste them to your backup drive. Going forward, you can make incremental backups (updating only the files that have been changed) via FreeFileSync, using its mirror (one-way) option.

Thank you for the tip

JSWolf · 08-03-2024, 06:43 PM

Quote:

Originally Posted by Moogwi

Hi,

My Calibre library size like 100 Go (75k of .epub)

but it takes too long to backup:

my config:

OS: Debian GNU/Linux 12 (bookworm) x86_64
Host: Mini IT13
Kernel: 6.9.12-1-liquorix-amd64
CPU: 13th Gen Intel i9-13900H (20) @ 3.001GHz
GPU: Intel Raptor Lake-P [Iris Xe Graphics]
Memory: 3346MiB / 31858MiB

I'm trying to backup on SSD NTFS

Use the program FreeFileSync to sync your backup. The first time It will take the full time to backup. But after that it will only backup what's new or changed since the last sync.

Also, you can delete the export files as you won't need them.

https://freefilesync.org/download.php

Moogwi · 08-03-2024, 07:36 PM

Quote:

Originally Posted by JSWolf

Use the program FreeFileSync to sync your backup. The first time It will take the full time to backup. But after that it will only backup what's new or changed since the last sync.

Also, you can delete the export files as you won't need them.

https://freefilesync.org/download.php

Thank you ! I appreciate ^^

BetterRed · 08-03-2024, 09:06 PM

Quote:

Originally Posted by theducks

The old way was to manually backup the ENTIRE CONFIGURATION FOLDER and Library folder(s)

I've been using file sync tools for data backup for almost 30 years, I switched to GoodSync in 2007. My calibre data (libraries, configuration, portable editions, etc) get backed up because of where they are, not because of what they are.

BR

Deskisamess · 08-24-2024, 03:25 PM

Can someone clarify about the "configuration" folder? I don't seem to be able to find it.

Sirtel · 08-24-2024, 03:40 PM

Quote:

Originally Posted by Deskisamess

Can someone clarify about the "configuration" folder? I don't seem to be able to find it.

On Windows, it's usually in User/AppData/Roaming - the entire folder named calibre. If you're not on Windows, click Preferences -> Miscellaneous -> Open calibre configuration folder, to see where it is.

Deskisamess · 08-24-2024, 03:54 PM

Thanks, found it. There is also a "Calibre-ebook.com" folder.

I needed a current backup of my books, and in the past used the Export option.

My use of Windows file structure is very rusty, and it seems more difficult than it was in the past.

Sirtel · 08-24-2024, 04:04 PM

Quote:

Originally Posted by Deskisamess

Thanks, found it. There is also a "Calibre-ebook.com" folder.

I needed a current backup of my books, and in the past used the Export option.

My use of Windows file structure is very rusty, and it seems more difficult than it was in the past.

The config folder is the one named calibre. I don't know what the other one is, something to do with the viewer when used separately, perhaps?

BetterRed · 08-24-2024, 08:04 PM

Quote:

Originally Posted by Sirtel

On Windows, it's usually in User/AppData/Roaming - the entire folder named calibre. If you're not on Windows, click Preferences -> Miscellaneous -> Open calibre configuration folder, to see where it is.

The "…->Open calibre configuration folder" button is available in all versions of Calibre, irrespective of OS.

Because the configuration folder can be relocated via the CALIBRE_CONFIG_DIRECTORY environment variable, I always tell people to use the button. I keep mine configuration folder adjacent to my calibre libraries and related folders and files.

Quote:

Originally Posted by Sirtel

The config folder is the one named calibre. I don't know what the other one is, something to do with the viewer when used separately, perhaps?

Yes, calibre-ebook.com is related to the Viewer, but it's created irrespective of how the viewer is started, from the library manager or directly. IIRC one of the Qt components creates it.

BR

Ghostcat · 08-25-2024, 03:07 AM

As a retired IT professional, I am reluctant to rely purely on increment backups. It is far too easy to propagate a, undetected, error. Then you're up the creak without a paddle.

Personally, while I use rsync as my main backup strategy, I also use Calibre's Export to do a full backup once a month. (My library is small enough that I can do this in a reasonable time.) Copies of this backup are keep on: the original machine, a USB stick and in the cloud (as an offsite backup).

Just because I'm paranoid doesn't mean Murphy isn't out to get me,

As to the OP; have your tried running TAR on your library directory?

08-03-2024, 05:38 PM	#1
Moogwi Junior Member Posts: 7 Karma: 10 Join Date: Sep 2023 Device: Kobo Libra H2O	backup Hi, My Calibre library size like 100 Go (75k of .epub) but it takes too long to backup: my config: OS: Debian GNU/Linux 12 (bookworm) x86_64 Host: Mini IT13 Kernel: 6.9.12-1-liquorix-amd64 CPU: 13th Gen Intel i9-13900H (20) @ 3.001GHz GPU: Intel Raptor Lake-P [Iris Xe Graphics] Memory: 3346MiB / 31858MiB I'm trying to backup on SSD NTFS

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
How and what to backup	HibDelyatt	Calibre	7	12-18-2021 09:30 AM
Not enough space to export/backup library. How to do partial backup?	droopy	Calibre	68	02-10-2021 04:00 AM
How do I backup CC?	HarryT	Calibre Companion	3	07-07-2014 11:55 AM
Backup	transmitthis	Calibre	2	02-11-2012 02:22 PM
Backup - how???	vxf	Sony Reader	0	06-23-2010 12:22 PM

08-03-2024, 05:57 PM	#2
Sirtel Grand Sorcerer Posts: 11,449 Karma: 230178176 Join Date: Jan 2014 Location: Estonia Device: Kobo Sage & Libra 2	Export/Import is meant for data transfer between two computers, not for backup. For backup just copy both the library folder and the configuration folder, and paste them to your backup drive. Going forward, you can make incremental backups (updating only the files that have been changed) via FreeFileSync, using its mirror (one-way) option.

08-03-2024, 06:02 PM	#3
theducks Well trained by Cats Posts: 30,441 Karma: 58055868 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	Calibre Export creates a custom Spanned Zip 100G is a huge amount of data to process and compress. The old way was to manually backup the ENTIRE CONFIGURATION FOLDER and Library folder(s) Export just keeps all the pieces together. Including multiple Libraries if Calibre is configured that way Another common way is to use something like FreeFileSync IN MIRROR MODE ONLY After the initial run, it does an incremental (just the changed pieces)

08-03-2024, 06:04 PM	#4
Sirtel Grand Sorcerer Posts: 11,449 Karma: 230178176 Join Date: Jan 2014 Location: Estonia Device: Kobo Sage & Libra 2	Given how many posts we've had here complaining about not being able to restore their libraries from the export/import files, I would not use that method for backup.

08-24-2024, 03:25 PM	#10
Deskisamess Wizard Posts: 2,681 Karma: 43102409 Join Date: Sep 2012 Location: Ohio Device: iPhone 7+, iPad mini, 2021 iPad Pro 12.9",Paperwhite 6.8"	Can someone clarify about the "configuration" folder? I don't seem to be able to find it.

08-24-2024, 03:54 PM	#12
Deskisamess Wizard Posts: 2,681 Karma: 43102409 Join Date: Sep 2012 Location: Ohio Device: iPhone 7+, iPad mini, 2021 iPad Pro 12.9",Paperwhite 6.8"	Thanks, found it. There is also a "Calibre-ebook.com" folder. I needed a current backup of my books, and in the past used the Export option. My use of Windows file structure is very rusty, and it seems more difficult than it was in the past.

08-25-2024, 03:07 AM	#15
Ghostcat Connoisseur Posts: 54 Karma: 582370 Join Date: Apr 2023 Device: Kobo Clara 2E	As a retired IT professional, I am reluctant to rely purely on increment backups. It is far too easy to propagate a, undetected, error. Then you're up the creak without a paddle. Personally, while I use rsync as my main backup strategy, I also use Calibre's Export to do a full backup once a month. (My library is small enough that I can do this in a reasonable time.) Copies of this backup are keep on: the original machine, a USB stick and in the cloud (as an offsite backup). Just because I'm paranoid doesn't mean Murphy isn't out to get me, As to the OP; have your tried running TAR on your library directory?

Advert

Advert