more workers to speed up autoadd?

[jeffery]

is there a way to get more workers (calibre-parallel.exe) going when auto adding books?

Calibre is barely using the cpu, it's barely using the drive, and it's using very little memory. all good things. but I would like to speed up the adding of books.

I know I can drag-n-drop books but it doesn't like having too many given to it at one time this way. I have the options set to 12 workers for this. Calibre locks up and then I have to try the whole batch again because It doesn't remove the original files it added, no way to know which ones got added. so while this is faster, you have to tend to it the whole time, adding small batches of books at a time. not a real option when dealing with a couple hundred thousand books (been collecting a long time, but having to recreate library. I've learned my lesson and will be making a backup of the library when completed).

Auto-add works pretty good. but Calibre stops responding after a while, though it keeps adding the books, one at a time. it's a very slow process with only one entry being worked on at a time.

So I am wondering now, is it possible to make calibre use more workers to get the job done faster? I still have plenty of computer left for it to use.

6 core intel i7-6850k
32GB ram
512GB nvme m.2 drive
many Tb of storage
one worker process doing the job. it's like trying to empty the Sahara with a hand trowel, when there's a fleet of bucket loaders sitting idle behind you.

[kovidgoyal]

Auto add is not meant for bulk adding You want to add large numbers of books, right click the add books button and choose one fo the bulk adding options.

[kovidgoyal]

Although I have to say that tyring to add a couple of hunfred thousand books in one go on a windows machine is unlikely to work. Split it into batches of ~1000 you'll have the best throughput you are likely to get htat way.

[jeffery]

Originally Posted by kovidgoyal

Auto add is not meant for bulk adding You want to add large numbers of books, right click the add books button and choose one fo the bulk adding options.

Kovid, First let me say thank you. You've created a great program here. I can't imagine using anything else.

but, the add books button locks up for me as well pretty reliably. I find I have to limit to a few hundred books per attempt to keep that from happening.