Tips for loading a large library onto a KO2

[HarryT]

I bought my KO2 mainly so I could have a larger proportion of my Ebook library (which is in Calibre) on it. I’ve learnt a few things over the last 48h which may be of interest to others, so I thought I’d share them.

Firstly, you need to know how to check if the Kindle is currently busy indexing books. To do this, tap the “search” box from the library screen, and enter a nonsense search string. When the results screen appears (which will show zero matches if you’ve entered a random nonsense search string) if “Text in Books” is highlighted, tap it, and it will say “NNN books not yet indexed” (should that be the case). Tap that line, and it’ll show you what the books are.

1. The Kindle’s behaviour is erratic while it’s indexing. Tap a book in the library, and it may take a very long time to open, or the entire device may appear to hang up. That’s because the CPU is busy indexing. If it does appear to have hung up, press the power button, wait about a minute, then press it again. That should bring things back to life again. Because of this, I’d strongly suggest waiting for the Kindle to finish indexing before using it.

2. Indexing is CPU-intensive, and hence power-hungry. When indexing, the Kindle will probably drain the battery in well under a day. Leave it connected to a wall-charger while it’s indexing.

3. The Kindle can index approximately 100 books an hour (depending on their size). Do as I (foolishly) did and load 2000 books at once, and the Kindle will be tied up for around 24h indexing them! It’s a lot better to load batches of around 100 books at a time and wait for each batch to finish indexing before loading the next batch. Another reason to do this is...

4. The indexing process can very occasionally hang up (it’s not happened to me on the KO2, but it has on earlier Kindles). If the number of books remaining to be indexed freezes for an hour or more, try rebooting the Kindle. If that still doesn’t free up the indexer, you’ll need to delete the books that are awaiting indexing and re-add them. That’s why it’s advisable to add your books in fairly small batches, and also add them in identifiable batches (eg authors starting with a certain letter), so you’ll know what to delete! Before panicking, though, note that very large books (such as some of the enormous Delphi collections) can take a very long time to index. If your recent batch of books contains a 50MB Delphi omnibus, don’t worry if the indexer appears to freeze for half an hour - it’s just processing that huge book.

5. The more books you have on your Kindle, the longer newly-added books will take to index. I’ve found that with 3000 books on my KO2, indexing is about 50% slower than it was when the device was empty.

After two days, I’ve loaded authors with surnames up to the letter “I” on my Kindle (some 3500 books), so it’s probably going to take another 3 or 4 days to finish loading my library.

The good news is that the Kindle performs very well with large numbers of books in the library (provided it’s finished indexing them!). Nothing obvious slows down in normal use. Searches are very quick, and books open virtually instantly.

Based on what I’ve loaded so far, a rough calculation suggests that it would probably be possible to store around 13-14000 books on the 32GB KO2. A significant proportion of my books, though, are (very) large omnibus editions, which pushes my average book size up a lot. If you read short novels, you could probably fit 20,000+ on!

Hope that’s of some interest to others.

[93terp]

Great info, thanks Harry!

[selianth]

Originally Posted by HarryT

3. The Kindle can index approximately 100 books an hour (depending on their size). Do as I (foolishly) did and load 2000 books at once, and the Kindle will be tied up for around 24h indexing them! It’s a lot better to load batches of around 100 books at a time and wait for each batch to finish indexing before loading the next batch. Another reason to do this is...

I was adding books 10 at a time this afternoon and having to wait about 20 minutes between batches for the indexing to finish. These were all azw3, with embedded fonts but otherwise unremarkable. (300-page thrillers for example.) I only have 300 books on the Kindle so it's not that there's a massive index already. So, I'd say "ymmv" on this point.

[FizzyWater]

This is good to know. I also had hoped to load a larger number of books on mine.

[HarryT]

Whichever figure one finds to be correct, the fundamental point is that it's a slow process!

[vrf]

Not that I'll every need to load this many books on, but once the indexing is complete for a massive library, is there any effect on the responsiveness of the device in day to day use? I suppose, at the very least, searching for a book would be slower.

[HarryT]

Originally Posted by vrf

Not that I'll every need to load this many books on, but once the indexing is complete for a massive library, is there any effect on the responsiveness of the device in day to day use? I suppose, at the very least, searching for a book would be slower.

I specifically address this point in the ante penultimate paragraph of my original post. Did you take the time to read it?

[vrf]

Yeah, caught me red-handed. I'm a skimmer. Thanks for the report.

Seems I've heard of other users over the years say that operation does slow down with large libraries, but maybe the newer processor is able to keep in front of that many files.

[Sirtel]

Originally Posted by vrf

Yeah, caught me red-handed. I'm a skimmer. Thanks for the report.

Seems I've heard of other users over the years say that operation does slow down with large libraries, but maybe the newer processor is able to keep in front of that many files.

I have over a thousand books on both my Voyage and the old Oasis. The only thing that's slowed down is adding or removing books into/from collections. I've not noticed any particular slowness elsewhere (searching included).

Now my old basic non-touch Kindles (from 2011 and 2012) did slow down considerably with several hundred books on them.

[Barty]

Amazon needs to address the hang while indexing problem. The only reason I would buy the 32gb is to dump my entire library in one go. If I have to dribble it out like this I'm not going to bother.

My habit is to keep a few books I'm reading on the kindle. After a while I accumulate about 100 on the device, but that's about it. Occasionally, in a fit of enthusiasm I may load 5, 10 at a time, and it's a pain when the indexing hangs and I have to delete and load one at a time to see which the offending book is.

The software needs to monitor the process and if it determines the indexing on a book isn't progressing as expected to put it at the end of queue and move on. It should show you the book(s) being indexed and % completed and let you stop indexing on some books and maybe just let them go unindexed.

[mathil]

This does not have to do with the KO2 specifically, but seeing as it's being discussed here - I know the device needs to be in sleep mode for the books to be processed, but does it also need to be charging?

I had a very vague idea of the Kindle indexing process before a few days ago (I own a PW3 but don't use it much, 95% of my library is in my Glo HD), when I read something similar to the post above in one of the new Oasis' threads. I'd decided to send a portion of my library (400 books, give or take) to my PW3 a couple of weeks ago as I thought it would be reasonable to do so in case I couldn't use my Kobo for some reason, and when I found out about this indexing thing I searched for the random string as advised, and found out that none of the books I'd loaded weeks before had been indexed. The PW3 had been in sleep mode for the vast majority of that time, excluding the minutes I took for adding the new books to collections etc. (where the device *did* seem to be less reactive than usual...). When I connected the PW to the charger and put it to sleep, I noticed it started indexing books, and finished some hours later.
This is a bummer, to be honest. I don't mean to start a Kobo/Kindle comparison as I know both devices have their own good and bad sides, but I don't understand why this resource-heavy and time-consuming process has to be mostly hidden from the user when it noticeably impacts the device's performance and battery life...

[HarryT]

Originally Posted by mathil

This does not have to do with the KO2 specifically, but seeing as it's being discussed here - I know the device needs to be in sleep mode for the books to be processed, but does it also need to be charging?

It needs to be neither asleep nor charging. It will index books while awake, and while not connected to a charger. However, it will index very, very slowly if you're reading at the same time. If you leave it, awake, though, on the "NNN books not yet indexed" page, it indexes at full speed - that's how I check on indexing progress.

[HarryT]

OK, it's now 1224 and I've just added another batch of 364 books (my "J" and "K" authors). Let's see how long these take to index. I'm leaving my Kindle connected to USB power, and sitting on the "Items Not Yet Indexed" page.

EDIT: 1h elapsed, and there are now 225 books left to index, so it's processed 139 books in an hour.

EDIT: 2h elapsed, and 111 books left to index, so 114 books processed in the second hour.

EDIT: 3h, and 47 books remaining, so only 64 books processed in this hour. That's due to processing some large omnibus editions, such as the Delphi complete works of M.R. James.

EDIT: 3h 20m and we're stuck on 46 books to go. This is almost certainly because of the huge 33MB Delphi "Complete Works of Henry James" omnibus, which is taking ages to process.

EDIT: 3h 30m and we're still stuck on 46. Hope it hasn't hung up! .

EDIT: 3h 35m and we're on the move again. It was the Henry James omnibus (it's now disappeared from the list of unindexed books). Show what an impact very large books can have - that one book took half an hour to index.

EDIT: Finished at 1620, so that took 3h 56m to index 364 books, or an average of one and a half books a minute. That included several extremely large books.

[John Hopper]

Originally Posted by HarryT

OK, it's now 1224 and I've just added another batch of 364 books (my "J" and "K" authors). Let's see how long these take to index. I'm leaving my Kindle connected to USB power, and sitting on the "Items Not Yet Indexed" page.

EDIT: 1h elapsed, and there are now 225 books left to index, so it's processed 139 books in an hour.

EDIT: 2h elapsed, and 111 books left to index, so 114 books processed in the second hour.

EDIT: 3h, and 47 books remaining, so only 64 books processed in this hour. That's due to processing some large omnibus editions, such as the Delphi complete works of M.R. James.

EDIT: 3h 20m and we're stuck on 46 books to go. This is almost certainly because of the huge 33MB Delphi "Complete Works of Henry James" omnibus, which is taking ages to process.

EDIT: 3h 30m and we're still stuck on 46. Hope it hasn't hung up! .

EDIT: 3h 35m and we're on the move again. It was the Henry James omnibus (it's now disappeared from the list of unindexed books). Show what an impact very large books can have - that one book took half an hour to index.

EDIT: Finished at 1620, so that took 3h 56m to index 364 books, or an average of one and a half books a minute. That included several extremely large books.

That's a really helpful post, Harry. I finished downloading my 2351 books on my KO2 last night after three days. Before I plugged it in overnight there were 570 books to be indexed, and only down to 566 books in the morning despite charging all night. But then it went down more quickly. It's now on 420 books.

[HarryT]

Originally Posted by John Hopper

That's a really helpful post, Harry. I finished downloading my 2351 books on my KO2 last night after three days. Before I plugged it in overnight there were 570 books to be indexed, and only down to 566 books in the morning despite charging all night. But then it went down more quickly. It's now on 420 books.

I've found that it seems to go a lot quicker if you keep the Kindle awake. It seems to slow to a crawl when the Kindle goes to sleep. When I was doing the batch of books in my previous post, I had the Kindle plugged into my laptop's USB port, but "ejected" (ie it was powered, but not in USB drive mode), and I refreshed the page every 5 minutes to keep it awake.