App with deep search functionality?

[BionicGecko]

Hi all,

I have an iPad which I currently use for reading PDFs or picture-heavy ePubs. One example of this is cookbooks; I have acquired dozens of cookbooks throughout the years, and I like using the iPad in the kitchen to display the recipe I'm currently cooking.

One functionality I'm missing however is deep search throughout all the books in the library; for instance, suppose I have an ingredient in the fridge I'd like to use, I would like to be able to search across all my cookbooks for mentions of this ingredient, ideally with some context (e.g. chapter name or a snippet of text surrounding the word).

So far all the applications I've tried (Kindle, Apple Books, Documents) will either search the library for book titles, or when inside a book, will search within the current book. I haven't found an app which will scan the contents of all the books within the library.

Are you guys aware of such an app? Ideally with support for both PDFs and ePubs?

I also recently ordered an Onyx Boox device, so Android apps are of interest to me as well.

Thanks!

[BionicGecko]

Answering (somewhat) my own question here, in case someone else is looking for the same thing.

So it seems that unfortunately no such reader exists for iOS (perhaps there is room in the market for an enterprising iOS app developer?) There is however a very workable solution for Android in the form of this app: Search in Books.

While not a reader per se, this little app does exactly what its name says, i.e. it searches across multiple (and individually selectable) ePubs and PDFs for specific words and provides a breakdown by book, with some textual context around each instance found. Individual search results can either be expanded for more context in plain text, or viewed in the original ePub of PDF formatting.

This might not be sufficient to everyone, as it is rather a search tool with integrated viewer than a full-fledged e-reader app; it is also an old, unsupported application which has to be sideloaded as it is not available on the Play Store. But it is perfect for my use case of searching and displaying recipes.

Hope it helps someone else!

[tomsem]

It seems odd that there seem to be no iOS apps that do this simple thing that even lowly Kindles do. I suppose it is just too easy to do in memory searches, and that adding a persistent index of all downloaded content seems like a lot of work that almost nobody would appreciate.

It would be relatively trivial to add book metadata to app index to make that content searchable with Spotlight but it would not solve this more general use case.

I think developers of BookFusion could justify adding such a feature, especially if it were tied to a paid subscription (which can be as little as $19/yr). They update the app frequently, and have an Android app too.

[BionicGecko]

Originally Posted by tomsem

I think developers of BookFusion could justify adding such a feature, especially if it were tied to a paid subscription (which can be as little as $19/yr). They update the app frequently, and have an Android app too.

This feature was already suggested to the BookFusion developers two years ago; it doesn’t seem like there is much interest in implementing it.

https://bookfusion.featureos.app/p/full-text-search

[tomsem]

Too bad, though it's not clear how to gauge interest from there. I would up vote it but it did not recognize the credentials I use for the app and I did not want to create another account just to comment, and then get confused as to which was which.

[Quoth]

Calibre has FTS (full Text Search). Ability for that to work with a PDF depends on the PDF. It beats the Kindle search. Mac/Windows/Linux.

[tomsem]

Originally Posted by Quoth

Calibre has FTS (full Text Search). Ability for that to work with a PDF depends on the PDF. It beats the Kindle search. Mac/Windows/Linux.

That's great but the actual use case is to do it with an iOS app.

[BionicGecko]

Originally Posted by Quoth

Calibre has FTS (full Text Search). Ability for that to work with a PDF depends on the PDF. It beats the Kindle search. Mac/Windows/Linux.

Originally Posted by tomsem

That's great but the actual use case is to do it with an iOS app.

Thank you Quoth for the suggestion. tomsem, you are right, however it turns out there is an interesting twist to this.

While I indeed thought at first that this was not a workable solution, after a bit of digging I discovered that the Calibre full-text search functionality mentioned by Quoth made its way into the Calibre Content Server in December last year. I never used the Calibre Content Server, but from my understanding it is essentially a web server making the contents of the whole Calibre library accessible to any web browser, e.g. Safari (or any third-party browser) on the iPad for my use case.

I didn't have time yet to install it on my NAS and test it, however this looks like a promising path to explore.

[tomsem]

Originally Posted by BionicGecko

Thank you Quoth for the suggestion. tomsem, you are right, however it turns out there is an interesting twist to this.

While I indeed thought at first that this was not a workable solution, after a bit of digging I discovered that the Calibre full-text search functionality mentioned by Quoth made its way into the Calibre Content Server in December last year. I never used the Calibre Content Server, but from my understanding it is essentially a web server making the contents of the whole Calibre library accessible to any web browser, e.g. Safari (or any third-party browser) on the iPad for my use case.

I didn't have time yet to install it on my NAS and test it, however this looks like a promising path to explore.

Yes, I've never bothered to make CCS accessible over the internet generally, and almost never have any reason to use it otherwise. And of course you need to have the host machine up and running calibre.

I would still want local, on-device content search of the content the app has currently downloaded.

[jdege]

I keep my ebooks in a Calibre repository on a Linux server.

I was faced with this problem, trying to remember which of the 60+ Percy Fitzhugh novels contained a comment I remembered.

But this was Linux. My epubs are really just zip files, so a command line using "find", "xargs", and "zipgrep" worked just fine.

[BionicGecko]

Originally Posted by tomsem

Yes, I've never bothered to make CCS accessible over the internet generally, and almost never have any reason to use it otherwise. And of course you need to have the host machine up and running calibre.

I would still want local, on-device content search of the content the app has currently downloaded.

Indeed nothing beats on-device full-text search, but I'll take the next best thing. One good thing about my use case is that I don't need to expose my library over the internet, only locally on my LAN - I rarely cook outside the house! But I agree with you, it is a very clunky solution.

Originally Posted by jdege

I keep my ebooks in a Calibre repository on a Linux server.

I was faced with this problem, trying to remember which of the 60+ Percy Fitzhugh novels contained a comment I remembered.

But this was Linux. My epubs are really just zip files, so a command line using "find", "xargs", and "zipgrep" worked just fine.

This could be an option as well, my "file server" at home is just a repurposed Raspberry Pi 3 running an OpenMediaVault (Linux) image. One key requirement for me however is to be able to quickly visualize the search results inside several books in a short timeframe; having to manually open each individual book and finding the relevant page (or searching again for the word inside the book) is too much hassle for me!

[BionicGecko]

Just a follow-up post on this topic, I finally got around to playing with the Calibre Content Server, and unfortunately it is not a workable solution for me.

The Full-Text Search functionality only returns a list of books containing the looked-up word, with context for the first instance of the word found in the book; it won't mention how many instances of the word are found in the book, or provide context for the other instances found.

Selecting a book will actually download the book to the browser cache, which is not an ideal place to store cookbooks as they can be very large; once the book is downloaded, epubs will open at the first instance of the word in that book, while PDFs are simply opened at the beginning of the book.

Unfortunately I returned the Android device I recently bought, so the "Search in Books" solution mentioned above is no longer available to me. I'm back to square one!

[skillachie]

Originally Posted by BionicGecko

This feature was already suggested to the BookFusion developers two years ago; it doesn’t seem like there is much interest in implementing it.

https://bookfusion.featureos.app/p/full-text-search

This is not correct, it is planned but we are working through the backlog and new features based on similar requests but prioritizing the ones that got ranked higher by readers. We already scoped how it would work from a UI/UX perspective, just a matter of getting to it.

To do full index of every page across all books in a readers library will use a very large index size. You can imagine it being significantly large for readers with 10k plus books. As a result it will be implemented in a tiered manner. But once we get to it we will reach back out. The last 2 years we have not been idle though, we had to write EPUB2/3 engines from the ground up, CBR/CBZ and other changes . https://www.reddit.com/r/BookFusion/...b-dcd3bcb0f500


You might also appreciate the image based highlights not possible across iOS , Android and Web. I assume you are reading for research or knowledge and so this feature will come in handy. We support 3 different platforms as well so it takes some time and we rely on support from the readers that pay to use the service.



In the interim we implemented full text search across all highlights for all books that is done on the dedicated highlights tab. In web it can be found at https://www.bookfusion.com/library/highlights

[skillachie]

Originally Posted by tomsem

It seems odd that there seem to be no iOS apps that do this simple thing that even lowly Kindles do. I suppose it is just too easy to do in memory searches, and that adding a persistent index of all downloaded content seems like a lot of work that almost nobody would appreciate.

It would be relatively trivial to add book metadata to app index to make that content searchable with Spotlight but it would not solve this more general use case.

I think developers of BookFusion could justify adding such a feature, especially if it were tied to a paid subscription (which can be as little as $19/yr). They update the app frequently, and have an Android app too.

Agreed. This is definitely planned and will be implemented. However, index size will be massive for some library sizes.

We already integrated book metadata for downloaded books into Spotlight. Details at the link. However, the data for each page of a library of unknown size, think libraries that have more than 30,000 books would not work for spotlight or even on device index. It would unfortunately crash at some point.

https://blog.bookfusion.com/bookfusi...-support-more/

Coming soon, just working through the features that got ranked higher based on readers input for now.

[BionicGecko]

Originally Posted by skillachie

This is not correct, it is planned but we are working through the backlog and new features based on similar requests but prioritizing the ones that got ranked higher by readers. We already scoped how it would work from a UI/UX perspective, just a matter of getting to it.

Apologies for misunderstanding the context of that thread on featureos.app; I'm glad to know you guys are working on it! Will keep my eyes peeled for the update

Originally Posted by skillachie

You might also appreciate the image based highlights not possible across iOS , Android and Web. I assume you are reading for research or knowledge and so this feature will come in handy. We support 3 different platforms as well so it takes some time and we rely on support from the readers that pay to use the service.

My use case for using FTS is rather modest to be honest, I'm not sure I would qualify my objective as "research or knowledge" but I'll certainly be happy to join the ranks of your paying customers when it is available!