How to export indexes from indesign to epub

[Ryn]

Originally Posted by KevinH

I see you found your solution but for the record there are ps2txt and ps2ascii you can use to display these as well as this useful article I used in the past:

https://www.cs.waikato.ac.nz/~ihw/pa...tract-Text.pdf

It prepends a short and sweet extra postscript function to the original postscript which redefines the show methods to give you text output that would be easier to parse.

FWIW, I find working with "ps printer device" can extract text electronically that is very hard to get to in other ways without a scanner.

Thanks for this. As the subject of postscript is entirely new to me, I am saving this for a moment when I have the opportunity to delve deeper into it.

[DiapDealer]

I've always felt that traditional indices in ebooks were a bit pointless. Anachronistically so. If print book makers could have made pages automatically turn and words on those pages to glow simply by saying a word aloud, then they'd have done so, and print indices would probably never have become a thing. And we wouldn't now be seeing people being forced by client dollars to try to simulate what a simple search engine can do with hypertext markup and millions of hardcoded links to and fro.

But I digress.

P.S. I've heard the "if you don't know what you need to search for" argument before, and I don't quite buy it. People who have no idea what they're looking for typically aren't looking for anything. And even if they were, manually wading through enormous, alphabetized, electronic indices is unlikely to focus their efforts very much.

[Ryn]

Originally Posted by DiapDealer

P.S. I've heard the "if you don't know what you need to search for" argument before, and I don't quite buy it. People who have no idea what they're looking for typically aren't looking for anything. And even if they were, manually wading through enormous, alphabetized, electronic indices is unlikely to focus their efforts very much.

I think this is a limited way of perceiving the breadth with which readers engage with books. From personal experience, I can recall spending days fascinated with the mysteries of an encyclopedia. I wasn't looking for anything in particular, but was simply curious about learning new things.

In a way, I was plumbing new depths of understanding of the world I was born into. Not by the focused act of searching, but by discovery.

I think some - not many, but some - books lend themselves to that wideranging form of learning. I think for instance, that people who are strongly motivated to deepen their understanding of their religion are much helped by guidance in the form of an index, or similar means.

[DiapDealer]

Originally Posted by Ryn

I think this is a limited way of perceiving the breadth with which readers engage with books.

No. My eyes are wide open.

Originally Posted by Ryn

From personal experience, I can recall spending days fascinated with the mysteries of an encyclopedia. I wasn't looking for anything in particular, but was simply curious about learning new things.

Same. Most of my childhood was spent in front of a massive set of the World Book Encyclopedia (complete with the Childcraft addons and annual supplements). I read them all from cover to cover (at random) because I was incredibly curious about learning everthing. Indices didn't enter into that picture, or that joy I experienced. By the time I was old enough to start needing to use an index, I would have killed for a search engine to focus my efforts. Now that ebooks are here (complete with search engines), I have zero interest in using someone else's curated, compiled points of interest in electronic indexes. I never don't know what I want to search for in a particular book anymore. Because I'm rarely searching for anything in a book I've not already read from cover to cover.

The fact of the matter is: only print index lovers love electronic indices. *shrug*

[DiapDealer]

But I'm not here to discourage anyone from their electronic "indexical" pursuits. Just rambling, really. They're no skin off my back.

[Ryn]

Originally Posted by DiapDealer

No. My eyes are wide open.

Same. Most of my childhood was spent in front of a massive set of the World Book Encyclopedia (complete with the Childcraft addons and annual supplements). I read them all from cover to cover (at random) because I was incredibly curious about learning everthing. Indices didn't enter into that picture, or that joy I experienced. By the time I was old enough to start needing to use an index, I would have killed for a search engine to focus my efforts. Now that ebooks are here (complete with search engines), I have zero interest in using someone else's curated, compiled points of interest in electronic indexes. I never don't know what I want to search for in a particular book anymore. Because I'm rarely searching for anything in a book I've not already read from cover to cover.

The fact of the matter is: only print index lovers love electronic indices. *shrug*

I'm just saying that the index, especially in encyclopedic works, can function just as the encyclopediae we both loved so much growing up.

Another example would be an atlas, which many people don't use to find stuff but rather to educate themselves, to wander, or to heuristically discover new territory. I have yet to meet an atlas I have memorized, and use google earth to much the same effect.

Granted, indexes are clunky, abstract, and often quite subjective. Still, in the correct contexts, be it technical works, religious tomes, or dictionaries (basically one immense index), they can be quite useful.

The fact that you "never don't know what [you] want to search for in a particular book anymore" really is neither here nor there.

[DiapDealer]

Originally Posted by Ryn

The fact that you "never don't know what [you] want to search for in a particular book anymore" really is neither here nor there.

Sure it is.

You may not agree with it, but it's just as valid and relevant as your assertion that electronic indices can't always be replaced with a search engine.

Originally Posted by Ryn

Granted, indexes are clunky, abstract, and often quite subjective.

This was the better stopping point. It is my entire premise. The only thing missing is mention of the index's natural electronic successor: the search engine. Not everything needs to survive the medium shift. The joy and wonder of discovery and learning in the electronic era will easily survive the exclusion of what is, in essence, a vestigial print appendage.

[Hitch]

Originally Posted by DiapDealer

Sure it is.

You may not agree with it, but it's just as valid and relevant as your assertion that electronic indices can't always be replaced with a search engine.

Hitch hesitantly raises her hand to address the assembly...

Index nerd here! I love me some indices and EVEN WHEN it's in an ebook, where, indeedy, it's useless, I like the fact that I can assess the index and see how many references to topic A were worth mentioning in the index; how many references to John Doe and so forth.

This was the better stopping point. It is my entire premise. The only thing missing is mention of the index's natural electronic successor: the search engine. Not everything needs to survive the medium shift. The joy and wonder of discovery and learning in the electronic era will easily survive the exclusion of what is, in essence, a vestigial print appendage.

Yes, but as we all know, that's not an index, it's a concordance. Not the same thing. I would be the first to agree that by and large, including RPN-driven indices is worthless, but a search engine is not necessarily the natural electronic successor to an index; it's the successor to a concordance and as anyone who's tried to find something on Google knows, more isn't always better.

Hitch

[DiapDealer]

Originally Posted by Hitch

Yes, but as we all know, that's not an index, it's a concordance. Not the same thing.

Quite debatable whether electronic search engine = print concordance. I would argue that the ability to search the entire text of a work has no print-based counterpart. But the ability to search a book's text certainly replaces the necessity for either a concordance or an index, in my opinion (and in my experience, since I've never once looked at, clicked on, or otherwise engaged with an electronic index or concordance). That's why searching is such a game changer in the P2E medium shift.

Originally Posted by Hitch

I would be the first to agree that by and large, including RPN-driven indices is worthless,

Ding, ding ding.

Originally Posted by Hitch

but a search engine is not necessarily the natural electronic successor to an index; it's the successor to a concordance

Ok. Perhaps successor was a poor choice of words. Suffice it to say that it is my opinion that a search engine eliminates the need for either (again: in my experience).

Originally Posted by Hitch

and as anyone who's tried to find something on Google knows, more isn't always better.

Nope. Can't agree there. I'll take way too much information over someone else's notions of what they think I should be focusing on any day of the week. I'm not one who really believes (or succumbs to) information overload. Give me raw, unadulterated, and uncurated search results (whether online, or limited to the text of an ebook) and I'll take it from there--thank you very much.

Another part of my problem with electronic indices and concordances stems from the fact that their entire reason for being has been changed entirely in the electronic medium shift. They went from from being purely reference-based, to purely navigation-based. Navigation aids I don't need. Page-turns and searching suffice.

[Tex2002ans]

Originally Posted by Hitch

Oh, you are being a [something or other]. You know I do, in fact, read all your stuff. (Go ahead, I defy you to find other human beings that do!). I just don't always remember every single thing. And...well, yes, I may on occasion skim. It's not like you write 300-word Blogger posts, now, is it. You're like the Anti-Twitter.

Originally Posted by Hitch

And there it is. :-) That stuff is just bloody tedious. I think it would be fun to write programming or clips, etc., to do it...but HAVING to do it, commercially, is the dog's south end.

Only had to fix a bajillion of those rotten indexes that someone else created (and that was bad enough!).

If it's a project I'm working on from scratch, I insist on unlinked indexes.

Originally Posted by Ryn

Laura mentioned another script that might actually serve my purpose even better: LiveIndex, found here: https://www.id-extras.com/products/liveindex/

I'm mentioning it, just in case anyone else ever comes across a similar use case.

Nice. I'll add it to my list.

Originally Posted by Ryn

This passive searching allows for a deeper sense of discovery in books that are more encyclopedic in scope.

Not relevant to the vast majority of books that reaches our devices, I would be the first to agree, but in some cases, very much a desirable addition.

Yep. An Index also lets you find more "broad" concepts, not necessarily worded in the raw text itself.

Take this for example:

famous philosophers
	Aquinas, 10
	Aristotle, 1, 5, 60, 199
	Socrates, 20, 60

You might search for the word "philosopher", then have to sift through 100 (irrelevant) "philosopher" hits. And within the text, "Aquinas", "Aristotle", or "Socrates" might not appear near the word "philosopher" at all.

Search (in ebooks) also doesn't typically match related words like: "philosophy" or "philosophies" or "philosophical".

A good Indexer would be able to pre-categorize + organize the information, throwing out a lot of the "irrelevant hits", while at the same time combining all those "related words" together.

And as Hitch said, you could use the index to get a very broad overview of WHAT information is covered in a given book. Even the size of the entries can tell you how "important" an author thinks a topic is. For example, the author may consider Aristotle to be more important than Aquinas (4 vs. 1).

Note: Me + Hitch (and others) discussed the pros/cons of Indexes/Search at extreme length in the 2016 "Sick of Amazon Kindle books without Page Numbers" thread.

Originally Posted by DiapDealer

Quite debatable whether electronic search engine = print concordance. I would argue that the ability to search the entire text of a work has no print-based counterpart. But the ability to search a book's text certainly replaces the necessity for either a concordance or an index, in my opinion (and in my experience, since I've never once looked at, clicked on, or otherwise engaged with an electronic index or concordance). That's why searching is such a game changer in the P2E medium shift.

Around the time of that famous 2016 thread, I read "How to Read a Book" by Mortimer Adler.

Absolutely fantastic title. When I first heard of it, I thought:

"Who the heck doesn't know how to read a book?"

Well, I didn't know... I didn't know... And it completely changed the way I read Non-Fiction + view Indexes.

Here's one blog article also discussing the book:

"How to Read a Book: The Ultimate Guide by Mortimer Adler"

* * *

And here's a relevant excerpt of Chapter 4, "The Second Level of Reading: Inspectional Reading":

Spoiler:

Inspectional Reading I: Systematic Skimming or Pre-reading

Let us return to the basic situation to which we have referred before. There is a book or other reading matter, and here is your mind. What is the first thing that you do?

[...] First, you do not know whether you want to read the book. You do not know whether it deserves an analytical reading. But you suspect that it does, or at least that it contains both information and insights that would be valuable to you if you could dig them out.

Second, let us assume-and this is very often the case*that you have only a limited time in which to find all this out.

In this case, what you must do is skim the book, or, as some prefer to say, pre-read it. Skimming or pre-reading is the first sublevel of inspectional reading. Your main aim is to discover whether the book requires a more careful reading. Secondly, skimming can tell you lots of other things about the book, even if you decide not to read it again with more care.

Giving a book this kind of quick once-over is a threshing process that helps you to separate the chaff from the real kernels of nourishment. You may discover that what you get from skimming is all the book is worth to you for the time being. It may never be worth more. But you will know at least what the author's main contention is, as well as what kind of book he has written, so the time you have spent looking through the book will not have been wasted.

[...]

2. STUDY THE TABLE OF CONTENTS to obtain a general sense of the book's structure; use it as you would a road map before taking a trip. It is astonishing how many people never even glance at a book's table of contents unless they wish to look something up in it. In fact, many authors spend a considerable amount of time in creating the table of contents, and it is sad, to think their efforts are often wasted.

It used to be a common practice, especially in expository works, but sometimes even in novels and poems, to write very full tables of contents, with the chapters or parts broken down into many subtitles indicative of the topics covered. Milton, for example, wrote more or less lengthy headings, or "Arguments," as he called them, for each book of Paradise Lost. Gibbon published his Decline and Fall of the Roman Empire with an extensive analytical table of contents for each chapter. Such summaries are no longer common, although occasionally you do still come across an analytical table of contents. One reason for the decline of the practice may be that people are not so likely to read tables of contents as they once were. Also, publishers have come to feel that a less revealing table of contents is more seductive than a completely frank and open one. Readers, they feel, will be attracted to a book with more or less mysterious chapter titles-they will want to read the book to find out what the chapters are about. Even so, a table of contents can be valuable, and you should read it carefully before going on to the rest of the book.

[...]

3. CHECK THE INDEX if the book has one-most expository works do. Make a quick estimate of the range of topics covered and of the kinds of books and authors referred to. When you see terms listed that seem crucial, look up at least some of the passages cited. (We will have much more to say about crucial terms in Part Two. Here you must make your judgment of their importance on the basis of your general sense of the book, as obtained from steps 1 and 2.) The passages you read may contain the crux-the point on which the book hinges-or the new departure which is the key to the author's approach and attitude.

As in the case of the table of contents, you might at this point check the index of this book. You will recognize as crucial some terms that have already been discussed. Can you identify, for example, by the number of references under them, any others that also seem important?

Even just skimming an Index (or well-designed Table of Contents) can give you lots of helpful information.

This is why I mostly don't mind leaving unlinked indexes in ebooks (they don't hurt, and can only help, even in ways that pure search can't accomplish).

[DiapDealer]

Second, let us assume-and this is very often the case*that you have only a limited time in which to find all this out.

Bad assumption. I haven't been in a situation where I have a limited amount of time to "find this all out" by reading a book in 30+ years. Fiction or non-fiction: I have all the time in the world to read anything I choose to read (and everything I read IS my choice for my own edification). So what need would I have of "skimming or pre-reading" techniques (with or without an index)? What need would I have of any kind of pre-reading preparation, period? RTFM.

We are in agreement, though: they don't hurt anything by being in the ebook (with or without links). I just think that when it comes to ebooks, the vast majority of those who truly appreciate a well-designed e-index are those who also create well designed e-indices. Heck, I think the same's probably true for print e-indices when it comes down to it. You're putting on a major production for the world's tiniest audience. If your time is truly money, then this is a good place to stop wasting it, IMO.

[Tex2002ans]

Originally Posted by DiapDealer

We are in agreement, though: they don't hurt anything by being in the ebook (with or without links). I just think that when it comes to ebooks, the vast majority of those who truly appreciate a well-designed e-index are those who also create well designed e-indices. Heck, I think the same's probably true for print e-indices when it comes down to it.

Agreed.

In my mind, I'm saving linkifying indexes as a "future-me" problem.

I've had the methods floating around in my head for years now, and I know all the individual steps work... it's just getting around to coding it up.

And as always, there's an XKCD for that:

https://xkcd.com/1205/

Is it worth the time for how often I do it? Not really.

But it'll happen eventually, and when it does, then I can automate through the entire batch of books.

And if you already have RPNs in there for Accessibility, then linkifying your Index shouldn't be that much more work. Just a lot of regex elbow grease!

Originally Posted by DiapDealer

You're putting on a major production for the world's tiniest audience. If your time is truly money, then this is a good place to stop wasting it, IMO.

Pretty agreed.

The benefits of a "linked index" over an "unlinked index"... not much.

The "unlinked index" is effectively equivalent to the print book.

The "linked index" is a slight enhancement, but its practicality is low (because screens =/= pages).

If the link pointed to the exact location, now THAT would be a meaningful enhancement... but the effort to create that is enormous (and 99% of indexes are created externally and given as a list of page numbers, not generated within the document itself).

And search (engines) get you to the exact location with zero extra effort for the bookmaker... but doesn't include the unique benefits of a curated index.

Complete Side Note: Algorithms + Natural Language Processing are also constantly getting better at determining keywords, plus detecting related words. Two interesting videos on this stuff:

• "Vectoring Words (Word Embeddings) - Computerphile"
• "GPT3: An Even Bigger Language Model - Computerphile"

This type of stuff is already being used in search engines, generating text, condensing news articles, telling "moods", detecting spam/malintent, and even generating categories, etc.

Perhaps there will be some future tech that can get fed an Index of entries+pages, then go backwards to linkify at the sentence/word-level.

Originally Posted by DiapDealer

Bad assumption. I haven't been in a situation where I have a limited amount of time to "find this all out" by reading a book in 30+ years. Fiction or non-fiction: I have all the time in the world to read anything I choose to read [...]

What? Everyone has limited time. 24 hours in a day, X amount of days before death.

Originally Posted by DiapDealer

So what need would I have of "skimming or pre-reading" techniques (with or without an index)? What need would I have of any kind of pre-reading preparation, period? RTFM.

Sounds to me like you "don't know how to read a book"! Maybe you should read that one!

Anyway, earlier in the book he lays out a lot of groundwork on:

• how to more effectively learn/absorb information
• goals of reading (entertainment/information/understanding)

Before even reading, you can quickly analyze the work at a very high/surface level, like looking at:

• the order of chapters.
  
  • Chronological or Topical?
    
  • Linear (like a Fiction book)?
    
  • Standalone chapters (like a manual)?
    
• how chapters are organized.
  
  • Parts, Chapters, Subchapters.
    
  • Wars > Battles > Skirmishes
• how many pages dedicated to each chapter/section/topic?

Prepping yourself for what's to come gives you:

• some "main/important topics" to focus on and where they're located
• potential questions to "ask" before you begin reading

so when you hit those points, you may absorb the information better by having this outline/framework in your mind ahead of time.

For example, let's say you have multiple history books about a civil war:

• One author may explain in purely chronological order (this lead to this lead to this).
• The other by main battles (here were the most important turning points).
• The other by topic (Economic, Social, Military).

Each method of organizing emphasizes different facets of the war:

• A purely chronological book will most likely cover broad strokes (and hop around).
  
  • Probably names major events, and focuses on general "feelings", but ignores a lot of little minutiae (because you have entire years to cover!).
• Main battles.
  
  • Which battles are included and not the others?
    
  • Are these chronological? Or ordered by "importance"?
    
    • Why was battle X more important than Y?
      
    • Did X cause Y? Or were they just occurring at the same time in different locations?
• Topical, you may want to focus on reading only one section of the book.
  
  • For example, I'm extremely knowledgeable in Economics, so as I skim through the headings/subheadings, I'll see what the historian believes were the important points. (Hint: Historians are generally pretty awful at economics! )

Instead of you hopping in completely blind, letting whatever random knowledge smack you in the face, hoping something sticks as the sentences whiz by... you can use some of these extra tools to enhance your reading... Indexes included!

So learn how to read a book!

[DiapDealer]

I admit I have a healthy skepticism/disdain for those who believe they can teach people to learn better/faster. For one, there's the old addage that "those who can't... teach." And two, my personal growth is not a race I'm trying to "win". Mainly because there's no finish line I'm trying to get to. I'm perfectly satisfied with my own techniques and progress.

I'm not a goal-oriented reader. I'm not trying to accomplish anything with books. Knowledge is just a side-effect of my hobby. Reading one after another (from preface to afterword) is the only plan I need. I need no help with that plan.

[DiapDealer]

But that's enough. This inefficient-reading slacker has better things to do than argue with goal-oriented readers.

[Hitch]

Originally Posted by Tex2002ans

Agreed.

In my mind, I'm saving linkifying indexes as a "future-me" problem.

I've had the methods floating around in my head for years now, and I know all the individual steps work... it's just getting around to coding it up.

What WOULD work for a "real" electronic linked index would be something that actually uses the tags--whether INDD or Word or Bob's Big Word Processor--and places a tag or anchor where the INDD mark is placed. That would work.

Then the reader could tap, read, and then, you'd have to be utterly reliant on the kindness of "back" functionality to take you back to the entry in the index where you were exploring. There wouldn't be any realistic, viable way to encode "back to index" functions--but I'm not sure, in 2020, that we need that. Even in 2009, the Kindle had "back" and so too does pretty much every device today.

Early in the biz, in 2010, say, we had to worry about back, but today, we really don't.

I mean...as a non-programmer and someone who has never gotten "inside" what INDD does (or Word) to that level of detail, when a spot is indicated/marked/tagged as "this index entry goes here," how hard would that be? Does anyone know? It can't simply tag the top of the "page," because that can move, as the file is revised. Forward, backward, whatever. So the mark must be at the location, not the page.

You'd think that somebody, anybody, would be working on this. (Now I'll find out from somebody here that this already exists. {sigh}).

Hitch