MobileRead Forums
Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Editor
Reload this Page Find Repeated/Duplicate Text

Notices

Reply
Page 1 of 2 1 2
 
Thread Tools Search this Thread
Old 05-17-2024, 06:17 PM   #1
mikapanja
Perfectionist
mikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentameter
 
Posts: 72
Karma: 12802
Join Date: Apr 2014
Device: none
Find Repeated/Duplicate Text
Is there a way to find (and possibly highlight) repeated word groups in ePubs?
mikapanja is offline   Reply With Quote
Old 05-17-2024, 07:38 PM   #2
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 30,506
Karma: 58055868
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
try
(\w+)\s\1
theducks is offline   Reply With Quote
Advert
Old 05-17-2024, 08:26 PM   #3
mikapanja
Perfectionist
mikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentameter
 
Posts: 72
Karma: 12802
Join Date: Apr 2014
Device: none
Quote:
Originally Posted by theducks View Post
try
(\w+)\s\1
No go. It only finds adjacent words of which the first one ends in and the second one starts with one or more identical letters, e.g. go on or he headed back...
mikapanja is offline   Reply With Quote
Old 05-17-2024, 08:58 PM   #4
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 30,506
Karma: 58055868
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
Quote:
Originally Posted by mikapanja View Post
No go. It only finds adjacent words of which the first one ends in and the second one starts with one or more identical letters, e.g. go on or he headed back...
a space after should stop that. I added a starting space required as well (this won't find if the first and second word are the match
Code:
\s(\w+)\s\1\s
theducks is offline   Reply With Quote
Old 05-17-2024, 09:19 PM   #5
mikapanja
Perfectionist
mikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentameter
 
Posts: 72
Karma: 12802
Join Date: Apr 2014
Device: none
Quote:
Originally Posted by theducks View Post
a space after should stop that. I added a starting space required as well (this won't find if the first and second word are the match
Code:
\s(\w+)\s\1\s
This one finds identical words adjacent to one another, e.g. my my. I'd like it to find non-adjacent repeated word groups, i.e scattered throughout the text.

Sorry if I was unclear.
mikapanja is offline   Reply With Quote
Advert
Old 05-17-2024, 09:37 PM   #6
BetterRed
null operator (he/him)
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 21,111
Karma: 27620706
Join Date: Mar 2012
Location: Sydney Australia
Device: none
@mikapanja - Do you want to find and highlight all occurrences of let's say: "cathode ray tube" ?

BR
Last edited by BetterRed; 05-17-2024 at 09:52 PM.
BetterRed is offline   Reply With Quote
Old 05-17-2024, 09:42 PM   #7
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 30,506
Karma: 58055868
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
Actually, Spell check can do that.
remove the tick for only misspelled words
Double click on the word (left side),
repeat it (the d-click) jumps to the next occurrence
theducks is offline   Reply With Quote
Old 05-17-2024, 10:38 PM   #8
mikapanja
Perfectionist
mikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentameter
 
Posts: 72
Karma: 12802
Join Date: Apr 2014
Device: none
Quote:
Originally Posted by BetterRed View Post
@mikapanja - Do you want to find and highlight all occurrences of let's say: "cathode ray tube" ?

BR
Exactly, BR, word groups, not words. Hence the spell check, as suggested by theducks, won't work.
mikapanja is offline   Reply With Quote
Old 05-17-2024, 10:39 PM   #9
BetterRed
null operator (he/him)
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 21,111
Karma: 27620706
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Sigil's PageEdit might be a better starting point, it's a lightweight wysiwyg content editor. I'll have a look-see

BR
Last edited by BetterRed; 05-17-2024 at 10:45 PM.
BetterRed is offline   Reply With Quote
Old 05-17-2024, 11:12 PM   #10
BetterRed
null operator (he/him)
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 21,111
Karma: 27620706
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Quote:
Originally Posted by BetterRed View Post
Sigil's PageEdit might be a better starting point, it's a lightweight wysiwyg content editor. I'll have a look-see
Marginal, probably not worth bothering with.

But, let's say you could highlight all the "cathode ray tubes", what would you do next?

BR
BetterRed is offline   Reply With Quote
Old 05-17-2024, 11:48 PM   #11
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 44,650
Karma: 24495948
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
There's no such functionality. You would need to write a plugin for it.
kovidgoyal is offline   Reply With Quote
Old 05-18-2024, 12:02 AM   #12
mikapanja
Perfectionist
mikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentameter
 
Posts: 72
Karma: 12802
Join Date: Apr 2014
Device: none
Quote:
Originally Posted by BetterRed View Post
But, let's say you could highlight all the "cathode ray tubes", what would you do next?
Even finding the group would be more than enough.

It would also be useful to be able to define the group size (say, 3-5 words, or 3 words and larger).

Quote:
Originally Posted by kovidgoyal View Post
There's no such functionality. You would need to write a plugin for it.
I was hoping one of our resident experts could come up with a relevant regex or function. I know it's doable in Word via macro, albeit on a sentence level.
mikapanja is offline   Reply With Quote
Old 05-18-2024, 01:12 AM   #13
BetterRed
null operator (he/him)
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 21,111
Karma: 27620706
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Quote:
Originally Posted by mikapanja View Post
I know it's doable in Word via macro, albeit on a sentence level.
No need for macros in Word, standard Ctrl+H Find & Replace on entire document or selection e.g. paragraph, page, section etc, viz:

Click image for larger version Name: Screenshot 2024-05-18 134952.jpg Views: 604 Size: 753.1 KB ID: 208356

And when you've done, set Highlight to None and repeat F&R.

BR
BetterRed is offline   Reply With Quote
Old 05-18-2024, 08:50 AM   #14
mikapanja
Perfectionist
mikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentametermikapanja can solve quadratic equations while standing on his or her head reciting poetry in iambic pentameter
 
Posts: 72
Karma: 12802
Join Date: Apr 2014
Device: none
Quote:
Originally Posted by BetterRed View Post
No need for macros in Word, standard Ctrl+H Find & Replace on entire document or selection e.g. paragraph, page, section etc, viz:

Attachment 208356

And when you've done, set Highlight to None and repeat F&R.
That works in case you know what you are looking for. But what if you don't, which was my original thought? E.g. if you want to know if any 5-word group is repeated in the text.
mikapanja is offline   Reply With Quote
Old 05-18-2024, 09:27 AM   #15
Doitsu
Grand Sorcerer
Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.
 
Doitsu's Avatar
 
Posts: 5,645
Karma: 23456789
Join Date: Dec 2010
Device: Kindle PW2
Quote:
Originally Posted by mikapanja View Post
That works in case you know what you are looking for. But what if you don't, which was my original thought? E.g. if you want to know if any 5-word group is repeated in the text.
For those kind of searches you'll need to use a concordance tool. For example, Laurence Anthony's AntConc (freeware).
  • Unzip the epub file.
  • If the folder contains .xhtml files change their file extensions to .html.
  • Open AntConc, select Open file(s) as 'Quick Corpus', select .html as the file type and then select the extracted .html files.
  • Click the N-Gram tab, select the desired number of words and click Start.
I've attached a sample screenshot of the output.
Attached Thumbnails
Click image for larger version Name: ngrams.png Views: 344 Size: 31.8 KB ID: 208363  
Doitsu is offline   Reply With Quote
Reply
Page 1 of 2 1 2

« Previous Thread | Next Thread »
Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Repeated text after page turn. Pawel1212 Onyx Boox 6 01-13-2023 04:09 AM
Replace repeated item with the number of times it is repeated 1v4n0 Sigil 3 04-01-2021 05:41 PM
Find duplicate books MOJOJE Library Management 1 08-13-2020 06:59 PM
Repeated text pdf to epub conversion magicman1223 Conversion 3 04-25-2014 03:02 PM
Find duplicate books... silentguy Calibre 10 12-10-2010 12:03 PM


All times are GMT -4. The time now is 01:51 PM.

Contact Us - FAQ - Guidelines - Privacy - Top

MobileRead.com is a privately owned, operated and funded community.