Regular expressions

[G2B]

I am experimenting with the search & replace in calibre.


Conversion of txt files often leaves too many line breaks, even with heuristic processing.

When the </p><p class="whatever"> line breaks have small letters on both sides, they can always be removed and substituted with a space.

'[a-z]</p><p class="whatever">[a-z]' finds all those repeats but when I replace with '[a-z] [a-z]' the conversion also removes and replaces every letter before and after the code with the string '[a-z]' which is not what I intended.

I have to include '[a-z] in the search string to find the correct line breaks. If I don't, I'll delete every line break in the book, which imo makes it pretty much unreadable.

Is there a way to include those letters before and after line break in the search string, but to exclude them from substitution?

TIA.

[sjfan]

Originally Posted by G2B

I am experimenting with the search & replace in calibre.


Conversion of txt files often leaves too many line breaks, even with heuristic processing.

When the </p><p class="whatever"> line breaks have small letters on both sides, they can always be removed and substituted with a space.

'[a-z]</p><p class="whatever">[a-z]' finds all those repeats but when I replace with '[a-z] [a-z]' the conversion also removes and replaces every letter before and after the code with the string '[a-z]' which is not what I intended.

I have to include '[a-z] in the search string to find the correct line breaks. If I don't, I'll delete every line break in the book, which imo makes it pretty much unreadable.

Is there a way to include those letters before and after line break in the search string, but to exclude them from substitution?

TIA.

Put parens around parts of the search string to form groups. Use \# in the replace string to replace with the content of that group.

Search for:

([a-z])</p><p class="whatever">([a-z])

Replace with:

\1 \2

[deback]

Search for this:

([a-z])</p>\s+<p class="whatever">([a-z])

The \s+ in the middle is for the space between the two lines and is necessary to find what you're looking for.

Replace with this:

\1 \2

You might also want to do two searches and drop the ([a-z]) at the end, though, because you'll find there are a lot of lines that need to be connected where the second line doesn't start with a small letter.

[gbm]

I recommend useing the editor for finding split paragraphs and line feeds, this is the base search and replace I use:

Search

</p>\s*<p[^>]+>([a-z])

replace:

 \1

note the space at the beginning of the replace.

For extra line feeds I use a regex-fuction.

search:

<p class="(.*?)">(.*?)</p>|<div class="(.*?)">(.*?)</div>

regex-fuction

def replace(match, number, file_name, metadata, dictionaries, data, functions, *args, **kwargs):
    return match.group().replace('\n', ' ')

bernie

[G2B]

Originally Posted by gbm

I recommend using the editor for finding split paragraphs and line feeds, this is the base search and replace I use:

Search

</p>\s*<p[^>]+>([a-z])

replace:

 \1

note the space at the beginning of the replace.


bernie

Thanks for the commenting.
I tried that, and despite that your string only shows [a-z] small letters, the editor also takes me to sentence line breaks that start with [A-Z] capitalized letters.

There seems to be something wring with my calibre. Heurisitic processing splits pages instead of removing line breaks. I have uninstalled and reinstalled, but same problems persist.

[theducks]

because you did not include something that limits which beginning term traps
I check for an alpha, optionally followed by: comma, either quote.

This still fails in a few cases like Mr., where the alpha
is upper case like A.M.
I fix those few by hand

<sample clipped from Sigils saved search FILE which includes additional escapes>

70\Name=Cleanup/Joins/Join to lower
70\Find="([[:alpha:],][\"\x201d]*)</p>\\s*<p\\b[^>]*>([a-z\x201c\"])"
70\Replace=\\1 \\2
71\Name=Cleanup/Joins/Join to upper
71\Find="([[:alpha:],]\x201d*)</p>\\s*<p\\b[^>]*>([\"\x201c]*[A-Z])"
71\Replace=\\1 \\2

[deback]

Originally Posted by G2B

I tried that, and despite that your string only shows [a-z] small letters, the editor also takes me to sentence line breaks that start with [A-Z] capitalized letters.

Check the Case Sensitive box, and then it will work correctly.

[G2B]

Originally Posted by deback

Check the Case Sensitive box, and then it will work correctly.

Oops. Thanks.