Regular expressions, Calibre and you- an introduction (Archived)

[Manichean]

Originally Posted by da_jane

If I want to remove multiple lines of text, do I enclose my reg expressions in parentheses and then separate the sets by a ?

If I understand you correctly, you want to do something like

(set1)?(set2)

That would match the strings "set1set2" and "set1set1set2", as the question mark is interpreted as a quantifier to be applied to the first group. There are two ways you can match multiple lines, both depend on you knowing where the linebreaks are (or could be).
The first way ist to use whitespaces: "\s*?" will match a linebreak, as, on windows, it is displayed as two whitespaces (carriage return and linefeed). Put that in the place(s) you expect a linebreak to happen- it will let the whole expression match even if there isn't any linebreak, that's why I used the "*" quantifier.
The second way would be using the dot and the dotall- flag, as in you'd put ".*?" where you expect linebreaks to happen, and append or prepend your whole expression with the flag construct, "(?s)". The same remarks as in the first way apply, plus a caveat: be careful when using this, as the dot doesn't only match whitespaces, but any characters. You might accidentally match more than you intended to.

For the record, I think you got confused at the point where I was talking about flags, right? If you have suggestions how to improve the tutorial so that doesn't happen, I'd be glad to hear them.

[bucsie]

Hi

Could you please explain the regular expression used for detecting headers?

(?i)(?<=<hr>)((\s*<a name=\d+></a>((<img.+?>)*<br>\s*)?\d+<br>\s*.*?\s*)|(\s*<a name=\d+></a>((<img.+?>)*<br>\s*)?.*?<br>\s*\d+))(?=<br>)

I am not a newbie with regex-es, but it's difficult to follow, and there are a couple of things I don't understand, like what is this for: (?<= or this one: (?=

[kacir]

Originally Posted by bucsie

I am not a newbie with regex-es, but it's difficult to follow, and there are a couple of things I don't understand, like what is this for: (?<= or this one: (?=

those funny non-standard RE constructs are described at
http://docs.python.org/library/re.html

(?i)   # switch ignorecase on
(?<=<hr>)   # lookbehind assertion. It means the following RE can only match if <hr> precedes it. The <hr> string will not be part of the resulting match
(      # beginning of the main RE
   (   # beginning of Group A Re
      \s*<a name=\d+></a>
      (
         (<img.+?>)*   # +? is non greedy quantifier. It means match as little characters as possible, but at least one character. I personally would have written (<img[^>]+>)* here (see the first post for explanation)

         <br>
         \s*
       )?
      \d+
      <br>
      \s*
      .*?   # *? is non greedy quantifier. It means match as little characters as possible 
      \s*
   )   # end of of Group A Re
   |   # Group A OR Group B will be matched
   (   # beginning of Group A Re
      \s*
      <a name=\d+></a>
      (
         (<img.+?>)*   # +? is non greedy quantifier. It means match as little characters as possible, but at least one character. I personally would have written (<img[^>]+>)* here 
         <br>
         \s*
      )?
      .*?   # *? is non greedy quantifier. It means match as little characters as possible 
      <br>
      \s*
      \d+
   )   end of of Group B Re
)      # end of the main RE
(?=<br>)   # lookahead assertion. It means that the preceding RE can only match if it is followed by a <br>. The <br> string will not be part of the resulting match

[mediax]

This thread (or an edited version of it) would make a good sticky

[Manichean]

The first post is available in the user manual, which is why I haven't thought it necessary to sticky this. However, if someone feels differently, I won't mind.

[mediax]

RTBM, for God's sake - RTBM !!!