Help: Tag Mapper and RegExpressions

[jecilop]

SOLVED as of post #16.

I need some help properly coding a Replace/Pattern rule. If you can or will provide input, I REALLY appreciate it.

Kovid provided some help when I requested the feature. I didn't realize it was already there, but it's definitely not intuitive. He gave me a link to the types of expressions I would need to use ( http://manual.calibre-ebook.com/regexp.html ), but I find it overwhelming and cannot put two and two together on it. I HAVE TRIED .

I have two situations noted below that I want to set up in rules. (I included a 3rd that he helped with for reference). If you know how, PLEASE provide the correct string needed for the "Replace if matches pattern" rule. Please do not provide just a guess as that can mess everything up as soon as I test it unless I create a separate library.
If I can get these, I think I can better learn more of this as I know there will be other patterns I'll want to work with.

Pattern 1: (solution provided by Kovid)
Tags: FIC098098, FIC090909, FIC######
Pattern: FIC([0-9]+)
Replace With: FICTION, \1
Yields: FICTION, 098098, 090909, etc.....
I just replaced with FICTION only and decided to eliminate the unnecessary numbers instead.

Pattern 2:
Tags: fiction / action / suspense - horror,
fiction / fantasy / action - adventure,
Pattern:
Replace With:
GOAL/Yields: fiction, action, suspense - horror, fantasy, action - adventure
Essentially, I want to turn the "Space/Space" into a comma which creates separate tags. I ONLY want this where a / exists with a space either before or after and NOT with a character of any sort next to it like "fiction/action/suspense". This would STAY "fiction/action/suspense" .

Pattern 3: although I MIGHT be able to figure this one out if I have a solution to Pattern 2.
Tags: action & adventure, science & fantasy, mystery & crime, thriller & horror
Pattern:
Replace With:
GOAL/Yields: action, adventure, science, fantasy, mystery, crime, thriller, horror (ie, separate tags)
As in Pattern 2: ONLY "Space&Space" would be replace with a comma to make separate tags. If "&" is used WITHOUT a space before or after, then I don't want that changed (ie, walk&run stays walk&run)

Thanks ahead of time if you can...

[fidvo]

The second pattern is surprisingly simple:

Pattern: " / " (not including the quotes)
Replace With: "," (not including the quotes)

I just tested, and Calibre is smart enough to identify duplicate tags generated by the replace and eliminate them in post-processing.

I haven't tried the third pattern, but it should be similar.

[jecilop]

Originally Posted by fidvo

The second pattern is surprisingly simple:

Pattern: " / " (not including the quotes)
Replace With: "," (not including the quotes)

I just tested, and Calibre is smart enough to identify duplicate tags generated by the replace and eliminate them in post-processing.

I haven't tried the third pattern, but it should be similar.

Just to be clear, you typed SPACE/SPACE without the quotes and it resulted in the new, multiple tags? That would make sense, but it seemed for more complicated because it's a symbol which tends to imply other things in the expressions.

[davidfor]

Whether you need to escape a character in a regex expression depends on the regex flavour. In this case it is Python, and the forward-slash does not need to be escaped. If it did, the search expression would be " \/ " (Space backslash forward-slash space). But, escaping an extra character doesn't usually harm.

For the last one it is " & " (Space ampersand space).

And before you do this, everyone here would recommend taking a backup of the library. Simply close calibre and take a copy of the library directory. Then, when you mess up, you can restore the backup and start again.

[BetterRed]

Originally Posted by davidfor

And before you do this, everyone here would recommend taking a backup of the library. Simply close calibre and take a copy of the library directory. Then, when you mess up, you can restore the backup and start again.

This member of everyone group wouldn't. If you 'mess up' then restoring an entire library, especially from the cloud, can take a long time.

So I would suggest copying a relevant subset if the library to an empty library and use it for testing - if you mess up, recreate it from main library.

Then backup your library, and apply what you learnt on the test library. This will reduce the chances if needing to restore from backup significantly.

BR

[jecilop]

@fidvo,

I just ran tests on this. It doesn't work for me.

I choose the rule: Replace..... Matches Pattern..... then the only thing in the field is SPACE/SPACE.

I told it to replace with a ,(comma).

Nothing changes when running. When I go back to the rule, the leading space before the / has disappeared; so does the leading if possibly not necessary space before the comma. I fix, yet it returns.

Are you sure this is all you typed in and had success with?

I've attached pics in case this isn't clear.

[jecilop]

Sorry, post 6 was directed to @fidvo. - NOT @davidfor

[jecilop]

@fidvo,

I just ran another test. I have to wonder if you actually tested a Replace/Matches Pattern rule.

As set up, that only works for a tag that is ONLY a forwardslash / .

I created a tag " / " . I easily replaced that with just a comma using Matches Pattern.
BUT, I'm talking about a forwardslash that is CONTAINED IN A TAG EXPRESSION. Therefore, there has to be something else to code to convey this.


A simple SPACE/SPACE and comma replace works with Search and Replace, but it doesn't work in the Tag Mapper which is the preference as my rules will "fix" the items such separation would create so I'd like it to be automatic vs the Search and Replace method that would require running it on tens of thousands of entries all at once since it would be more time consuming to "find" tags expressions containing this.

[davidfor]

Somewhere in the code, something is probably trimming the leading spaces. Try "\s/\s". The "\s" means "any whitespace". That will match spaces and other characters such as tabs, but should be safe for what you are doing.

Edit:

Actually "\ /\ " (backslash space slash backslash space) will probably work as well. But, that risks the trailing space being lost. And it just looks weird to me.

[jecilop]

Neither of these work.
I tried with an actual s (just in case) and then with a black space instead of the s as noted in your edit.

The "s" one doesn't change anything. The second gets an invalid error as "not a regular expression"

[davidfor]

"\s/\s" (that is: backslash s forward-slash backslash s) should work. I can't do any playing now, so I'll try at home tonight if no-one else chimes in.

[jecilop]

Yep, I did that EXACTLY (no quotes)...no go.

[fidvo]

The difference could be that I tested it in the Search and Replace in the Bulk Edit Metadata window. I didn't try it in the tag mapper.

[fidvo]

I figured out why it works in the Search and Replace but not the tag matcher.

The S&R looks for an instance of the pattern, while the tag mapper tries to match to the complete tag.

Although I don't have a complete solution, here's a partial solution.

Pattern: (.+) / (.+)
Replace: \1,\2

This will work in cases where there's only one division (space-slash-space) in each tag. Maybe someone smarter than me can figure out how to fix it so it works for an arbitrary number of divisions.

[kovidgoyal]

Originally Posted by fidvo

This will work in cases where there's only one division (space-slash-space) in each tag. Maybe someone smarter than me can figure out how to fix it so it works for an arbitrary number of divisions.

That is all that is needed. Tag mapper rules apply repeatedly to their own output (upto a hard limit of twenty times)

So the solution is simple:
pattern: ([^/]+) / ([^/]+)
replace: \1, \2

This will cause: a / b / c /d
to become: a, b, c, d

The only case it does not handle is leading and trailing slashes, which I leave as an exercise for the reader.

I might look into adding a separate split action in addition to replace, which will make this particular class of operations more intuitive.