Templates: various questions not worth their own thread

[chaley]

Originally Posted by ownedbycats

Sorry, not too clear. Currently this returns the first alphanumeric character of title_sort. I thought it'd be possible to combine all the numeric characters into a single 0-9 category.

Still confused. What does it mean to combine numbers into a category? Can you give some examples?

[ownedbycats]

For the tag browser. I don't have that many book titles starting with numeric characters specifically so I thought to make a single 0-9 instead of 0, 1, 2, 4, 5, 9...

[chaley]

Originally Posted by ownedbycats

Attachment 186677

For the tag browser. I don't have that many book titles starting with numeric characters specifically so I thought to make a single 0-9 instead of 0, 1, 2, 4, 5, 9...

I think this does what you want. Two versions

TPM:

{title_sort:'contains(t=uppercase(re($, '^\W*(\w).*$','\1')), '\d', '0-9', t)'}

GPM:

program:
	t = uppercase(re($title_sort, '^\W*(\w).*$','\1'));
	if '\d' in t then '0-9' else t fi

[ownedbycats]

That works. Thank you.

A minor question about the template tester. Heres a template I am making for my 'mark as read' action chain:

program:
	
	if 
		check_yes_no('#onkobo', 1, 0, 0)
		then format_date(today(),'iso')
else $$#lastread
fi

In the screenshot is a book that is affected by the if statement and returns 'today'.

However, changing $$#lastread (raw field) to $#lastread in the else statement changes it slightly. Why does this occur?

edit: wait, I think it's because time elapses...

[chaley]

Originally Posted by ownedbycats

However, changing $$#lastread (raw field) to $#lastread in the else statement changes it slightly. Why does this occur?

edit: wait, I think it's because time elapses...

Yes, time does pass.

However $$X and $X are not the same thing. The raw_field version ($$) returns whatever is the raw data for the field. The field version ($) returns a formatted value. How those two differ depends on many things such as tweaks, column formats, and calibre default formats.

[ownedbycats]

#lastread is a column that shows just the date, so that $#lastread returns only the date (giving me a timestamp of 00:00:00 if I actually run it to modify the column) and $$#lastread gave me the full timestamp.

[capink]

@chaley: is there a way to make the template output new lines. For example if I want to output all formats for a book separated by a new line? Or do I have to wrap the call to the formatter to maybe replace '\\n' with a new line (in Action Chains)?

Edit: Replacing '\\n' with newlines would be a really bad idea it can mess up file paths which can have the same character. I will think of a safer representation for a new line. Maybe a new_line() function that inserts a non-printable character.

[chaley]

Originally Posted by capink

@chaley: is there a way to make the template output new lines. For example if I want to output all formats for a book separated by a new line? Or do I have to wrap the call to the formatter to maybe replace '\\n' with a new line (in Action Chains)?

Edit: Replacing '\\n' with newlines would be a really bad idea it can mess up file paths which can have the same character. I will think of a safer representation for a new line. Maybe a new_line() function that inserts a non-printable character.

The lexical analyser doesn't do any character escaping. The exception, which it gets wrong, is escaping quotes for example 'aa\'bb' that ends up as the string aa\'bb. Yes, the backslash is still there, which is wrong. I am looking for ways to fix this without breaking all strings containing backslashes.

If we use a "newline()" function, and presumably strcat(), then why not have it put a real newline in the string?

Note that by default leading and trailing whitespace is stripped (strip()). If you don't want that then call the formatter with strip_results=False.

EDIT: if strip_results is True then all white space, including newlines, is turned into a single space.

[chaley]

Here is a test version of formatter.py that supports a newline() function. It returns an actual newline character. As I said above, if you call the formatter with strip_results=False then the character(s) will not be removed.

I tested it with this template and it worked

program: strcat('a', newline(), 'b')

[capink]

It is working. I also strip the leading and trailing white space after that using strip().

def safe_format(fmt, kwargs_, error_value, book, **kwargs):
    output = SafeFormat().safe_format(fmt, kwargs_, error_value, book, strip_results=False, **kwargs)
    return output.strip()

[chaley]

Originally Posted by capink

It is working. I also strip the leading and trailing white space after that using strip().

def safe_format(fmt, kwargs_, error_value, book, **kwargs):
    output = SafeFormat().safe_format(fmt, kwargs_, error_value, book, strip_results=False, **kwargs)
    return output.strip()

The change is now in calibre source.

In addition, I improved the performance of the formatter parser by using table lookup instead of a series of 'if's to generate the interpretable syntax tree. Could you use the new version? I don't think I broke anything, but I have said that before.

[capink]

Thanks. I am using it now and it seems to be working OK. But newline() is not highlighted and does not appear in the function reference, but it works as expected.

[chaley]

Originally Posted by capink

Thanks. I am using it now and it seems to be working OK. But newline() is not highlighted and does not appear in the function reference, but it works as expected.

It is a function, not a keyword, so it isn't highlighted. I thought about making it a keyword but decided against it to avoid clashes with identifiers in existing templates.

You are right, I forgot to add the function to the function list. I will do that.

Question: for most formatter calls the newline() function won't do anything because the result has all contiguous white space converted to a single blank. It is then stripped. For example the template result "\nA \r\t\n B " becomes "A B". I am thinking of changing that behavior for GPM templates (but not SFM templates), removing the conversion-to-one-blank step. If I do then the above string would become "A \r\t\n B" (the leading and trailing whitespace is stripped but internal whitespace is not modified). I could also do strip(' '), removing leading & trailing blanks but leaving other whitespace intact. In this case the example result would be "\nA \r\t\n B" This change would make the newline() function generally usable and would permit formatting using tabs. (Of course that would require adding a tab() function.)

It is possible that someone has GPM templates that depend on the current callapse-white-space behavior, but I think the chances are very low, and in any event the template can be changed not to depend on collapsing spaces together.

I lean toward removing collapsing internal white space and stripping only blanks. What do you think?

BTW: the above change would mean you don't need to set strip_results because it would still do the strip() that you are doing.

[chaley]

Another thought: instead of newline() why not have character('name') where 'name' is the letter used in the escape? That lets us add more characters without risk of breaking things or needing to introduce more functions.

We could adapt names from the python standard escape sequence table.

\\ - backslash
\' - quote (')
\" - double_quote (")
\b - backspace
\n - newline (LF)
\r - return (CR)
\t - tab (TAB)

Examples:

character('newline') returns '\n'
character('return') returns '\r'
character('tab') returns '\t'
character('backspace') returns '\b'
character('backslash') returns '\\'

[capink]

Originally Posted by chaley

It is a function, not a keyword, so it isn't highlighted. I thought about making it a keyword but decided against it to avoid clashes with identifiers in existing templates.

You are right, I forgot to add the function to the function list. I will do that.

Functions turn into blue once I add the parenthesis. I think adding to the function list would do it. e.g. In the screenshot below, the strcat is blue while the newline is not.

Originally Posted by chaley

Question: for most formatter calls the newline() function won't do anything because the result has all contiguous white space converted to a single blank. It is then stripped. For example the template result "\nA \r\t\n B " becomes "A B". I am thinking of changing that behavior for GPM templates (but not SFM templates), removing the conversion-to-one-blank step. If I do then the above string would become "A \r\t\n B" (the leading and trailing whitespace is stripped but internal whitespace is not modified). I could also do strip(' '), removing leading & trailing blanks but leaving other whitespace intact. In this case the example result would be "\nA \r\t\n B" This change would make the newline() function generally usable and would permit formatting using tabs. (Of course that would require adding a tab() function.)

It is possible that someone has GPM templates that depend on the current callapse-white-space behavior, but I think the chances are very low, and in any event the template can be changed not to depend on collapsing spaces together.

I lean toward removing collapsing internal white space and stripping only blanks. What do you think?

BTW: the above change would mean you don't need to set strip_results because it would still do the strip() that you are doing.

Sounds very good to me.

Originally Posted by chaley

Another thought: instead of newline() why not have character('name') where 'name' is the letter used in the escape? That lets us add more characters without risk of breaking things or needing to introduce more functions.

We could adapt names from the python standard escape sequence table.

\\ - backslash
\' - quote (')
\" - double_quote (")
\b - backspace
\n - newline (LF)
\r - return (CR)
\t - tab (TAB)

Examples:

character('newline') returns '\n'
character('return') returns '\r'
character('tab') returns '\t'
character('backspace') returns '\b'
character('backslash') returns '\\'

That is definitely much better.