Register Guidelines E-Books Today's Posts Search

Go Back   MobileRead Forums > E-Book Software > Calibre > Library Management

Notices

Reply
 
Thread Tools Search this Thread
Old 02-24-2023, 04:59 PM   #1
ownedbycats
Custom User Title
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 9,498
Karma: 64500003
Join Date: Oct 2018
Location: Canada
Device: Kobo Libra H2O, formerly Aura HD
[FanFicFare] Metadata Management for Fanfiction

As requested by JimmXinu.

Quote:
Originally Posted by Amalthia View Post
I'm now curious to know how other people use calibre to normalize their metadata, only I'm not sure this thread is the place to find out?
There's a few examples on this page:
https://github.com/JimmXinu/FanFicFa...dataManagement

Here's one I added (and use sometimes):

Code:
Removing redundant character entries
Some fandoms have characters of indeterminate gender (e.g. player characters in video games), and authors sometimes add both a general and gender-specific entry.

[defaults]
exclude_metadata_pre:
## Removes "CharacterName" if Male or Female CharacterName is already present.
 characters=~^CharacterName&&characters=~^(Male|Female) CharacterName
Also this, specifically for AO3:

Code:
More selective chapterslashtotal [Archive of Our Own]
For Archive of Our Own, this will display '5/10' in its entirety, but only '5' for '5/5' or '5/?'. Replaces the default 'Chapters' on the titlepage.

[archiveofourown.org]
chapterslashtotal_label: Chapters
titlepage_entries: seriesHTML,category,genre,language,characters,ships,status,datePublished,dateUpdated,dateCreated,rating,warnings,chapterslashtotal,numWords,publisher,description
add_to_replace_metadata:
 chapterslashtotal=>^(\d+)/\1$=>\1
 chapterslashtotal=>/\?$=>
For Mass Effect, I prefer to split Trilogy and Andromeda (they're literally in different galaxies, lol). But there's plenty of stories with ambiguous entries, mostly ffnet or before AO3 split it. So for these, I merge all the ambigious entries to Mass Effect, then check for a few Andromeda-specific characters (added as needed). If it doesn't match those, it falls back to Trilogy. Never got a miscategorization with this.

This could probably also apply to other fandoms, though I couldn't tell you of any off-hand.

Code:
## Splitting up Mass Effect into Trilogy and Andromeda 
 category=>^(Mass Effect - Various Authors|Mass Effect \d|Mass Effect - All Media Types)$=>Mass Effect
 category=>^Mass Effect$=>Mass Effect: Andromeda&&characters=~(Ryder|Jaal Ama Darav)
 category=>^Mass Effect$=>Mass Effect Trilogy

Last edited by ownedbycats; 02-25-2023 at 11:48 AM.
ownedbycats is offline   Reply With Quote
Old 02-25-2023, 02:13 AM   #2
Amalthia
Wizard
Amalthia does all things with Zen-like beautyAmalthia does all things with Zen-like beautyAmalthia does all things with Zen-like beautyAmalthia does all things with Zen-like beautyAmalthia does all things with Zen-like beautyAmalthia does all things with Zen-like beautyAmalthia does all things with Zen-like beautyAmalthia does all things with Zen-like beautyAmalthia does all things with Zen-like beautyAmalthia does all things with Zen-like beautyAmalthia does all things with Zen-like beauty
 
Amalthia's Avatar
 
Posts: 1,165
Karma: 32196
Join Date: Jan 2007
Location: Anchorage, AK
Device: Sony Reader PRS-505, PRS-650, PRS-T3, Pocketbook HD2
Thanks for sharing what you're doing with your metadata.

Right now I'm mostly using replace_metadata to normalize the fandom tags, so I can use generate cover to insert fandom covers. Yesterday, I realized I could do something similar with Genre/freeformtags but it's not without risks and if I went at it full speed ahead my lines of code would reach thousands, which is a bit much.

For now, I'm using this to replace genre metadata.

Code:
replace_metadata:
 genre=>.*[Nn]o beta.*=>Not Beta Read
 genre=>.*[Uu]nbeta.*=>Not Beta Read
 genre=>.*[Ss]low [Bb]urn.*=>Slow Burn
 genre=>.*slowburn.*=>Slow Burn
 genre=>.*[Ff]ix[Ii]t.*=>Fix-It
 genre=>^[Ff]ix [Ii]t=>Fix-It
I am flat out removing these freeformtags with this code.

Code:
exclude_metadata_pre:
 warnings==Author Chose Not To Use Archive Warnings
 warnings==No Archive Warnings Apply
 warnings==Graphic Depictions Of Violence
 warnings==Creator Chose Not To Use Archive Warnings
 warnings==Additional Warnings In Author's Note
 freeformtags,genre==Other Additional Tags to Be Added
 freeformtags,genre=~^I .*
 freeformtags,genre=~^[Aa]dditional.*
 freeformtags,genre=~^[Aa]uthor.*
 freeformtags,genre=~^[Aa]nd.*
 freeformtags,genre=~^on .*
 freeformtags,genre=~^[Aa]lso.*
 freeformtags,genre=~^[Aa]lmost.*
 freeformtags,genre=~^NOTE.*
 freeformtags,genre=~^I'm.*
 freeformtags,genre=~^this.*
 freeformtags,genre=~^[Ii]n .*
 freeformtags,genre=~^[Aa] .*
 freeformtags,genre=~^[Aa]t .*
 freeformtags,genre=~^[Ww]ordcount.*
 freeformtags,genre=~^[Ww]ill .*
 freeformtags,genre=~^[Ss]ome .*
 freeformtags,genre=~^[Ss]o .*
 freeformtags,genre=~^[Oo]h .*
 freeformtags,genre=~^[Oo]f .*
 freeformtags,genre=~^[Ww]ell .*
 freeformtags,genre=~^[Tt]here.*
 freeformtags,genre=~^[Aa]ka .*
 freeformtags,genre=~^[Ii]dk .*
 freeformtags,genre=~^[Aa][Ll][Ll] .*
 freeformtags,genre=~^[Ii]t's .*
 freeformtags,genre=~^NOTE.*
 freeformtags,genre=~^[Mm]ore .*
 freeformtags,genre=~^[Ll]ike .*
 freeformtags,genre=~.*[Tt]ags .*
 freeformtags,genre=~^[Bb]ut .*
 freeformtags,genre=~.*because.*
 freeformtags,genre=~^[Hh]e.*
 freeformtags,genre=~^[Ss]he.*
I choose these because the tend to be author's opinions or just random phrases that don't describe the story. Or worse, the author's say something like "more tags to come."

If I thought more author's cared, I'd recommend that they split up their tags to use "Fix-It" "Jedi Order," instead of using "Jedi Order Fix-It." This makes it easier to find all "Fix-It" fics, or all "Jedi Order" you can select on multiple tags in the browser in Calibre. but if they are mushed together it gets lost.

Amalthia
Amalthia is offline   Reply With Quote
Advert
Old 02-25-2023, 02:14 AM   #3
culytera
Zealot
culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.
 
Posts: 121
Karma: 295674
Join Date: Jul 2021
Device: iPhone
Quote:
Originally Posted by Amalthia View Post
I'm now curious to know how other people use calibre to normalize their metadata, only I'm not sure this thread is the place to find out?
I separate the genre tags into category and freeform tags instead so they don't flood Calibre's tag browser while still keeping the freeform tags in order. The freeform tags will only show up on the fic's cover page, on a custom column and the epub itself. The only tags that show up on the tag browser now are the category/fandom, romantic relationships (the relationship tags that use "/"), the rating, the fic's status (whether it's in-progress/complete), and the grouped word count. So at the moment, my library has 59 tags.

If I want to look up the freeform tags on Calibre, I can still search for them since they're in a custom column with the long text type.

I use MapleRead to read epubs and I want to see the tags by order of their type so I also put a prefix depending on the tag. If it's a category it's [c], for the relationships it's [p], rating [r], status [s], grouped word count [w].

How I add the prefixes:
Spoiler:
Code:
add_to_extra_valid_entries:,prefix_ships,prefix_category,prefix_rating,prefix_status,prefix_wordcount

include_in_prefix_ships:ships
include_in_prefix_category:category
include_in_prefix_rating:rating
include_in_prefix_status:status
include_in_prefix_wordcount:numWords

replace_metadata:
 ## put these at the end of replace_metadata
 prefix_ships=>^(.*)=>[p] \1
 prefix_category=>^(.*)=>[c] \1
 prefix_rating=>^(.*)=>[r] \1
 prefix_status=>^(.*)=>[s] \1
 prefix_wordcount=>^(\d+|([0-4],\d{3})*|5,000)$=>[w] 0 - 5000
 prefix_wordcount=>^((?!5,000)([5-9],\d{3})*|10,000)$=>[w] 5001 - 10000
 prefix_wordcount=>^((?!10,000)(1\d,\d{3})*|20,000)$=>[w] 10001 - 20000
 prefix_wordcount=>^((?!20,000)([2-3]\d,\d{3})*|40,000)$=>[w] 20001 - 40000
 prefix_wordcount=>^((?!40,000)([4-7]\d,\d{3})*|80,000)$=>[w] 40001 - 80000
 prefix_wordcount=>^((?!80,000)(([8-9]\d)|(\d{3,}))(,\d{3})+)$=>[w] 80000+

 ## ships, category, rating, status with the added prefixes in Calibre's tags, the freeform tags don't flood the tag browser
include_subject_tags:prefix_ships,prefix_category,prefix_rating,prefix_status,prefix_wordcount


I remove characters if they're already mentioned in the relationship tag (since you have to manually input each character, I don't cover all of them, only the ones with multiple fics in my library):
Code:
exclude_metadata_pre:
 characters=~CHARACTER_NAME&&ships=~CHARACTER_NAME
 ## The below line specifically to exclude relationship tags that don't have / or & since some authors put "ship_name relationship" in the relationship tags and I want the tags to be consistent and not redundant
 ships!~(.*)[/&](.*)
I also have the fic's published and updated dates reformatted (thanks to JimmXinu for this):
Code:
datePublished_format:%%B %%d, %%Y
dateUpdated_format:%%B %%d, %%Y

add_to_extra_valid_entries:,dateRange

include_in_dateRange:datePublished, dateUpdated

add_to_replace_metadata:
# diff dates     1pmonth    2pday        3pyear   4umonth    5uday        6uyear
 dateRange_LIST=>([^\s]+)\s+(\d{1,2}),\s+(\d{4}), ([^\s]+)\s+(\d{1,2}),\s+(\d{4})=>\1 \2, \3-\4 \5, \6
# same year      1          2            3        4          5
 dateRange_LIST=>([^\s]+)\s+(\d{1,2}),\s+(\d{4})-([^\s]+)\s+(\d{1,2}),\s+\3=>\1 \2-\4 \5, \3
# same month     1          2              3            4
 dateRange_LIST=>([^\s]+)\s+(\d{1,2})-\1\s+(\d{1,2}),\s+(\d{4})=>\1 \2-\3, \4
And with the rest of my personal.ini, this is pretty much what the results will look like. (First image)

Since I put these all in custom columns, I could also use them as values for the Generate Cover plugin. You could even choose to show only one relationship tag instead of all of them. Here's how that layout was achieved:
Code:
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯​
{#ao3_category}
{#ao3_ships:sublist(0,-1,\,)}
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯​
{#ao3_rating} • {#ao3_words} • {#ao3_chapters}
{#ao3_date}
Attached Thumbnails
Click image for larger version

Name:	calibre.jpg
Views:	731
Size:	408.8 KB
ID:	199915   Click image for larger version

Name:	generate_cover.jpg
Views:	706
Size:	214.1 KB
ID:	199916  
culytera is offline   Reply With Quote
Old 02-25-2023, 11:06 AM   #4
dailyalice
Geek of All Trades
dailyalice rocks like Gibraltar!dailyalice rocks like Gibraltar!dailyalice rocks like Gibraltar!dailyalice rocks like Gibraltar!dailyalice rocks like Gibraltar!dailyalice rocks like Gibraltar!dailyalice rocks like Gibraltar!dailyalice rocks like Gibraltar!dailyalice rocks like Gibraltar!dailyalice rocks like Gibraltar!dailyalice rocks like Gibraltar!
 
dailyalice's Avatar
 
Posts: 55
Karma: 100878
Join Date: Aug 2012
Location: Minneapolis
Device: Kindle Oasis (3rd gen)
Love to see what other people are doing. Thanks everybody.

I dislike the FanFicFare's default behavior that dumps a ton of data into the main calibre Tags column/field so I actually edit my personal.ini so FanFicFare doesn't put anything in subject_tags. (I know it works for some people since these fields/columns are easily searched, but I like having things in separate columns/fields.)

Instead I have a bunch of custom columns for the data. Relationships column, Fandom column, Warnings column, etc.

The most important custom column, that I always create first, is the "Saved Metadata" column so I always have all available data from archives in my library. That way if I mess around with adding new columns or changing them, I can always fill them from Saved Metadata without having to redownload fic.

Though it is best to figure out what metadata you value the most and how you want it presented sooner rather than later, especially if you make anthologies (which won't work with Saved Metadata). Or if you want data to look a certain way on title pages. You can inject new generated covers into epubs without too much trouble, but not title pages.

Here's my post over in the FanFicFare thread about how I have a tag column that's unedited AO3 freeform tags called Tags (AO3) and another tag column currently named Tags (Edited) which is "only" about 450 tags, some pulled straight from AO3's freeform tags, some modified using replace_metadata.

Basically I only keep the tags from AO3's tags that I find useful. Any time I hear someone say they want to try to normalize all the AO3 tags, I point out how futile it is. You can use the data! But my Tags (AO3) column has over 30,000 tags in it (I wish I were exaggerating). Tags (Edited) has 450 or so which is much more manageable to say the least.

I keep Tags (AO3) around so I have the data and can look at it or search it if I want to. And to have that information on title pages. (But I mostly ignore it in the tag browser.)

Quote:
Originally Posted by culytera View Post
If I want to look up the freeform tags on Calibre, I can still search for them since they're in a custom column with the long text type.
I'm intrigued by this. If you dump freeform tags into a long text column, do the tags then show up in the order they're written on AO3? I'm guessing they would.

Because that's a thing I'd like to have on my title pages: the freeformtags as posted. When you put them in a tags column, they're alphabetized which would be fine if people actually used them as tags but of course some authors practically tell stories in the tags (and yes, I like to keep the stories I suppose).
dailyalice is offline   Reply With Quote
Old 02-25-2023, 11:23 AM   #5
ownedbycats
Custom User Title
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 9,498
Karma: 64500003
Join Date: Oct 2018
Location: Canada
Device: Kobo Libra H2O, formerly Aura HD
Quote:
Originally Posted by dailyalice View Post
I'm intrigued by this. If you dump freeform tags into a long text column, do the tags then show up in the order they're written on AO3? I'm guessing they would.

Because that's a thing I'd like to have on my title pages: the freeformtags as posted. When you put them in a tags column, they're alphabetized which would be fine if people actually used them as tags but of course some authors practically tell stories in the tags (and yes, I like to keep the stories I suppose).
If culytera is referring to the longtext metadata column—not 100% on this, wording's a bit unclear—tags are unsorted and looks something like this. (story in question)

Code:
<p><span class='label'>Freeform Tags</span>: <div class='metadata list' id='freeformtags'><ul>
<li>Shadowscales</li>
<li>Original Character(s)</li>
<li>character backstory</li>
<li>Implied/Referenced Alcohol Abuse/Alcoholism</li>
<li>Alcohol</li>
<li>Minor Character Death</li>
</ul></div><p>

Last edited by ownedbycats; 02-25-2023 at 11:37 AM.
ownedbycats is offline   Reply With Quote
Advert
Old 02-25-2023, 11:40 AM   #6
ownedbycats
Custom User Title
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 9,498
Karma: 64500003
Join Date: Oct 2018
Location: Canada
Device: Kobo Libra H2O, formerly Aura HD
Quote:
Originally Posted by dailyalice View Post
Because that's a thing I'd like to have on my title pages: the freeformtags as posted. When you put them in a tags column, they're alphabetized which would be fine if people actually used them as tags but of course some authors practically tell stories in the tags (and yes, I like to keep the stories I suppose).
Try something like this. It copies the freeformtags to a custom entry. Then you can edit the custom entry to have it only affect the column mapping and leave titlepage intact.

Hopefully I got it right, I'll ask in the FFF thread. Seems good

Code:
[defaults]

custom_columns_settings: 
 editedfreeform=>#yourtagcolumn

# 'add_to_' concats on to existing param, thus the leading comma
add_to_extra_valid_entries:,editedfreeform

include_in_editedfreeform:freeformtags

replace_metadata: 
 editedfreeform=>SomeTag=>SomeEditedTag

exclude_metadata_pre:
 editedfreeform=~SomeConversationalTagThatMakesNoSenseWhenSorted

Last edited by ownedbycats; 02-25-2023 at 11:58 AM.
ownedbycats is offline   Reply With Quote
Old 02-25-2023, 10:16 PM   #7
culytera
Zealot
culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.
 
Posts: 121
Karma: 295674
Join Date: Jul 2021
Device: iPhone
Quote:
Originally Posted by dailyalice View Post
Love to see what other people are doing. Thanks everybody.

I dislike the FanFicFare's default behavior that dumps a ton of data into the main calibre Tags column/field so I actually edit my personal.ini so FanFicFare doesn't put anything in subject_tags. (I know it works for some people since these fields/columns are easily searched, but I like having things in separate columns/fields.)

...

I'm intrigued by this. If you dump freeform tags into a long text column, do the tags then show up in the order they're written on AO3? I'm guessing they would.

Because that's a thing I'd like to have on my title pages: the freeformtags as posted. When you put them in a tags column, they're alphabetized which would be fine if people actually used them as tags but of course some authors practically tell stories in the tags (and yes, I like to keep the stories I suppose).
Do you have the keep_in_order_ settings in your personal.ini setup?

Mine looks like this:
Code:
keep_in_order_category:true
keep_in_order_characters:true
keep_in_order_additional_characters:true
keep_in_order_dateRange:true
keep_in_order_fandoms:true
keep_in_order_freeformtags:true
keep_in_order_genre:true
keep_in_order_ships:true
keep_in_order_ao3categories:true
I don't even know if some of these are still important but I just keep them like that and it works for me.

My include_subject_tags only have the prefixed tags so Calibre's tag browser doesn't get flooded by all sorts of tags that'll just confuse me.
Code:
include_subject_tags:prefix_ships,prefix_category,prefix_rating,prefix_status,prefix_wordcount
My titlepage settings aren't complicated thanks to the freeformtag separation, it's neat enough for my use case. I don't need to edit the freeform tags anymore since they don't flood the tag browser and they're already in their original order.
Code:
titlepage_entries:<div class="inline">,seriesHTML,category,rating,warnings,</div>,ships,additional_characters,freeformtags


## metadata ${title}, etc.  Make sure to keep at least one space
## at the start of each line and to escape % to %%.


titlepage_start:<?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
 <html xmlns="http://www.w3.org/1999/xhtml">
 <head>
 <title>${title} by ${author}</title>
 <link href="stylesheet.css" type="text/css" charset="UTF-8" rel="stylesheet"/>
 <meta name="calibre-html-render-data" content='{"size": "A2", "margins": [0, 0, 0, 0]}'/>
 </head>
 <body class="fff_titlepage" style="font-size: 2.5vmax;">
 <div class="title">
 <h1><a href="${storyUrl}">${title}</a> by ${authorHTML}</h1>
 </div>
 <hr/>
 <dl class="tags">


## ${id}, ${label}, ${value}
titlepage_entry:
 <dt><b>${label}:</b></dt> <dd>${value}</dd>


## ${id}, ${label}, ${value}
## Used with entries listed in wide_titlepage_entries, typically description, storyUrl, authorUrl
titlepage_wide_entry:


## When using tables, make these span both columns.
wide_titlepage_entries:


## ${id}, ${value}
## For example, if you've set description_label:
## (that is, nothing)
titlepage_no_title_entry:


## metadata
titlepage_end:
 <dl class="stats">
 <dt><b>Words & Chapters:</b> ${numWords} &bull; ${chapterslashtotal}</dt>
 <dt><b>Date:</b> ${dateRange}</dt>
 </dl>
 </dl>
 <hr/>
 <div class="tag-summary-content">${description}</div>
 </body>
 </html>
I also have the (mostly) unmodified metadata for relationships, characters (where I don't exclude characters already mentioned in ships), category/fandom and rating in custom columns for reference, though they're only shown in the tag browser. (See attached for my additional tags column's settings, book details, my custom columns, and what my tag browser looks like)

Code:
add_to_custom_columns_settings:
 category=>#ao3_category
 characters=>#ao3_characters
 additional_characters=>#ao3_additional_characters
 dateRange=>#ao3_date
 dateUpdated=>#ao3_dateupdated
 freeformtags=>#ao3_additional_tags
 numWords=>#ao3_words
 rating=>#ao3_rating
 ships=>#ao3_ships
Attached Thumbnails
Click image for larger version

Name:	additional_tags_column_settings.jpg
Views:	579
Size:	80.2 KB
ID:	199940   Click image for larger version

Name:	book_details.jpg
Views:	623
Size:	616.7 KB
ID:	199941   Click image for larger version

Name:	custom_columns.jpg
Views:	574
Size:	215.8 KB
ID:	199942   Click image for larger version

Name:	tag_browser.jpg
Views:	520
Size:	72.9 KB
ID:	199943  
culytera is offline   Reply With Quote
Old 02-25-2023, 10:31 PM   #8
ownedbycats
Custom User Title
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 9,498
Karma: 64500003
Join Date: Oct 2018
Location: Canada
Device: Kobo Libra H2O, formerly Aura HD
Unfortunately, keep_in_order only really works for title page; taglike columns always sorted.
ownedbycats is offline   Reply With Quote
Old 02-25-2023, 10:43 PM   #9
culytera
Zealot
culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.
 
Posts: 121
Karma: 295674
Join Date: Jul 2021
Device: iPhone
Quote:
Originally Posted by ownedbycats View Post
Unfortunately, keep_in_order only really works for title page; taglike columns always sorted.
It seems to follow the order if you check "contains names". Its separator will change to an ampersand in book details, but if you don't want to see that, you could separate them by line instead (preferences -> look & feel -> book details -> categories on separate lines).

Though I prefer to just use the search bar instead of treating them as a tag so I don't have to clean them up.
Attached Thumbnails
Click image for larger version

Name:	addtags2.jpg
Views:	482
Size:	94.6 KB
ID:	199945  
culytera is offline   Reply With Quote
Old 02-25-2023, 10:46 PM   #10
ownedbycats
Custom User Title
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 9,498
Karma: 64500003
Join Date: Oct 2018
Location: Canada
Device: Kobo Libra H2O, formerly Aura HD
Ah, that makes sense.

In theory, I think it might be possible to use a gpm template to regex a full-text column and display it as an unsorted column. In practice, I'm not quite good enough at templates or regexes to try this. And it'd be the unmodified tags, which would make it kinda messy.

I did use a regex search & replace to populate my #fanficratings column from the longtext rather than run a 'update from column' on everything, though.

search: (?s).*(<p><span class='label'>Rating<\/span>: <div class='metadata' id='rating'>(.*?)<\/div><p>).*
replace with: \2

(regex101 for demonstration)

Click image for larger version

Name:	2023-02-25 23_47_58-Editing metadata for one book.png
Views:	491
Size:	48.7 KB
ID:	199946

Last edited by ownedbycats; 02-25-2023 at 10:50 PM. Reason: adding regex101 link
ownedbycats is offline   Reply With Quote
Old 02-26-2023, 08:49 PM   #11
culytera
Zealot
culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.
 
Posts: 121
Karma: 295674
Join Date: Jul 2021
Device: iPhone
Quote:
Originally Posted by ownedbycats View Post
Ah, that makes sense.

In theory, I think it might be possible to use a gpm template to regex a full-text column and display it as an unsorted column. In practice, I'm not quite good enough at templates or regexes to try this. And it'd be the unmodified tags, which would make it kinda messy.

I did use a regex search & replace to populate my #fanficratings column from the longtext rather than run a 'update from column' on everything, though.

search: (?s).*(<p><span class='label'>Rating<\/span>: <div class='metadata' id='rating'>(.*?)<\/div><p>).*
replace with: \2
Interesting method. So you just put all the metadata pulled by FanFicFare into one long text custom column? Is the search & replace afterwards automated?

I wonder if that's faster than filling each custom column via FanFicFare.

Last edited by culytera; 02-26-2023 at 08:54 PM.
culytera is offline   Reply With Quote
Old 02-26-2023, 09:14 PM   #12
ownedbycats
Custom User Title
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 9,498
Karma: 64500003
Join Date: Oct 2018
Location: Canada
Device: Kobo Libra H2O, formerly Aura HD
Quote:
Originally Posted by culytera View Post
Interesting method. So you just put all the metadata pulled by FanFicFare into one long text custom column? Is the search & replace afterwards automated?
It's an option in FanFicFare. See Update Calibre Metadata from Saved Metadata Column.

I created the #fanficrating column and set it in FFF, but was too lazy to check 'new only' on all the other columns to avoid replacing everything else. So I manually ran the search & replace to populate it on my existing stories. New/updated stories are automatically filled out.

Last edited by ownedbycats; 02-26-2023 at 09:18 PM.
ownedbycats is offline   Reply With Quote
Old 02-26-2023, 10:22 PM   #13
culytera
Zealot
culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.culytera ought to be getting tired of karma fortunes by now.
 
Posts: 121
Karma: 295674
Join Date: Jul 2021
Device: iPhone
Quote:
Originally Posted by ownedbycats View Post
It's an option in FanFicFare. See Update Calibre Metadata from Saved Metadata Column.

I created the #fanficrating column and set it in FFF, but was too lazy to check 'new only' on all the other columns to avoid replacing everything else. So I manually ran the search & replace to populate it on my existing stories. New/updated stories are automatically filled out.
Ahh I thought you were using the regex to populate the metadata for all works. My process was always to just redownload the fic to update/fix any metadata since I also have to adjust the title page.

Now I'm just trying to optimize the process of downloading and updating the library.

Last edited by culytera; 02-26-2023 at 10:40 PM.
culytera is offline   Reply With Quote
Old 02-26-2023, 11:56 PM   #14
ownedbycats
Custom User Title
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 9,498
Karma: 64500003
Join Date: Oct 2018
Location: Canada
Device: Kobo Libra H2O, formerly Aura HD
The other problem with search & replace - longtext column stores original values, so replace_metadatas have no effect. In this specific case, I also wanted to see what all the ratings were before deciding how to 'standardize' them:

Code:
## Ratings
 rating=>^(K|K\+)$=>General Audiences
 rating=>^Teen And Up Audiences$=>Teen and Up Audiences
 rating=>^T$=>Teen and Up Audiences
 rating=>^Mature$=>Mature Audiences
 rating=>^M$=>Mature Audiences
Anthologies are a bit of a special case, though. They basically concat all the ratings and return them as a string.

What I did in this case was use Action Chains to run a single-field edit with this gpm template:

Code:
program:
## list_remove_duplicates probably isn't necessary
	list = list_remove_duplicates($#fanficrating, ',');

	first_non_empty(
		if 'Explicit' inlist list then 'Explicit' fi,
		if 'Mature Audiences' inlist list then "Mature Audiences" fi,
		if "Teen and Up Audiences" inlist list then "Teen and Up Audiences" fi,
		if "General Audiences" inlist list then "General Audiences" fi,
		if "Not Rated" inlist list then "Not Rated" fi
	)
(I later improved the template, using a custom function. The provided one is easier to set up.. )

Last edited by ownedbycats; 02-26-2023 at 11:58 PM.
ownedbycats is offline   Reply With Quote
Old 04-03-2023, 07:44 PM   #15
ownedbycats
Custom User Title
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 9,498
Karma: 64500003
Join Date: Oct 2018
Location: Canada
Device: Kobo Libra H2O, formerly Aura HD
For Archive of Our Own: here is my method for copying collections and collection URLs to the series column. I use an Mass Effect fic exchange as an example.

Notes:

- This requires a version of FanFicFare released after April 4, 2023 (as of posting date, the latest test version) to make use of a static value in 'include_in.'

- If there's both series and collection, you'll need to pick which one you want. Use exclude_metadata_pre to remove the unwanted one using a storyUrl conditional.


Code:
[archiveofourown.org]

## This creates an additional 'collectionUrl' entry.
add_to_extra_valid_entries:,collectionUrl

## Copies collections to series
include_in_series:series,collections

## Creates a placeholder collectionUrl. I chose to use 'abc' for easy replacement (no escapes!).
include_in_collectionUrl:"abc"

## Adds collectionURL to series.
include_in_seriesUrl:seriesUrl,collectionUrl

include_metadata_pre:
## Include specific collections as series. Here, I've included both an exact version (==) and regex (=~). 
 collections==Spectre Requisitions 2023
 collections=~(Spectre Requisitions \d{4}|Spec Recs|Spec Recs \d{4})

add_to_replace_metadata:
## I apply transformations to 'collections' but applying them to 'series' should also work
 collections=>Spec Recs=>Spectre Requisitions

## Here, change the placeholder 'abc' urls to the collection URLs based on a conditional. 
 collectionUrl=>abc=>https://archiveofourown.org/collections/SpecReqs2023&&collections==Spectre Requisitions 2023

exclude_metadata_post:
## If the placeholder isn't changed during replace_metadata, this then removes it to prevent general breakage
 collectionUrl==abc

Last edited by ownedbycats; 04-03-2023 at 10:55 PM.
ownedbycats is offline   Reply With Quote
Reply


Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Library management issues after editing metadata dra Library Management 26 04-11-2020 05:10 PM
Best strategy for metadata management for Kobo using Calibre? ceridwen Kobo Developer's Corner 16 05-13-2019 02:51 AM
Will automatic metadata management also transfer updated cover images to device? TheSacredSoul Library Management 2 01-11-2013 06:05 AM
Metadata Management on Android not working? TheStretchedElf Devices 0 08-08-2012 09:10 AM
Automatic Metadata Management gxxshock Calibre 2 12-28-2008 12:48 PM


All times are GMT -4. The time now is 10:45 PM.


MobileRead.com is a privately owned, operated and funded community.