Plugin ideas... add yours here

[sherman]

Hi folks.

Well, when I purchase a book, I like to retain all it's formatting and don't like doing an epub-to-epub conversion. However, most publishers don't do a "perfect" epub (in my opinion) and usually require a few css and xpgt tweaks before I load it on to my reader.

However, the publishers use a master template, so the changes I like to make to one book are identical to any other book by that publisher. This of course gets very repetitive very quickly...

So I was wondering if it were possible if someone was interested in making a plugin that would open an epub, run a series of find/replace operations on the css and xpgt files within and repackage the epub -- basically an "automated" Tweak Epub feature.

Example:
For some reason, Penguin books scale down the fonts by one "size". so a typical body paragraph has the following style

.tx {
font-size: small;
text-align: justify;
text-indent: 15px;
}

So therefore, when I make my alterations x-large becomes xx-large, large to x-large, medium to large, delete small, x-small to small, xx-small to x-small. I also like to change the indent to around 1.3em.

I also like to change the margins in the page-template.xpgt to 5px.



Any thoughts?

[garcle]

OK the Title Goal ("Scrape and Update a Custom field PLugin") is probably way too generic. But I have not let go the idea of having a custom DCC column in my Calibre Library (it would be useful to me).

So a Plugin that takes the ISBN of the selected record and hits (for example)the Classify web service, and then "beautiful Soups" the result to grab the DCC, and then saves it back into the nominated custom column. would be awesome

I have been tinkering and have figured out how to use the Beautiful Soup py Module to target response XML and to Lift a certain element within the response. My difficulty comes with getting my head around how to perform an update of a custom column associated with the selected record. (ie I'm having trouble navigating the data updating code of Calibre.)

A Plugin of this kind, while initially being targeted at a specific case, could potentially be expanded to become a generic plug-in for fetching and populating custom columns of various types from various services. Some other examples might be:-
- a cost column to track how much I have spent on my ebooks.
- an IMDB movie column with a link to movies made of the book.
- a country column denoting which country the author came from.
- a notification service, which alerts you when a new book in the current book series is published.


Oh and I'm aware of the 'fragility" issue, however Classify, for instance, is more a web services than an http:html response and one could expect that the xml element names would remain constant even if their position in the response changed.

[kiwidude]

Originally Posted by sherman

So I was wondering if it were possible if someone was interested in making a plugin that would open an epub, run a series of find/replace operations on the css and xpgt files within and repackage the epub -- basically an "automated" Tweak Epub feature.

Good old Palmy North eh? Grew up near Stratford in Taranaki myself...

I don't have a need for such a plugin myself but I can understand the issue and technically there is no great difficulty. One question that comes to mind if someone was to assist you with writing it is the nature of the find/replace operations and how those are specified. Could you get away with substituting an entire css file? Or do you substitute just specific styles? Maybe if you could list (as an example) all of the substitutions you do currently that you require, put them either in an attachment or spoiler section for someone to take a look at.

If the find/replace operations are quite complex then you are probably going to want them specified using regular expressions for instance.

[kiwidude]

Originally Posted by garcle

OK the Title Goal ("Scrape and Update a Custom field PLugin") is probably way too generic. But I have not let go the idea of having a custom DCC column in my Calibre Library (it would be useful to me)...

This does sound a VERY interesting idea, particularly if extensible via the BeautifulSoup type approach. I like your list of example usages too, very creative.

I've got the GoodReads plugin as next on my todo list to investigate so that will keep me out of trouble for a few days. But if no-one else has stepped in after that to help you with this then I could be tempted to offer help if needed, sounds an interesting challenge.

[sherman]

Originally Posted by kiwidude

Good old Palmy North eh? Grew up near Stratford in Taranaki myself...

I don't have a need for such a plugin myself but I can understand the issue and technically there is no great difficulty. One question that comes to mind if someone was to assist you with writing it is the nature of the find/replace operations and how those are specified. Could you get away with substituting an entire css file? Or do you substitute just specific styles? Maybe if you could list (as an example) all of the substitutions you do currently that you require, put them either in an attachment or spoiler section for someone to take a look at.

If the find/replace operations are quite complex then you are probably going to want them specified using regular expressions for instance.

The 'naki eh? I was born and raised in New Plymouth -- Bell Block to be exact. Just shifted back actually, need to update my profile


But anyway, back on topic...

I tend to prefer replacing specific styles rather than substituting in a new stylesheet.
It wouldn't need anything too exotic, basic search is all that's necessary, although regular expressions would be much more flexible and powerful.

My idea was to use a basic text file with a series of search/replace terms that the plugin would perform on the .css and .xpgt files found in the epub.

So, for the Penguin example, the text file could look a little something like this:

Spoiler:

[css]
(search)font-size: x-large (replace)font-size: xx-large
(search)font-size: large (replace)font-size: x-large
(search)font-size: medium (replace)font-size: large
(search)font-size: small (replace)
(search)font-size: xx-small (replace)-small
(search)font-size: x-small (replace)small
(search)font-size: -small (replace)x-small
(search)text-indent: 15px (replace)text-indent: 1.3em
(search)margin-right: 25px (replace)

[xpgt]
(search)20px or 30px (replace)5px


//of course, this wouldn't be the syntax actually used

[jtintle]

Hello, just had a thought of a plugin idea for Calibre:
I wish I could set calibre to take my list of authors, and find new releases from an author and let me know in a popup box, could do it once a week, maybe by checking against Amazon or something like that....I don't know if anyone else would be interested in it, just thought i would throw it out there

[kiwidude]

Originally Posted by jtintle

Hello, just had a thought of a plugin idea for Calibre:
I wish I could set calibre to take my list of authors, and find new releases from an author and let me know in a popup box, could do it once a week, maybe by checking against Amazon or something like that....I don't know if anyone else would be interested in it, just thought i would throw it out there

Nice idea. I wonder if there is a "better" source of that data for new releases in terms of identifying for an author what they have released recently/upcoming etc. FantasticFiction is fine but the data is unstructured, I dunno if Amazon has a "structured" source of the data either.

I keep my favourite authors in a user category, I think it would be great from time to time to be able to use a plugin to find out what books they have coming out or that I may have missed without having to one by one look them up...

[ElizabethN]

Originally Posted by kiwidude

Nice idea. I wonder if there is a "better" source of that data for new releases in terms of identifying for an author what they have released recently/upcoming etc. FantasticFiction is fine but the data is unstructured, I dunno if Amazon has a "structured" source of the data either.

I keep my favourite authors in a user category, I think it would be great from time to time to be able to use a plugin to find out what books they have coming out or that I may have missed without having to one by one look them up...

I use this site as well as FictFact and Amazon to try to stay caught up with releases, you can be alerted by series or author.

http://www.fictfact.com/

[kiwidude]

Originally Posted by ElizabethN

I use this site as well as FictFact and Amazon to try to stay caught up with releases, you can be alerted by series or author.

http://www.fictfact.com/

Very cool, thanks for sharing that, not seen that site before. If nothing else it will be added to my "Search the Internet" plugin as another interesting site. I'm wary of being sent weekly emails for the endless spam opportunities but it is nice to know the option is there.

[marvin_2]

Originally Posted by kiwidude

Excuse me for being a bit slow here but I wanted to clarify which "sort" you meant - Calibre has several of them

Indeed it does. I was hoping a plugin might help to keep an overview of the various options, and their current settings. For a start, it would help to have a plugin (the quick-prefence plugin?) manage the settings currently hosted in the "Tweaks" section (author_sort setting, title_series_sorting, possibly title_sort_articles).

Originally Posted by kiwidude

Did you mean the value of the tweak "sort_columns_at_startup"?

That would be a good candidate, series_index_auto_increment could be another.

Originally Posted by kiwidude

Allowing you to apply the value there (and possibly other sorts) in your library view after you have altered your sort columns manually?[...]
You mentioned the author sort by in the metadata dialogs. This is something I never touch. Could you explain your workflow and exactly which sorts you find yourself altering?

It appears that the Tweaks only affect new book records, the values for Author_sort and Title_sort need to be updated manually. Since there is a function in the Bulk-Metadata Editor, there may be a routine a plugin in call to automatically update these columns. If there isn't, the scope of updating metadata is probably beyond what a plugin should do - I fully agree with your view on this.

[kiwidude]

Originally Posted by marvin_2

Indeed it does. I was hoping a plugin might help to keep an overview of the various options, and their current settings. For a start, it would help to have a plugin (the quick-prefence plugin?) manage the settings currently hosted in the "Tweaks" section (author_sort setting, title_series_sorting, possibly title_sort_articles).

My initial response to this is that unfortunately anything you find in Tweaks is not going to be a candidate for "changing" in a plugin. I'm sure Kovid could explain why some things are in Tweaks rather than Preferences, but certainly one of the reasons I can think of is for Calibre-wide settings that will need a restart of the application to change. Very often the value is only checked at Calibre startup to initialise some behaviour, and hence changing the 'tweak' value and not restarting results in either no difference or inconsistent behaviour.

So at best in the plugin choosing from multiple values in a dropdown menu to change a tweak value would restart Calibre after making the change. However because of this restart requirement I would again ask the question what tweak values you feel the need to frequently change and why? It may be that if Kovid agrees with you the setting could be moved from Tweaks to Preferences - in which case a convenient way to swap values makes sense for the Quick Preferences plugin. Or it may be that there is a different way to approach what you are trying to achieve.

My overriding sense of what you are asking is not really for the ability to quickly change between tweak settings, but more for a GUI to change the values in Tweaks?

It appears that the Tweaks only affect new book records, the values for Author_sort and Title_sort need to be updated manually. Since there is a function in the Bulk-Metadata Editor, there may be a routine a plugin in call to automatically update these columns. If there isn't, the scope of updating metadata is probably beyond what a plugin should do - I fully agree with your view on this.

Right, that is a great example of where you would not want to be switching between tweak values. You pick one and you stick with it, or else your library ends up a mish-mash. This happened to me when I first changed the author sort tweak having initially imported some books then deciding I preferred LN, FN. For that particular tweak there are "known remedies" published on this forum involving the Manage Authors or S&R dialogs.

And no, updating book metadata is certainly not in the scope of the "Quick Preferences" plugin. Although if someone comes up with some scenario where they want to right-click apply some change to book metadata using a different plugin that is certainly feasible.

[kiwidude]

Another idea I had for a plugin yesterday - a "Manage Series" plugin.

With the future of the LibraryThing metadata download plugin questionable at best (and frequently mish-mashed quality of data even when it was working) the onus is going to be more and more on users to edit the series metadata themselves manually. I'm not a fan of the careful [] bracket editing required in Calibre currently when you renumber something - it is fine for an ad-hoc edit but anyone who has tried to renumber a large series knows it quickly becomes painful.

My idea is a right-click option which will present the books you have selected in a list, allowing you to move them up/down to change their order in the "series". You would be able to specify a series name (defaulted from first found in the selection if any) and a starting number (defaulting at 1). It would display read-only columns of title, author, pubdate and the "current" series value, plus a new series value based on order in the list as you move stuff around.

Perhaps also supporting some left/right buttons for when you want an item to have a "x.1" number in the middle of the main sequence. Maybe also a button to bring up the FantasticFiction webpage for the author for those not using the "Search the Internet" plugin or who opened this dialog first...

This might also be useful to quickly create a "fake" series for an author where you want to read in order of "published date" but find the data sourced into Calibre is incorrect (as I do) due to the wrong ISBNs being picked up or data missing.

If it sounds of interest let me know.

[ElizabethN]

Originally Posted by kiwidude

Another idea I had for a plugin yesterday - a "Manage Series" plugin.

With the future of the LibraryThing metadata download plugin questionable at best (and frequently mish-mashed quality of data even when it was working) the onus is going to be more and more on users to edit the series metadata themselves manually. I'm not a fan of the careful [] bracket editing required in Calibre currently when you renumber something - it is fine for an ad-hoc edit but anyone who has tried to renumber a large series knows it quickly becomes painful.

My idea is a right-click option which will present the books you have selected in a list, allowing you to move them up/down to change their order in the "series". You would be able to specify a series name (defaulted from first found in the selection if any) and a starting number (defaulting at 1). It would display read-only columns of title, author, pubdate and the "current" series value, plus a new series value based on order in the list as you move stuff around.

Perhaps also supporting some left/right buttons for when you want an item to have a "x.1" number in the middle of the main sequence. Maybe also a button to bring up the FantasticFiction webpage for the author for those not using the "Search the Internet" plugin or who opened this dialog first...

This might also be useful to quickly create a "fake" series for an author where you want to read in order of "published date" but find the data sourced into Calibre is incorrect (as I do) due to the wrong ISBNs being picked up or data missing.

If it sounds of interest let me know.

I'd be interested.

[DoctorOhh]

Originally Posted by kiwidude

Another idea I had for a plugin yesterday - a "Manage Series" plugin.

Sounds like a good idea.

Originally Posted by kiwidude

You would be able to specify a series name (defaulted from first found in the selection if any) and a starting number (defaulting at 1).

Bulk metadata edit does this now it allows the renaming of a large series quickly by just selecting the series in order before opening the metadata edit window.

Originally Posted by kiwidude

Perhaps also supporting some left/right buttons for when you want an item to have a "x.1" number in the middle of the main sequence.

This would be nice and is missing from the bulk metadata edit feature.

I also think the rest of your ideas sound good.

[Perkin]

This would be a good plugin.
I have another 'feature', as well as the left/right to add small increments, how about one feature to skip some numbers (for when you've got books missing and haven't got 'dummy' records) and then all subsequent numbers are adjusted as well.