[GUI Plugin] Highlights to Obsidian

[jm289765]

Highlights to Obsidian is a plugin that formats and sends highlights to the Obsidian.md markdown editor.

Main Features:

• Send all highlights to Obsidian
• Only send new highlights or highlights of selected books
• Customizable formatting
• Optional header to be attached to the beginning of each batch of highlights
• When multiple highlights are sent to the same file, user can specify how they will be sorted

Formatting options:

Spoiler:

Book Data:

• {title}: Title of the book the highlight is in.
• {authors}: Authors of the book the highlight is in.
• {bookid}: The book's ID in calibre.

Highlight Data:

• {highlight}: The highlighted text.
• {blockquote}: The highlighted text, formatted as a blockquote. An arrow and a space "> " are added to the beginning of each line.
• {notes}: The user's notes on this highlight, if any notes exist. There is a config option that allows you to set different formatting depending on whether a highlight includes notes.
• {url}: A calibre url to open the ebook viewer to this highlight. Note that this may not work if your library's name contains unsafe URL characters. Numbers, letters, spaces, underscores, and hyphens are all safe.
• {location}: The highlight's EPUB CFI location in the book. For example, "/2/8/6/5:192". As a sort key, this will order highlights by their position in the book.
• {timestamp}: The highlight's Unix timestamp. As a sort key, this will order highlights by when they were made.
• {uuid}: The highlight's unique ID in calibre. For example, "TlNlh8_I5VGKUtqdfbOxDw".

Time Data:

• {date}: Date the highlight was made, formatted as YYYY-MM-DD.
• {time}: Time the highlight was made, formatted as HH:MM:SS.
• {datetime}: Date and time highlight was made, formatted as YYYY-MM-DD HH:MM:SS.
• {day}: Day of the month the highlight was made, as in 03 or 17.
• {month}: Month the highlight was made, as in 04 for April or 10 for October.
• {year}: Full year the highlight was made, as in 2022.
• {hour}: Hour the highlight was made, based on a 24-hour (not 12-hour) system.
• {minute}: Minute the highlight was made.
• {second}: Second the highlight was made.
• {utcnow}: current time, formatted same as {datetime}.
• {datenow}: Current date, formatted same as {date}.
• {timenow}: Current time, formatted same as {time}.
• {timezone}: The timezone that your computer is currently set to. Note that this may not always match the timezone the highlight was made in. Also note that this might use the full name "Coordinated Universal Time" instead of the abbreviation "UTC".
• {utcoffset}: The UTC offset of your computer's current time zone. For example, UTC time gives +0:00. EST time can be -4:00 or -5:00, depending on daylight savings time.
• All time options use UTC by default. To use your computer's local time zone instead, add "local" to the beginning: {localdate}, {localtime}, {localdatetime}, {localday}, {localmonth}, {localyear}, {localhour}, {localminute}, {localsecond}, {localnow}, {localdatenow}, {localtimenow}.

H2O Data:

• {totalsent}: The total number of highlights being sent.
• {booksent}: The total number of highlights being sent to this Obsidian note. If a large note is split into multiple smaller notes, {booksent} will give the total being sent to all of those smaller notes.
• {highlightsent}: This highlight's position in the highlights being sent to this note. For example, "{highlightsent} out of {booksent}" might result in "3 out of 5".

Note: For an example of how to use these, see the default format settings in the plugin's config.

Special Notes:

• After installing, go to Preferences -> Toolbars & menus -> The main toolbar. The Highlights to Obsidian menu button is listed as H2O.
• You can set keyboard shortcuts in Preferences -> Shortcuts -> H2O.
• Source code
• Version history

[petzi]

for this important plugin! I love the feature that each highlight has a link to open up the book in Calibre and to scroll to the exact location of the highlight!

What follows are some suggestions to improve your plugin:

To get better control of sending highlights to Obsidian, I suggest adding the option "Send Highlights of Selected Books to Obsidian". This would confine sending the highlights only of selected books in the Calibre library.

In addition, of the highlight's timestamp, I would need as a sort-key the location of the highlight in the book: Highlights in the first chapter appear before highlights in the second chapter, even if chapter one was later highlighted than chapter two.

The time when highlights were sent does not reflect my local time (UTC + 1). I had to set it manually one hour earlier to get only the latest highlights.

Provide an image (the Obsidian logo?) for the H20 button in the toolbar to get a nicer view in the UI.

Your plugin does not appear in the list of Calibre plugins, even after the installation via the file upload. (Calibre Config -> Advanced Plugin -> Get new plugins).

[jm289765]

@petzi - Thanks for the suggestions. I've added them in v1.1.0.

Originally Posted by petzi

In addition, of the highlight's timestamp, I would need as a sort-key the location of the highlight in the book: Highlights in the first chapter appear before highlights in the second chapter, even if chapter one was later highlighted than chapter two.

To do this, type "location" (without quotes) into the new Sort Key box at the bottom of the config menu.

Unfortunately, it looks like I can't update my first post until I have at least 10 posts. Since I'm not willing to make a bunch of otherwise useless posts, I'll just attach the updated .zip here.

Release notes here. Updated list of formatting options here.

moved attachment

[jm289765]

Originally Posted by petzi

Your plugin does not appear in the list of Calibre plugins, even after the installation via the file upload. (Calibre Config -> Advanced Plugin -> Get new plugins).

This plugin needs to be added to the Plugin Index. Unfortunately, I am unable to PM a mod to request that it be added.

If any Calibre forum mod sees this, can you please add Highlights to Obsidian to the plugin index? Index entry file is attached.

[kovidgoyal]

done.

[DaltonST]

The index is updating hourly from the oldest version in the Original Post because the OP apparently (per his post) has no security to edit his Original Post.

Would a moderator please move the latest version from https://www.mobileread.com/forums/sh...00&postcount=3 to the Original Post so the users can actually get it? Ph.D. students and the like don't have time to hang out in MR in order to discover that their Obsidian plugin for Calibre Annotations is obsolete because MR won't let the developer update it himself because he hasn't posted enough non-value-added comments.

DaltonST

[petzi]

Thanks for the update with features I have suggested. Nice button image!

[petzi]

Hi,

Another suggestion: In addition to the two fields for body formats (with note and without note), please provide a third field for a template, especially a YAML header at the beginning of the file. I am using the Obsidian Templater plugin to create a complex YAML header for my Obsidian notes.

The conversion with Templater with your plugin works well, but currently I get the content of the template with every highlight because I have to include it into the body format field.

Thanks in advance, Peter

[jm289765]

Can a mod please update the original post's .zip file to this version?

Version 1.2.0

• split config menu into two separate menus (it was getting crowded)
• added note headers. similar to the note body, but only sent once each time you send a group of highlights.
• added {utcnow} and {localnow} formatting options

Version 1.1.1

• when saving config, h2o will now verify that the last send time is valid
• changed default sorting to highlight's location in book instead of timestamp
• fixed error when trying to send a large amount of text and/or highlights from a single book
• added a file name length limit of 180 characters

[jm289765]

Originally Posted by petzi

a YAML header at the beginning of the file

With the way I've implemented headers, it'll append another header to the note each time you send highlights. To have a single header at the start of the file, you'll need to send highlights once with headers enabled, and then disable them.

Also worth noting that you can delete the body format but leave the header. Then you'll send only one thing to each note, regardless of how many highlights you have.

[petzi]

It worked!

There are minor issues:

1. Whenever I call "config" or "Configure this plugin" I got the following window: (macOS, Ventura 13.2.1, Intel)

See the first image:
https://imgur.com/a/RFF34VP

The big appearance of this window it not nice, but - as I said - it works.

2. I am using emojis in my title, but I noticed I could not use them in the template field. It is not a big deal, as I can add this part of the title in Obsidian. But still, it defeats the automatic routine.

Some other possible improvements to streamline the workflow:

I am using preferentially the option "Send all highlights of selected books" after I have read a book.

(a) It would be charming to have the possibility of a shortcut for this option. Currently, I could only find three possible shortcut options for

• Resend Highlights to Obsidian
• Send New Highlights of Selected Books
• Send New Highlights to Obsidian

See the second image:
https://imgur.com/a/RFF34VP

In this context I did not understand the last paragraph in your help window:

You can set keyboard shortcuts in Preferences-> Shortcuts-> H20. Some available keyboard shortcuts include CTRL+S, CTRL+E, CTRL+G, CTRL+H, CTRL+], and CTRL+K.

(b) As I am choosing always to transfer all the highlights of just one book, the warning window bothers me:

Are you sure you want to send ALL highlights of the selected books to obsidian? This cannot be undone.

Maybe you could provide an option to hide this dialog?

(c) Similar with the success window: it is nice to know the number of highlights, but my workflow is:

1. Go to Calibre and select the book
2. Transfer all the highlights of this one book (if I had already transferred highlights of this book, I delete the corresponding file in Obsidian. This is much easier as fuddling with "Last time the highlights were sent")
3. The program changes to Obsidian and I am going to revise my highlights.
4. BUT I have to go back and confirm the success dialog of the H20 plugin.

I am not sure what would be a practical solution. Another option to hide this window? Or better: To provide this information into the Obsidian file? (At the beginning or at the end of the highlights?)

I apologize for these many suggestions, but I really love your plugin. It is for me a game changer, especially as Calibre provides very robust exact links to the highlights. These links are important as one has to go back many times to the original text to see the context of the highlights. (I am planning to write an article for my German weblog about your plugin. I will send you the link when this has happened.)

All the best
Peter

[jm289765]

Originally Posted by petzi

1. Whenever I call "config" or "Configure this plugin" I got the following window: (macOS, Ventura 13.2.1, Intel)

See the first image:
https://imgur.com/a/RFF34VP

The big appearance of this window it not nice, but - as I said - it works

Calibre stores window sizes in a file. This lets you resize a window, close it, and reopen it, and it'll remember the size you set it to. Your config window kept the size set by the previous version of H2O. You might be able to fix it by resizing the window with your mouse.

If that doesn't work, try this:

1. In calibre, go to Preferences -> Miscellaneous -> Open calibre configuration folder. Click this and then exit calibre completely.
2. In calibre's configuration folder, there should be a file named "gui.json". Make a backup of it (to be safe) and then open it.
3. Use Ctrl+F to search for "Highlights to Obsidian". There should be 2 results.
   The second result should look like this:
   

     "geometry-of-plugin config dialog:User interface action:Highlights to Obsidian": {

4. Delete the bracketed area after and including this line. The part you delete should look similar to this:
   

     "geometry-of-plugin config dialog:User interface action:Highlights to Obsidian": {
       "frame_geometry": {
         "height": 206,
         "width": 208,
         "x": 660,
         "y": 315
       },
       "full_screened": false,
       "geometry": {
         "height": 176,
         "width": 208,
         "x": 660,
         "y": 345
       },
       "maximized": false,
       "normal_geometry": {
         "height": 176,
         "width": 208,
         "x": 660,
         "y": 345
       },
       "qt": {
         "__class__": "bytearray",
         "__value__": "AdnQywADAAAAAAKUAAABOwAAA2MAAAIIAAAClAAAAVkAAANjAAACCAAAAAAAAAAABgAAAAKUAAABWQAAA2MAAAII"
       },
       "screen": {
         "depth": 32,
         "device_pixel_ratio": 1.25,
         "geometry_in_logical_pixels": {
           "height": 864,
           "width": 1536,
           "x": 0,
           "y": 0
         },
         "index_in_screens_list": 0,
         "manufacturer": "",
         "model": "",
         "name": "\\\\.\\DISPLAY1",
         "serial": "",
         "size_in_logical_pixels": {
           "height": 864,
           "width": 1536
         },
         "virtual_geometry": {
           "height": 864,
           "width": 1536,
           "x": 0,
           "y": 0
         }
       }
     },

5. Notice that this section:
   • starts with "geometry-of-plugin config dialog:User interface action:Highlights to Obsidian"
   • ends with two lines containing one curly bracket each, followed by one line that has a curly bracket and comma.
6. After deleting this and saving the file, calibre will recalculate the default minimum size of the config window.

Important: Make sure you close calibre before modifying gui.json. If you have calibre open, it'll undo your changes.

@petzi, your other suggestions should be done within a day.

[jm289765]

Originally Posted by petzi

2. I am using emojis in my title, but I noticed I could not use them in the template field. It is not a big deal, as I can add this part of the title in Obsidian. But still, it defeats the automatic routine.

Emojis in book titles and in format templates seem to work for me. I tried the using 🌦️ and 🚗, copy/pasted from Wikipedia:

• emojis in title, body, and header format templates are successfully sent to Obsidian
• when the book's title has an emoji, it's successfully sent to Obsidian by using the {title} format option in title, body, and header templates
• emojis can also be used in the vault name

I also tried each of these (except vault name) with the text á͈̗̙̕ͅb̝̘͍̽͌̄͝c̷̝̣̭̣̯͉̺͔̋ͯ͐̈. It worked, but when it was in the title of the note sent to Obsidian, it always made a new note instead of adding to the existing one. This is probably an Obsidian problem, not something I can fix.

[DaltonST]

Originally Posted by jm289765

Can a mod please update the original post's .zip file to this version?

Version 1.2.0

• split config menu into two separate menus (it was getting crowded)
• added note headers. similar to the note body, but only sent once each time you send a group of highlights.
• added {utcnow} and {localnow} formatting options

Your Original Post never got updated, and is still Version 1.1.0.

In order to ensure that a moderator actually reads your posted request, it would be prudent to make a unique posting with your request as the title. Or, if you have security to do so, send them a Private Message. However, since you need 10 basic posts in MR to gain security to edit your own Original Post, a separate post for each new release you publish will get you to that threshold over time.


DaltonST

[jm289765]

Version 1.3.0

• Improved appearance of config menus
• Added options to disable confirmation and success popups when sending highlights
• Enabled making keyboard shortcuts for "Send all" functions
• Added formatting options for how many highlights were sent: {totalsent}, {booksent}, {highlightsent}
• Added formatting options for hour, minute, and second the highlight was made
• Added better formatting options for current time

• {totalsent}: The total number of highlights being sent.
• {booksent}: The total number of highlights being sent to this Obsidian note.
• {highlightsent}: This highlight's position in the highlights being sent to this note. For example, "{highlightsent} out of {booksent}" might result in "3 out of 5".