Kobo highlights grouped by chapter

[ironhak]

Hello everyone, using calibre Kobo Utilies I can extract highlights of a book and the output is something like this:

The Richest Man in Babylon by George S. Clason

Book last read: 2023-09-17 12:45:32
Percentage read: 23%


Chapter 6: The Man Who Desired Gold
Highlight
Chapter progress: 18.64%
Highlight: appeared at the open door. Her furtive glances in his direction reminded him that the meal bag was almost empty and he should be at work finishing the chariot—hammering and hewing, polishing and painting, stretching taut the leather over the wheel rims, preparing it for delivery—so he could collect from his wealthy customer.
			 Nevertheless, his fat, muscular body


Chapter 6: The Man Who Desired Gold
Highlight
Chapter progress: 18.64%
Highlight: liberality, my good friend,” began Kobbi with an elaborate salute. Yet, it does appear they have already been so generous thou needest not to labor. I rejoice with thee in thy good fortune. More, I would even share it with thee. Pray, from thy purse which must be bulging else thou wouldst be busy in your shop, extract but two humble shekels and lend them to me until after the noblemen’s feast this night. Thou wilt not miss them ere they are returned.”
			 “If I did have two shekels,” Bansir responded gloomily,


Chapter 6: The Man Who Desired Gold
Highlight
Chapter progress: 19.49%
Highlight: satisfied to work long hours and spend our earnings freely. We have earned much coin in the years that have passed, yet to know the joys that come from wealth, we must dream about them. Bah! Are we more than dumb sheep? We live in the richest city in all the world. The travelers do say none equals it in wealth. About us is much display of wealth, but of it we ourselves have naught. After half a lifetime of hard labor, thou, my best of friends, hast an empty purse and sayest to me,


Chapter 7: The Richest Man in Babylon
Highlight
Chapter progress: 23.73%
Highlight: than we. You have become the richest man in all Babylon while we struggle for existence. You can wear the finest garments and you can enjoy the rarest foods, while we must be content if we can clothe our families in raiment that is presentable and feed them as best we can. Yet once we were equal. We studied under the same master. We played in the same games. And in neither the studies nor the games did you outshine us. And in the years since, you have been


Chapter 7: The Richest Man in Babylon
Highlight
Chapter progress: 23.73%
Highlight:  bare existence in the years since we were youths, it is because you either have failed to learn the laws that govern the building of wealth or else you do not observe them. Fickle Fate is a vicious goddess who brings no permanent good to anyone. On the contrary, she brings ruin to

Is there a way to group highlgiths by chapter? So having something like:

### Chapter 1 name
- Highlight 1
> Eventual note to highlight 1
- Highlight 2
> Eventual note to highlight 2
- Highlight 3
> Eventual note to highlight 3

### Chapter 2 name 
- Highlight 4
> Eventual note to highlight 4
- Highlight 5
> Eventual note to highlight 5
- Highlight 6
> Eventual note to highlight 6

Thank's.

[JSWolf]

No way to do as you want.

[Quoth]

You'd need to use Regex in a suitable text editor, like Notepad++ on Windows (no relation to Windows Notepad), or a wide range of editors on Linux. I don't know which text editor on Mac, but likely there is one for programmers that does regex.

[ironhak]

Originally Posted by Quoth

You'd need to use Regex in a suitable text editor, like Notepad++ on Windows (no relation to Windows Notepad), or a wide range of editors on Linux. I don't know which text editor on Mac, but likely there is one for programmers that does regex.

Regex is basically an enhanced find & replace, how can you possibly use regex to group highlights based on chapter?

[ironhak]

Thank's to some AI help I was able to create a code that does exactly what I want:
Python Code:

Spoiler:

import re

# Read the input.txt file
with open('input.txt', 'r') as file:
    text = file.read()

# Initialize variables to store the data
chapters = []

# Split the text into chapters based on the "Chapter X" pattern
chapter_texts = re.split(r'Chapter \d+:', text)[1:]

# Function to remove leading spaces from each line
def remove_leading_spaces(text):
    lines = text.split('\n')
    cleaned_lines = [line.lstrip() for line in lines]
    return '\n'.join(cleaned_lines)

# Function to replace \n\n with \n\n  (two spaces)
def replace_double_newline(text):
    return text.replace('\n\n', '\n\n  ')

# Loop through each chapter text to extract information
for chapter_text in chapter_texts:
    chapter_data = {}
    
    # Extract Chapter name
    chapter_name = re.search(r'(.+?)\n', chapter_text)
    if chapter_name:
        chapter_data['Chapter name'] = chapter_name.group(1).strip()
    
    # Extract Highlight text and remove leading spaces
    highlight_info = re.search(r'Highlight:(.*?)\n(?:Notes:|Chapter \d+:|$)', chapter_text, re.DOTALL)
    if highlight_info:
        highlight_text = remove_leading_spaces(highlight_info.group(1)).strip()
        chapter_data['Highlight'] = replace_double_newline(highlight_text)
    
    # Extract Notes if present and remove leading spaces
    notes_info = re.search(r'Notes:(.*?)(?:(?=\nChapter \d+:)|(?=\nNotes:)|$)', chapter_text, re.DOTALL)
    if notes_info:
        notes_text = remove_leading_spaces(notes_info.group(1)).strip()
        chapter_data['Notes'] = replace_double_newline(notes_text)
    
    # Append chapter data to the chapters list
    chapters.append(chapter_data)

# Create a Markdown file
with open('output.md', 'w') as md_file:
    current_chapter = None
    for chapter in chapters:
        if chapter['Chapter name'] != current_chapter:
            md_file.write(f'### {chapter["Chapter name"]}\n')
            current_chapter = chapter['Chapter name']
        
        if "Highlight" in chapter:
            md_file.write(f'- {chapter["Highlight"]}\n')
        
        if "Notes" in chapter:
            md_file.write(f'  > {chapter["Notes"]}\n\n')

print("Conversion completed. Markdown data saved to 'output.md'.")

Sample input of highlights extracted with Kobo Utilities

Spoiler:

The Richest Man in Babylon by George S. Clason

Book last read: 2023-09-17 16:41:41
Percentage read: 28%


Chapter 7: The Richest Man in Babylon
Highlight
Chapter progress: 27.97%
Highlight: And lastly, you have learned to make gold work for you. You have taught yourself how to acquire money, how to keep it, and how to use it. Therefore, you are competent for a responsible position. I am becoming an old man. My sons think only of spending and give no thought to earning. My interests are great, and I fear too much for me to look after. If you will go to Nippur and look after my lands there, I shall make you my partner and you shall share in my estate.’
			 “So, I went to Nippur and took charge of his holdings, which were large. And because I was full of ambition


Chapter 7: The Richest Man in Babylon
Annotation
Chapter progress: 27.97%
Highlight: I did share in his estate as he had arranged under the law.” 
			 
			 So spake Arkad, and when he had finished his tale, one of his friends said, “You were indeed fortunate that Algamish made of you an heir.”

Notes: Text wirh paeagraph break


Chapter 7: The Richest Man in Babylon
Highlight
Chapter progress: 28.81%
Highlight: I would do it. If on the seventh day I passed by without remembering, I would not say to myself, Tomorrow I will cast two pebbles which will do as well. Instead, I would retrace my steps and cast the pebble. Nor on the twentieth day would I say to myself, Arkad, this is useless. What does it avail you to cast a pebble every day? Throw in a handful and be done with it. No, I would not say that nor do it. When I set a task for myself, I complete it. Therefore, I am careful 


Chapter 7: The Richest Man in Babylon
Annotation
Chapter progress: 28.81%
Highlight: 	 “You had strong willpower to keep on after you lost your first year’s savings. You are unusual in that way,” 
Notes: Some note


Chapter 7: The Richest Man in Babylon
Annotation
Chapter progress: 29.66%
Highlight: thou also will be numbered among them. Therefore, invest thy treasure with greatest caution that it be not lost. Usurious rates of return are deceitful sirens that sing but to lure the unwary upon the rocks of loss and remorse. Provide also that thy family may not want should the gods call thee to their realms. For such protection it is always possible to make provision with small payments at regular intervals. 
			 “Therefore, the provident man delays not in expectation of a large sum becoming available for such a wise purpose. Counsel with wise men. Seek the advice of men whose daily work is handling money. Let them save you from such an error as I myself made
Notes: AGAIN


Chapter 8: Seven Cures for a Lean Purse 
Annotation
Chapter progress: 32.2%
Highlight: “We shall now consider the first cure.”
			 The First Cure: Start thy purse to fattening.
			 Arkad addressed a thoughtful man in the second row. 
			 “My good friend, at what craft workest thou?”
			 “I,” replied the
Notes: This is text wirh paragraph break


Chapter 9: The First Cure: Start thy purse to fattening.
Annotation
Chapter progress: 33.05%
Highlight: 	 Thereupon, they agreed that it was so. 
			 “Then,” continued Arkad, “if each of you desireth to build for himself a fortune, is it not wise to start by utilizing that source of wealth which he already has established?” 
			 To this they agreed.
			 Then Arkad turned to a humble man who had declared himself an egg merchant. “If 
Notes: Hello

Output: properly formatted to account even for paragraph breaks inside the highlighted text:

### The Richest Man in Babylon
- And lastly, you have learned to make gold work for you. You have taught yourself how to acquire money, how to keep it, and how to use it. Therefore, you are competent for a responsible position. I am becoming an old man. My sons think only of spending and give no thought to earning. My interests are great, and I fear too much for me to look after. If you will go to Nippur and look after my lands there, I shall make you my partner and you shall share in my estate.’
“So, I went to Nippur and took charge of his holdings, which were large. And because I was full of ambition
- I did share in his estate as he had arranged under the law.” 

  So spake Arkad, and when he had finished his tale, one of his friends said, “You were indeed fortunate that Algamish made of you an heir.”
  > Text wirh paeagraph break

- I would do it. If on the seventh day I passed by without remembering, I would not say to myself, Tomorrow I will cast two pebbles which will do as well. Instead, I would retrace my steps and cast the pebble. Nor on the twentieth day would I say to myself, Arkad, this is useless. What does it avail you to cast a pebble every day? Throw in a handful and be done with it. No, I would not say that nor do it. When I set a task for myself, I complete it. Therefore, I am careful
- “You had strong willpower to keep on after you lost your first year’s savings. You are unusual in that way,”
  > Some note

- thou also will be numbered among them. Therefore, invest thy treasure with greatest caution that it be not lost. Usurious rates of return are deceitful sirens that sing but to lure the unwary upon the rocks of loss and remorse. Provide also that thy family may not want should the gods call thee to their realms. For such protection it is always possible to make provision with small payments at regular intervals. 
“Therefore, the provident man delays not in expectation of a large sum becoming available for such a wise purpose. Counsel with wise men. Seek the advice of men whose daily work is handling money. Let them save you from such an error as I myself made
  > AGAIN

### Seven Cures for a Lean Purse
- “We shall now consider the first cure.”
The First Cure: Start thy purse to fattening.
Arkad addressed a thoughtful man in the second row. 
“My good friend, at what craft workest thou?”
“I,” replied the
  > This is text wirh paragraph break

### The First Cure: Start thy purse to fattening.
- Thereupon, they agreed that it was so. 
“Then,” continued Arkad, “if each of you desireth to build for himself a fortune, is it not wise to start by utilizing that source of wealth which he already has established?” 
To this they agreed.
Then Arkad turned to a humble man who had declared himself an egg merchant. “If
  > Hello

Given I'm not a coder and ChatGPT did almost everything, if someone is willing to improve the thing it would be nice, for now it seems to work.

[Quoth]

No, a human did it and the Open AI people scraped it.