Simplifying Kindle notebook exports

[archz2]

How can I simplify exported highlights which are exported?
Every highlight has a location number and chapter number.





A simplified and easier version required looks like.


Chapter 1

Highlight 1
Highlight 2
Highlight 3
Highlight 4



Chapter 2

Highlight 1
Highlight 2
Highlight 3
Highlight 4


Manually cleaning the location number is tedious. Any free tool to do the cleanup?

[mergen3107]

Try KindleMate. It has flexibility on what parameters you want to export.

[tomsem]

This looks like the HTML export from the Kindle app, not from Kindle?

I would just open with Word (or anything that imports HTML) and spend a few minutes editing it. With some scripting skills you could probably automate edit of the HTML since it has consistent structure/tagging.

I've never used KindleMate (it is Windows only), but it relies on the Clippings file, which does not log chapter headings at all (nor does direct export from Kindle). I assume that's why you are using the Kindle app export.

[archz2]

Originally Posted by mergen3107

Try KindleMate. It has flexibility on what parameters you want to export.

Thanks! Quite useful. How do I enable the chapter and subheading also for exporting? I couldn't find any option. Their user guide and FAQ doesn't cover this aspect either.

[archz2]

Originally Posted by tomsem

This looks like the HTML export from the Kindle app, not from Kindle?

I would just open with Word (or anything that imports HTML) and spend a few minutes editing it. With some scripting skills you could probably automate edit of the HTML since it has consistent structure/tagging.

I've never used KindleMate (it is Windows only), but it relies on the Clippings file, which does not log chapter headings at all (nor does direct export from Kindle). I assume that's why you are using the Kindle app export.

Yes exactly. It's an html export from the app.
I'm using Kindle app export because there's no share highlights button appearing in my Kindle device. This has happened with many books that I shared through send-to-kindle service. The export highlights option shows up only in the app.

And there's one book that I sideloaded through USB. Surprisingly, export highlights button is showing up for it. But unfortunately, it only exports a blank page with name, author and photo of the book at the top.

[archz2]

Any tool to clean the html file so that it puts only one heading and classifies the highlights accordingly under one chapter and one subheading?

[tomsem]

It appears this will happen with ToC's with more than one level: single level ToCs seem to work okay, chapter names are in sectionHeading.

[archz2]

Originally Posted by tomsem

It appears this will happen with ToC's with more than one level: single level ToCs seem to work okay, chapter names are in sectionHeading.

In the screenshot I shared, the chapter name is appearing with every single highlight. Chapter name is a first/one level category. Right?

How to make it appear only once, and then all the highlights are organized under a single chapter heading?

[tomsem]

Originally Posted by archz2

In the screenshot I shared, the chapter name is appearing with every single highlight. Chapter name is a first/one level category. Right?

How to make it appear only once, and then all the highlights are organized under a single chapter heading?

Yes, it's clear what you are asking for. But in my experiments, some books export as you would wish, and the ones that do not have an additional level of hierarchy to them (for example collected works, omnibus editions etc.).

[DNSB]

Originally Posted by archz2

In the screenshot I shared, the chapter name is appearing with every single highlight. Chapter name is a first/one level category. Right?

How to make it appear only once, and then all the highlights are organized under a single chapter heading?

I've seen quite a few books where the book name is wrapped in h1 tags, the parts are wrapped in h2 tags and the chapters are wrapped in h3 tags.

[tomsem]

Okay here is a first take at a script to add a chapter section header and remove the chapter strings from the note headers. It adds a bullet character to the chapter strings so they are set off from the 'higher level' section header. It will output new html files with '-new' appended to the original filename.

If it does not find 'chapter pattern' in the note headings, it will not do anything.

You can use calibre to run it:

[path to calibre executables]calibre-debug gather_chapter_notes.py html1 [html2, ...]

gather_chapter_notes.py

from re import match, DOTALL
from sys import argv

from bs4 import BeautifulSoup

chapter_pattern = r".*? - (.*?)( > ).*"


def gather_chapter_notes(html: str):
    soup = BeautifulSoup(html, 'html.parser')
    title_insert = {}
    remove_these = set()
    for note_heading in soup.find_all('div', class_='noteHeading'):
        content = note_heading.contents[-1]
        if matches := match(chapter_pattern, content, flags=DOTALL):
            title, token = matches.groups()
            if title not in title_insert:
                title_insert[title] = note_heading
            remove_these.add(f'{title} > ')

    for title, node in title_insert.items():
        title_section = soup.new_tag('div', attrs=[('class', 'sectionHeading')])
        title_section.string = f'● {title}'
        node.insert_before(title_section)

    html = str(soup)
    for remove_this in remove_these:
        html = html.replace(remove_this, '')
    return html


for arg in argv[1:]:
    with open(arg) as f:
        html_text = f.read()

    new_html = gather_chapter_notes(html=html_text)

    with open(arg.replace('.html', '-new.html'), 'w') as f:
        f.write(new_html)

[archz2]

Originally Posted by tomsem

Okay here is a first take at a script to add a chapter section header and remove the chapter strings from the note headers. It adds a bullet character to the chapter strings so they are set off from the 'higher level' section header. It will output new html files with '-new' appended to the original filename.

If it does not find 'chapter pattern' in the note headings, it will not do anything.

You can use calibre to run it:

[path to calibre executables]calibre-debug gather_chapter_notes.py html1 [html2, ...]

gather_chapter_notes.py

from re import match, DOTALL
from sys import argv

from bs4 import BeautifulSoup

chapter_pattern = r".*? - (.*?)( > ).*"


def gather_chapter_notes(html: str):
    soup = BeautifulSoup(html, 'html.parser')
    title_insert = {}
    remove_these = set()
    for note_heading in soup.find_all('div', class_='noteHeading'):
        content = note_heading.contents[-1]
        if matches := match(chapter_pattern, content, flags=DOTALL):
            title, token = matches.groups()
            if title not in title_insert:
                title_insert[title] = note_heading
            remove_these.add(f'{title} > ')

    for title, node in title_insert.items():
        title_section = soup.new_tag('div', attrs=[('class', 'sectionHeading')])
        title_section.string = f'● {title}'
        node.insert_before(title_section)

    html = str(soup)
    for remove_this in remove_these:
        html = html.replace(remove_this, '')
    return html


for arg in argv[1:]:
    with open(arg) as f:
        html_text = f.read()

    new_html = gather_chapter_notes(html=html_text)

    with open(arg.replace('.html', '-new.html'), 'w') as f:
        f.write(new_html)

Thanks. How do I use this script to clean the html file? Can you please share the steps?

[Quoth]

Originally Posted by archz2

Thanks. How do I use this script to clean the html file? Can you please share the steps?

It's in the post you quoted

[path to calibre executables]calibre-debug gather_chapter_notes.py html1 [html2, ...]

[archz2]

I don't understand. If I double click the file titled 'gather_chapter_notes.py' in the zip, nothing happens.

I don't know any coding language.

[Quoth]

You don't double click it. It's a console/terminal command.