[GUI Plugin] Hyphenate This!

[shamanNS]

Originally Posted by nem_mil

Right, but the dictionaries for two alphabets are already in two distinct files, recognized by Calibre as two separate languages. I got them here: https://devbase.net/dict-sr/

So, the Latin one is tagged as "Serbo-Croatian" (the file is titled hyph_sh.dic), and I run the Latin book only through that dictionary, never through the other one. The Cyrillic one is hyph_sr and I use it only for the Cyrillic books, but it doesn't work. From what I understand, Serbo-Croatian and Serbian are seen by Calibre and Hyphenate This as two separate languages, which would mean that I never use one dictionary for two alphabets, right?

Here is how it looks on my Calibre: https://ibb.co/BCmCBR4



Why do you say that it doesn't work properly?

Sorry, I didn't pay sufficient attention while reading your previous comment. I wrongly interpreted and misunderstood that you've edited the dictionary file (to switch it from representing "serbo-croatian" to "serbian latin") not the ebooks language metadata.


Anyways, I just did couple of test in Calibre and I've figured out what is the problem: this plugin doesn't like the character encoding used on that "hyph_sr.dic" (for Serbian Cyrillic). Converting the encoding to UTF-8 (that gets you the exact same file as the one attached by @BeckyEbook) fixes the problem.

While testing things I've realised that Calibre's built-in "Add soft hyphens" option in "Polish books" tool / plugin actually uses the same file that you've downloaded (with the same character encoding) and hyphenation works correctly when using "Polish books" option.

Long story short: download and use (add to HyphenateThis plugin) the file attached by @BeckyEbook and it will work correctly for Serbian Cyrillic books.:

fixed Serbian Cyrillic hyphenation dictionary


What I've written previously still stands: you can't "install" at the same time two hyphenation dictionaries if both get imported as "serbian". That issues was known to me every since this HyphenateThis plugin has first appeared.
You've gotten around it by changing book's language metadata to "Serbo-Croatian" and using dictionary for that. IMHO that workaround (mangling each Serbia Latin book's language metadata) is not acceptable. If I read serbian language books I would probably prefer continuously installing the correct serbian hyphenation dictionary .. or even using Calibre's "Polish Books" tool since that seams to be able to figure out when to use which of the two hyphenation dictionaries.

Hyphenation dictionaries that Polish Books uses can be found at:

C:\Program Files\Calibre2\app\resources\hyphenation\dictionaries.tar.xz

[JSWolf]

Don't use soft-hyphens for ePub. They do not work well enough. Use a hyphenation dictionary that does work properly. But until I get the language codes, I am not going to try to build an install file for the hyphenation dictionaries.

[shamanNS]

Nobody cares what you think should or shouldn't be used and done

As for language codes: if you were more specific what form of language codes ("sr", "srp", "sr-RS","sr-Latn-RS", "sr-Cyrl-RS"...) you're expecting to get maybe someone would have already provided them to you. Also, you could've Googled that information 9999 times already since yesterday.

[JSWolf]

Originally Posted by shamanNS

Nobody cares what you think should or shouldn't be used and done

As for language codes: if you were more specific what form of language codes ("sr", "srp", "sr-RS","sr-Latn-RS", "sr-Cyrl-RS"...) you're expecting to get maybe someone would have already provided them to you. Also, you could've Googled that information 9999 times already since yesterday.

But I need to know what it is these eBooks are using. Sure I could have looked it up. But then I could have been wrong.

I'm trying to do what should work instead of what may not work.

[shamanNS]

Used where / for what?

1) dc:language in .opf
2) xml:lang attribute on XHTML elements
3) stuff like Hunspell and OpenOffice / LibreOffice dictionaries

"dc:language" most of the time will have either 2 letter language code ("sr") or 3 letter ISO_whatever variant ("srp"). The same value is used for both Latin or Cyrillic script / alphabets epubs.
There aren't 2 or 3 letter codes that indicate the script used.

Stuff like Windows locale and Dot NET locale that support "extended language codes" use that form of "2 letter language code + 4 letter script + 2 letter country code" ( so "sr-Latn-RS" and "sr-Cyrl-RS")

No idea how Kobo's hyphenation dictionaries "encode" that type of info. I've noticed that for example KOReader has hyphenation rules only for Serbian Cyrillic and not for Serbian Latin.

[nem_mil]

I just tried the file uploaded by BeckyEbook and it works perfectly!!! Thank you all for your help

I was even thinking about using some condensed fonts to solve the issue since justified text leaves so much space between words in Serbian Cyrillic, but this is a much better solution.

[FelixKrull]

I just discovered a bug or something:

I have three dictionaries installed, and hyphenating works perfectly.

Now I added a forth dictinonary (spanish, from here: https://extensions.libreoffice.org/e...h-dictionaries)

and the plugin stops working with an error:



After removing the dictionary everything works fine again.

[JSWolf]

Originally Posted by FelixKrull

I just discovered a bug or something:

I have three dictionaries installed, and hyphenating works perfectly.

Now I added a forth dictinonary (spanish, from here: https://extensions.libreoffice.org/e...h-dictionaries)

and the plugin stops working with an error:



After removing the dictionary everything works fine again.

You could try removing one of the three working dictionaries and then add in the Spanish dictionary in case it's 4 dictionaries that don't work.

[FelixKrull]

Thanks, I'll try this (thought it would be awkward as I need all of the other dictionaries...)

[JSWolf]

Originally Posted by FelixKrull

Thanks, I'll try this (thought it would be awkward as I need all of the other dictionaries...)

You can always delete the Spanish dictionary and add back the one you've removed.

[Miq 528]

Hello, I am going to self publish a book and this plugin will improve it a lot, thank you.

I can touch some things in the html, but my relative ignorance makes me ask this: How can I tag a phrase or word that I want to not being hyphenated?

In the configuration of your plugin I have seen that of:

"Comma separated list of tags to:
Ignore
h1, h2, h3"

I don't know how to put this into the html, but I think that with one example would be enough, here is the (shortened) Calibre html of the title page, could you put there that tag for that the title and the "Author's name" not be hyphenated?
Thank you.
Miqbenn.


<?xml version='1.0' encoding='utf-8'?>
<html xmlns="http://www.w3.org/1999/xhtml" lang="es" xml:lang="es">
<head>
<title>Unknown</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
<link rel="stylesheet" type="text/css" href="stylesheet.css"/>
<link rel="stylesheet" type="text/css" href="page_styles.css"/>
</head>
<body class="calibre">
<p class="block_">*</p>
<p class="block_1">The Daughter of the Artisan</p>
<p class="block_1">Author's name</p>
<p class="block_">*</p>
</body></html>

[chancerlane]

Can any of you guy's direct me to a good English Hyphenation Dictionary? OP's post is slightly confusing to me and couldn't find a standard English Hyphenation dictionary is there a good one anyone can give me a direct link to?

[waka]

This works great! Thank you very much!
It improves the reading experience a lot for the kindle's default reader.
I might not even jailbreak my device if I know this trick earlier!

[Ialou]

I have an AresNote likebook ereader. He is basically a modified android. It has a hyphenation system, but it doesn't work well in my language. Is there any way to install the dictionary on the new device? Can anyone tell me?

Originally Posted by SauliusP.

Hyphenate This! will add soft hyphens to your ebook and add even better feel of a real book!

Supports EPUB and AZW3 formats (no MOBI, even with KF8 inside, convert instead).

If you have a Kindle (with AZW3/KF8 support) this plugin will explode the book, add soft-hyphens and rebuild it back.

Use hyphenation dictionaries from Apache OpenOffice Extensions. Download one and add to the plugin via its settings.

This plugin is primarily targeted for Kindle users reading AZW3/KF8 format books, as Kindle does not support hyphenation itself. However, recent firmwares added support of soft-hyphenation. So if the book is pre-hyphenated, Kindle will display it correctly. Text search and other features remain. Note, that if you hyphenate EPUB or AZW3 and convert to "old" MOBI, hyphenation won't work.

Some EPUB readers have native hyphenation, but if you read some exotic language (like me), hyphenation support might be poor or not present at all. Luckily, Libre/Open Office dictionaries are implemented for quite a lot of languages.

Note. Not all EPUB readers support soft-hyphens in the way expected. Some hyphenate, but do not show dashes. Some display correctly, but lack search feature. So try it yourself and decide if it is any good. As per discussion further in this thread:

• Sony devices seem to split text on soft-hyphens, but do not display dashes. Not acceptable for Sony users.
• Kobo seems to display hyphenation correctly, but text search is ruined.

CAUTION! In versions before (and including) 0.9.26 of Calibre there is a flaw with AZW3 explosion/rebuilding. You might loose picture content. So please back-up your AZW3, if it is the only and original version of the book you have!

CAUTION! In versions before 0.9.24 of Calibre there is a flaw in support of AZW3 explode and rebuild workflow. TOC might be corrupted as well as quick jump through chapters! Might not be the case for you, but be warned!

Illustrations. I have added screenshots from my Kindle with English book. However, English is quite compact and hyphenation does not show all its beauty. So I've also added two screenshots with Lithuaniant text, where hyphenation is more obvious. Of course, text will look like wingdings to most of you, but just try too see the difference :-)

Spoiler:

English text, original:


English text, soft-hyphenated:


Lithuanian text, original:


Lithuanian text, soft-hyphenated:

User Guide

Spoiler:

Install plugin and download "OXT" dictionaries from link above. Open plugin's settings via menu and add those dictionaries. After dictionaries are added to the plugin, downloaded files are removed, plugin stores hyphenation information inside it's settings directory.

NOTE. You may also add hyphenation dictionary directly, i.e. appropriate "DIC" file, extracted from "OXT" (OXT is simly a ZIP file). "DIC" file must be named "hyph_<language code>.dic". E.g. "hyph_en_US.dic" or "hyph_ru.dic".

NOTE 2. I have tested lots of "OXT" dictionaries. Surprisingly, some of them include hyphenation file, but it is not included in the descriptor (plugin uses descriptor to find out the hyphenation dictionary inside "OXT" archive). So if you add "OXT" dictionary, but no new dictionary appears in the list, try to open "OXT" file with some archive manager and search for "hyph*.dic" file there. If it is present, extract it and add directly. If not, you're not lucky.

Settings window:



Simple part:
Install or remove dictionaries here and specify the minimum length of the word to be hyphenated.

Advanced part:

Hyphenation limits

Some of the hyphenation dictionaries contain special directions: LEFTHYPHENMIN and RIGHTHYPHENMIN. They limit syllable length on either left or right side of word. Example in the picture is 2 characters on left (overwritten with 3) and 3 characters on right for English dictionary. Some dictionaries do not contain these directions, then default limit is 2. If you don't like default or included limits, you can edit limits for each dictionary separately by marking "Override" tick mark.

Tags to ignore/parse

Some people pointed out, that there is no real (and aesthetic) need to hyphenate chapter names. Those are usually enclosed in heading tags: h1, h2 etc. I have added possibility to ignore any tags. Defaults are three headings.
You might also want to hyphenate only particular tags' content. In the example these are p and td (paragraph and table cells).
Special note. If in the "parse" tags you enter p, that means all paragraphs will be parsed and hyphenated, including their child tags, like span, em, strong etc. If you want some special tags to be ignored inside p, add them to "ignore" list. In such case you might configure some particular tags inside p to be ignored, like em, for example.

Custom column

Hyphenation status can be saved to custom column of type "Text, column shown in the tag browser". User also can define, what to write to that custom column, when hyphenation was performed and when hyphens were removed. If column name is empty, status is not written anywhere.

Next, everything is simple. Choose book with EPUB and/or AZW3 formats, click plugin's icon, choose one of the formats and click OK. Book will be hyphenated. There is also handy action to remove soft-hyphens from book via menu.

Version history:

Spoiler:

Version 0.1.3 2020-10-01
Compatibility upgrade for Python 3 and Calibre 5

NB : Versions 0.1.0 ->0.1.2 defunct betas

Version 0.0.9 2019-11-14

Version 0.0.9 2019-11-14
Fix for uppercase dictionary description
(prevented to add new Russian and Swedish dictionaries).

Version 0.0.8 2014-08-08
Get ready for Calibre 2 with Qt5!

Version 0.0.7 2013-04-22
Fix for unicode support of custom text in hyphenated custom column.

Version 0.0.6 2013-04-09
Added custom column to save hyphenation status.

Version 0.0.5 2013-03-29
Community requests and other enhancements

• Added limits of syllable splits on left and right sides.
• Added override of the syllable limits via settings.
• Added "tags to ignore" and "tags to parse" lists via settings.
• Added nice icon and generally beautified settings dialogue (gets complex).

Inspired by active interest and donations (of course).

Version 0.0.4 2013-03-26
Shortened toolbar button label as per community requests.

Version 0.0.3 2013-03-19
Fixed some issues on user feedback.
Added internal Calibre's HTML parser to avoid encoding problems.
Changed text parsing to XML parsing, much faster and efficient.

Version 0.0.2 2013-03-18
Fixes of the FAIL of first release.

Version 0.0.1 2013-03-18
The very first version of the plugin.
Soft-hyphenation of EPUB and AZW3.

[JSWolf]

Originally Posted by Ialou

I have an AresNote likebook ereader. He is basically a modified android. It has a hyphenation system, but it doesn't work well in my language. Is there any way to install the dictionary on the new device? Can anyone tell me?

The problem is that a lot of Reading programs do not like soft-hyphens. But you can try to see if they work with the program you are using to read eBooks.

The other thing is, if you are going to ask for help, you need to give enough information for someone to actually help. What language do you read eBooks?