Pocketbook dictionary format revisted

[InMyPocket]

Hi,


I "hacked" the 3.1.3 version of the Penelope tool : https://github.com/pettarin/penelope to support XDXF file instead of the suggested XML format. (The generated file extension is .xdxf and the tags are replaced by the XDXF standard).
It also manage some problems with encoding and escaping done by the original Penelope tool.

The file is availables here :
https://gofile.io/d/mTBOqX

penelope-3.1.3-XDXF-InMyPocket.zip


I also created a bundle with the tools required to convert files to Pocketbook .dic file.
It includes:
* the above hacked penelope tool;
* the Pocketboot converter tool with several locales;
* a "sed" tool for Windows to make some quick cleaning when necessary.

This bundle also included a demo of 2 Stardict files and the 2 converted files:
* dict-fr.zip : is a Stardict version of the Wiktionnaire (date 01 feb 2021) from this project: https://github.com/BoboTiG/ebook-reader-dict
* Nouveau Littre 2011 (from Bookeen by Penelope)_reconstructed.Stardict.zip generated by Marksimus (see https://www.mobileread.com/forums/sh...&postcount=276)

Two ".bat" are provided to automate the convertion process explained in the examples below.


The bundle is available here:
https://gofile.io/d/R5Oin6


Examples:

################
Wiktionnaire
################

First get the Wiktionnaire here (in Stardict format):
https://github.com/BoboTiG/ebook-reader-dict
https://github.com/BoboTiG/ebook-rea...eleases/tag/fr

1) Convert the french wiktionary (Stardict format) to XDXF
python penelope -i dict-fr.zip -j stardict -o TEMPDIC -p xml -f fr -t fr -d

2) Make some cleaning and truncate lines to avoid crash of Pocketbook "convert.exe" tools.
sed -f XML2XDXF.txt TEMPDIC.xdxf > Wiktionnaire.xdxf


XML2XDXF.txt content:

s/&lt;/</g
s/&gt;/>/g
s/<dictionary>/<xdxf lang_from="fr" lang_to="fr" format="visual">\n<full_name>Wiktionnaire<\/full_name>\n<description>Wiktionary FR<\/description>\n<lexicon>/
s/<\/dictionary>/<\/lexicon>\n<\/xdxf>/
s/<entry>/<ar>/g
s/<\/entry>/<\/ar>/g
s/<key>/<k>/g
s/<\/key>/<\/k>/g
s/<full_name>.*<\/full_name>/<full_name>Wiktionnaire<\/full_name>/
s/\(<\/li>\)/\1\n/g
s/\(<\/*\)\(ol\|def\)\(>\)/\1\2\3\n/g
s/#\([0-9a-fA-F]\)\{6\}//g
s/<li>/<li># /g
s/\(<i>\)*(Siècle à préciser)\(<\/i>\)*//g
s/\(<i>\)*(Date à préciser)\(<\/i>\)*//g

3) Convert to PB format
convert Wiktionnaire.xdxf fr



######################
Nouveau Littré:
######################

First get the Nouveau Littré (in stardict format) from this post:
https://www.mobileread.com/forums/sh...&postcount=276
Download link:
https://markismus.stackstorage.com/s/ueuYYszltpoI9yn

1) Convert the french wiktionary (Stardict format) to XDXF

python penelope -i "Nouveau Littre 2011 (from Bookeen by Penelope)_reconstructed.Stardict.zip" -j stardict -o TEMPDIC -p xml -f fr -t fr

2) Make some cleaning and truncate lines to avoid crash of Pocketbook "convert.exe" tools.

sed -e "s/<f>\xe2\x99\xa6<\/f>/\n<br\/>#/g" -e "s/<f>\xe2\x80\xa2<\/f>/\n<br\/>#/g" -e "s/<\/ar>/<\/ar>\n/g" -e "s/<full_name>Dictionary Name<\/full_name>/<full_name>Nouveau Littr\xc3\xa9 2011-PM<\/full_name>/g" TEMPDIC.xdxf > NouveauLittre.xdxf

3) Convert to PB format
convert NouveauLittre.xdxf fr

[gosertje]

@Markismus

Hello,

I am not so good in these things. I managed to get some dictionaries (*.dic) you made working on my InkPad 3.
Now, when reading English I can check words with Webster or Oxford.
Thank you very much.

Do you also have a English - Dutch dictionary for a InkPad 3, or could you make one for me (Or provide a kind of manual how to do it)
I would be very much obliged.

I found this one in stardict format for what it is worth.

thanks

gosertje

[InMyPocket]

Originally Posted by gosertje

@Markismus

Hello,

I am not so good in these things. I managed to get some dictionaries (*.dic) you made working on my InkPad 3.
Now, when reading English I can check words with Webster or Oxford.
Thank you very much.

Do you also have a English - Dutch dictionary for a InkPad 3, or could you make one for me (Or provide a kind of manual how to do it)
I would be very much obliged.

Hi,

As far that I understood, Marksimus moved to a Kobo reader. So not sure he will continue to invest effort for the Pocketbook world...

But, by chance I'm just testing convertion of bilingual dictionnaries from the Wikdict project (see wikdict.com). I'm converting files from the Kobo format (I couldn't achieve any usable result with the stardict format for bilingual dictionaries). I'm pretty satisfied with my EN-FR version. I'll try to make de conversion for EN-NL in the next few days. And I'll post it here.

[Markismus]

@gosertje It is quite hard to find a decent English-Dutch dictionary. My mother's been complaining for years, that the Kobo dictionary never has the words she's looking up. And that is a commercial dictionary. The Pocketbook also has its Lingvo dictionaries: Apparently, you found their English words not enough, too, if you're using the Oxford 2nd Edition.

So the conversion isn't a problem. It's the dictionary. Find me a good one and I'll convert it for you.

@InMyPocket Good idea to rip the Kobo dictionaries. I haven't looked into that, but I know that there is a whole toolset over at the Kobo section that hacks into their dictionaries. If you have it, I would like to have the xdxf- or xml- or stardict format from your Kobo conversion.

If anyone has decent English-Dutch dictionary: If you link it to me in a PM, I'll rip it so that nothings links back to you.

[InMyPocket]

Hi,

I converted the EN-NL and the EN-FR dictionnaries from the KOBO version of Wikdict.com project (see also https://www.mobileread.com/forums/sh...d.php?t=331252).

Once the source files are unzipped, I used almost the *nix sed tool only. I will document the process later on.

I changed the presentation/formating to display the language (en, fr, nl) of the definition. I think that it is much convenient this way. Let me know what your think.

Pocketbook .dic and .xdxf file of EN-NL and EN-FR can be downloaded here:
https://gofile.io/d/lC0gnJ

Enjoy !

[InMyPocket]

Hi,

I updated the dictionary files and added some new. So these dictionaries can be downloaded:

EN-FR
FR-EN
EN-NL
NL-FR
FR-NL
NL-FR

I also added the scripts files (Windows .bat) I used to make de convertion. They use the some tools of "bundle" I posted few days ago: sed and PB converter.

[gosertje]

Hi Markismus and InMyPocket,

Thank you very much for the quick response and results.
I will start to use the one coverted from the Wikdict.com

@Markismus: I guess I have the same problem as your mother, I can never find the translation for the words I am looking for. So maybe the issue is in the dictionairy itself and I need to look for a really good one.

I have a good eng-nl dictionary, but that is .epub format.
I guess that cannot be converted.

[InMyPocket]

Originally Posted by gosertje

I have a good eng-nl dictionary, but that is .epub format.
I guess that cannot be converted.

Send me this file, I will take a look at it. As EPUB is essentially an HTML format (as th Kobo dic). If the file contains enough, there is good chance that I'll be able to convert.

[InMyPocket]

Concerning the Wikdict dictionaries I converted, I saw that the EN-xx file miss some words. Ex: words/expressions between "trough" and "trouser press" are missing: trouser, troupe,...
It is an error in the source file, not I the convertion process. I sent an email to Karl Bartel, the maker of these file to raise the problem. He would probably be able to fix that easily... Once done, I'll update the dictionaries for PB.

[gosertje]

Hi,

Here is the file.
Would be great if you manage to convert it
It is a pretty good dictionary.
It is also on my old Sony PRS-T2 reader.

Maybe also something for Markismus mother.

[Markismus]

@gosertje I've written an extension to the script, to convert these epub-dictionaries. If you have more, especially Van Dale, please share: I love to test the script and convert more.

The result in Koreader looks like this:


The files can be found at my share on pCloud.

[Markismus]

I've also included the Kobo conversion of the dictionary. If someone would test that one on a Kobo, I would be quite grateful.

Example instructions are here.

Basically, you download the Kobo English-Dutch dictionary via the menu, connect the e-reader to your PC/laptop and browse for the directory .kobo/dict. You rename it to dicthtml.zip.backup and replace it with the dictionary that I compressed. (Be sure to unzip it, before you use it. I only used the zip-file to give it a comprehensive name.)

Point 7 and 8 from the linked howto:

7. Now use zip compression to zip all your gzipped html files, and the words file, into a single file called dicthtml.zip, e.g.,

7z a -tzip dicthtml.zip *.html words

On Unix this is:

zip dicthtml.zip *html words

This will create a dictionary file that will replace the English dictionary on the Kobo. If you want to replace a different language dictionary, then use the appropriate suffix, e.g., dicthtml-de.zip for German, dicthtml-nl.zip for Dutch.

8. Copy the new dictionary file to the Kobo, into the directory .kobo/dict (backup the existing dictionary first!). You may need to open a book and then flip between dictionaries in order to get your new dictionary's index to be loaded and recognised by the Kobo (you only need to do this once after replacing the dictionary).

[gosertje]

It works, thanks a lot
Now I have a decent dictionary

I enclose the other ones I have.

btw: just cosmetics, is it possible to make the font size smaller?
Gives you more text in the dictionary pop-up (less scrolling)

[Markismus]

@Gosertje Hardstikke mooi!

The Pocketbook converter strips most tagging. Only bold and italics remain, IIRC. If the font size can be reduced, it will have to be done within the Pocketbook environment.

Of course, you could install Koreader....It allows you to change font size, font face and you can even add your custom styling, it you want. The Pocketbook dictionary format is a very basic format. If you want more negative adjectives, you'll only have to search this forum.

[gosertje]

@MarkisMus bedankt.

So you are not in favour of Pocketbook.
I must say I am quite happy with my InkPad 3, especially I like the format.
But of course I am not a real expert.
I used to have a Sony PRS-T2 which was also OK, but smaller and no backlight.

Is it much effort to convert the other dictionaries also, in case I read some French or German, would be appreciated.