[Plugin] ODTImport

[Doitsu]

ODTImport: Import ODT documents into Sigil as epubs.
(based on Writer2LaTeX)

Current Version: "0.4.0"

This plugin is a very simple Writer2LaTeX 1.4 wrapper, which allows you to import OpenOffice ODT files.

Credits: Since I'm not that familiar with the ODT file format and Writer2LaTeX, I asked st_albert for help with the configuration and stylesheet files and he kindly provided sample configuration/stylesheet files (config.xml and epub.css) as well as an ODF text document template file (custom-styles.ott).
If you want to test the template file, unzip Custom-Stylesheet.zip and add the custom styles in LibreOffice/OpenOffice via Styles > Load Styles > From file > ODF Text Document Template > custom-styles.ott. This will add a number of custom styles that all start with custom, e.g. custom-body-text.
Note that these styles are not intended as all purpose styles, they're merely provided to demonstrate how to map custom styles to stylesheet classes.
For example, the following line in config.xml maps the LibreOffice/OpenOffice custom-body-text style to the p.custom-bodytext class in epub.css.

<xhtml-style-map name="custom-body-text" family="paragraph" element="p" css="custom-bodytext" />

If you don't want to use the default config/css files, you can simply delete them from the plugin folder to force Writer2LaTeX to use the default settings. (To display the ODTImport plugin folder select Edit > Preferences > Open Preferences Location.)
In this case you might find the RemoveInLineStyles plugin helpful, which will allow you to convert all inline styles to classes.

Warning: Like all other input plugins this plugin will destroy the contents of the currently loaded ePub; make sure to only run it when no epub is loaded.

System requirements:

If you're using an older Sigil version, you'll need to install a Python interpreter and select its path in the Manage Plugins dialog box. Since Writer2LaTeX is a Java app, you'll also need to install Java, if you haven't already done so and the Java executable path must be added to the system path. I.e., if you enter java -version in a DOS/terminal window you should get a version number. Otherwise the plugin won't work.
If you don't get a version number, please read the troubleshooting section of the epubcheck (Java) plugin.

Installation

1. Select Manage Plugins from the Plugins menu. In the Manage Plugins dialog box, select Use Bundled Python, if it isn't already selected. (If your Sigil version doesn't have a Use Bundled Python option, click one of the Auto buttons to detect the path or Set to manually select the Python interpreter path. )
2. Click Add Plugin and select ODTImport_v0.1.zip. This will install the plugin, which you can select via Plugins > input > ODTImport.

(ODTImport_v0.4.0.zip comes with Writer2LaTeX 1.7.)

Troubleshooting: If you get a WindowsError: [Error 2] error message, the Java binary couldn't be found. Installing/updating Java and re-booting your machine should take care of this problem.

License: GNU General Public License v3 (GPL-3)

[st_albert]

I just gave it a pretty good workout in 0.9.5 on Kubuntu 16.10 and it works great. I was even able to replace the config.xml and epub.css with my own production versions with no trouble.

This makes it essentially as easy for me to go from LibreOffice to Sigil as it would be to use the LO writer2xhtml extension, but with greater flexibility, and no worries if the LO version gets ahead of the writer2xhtml extension version, and no longer functions. (As has happened from time to time in the past).

For that matter, I suppose the import plugin should work with .odt files from other sources than LibreOffice or OpenOffice, perhaps with some tweaks of the config.xml file.

Great work!

Albert

[roger64]

Hi

Congratulations!

As a first try, both of you pretty welll succeeded. I imported quite a complex odt file with some titles, lots of footnotes... It was converted straight out the box using the provided "custom" style-sheet.

Now I will make further tries to tweak it to my own style-sheet and style-mappings. As we do not have the initial front panel writer2xhtml, provides, I think it will probably be necessary to use the metadata editor of Sigil to complete the missing fields we can't fill in LibreOffice.

[roger64]

I did a second test with the same -complex- odt file.

This time I changed three elements
- writer2latex.jar for version 1.5.2.alpha which is quite stable BTW
- config.xml from my -renamed- writer2xhtml.xml file
- epub.css from my -renamed- writer2xhtml-styles.css

The result was even better than the previous one because I could use my custom settings.

To get 100% of what I have with writer2xhtml, I should find a way to select some missing options. With writer2xhtml, these are usually selected on the panel (see screenshot). Using writer2latex there is probably a different way to select or express our choice before processing.

It mainly concerns:
- EPUB2 or EPUB3?
- values in px or cm
- one font-family for all the styles (and not for each style)
- Document division (splitting the html file along h1 or h2 headings, etc.)
- producing a toc.ncx

Have you an idea how to do it?

[Doitsu]

Originally Posted by roger64

It mainly concerns:
- EPUB2 or EPUB3?
- values in px or cm
- one font-family for all the styles (and not for each style)
- Document division (splitting the html file along h1 or h2 headings, etc.)
- producing a toc.ncx

Have you an idea how to do it?

As I've mentioned in the plugin description, I'm only familiar with the basic feature of Writer2Latex and using an alpha version in a production environment is generally not advisable.

Hopefully, st_albert will chime in with some helpful feedback.

As for your points:

- EPUB2 or EPUB3?

For ePub3 output you'll need to change -epub command line parameter to epub3.

Locate the following block in plugin.py and replace it with the following version:

Spoiler:

        # define command line arguments
        if os.path.isfile(w2l_config_path) and os.path.isfile(w2l_css_path):
            args = ['java', '-jar', w2l_path, '-epub3', '-config=' + w2l_config_path, '-stylesheet=' + w2l_css_path, odt_file] 
        elif os.path.isfile(w2l_config_path) and not os.path.isfile(w2l_css_path):
            args = ['java', '-jar', w2l_path, '-epub3', '-config=' + w2l_config_path, odt_file]
        elif not os.path.isfile(w2l_config_path) and os.path.isfile(w2l_css_path):
            args = ['java', '-jar', w2l_path, '-epub3', '-stylesheet=' + w2l_css_path, odt_file] 
        else:
            args = ['java', '-jar', w2l_path, '-epub3', odt_file]

- values in px or cm

Should be enabled with the the following line in config.xml:

<option name="convert_to_px" value="true" />

(It was set to false in the sample config file.)

- one font-family for all the styles (and not for each style)

No idea. Why don't you define a document font for the body element?

- Document division (splitting the html file along h1 or h2 headings, etc.)

AFAIK, this is defined via the following config.xml parameter:

<option name="split_level" value="1" />

1 = h1; change this to the desired heading level number.

- producing a toc.ncx

Should be enabled by the following option:

<option name="include_ncx" value="true" />

Note that I haven't tested these settings with the alpha version.

[roger64]

@Doitsu
Thanks very much for your detailed answer. I will try all this tomorrow.

I already select a font-family in the body of epub.css but that's not enough to get rid of all the unwanted font-family which are set in the styles. It's also like that for writer2xhtml and there is a need to order it specifically somewhere.

As for this alpha version, I have been using it without any trouble for nearly a year... as a hobbyist.

[st_albert]

I think Doitsu covered most everything I know about configurations. As for controlling splits in the epub, I will add that there is another config option which more-or-less determines how to handle explicit page breaks in the ODF document. It is

  <option name="page_break_split" value="explicit" />

I say "more or less" because in my experience it seems to interact with the "split level" option, and whether or not the heading level specified does or does not include a page-break option in the LO style definition. If you find you're getting blank pages in the epub, tweak these values. Likewise, if you are not getting splits where you think you should.

@roger64: By the way, what does ver 1.5.2-alpha bring to the table vs. 1.4?

I know both Doitsu and Roger64 are already familiar with this, but for those new to writer2latex, here is the user manual:

[st_albert]

Oh, and as for metadata, I see from the user manual that there is a way to add it to the .odt file via the writer2xhtml extension, but I don't see how to do so if using the stand-alone (and therefore, the Sigil plug-in).

This is an area where I'm blind. I never used it in the LO/OO extension, because I just edit the content.opf directly in Sigil. I gave my boss a "skeleton" file with blank metadata entries that we need to use. For a given book, she fills in what she wants, saves it as a .txt file, and I copy and paste it into content.opf as needed. There is a lot of metadata included. But, as JSWolf will no doubt point out, no one makes use of it at present. My boss wants it there, though, for the future.

My feeling is, if you will excuse a slightly off-color analogy, that it is like losing bladder control while wearing a dark suit. It gives you a nice warm feeling, but nobody notices.

That said, if anyone finds the way to include metadata via the plugin, I'd like to know how.

[roger64]

Up to now (till I find another odt file with more complex and untried features), everything seems to be working fine.

Two things are missing but can be easily dealt with :
1. - there seems to be no way to add automatically "resources" (fonts we wish to embed). We can however insert automatically any @font-face declaration.
2. - no metadata editor in the plugin but there is no real need for it(see further down)


The missing option I was looking for was:

 <option name="use_default_font" value="true" />

@st_albert
road map
Henrik Just published his road map from 1.4 to 1.6.

for writer2xhtml, the two planned items have been implemented with version 1.5.2.alpha
- Support for EPUB3 (including metadata)
- toolbar for launching writer2xhtml or writer2latex from LibreOffice

for writer2latex, I do not know since it's the first day I use it.

metadata editor
Of course since Sigil implemented a brand new metadata Editor, it seems spurious to try to double it in the odt plugin.

writer2xhtml has one which is useful because it's used within LibreOffice (on the lower part of the screenshot above, click on "Edit document properties").

usermanual
I was aware of the existence of this manual, but every time I tried to read it, I found it overwhelming.

[roger64]

Using this plugin, I had a closer look at my config.xml file (coming from writer2xhtml). It was a total mess.

So I reordered all its options following the order of the user manual. I added the names of each group, using also the names provided by the user manual.

Spoiler:

<?xml version="1.0" encoding="UTF-8" ?>
<config>
<!-- Style options -->
<option name="template_ids" value=",,," />
<option name="pretty_print" value="true" />
<option name="no_doctype" value="false" />
<option name="encoding" value="UTF-8" />
<option name="hexadecimal_entities" value="true" />
<option name="use_named_entities" value="false" />
<option name="add_bom" value="false" />
<option name="multilingual" value="false" />
<option name="separate_stylesheet" value="false" />
<option name="custom_stylesheet" value="" />
<!-- Control the conversion of formatting -->
<option name="formatting" value="ignore_hard" />
<option name="frame_formatting" value="ignore_hard" />
<option name="section_formatting" value="convert_all" />
<option name="table_formatting" value="convert_all" />
<option name="table_size" value="relative" />
<option name="list_formatting" value="css1" />
<option name="tabstop_style" value="2em" />
<option name="use_default_font" value="true" />
<option name="default_font_name" value="Linux Libertine O" />
<!-- Handling of dimensions -->
<option name="convert_to_px" value="false" />
<option name="scaling" value="100%" />
<option name="column_scaling" value="100%" />
<option name="image_size" value="relative" />
<option name="relative_font_size" value="true" />
<option name="font_scaling" value="100%" />
<!-- Options for special content -->
<option name="formulas" value="image+starmath" />
<option name="use_mathjax" value="false" />
<option name="embed_svg" value="true" />
<option name="embed_img" value="false" />
<option name="endnotes_heading" value="Notes" />
<option name="footnotes_heading" value="Notes" />
<option name="use_dublin_core" value="true" />
<option name="notes" value="true" />
<option name="display_hidden_text" value="false" />
<option name="include_toc" value="true" />
<option name="include_ncx" value="true" />
<option name="float_objects" value="true" />
<!-- AutoCorrect options -->
<option name="ignore_double_spaces" value="false" />
<option name="ignore_empty_paragraphs" value="false" />
<option name="ignore_hard_line_breaks" value="false" />
<!-- File options -->
<option name="external_toc_depth" value="3" />
<option name="split_level" value="2" />
<option name="repeat_levels" value="5" />
<option name="page_break_split" value="styles" />
<option name="split_after" value="0" />
<option name="image_split" value="none" />
<option name="cover_image" value="true" />
<option name="save_images_in_subdir" value="false" />
<option name="uplink" value="" />
<!-- Options specific for spreadsheet documents -->
<option name="calc_split" value="false" />
<option name="display_hidden_sheets" value="false" />
<option name="display_hidden_rows_cols" value="false" />
<option name="display_filtered_rows_cols" value="false" />
<option name="apply_print_ranges" value="false" />
<option name="use_title_as_heading" value="true" />
<option name="use_sheet_names_as_headings" value="true" />
<!-- Options for batch conversion -->
<option name="directory_icon" value="" />
<option name="document_icon" value="" />
<!-- Style maps -->
<xhtml-style-map after="" before="" block-css="(none)" block-element="dl" css="(none)" element="dt" family="paragraph" name="List Heading" />
<xhtml-style-map after="" before="" block-css="(none)" block-element="" css="(none)" element="address" family="paragraph" name="Sender" />
<xhtml-style-map after="" before="" block-css="(none)" block-element="dl" css="(none)" element="dd" family="paragraph" name="List Contents" />
<xhtml-style-map after="" before="" block-css="(none)" block-element="blockquote" css="(none)" element="p" family="paragraph" name="Quotations" />
<xhtml-style-map after="" before="" block-css="(none)" block-element="" css="(none)" element="hr" family="paragraph" name="Horizontal Line" />
<xhtml-style-map after="" before="" block-css="(none)" block-element="" css="(none)" element="pre" family="paragraph" name="Preformatted Text" />
<xhtml-style-map after="" before="" block-css="(none)" block-element="" css="(none)" element="p" family="paragraph" name="Text body" />
<xhtml-style-map after="" before="" css="(none)" element="em" family="text" name="Emphasis" />
<xhtml-style-map after="" before="" css="(none)" element="var" family="text" name="Variable" />
<xhtml-style-map after="" before="" css="(none)" element="code" family="text" name="Source Text" />
<xhtml-style-map after="" before="" css="(none)" element="strong" family="text" name="Strong Emphasis" />
<xhtml-style-map after="" before="" css="(none)" element="kbd" family="text" name="User entry" />
<xhtml-style-map after="" before="" css="(none)" element="samp" family="text" name="Example" />
<xhtml-style-map after="" before="" css="(none)" element="dfn" family="text" name="Definition" />
<xhtml-style-map after="" before="" css="(none)" element="cite" family="text" name="Citation" />
<xhtml-style-map after="" before="" css="(none)" element="tt" family="text" name="Teletype" />
<xhtml-style-map css="(none)" element="sup" family="attribute" name="superscript" />
<xhtml-style-map css="(none)" element="b" family="attribute" name="bold" />
<xhtml-style-map css="(none)" element="i" family="attribute" name="italics" />
</config>

Options values are mine and can of course be modified. Consult the user manual from page 25.

[Arios]

Thank you Doitsu, as well as st_albert

This plugin is a sagacious move that seems to work pretty well, even with this initial version.

Greetings!

[roger64]

I. - Front panel

DiapDealer introduced a brilliant front panel with the version 0.2 of his DocXimport plugin (see screenshot). Such a kind of front panel would also very well suit ODTimport.

1. - Choice between EPUB2 and EPUB3 is now difficult for the average user. It could be made much easier (provided we use the 1.5.2. version)

2. - There are two major configuration files interesting the user: config.css and epub.css. However we can imagine there could exist several files of this kind, varying according the intended output (with or without fonts, dropcaps, illustrated book, etc.). This way the user would be able to choice which config.css or epub.css to use just by selecting a new path. If nothing is selected, it would mean that he intends to use the two current files.

II. - Other goodies

Further, there are some very few "weaknesses" of writer2xhtml that I know how to address everytime I produce an EPUB. However, they may pose a problem for beginners.

Resources:
Now, when I wish to embed fonts, the plugin allows me only to insert some @font-face declarations in epub.css. It would be good to be able to also designate and import in the EPUB the content of a "Resource" folder with fonts and the com.apple.ibooks.display-options.xml file. We may need to tick a box and select a path for this.

cm to em:
writer2xhtml offers the choice of px or cm for style sheet dimension units. Neither is really satisfying (relative units are better). I use the DiapDealer Toolbag in the Calibre Editor to convert cm to em in all style sheets. The conversion is not mathematically rigorous but practically it is very useful. We can imagine it could be automatically performed once the conversion to EPUB has been done provided the user ticked previously such an option.

xhtml files
writer2xhtml 1.5.2. produces EPUB3 with html(5) files (but EPUB2 with xhtml files). Theses files respect also the xhtml format that Epubcheck now recommends to write in such a case. Sigil can rename all the text files extension from html to xhtml. If this was done automatically after EPUB conversion, this would be a plus.

[Doitsu]

Hi roger64,

Thanks for your feedback.

As the plugin didn't generate much interest and was primarily designed for experienced Writer2LaTeX users who usually have the technical skills to modify/replace the default config and css files and the very simple Python plugin wrapper code, I'm somewhat reluctant to add a proper GUI, but I might add plugin prefs support that'll allow the user to modify some of the default settings.

Originally Posted by roger64

1. - Choice between EPUB2 and EPUB3 is now difficult for the average user.

I might add this as plugin preference settings. However, as I've mentioned before, it's usually not a good idea to use an alpha version in a production setting.

Originally Posted by roger64

2. - There are two major configuration files interesting the user: config.css and epub.css. However we can imagine there could exist several files of this kind [...]

AFAIK, most Writer2LaTeX users use only one set of config/css files. IMHO, having a proper GUI for config/css file selection would be a nice-to-have feature, however, it's not essential.

Originally Posted by roger64

When I wish to incorporate fonts, up to now the plugin allows me to insert only some @font-face declarations in epub.css. It would be good to be able to also designate and import in the EPUB the content of a "Resource" folder with fonts and the com.apple.ibooks.display-options.xml file.

Both are features that aren't natively supported by Writer2LaTeX and adding the com.apple.ibooks.display-options.xml file is only needed for iBooks epub3 books, which aren't that well supported outside of iBooks.
Besides there's also dedicated AddiBooksXML plugin by DiapDealer.

I might add a user font path to the plugin prefs settings and automatically add all fonts in that folder to the epub.

Originally Posted by roger64

cm to em:

If you specify your own stylesheet, it shouldn't have any cm values in it and using Writer2LaTeX without a custom stylesheet isn't recommended anyway. I.e., don't expect support for this feature.

Originally Posted by roger64

xhtml files

AFAIK, the epub3 standard supports both xhtml and html files and only requires that each type have the proper extension and file type declaration. I.e. it does not require XHTML files nor XHTML file extensions for HTML(5) files.
Besides it's not possible to rename files with plugins without breaking links. (Only the GUI rename feature will properly update links.)

[roger64]

Thanks to you.

The ODTimport plugin is certainly good for writer2latex users but also for common writer2xhtml users - hey I am one and I guess Arios is one too - and the number of its users can be greatly expanded to the whole community of writer2xhtml users.

I have been using writer2xhtml for some years (4, 5?) and kept alive a MR thread about it (in French). By experience, I know that the main problem for explaining to someone how to use it is to make him understand the not so clear (say confusing) writer2xhtml GUI with different panels.

With ODTimport, this problem magically disappears: It's enough to provide two files (config and epub) and the result is predictible, guaranteed. This is great. Once a beginner manages to obtain a nice result easily, the more chances he gets used to this software.

That's why, I propose to offer an option to get rid of the very few remaining stumbling blocks. Paris has not been done in one day, and we will have time to speak again about it. Anyway, you know I am very keen to test it.

Thanks again for your very nice work.

[Doitsu]

I've slightly updated the plugin for the Writer2LaTeX 1.5.2 alpha version, which supports ePub3 output. You can also automatically add a com.apple.ibooks.display-options.xml file for iBooks if you enable the ibooks_xml value in the preferences json file.

The new version also comes with a general purpose stylesheet provided by roger64 who also streamlined the configuration file.

It also has Sigil plugin prefs support. You can set the following values:

{
  "output_format": "epub3", 
  "ibooks_xml": true,
  "initialdir": "C:/Users/User/Downloads"
}

"output_format" can be epub or epub3; "ibooks_xml" can be true or false and "initialdir" can be any valid odt document folder path.

Disclaimer: Since Writer2LaTeX 1.5.2 is still an alpha version, I don't recommend using it in a production environment. If you need to generate ePub3 books, I'd recommend using Writer2LaTeX 1.4 and the Sigil ePub3 output plugin.