Possible scripting engine for Sigil

[Valloric]

[Discussion moved into it's own thread from here]

I've been thinking about exposing the internals of Sigil to an embedded version of Python (or Lua, or C# through embedded Mono).

What do others think of this?

[crutledge]

I've been thinking about exposing the internals of Sigil to an embedded version of Python (or Lua, or C# through embedded Mono). This should let you do what your asking, and much more.

What do others think of this?

JGSoft makes an excellent text clip manager and editor.
http://www.acetext.com/

The operation is straight foward and simple and requires no knowledge of any programming language. The price is reasonable and they provide full interface data. A fullup trial version is availabe with full documentation.

By following instructions any text string can be inserted at the caret or before/after any highlighted text. AceText works with SIGIL as it stands (by naming SIGIL in the Peferences) and changing the HTML becomes simplicity itself. All text clips can be saved in named Collections to be used any time. A fullup CSS can be saved and inserted into the HTML with a simple placement of the caret in the editor and a double-click on the named Clip. Preformatted tags can be saved and inserted as desired. Repetitive programming becomes a simple PLACE CARET - DOUBLE CLICK. Sharing of Clip Collections is simple.

It would be nice if the interface were explicit to preclude any problems with future changes to SIGIL.

DISCLAIMER
I am in no way connected to JGSoft other than as a very satisfied user of their tools for a very long time. Being sometimes very slow, I especially appreciate their excellent documentation.

[Valloric]

Originally Posted by crutledge

JGSoft makes an excellent text clip manager and editor.
http://www.acetext.com/

Um... thanks for pointing out this application crutledge, but what does this have to do with Sigil one day embedding a scripting language?

That application can be used to insert snippets, but an embedded scripting language could offer much more. I'm asking how do people feel about that feature, and what language would they like to see embedded.

[crutledge]

Originally Posted by Valloric

Um... thanks for pointing out this application crutledge, but what does this have to do with Sigil one day embedding a scripting language?

That application can be used to insert snippets, but an embedded scripting language could offer much more. I'm asking how do people feel about that feature, and what language would they like to see embedded.

It means nothing in the above context. A scripting language for those familiar with Python, etc., will allow the application of strange and wonderous things. But for those of us who don't care to learn another language (maybe only me), this "snippit inserter" is just what we need and would like to see it, or something very like it, solidly interfaced to SIGIL.

Each book is an individual work and must be approached as such. Scripting is for handling multiple books in exactly the same manner. It seems to me that this is the provence of SIGIL itself. A scripting language seems to be for making changes to the SIGIL output. Which is fine. However, if SIGIL produces a clean solid HTML file, with the ability to insert a TOC and other desired links, e.g. a linked LIST of ILLUSTRATIONS and will let me go in and format the book as I see fit, I will be as happy as a Blujay in a corn patch. I will then bless thee and thine to the seventh generation and shout thy praise from the roof tops.

Thanks for all your work. You are totally dedicated.

Charlie

[Valloric]

Originally Posted by crutledge

Scripting is for handling multiple books in exactly the same manner. It seems to me that this is the provence of SIGIL itself. A scripting language seems to be for making changes to the SIGIL output.

Well I had something different in mind... the embedded scripting language could interface to some kind of Sigil API that would provide access to all the metadata, XHTML files, images files etc... so one could write scripts to etc. remove page numbers from scanned books, or do other sorts of boring and easily automated tasks. Honestly you could do a LOT with that. I'm sure people would find more uses for it than I could imagine.

It's not something that would be implemented any time soon, but it's interesting. Of course, if the community expresses a deep desire for this kind of functionality, that would give it more priority.

[kovidgoyal]

Shouldn't you more or less get that for free with QtScript?

[rogue_ronin]

I'd like to see you do it NoteTab style. Check out the Clip language -- and the embedded clip window that lets you write scripted macros.

I don't expect that you'll recreate that language, just look at how it works -- it's really functional, with libraries of macros displayed. It can pretty much do anything at all. I wrote a full ebook editing and file-management suite in it.

I'd personally prefer to avoid a scripting language that looks like C code. It should look like BASIC actually. Something slightly more abstracted means that even doofii like me can code complex macros.

m a r

[Valloric]

Originally Posted by kovidgoyal

Shouldn't you more or less get that for free with QtScript?

QtScript is based on JavaScript. I hate JavaScript.

Originally Posted by rogue_ronin

I'd like to see you do it NoteTab style. Check out the Clip language -- and the embedded clip window that lets you write scripted macros.

I'll take a look at it.

[JohnnyD]

Originally Posted by Valloric

Well I had something different in mind... the embedded scripting language could interface to some kind of Sigil API that would provide access to all the metadata, XHTML files, images files etc... so one could write scripts to etc. remove page numbers from scanned books, or do other sorts of boring and easily automated tasks. Honestly you could do a LOT with that. I'm sure people would find more uses for it than I could imagine.

How about something with special keywords, some for document flow events (like "start chapter" or "start paragraph" or "next word" or even "next character") and some string handling functions (like "first character" and "substring" etc.)

It has been done a zillion times before, but it is the first method which springs to mind when I think about how to automate the proces of inserting "drop capitals"

[dmikov]

Originally Posted by Valloric

Um... thanks for pointing out this application crutledge, but what does this have to do with Sigil one day embedding a scripting language?

That application can be used to insert snippets, but an embedded scripting language could offer much more. I'm asking how do people feel about that feature, and what language would they like to see embedded.

I would vote for JavaScript, not that I like it, but working with c# all day long wont' have problem switching, so I would imagine Java or c++ programmers.
Basic sounds good, but it's just more typing and non case sensitive - dubious advantage.
If I had to learn new scripting or scripting like language I would vote for ruby. Had a very tough time learning lua.

[rogue_ronin]

Originally Posted by dmikov

I would vote for JavaScript, not that I like it, but working with c# all day long wont' have problem switching, so I would imagine Java or c++ programmers.
Basic sounds good, but it's just more typing and non case sensitive - dubious advantage.
If I had to learn new scripting or scripting like language I would vote for ruby. Had a very tough time learning lua.

It is "more typing" that makes it easier. I can remember what GetDocumentName means (NoteTab example.) When I drop to, say, bash, I have to use Google whenever I want to script something. What the hell is a '[', and where can I find documentation? It isn't an enclosure, that's for sure. It's actually a word of some sort, related to 'if'.

When I said BASIC, I basically (hah!) meant that it should be relatively easy to learn and simple in syntax. The example I gave with NoteTab falls into that paradigm. It can do amazing stuff, but it is really easy to learn. Over time, it added more than just text manipulation -- it moved into regex, file management, disk-access, web-capability, and more.

The point is to make it accessible to non-programmers. Folks editing and formatting their favorite books are not likely to be conversant with the ins and outs of proper programming. But if you can come at it from a simpler format, you'll get a lot more people contributing useful macros. And you can always add complexity.

I'm not a complete idiot, and I've programmed macros for years -- but when I start seeing massively indented bracket outlines, I just turn right off, lose enthusiasm. When I switched to Linux, I looked for a text editor that was programmable, comparable to NoteTab. I found Nedit, and a few others -- but they all required knowledge of C, or C structuring, familiarity with low-level use of environment variables, sed, etc. They were programmer's editors (which is natural, considering the nature of Linux.) I still keep a Virtual Machine of Win2K in order to run the latest NoteTab because I can't find a replacement (what, 8 or 9 years later...)

A macro in a text (or HTML, or writer's) editor is a list of instructions, manipulating (in this case) text and variables -- do this, then this, add it to the other thing, and put it over here. I feel that it should be as straightforward and simple as possible, while still retaining capacity to do anything at all that the host editor can do. Add hooks to other scripting languages if you want, but keep the native language simple.

BASIC is a good example of this sort of thinking (and Clip, from NoteTab) -- javascript is not, nor C*. Perhaps AppleScript, too, is a good example. What I remember of it was some sort of 'natural' language syntax. I thought it was a little loose, but better too loose than too strict.

Book editing is much closer to artistry (writing) than programming (as it is usually performed.) Make tools that enable the likely sort of people that will be using the editor.

$.02

m a r

[JohnnyD]

Originally Posted by rogue_ronin

The point is to make it accessible to non-programmers. Folks editing and formatting their favorite books are not likely to be conversant with the ins and outs of proper programming. But if you can come at it from a simpler format, you'll get a lot more people contributing useful macros. And you can always add complexity.

...

Originally Posted by rogue_ronin

A macro in a text (or HTML, or writer's) editor is a list of instructions, manipulating (in this case) text and variables -- do this, then this, add it to the other thing, and put it over here. I feel that it should be as straightforward and simple as possible, while still retaining capacity to do anything at all that the host editor can do. Add hooks to other scripting languages if you want, but keep the native language simple.

What kind of scripting engine do we need? A general purpose one? Or something dedicated? A general purpose one is relatively easy to add to Sigil, but has a steep learning curve. A dedicated one, directed at epub-writers (or "composers") should be easy to use, but takes a lot of effort to build into Sigil.

Allthough... I guess that most actions you want to automate have something to do with search&replace. Next to that, you would like some knowledge of where you are in a document. The typical topography of a book-like document is something like "book - chapter - paragraph - sentence - word - character".

Being able to search&replace text on a specific character or specific word in a specific sentence from a specific paragraph in a specific chapter could be really awesome. And relatively easy to build into Sigil... What's more, it is something a book writer/maker would know how to use...

[Dave Berk]

I'd vote for python, since it appear that we have many python expert around here (Kovid, Igor, user_none, etc). Also it would facilitate reusing code from Calibre in Sigil (I think).

[dmikov]

Originally Posted by rogue_ronin

It is "more typing" that makes it easier. I can remember what GetDocumentName means (NoteTab example.) When I drop to, say, bash, I have to use Google whenever I want to script something. What the hell is a '[', and where can I find documentation? It isn't an enclosure, that's for sure. It's actually a word of some sort, related to 'if'.

When I said BASIC, I basically (hah!) meant that it should be relatively easy to learn and simple in syntax. The example I gave with NoteTab falls into that paradigm. It can do amazing stuff, but it is really easy to learn. Over time, it added more than just text manipulation -- it moved into regex, file management, disk-access, web-capability, and more.

The point is to make it accessible to non-programmers. Folks editing and formatting their favorite books are not likely to be conversant with the ins and outs of proper programming. But if you can come at it from a simpler format, you'll get a lot more people contributing useful macros. And you can always add complexity.

I'm not a complete idiot, and I've programmed macros for years -- but when I start seeing massively indented bracket outlines, I just turn right off, lose enthusiasm. When I switched to Linux, I looked for a text editor that was programmable, comparable to NoteTab. I found Nedit, and a few others -- but they all required knowledge of C, or C structuring, familiarity with low-level use of environment variables, sed, etc. They were programmer's editors (which is natural, considering the nature of Linux.) I still keep a Virtual Machine of Win2K in order to run the latest NoteTab because I can't find a replacement (what, 8 or 9 years later...)

A macro in a text (or HTML, or writer's) editor is a list of instructions, manipulating (in this case) text and variables -- do this, then this, add it to the other thing, and put it over here. I feel that it should be as straightforward and simple as possible, while still retaining capacity to do anything at all that the host editor can do. Add hooks to other scripting languages if you want, but keep the native language simple.

BASIC is a good example of this sort of thinking (and Clip, from NoteTab) -- javascript is not, nor C*. Perhaps AppleScript, too, is a good example. What I remember of it was some sort of 'natural' language syntax. I thought it was a little loose, but better too loose than too strict.

Book editing is much closer to artistry (writing) than programming (as it is usually performed.) Make tools that enable the likely sort of people that will be using the editor.

$.02

m a r

I have to disagree with you on this one. Easy macro languages are hardest to learn, they are usually proprietary to one piece of software and there fore hard rationalize investing too much time into. Add to it, that vbscript or jscript or ruby or etc has oh around thousands books to learn and NoteTab or whatever < 1, situation becomes even more hard on developer to support it.

I like notepad++ and ultraedit. I would use notepad++ since it's free and has good plugins, but I am using ultraedit since on top of macro language (which both have) it has scripting with a normal language.
I think Microsoft Office can be commanded for at least this accomplishment of including nice scripting capability. Hate their code storage paradigm though.

[Valloric]

Originally Posted by dmikov

I have to disagree with you on this one. Easy macro languages are hardest to learn, they are usually proprietary to one piece of software and there fore hard rationalize investing too much time into. Add to it, that vbscript or jscript or ruby or etc has oh around thousands books to learn and NoteTab or whatever < 1, situation becomes even more hard on developer to support it.

I have to say I completely agree with this. I really don't feel like creating a proprietary scripting language for Sigil and I think it would be best sticking to something tried-and-true. The industry seems to be shifting away from such domain-specific languages and heading towards some common ground. Lua is becoming the de facto standard for scripting in game development, and numerous (scriptable) desktop applications are starting to integrate Python.

So Python would currently be the front-runner. There are countless books on it and everyone and their grandmother seems to know it (or are learning it).