|
![]() |
|
Thread Tools | Search this Thread |
![]() |
#31 |
Sigil Developer
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 7,803
Karma: 5446592
Join Date: Nov 2009
Device: many
|
The unicode standard is not carved in stone and is in its 15th revision. Would you believe that new emojiis are one of its driving issues.
And AI and text analytics are some of the areas pushing most strongly for search ability and a using precomposed forms. Last edited by KevinH; Yesterday at 05:09 PM. |
![]() |
![]() |
![]() |
#32 |
Evangelist
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 436
Karma: 77256
Join Date: Sep 2011
Device: none
|
I can understand the usefulness for most users. It just seems there are enough potential issues with some languages, perhaps potentially more for non-Western, partially given the opinion of some it seems of Unicode in general, that I strongly feel it should be a setting and not something strictly enforced.
|
![]() |
![]() |
![]() |
#33 |
Sigil Developer
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 7,803
Karma: 5446592
Join Date: Nov 2009
Device: many
|
Understood. As I said, given demand, I will create an environment variable to disable it once we get things working completely.
|
![]() |
![]() |
![]() |
#34 |
Evangelist
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 436
Karma: 77256
Join Date: Sep 2011
Device: none
|
ALERT: Potential Issues with Sigil 2.2.X and rtl languages and Normalization Forms
Great. I wasn't sure if the variable was either possibly planned or certain. Thank you very much.
|
![]() |
![]() |
![]() |
#35 |
Evangelist
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 436
Karma: 77256
Join Date: Sep 2011
Device: none
|
I’m not that familiar but I mention such since it seems to be relevant here and it seems that Sigil and some eReaders may not be supporting such.
Is canonical Unicode equivalence relevant here? Not necessarily storing the text in some normalized form, but it seems maybe the spec requires that apps normalize text just for the purpose of processing? So it seems maybe search should function the same way in apps as required by such? |
![]() |
![]() |
![]() |
#36 |
Sigil Developer
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 7,803
Karma: 5446592
Join Date: Nov 2009
Device: many
|
Easy to do when processing an external file but any position information of a match location may then not match the original file because of combined characters shrinking length of its string representation. This makes replacement a big issue.
This is even more of a problem when done inside an editor that does not force normalize the strings in advance. Any match information (position and if match or not) will be incorrect unless underlying file is properly normalized the same way as the search string. This makes search and replacement impossible. |
![]() |
![]() |
![]() |
#37 |
Evangelist
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 436
Karma: 77256
Join Date: Sep 2011
Device: none
|
It indeed seems challenging.
It seems that reading systems normalize text for the purpose of search sort or other and platforms seem to have APIs for handing such. I haven't used any readers other than Apple Books in a while. Yet it seems to do ok with canonical equivalence and diacritic insensitive search. If an editor should behave the same way I'm not sure but as an option maybe it'd be useful? Not really familar but does it seem this might help? https://doc.qt.io/qtforpython-6.6/Py...aryFinder.html |
![]() |
![]() |
![]() |
|
![]() |
||||
Thread | Thread Starter | Forum | Replies | Last Post |
Potential Issues Upgrading? from 3.21 :O | rebeltaz | Calibre | 22 | 03-20-2022 11:19 PM |
New to Using Sigil and Having Issues | jester1972 | Sigil | 20 | 04-30-2017 09:24 AM |
New Issues in Sigil 0.9.3 | jafprrr | Sigil | 11 | 03-10-2016 11:59 AM |
issues with sigil 0.8.4 | eregs | Sigil | 2 | 02-27-2015 08:01 AM |
Support for RTL Languages | Gonidae | Calibre | 1 | 10-05-2012 05:13 AM |