[Kindle Touch] Localization

[JustAMan]

Originally Posted by eureka

Fork repo on Bitbucket, add your tool and make a pull request. I'll then give you commit rights at the original repo.

OK, I'll try to figure it all out.

BTW, just finished generating reports on current transifex status... Much more comforting in terms of producing the same output as my previous tool report.zip

UPD. Tool committed to my own fork and sent via pull request to yours repo.

[ixtab]

@JustAMan: thanks for the update. The newest version works for me, but it indeed seems to miss almost all of the properties that cause problems. There has been no relevant change in transifex, so it must be some problem with your tool (?).

@all: the Java tool (and therefore, binary package creation) has been updated slightly. Most importantly, system locales are not written to /usr/share/locale/<LOC_BASE> anymore, but to /usr/share/locale/<LOC_FULL> (e.g. "de" -> "de_DE"). As far as I have tested, this works with no problems, but for some of the previous packages, it requires a reinstallation (=uninstall package using old uninstaller, reinstall using new installer). The First post of this topic has been changed accordingly to make this clear.

[JustAMan]

Originally Posted by ixtab

@JustAMan: thanks for the update. The newest version works for me, but it indeed seems to miss almost all of the properties that cause problems. There has been no relevant change in transifex, so it must be some problem with your tool (?).

Did you do "tx pull -a" after you did "git pull/fetch/whatever"?
It solved my doubts as it pulled a lot of fresh stuff...

BTW, it looks like I was able to fix russian localization, at least I got home screen working, .txt/.prc reader working, .pdf reader working, settings working. Diff for latest translation pulled from transifex: ru-fix.zip
What should I do with that?

What isn't there is I'm not able to select default dictionary for Russian the way I'm able to select one for English... Might not be the issue with localization itself, dunno.

I cannot test other stuff I suppose is over-translated: that is periodical things, timezone selection and social stuff. Didn't check Amazon Store, too.
Any ideas how to test those?

Some stuff is translated wrong (probably people that did that translation didn't understand what they were translating or were using Google Translate ), some stuff isn't translated at all (e.g. dialog "turn on WiFi?" is in English, but it might be JS-generated, dunno again).

[JustAMan]

Hey, just read first post again... You claim that update-base is necessary for the stuff to work, but I don't recall installing it... And my device is localized. Huh?

UPD. Made sure I didn't have that base package installed...

[root@kindle root]# ls /usr/lib/locale/ -la
drwxrwxr-x    2 root     root          1024 Nov  3 22:58 .
drwxr-xr-x   27 root     root         16384 Nov  4 00:05 ..
-rw-r--r--    1 root     root       8318256 Nov  3 22:58 locale-archive
[root@kindle root]#

Something really weird is going on...

[ixtab]

@JustAMan:
your /usr/lib/locale/ layout is indeed expected, as the Kindle *itself* already contains the ru_RU files.

There is nothing weird at this point. I'm just trying to make sure that people who installed a previous version of their locale are actually uninstalling before installing it again...

to summarize: the (system) ru_RU locale is *already contained* on the Kindle. Localization packages will not touch it currently. See "localedef --list-archive".

[JustAMan]

Okay, if you say everything is fine than everything is fine I'm complete noob in Linux locale stuff.

[eureka]

Originally Posted by JustAMan

BTW, it looks like I was able to fix russian localization, at least I got home screen working, .txt/.prc reader working, .pdf reader working, settings working. Diff for latest translation pulled from transifex: Attachment 81801
What should I do with that?

You should push changed resources to Transifex:

• register on transifex.net
• apply for membership in Russian translation team of KT Framework resources
• after approving, pull resources from Transifex (just in case) in your clone of Git repo, apply your diff (if it's not applied yet), then push changed ru_RU resources back to Transifex (like tx push --resource=kindle-touch-framework.\* --language=ru_RU)
• commit your changes and push them to to Bitbucket

Originally Posted by JustAMan

some stuff isn't translated at all (e.g. dialog "turn on WiFi?" is in English, but it might be JS-generated, dunno again).

WiFi dialog is Pillow-driven, so yes, it's JS part.

[ixtab]

@JustAMan: I just made sure to have the most recent translations (in fact, I removed the entire directory and did a "tx pull -a -s"), then ran your report tool again. From what I can tell, the output of the tool is now *extremely* helpful. I'm pretty sure that if all of the reported messages weren't translated, the russian locale as a whole should work).

I think that before we go anywhere else, we (=you, or other ru_RU translators) should update the russian translation to reflect this newly gained knowledge, and test whether the updated translations work.

Your tool will still be very useful once ru_RU is at a working 100%. Actually, it'll probably even be more useful than comparisons between "en" and "de": a few translated resources also happen to be the same between "en" and "de", even if that doesn't imply a "do-not-translate" situation, like "OK" or "International". Russian may serve as a better discriminator in these cases, because there is a lower chance of translated strings equalling the original.

[JustAMan]

ixtab,
I've pushed the fixes for ru_RU locale to Transifex & git repo.
Might be worth publishing at the first page...

And I thought about having a list of "do-not-take-from-translation" properties built in your locale*.jar building stuff. Is it possible? I think it might be easy enough to get from my checking tool, I mean, to make it produce such a list

BTW, anyone has newspapers/blogs on their KTouch? I don't and I don't have ability to get some as I do not own credit card suitable for Amazon, and they won't let me subscribe even for free trial without that card... So I cannot test the locale for newspapers and blogs. I was able to test it for magazines, though.

[ixtab]

@JustAMan: cool

I'm not modifying the front page yet, for various reasons: first, we need to make sure that it is indeed working, so give yourself and other ru_RU users another day or two to test it. Second, I also didn't make an announcement that de is "complete" there, for the simple reason that: third, none of our translations can be considered 100% until we have the JS part in there as well. I see that eureka is making progress, but I'll wait until he says the tool is stable...

That said, I have updated the Java tool to include all the logic for handling blacklisted entries. See the big green chunks on https://bitbucket.org/katey_hack/kin...t/7061dd1697f1 . It's working, now all we need to have is the data. Could you implement in your tool a "negated" mode, where it outputs (almost) the exact opposite of what it's doing now? I.e., it takes a reference translation, and then outputs all Strings which are identical between en_US and the translation.

Ideally, the output would look something like this:

// com.amazon.ebook.booklet.periodicals.blogs.ui.views.home.resources.BlogHomeWidgetResources#event.gesture.previous=east
set.add(new ResourceEntry("com.amazon.ebook.booklet.periodicals.blogs.ui.views.home.resources", "BlogHomeWidgetResources","event.gesture.previous"));

The first line is only for human verification of the value, the second line could be directly included in the code.

Any other format that can easily be transformed into this via some regex magic will do as well.

Спасибо!

[eureka]

Originally Posted by ixtab

... none of our translations can be considered 100% until we have the JS part in there as well. I see that eureka is making progress, but I'll wait until he says the tool is stable...

I'm glad to hear that I still didn't exhausted your patience Tool is already made up to producing [incomplete] compile output. It outputs locales directory structure and JS files with JS object inside, but JS file lacks top-level variable assignment and wrapping of string into constructor call (where it's needed) as it's a metadata and metadata handling isn't implemented yet. (So, tool isn't ready yet.)

Note: tool takes two .properties to produce output JS file (for the case when some strings in localized .properties are empty or missing and they should be taken from original .properties).

BTW, I've seen your addition at KT hacking wiki page about a displayed clock's 24-hour format and it's relation to current locale (i.e. to locale definition). Are you still not convinced to provide real locale definitions?

[ixtab]

Originally Posted by eureka

BTW, I've seen your addition at KT hacking wiki page about a displayed clock's 24-hour format and it's relation to current locale (i.e. to locale definition). Are you still not convinced to provide real locale definitions?

Haha!

That entry wasn't added by me, but I also noticed it, and it directly scared the shit out of me. However, I have immediately tested it with the polish locale, and even though the system locale is symlinked to en_US, the 24-hour clock is displayed

So for the time being, I still don't see a need for it. If someday we realize that there is indeed a need to go for them, we can.

[eureka]

Originally Posted by ixtab

Ideally, the output would look something like this:

// com.amazon.ebook.booklet.periodicals.blogs.ui.views.home.resources.BlogHomeWidgetResources#event.gesture.previous=east
set.add(new ResourceEntry("com.amazon.ebook.booklet.periodicals.blogs.ui.views.home.resources", "BlogHomeWidgetResources","event.gesture.previous"));

The first line is only for human verification of the value, the second line could be directly included in the code.

Any other format that can easily be transformed into this via some regex magic will do as well.

Idea: with resembling information one could make UserJS for hiding blacklisted strings from Transifex translation's web-editor. So, JSON output would be good (but I'm not insisting, just suggesting). And firmware version should be included also.

Originally Posted by ixtab

Haha!

That entry wasn't added by me, but I also noticed it, and it directly scared the shit out of me. However, I have immediately tested it with the polish locale, and even though the system locale is symlinked to en_US, the 24-hour clock is displayed

Then it was a bit rude from my side. I've confused you with some IP address and was too lazy to check the info. Sorry for that

[JustAMan]

@ixtab:
Done. That was easy This information was already gathered, so it just needed representation coded.

I've pushed my changes to bitbucket. Give a try via "python checkit.py -a blacklist", however I recommend setting "ru_RU" as "model language" as it produces less blacklist entries than "de" (that is, add "-m ru_RU" to command line).

It saves the blacklist report by default to "report/5.0.0/blacklist-model_language.txt".

BTW you guys posted quite a few messages while I tried to report my progress

Originally Posted by eureka

Idea: with resembling information one could make UserJS for hiding blacklisted strings from Transifex translation's web-editor. So, JSON output would be good (but I'm not insisting, just suggesting).

No problem, just provide some sample output like ixtab has done. I'm not that familiar with UserJS stuff (btw it implies that translator uses Opera, right? hehe, that's good I'm Opera fan)
JSON is easy in Python, you know ("import json" and you're almost there )

Originally Posted by eureka

And firmware version should be included also.

Hmm... That's goes to ixtab I suppose My tool produces reports and stores them under specified version folder (only 5.0.0 currently - no other versions in src/ are present now).

[ixtab]

Once the blacklist is complete, I suggest to re-create all source files using the appropriate filter. This would allow for two things at the same time:

• if done from 5.0.0 jars and from 5.0.3 jars, allow us to check whether anything has changed, and if so, to decide on how we should handle that.
• entirely remove blacklisted entries from Transifex.

As to the system locale issue, I'm currently installing a VM with all existing utf8 locales. It's still installing, but I'll post here once I figured out the implications of trying to add the full system locale to each package. Two things are already sure for now:

• It will bloat each installer package by (roughly) a factor of 10.
• It seems like there isn't a utf8 system definition for every single territory out there. From memory, all current translations should have an existing corresponding UTF8 locale, but this cannot be guaranteed for all locales.

So, any thoughts or suggestions?