FW 5.3.x - 5.7.x Fonts Hack

[NiLuJe]

Hi .

You'll find here the FW 5.x (KT/PW/PW2/KT2/KV/PW3/KOA/KT3/KOA2/PW4) port of the Fonts hack .
I say 'port', but some of it's *cough* 'inspired' *cough* from ixtab's excellent Kindle Touch Font Hack, some of it from the FontConfig KUAL extension, and most of it is based on the findings of multiple people on this forum, so, thanks everyone!

NOTE: You'll need to be running at least FW 5.3.x for this to work (to avoid issues, you won't be able to install it on unsupported FW versions anyway).

What Does It Do?:

This will allow you to customize the fonts used by your Kindle, be it in books, or the UI, and optionally grant you deeper control over the font rendering engine. On FW 5.3.x, it serves as a wrapper around the alt fonts trick with an added layer of safety, and on FW 5.4.x, it allows you to continue using custom fonts without embedding them in each book, despite the changes in the Aa menu's behavior.

Latest Updates (06/04/2019):

Fonts v0.14.N (FW 5.7.x support, updated binaries).

PSA: All downloads have been moved to the Snapshots thread!

IMPORTANT NOTE REGARDING UPDATES:

Spoiler:

Here are general update instructions for this hack:

• No need to run the Update_*_uninstall.bin updates, ever.
• You simply have to apply the latest Update_*_install.bin update.

INSTALL:

Fonts:

Spoiler:

First of all, make sure your device is JailBroken.
If you own a PW1, it might also be a very good idea to make sure you have the Rescue Pack installed,
because both the ALT_FONTS stuff and this can have a pretty intrusive impact on the framework & boot process, so better be safe than sorry .

Playing with fonts on a Kindle has always been a great way to get the framework in a mood, especially since the K3, so I'll say it right away: this deserves the title hack, so I'd strongly recommend being handy with the basics of how all of this works, how it might go wrong, and how to fix it, and at the very least always have an SSH shell ready & open when you're testing your setup. Most of the instructions will make that clear, and provide methods that will leave you time to recover from mistakes if you have a shell available.

First of all, an important notice for FW >= 5.4.x users: you will not see your custom fonts in the Aa menu on this device. Currently, the only way to actually use your custom fonts is thus to use the font overrides feature.

If you intend to use the Font Overrides KUAL menu:

You will need the Python package. Make sure it's installed *before* the Fonts hack.

As usual, you'll need MRPI for the install process.

Download the K5 Fonts Hack package from the Snapshots thread, and unpack it. In here, you'll find a few files.

First, upload the appropriate install .bin file for your device (Update_linkfonts_0.14.N_install_pw2_kt2_kv_pw3_koa _kt3_koa2_pw4_kt4.bin for a PW2 or a KT2 or a KV or a PW3 or a KOA or a KT3 or a KOA2 or a PW4 or a KT4, the other one for a Touch or a PW1) to the mrpackages directory of your Kindle.

Now, eject & unplug your Kindle, and run MRPI via KUAL: Helper -> Install MR Packages. It should take a couple dozen of seconds.
NOTE: Since I'm not sure if this hack will work on earlier firmwares than 5.3.1, the installer will refuse to launch if you're using an older firmware.

And that's it for the installation . Now for a quick look at what you can do with it… All of the settings/tweaks I'll talk about are found in the Fonts KUAL menu that the hack installs, so, if you're not already using KUAL, now would be a good time to correct that

The basics: adding new font choices to the Aa menu (with some caveats on FW >= 5.4, read everything carefully). It's as simple as dropping your font files (no need to rename the files) in the linkfonts/fonts folder at the root of your Kindle.
NOTE: If you've already played with the USE_ALT_FONTS tweak, you were using the fonts folder, not linkfonts/fonts. This works, too, but is discouraged for various reasons mostly related to full official updates, see the NOTES section at the bottom .
Once you're done adding new fonts, run the Fonts -> Fonts Hack Behavior > Update fontconfig cache KUAL button to let the Kindle know about them.
Once the framework has restarted, you should now see your new fonts in the Aa menu (except on FW >= 5.4.x, but you do need to follow this step to add new fonts)! Make sure you try them both in 'old' Mobi7 files, *and* in KF8 files: if the UI freezes when you try to turn the page, or if the framework restarts when you open the Aa menu, congratulations, you've found a font the framework chokes on! (Running restart lab126_gui should throw you back to the Home screen, and let you kill the offending font. Typically, this is this kind of stuff using a custom FT override might help with.).
Once you're done mucking around, and after checking that everything seems to be working okay, I'd recommend doing a full, proper restart, just to lower the chances of something wonky happening at boot.

If you're familiar with the USE_ALT_FONTS tweak, you're also aware that it makes the full range of CJK fonts visible in the Aa menu. To remove them (all) from the Aa menu, run the Fonts -> Fonts Hack Behavior > Hide CJK Fonts KUAL button, and then restart the framework (via the Fonts > Restart the framework now KUAL button) or your device.

Like my K2/3/4 Fonts hack, you will also have a choice of three different FreeType libraries to choose from in the Fonts -> FreeType Override KUAL menu. Freetype's job is to render the fonts, so enabling this can have a drastic impact on the 'look' of your fonts. It can also help avoid a lot of weird issues with custom fonts. I'm personally usually running the BCI override.
To give you a rough idea of the differences between the three choices: BCI should be the closest to Amazon's, it uses the native bytecode to hint TrueType fonts; AutoHint instead relies on freetype's own code to automatically hint fonts (it can help with fonts with broken bytecode, or at least bytecode that freetype chokes on); and light is basically autohint with a twist, might be a little closer to BCI.
(I'll refer you to the freetype & fontconfig documentation if you want more info about all this).
You'll need to restart the X server (via the Fonts > Restart the X server now KUAL button) to take an FT override into account. Make sure your system is in a sane state (ie. it cold boots properly) before using an FT override (meaning don't try to do a billion things at once, take it step by step), and make sure the X server restarts properly before rebooting your Kindle.
To be perfectly honest, I'm usually doing these configuration tests with the device plugged, and an SSH shell available & ready, to be able to catch and stop anything wonky happening before it escalates into more annoying/time consuming stuff… (Running stop x should take care of the usual suspects, and leave you ample time to fix stuff without playing against the clock).

And finally, and of particular importance for FW >= 5.4.x users, you'll also be able to override the default fonts for the four basic families (serif (Caecilia Regular), sans (Helvetica Neue LT), mono, condensed (Caecilia Condensed)), plus the fallback font (Code2000). This is handled via the Font Overrides KUAL menu. Each basic family has a submenu for Amazon fonts, and Custom fonts. Those submenus are blank by default. This is why there is a Font Overrides > Populate this menu button . Once you see the 'Finished rebuilding menu' message at the bottom of your screen, go back to the Font Overrides menu (if it's still blank -- which might happen if you have a heavy KUAL extensions folder, or a lot of custom fonts --, hit the Font Overrides > ** Refresh ** button).
It should now be pretty self-explanatory: clicking a font in a family submenu will set this font as the default for this family! Run the Fonts > Restart the framework now KUAL button to take these changes into account.
Of course, make sure the font you want to use actually works correctly on your setup, before shooting yourself in the foot .
It goes without saying, but do not override a default family with a font that only works with a custom FT lib, that's only asking for trouble!
Keep in mind that this KUAL menu can be pretty large, so if you feel it's slowing down KUAL cache refreshes too much, feel free to blacklist it (extension name linkfonts-ovr) via the KUAL_search_exclude_paths setting in KUAL's config .

Also, for those a bit more familiar with fontconfig, you're welcome to take a look at the examples in linkfonts/etc/conf.avail, a few of those can be pretty useful (more fine-grained control, making some fonts bigger, …) .

IMPORTANT NOTES & TROUBLESHOOTING:

Spoiler:

Don't try to force a custom update by rebooting your Kindle. You should always install custom hacks via the Settings page. If the 'Update Your Kindle' link is greyed out, your firmware is too old, you did something wrong, or you have an unknown Kindle model (in which case, contact me!). Don't try to force an install by rebooting. It'll, at best, fail.

Another thing: every time I mention rebooting or restarting your device, you need to do it with the device completely *unplugged*. It may sound strange, but it affects a number of things in weird and interesting ways .

On the PW2, you'll need KUAL >= 2.3 (or a recent enough snapshot).

IMPORTANT NOTICE REGARDING OFFICIAL UPDATES: If you're NOT using version 0.5.N or newer, or if you're using the USE_ALT_FONTS trick without this hack, I strongly urge you to change back your reader font to a default one (Caecilia), and to remove the USE_ALT_FONTS file and to remove/rename the fonts folder on the root of your Kindle before applying official updates, that will potentially save you a world of hurt…

Even with this hack, I'd recommended going back to a default font in the reader, and if you're still using the default fonts folder instead of the hack-specific one (linkfonts/fonts), then to take this opportunity to move to the hack-specific one before updating [There's a KUAL button for that (Fonts -> Fonts Hack Behavior > * Migrate your fonts directory *) ].

IMPORTANT NOTICE FOR FW >= 5.4.x USERS: I repeat, currently, the only way to actually use your custom fonts on your device is to use the Font Overrides feature. If this is still a bit muddy for you, check the first part of this post for a barebones version of the best procedure to follow on these new devices/firmware versions.
And for the truly adventurous, you can look at ameho's post on how the 'official' custom font support works on FW 5.4.x….

The fallback override is full of fun & interesting quirks, if anything feels wonky, do read the comments in the config shard (linkfonts/etc/fc-override-fallback.tpl), or take a look at this thread, where I tested it for the first time.

As will be mentioned in a bit, unless you know & understand the risks, I'd recommend staying away from the Sans override: due to its effect on a wide range of things in the UI, it's far more volatile than the rest.

On a related note, if you intend to delve into font tweaking, those two posts should give you a bit of insight into the process (especially if you intend to tweak the line-height of your font): a nice recap from mandy314, and a couple of Fontforge screenshots of a before/after. Good luck .

Overriding the sans family (or just Helvetica or Futura) might lead to a number of weird (and potentially 'fatal') issues, because these fonts are used in various places in the UI, and at points where the userstore might not be available. You've been warned, but it's generally a very bad idea .
Note that since FW 5.7.2, the situation should be much safer, because it's now Amazon Ember that's used all over the place, and not Helvetica and/or Futura .

On the subject of Code2000, note that it's only used as a last resort fallback. If you happen to install a custom font supporting a specific script, it will get picked up to render that script over Code2000, no matter what your fallback override is set to. On the other hand, for complex scripts, it might render with some minor to showstopping layout issues: this is what the fallback override was designed to fix .


ChangeLog:

Spoiler:

• Fonts:
  • v0.1.N:
    • First release .
  • v0.2.N:
    • Rebuilding the Font Overrides menu is now much faster.
    • Be a little less stupid about when to start/stop the hack, it might help avoid corner case issues…
    • Add a 'Restart the X server' button to help testing FT overrides in a slightly more controlled environment.
    • Made the 'restart framework' & 'update fontconfig cache' KUAL buttons slightly less stupid.
    • Made the doc scarier. Booh!
  • v0.3.N:
    • Updated binaries (updated tc).
  • v0.4.N:
    • Handle the USE_ALT_FONTS file in a smarter way, to avoid breaking X or worse after a full official update…
    • Updated binaries (updated tc).
  • v0.5.N:
    • Updated FT binaries (fix the clipping issues with synthetic emboldened fonts)
    • Recommend using our own fonts directory in the doc to avoid issues after a full official update, and add a KUAL button to automatize the migration.
  • v0.6.N:
    • Updated binaries (updated tc & ft).
  • v0.7.N:
    • Initial PW2 support (font overrides only).
    • Updated binaries (updated tc & ft).
    • Some font overrides tweaks & fixes.
  • v0.8.N:
    • Fix the crashes when switching to USB mass storage with an FT override enabled on FW >= 5.4
    • Updated binaries (updated tc & ft).
  • v0.9.N:
    • Updated binaries (updated tc & ft).
    • Handle the 'new' PW2 models.
    • Add a new font override entry in the menu: fallback, to override Code2000 (and Helvetica when it's used as a fallback).
    • Fix the serif override being a bit too eager. It doesn't affect Palatino & Baskerville anymore, as was always intended.
    • Made the actual generation of the fontconfig shard much faster when enabling a font override.
    • Fix the query button in the sans override menu so that it actually does its job.
    • The freetype libraries used for the FT override are now built w/ harfbuzz support, which may help when autohinting some CFF fonts.
    • Add the extra stock Japanese & Chinese fonts to the CJK blacklist
  • v0.10.N:
    • Handle FT overrides on FW >= 5.6.1
    • Updated binaries (updated tc & ft).
  • v0.11.N:
    • Updated binaries.
    • Fix a mostly harmless but broken and inconsistent behavior on stop when using an FT override (in some circumstances).
  • v0.12.N:
    • Updated binaries (updated tc & ft).
    • Fix eips handling on the KT2.
  • v0.13.N:
    • Updated binaries (updated tc & ft).
    • Proper handling of the changes introduced in FW 5.6.5 (i.e., Bookerly).
  • v0.14.N:
    • Updated binaries (updated tc & ft).
    • Proper handling of the changes introduced in FW 5.7.2 (i.e., Ember).
• Doc: $Id: FT_MR_THREAD 16004 2019-06-04 17:09:41Z NiLuJe $

[NiLuJe]

With the usual caveats of viewing eInk screenshots on an LCD screen, here are a few useless screenshots of the KUAL menus & the aA menu, with the BCI FreeType override.

Another note: since I'm not sure how this all behaves on anything older than FW 5.3, the update package requires FW 5.3.1, so if it mysteriously disappears from your device when you try to install it, it's because you're using an older firmware .

This was developed for KUAL 2.1, so make sure you have the latest version of KUAL installed .

A good resource for free fonts is Google's Web Fonts DB: http://www.google.com/fonts/ && https://code.google.com/p/googlefontdirectory/ (Thanks to knc1 for reminding me of that ).

EDIT: The last screenshot illustrates the behavior of the font override feature on FW 5.4.x: Serif (Caecilia) has been overridden by Adobe Garamond Pro, and condensed (Caecilia Condensed) has been overridden by Crimson.

[NiLuJe]

And before someones asks:

If I don't care about default fonts overrides, and custom freetype libs, what does this do better than just using USE_ALT_FONTS manually?

Well, not much, but a bit: a nicer UI via KUAL, and a saner & faster handling of cache updates & framework restarts (we never switch the rootfs rw, we only update the cache of custom fonts so it's much faster, and it's saved/restored at boot/shutdown to avoid having to rebuild it every boot [the fontconfig cache lives in a tmpfs]). Also, the KF8 reader is restarted, so the initial testing of new fonts in all kinds of content is much faster because it doesn't initially require a full restart.

Speaking of, with the CJK fonts hidden, 15 custom font families is a good maximum number if you want to keep the Aa menu manageable .

And finally: I haven't tested the browser and/or mesquite apps, so I do not know how these will behave. AFAICT, it should just work, but the font overrides might not take there.

[Mailyfesux]

I tried the BCI override, now my kindle pw at version 5.3.6 is saying Repair needed. Is it bricked? How can I fix it?

[ixtab]

Originally Posted by Mailyfesux

I tried the BCI override, now my kindle pw at version 5.3.6 is saying Repair needed. Is it bricked? How can I fix it?

Boot into diags, undo all changes that you did, and to be on the safe side, blast mmcblk0p3, too.

Refer to (the second part of) this file for the required steps - but only perform the ones that you need. You don't have to overwrite neither the kernel nor the partition. And once you're done, make sure to reinstall the jailbreak bridge and the rescue pack.

[NiLuJe]

@Mailyfesux: Argh. What ixtab said.

What did you try, in as much details as possible, and in what order? (ie. did you have custom fonts installed, if so how many and which ones, was the fontconfig cache up to date, etc.).

Also, where (when) in the boot process did it first start to go wrong, if you can tell (ie. tree, spinning circle, progress bar?)

EDIT: For the record, and if you don't have the rescue pack installed, the most conservative (ie. blasting the least amount of stuff) way to recover from this would be to:

1/ Disable the hack: delete USE_ALT_FONTS and linkfonts/auto
1b/ You might want to clear the fonts directory, too.
2/ Reset the very, very annoying restart counter as described in the bottom of the page ixtab linked to.

For point 2/, if you don't have the rescue pack installed, all is not lost. Every startup script I write start by looking for an 'emergency' script, for just such an occasion. The fonts hack will try to launch linkfonts/bin/emergency.sh. So just put the commands shown in ixtab's howto in there as a simple shell script (unix line endings), like so:

#!/bin/sh

rm -f /var/local/upstart/lab126_gui.restarts
rm -f /var/run/upstart/lab126_gui.restarts

# Hell, might need this too...
rm -f /var/local/upstart/x.restarts
rm -f /var/run/upstart/x.restarts

# Hai, PW2!
rm -f /var/local/upstart/mesquite.restarts
rm -f /var/run/upstart/mesquite.restarts

[Mailyfesux]

I went into fonts, installed about 12 fonts. updated the fontconfig cache, restarted the device, set the bci override, and then reset the device which gave the Repair Needed screen.

The repair needed screen showed up during the spinning circle

I am still currently in the process of trying to unbrick it.

[NiLuJe]

@Mailyfesux: Assuming you were using the KUAL menu, did all the fonts work (especially in KF8 stuff) after the framework restart triggered by the fontconfig cache update?

Did you ever encounter any data corruption issue with your device? (broken books, books breaking the indexing process, your OS complaining about the filesystem when mounting the device over USB, ...)

EDIT: Or, more obvious, is the userstore full/nearly full?

[Mailyfesux]

I did not try the KF8 stuff. but the fonts I added worked. There was no previous data corruption. I dont believe the userstore was full.

I disabled the font hack via your instructions and now everything is fine.

[knc1]

Originally Posted by Mailyfesux

I did not try the KF8 stuff. but the fonts I added worked. There was no previous data corruption. I dont believe the userstore was full.

I disabled the font hack via your instructions and now everything is fine.

Would you install this button:
https://www.mobileread.com/forums/sho...d.php?t=219454
and post the text document created with your next post please?

[NiLuJe]

@Mailyfesux: Glad you got it running again, and sorry for the bad experience :/.

I'm a bit stumped as to why it would blow up like that, so If you could PM me some logs, that might help shedding some more light on this... (run ;dm from the Home page's search bar, it will dump the logs in a file named all_logs_<blahblah>.txt in the documents folder).

[richy1989]

I want to get a new font to modify(override) the default fonts,what can I do to make it work with the Fonts hack?
I don't want it become"Your kindle is Repair needed" ,I am in china,it is difficult to repair it by myself.

[NiLuJe]

@richy1989: Cheap quote incoming. Do or do not, there is no try.

If you do want to use it, just follow the first post, everything's in there (don't forget to install Python beforehand, the Font Override menu depends on it) .

If the K5 mentioned in your profile is a Touch, you'll always have Kubrick if something goes seriously wrong.

[richy1989]

Originally Posted by NiLuJe

@richy1989: Cheap quote incoming. Do or do not, there is no try.

If you do want to use it, just follow the first post, everything's in there (don't forget to install Python beforehand, the Font Override menu depends on it) .

If the K5 mentioned in your profile is a Touch, you'll always have Kubrick if something goes seriously wrong.

Well!
Let me try it!
Thank you!

And my device is Paperwhite, not a Touch

[knc1]

Originally Posted by richy1989

And my device is Paperwhite, not a Touch

In that case, Kubrick will not be of any help.
Hint: Keep your serial port cable handy.