kobopatch - An improved patching system

[geek1011]

kobopatch
An improved patching system

------

Continued from https://www.mobileread.com/forums/sh...d.php?t=296597

Features/improvements over patch32lsb:

• Faster
• No temp files
• Preserves all file metadata, including ownership and xattrs
• More checks and better error messages (including patch_group and syntax validation)
• Optional new format which is more powerful and consistent (demo)
  • New instructions like FindReplaceString, ReplaceZlib, FindZlib, FindZlibHash, FindBLX, ReplaceBLX, ReplaceBytesNOP, FindBaseAddressHex, FindBaseAddressSymbol, ReplaceBytesAtSymbol
  • Can specify offsets using integers as well as hex
  • More consistent escaping
  • More
• Override patch_enable options from a single config file (to make it easier to upgrade)
• Multiple patch files for a single binary (to make it easier to upgrade)
• No external dependencies (like 7-zip)
• Fully backwards-compatible
• Extensible
• Can be embedded in another application more easily (I might even make a web ui)
• Zlib support
• Assembly support
• Simplified BLX replacement
• Simplified replacement of bytes based on a symbol name (AKA no more recalculating offsets every version for replacing values in functions)
• Support for compiling and adding translation files
• Support for adding custom files to KoboRoot.tgz
• Patches are automatically checked to make sure they apply before each release

Usage:

1. Download the zip for your firmware version from here.
2. Download the firmware from here to the src folder. The zip should be called something like kobo-update-1.2.3456.zip.
3. Install the firmware on your kobo BEFORE proceeding with the next steps.
4. Enable patches in the files in the src folder (or use the overrides in kobopatch.yaml to keep your options separate).
5. Run kobopatch.bat on Windows, or kobopatch.sh on Linux.
6. If the patching succeeded, a file named KoboRoot.tgz will be created in the out folder. Copy it to the .kobo folder of your device.

Saving patch_enable options separately:
See kobopatch.yaml in the patch zip for instructions.

Reporting bugs:
You can either reply here or open an issue on GitHub. Please provide log.txt along with your report.

Code | Downloads | Patch Downloads | Patch Requests

[geek1011]

FAQ:

What should the overrides option in kobopatch.yaml look like?
Here is mine as an example:

Spoiler:

overrides:
  src/nickel.patch:
    Custom synopsis/details line spacing: yes
    Custom Header menubar - reduce height by 33%: yes
    Increase The Cover Size In Library: yes
    Increasing The View Details Container: yes
    Reading stats/Author name cut when the series is showing bug fix: yes
    Increase size of Kepub chapter progress chart: yes
  src/libadobe.so.patch:
    Remove PDF map widget shown during panning: yes
  src/libnickel.so.1.0.0.patch:
    Custom font sizes: yes
    Freedom to advanced fonts control: yes
    Custom Sleep/Power-off timeouts: yes
    Fix three KePub fullScreenReading bugs: yes
    Always display chapter name on navigation menu: yes
    Clock display duration: yes
    New home screen footer rename: yes
  src/librmsdk.so.1.0.0.patch:

Is this stable?
It is. It has many automated tests, both at compile-time and at runtime. I periodically run manual tests comparing the output of all the patches with the existing patching system. I have also used this on my kobos for over 3 months with 4 different firmware versions.

What is YAML?
YAML is a superset of JSON.

Can I use this with old patches?
Yes.

How can I update kobopatch?
Download the latest binaries from the release page and put them in the bin folder.

How does kobopatch convert from the old patch zips?
First, it extracts the zip and copies the patches to a new folder. Second, it processes the patch files and uncomments instructions which are commented but it recognizes (like patch_group). This allows compatibility to be maintained with the original patcher for now. Finally, it puts kobopatch and the default config in the folder and zips it.

When is zlib support coming?
zlib support was implemented on July 9, 2018.

How can I convert the patch32lsb format to the new kobopatch format?
You can use this tool. Note that this tool is still alpha quality and needs manual checking after conversion. It also has issues with binary patches (e.g. the zlib ones). It should be stable by v1.0.0.

For individual patches, you can manually convert according to the samples included in the kobopatch repository. Manually converted patches currently are stable, including the binary ones.

How do I add translation files?
First, put the ts files in the src folder. Second, update kobopatch.yaml to match. If you haven't already done so, install the QT Linguist tools. Finally, run kobopatch normally.

Why does my keyboard have the wrong letters when I enable the "Rename new home screen footer" patch?
This is a side effect of some of the string-related patches (especially the "Rename new home screen footer" patch). Due to the way QT stores strings, sometimes, it will use part of another string to save space, and as a result, another string may change as well (such as the keyboard).

Why does kobopatch say a FindReplaceString with special characters has the wrong replacement length?
QT uses the UTF-8 string encoding, which encodes special unicode characters as two bytes. This means that many special unicode characters (like accents, upside-down chars, fancy arrows, etc) may take up two bytes.

[geek1011]

Changelogs
See https://github.com/geek1011/kobopatch/releases .

[geek1011]

Tips and Tricks

Separating custom patches from the included ones
To make upgrading easier, as well as using the overrides section of kobopatch.yaml, you can also put custom patches which are not included into their own file. To do this, make a new file in the src folder, for example nickel.custom.patch (for patch32lsb format), or nickel.custom.yaml (for the new format). Afterwards, add a line in the patches section of kobopatch.yaml corresponding to your new file. Here is an example of mine:

Spoiler:

## Works with kobopatch v0.6 or higher.
## You can update kobopatch by downloading the latest release from https://github.com/geek1011/kobopatch/releases. 
version: 4.9.11311
in: src/kobo-update-4.9.11311.zip
out: KoboRoot.tgz
log: log.txt

patchFormat: patch32lsb

# Note the custom nickel.oren64.patch and libnickel.so.1.0.0.geek1011.patch files which patch the same binary.
patches:
  src/nickel.patch: usr/local/Kobo/nickel
  src/nickel.oren64.patch: usr/local/Kobo/nickel
  src/libadobe.so.patch: usr/local/Kobo/libadobe.so
  src/libnickel.so.1.0.0.patch: usr/local/Kobo/libnickel.so.1.0.0
  src/libnickel.so.1.0.0.geek1011.patch: usr/local/Kobo/libnickel.so.1.0.0
  src/librmsdk.so.1.0.0.patch: usr/local/Kobo/librmsdk.so.1.0.0

overrides:
  src/nickel.patch:
    Custom synopsis/details line spacing: yes
    Custom Header menubar - reduce height by 33%: yes
  src/nickel.oren64.patch:
    Increase The Cover Size In Library: yes
    Increasing The View Details Container: yes
    Reading stats/Author name cut when the series is showing bug fix: yes
    Increase size of Kepub chapter progress chart: yes
  src/libadobe.so.patch:
    Remove PDF map widget shown during panning: yes
  src/libnickel.so.1.0.0.patch:
    Custom font sizes: yes
    Freedom to advanced fonts control: yes
    Custom Sleep/Power-off timeouts: yes
    Fix three KePub fullScreenReading bugs: yes
    Always display chapter name on navigation menu: yes
    Clock display duration: yes
  src/libnickel.so.1.0.0.geek1011.patch:
    New home screen footer rename: yes
  src/librmsdk.so.1.0.0.patch:

[geek1011]

Discussion Questions

1. Is there any additional features you would like?
2. Is there anything you would like to change?
3. How do you want the zlib patching to work (format, usecases, etc)
4. How much would you value a web interface (like the old KHD)?
5. Is there anything else you want to ask/say?

[Terisa de morgan]

I would like to be able to include the updated translations in a directory so they are added to nickel. At this moment, I've modified the bat file for adding them, but I would like if they are included from the beginning.

[geek1011]

Originally Posted by Terisa de morgan

I would like to be able to include the updated translations in a directory so they are added to nickel. At this moment, I've modified the bat file for adding them, but I would like if they are included from the beginning.

So you want to be able to have a dir which gets added to KoboRoot.tgz after the patches are applied, right?

[sherman]

Originally Posted by geek1011

So you want to be able to have a dir which gets added to KoboRoot.tgz after the patches are applied, right?

Speaking for myself, I'd love this as well, so I could include modified hyphenation dictionaries in one step.

[Terisa de morgan]

Originally Posted by geek1011

So you want to be able to have a dir which gets added to KoboRoot.tgz after the patches are applied, right?

It's not directly added but processed. Tomorrow I'll attach the updated bat so you can check it.

[geek1011]

Thanks for the feedback. As for kobopatch itself, is it working well for you?

[jcn363]

Originally Posted by geek1011

Thanks for the feedback. As for kobopatch itself, is it working well for you?

So you want to be able to have a dir which gets added to KoboRoot.tgz after the patches are applied, right?

Yes, please.

Can we have groups of settings in kobopatch.yaml?

Spoiler:

overrides:
  normal: no
    src/nickel.patch:
      Custom synopsis/details line spacing: yes
      Custom Header menubar - reduce height by 33%: no
      ...
  reduced: yes
    src/nickel.patch:
      Custom synopsis/details line spacing: yes
      Custom Header menubar - reduce height by 33%: yes
      ...

[Terisa de morgan]

I have a question. There are some patches that, aside of been enabled or no, accept some customization (Custom footer (page number text), for example). How would you handle it here?

[geek1011]

Originally Posted by Terisa de morgan

I have a question. There are some patches that, aside of been enabled or no, accept some customization (Custom footer (page number text), for example). How would you handle it here?

I wouldn't. That's currently beyond the scope of kobopatch, because there is no clean way it can be implemented (covering edge cases, validation) and it would make it a bit confusing to use. I also wanted to keep as much logic as possible in the patch files itself to make it easier to maintain (it's currently the same number of lines of code as the original one). More complexity means more bugs and less checks. The way I was considering was a variables section in kobopatch.yaml which is substituted into all patches which match, but I decided against it. Also keep in mind that users may not necessarily know how to use the feature.

If one of you can think of a better way to implement this, I'd be happy to consider it.

Also, what would you think of a print instruction to show a message to the user during patching? This would be useful to inform about dangerous patches or things like that.

[geek1011]

Originally Posted by jcn363

No problem!

Originally Posted by jcn363

Can we have groups of settings in kobopatch.yaml?

Spoiler:

overrides:
  normal: no
    src/nickel.patch:
      Custom synopsis/details line spacing: yes
      Custom Header menubar - reduce height by 33%: no
      ...
  reduced: yes
    src/nickel.patch:
      Custom synopsis/details line spacing: yes
      Custom Header menubar - reduce height by 33%: yes
      ...

I'm not quite sure I understand what you are asking.

[OfficerAction]

Thanks! Nice work