Full vs. incremental updates

[Renate]

Ok, most of you all know this stuff, but just to restate the obvious.

A full update has all (most) of the system stuff you need on your device to run.
It makes no presumptions of what system data you currently have on your device.
On the other hand, it can refuse to update to an older version.
On Onyx a full update is about 1.5 GB or so.

An incremental update contains modifications to existing sytem data on your device.
It presumes that it is modifying a specific, known, last installed, full version.
If you have modified anything by rooting or hacking there may be complications.
On Onyx an incremental update is about 0.5 GB or so.

To make things a bit more complicated, a single update may be partially full and partially incremental.

As an example, here is the last Poke5P full update:

Partition         New   Operations
--------------  ------  --------------------------------------
boot            96.0 M  REPLACE[3], REPLACE_XZ[6], REPLACE_BZ[39]
dtbo            8.00 M  REPLACE_XZ[1], REPLACE_BZ[3]
product          389 M  REPLACE[26], REPLACE_XZ[168], REPLACE_BZ[1]
system          2.09 G  REPLACE[59], REPLACE_XZ[992], REPLACE_BZ[21]
system_ext       286 M  REPLACE[13], REPLACE_XZ[127], REPLACE_BZ[3]
vbmeta          8.00 k  REPLACE_XZ[1]
vbmeta_system   4.00 k  REPLACE_XZ[1]
vendor           487 M  REPLACE[1], REPLACE_XZ[242], REPLACE_BZ[1]
xbl             3.01 M  REPLACE_XZ[2]

You can see that it does not contain every single partition.
But the ones that it does have, it fully overwrites.
If you replaced the boot image for Magisk you won't have root afterwards, but it won't explode.

Here is a Poke5P incremental update:

Partition         Old     New   Operations
--------------  ------  ------  --------------------------------------
abl                      148 k  REPLACE_XZ[1]
boot            96.0 M  96.0 M  ZERO[40], SOURCE_COPY[1], PUFFDIFF[1], BROTLI_BSDIFF[1]
dtbo            8.00 M  8.00 M  ZERO[4], BROTLI_BSDIFF[1]
product          389 M   637 M  ZERO[3], REPLACE_XZ[41], REPLACE_BZ[78], SOURCE_COPY[310], PUFFDIFF[6], BROTLI_BSDIFF[56]
recovery                96.0 M  ZERO[35], REPLACE[3], REPLACE_XZ[9], REPLACE_BZ[1]
system          2.09 G  1.92 G  ZERO[5], REPLACE_XZ[67], REPLACE_BZ[147], SOURCE_COPY[1876], PUFFDIFF[30], BROTLI_BSDIFF[293]
system_ext       286 M   284 M  ZERO[2], REPLACE_XZ[3], REPLACE_BZ[25], SOURCE_COPY[252], PUFFDIFF[6], BROTLI_BSDIFF[110]
vbmeta          8.00 k  8.00 k  BROTLI_BSDIFF[1]
vbmeta_system   4.00 k  4.00 k  BROTLI_BSDIFF[1]
vendor           487 M   487 M  ZERO[3], SOURCE_COPY[800], BROTLI_BSDIFF[203]
xbl             3.01 M  3.01 M  ZERO[4], REPLACE_XZ[1], SOURCE_COPY[5]

It modifies lots of things and if those things aren't how it expects there are going to be problems.
Note that abl is fully replaced. If you were looking just for that you could use a payload dumper and get a clean copy.
The other files, like boot, can not be extracted except in the context of a device updated to a specific full update.

(These printouts are from a work-in-progress utility.)

[Renate]

I've updated the previous post a few times as I've made advances on the utility.

If you do downloading of updates yourself, it's probably good if you note down the download URL. You might want to rollback to an earlier version sometime. Also, some things like abl are being clamped down with removing fastboot flash from the bootloader. Note that there are anti-rollback provisions in these updates. There may be issues getting them to work.

Anti-rollback 2024-07-01 15:50:32

[DanyDanman]

Just checking. If you have a full backup made with EDL. And want to rollback to an earlier version sometime can I?

Will the Anti-rollback pose a problem?

Even further down the line might EDL access be blocked off?

Thank you for your hard work!

[Renate]

A full, raw EDL backup (with a combined size the same as your flash, 32GB or 128 GB) will always go back on your device just fine. If you flash it on your friend's device you have just overwritten the device ID, WiFi and Bluetooth MACs and who knows what else.

Do not ever "wipe" your data. That destroys your keys and even a raw backup will not save you.

Although the (freely available) documentation is poor, I presume that additional efuses on your device could be blown. If the OEM were to release some evil update they could blow fuses and clamp down on a lot of things. That would be evil.

The anti-rollback would not necessarily be a big deal, but it might require some effort. We've seen that we can flash the "wrong" software on a device (e.g. flash Poke5P on a Poke5). The normal update process checks the metadata and signing on the update.zip, but I update without that anyway.

[Renate]

I've been using my autoupdate in recovery to flash update.zip from a USB flash drive.
It works just fine and I've even flashed my poor Poke5 to Chinese and back again.
I can be so brave because I have a full raw backup, which I have EDL'ed in a few times.

But incremental updates can be problematical.
They need a lot of elbow room to rearrange the super partition.
Probably more space than you have RAM unless you have something with 4 GB or more.

Well, I'm working on this:

01/01/1980  00:00       670,998,528 product_a-cow-img.img.0000
01/01/1980  00:00     1,318,719,488 system_a-cow-img.img.0000
01/01/1980  00:00       298,991,616 system_ext_a-cow-img.img.0000
01/01/1980  00:00       451,747,840 vendor_a-cow-img.img.0000

As you can see, you need about 2.75 GB elbow room.
I could probably get this to work if I formatted a flash drive with ext2.

[Quyen607]

Hi Renate, Can we extract a full raw img backup file on PC to research it? is there any way to modify something in the img file?

[Renate]

Yes and yes. If this has nothing to do with this thread, start another.

[denisuu]

Originally Posted by Renate

If you flash it on your friend's device you have just overwritten the device ID, WiFi and Bluetooth MACs and who knows what else.

Interesting Thread!

I always believed that a MAC address was "burned-in" and could only be spoofed, not changed. Does it also exist in software? I thought that serial numbers, IMEIs, and MAC addresses couldn't be altered, only spoofed.

[Renate]

The processor chip serial number is burned in.
That's the one you get back with EDL, not the one you see on "adb devices".

I've never really dived into stuff like IMEI, but on phones I believe that's all in signed partitions that you can't really patch yourself.

I've been working on these incremental updates. I've patched the updater so that it doesn't reuse free areas of super. That makes the needed "elbow room" 3.5 GB. I've tried using vfat, exfat, ntfs, ext2, ext4. I'm still working on this.

[Never_Sm1le]

Originally Posted by denisuu

Interesting Thread!

I always believed that a MAC address was "burned-in" and could only be spoofed, not changed. Does it also exist in software? I thought that serial numbers, IMEIs, and MAC addresses couldn't be altered, only spoofed.

From my experience with Xiaomi devices, they are stored in either efs or persist partition, and with edl I see my poke 5s also contain /persist. Maybe it's in there.

[Renate]

I've been beating my head on how to do updates in recovery. The update process itself is unbelievably complicated. The device-mapper is heavily involved. I've had to do a deep dive into that (mostly undocumented) subject. My usage case is to be able to take a heavily modified system, restore it to stock with a full update, then update it with an incremental update, Of course your own rooting/modifications have to be performed again adapting to whatever changes were made. I just got 3.5.1 and 3.5.3 installed 100% ok, but I get a 900e (system crash). Oh, well, reflash to the last full backup.

[Quyen607]

Wow, you got 3.5.1 and 3.5.3 installed by Recovery/Update OK. How about the system crash.

[Renate]

I have to do more work on this. I don't know what the problem is yet. The updates are successful and all the hashes check.

The update engine allocates files for spare space but then it accesses this space using a device mapper (linear target). That means that I have to mount the flash (internal or external) by device mapping it first and then mounting the dm. My dm is mapped by the update engine's dm. Apparently that is the only way to share a physical device between a file system and a mapper. That works fine now.

[Renate]

I finally got my Poke5 to 3.5.3

In short, I used my (always improving) custom recovery and a flash drive to switch from my custom version to stock 3.5.1
Actually, I used the micro SD slot on the Poke5. I forgot that I had it.
I had been trying for weeks to do incremental updates.
They are amazingly complex and don't work reliably (or at all).
So running the stock 3.5.1, I pushed 3.5.3 incremental update.upx (still encrypted) to sdcard, then:

am start -a onyx.settings.action.firmware

It rebooted fine into stock 3.5.3
Since my mods are all done on parts of super, I went to look what was there.
Santa's elves had certainly left a mess there!

====== Slot 1 ======
13000 Header v10.2
13100 P0 system_b (readonly updated) E0 G1
13134 P1 system_ext_b (readonly updated) E1 G1
13168 P2 product_b (readonly updated) E2-5 G1
1319c P3 vendor_b (readonly updated) E6 G1
131d0 P4 system_b-cow () E7 G2
13204 E0 2048 - 4033624 (4031576) D0
1321c E1 4392712 - 4974408 (581696) D0
13234 E2 4978120 - 5775576 (797456) D0
1324c E3 4033624 - 4392712 (359088) D0
13264 E4 4974408 - 4978120 (3712) D0
1327c E5 6772024 - 6917208 (145184) D0
13294 E6 5775576 - 6772024 (996448) D0
132ac E7 6917208 - 8388608 (1471400) D0
132c4 G0 default ()
132f4 G1 qti_dynamic_partitions_b ()
13324 G2 cow ()
13354 D0 super

So I cleaned it up (with my newly updated lputil) so:

====== Slot 1 ======
13000 Header v10.2
13100 P0 system_b (readonly) E0 G1
13134 P1 system_ext_b (readonly) E1 G1
13168 P2 product_b (readonly) E2 G1
1319c P3 vendor_b (readonly) E3 G1
131d0 E0 2048 - 4033624 (4031576) D0
131e8 E1 4033624 - 4615320 (581696) D0
13200 E2 4615320 - 5920760 (1305440) D0
13218 E3 5920760 - 6917208 (996448) D0
13230 G0 default ()
13260 G1 qti_dynamic_partitions_b ()
13290 D0 super

And it even booted up!

Now I just have to put all my mods back in.

[Quyen607]

wow, you did an amazing work, can we switch from Poke5 3.5.3 to Poke5P 3.5.3 by your way:
1. Flash to Poke5P 3.5.1
2. Put Poke5P 3.5.3 update.upx to Sdcard , it will not boot up to system, boot to adb, then:

am start -a onyx.settings.action.firmware

3. Clean it up