Register Guidelines E-Books Today's Posts Search

Go Back   MobileRead Forums > E-Book Readers > Amazon Kindle > Kindle Developer's Corner

Notices

Reply
 
Thread Tools Search this Thread
Old 01-22-2016, 10:22 AM   #16
knc1
Going Viral
knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.
 
knc1's Avatar
 
Posts: 17,212
Karma: 18210809
Join Date: Feb 2012
Location: Central Texas
Device: No K1, PW2, KV, KOA
At this point, the kernel is running multi-task.
The messages are inter-leaved (normal).
Here is what it said:
Code:
EXT2-fs error (device mmcblk0p1): ext2_check_page: x1
"bad entry in directory #4332: : rec_len is too small for name_len -
offset=0, inode=3814195200, rec_len=124, name_len=1490"
Translation:
Only part of the new 'main' file system is written (we could have guessed that from the repeated update looping).

This is progress, really, it is.

I have 'life' to deal with today - more tomorrow morning.

But if your curious, we will be using dd to copy the new mmcblk0p1 image.

Meanwhile,
Download a brand new, fresh, copy of the currently listed PW-1 update package from Amazon.
(For some reason, I don't trust your current copy. )

Also get the most recent KindleTool from NiLuJe's snapshots thread.
We need to take that new update package apart to get at the main filesystem image.

You will need your device's serial number.
Also use the KindleTool with your serial number to find the password for your diags login.

We will take apart the Update_*.bin package

Use the 'Recovery Menu' to export USB storage again
Copy the rootfs.img file to USB storage
eject, etc...

Use the 'Recovery Menu' function to "exit to shell login"
The 'dd' the rootfs.img file onto /dev/mmcblk0p1

More exact details tomorrow.

Last edited by knc1; 01-22-2016 at 10:34 AM.
knc1 is offline   Reply With Quote
Old 01-22-2016, 05:11 PM   #17
Whitefox
Member
Whitefox began at the beginning.
 
Posts: 13
Karma: 10
Join Date: May 2014
Device: kindle 4, kindle paperwhite
Sorry I didn't want to bother you

For now I downloaded KindleTool
Code:
$ ./kindletool version
./kindletool (KindleTool) v1.6.4 built by niluje@ajulutsikael on Gentoo with GCC 4.9.2 on May  7 2015 @ 22:16:21 against libarchive 3.1.2 & nettle 3.1.1
Obtained the root password and downloaded the update file (twice just in case )
Code:
$ sha256sum *.bin
6be62d30df26894b08083dbc55d23774756710c3a44c9a1d0075dd02d12090c9  update_kindle_5.6.1.1_b.bin
6be62d30df26894b08083dbc55d23774756710c3a44c9a1d0075dd02d12090c9  update_kindle_5.6.1.1.bin
Whitefox is offline   Reply With Quote
Old 01-22-2016, 07:09 PM   #18
knc1
Going Viral
knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.
 
knc1's Avatar
 
Posts: 17,212
Karma: 18210809
Join Date: Feb 2012
Location: Central Texas
Device: No K1, PW2, KV, KOA
Good thing we work for free.

We both have spent far more time on this device than it is worth.
But at some point, things go beyond reason into being a pure challenge.

It should be fixed tomorrow, if it is fixable at all.

I'll come up with a small script that copies the rootfs.img file one erase block worth of data at a time, then read it back to verify it.

This does not act like a bad flash device, it always wrote the Kernel on every loop that we looked at.
knc1 is offline   Reply With Quote
Old 01-28-2016, 01:09 PM   #19
Whitefox
Member
Whitefox began at the beginning.
 
Posts: 13
Karma: 10
Join Date: May 2014
Device: kindle 4, kindle paperwhite
From your indication I wrote this script, should this work for our purpose ?

Code:
#!/bin/bash
src=$1
dst=$2
tmp1="./tmp1.img"
tmp2="./tmp2.img"

if test $# -lt 2 
then
	echo "Missing parameter"
	exit
fi
size=`wc -c $src | cut -d' ' -f1 ` # in bytes
size=$[ ($size+1024) / 1024 ]
i=0
echo "Image size $size kb"
while test  $size -gt $i
do	
	echo -e "\t$i"	
	#Write from the image to the partition
	dd if=$src of=$dst bs=1024 count=1 skip=$i seek=$i
	#Copy the last read kb to a temp file
	dd if=$src of=$tmp1 bs=1024 count=1 skip=$i
	#Copy the last written kb to another file
	dd if=$dst of=$tmp2 bs=1024 count=1 skip=$i
	#Compare the difference between the files
	val=`diff $tmp1 $tmp2| wc -l` 
	if test $val -ne 0
	then
		echo -e "\tError at $i"
		base64 $tmp1 | head
		echo ""
		base64 $tmp2 | head
	fi
	i=$[$i+1]
done
rm $tmp1 
rm $tmp2
Whitefox is offline   Reply With Quote
Old 01-28-2016, 01:31 PM   #20
knc1
Going Viral
knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.
 
knc1's Avatar
 
Posts: 17,212
Karma: 18210809
Join Date: Feb 2012
Location: Central Texas
Device: No K1, PW2, KV, KOA
Phooey!
I forgot entirely about this thread.

- - - -

Close, but troubles with a bunch of details.
Plus, it has to run under the build of Busybox Ash that is on the Kindle.

Give me another day, now that I have been reminded of it (I can both fix it and test it on PW1).
knc1 is offline   Reply With Quote
Old 01-29-2016, 10:47 AM   #21
knc1
Going Viral
knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.
 
knc1's Avatar
 
Posts: 17,212
Karma: 18210809
Join Date: Feb 2012
Location: Central Texas
Device: No K1, PW2, KV, KOA
Quote:
Originally Posted by knc1 View Post
- - - - -
Download a brand new, fresh, copy of the currently listed PW-1 update package from Amazon.
(For some reason, I don't trust your current copy. )

Also get the most recent KindleTool from NiLuJe's snapshots thread.
We need to take that new update package apart to get at the main filesystem image.

You will need your device's serial number.
Also use the KindleTool with your serial number to find the password for your diags login.
- - - - -
This is where we left off - -

You used the "Recovery Menu" to export USB and deleted anything *.bin and *.dat

You then tried to boot the 'main' OS with the result of:
Code:
kinit: name_to_dev_t(/dev/mmcblk0p1) = dev(179,1)
- - - -
Kernel panic - not syncing: Attempted to kill init!
Which means the filesystem for the 'main' OS is trash.

But we need to get this thing running something!

Let us try the 'diags' OS - -

Catch the u-boot prompt again -
bootm 0xE41000
(Since we don't have any update files on it - it should boot to a login prompt - **should**)

If it does, then use the username 'root' and the password you found with KindleTool to log in.

Make sure that you can get that far along, we need the 'diags' system to run our testing copy script.

Meanwhile, I am working on the script itself.
With only the Ash command line provided by the 'diags' build of Busybox 1.17.1 - it is more than a bit arcane -
But I figured out how to write the copy loop controls, so the rest of the script is now a downhill effort.

Last edited by knc1; 01-29-2016 at 10:54 AM.
knc1 is offline   Reply With Quote
Old 01-29-2016, 11:52 AM   #22
Whitefox
Member
Whitefox began at the beginning.
 
Posts: 13
Karma: 10
Join Date: May 2014
Device: kindle 4, kindle paperwhite
Yes I can login

env
Spoiler:
Code:
[root@kindle root]# env
HISTFILESIZE=1000
INPUTRC=/etc/inputrc
USER=root
HOSTNAME=kindle
HOME=/tmp/root
PAGER=/bin/more 
PS1=[\u@\h \W]\$ 
LOGNAME=root
TERM=vt102
PATH=/usr/local/bin:/bin:/usr/bin:/usr/sbin:/sbin:/usr/local/python/bin
DISPLAY=:0.0
LANG=en_US.UTF-8
HISTSIZE=1000
SHELL=/bin/sh
LC_ALL=en_US.UTF-8
PWD=/var/tmp/root
EDITOR=/bin/vi


ls /usr/bin
Spoiler:
Code:
[root@kindle root]# ls /usr/bin/
[               getconf         pmic            tee
[[              getent          preload         test
awk             getopt          preload_static  time
basename        hd              printf          timeout
clear           head            pwdx            top
cmp             i2cutil         rename          tr
cut             id              renice          tset
cyttspfwloader  ifplugd         reset           tsl
ddate           kbd_mode        rev             tty
dirname         kdb             rpcinfo         ul
dosattr         kdb_static      rtcwake         unzip
du              killall         script          uptime
env             ldd             seq             volname
evtest          less            setfdprm        waitforkey
expr            line            setsid          wc
fdformat        logger          setterm         which
find            logname         showkey         write
flock           md5sum          showlog         xargs
free            mkfifo          sort            yes
fusermount      nc              tail
gasgauge-info   parselog        tailf
Whitefox is offline   Reply With Quote
Old 01-29-2016, 02:12 PM   #23
knc1
Going Viral
knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.
 
knc1's Avatar
 
Posts: 17,212
Karma: 18210809
Join Date: Feb 2012
Location: Central Texas
Device: No K1, PW2, KV, KOA
You used the KindleTool to find your password.
Now use the KindleTool to extract the files from the Amazon update package.
(kindletool --help gives you the gory details)

At the top level of the output tree created, you will find a file named: rootfs.img
That's the baby we want to write to flash (or find out why not).

- - - -

That much above was alright.
The rest of the information, and the script, was trash.

Last edited by knc1; 01-30-2016 at 08:11 AM.
knc1 is offline   Reply With Quote
Old 01-29-2016, 03:08 PM   #24
knc1
Going Viral
knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.
 
knc1's Avatar
 
Posts: 17,212
Karma: 18210809
Join Date: Feb 2012
Location: Central Texas
Device: No K1, PW2, KV, KOA
That first version is wrong.

And at least one copy was downloaded.
Whoever got it, please discard the script.

Last edited by knc1; 01-30-2016 at 08:12 AM.
knc1 is offline   Reply With Quote
Old 01-29-2016, 04:45 PM   #25
knc1
Going Viral
knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.
 
knc1's Avatar
 
Posts: 17,212
Karma: 18210809
Join Date: Feb 2012
Location: Central Texas
Device: No K1, PW2, KV, KOA
Phooey.
Busybox dd truncates the output file and does not accept the 'notrunc' option.

So this will take a script (almost done now) plus a copy of dd from the Debian coreutils package for armel Wheezy (yup it runs, I just checked it).

I also have a working copy of dc3dd to toss into the mix (it does file forensics while copying
- including write confirmation).

Example:
Spoiler:

Code:
$ dc3dd if=/dev/zero hof=zeros.bin hash=md5 ssz=1024 cnt=14

dc3dd 7.0.0 started at 2016-01-31 00:14:21 -0600
compiled options:
command line: dc3dd if=/dev/zero hof=zeros.bin hash=md5 ssz=1024 cnt=14
sector size: 1024 bytes (set)
14336 bytes (14 K) copied (100%), 0.201406 s, 70 K/s                          
output hashing (100%)                                                         

input results for pattern `00':
   14 sectors in
   b211778b80f6d441b6cf61ada776fc6d (md5)

output results for file `zeros.bin':
   14 sectors out
   [ok] b211778b80f6d441b6cf61ada776fc6d (md5)

dc3dd completed at 2016-01-31 00:14:21 -0600
All ones is the flash version of 'erased':
Code:
$ dc3dd pat=FF hof=ones.bin hash=md5 ssz=1024 cnt=14 nwspc=on verb=on

dc3dd 7.0.0 started at 2016-01-31 00:27:09 -0600
compiled options:
command line: dc3dd pat=FF hof=ones.bin hash=md5 ssz=1024 cnt=14 nwspc=on verb=on
sector size: 1024 bytes (set)
14336 bytes (14 K) copied (100%), 0.201417 s, 70 K/s                          
output hashing (100%)                                                         
input results for pattern `FF':
   14 sectors in
   6b2cbf063cad35950b55aee7b150b32c (md5)
output results for file `ones.bin':
   14 sectors out
   [ok] 6b2cbf063cad35950b55aee7b150b32c (md5)
dc3dd completed at 2016-01-31 00:27:10 -0600
Or a repeating text pattern:
Code:
$ dc3dd tpat="lab126 sucks! " hof=lab126.bin hash=md5 ssz=1024 cnt=14 nwspc=on verb=on

dc3dd 7.0.0 started at 2016-01-31 00:33:09 -0600
compiled options:
command line: dc3dd tpat=lab126 sucks!  hof=lab126.bin hash=md5 ssz=1024 cnt=14 nwspc=on verb=on
sector size: 1024 bytes (set)
14336 bytes (14 K) copied (100%), 0.201305 s, 70 K/s                          
output hashing (100%)                                                         
input results for pattern `lab126 sucks! ':
   14 sectors in
   879083cc441263e8205581d6dd633a6c (md5)
output results for file `lab126.bin':
   14 sectors out
   [ok] 879083cc441263e8205581d6dd633a6c (md5)
dc3dd completed at 2016-01-31 00:33:09 -0600
Yielding:
Code:
$ hexedit lab126.bin
00000000   6C 61 62 31  32 36 20 73  75 63 6B 73  21 20 6C 61  62 31 32 36  20 73 75 63  6B 73 21 20  lab126 sucks! lab126 sucks!
0000001C   6C 61 62 31  32 36 20 73  75 63 6B 73  21 20 6C 61  62 31 32 36  20 73 75 63  6B 73 21 20  lab126 sucks! lab126 sucks!
00000038   6C 61 62 31  32 36 20 73  75 63 6B 73  21 20 6C 61  62 31 32 36  20 73 75 63  6B 73 21 20  lab126 sucks! lab126 sucks!
00000054   6C 61 62 31  32 36 20 73  75 63 6B 73  21 20 6C 61  62 31 32 36  20 73 75 63  6B 73 21 20  lab126 sucks! lab126 sucks!
00000070   6C 61 62 31  32 36 20 73  75 63 6B 73  21 20 6C 61  62 31 32 36  20 73 75 63  6B 73 21 20  lab126 sucks! lab126 sucks!
Looks useful, I may play with that instead of using dd, later.
The dc3dd utility can hash its input and read-back compare with the hash of the output but it can't do partial blocks (sectors in en_DC3DD).
The dd command I have figured out. Why push my luck?

Last edited by knc1; 01-31-2016 at 02:04 AM.
knc1 is offline   Reply With Quote
Old 02-01-2016, 09:38 AM   #26
knc1
Going Viral
knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.
 
knc1's Avatar
 
Posts: 17,212
Karma: 18210809
Join Date: Feb 2012
Location: Central Texas
Device: No K1, PW2, KV, KOA
Looking good so far (output when run by Q.A. script):
Spoiler:

Code:
both full blocks, test: c-w/r/c
Termination on block boundary.
Blocks: 3, plus Bytes: 0, Total skips: 0, Total bytes: 12288

both full blocks, test: c
Termination on block boundary.
Blocks: 3, plus Bytes: 0, Total skips: 3, Total bytes: 12288

both partial blocks, test: c-w/r/c
Termination on a partial block boundary
Blocks: 3, plus Bytes: 2048, Total skips: 0, Total bytes: 14336

both partial blocks, test: c
Termination with a partial block
Blocks: 3, plus Bytes: 2048, Total skips: 4, Total bytes: 14336

Following are Warnings because they might be intended usage
That is: Changing only an initial portion of the destination

both full blocks, *.src < *.dst, test: c-w/r/c
Warning: Early EOF on test03.src
Blocks: 2, plus Bytes: 0, Total skips: 0, Total bytes: 8192

both full blocks, *.src < *.dst, test: c
Warning: Early EOF on test03.src
Blocks: 2, plus Bytes: 0, Total skips: 2, Total bytes: 8192

both partial blocks, *.src < *.dst, test: c-w/r/c
Warning: Early EOF on test04.src, common bytes: w/r/c
Blocks: 2, plus Bytes: 2048, Total skips: 0, Total bytes: 10240

both partial blocks, *.src < *.dst, test: c
Warning: Early EOF on test04.src, common bytes: match
Blocks: 2, plus Bytes: 2048, Total skips: 3, Total bytes: 10240

Still need to test error paths, but should be done today.

Edit: Of course, it was a bigger job than just that due to think'os along the way.
**should**

Last edited by knc1; 02-02-2016 at 05:48 PM.
knc1 is offline   Reply With Quote
Old 02-02-2016, 06:01 PM   #27
knc1
Going Viral
knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.
 
knc1's Avatar
 
Posts: 17,212
Karma: 18210809
Join Date: Feb 2012
Location: Central Texas
Device: No K1, PW2, KV, KOA
CopyCmp (compare-write/readback/compare) script

At last!
This turkey finally pasted its Q.A. testing:
Spoiler:

Code:
=== Normal output when both sizes equal
=== ending on full block or partial block

=== Src and Dst both 3 full blocks, test: c-w/r/c
Termination on block boundary. (0+0.0+0)
Blocks: 3, plus Bytes: 0, Total skips: 0, Total bytes: 12288
test01.dst: Before: 12288; After: 12288

=== Src and Dst both 3 full blocks, test: c
Termination on block boundary. (0+0.0+0)
Blocks: 3, plus Bytes: 0, Total skips: 3, Total bytes: 12288
test01.dst: Before: 12288; After: 12288

=== Src and Dst both 3-1/2 blocks, test: c-w/r/c
Termination on partial block boundary; w/r/c (0+1.0+1)
Blocks: 3, plus Bytes: 2048, Total skips: 0, Total bytes: 14336
test02.dst: Before: 14336; After: 14336

=== Src and Dst both 3-1/2 blocks, test: c
Termination on partial block boundary; match (0+1.0+1)
Blocks: 3, plus Bytes: 2048, Total skips: 4, Total bytes: 14336
test02.dst: Before: 14336; After: 14336

=== Following are "Notice"s because they might be intended usage.
=== That is: Changing only an initial portion of the destination.

=== Src 2 full blocks, Dst 3 full blocks, test: c-w/r/c
Notice: EOF on test03.src, full block on test03.dst (0+0.1+0)
Blocks: 2, plus Bytes: 0, Total skips: 0, Total bytes: 8192
test03.dst: Before: 12288; After: 12288

=== Src 2 full blocks, Dst 3 full blocks, test: c
Notice: EOF on test03.src, full block on test03.dst (0+0.1+0)
Blocks: 2, plus Bytes: 0, Total skips: 2, Total bytes: 8192
test03.dst: Before: 12288; After: 12288

=== Src 2-1/2 blocks, Dst 3-1/2 blocks, test: c-w/r/c
Notice: Partial block on test04.src, common bytes: 2048; w/r/c (0+1.1+0)
Blocks: 2, plus Bytes: 2048, Total skips: 0, Total bytes: 10240
test04.dst: Before: 14336; After: 14336

=== Src 2-1/2 blocks, Dst 3-1/2 blocks, test: c
Notice: Partial block on test04.src, common bytes: 2048; match (0+1.1+0)
Blocks: 2, plus Bytes: 2048, Total skips: 3, Total bytes: 10240
test04.dst: Before: 14336; After: 14336

=== Src 3-1/4 blocks, Dst 3-1/2 blocks, test: c-w/r/c
Termination on partial block boundary; w/r/c (0+1.0+1)
Blocks: 3, plus Bytes: 1024, Total skips: 0, Total bytes: 13312
test05.dst: Before: 14336; After: 14336

=== Src 3-1/4 blocks, Dst 3-1/2 blocks, test: c
Termination on partial block boundary; match (0+1.0+1)
Blocks: 3, plus Bytes: 1024, Total skips: 4, Total bytes: 13312
test05.dst: Before: 14336; After: 14336

=== Src 3 blocks, Dst 2-3/4 blocks, test: c-w/r/c
Notice: Partial block on test06.dst, common bytes: 3072; w/r/c (1+0.0+1)
Blocks: 2, plus Bytes: 3072, Total skips: 0, Total bytes: 11264
test06.dst: Before: 11264; After: 11264

=== Src 3 blocks, Dst 2-3/4 blocks, test: c
Notice: Partial block on test06.dst, common bytes: 3072; match (1+0.0+1)
Blocks: 2, plus Bytes: 3072, Total skips: 3, Total bytes: 11264
test06.dst: Before: 11264; After: 11264

=== Typically error conditions (source and destination size mis-match).
=== But since EOF is only recognized when encountered, the destination
=== blocks prior to the current block have been modified.

=== Src 3 blocks, Dst 2 blocks, test: c-w/r/c
Notice: full block on source, EOF on test07.dst (1+0.0+0)
Blocks: 2, plus Bytes: 0, Total skips: 0, Total bytes: 8192
test07.dst: Before: 8192; After: 8192

=== Src 3 blocks, Dst 2 blocks, test: c
Notice: full block on source, EOF on test07.dst (1+0.0+0)
Blocks: 2, plus Bytes: 0, Total skips: 2, Total bytes: 8192
test07.dst: Before: 8192; After: 8192

=== Src 3 blocks, Dst 3-1/4 blocks, test: c-w/r/c
Notice: EOF on test08.src, partial block on test08.dst (0+0.0+1)
Blocks: 3, plus Bytes: 0, Total skips: 0, Total bytes: 12288
test08.dst: Before: 13312; After: 13312

=== Src 3 blocks, Dst 3-1/4 blocks, test: c
Notice: EOF on test08.src, partial block on test08.dst (0+0.0+1)
Blocks: 3, plus Bytes: 0, Total skips: 3, Total bytes: 12288
test08.dst: Before: 13312; After: 13312

=== Src 2 blocks, Dst 3 blocks, test: c-w/r/c
Notice: EOF on test09.src, full block on test09.dst (0+0.1+0)
Blocks: 2, plus Bytes: 0, Total skips: 0, Total bytes: 8192
test09.dst: Before: 12288; After: 12288

=== Src 2 blocks, Dst 3 blocks, test: c
Notice: EOF on test09.src, full block on test09.dst (0+0.1+0)
Blocks: 2, plus Bytes: 0, Total skips: 2, Total bytes: 8192
test09.dst: Before: 12288; After: 12288

=== Src 3-1/2 blocks, Dst 3 blocks, test: c-w/r/c
Notice: partial block on source, EOF on test010.dst (0+1.0+0)
Blocks: 3, plus Bytes: 0, Total skips: 0, Total bytes: 12288
test010.dst: Before: 12288; After: 12288

=== Src 3-1/2 blocks, Dst 3 blocks, test: c
Notice: partial block on source, EOF on test010.dst (0+1.0+0)
Blocks: 3, plus Bytes: 0, Total skips: 3, Total bytes: 12288
test010.dst: Before: 12288; After: 12288

Command shell scripting is slow, but programming flash erase blocks is even slower -
So the script first compares each source and destination block pair (or fractional common parts) and just skips to the next block if the same.

The attached tar-ball has it all -
dd and dc3dd for both x86 and armel
the copycmp.sh script
and the tests directory with reference input files (made with dc3dd).

Tomorrow - write up directions on how to use this invention.
(The kindle's updater might get stuck in a loop on errors, but this script will not - and will tell us more than we probably wanted to know.)

Note:
Does copycmp file-to-device, device-to-file, file-to-file, and I suppose device-to-device (that last use is untested).
Attached Files
File Type: gz copycmp.tar.gz (264.1 KB, 285 views)

Last edited by knc1; 02-03-2016 at 02:03 PM.
knc1 is offline   Reply With Quote
Old 02-03-2016, 02:45 PM   #28
Whitefox
Member
Whitefox began at the beginning.
 
Posts: 13
Karma: 10
Join Date: May 2014
Device: kindle 4, kindle paperwhite
Awesome, I don't understand half of it, but awesome
Whitefox is offline   Reply With Quote
Old 02-03-2016, 02:58 PM   #29
knc1
Going Viral
knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.
 
knc1's Avatar
 
Posts: 17,212
Karma: 18210809
Join Date: Feb 2012
Location: Central Texas
Device: No K1, PW2, KV, KOA
Quote:
Originally Posted by Whitefox View Post
Awesome, I don't understand half of it, but awesome
Still working on the directions and an example or two.

I just used it to copy device-to-file ('diags' partition to backup file).
Took about 8 minutes, 3-1/2 minutes running the script, the rest waiting for the eMMC writes to complete. Lots of 'wait' time, it was only using about 7% cpu.

Now I'll repeat the command, see how long it takes to only compare (since all the blocks **should** be a correct copy).

Then, try the same thing with dc3dd, set to hash-check each block written.

But I am pretty sure already that this script is working as intended.

Last edited by knc1; 02-03-2016 at 03:00 PM.
knc1 is offline   Reply With Quote
Old 02-03-2016, 04:23 PM   #30
knc1
Going Viral
knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.
 
knc1's Avatar
 
Posts: 17,212
Karma: 18210809
Join Date: Feb 2012
Location: Central Texas
Device: No K1, PW2, KV, KOA
This is a worked example of using the copycmp.sh script and the dc3dd utility attached to post:
https://www.mobileread.com/forums/sho...6&postcount=27

In this example, I am doing device-to-file copy/compare and in the problem situation presented in this thread, the use will be file-to-device.

This example is run on a PW-1 with USBnetworking access (ssh terminal) to the command line as user: 'root'.
My ssh setup includes a hosts file entry for: 'ken1' and an ssh config file address short-cut: 'kpw' using pubkey authentication (no manual login).
(All of which I described in one of my threads on using kindle/ssh.)

Copy the tar-ball from the PC to the root of the Kindle's USB storage:
Code:
$ scp copycmp.tar.gz kpw:/mnt/us
copycmp.tar.gz                                100%  264KB 264.1KB/s   00:00
Open an ssh terminal to the 'remote' (about 2 feet) Kindle PW-1:
Code:
$ ssh kpw
Welcome to Kindle!

#################################################
#  N O T I C E  *  N O T I C E  *  N O T I C E  # 
#################################################
Rootfs is mounted read-only. Invoke mntroot rw to
switch back to a writable rootfs.
#################################################
[root@kindle root]#
Unpack the release archive in the USB storage root:
Code:
[root@kindle root]# cd /mnt/us
[root@kindle us]# tar -xzf copycmp.tar.gz
The script requires the full featured dd utility from (Debian) CoreUtils package.
Check that it is going to run on this particular Kindle:
Code:
[root@kindle us]# cd copycmp
[root@kindle copycmp]# arm/dd --help
Usage: arm/dd [OPERAND]...
  or:  arm/dd OPTION
Copy a file, converting and formatting according to the operands.
- - - - -
Looks like that will work.

Return to root's home directory.
Check its real full pathname:
Code:
[root@kindle copycmp]# cd
[root@kindle root]# pwd
/tmp/root

[root@kindle root]# df /tmp
Filesystem           1K-blocks      Used Available Use% Mounted on
tmpfs                    32768       268     32500   1% /var
The 'tmpfs' filesystem type is implemented in ram and everything in it will 'go away' when the Kindle is rebooted.
The script uses space in the /tmp tree for its block sized working files.
Shown above is that tree portion has about 32Mbytes free - more than enough.

We will run the script (and it will run dd) from the current directory in ram.
Just copy the two things we need to our current location (the period at the end of the line is part of the command).
Code:
[root@kindle root]# cp /mnt/us/copycmp/copycmp.sh /mnt/us/copycmp/arm/dd .
[root@kindle root]# ls
copycmp.sh  dd
For this example, the contents of the 'diags' partition will be copied to a file of all zeros.
The file needs to be the same size as the 'diags' partition.
Linux makes that information available in the /proc tree and a bit of command line math converts from 1K blocks to bytes.
Code:
[root@kindle root]# cat /proc/partitions
major minor  #blocks  name
   7        0    1398776 loop0
   7        2         72 loop2
   7        3        564 loop3
   7        4      67788 loop4
   7        5         32 loop5
   7        6       1884 loop6
   7        7       5352 loop7
 179        0    1921024 mmcblk0
 179        1     358400 mmcblk0p1
 179        2      65536 mmcblk0p2
 179        3      65536 mmcblk0p3
 179        4    1398784 mmcblk0p4
We are only interested in the byte-size of p2, so:
Code:
[root@kindle root]# set $(cat /proc/partitions | grep mmcblk0p2) ; echo $(($3*1024))
67108864
Make sure we have enough free space on USB storage for our output files:
Code:
[root@kindle root]# df -h /mnt/us
Filesystem                Size      Used Available Use% Mounted on
fsp                       1.3G    166.7M      1.2G  12% /mnt/us
Since the copycmp script will neither truncate not extend the destination, we need to have a pre-existing destination file for the partition's contents:
Code:
[root@kindle root]# dd if=/dev/zero of=/mnt/us/blk0p2.img bs=67108864 count=1
1+0 records in
1+0 records out
67108864 bytes (64.0MB) copied, 96.629396 seconds, 678.2KB/s
Time to run the copycmp script:
Code:
[root@kindle root]# time ./copycmp.sh /dev/mmcblk0p2 /mnt/us/blk0p2.img
Termination on block boundary. (0+0.0+0)
Blocks: 16384, plus Bytes: 0, Total skips: 10660, Total bytes: 67108864

real	8m 51.76s
user	3m 27.99s
sys	    4m 7.89s
Normally, you would not use the 'time' function, you would just run the script.
Note that we also learn that the partition has 10,660; 4096 byte blocks of zeros in it.

While the script was executing, I logged into the Kindle with another ssh terminal session to check on cpu usage:
Code:
[root@kindle us]# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
- - - - -
root     17943  6.9  0.2   2332   644 pts/0    RN+  00:46   0:22 /bin/sh ./copycmp.sh /dev/mmcblk0p2 /mnt/us/blk0p2.img
- - - - -
The eMMC flash is so slow to program that we are using less than 10% of the cpu time to run the script.

The copycmp script has already done a write-readback-compare of each block it wrote.
But just for test purposes, we will run the command again.
Since the file should be a bit-wise exact copy of the partition, the script will just read and compare.
A handy feature that can save a lot of writing to the flash when just updating an image.
Again, you would probably not run the script under the timing utility:
Code:
[root@kindle root]# time ./copycmp.sh /dev/mmcblk0p2 /mnt/us/blk0p2.img
Termination on block boundary. (0+0.0+0)
Blocks: 16384, plus Bytes: 0, Total skips: 16384, Total bytes: 67108864

real	6m 36.85s
user	2m 52.15s
sys	2m 45.36s
That is the expected output.

The netBSD dc3dd utility can also do write with read-back verification (by any of several hash functions).
For comparison with the above, get a copy of it from the (opened) release tar-ball and give it a try (with a different output filename).
Code:
[root@kindle root]# cp -a /mnt/us/copycmp/arm/dc3dd .
[root@kindle root]# time ./dc3dd if=/dev/mmcblk0p2 hof=/mnt/us/blk0p2-dc3dd.img hash=sha1 ssz=4096 nwspc=on verb=on

dc3dd 7.1.614 started at 2012-10-04 01:24:05 +0717
compiled options:
command line: ./dc3dd if=/dev/mmcblk0p2 hof=/mnt/us/blk0p2-dc3dd.img hash=sha1 ssz=4096 nwspc=on verb=on
device size: 131072 sectors (probed)
sector size: 4096 bytes (set)
67108864 bytes (64 M) copied (100%), 35.4048 s, 1.8 M/s                       
output hashing (100%)                                                         
input results for device `/dev/mmcblk0p2':
   16384 sectors in
   0 bad sectors replaced by zeros
   f40172decf43e08bf93724c55eb4a9c2a24a8825 (sha1)
output results for file `/mnt/us/blk0p2-dc3dd.img':
   16384 sectors out
   [ok] f40172decf43e08bf93724c55eb4a9c2a24a8825 (sha1)
dc3dd completed at 2012-10-04 01:24:40 +0717

real	0m 35.42s
user	0m 5.90s
sys	    0m 2.31s
Hmm...
About 9 minutes for the script and about 35 seconds for the compiled dc3dd utility.
With the dc3dd utility we only get a pass/fail indication on the over-all file (or partition - it can write to devices also).
With the much slower script we (can) get block-by-block status messages on failure(s).

So it would seem that a reasonalbe approach would be to run the needed copy with dc3dd first.
If that passes its hash-sum testing, your done.
If not, then run the same update with the copycmp script and find out more details of where/what is failing.
knc1 is offline   Reply With Quote
Reply


Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Kindle Paperwhite 1st Gen doesn't sleep? ForsCream Amazon Kindle 3 12-11-2013 09:49 AM
Kindle paperwhite doesn't recognize book tsumaster Amazon Kindle 2 07-22-2013 09:59 AM
Troubleshooting Kindle Paperwhite search doesn't find anything PeterBr Amazon Kindle 5 06-05-2013 04:30 PM
Troubleshooting Paperwhite doesn't display publisher font but Kindle 3 does EbokJunkie Amazon Kindle 6 04-16-2013 02:47 PM
Kindle Keyboard upgrade - Paperwhite laknights General Discussions 15 10-14-2012 05:45 AM


All times are GMT -4. The time now is 11:41 AM.


MobileRead.com is a privately owned, operated and funded community.