Kindle Touch serial port recovery.

[geekmaster]

NEWS FLASH: A Kindle Touch can now be debricked in cases like mine by using only the USB Drive, even if it is only visible to the host PC for a few seconds during the reboot cycle like mine was. It has been reported to work if your Kindle is stuck at the startup screen with no progress bar, and it also works if you kindle is stuck at the dreaded "Repair Needed" screen (like mine was). No serial port connection is required, so no need to open up the Kindle back cover.

UPDATE: Success! My kindle touch works again. I was able to repair my "bricked" (unbootable) kindle touch without using a backup copy of the root partition (mmcblk0p1). Read the rest of this thread for history and details of this technical adventure. Let's begin this story at the beginning:

Yes, I need it. I did something stupid and now my Touch will not boot. It sticks at the "Repair Needed" screen. EDIT: There are reports that at least three developers have bricked their kindle touch after making small changes to startup scripts (like I did).

I do have a backup copy of my root partition, which I was sure to do before making any of my own changes to root.

Anyway, while it is booting, the USB drive is visible for about 10 seconds. By testing in a loop for a file to show up in the mount for the USB drive, then creating an ENABLE_DIAGS file, I was able to boot to the diagnostics partition. Not much useful there, unlike the Kindle 4. Time to crack it open and make a serial cable.

After many hours of messing around with little success, and even after building level converters for 1.8v TTL TxD to 3.3v TTL RxD, and 3.3v TTL TxD to 1.8v TTL RxD, it still did not work. Time to get out the oscilloscope.

With the scope I could see that when attached to the USB serial adapter, the kindle TxD output logic zero was way too high. After much testing, I discovered that the USB serial adapter had an LED attached directly to the RxD pin with no buffer or isolation. The 1.8v TTL TxD from the kindle could not sink that much current. It could only get what should be near 0v down to about 2.5v.

I then unsoldered the surface mount LED on the USB adapter. Then the adapter needed an external pullup resistor, which was 2.2K ohms in my level shifter circuit. Even without the LED, the kindle could only get the output down to about 1.8v. I increased the value of the pullup resistor to 22K ohms, and now the kindle could get the logic 0 down to about 0.7v, and (finally) there was serial data on my terminal screen. Here it is:

Spoiler:

PHP Code:

U-Boot 2009.08-lab126 (Nov 03 2011 - 11:56:43)

CPU:   Freescale i.MX50 family 1.1V at 800 MHz
mx50 pll1: 800MHz
mx50 pll2: 400MHz
mx50 pll3: 216MHz
ipg clock     : 50000000Hz
ipg per clock : 50000000Hz
uart clock    : 24000000Hz
ahb clock     : 100000000Hz
axi_a clock   : 400000000Hz
axi_b clock   : 200000000Hz
weim_clock    : 100000000Hz
ddr clock     : 800000000Hz
esdhc1 clock  : 80000000Hz
esdhc2 clock  : 80000000Hz
esdhc3 clock  : 80000000Hz
esdhc4 clock  : 80000000Hz
MMC:  FSL_ESDHC: 0, FSL_ESDHC: 1
Board: Whitney WFO
Boot Reason: [POR]
Boot Device: MMC
Board Id: XXXXXXXXXXXXXX
S/N: B011XXXXXXXXXXXX
DRAM:  256 MB
Using default environment

In:    serial
Out:   logbuff
Err:   logbuff
Quick Memory Test 0x70000000, 0xfffe000
POST done in 13 ms
Hit any key to stop autoboot:  0
## Booting kernel from Legacy Image at 70800000 ...
   Image Name:   Linux-2.6.31-rt11-lab126
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    5090112 Bytes =  4.9 MB
   Load Address: 70008000
   Entry Point:  70008000
   Verifying Checksum ... OK
   Loading Kernel Image ... OK
OK
Starting kernel ...
mxc_spi mxc_spi.0: chipselect 0 already in use
mxc_spi mxc_spi.0: chipselect 0 already in use
mxc_spi mxc_spi.2: chipselect 0 already in use
2.6.31-rt11-lab126 #1 Thu Nov 3 12:14:23 PDT 2011 armv7l
INFO:Loaded module /lib/modules/eink_fb_waveform.ko  (35728 bytes)
mxc_epdc_fb mxc_epdc_fb: not use_builtin_cmap !!
INFO:Loaded module /lib/modules/mxc_epdc_fb.ko default_panel_hw_init=1 default_update_mode=1 (45148 bytes)
INFO:eink initialized... (486400 bytes)
INFO:Using default keypad setting. (not passing "kb_rev" to module)
INFO:Loaded module /lib/modules/whitney_button.ko  (9268 bytes)
Press [ENTER] for recovery menu...       0 /INFO:*** Partition table verified for /dev/mmcblk0 ***
INFO:Checking for updates... (auto-pilot mode)
/dev/mmcblk0p4:
CHS=4/16/105536 size=3458203648 bytes
flag type      first       last  lba first   lba size
Partition p1:
0x00 0x0b         16 <large>            16    6754288
    CHS: 0/1/1 - <large>
Partition p2:
Partition p3:
Partition p4:
INFO:Setup loop device /dev/loop0 for /dev/mmcblk0p4 + 8192
INFO:No update*.bin found; no update needed.
INFO:no updates found.
BOOTING DEFAULT.
  argc == 10
  argv[0]: "kinit"
  argv[1]: "consoleblank=0"
  argv[2]: "rootwait"
  argv[3]: "ro"
  argv[4]: "ip=off"
  argv[5]: "root=/dev/mmcblk0p1"
  argv[6]: "quiet"
  argv[7]: "eink=fslepdc"
  argv[8]: "video=mxcepdcfb:E60,bpp=8"
  argv[9]: "console=ttymxc0,115200"
  argc == 4
  argv[0]: "IP-Config"
  argv[1]: "-i"
  argv[2]: "Linux kinit"
  argv[3]: "ip=off"
IP-Config: no devices to configure
kinit: do_mounts
kinit: name_to_dev_t(/dev/mmcblk0p1) = dev(179,1)
kinit: root_dev = dev(179,1)
kinit: /dev/root appears to be a ext3 filesystem
kinit: trying to mount /dev/root on /root with type ext3
kinit: Mounted root (ext3 filesystem) readonly.
info firsttime:mount_rw:time=4600:Mounting root RW for first boot
info firsttime:mount_ro:time=4960:Mounting root RO
info modules:modprobe:loading module g_file_storage:
init.exe: recevent pre-start process (529) terminated with status 1
info system:start:time=5160:
info hardware:no_wan::WiFi only device. No WAN found
info modules:modprobe:loading module fuse:
info system:config:platform=yoshi,board=whitney,rev=20,proto=N,wifi=Y,wan=N:
info modules:modprobe:loading module ppp_async:
info modules:modprobe:loading module whitney_button:
info system:done:time=6630:
info zforce:start:version=2.0b0r12:
info userstore:run:time=6980,action=start:
info userstore:found_shutdown_flag::Found SHUTDOWN flag file
info display:started:time=7150:
info battery:charged:cap=100,mV=4126,mAH=1474:



Welcome to Kindle!

kindle login: info hardware:no_wan::WiFi only device. No WAN found
info X:load:time=10700:
info X:xorg.conf:Generating xorg.conf:
MAC from kernel xx:xx:xx:xx:XX:XX
info X:makexconfig:Whitney Input device detection status:
info X:makexconfig:Keypad: OK:
info X:makexconfig:Touch device: OK:
info X:makexconfig:Accelerometer device: N/A !!:
info X:xorg.conf:xorg.conf generated:
info X:starting:time=12390:
wmi_control_rx() : Unknown id 0x101e
info X:started:time=17440:
info blanket_f:module already loaded:splash:
info X:keyboard:Disabling keyboard repeat on Home Key:
info locale:read:lang=en_US.utf8,lc_all=en_US.utf8:Retrieved Language
crit lab126_gui:reboots=2,max=2:shutting down and showing customer service page:
info shutdown:begin:level=0,halt=1,special=cust_service:
info upstart_f:job_already_stopped:job=passwdlg:
info upstart_f:jobs_stopped:elapsed_ms=0,timeout_ms=20000:
info blanket_f:loading blanket module:shutdown:
info shutdown_showimage:screen:splashing screen:cust_service
info shutdown_showimage:screen:waiting for cust_service screen to come up:
info upstart_f:stopping_job:job=lab126:
info upstart_f:stopping_job:job=x:
Reset Cause readback: 0x2
info upstart_f:jobs_stopped:elapsed_ms=4360,timeout_ms=45000:
info shutdown:cust_service:starting:
info ss_cust_service:nothing to do for this shutdown mode:"cust_service":
info ss_cust_service:completed successfully::
info shutdown:cust_service:done:
info upstart_f:stopping_job:job=filesystems:
info filesystem:unmount_userstore:Unmounting UserStore:
info userstore:run:time=28720,action=stop:
info userstore:found_bootup_flag::Found BOOTUP flag file
info filesystem:rotate_log:Rotating Logs:
boot: W def:hlt:halt=Device,version=000000:
info filesystem:unmount_varLocal:Unmounting VarLocal:
info upstart_f:jobs_stopped:elapsed_ms=3940,timeout_ms=30000:
info shutdown:mounting root filesystem read-only::
info shutdown:syncing disks::
info shutdown:end:level=0,halt=1:
System halted.
Halting ... 


Next step: use this working serial port to do the recovery, but I need some sleep before I do something even more stupid.

Stupid is as stupid does, and all that...

By the way, being in a hurry and not having an official connector, I first tried cutting down a microSDHC adapter that had the same pin spacing. No obvious way to attach it though. Next I cut down a telephone extension cord plug (USA style pinout). After test fitting, I could see that it would snap in place and stay, if only I had cut off just a little less plastic in a couple of carefully controlled locations.

I ended up using some old IDE ribbon cable (a strip of 3 wires about 6 inches long). I stripped and solder-tinned both ends, then soldered one end to the pins in the kindle socket. The other end went first to my adaptor, then later to my level shifter circuit. The circuit uses open space (rat's nest, or point-to-point) wiring method, which works fine if the component leads are cut short enough and spaced safely apart. That is the way TVs and radios were wired in the old vacuum tube days, with the assistance of terminal strips where needed.

The 3-wire ribbon cable is held in place with some tape. I think I will leave it attached and tucked safely inside when I replace my back cover.

I will post my progress when I resume this recovery after some sleep.

[geekmaster]

And here is the script that was damaged, and has now been restored:

[root@kindle root]# cat /etc/upstart/framework

Spoiler:

PHP Code:

#! /bin/sh

# check for framework disabled

source /etc/upstart/functions
JHOME=/usr/java
JLIB=/usr/java/lib
EHOME=/opt/amazon/ebook
LIBHOME=/opt/amazon/ebook/lib

# construct clazzpath from lib, booklet, testlibdir
LPATH=$LIBHOME/concierge-1.0.0.jar:$LIBHOME/icu4j.jar:$LIBHOME/utilities.jar:$LIBHOME/json_simple-1.1.jar

JAR_LIST=$(find $EHOME/lib $EHOME/booklet /usr/local/ebook/lib -name \*.jar 2>/dev/null)
CPATH=$(echo $JAR_LIST | sed -e 's/ \+/:/g')

# heap changes

if [ "$(awk '/^MemTotal:/ {print $2}' </proc/meminfo)" -gt 255000 ]; then
  HEAP="-Xmx20m -Xgc:youngGen=3m"
else
  HEAP="-Xmx16m -Xgc:youngGen=1m"
fi

WAN_EXISTS=true
WIFI_EXISTS=true
f_wan || WAN_EXISTS=false
f_wifi || WIFI_EXISTS=false

AOPTS="-Ddebug=1 \
-Dxargs=$EHOME/bin/init.xargs \
-Dcheck_comm_stack=true \
-Dawt.eventThreadPriority=7 \
-Dgci.eventThreadPriority=7 \
-Dfeature.wifiPresent=$WIFI_EXISTS \
-Dfeature.wanPresent=$WAN_EXISTS \
-Dsun.net.client.defaultReadTimeout=120000 \
-Dhttp.keepalive.timeout=60000 \
-Dhttp.maxConnections=16 \
-Dallow_demo=false \
-Djava.awt.SyncOnPaint=false \
-Dextkeyboard=false \
-Dconfig=$EHOME/config/framework.$(f_platform).conf \
-DPLATFORM_CLASS_FILE=$EHOME/config/platform.conf \
-DENABLE_SEARCH_INDEXING_THREAD=true \
-Dpolicy.home=$EHOME/security \
-Djava.security.manager \
-Djava.awt.DebugPaint=false \
-DUSE_KWGRAPHICS=false \
-Dx11.sync=true \
-Dx11.dpi=72 \
-Dbuttons.noxor=true \
-Dcc_transport_jni=true \
-Dawt.clickRadius=25 \
-Dawt.multiClickRadius=100 \
-DappContext.disposeClearsSoftReferences=true \
-Djavax.microedition.plaf.platformfactory=com.amazon.agui.swing.plaf.kindle.KindlePlatformFactory"

# enable more cvm crash logging/reporting
export CVM_LOG_LEVEL=4

APP=ch.ethz.iks.concierge.framework.Framework

# XXX - are any of these needed anymore?

# use either jvmpi or jvmti, not both
#IHPROF="-Xrunhprof:heap=all,thread=y,file=/mnt/base-us/framework.$$.hprof.txt"
#IHPROF="-Xbootclasspath/a:$JLIB/java_crw_demo.jar \
#                -agentlib:jvmtihprof=cpu=times,heap=sites,thread=y,file=/mnt/us/framework.$$.hprof.txt"

#DEBUG="-agentlib:jdwp=transport=dt_socket,server=y,address=8000,suspend=y -Xdebug -DMinimumKindletTimeoutMillis=60000"

if [ -f /mnt/us/PROFILE_CVM_NB ]
then
    # NetBeans profiling
    DEBUG="-agentlib:profilerinterface=lib/profiler/lib,5140"
    LPATH=$LPATH:$JLIB/profiler/lib/jfluid-server.jar:$JLIB/profiler/lib/jfluid-server-cvm.jar

elif [ -f /mnt/us/DEBUG_CVM_JDWP ] || [ -f /mnt/us/DEBUG_CVM_JDWP_SUSPEND ]
then
    SUSPEND=n
    if [ -f /mnt/us/DEBUG_CVM_JDWP_SUSPEND ]
    then
        SUSPEND=y
    fi
     # N.B. the line below does not appear in /proc/$(pidof cvm)/cmdline, even though debugging is enabled
     DEBUG="-agentlib:jdwp=transport=dt_socket,server=y,address=8000,suspend=$SUSPEND -Xdebug -DMinimumKindletTimeoutMillis=60000"
fi

JOPTS="-Dsun.awt.exception.handler=com.amazon.agui.swing.event.EventQueueExceptionHandler \
-Xbootclasspath/a:$JLIB/localedata.jar:$JLIB/charsets.jar:$LPATH \
-Dsun.boot.library.path=$JLIB \
-cp $CPATH"

if [ -f /mnt/us/DEBUG_CVM_CRASH ]; then
  CRASH=/mnt/us/DEBUG_CVM_CRASH
  export CVM_CRASH_DEBUG=1
  f_log D framework crash_debug "" "To disable, remove ${CRASH}"
fi

OPTS="$HEAP $JOPTS $IHPROF $DEBUG $AOPTS"
rm -rf /var/local/java/default

# Capture stdout/stderr and log it.
# Unfortunately, if framework is killed, it might not kill cvm.
# If this happens, we rely on framework.conf to do a killall cvm.
# Nasty? Yes, but until cvm can directly log its stderr/stdout to
# syslog this is the best we can do.
$JHOME/bin/cvm $OPTS $APP 2>&1 | logger -p local2.debug
exit 1 


It still does not boot though...

I wonder if it has a permissions problem...

EDIT: I just rebooted and checked the file:
ls -al /etc/upstart/framework
-rwxr-xr-x 1 root root 3611 Nov 3 2011 /etc/upstart/framework

That looks okay to me. I am certain that it has been put back to the way it was before I changed anything on the root partition. That is the original file that had been renamed and replaced by an editted copy. It was restored by deleting the copy and renaming the original file back to its original name.

Does anybody have any ideas why it still won't boot?

[yifanlu]

When in doubt restore mmcblk0p1 from backup and nuke mmcblk0p3

[geekmaster]

I was able to boot to diagnostic mode by putting ENABLE_DIAGS on the USB Drive. Then in the "Exit, Reboot or Disable Diags" menu there is an option to exit to a Login Prompt. I selected that and now my serial port has a root shell booted from the diagnostic partition.

[root@kindle /]# mount

rootfs on / type rootfs (rw)
/dev/root on / type ext3 (ro,noatime,nodiratime,data=writeback)
proc on /proc type proc (rw,relatime)
sysfs on /sys type sysfs (rw,relatime)
tmpfs on /dev type tmpfs (rw,relatime,mode=755)
tmpfs on /dev/shm type tmpfs (rw,relatime)
devpts on /dev/pts type devpts (rw,relatime,gid=5,mode=620)
tmpfs on /var type tmpfs (rw,relatime,size=0k)
/dev/mmcblk0p3 on /var/local type ext3 (rw,relatime,errors=continue,data=writeback)
/dev/loop/0 on /mnt/base-us type vfat (rw,noexec,noatime,nodiratime,fmask=0022,dmask=0022,codepage=cp437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro)
fsp on /mnt/us type fuse.fsp (rw,nosuid,nodev,noatime,user_id=0,group_id=0)

I am glad that I refrained from making root partition changes before backing up mmcblk0p1, so I do not have to ask for a copy from another Touch owner. My recovery procedure looks promising at this point.

I have some Christmas shopping to do now, so I will continue this recovery later.

[geekmaster]

So far, I can run framebuffer-based apps booted from the diag partition. Way more available resources than in the "gui framework" partition. Still have not figured out how to reset the "unsuccessful sequential reboots" counter to below the maximum allowed, so I can boot past the "repair needed" screen on the main partition.

The simple brute-force "nuclear option" would be to reformat the /var partition, but I am seeking a "kindler gentler" way. I can always use the "nuke mmcblk0p3" option later if necessary.

[geekmaster]

It is all fixed now. I repaired my kindle touch using the serial port. I did NOT restore my backup copy of the root partition (mmcblk0p1). That means that if you "slightly" break your startup scripts, you can boot to diag mode and repair them. In my case, I made a backup copy of the script before I changed it, but without a backup you could still manually edit out your damaging changes.

A complication was that I had to reset a boot counter manually. The file /var/local/upstart/lab126_gui.restarts had the value "2" in it. I did "echo 0 > /var/local/upstart/lab126_gui.restarts" to reset the boot counter. [Thanks for the suggestion, nueva!]

I had also cleaned up my USB drive from an export in diags (a bit too thoroughly). I had deleted an xml file needed to exit diags mode. I was able to force normal boot mode with the idme command, and I got my desktop back. I tried diags again, but it still could not exit. I replaced the missing xml file from a backup copy to the diags usb export drive, then exited properly. All is well now, and I can exit diags correctly without forcing it with the idme command.

I will [maybe] do a video like seaniko7 did for the k3, where I intentionally damage my scripts so it gets stuck in the "Repair Needed" screen at bootup. Then I will repair it WITHOUT a root partition backup copy. (After the holiday rush). EDIT: Now that the cover is back on my Kindle Touch, going through all this again just for a video seems like it would consume a lot of time that I would rather spend developing new code, so I may postpone this until I *accidentally* brick my touch again.

I plan to do another thing that seaniko7 did for his own personal use -- add usbserial to my startup scripts, exept that my version will be conditional on a trigger file (ENABLE_USBSERIAL). That should allow doing all this repair stuff over usbserial (not usbnet) instead of using a serial cable.

If that works, a packaged USBSERIAL hack would be an excellent add-on for all kindles (conditional on the trigger file, like how NiLuJe's USBNET works), as a safety precaution for "easy" debricking without opening the kindle.

[ixtab]

Originally Posted by geekmaster

I plan to do another thing that seaniko did for his own personal use -- add usbserial to my startup scripts, exept that my version will be conditional on a trigger file (ENABLE_USBSERIAL). That should allow doing all this repair stuff over usbserial (not usbnet) instead of using a serial cable.

If that works, a packaged USBSERIAL hack would be an excellent add-on for all kindles (conditional on the trigger file, like how NiLuJe's USBNET works), as a safety precaution for "easy" debricking without opening the kindle.

Oh yeah, that is definitely a very good plan! While I'm being cautious with my edits, there is always some chance of bricking the device by accident. You managed to recover from there, but I would be totally screwed in that case

So +1 from my side, most definitely.

[geekmaster]

In retrospect, the "right" (but slow) way to go would have been to order the parts I need instead of just hacking something together with the parts I already had.

In particular, it would be a lot easier to just order a known-working 1.8v USB serial adapter instead of attaching resistors and diodes to a 3.3v USB adapter to shift the logic voltage levels. In addition to shifting logic levels, the USB adapter also had to be modified to allow the kindle TxD output to pull the adapter RxD input down to a logic zero (by removing an LED).

Also, it would be much more "professional" to use the right connector instead of soldering wires to the serial port pins inside the kindle.

But a quick hack often gets the job done much faster than waiting for the "right" parts to arrive (especially cheap parts from China).

The serial cable has now been unsoldered from my kindle touch, and the back cover is installed. Now I can hopefully leave the back cover on it for awhile.

[geekmaster]

Because the kindle touch GUI environment does not leave much RAM memory for native mode apps, especially when using a loop mounted file system, my current plans are to use only the USB Drive partition, and use only apps that will run from /mnt/us/bin and do not depend on symlinks (such as static-linked linux apps).

For development, I am using nano and tcc, which both run fine without needing any symlinks.

I *really* do not want to go through this again, even though I actually know how to fix it and it would be relatively fast and painless next time. I would rather spend my spare time making new stuff.

[ixtab]

Any news on that usbserial recovery mode idea?

I'm not savvy enough to implement it, but I'm sure that it would be terribly helpful for the folks out there trying to un-brick their devices... Like https://www.mobileread.com/forums/sho...9&postcount=19 for example. We're still poking around in thin air when we don't know what the device is doing

[MatzeMatz]

And I thought "just translating" stuff is an easy way to get firm with the device.
I also thought it's a brick safe method - but I was wrong

[geekmaster]

I think that the startup scripts check the md5sum on some things to make sure they did not get "corrupted". Any changes you make to protected stuff would need to have their md5sum recomputed and stored, or it the boot process may abort.

At least that is what it *looked* like it was doing when I looked in there...

A *lot* of developers are bricking their kindle touch lately. It is way too easy to do that...

[yifanlu]

People are bricking because upstart is very delicate. If one process fails, the rest won't even start (since most depend upon one another). There's no corruption check. People are just modifying files which causes some upstart script to fail.

[geekmaster]

I downloaded the upstart PDF manual. I am reading it now. I need to understand this before I touch any more scripts in the upstart directory...

< upstart

[MatzeMatz]

About the 1.8V USB <-> serial adapter:
Any suggestion about how I can detect if it's compatible (1.8V) adapter?
What about the "Digitus DA-70156"?
(e.g. from German Amzon: Digitus DA-70156)
Or any other suggestions?