Aura H2O tps65185_int_func under-voltage on DCDC1 detected !

[DomesticExtremis]

This is a continuation of the thread titled Kobo Aura H2O frozen and unresponsive e-reader, which needs a more technical investigation than seen in the main forum.

Just to summarise:

The problem emerged when deleting an article from Pocket and returning to main article list.
Screen refresh typically goes all white, all black then the new page data is displayed.
This time the screen froze as it was turning black and remains there with most of the new image in inverse video (white on black) and a few lines of the article still showing around one inch from the top edge, just below the text "ARTICLES FROM POCKET".

The is no visible evidence of damage to the screen or substrate and the frontlight still functions.

The device is unresponsive except for being able to power off (long press) and power back on.
When powered back on, the switch light comes on constantly for a few seconds, then flashes once a second or so for around ten seconds. The front light illuminates and then nothing.

The device is quite old, dating back to 2013/14, was obtained as a hand me down and has been used on a daily basis since birth. As far as I know it has never suffered any traumatic falls or drops, just the occasional firmware 'upgrade'.

After seeking advice on the MobileRead forum (https://www.mobileread.com/forums/sh...d.php?t=322950), it was suggested that it may be due to a failing or corrupt SD card. I requested a SD card image and awaited the arrival of some new 16GB microSD cards onto which to put it.

Accessing the SD card requires the opening of the device. Since it is ostensibly waterproof, this is no mean feat and destroy the water-tightness of the casing. Removing the front bezel is tricky and requires a lot of care not to damage either it or the components inside. The gloop use to seal and stick down the bezel was cleaned away using Zippo lighter fluid. In my case the frontilight lightguide (the dark, translucent bezel underneath the front bezel) remained attached to the screen, unlike in many of the teardowns where is becomes magically detached without effort or explanation (looking at you iFixit/Instructables).
Four screws at the corners of the main screen/board assembly can now be removed ant the whole thing tipped out of the rear casing to reveal the main board and gain access to the SD card.

While waiting for parts as noted above, the following were tried:

i) endless messing with the power button;
ii) attempts to do a factory reset by holding down the power button and tapping the touch screen at the bottom corners;
ii) disconnecting the battery and reconnecting it to see if it unlocked any locked up BIOS/firmware chip;
iv) reseating the flat flex cables for the screen and frontlight connector.

None of these were successful.

With a multimeter, I confirmed that the battery voltage was good (4.1V) and found several power rails were present - 1.1V, 3V and 12V (at the fronlight connector).
So the board seems to be alive at least and producing the correct voltages for the main chip/RAM, the other chips and the frontlight illumination.

So the new cards arrived and the SD card image dd'd on ti it as per the instructions on the MR forum, the data partition was resized to fill the available space and it was inserted into the device.
Sadly, there was no change - the problem was not the SD card.

[DomesticExtremis]

The next suggestion was a damaged screen.
However, detaching the board from the main screen and inspecting the substrate reveals no sign of damage. In addition, the device had not bee dropped and has no visible faults on the image. It failed while in use, so I think the screen may not be the problem.
New screens are available, but too costly to try for the moment without knowing for sure that a new unit is required.
However the now opened and non-watertight device has two sets of serial headers on the main board, so I obtained an FTDI USB to serial module and used that to try and see what is going on during the initial boot process...

[DomesticExtremis]

Having obtained an FT232RL module from Ebay (https://www.ebay.co.uk/itm/FT232RL-3...53.m2749.l2649) and carefully followed the instrucions in this thread (https://www.mobileread.com/forums/sh...d.php?t=243647) I was able to obtain a trace of the messages for boot up and power down from the e-reader:

U-Boot 2009.08-dirty-svn ( 7月 16 2014 - 17:36:26)


CPU:   Freescale i.MX50 family 1.1V at 800 MHz

mx50 pll1: 800MHz

mx50 pll2: 400MHz

mx50 pll3: 216MHz

ipg clock     : 66666666Hz

ipg per clock : 66666666Hz

uart clock    : 24000000Hz

ahb clock     : 133333333Hz

axi_a clock   : 400000000Hz

axi_b clock   : 200000000Hz

weim_clock    : 100000000Hz

ddr clock     : 266666666Hz

esdhc1 clock  : 80000000Hz

esdhc2 clock  : 80000000Hz

esdhc3 clock  : 80000000Hz

esdhc4 clock  : 80000000Hz

Board: MX50 RDP board

Boot Reason: [POR]

Boot Device: SD

I2C:   ready

DRAM:  512 MB

MMC:   FSL_ESDHC: 0, FSL_ESDHC: 1, FSL_ESDHC: 2

In:    serial

Out:   serial

Err:   serial

[_get_sd_number] g_sd_number:2


MMC read: dev # 2, block # 1023, count 1 partition # 0 ... 

1 blocks read: OK


MMC read: dev # 2, block # 1024, count 1 partition # 0 ... 

1 blocks read: OK

PCB ID is 41

ESDin=0,UPGKey=-1,PWRKey=0

ram p=70000000,size=536870912


MMC read: dev # 2, block # 14335, count 1 partition # 0 ... 

1 blocks read: OK


MMC read: dev # 2, block # 14336, count 13205 partition # 0 ... 

13205 blocks read: OK


MMC read: dev # 2, block # 18431, count 1 partition # 0 ... 

1 blocks read: OK

no "logo" bin header


MMC read: dev # 2, block # 45055, count 1 partition # 0 ... 

1 blocks read: OK

no "logo" bin header

Kernel RAM visiable size=505M->505M

init TPS65185 power ...

Relock PLL1 to 1GHz ...

mx50 pll1: 1000MHz

mx50 pll2: 400MHz

mx50 pll3: 216MHz

ipg clock     : 66666666Hz

ipg per clock : 66666666Hz

uart clock    : 24000000Hz

ahb clock     : 133333333Hz

axi_a clock   : 400000000Hz

axi_b clock   : 200000000Hz

weim_clock    : 100000000Hz

ddr clock     : 250000000Hz

esdhc1 clock  : 80000000Hz

esdhc2 clock  : 80000000Hz

esdhc3 clock  : 80000000Hz

esdhc4 clock  : 80000000Hz

Hit any key to stop autoboot:  0 


MMC read: dev # 2, block # 2047, count 1 partition # 0 ... 

1 blocks read: OK

no kernel image signature !


MMC read: dev # 2, block # 2048, count 8192 partition # 0 ... 

8192 blocks read: OK

## Booting kernel from Legacy Image at 70800000 ...

   Image Name:   r7407_#3027 Aug 13 17:39:39

   Created:      2014-08-13   9:39:41 UTC

   Image Type:   ARM Linux Kernel Image (uncompressed)

   Data Size:    1953688 Bytes =  1.9 MB

   Load Address: 70008000

   Entry Point:  70008000

   Loading Kernel Image ... OK

OK


Starting kernel ...


Uncompressing Linux... done, booting the kernel.
drivers/video/mxc/lk_tps65185.c(556):tps65185_int_func under-voltage on DCDC1 detected !

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=0!

1+0 records in
1+0 records out
512 bytes (512B) copied, 0.000217 seconds, 2.3MB/s
cannot open /dev/null
drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=1!

dosfsck 3.0.6, 04 Oct 2009, FAT32, LFN
There are differences between boot sector and its backup.
Differences: (offset:original/backup)
  489:8d/93, 490:ef/03, 491:02/81, 492:d2/03, 493:00/2a
  Not automatically fixing this.
drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=2!

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=3!

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=4!

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=5!

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=6!

/dev/mmcblk0p3: 207 files, 6669/3634233 clusters
drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=7!



(none) login: drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=8!

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=9!

[PROGRESS_BAR-3009] No progess ...

epdfbdc_set_width_height(1439): new DCSize(3342336)>original DCSize(3317760) !

sh: you need to specify whom to kill
killall: fickel: no process killed
killall: udhcpc: no process killed
killall: wpa_supplicant: no process killed
killall: ntpd: no process killed
drivers/video/mxc/lk_tps65185.c(556):tps65185_int_func under-voltage on DCDC1 detected !

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=0!

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=1!

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=2!

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=3!

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=4!

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=5!

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=6!

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=7!

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=8!

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=9!

drivers/video/mxc/lk_tps65185.c(556):tps65185_int_func under-voltage on DCDC1 detected !

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=0!

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=1!

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=2!

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=3!

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=4!

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=5!

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=6!

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=7!

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=8!

drivers/video/mxc/lk_tps65185.c(1352):wait power on timeout lpcnt=9!

Power down.

Which is interesting, particularly this message:

drivers/video/mxc/lk_tps65185.c(556):tps65185_int_func under-voltage on DCDC1 detected !

Note that I tried this with just battery power and with USB cable connected and got the same result, indicating that is its not a battery problem.

Clearly, the system is trying to power up the TPS65185, which is the diisplay driver, failing and then abandoning the boot process.

[DomesticExtremis]

So the task is to figure out what is causing this error and preventing the boting of my ereader.

It is plausible that a broken and shorted substrate could pull down a voltage somewhere and cause the error seen, so we cannot eliminate that fully yet. However, it is far more likely that either the chip itself is faulty or that some support component or power supply circuitry for it is faulty.

The chip is from Texas Instruments and is well documented (http://www.ti.com/product/TPS65185/technicaldocuments).
There is a data sheet (http://www.ti.com/lit/gpn/tps65185) and a guide on understanding undervoltage conditions (http://www.ti.com/lit/pdf/slva769), which might give me some clues as to how to proceed. In addition, they seem readily available on Ebay for just a few kopecs.

I should add that I am no electronics engineer and will take a while to understand these documents. Anybody who knows more than me or has any tips and tricks on how to proceed is welcome to chip in.

[pazos]

Originally Posted by DomesticExtremis

So the task is to figure out what is causing this error and preventing the boting of my ereader.

It is plausible that a broken and shorted substrate could pull down a voltage somewhere and cause the error seen, so we cannot eliminate that fully yet. However, it is far more likely that either the chip itself is faulty or that some support component or power supply circuitry for it is faulty

The PMIC seems fine (it is already initialized in u-boot) and all it does is fault handling using a https://en.wikipedia.org/wiki/Undervoltage-lockout threshold.

If you have the equipment you can measure PMIC Vin and see if it is above minimum (3.0v).

What I'm not able to figure out is the other error (this one on the EPDC framebuffer driver):

epdfbdc_set_width_height(1439): new DCSize(3342336)>original DCSize(3317760) !

I know it is tagged as an error in kernel code but I'm not sure if that is the cause, a consequence of another circuitry error or totally unrelated.

I'm wondering what happens if you boot your device without the ribbon cable connected to the screen (do it at your own risk, ofc).

[MGlitch]

It amazes me that you have all the equipment to do everything you've done in various posts on this single topic, and yet you do not have a camera to allow for a second set of eyes on the substrate.

[DomesticExtremis]

Originally Posted by pazos

The PMIC seems fine (it is already initialized in u-boot) and all it does is fault handling using a https://en.wikipedia.org/wiki/Undervoltage-lockout threshold.

If you have the equipment you can measure PMIC Vin and see if it is above minimum (3.0v).

What I'm not able to figure out is the other error (this one on the EPDC framebuffer driver):

epdfbdc_set_width_height(1439): new DCSize(3342336)>original DCSize(3317760) !

I know it is tagged as an error in kernel code but I'm not sure if that is the cause, a consequence of another circuitry error or totally unrelated.

I'm wondering what happens if you boot your device without the ribbon cable connected to the screen (do it at your own risk, ofc).

Yes I noticed that too, but I think it is a secondary problem at the moment. I will try your suggestion at some point.

[DomesticExtremis]

Originally Posted by MGlitch

It amazes me that you have all the equipment to do everything you've done in various posts on this single topic, and yet you do not have a camera to allow for a second set of eyes on the substrate.

LOL.
I do have a camera, quite a nice one of which I am very fond.
Sadly it is also on the healing bench awaiting a period in my life of sufficient length for me to disassemble it and try to fix the problem.
The lens extension is stuck...a common fault, apparently.
It's a Canon PowerShot G-11. I have some guidance from YouTube, but getting it apart is fiddly and I don't want to do half the job and then come back to it some weeks later having forgotten where all the bits go. Normally I would take pictures but I can't do that for without a camera
Life (and other stuff breaking) is getting in the way of saving the planet at the moment...

[DomesticExtremis]

Good news - the datasheet is fantastically detailed (and in many places utter gobbledegook to me )

DCDC1 is an internal switching boost converter which only affects 3 or 4 pins and relies on a relatively small amount of external components.
See the attached section of the block diagram (pictures,yay!).

Moreover, there is quite a clever power up/power down sequence. Correct functioning of DCDC1 is tested early on so if not regulating properly the chip will stop the power up sequence and shutdown nicely - thus protecting the other circuitry.

So either the internal MOSFET is burned, one of the support components for the boost converter has failed or there is insufficient voltage on the supply, I think .

[pazos]

Originally Posted by DomesticExtremis

Yes I noticed that too, but I think it is a secondary problem at the moment. I will try your suggestion at some point.

Ok, If you're serious with this you can try to update the e-ink screen from u-boot. I remember that u-boot have some nice commands to tinker with the epd. Sadly it has a watchdog that will trigger a reset if a certain flag isn't cleared in a few seconds, so you might need to modify the environment and hook there the commands used to refresh the screen.

I'm attaching my own notes here: https://www.mobileread.com/forums/sh...3&postcount=10. (and following posts). Do not attempt to repurpose the u-boot / kernel trees for your device as they come from a slightly different product (Aura HD)

My humble guess: your screen is borked. It could be a crack on the substrate or a single failure that make your screen eats more than 16V, triggering the UVLO

[DomesticExtremis]

Thanks for that, useful info.

I'm not sure we have UVLO - that would mean insufficient input voltage to the ehtire chip. This has to be measured good prior to attempting the start up of the DCDC1 module. Power up of DCDC1 is the bit that is failing.
That is not to say that the screen is not borked or cracked (I can't see any crack and it failed in mid refresh), but the boot messages suggest something to do with the chip (possibly mistakenly, it's a dumb computer after all). I'm 90% sure the screen is good at the moment.
The data sheet says this (and much more):

8.3 Feature Description
8.3.1 Wake-Up and Power-Up Sequencing
The power-up and power-down order and timing is defined by user register settings. The default settings support the E Ink Vizplex panel and typically do not need to be changed.
In SLEEP mode the TPS65185x is completely turned off, the I 2 C registers are reset, and the device does not accept any I 2 C transaction. Pull the WAKEUP pin high with the PWRUP pin low and the device enters STANDBY mode which enables the I 2 C interface. Write to the UPSEQ0 register to define the order in which the output rails are enabled at power-up and to the UPSEQ1 registers to define the power-up delays between rails. Finally, set the ACTIVE bit in the ENABLE register to 1 to execute the power-up sequence and bring up all power rails.
Alternatively pull the PWRUP pin high (rising edge).
After the ACTIVE bit has been set, the negative boost converter (VN) is powered up first, followed by the positive boost (VB). The positive boost enable is gated by the internal power-good signal of the negative boost. Once VB is in regulation, it issues an internal power-good signal and after delay time UDLY1 has expired, STROBE1 is
issued. The rail assigned to STROBE1 will power up next and after its power-good signal has been asserted and delay time UDLY2 has expired, STROBE2 is issued. The sequence continues until STROBE4 has occurred and the last rail has been enabled.
...

So VB (our DCDC1 circuit) is powered up second after VN (DCDC2, should be -16V). I could try seeing if VN comes on and has the correct voltages, if they are wrong that might indicate a problem with the screen....

[NiLuJe]

@pazos: IIRC, that error is "normal", and related to the width/height swap because of the Landscape panel on the H2O. I certainly remember seeing it on my device as part of a normal and working boot process (quite possibly when pickel or nickel rotates the fb) .

EDIT: Yup, fbdepth -r 0 -> fbdepth -r 3 triggers it, too.

[FBInk] Detected a Kobo Aura H2O (370 => Dahlia @ Mark 5)
Variable fb info: 1440x1080 (1440x2304), 8bpp @ rotation: 0 (Upright, 0°)
Fixed fb info: ID is "mxc_epdc_fb", length of fb mem: 6684672 bytes & line length: 1440 bytes

Current bitdepth is already 8bpp!

Switching fb to 8bpp (current bitdepth) @ rotation 3 . . .
Setting bitdepth to 8bpp
Setting grayscale to 1
Setting rotate to 3 (Counter Clockwise, 270°)
Setting rotate to 1 (Clockwise, 90°) to account for kernel rotation quirks
Bitdepth is now 8bpp (grayscale: 1) @ rotate: 3 (Counter Clockwise, 270°)

Variable fb info: 1080x1440 (1088x3072), 8bpp @ rotation: 3 (Counter Clockwise, 270°)
Fixed fb info: ID is "mxc_epdc_fb", length of fb mem: 6684672 bytes & line length: 1088 bytes


epdfbdc_set_width_height(1439): new DCSize(3342336)>original DCSize(3317760) !

[pazos]

Good to know @NiLuJe. So totally unrelated it was.

[DomesticExtremis]

Originally Posted by pazos

Good to know @NiLuJe. So totally unrelated it was.

Ditto. One less thing to worry about.

[DomesticExtremis]

Originally Posted by pazos

I know it is tagged as an error in kernel code but I'm not sure if that is the cause, a consequence of another circuitry error or totally unrelated.

I took a poke around in your source archive and looked for the DCDC1 error.
The way the function is structured, and the way the interrupt register holds its error codes means that there can only be single error reported and it seems to be detected unambiguously.
That is to say we do not have a hidden UVLO or DCDC2 error - the one reported is the one we have.