Bone VDD_3V3EXP Disable Issues

If I disable LDO3 in the TPS65217, VDD_3V3A should turn off. VDD_3V3A
turning off should disable U8 linear regulator causing VDD_3V3EXP to
turn off.

It doesn't.

Disabling LDO3 in the TPS65217 causes VDD_3V3A to drop to approximately
2.4 V. This is not low enough to disable the U8 linear regulator's
enable pin. Further, disabling LDO4 after this only causes VDD_3V3A to
drop to approximately 1.2 V. Again, this is not enough to disable U8
linear regulator's enable pin.

I've tested on an A3 and an A6 bone.

I realize there's grave impacts to disabling LDO3 or LDO4, ie: much of
the board won't work any more although the ARM core and DDR will still
be operating. That's OK for my experiments.

How can I turn off VDD_3V3EXP from software without completely turning
off all rails on the TPS65217? Is it possible?

I'm trying to do this in order to debug an issue we see with a custom
cape at power down when running on 3.7 V lithium battery. With our cape
attached, VDD_3V3EXP latches on even when the rest of the BeagleBone is
powered off. This is due to SYS_5V staying present due to the battery
and U8 linear regulator not powering down before the latch action
occurs. The TPS65217 and AM3359 are both in the proper states, I can
boot due to power button press once in the OFF state (defined by the
TPS65217 data sheet). But keeping VDD_3V3EXP running will ruin the
battery life while in the OFF state.

On the cape, we pull up to VDD_3V3EXP for things like MMC/SD and I2C.
I believe this is part of the problem as there may be power back feeding
through the pull-ups enough to cause the latch situation. But we
believe we're following the recommended pull-up strategy documented in
the Bone SRM.

Has any one else seen issues like this when attempting to enter the
TPS65217 OFF state with capes attached that use pull-ups to VDD_3V3EXP
while running on 3.7 V lithium battery power? If so, what capes work?

Yes, I realize this is quite a corner case. Sadly, it's an important
one for me.

Thanks,
Andrew

You are correct. This is something we just ran into yesterday and we are looking at a solution. The solution is being tested and will not be something that shows up in the near future, but hopefully soon. It MAY be something that could be done with a wire mod, but it is too early to tell. Should know more by Monday.

Gerald

You are correct. This is something we just ran into yesterday and we
are looking at a solution. The solution is being tested and will not
be something that shows up in the near future, but hopefully soon. It
MAY be something that could be done with a wire mod, but it is too
early to tell. Should know more by Monday.

Thanks! So I'm not crazy, that's good to know. :slight_smile:
Even if the wire mod is complex, it'd be great if you could share it.

I ran into a similar finding on the A6 board. Interestingly, the A5C works correctly. These are with a battery connected.

I’m investigating a proper shutdown and power-off if the main 5V is removed. I have a Li-Ion battery connected to keep everything running until a proper shutdown/power-off can occur.
Using the method in “Re: [beagleboard] Chip power down and PMIC_POWER_EN” while in Uboot, I can see that the A5C shuts everything down including the VDD_3V3B supply (see schematic). The A6 does not.

Now if there is no battery connected, they both power-off.

So what’s happening?

Brad

Did you look here?

http://www.elinux.org/Beagleboard:BeagleBoneBlack#Board_Revisions_and_Changes

3) Moved the enable for the VDD_3V3B regulator to VDD_3V3A rail. Change was made to reduce the delay between the ramp up of the 3.3V rails. No evidence of this being an issue, but it really needs to be as close to the same as possible.

So, you have to shutdown VDD_3V3A to shutdown the VDD_3V3B.

Gerald

Thanks for replying.

Yes I have seen that note. And when the battery is not connected the VDD_3V3A does shutdown, but not when a battery is connected (A6 version). When I measure the voltage on VDD_3V3A after the ‘shutdown’, it is about 2.4V (it is 3.3V when ON). Is there a path for leakage back to that node that is keeping the voltage up?

The registers in the PMIC are the same for both board versions. I have not changed them from what comes with the BBB.

I used this method to test the shutdown:

Uboot example:

  1. Boot to the uboot prompt by pressing any key before the kernel boots
  2. Set the PMIC i2c status register to turn OFF when EN line goes down. (i2c, memory write, device 24, address A, value 0x80)

    i2c mw 24 A 80

  3. Set the memory base to the RTC registers

    base 44e3e000

  4. Enable PWR_ENABLE_EN bit (memory write, offset address 98, value in hex 0x00010000)

    mw 98 10000

  5. Enable ALARM2 interrupt once (memory write, offset address 48, value in hex 0x00000010)

    mw 48 10

  6. Copy over RTC current time to ALARM2 times for hour/day/month/year (memory copy, from address 8, to address 88, 4 times)

    cp 8 88 4

  7. This next part is where it gets slightly more complicated because you are going to manually set the ALARM2 minutes to one or two minutes after the current time.
  8. Read the current seconds, minutes. This is easy to manually do because they are in BCD and not in hex. (memory display, address zero, 2 times)

    md 0 2
    44e3e000: 00000008 00000011 (8 seconds, 11 minutes)

  9. Write to the ALARM2 minutes (seconds should already be zero) (memory write, address 84, value 0x00000012)

    mw 84 12 (12 minutes)1. Wait until the current time catches up to the ALARM2 time and the system will power down. If you took too long doing step 7.2 and the time has already passed, just do step 7.2 again.

  10. While you’re waiting you can keep checking the current time ( this will show all 6 fields, second, minute, hour, day, month, year)

    md 0 6

  11. And if you think you have passed your ALARM2 time you can compare above results with ALARM2 time. If so, go to step 7.2. If you went onto a new hour, you might have to start from step 6.

    md 80 6

Thanks,

Brad

The change only affected when the 3V3B rail was enabled by enabling it with the 3V3A rail. So, I can’t see how they are related. Both LDOs are powered by the same SYS_5V bus.

What is the level of the SYS_5V bus?

The battery in this case is now supplying all of the LDOs and switchers. So I am thinking that maybe we need to make sure all rails are actually tuned off via SW.

Gerald

The SYS_5V bus has the Battery supply, approx. 3.8V

How can I make sure “all rails are actually turned off via SW”?

I’ll go through the procedure and measure all the voltages and report back.

Brad

Check the registers in the datasheets. There is a away to group which rails are up and down when you go into the power down mode. However, it changes back to default when you power back up so you have to write them again…

Gerald

The SYS_5V bus has the Battery supply, approx. 3.8V

How can I make sure "all rails are actually turned off via SW"?

I'll go through the procedure and measure all the voltages and report back.

Are you running a cape during your tests or not?
If so, does your cape have any pull-up resistors on it?

Are we talking about white or black bones? I've lost that in this
conversation. My original experiments were all with white bones.

Thanks,
Andrew

PS: Please keep me CC'ed, I'm no longer subscribed to the list.

This is without a cape and it is with the BB Black. That's probably why I
was confused about the dates and board versions you referred to.

Brad

Ok, here’s what I did:

No cape

Debug serial connected to desktop computer

  1. Connect battery as designed

a. Batt+ to TP5

b. Batt+ to TP6

c. 10k between TP7 and TP8

d. Gnd to TP8

  1. Connect +5V to power plug

  2. Let BBB boot up

  3. Login

  4. Disconnect +5V

  5. Dump PMIC registers (in order) (same for both boards)

a. 0xE2

b. 0x3E

c. 0

d. 0

e. 0xB1

f. 0x80

g. 0xB2

h. 0x01

i. 0

j. 0

k. 0x80

l. 0

m. 0x7F

n. 0x0C

o. 0x18

p. 0x02

q. 0x09

r. 0x06

s. 0x09

t. 0x38

u. 0x26

v. 0x3F

w. 0x7F

x. 0

y. 0x03

z. 0x15

aa. 0x5F

bb. 0x32

cc. 0x40

dd. 0x20

ee. 0

  1. Shutdown –r now

  2. Boot to Uboot prompt

  3. “i2c mw 24 a 80” (set PMIC status register to turn OFF when PMIC_POWER_EN goes down)

  4. “base 44e3e000”

  5. “mw 98 10000”

  6. “mw 48 10”

  7. “cp 8 88 4”

  8. “md 8 4”

  9. “md 88 4” to check that the data transferred

  10. “md 0 2”

  11. “mw 84 xx”, time in future for Alarm2 to act

  12. Wait for the shutdown

  13. Measure voltages:

a. A6:

i. Vbatt = 4.03

ii. SYS_5V = 4.00

iii. VDD_5V = 0

iv. VDD_3V3B = 3.39

v. VDD_3V3A = 2.27

vi. VDDS_DDR = 0

vii. VDD_MPU = 0

viii. VDD_CORE = 0

ix. VDD_1V8 = 0.16

x. VRTC = 0.16

xi. VDD_3V3AUX = 0.16

b. A5C

i. Vbatt = 4.03

ii. SYS_5V = 4.03

iii. VDD_5V = 0

iv. VDD_3V3B = 0

v. VDD_3V3A = 0

vi. VDDS_DDR = 0

vii. VDD_MPU = 0

viii. VDD_CORE = 0

ix. VDD_1V8 = 0

x. VRTC = 0

xi. VDD_3V3AUX = 0

So those are my results. Any thoughts?

Brad

An update:

For the A6 (Beaglebone Black) version, I connected U4 pin 1 (enable) back to VDD_3V3AUX (same as A5C). It now shuts down correctly while on battery. This was verified on two A6 BBB.

The mystery is what causes the VDD_3V3A to hang up on the A6 version when shutting down while on battery.

Brad,

Thank you for this post. I thought I was going madder than usual when the production units wouldn’t turn off but the prototypes did!

all it required was lifting pin 1 of U4 and soldering a wire-wrap wire to the power led anode (side furthest from the edge of the board)

you have as they say “saved my bacon”

Best Wishes,

Darren

Does anyone know if there is a software solution to this issue? I haven’t seen any changes in the recent hardware revisions to address this. We are currently planning on deploying a couple of dozen BBB and I would prefer not having to make hardware mods.

Terry

Does anyone know if there is a software solution to this issue? I haven’t seen any changes in the recent hardware revisions to address this. We are currently planning on deploying a couple of dozen BBB and I would prefer not having to make hardware mods.

I’ve just been looking into the PMIC for another user and this is probably a configuration issue. The PMIC supports several modes (page 15 of the TPS65217C datasheet). I believe the PMIC isn’t configured to enter OFF mode, but rather it is entering SLEEP mode which means any rails not controlled by the power-down sequencer will remain enabled in SLEEP mode.

Here is what I think the solution might be:

Add “pmic-shutdown-controller” as shown in
/Documentation/devicetree/bindings/regulator/tps65217.txt.

In the V3.15.1-bone2 kernel, /drivers/mfd/tps65217.c line 213, the
comments “Set the PMIC to shutdown on PWR_EN toggle”. This should work for all kernel V3.8 onwards.

Reading “Power Down Sequence” on page 18 (TPS65217C datasheet), this will initiate the power
down sequence and leave the PMIC in OFF mode.

I want to confirm this with Robert Nelson first before proceeding; however, you can add "pmic-shutdown-controller” as described in the kernel docs to /arch/arm/boot/dts/tps65217.dtsi and see if this works for you.

Regards,
John

I just learned the hard way that there’s a shutdown difference in the Rev C boards vs. Rev A5, so I’m glad to find this thread. I confirmed with ‘i2cdump -f 0 0x24’ that register 0x0Ah = 84 which means it IS configured to enter OFF mode when the button is pushed. Should the ‘pmic-shutdown-controller’ code change still be made? Is there a better way to get the PMIC into OFF mode when the button is pushed? I really need this function so the battery isn’t drained during extended off time. I’m 2 for 2 with Rev C boards that act this way, so I assume it’s common.
Thanks for any help, JP

Pinging this group again in hopes for some advice or info or updates:
Has anyone successfully shutdown the PMIC from software while on battery pwr?
I can confirm the board mod works, but now I’m facing a qty 100 order! That’s too much rework for me, so I can’t place an order until I have a reasonable resolution.
Thanks, JP

The PMIC is not the problem here, it properly shuts down all DCDC and LDO supplies, and requires no configuration other than voltage adjustments when desired, and setting the OFF bit before using the RTC to request shutdown. (“SLEEP state” of the PMIC, aka “RTC-only sleep”, is not supported since vdds was moved from LDO3 to LDO1 in rev A6A.)

The issue is the interconnection of the 3v3a and 3v3b power domains, resulting in significant leakage current between them (e.g. via protection diodes) whenever one is powered while the other is not. The 3v3b → 3v3a leakage was of course exactly the reason for moving the enable-signal of the 3v3b regulator (U4) from 3v3aux (LDO2) to 3v3a. However, while this resolved the issue at boot, it did not resolve it at shutdown when running on dc power, and made it far worse when running on battery.

The problem is that when the 3v3a supply (LDO4) is disabled, the 3v3b regulator remains enabled until 3v3a drops below the threshold voltage of the enable-input of U4, which is far below 3.3V (specified to be somewhere between 0.4V (at 25 ͏°C) and 2V). As a result, current will start to flow from 3v3b to 3v3a, and apparently enough current to keep it logic-high in the opinion of the 3v3b-regulator (despite the ~375 ohm discharge resistor the PMIC applies when LDO4 is disabled!).

Thus, once turned on, the 3v3b regulator manages to keep itself enabled indefinitely as long as it is supplied from SYS_5V. When entering off-state, the PMIC automatically connects SYS to battery power rather than DC- or USB-supply. If there’s no battery, then the capacitors on SYS will drain pretty fast hence the 3v3b shutdown is not delayed much. If there’s a battery, then you’re pretty screwed.

It is very important to note that this issue is not merely one of battery lifetime; this leakage current may damage the processor.

To illustrate all this, let’s stare at some pretty pictures.

Here’s a capture of various power terminals during boot and shutdown while on DC power (click to zoom):

I’ve marked the PMIC sequencer “strobes” with vertical dashed lines, partially cut away when irrelevant or visually interfering with the signal plots. Unless noted otherwise, the interval between strobes is 1 ms.

As I mentioned, in off-state SYS is tied to BAT, and when DC-powered they hover around 1V for some reason (they’ll drop if you try to drain power from either, but once you stop they bounce back to 1V). All regulated supplies are off. Once the PMIC has detected a wakeup event, it connects SYS to the DC power supply, asserts the wakeup signal, and the sequencer powers up the “always-on” supplies:

  • strobe 15: LDO1 (rtc, vdds)

  • strobe 14: unused
    The AM335x RTC asserts PW_EN and once debounced by the PMIC (~50 ms) the sequencer completes the power-on sequence:

  • strobe 1: DCDC1 (ddr)

  • strobe 2: LDO3 (1v8)

  • [5 ms delay]

  • strobe 3: LDO2 (3v3aux = power led)

  • strobe 4: LDO4 (3v3a, enables 3v3b)

  • strobe 5: DCDC2 (mpu), DCDC3 (core)

  • strobe 6: unused

  • strobe 7: unused
    Meanwhile, since BATMON isn’t connected to BAT, the PMIC doesn’t really get what’s going on there. It seems to attempt to charge it for a while, then gives up, and later BAT somehow manages to drop below 0V and stay there (behaviour is quite different if BATMON is connected to BAT, in which case it’s pulsed a few times then drops to about 0.2V - 0.4V, apparently depending on system load).

When reentering off-state the PMIC shuts down the supplies in reverse order, with 1 ms between strobes 1 and 14. Strobe 7 is relevant this time because it marks when shutdown is initiated, and as a result SYS is disconnected from DC power and connects to BAT, which immediately shoots up to SYS level. Note however that 3v3b is not disabled at strobe 4 but stays slightly below SYS until long after the PMIC has completed shutdown. About 20 ms after it was supposed to, it finally begins to drop to zero while SYS, now unloaded and heavily supported by fat capacitors, begins its long and slow journey back towards the 1V.

For comparison, the same plot but now powered at 3.6V through BAT (using a variable power supply).

When BAT was powered up, SYS started tracking it once it reached 1V (hmm, sounds familiar), but the PMIC initially remains in off-state. The boot and shutdown are essentially the same, except SYS follows BAT continuously. As a result, once 3v3b is turned on, it remains on until power to BAT is cut.

Bottom line: if you want to run on battery power, this is a serious problem. You’ll need to keep the PMIC in active-state, though you can minimize power by commanding the ethernet phy and hdmi framer to power themselves down, and then enter a deepsleep mode. Alternatively, you’d need to patch the hardware, or have an external circuit detect this situation (3v3b on while 1v8_adc off) and disconnect the battery to resolve it.

Hi Matthijs,

The PMIC is not the problem here, it properly shuts down all DCDC and LDO
supplies, and requires no configuration other than voltage adjustments
when
desired, and setting the OFF bit before using the RTC to request
shutdown.
("SLEEP state" of the PMIC, aka "RTC-only sleep", is not supported since
vdds was moved from LDO3 to LDO1 in rev A6A.)

The issue is the interconnection of the 3v3a and 3v3b power domains,
resulting in significant leakage current between them (e.g. via
protection
diodes) whenever one is powered while the other is not. The 3v3b -> 3v3a
leakage was of course exactly the reason for moving the enable-signal of
the 3v3b regulator (U4) from 3v3aux (LDO2) to 3v3a. However, while this
resolved the issue at boot, it did not resolve it at shutdown when
running
on dc power, and made it far worse when running on battery.

The problem is that when the 3v3a supply (LDO4) is disabled, the 3v3b
regulator remains enabled until 3v3a drops below the threshold voltage of
the enable-input of U4, which is far below 3.3V (specified to be
somewhere
between 0.4V (at 25 ͏°C) and 2V). As a result, current will start to flow
from 3v3b to 3v3a, and apparently enough current to keep it logic-high in
the opinion of the 3v3b-regulator (despite the ~375 ohm discharge
resistor
the PMIC applies when LDO4 is disabled!).

Thus, once turned on, the 3v3b regulator manages to keep itself enabled
indefinitely as long as it is supplied from SYS_5V. When entering
off-state, the PMIC automatically connects SYS to battery power rather
than
DC- or USB-supply. If there's no battery, then the capacitors on SYS will
drain pretty fast hence the 3v3b shutdown is not delayed much. If there's
a
battery, then you're pretty screwed.

It is very important to note that this issue is not merely one of battery
lifetime; this leakage current may *damage the processor*.

To illustrate all this, let's stare at some pretty pictures.

Here's a capture of various power terminals during boot and shutdown
while
on DC power (click to zoom):

<https://lh3.googleusercontent.com/-aWEH_7JAEbw/VUBn8aVmaoI/AAAAAAAAACE/5RFxipIqG_E/s1600/dc.png>

I've marked the PMIC sequencer "strobes" with vertical dashed lines,
partially cut away when irrelevant or visually interfering with the
signal
plots. Unless noted otherwise, the interval between strobes is 1 ms.

As I mentioned, in off-state SYS is tied to BAT, and when DC-powered they
hover around 1V for some reason (they'll drop if you try to drain power
from either, but once you stop they bounce back to 1V). All regulated
supplies are off. Once the PMIC has detected a wakeup event, it connects
SYS to the DC power supply, asserts the wakeup signal, and the sequencer
powers up the "always-on" supplies:

   - strobe 15: LDO1 (rtc, vdds)
   - strobe 14: unused

The AM335x RTC asserts PW_EN and once debounced by the PMIC (~50 ms) the
sequencer completes the power-on sequence:

   - strobe 1: DCDC1 (ddr)
   - strobe 2: LDO3 (1v8)
   - [5 ms delay]
   - strobe 3: LDO2 (3v3aux = power led)
   - strobe 4: LDO4 (3v3a, enables 3v3b)
   - strobe 5: DCDC2 (mpu), DCDC3 (core)
   - strobe 6: unused
   - strobe 7: unused

Meanwhile, since BATMON isn't connected to BAT, the PMIC doesn't really
get
what's going on there. It seems to attempt to charge it for a while, then
gives up, and later BAT somehow manages to drop below 0V and stay there
(behaviour is quite different if BATMON is connected to BAT, in which
case
it's pulsed a few times then drops to about 0.2V - 0.4V, apparently
depending on system load).

When reentering off-state the PMIC shuts down the supplies in reverse
order, with 1 ms between strobes 1 and 14. Strobe 7 is relevant this time
because it marks when shutdown is initiated, and as a result SYS is
disconnected from DC power and connects to BAT, which immediately shoots
up
to SYS level. Note however that 3v3b is *not* disabled at strobe 4 but
stays slightly below SYS until long after the PMIC has completed
shutdown.
About 20 ms after it was supposed to, it finally begins to drop to zero
while SYS, now unloaded and heavily supported by fat capacitors, begins
its
long and slow journey back towards the 1V.

For comparison, the same plot but now powered at 3.6V through BAT (using
a
variable power supply).
  
<https://lh3.googleusercontent.com/-igtCZe5bffg/VUBoUPZ5PKI/AAAAAAAAACM/xKL_ZMFQLso/s1600/bat.png>

When BAT was powered up, SYS started tracking it once it reached 1V (hmm,
sounds familiar), but the PMIC initially remains in off-state. The boot
and
shutdown are essentially the same, except SYS follows BAT continuously.
As
a result, once 3v3b is turned on, it remains on until power to BAT is
cut.

Bottom line: if you want to run on battery power, this is a serious
problem. You'll need to keep the PMIC in active-state, though you can
minimize power by commanding the ethernet phy and hdmi framer to power
themselves down, and then enter a deepsleep mode. Alternatively, you'd
need
to patch the hardware, or have an external circuit detect this situation
(3v3b on while 1v8_adc off) and disconnect the battery to resolve it.

You seem to be talking about the BBB, correct?
My initial work was on BBW back in 2012 (prior to BBB release).

It sounds like BBB has similar issues to BBW with running on battery but
that the issues are slightly different.

I'm not subscribed to this list any longer so please keep me in CC.
Thanks!
-Andrew