Bone A6 (and others) PHY LEDs after reboot

GoogleGroups
#1

On Beaglebone A6, the Ethernet PHY LEDs should indicate:
Orange -> ON = link, OFF = no link.
Green -> ON = 100 Mbit, OFF = 10 Mbit, only active if Orange is ON.
On activity, Green LED should invert for a very short period.

For example, when on a 100 Mbit network, at boot part way into u-boot
loading when the PHY and other networking is set up, both Green and
Orange LEDs should be on and the Green LED should blink off briefly
during network activity.

This works great.

Up until a reboot or other warm software reset (u-boot 'reset' command,
Linux 'reboot', or similar), for me. After a warm software reset I
observe the LEDs to have inverted themselves, now my 100 Mbit network
link is shown by both LEDs being off and removing the cable turns both
LEDs on. A further reboot or warm reset inverts it back to how it was
before.

Has any one else observed this?

It also affects A5 Bones, but their LED configuration is different and
less obvious due to the operation of the Orange LED being changed for
the A6 rev.

Thanks,
Andrew

#2

On Beaglebone A6, the Ethernet PHY LEDs should indicate:
Orange -> ON = link, OFF = no link.
Green -> ON = 100 Mbit, OFF = 10 Mbit, only active if Orange is ON.
On activity, Green LED should invert for a very short period.

No, I'm wrong here.

The correct LED activity for an A6 bone is:
Green -> ON = Link, OFF = no link, blinks off briefly on activity.
Orange -> ON = 100 Mbit, OFF = 10 Mbit.

For example, when on a 100 Mbit network, at boot part way into u-boot
loading when the PHY and other networking is set up, both Green and
Orange LEDs should be on and the Green LED should blink off briefly
during network activity.

This works great.

Up until a reboot or other warm software reset (u-boot 'reset'
command, Linux 'reboot', or similar), for me. After a warm software
reset I observe the LEDs to have inverted themselves, now my 100 Mbit
network link is shown by both LEDs being off and removing the cable
turns both LEDs on. A further reboot or warm reset inverts it back
to how it was before.

My tentative fix is increasing the PRM_DEVICE.PRM_RSTTIME.RSTTIME1
register bitfield from the default of 0x6 to 0x80 seems to fix things
for me. It's measured in counts of SYS_CLK, which may be the 32 kHz
clock but also may not be. If someone could clarify SYS_CLK operation
for me, I'd appreciate it, the TRM isn't super clear to me.

With the default setting of 0x6, the SYS_RESETn only goes low for 800
ns during a software initiated warm reset. At 0x80, SYS_RESETn is low
for almost 6 us. At 0xFF, SYS_RESETn is low for about 11 us.

The SMSC LAN8710 PHY data sheet states a minimum duration for nRST of
100 us in order to perform a hardware reset of the PHY. We're not even
close to that.

Alternatively, u-boot could issue a software reset to the PHY, but I've
not yet investigated this or if:
1. u-boot may already do this
2. software reset of the PHY may not affect LED settings

Scope plots: http://imgur.com/a/CE3Hi

-Andrew

#3

Andrew,

Wow, that’s wacky. I actually haven’t observed this particular phenomenon, but I have noticed that the SMSC 8710 doesn’t always detect network activity after plugging in a network cable. More specifically, the LEDs don’t light up, and no link is detected. There is a patch in the mainline Linux kernel that cures this problem by disabling the FlexPWR feature:

http://permalink.gmane.org/gmane.linux.network/244262

When this problem occurs, a warm software reboot (e.g. ‘reboot’) doesn’t break the SMSC 8710 out of this state. Neither U-boot nor the Linux driver issues a software reset to the PHY. I did experiment by adding a software reset to the SMSC 8710 in the Linux driver, and this seemed to kick the PHY out of the state.

It sounds like you have the right fix, but perhaps disabling the FlexPWR feature might cure the problem too? I would also venture to guess issuing a software reset to the 8710 would too.

I asked SMSC about the FlexPWR issue, and this is what I got back:

"The comment in the kernel driver is not correct.

It is not the bug. It is that the author did not understand how the device EDPD works.SMSC contacted the developer who put this comment in and explained how the device functions.

Since the device expects 2 NLPs in 64ms apart, many devices do not do this hence the device cannot wake from EDPD.

#4

Wow, that's wacky. I actually haven't observed this particular
phenomenon, but I have noticed that the SMSC 8710 doesn't always
detect network activity after plugging in a network cable. More
specifically, the LEDs don't light up, and no link is detected.
There is a patch in the mainline Linux kernel that cures this problem
by disabling the FlexPWR feature:

http://permalink.gmane.org/gmane.linux.network/244262

When this problem occurs, a warm software reboot (e.g. 'reboot')
doesn't break the SMSC 8710 out of this state. Neither U-boot nor
the Linux driver issues a software reset to the PHY. I did
experiment by adding a software reset to the SMSC 8710 in the Linux
driver, and this seemed to kick the PHY out of the state.

That's interesting. Are you able to reproduce this issue of not
getting link from u-boot? Taking Linux out of the equation?

The 8710 data sheet says the default state for the EDPWRDOWN register
bit is 0, with this ability to perform Energy Detect Power-Down
disabled. Who is changing that register setting in your case?

Marek's patch seems somewhat useless if that register bit isn't ever
getting enabled in the first place. Unless this is another case of
Linux protecting itself from the bootloader (which isn't a bad idea, I
guess...).

I don't believe I've seen this issue before but now I'm going to try to
see if I can reproduce it.

It sounds like you have the right fix, but perhaps disabling the
FlexPWR feature might cure the problem too? I would also venture to
guess issuing a software reset to the 8710 would too.

I asked SMSC about the FlexPWR issue, and this is what I got back:

These details don't seem to appear in the SMSC data sheet...

"The comment in the kernel driver is not correct.

It is not the bug. It is that the author did not understand how the
device EDPD works.SMSC contacted the developer who put this comment
in and explained how the device functions.****

** **

Since the device expects 2 NLPs in 64ms apart, many devices do not do
this hence the device cannot wake from EDPD.****

==============================================================================
****

Below is an explanation of how their Energy Detect and Power Down
works:****

** **

ENERGYON algorithm: There are two state machines involved.****

** **

The signal valid generation is as below:****

1. If a NLP is detected and then starts a 64ms timer.****

2. Checks for pulse going low and waits 2us.****

3. If a second pulse is detected before the 64ms timer completes, a
valid signal is determined and ENERGYON goes high (taking out of
Energy Detect Power Down).****

4. If no second pulse is detected before the 64ms timer completes,
keep ENERGYON low and go back to step 1.****

** **

If auto-neg on:****

When auto-neg state machine enters detection state, a 256ms timer
starts. If timer expires ENERGYON goes low.****

If auto-neg transitions to another state before 256ms timer is
aborted and ENERGYON remains high. Also, when signal valid is high,
ENERGYON goes high.* ***

** **

If auto-neg is off:****

If the link is not ON for a duration of 256ms the ENERYON goes low.
When signal valid goes high, ENERGYON goes high.****

** **

Some of the switches/network cards may not send the NLP 64ms apart.
We did see some cards send 256ms and even some send 1 sec apart. With
such partner card our device may not wake from EDPD.*"*

Any pointers on cards / switches that perform the 256 ms or 1 second
version? Getting one of those might help me test.

#5

Wow, that’s wacky. I actually haven’t observed this particular
phenomenon, but I have noticed that the SMSC 8710 doesn’t always
detect network activity after plugging in a network cable. More
specifically, the LEDs don’t light up, and no link is detected.
There is a patch in the mainline Linux kernel that cures this problem
by disabling the FlexPWR feature:

http://permalink.gmane.org/gmane.linux.network/244262

When this problem occurs, a warm software reboot (e.g. ‘reboot’)
doesn’t break the SMSC 8710 out of this state. Neither U-boot nor
the Linux driver issues a software reset to the PHY. I did
experiment by adding a software reset to the SMSC 8710 in the Linux
driver, and this seemed to kick the PHY out of the state.

That’s interesting. Are you able to reproduce this issue of not
getting link from u-boot? Taking Linux out of the equation?

I haven’t tried that yet, but I don’t think U-Boot sets the EDPWRDOWN register so this problem shouldn’t happen at startup.

The 8710 data sheet says the default state for the EDPWRDOWN register
bit is 0, with this ability to perform Energy Detect Power-Down
disabled. Who is changing that register setting in your case?

The Linux driver. In drivers/net/phy/smsc.c, see smsc_phy_config_init():

/* Enable energy detect mode for this SMSC Transceivers */
rc = phy_write(phydev, MII_LAN83C185_CTRL_STATUS,
rc | MII_LAN83C185_EDPWRDOWN);

Marek’s patch seems somewhat useless if that register bit isn’t ever
getting enabled in the first place. Unless this is another case of
Linux protecting itself from the bootloader (which isn’t a bad idea, I
guess…).

I don’t believe I’ve seen this issue before but now I’m going to try to
see if I can reproduce it.

It sounds like you have the right fix, but perhaps disabling the
FlexPWR feature might cure the problem too? I would also venture to
guess issuing a software reset to the 8710 would too.

I asked SMSC about the FlexPWR issue, and this is what I got back:

These details don’t seem to appear in the SMSC data sheet…

"The comment in the kernel driver is not correct.

It is not the bug. It is that the author did not understand how the
device EDPD works.SMSC contacted the developer who put this comment

in and explained how the device functions.****

Since the device expects 2 NLPs in 64ms apart, many devices do not do

this hence the device cannot wake from EDPD.****

==============================================================================

Below is an explanation of how their Energy Detect and Power Down

works:****

ENERGYON algorithm: There are two state machines involved.****

The signal valid generation is as below:****

  1. If a NLP is detected and then starts a 64ms timer.****

  2. Checks for pulse going low and waits 2us.****

  1. If a second pulse is detected before the 64ms timer completes, a
    valid signal is determined and ENERGYON goes high (taking out of

Energy Detect Power Down).****

  1. If no second pulse is detected before the 64ms timer completes,

keep ENERGYON low and go back to step 1.****

If auto-neg on:****

When auto-neg state machine enters detection state, a 256ms timer

starts. If timer expires ENERGYON goes low.****

If auto-neg transitions to another state before 256ms timer is
aborted and ENERGYON remains high. Also, when signal valid is high,

ENERGYON goes high.* ***

If auto-neg is off:****

If the link is not ON for a duration of 256ms the ENERYON goes low.

When signal valid goes high, ENERGYON goes high.****

Some of the switches/network cards may not send the NLP 64ms apart.
We did see some cards send 256ms and even some send 1 sec apart. With

such partner card our device may not wake from EDPD."

Any pointers on cards / switches that perform the 256 ms or 1 second
version? Getting one of those might help me test.

I’m using a TrendNet POE TPE-S44. I’d imagine some of the cheaper switches out there may also cause the problem to occur.

#6

<snip>

Up until a reboot or other warm software reset (u-boot 'reset'
command, Linux 'reboot', or similar), for me. After a warm software
reset I observe the LEDs to have inverted themselves, now my 100 Mbit
network link is shown by both LEDs being off and removing the cable
turns both LEDs on. A further reboot or warm reset inverts it back
to how it was before.

Has any one else observed this?

I'm worried that no one else has seen this. I'm searching for someone
to verify my findings below. If you have an A5 or A6 bone, could you
please help me out?

The following is how to reproduce on A5 and A6 bones with current (as
of this morning) mainline U-Boot 2013.01-rc1-00071-g1cc619b and no
uEnv.txt using the 'am335x_evm_config' and USB power only (I do not
allow Linux to load):

A3: at first boot when connected to 100 Mbit Ethernet, just the orange
Ethernet LED is on and the green LED is off, then after u-boot loads the
networking components just the green LED is on and the orange LED is
off. Executing a u-boot 'reset' command does not change the LED
orientation. This is as expected, u-boot 'reset' command (a warm
software reset) should not change the LED orientation even though the
SMSC PHY's nRST line gets pulled and thus the PHY should resample its
strapping resistors.

A5: at first boot when connected to 100 Mbit Ethernet, just the orange
Ethernet LED is on and the green LED is off, then after u-boot loads the
networking components just the green LED is on and the orange LED is
off. However, executing a u-boot 'reset' command DOES change the LED
orientation. After the 'reset' command, both the orange and green LEDs
are on until u-boot loads the networking components and then both LEDs
turn off. A further u-boot 'reset' command reverts to the original LED
configuration and operation. This is NOT expected, u-boot 'reset'
command should NOT change the orientation of the Ethernet LEDs.

A6: at first boot when connected to 100 Mbit Ethernet, no Ethernet LEDs
are on, then after u-boot loads the networking components both the
orange and green LEDs are on. However, executing a u-boot 'reset'
command DOES change the LED orientation. After the 'reset' command,
both the orange and green LEDs are off until u-boot loads the
networking components and then both LEDs turn on. A further u-boot
'reset' reverts to the original LED configuration and operation. This
is NOT expected, u-boot 'reset' command should NOT change the
orientation of the Ethernet LEDs.

I'm working on patch to be submitted to mainline u-boot to either
lengthen the PRM_DEVICE.PRM_SRTTIME.RSTTIME1 register's default from
0x06 to at least 0x80 (determined from some quick testing and results
shown in previous scope plots to this thread) or to provide a software
reset to the PHY (which has not yet been tested). If lengthening the
RSTTIME1 value is performed, the nRST line still is not asserted to the
SMSC PHY for longer than the SMSC recommended minimum, which seems OK,
but isn't ideal. The ideal solution is a change to the nRST PHY
circuit to ensure that nRST is asserted for at least 100 us when ever
it's asserted.

A caveat on the A3, if I reduce the RSTTIME1 register to lower than the
default 0x06, I can make the A3 act like the A5. It seems the A3 I
have to test with is slightly less sensitive, but still exhibits the
issue with low values of RSTTIME1. Due to this, not all bones may show
this issue. Bones, in general, may be dancing on the limit of the SMSC
PHY's minimum nRST asserted duration. My theory is that with very
short nRST durations, the PHY's sampling of the strapping resistors is
being cut short and thus the LED orientation (active low vs high) gets
inverted. Why this reverses itself on a warm software reset still
boggles me.

I'm not confident in sending a patch for this unless someone else can
verify my observations above. Could a few intrepid souls who have
either A5 or A6 bones and 100 Mbit Ethernet switches please try to
reproduce this issue? It should only take you a moment and will not
harm your SD card. Just stop the boot at the u-boot prompt and observe
the LEDs, then issue a 'reset' command to u-boot and again stop the
boot and observe the LEDs.

Thanks,
Andrew

#7

Well, I've sent a patch anyway[1].

[1]:http://patchwork.ozlabs.org/patch/197551/

If anyone tests my exact steps and verifies the issue, try again with
that patch applied. Should fix it.

-Andrew

#8

I’m not confident in sending a patch for this unless someone else can
verify my observations above.

Well, I’ve sent a patch anyway1.

If anyone tests my exact steps and verifies the issue, try again with
that patch applied. Should fix it.

Andrew, your patch makes me wonder if it would be possible to disable the green and amber LED’s on the ethernet port entirely. Can that be done in software?

  • Grant
#9

My patch just extends the reset assertion duration to fix the issue.
The LEDs were screwing themselves up, I believe because the reset
assertion was under 1 us and the pins were having to transition from
being outputs to being inputs on the falling edge and then sample on
the rising edge. If there's any capacitance on the line, they might
read the wrong way since the pull resistors are pretty weak. With a 6
us reset assertion, they get more time to bleed the capacitance to the
proper value of the pulls.

I'm not sure you can disable the green LED, it should assert based on
CRS, not sure there's a register to disable that. The orange LED can
be forced off, if you don't mind running at 10 Mbit on an A6 bone (or
100 Mbit on A5 or earlier) via register settings or just hooking up to
another end at those speeds.

-Andrew

#10

I’m not sure you can disable the green LED, it should assert based on
CRS, not sure there’s a register to disable that. The orange LED can
be forced off, if you don’t mind running at 10 Mbit on an A6 bone (or
100 Mbit on A5 or earlier) via register settings or just hooking up to
another end at those speeds.

-Andrew

Can all 3 LED’s on the Beaglebone be disabled by making a physical modification to the board? If so, how difficult would it be?

  • Grant
#11

Which "all 3 LEDs" do you mean?

On my beaglebones there's 4 user LEDs, a power LED, and 2 Ethernet
LEDs. You can disable any or all of them by making hardware changes,
yes. Desoldering the resistors in series with them will render them
dark.

You can disable the user LEDs in software by just not having the kernel
set them up (the heartbeat and the mmc0 trigger).

You can disable the power LED by turning off LDO2 in the TPS65217 via
the I2C interface.

-Andrew

#12

Can all 3 LED’s on the Beaglebone be disabled by making a physical
modification to the board? If so, how difficult would it be?

Which “all 3 LEDs” do you mean?

On my beaglebones there’s 4 user LEDs, a power LED, and 2 Ethernet
LEDs. You can disable any or all of them by making hardware changes,
yes. Desoldering the resistors in series with them will render them
dark.

You’re right, I already disabled my user LED’s and forgot they were there.

You can disable the user LEDs in software by just not having the kernel
set them up (the heartbeat and the mmc0 trigger).

You can disable the power LED by turning off LDO2 in the TPS65217 via
the I2C interface.

Is this something that can be done without extra hardware?

  • Grant
#13

No 'extra' hardware is required. For the power LED, you can either
remove the resistor in series with the LED or command the TPS65217 (the
power management IC) to turn off LDO2 output (there's a register for
that).

For the Ethernet LEDs, simply remove the two series resistors (1 for
orange, 1 for green). Or don't use Ethernet (on an A6 bone where
LEDs 'do the right thing')...

It's kind of the opposite of 'extra' hardware :wink:

-Andrew

#14

You can disable the power LED by turning off LDO2 in the TPS65217
via the I2C interface.

Is this something that can be done without extra hardware?

No ‘extra’ hardware is required. For the power LED, you can either
remove the resistor in series with the LED or command the TPS65217 (the
power management IC) to turn off LDO2 output (there’s a register for
that).

Via what software interface is that command sent? Is it done in the kernel or userspace?

  • Grant
#15

The kernel can do it. Or you can have u-boot do it via your uEnv.txt
file via some i2c memory writes to the TPS65217. I don't believe you
can change TPS65217 registers via sysfs to disable an output but I don't
currently have a Beaglebone with me to test that out on (it might be
possible).

The data sheet for the TPS65217 shows all the registers and what each
bit means, just be aware some registers (ie: the ones you'll be
interested in) have a "password" to unlock which uses the password
protection register. All of this is documented in the data sheet.

You're probably mosts interested in the 'LDO2 Control Register' and
the 'Enable Register' if you want to change the brightness or turn off
the power LED on A6 or older Beaglebones.

-Andrew

#16

You can disable the power LED by turning off LDO2 in the
TPS65217 via the I2C interface.

Is this something that can be done without extra hardware?

No ‘extra’ hardware is required. For the power LED, you can either
remove the resistor in series with the LED or command the TPS65217
(the power management IC) to turn off LDO2 output (there’s a
register for that).

Via what software interface is that command sent? Is it done in the
kernel or userspace?

The kernel can do it. Or you can have u-boot do it via your uEnv.txt
file via some i2c memory writes to the TPS65217. I don’t believe you
can change TPS65217 registers via sysfs to disable an output but I don’t
currently have a Beaglebone with me to test that out on (it might be
possible).

The data sheet for the TPS65217 shows all the registers and what each
bit means, just be aware some registers (ie: the ones you’ll be
interested in) have a “password” to unlock which uses the password
protection register. All of this is documented in the data sheet.

You’re probably mosts interested in the ‘LDO2 Control Register’ and
the ‘Enable Register’ if you want to change the brightness or turn off
the power LED on A6 or older Beaglebones.

Putting this together from scratch is over my head for sure. Do you know of an example online of anyone doing this in the kernel or uEnv.txt?

  • Grant
#17

u-boot command lines to turn off LDO2 would be:

U-Boot# i2c mw 24 b 6b
U-Boot# i2c mw 24 16 fe

The first memory write is the password unprotecting, the second
disables LDO2 in the ENABLE register. Try it from the u-boot command
line. Then just put those two commands into a uEnv.txt and make sure
they get executed. The 'U-Boot# ' part is the prompt.

Decoding the command:
i2c - i2c command set
mw - memory write
24 - i2c address of the tps65217
b or 16 - register to write to in the tps65217
6b or fe - data to write to said register

The 'i2c help' is useful in u-boot, too.

Tested on an A3 that's now running with the power LED off (and all
other LEDs off too, since it's not on Ethernet and I'm at the
u-boot prompt) with mainline u-boot 2013.01-rc1. Should also work fine
with the version of u-boot shipping with any bones.

-Andrew

#18

Correction, A3 had the orange Ethernet LED still on in my testing, cause
it's an A3 and the Ethernet LEDs do the wrong thing.

-Andrwe

#19

I've withdrawn my patch. It doesn't fix the problem of inverted
Ethernet LEDs with some (admittedly custom) capes that use
SYSRESET_n as a reset attached on some A6 bones.

I still believe the root cause is the reset duration along with the PHY
configuration done by u-boot. The reason the reset duration is so
short is because the reset circuit is using the 24 MHz clock as its
input clock (due to boot pin pull resistors). This leads to reset
assertion measured in hundreds of nano seconds or single digit micro
seconds. The SMSC PHY wants a minimum of 100 usec reset duration.

Regarding the use of the nRESETIN_OUT line, the AM335x TRM states on
page 534:
"It is recommended that this signal be used as input only (do not
connect to other devices as a reset) to implement a push button reset
circuit to the AM335x, or an output only to be able to reset other
devices after an AM335x reset completes."

The BeagleBone does not abide by this recommendation. BeagleBone uses
nRESETIN_OUT as both an input for a push-button warm reset and as an
output to reset other devices on board during a warm reset.
But reading through how nRESETIN_OUT works, if it's used as a
push-button to reset the AM335x along with any other devices on the
board (such as the PHY, so you can have just one reset button) it seems
unavoidable that nRESETIN_OUT gets used as both an input and an
output. This conflicts with the TRM recommendation cited above!

It seems that according to the AM335x TRM, if you want to use a
push-button reset input that an external reset controller should be
used in order to abide by the recommendation cited above. It also
seems that in order to have external devices (like a PHY) held in reset
for their minimum reset duration (100 usec in the SMSC PHY case) that
an external reset controller is required as the maximum duration that
nRESETIN_OUT can be asserted for is approximately 11 usec due to the
one byte size of RSTTIME1. The ideal case would be an external reset
controller that took the input of the push button and then proceeded to
hold all on-board devices except the AM335x in reset for the longest of
all devices being reset's minimum duration while holding the AM335x in
reset for that duration plus a small amount so that the AM335x does not
start executing until all on-board devices have been released from
reset. This adds cost and board space and thus I understand why the
BeagleBone did not implement it. I don't believe the use of an
external reset controller is the intention of TI.

Am I making any sense?

In the mean time, I'm going to see if I can "fix" the way u-boot
configures the PHY so as to ensure proper LED orientation.

Thanks,
Andrew

#20

I don't believe I can "fix" this via SMSC PHY registers. It appears
that the only way to set the SMSC PHY LED operation is with the
strapping resistors. Thus, it would be nice if the SYSRESET_n would
stay asserted for > 100 usec. This seems to require a hardware change
in order to provide the minimum reset duration for the SMSC PHY.

It should be possible to extend the SYSRESET_n duration via a circuit
on a cape. If we're successful implementing that, I'll share.

Thanks,
Andrew