Beagle-X15, Debian and reproducible-builds

A linux kernel supportting BeagleBoard-X15 is now in Debian stretch
(Debian's upcoming stable release), with support for USB, eMMC, uSD and
ethernet. Unfortunately, eSATA doesn't seem to work when compiled as a
module, although I have it working as a built-in with linux 4.5-rc6.

BeagleBoard-X15 is enabled in debian's u-boot packages, although there
is still considerable work needed send the patches mainline.

Debian-installer images are built, but still require additional modules
to be enabled in the udebs (special packages used only for
debian-installer), and I've filed a bug report with a patch, but will
need to do a little more work to actually get the patch accepted:

  https://bugs.debian.org/815848
  https://d-i.debian.org/daily-images/armhf/daily/netboot/SD-card-images/

I've been running it as a build machine for two and a half weeks as part
of the https://tests.reproducible-builds.org infrastructure. It has
built over 3000 packages in that time. Of the 18 build machines, it's
the sole TI based processor.

  https://jenkins.debian.net/munin/debian.net/bbx15-armhf-rb.debian.net/index.html

The BeagleBoard-X15 is keeping reasonable pace with some quad-core and
octa-core systems with 2GB of ram, as well as quad-core systems with 4GB
of ram:

  https://jenkins.debian.net/view/reproducible/job/reproducible_nodes_info/lastBuild/console

It has crashed a couple times, possibly due to eSATA issues. It's
currently running 4.5-rc6, so plan to upgrade the kernel soon.

The fact that it doesn't boot after power cycle without manually
pressing the power button (or hacking the hackware, as discussed on this
list earlier) does take it's toll on uptime, and prevents methods such
as powercycle on ping failure. Someone on irc suggested using the jtag
pins as a way to remotely power cycle it, but I haven't delved into that
yet.

It's a bit tricky with only one board to do u-boot and debian-installer
testing while still running as part of the reproducible-builds
infrastructure, but it has worked out alright for the most part.

Thanks everyone!

live well,
  vagrant

A linux kernel supportting BeagleBoard-X15 is now in Debian stretch
(Debian's upcoming stable release), with support for USB, eMMC, uSD and

Weeee.. thanks.

ethernet. Unfortunately, eSATA doesn't seem to work when compiled as a
module, although I have it working as a built-in with linux 4.5-rc6.

weird. I could swear that omap2plus_defconfig did work with eSATA last
I tested... probably built in though..

Could you file a https://bugzilla.kernel.org/ bug with details and
.config involved?

Ccying l-o to keep list informed.

Hi Vagrant,

ethernet. Unfortunately, eSATA doesn't seem to work when compiled as a
module, although I have it working as a built-in with linux 4.5-rc6.

weird. I could swear that omap2plus_defconfig did work with eSATA last
I tested... probably built in though..

Could you file a https://bugzilla.kernel.org/ bug with details and
.config involved?

Haven't created a bugzilla account yet...

I just tired with omap2plus_defconfig with CONFIG_ATA=m and CONFIG_SATA_AHCI_PLATFORM=m
on v4.5 + u-boot 2016.03 and eSATA worked fine.

I need to know the exact .config that you used to debug the issue. Thanks.

Thanks for following up. I recently tried with 4.6-rc1 and the attached
config, which is based on Debian's multiplatform builds of the kernel.

I'm also starting to suspect some of the patches applied to u-boot
2016.03 might be interfering, which were forward-ported from ti's
u-boot; will try without those u-boot patches applied.

live well,
  vagrant

config-4.6.0-rc1 (175 KB)

[...]

I'm also starting to suspect some of the patches applied to u-boot
2016.03 might be interfering, which were forward-ported from ti's
u-boot; will try without those u-boot patches applied.

Umm.. you should not be needing to patch upstream u-boot at all. we are
posting all support patches upstream anyways. best at this point, from
upstream kernel perspective is to stay on pure upstream u-boot master..
things should eventually settle down in upstream u-boot eventually, but
not yet.

Vagrant,

Hmm... would'nt this break sata u-boot support as a result? not to
mention setup a dependency between bootloader and kernel (not that it
is not already there..)...

I'm also starting to suspect some of the patches applied to u-boot
2016.03 might be interfering, which were forward-ported from ti's
u-boot; will try without those u-boot patches applied.

Umm.. you should not be needing to patch upstream u-boot at all.

Ok, so I tried 2016.03, and that booted fine. Athough eSATA still didn't
work when compiled as a module.

we are posting all support patches upstream anyways. best at this
point, from upstream kernel perspective is to stay on pure upstream
u-boot master.. things should eventually settle down in upstream
u-boot eventually, but not yet.

Tried u-boot master, and it just hangs:

  U-Boot SPL 2016.03 (Mar 31 2016 - 17:15:32)
  DRA752 ES1.1
  Trying to boot from MMC2_2

Unfortunately, I tested booting from eMMC, and now need to figure out
how to get it to boot from SD... or some way of reverting back to a
working u-boot SPL on eMMC.

live well,
  vagrant

testing latest master: v2016.03-523-g080c499df689

Looks like SD card with fat partitions boots u-boot:
http://pastebin.ubuntu.com/15571915/ with a newer rev of board.

http://pastebin.ubuntu.com/15571902/ with the A.2 rev of the board.

Attached should be a boot log on the BeagleBoard-X15 booting with u-boot
master (080c499df689e8c42df70de44502c0d71533dda8) and linux 4.6-rc1 with
the config provided earlier in the thread.

No luck with Roger Quadros's patch removing the extra init_sata call in
u-boot, although I only tested that with u-boot 2016.03.

Since it's possibe the initramfs doesn't have all the necessary modules
to get eSATA working, I'll try testing with rootfs on microSD or eMMC
later today or tomorrow, so I can be confident it has all the modules
the kernel was built with when debugging the eSATA issues.

live well,
  vagrant

boot-4.6-rc1.log.txt.gz (9.25 KB)

Why would it break sata u-boot support? init_sata() is being done already
as part of scsi_init() as mentioned in the patch commit message.

cheers,
-roger

we are posting all support patches upstream anyways. best at this
point, from upstream kernel perspective is to stay on pure upstream
u-boot master.. things should eventually settle down in upstream
u-boot eventually, but not yet.

Tried u-boot master, and it just hangs:

  U-Boot SPL 2016.03 (Mar 31 2016 - 17:15:32)
  DRA752 ES1.1
  Trying to boot from MMC2_2

Unfortunately, I tested booting from eMMC, and now need to figure out
how to get it to boot from SD... or some way of reverting back to a
working u-boot SPL on eMMC.

testing latest master: v2016.03-523-g080c499df689

Looks like SD card with fat partitions boots u-boot:
Ubuntu Pastebin with a newer rev of board.

Ubuntu Pastebin with the A.2 rev of the board.

Attached should be a boot log on the BeagleBoard-X15 booting with u-boot
master (080c499df689e8c42df70de44502c0d71533dda8) and linux 4.6-rc1 with
the config provided earlier in the thread.

No luck with Roger Quadros's patch removing the extra init_sata call in
u-boot, although I only tested that with u-boot 2016.03.

Since it's possibe the initramfs doesn't have all the necessary modules

you would need "phy_ti_pipe3" along with the AHCI and scsi modules.

to get eSATA working, I'll try testing with rootfs on microSD or eMMC
later today or tomorrow, so I can be confident it has all the modules
the kernel was built with when debugging the eSATA issues.

That is a good idea.

However SATA seems to be failing right at u-boot level from your log.
That doesn't seem right.

2016-03-31_22:45:17.40524 SATA link 0 timeout.
2016-03-31_22:45:17.40530 AHCI 0001.0300 32 slots 1 ports 3 Gbps 0x1 impl SATA mode
2016-03-31_22:45:17.40534 flags: 64bit ncq stag pm led clo only pmp pio slum part ccc apst

Can you please try to power cycle board and hard drive always till we
have got the eSATA issue figured out?

cheers,
-roger

[..]

diff --git a/board/ti/am57xx/board.c b/board/ti/am57xx/board.c
index 042f9ab..34c5161 100644
--- a/board/ti/am57xx/board.c
+++ b/board/ti/am57xx/board.c
@@ -264,7 +264,6 @@ int board_init(void)

int board_late_init(void)
{
- init_sata(0);
   /*
    * DEV_CTRL.DEV_ON = 1 please - else palmas switches off in 8 seconds
    * This is the POWERHOLD-in-Low behavior.

Hmm... would'nt this break sata u-boot support as a result? not to
mention setup a dependency between bootloader and kernel (not that it
is not already there..)...

Why would it break sata u-boot support? init_sata() is being done already
as part of scsi_init() as mentioned in the patch commit message.

OK, I missed that..

testing latest master: v2016.03-523-g080c499df689

Looks like SD card with fat partitions boots u-boot:
Ubuntu Pastebin with a newer rev of board.

Ubuntu Pastebin with the A.2 rev of the board.

Attached should be a boot log on the BeagleBoard-X15 booting with u-boot
master (080c499df689e8c42df70de44502c0d71533dda8) and linux 4.6-rc1 with
the config provided earlier in the thread.

No luck with Roger Quadros's patch removing the extra init_sata call in
u-boot, although I only tested that with u-boot 2016.03.

Since it's possibe the initramfs doesn't have all the necessary modules

you would need "phy_ti_pipe3" along with the AHCI and scsi modules.

Yeah, that's present and loaded, even. along with ocp2scp and
phy_omap_control, the others that usually seem to be required as far as
I can tell.

to get eSATA working, I'll try testing with rootfs on microSD or eMMC
later today or tomorrow, so I can be confident it has all the modules
the kernel was built with when debugging the eSATA issues.

That is a good idea.

Unfortunately, even with rootfs on eMMC, with all the modules built with
the kernel available, it still doesn't detect the eSATA SSD.

However SATA seems to be failing right at u-boot level from your log.
That doesn't seem right.

2016-03-31_22:45:17.40524 SATA link 0 timeout.
2016-03-31_22:45:17.40530 AHCI 0001.0300 32 slots 1 ports 3 Gbps 0x1 impl SATA mode
2016-03-31_22:45:17.40534 flags: 64bit ncq stag pm led clo only pmp pio slum part ccc apst

I've not seen SATA work from mainline u-boot on the BeagleBoard-X15.

Can you please try to power cycle board and hard drive always till we
have got the eSATA issue figured out?

Power cycling didn't help with my most recent test, using linux 4.6-rc2
and u-boot 2016.03 (with a small patch to support
config_distro_bootcmd).

The attached kernel config with linux 4.6-rc2 doesn't work with
OMAP_OCP2SCP, OMAP_CONTROL_PHY and TI_PIPE3 as modules, but works as
builtins.

I did at one point get the impression that the eSATA port provides power
off of the USB rail... perhaps there is an ordering issue with bringup
of USB vs. eSATA when loaded as a module?

config-4.6.0-rc2.gz (42.5 KB)

testing latest master: v2016.03-523-g080c499df689

Looks like SD card with fat partitions boots u-boot:
Ubuntu Pastebin with a newer rev of board.

Ubuntu Pastebin with the A.2 rev of the board.

Attached should be a boot log on the BeagleBoard-X15 booting with u-boot
master (080c499df689e8c42df70de44502c0d71533dda8) and linux 4.6-rc1 with
the config provided earlier in the thread.

No luck with Roger Quadros's patch removing the extra init_sata call in
u-boot, although I only tested that with u-boot 2016.03.

Since it's possibe the initramfs doesn't have all the necessary modules

you would need "phy_ti_pipe3" along with the AHCI and scsi modules.

Yeah, that's present and loaded, even. along with ocp2scp and
phy_omap_control, the others that usually seem to be required as far as
I can tell.

to get eSATA working, I'll try testing with rootfs on microSD or eMMC
later today or tomorrow, so I can be confident it has all the modules
the kernel was built with when debugging the eSATA issues.

That is a good idea.

Unfortunately, even with rootfs on eMMC, with all the modules built with
the kernel available, it still doesn't detect the eSATA SSD.

However SATA seems to be failing right at u-boot level from your log.
That doesn't seem right.

2016-03-31_22:45:17.40524 SATA link 0 timeout.
2016-03-31_22:45:17.40530 AHCI 0001.0300 32 slots 1 ports 3 Gbps 0x1 impl SATA mode
2016-03-31_22:45:17.40534 flags: 64bit ncq stag pm led clo only pmp pio slum part ccc apst

I've not seen SATA work from mainline u-boot on the BeagleBoard-X15.

Can you please try to power cycle board and hard drive always till we
have got the eSATA issue figured out?

Power cycling didn't help with my most recent test, using linux 4.6-rc2
and u-boot 2016.03 (with a small patch to support
config_distro_bootcmd).

The attached kernel config with linux 4.6-rc2 doesn't work with
OMAP_OCP2SCP, OMAP_CONTROL_PHY and TI_PIPE3 as modules, but works as
builtins.

OK. This is the magic configuration that is needed for SATA to work.
When earlier I said that TI_PIPE3 must be built in, that would imply that
OMCP_OCP2SCP and OMAP_CONTROL_PHY will be built in due to module dependency.

I did at one point get the impression that the eSATA port provides power
off of the USB rail... perhaps there is an ordering issue with bringup
of USB vs. eSATA when loaded as a module?

We don't expect eSATA to provide power to the SATA drive. You will have to
power it externally.
Maybe that is the issue why it isn't working at u-boot and the dependency
with USB driver?

cheers,
-roger

Correcting my earlier statement. We do support eSATA drives that can run
@5V/1A power.

I've send the below patch to linux-omap and that reduces USB dependency
on EXTCON and might fix your issue as well.

The attached kernel config with linux 4.6-rc2 doesn't work with
OMAP_OCP2SCP, OMAP_CONTROL_PHY and TI_PIPE3 as modules, but works as
builtins.

OK. This is the magic configuration that is needed for SATA to work.
When earlier I said that TI_PIPE3 must be built in, that would imply that
OMCP_OCP2SCP and OMAP_CONTROL_PHY will be built in due to module dependency.

I did at one point get the impression that the eSATA port provides power
off of the USB rail... perhaps there is an ordering issue with bringup
of USB vs. eSATA when loaded as a module?

...

Correcting my earlier statement. We do support eSATA drives that can run
@5V/1A power.

I've send the below patch to linux-omap and that reduces USB dependency
on EXTCON and might fix your issue as well.

Thanks! Unfortunately, no such luck when built as modules with the
debian config with mainline linux 541d8f4d59d79f5d37c8c726f723d42ff307db57:

  [ 5.195698] ahci 4a140000.sata: couldn't get PHY in node sata: -517
  [ 5.207222] ahci 4a140000.sata: SSS flag set, parallel bus scan
  disabled
  [ 5.207244] ahci 4a140000.sata: AHCI 0001.0300 32 slots 1 ports 3
  Gbps 0x1 impl platform mode
  [ 5.207251] ahci 4a140000.sata: flags: 64bit ncq sntf stag pm led clo
  only pmp pio slum part ccc apst
  [ 5.229994] ata1: SATA max UDMA/133 mmio [mem 0x4a140000-0x4a1410ff]
  port 0x100 irq 321
  [ 15.231005] ata1: softreset failed (1st FIS failed)
  [ 25.234595] ata1: softreset failed (1st FIS failed)
  [ 60.238725] ata1: softreset failed (1st FIS failed)
  [ 60.243852] ata1: limiting SATA link speed to 1.5 Gbps
  [ 65.414439] ata1: softreset failed (device not ready)
  [ 65.419747] ata1: reset failed, giving up

Will test if it still works as a built-in later, and linux-next for good
measure.

live well,
  vagrant