CPU in HYP mode - to enable KVM

BeagleBoard-X15
1

Hi,

I’m trying to enable KVM on the board, but it is not possible, because the CPU works in SVC and not in HYP mode.

I tried to play around with the U-boot code, and defined CONFIG_ARMV7_VIRT and CONFIG_ARMV7_NONSEC, which are needed for the HYP mode.
All would go well, but CONFIG_ARMV7_NONSEC requires the functions “smp_set_core_boot_addr” and “smp_waitloop”, which are not defined for the BBX15. I have looked at the implementations for other boards, but they all are different. Tried to define them as empty functions (as in the Broadcom “bcm_ep” case) but that did not work - the board didn’t even start booting.

Does anyone have any idea or a hint as for how I should define these functions for the BBX15?

Thanks!
Leonid.

2

humm, i wonder if this could help you activate hyp mode
http://lists.denx.de/pipermail/u-boot/2015-January/202920.html

3

Sorry for the very late reply - I was away from the board for over a week.

This patch doesn’t seem to work: after turning the board on, it just turns itself off after a few seconds. Thanks for the lead though! Trying different ideas now. If anyone has a suggestion, I’d love to hear!
By the way, why is the HYP mode not enabled by default? If the CPU supports virtualization…

4

There's a secure monitor call to enter hypervisor mode, see TRM section 33.4

5

Thanks for pointing me to the docs, but isn’t that the same as:

#ifndef CONFIG_OMAP34XX
ENTRY(save_boot_params)
ldr r1, =OMAP_SRAM_SCRATCH_BOOT_PARAMS
str r0, [r1]
mov r0, lr
ldr r12, =0x102
.word 0xe1600070
b save_boot_params_ret
ENDPROC(save_boot_params)
#endif

?
…Which is what I did based on this patch.

6

Thanks for pointing me to the docs, but isn't that the same as:

#ifndef CONFIG_OMAP34XX
ENTRY(save_boot_params)
    ldr r1, =OMAP_SRAM_SCRATCH_BOOT_PARAMS
    str r0, [r1]
    mov r0, lr
    ldr r12, =0x102
    .word 0xe1600070
    b save_boot_params_ret
ENDPROC(save_boot_params)
#endif

it is the correct secure monitor call, but why is there a branch located
after it? it will never get there since it'll jump to r0 in hypervisor mode

...Which is what I did based on this
<http://lists.denx.de/pipermail/u-boot/2015-January/202920.html&gt; patch.

That patch looks okay to me, but a critical difference seems to be that
there the alternative to the smc is "bx lr", which is good since the
smc[r12=0x102] behaves like "bx r0 in hyp mode". If you want to go
somewhere else you'll need to load that destination in r0. Since you have
live registers (at least lr) you'll want to stash those somewhere first,
and make r0 point to a trampoline which restores it before branching to
your intended destination.

The "this is a great place" comment in the patch you linked to is worth
reflecting on: you want to do this call in some place where preferably
there are no live registers at all.

7

Thanks for the explanation, but it is still a bit unclear to me (first time touching any bootloader code). I know that the solution is very simple here, but I can’t get it yet…

I can not apply that patch as-is, because meanwhile that file have changed. Could you please explain how should I go about it?

Thanks!

8

I just inspected the code: it turns out this function is the very very
first thing called by SPL and its purpose it to ensure no register is
live at the end of it, by saving any relevant info the ROM bootloader
may have passed in registers.

They formerly used to call it as a subroutine, but since some platforms
had some need to save the original LR they changed the call and return
to use direct jumps instead:
https://github.com/u-boot/u-boot/commit/e11c6c279d823dc0d2f470c5c2e3c0a9854a640f

Hence, the correct way to port the patch would be:

#ifdef CONFIG_DRA7XX_HYPERVISOR_ON
  adr r0, save_boot_params_ret
  .word 0xe1600070 @ smc #0
#else
  b save_boot_params_ret
#endif

9

I get the impression my post got corrupted or something… in my gmail I see part of an old draft appended while on google groups I only see the old draft…

I was supposed to say:

To add to that, there might be a very remote that save_boot_params_ret is beyond the range of adr. The safe alternative would be:

10

Thanks again for your help. I have made the changes that you can see below, but it still does not work. I assumed that you missed the “ldr r12, =0x102” line, since obviously that value should be loaded there somehow. The result: the board shuts down right after powering on.

ENTRY(save_boot_params)
ldr r1, =OMAP_SRAM_SCRATCH_BOOT_PARAMS
str r0, [r1]

  • b save_boot_params_ret
    +#ifdef CONFIG_DRA7XX_HYPERVISOR_ON
  • ldr r0, =save_boot_params_ret
  • ldr r12, =0x102
  • .word 0xe1600070 @ smc #0
    +#else
  • b save_boot_params_ret
    +#endif
    ENDPROC(save_boot_params)
    #endif
11

I will try and give it a shot later this week or next, but I think a
bunch of registers get clobbered by ROM code and left to us in
non-secure world to protect.

12

Thanks again for your help. I have made the changes that you can see
below, but it still does not work. I assumed that you missed the "ldr
r12, =0x102" line, since obviously that value should be loaded there
somehow.

Yeah sorry

The result: the board shuts down right after powering on.

Then I think it's really time for JTAG

Matthijs

13

There are no live regisers at the point this call is done: r0-r14 may all
be freely clobbered by the monitor call if it wants to.

14

You could also try the original patch against the version of u-boot it was
based off. Assuming he posted a patch that worked, you can then rebase all
changes to u-boot since then on top of that (fixing the conflict around
commit e11c6c2 by changing the call to what you have now) and use git
bisect to figure out what broke it.