Ti's RPMsg Examples Significantly Changed

GoogleGroups
61

I will work to enable uio_pruss functionality, and I think that is what you want, not just getting remoteproc out.

Please don’t do that. Robert Nelson has the “bone” kernel for this purpose which supports pruss_uio by default and does not have RemoteProc/RPMSG installed. The “ti” kernel however does the reverse, with the RemoteProc/RPMSG installed by default and pruss_uio no installed. I believe the two frameworks conflict so it is not possible to have both installed.

It sure seems to me that if both can exist in the source tree and be selected at runtime with configuration (ideally via device tree, switchable later by loading and unloading modules), that would be the ideal solution. It sounds like John is saying this is technically not possible, but I feel like it should be (it is just code, after all).

If you read the discussion, TJF and William don’t want to build a custom kernel. So since you cannot have pruss_uio and RemoteProc/RPMSG, configured simultaneously, this is the dilemma we are facing. Currently Robert Nelson has configured the “bone” kernel to have pruss_uio dn the “ti” kernel to have RemoteProc/RPMSG. William is concerned that the “ti” kernel has more features than the “bone” kernel. Solution is to ask Robert Nelson to add the missing features to the “bone” kernel.

Regards,
John

62

I'm not sure specifically what's preventing the two from being configured simultaneously, so long as both their code doesn't execute simultaneously. It seems one or the other or both can be modified to coexist in the configuration, and that may be the best way to ensure the kernel supports all users.

We have the source, it should be possible. The kernel can support multiple ethernet drivers configured simultaneously, why not multiple PRU drivers?

63

Think about what you are proposing. When the kernel loads, it discovers devices and then loads the appropriate drivers. Now what happens when it discovers the PRU, which driver does it load, PRUSS_UIO or RemoteProc?

Regards,
John

64

#1 It’s not as if Robert doesn’t have enough on his plate already.

#2 Forcing the need for two different release kernels is not the proper way to go about solving this problem.

65

Think about what you are proposing. When the kernel loads, it discovers devices and then loads the appropriate drivers. Now what happens when it discovers the PRU, which driver does it load, PRUSS_UIO or RemoteProc?

Why not give it a configuration choice? Both modules can be optional,
load or not load only one. Develop your application to match.

You already do this with the pin overlays/pin mux.

Can you decouple it from the main program/OS and then have resources
that you need?

Harvey

66

Doesn't it "discover" devices by looking at the device tree?

67

Doesn’t it “discover” devices by looking at the device tree?

For many devices, the kernel is sort of unaware until a device tree is loaded. Now by “unaware” I just mean that the kernel has been compiled with that module as a loadable module ( non static ), and a “hook” has been written into the main board file for specific modules. Which load when the device tree “hook” status has been changed to “Okay”.

But you’ve probably been one or two of these yourself already. But here: https://github.com/beagleboard/bb.org-overlays/blob/master/src/arm/uio_pruss_enable-00A0.dts#L22

As an example.

68

Been, seen. It’s all the same :wink:

69

Doesn't it "discover" devices by looking at the device tree?

Ideally, yes, but Robert said there is a conflict when both frameworks are built in the same kernel. Currently, both frameworks are not built into the kernel and both are configured as Loadable Modules. Also, it should be possible to have a separate Devicetree for each framework, but with the build conflict Robert is unable to do this at this time. My understanding is this is why the “bone” kernel exists. If Robert is able to resolve the build conflict, then is the “bone” kernel even necessary? Remember, the “ti” kernel is not just for the BBB, but also for the BeagleBoard-x15 so the RemoteProc/RPMSG is required to support the DSPs and CortexM4s, so kill the idea that RemoteProc/RPMSG is going away, it just isn’t. There is no pruss_uio equivalent for the DSPs or CortexM4s. Even more important, TI have even bigger processors that have 4 Cortex-A15 ARM cores and 8 DSPs and the same “ti” kernel is used for that processor.

Hopefully Robert can chime in on this issue and clear up this issue.

Regards,
John

70

@John3909:

Why don’t you use the “bone” kernel which pruss_uio as default.

That’s what I do.

The “ti” kernel has RemoteProc/RPMSG as default. I don’t understand you problem here.

There’ll be no more problem when we follow my proposal and remove the remoteproc framework from the mainstream. But in the current situation the problem is to find an answer when a user asks “How to test a ti-rt kernel with libpruio?”.

Look at the BeagleLogic development blog, where he explains the throughput problem with pruss_uio. When he changed to RemoteProc/RPMSG, the throughput increased dramatically.

Why should I spent time on that investigation? Better convince me by telling me: how much do you think your thingy can increase libpruio throughput?

I continue to say that you are using RemoteProc/RPMSG incorrectly.

Obviously you have no idea how libpruio works and you have no experience on the usecase it’s made for. Do you really think you can give a professional opinion on correct or incorrect? I’m sure I use it correct, because I don’t use it at all, since it doesn’t fulfill the requirements.

You shouldn’t have a tight control loop between the PRU and ARM because this makes no sense. Linux is non deterministic so why would you want to compromise the PRU by making it dependent on the communications with Linux. Either use one PRU for the control loop and another for communicating with Linux, or use DMA to pass data between PRU and ARM.

It makes no sense to discuss that in detail here, since you obviously have no idea on rapid prototyping controllers. (I can give you a private lesson if you like.)

These are generally toys. The vast majority of drivers are Kernel based drivers.

I don’t know all the other drivers you stated here. But in case of one-wire your “generally toys” (all four) do the heavy work in real world projects. Because your “vast majority” (I count one) isn’t able to perform a simple broadcast trigger for multiple Dallas sensors, and misses lots of other features like setting limits, … In my experience, and in case of one-wire usecase, the kernel driver is the toy. And there’re much and good reasons why the userspace drivers are under continuous development (and the kernel driver isn’t).

Try a little bit thinking outside your box.

Once you compromise physical security, you have no security, period, so this is a silly point to make. There a many Linux device drivers that rely on firmware and these are all done is a secure way. The purpose of a kernel driver is to validate the user parameters and prevent operations outside well defined limits. Userspace drivers have no such validation and can do whatever they please, hence no security.

My libpruio driver has lots of validations. Ie. you can find many posts at this forum regarding a validation bug, where the maximum sampling rate for ADC sampling was calculated too much on the save side. Do you really think that only kernel code can validate?

But please read my statement again. I’m not talking about physical security, nor about arbitrary periperals. I’m talking about the PRUSS and their firmware, which can access all CPU memory without any kernel protection. I’m talking about software security.

Hence why you want to Kernel based driver to validate the firmware. Again, userspace driver can place whatever code it wants on the PRU.

Right, whatever code it wants. Ie. a virus. And it’s really hard to find a virus running on the PRUSS. Currently, I see no way how a kernel driver should validate if the firmware is free of virus. And when firmware is loaded from files and started by command line, it’s much more easy to run a virus on the PRUSS, in contrast to loading and running it by prussdrv. With remoteproc, it even doesn’t need any command line action: just copy the virus to /lib/firmware and name it am335x-pru$[0|1]-fw, and it will run at next boot time. This is a massive safety risc in the current remoteproc concept. It shouldn’t be that easy to install and run malware.

TI have several processors that have PRU, DSP, CortexM4 in addition to one or more ARM processors. Just look at the BeagleBoard-x15 for example.

How is this related to my statement you qouted?

This framework works for me so I certainly don’t want it removed.

Fine, go ahead. (BTW: which projects did you realise yet?)

Once you read my statements again, you’ll find out that I only want to remove a dangerous software from the mainstream, but not completely. So you can still use your be-loved thingy. Just load it as an option.

I think it would be more productive to make suggestions on how to improve the RemoteProc/RPMSG. BTW, I’m sure you don’t have any problem with RemoteProc, because it is just loading and starting/stopping the firmware on the PRU. PRUSS_UIO has a similar firmware loader. So perhaps we should concentrate on Virtio, vring or RPMSG.

Once again, I’ve problems with remoteproc. And I’ve problems with the framework and its design. It doesn’t make sense to talk about details. In order to make this framework usable, it needs talk about general issues, like

Do we really need PRU access from kernel space?
If so, how to handle the additional riscs?
…

And this needs other than traditional thinking, since riscs regarding the PRUSS are different from riscs handled by any kernel driver before.

But this discussion is pointless, since Suman said that he don’t want to start from scratch. (He may change his mind when he reconsiders the riscs of the current design.)

Once again, why not install the “bone” kernel which has pruss_uio as default. Why do you insist on installing the “ti” kernel which has RemoteProc/RPMSG as default and then insist on removing RemoteProc/RPMSG. This makes no sense to me.

See my answer above. TI kernels do not only differ from remoteproc. Some of my users want to test the real-time features, which aren’t in the bone channel. I try to help them. I try to find a way that prefers corporate development to mutual exclusion by downgrading.

The reason it was added to mainstream was to encourage support by other vendors which has already started.

Obviously too early.

Since pruss_uio is only supported on one platform, it shouldn’t be included in mainstream.

An unsave, experimental framework also shouldn’t be included in mainstream. I’d try to make both optional and choise in device tree when enabling the PRUSS.

BR

71

On the contrary, perhaps you should explain the use case so everyone here can understand what it is you cannot do with RemoteProc/RPMSG. Give some examples of how you do this with libpruio. I just don’t understand the need for communications with Linux in a tight control loop, but I’m hoping you can enlighten us on the issue and hopefully move this discussion forward. Think of this discussion as a cooperative one were everyone should ultimately benefit.

Regards,
John

72

I think there is a LOT of misconception as to what uio_pruss is. Or what any UIO driver is - period.

#1 a UIO driver is kernel side first. There is no userspace side driver with out a UIO kernel driver stub. So, if you create your own kernel side “stub”( as done with uio_pruss ) it can be as complex, or as simple as needed.

#2 I always treat security risks as if an intruder has already gained access to at least a regular user account. When you think of the problem from this angle, you start thinking how it’s best to write programs that are hard to compromise. So, if you’re concerned about uio_pruss being a security risk. Then write your userspace program in two halves. root / superviser service + regular user / worker program. Worker program can NOT do anything the supervisor service will not let it do. Passed that, if an intruder gains access to root. You’re already screwed, so you can relax :wink:

#3 Someone, or some people do not understand how fast userspace can be, when you write proper applications with performance in mind. You can tell me until you’re blue in the face that implementations including, and using /dev/mem/ + mmap() are a bad idea, because of the “reindeer effect”, or whatever. The thing is, I already know what the good, and bad implications of using /dev/mem/ + mmap() are. So why would I need anyone elses advice on it ? Hint: I don’t. If you think it’s a bad idea, then fine. Do not use it. Let the big boys write the performant applications. But do not try and pee on my leg, and tell me it’s raining. Because I do have a brain of my own.

Passed that, sticking to API calls that do not copy data to, and from the kernel. Or otherwise stall an application because of context switching, or any number of reasons - is best practice. But you have to know this to begin with, and then bother in trying to make things work the way you need, or want them to work. Not just toss something together, and complaining when it does not live up to your performance standards. e.g. You should probably consider learning how to write code properly . . . Or let someone else do it.

So, what does /dev/mem/ + mmap() have to do with the PRU’s ? Nothing really. Except if you expect to keep up with programmable high speed peripherals. You’re going to need every bit of speed your application can muster. Then, if you can not trust me when I say that /dev/mem/ + mmap() is easily 100’s or 1000’s times faster than standard libc / POSIX calls / sysfs. Then go experiment on your own to prove to yourself. Me, again, I do not need any advice. I’ve already tested on my own.

73

So reading the UIO docs;

https://www.kernel.org/doc/htmldocs/uio-howto/about.html

"For some hardware that has more than one interrupt source internally, but not separate IRQ mask and status registers, there might be situations where userspace cannot determine what the interrupt source was if the kernel handler disables them by writing to the chip’s IRQ register. In such a case, the kernel has to disable the IRQ completely to leave the chip’s register untouched. Now the userspace part can determine the cause of the interrupt, but it cannot re-enable interrupts. Another cornercase is chips where re-enabling interrupts is a read-modify-write operation to a combined IRQ status/acknowledge register. This would be racy if a new interrupt occurred simultaneously.”

To address this issue, you are required to write a UIO Kernel Module:

https://www.kernel.org/doc/htmldocs/uio-howto/custom_kernel_module.html

Now with reference to handling interrupts:

https://www.kernel.org/doc/htmldocs/uio-howto/adding_irq_handler.html

“If, on the other hand, your hardware needs some action to be performed after each interrupt, then you must do it in your kernel module. Note that you cannot rely on the userspace part of your driver. Your userspace program can terminate at any time, possibly leaving your hardware in a state where proper interrupt handling is still required.”

Regarding /dev/mem, how does a userspace app access /dev/mem? In a secure system, you shouldn’t be running your app as root and there should be no member of the kmem group. This feature is more for hacking and debugging purposes, not serious application development.

Regards,
John

74

@John3909:

It makes no sense to discuss that in detail here, since you obviously have no idea on rapid prototyping controllers. (I can give you a private lesson if you like.)

On the contrary, perhaps you should explain the use case so everyone here can understand what it is you cannot do with RemoteProc/RPMSG. Give some examples of how you do this with libpruio. I just don’t understand the need for communications with Linux in a tight control loop, but I’m hoping you can enlighten us on the issue …

I think Suman did already know what libpruio (and many other high performance PRU projects) need. But here’s a private lesson for you: There is no “need for communications with Linux in a tight control loop”. We need direct memory access between PRU and ARM CPU. No Linux inbetween. We need to bypass all that vring and RPMsg magic. We gain for speed and small memory footprints. We want to save resources for future extensions and we don’t want to spend resources for features that we neither need nor want.

… and hopefully move this discussion forward.

To get this discussion forward, we have to move backwards. Neither Suman nor myself want this. So the best way to solve the conflict is, that we get both of our concepts optional. The user should decide if he wants to drive a Jaguar or a Bentley.

Since the Bentley has an open trunk where any agressor can easy hide a big bomp, it is a MUST-HAVE to get it out of the mainstream. Remeber, PRUSS can access ALL memory. A PRU virus can even override kernel space memory or manipulate kernel drivers. All the safety strategies you explained to William are pointless when a PRU virus overrides your instructions. With network access, a small loader on the PRU could exchange the complete system, auto-started by remoteproc at boot time. And all that vring and RPMsg magic make it even more easy to develop such a malware. Currently, remoteproc is some kind of VCE (virus construction environment), inbuild in the kernel source.

I wonder how you can speak about safety and support such a concept at the same time. If you’re not just declaiming phrases you learned at highschool, if you have a brain of your own and you’re using it, if you really were interested in security, you would second my proposal to make remoteproc optional, and remove it from mainstream.

Think of this discussion as a cooperative one were everyone should ultimately benefit.

Fine that we are at one with this, and you are also ready for corporate development, now.

BR

75

@John3909:

It makes no sense to discuss that in detail here, since you obviously have no idea on rapid prototyping controllers. (I can give you a private lesson if you like.)

On the contrary, perhaps you should explain the use case so everyone here can understand what it is you cannot do with RemoteProc/RPMSG. Give some examples of how you do this with libpruio. I just don’t understand the need for communications with Linux in a tight control loop, but I’m hoping you can enlighten us on the issue …

I think Suman did already know what libpruio (and many other high performance PRU projects) need. But here’s a private lesson for you: There is no “need for communications with Linux in a tight control loop”. We need direct memory access between PRU and ARM CPU. No Linux inbetween. We need to bypass all that vring and RPMsg magic. We gain for speed and small memory footprints. We want to save resources for future extensions and we don’t want to spend resources for features that we neither need nor want.

While I haven’t tried this, I don’t believe there is anything in the RemoteProc/RPMSG framework that prevents you from doing this now. You can still use RemoteProc to load/start/stop your PRU firmware and you don’t have to use virtio or rpmsg, but rather create a new KM to do just this. Suman does this make sense?

… and hopefully move this discussion forward.

To get this discussion forward, we have to move backwards. Neither Suman nor myself want this. So the best way to solve the conflict is, that we get both of our concepts optional. The user should decide if he wants to drive a Jaguar or a Bentley.

Since the Bentley has an open trunk where any agressor can easy hide a big bomp, it is a MUST-HAVE to get it out of the mainstream. Remeber, PRUSS can access ALL memory. A PRU virus can even override kernel space memory or manipulate kernel drivers. All the safety strategies you explained to William are pointless when a PRU virus overrides your instructions. With network access, a small loader on the PRU could exchange the complete system, auto-started by remoteproc at boot time. And all that vring and RPMsg magic make it even more easy to develop such a malware. Currently, remoteproc is some kind of VCE (virus construction environment), inbuild in the kernel source.

So how do you propose to modify the PRU firmware without root access? On the other hand, your libpruio app runs in userspace, so it is easy to swap out that app and bypass all security.

I wonder how you can speak about safety and support such a concept at the same time. If you’re not just declaiming phrases you learned at highschool, if you have a brain of your own and you’re using it, if you really were interested in security, you would second my proposal to make remoteproc optional, and remove it from mainstream.

Your childish antics say more about you and this isn’t helpful in this discussion. Keep the discussion professional and reframe from getting personal. You really are not listening, RemoteProc/RPMSG are here to say because of all the other processors TI support and several other vendors have decided to support this framework also. Nothing you say is going to change that. All you can do is request that it is improved to suite your needs, or you will have to build your own custom kernel without RemoteProc/RPMSG to support libpruio. RemoteProc/RPMSG will be the standard interprocessor communications framework in Linux. Take this chance to influence the development. Everything else is futile.

Think of this discussion as a cooperative one were everyone should ultimately benefit.

Fine that we are at one with this, and you are also ready for corporate development, now.

This is so silly and you should be better than this.

Regards,
John

76

@John3909:

It makes no sense to discuss that in detail here, since you obviously have no idea on rapid prototyping controllers. (I can give you a private lesson if you like.)

On the contrary, perhaps you should explain the use case so everyone here can understand what it is you cannot do with RemoteProc/RPMSG. Give some examples of how you do this with libpruio. I just don’t understand the need for communications with Linux in a tight control loop, but I’m hoping you can enlighten us on the issue …

I think Suman did already know what libpruio (and many other high performance PRU projects) need. But here’s a private lesson for you: There is no “need for communications with Linux in a tight control loop”. We need direct memory access between PRU and ARM CPU. No Linux inbetween. We need to bypass all that vring and RPMsg magic. We gain for speed and small memory footprints. We want to save resources for future extensions and we don’t want to spend resources for features that we neither need nor want.

While I haven’t tried this, I don’t believe there is anything in the RemoteProc/RPMSG framework that prevents you from doing this now. You can still use RemoteProc to load/start/stop your PRU firmware and you don’t have to use virtio or rpmsg, but rather create a new KM to do just this. Suman does this make sense?

So, thinking a little more about this, this new KM would support mmap() so that your userspace app/lib could access this memory directly. Linux is out of the way and yet everything is still secure :wink:

Regards,
John

77

@John3909:

While I haven’t tried this, I don’t believe there is anything in the RemoteProc/RPMSG framework that prevents you from doing this now. You can still use RemoteProc to load/start/stop your PRU firmware and you don’t have to use virtio or rpmsg, but rather create a new KM to do just this. Suman does this make sense?

So, thinking a little more about this, this new KM would support mmap() so that your userspace app/lib could access this memory directly. Linux is out of the way and yet everything is still secure :wink:

Good John, you’re closing up to this discussion. You’re at this point now (three days ago):

[Suman: Right, the basic infrastructure is already there and you would need only to build upon it. For eg., at the moment, all you would need is a small kernel module, use pruss_request_mem_region to gain ownership of PRU memories, export it to userspace and use it however you want. The interrupts would have to go through kernel though. Maybe it is the same module that exports the above desired sysfs interfaces too.]

Am I reading right here? My message is:

  • Reduce resource consumption on a small SoC.
    And your advice is:

  • Add a further module to a bloated framework.

Go ahead! Once you’re on track, it wouldn’t need any “childish antics” any more. I can hardly wait reading your opinion on the PRU virus issues.

BR

78

This is a valid design pattern and you did very impressive things with
it, but it's not compatible with the basic design of Linux hardware
integration: I don't know of a single mainline kernel driver designed
this way. The whole point of a kernel driver is that it abstracts the
hardware and presents the unified abstraction to the rest
(open/read/write/ioctl/skbuf/etc). Of course this costs performance,
and so the development identifies those bottlenecks and tries to come
up with more abstractions like no-copy networking, etc.
The challenge for those abstractions is to provide speed while still
maintaining security, e.g. by disallowing full control of the entire
physical address space. There have to be address space checks on those
kernel-userspace transitions, and they will use up some cycles; you
can optimize them but can't just not do them.

It seems to me that you just don't want the Linux machinery to
intervene: your design does feel like a bare metal, tight control
loop, using Linux mostly for setup, program loading and
administration.

Also, I would be personally obliged to you if you could refrain from
commenting on other people's motivations and abilities, and just keep
focused on the technical points everyone is making. The discussion is
much more productive this way.

79

This message from Przemek didn’t get copied to everyone but I thought he had an interesting prospective to this discussion.

Regards,
John

80

PRUSS can access ALL memory. A PRU virus can even override kernel space memory or manipulate kernel drivers. All the safety strategies you explained to William are pointless when a PRU virus overrides your instructions.

I do not think the strategy I discussed would be pointless. If your worker application was required to speak with a supervisor service, and the supervisor was the only part that had direct access to the PRU. There would be no problem, unless your attacker gained root, at which point you’re already too far gone.

Also, /dev/mem/ + mmap() are identical to the PRU in relation to having direct access to any memory address. Which is why many people have concerns when it is used.