Large Arrays in DDR via PRU. Does prussdrv_map_extmem() always give contiguous physical addresses?

Hi,

I’ve got PRU0 of my BBB pulling in 12 Bytes of data from a ADC with a sampling frequency of 31250hz.

I want to stash all of this data in an array that lives in DDR memory. I would like to make the array 0x4000 bytes (16kB) large. This would give me 1365 12 byte records and about 43ms of data to play with.

I’m currently using prussdrv_map_extmem() to hook me up with the DDR memory. It provides me with a memory block that is 256kB, so I have plenty of room.

The question I have is… can I count on the physical address in this 256kB block of memory to be contiguous?

I’m pretty new to linux memory management, but I see that the “page size” is only 4kB… so I imagine that the virtual memory system might chop my 256kB block up into bits which might not necessarily have contiguous addresses.

If I can count on contiguous addresses, this radically simplifies the memory access code in the PRU. If I can’t… well, then I can’t, and I’ve got to figure something else out.

When I run experiments I see that the physical addresses are contiguous, but are they guaranteed to be? or has this just been “lucky”?

Thanks,

Bill

When I run experiments I see that the physical addresses are contiguous, but are they guaranteed to be? or has this just been “lucky”?

If memory serves, I think they have to be. But, if as you say you’ve tested and they’ve come up contiguous already, then I’m very sure this can not change. Not unless they memory is somehow freed using free(), and I do not think that is possible . . .

I have been using the BeagleBone PRU and have been accessing and mapping memory this way and thus far it seems as though it is one contiguous address. I am relatively new to Beaglebone so maybe one of the more experienced may want to step in on this one.

Once I have used prussdrv_get_phys_addr to get the physical address it seems to store to one contiguous memory and is limited to 8mb.

I believe it is allocated contiguous from uio_pruss as well.

If you dig through the user-mode uio_pruss code, you will see that the
PRU itself, the PRU memory and control registers, and the large shared
buffer is all accessed via a mmap on the /dev/uio_pruss* device.

If you dig in the PRU kernel driver, you will see the dram shared
buffer is allocated and handled as DMA memory. The
dma_alloc_coherent() call used to allocate the memory grabs a
physically contiguous chunk, provides physical and virtual addresses,
and sets the page caching flags as needed for the architecture.

Charles,

Thanks so much for a definitive answer to this question!!!

Perhaps even more than the answer, I am fascinated by the account you give of how you know this to be the case!

I would like to have a better understanding of kernel drivers and DMA memory. As an educational exercise I would like dig through the PRU kernel driver and see this code for myself?

Could you tell me where I can find it?

Thank you very much.

Bill

Bill, I have been studying this myself, it is complex, and the learning curve looks to be pretty steep to a newbie embedded programmer.
Here’s as clear of an explanation as I have been able to find:

http://processors.wiki.ti.com/index.php/PRU-ICSS_Remoteproc_and_RPMsg

I’ve got the PRU Cape and that combined with the Debian distro and version 4 kernel it is pretty easy to run the example code using the kernel driver. Understanding what is going on is another story. If you go to exploringbeaglebone.com and look in the chapter “Extra Content Linux Kernel Programming” there are 3 articles which guide you through the workings of kernel drivers.

Regards,
Greg

Could you tell me where I can find it?

http://processors.wiki.ti.com/index.php/PRU_Linux_Application_Loader_API_Guide

You can also read through the code . . . but I will mention the last time I was reading through the code, and reading up on this API, the amount of memory used is not limited to 8k or whatever is default as shipped.

Another interesting thing I realized, is that mmap() can map any register through /dev/mem/ so should be easy to access the PRU scratchpad from “userspace”. Meaning, you still have the latency from Linux to memory, but PRU accesses should be much faster. Not to mention having the ability of “broadsiding” from one PRU to the other . . .granted, I’m having a hard time tryign to think of a usecase where that might be useful . . .

I would recommend staying away from remoteproc and rpmsg at least until it’s out of experimental.

RemoteProc and RPMSG work just fine. By experimental, it means the interface may change and currently TI are adding support for PRU/Ethernet which will mean some rework. What this means is that when you update to a newer kernel, you may have to change your code, but the change will probably be trivial.

Regards,
John

remoteproc definitely has some bad behavior. But what I am doing is just as experimental, and no motors (all data transfer) involved.

What is currently considered a reliable methodology for getting the most out of the PRUs?
Also, what about C versus assembler? Will C alone suffice, or is there real need to do some assembler?
I’ve found that there are actually two different assemblers, the older pasm (apparently no longer supported), and the assembler part of clpru.
If you are just starting out in PRU work, should pasm even be considered?

Regards,
Greg

Hi Soapy,

Yeah John always seems to take things personally, or out of context. I have no problem what so ever with anyone using whatever they want. Including the OP. My comment were merely to point out that remoteproc / rpmsg are not finished, have known issues, and are a pain in the backside to initially get working.

So for someone using prussdrv, it is probably a bad idea to even start thinking about using remoteproc / rpmsg. Unless they’re just experimenting . . . where then it could be a good learning experience I suppose. Me . . . I’d rather something were fully functional and well documented before I invest my time into it. remoteproc is neither of these.

remoteproc definitely has some bad behavior. But what I am doing is just as experimental, and no motors (all data transfer) involved.

What is currently considered a reliable methodology for getting the most out of the PRUs?

Whatever works for you. Me, I opted for C in Linux of course, and ASM for the PRU’s because it is the best documented. But I’ve experimented with it all including attempting remoteproc . . .

The assembler I decided to use was the one included with the am335x_pru_package. I have not really spend a lot of time writing PRU assembly yet, but was reading up on the registers, etc, and then got busy with other things. But I did start to understand the prussdrv driver, and API set . . .

Anyway, pick something, and go with it. If you ask me the old way of using the PRU is the best way to go right now. Later, when remoteproc / rpmsg is mainline, and perhaps well documented. Then I personally plan on giving it a more serious look. Right now, I consider it a toy . . .

William, what are you talking about? Why would I take what you say personally? You make these blanket statements about a technology you say you don’t know how to use and recommend that everyone else not use this technology. If you want to stay with a dead technology, that is your call, but there is no reason for anyone to stay away from RemoteProc/RPMSG. Manufacturers other than TI have embraced this technology which open up a range of cores you can interact with, such as DSP, CortexM4, PRU, etc. Yes, I know you told me you have no interest in the x15 so this is probably not important to you and I respect that.

Soapy, I use the drivers from Starterware and adapt them to work on the PRU, so I always use the C compiler. There is a github repo were several of the Starterware drivers have been ported to the PRU. This will save you a bunch of time.

Regards,
John

Hi John, are you referring to the modules/drivers:

pruss_remoteproc.ko
virtio_rpmsg_bus.ko
rpmsg_pru.ko

The above plus some examples are already loaded and built in the latest “testing” images which use kernel 4.
pruss_remoteproc is showing up in lsmod without doing anything at all. The Debian 8.3 based distro boots up with pruss_remoteproc inserted.

If you are referring to something different or in addition to the above, could you please post an example link to github?

Regards,
Greg

William, what are you talking about? Why would I take what you say personally? You make these blanket statements about a technology you say you don’t know how to use and recommend that everyone else not use this technology. If you want to stay with a dead technology, that is your call, but there is no reason for anyone to stay away from RemoteProc/RPMSG. Manufacturers other than TI have embraced this technology which open up a range of cores you can interact with, such as DSP, CortexM4, PRU, etc. Yes, I know you told me you have no interest in the x15 so this is probably not important to you and I respect that.

So, using something consistent , that is well documented, and has been proven to work over the last several years does not make it “dead tech”. It makes it something that actually works for many people. No one cares what TI adopts, except perhaps you, and TI. People care what works, with the least amount of resistance.

So hey lets put the squash on this right now. Why don’t you write us some code in the next day or two that blinks the USR leds in some kind of pattern that proves the remoteproc / rpmsg is actually functional / usable. As no one cares if you can write 100 "hello world " messages into /var/log/messages . . . I can do that with a bash script and no PRU . . .

You do that, and I’ll concede that remoteproc is at least semi useful.

http://processors.wiki.ti.com/index.php/PRU_Training:_Hands-on_Labs#LAB_6:_Blinking_LEDs_with_RPMsg_from_Linux_User_Space

Hi William, please see the above. I have the PRU Cape, but I haven’t got this far in the labs. The other labs with remoteproc and other associated modules works so far.

Regards,
Greg

Not to add add data points to the flames but -

That example code in that link works fine. Was able to use that sample code to
build a rpmsg simulator for other hw using the PRU. The PRU firmware creates 2
rpmsg channels that gets messages sent between then. Setup is:
Kernel driver A is a I2C driver attaches to one rpmsg channel. It reads data
from the I2C channel and forwards it to the PRU using that rpmsg channel.

Kernel driver B is rpmsg/IIO driver that attaches to the other rpmsg channel
and sends out IIO data based on the rpmsg packets from the PRU.

Having used both UIO_pruss and remote proc/rpmsg -
- remoteproc/rpmsg is a lot easier to use with standard kernel interfaces
(i.e. tying in I2C, etc).
- remoteproc/rpmsg really works better with the C compiler which adds a layer
of overhead compared to pure ASM. The UIO_pruss interface gives a raw
processor which simplifies pure ASM code.
- Passing large amounts of data using rpmsg could have considerable overhead.
For example, I would just do just pure remoteproc or stay with UIO for my
PRUSS driven video capture code. But for slower data rates, rpmsg could
simplify things as the buffer management/interrupt/driver attachment code is
done for you.
- Just keep in mind resources are limited on the PRUSS and accessing resources
outside of the PRUSS can be expensive.

Hi William, please see the above. I have the PRU Cape, but I haven’t got this far in the labs. The other labs with remoteproc and other associated modules works so far.

Hi Soapy,

Thanks, but I have been aware of that guide for some time now. The example, blinks a PRU cape LED, not one of the USR leds on beaglebone. That’s problem #1( not everyone wants to buy yet more hardware to explore an experimental concept).

Problem #2 the guide is based on code composer studio. If you can not see the problem with that, then chances are I will probably never convince you it is a problem. For many of us though, a requirement of using CCS is a major problem.

Problem #3, there is no in depth code explanation, just setup “do x.y.z exact steps”, and thats it. So, for me, no code explanation is not really the problem. The problem is no one bothers explaining how the rpmsg mechanism works in conjunction with our specific hardware. That is to say, many of us already know how the hardware works, but how do “WE” control the hardware through this software ? 1 or 2 good examples would go a very long ways here.

Problem #4, as ybeagle3 mentions above, performance. If we’re wanting to use a PRU or two, chances are pretty good we want / need performance. Granted, there are probably ways to tweak rpmsg, but at some point one has to wonder why even bother.

Problem #5, and possibly the biggest turn off for me aside from very little documentation. This software is experimental, incomplete, and has no proven track record. Which means, no one knows how stable the software is right now. Anyone saying they know if full of it. While on the other hand, the UIO PRU based software has been around a good long time, is proven, and is probably in many commercial grade applications. Not to mention scientific applications, etc.

Anyway, I still think remoteproc / rpmsg is a really cool idea - In concept. In reality though it has a very long ways to go, and no telling if it will ever make it passed the experimental stage. Then if it does, how long will it take ?

Another thing. This hardware( beaglebone ) is an open sourced design. So who in their right mind thought it would be a good idea to demonstrate such a cool idea using a closed source toolchain / toolset ? Oh, right. The same company who says they do not support the PRU’s to begin with . . .

Do also keep in mind, that I actualy am PRO TI . . .

Hi Greg,

I’ve pointed this out to William before, but he doesn’t like to use CCSV6, so you are wasting your time :wink:

Hi William,

By dead technology, I mean there is no further development expected. All future efforts will focus on RemoteProc/RPMSG. For someone looking to start development with remote processor communications, why spend the time learning a technology that isn’t going anywhere? Rather, spend time learning to use a technology that will be enhanced and supported in future kernels.

Regards,
John

Hi Greg,

I’ve pointed this out to William before, but he doesn’t like to use CCSV6, so you are wasting your time :wink:

Actually John, I was well aware of that guide long before you ever mentioned it.

Hi William,

By dead technology, I mean there is no further development expected. All future efforts will focus on RemoteProc/RPMSG. For someone looking to start development with remote processor communications, why spend the time learning a technology that isn’t going anywhere? Rather, spend time learning to use a technology that will be enhanced and supported in future kernels.

That’s not for you, or even TI to decide. That is for the community to decide. Especially since TI has pretty much refused to support the PRU since day one. As for what who does when. You seem to conveniently selectively reading what I write in my post. As I’ve said like 500 times already. I do not care what people use. I will however come into a post like this where the OP originally posted in relation to something am335x_pru_package specific. and you or anyone for that matter jumps in suggesting to use remoteproc / rpmsg. Why ?

Because obviously this person has a time investment into the am335x_pru_package already. I’d like to try and save them from chasing yet another rabbit down yet another useless rabbit hole. As many people here use this software in products, school projects, or just something very serious in the hobbyist circles. They do not care what you think about some experimental bit of software that has yet to be proven . . .

You’re like TJP or whatever his name. Everything is a nail, except instead of prusio, or whatever it is, remoteproc is your hammer.