PRUSS extRAM vs ddrMem; are they different memories or both part of the 500Mbytes DDRL3?

I am trying to figure out memory mappings in the PRU, and especially ddrMem vs extRAM, and although there is information here and there, I have yet to find a cohesive explanation. I have found a few things to share, and perhaps someone more knowledgeable can help fill in some unanswered questions. First, a small caution, the Makefiles in the individual directories of TI’s three examples in their am335x_pru_package are missing the -V3 flags that should be passed to pasm_2; they are not missing in the Makefile one level up, since I used the lower level Makefiles, this error lost me almost a week. Irrespective, re memory.

It appears to me that ddrMem accessed at 0x80000000 in PRU space is different than extRAM that is accessed at an address in PRU space found from >cat /sys/class/uio/uio0/maps/map1/addr which for me gives 0x9f2c0000. The extRAM size is easily found using >cat /sys/class/uio/uio0/maps/map1/size (0x40000 for me as I haven’t increased it yet). However, I can’t find anywhere discussion on what the size of ddrMem can be safely used. In user space I map ddrMem in userspace using:

mem_fd = open("/dev/mem", O_RDWR);
/* map the DDR memory /
ddrMem = mmap(0, 0x0FFFFFFF, PROT_WRITE | PROT_READ, MAP_SHARED, mem_fd, DDR_BASEADDR);
if (ddrMem == NULL) {
printf(“Failed to map the device (%s)\n”, strerror(errno));
close(mem_fd);
return -1;
}
ddrMem_int = (unsigned int
) ddrMem;
memset(ddrMem, 0, 0x2000);

where DDR_BASEADDR is 0x80000000. This is based on the TI example. It appears the size of the memory is 0x10000000 or about 256Mbytes. However, if one changes the memset to memset(ddrMem, 0, 0x0FFFFFFF); (DO NOT DO THIS), it renders the BBB unuseable; it needs to be powererd down, disconnected from power, and then powered back up before it is useable. It would be nice to know how ddrMem could be used. I am working on figuring out access speeds to each, and from various places, it appears extRAM is always at memory locations where it can be accessed using Direct Memory Access. Can someone tell me the maximum size that can be used when loading uio_pruss; I saw it somewhere, but can’t find the posting again? Also, it would be nice if the size could be set in a uio_pruss device tree overlay at boot time; I can’t find info re this at all. Accessing ddrMem using SBCO seems to take 4 cycles for the first register, and then one cycle per additional register; that is the command:
SBCO r0, CONST_DDR, PARAM_OFFSET+0, 32 takes 11 cycles; I was expecting 9 cycles (SBCO and SBBO commands normally take 2 cycles for the first register and 1 for each additional register, as opposed to LBCO and LBBO commands which take 4 cycles for the first register, and 1 cycle per additional register - it sure would be nice if TI gave us a table for disassembling the op codes and how many cycles per code). I don’t understand yet where I am losing the 2 cycles. Just for some more info re memory mapping, noting the pid of my user space process is 1354, then doing sudo cat /proc/1354/maps gives:

00008000-0000a000 r-xp 00000000 b3:02 14116 /home/martin/programming/pru/pru_test/logic_test
00012000-00013000 rw-p 00002000 b3:02 14116 /home/martin/programming/pru/pru_test/logic_test
00013000-00034000 rw-p 00000000 00:00 0 [heap]
a6d00000-a6d40000 rw-s 00001000 00:05 5333 /dev/uio0
a6d40000-b6d40000 rw-s 80000000 00:05 2292 /dev/mem
b6d40000-b6d80000 rw-s 00001000 00:05 5333 /dev/uio0
b6d80000-b6e00000 rw-s 00000000 00:05 5333 /dev/uio0
b6e00000-b6e40000 rw-s 00001000 00:05 5333 /dev/uio0
b6e40000-b6ec0000 rw-s 00000000 00:05 5333 /dev/uio0
b6ec0000-b6f98000 r-xp 00000000 b3:02 34894 /lib/arm-linux-gnueabihf/libc-2.13.so

and so on. The pru base addresses are b6d80000-b6e00000 and the addresses of the ddrMem are a6d40000-b6d40000, but again do not assume you can use all of these. There are also a number of mappings that I don’t understand (such as b6d40000-b6d80000, b6e00000-b6e40000, and b6e40000-b6ec0000).

Finally, I have had some success with including:

#include “/home/martin/programming/pru/am335x_pru_package/pru_sw/app_loader/interface/__prussdrv.h”

extern tprussdrv prussdrv;

to get at the prussdrv struct using gdb. This is quite helpful in understanding some of the PRU addresses.

Also, including
#include “logic_addresses.h” // for PRU relative addresses used in debug

where logic_addresses.h has:

#define PDATA0 0x00000
#define PINST0 0x34000
#define PCNTL0 0x22000
#define PDBUG0 0x22400
#define PDATA1 0x02000
#define PINST1 0x38000
#define PCNTL1 0x24000
#define PDBUG1 0x24400
#define PBASE 0x00000
#define PINSTC 0x20000
#define PIEP 0x2E000
#define PSHARE 0x10000

static unsigned int pdata0 = PDATA0;
static unsigned int pinst0 = PINST0;
static unsigned int pcntl0 = PCNTL0;
static unsigned int pdbug0 = PDBUG0;
static unsigned int pdata1 = PDATA1;
static unsigned int pinst1 = PINST1;
static unsigned int pcntl1 = PCNTL1;
static unsigned int pdbug1 = PDBUG1;
static unsigned int pbase = PBASE;
static unsigned int pinstc = PINSTC;
static unsigned int piep = PIEP;
static unsigned int pshare = PSHARE;

to allow one to get at the debug registers using gdb in user space (they are not accessible in PRU space as far as I can tell - if anyone knows how to get at them, please share it with us).

It appears TI is not interested in supporting the PRU on BBB:

"Hi,

The PRU support package that you refer is not developed or supported by TI. You can ask for information about it at http://beagleboard.org/Community/Forums"

so we are pretty well on our own; it would be a shame if after spending considerable time to understand and use the PRU’s TI decided to drop them - we can only hope this would not be the case. It might be nice if the PRU info could be collected into one location or wiki, as currently it is spread over many different locations with much of the info in forum posts being simple one liners (with the notable exception being Derek Malloy’s web site http://exploringbeaglebone.com/chapter13). My background is hardware, so I wouldn’t know how to set a wiki up, but if someone could look after it, I would be able to contribute. Anyways, thanks for any help that experienced PRUers can give, especially with respect to a uio_pruss device tree overlay and the usable size of ddrMem.

For any interested, I found the 2 lost cycle counts; I had changed my cycle-counting macros to always load PCNTL0 for both PRUS using a MOV op code which added 2 cycles of over-head I hadn’t corrected for. The
SBCO r0, CONST_DDR, PARAM_OFFSET+0, 32 does take 9 cycles as expected. If the ddrMem was big enough, it would be an option for sending data quickly to user space.

I have been able to figure out some of the memory available and mapping for the PRUSS.

ddrMem and extRAM are both part of the 500MByte DDR memory on the L3 buss. This memory has a physical location starting at 0x80000000 and goes to 0x9fefffff. It can be seen by doing >sudo cat /proc/iomem (a very useful command for checking physical addresses). The last three lines for my BBB are:

80000000-9fefffff : System RAM
80008000-807fda2f : Kernel code
8083c000-8093d13f : Kernel data

notice that the kernel code starts at 0x80008000 so one certainly can’t write at the that address or above. Using memset and many trials after mapping my ddrMem, I found a memset(ddrMem, 0, 0x6000); didn’t seem kill my BBB, but a memset(ddrMem, 0, 0x7000) did; I might suggest to be safe one should restrict ddrMem to 0x4000 (16 kBytes) until TI or beagleboard.org are kind enough to tell us what the actual limit is. As far as I can tell, one can write into this memory as fast as one can write into extRAM (please correct me if this is incorrect).

extRAM is near the top of the ddrMem addressing range. I believe it is normally in the range of DMA access, but I can’t find much about this. Its address can be found using

sudo cat /sys/class/uio/uio0/maps/map1/addr (which for me now gives 0x9d000000)
and its size can be found using
sudo cat /sys/class/uio/uio0/maps/map1/size (which for me now gives 0x800000 as I increased the default)
It’s size can be changed by either unloading and reloading the module uio_pruss to change from the default, or as found by Derek Malloy, uio_pruss.c can be recompiled with a different default. If the new uio_pruss.ko file is copied over /lib/modules/3.8.13-bone70/kernel/drivers/uio/uio_pruss.ko (for me) then the default will be used after rebooting so the module doesn’t need to be reloaded each time. Note that for 3.8.13-bone70 should be replaced by whatever you get from >uname -r for your BBB. The maximum size for extRAM is 0x800000 (8.39Mbytes or 2.2Mwords). An example of using this memory as a buffer was written by Derek Malloy and can be found at: http://exploringbeaglebone.com/chapter13 .

Summary: use extRAM for buffers, keep ddrMem for passing parameters back and forth to user space or for small amounts of data.
This took me considerable time to find out; it really should have been explained by the experienced PRUers; I’m hoping this explanation will save you others the time I lost.

One note of caution: PRUSS seem to be able to write anywhere in memory so it is easy to kill a BBB; so far powering down and re-booting has always brought me back, but I am seeing some funny behaviour. Getting back on line seems faster with a wireless connection compared to USB or a WiFi connection.