Unable to access custom memory in user space

Hi,

I'm trying to access a memory device (256KB) we added to our OMAP3-
based custom board under Linux. I'm able to access (reads and writes)
it under u-boot without any problems. That memory device is allocated
by the Xilinx Spartan 6 and hooked up to CS2 on the GPMC and mapped to
physical address 0x10000000. Under Linux, I wrote a little user-space
application using mmap to access it. On our previous board which uses
the TI Davinci processor (6446), the application works fine using
mmap. On this OMAP3-based custom board, all I get are 0's on the
reads and writes; no error conditions though. Do I need to "register"
the physical address (0x10000000) with the Linux kernel for mmap to
work correctly? If so, how do I go about doing so? I searched the
kernel and see that arch/arm/mach-omap2/io.c contains virtual and
physical addresses for various memory space regions of the OMAP3; I
assume this is used by kernel drivers only via ioremap. For what I'm
doing, I'm not writing any driver and am just using mmap in a user-
space application to access the memory device. Any advice on what I'm
doing wrong; this area is not-cacheable so I added the "O_SYNC"
option. Here's a snippet of my sample application. Thanks. Regards,
Andy

  volatile unsigned char *p;
  size_t length = 0x10000;
  int prot = PROT_READ | PROT_WRITE;
  int flags = MAP_SHARED;
  int fd = open("/dev/mem", O_RDWR | O_SYNC);
  off_t offset;
  unsigned int offset_addr;
  int index;

  sscanf(argv[1], "%x", &offset_addr);
  offset = (off_t)(offset_addr & 0xffff0000);
  index = offset_addr & 0xffff;

  if (fd < 0) {
        perror("open");
        return 1;
  }

  p = (unsigned char *)mmap(NULL, length, prot, flags, fd, offset);
  if (p == MAP_FAILED) {
        perror("mmap");
        return 4;
  }

  // process p[index]

You might need to add a mem= kernel argument to tell the kernel that
the section of memory is reachable.

You might need to add a mem= kernel argument to tell the kernel that
the section of memory is reachable.

Thanks for the tip; I'll try it when I get back to work on Monday.
However, I thought that parameter was reserved for RAM (DDR2). For
example, if the board has 128MB and I want to reserve 28MB for DSPLINK
amd CMEMK for DSP operation, then "mem=100M" would be passed from u-
boot to only allow the kernel to use 100MB. I'm not sure if you
really meant the "memmap" parameter; for example, for my case (256KB
at 0x10000000), would it be "memmap=256K@256M". According to what I
found by googling, I see the following kernel parameters:

1355 mem=nn[KMG] [KNL,BOOT] Force usage of a specific amount of
memory
1356 Amount of memory to be used when the kernel is not able
1357 to see the whole system memory or for test.
1358 [X86-32] Use together with memmap= to avoid physical
1359 address space collisions. Without memmap= PCI devices
1360 could be placed at addresses belonging to unused RAM.
1361
1362 mem=nopentium [BUGS=X86-32] Disable usage of 4MB pages for
kernel
1363 memory.
1364
1365 memchunk=nn[KMG]
1366 [KNL,SH] Allow user to override the default size for
1367 per-device physically contiguous DMA buffers.
1368
1369 memmap=exactmap [KNL,X86] Enable setting of an exact
1370 E820 memory map, as specified by the user.
1371 Such memmap=exactmap lines can be constructed based on
1372 BIOS output or other requirements. See the memmap=nn@ss
1373 option description.
1374
1375 memmap=nn[KMG]@ss[KMG]
1376 [KNL] Force usage of a specific region of memory
1377 Region of memory to be used, from ss to ss+nn.
1378
1379 memmap=nn[KMG]#ss[KMG]
1380 [KNL,ACPI] Mark specific memory as ACPI data.
1381 Region of memory to be used, from ss to ss+nn.
1382
1383 memmap=nn[KMG]$ss[KMG]
1384 [KNL,ACPI] Mark specific memory as reserved.
1385 Region of memory to be used, from ss to ss+nn.
1386 Example: Exclude memory from 0x18690000-0x1869ffff
1387 memmap=64K$0x18690000
1388 or
1389 memmap=0x10000$0x18690000

You might need to add a mem= kernel argument to tell the kernel that
the section of memory is reachable.

Thanks for the tip; I'll try it when I get back to work on Monday.
However, I thought that parameter was reserved for RAM (DDR2). For
example, if the board has 128MB and I want to reserve 28MB for DSPLINK
amd CMEMK for DSP operation, then "mem=100M" would be passed from u-
boot to only allow the kernel to use 100MB. I'm not sure if you
really meant the "memmap" parameter; for example, for my case (256KB
at 0x10000000), would it be "memmap=256K@256M".

I have not done this, but I remember seeing a thread on DSP that
indicated that the kernel needs TWO mem= args to make the DSP memory
area available to the kernel. IIRC the DSP-specific one had an address
and a size. I might be totally wrong.

*brief search through bookmarks*

This is what I was thinking of

http://www.angstrom-distribution.org/demo/beagleboard/

See the readme below the file listing.

I have not done this, but I remember seeing a thread on DSP that
indicated that the kernel needs TWO mem= args to make the DSP memory
area available to the kernel. IIRC the DSP-specific one had an address
and a size. I might be totally wrong.

*brief search through bookmarks*

This is what I was thinking of

http://www.angstrom-distribution.org/demo/beagleboard/

See the readme below the file listing.

Sorry I tried your suggestion and it still doesn't work. I believe
the mem parameter is only used to tell the kernel what is available to
it for use, what it can manage (for malloc, etc). I don't think you
need to tell the kernel via the boot parameters your "other custom
memory"; the CS settings on the GPMC should resolve the addressing
automatically. Does anyone else have any advice? I wonder if the
kernel is treating this custom memory range as cacheable; however, I
open /dev/mem with O_SYNC to make sure it doesn't cache the area. I'm
going to try to post this in another group or forum.

I have not done this, but I remember seeing a thread on DSP that
indicated that the kernel needs TWO mem= args to make the DSP memory
area available to the kernel. IIRC the DSP-specific one had an address
and a size. I might be totally wrong.

*brief search through bookmarks*

This is what I was thinking of

http://www.angstrom-distribution.org/demo/beagleboard/

See the readme below the file listing.

I tried your suggestion and it still doesn't work. I believe the mem
parameter is used to tell the kernel what memory is available to it
for automatic management (i.e. malloc). For what I'm doing, in which
we added 256KB via a FPGA device on the custom board, I don't think we
need to let the kernel know; the CS settings on the GPMC for this
custom memory should automatically handle access to it under Linux.
Can anyone else help or have any suggestion on what else I need to do
to access that memory under Linux. I'm going to post this issue in
another group or forum. Thanks.

why don’t you hack the kernel to enable memory mapping for the CS? A lot of examples in the kernel. 5 minutes to work

2010/6/21 Andy Ngo <andyngo.work@gmail.com>

I have already verified the mapping is enabled by looking under Linux
at the 7 GPMC CONFIG registers associated with CS2 that it's attached
to. I don't need to "hack" anything because I have verified that arch/
arm/mach-omap2/gpmc.c (function gpmc_mem_init) is "Reserving all
regions that has been set up by bootloader". Hence, the kernel is
registering the memory mapping of any active CS that was set up by u-
boot; no redundant setup is required again. My problem is that I
can't access the data correctly (all 0's) under Linux; under u-boot, I
have no problem.

I believe this is not enough. Take a look at board-overo.c where they enable mapping for lan9221 controller

2010/6/22 Andy Ngo <ndno72-omap@yahoo.com>

Thanks... I see you what you mean. In overo_init_smsc911x, it calls
the following to request resource for the chip select:

  if (gpmc_cs_request(OVERO_SMSC911X_CS, SZ_16M, &cs_mem_base) < 0) {
    printk(KERN_ERR "Failed request for GPMC mem for smsc911x\n");
    return;
  }

I thought I tried this and it still doesn't work; I'll verify it again
tomorrow when I get back to my work lab. When I use /dev/mem and mmap
to access that region, does it use the reserved resource above? I
thought the above was just to reserve the resources for a driver (like
the smc911x).

Never mind, I finally figured it out. The 256KB memory is created
from the Xilinx FPGA we have on our custom board. We hooked up the
sys_clkout2 output from the OMAP to the FPGA as its clocking source.
It works fine under u-boot (I configured sys_clkout2 to 27MHz, use the
54MHz with a divider of 2). However, when I boot up to Linux, this
clock source is messed up and the 256KB memory is unaccessable. I
tried playing around with clock.c, clock34xx.c and clock34xx.h to make
sure sys_clkout2 is not disturbed and is not disabled but had no
luck. So I just reconfigured the kernel with the
CONFIG_OMAP_RESET_CLOCKS option turned off and now I can access the
memory correctly. Thanks everyone for you help.

Regards,
Andy