DSS framebuffer address question

We're trying to set up a board where the OMAP can read the framebuffer
data from an FPGA. Are there any hardware or software constraints on
the input address range for framebuffers? I see that I can specify the
physical address in the current DSS2 bootargs to be something like
"omapfb.vram=0:2M@5000000", and once I tweaked the code to add
0x5000000 as allowable VRAM space, it accepted it. But it does not
seem to be happy to actually read data from that address.
I'm working my way through the DSI section of the SPRUF98B, but if
someone else has info on this, I'd appreciate it.