tilcdc FIFO underflow

Hi there,

i was able to get my 10.1 custom Display to work with the RGB Pins on the beaglebone black.

I am using the latest Debian Jessie image from beagleboard.org and the X11 Desktop has the right colors and the right size.

So i assume that the Timings from my display are correct.

panel {

status = “okay”;

compatible = “ti,tilcdc,panel”;

pinctrl-names = “default”;

pinctrl-0 = <&bb_lcd_lcd_pins>;

panel-info {

ac-bias = <255>;

ac-bias-intrpt = <0>;

dma-burst-sz = <16>;

bpp = <32>;

fdd = <0x80>;

tft-alt-mode = <0>;

stn-565-mode = <0>;

mono-8bit-mode = <0>;

sync-edge = <0>;

sync-ctrl = <1>;

raster-order = <0>;

fifo-th = <0>;

};

display-timings {

native-mode = <&timing0>;

timing0: 1280x800 {

clock-frequency = <70000000>;

hactive = <1280>;

vactive = <800>;

hfront-porch = <80>;

hback-porch = <60>;

hsync-len = <20>;

vback-porch = <12>;

vfront-porch = <8>;

vsync-len = <3>;

hsync-active = <0>;

vsync-active = <0>;

};

};

};

But when i play a video or when i have many frames on the display i get some screen flickers.

On the log via UART i see the following:

1- When booting up: of_graph_get_next_endpoint(): no port node found in /ocp/lcdc@4830e000

2- When video or many frames are displayed:

tilcdc 4830e000.lcdc: tilcdc_crtc_irq(0x00000020): FIFO underflow

tilcdc 4830e000.lcdc: tilcdc_crtc_irq(0x00000004): Sync lost

Any idea?

Is the DDR flash rate to slow? Or the DMA buffer not big enougth?

Hi Michael,

we had very similar issue, which we encountered when updating to gstreamer 1.6. Using git bisect we were able to find a root cause and prepare simple test program, which reliably reproduces the issue:

#include <string.h>
#include <stdint.h>
#include <stdlib.h>

int main(int argc, char **argv)
{
int size;
uint8_t *memblock;

if (argc < 2)
return -1;

size = atoi(argv[1]);

memblock = malloc(size);

while (1)
{
memset(&memblock[0], 0, size);
}

free(memblock);

return 0;
}

starting this program will cause crazy flickering, …

We were able to reproduce the issue with

  1. debian originally provided on beaglebone using memtest utility for carlos. Kernel version was:

Linux version 3.8.13-bone70 (root@a5-imx6q-wandboard-2gb) (gcc version 4.6.3 (Debian 4.6.3-14) ) #1 SMP Fri Jan 23 02:15:42 UTC 2015

  1. latest version of debian from beaglebone website Kernel version was:

Linux version 4.1.15-ti-rt-r43 (root@a4-imx6q-wandboard-2gb) (gcc version 4.9.2 (Debian 4.9.2-10) ) #1 SMP PREEMPT RT Thu Jan 21 20:13:58 UTC 2016

so it seems to be existing for ages.

Did you in the mean time managed to find a solution for your problem?

Thanks,
Radek

starting this program will cause crazy flickering, …

I’m surprised that code is not causing more of a problem than just “flickering” - Whatever that means. In fact, I’m surprised that code would compile at all without a compiler error. As &variable[0] probably is not what you think it is.

Also, for the record, infinitely allocating an arbitrary amount of dynamic memory, in a loop, without any kind of a pause is going to eat up an incredible amount of CPU cycles. As in that code will very likely use as close to 100% CPU as the system will allow it. Which again, is no wonder you’re “flickering” . . .

You lucky your program does not randomly crash for using malloc() on the same dynamic memory over, and over, without using free(), or instead using realloc() . . .

I’m surprised that code would compile at all without a compiler error. As &variable[0] probably is not what you think it is.

I see no issue here. Can you elaborate?

Also, for the record, infinitely allocating an arbitrary amount of dynamic memory, in a loop, without any kind of a pause is going to eat up an incredible amount of CPU cycles. As in that code will very likely use as close to 100% CPU as the system will allow it. Which again, is no wonder you’re “flickering” . . .

The code is not allocating anything in a loop.

You lucky your program does not randomly crash for using malloc() on the same dynamic memory over, and over, without using free(), or instead using realloc() . . .

Because it’s not calling malloc() over and over…

The issue here is that if you run memset() in a tight loop (which at least one gstreamer1.6 audio decoder plugins does), it seems to clog up the data bus enough for the DMA to fail.

Because it’s not calling malloc() over and over…

Yeap, you’re right it’s calling memset() every loop. Not sure why I was thinking it was calling malloc() every loop. But the point is, it does not matter what is being called in the loop. any system / API call in the loop, without some form of a sleep() will eat up a lot of CPU cycles. Again, as much as the system will allow it to, which is usually in the mid to high 90%'s . . .

I’m surprised that code would compile at all without a compiler error. As &variable[0] probably is not what you think it is.

I see no issue here. Can you elaborate?

So, &variable ← beginning of an array, equivalent to variable[0], but what is &variable[0] ? A multi dimensional array ? that’s not what was defined . . . to be 100% honest I’m not sure what it does, but the first thing that comes to mind is “undefined behavior”.

I would also like to point out that the free() after the loop ( end of program ) is not necessary. Modern Linux versions deal with this automatically. I do not suppose it’ll hurt, but figured I’d point that out.

Because it’s not calling malloc() over and over…

Yeap, you’re right it’s calling memset() every loop. Not sure why I was thinking it was calling malloc() every loop. But the point is, it does not matter what is being called in the loop. any system / API call in the loop, without some form of a sleep() will eat up a lot of CPU cycles. Again, as much as the system will allow it to, which is usually in the mid to high 90%'s . . .

Yes it’s definitely not nice to call memset() in such a tight loop, but the program is just for demonstration purposes. As mentioned at least one gstreamer 1.6 audio decoder plugins will call memset() consecutively fast enough to cause issues with the lcdc dma. And thus screen flicker.

So, &variable ← beginning of an array, equivalent to variable[0], but what is &variable[0] ? A multi dimensional array ? that’s not what was defined . . . to be 100% honest I’m not sure what it does, but the first thing that comes to mind is “undefined behavior”.

I won’t go into details, but “&variable” is not the same as “variable[0]”. (You might be able construct some special/silly case tho where the content of variable[0] is equivalent to &variable).

And “&variable[0]” is equivalent to “variable”. The latter might have been more readable in the context of the code, but both version are valid C. Basically you are getting the address of the first element of the memory block, which is the starting address of the memory block itself, which is what memset expects.

That’s not what I’ve been taught over the last 20 years. In C variable[0] is essentially a pointer to the start of variable. Which if you want to get technical is not the “adddress of” variable. Regardless &variable should give the starting address of variable. So at minimum &variable[0] is a bit superfluous which makes the code harder to read through . . . and is something I’d never do personally.

Is that nick picky enough ? :wink:

Also to nickpick even more. memset() expects type unsigned character, and uint8_t *, and unsigned char * are not really equivalent . . . which is a source of even more errors :wink: This I’ve had personal hands on with :wink:

In C variable[0] is essentially a pointer to the start of variable.

No, that’s plain wrong.

Also to nickpick even more. memset() expects type unsigned character, and uint8_t *, and unsigned char * are not really equivalent . . . which is a source of even more errors :wink: This I’ve had personal hands on with :wink:

No, glibc, uclibc, eglibc, even avr-libc all have the same signature, which is the following:
void *memset(void *s, int c, size_t n);

Also this is the last post from me regarding C stuff, since it’s off topic.

Hi William,

Let’s not discuss the C of the program, as:

  1. The mem-test program behaves the same even if it contains “memset(memblock, 0, size);”
  2. The mem-test program behaves the same even if you run it using “nice” at very low priority
  3. The mem-test program is not accessing directly any HW and runs as user nobody
  4. The mem-test program does not cause any side-effects on x86 Ubuntu PC, Freescale iMX6 ARM or Broadcom BCM958305k dev kit
  5. Same behavior is reproducible with gstreamer-1.6 and decoding WMA

I believe Linux kernel should never allow process running as nobody at very low priority without any access to HW to damage experience of other process (such as X server) running as root with much higher priority, …

Thanks,
Radek

Hi William,

I would like to point out, that free will never even be reached as there is an ifinite loop before it :slight_smile:

It is just a good engineering practice, to write always free for every malloc, in case somebody later on modifies/extends code later on to avoid memory-leaks within the program.

Thanks,
Radek

However . . since you do not get to respond to something and then just tell me to stop talking . . .

In C variable[0] is essentially a pointer to the start of variable.

*No, that's plain wrong.*

Bullshit.

Also to nickpick even more. memset() expects type unsigned character, and uint8_t *, and unsigned char * are not really equivalent . . . which is a source of even more errors :wink: This I’ve had personal hands on with :wink:

*No, glibc, uclibc, eglibc, even avr-libc all have the same signature, which is the following:* void *memset(void *s, int c, size_t n);

Here I mispoke ( mis-wrote ) but I was not speaking to the function prototype. But instead the value that memset() “sets” is interpreted as unsigned char *. So, mixing uint8_t *, and unsigned char * is bad mojo. It’ll work, until you expect it to, and then your code will start randomly misbehaving. That is to say that such code may / may not work as expected, or in other words. undefined behavior. I am not however sure if there is in fact anything in the standard regarding that, but I do know from first hand experience that such code, that is not consistent in types, is not 100% reliable. Which is a good reason to use compiler options -Wall, and such.

@Radek

It is just a good engineering practice, to write always free for every malloc, in case somebody later on modifies/extends code later on to avoid memory-leaks within the program.

There would be no memory leak. knowing what your code does is in fact good engineering, so omitting useless code because you know it’s useless is not bad form. It’s known as knowing what you’re doing :wink:

Anyway, I’m not sure how useful this “test code” really is. This is not something any useful application would ever do . . .other than apparently making a beaglebone in certain situations “flicker” . . .

Anyway, I’m not sure how useful this “test code” really is. This is not something any useful application would ever do . . .other than apparently making a beaglebone in certain situations “flicker” . . .

This is exactly what ffmpeg does when decoding WMA. Not so harshly - it does a lot of computation in between, but harsh enough to cause beaglebone display to flicker.

Radek

One last point. In the case of Debian. You do not really have to worry much about how large your stack size for any application is. So complicating an application by dynamically allocating memory( heap versus stack ) is not necessarily needed.

Anyway, I’m not 100% sure all Linux distro’s behave in this manner, but know for a fact this is the case with Debian as I spent a good amount of time reading / researching this for a project of mine.

Not only that, but it is of course possible to structure code in a way that your stack stays small to begin with. Either way, Debian can if needed allocate up to 100% system for a single applications stack if needed. Of course the only way this will ever happen is if tested with specific code . . .

Hi Michael,

As the post above will be probably often found by google, I will post here also a temporary solution colleague of mine found to the flickering problem.

devmem2 0x4c000054 w 0x00FFFFF10

It is probably not ideal - there should be a way how to prioritize just the LCD transfers (according to the TI wiki), but the values there do not work. This fix just ensures that any “too old” command gets executed sooner than it would otherwise, maybe that is a good thing anyway.

However, I believe I understand all the various issues we’ve seen (although I could still be wrong - but this at least makes sense):

static image shift

  • here, the DMA for transferring data to LCD fifo is blocked for a short period of time once, but then it continues normally. This produces the fifo-underflow interrupt once, the data for display are few pixels late, but then all works normally so the image is just shifted (although not too visible in the photo, the “extra” pixels at top-left corner are black, and not white if the image was just wrapped).

full flickering

  • in this case, the memory load is so severe that the DMA is blocked too often and can’t supply steady stream of data; producing fifo-underflow interrupts all over. The image still starts from the top-left corner as usual, but then the rest of the “pixel stream” contains some extra 0s so the testimage produces diagonal patterns (with varying error, hence the flickering).

So these 2 are the same issue, just with different outcome.

The problem is the DDR command prioritization logic in our memory controller, the memset() loop manages to starve the DMA data stream. The “fix” just tweaks the REG_PR_OLD_COUNT value - now if there is a DDR command in the memory controller fifo for too long, it will get executed (out of the prioritized ordering).

(I can probably also explain why it ± works with >2k buffers; but there I’m not fully sure yet.)

All this is based on this info:
Thanks, Radek On Tuesday, March 8, 2016 at 6:33:00 PM UTC+1, Radek Dostál wrote: