suggest benchmark

Hey guys!

I have a number of boards with different ARM CPUs like: ARM9, OMAP3530 (600 and 720MHz versions), I.MX515 at babbage.

So I want to test them for some kind of performance if this term fits to. I need some open source benchmark with could show of the calculation speed and memory throughput also.

I’ve found one benchmark, compiled it under x86 and it took an hour to complete all the tests resulting in some “magic” number which has to be compared with something. The problem I see is that at 200MHz ARM9 this benchmark will work during a day and I don’t need this. Something lightweight and fast I like more. I am not a scientist and not going to launch my boards to space trip, so any special requirements are absent.

Any suggestions?

Max

Maybe one of the tests in this suite:

http://global.phoronix-test-suite.com/?k=profile&u=robertcnelson-9745-20331-820

(the full suite test will take about a day on a beagle..)

Regards,

I’ve used nbench to benchmark x86 Arm and PPC boards. Its probably not the most scientific but it gives you a good comparison between the boards, and it only takes a few minuets to run the test.

http://www.tux.org/~mayer/linux/bmark.html

Kevin

Kevin Holland <habib_15@hotmail.com> writes:

I've used nbench to benchmark x86 Arm and PPC boards. Its probably
not the most scientific but it gives you a good comparison between
the boards, and it only takes a few minuets to run the test.

Nbench is not a good benchmark. There are no good benchmarks. There
is no substitute for testing whatever you are planning on running on
the hardware.

nbench is even worse than many benchmarks as it relies on the math
library for sin and cos. So in some parts all you're measuring is the
performance of that library, which is extremely poor in many distros
that don't even care to compile libm with VFP for platforms that
support it.

Laurent

Laurent Desnogues <laurent.desnogues@gmail.com> writes:

Especially if they are using the CSL toolchain, that has a 'generic' libm, so no VFP for people using harry potter toolchains.

Well, drystone?

2010/9/14 Måns Rullgård <mans@mansr.com>

Even worse than nbench. It depends on libc string functions and also
perfectly fits in L1 caches.

Follow Mans' advice: pick the application you're most interested in and
measure its performance.

Laurent

Well, drystone?

optimize strcmp() and dhrstone results will skyrocket :slight_smile:

Well you can find flaws in all benchmark tests, but I still feel that if you just want a quick comparisons of systems (board and OS of choice) nbench is fine. Most people will not want to recompile the math library, they will either create their own math functions specific to hardware or use the default.

I agree that nbench is a bad test if you have something very specific you are trying to gauge like math capabilities or memory throughput, so like others have stated then you need to make a custom test yourself or test the specific program you had in mind. But I do think that nbench is good for a quick feeler test just to do a comparision.

If I could use the same applications at ARM9 and Cortex-A8…

For example, when I ran Rabbit video on OMAP3530 600MHz it was slower than at OMAP3530 720MHz. But i.mx515 shows this video more smother because it runs at 800MHz. So, this is can be the benchmark, but how can I show such result to my clients? I need some numeric values to be compared.

2010/9/14 Koen Kooi <koen@beagleboard.org>

Laurent Desnogues <laurent.desnogues@gmail.com> writes:

Maxim Podbereznyy wrote:

If I could use the same applications at ARM9 and Cortex-A8..

For example, when I ran Rabbit video on OMAP3530 600MHz it was slower
than at OMAP3530 720MHz. But i.mx515 shows this video more smother
because it runs at 800MHz. So, this is can be the benchmark, but how can
I show such result to my clients? I need some numeric values to be compared.

easy, decode 100 frames, sum the decoding time and divide by 100, show that
to your client.

Maxim Podbereznyy <lisarden@gmail.com> writes:

If I could use the same applications at ARM9 and Cortex-A8..

For example, when I ran Rabbit video on OMAP3530 600MHz it was slower than
at OMAP3530 720MHz. But i.mx515 shows this video more smother because it
runs at 800MHz. So, this is can be the benchmark, but how can I show such
result to my clients? I need some numeric values to be compared.

Measure the time to decode a video at max speed.

any guidance how to decode the exact number of frames or measure a time precisely?

2010/9/14 Måns Rullgård <mans@mansr.com>

Maxim Podbereznyy wrote:

any guidance how to decode the exact number of frames or measure a time
precisely?

measuring the full decode time of a video does not need to be awfully
precise, unix "time" command should do fine...

Wouldn't something like mplayer work:

mplayer -benchmark -vo null -nosound file_to_test.mpg

It should, if I remember correctly, output the time it took to decode
all the frames in the file as fast as possible. Take # of frames and
divide by the time to get an average FPS. Repeat more than 3 times to
get an average and stdev. Volia.

Note that the "-vo null -nosound" only tests the video codec
performance and doesn't test audio codec nor how fast you can get the
frame on the display. Now, if you use "-vo null -ao null", you'll
benchmark the video AND audio codec IIRC.

Downside is you'll need to compile mplayer for each platform, which is annoying.

Also note this is all theory. I haven't tried this yet on an ARM device.

Take care,
Mark Crichton

Well, using the mplayer seems reasonable. The only problem that it does not work at default rootfs supplied with i.mx51 Dev-kit and I really have no idea how to compile for i.mx51. There is no OE for i.mx51 and this is pity.

2010/9/14 Mark Crichton <crichton@gmail.com>

Well, using the mplayer seems reasonable. The only problem that it does not work at default rootfs supplied with i.mx51 Dev-kit and I really have no idea how to compile for i.mx51.

It's the very same cortex-A8 inside, so using the same rootfs should work. And there's a machine description for efikamx in OE, which is mx51 based.

regards,

Koen