[Battlemesh] Battlemesh v6 conclusions

[Axel Neumann]

Hi


On 07/31/2013 09:45 PM, NicoEchániz wrote:
> On 07/30/2013 06:58 AM, Pau wrote:
>> --
>> ./p4u
>>
>>
>> On Tue, Jul 30, 2013 at 11:09 AM, Clauz<clauz at ninux.org
>> <mailto:clauz at ninux.org>>  wrote:
>>
>>      Hi!
>>      I'll try to reply inline.
>>
>>      On 07/29/2013 12:11 PM, Pau wrote:
>>      >  There are not "official" results, just raw data. As far as I know Nico
>>      >  and Clauz (the testing team) did some initial analisys, which pointed
>>      >  that BMX6 was the protocol with better results in general followed by
>>      >  OLSR in the bandwidth corner-to-corner test and Batman-adv in the time
>>      >  convergence test. But maybe they can clarify it.
>>
>>      Well, this, at least from my side, was just a sensation from looking at
>>      the screen while the tests where performed, so it's not too fair to make
>>      any big claim before having a real analysis done on the data.
>>

I spend some (more than wanted) time on this last month but then run out 
of time to complete it :-(
Now I just uploaded the current status to a github project here:
https://github.com/axn/wbm2pdf
where you can find scripts that analyse the ping measurements (random, 
mobile, and running node tests) and create a big pdf
https://raw.github.com/axn/wbm2pdf/master/tex/wbmv6.pdf

I did do some assumptions to analyze the data which may be questionable, 
especially when comparing layer 2 with layer 3 routing protocols. I 
tried to justify them in Section 3.1 which should be considered when 
looking at the resulting graphs.

I hope I find some more time this month to include also some graphs for 
the netperf tests.
Anybody willing to contribute is welcome.

The idea is also to have tools that allow us to create instant and 
intermediate results during the next battlemeshes so that protocol 
developers and testbed drivers can get periodic feedback.


>> Ok! However I remember Nico did some python script which already showed
>> some results (at least in the bandwith test).
> I pasted what I did that day here:
>
> http://pastebin.com/DpktA56z
>
>
> It was a hacky parse put together in a rush to try to have some averages
> on the netperf tests.
>
>
> If you run it on the collected data you will get something like this:
>
> $ ./np_parse.py "netperf_test1/screenlog.netperf"
> [['babel', 831.1666666666666, 3, 1641.43],
> ['batadv', 833.1899999999999, 7, 1146.91],
> ['bmx6', 2695.2912499999998, 8, 3945.15],
> ['olsr', 2918.0000000000005, 8, 3507.81]]
>
> $ ./np_parse.py "netperf_test2/screenlog.netperf"
> [['babel', 836.2333333333335, 3, 1216.47],
> ['batadv', 989.8333333333334, 3, 1148.42],
> ['bmx6', 2707.9880000000003, 5, 4644.81],
> ['olsr', 3647.4300000000003, 4, 4300.46]]
>
> $ ./np_parse.py "netperf_test3/screenlog.netperf"
> [['babel', 826.5674999999999, 4, 895.01],
> ['batadv', 402.7757142857144, 7, 590.57],
> ['bmx6', 1979.8442857142857, 7, 2788.54],
> ['olsr', 1866.6214285714286, 7, 3713.86]]
>
>
> Each list is showing:
> [protocol, average_throughput, samples, max_throughput]
>
>
> Information on what was happening during each test should be somewhere
> else :)
>


I think this are the only written down notes on what indeed happened:
http://downloads.battlemesh.org/WBMv6/test_data/BATTLEMESH_V6_TESTS_NOTES

Apart from these technical details ;-)
http://downloads.battlemesh.org/WBMv6/test_data/scripts0/

And there are descriptions of tests here:
http://battlemesh.org/BattleMeshV6/Tests
describing measurement objectives and scenarios for
- basic stats test
- The channel surfer test
- Convergence time test
- Don't cross the streams test
But I am not sure to which extend the executed tests match these 
descriptions.

BTW: There is an old thread discussing the same questions here:
http://ml.ninux.org/pipermail/battlemesh/2013-April/002345.html







>
> cheers!
> Nico
> _______________________________________________
> Battlemesh mailing list
> Battlemesh at ml.ninux.org
> http://ml.ninux.org/mailman/listinfo/battlemesh
>