[Battlemesh] Battlemesh v6 conclusions

Pau hakais at gmail.com
Thu Sep 12 07:48:07 UTC 2013


Great work Axel.
I have done just a quick read for the moment and it seems very
complete and accurated. However for somebody who is not very involved
in the analisys is very hard to understand what is going on, would be
very nice to have a conclusions page summarizing the results of the
tests (yes! a summary of the summary for dummies).

Cheers
--
./p4u


On Wed, Sep 11, 2013 at 1:48 PM, Axel Neumann <neumann at cgws.de> wrote:
>
> Hi
>
>
>
> On 07/31/2013 09:45 PM, NicoEchániz wrote:
>>
>> On 07/30/2013 06:58 AM, Pau wrote:
>>>
>>> --
>>> ./p4u
>>>
>>>
>>> On Tue, Jul 30, 2013 at 11:09 AM, Clauz<clauz at ninux.org
>>> <mailto:clauz at ninux.org>>  wrote:
>>>
>>>      Hi!
>>>      I'll try to reply inline.
>>>
>>>      On 07/29/2013 12:11 PM, Pau wrote:
>>>      >  There are not "official" results, just raw data. As far as I know
>>> Nico
>>>      >  and Clauz (the testing team) did some initial analisys, which
>>> pointed
>>>      >  that BMX6 was the protocol with better results in general
>>> followed by
>>>      >  OLSR in the bandwidth corner-to-corner test and Batman-adv in the
>>> time
>>>      >  convergence test. But maybe they can clarify it.
>>>
>>>      Well, this, at least from my side, was just a sensation from looking
>>> at
>>>      the screen while the tests where performed, so it's not too fair to
>>> make
>>>      any big claim before having a real analysis done on the data.
>>>
>
> I spend some (more than wanted) time on this last month but then run out of
> time to complete it :-(
> Now I just uploaded the current status to a github project here:
> https://github.com/axn/wbm2pdf
> where you can find scripts that analyse the ping measurements (random,
> mobile, and running node tests) and create a big pdf
> https://raw.github.com/axn/wbm2pdf/master/tex/wbmv6.pdf
>
> I did do some assumptions to analyze the data which may be questionable,
> especially when comparing layer 2 with layer 3 routing protocols. I tried to
> justify them in Section 3.1 which should be considered when looking at the
> resulting graphs.
>
> I hope I find some more time this month to include also some graphs for the
> netperf tests.
> Anybody willing to contribute is welcome.
>
> The idea is also to have tools that allow us to create instant and
> intermediate results during the next battlemeshes so that protocol
> developers and testbed drivers can get periodic feedback.
>
>
>
>>> Ok! However I remember Nico did some python script which already showed
>>> some results (at least in the bandwith test).
>>
>> I pasted what I did that day here:
>>
>> http://pastebin.com/DpktA56z
>>
>>
>> It was a hacky parse put together in a rush to try to have some averages
>> on the netperf tests.
>>
>>
>> If you run it on the collected data you will get something like this:
>>
>> $ ./np_parse.py "netperf_test1/screenlog.netperf"
>> [['babel', 831.1666666666666, 3, 1641.43],
>> ['batadv', 833.1899999999999, 7, 1146.91],
>> ['bmx6', 2695.2912499999998, 8, 3945.15],
>> ['olsr', 2918.0000000000005, 8, 3507.81]]
>>
>> $ ./np_parse.py "netperf_test2/screenlog.netperf"
>> [['babel', 836.2333333333335, 3, 1216.47],
>> ['batadv', 989.8333333333334, 3, 1148.42],
>> ['bmx6', 2707.9880000000003, 5, 4644.81],
>> ['olsr', 3647.4300000000003, 4, 4300.46]]
>>
>> $ ./np_parse.py "netperf_test3/screenlog.netperf"
>> [['babel', 826.5674999999999, 4, 895.01],
>> ['batadv', 402.7757142857144, 7, 590.57],
>> ['bmx6', 1979.8442857142857, 7, 2788.54],
>> ['olsr', 1866.6214285714286, 7, 3713.86]]
>>
>>
>> Each list is showing:
>> [protocol, average_throughput, samples, max_throughput]
>>
>>
>> Information on what was happening during each test should be somewhere
>> else :)
>>
>
>
> I think this are the only written down notes on what indeed happened:
> http://downloads.battlemesh.org/WBMv6/test_data/BATTLEMESH_V6_TESTS_NOTES
>
> Apart from these technical details ;-)
> http://downloads.battlemesh.org/WBMv6/test_data/scripts0/
>
> And there are descriptions of tests here:
> http://battlemesh.org/BattleMeshV6/Tests
> describing measurement objectives and scenarios for
> - basic stats test
> - The channel surfer test
> - Convergence time test
> - Don't cross the streams test
> But I am not sure to which extend the executed tests match these
> descriptions.
>
> BTW: There is an old thread discussing the same questions here:
> http://ml.ninux.org/pipermail/battlemesh/2013-April/002345.html
>
>
>
>
>
>
>
>
>>
>> cheers!
>> Nico
>> _______________________________________________
>> Battlemesh mailing list
>> Battlemesh at ml.ninux.org
>> http://ml.ninux.org/mailman/listinfo/battlemesh
>>
>
> _______________________________________________
> Battlemesh mailing list
> Battlemesh at ml.ninux.org
> http://ml.ninux.org/mailman/listinfo/battlemesh


More information about the Battlemesh mailing list