[Battlemesh] tests for downloads ? and tests follow-ups

[Axel Neumann]

hi

In the following you'll find a quick summary of the measurements Henning and I 
did in the last night of the battlemesh!

More detailed numbers can be found here (.pdf):
http://dabax.net/wbm/axel/summary/WBMv4-A-summary-details-x01.pdf
and here (.ods): 
http://dabax.net/wbm/axel/summary/WBMv4-A-summary-details-x01.ods

The bash commands to evauate all the numbers summarized in the latter link and 
some more remarks on the measurements are given here: 
http://dabax.net/wbm/axel/summary/wbm4-eval-notes.txt

By playing with the counted 0/1 values in the latter spreadshet you can 
exclude certain measurements from the summary line. This way you can focus the 
evaluation on specific scenarios (like only static or only mobile scenarios). I 
suggest to dig around a bit, its very interesting...

There are also graphs illustrating the protocol-traffic overhead over time for 
each measurement comparing the different protocols here (*.png):
http://dabax.net/wbm/axel/summary/

An approximate map of the deployed node locations is here:
http://dabax.net/wbm/axel/summary/node-location.jpg

The script to gather ping and top data is here:
http://dabax.net/wbm/axel/script/ping-test.sh

All raw data can be found here:
http://dabax.net/wbm/axel/raw-data/


Most important remark:
Due to different configurations there are two groups of protocols.
Group A: Babel, Olsr, Bmx6 (33 nodes, different channels, lan & wlan interface
Group B: Batman, Batman-Adv (25 nodes, one channel,  only wlan interface
Protocols from group A can NOT be compared with protocols from group B!


#########################################
############################

Packet Loss (all scenarios):

Group A:
Bmx6 59.5% 
Olsrd 64.5% 
Babel 71.0% 

Group B:
Batman-Adv 71.3% 
Batman       74.4% 


Packet Loss (only two rather fast mobile scenarios):

Group A:
Babel 78.0% 
Bmx6 80.0% 
Olsrd 81.5% 

Group B:
Batman       77.5% 
Batman-Adv 82.0% 


############################

Average Protocol Traffic Overhead in Bytes/Second (all scenarios)

Group A:
Olsrd 591  Bytes/sec  1.3 Packets/sec   
Bmx6 954  Bytes/sec  4.8 Packets/sec  
Babel 2029 Bytes/s    12.5 Packets/sec 

Group B:
Batman-Adv 1240 Bytes/sec 21.7 Packets/sec
Batman       1308 Bytes/sec 20.0 Packets/sec 


############################

Average Cpu load @ Fonera2100 (all scenarios)

Group A:
Olsrd 1.8%
Bmx6 4.2%
Babel 7.2%

Group B:
Batman      3.8%
Batman-Adv ?


############################

Average Memory consumption @ Fonera2100 (all scenarios)

Group A:
Babel 8.8%
Bmx6 10.0%
Olsrd 10.9%

Group B:
Batman      8.0%
Batman-Adv ?



############################

Average Recorded number of loops per 1500 pings (all scenarios)

Group A:
Olsrd  30.8
Bmx6  0
Babel  0

Group B:
Batman      13.8
Batman-Adv ?

############################

Average Memory consumption @ Fonera2100 (all scenarios)

Group A:
Babel 8.8%
Bmx6 10.0%
Olsrd 10.9%

Group B:
Batman      8.0%
Batman-Adv ?


############################


Its very interesting to look deeper into what happened. For example for me it 
was interesting how much protocols made use of channel switching (reduce 
interference) at the cost of at least two additional hops.

Average Number of detected ping RecordRoutes via channels 1 or 11 nodes  
Given as average per measurement      (all scenarios):

Group A:
Bmx6  10.4
Olsrd  10.2
Babel  8.9

Or: Its also interesting to node how captured protocol traffic overhead changed 
between the first measurements (e.g. log-07-BytesPer10Seconds.png ) and the 
last measurements (e.g. log-21-BytesPer10Second.png around 7am when there was 
much less radio interference from notebooks).


############################


Any comments, suggestoins, or other evaluations are very welcome.

Special thanks to Nico, Pieter, ... and all the others for setting this all up

/axel


On Donnerstag 24 März 2011, Axel Neumann wrote:
> Hi
> 
> On Donnerstag 24 März 2011, Pau wrote:
> > I have also some tests results, I can upload it in raw format. But this
> > link does not work for me.
> 
> Until downloads.battlemesh.org is available I put the raw I have into your
> server at http://dabax.net/wbm/
> 
> Some quick and important remarks right now !!!!!!!!!!!!
> 
> In total there were 33 foneras actively meshing.
> This includes:
> 25 foneras meshing on channel 6 .
> 4 meshing on channel 1
> 4 meshing on channel 11.
> 
> Babeld, Olsrd and bmx6 were meshing using wlan and lan interface.
> This was not the case with batman and batman-adv which were only meshing on
> wlan interface.
> 
> We just found out about this fact in the early morning and were to tired to
> fix it. sorry,
> 
> Therefore, You can compare babeld, olsrd, and bmx6 with each other.
> You can also compare batman with batman-adv. But its unfair to compare
> between protocols of this two groups.
> 
> 
> Thre tcpdumps (*.rawdata) were created from my notebook which was connected
> directly to fonera 10.
> 
> All remaining data was collected on node 10 using the following script:
> http://dabax.net/wbm/script/ping-test.sh
> 
> each measurement was given a number which is the first number in a file
> name. So the file names can be read like:
> 
> log-<experimentNumber>-<destinationNodeNumber>-<protocolPrefix>.<dataType>
> 
> 
> I just came back this this morning and had no time yet for fourther post-
> processing.  more tomorrow...
> 
> 
> /axel
> 
> > 2011/3/24 Xavier Carcelle <xavier.carcelle at gmail.com>
> > 
> > > Hola battlemesh folks,
> > > 
> > > Alex, Markus : Have you able to push the 50M of logs from your tests
> > > to downloads.battlemesh.org for us to analyze ?
> > > 
> > > For further tests ready for next battlemesh or "intermediate"
> > > battlemesh (4.x this summer for instance), I believe we should have :
> > > 
> > > *VoIP server with siphone on mobile-meshed laptops (I have a setup w/
> > > Asterisk and Twinkle working)
> > > *Streaming server : Icecast somewhere (kloschi has a set-up that we
> > > can use and Ursula has a set-up to try following the Paris demo audio
> > > over mesh)
> > > *Multicast server : From Simone
> > > *Buffer tests : large transfer over the mesh and monitoring of the
> > > buffers w/ pings, tcpdumps, pingplotter ... bloat monitoring
> > > *To-the-outside-world : maintaining sessions to ssh, https ... servers
> > > 
> > > Cheers,
> > > 
> > > Xavier.
> > > _______________________________________________
> > > Battlemesh mailing list
> > > Battlemesh at ml.ninux.org
> > > http://ml.ninux.org/mailman/listinfo/battlemesh
> 
> _______________________________________________
> Battlemesh mailing list
> Battlemesh at ml.ninux.org
> http://ml.ninux.org/mailman/listinfo/battlemesh