[Battlemesh] Tests

Mon May 12 11:51:24 CEST 2014

Hi Dave,

sorry for my late answer I have now read all of your comments about the tests and find it very useful indeed!

I like the idea of using netperf-wrapper as a testing tool and after a short look some of the scenarios could be done with it but I would like to hold a short meeting with you to transfer your knowledge about usage/evaluation and maybe also compare it to gperf or other tools.

Unfortunately the link http://snapon.lab.bufferbloat.net/~d/weakening_mac_loss_recovery.pdf did not work for me.

Best wishes, Amadeus

>---- Original Message ----
>From: Dave Taht <dave.taht at gmail.com>
>To: "Battle of the Mesh Mailing List" <battlemesh at ml.ninux.org>
>Sent: Sa, Mai 3, 2014, 9:35 PM
>Subject: Re: [Battlemesh] Tests
>
>A couple more comments:
>
>In scenario 3s and 5 it is unclear if what babel calls "diversity
>routing" (and batman calls
>something else that I can't remember) will be used.
>
>http://www.pps.univ-paris-diderot.fr/~jch/software/babel/wbmv4.pdf
>
>My assumption is that a single wifi channel will be used. Using
>multiple channels is
>generally a huge win, but would need more routers (as in scenaro 5) to
>achieve interesting effects.
>
>If it is the first case, the diagram on 3 is incorrect for a 5m
>radius, as all routers would
>have direct reachability unless they are on different ssids.
>
>If it is the second case, n7 would have to support 6 channels, and the
>others, 3.
>
>
>
>
>
>On Sat, May 3, 2014 at 9:45 AM, Dave Taht <dave.taht at gmail.com> wrote:
>> On Sat, May 3, 2014 at 7:34 AM, Amadeus Alfa
>> <amadeus at chemnitz.freifunk.net> wrote:
>>> Hello all,
>>>
>>> BattleMesh v7 tests are now available online:
>>> http://battlemesh.org/BattleMeshV7/Tests
>>>
>>> Please let us know if you have any comments and what firmware requirements
>>> are necessary.
>>
>> the netperf-wrapper test can be somewhat easily used in scenarios 2
>> and 5. Requirements
>> are a netperf netserver compiled with --enable-demo on the server(s),
>> and python on the
>> clients. (only plotting requires py-qt4 and python-matplotlib)
>>
>> To tackle 5 in a topology like this:
>>
>> client1 -> server1
>>           -> server2
>>
>> client1 $: netperf-wrapper -l 300 -H server1 -H server2 -H server1 -H
>> server2 rtt_fair_be
>>
>> To test
>>
>> client1 -> server1
>> client2 -> server2
>>
>> client1 $: ssh client2 'netperf-wrapper -l 300 -H server1 -H server2
>> -H server1 -H server2 -t client2-babel rtt_fair_be' &
>> client2 $: netperf-wrapper -l 300 -H server1 -H server2 -H server1 -H
>> server2 -t client1-babel rtt_fair_be
>>
>> To meet the spec more exactly, to do a simpler single stream tcp
>> upload or tcp download test rather than 2 flows
>> to/from each server as these do, use "tcp_upload" or "tcp_download" as
>> the test name, and only a single -H target.
>> Writing other tests is rather straightforward also. The most often
>> used test is rrul_be actually.
>>
>> To exercise the wifi hardware queues, "rtt_fair" will test the BE and BK queues.
>>
>> (the -t title option above distinguishes between test runs). There is
>> also a batching facility that can be used to
>> capture a lot more data about the local structure of the qdisc and
>> network, and completely automate a test series.
>>
>> In all cases latency and loss are fairly closely measured, and the end
>> results can be easily combined together
>> on a variety of cdf, box, q-q or other plots. There are simply
>> extraordinary numbers of graphs and data available
>> in the netperf-wrappers format now...
>>
>> As for scenario 2 (powerline), I would argue for more different
>> manufacturers hardware as the two papers I know
>> of on it give wildly varying and oft-terrible results, and I'd be
>> reluctant to draw conclusions from only one manufacturer.
>>
>> http://caia.swin.edu.au/reports/130121A/CAIA-TR-130121A.pdf
>>
>> http://caia.swin.edu.au/reports/130417A/CAIA-TR-130417A.pdf
>>
>> Does anyone have any MOCA gear yet?
>>
>> I have a few comments the structure of the tests.
>>
>> 0) Given the quality of analysis we're getting back from
>> netperf-wrappers, I'd like that, rather than (or in addition to) iperf
>> be used. Get it at: https://github.com/tohojo/netperf-wrapper
>>
>> 1) A single long unidirectional tcp flow is not how the internet works & neither
>> test specifies the direction of the traffic. A single tcp stream is
>> not particularly
>> revealing, and various events (RTOs, reorders etc) are not easily
>> recovered from. A
>> a simultaneous continuous ping measurement gives a better idea of what's going
>> wrong on the path, in addition to the transfer.
>>
>> A string of chrome web page benchmarker runs might be useful to obtain, in order
>> to generate more realistic network traffic from a list of urls.
>>
>> 2) 20 minutes is a heck of a long time to run a test. It should be
>> good to detect
>> long period oscillations like the babel-rtt work showed, however. (I used -l 300
>> above as that's 5 minutes)
>>
>> 3) The client nor server driving the test is not specified. My
>> assumption is that
>> each would be a reasonably fast box connected via ethernet to the
>> routers, running
>> a given kernel version, with a specific tcp (cubic) used, with an
>> underlying qdisc
>> of pfifo_fast (or sch_fq if the kernel is recent enough). Changing the tcp might
>> be useful as well as a separate experiment....
>>
>> As for scenario 3, while I agree it's a good measurement of baseline
>> load induced by the routing protocol,
>> it's not a measure of what happens with route updates when routes
>> change, due to load, interference or other factors.
>>
>> While it involves a lot of exercise, I was bemused by the testing
>> method outlined in this paper, so
>> perhaps there could be a node mobility test of some sort?
>>
>> http://snapon.lab.bufferbloat.net/~d/weakening_mac_loss_recovery.pdf
>>
>> "The experiment is performed in a two storied Japanese style house
>> built of wood. As depicted in Fig. 2, the server and AP are located in
>> the 2nd floor. The terminal moves between the far position in the 1st
>> floor and the near position in the 2nd floor. The distance between the
>> terminal and AP is about 1.2 meter for the near position and about 10
>> meter for the far position. During a test run for 120 seconds, the
>> terminal moves in the following way.
>>
>> * It starts communication at the far point and keeps the position for
>> 30 seconds.
>>
>> * Then, it moves from the far position to the near position. The
>> duration spent for the moving is about 15 seconds.
>>
>> * It communicates with the server at the near position for 30 seconds.
>>
>> * Again, it moves from the near position to the far position spending
>> about 15 seconds.
>>
>> * In the last part, the terminal keeps the communication for 30
>> seconds at the far position. "
>>
>> (my daydream is that the mobility is done by a robot, but...)
>>
>>
>>>
>>> @Eloi I have adopted 2 tests that should be of interest. :-)
>>>
>>> Best wishes, Amadeus
>>>
>>>
>>>
>>> ---- Original Message ----
>>> From: Eloi Carbó Solé <eloicaso at gmail.com>
>>> To: "Battle of the Mesh Mailing List" <battlemesh at ml.ninux.org>
>>> Sent: Mi, Apr 30, 2014, 6:05 PM
>>> Subject: Re: [Battlemesh] Tests
>>>
>>> Hi!
>>>
>>> I am doing my final degree thesis wibed[0] that will be presented by Pau[1]
>>> and I am adopting the WBMv6 tests in order to check the mesh network
>>> behaviour. One of the main points of this thesis will be to validate the
>>> network, testing and monitoring its behaviour applying these tests. My idea
>>> is also to adopt and use WBMv7 tests too, so I am really interested in being
>>> in touch with those persons who will discuss and perform the experiments and
>>> tests.
>>>
>>> Best regards,
>>>
>>>
>>> [0] https://wiki.confine-project.eu/wibed:start
>>> [1] http://battlemesh.org/BattleMeshV7/Agenda
>>>
>>>
>>> On Tue, Apr 29, 2014 at 8:17 PM, Amadeus Alfa
>>> <amadeus at chemnitz.freifunk.net> wrote:
>>>>
>>>> Hi Colleen,
>>>>
>>>> thank you for taking care of testing ;-) The test scenarios and planning
>>>> will be available for review by the end of this week (will send PDF files to
>>>> the list). I would love to see your idea of tapping 802.11s in one of the
>>>> tests - could you please provide some more information so that I can create
>>>> a short description out of it?
>>>>
>>>> Also, we are still looking for testing supporters since there is only one
>>>> person right now trying to coordinate tasks.
>>>>
>>>> Best wishes, Amadeus
>>>>
>>>>
>>>> >---- Original Message ----
>>>> >From: Colleen T <colleen at cozybit.com>
>>>> >To: battlemesh at ml.ninux.org
>>>> >Sent: Di, Apr 29, 2014, 9:33 PM
>>>> >Subject: [Battlemesh] Tests
>>>> >
>>>> >Hello everyone,
>>>> >
>>>> >I am looking forward to coming.  I have been looking at the
>>>> >Battlemesh6 test page.  Will the tests be very similar this year?  Do
>>>> >we run them over the course of the week?
>>>> >
>>>> >I will be bringing some TPLink mr3020's -- not sure how many yet --
>>>> >and testing open80211s =)
>>>> >
>>>> >Cheers,
>>>> >Colleen
>>>> >_______________________________________________
>>>> >Battlemesh mailing list
>>>> >Battlemesh at ml.ninux.org
>>>> >http://ml.ninux.org/mailman/listinfo/battlemesh
>>>>
>>>> _______________________________________________
>>>> Battlemesh mailing list
>>>> Battlemesh at ml.ninux.org
>>>> http://ml.ninux.org/mailman/listinfo/battlemesh
>>>
>>>
>>>
>>>
>>> --
>>> Eloi Carbó Solé.
>>>
>>>
>>> _______________________________________________
>>> Battlemesh mailing list
>>> Battlemesh at ml.ninux.org
>>> http://ml.ninux.org/mailman/listinfo/battlemesh
>>>
>>
>>
>>
>> --
>> Dave Täht
>>
>> NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.article
>
>
>
>-- 
>Dave Täht
>
>NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.article
>_______________________________________________
>Battlemesh mailing list
>Battlemesh at ml.ninux.org
>http://ml.ninux.org/mailman/listinfo/battlemesh