[Battlemesh] Tests

Dave Taht dave.taht at gmail.com
Mon May 12 18:56:06 CEST 2014 

On Mon, May 12, 2014 at 2:51 AM, Amadeus Alfa
<amadeus at chemnitz.freifunk.net> wrote:
> Hi Dave,
>
> sorry for my late answer I have now read all of your comments about the tests and find it very useful indeed!

Sorry for my late reply, I am in the PDT timezone.

> I like the idea of using netperf-wrapper as a testing tool and after a short look some of the scenarios could be done with it but I would like to hold a short meeting with you to

Cool.

>transfer your knowledge about usage/evaluation and maybe also compare it to gperf or other tools.

I am in irc now as "dtaht_nuc". Can also do a hangout or webrtc.

>
> Unfortunately the link http://snapon.lab.bufferbloat.net/~d/weakening_mac_loss_recovery.pdf did not work for me.

Snapon crashed over the weekend, try it now.

> Best wishes, Amadeus
>
>
>>---- Original Message ----
>>From: Dave Taht <dave.taht at gmail.com>
>>To: "Battle of the Mesh Mailing List" <battlemesh at ml.ninux.org>
>>Sent: Sa, Mai 3, 2014, 9:35 PM
>>Subject: Re: [Battlemesh] Tests
>>
>>A couple more comments:
>>
>>In scenario 3s and 5 it is unclear if what babel calls "diversity
>>routing" (and batman calls
>>something else that I can't remember) will be used.
>>
>>http://www.pps.univ-paris-diderot.fr/~jch/software/babel/wbmv4.pdf
>>
>>My assumption is that a single wifi channel will be used. Using
>>multiple channels is
>>generally a huge win, but would need more routers (as in scenaro 5) to
>>achieve interesting effects.
>>
>>If it is the first case, the diagram on 3 is incorrect for a 5m
>>radius, as all routers would
>>have direct reachability unless they are on different ssids.
>>
>>If it is the second case, n7 would have to support 6 channels, and the
>>others, 3.
>>
>>
>>
>>
>>
>>On Sat, May 3, 2014 at 9:45 AM, Dave Taht <dave.taht at gmail.com> wrote:
>>> On Sat, May 3, 2014 at 7:34 AM, Amadeus Alfa
>>> <amadeus at chemnitz.freifunk.net> wrote:
>>>> Hello all,
>>>>
>>>> BattleMesh v7 tests are now available online:
>>>> http://battlemesh.org/BattleMeshV7/Tests
>>>>
>>>> Please let us know if you have any comments and what firmware requirements
>>>> are necessary.
>>>
>>> the netperf-wrapper test can be somewhat easily used in scenarios 2
>>> and 5. Requirements
>>> are a netperf netserver compiled with --enable-demo on the server(s),
>>> and python on the
>>> clients. (only plotting requires py-qt4 and python-matplotlib)
>>>
>>> To tackle 5 in a topology like this:
>>>
>>> client1 -> server1
>>>           -> server2
>>>
>>> client1 $: netperf-wrapper -l 300 -H server1 -H server2 -H server1 -H
>>> server2 rtt_fair_be
>>>
>>> To test
>>>
>>> client1 -> server1
>>> client2 -> server2
>>>
>>> client1 $: ssh client2 'netperf-wrapper -l 300 -H server1 -H server2
>>> -H server1 -H server2 -t client2-babel rtt_fair_be' &
>>> client2 $: netperf-wrapper -l 300 -H server1 -H server2 -H server1 -H
>>> server2 -t client1-babel rtt_fair_be
>>>
>>> To meet the spec more exactly, to do a simpler single stream tcp
>>> upload or tcp download test rather than 2 flows
>>> to/from each server as these do, use "tcp_upload" or "tcp_download" as
>>> the test name, and only a single -H target.
>>> Writing other tests is rather straightforward also. The most often
>>> used test is rrul_be actually.
>>>
>>> To exercise the wifi hardware queues, "rtt_fair" will test the BE and BK queues.
>>>
>>> (the -t title option above distinguishes between test runs). There is
>>> also a batching facility that can be used to
>>> capture a lot more data about the local structure of the qdisc and
>>> network, and completely automate a test series.
>>>
>>> In all cases latency and loss are fairly closely measured, and the end
>>> results can be easily combined together
>>> on a variety of cdf, box, q-q or other plots. There are simply
>>> extraordinary numbers of graphs and data available
>>> in the netperf-wrappers format now...
>>>
>>> As for scenario 2 (powerline), I would argue for more different
>>> manufacturers hardware as the two papers I know
>>> of on it give wildly varying and oft-terrible results, and I'd be
>>> reluctant to draw conclusions from only one manufacturer.
>>>
>>> http://caia.swin.edu.au/reports/130121A/CAIA-TR-130121A.pdf
>>>
>>> http://caia.swin.edu.au/reports/130417A/CAIA-TR-130417A.pdf
>>>
>>> Does anyone have any MOCA gear yet?
>>>
>>> I have a few comments the structure of the tests.
>>>
>>> 0) Given the quality of analysis we're getting back from
>>> netperf-wrappers, I'd like that, rather than (or in addition to) iperf
>>> be used. Get it at: https://github.com/tohojo/netperf-wrapper
>>>
>>> 1) A single long unidirectional tcp flow is not how the internet works & neither
>>> test specifies the direction of the traffic. A single tcp stream is
>>> not particularly
>>> revealing, and various events (RTOs, reorders etc) are not easily
>>> recovered from. A
>>> a simultaneous continuous ping measurement gives a better idea of what's going
>>> wrong on the path, in addition to the transfer.
>>>
>>> A string of chrome web page benchmarker runs might be useful to obtain, in order
>>> to generate more realistic network traffic from a list of urls.
>>>
>>> 2) 20 minutes is a heck of a long time to run a test. It should be
>>> good to detect
>>> long period oscillations like the babel-rtt work showed, however. (I used -l 300
>>> above as that's 5 minutes)
>>>
>>> 3) The client nor server driving the test is not specified. My
>>> assumption is that
>>> each would be a reasonably fast box connected via ethernet to the
>>> routers, running
>>> a given kernel version, with a specific tcp (cubic) used, with an
>>> underlying qdisc
>>> of pfifo_fast (or sch_fq if the kernel is recent enough). Changing the tcp might
>>> be useful as well as a separate experiment....
>>>
>>> As for scenario 3, while I agree it's a good measurement of baseline
>>> load induced by the routing protocol,
>>> it's not a measure of what happens with route updates when routes
>>> change, due to load, interference or other factors.
>>>
>>> While it involves a lot of exercise, I was bemused by the testing
>>> method outlined in this paper, so
>>> perhaps there could be a node mobility test of some sort?
>>>
>>> http://snapon.lab.bufferbloat.net/~d/weakening_mac_loss_recovery.pdf
>>>
>>> "The experiment is performed in a two storied Japanese style house
>>> built of wood. As depicted in Fig. 2, the server and AP are located in
>>> the 2nd floor. The terminal moves between the far position in the 1st
>>> floor and the near position in the 2nd floor. The distance between the
>>> terminal and AP is about 1.2 meter for the near position and about 10
>>> meter for the far position. During a test run for 120 seconds, the
>>> terminal moves in the following way.
>>>
>>> * It starts communication at the far point and keeps the position for
>>> 30 seconds.
>>>
>>> * Then, it moves from the far position to the near position. The
>>> duration spent for the moving is about 15 seconds.
>>>
>>> * It communicates with the server at the near position for 30 seconds.
>>>
>>> * Again, it moves from the near position to the far position spending
>>> about 15 seconds.
>>>
>>> * In the last part, the terminal keeps the communication for 30
>>> seconds at the far position. "
>>>
>>> (my daydream is that the mobility is done by a robot, but...)
>>>
>>>
>>>>
>>>> @Eloi I have adopted 2 tests that should be of interest. :-)
>>>>
>>>> Best wishes, Amadeus
>>>>
>>>>
>>>>
>>>> ---- Original Message ----
>>>> From: Eloi Carbó Solé <eloicaso at gmail.com>
>>>> To: "Battle of the Mesh Mailing List" <battlemesh at ml.ninux.org>
>>>> Sent: Mi, Apr 30, 2014, 6:05 PM
>>>> Subject: Re: [Battlemesh] Tests
>>>>
>>>> Hi!
>>>>
>>>> I am doing my final degree thesis wibed[0] that will be presented by Pau[1]
>>>> and I am adopting the WBMv6 tests in order to check the mesh network
>>>> behaviour. One of the main points of this thesis will be to validate the
>>>> network, testing and monitoring its behaviour applying these tests. My idea
>>>> is also to adopt and use WBMv7 tests too, so I am really interested in being
>>>> in touch with those persons who will discuss and perform the experiments and
>>>> tests.
>>>>
>>>> Best regards,
>>>>
>>>>
>>>> [0] https://wiki.confine-project.eu/wibed:start
>>>> [1] http://battlemesh.org/BattleMeshV7/Agenda
>>>>
>>>>
>>>> On Tue, Apr 29, 2014 at 8:17 PM, Amadeus Alfa
>>>> <amadeus at chemnitz.freifunk.net> wrote:
>>>>>
>>>>> Hi Colleen,
>>>>>
>>>>> thank you for taking care of testing ;-) The test scenarios and planning
>>>>> will be available for review by the end of this week (will send PDF files to
>>>>> the list). I would love to see your idea of tapping 802.11s in one of the
>>>>> tests - could you please provide some more information so that I can create
>>>>> a short description out of it?
>>>>>
>>>>> Also, we are still looking for testing supporters since there is only one
>>>>> person right now trying to coordinate tasks.
>>>>>
>>>>> Best wishes, Amadeus
>>>>>
>>>>>
>>>>> >---- Original Message ----
>>>>> >From: Colleen T <colleen at cozybit.com>
>>>>> >To: battlemesh at ml.ninux.org
>>>>> >Sent: Di, Apr 29, 2014, 9:33 PM
>>>>> >Subject: [Battlemesh] Tests
>>>>> >
>>>>> >Hello everyone,
>>>>> >
>>>>> >I am looking forward to coming.  I have been looking at the
>>>>> >Battlemesh6 test page.  Will the tests be very similar this year?  Do
>>>>> >we run them over the course of the week?
>>>>> >
>>>>> >I will be bringing some TPLink mr3020's -- not sure how many yet --
>>>>> >and testing open80211s =)
>>>>> >
>>>>> >Cheers,
>>>>> >Colleen
>>>>> >_______________________________________________
>>>>> >Battlemesh mailing list
>>>>> >Battlemesh at ml.ninux.org
>>>>> >http://ml.ninux.org/mailman/listinfo/battlemesh
>>>>>
>>>>> _______________________________________________
>>>>> Battlemesh mailing list
>>>>> Battlemesh at ml.ninux.org
>>>>> http://ml.ninux.org/mailman/listinfo/battlemesh
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Eloi Carbó Solé.
>>>>
>>>>
>>>> _______________________________________________
>>>> Battlemesh mailing list
>>>> Battlemesh at ml.ninux.org
>>>> http://ml.ninux.org/mailman/listinfo/battlemesh
>>>>
>>>
>>>
>>>
>>> --
>>> Dave Täht
>>>
>>> NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.article
>>
>>
>>
>>--
>>Dave Täht
>>
>>NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.article
>>_______________________________________________
>>Battlemesh mailing list
>>Battlemesh at ml.ninux.org
>>http://ml.ninux.org/mailman/listinfo/battlemesh
>
> _______________________________________________
> Battlemesh mailing list
> Battlemesh at ml.ninux.org
> http://ml.ninux.org/mailman/listinfo/battlemesh



-- 
Dave Täht

NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.article

More information about the Battlemesh mailing list