[Battlemesh] tests for WBMv3

[L. Aaron Kaplan]

On May 26, 2010, at 1:10 PM, Alex wrote:

> Am Wed, 26 May 2010 12:50:55 +0200
> schrieb "L. Aaron Kaplan" <aaron at lo-res.org>:
> 
>> 
>> On May 26, 2010, at 12:39 PM, elektra wrote:
>> 
>>> Hello Luca -
>>> 
>>>> what if you want
>>>> to compare three or more protocols? Set up a sort of tournament? 
>>> 
>>> exactly. Lets keep in mind that the boxes are CPU and memory
>>> constrained. I see no problem running two protocols, given that the
>>> protocol implementation is usable. In order to compare convergence
>>> with mobile nodes this the only feasible scenario, IMHO.
>>> 
>> I am not convinced about running three protocols in parallel on the
>> same box.
>> 
>> One after the other would make sense. But in parallel??
>> 
>> Does not make so much sense IMHO especially since it is against the
>> repeatability argument.
> 
> sorry, this is  plain wrong.
> 

listen again to the argument please...I will repeat it below.

> As we cannot repeat conditions, we have to test at least 2 protocols at
> the same time.
> 
> In multiple runs, we can compare.

Sorry I guess your objection has a logical flaw :)
If you can not compare two runs, why in hell should you be able to compare 10 runs?

If you would compare 1000nds of runs - OK, maybe the law of big numbers kicks in.
But I doubt you will have the time for that.

> 
> We can setup simple 2.4 GHz TV-Transmitters to change the enviroment
> noise, to compare under different conditions, but as you already wrote,
> we cannot repeat anything, so we have to test parallel.
> 

As I said before - you can design a network so that you have enough error margin left 
(by using attenuators or shielding or intentional bad cables (--> how about measuring the signal on the wifi cards?) )
for "good" links and make "bad" links really bad. Hence you have a reference setup of how it should route.
And then you make static routes there.
Therefore you have somewhat of a reference which links are to be considered good.

Alex, what you are proposing will simply generate some surprising results but nobody will understand why things are like they are.
It could be that one protocol has a burst of traffic , which in effect creates in the air collisions for the other (parallel) protocol.
So, how do you compare or repeat or debug this situation now? I would be really interested in know that.
Sure, there might be a nice hack for a protocol:

void win_the_battle() {

  if(cmd_line_arg(optarg, "--spam-other-protocols")) {
    establish_optimum_paths();
    stop_updating_routing_table();

    for (int i=0; i < 1000000; i++) {
      p = gen_spam_packet();
      send_out(p);
    }
    sleep(1);
    start_updating_routing_table(); 
    declare_victory();
  }
}

Sure way to "win" in case you really still believe this is a "battle".

> Remember,  it is a Campground, not a lab, it's a Battle, not a free hug
> party.
> 

"battle" bla.. 
No seriously , IMHO if you do all the effort - you should do it properly and *learn* from the results. That's the real value of it!
Or if you just want to have a messy rugby ground (in the wireless field) you will have to accept the fact that the results will not mean much. If you manage to create a repeatable messy situation, then great! I am on your side. But I doubt it.

Since I most probably can not come in person I would really benefit from good, proper documentation and repeatability.
A wireless rugby ground might be the dream of a few, but really, it serves no future purpose.

a.

PS: I don't hope it rains :)