[Battlemesh] battlemesh v6 test data postprocessing

Juliusz Chroboczek jch at pps.univ-paris-diderot.fr
Sat May 17 17:37:37 CEST 2014


>> The pair of nodes was chosen randomly at each single test.  And, we
>> didn't collect enough measurements to obtain the same set of nodes
>> for all the protocols.

> Oh, so nothing to worry about.  This data is clearly invalid.

Clauz,

Sorry if that was strongly worded, but please realise that this is not
a useful test setup.

The tests that we used to do at Battlemesh in the olden times served
two purposes:

  (1) find out how the different routing protocols are doing in
      real-world scenarios;
  (2) uncover issues and help the protocol developers work them out.

Unfortunately, the last time I got any useful data from Battlemesh was
in Catalogna.

Your test fails to achieve either of the points above.  Actually, they
are worse than useless, since they might be misleading and cause us to
spend time chasing nonexistent issues.

  (1) random tests on such a small sample do not provide any statistically
      meaningful results;
  (2) without a clear description of the scenarios, there is no way to
      work out what is going on.

I'd really like to encourage everyone present at WBM this year to work
on providing useful results, both to users and to protocol developers.

-- Juliusz



More information about the Battlemesh mailing list