[Battlemesh] No results again

[Pau]

Thank you for your efforts Federico, I saw you working hard and taking
several responsibilities.

In general I agree with all you say. However I think it is not fear to
blame the firmware 'wibed' as the main cause for the experimentation
failing. In my opinion, the main problem has been the low involvement of
the people. From a total of 40-60 nerds only a group of 4-5 were really
taking care of the whole tasks for the experimentation: preparing the
hardware, preparing the firmware, preparing the experiments and so on.
And the most frustrating part, for me, was that once the testbed was
really running no one was interested on making the experiments.

There have been now three consecutive years using Wibed, and there have
been several improvements year after year. The funny part here is that
in Leipzig we manage to make experiments and generate results using
WiBed. So please don't say it was never working.

The main problems we had this year, under my point of view, were:

1) Using a single 2.4GHz radio for management network is not enough,
most of the problems were because of the weakness of the network. It was
really hard or even impossible to distribute the Overlays and execute
the remote commands. The last day we change to dual-band radio for
management and suddenly it worked much better.

2) The overlay for experimentation was not ready and it required new
coding. For instance we implement a new IP approach, which required
implementation, testing, failing and fixing. But it has nothing to do
with Wibed.

We can start a new firmware again the next year, going back to the old
approach and installing everything into the ROM. However flashing all
nodes (~40) is not an easy and quick task (we already know this). Thanks
to the approach of Wibed (use a generic ROM and install the protocols
and testing suit into an external USB overlay distributed by a central
controller), we could install and test different overlays without the
need of re-flashing everything.

In addition, it implements security mechanisms, such as auto-remove the
overlay and go back to the stock ROM in case of a failure. For instance
in some moment we distributed a wrong command and all nodes were
inaccessible. But in no more than 20 minutes, all nodes were again
online and working, without the need of re-flashing.

Anyway, the next year the firmware/testing team will choose if using
WiBed or if start a new firmware. I strongly think it would be worth to
keep using and improving it, but it is not up to me.

What I really think we should discuss, is the fact that people seem to
be no longer interested on running experiments. It might be fine, but we
should put this on the table and maybe change the focus of this
nice-wonderful event.

Thanks to everyone who was there, it has been a really nice experience.

Cheers.

On 09/05/16 00:35, Nemesis wrote:
> Hi everyone,
> 
> I'm sorry to disappoint who's following this list and who was waiting
> for results, but this year we have none.
> 
> We tried very hard, but a series of mistakes and technical issues
> delayed the tests until it was too late.
> 
> I understand what we are doing is hard, but I really do not buy nor
> endorse the argument that we did not have enough people involved or is
> just too hard, because we had several people participating and 3-5
> people constantly working on the testbed. I remember especially Guido
> Ibarren and Alessandro Gnagni working tirelessly.
> 
> We have to realize and accept we have failed for precise reasons, the
> approach we used is not effective. Please do not get aggressive or
> hyper-defensive.
> 
> Having worked on the testbed actively, I also failed, so I also assume
> responsibility for failure.
> 
> To sum up, I attribute the failure mostly to these reasons:
> 
> * slow iteration: for every iteration we had to wait wibed nodes to be
> ready, but every time there was a different problem to debug because
> nodes were not becoming ready
> * lack of communication; I tried hard doing daily briefings and FAILED
> miserably; it just won't happen if we don't schedule them in advance and
> enforce them by taking the microphone and asking all the poeple involved
> to stop doing what they are doing and go there to report what problem
> they have and how other people can help
> * configuration mess: complex configuration, too many changes
> 
> I really love the battlemesh: there has never been another event I
> attended 5 years in a row. I love it also because of the test battle.
> Without this peculiar feature the event would be just another conference
> and I'm not sure I would be willing to come every frigging year: there
> are so many conferences I want to attend but I can't got to all of them,
> I have to choose and I allocate a slot to the battlemesh every year
> because its unique mixture of technical talks and live hacking.
> 
> Therefore I don't want to give up the battle and I don't think getting
> results is a mission impossible. That's why at the next battlemesh I
> want to try a different approach.
> 
> After having worked with wibed I can say without doubts that I'm not
> interested in fixing it. At work I manage thousands of OpenWRT devices
> every day without any issue, configurations get updates in minutes.
> I think other solutions to accomplish the same tasks exist, are simpler
> to use, better maintained and more effective.
> 
> I also think we should not hold a monopoly on the testbed, especially if
> the current approach has failed more than once. There are about a
> hundred people coming to the battlemesh, I firmly believe we can afford
> having 2 to (max) 3 testbeds taken care by different groups of people
> who can then run the tests they are most interested in.
> 
> Last year (battlemesh v8 in Slovenia) the failure of the wibed testbed
> was attributed to me and other people who worked on a manually run
> testbed which according to them split the workforce.
> This year we all worked on a single testbed, including all the routing
> protocol developers that were present, and it didn't work.
> 
> Next year please do not attack or blame those who want to try a
> different approach.
> 
> I'm really looking forward to the next battlemesh and I hope we'll all
> be reasonable people, find solutions, compare approaches and use the
> most effective ones that allows us to get results.
> 
> Federico
> 
> 
> 
> _______________________________________________
> Battlemesh mailing list
> Battlemesh at ml.ninux.org
> http://ml.ninux.org/mailman/listinfo/battlemesh
> 

-- 
./p4u

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://ml.ninux.org/pipermail/battlemesh/attachments/20160509/cc8940d8/attachment.pgp>