[Nodeshot] TX, RX rate

[nemesis]

 On Fri, 08 Nov 2013 12:36:31 +0100, Clauz <clauz at ninux.org> wrote:
> On 11/08/2013 12:17 PM, nemesis wrote:
>>
>> [CUT]
>>
>>
>>>> So to summarize, the questions are:
>>>
>>> Summarizing my opinions below:
>>>
>>>> * should every interface type have tx_rate and rx_rate or not?
>>>
>>> No.
>>>
>>>> * if not, which types?
>>>
>>> Only wireless 802.11 and wired ethernet.
>>>
>>>> * should link objects have tx_rate and rx_rate?
>>>
>>> Yes.
>>>
>>>> * if yes how should those values be determined?
>>>
>>> min_rate and max_rate instead of computing the average.
>>
>> do you mean:
>>
>> * min_tx_rate
>> * min_rx_rate
>> * max_tx_rate
>> * max_rx_rate
>
> No, this means that you have a "rx" and "tx" side. How do you choose
> which side is rx and which side is tx? What I was suggesting is
> replacing the pair (rx_rate, tx_rate) with the pair (min_rate, 
> max_rate).

 Ok, now I got it, thank you, sounds good.

>> Sounds good. On the UI we could show an average with
>> besides/below/onrollover all these values.
>>
>> Will it be problematic to update all these info very often?
>>
>> Immagine the flow:
>>
>> 1. system goes to find the tx/rx, dbm, ecc.
>> 2. updates Interface A
>> 3. updates Interface B
>> 4. updates Link object
>>
>> They're not very complex queries though, it should be ok.
>
> It depends on how you implement point 1. Using ssh to connect to all
> nodes very often looks to me like adding a lot of overhead to the 
> network.
>
>> I was thinking to implement some realtime updating too, for example, 
>> you
>> open up a device in the web interface, and you get real-time data
>> through a websocket connection between the browser and the server, 
>> the
>> server retrieves the info via one of the "connectors" (SSH, SNMP, 
>> HTTP,
>> ecc) every x secs and sends it back to the all the listening 
>> browsers on
>> that channel (so each opened device page will be a channel in order 
>> to
>> avoid crashing the server in case of many users watching the same 
>> page).
>
> wow!
>
>> I'm worried that the final result will be heavy from a hardware 
>> resource
>> point of view (many queries, often). I already see that when 
>> performing
>> some operations many queries are generated, but maybe we can take 
>> care
>> of optimization later.
>
> The overhead might be related to the connection method (e.g. ssh), I
> don't think actual queries are heavy.

 We can use SNMP or any other protocol that feels lighter.
 Do you have the link to the work that is being done on NodeDB?

 Regarding the connection method, there is a lot we can do as a group.

 I want to explain to everyone what I have been done latelly with the 
 help of Fabys, Lux and Matteo (Ninux.org).

 Check these out:
 https://github.com/nemesisdesign/nodeshot/blob/refactoring/nodeshot/networking/connector/classes/ssh.py
 https://github.com/nemesisdesign/nodeshot/blob/refactoring/nodeshot/networking/connector/classes/UbiquitiAirOS.py
 https://github.com/nemesisdesign/nodeshot/blob/refactoring/nodeshot/networking/connector/classes/OpenWRT.py

 These classes could be moved to an ssh/ folder;

 Then we could do more folders, one for each protocol/method we use, eg: 
 snmp, munin

 For each firmware we want to use we will have to write classes similar 
 to (for example) UbiquitiAirOS, which implement the same methods, but 
 instead of using SSH we'll use the specific library for the protocol we 
 want to use.

 This is basically an abstraction layer, in a similar way to how famous 
 frameworks have built ORMs (Object Relational Mappers) for popular 
 relational databases.

 For example, in Django you can do:
 Device.objects.all()
 while in Ruby on Rails you can do:
 Device.all

 And these calls will translate to an SQL query that is compatible with 
 the database specified in the project configuration.

 With this thing i called "connector" system I want to achieve something 
 similar, my final goal is to do:

 # retrieve device object from DB with an SQL query
 device = Device.objects.get(mac='00:15:6d:9f:cf:ec')
 # retrieve wireless channel by connecting to the device through the 
 network
 device.connector.get_wireless_channel()

 To retrieve information of a device through the network. The system 
 will know which protocol and settings to use based on the information 
 contained in the database for that specific device, so each device might 
 use different connector classes if needed, or a device might even 
 specify more connectors with an order of preference (eg: try SNMP, then 
 SSH if SNMP doesn't work).

 Right now we have something similar, with a slightly uglier syntax:

 # retrieve device object from DB with an SQL query
 device = Device.objects.get(mac='00:15:6d:9f:cf:ec')
 # retrieve first connector of a device, then use the connector to 
 retrieve the wireless channel
 device.device_login_set.all()[0].connector.get_wireless_channel()


>> Pheraphs here there are some optimization or profiling experts?
>
> not me :)
>
>> Anyone has experience in profiling / automated performance testing /
>> stress test?
>
> To get an idea of what is happening perhaps we can just run "top" on 
> the
> devices while running "information fetching" scripts.
>

 That is surely going to be the first step of this process.


 Federico