[ninux-dev] Info libre-mesh, openwrt, nodeshot, netengine

[Gioacchino Mazzurco]

Jow di openwrt stava dando info interessanti su IRC riguardo alle api JSON di 
ubus per prendere le info e/o controllare il device, credo che vi possa 
interessare

[16:54:40] <karlp> jow_laptop: I'm trying to follow 
http://wiki.openwrt.org/doc/techref/ubus#access.to.ubus.over.http but getting 
a bit stuck,
[16:54:50] <karlp> I can use ubus -v call file read '{"path": "/tmp/tfile"}' 
from the cli, and that works well,
[16:54:58] <karlp> but curl --data "method=list" http://eg-134867.local/ubus/
[16:55:11] <karlp> and curl --data-urlencode 
"method=call&object=file&function=read&data={'path':'/tmp/tfile'}" 
http://eg-134867.local/ubus don't work at all
[16:55:26] <karlp> I just get {"jsonrpc":"2.0","id":null,"error":
{"code":-32700,"message":"Parse error"}} returned all the time
[16:58:29] <cyrusff> karlp: you have to send json in the post-body
[16:58:38] RealOpty [~openwrt at 24.116.69.215] has joined #openwrt-devel
[16:58:39] <cyrusff> see jsonrpc-specs
[16:58:52] <karlp> wher eis jsonrpc-specs?
[16:58:56] <cyrusff> google ;)
[16:59:06] <cyrusff> version 2.0
[16:59:09] <cyrusff> for the specs
[17:00:07] <karlp> ok, wasn't aware this was any standard thing :)
[17:00:49] <karlp> the access.ubus.over.http section was just showing 
"method=call&sid=...&object=...&function=...&data=..."
[17:00:54] <cyrusff> yeah you can actually find implementations for most major 
programming languages
[17:01:16] <cyrusff> this wiki page looks wrong if you ask me
[17:01:50] <jow_laptop> yeah that wiki page is completely wrong
[17:01:56] <karlp> that would agree with what I was seeing :)
[17:02:03] <jow_laptop> its also way more complicated
[17:02:20] <jow_laptop> for the current ubus via http you need to setup acls 
and login to be able to see anything
[17:02:44] <jow_laptop> unless you disable authentication in uhttpd-mod-ubus 
with the "-a" flag
[17:03:05] <karlp> so /etc/config/uhttpd just has "option ubus_prefix '/ubus'"
[17:03:14] <karlp> and that won't actually do anything without further config 
then?
[17:03:58] <jow_laptop> it will
[17:04:14] <jow_laptop> but you'll get permission denied for all accesses 
because you need to setup acls first
[17:04:18] <jow_laptop> moment...
[17:06:35] <jow_laptop> #1 an acl definition in 
/usr/share/rpcd/acl.d/whatever.json - http://pastebin.com/Pc9GgSsv
[17:07:19] <karlp> ok, and jsonrpc docs say "{"jsonrpc": "2.0", "method": 
"subtract", "params": {"subtrahend": 23, "minuend": 42}, "id": 3}" for calls 
with named params
[17:07:36] <karlp> how does that translate to ubus call <path> <method> 
[<message with further named args>]
[17:07:50] <jow_laptop> #2 a login account in /etc/config/rpcd: 
http://pastebin.com/AQYBzxqD
[17:08:45] <karlp> ok, I already have #2 out of the box on current trunk it 
seems
[17:09:14] <jow_laptop> nope, only an unauthenticated acl which just grants 
enough rights to allow you to call login
[17:09:44] <karlp> what should "wahtever" be, in /usr/share/rpcd, there's 
already unauthenticated.json,
[17:09:53] <jow_laptop> name is arbritary
[17:09:58] <jow_laptop> the files are read via globbing
[17:10:09] <jow_laptop> but you can call it "superuser" or "root" for clarity
[17:11:07] <karlp> you mean there can only be one file?
[17:11:18] <jow_laptop> there can be as many as you like
[17:11:23] <jow_laptop> they're all merged
[17:12:15] <karlp> so the top level key in blah.json and whatever.json is used 
rather thean the filename then I presume, and where do _those_ match to?
[17:12:42] <jow_laptop> to the "list read" and "list write" keys in config 
login of /etc/config/rpcd
[17:12:50] <jow_laptop> those map acl roles to user accounts
[17:12:53] <karlp> right, thanks :) that's the bit I was missing
[17:13:33] <jow_laptop> on to step #3 - perform a login: 
http://pastebin.com/awTkKHHU
[17:14:02] <jow_laptop> the sessionid "00000000000000000000000000000000" is a 
speciall nul-session which just has enough access rights for the session.login 
ubus call
[17:14:13] <karlp> does the number of those 0s matter?
[17:14:18] <jow_laptop> yes, 32
[17:16:19] <karlp> ok, calling the login without setting up the acls gives me 
no erorrs, just a {"jsonrpc":"2.0","id":1,"result":[6]} which is ok, I didn't 
follow the rules, but is that what's meant to be received if you don't have 
access?
[17:17:36] <karlp> in your example superuser.json file, it lists ubus: * and 
uci: *
[17:17:50] <jow_laptop> yes, code 6 is the ubus return code - 
UBUS_STATUS_PERMISSION_DENIED
[17:18:09] <karlp> is it correct to presume that ubus: [file,log, service] 
would be valid too? and do you need uci:* when uci is avialable via ubus as 
well?
[17:18:20] <karlp> that's just for the existing json-rpc for uci, from luci1 
right?
[17:19:13] <jow_laptop> ubus: { file: [ "*" ], log: [ "*" ], service: [ "*" ] } 
would be valid
[17:19:38] <jow_laptop> it would grant access to ubus call file *, ubus call 
log *, ubus call service *
[17:19:47] <karlp> ok, that makes sense.
[17:20:07] <jow_laptop> the uci: scope is used for the uci api provided by 
rpcd
[17:20:18] <jow_laptop> to allow defining per-file permissions
[17:20:29] <jow_laptop> because using the ubus scope you can onyl say uci set 
is allowed or not allowed
[17:20:47] <jow_laptop> but not specify that it is allowed to e.g. modify 
/e/c/system but not /e/c/network
[17:21:23] <jow_laptop> if you do not use the ubus uci api then you do not 
need to define uci: acls either
[17:22:38] <karlp> ok, do you have an example of making a call now that I have 
the ubus_rpc_session key now?
[17:22:45] <jow_laptop> http://pastebin.com/KiixX6Vt
[17:23:54] <jow_laptop> so the json container format is:   { "jsonrpc": "2.0", 
"id": <unique-id-to-identify-request>, "method": "call", "params": [ 
<ubus_rpc_session>, <ubus_object>, <ubus_method>, { <ubus_arguments> } ] }
[17:24:01] <karlp> yay, it worked.
[17:24:26] <karlp> ok, thanks heaps, now to spend a bunch of time writing into 
the wiki :|
[17:24:43] <jow_laptop> the "id": key is merely echo'ed by the server, so its 
most not be strictly unique
[17:24:45] <karlp> I'll just replace the whole section in 
http://wiki.openwrt.org/doc/techref/ubus#access.to.ubus.over.http ok?
[17:25:00] <jow_laptop> its mainly intended for client software to easily 
relate responses to previously made requests
[17:25:03] <jow_laptop> yep
[17:25:14] <jow_laptop> *it must not be strictly unique
[17:25:44] <jow_laptop> its type is also not specified afair
[17:26:02] <jow_laptop> can be an sha1 hash, md5 sum, sequence counter, unix 
timestamp, ...
[17:26:09] <jow_laptop> either number or string
[17:26:29] <karlp> I was looking into this to try and make some web pages 
simpler than calling luci1 lua controllers to do stuff and squirt responses 
back, I guess this moves all the coding into my javascript instead now :)
[17:26:38] <jow_laptop> yes
[17:26:52] <jow_laptop> there is also some initial luci2 prototype implemented 
in js
[17:27:03] <karlp> yeah, I've seen teh discussions a bit, haven't tried it out 
yet
[17:27:25] <karlp> eventually there'll be a magical ubus-websockets right? ;)
[17:27:32] eva42_ [~eva42 at c-68-56-185-188.hsd1.fl.comcast.net] is now known as 
eva42
[17:27:37] aversario [~chatzilla at p549B3233.dip0.t-ipconnect.de] has joined 
#openwrt-devel
[17:27:40] <jow_laptop> dunno, if someone implements it - why not
[17:28:00] <jow_laptop> as far as I understood it, just the transport layer 
would change
[17:28:04] <jow_laptop> the json protocol would remain
[17:28:46] <karlp> yeah, not my strength either
[17:29:11] <jow_laptop> one thing I will add soon is to add CORS headers to 
uhttpd-mod-ubus
[17:29:28] <jow_laptop> so ajax2 capable browser can access it from anywhere, 
not just within the same origin domain
[17:29:54] <jow_laptop> this way you can host the webui somewhere else and can 
login using ip + user + pass to any router
[17:30:00] <karlp> I've got an mqtt broker on my device, just because it plays 
with more people than ubus/dbus, at least from my poitn of view, and I want to 
get a lot of that out into the browser, we've been doing writing out to files 
and lua controllers to read them in again, and delete old ones, which is a bit 
lunky.
[17:30:16] <karlp> sounds interesting
[17:30:28] <karlp> I need to make sure https works too really,
[17:30:40] <jow_laptop> yeah, thats more or less a precondition
[17:31:55] <karlp> at least it only needs _a_ cert, it can be the same cert on 
all devices
[17:32:02] <jow_laptop> btw, here's a more complicated acl file as used by 
luci2 atm: 
http://git.openwrt.org/?p=project/luci2/ui.git;a=blob;f=luci2/share/acl.d/luci2.json
[17:32:25] trapier [~trapier at cpe-174-097-188-117.nc.res.rr.com] has joined 
#openwrt-devel
[17:32:25] <jow_laptop> instead of granting access to anything it defines 
multiple scopes needed for various pages
[17:32:32] <karlp> thanks, that's a good example to help understand what you 
can do
[17:33:20] <jow_laptop> and an example guest account: 
http://pastebin.com/gfcMDY7p
[17:33:23] <karlp> do those then need different users in /etc/config/rpcd for 
each of them?
[17:33:38] <jow_laptop> read only access to everything except the software 
tasks (opkg calls etc.)
[17:33:48] <jow_laptop> no write access
[17:33:57] <karlp> so $p$xxxx is passwords from /etc/passwd
[17:34:11] <jow_laptop> $p$<user> refers to /etc/shadow
[17:34:12] <karlp> what's $1? is this some traditional linux password thing I 
should know from somewhere else?
[17:34:26] <jow_laptop> $1$ is just a crypt() hash
[17:34:32] <jow_laptop> using sha1
[17:34:37] <jow_laptop> the same stuff used in /etc/shadow
[17:34:53] <jow_laptop> you can generate it using e.g. "uhttpd -m secret"
[17:35:30] JyZyXEL [~foo at a88-115-40-53.elisa-laajakaista.fi] has joined 
#openwrt-devel
[17:35:46] <karlp> so in the big luci2.json, those are just _definitions_ of 
sections and access,
[17:35:47] <jow_laptop> or on a normal linux using "mkpasswd -m md5 secret"
[17:35:54] <jow_laptop> right
[17:36:07] <karlp> you still need to setup the rpcd users to give accesses to 
each of those blocks
[17:36:14] <jow_laptop> yes
[17:36:47] <karlp> ok, my brain's about full for right now, I'll try and 
digest a bit of this and write it up, but there'll probably still be mistakes 
on the wiki when I'm finished :)
[17:36:49] <jow_laptop> and the default account shipped is 
http://pastebin.com/wrBkeHix
[17:37:20] <jow_laptop> means: username root, password from root (same as ssh 
etc.) and read+write access to any definition in any 
/usr/share/rpcd/acl.d/*.json
[17:37:43] <karlp> oh, ok, and because the default _definitions_ are that 
unauthenticated.json, you can login, but you can't actually do anyuthing even 
after logging in
[17:37:52] <jow_laptop> exactly
[17:38:04] <jow_laptop> if you issue via ssh "ubus call session list"
[17:38:18] <jow_laptop> you should see a dump of all sessions including their 
effective acls
[17:40:19] <mhei> Hi all. I would like to play a little bit with an own 
instance of a buildbot. Is the configuration/scripts which drives the buildbots 
somewhere documented. I didn't find anything at the wiki/google...
[17:43:37] <karlp> one more thing, currently, with the lua controllers, once 
you're logged in, you're in, and I don't hve to do anything to handle session 
expiry/timouts,
[17:43:59] <karlp> do you have any advice on how to handle this via the 
jsonrpc interface?
[17:44:20] <jow_laptop> sessions timeout after by default 300s
[17:44:31] <karlp> I saw that I got :-32002, Access denied when the session 
had expired
[17:44:46] <jow_laptop> whenever you use the session, e.g. performing a call, 
the coutdown is reset
[17:45:07] <karlp> just hinking about how to handle this in the UI actually,
[17:45:26] <jow_laptop> luci2 performs aheartbeat every 5 seconds
[17:45:51] <jow_laptop> it calls session.access on its own sessions to see 
whether it is still logged in
[17:45:59] <jow_laptop> if not, the login prompt will pop up
[17:46:35] <jow_laptop> so as  long as the network connectivity works, the 
session never expires
[17:48:39] <karlp> ok, that's if I'm in the luci2 ui. fair enough :)
[17:48:55] <karlp> I was looking at using this in our existing luci app, but I 
guess that's my problem then :)
[17:48:57] <jow_laptop> "timeout": is a property of the login call btw, so you 
can specify something higher - like 3600s
[17:49:15] <jow_laptop> I plan to switch luci1 to ubus sessions as well
[17:49:43] <karlp> before or after barrier breaker gets released? ;)
[17:49:47] <jow_laptop> but not decided yet as this would pull in rpcd and 
uhttpd-mod-ubus as requirements, making it bigger again
[17:50:04] <karlp> would be a nicer position to go forward from though,
[17:50:17] <jow_laptop> on the other hand I got remove a lot of server side 
data gathering code from the controlers and do it in js instead
[17:50:22] <jow_laptop> *I could
[17:54:44] RooTer [~rooter at 87-205-93-248.adsl.inetia.pl] has quit IRC: Ping 
timeout: 252 seconds
[18:00:11] <karlp> so, the /ubus url, that's not really actually _ubus_, 
that's rpcd? is that correct?
[18:00:47] <jow_laptop> no, uhttpd
[18:01:03] <jow_laptop> rpcd is just a service which happens to provide the 
serssion.* namespace which uhttpd uses to aurthenticate
[18:01:06] <jow_laptop> brb
[18:02:19] Mazilo [~quassel at 2601:0:8d80:82f:224:8cff:fec3:4a28] has quit IRC: 
Ping timeout: 240 seconds
[18:03:04] Moon_Cheetah [~Moon_Chee at 222.211.89.148] has joined #openwrt-devel
[18:04:30] <karlp> the reason I asked, is that ubus from the command line 
doesn't need any logins or acls, but uhttpd-mod-ubus does, and those acls are 
setup in the rpcd directory, not any ubus directory
[18:08:47] Moon_Cheetah [~Moon_Chee at 222.211.89.148] has quit IRC: Remote host 
closed the connection
[18:10:01] kako [~kako at host31-54-89-15.range31-54.btcentralplus.com] has quit 
IRC: Quit: Verlassend
[18:14:32] <jow_laptop> yes, basically for any request received by /ubus, 
uhttpd runs "ubus call session access '{ ubus-rpc-session, requested-object, 
requested-method }'
[18:14:48] <jow_laptop> and however is providing the ubus session.* namespace 
is in charge of implementing the acl
[18:14:53] <jow_laptop> this happens to be rpcd
[18:15:05] <jow_laptop> *whoever
[18:15:34] <jow_laptop> in the very beginning the sessioning was part of 
uhttpd but we decided it belongs into a separate service
[18:16:05] <jow_laptop> because the ubus-json-http bridge is just one of 
various possible access methods for ubus
[18:16:47] <jow_laptop> another theoretical access method would be a tcp 
server directly speaking blobmsg, instead of using the json<>blobmsg 
conversion with http as transport
[18:17:00] <jow_laptop> that json+http approach was implemented to be browser 
friendly
[18:17:33] <jow_laptop> for native client applications, e.g. some Qt program 
running on a mobile phone there'd be more efficient protocols, like plain tcp + 
tls + blobmsg