[Battlemesh] ImageBuilder frontend projects, or how to generate custom OpenWrt images

Russell Senior russell at personaltelco.net
Tue Oct 22 00:02:03 CEST 2019


On Sat, Oct 19, 2019 at 6:00 AM Baptiste Jonglez <
baptiste at bitsofnetworks.org> wrote:

> Hi,
>
> On 30-09-19, Russell Senior wrote:
> > > As a side-note, does anybody use uci-defaults scripts?
> > > https://openwrt.org/docs/guide-developer/uci-defaults
> > >
> > > It seems like the best way to implement customization without having to
> > > update file templates with each OpenWrt release, but during my quick
> > > overview tour I haven't noticed any project using this method.
> > >
> >
> > Yes, we use it. Iirc, one thing you need to be aware of is that the
> > uci-defaults scripts run after any restored settings. This isn't a
> problem
> > for us, because we never save settings when reflashing. I have some
> > infrastructure that saves the initial uci config and diffs files in the
> > overlayfs, so that any local changes can be noticed and folded into
> > replacement firmware.
>
> Thanks for this input!
>
> I take it that you generate firmware images then, which tool do you use?
> Just the imagebuilder + uci-defaults scripts, or something more elaborate?
>

I'm only slightly embarrassed to say that I just use the normal OpenWrt
build system. We are generally building OpenWrt master HEAD, so we are
subjected to more upstream volatility than if we were sticking to a stable
branch. We have approximately a dozen each of about 5 different gateway
devices in our network, and building images has been a semi-manual
process.  We have a database of node configuration values available as json
from an api [1], and a script [2] that extracts values from the database
and inserts them into a template for the files overlay. The template uses
uci-defaults scripts. The reason we moved to using files/etc/uci-defaults/
scripts instead of static files/etc/config/ files is so that we don't
lazily miss OpenWrt config changes.

Our process is something like this:

 a) we git pull the OpenWrt tree, and update feeds;
 b) we build an image with an auto-generated files overlay tree and test it
on a local device in our test bed and make sure it basically works;
 c) we auto-generate the files overlay tree for a remote device;
 d) we cp -a that files overlay to a files-orig tree;
 e) we look at the build artifacts from the last version we flashed on the
remote devices and look at the diff -ruN between the files-orig and files
we used in generating that prior version and apply the patch (sometimes
with minor changes or fixups) to the newly auto-generated files overlay;
 f) we exploit the squashfs+overlayfs file system on the remote device to
check to see what we might have adjusted since the last time we flashed
[3], and make whatever adjustments that seem appropriate to the new files
overlay (we capture the initial uci config shortly after first-boot [4],
and have diffutils available on the devices);
 g) we build the image with the modified files overlay and scp it to the
remote device and "sysupgrade -v -n" it, and make sure it comes back up, is
phoning home, and seems to be working;
 h) we save the build artifacts for that remote device including:
  1) the bin directory;
  2) the .config
  3) the output of the scripts/diffconfig.sh;
  4) files-orig tree (the auto-generated tree);
  5) files tree (actually used in the build);

Because there are changes in OpenWrt over time, this process requires human
oversight to get right with a high probability. The consequences of getting
it wrong are painful enough (down time and driving across town in order to
recover) that we have been reluctant to further automate it. At our scale
and development pace, it works for us.

Because it is almost always me building the images, and I have the full
build system available, it doesn't really make sense to use the
imagebuilder.

[1] https://personaltelco.net/api/v0/hosts
[2] https://github.com/personaltelco/ptp-openwrt-files
[3]
https://github.com/RussellSenior/ptpcab/blob/master/ptp-utils/files/since-last-flash.sh
[4]
https://github.com/personaltelco/ptp-openwrt-files/blob/master/etc/uci-defaults/ptp.zzz.defaults
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ml.ninux.org/pipermail/battlemesh/attachments/20191021/8d4668d1/attachment.html>


More information about the Battlemesh mailing list