Format of /etc/tertf/mac_vendor.db

ckuethe · 2022-01-25T01:22:05.956Z

As has been noted elsewhere mac_vendor.db is pretty huge and uses up lots of space in /overlay (edit: at least on my MT300N-v2)

I realize that others might be interested in prefix-to-vendor mappings, but I know what 4 devices connect to my travel router. I’d just as soon truncate the database so that all prefixes come from unknown vendors, and use that 920kB of space for something else.

Thus: what’s the on-disk format of this file, so I can create a new one and reclaim the disk space.

wcs2228 · 2022-01-25T03:22:36.409Z

I checked my GL-MV1000W, the mac_vendor.db file is actually in /rom as part of the firmware. In such case, the overlay /etc/tertf//mac_vendor.db file is a “virtual” file and, if you delete it, it only sets an overlay flag and the rom “real” file is still taking up storage.

The file is in sqlite3 format.

ckuethe · 2022-01-25T06:17:46.664Z

Awesome, thanks.

On my MT300N-v2, it’s most definitely in /overlay. Emptying it out and vacuuming the resulting database frees up about 920kB

opkg update
opkg install sqlite3-cli
echo "delete from MAC_VENDOR; VACUUM;" | sqlite3 /etc/tertf/mac_vendor.db
opkg remove --autoremove sqlite3-cli

elorimer · 2022-01-25T12:50:14.291Z

Aren’t you both right? It’s in /rom, and on boot it is copied to /overlay and takes up the limited space there. So deleting it frees up space, but you have to do that each time it boots.

Alternatively, you could build your own firmware with a vacuumed version.

wcs2228 · 2022-01-25T14:17:33.190Z

The Mount Points section in the following URL describes /rom and /overlay, which indicates that /rom is not copied to /overlay:

OpenWrt Wiki – 18 Jan 10

The OpenWrt Flash Layout

The OpenWrt Flash Layout The embedded devices (routers and such) OpenWrt/LEDE (Linux Embedded Development Environment) has mainly targeted since its inception, use flash memory as the form of non-volatile memory for the persistent storage of the...

Post 7 of the following thread also describes how /overlay works:

OpenWrt Forum – 29 Nov 17

Looking for tutorial for extroot, /mnt, /overlay, etc

LEDE on embeded devices with low flash it uses two filesystems. one is SquashFs, that allow to put tight all the ROM part, that is basically the firmware. and when it boot for first time it put the initial configuration files on OverlayFS, this is...

ckuethe · 2022-01-25T18:50:46.441Z

elorimer:

Aren’t you both right? It’s in /rom, and on boot it is copied to /overlay and takes up the limited space there. So deleting it frees up space, but you have to do that each time it boots.

It’s not copied on boot, at least not on my MT300N-v2. Maybe it is on first boot, but here’s what I see after truncating and vacuuming the database, and rebooting:

root@router:/# ls -il /etc/tertf/* /rom/etc/tertf/* /overlay/upper/etc/tertf/*
3177 -rw-r--r--    1 root     root         12288 Jan 25 06:15 /etc/tertf/mac_vendor.db
4437 -rw-r--r--    1 root     root           472 Jan 25 07:04 /etc/tertf/tertfinfo_bak
3177 -rw-r--r--    1 root     root         12288 Jan 25 06:15 /overlay/upper/etc/tertf/mac_vendor.db
4437 -rw-r--r--    1 root     root           472 Jan 25 07:04 /overlay/upper/etc/tertf/tertfinfo_bak
 767 -rw-r--r--    1 root     root        950272 Feb 15  2021 /rom/etc/tertf/mac_vendor.db

There was no tertfinfo_bak in the /rom squashfs, so one was created, and that write operation was trapped, and redirected into /overlay/upper. After disassembling /usr/bin/gltertf (the only binary in /usr to reference mac_vendor.db), it appears that sqlite is doing something clever/stupid since gltertf is the only thing that opens mac_vendor.db and only to do a SELECT. And now that that file was written, it’s being served from /overlay/upper rather than /rom.

elorimer:

Alternatively, you could build your own firmware with a vacuumed version.

Too much work. I don’t want to maintain build infra for my various gl.inet products. Easier and more maintainable to keep a shell script to apply my local changes: opkg update, install my ssh keys, install some packages, configure things, patch out undesirable behavior from the startup scripts… So I can either temporarily install sqlite to truncate the database, or just keep a copy of the truncated database for my configurator script to install.

ckuethe · 2022-03-07T20:53:07.217Z

An even easier workaround - which does not require installing sqlite and that doesn’t cripple the behavior of tertf - is this:

cd /overlay/upper/etc/tertf
rm -f mac_vendor.db
ln -s /rom/etc/tertf/mac_vendor.db .
reboot

Ideally the database should be opened read-only which would mean that nothing would be written to /overlay. This might be fixed in fw4 but until that happens, a symlink to /rom will cause writes to fail but reads still work.

elorimer · 2022-03-07T21:35:14.059Z

Yes, that saves memory in overlay, but you still have 1M of it in /rom. The trick is to get that out of /rom, make the firmware smaller, so more flash is available in overlay from the get go. Isn’t it?

So, I can produce a shrunk mac_vendor.db, but now how do I inject that into imagebuilder to replace what is there?

wcs2228 · 2022-03-07T22:39:52.623Z

Can you inject it into imagebuilder as a custom file?

OpenWrt Wiki – 19 Dec 16

Using the Image Builder

Using the Image Builder See also: Image Builder frontends, Using the toolchain, Quick image building guide The Image Builder (previously called the Image Generator) is a pre-compiled environment suitable for creating custom images without the need...

I do not work for and I do not have formal association with GL.iNet

elorimer · 2022-03-08T21:31:11.062Z

Haven’t figured out how. I took the 928kB mac_vendor.db and vacuumed it down to 12kB. Then I copied it over to imagebuilders /files directory as /files/etc/tertf/mac_vendor.db, and then built a new firmware with no other changes. I ended up with the 928kB mac_vendor.db; I thought it might have taken my vacuumed version instead.

wcs2228 · 2022-03-08T23:39:09.995Z

Imagebuilder may only process complete packages, not individual files.

You can try the following:

Build the new firmware without gl-tertf, which other packages do not have dependency on
Install the firmware, then install the gl-tertf package manually
Replace the mac_vendor.db file with the empty file

I do not work for and I do not have formal association with GL.iNet