BanIP on AX1800 after upgrade to 4.1 does not load dowloaded lists

mk808 · February 22, 2023, 9:05pm

Recently upgraded to 4.1 Firmware and BanIP no longer loads lists.

No obvious error in log files however report shows 3 IPSets with 0 IPs/Prefixes

Files are being retrieved via curl and deposited in /tmp/banIP-backup as per norm however no lists are generated in /tmp/banIP-Report

No error files in any of the verbose logs , the only error file i can find is in syslog

“daemon.err banip.sh[17204]: sh: out of range”

Local whitelists and blacklists etc still function normally but it appears not to be able to process downloaded lists after upgrade

Bitty_Bytes · February 23, 2023, 12:15am

What version of banIP are you running? I took a quick glance for “out of range” errors and did not find anything (other than an unrelated fixed bug report). Do you have plenty of memory available to process the blocklists? I know the firmwares are getting chunkier now with each upgrade and it sounds like you kept prior packages on the upgrade.

Have you tried to uninstall it and reinstall?

stop all banIP related services with /etc/init.d/banip stop
optional: remove the banip package (opkg remove banip)

Also, did you get any information from the CLI commands for status, report or lists that was out of the ordinary?

Syntax: /etc/init.d/banip [command]

Available commands:
	start           Start the service
	stop            Stop the service
	restart         Restart the service
	reload          Reload configuration files (or restart if service does not implement reload)
	enable          Enable service autostart
	disable         Disable service autostart
	enabled         Check if service is started on boot
	refresh         Refresh ipsets without new list downloads
	suspend         Suspend banIP processing
	resume          Resume banIP processing
	query           <IP> Query active banIP IPSets for a specific IP address
	report          [<cli>|<mail>|<gen>|<json>] Print banIP related IPset statistics
	list            [<add>|<add_asn>|<add_country>|<remove>|<remove_asn>|<remove_country>] <source(s)> List/Edit available sources
	timer           [<add> <tasks> <hour> [<minute>] [<weekday>]]|[<remove> <line no.>] List/Edit cron update intervals
	version         Print version information
	running         Check if service is running
	status          Service status
	trace           Start with syscall trace

github.com

dibdot/packages/blob/master/net/banip/files/README.md

<!-- markdownlint-disable -->

# banIP - ban incoming and/or outgoing ip adresses via ipsets

## Description
IP address blocking is commonly used to protect against brute force attacks, prevent disruptive or unauthorized address(es) from access or it can be used to restrict access to or from a particular geographic area — for example.  

## Main Features
* Support of the following fully pre-configured domain blocklist sources (free for private usage, for commercial use please check their individual licenses)

| Source              | Focus                          | Information                                                                       |
| :------------------ | :----------------------------: | :-------------------------------------------------------------------------------- |
| asn                 | ASN block                      | [Link](https://asn.ipinfo.app)                                                    |
| bogon               | Bogon prefixes                 | [Link](https://team-cymru.com)                                                    |
| country             | Country blocks                 | [Link](https://www.ipdeny.com/ipblocks)                                           |
| darklist            | blocks suspicious attacker IPs | [Link](https://darklist.de)                                                       |
| debl                | Fail2ban IP blacklist          | [Link](https://www.blocklist.de)                                                  |
| doh                 | Public DoH-Provider            | [Link](https://github.com/dibdot/DoH-IP-blocklists)                               |
| drop                | Spamhaus drop compilation      | [Link](https://www.spamhaus.org)                                                  |
| dshield             | Dshield IP blocklist           | [Link](https://www.dshield.org)                                                   |

This file has been truncated. show original

If you don’t get a fix here from AX1800 users, you can always pop-over to the banIP thread with your version and error message and ask. I have banIP running on a Spitz (3.215) on v0.3.11 with a half dozen lists/30k IP’s at about 80%/97MB of ram used and an RPI with it that have been running fine for several months now while waiting for a little more traction on the 0.8.x builds of banIP for 22.0.3.

mk808 · February 23, 2023, 12:59am

Its running 0.7.10. When i did the upgrade to 4.1 firmware it wiped the box the only plugins I have running are adguard and banip.

Memory isn’t an issue, I am just testing with Firehol2 list which isn’t that large.

Done multiple clean installs … the annoying thing is there isn’t any obvious error i can find in log files. I think i will try downgrading back to 3 firmware where it was working.

Bitty_Bytes · February 23, 2023, 1:21am

Under Settings>Advanced Settings, do you have verbose logging enabled?

Did any of the following commands show anything useful? I have v0.7.10 running on my Pi and it shows quite a bit of detail:

/etc/init.d/banip refresh
logread

/etc/init.d/banip status

/etc/init.d/banip report

/etc/init.d/banip list

mk808 · February 23, 2023, 1:50am

They all are working apart from the lack of IP’s

Wed Feb 22 15:42:25 2023 user.info banIP-0.7.10[31801]: start banIP processing (refresh)
Wed Feb 22 15:42:25 2023 user.debug banIP-0.7.10[31801]: directory ‘/tmp’ is used
Wed Feb 22 15:42:25 2023 user.debug banIP-0.7.10[31801]: f_tmp ::: tmp_base: /tmp, tmp_dir: /tmp/tmp.ocMoLp, pid_file: /var/run/banip.pid
Wed Feb 22 15:42:25 2023 user.debug banIP-0.7.10[31801]: f_env ::: auto_detect: 1, fetch_util: /usr/bin/curl, fetch_parm: --connect-timeout 20 --silent --show-error --location -o, src_file: /tmp/ban_sources.json, log_terms: dropbear sshd luci nginx, interfaces: wan , devices: eth0, subnets: xxx.xxx.xxx.xxx/22, ip_devices: eth0 eth1 eth2 eth3 eth4 bond0 br-guest br-lan wgserver wlan0 wlan1 , protocols (4/6): 1/0
Wed Feb 22 15:42:25 2023 user.debug banIP-0.7.10[31801]: f_ipset ::: name: -, mode: initial, out_rc: 0
Wed Feb 22 15:42:25 2023 user.debug banIP-0.7.10[31801]: f_ipset ::: name: maclist_6, mode: flush, out_rc: 4
Wed Feb 22 15:42:25 2023 user.debug banIP-0.7.10[31801]: f_ipset ::: name: whitelist_6, mode: flush, out_rc: 4
Wed Feb 22 15:42:25 2023 user.debug banIP-0.7.10[31801]: f_ipset ::: name: blacklist_6, mode: flush, out_rc: 4
Wed Feb 22 15:42:25 2023 daemon.err banip.sh[31801]: sh: out of range
Wed Feb 22 15:42:25 2023 user.debug banIP-0.7.10[31801]: f_ipset ::: name: whitelist_4, mode: create, ipver: inet, settype: src+dst, count(sum/ip/cidr/mac): /-1/1/0, time: 0, out_rc: 0
Wed Feb 22 15:42:25 2023 user.debug banIP-0.7.10[31801]: f_ipset ::: name: firehol2_4, mode: refresh, count(sum/ip/cidr/mac): 0/0/0/0, time: 0, out_rc: 4
Wed Feb 22 15:42:27 2023 user.debug banIP-0.7.10[31801]: f_ipset ::: name: firehol2_4, mode: backup, out_rc: 0

Bitty_Bytes · February 23, 2023, 3:23am

It sounds like you’ve installed several times, but other than checking dependencies, using the --force-maintainer command, trying an alternate blocklist set and/or moving your backup/report directories to a specified directory, looking at the banip.sh file did not give any indication of where the failure is occuring.

opkg update
opkg install libc, jshn, jsonfilter, ip, ipset, iptables, ca-bundle

opkg install banip --force-reinstall --force-maintainer

mkdir /opt/banip/backup
mkdir /opt/banip/report

Change directories for report and backup under Settings>Additional Settings to the new locations

/opt/banip/backup
/opt/banip/report

That is an odd error. Hopefully one the above items at least gets the report generated and IP’s populated.

mk808 · February 23, 2023, 7:14am

Appreciate the help, however, no joy and i am stumped and I think i will try to roll back to previous firmware version.

I just did a complete reset of box and reloaded 4.1 from scratch and started with a completely clean install and banip did not work out of the box. At this point i think it has to be the firmware version. I have multiple small GL Mango boxes running ver 3 and no issue whatsoever.

i7bar · January 2, 2025, 2:43am

Hello,
just to wanted to mention that I am getting the same error on AX1800, the latest 4.6.8 firmware - the empty IPset and this is upsetting.

i7bar · January 4, 2025, 9:51pm

Hello,

I spent a bit of time and looked into this problem and I must say BanIP does not work on AXT1800 (please note the topic starter is speaking about AX1800 but I think it makes no difference in this case) v.4.0.1 - 4.6.8. And I believe the reason is the strange behavior of ipset. Therefore, I have a question for Gl.iNet staff - does AXT1800 firmware includes the standard version of ipset or this is somehow modified one?

Explanation:

AXT1800 v.4.0.1 - 4.6.8 is built on the top of OpenWrt 21.02-SNAPSHOT r16399+159 ->
OpenWrt 21.02-SNAPSHOT r16399+173-c67509efd7 and uses kernel 4.4.6

I unpacked AXT1800 v.4.0.1, 4.0.2, 4.0.3 firmware and tried the included versions of ipset (together with ipset I copied from my 2 working ATX running v4.2 and 4.6 firmware - they have different timestamps but behave identically)

For comparison sake I was using GL.iNet GL-MT2500 running v4.2.0 built on the top of OpenWrt 21.02-SNAPSHOT r15812+878-46b6ee7ffc and running kernel v. 5.4.211

First of all, ipset running on AXT1800, compalins on the protocol mismatch (a screenshot showing output of ipset running on AXT1800 and MT2500, side by side:

Second and the most important observation: when one run the command "ipset list" the reported ipsets dont include the line "Number of entries" as shown on the following screenshot:

Interesting that x64 OpenWrt 21.02.0-rc3 r16172-2aba3e9784" build does not have this problem either and properly returns existing ipsets:

banip.sh reads info about existing ipsets and extracts the number of entries as shown before:

>cnt="$(printf "%s\n" "${src_list}" | awk '/^Number of entries:/{print $4}')"

and uses the var. cnt latet in f_iptables call. Since cnt is set to NULL, nill or None (whatever programming language you speak) , when accessed the error "out of range" is generated, end of story.

So at this point before I spent more time on I would like to know if someone knows why ipset included in AXT1800 firmware returns incomplete info i.e. the entry "Number of entries" is missing in ipset reports?

Thanks!

i7bar · January 4, 2025, 10:20pm

A quick update - ipset relies on libipset.so.13.1.0 (in all cases: AXt1800, MT2500, x64) and it does contain the line "Number of entries". And then it brings me to a conclusion that the reason why "Number of entries" is not included in ipset report is the ipset protocol mismatch, mentioned before.

And if this is correct then it will be the result of using older (v.4) kernel with newer versions of ipset and it will affect all Gl-iNet routers (not only AX1800 and AXT1800) using firmware build in similar way.

It also means that if we want to run banip on this kind of firmware then the script has to be modified and the line

cnt="$(printf "%s\n" "${src_list}" | awk '/^Number of entries:/{print $4}')"

will have to be replaced with a piece of code, counting the number of members instead of relying on existence of the line "Number of entries" in ipset report.

And when that is done, it still will not guaranty success because after that some other issues related to protocol mismatch may surface.

I hope I am wrong and someone can correct me.

Thanks.

bruce · January 10, 2025, 8:31am

The AXT1800 with the v4.7.0 looks like no issue.

I will ask my colleagues to check if this issue is wrong or not.

i7bar · January 11, 2025, 4:12am

Hello Bruce,

You are right, the behavior of ipset has changed and now the latest one, coming with v.4.7 firmware, works correctly. The problem is BanIP is not available for v.4.7 and many users (I am one of them) will continue using older (because it is very stable) firmware.
All in all, this is not a drama, just saying.

bruce · January 15, 2025, 2:52am

The v4.7.0 banip plugins has been updated, please test it again.