Ytdyklly

I would like to explain what I'm currently working on briefly and explain issues I need to address.

We have a device (a TP-link Access Point) that we have our custom linux kernel running (which is a modified OpenWrt OS). In this device we use LUA and shell scripts to enforce certain rules. We use iptables firewall and with some research I've been able to achieve these:

• Create main iptables chains to keep track of uploaded and downloaded data separately. Note that this accounts cumulative data traffic.




• Create user specific sub-rules that count Upload/Download because iptables doesn't create necessary rules to account a certain user (by IP) automatically.




• At some point I noticed that when access point is rebooted, I would lose all iptables chains. I added a script into /etc/init.d to init quota (this would add main iptables chains for accounting traffic.)




• I noticed I would lose IP-specific sub rules too. To solve this, I used dnsmasq. When dnsmasq leases an IP to a device (whether it being new/old lease), a script creates specific iptables rule so that this user's usage is counted. This was possible thanks to dnsmasq having an option to run a script which is triggered on lease.




• I've also created two different scripts which creates json files that has IP, Download & Upload (in bytes) in it. These are going to be used in our IOS/Android app to provide user some functionalities such as viewing reports, creating quota and restricting usage etc.

Up to this point, I can keep track of which client downloaded/uploaded how much data. I can restore my iptables rules after a reset and when a device is connected I add it's IP address with relevant iptables rules. However, I have some issues:

• My computer which I use in the office (Win10/Ubuntu) is connected to AP I am using to test constantly and for some reason unless I manually disconnect, take a static IP, then turn DHCP client on to trigger a DHCP lease from AP, it doesn't trigger my script. This means that after AP is restarted, I have down-time on my traffic accounting for some users.




• Even if I create main and sub-rules with properly triggered scripts, it still resets the counters. I need to somehow keep my counter going between AP resets, client disconnects etc.




• iptables AFAIK doesn't support traffic accounting by MAC address by default. I know there are alternatives, such as iptables modules that may achieve that. However, I am supposed to not use any external libaries/packages due to device's limitations. (e.g it has 16 mb flash memory in total)

I am aware I can use crontab to save current usage, reset counters, have timestamps and read logs to have healthy usage statistics. However this means a lot of parsing and I also think this will not be efficient disk usage-wise. We may probably use cloud (which we already use for various reasons) to save logs.

Also about MAC and IP thing, I realized DHCP leases same IP to same device, it probably has something to do with the way it was implemented in the first place. My computer is given 192.168.42.244 every time, for example. The reason I mention that is, I could probably just have a file where I save MAC and IP addresses, periodically test if their iptables rules are up, if not, add rules to ensure they are being accounted. Then, I feel like running a crontab task like every minute would probably not be very efficient.

I think whole these can be achieved properly, I just feel a bit lost when to start accounting effectively (during lease/ on boot/ periodically), also how to distinguish devices (in case a different IP is leased to same device for some reason) and lastly what should I relay on most? arp? dhcp.leases? a client_list file that I maintain myself?

I am also sharing some scripts that I use, for somebody else that works on a similar thing.

This is init_quota, comment is pretty self explanatory:

#!/bin/sh

# this script creates relevant chains to account traffic



iptables -N TRAFFIC_ACCT_IN

iptables -N TRAFFIC_ACCT_OUT

iptables -I FORWARD -i eth0 -j TRAFFIC_ACCT_IN

iptables -I FORWARD -o eth0 -j TRAFFIC_ACCT_OUT

This is /etc/detect_new_device.sh which is triggered on a DHCP lease. (dnsmasq option)
dnsmasq calls this script with arguments seen on comments.
Creates "sub-rules" that I mentioned above. At first I would use "add|old" trigger, but it wasn't consistent, so I made it such that it adds the rule if a rule doesn't currently exist for that specific IP.
I also used this to save mac/ip/device name in a text file for further possible use.
Note that it updates IP of a device (distinguished by MAC address) by removing previous match and adding a new line.

#!/bin/sh                                                                                                                                                                                                   



# This script detects new DHCP lease to trigger relevant iptables command.                                                                                                                                  

#                                                                                                                                                                                                           

# $1: add | old                                                                                                                                                                                             

# $2: MAC address                                                                                                                                                                                           

# $3: IP address                                                                                                                                                                                            

# $4: device name                                                                                                                                                                                           



if [ ! -f /root/quota/client_list ] ; then                                                                                                                                                                  

 touch /root/quota/client_list                                                                                                                                                                              

fi                                                                                                                                                                                                          



sed -i "/$2/d" /root/quota/client_list                                                                                                                                                                      



echo -n "$2 " >> /root/quota/client_list                                                                                                                                                                    

echo -n "$3 " >> /root/quota/client_list                                                                                                                                                                    

echo -n "$4 " >> /root/quota/client_list                                                                                                                                                                    

echo "" >> /root/quota/client_list                                                                                                                                                                          



iptables -L TRAFFIC_ACCT_IN -n -v -x | grep -q "$3"                                                                                                                                                         

chain_exists=$?                                                                                                                                                                                             



if [ "$chain_exists" -ne 0 ]; then                                                                                                                                                                          

  iptables -A TRAFFIC_ACCT_IN --dst "$3"                                                                                                                                                                    

  iptables -A TRAFFIC_ACCT_OUT --src "$3"                                                                                                                                                                   

fi             

I also use this all_usage.sh which lists and formats traffic report into a json file:

#!/bin/sh



iptables -L TRAFFIC_ACCT_IN -v -x -n | awk '$1 ~ /^[0-9]+$/ { printf "%sn", $8}' >> ip_list.txt

ips="/root/quota/ip_list.txt"



rm result.txt

touch result.txt

echo "[ " >> tmp_result.txt



while IFS='' read -r line || [[ -n "$line" ]]; do



  ip_add=$line

  bytes_down=$(iptables -L TRAFFIC_ACCT_IN -v -x -n | grep $ip_add | awk '$1 ~ /^[0-9]+$/ {printf "%sn", $2}')

  bytes_up=$(iptables -L TRAFFIC_ACCT_OUT -v -x -n | grep $ip_add | awk '$1 ~ /^[0-9]+$/ {printf "%sn", $2}')

  template='{"ip":"%s","down":"%s","up":"%s"}'

  current_string=$(printf "$template" "$ip_add" "$bytes_down" "$bytes_up")

  echo "$current_string," >> tmp_result.txt

done < "$ips"



sed '$s/,$//' tmp_result.txt >> result.txt

echo "]" >> result.txt



rm ip_list.txt tmp_result.txt

result.txt looks like this:

[

{"ip":"192.168.42.227","down":"485223","up":"163955"},

{"ip":"192.168.42.191","down":"0","up":"0"},

{"ip":"192.168.42.186","down":"602397","up":"61924"},

{"ip":"192.168.42.210","down":"28323","up":"4987"},

{"ip":"192.168.42.244","down":"420916885","up":"20119892"},

{"ip":"192.168.42.221","down":"0","up":"0"},

{"ip":"192.168.42.197","down":"26812","up":"783"}

]