I pay for internet which is shared among a few people in a couple of houses and sometimes it gets hogged completely so I really need to know whats happening. My current setup consists of one TP-link ADSL router, and then a few access points.
1) How many people?
2) What line speed / type ISP package?
3) What kind of AP's?
These factors (in addition to "bandiwidth hogs") all play a role in performance and are worth considering.
How could I go about setting something up which would tell me whats going where, and how much usage is occurring from individual sources? Preferably in real time too. DO I need some sort of dedicated server, maybe a new router?
Time and money come into play here. For a (relatively) quick and pretty solution this (imho) works well:
http://community.smoothwall.org/forum/viewtopic.php?f=26&t=31111
It requires a "dedicated pc" running
http://www.smoothwall.org/ but doesn't have to be fancy.
With it you can see (in real time by way of pretty graphs) who's online, how much they're using and set monthly limits "per users ip address"
There's also a QOS feature that you can use to throttle / shape individual users. This is important as its one thing finding the "hoggers" but then what?
As mentioned by TheGrove
http://www.pfsense.org/ is another option that will also require a dedicated PC and has the following options as far as bandwidth monitoring go:
http://doc.pfsense.org/index.php/How...width_usage? It can also do the QOS you require.
Of these
http://bandwidthd.sourceforge.net/ is my favorite for gathering usage trends but wont by itself allow you to throttle or shape. Here's a useful guide for getting it up and running on a Linux machine:
http://community.spiceworks.com/how_to/show/1261-how-to-use-bandwidthd-to-monitor-your-network
Re Mikrotik as others have mentioned this is a great product for the price and if you have the cash I'd start here
http://www.scoopdistribution.co.za/product_info.php?cPath=100_119&products_id=1281 as it has a built in radio card which is useful for testing stuff like for example (as mentioned)
http://wiki.mikrotik.com/wiki/Manual:User_Manager , unfortunately their online demo (in the above link) is down at the moment but maybe check it out when back up; it should do what you're after. (its unf not currently under active development and as a result lacking a little in documentations)
http://forum.mikrotik.com/viewtopic.php?f=10&t=8411&sid=e0a5cf571ab3ed66f58611dcc06a5044
Mikrotik has a lot of very useful features relating to what you're after. For example you can create simple queues for each user and set bandwidth limits based on time, speed and usage per user or a whole subnet. (these simple queues will also tell you who's used how much but stats will be lost if rb is rebooted.) Also make sure you separate local from internet traffic if need be - you can do this by creating the appropriate "mark packet" rules in /ip firewall mangle.
Using this
http://forum.mikrotik.com/viewtopic.php?t=67731&p=344651 script for example at a customers cafe (I may try user manager next but for now this is working well) I limit users who exceed 100mb of data to 75/75kbps, the script runs every 2.5 minutes with another script running every 3 hours or so to reset the counters back to 0.
Another useful way to gather stats on who's using what and when is with Mikrotiks TrafficFlow, this will require a collector such as
http://www.pmacct.net/ that will store the info in an SQL data base that can then be graphed with something like
https://github.com/Daniel15/pmacct-frontend/ here's a screenshot
http://stuff.dan.cx/images/projects/pmacct/month.png (busy experimenting with this myself)
Should you decide to upgrade your AP's as well as your router these
http://dl.ubnt.com/datasheets/unifi/UniFi_AP_Datasheet.pdf come with features (out the box) that can also (to a degree) do what you're after.
For real time bandwidth stats
http://www.ntop.org/products/ntop/ is defiantly worth a mention though I personally found it gave too much info about things that were not directly related to what I was after (that being pretty much the same thing as you).
Also worth mentioning (while perhaps overkill) is
http://freeradius.org/ in conjunction with a Mikrotik Router Board. (Haven't gotten round to trying this myself yet) For a paid for product based on the same thing (RADIUS) check out
http://www.dmasoftlab.com/cont/home
For graphing
http://www.cacti.net/ looks great but from what I can tell won't (or I cant get it to) graph on a "per user" rather than "per device" basis.
If your users don't mind being limited to http only ie no P2P (which they probably will) then a proxy server using
http://www.squid-cache.org/ along with
http://sarg.sourceforge.net/ to monitor usage works a treat. (you may have to let them know they're being monitored depending on your setup / terms of use etc.) Here a working demo / example
http://sarg.sourceforge.net/squid-reports/index.html and here's
https://sourceforge.net/p/sarg/discussion/363374/thread/e6e3b6d0/ a post to help you if you get stuck (as I did) with creating day / month / year reports.
I find a combination of Ubiquiti (for wireless) with Mikrotik (for routing) and a spare pc running Linux (for administration) is the combo that works for me. Don't forget
http://www.nagios.org/ for alerting you via sms or email when your RB and or AP(s) go offline.
If I were you and had an interest in the above, some spare time with about R700 to spare and access to 1 or 2 old pc's; I'd say try em all till you find a solution that fits your needs.
And if you haven't done so already why not get wugged while you're at it?
http://www.wug.za.net/wiki/index.php?title=Current_SA_WUGs
Hope this helps; also very keen to hear how others have achieved a (relatively simple to implement) open source solution to providing minute / hour / day / week / month / year usage stats on a "per user" rather than "per router / device" basis that can ideally be in graph format.
