Dual PPPoE connections with static routing on Kubuntu

I decided to write this simple HOWTO so people can set up international and local ADSL accounts using PPPoE on (K)ubuntu.
I'm assuming that the international account is an expensive SAIX based one and the local one is a cheap IS based on but it will work for any ISP account.
If you're using Ubuntu you may have to replace references to "sudo kedit" with the Ubuntu equivalents such as "gksudo" or "sudo gedit" since KDE and it's apps probably won't be installed by default.
I prefer good old vi because it's practically universal across *nix machines but let me not start a flame war.

Install required packages

• pppoeconf
  You need to make sure that you have pppoeconf installed.
  It's not essential but I'm not the type that likes writing half a dozen PAP and CHAP configs to set up a connection.
  dpkg -s pppoeconf will tell you if it's installed.
  To install it if it's missing : sudo apt-get install pppoeconf
  
• RP-PPPoE
  Install the PPPoE protocol from Roaring Penguin.
  http://www.roaringpenguin.com/en/penguin/openSourceProducts/rpPppoe
  The installation instructions are in the download file but you don't need to run the GUI or use it to set up a PPPoE connection. All we need it for is the protocol which is in a library.
  Do the following :
  tar zxf rp-pppoe-3.8.tar.gz
  cd rp-pppoe-3.8/src
  ./configure
  sudo make install
  

Configure ADSL router/modem
Make sure your ADSL modem is in bridge mode. I won't discuss how to do that since it's different for each modem and it will be part of the modem documentation anyway.

Create PPPoE accounts
Run pppoeconf and set up the international account.
Accept the defaults provided by pppoeconf and say yes to the start at boot option.
Now test the connection with pon dsl-provider
Your Internet connection should be working and if you list the interfaces with ifconfig you should see ppp0 listed.
Disconnect ppp0 before continuing. poff dsl-provider

Now is where we need to start with some custom changes because dual ADSL connections are practically unheard of overseas and pppoeconf only caters for one account in one config file (dsl-provider).
Move the dsl-provider file located in /etc/ppp/peers to "international".
sudo mv /etc/ppp/peers/dsl-provider /etc/ppp/peers/international
We need to do this because when we set up the local account it will overwrite dsl-provider.

Run pppoeconf again and set up the local account but make sure you answer no to the "Use peer DNS" question. We only want to use the DNS servers provided from the ISP from the international account. Say yes to the start at boot option.
Move the dsl-provider config to "local".
sudo mv /etc/ppp/peers/dsl-provider /etc/ppp/peers/local

Let's edit the international and local configs by hand.
I'll only list the parameters that you need to check.
A hash before the line means the line is commented out.

For the international connection :
sudo kedit /etc/ppp/peers/international
noipdefault
defaultroute
replacedefaultroute
usepeerdns

For the local connection :
sudo kedit /etc/ppp/peers/local
noipdefault
# defaultroute
# replacedefaultroute
# usepeerdns

If you want to manage the connections manually or test them you can use the following commands.
pon international
poff international
pon local
poff local

Depending on what order you do then ppp0 and ppp1 can swap around at will which is not desirable and we will fix it in the next section.
Also since the DNS entries in /etc/resolv.conf are only being set up by the international account you won't be able to browse any web sites unless it is connected.
For the moment the local account won't route any traffic because it is not the default route.

Configure interfaces
As we saw in the previous section we have a problem where ppp0 and ppp1 can swap depending on which account was connected first and it's not desirable especially for routing purposes.
To fix this we need to set up the accounts so that we can use interface names like ppp0 and ppp1 instead.
Let's make the necessary changes to /etc/network/interfaces.
sudo kedit /etc/network/interfaces

If you scroll down to the bottom of the file you should see a block that looks like the following :
auto dsl-provider
iface dsl-provider inet ppp
provider dsl-provider

What we're going to do is duplicate the block and replace the interface names with ppp0 and ppp1 for the international and local accounts respectively.

Simply replace the block with the following :
auto ppp0
iface ppp0 inet ppp
provider international

auto ppp1
iface ppp1 inet ppp
provider local


Make sure that the line pre-up /sbin/ifconfig eth0 up # line maintained by pppoeconf is at the bottom of the file!

The auto ppp* tells the system to start the connections at boot time.
Now if we want to control the connections we can use the ifup and ifdown commands (interface up and interface down).
Test the connections to make sure there are no problems.
sudo ifup ppp0
sudo ifup ppp1
ifconfig should list both interfaces
sudo ifdown ppp0
sudo ifdown ppp1

Don't use pon or poff from now on unless you want to create a mess - stick with ifup and ifdown. pon and poff don't worry about interface names so if you run pon three times for international you'll end up with ppp0, ppp1 and ppp2.

Static routing
Now the fun part where we actually get the traffic flowing.
We should have two interfaces configured and working.
ppp0 is the international interface and is the default route through which all traffic will flow.
ppp1 is our local interface and we need to tell the system to route all the South African traffic through the interface instead of the international interface.
We'll use static routing to do this and I'll do it in steps so as to not cause too much confusion.

Download Armin's list of local routes from http://alm.za.net/ip/localroutes4.txt
The list is updated every 24 hours and we'll use it instead of duplicating his work.
It contains all the network addresses that are specific to South Africa.
Place the file in /etc/ppp/ sudo mv localroutes4.txt /etc/ppp

We need to create a small script that tells the system to add the routes to the interface when if comes up.
Create a file in /etc/ppp/ip-up.d/ called "zanet".
sudo kedit /etc/ppp/ip-up.d/zanet

Paste the following in the file :

#!/bin/sh -e
# Called when a new interface comes up

# add custom routing for zanet (local South Africa) on ppp1 device
if [ "$PPP_IFACE" = "ppp1" ]
then
cat /etc/ppp/localroutes4.txt | sed s/'\$LOCAL'/$PPP_IFACE/ | while read localroute
do
# Change net to host for /32 addresses
# This is a bug workaround - Armin must fix his list script.
if [[ $localroute =~ "/32" ]]
then
localroute=`echo $localroute | sed s/net/host/`
$localroute
else
$localroute
fi
done
fi

# We need to route DNS lookups via ppp0 (SAIX) instead of ppp1 (IS)
# The static routing will cause the DNS lookups to be made through the local IS
# connection and the SAIX network will block the lookups because they don't originate
# from their network.
if [ "$PPP_IFACE" = "ppp0" ]
then
cat /etc/resolv.conf | sed s/nameserver// | sed s/' '// | while read nameserver
do
route add -host $nameserver $PPP_IFACE
done
fi


Static routing should now work unless you made a mistake or I forgot something.
Test it by starting ppp0 and then ppp1.
If run route -n you should see huge list of static routes for the ppp1 interface and only a few for ppp0.

Automatic localroute updates
You could get wget to retrieve a new list of local routes every so often manually or with a cronjob but I'll leave that up to you.

Automatic interface reconnection
If an interface drops the ppp daemon will try to reconnect it but eventually it will give up.
Below is a simple script that I run every 5 minutes from a user cron job that attempts to bring the interface up again.
Sometimes the interface is down but there seems to be some sort of dirty state in the pppoe library so I do an ifdown before an ifup which fixes the problem.

#!/bin/bash
PATH=$PATH:/sbin

# Make sure ppp0 is still up
if [ "`ifconfig | grep ppp0`" == '' ]
then
sudo ifdown ppp0
sudo ifup ppp0
fi

# Make sure ppp1 is still up
if [ "`ifconfig | grep ppp1`" == '' ]
then
sudo ifdown ppp1
sudo ifup ppp1
fi


The crontab syntax is :
*/5 * * * * sh /path-to-some-place/check_connections.sh
You'll first need to chmod +x the file of course.

You should now have a dual connection setup which will start automatically at boot time.
I set up Shorewall for some extra protection - if anyone wants to know how to configure it for dual connections I'll add it but it is explained in th docs quite well already.
I hope this information will be useful to someone.

Get in there you good thing :thup:

It works, but after a while I have to manually shutdown and reinstate the ppp ports by using sudo ifdown then sudo ifup. And I've set up the cron to run the check connection script every 5 minutes. For some reason I am suspicious about it.

Also is it meant to show these error messages.


EDIT: Clearly the issue isn't the script but a Keep Alive For How Long type issue. Also when I run ifconfig, ppp0 and ppp1 are still up.

Paul_S: Have you tried to use hellanzb with this setup?

I get the following error: DNS lookup failed for hostname: news.is.co.za

EDIT: International is WebAfrica and Local is OpenWeb

What do the first two lines mean?

A plog when the interface is working

These instructions, unsurprisingly, also work with Debian. And there too I am faced with the same problem.

The DNS lookup problem for news.is.co.za?
I'm not sure why that is happening and I'm currently using a single ISP so I can't test it however just use the IP address for the server instead.
news.is.co.za => 196.26.208.123

Hmmm. That makes sense. It dawned on me that this could be the problem as the error manifests itself by resolving failing. If there is a download going on at that point it continues and does not fail. Will try this this evening by setting up local again and manually enter the primary and secondary DNS servers.

It seems the trick is to have usepeerdns for the local account and to switch ppp0 and ppp1 around, ie ppp0 as local and ppp1 as international. I'll see overnight if this stands the test of time.

EDIT: Nope didn't help. Though it does take longer for it to lose its resolving powers :

I think my one man conversation ends here. The problem I think had to do with the fact that my modem/router settings had the IP leased to the computer via DHCP for 3600 seconds. I set this to a never ending lease (0 seconds) and since this evening 19h00 I have had no loss of connection. Again it will be left overnight and I will see in the morning. But I am confident that this has solved the problem.

EDIT: I can say now that the problem is solved on Kubuntu and Debian.

Now my next task is to get the instructions to work under Fedora.

Another small script for automatically updating the local routes file.
Stick into a root cronjob that runs once every while (not more than once every 24 hours since the source is only updated once per day at 6 AM).


#!/bin/bash

cd /etc/ppp
wget http://alm.za.net/ip/localroutes4.txt

# If there is already a localroutes file replace it with the new one.
if [ -f /etc/ppp/localroutes4.txt.1 ]
then
# Make sure that we didn't get an empty or nearly empty list
# More than 1000 characters will qualify as a valid amount of local routes
# Rather keep the old list if the new one is too small
if [ `cat /etc/ppp/localroutes4.txt | wc -c` -gt 1000 ]
then
mv localroutes4.txt.1 localroutes4.txt
fi
fi

Hmmm does the routing prevent access to the router?

Jongi said:

Hmmm does the routing prevent access to the router?

Click to expand...

No it shouldn't.
It's quite likely you have a firewall blocking the connection.

sudo iptables -L will show you the rules if there are any.
If the default policy on the default tables is set to ACCEPT and there are no rules in the tables then it should work.

Hi Paul, Jongi

In your environment where the DSL modem is in bridging mode and the linux PC is running a firewall (shorewall) with dual PPPoE connections shown in the configuration below

Internet == DSL modem(IP:?) ==(eth0:IP:?) LinuxPC (eth1:IP:192.168.1.x)== local lan

1. How is the ip addressiung structured on the link between the DSL and the outside network on the linux box ?
2. Is their a performance hit agains using this method as apposed to using the vanilla dsl modem with a single PPPoE connection?

Any help appreciated

1. How is the ip addressiung structured on the link between the DSL and the outside network on the linux box ?

Click to expand...

For every PPPoE connection a pppX device is created on the Linux box and assigned a local and remote IP address.
It's the same as a PPP connection using a 56K dialup setup except it's over Ethernet.
The connection still runs over the Ethernet between the Linux box and the ADSL modem/router but you don't have to worry about that side.

So you'll end up with two IP addresses on each side of each PPPoE connection.
A local IP will be assigned to the pppX device on the Linux box and a remote IP will be assigned on the ISP side.
The ADSL modem/router doesn't actually know anything about the IPs for the connection - it just handles the connection at the Ethernet level. Think of the connection as a tunnel that passes through the ADSL modem/router.

2. Is their a performance hit agains using this method as apposed to using the vanilla dsl modem with a single PPPoE connection?

Click to expand...

There shouldn't be any noticable performance hit unless you start setting up dozens of PPPoE connections. The only thing is that the PPPoE connections will share the physical connection and therefore you won't get 4 * 384kbps if you set up 4 PPPoE connections on a 384kbps ADSL account.

Thanks Paul,

You have answered my question regarding the performance but I am unsure as to what IP address the dsl modem would have. I presume it would need one for management and monitoring

Thanks

time4net said:

Thanks Paul,

You have answered my question regarding the performance but I am unsure as to what IP address the dsl modem would have. I presume it would need one for management and monitoring

Thanks

Click to expand...

Yes, you'd need an IP on the ADSL modem for management but it doesn't affect the PPPoE protocol which runs on top of the Ethernet connection.

One would normally use a private network address like 192.168.1.254 for the ADSL modem/router and another IP in the 192.168.1.1-253 range for the Ethernet interface(s) on the Linux box.

thanks, all the software is now loaded, I just need to restructure the wiring this evening to test properly.

Thanks for this awesome guide It works perfectly. However, I'm having trouble getting my main computer (which is running Kubuntu 7.10) to talk to the router, which is running Ubuntu 7.10. I've tried setting the gateway on the main computer to the IP address of the router, and it doesn't seem to work. I tried pinging a few websites from the main computer to test it, and while it seems (don't take my word on this, though, maybe it was just cached somewhere :-/) that it can resolve domain names to IP addresses, it doesn't actually forward any of the requests from my main computer to the outside world. What could be wrong?

what is the output of ifconfig?