7b8cda88d11751d7d632b3d548028bf9c0af8ebf
By:
animaexos

Floating iP gateway as default router and source address for outbound connections

October 31, 2015 6.4k views
Networking Security High Availability Scaling Load Balancing FreeBSD

Community!

I am curious if it is possible to setup anchor ip gateway as default router so all outbound traffic from droplet shows to external services as coming from floating ip.

This will greatly simplify my external system firewall setup since floating ip never changes; I can allow all connections from floating ip.

I tried adding the anchor ip gateway as the defaultrouter in /etc/rc.conf and reboot but this did not work. (curl -s http://169.254.169.254/metadata/v1/interfaces/public/0/anchor_ipv4/gateway)

I am using freebsd, but welcome any linux recommendations as well. There may be multiple ways to achieve this (e.g. routes, routed, iptables), but what is the CLEANEST approach, and is this even possible on Digital Ocean?

3 comments
  • good question - I'm also curious about whether this is possible for a HA setup I have in mind ..

    (I have already been told that floating IP's can't have PTR records or forward SMTP traffic so I'm a bit concerned whether they are 'fit for purpose' ..)

  • @animaexos After struggling with this, I achieved it using iptables.

    Run this command from your server to retrieve anchor IP address

    $ curl -s http://169.254.169.254/metadata/v1/interfaces/public/0/anchor_ipv4/address
    

    Then run this command to redirect all outgoing traffic to your floating ip.

    $ iptables -t nat -A POSTROUTING -p tcp  -o eth0 -j SNAT --to-source ANCHOR_IP
    
  • To build on what @marcoscvp90 said, in PF the same rule would be:

    nat pass on $NIC to any { port $PORT|$PORTRANGE } -> $ANCHOR_IP
    

    port is optional if you want to nat all traffic through the floating ip.

10 Answers

Hello guys,

I'm struggling to get this to work. First, I obtain the anchor IP by using the command:

$ curl -s http://169.254.169.254/metadata/v1/interfaces/public/0/anchor_ipv4/address

For example, let's say that it's 10.10.0.1 - i'll then use the following command:

$ iptables -t nat -A POSTROUTING -p tcp  -o eth0 -j SNAT --to-source 10.10.0.1

Are there any further steps required? When I ping another machine, instead of the floating IP address, it appears to come from the droplets primary IP address. I have monitored this using tcpdump on the other droplet:

sudo tcpdump -i ethX icmp and icmp[icmptype]=icmp-echo

If anyone could point me in the right direction, it would be hugely appreciated!

Thanks,

James

@marcoscvp90 Thanks for this info.

I am trying to use in a postfix config; I want to use the floating ip as the receiving address of my server. In the email world is very important the ip reputation. I want to use floating ip to keep all the reputation.

Using your iptables recomendation, when postfix try to connect to the internet it gets "Network not reachable".

Its necessary to add a new ip route?

thanks,

Roberto

@Brants the ping packets are not being SNATed because you are using -p tcp and they are icmp. If you replace that parameter with -p all, it will work for any protocol coming out of eth0.

@robertomatute I believe that DigitalOcean is blocking all outbound TCP on port 25 from the floating IPs. This is annoying, since I also wanted to use the floating IP feature to support outbound SMTP.

  • @b7 thanks for the information. I hope DigitalOCean let use the floating ip for smtp soon!

    • We are in the same situation:

      $ iptables -t nat -A POSTROUTING -p tcp -o eth0 -j SNAT --to-source ANCHOR_IP

      ....works fine for us except that we cannot send outbound email via SMTP.

      Wish DO would remove this restriction - Or maybe there is another IPtables solution?

      • There is no way to circumvent it with iptables, since DO is actively blocking it at the other end of the floating IP. I opened a support ticket and they confirmed the block was intentional.

        Without reverse DNS configuration for floating ips, this would still not be a viable solution for a production MX server even without the block. A real pity, since with both of those features, DO would have the best setup for running an in-cloud MX server of any provider.

        • The reason I started setting up my DO server was for a mail-server setup over a floating IP. How is this not an important usecase?

We used the command provided above to set the default From
IP address on outbound connections to the floating IP address:

iptables -t nat -A POSTROUTING -p tcp  -o eth0 -j SNAT --to-source <ANCHORIP>

where <ANCHOR_IP> should be replaced by your actual anchor IP address. The anchor
IP address can be obtained with the following command (which hangs if you have
already executed the command above!)

/usr/bin/curl -s http://169.254.169.254/metadata/v1/interfaces/public/0/anchor_ipv4/address

However, we then ran into the Digital Ocean block on SMTP connections
originating from a floating IP address. This wasted days of our time!
We eventually solved this by adding an extra rule to the POSTROUTING
table as follows:

iptables -t nat -A POSTROUTING -p tcp --dport 587 -o eth0 -j SNAT --to-source <FIXEDIP>

where <FIXEDIP> is the fixed IP address of your droplet (as appears in
your droplet list in the Digital Ocean web management interface).
This iptables command says to make SMTP a special case and to use the droplet's
fixed IP address as a From address for outbound SMTP connections.

You have to issue the commands in the correct order so that the special case
SMTP rule ends up BEFORE the catchall rule in the resulting POSTROUTING rule
chain. So do them in this order:

iptables -t nat -A POSTROUTING -p tcp --dport 587 -o eth0 -j SNAT --to-source <FIXEDIP>
iptables -t nat -A POSTROUTING -p tcp  -o eth0 -j SNAT --to-source <ANCHORIP>

If you are engineering a general solution, put in a rule for each port that
is commonly used for SMTP:

# Assumes the POSTROUTING chain is empty at this point.
iptables -t nat -A POSTROUTING -p tcp --dport 25 -o eth0 -j SNAT --to-source <FIXEDIP>
iptables -t nat -A POSTROUTING -p tcp --dport 465 -o eth0 -j SNAT --to-source <FIXEDIP>
iptables -t nat -A POSTROUTING -p tcp --dport 587 -o eth0 -j SNAT --to-source <FIXEDIP>
iptables -t nat -A POSTROUTING -p tcp --dport 2525 -o eth0 -j SNAT --to-source <FIXEDIP>
iptables -t nat -A POSTROUTING -p tcp --dport 2526 -o eth0 -j SNAT --to-source <FIXEDIP>
iptables -t nat -A POSTROUTING -p tcp  -o eth0 -j SNAT --to-source <ANCHORIP>

If all goes well, you will end up with a POSTROUTING chain that looks like this:

[root@www1 ~]# iptables -t nat --list
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
SNAT       tcp  --  anywhere             anywhere             multiport dports smtp,urd,submission,ms-v-worlds,ema-sent-lm /* 455 Use fixed IP for SMTP outbound */ to:<FIXEDIP>
SNAT       tcp  --  anywhere             anywhere             /* 456 Use floating IP for outbound */ to:<ANCHORIP>
[root@www1 ~]# 

Reminder: You can delete the first entry in a chain like this:

iptables -t nat -D POSTROUTING 1

Reminder: If you want your new iptables to survive a reboot,
save it using a command something like this:

/sbin/service iptables save

If you are coding all this in Puppet, the following might be helpful:

firewall { '455 Use fixed IP for SMTP outbound':
    table    => 'nat',                  # -t nat
    chain    => 'POSTROUTING',          # -A POSTROUTING
    proto    => 'tcp',                  # -p tcp
    dport    => [25,465,587,2525,2526], # --dport 587 (etc)
    outiface => 'eth0',                 # -o eth0
    jump     => 'SNAT',                 # -j SNAT
    tosource => $fixed_ip_address,      # --to-source $fixed_ip_address

    # It seems like a good idea to ensure that the rule to allow established
    # connections to continue unharmed is implemented before we monkey around
    # with anything else. Perhaps this 'require' isn't needed, but I've
    # included it to be on the safe side.
    #
    require  => Firewall['002 accept related established rules'],
}


firewall { '456 Use floating IP for outbound':
    table    => 'nat',                # -t nat
    chain    => 'POSTROUTING',        # -A POSTROUTING
    proto    => 'tcp',                # -p tcp
    outiface => 'eth0',               # -o eth0
    jump     => 'SNAT',               # -j SNAT
    tosource => $anchor_ip_address,   # --to-source $anchor_ip_address

    # It is essential that the firewall rule to allow all established
    # connections to continue unharmed is implemented before the floating
    # IP address is made the default for all outbound traffic because if
    # you don't do this, it seems that existing connections suddenly
    # switch to the floating IP address mid-session! Apart from anything else,
    # this causes all SSH sessions to the node to fail, which causes
    # any manual Puppet SSH session to be terminated!
    #
    require  => Firewall['002 accept related established rules'],
}

I hope this saves someone some time!

Ross Williams

edited by kamaln7

Any issues related to router can be resolved here: 192.168.0.1

Have another answer? Share your knowledge.