Related

Product Managed Kubernetes on DigitalOcean Product
Why are snapshots so slow Question Question
Connect to DB installed into dedicated VPS (private networking) Question Question

Question

How to customize firewall rules for managed kubernetes service?

Posted May 29, 2019 5.9k views
DebianDigitalOceanFirewallKubernetes

Hi. I recently started to use DigitlaOcean managed Kubernetes. and Just realized that when I try to modify my node pools (adding or deleting existing one), It also refreshes my firewall setup together.

I need to customize this firewall rule for my services. and Rollback of firewall rules can be DISASTER in my case.

Is there any way to define custom ports on firewall and lock it to avoid unexpected rollback?

I’m pretty sure that this can be a common issue for every users who planning to use DO managed Kubernetes.

I’ll wait for your reply. Thanks.

Add a comment
yechan
By:
yechan

Related

Product Managed Kubernetes on DigitalOcean Product
Why are snapshots so slow Question Question
Connect to DB installed into dedicated VPS (private networking) Question Question

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
3 answers
yechan May 30, 2019

Hi Ethan.

Thanks for the reply. Creating separated firewall rules with k8s tag worked fine for adding custom port access.

But looks like it is still impossible to override existing firewall rule which already defined on auto generated one.

For example, I want to only allow access to NodePort (port 30000 ~ 32767) from specific IPs. Not just opening to public. But seems defining these custom rules on separated firewall doesn’t really override existing one.

Can you also help me to solve this issue?

Thanks again.

Reply Report
  • Macil March 3, 2020

    I’m having the same problem as yechan above. As efox suggested, I can make a second firewall that applies to the Kubernetes cluster’s droplets, and I can add rules to it that open up extra ports, but it’s impossible for me to close any ports with the new firewall. It seems that DigitalOcean combines all of the additive whitelist rules from all of the firewalls affecting a droplet, so it’s impossible to block anything by adding a new firewall.

    I want to make it so all of my Kubernetes nodes have their services exposed only to the private network and not on the private internet, so that way all access has to go through the load balancers. I believe this is the standard behavior of Kubernetes on other hosts, so I’m a little confused that this seems not to be possible on DigitalOcean.

    Reply Report
    • benoitm September 17, 2020

      That’s a showstopper for us. In our business (aviation domain), we enforce a strict security policy.

      We were planning to do the same: keep our cluster in a VPC and prevent any direct access from outside. All access goes through either a bastion host or our WAF. What’s the point of a VPC, otherwise…

      Reply Report
    • evk02 September 26, 2020

      I am having the exact same issue - can’t permanent close the NodePorts to the public internet and ensuring that it is only available to the load balancer on the VPC.
      I have raised an idea for this - please vote for it:
      https://ideas.digitalocean.com/ideas/NETSECX-I-15

      Reply Report
    • Macil September 27, 2020

      I found a way to restrict access to the droplets so that the ports can only be accessed by the load balancer and not the whole internet.

      This config is for Istio specifically, to limit access to your ingress to just the load balancer (and cluster), but if you’re not using Istio, you’ll want to update the metadata.namespace and spec.podSelector.matchLabels parts to match the pods you want to restrict access to.

      apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: gateway-limiter
  namespace: istio-system
spec:
  podSelector:
    matchLabels:
      istio: ingressgateway
  policyTypes:
  - Ingress
  ingress:
  - from:
    # allow access from private network and load balancer
    - ipBlock:
        cidr: 10.0.0.0/8
    - ipBlock:
        cidr: 172.16.0.0/20
    - ipBlock:
        cidr: 192.168.0.0/16
  - from:
    # allow access from any pods within the cluster
    - namespaceSelector: {}
      Reply Report
efox May 29, 2019

Hey there,

The firewall created automatically by your DOKS cluster has a controller watching its state, helping to ensure the rules needed for the cluster to function stay in-place. Any time reconciliation of the cluster occurs (from adding/recycling nodes, etc.) the state is checked and if any changes are made the firewall is replaced entirely.

Right now, this means no changes to that firewall are recommended. To add additional firewall rules to your DOKS cluster you’ll need to create a separate Cloud Firewall and point it toward your DOKS Cluster’s tag. You can modify this Firewall that isn’t watched by a controller as much as you want, reconciliation will have no effect on it. Creating this via doctl would look something like:

doctl compute firewall create --inbound-rules="protocol:tcp,address:0.0.0.0/0,ports:22" --tag-names=k8s:CLUSTER_UUID --name=<your_name>

Our DOKS Engineering team is working on a more robust firewall implementation to remove these middle-steps, this is something we’re actively investigating and don’t have an exact ETA on release. Be sure to continue watching for updates to DOKS as this is bound to come in future releases.

If you have any other questions or need anything else, just write back in and let us know.

Regards,
Ethan - Developer Support Engineer II @ DigitalOcean

Reply Report
  • sigi324 June 18, 2019

    If that is the case, it should be documented very prominently!

    I thought i’m crazy and had to debug it which wasted my time :(, after all there is nothing preventing me modifying the default created firewall for k8s.

    Reply Report
filiw December 2, 2020

Relevant reading:

https://www.digitalocean.com/docs/kubernetes/resources/managed/#worker-node-firewalls

It explains how to create a new cloud firewall, attach it to your DOKS cluster and how to alter the default “allow all ipv4+ipv6 traffic”-rule that is automatically created when defining a nodeport.

Reply Report
Submit an Answer

Related

Looking for something else?

Ask a new question Search for more help