VLANing your network with L3 switches

While consumer-grade hardware allows you to create a guest WiFi network, fairly inexpensive used enterprise gear gives you far better control. Segmenting your network with VLANs allows you to use your L3 switch as a hardware firewall, isolating devices from each other where desirable.

I aggregate all the VLANs on the core switch, then have 1 VLAN that is connected to the router/NAT, which greatly simplifies configuration on the NAT/firewall, and ensures you can route between VLANs at switch-wire-speed. This also allows me to do a router-on-a-stick configuration which vastly simplifies the NAT/firewall configuration, and moves the VLAN firewall to the switch ASICs.

At “wire-speed” is important: This means devices on different VLANs can talk to each other as fast as port speeds allow. Alternatively, if you put your NAT/firewall/router on each VLAN, this device becomes responsible for handling all traffic, which can quickly become a bottleneck.

I’ve been using Brocade ICX series switches since they let me do L3 routing between VLANs on switch hardware.

VLAN Definitions

Configuration

Let’s get started.

  1. Set up the switch: factory reset, update to latest firmware, get console access. Enable ssh access.
    enable
    configure terminal
    crypto key gen rsa mod 2048
    crypto key generate ec label system
    crypto-ssl certificate generate
    web-management https
    no web-management http
    
  2. Define the VLANs on the switch. I’m using 1/1/10 for an Access Point that will be on the management network, but also be able to put WiFi clients on various VLANs as necessary.
    vlan 10 name management
      tag ethernet 1/1/1 to 1/1/10
      router-interface ve 1
      exit
    interface ve 10
      ip address 10.0.10.1/24
      exit
    vlan 20 name workstations
      untag ethernet 1/1/10 to 1/1/24
      router-interface ve 20
      exit
    vlan 30 name iot
      tag ethernet 1/1/10
      router-interface ve 30
      exit
    vlan 40 name guest
      tag ethernet 1/1/10
      router-interface ve 40
      exit
    vlan 50 name firewall
      untag ethernet 1/2/1
      router-interface ve 50
      exit
    int eth 1/1/10
      dual-mode 10
      # now 1/1/10 is untagged on vlan10, tagged on vlans 20, 30 and 40
      exit
    
  3. Next, we set the IPs on each ve interface, and set up the DHCP relay agent to send traffic to the DHCP server at 10.0.10.5. Ensure you set up DHCP scopes with the appropriate IP ranges; the relay agent will tag the necessary information to ensure the client goes in the right scope. I do NOT use the built-in Brocade DHCP server since it doesn’t properly act as an authoritative DHCP server, which prevents all clients from working properly.
    interface ve 10
      ip address 10.0.10.1/24
      exit
    interface ve 20
      ip address 10.0.20.1/24
      ip helper-address 1 10.0.10.5
      exit
    interface ve 30
      ip address 10.0.30.1/24
      ip helper-address 1 10.0.10.5
      exit
    interface ve 40
      ip address 10.0.40.1/24
      ip helper-address 1 10.0.10.5
      exit
    
  4. While the switch will route between the ve interfaces, it also needs to be able to route clients to the internet as needed. So on the switch, we add a static route:
      ip route 0.0.0.0/0 10.0.50.254
    

    and then on the Ubiquiti EdgeRouter we add a static route for internal VLANs (for example):

      delete interfaces ethernet eth0 address 10.0.1.1/24
      set interfaces ethernet eth0 address 10.0.50.254/24
      set protocols static route 10.0.0.0/16 next-hop 10.0.50.1 description locationname-core
      set protocols static route 10.0.0.0/16 next-hop 10.0.50.1 distance 1
    
  5. Next, firewall the VLANs. Here’s a series of commands that sets ACLs for the Guest VLAN that blocks access to internal VLANs (except an internal DNS server at 10.0.10.6, and a DHCP server at 10.0.10.5), allows other internal VLANs to access devices on this VLAN (established connections), and allows Internet access.
      no ip access-list extended VLAN40-out
      ip access-list extended VLAN40-out
      deny tcp any host 10.0.40.1 eq ssh
      deny tcp any host 10.0.40.1 eq telnet
      deny tcp any host 10.0.40.1 eq http
      deny tcp any host 10.0.40.1 eq 443
      permit udp any any eq bootps
      permit udp 10.0.40.0 0.0.0.255 host 10.0.10.5 eq bootpc
      permit udp any 10.0.10.6 255.255.255.255 eq dns
      permit tcp any 10.0.10.6 255.255.255.255 eq dns
      permit icmp any host 10.0.10.6 echo
      remark permit tcp any any gt 1023 established
      remark allow any remote vlan to access 40.50
      permit tcp any host 10.0.40.50
      permit tcp host 10.0.40.50 any gt 1023 established
      remark allow 40.100 to access 40.50
      permit tcp host 10.0.40.100 host 10.0.40.50
      remark permit tcp any 10.0.40.50 eq http
      deny ip any 10.0.0.0 0.0.255.255
      permit ip any any
      interface ve 40
      ip access-group VLAN40-out in
      exit
    
  6. Here’s a series of commands that sets ACLs for the IOT VLAN set that blocks internet access, but gives other internal VLANs access to devices on this network (established connections)
      no ip access-list extended VLAN30-out
      ip access-list extended VLAN30-out
      remark Block all access to the switch's ve interface
      deny tcp any host 10.0.30.1 eq ssh
      deny tcp any host 10.0.30.1 eq telnet
      deny tcp any host 10.0.30.1 eq http
      deny tcp any host 10.0.30.1 eq 443
      remark allow hosts on the subnet to reach dhcp server
      permit udp any any eq bootps
      permit udp 10.0.30.0 0.0.0.255 host 10.0.10.5 eq bootpc
      remark allow access to ntp on 10.0.10.6
      permit udp 10.0.30.0 0.0.0.255 host 10.0.10.6 eq 123
      remark allow hosts on the subnet to do dns lookups
      permit udp any 10.0.10.6 0.0.0.0 eq dns
      remark allow hosts on the subnet to ping the dns server for debugging purposes
      permit icmp any host 10.0.10.6 echo
      remark allow other vlans to access hosts on this network
      permit tcp any any gt 1023 established
      remark allow hosts on other internal VLANs to access devices on this IOT network
      permit icmp 10.0.30.0 0.0.0.255 any echo-reply
      remark allow any remote host to access 10.0.30.13 on this VLAN on tcp/7442
      permit tcp any 10.0.30.13 0.0.0.0 eq 7442
      remark block outbound access to all local VLANs
      deny ip any 10.179.0.0 0.0.255.255
      remark deny everything else
      deny ip any any
      vlan 30
      ip access-group VLAN30-out in
      exit
    
  7. Enable PoE on the AP port
      int eth 1/1/10
     port-name r710-upstairs
     inline power
      exit
    
  8. Configure your AP with the appropriate SSIDs.

  9. Connect a client to VLAN20 and ensure it gets DHCP, ping the DNS server and run a traceroute.
      ping 10.0.10.6
      Pinging 10.0.10.6 with 32 bytes of data:
      Reply from 10.0.10.6: bytes=32 time=3ms TTL=128
      Reply from 10.0.10.6: bytes=32 time=4ms TTL=128
      tracert 4.2.2.1
      Tracing route to a.resolvers.level3.net [4.2.2.1]
      over a maximum of 30 hops:
     1     1 ms     2 ms     1 ms  workstation-core [10.0.20.1]
     2     1 ms     1 ms     2 ms  edge.lan [10.0.50.254]
     3     3 ms     2 ms     3 ms  some.isp.router [x.x.x.x]