A flat network works beautifully right up until the moment you understand what it is. A few years back I walked into a hosting datacenter in Brazil where every machine could reach every other machine, no questions asked. Customer servers, management interfaces, staff laptops, the billing box, all of it sharing one big happy broadcast domain. It had run like that for years and nothing was on fire. That was the problem, because “nothing is on fire” is a terrible reason to sleep well.
Table of contents
Open Table of contents
The short version
- The datacenter ran on a flat, unmanaged network where everything could talk to everything, with no segmentation and no real edge control.
- Over the following year or so we swapped the dumb switches for managed Layer 3 gear, carved the network into VLANs, and put proper routers and a firewall in front of it.
- We built internal and external security policy and real VPN access rules for staff and partners.
- The goal was never to make it faster. It was to make the blast radius of any single mistake or breach small.
- “It works” tells you packets arrive. Whether it’s safe tells you what happens on your worst day. Those are not the same measurement.
The context, and the quiet problem
Hosting is a trust business dressed up as a bandwidth business. Customers hand you their servers and assume the guy in the next rack cannot poke at their management port. On a flat network that assumption is a polite fiction. Nothing separated customer traffic from the internal management plane, so a compromised customer VM could ARP-scan the whole segment and find the switches, the storage, the staff machines, everything.
Nobody had abused this yet, as far as we knew. But security is not about what happened, it is about what is reachable, and on that network everything was. The edge was just as casual: no serious border firewall doing stateful inspection, and remote access that leaned on the network being “internal enough.” For a company selling reliability, the gap between the pitch and the plumbing bothered me.
How the rebuild actually worked
We did it in layers, from the core outward, because you cannot segment a network you cannot see.
First, the unmanaged switches went in the bin and managed Layer 3 switches took their place. That single change gave us VLANs, and VLANs are where the real work started. We split the flat space into separate broadcast domains by role: customer-facing, management, storage, staff. Traffic that used to flow sideways for free now had to pass through a router to go anywhere useful, and routers are where you get to say no.
The routers handled inter-VLAN routing and policy. A firewall sat at the border doing edge control and stateful filtering, so the datacenter finally had a real front door instead of an open lobby. Then we built the access structure on top: internal and external security policy, and VPN access with actual rules about who gets in, from where, and to which segment. Staff and partners went from “you are on the LAN, welcome” to “you authenticate, and you reach only what your role needs.”
A simplified sketch of the segmentation intent looked roughly like this:
VLAN 10 management -> switches, iDRAC/IPMI, monitoring (staff-only, via VPN)
VLAN 20 storage -> SAN/backup traffic (no customer access)
VLAN 30 customers -> customer servers (internet-facing, isolated)
VLAN 40 staff -> internal workstations (policy-controlled)
default deny between VLANs; allow only what a role actually needsThat last line is the one that mattered. Default deny. On the old network the implicit rule was “allow everything.” Flipping that default is the whole project in one sentence.
On the gear itself the config is unremarkable. Creating a VLAN and pinning a customer port to it is a handful of lines:
! create the customer VLAN and pin a port to it
vlan 30
name customers
interface GigabitEthernet0/5
switchport mode access
switchport access vlan 30
! between VLANs: permit only what a role needs, then deny the rest
ip access-list extended CUSTOMERS-IN
permit tcp any host 10.30.0.10 eq 443
deny ip any any logThe deny ip any any log at the bottom is the whole philosophy in one line. On the flat network that rule did not exist and everything was implicitly permitted. Here a packet has to match an explicit permit or it gets dropped and logged, which also means the logs finally tell you who is trying to reach what.
The one detail worth remembering
Segmentation is not really about the VLAN tags. It is about who owns the decision when two machines want to talk. On a flat network the network itself decides, and it always says yes. The moment you route between VLANs through a firewall, you decide, deliberately, one rule at a time.
That is more work. You will get paged because someone’s perfectly reasonable traffic is now blocked by a default-deny you wrote. That is not the system failing, that is the system finally asking a question it never used to ask. Every “why is this blocked” ticket was a connection that previously happened silently, with nobody’s permission. I would rather field the ticket than find out the hard way.
The lesson
The hard part was not the config. Routers and firewalls are well-documented, VLANs are decades old, none of this was clever. The hard part was cultural: convincing people that a network that had “always worked fine” was quietly a liability. Working and safe feel identical from the inside, right up until they violently do not.
Final takeaway
If everything on your network can reach everything else, you do not have a network, you have a shared fate. Segment by role, deny by default, put a real firewall at the edge, and make remote access prove itself. None of it makes the packets faster. All of it makes your worst day smaller, and in hosting, shrinking the worst day is the entire job.