I love complicated architectures, architectures that involve numerous components, failover constructions and what not. But sometimes simple architectures draw my attention and amaze me because of their straightforward and refreshing simplicity. Let's take load balancing as an example:
If you want to spread a service load between multiple server computers, you need some kind of load balancing. Load balancing can be approached in many ways, among them pretty exciting hardware like F5 Big IP and the likes. These machines are extremely good at what they do, offloading ssl, selecting the right server to defer to based on load statistics, etc, etc. If you want to do it right, you need 2 off course, for failover. Some architectures definitely require the use of a load balancer.
Then again, some architectures don't need all of that. And that leaves room for stupidly simple alternatives. One simplistic example is a usenet access service provider that hands out random server names (server1.usenetservice.nl ... server70.usenetservice.nl) to new subscribers. That is one heck of a round robin implementation! How can anybody not like that!?
Choose simple if you can and complex only if you must.