57 Simple Sys Admin Mistakes that Someday Youll Regret

When it comes to IT, simple mistakes can cost you big time. Whether that is immediately or not depends on a number of factors, and sometimes, when something has gone incredibly wrong, you wont even realize that the root cause was a snap decision or simple oversight you made months ago. If you parse this list, you might just avoid a costly mistake sometime in the future; or, you may see something that makes you realize you werent the only one who had that happen.

Networking
There are tons of simple mistakes, oversights, omissions, and bad ideas that can bite you in the networking realm. Here are some of our favourites. Whether they caused outages, security issues, poor performance, or just made troubleshooting a pain, weve been there, done that, bought the t-shirt. DNS 1. Forgetting to close a zone name with a trailing dot. The lack of one simple period stopped mail for 30K users once at a major company. 2. Leaving your zone open to perform a full zone transfer to any requestor just makes recon easy for attackers. Never permit zone transfers to any server except the ones you want hosting a copy. 3. Setting week long TTLs may reduce traffic, but when you make a typo and dont realize it for a couple of hours, explaining that it will take a week before that site is back up globally, is an uncomfortable conversation to have. 4. Keeping all your DNS servers on the same network is a quick path to losing everything when a routing table entry goes bad. 5. Using routing/NAT instead of split DNS is not the way to let your users access DMZ resources using external names; that is, not if you expect everything to work. 6. Circular recursion is when two different servers are configured to recurse to one another for unknown requests. You can take out an entire Ethernet segment if the DNS servers are robust enough as the request ping-pongs between them.

7. It may seem like a pain to create all the in-addr.arpa records so you can use PTR records internally, but when you are trying to find the source of some rogue traffic, youll be glad you did. Other networking issues 8. Yes, I know Microsoft told you 10 years ago that NetBIOS was going away. Guess what. Its still here. Not configuring WINS is a bad idea and can cause poor network performance. 9. Clear-text protocols should never be used, no matter how simple they are to set up or how limited you think their use will be. Ignore this rule and soon you will see domain creds crossing the wire. 10. Using non-standard ports is just a bad idea all around. You may think it lets you do more on your server, or provides some security, but all it really does is keep your customers from accessing the services. Way too many potential customers could use your web app if it was on 80 or 443, but their network blocks outbound access to anything else. 11. Legacy rules on the firewall must be cleaned up immediately. Open ports to unused ip.addrs are a disaster waiting to happen when a new box uses an old address. 12. NAT internal traffic out to the Internet, and Internet traffic to your DMZ for applications that support it, but never NAT internally. You will break stuff left and right, and some of that cannot be fixed. There once was a major corporation who ignored that, and wound up having to migrate 10,000 user accounts, twice, because they couldnt get a trust working through a NATd connection. 13. Using long xlate timeouts means that bad connections stick around for hours. Think of xlates in terms of seconds to minutes, but never in hours. 14. Bypassing the firewall is never the right answer. If you are going to allow systems to span the external and internal network, and they themselves are not firewalls, you may as well just get rid of the firewall entirely. Think of the money youll save on annual support. Youll need that to pay for incident response! 15. Never use a domain name that you do not own. Register anything you want to use, even if you think it will never be used externally. 16. Never use bogon addresses. They wont be bogons forever. RFC 1918 addresses and registered addresses you control are the only ones you should use.

17. Using a HOSTS file to get around something, and then never going back to fix the issue. Its never the HOSTS file, until you spend a few hours troubleshooting DNS before you finally remember that it actually is the HOSTS file.

User accounts and passwords

18. Go Bing Search default password lists, and then go change all the default passwords you never thought were a risk. 19. Never set a users password to never expire. We expire passwords so that when they are compromised, we limit the time the attacker can use the compromised credentials. Never is a mighty long time. 20. Dont forget to set a service accounts password to not expire, and then sch edule a regular reset. If a service accounts password expires, the next time that service needs to start, it wont. 21. Setting too complex a password requirement just leads users to write things down. 22. Requiring too many things in password (must start with a capital, must end in number, must be no more than 11 characters long, etc.) leads to passwords that will be written down.

23. Using the same password for multiple accounts means that an attacker has the master key as soon as he compromises one service. Use different passwords for your work, personal email, Facebook account, etc. 24. Never use a User account as a service account. You change your password, you break the service. 25. Leaving sAMAccountName blank may be fun and make scripting easier, but there are some systems that cannot use UPN. 26. Using non-routable UPNs works great internally, but the more services that move to the cloud, the more important a routable UPN will be. 27. Never set the local admin account password to be the same on all workstations. Someday, it will get out, and youll have to change it on every single machine in the company. Dont use a pattern that can be easily figured out either. 28. Stale accounts should be disabled and then deleted.

29. Weak passwords are worse than strong ones written down. An attacker has to find the piece of paper you wrote a password on; they can break a weak password from anywhere. 30. If you have a web service you offer to external users, like your customers, let them use their email address as their username. Its unique, you dont have to maintain it, and they will be much happier for it.

Speaking of Email
31. As much as they are spam magnets, you still need to establish the RFC 2142 addresses and monitor them. Comply with the RFCs. 32. The last thing you should do to a mail server before opening port 25 on the firewall is to check it for open relay. 33. The first thing you should do to a mail server after opening port 25 on the firewall is to check it for open relay. 34. Emailing a bunch of people, some of whom dont work for you? Use BCC. CCing hundreds of strangers happens by accident all the time. That they immediately drop your service and block your emails after this happens is not by accident. 35. Spellchecking is not optional! Enable it on your client, and check the spelling of your email, and your subject line! 36. Reply to all is almost never a good idea. Consider how much noise you deal with every day in your inbox, and do unto others as you wish they would do unto you.

Active Directory
37. Never set up a DC to suffer DNS islanding. A DC should never use itself for DNS; always point it at another DC. 38. Keep your admin accounts separate and use a second, normal account for your workstation use. Too many things like inheritance filtering can cause administrative accounts all kinds of problems with other services.

39. Monkeying with inheritance is a bad idea. Ive never seen a case where it was a good idea, but Ive seen literally hundreds where someone turned off inheritance years ago, and it caused major issues for the poor team that inherited the mess. 40. Repeat after me. You create a new subnet, you define it in AD and assign it to a site. It takes two extra minutes. Not keeping up with your sites will cause logon and other performance issues, can impact replication, and will be a multi-month project to clean up when it finally becomes the blocker to a major project.

Antivirus
41. Disabling antivirus should be a capital offense. 42. Not configuring proper exemptions based on vendor recommendations is a sure way to crush application performance, and will cause others to disable antivirus protection. 43. Not investigating failed updates immediately is asking for trouble, since usually the reason an update failed is either because the antivirus was disabled, or your machine is already infected.

Teach your Users well

44. Asking for their password is never acceptable. The first time you do, you can never again expect them to safeguard their credentials from potential attackers. 45. Using self-signed certs just trains your users to ignore certificate warnings and makes them click-through-happy. The last thing you want users to do is ignore warnings.

Other
46. Certificates expire. Not monitoring certificates will lead to outages, and mad scrambles to replace them. Set up a reminder for a couple of weeks ahead of expiration so you have plenty of time to renew. 47. Pings are not the enemy. Not allowing ICMP echo just makes troubleshooting and simple monitoring more difficult, and it violates RFC 792 and 4443. Dont violate the RFCs. 48. Not using NTP is just lazy. Pick a good time source and sync everything to it. It makes your logging correlation much easier, makes users happy, and prevents authentication from breaking.

49. Not verifying backups is playing Russian Roulette. You must perform regular test restores to ensure you have a good backup. 50. Deploying an untested patch is only fractionally worse than 51. Skipping a patch. 52. Encrypt everything that can leave the four walls of your network. That means laptop hard drives, USB keys, and network traffic. Unless its data you would post on your website, it should never go out the door (physical or virtua l), unless its encrypted. 53. Using unsupported configurations may be cheaper or faster, but unless you really are willing to forgo the option of vendor support, its just not worth it. Youll spend more money trying to troubleshoot an issue, or waste more time getting things back into a supported configuration than you ever saved by trying to cut corners. 54. I have personally seen at least three instances where letting support/maintenance lag or expire wound up costing the companies in question millions of dollars. Never let your software support expire until you have formatted every machine that was running that software. 55. Staying on legacy software versions may save you upgrade fees, but eventually you will find yourself in an unsupported configuration, or at a point where you want to do X, but you cannot because it wont support legacy Y. Youve just found that proverbial spot between the rock and the hard place. 56. Hardcoding anything just dont do it. 57. Documentation is a pain. We all hate it, we never do it, and yet we curse the guy who came before us that didnt keep good documentation. Pay it forward.

So there you have 57 far too common mistakes, omissions, shortcuts, bad ideas, and lapses in judgment that might seem like a good idea today, but will cost someone pain and suffering in the future. Dont be that guy. Do things right, the first time, and youll be much better for it, and so will your network.