Compliance: the new differentiator?

I thought I’d share with everyone an article I wrote for the Finance on Windows magazine. The magazine is targeted at Microsoft clients within the Finance sector in the UK and is aimed towards CIO/CTO level to discuss industry trends and issues being faced. As always comments are most welcome.

Compliance: the new differentiator?

It has been far too easy to point the finger of blame at the finance sector as a whole for the current economic situation. With both government and public calling for tighter controls, it is inevitable that stricter compliance measures will be introduced and enforced. Regardless of where the fault lies, regulation has become the latest watchword in the UK finance industry.

Many of the current auditing and compliance standards in the UK finance sector are largely voluntary and sector lead, unlike in America where strict standards (such as Sarbanes Oxley) were introduced in the wake of Enron and other past problems. However, with such large government involvement at the commercial ownership level, combined with recent proposals to shake up the regulatory overview structure, it is only a matter of time before stricter, compulsory standards will be introduced in the UK.

In the UK, the Financial Services Authority (FSA) has traditionally engaged with business in an advisory capacity, promoting the organisational benefits of compliance, and highlighting how high achieving companies are reaping the rewards of implementing and driving best practice across the finance industry. Through round table discussions with input from the industry, the FSA has been attempting to lead business towards the benefits of best practice. Historically, the financial services industry was built on trust. In many parts of the industry, trust might soon be supplemented by government regulation aimed at protecting customers, investors and the economy.

Whether it is voluntary or compulsory, the reality is that finance companies will not gain benefit from compliance until they regard it in a different light – many companies see compliance as a set of tick boxes that must be filled in by certain dates. Yet, failure in basic security and data retention systems within some of the UK’s largest banks and the FSA’s combined fines of £3 million show that continuous compliance is not just needed; it makes good financial sense.

Thanks to the shift in responsibility of directors and managers of finance organisations to provide greater transparency for regulators, the ability to report on customer, investor and employee data has never been more important.

Increasingly, finance companies are instigating separate compliance departments to ensure that records are kept, that issues are identified before a customer’s interests are damaged (or the regulator takes action) and that internal controls and regulatory processes are closely adhered to. But at the core, the financial industry must focus on security best practices as its primary goal. Do that right, and compliance will follow by default.

So what is your primary goal? Compliance, security, or both? The best way to understand the difference is to clearly define the two terms: security is a set of practices; compliance is the process to gauge the effectiveness of those practices. It is the understanding of the difference between these that brings about the concept of continuous compliance.

Organisations need to move away from a model where they throw large amounts of money and resources at deadlines, to one that reduces the cost of compliance and delivers lasting value. Businesses that closely and regularly monitor IT security, risk management and data security practices are not only able to achieve compliance by default, they also achieve improved operational efficiency and organisational agility with IT staff spending less time fighting fires and more time on projects of direct benefit to the bottom line.

In fact, more and more businesses are now citing best practice activities to differentiate themselves as a company and improve their position against their competitors. The goal is to achieve a culture of real-time transparency, continuous risk management and compliance and security. As companies progress along this continuum, they move to a state where they are beyond the static ‘check in the box’ IT risk management strategy and they can begin to see the cost savings and tangible benefits that this model brings.

The necessity of moving towards a dynamic continuous compliance model is underscored by the way in which compliance is generally introduced. Whether it is a law, a regulation or a published industry standard, it can take up to a year for that mandate to be written. Businesses need to create a continuous compliance environment that will ensure they remain one step ahead of real-time threat environments.

As an IT director or CFO, you have three main options in your approach to security and compliance: do nothing and wait for regulatory requirements to force action; do the bare minimum necessary to pass regular audits; or take a combined approach to security and best practice to lead the industry and achieve compliance as a default output.

Newspaper headlines show that the first option is no longer viable with the FSA enforcing and fining millions of pounds for lapses in security and compliance. So no matter the projected cost for compliance, it is surely cheaper than a £3 million fine.

Many companies will view the second option as the easy solution, but with more (not less) compliance and reporting requirements looming, this option will become increasingly expensive as the costs and effort of preparing for each audit begin to affect the day-to-day business processes.

The only viable approach is to implement leading edge security processes and solutions. Not only will your business and reputation be secure, but you will also be prepared for any compliance requirements that emerge from the current political storm.

Tim Eichmann, CIO, Parative

This article first appeared in the Autumn 2009 edition of Finance on Windows.

Active Directory Health Checks. A load of rubbish, or something you didn’t know was vital?

I’m involved in an extremely large Active Directory health check at the moment for a large multi-national company with over 1000 locations and just over 400 domain controllers. Complex? Yes. Broken? No. Needs help? Well nearly every AD does…

OK, so what is an ‘AD Health Check’ ?

From talking to people over the last week it seems there is a fair bit of confusion around what an AD health check actually entails, and some mystery around how to do one properly. I’m not going to pretend it’s straight forward, otherwise anyone would be able to do it and there wouldn’t be companies out there (like us, and including Microsoft themselves) who offer these health checks to organisations.

Most big companies are aware that Microsoft Premier Support offer the ADRAP (Active Directory Risk and Health Assessment Program, OK it’s not a perfect acronym). As well as the ADHC (AD Health Check) from Microsoft Consulting Services.
These use scripts and executables from the MS Support Tools to specifically test and report on the overall health of your AD.

But the basics of a health check are to test all of the behind the scenes components in your AD to find out what errors have crept into the environment. The most obvious test is your Replication, ensuring there are no replication errors and that your KCC (Knowledge Consistency Checker) has been doing it’s job in keeping AD replicating without your intervention.

The other big check is DNS, and now in my experience this is where a lot of the confusion lies, as a lot of AD admins know that AD relies on DNS but are not 100% sure of what the ‘_msdcs’ container does and why additional or missing entries in this container can affect the performance of your AD, or ultimately can bring down your whole AD.
This area of AD is also one area that can accumulate bad records as failed DCPROMO demotions will leave old GUIDs and NS entries in here if you don’t follow the manual cleanup process to the letter. And this is the most common spot that the first of the issues appear.

So who needs an AD health check then?

Well the simple answer is, anyone who uses Active Directory in their company.
If you own AD then you should be running tests and fixing errors on a regular basis. And I’m not talking about daily checks here, I’m talking about proper in-depth analysis that you would get from an ADRAP report. If you aren’t doing this at least every 3 months then you’ll be in for a shock when you finally do one.

The obvious question. Can’t I do this myself?

There is a lot you can do yourself without hiring in consultants, but the danger in this is that you may tend to overlook some issues that an external set of eyes will pickup on and point out.
These issues tend to be of the ‘somebody else’s problem’ variety. Douglas Adams was spot on in his ‘Hitchhikers Guide to the Galaxy’ series when he said in a satirical, but perfect observation, that people will tend to ignore anything, even if it’s right in front of them, if they think it is somebody else’s problem.
And therein lies the way that those small errors get into an AD. A domain controller goes down and the decision is made to just build a new one and give it a new name. Obviously it’s then up to ‘Bob’ to do the metadata cleanup and everyone else will just ignore any errors they see relating to this. Why? Becuase it’s not their job. Well if Bob gets halfway through the metadata cleanup and deletes the computer object in AD and runs the appropriate ntdsutil commands but then gets a phone call and never deletes the DNS entries, we now have an extra GUID in DNS and an extra name server (NS) entry, neither of which point to a valid server. Multiply these types of errors by the number of DC’s you own, multiplied by the number of years your AD has been running, and it is actually very easy to see how a regular spring clean of your AD is a very good idea.

But my AD is perfect, right?

I can honestly say that nearly every client I’ve been into to perform an AD Health Check has their local Domain Admins who will swear blind at the start of the engagement that their AD is running perfect and that there are absolutely no problems to be found. They will talk about strict change control, daily log checks and a whole list of reasons why it’s a waste of time and money to have me sitting there.

After about the first 10 minutes when the first test on DNS fails I hear the ‘somebody else’s problem’ reasons for why there are invalid NS records in DNS and why DNSLINT has crashed because of so many errors. The truth is ‘Bob’ probably should have done his job properly the first time, but most AD admins who have never had an AD health check done are not even aware of a multitude of different small errors that can float around in their environments, and usually don’t appreciate that these errors actually exist in their environment because Users can logon and everything seems to be replicating fine, so what’s the problem hey?

The general User complaint, ‘the network is running slow’.

OK so your AD is replicating and your users can logon. Is your AD performing as fast as it should? No. Do the users notice? You’d like to think not but when you get enough invalid DNS servers in their lookup list and they have to wait for the first 2 DNS servers that don’t exist anymore to timeout before they get a reply from a real server, then they begin to make that general complaint of your network being slow without being able to tell you exactly ‘what’ is slow. ‘Everything’ is the answer and most IT people just ignore it as a ‘user’ problem.
Well one of my last DNS checks found that DNS queries in a certain site were taking on average 32 seconds (32,267ms in the tools) to reply with a valid answer to a DNS query. Not sure about you, but if I was browsing the internet and it took 32 seconds to load up each new site I browsed I’d certainly be logging a helpdesk call saying ‘the network is really slow’.

What can you do right now to help?

Well you can start with the free tools from Microsoft to start to look for common problems in your environment. Install the Support Tools and the Resource Kit on your workstation and jump to a command prompt to get started.

DNS – Use DNSLINT.exe with /s and /ad and point them to the IP address of a domain controller.
dnslint /s 10.10.1.10 /ad 10.10.1.10
The /s tells the command to use that IP as the DNS source and not to go to the internet, the /ad tell it to perform Active Directory tests which will scan for the DC GUID records that AD uses to replicate, and test all servers in the Name Server list for your ‘_msdcs’ forward lookup zone.
DNSLINT is free but it’s not perfect. A lot of times when it finds a fatal error it will just crash. Normally because your NS list contains a server that doesn’t exist. In this case you need to look at the last entry and figure out which server should have been queried next, and this is the culprit.

Replication – Use REPADMIN.exe /replsummary to get a nice list of your DCs and their replication status. You can use the /syncall switch to try and force out a sync session but if KCC hasn’t already managed to fix the issue it’s usually something more serious.

File Replications (SYSVOL) – Microsoft have a GUI tool called UltraSound that will do quite good File Replication tests for you. Note that you need SQL 2005, or at least SQLEXPRESS to run UltraSound. But this free tools gives a good view of not just SYSVOL replication but any other FRS you have in the environment.

Domain Controllers – You can use DCDIAG.exe to test for basic DC settings and use the /fix switch to attempt simple but safe repairs for records that dcdiag find incorrect.
DCDIAG /fix is all you need for basic errors and to fix up some of the DNS records I’ve been talking about. Add /v for verbose to see what exactly it is testing.

Products you can buy to do it yourself.

Once you’ve used the free tools you will find a number of limitations. From this point forward it’s time to look at the commercial tools that make an AD diagnosis a walk in the park.

Quest Software – Spotlight on AD (SLAD).
This tool is first as it’s my favourite. On installation the simple pretty GUI interface belies the power that this tool has. In the top right of the topology diagram is a list of health checks. If you scheduled these to run every day, and you fixed the issues found, you would probably be 90% of the way to a green tick in all boxes for an AD Health Check.
SLAD also has a nice web based report interface (.NET 1.1 unfortunately, wish they would upgrade it so it works with my x64 web sites) where you can get a number of meaningful reports on DC health and replication times etc. Good for giving to management to show what a good job you are doing : ) (and to justify any cost to the bean counters)

NetPro – Active Directory Directory Troubleshooter (ADDT)
A good tool for clients with under 100 Domain Controllers (Note that there is a hard coded limit to 100 objects that can be tested with this tool, fine for most, but frustrating if you like the report format but are on-site with a very large client).
This tool has no AD topology GUI like Spotlight, but it does have an inmpressive range of Tests and Reports that you can throw against your AD and DCs. It has a very readable report format, though my only gripe is that it has no filters on the reports. If I’m doing a check on ‘DNS Replication Partners’ I’d really like to only get a report on the errors, not the 20 pages of things that are working in my environment! But all up if you can handle the limitations you’ll get a lot from this tool. About 80% of what you need.

Microsoft – Active Directory Topology Diagrammer (ADTD)
OK so this is actually a free tool (but you do need to own Visio to use it), and it’s not actually got anything to do with diagnostics, but I was happy to see that there is finally a tool that works (and is free) that will query your AD and build in Visio all of those annoying diagrams that we’ve all spent so long trying to create only to have them out of date 2 days later.
ADTD will create AD layouts, OU layouts with GPOs, etc in Visio from a stencil set. For all you AD architects out there make sure you go to MS (http://www.microsoft.com/downloads/details.aspx?familyid=cb42fc06-50c7-47ed-a65c-862661742764&displaylang=en&tm) and grab as copy. My favourite download this week : )

Conclusions?!

Well my personal conclusions from my last few weeks are this:
Most people enjoy their job and take pride in their AD, so it’s a bit of a kick in the old pride when your manager orders an AD Health Check. But do the right thing and give the guy the info he needs, watch what he does and learn from it. After all if you can pickup the method, and the tools (free is OK, commercial if you can afford it) then you can really impress your manager every quarter with your own AD health check that took you no more time than to schedule a few reports and paste them into a word document.
After all, isn’t it better to do that and impressed your boss, than to have your SYSVOL replication stop, not notice, and then have your whole AD fall on it’s butt and then have to call in a guy to fix it.

Recommendations

I personally use all of the above tools mentioned during a health check. There is no 1 silver bullet for a real end to end health check, and where one tools is lacking, another will excel.
When I’m on site I see what the errors are and then show the client what tool found what error and let them decide on how savvy they are with scripting and the free Microsoft tools. It’s either that or a pretty Spotlight GUI that they can leave on a plasma screen showing all green dots that their manager can walk past every day : )

Tim Eichmann

Welcome to the Active Directory & Virtualization blog

Hi and welcome to the Parative blog on Active Directory and Virtualization technologies, my name is Tim Eichmann.

A bit of background

I’m one of the Solution Architects for Parative in the AD and virtualization arenas. I’ve been working with AD since it’s initial beta release in 1999, back in the days of Windows 2000. I was lucky enough to be working for the Queensland government in Australia at the time and we got on the early adopter scheme with Microsoft to be one of the first in Queensland to take the leap from NT4 to Windows 2000 AD in all it’s glory.

I have to say that working with Microsoft was a great experience, especially since I had only been in IT a few years at that point. The early adopter programs give you access direct into Microsoft’s technical teams and we had a specialist fly from all the way from Ireland to Australia to help us design everything and standby when we pressed the ‘go’ button.

So I’ve been involved in the Active Directory migration field since it first began, and I enjoy it as much today as I did back then. Only now I know what AD is doing and there is a lot less trepidation in pressing buttons on migrations for thousands of users.

In the security field I work on user provisioning systems, active directory security models, and auditing and compliance standards such as Sarbanes Oxley (SOX), GCSX (COCO), PCI and many others.

I also work with various tools for security, auditing and compliance such as those by Quest Software. And virtualisation tools from VMWare, VizionCore and Provision Networks.

What I’ll be sharing with you

In my blog posts I’ll be talking about problems that people face during Active Directory migrations, I’ll be sharing tips and tricks that will help you prepare if you are about to start a migration, and I’ll also be posting pieces of PowerShell that I write for migrations and AD security.

I’ve got a lot of background in AD and I really love virtualisation, so please feel free to ask me any questions regarding migrations, AD security best practice, user provisioning, as well as server consolidation with virtualization, Physical 2 Virtual (P2V) technologies, anything really…

I look forward to hearing from you all soon,

     Tim