Exchange 2010 SP2 and Office 365. What works and what to look out for.

Exchange 2010 Service Pack 2 was released in Dec 2011 along with a few updates to the Microsoft Office 365 cloud service. This article contains a few points on whats new, what works better, and what to look out for if you are upgrading to Exchange 2010 SP2 and you are already connected to Office 365. I’ll be updating this article as we find new features or problems while we roll out our next wave of Office 365 customers.

What to watch for!

1. Changes to the MRSProxy

For those of you who have already deployed an Exchange 2010 co-existence server and connected successfully to Office 365 in Hybrid mode, if you are looking to install Exchange 2010 SP2 on your co-existence server then ensure you are aware of the changes to the MRSProxy in SP2. The Mailbox Replication Proxy (MRSProxy) is required in a Hybrid setup to move mailboxes from the on premise systems to Office 365. In Exchange 2010 SP1 you used to have to update the web.config file manually to enable the MRSProxy on the Client Access Server (CAS) that you were presenting to the internet. In SP2 there is now a new PowerShell switch to configure this, but the catch is, when you upgrade to SP2 your existing web.config file is overwritten so the MRSProxy is disabled after installing Ex2010 SP2! So if you have upgraded and now can’t do any remote mailbox moves, that is why.

To enable the MRSProxy is SP2 open your Exchange Management Shell PowerShell prompt and run:
Get-WebServicesVirtualDirectory | fl
This will return your existing CAS list so you can copy and paste the Identity values into the next command.
Set-WebServicesVirtualDirectory -Identity “EWS (Default Web Site)” -MRSProxyEnabled $true -MRSMaxConnections 100

This will enable the MRSProxy correctly in SP2. If you have changed the timeout values of the data move then you will again need to go to your web.config file and update the timeout value again. If you are not familiar with this it is the timeout value of the MRSProxy when performing a remove mailbox move. When you are performing bulk migrations of users to Office 365 it is a good idea to increase this so you don’t get failures during overnight data loads if you are using virtual machines for the Mailbox or CAS roles.
Open the web.config file located in D:\Program Files\Microsoft\Exchange Server\V14\ClientAccess\exchweb\ews
Go to the bottom of the file and locate the new smaller MRSProxy section and make the change shown:

<!– Mailbox Replication Proxy Service configuration –>
<MRSProxyConfiguration
DataImportTimeout=”00:20:00” />

We have found this helps especially in Proof of Concept environments where the virtualisation hosts may be slower than a production platform and the default 1 minute timeout can unfortunately happen causing the client to ask questions about stability. Better to tweak this up front and ensure a smooth test or data load.

2. The Hybrid Mode Wizard

This one is really just for people who already have a Hybrid mode setup working. Don’t run this wizard if you are already working with DirSync, ADFS and Exchange Federation. It is a case of “if it’s not broke, don’t fix it”. The way we have setup client in the past required a lot of extra work around service sub-domains and setting up Organization Trust & Org Relationships as well as Send & Receive Connectors. If you run this wizard it will setup a new set of names and you’ll end up with new SMTP suffixes, Org Trusts, Org Relationships etc. Only run this wizard if you have never setup hybrid mode in your environment! Read more below to see what it does.

3. The Exchange Deployment Guide is not up to date

It normally takes a few weeks from any update being released for the online Exchange Deployment Guide (http://technet.microsoft.com/en-gb/exdeploy2010/default.aspx#Index) to be updated. Currently if you follow the deployment guide it will still step you through the manual instructions for Exchange 2010 SP1, if you then use the Hybrid Wizard you will end up with conflicting settings in your manual version VS the wizard created objects. For now I’d recommend if you are trying this yourself to wait until the Deployment guide is updated as there are still some pre-requisites you must do before running the Wizard. If you need to get a POC running ASAP then either ignore the new wizard and follow the deployment guide or get someone like us to put in it for you. Unfortunately the Wizard isn’t 100% straight forward yet.

Exchange 2010 Deployment Guide

 

Whats Changed?

The new Hybrid Mode wizard

One of the key changes to Exchange SP2 are a new set of Schema extensions to hold more data for Hybrid mode installations since these are the most common people are going for.
Part of making this easier is a new Hybrid Wizard to help clients deploy a hybrid configuration into an existing Exchange 2010 organisation.

Here is what your EMC (Exchange Management Console) will look like after you install SP2.Exchange 2010 SP2 Hybrid Wizard

The new wizard steps you through the requirements for setting up the Hybrid configuration.

 Exchange 2010 SP2 Hybrid Wizard

The most important thing to note on here is the requirement for an externally trusted SSL certificate. Since you will need to put this certificate in a number of places, most notably Threat Management Gateway to publish the MRSProxy, we recommend you buy a Wildcard certificate. You can pick up a 5 year wildcard certificate from GoDaddy for very reasonable prices compared to VeriSign and GoDaddy is on the trusted cert list from Microsoft in Win7 and XP with the Root Certificates Update from Windows Updated. We’ve used GoDaddy certs ourselves and for a few clients to date without problems.

Whats it do?

The Hybrid Wizard is basically asking you for a lot of info up front and then running all of the commands you see in the Exchange Deployment guide in the background. When you get errors from the Hybrid Wizard you will need to open the log file located in D:\Program Files\Microsoft\Exchange Server\V14\Logging\Update-HybridConfiguration\
In this log you will see the individual commands running and their success or error status. The list of commands that are run is quite long and covers most of what you will see in the Exchange Deployment Guide, but includes setting up the Organisation Relationships and free/busy lookup as well as routing from cloud to on-premise and vice versa. One thing to note is how the routing is setup. All email will come into your on premise system and then be securely forwarded to Office 365, you do have an option during the wizard to choose whether Office 365 messages to the internet are delivered directly from 365 to the internet or whether it should go back through your on-premise system before going to the internet. You may need this scenario if you require Journalling of all emails or want to stamp disclaimers etc onto all outbound emails. At a later point things like auto signatures and disclaimers can be done from Office 365 with Tranport Rules (read Technet for some good examples).

Wrap Up

So that is a small look at the new Hybrid Wizard in Exchange 2010 SP2. We will post any new gotchas are we hit them in the field of SP2 updates and how they affect your Office 365 installation. If you have any good/bad stories of your own then feel free to email us at info@parative.com or if there is anything you’d like to see us discuss then just ask.

Thanks : )

Tim Eichmann, CIO, Parative

Performance Counter Problems Exchange 2010 SP1 RU3

Recently I had some unusual issues on a client site. We had an admin server that was running just the Exchange Management tools (EMC, EMS). Post installation of Exchange 2010 SP1 Rollup 3 we started getting performance counter errors in the event logs, about 15 errors at random points during the day. 

Exchange 2010
Source: MSExchange Common
EventID 106
Category (1) 
Performance counter updating error. Counter name is Total Admin Audit records saved, category name is MSExchange Admin Audit. Optional code: 2. Exception: The exception thrown is : System.InvalidOperationException: The requested Performance Counter is not a custom counter, it has to be initialized as ReadOnly.

 

Microsoft have various articles like http://support.microsoft.com/kb/982679/en-us which discuss various problems with their counters and the workaround is just to ignore them. Well that isn’t a very satisfactory answer when you have an event log full of red errors and a log management system alerting that the server is broken! 

Having done the usual Google search and found many people with various versions of our problem it turned out the fix wasn’t straight forward as we seemed to have been hit with multiple problems. 

Root cause was the performance counters were broken as everyone knows. We also were missing registry keys which meant not all of our PowerShell components were loading in the EMS. So in case anyone has this much fun with one of their servers, here are the fixes in order of running. 

Step 1. Fix your EMS PowerShell snapin to get all your commands back

When you open your Exchange Management Shell (EMS) your should be able to run Get-PSSnapIn and get the following appear:
Name : Microsoft.Exchange.Management.PowerShell.E2010
PSVersion : 1.0
Description : Admin Tasks for the Exchange Server 

Name : Microsoft.Exchange.Management.Powershell.Support
PSVersion : 1.0
Description : Support Tasks for the Exchange Server 

Name : Microsoft.Exchange.Management.PowerShell.Setup
PSVersion : 1.0
Description : Setup Tasks for the Exchange Server
Along with all of your standard PowerShell snapins. 

If any are missing, especially the Exchange Setup snapin, you won’t get very far. 

You need to edit the registry somewhat to fix this one.
Open Regedit and check out
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\PowerShell\1\PowerShellSnapIns]
If you don’t see the following then you’ll need a bit of work done. 

  

To save anyone some work just copy and paste the following into a .Reg file. Make sure you update the ModuleName path to equal where you installed your Exchange files. 

Start —> 

Windows Registry Editor Version 5.00 

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\PowerShell\1\PowerShellSnapIns]
@=”" 

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\PowerShell\1\PowerShellSnapIns\Microsoft.Exchange.Management.PowerShell.E2010]
“CustomPSSnapInType”=”Microsoft.Exchange.Management.PowerShell.AdminPSSnapIn”
“ApplicationBase”=”D:\\Program Files\\Exchange Server\\bin”
“AssemblyName”=”Microsoft.Exchange.PowerShell.Configuration, Version=14.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35″
“Description”=”Admin Tasks for the Exchange Server”
“ModuleName”=”D:\\Program Files\\Exchange Server\\bin\\Microsoft.Exchange.PowerShell.Configuration.dll”
“PowerShellVersion”=”1.0″
“Vendor”=”Microsoft Corporation”
“Version”=”14.0.0.0″ 

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\PowerShell\1\PowerShellSnapIns\Microsoft.Exchange.Management.PowerShell.Setup]
“ApplicationBase”=”D:\\Program Files\\Exchange Server\\bin”
“Description”=”Setup Tasks for the Exchange Server”
“ModuleName”=”D:\\Program Files\\Exchange Server\\bin\\Microsoft.Exchange.PowerShell.Configuration.dll”
“PowerShellVersion”=”1.0″
“Vendor”=”Microsoft Corporation”
“Version”=”14.0.0.0″
“AssemblyName”=”Microsoft.Exchange.PowerShell.Configuration, Version=14.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35″
“CustomPSSnapInType”=”Microsoft.Exchange.Management.PowerShell.SetupPSSnapIn” 

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\PowerShell\1\PowerShellSnapIns\Microsoft.Exchange.Management.Powershell.Support]
“CustomPSSnapInType”=”Microsoft.Exchange.Management.Powershell.Support.SupportPSSnapIn”
“ApplicationBase”=”D:\\Program Files\\Exchange Server\\bin”
“AssemblyName”=”Microsoft.Exchange.Management.Powershell.Support, Version=14.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35″
“Description”=”Support Tasks for the Exchange Server”
“ModuleName”=”D:\\Program Files\\Exchange Server\\bin\\Microsoft.Exchange.Management.Powershell.Support.dll”
“PowerShellVersion”=”1.0″
“Vendor”=”Microsoft Corporation”
“Version”=”14.0.0.0″ 

–> End. Copy in between the markers, update your ModuleName to your path and import into your registry. 

So that is fix 1. This will get your EMS PowerShell session back up and running with the full range of Exchange 2010 Commands that you need. 

Step 2: Remove the bad Performance Keys.

For this we are going to open up a EMS PowerShell prompt and run the following. 

  • Type add-pssnapin Microsoft.Exchange.Management.PowerShell.Setup and hit enter.
  • Go to your Exchange BIN folder:  cd “D:\Program Files\Exchange Server\Bin\perf\AMD64″ (Change the path to where your perf folder is in your Exchange install)
  • Type remove-perfcounters -definitionfilename .\eseperf.xml and press Enter.

You will now need to look very closely at the error in your event log 

 

As you can see we still had errors that were coming from the “MSExchange Admin Audit” performance counters. So far we had just removed the base ESE counters that covered most of our errors. 

  • To fix the last of the problems you have to go to the very specific counter you have a problem with.
    CD “D:\Program Files\Exchange Server\Setup\Perf”
  • We had to also run remove-perfcounters -definitionfilename .\AdminAuditPerfCounters.xml which then unloaded the last of our broken counters.
  • Keep an eye on your logs and look at what Category the errors come from and just remove the specific sets from the Setup\Perf folder.

If you don’t plan on monitoring any Performance Counters on the server then you are done. As this was an Admin console we didn’t want any Exchange specific perf counters so we stopped here after we had cleared all of the error from the Application Log. 

If you are reading this and you have a Hub/CAS/Mailbox server with this issue then you will want to reload the counters now so that they are working. 

Step 3 (Optional): Fix the Performance Counters.

Back at your PowerShell prompt: 

  • Type add-pssnapin Microsoft.Exchange.Management.PowerShell.Setup and hit enter.
  • Go to your Exchange BIN folder: cd “D:\Program Files\Exchange Server\Bin\perf\AMD64″ (Change the path to where your perf folder is in your Exchange install)
  • Type add-perfcounters -definitionfilename .\eseperf.xml and press Enter. This will reload the base ESE counters.
  • CD “D:\Program Files\Exchange Server\Setup\Perf”
  • Re-Add any other bad counters that you removed in the last step.
    add-perfcounters -definitionfilename .\AdminAuditPerfCounters.xml
  • This will reload the counters and fix any permission errors in the registry that were stopping the keys being read. It will also replace any missing or broken keys from the various Service Packs and Rollups you’ve been applying.

If you are after specific counters then you can reload any that you need from the \Program Files\Microsoft\Exchange Server\V14\Setup\Perf directory in a default installation. 

 

I really hope this will fix any issues you have with Performance Counters on your Exchange servers and Exchange management boxes. A good admin will always investigate why there are ERRORS in event logs. Never ignore them as ignorance is rarely bliss in an IT environment :)  

Thanks,
Tim 

Tim Eichmann, CIO, Parative 

Quest Archive Manager – Administrative Experience

Today I thought I would do a quick run through of the basic features of the administrative interface for Archive Manager from Quest Software.

Archive Manager is an email archiving platform which is quick to implement and generally once up and running requires minimal maintenance. Archive Manager is developed around standard Microsoft Technologies such as IIS and SQL Server so current Microsoft based support staff tend to have a very short learning curve for this product. I will look to cover architecture in a future article.

The main user and administrative interfaces for Archive Manager are all web based running on an IIS server so are typically accessed with Internet Explorer. The same URL is used to access the user and administrative portions of the web site. Archive Manager also integrates with Microsoft Office Outlook for user access but I will cover that later.

UserAdmin1

The above image shows the top of the user screen after accessing the Archive Manager web site. In the top right-hand corner the Administration button is visible. This button is only visible to users who have some form of administration right within Archive Manager.

Admin1

After clicking on the Administration button the administration page loads and presents the different configuration options available. Only options that a user has rights to use are visible. As I am the system administrator all options are currently shown here.

Below I am going to step through some of these options.

 

 

 

Login1

The Logins option allows the administrator to view all of the logins that have been imported from the directory and which domain they belong to.

Opening the properties of a login allows the users security role to be set (within Archive Manager) and whether this account is allowed to login to Archive Manager via the Active Option. All new users are automatically made active by default.

 

 

MailboxesOption

The mailboxes option list information about the mailboxes retrieved from the email servers. The summary screen shows the mailbox name, it’s email server and when the folder was last update. The Last Updated time shows when the mailbox was last touched by the service responsible for extracting and stubbing emails from mailboxes.

In the mailbox properties there is an option to enable store management. This option in simple terms turns archiving on/off for this mailbox. There are also options to set specific policies for this mailbox which override those set at the server level.

SecRolesMainThe security roles option allows administrators to customize the default four roles (Administrator, Manager, Resource and User) that are installed with Archive Manager or to create new customized security roles to meet your specific needs. When accessing the Archive Manager interfaces only the features that are enabled by the security roles are visible to users. As can be seen by looking at the properties of the one of the roles, there are a large number of security options to choose from.

MBXServerMain

The mailbox servers option shows the mailbox servers which are visible to Archive Manager, in this case a single Exchange 2010 Server. The summary screen shows the server type, whether it is enabled for store management and on what schedule.

The server properties screen enables the administrators to turn store management on/off and to set processing policies for the server. For a mailbox to be processed by Archive Manager, store management needs to be enabled both on the server and on the mailbox. The policies set here at the server level can be overridden on a user by user basis, but generally server policies are used.

Policy1The final option I am going to look at in this article are the Message Policies which fall into two general categories. Export tells Archive Manager to copy email out of mailboxes based on policy, this can be done immediately the next time a mailbox is processed or after a number of days. Strip Policies convert messages within the mailbox into Stub objects with a link to the full message within the archive. The strip policies can either strip the full message or just attachments. The strip policies also have options to include or exclude un-read messages, flagged messages and whether to delete the stubs after a period of time.

I have only looked at the basic administration functions here but as you can see it is a very straight forward interface to use and simple to support. There are some installable tools to assist with administration for tasks such as retention policy management, PST Import and Search Exports.

Exchange 2010 SP1 Mailbox Archive and Retention Policy Features

I thought I would take a better look at the new Exchange 2010 feature, the personal Archive which is effectively a second mailbox for a user. The personal Archive was introduced to Exchange server to give administrators an option for dealing with .pst files which traditionally have the following problems:

    1. Access to .pst files sitting on file shares are not supported.
    2. .pst files generally reside on workstations and as such are a problem to locate and search through for discovery requests.
    3. .pst files are prone to corruption.

In addition to the personal archive feature of Exchange server 2010, Microsoft have introduced retention and MoveToArchive policies which will enable administrators and users to control mailboxes sizes.

With the release of Exchange Server 2010 RTM the personal archive mailbox had to be created in the same mailbox database as the users mailbox but with the advent of Exchange Server 2010 SP1 it is now possible to store the personal archive mailbox in a different mailbox database which could be on a completely separate server. The one thing to note here however is that the personal archive feature of Exchange 2010 does not utilize any single instance storage so it is not as efficient as Quest Archive Manager or Symantec Enterprise Vault which can cut your overall storage costs. Also similar to a full user mailbox there is a quota which can be managed by administrators’ for the personal archive mailboxes.

imageWhen looking to archive-enable an existing mailbox the administrator is presented with an option to specify a specific database in which to create an archive. In this example I have a mailbox database to host the live user mailboxes and a separate mailbox database to host their archives on a second server.

image

The Exchange Server 2010 Archive cannot work with emails captured through the Journal process in Exchange. In the absence of a 3rd party product which normally captures Journaled content you are a limited to storing Journal mail in a dedicated mailbox which may or may not be managed by Exchange Server native functionality to keep it’s size under control.

The personal archive can have quota’s set in order to control size and disk usage by the archive. The default values for the archive vary between the RTM and SP1 versions of Exchange Server 2010

Version

Archive Warning Quota

Archive Quota

Exchange 2010 RTM Unlimited Unlimited

Exchange 2010 SP1

45Gb

50Gb

If enabling the archive for a new or existing user, the archive folder will be visible in Outlook 2010 and OWA 2010 straight away, however the retention policies will not be available to the user until the managed folder assistant has run which by default occurs overnight. However it is possible to force the issue by running the following command

Start-ManagedFolderAssistant –identity <MailboxID>

image

After running this command a user will have to logout of OWA or Outlook before the retention policies will be available.

The Exchange archive folder is only visible to users of Microsoft Office Outlook 2010 and Outlook Web App 2010 although Microsoft are looking to release an update for Outlook 2007 possibly in the first half of 2011 to add support. The Exchange archive folder however is not available to Outlook 2010 while working offline.

To assist administrators manage mailbox content Exchange server includes retention policies which can be applied to mailboxes. Retention policies are replacing managed folders which are de-emphasised from Exchange server 2007.

image

Retention policies consist of a number of Retention Policy Tags which with the advent of Exchange Server 2010 SP1 can be created and edited through the Management Console, and there are three types of policy Tag.

Default Policy Tag – These tags when added to a Retention Policy apply at the mailbox root and apply to all items in a mailbox unless overridden by a more specific folder level policy. There can be only one Default Policy Tag per retention policy which can include one of the available actions which are Delete and allow Recovery, Permanently Delete and Move to Archive..

Retention Policy Tag -  These tags can apply retention to a default mailbox folders such as inbox and can include actions such as Delete and allow Recovery and Permanently Delete. A Retention Policy Tag cannot include the action Move to Archive. As a result the only way of applying a Move to Archive action on a default folder would be via the Default Policy Tag. When applied to a default folder a Retention Policy Tag cannot be overridden by a User, however a user can set a Personal Tag on individual items within a default folder.

Personal Tag – These tags are ones that can be applied by a user and can include all of the available actions. However the user is not able to apply a personal tag to a default mailbox folder but can apply the tag to items within a default folder. Of course Personal Tags can be applied to the users custom folders. Personal Tags that include the Delete and Allow Recovery and Permanently Delete options appear under the Retention Policies menu options in Outlook and OWA where as Personal Tags with the Move to Archive option appear under the Archive Policy menu options. If a Personal Tag is applied it will always be processed in preference to Retention Policy Tags or Default Policy Tags even if it means that the message will be moved or deleted sooner or later than the settings specified in the Retention Policy Tag or the Default Policy Tag.

Unfortunately the retention policies are indiscriminate and cannot be applied by keywords in the message subject or body.

Retention policies should be planned to meet the companies retention in advance of deployment as it is best practice to keep the set of Retention Policy Tags and Retention Policies to a minimum and kept simple. From time to time however some Retention Policy Tags will fall out of use but there are some considerations that need to be accounted for:

  1. Removing a Retention Policy Tag from a Retention Policy makes it unavailable to users but items that are stamped with that tag are still processed by the Managed Folder Assistant with the properties of that tag.
  2. Deleting a Retention Policy Tag will remove the Tag from all items with that tag applied. This should be done with caution as this has the potential to generate a high workload for the Managed Folder Assistant which will consume server resources depending on the number of mailboxes and items affected by the tag removal.

The implementation of Archiving and Retention Policies is somewhat unique compared to the 3rd party offerings in the marketplace. Before taking the decision to implement Exchange Archiving to meet your compliance needs I would advise checking that the available functionality meets whatever local standards you need to work to. Features such as users being able to modify policies on specific items may exclude the solution from standing up in a court of law if required.

Exchange 2003 Routing during Quest QMM Migration and 5.4.4 No Route Found NDR

While working on a migration involving Exchange Server 2003 recently we encountered an interesting bug with the SMTP connectors in Exchange Server 2003 which is not publicly documented by Microsoft.

The diagram below demonstrates the configuration in Exchange Server 2003 that we were using

image

The configuration that we had in place was where the Exchange 2003 bridgehead server hosted multiple SMTP connectors, some of which were for specific routes but also included a connector for the default route.

What we were finding was that when an email was sent to a @source.local address we would get a 5.4.4 No Route found NDR report returned from the Boundary SMTP Smart host. As you can see from the above diagram the emails for @source.local should never be routed to the Boundary device. Using WinRoute we could see that the routing table on the bridgehead server hosting the connectors was correct and contained the entry for the specific route.

After a support call with Microsoft we discovered that there is a bug in Exchange Server 2003 with the way that the Categorizer handles SMTP connectors when there is an internet based connector present. I had not seen this during any previous migrations but it manifested itself for this client.

The resolution in this particular case was to move the more specific connector up onto Exchange Server 2007 as this particular environment had Exchange 2007 deployed so we used this architecture.

image

If you are not fortunate enough to have Exchange 2007 or Exchange 2010 running in conjunction with Exchange 2003 the only real alternative if this problem occurs is to deploy another Exchange 2003 server to act as a bridgehead for the more specific route and remove the specific route from the bridgehead server hosting the Internet based connector using an architecture similar to that shown below.

image

Quick Exchange Migration with QMM using Alternate Hosts

One of our clients recently had a requirement to migrate a subset of mailboxes from another company that had gone into administration. From the point of engagement we had few days to complete the design work and then from the point of start of implementation less than one month to build new servers into the existing Exchange 2010 infrastructure to support the users and then get the mailbox data migrated with a big bang migration over one weekend. This was a project with an absolute deadline that could not move.

So how did we do the Exchange migration?

The Environment

Well at the point of engagement our client had already created a number of mail enabled contacts in their Exchange organization and configured email coexistence. Additionally some of the transitioning users from one office had already been transitioned by hand.

The source environment consisted of two Exchange Server 2003 clusters which hosted the mailboxes we wished to transition, but also contained mailboxes we did not wish to transition. The mailboxes that were transitioning had the company attribute set to the name of our client. Overall there was around 160GB of mailbox data to migrate, the mailboxes also contained Enterprise Vault stubs which we needed to remove as they would no longer be valid.

The target environment was to be an Exchange 2007 CCR Cluster installed into a pre-existing Exchange 2010 infrastructure (fortunately we still had a temporary Exchange 2007 server in the environment). But due to timescales and wanting to get the data synchronization started as soon as possible we deployed a temporary Exchange 2007 mailbox server on a virtual platform to synchronize the mailbox data onto. When the physical hardware arrived it was built into an Exchange 2007 CCR Cluster.

The two environments were connected via a VPN but bandwidth was not great (around 6Mbps) so we had to take that into consideration without affecting user experience.

Migration Architecture

With this migration we were primarily concerned that there was not sufficient bandwidth between the source and target servers to complete the synchronization of data on time and also had concerns as to whether the source Exchange servers had sufficient disk space on one shared drive to allow potentially all of the transitioning data to queue.

The space issue is due to the way that QMM synchronizes data. In simple terms the agents used by QMM export mailbox data into PST files which are then normally copied to the target server. The agents are normally installed directly onto the Exchange servers.

We decided to use a feature of Quest Migration Manager for Exchange called Alternate Hosts which allowed us to provision a virtual machine in the source environment but a member of the target domain and to ensure that this machine had sufficient space to potentially queue all of the exported Email. The QMM agents would then be installed onto the alternate host rather than the source Exchange servers themselves. So the architecture we had is represented by the following diagram.

QMMArchitecture

The Migration Process

The first task with any QMM migration is to run directory synchronization. The main objective with the directory sync in this instance was to run it with the following objectives:

  1. Only synchronize migrating users… Solution = User an LDAP filter on the directory sync agent filtering by company attribute.
  2. Don’t synchronize manually migrated users… Solution = Use Directory sync exclusions.
  3. Mailbox enable migrating users… Solution = Allow Directory sync agent to mailbox enable accounts and merge the pre-existing contacts to maintain email reply-ability.

After all the users had been matched and mailboxes created the directory sync was turned off as in this instance coexistence was not required. There were a few exceptions that cropped up but then again there always is.

For the email data the original concept was that the target domain server would be deployed into the source environment with the QMM Agents. The agents would be configured such that any messages with Enterprise Vault messages classes would be skipped and so that the following would occur:

  1. Calendar data would be synchronized 24/7. As coexistence was not required this was configured one-way only.
  2. Mail data would be exported 24/7.
  3. The Agents would transfer what data they could from source to target out of hours which turned out to be around 10GB – 12GB over a weekday night.
  4. Once the bulk of the data had been exported we would copy it off to portable disks and transfer to the target environment. Note: This is a consultancy procedure that can be done but is not documented and may not be supported by Quest support and needs to be carefully planned as the order the data files arrive in the target is important. As it turned out we did something slightly different which is explained below.

The calendar synchronization completed it’s initial synchronization after about one full day and was allowed to continue with synchronization of delta information.

The bulk of the mailbox data extraction had completed for most mailboxes after about one week and although about 10-12GB was being transferred per night we now needed to move the queue of migrating data (around 80GB) from source to target. Instead of using a portable disk we took the decision to pull the alternate host back into the target domain along with other virtual machines that were being moved at the time.

Once the alternate host was back in our clients environment and provisioned a with a new IP address, the remaining 80GB of data was transferred to the target machine very quickly by the agents and the Quest agent imported the queue of data in around one day. We then left all the agents running 24/7 to keep up with deltas.

On the weekend of the transition we started the process in QMM to switch the mailboxes to the target environment and waited for that process to complete. As mailboxes were switched by QMM they were mailbox-moved from the temporary Exchange 2007 mailbox server onto the production Exchange 2007 CCR Cluster. Once all switches and mailbox moves were complete the CCR Cluster Storage Groups were seeded down to an Exchange 2007 SCR server for DR.

Migration complete Smile

About QMM Alternate hosts.

The use of alternate hosts is ideal under the following circumstances.

  1. When source or target servers, for whatever reason are unable to support the QMM agents.
  2. When for political or technical reasons the QMM agents cannot be installed on servers.
  3. If the source or target server is a CCR cluster. If the agents are deployed on a CCR cluster directly data needs to be transferred twice across the network, particularly for a target server to allow fail-over to occur and the migration to continue uninterrupted. The alternate host can get around this.

A couple of points about this article.

  1. The manual movement of the QMM Service Files while it will work is not a documented procedure and is normally done with field support guidance.
  2. The throughput figures given here may not be representative in any other environment. Data extraction and transfer rates are environment specific.
  3. The movement of the alternate host was completed with consultancy to ensure that QMM was shut down before the move and to ensure everything started again ok after the host move.
  4. Due to the urgent timescales required there were some risks carried with the project and no proof of concept was used.

Installing Blackberry Express with Exchange 2010 – Part 1

Today I am going to have a look at what is involved in installing Blackberry Express Server 5.0 with Exchange Server 2010.

What is Blackberry Express Server

Blackberry Express Server is supported to be installed directly on an Exchange 2003 SP2, 2007 SP1 or 2010 RU1 server in order to support up to 75 Blackberry Smartphones and is ideal for users who receive between 100 and 200 messages per day. Blackberry Express Server can be deployed to dedicated hardware for support up to 2,000 Smartphones. The only requirement for using Blackberry Express Server is that you have Internet enabled data plans with your Blackberry Devices.

Operating System Requirements for v5.0.1

  • Windows Server 2003 SP2 (32-bit or 64-bit)
  • Windows Server 2k3 R2 SP2 (32-bit or 64-bit)
  • Windows Server 2008 (32-bit or 64-bit)
  • Windows Server 2008 R2 (Except for Administration and Monitor components)
  • Windows Small Business Server 2003 or 2008

Hardware Requirements

User Support

       Requirements

Up to 75 Users hosted on Exchange Server or SBS

  • Standard Exchange / SBS Scaling
  • + 1.5 GB RAM

Up to 200 Users

  • Single Processor 2.0 GHz Xeon (two processors recommended)
  • 2GB RAM
  • 2 Drives (RAID 1)

Up to 500 Users

  • Two Processors, 2.0 GHz Xeon
  • 2GB RAM
  • 2 Drives (RAID 1)

Up to 1000 Users

  • Two Processors, 2.0 GHz Xeon
  • 3GB RAM
  • 2 Drives (RAID 1)

Up to 2000 Users

  • Two Processors, 2.8 GHz Xeon
  • 6GB RAM
  • 2 Drives (RAID 1) or 4 Drives RAID10

Messaging Platform Requirements

  • Microsoft Exchange 2010 with Rollup 1
  • Microsoft Exchange 2007 SP2
  • Microsoft Exchange 2003 SP2
  • Microsoft Exchange mixed environment (Exchange 2003 and 2007)

Software Requirements

Blackberry Enterprise Server Express requires one of the following software components

  • Microsoft Exchange 2003 SP2 System Manager
  • Microsoft Exchange MAPI Client and CDO 1.21 (CDO 1.21 Version 6.5.8146.0 or higher for Exchange 2010)

There are other requirements such as Java components but the Blackberry Express Server installation package can install those for you if they are not detected.

Database Requirements

Access to one of the following SQL servers

Supported SQL Servers

Database Server settings

  • MSDE 2000 SP3
  • MS SQL Server 2005 SP3 (32-bit or 64-bit)
  • MS SQL Server 2005 Express SP3
  • MS SQL Server 2008 SP1 (32-bit or 64-bit)
  • MS SQL Server 2008 Express SP1 (32-bit or 64-bit)
  • Collation set to default case-insensitive
  • Blackberry Configuration database collation set to default case-insensitive
  • Blackberry database notification system and Microsoft SQL Server collation must be the same.
  • Default collations are recommended though non-default collations are supported.
  • TCP/IP Protocols turned on
  • If using a remote database server ensure remote connections are enabled.

Configuring for Exchange 2010

Create a Service Account

  • Ideally in the domain where the Blackberry Enterprise Express server is to be installed create a domain account with a mailbox called BESAdmin.
  • Assign BESAdmin owner permissions to all Public Folders. BES requires public folders to support free/busy lookups.
  • BESAdmin must not be a member of Domain Admins.

Service Account Permissions

Run the following command to set store permissions for the BESAdmin account (this will need to be repeated if a new mailbox database is created in Exchange)

Get-MailboxDatabase | Add-ADPermission -user "BESAdmin" -accessrights extendedright -extendedrights receive-as,ms-exch-store-admin

Add the BESAdmin account to the View-one organization management role which allows the service account to see the Exchange configuration objects within Active Directory

Add-RoleGroupMember “View-only organization management” –member “BESAdmin”

The BESAdmin account also needs to be granted rights to be able to Send-as a user. The permission can be set at the domain level and allowed to inherit down.

Add-AdPermission example.com –inheritedobjecttype user –inheritancetype descendents –extendedrights send-as –user “BESAdmin”

Exchange Configuration

Client Throttling

Client throttling can affect performance of the Blackberry Enterprise Server so the throttling policy for the BESAdmin account should be modified so that it can be adjusted if required without affecting all users.

New-ThrottlingPolicy BESPolicy
Set-Mailbox “BESAdmin” –throttlingpolicy BESPolicy

Address Book Service Connections

The number of maximum number of address book service connections needs to be increased from 50 to 100000. This change affects all users

Browse to the Exchange install location. By default this is C:\Program Files\Microsoft\Exchange Server\V14 and locate the file  Microsoft.exchange.addressbook.service.exe.config under the BIN folder on each CAS server. Open this file in a text editor such as Notepad.

Locate and change MaxSessionsPerUser to 100000
Save and close the file and restart the Microsoft Exchange Address Book service.

Create Management Exchange 2010 Role Entry

Configure Management role entry for Exchange Web Services which allows the management of calendars
New-ManagementRoleAssignment –name “BES Admin EWS” –role applicationImpersonation –user “BESAdmin”

Configure Admin account encryption level (Blackberry Enterprise Express 5.0 pre SP1)

If you are running in a forest which has Windows Server 2008 Domain Controllers but the domain functional level is pre-Windows 2008 you may run into a problem where the account designated as the administrator account is unable to login to the Blackberry Administration Console.

The options are

  • Upgrade to Blackberry Enterprise Server Express 5.0 SP1 (Bundle 12)

  • Upgrade domain functional level to Windows 2008 (not reversible)

  • Configure the use of Kerberos DES encryption on Admin account account tab in Active Directory Users & Computers. This is the option I have chosen as part of the install process as I don’t have BES Express 5.0 SP1 to hand and I did not wish to update my domain to Windows 2008 just yet.

Configure SQL Server Permissions

The account used to install the Blackberry Enterprise Server Express software needs to be granted dbcreator rights on the SQL server hosting the Blackberry Configuration database (I install as the BESAdmin account as that ensures the service account has the correct connection). Additionally the BESAdmin account also needs sysadmin rights on the SQL server if the notification system is going to be used.

Firewalls and Network Requirements

The Blackberry Enterprise Server Express needs the following requirements to be met

  • Bi-directional use of port 3101 in order to maintain an outbound connection to an external server.
  • Access to a DNS server which can resolve internet addresses.
  • For proxy firewalls the proxy server needs to be transparent.

That’s all for the moment. I will follow up shortly with another post which quickly runs through the Blackberry Enterprise Server Express install process.

Exchange 2010 Remote PowerShell

It has been a while since I have posted so here goes for the first post of 2010.

PowerShell v2 has brought with it the ability to connect from remote machines as PowerShell is made available via an IIS published URL. This allows support engineers to connect to PowerShell over the internet and administer servers without having to open a VPN to the enterprise network.

I have been looking for a while around how to achieve this from a non-domain joined workstation so that I could administer client systems. Information on the internet never seemed to give the complete picture so I have documented it here. This example is connecting to Exchange Server 2010.

Step 1. Enable Remote PowerShell

Remote PowerShell first needs to be enabled on the Exchange 2010 CAS servers which are internet facing. This can be achieve by using the PowerShell command enable-psremoting.

RemotePS1  

The command describes what it is going to do and then prompts for confirmation on a couple of actions including a restart of the WinRM service.

Step 2. Enable Remote PowerShell for users

Remote PowerShell is enabled by default on user objects. You can confirm that is enabled by running the command: get-user –id <userID> ¦ select displayname,remotepowershellenabled ¦ fl

RemotePS2

The command set-user –id <userID> –remotepowershellenabled:$true will enable remote PowerShell for a user.

Step 3. Enable Authentication on the IIS Web SiteRemotePS3

We normally publish the Exchange CAS servers through Microsoft ISA Server 2006. By default the PowerShell virtual directory is enabled for Anonymous authentication only. Open IIS Manager on the Exchange 2010 CAS server, open the Authentication settings of the PowerShell virtual directory and enable Basic Authentication.

 

 

 

Step 4. Publish the PowerShell Virtual Directory on ISA ServerRemotePS4

As previously mentioned Exchange CAS servers are normally deployed behind a Microsoft ISA Server. Adding the PowerShell virtual directory to the publishing rules for Outlook Web App will expose the location to the internet. We normally have to ISA policies publishing OWA, one for Negotiated Authentication and one for Basic Authentication. The PowerShell virtual directory will need to be published via the Basic Authentication publishing rule.

That completes the configurations required to expose PowerShell to the internet. All that remains the establishing the connection to Exchange Server over the internet from PowerShell v2 installed on my workstation.

 

 

Step 5 – Set Session Options

If you are testing Remote PowerShell to a Proof of Concept or a live system that is still utilizing self-signed certificates you may need to establish some session options for the connection.

Open a PowerShell v2 Window on your client machine and enter the following (don’t close the PowerShell Window between the steps below)

$so = new-pssessionoption –skiprevocationcheck –skipcncheck -skipcacheck

Capture3

These options will allow the connection to ignore common problems when making connections while using self-signed certificates.

Step 6 – Enter CredentialsCapture2

In order to connect you need to enter some credentials in order to login to the remote system. This can be achieved by entering the command: $Cred = Get-Credential

In the dialog enter the credentials of a user who has remote PowerShell rights and also rights to administer Exchange set through the Role Based Access Control (RBAC).

 

 

 

Step 7 – Create a Remote PowerShell Session

Next we use the Session Options and the Credentials entered in the previous steps to establish the session with one of the CAS servers published through the ISA Server.

Enter the command: $PSSession = new-pssession –configurationname Microsoft.Exchange –connectionuri “https://<External FQDN>/PowerShell” –credential $Cred –authentication Basic –SessionOption $so

Where <External FQDN> is the internet address of your Exchange servers.

Capture4

Step 8 – Import the Remote PowerShell Session

The final step in the connection process is to import the session. This downloads the commands that you are allowed to use from the Exchange Servers. Only the commands that you are allowed to use based on your RBAC privileges are downloaded and made available.

To import the PowerShell session type the command : import-pssession $PSsession

Capture6

A cyan progress bar will appear at the top of the screen while the commands are downloaded.

Step 9 – Use Remote PowerShell

Once the command download is complete you can use Exchange PowerShell commands to administer the servers over the internet.

Capture1

Step 10 – Close the Remote PowerShell session

When you have completed your tasks at hand the PowerShell session should ideally be closed gracefully using the command : remove-pssession $PSSession

Capture7

That’s it for this article see you next time.

Quest QMM and Exchange Server 2007 SP2

An interesting development occured this week where we have found that Quest Migration Manager for Exchange v8.3 is not compatible with Exchange Server 2007 SP2 for customers who are running this QMM version and are tempted to upgrade their Exchange server versions.

As of the time of writting, Out of the Box QMM v8.4 is also not supported against Exchange Server 2007 SP2 however Quest have released an update for QMM v8.4 which patches the SQL database to make it compatible and supported with Exchange Server 2007 SP2.

When deploying QMM v8.4 install the Hotfix: MigrationManagerForExchange8.4Update20090828. It is not supported in deploy this hotfix against QMM v8.3.

This hotfix does not imply support with QMM for Exchange 2010 and does not synchronize any new attributes introduced as part of the Exchange 2010 schema updates which form part of Exchange 2007 SP2.

Exchange 2010 is Code Complete

It has been announced that the final code for Exchange Server 2010 has been signed off as complete so it should be with us soon as this is a big milestone towards RTM. By the looks of things Microsoft may launch Exchange Server 2010 at TechED Europe.

The announcement can be found on the Exchange Team Blog Here