Category Archives: Exchange 2007

AutoDL module updated yet again

Posted on by — 1 Comment ↓

Articles in the "AutoDL management module" series

  1. PowerShell module for managing automated distribution groups
  2. AutoDL module updated yet again [This article]

Edit 3/27/17:  Rather than add another post, this entry has been updated to reflect the addition of the Find Groups button added for group mirroring.

When automating distribution lists, eventually someone will want one that doesn’t use its own filter, but contains the same members as another group.  (Usually it is a security group when there is separation between DLs and security groups.)  The module has always supported group mirroring, but it required a specific syntax (the LDAP filter had to be prefixed with “guid:”) and that the object’s GUID be entered (without curly braces).

The module has been updated so that it is much easier to use group mirroring.  The GUI now has a separate field for entering the groups to mirror:

AutoDL Group Mirroring

A group’s filter now separates using either an LDAP filter or group mirroring.

You select the radio button for either using an LDAP filter or mirroring the membership of other groups.  You no longer manually enter the GUID of the object, but enter the distinguished name of the object and it will be converted to the objectGUID when saving the filter.  (Using the objectGUID allows membership updates to continue even if the source group is moved and its distinguished name changes.) You can also add groups to the list by clicking the Find Groups button and searching for them.  This functionality is provided by an external library that is included in the download.  Put the DLL in the same directory as the module and it will automatically be loaded.  I have added a check for the library, so if it isn’t found, the Find Groups button will simply be disabled and a label will be displayed next to it that says the dependent file could not be loaded.

When running Get-AutoDLFilter and Set-AutoDLFilter, the objectGUIDs will be converted into the current DNs for display.  The GUI output of Get-AutoDLFilter has also been updated to reflect whether it is using an LDAP filter or group mirroring and, if the latter, doesn’t display the sections for the display-formatted and raw filters.

The distinguished name of the object(s) will be validated when the focus leaves the text box (unless clicking the Cancel button), checking that it resolves to an existing group object.  If updating a group’s filter from the command line, the MirrorGroup parameter has been added to the Set-AutoDLFilter cmdlet.  The same DN validation occurs when updating from the command line, and the LDAPFilter and MirrorGroup parameters are in separate parameter sets so that they are mutually exclusive.

Any existing groups using group mirroring are compatible with this version:  Nothing has changed on the “back end,” only how the groups being mirrored get added to a filter has been updated.

Download the module and AD object picker library here:

  AutoDLManagement.zip (22.7 KiB)


If you want just the module, it is here:

  AutoDLManagement.psm1 (55.4 KiB)

Previous in series |

AutoDL management module updated

Posted on by — No Comments ↓

I have spent a fair amount of time updating the AutoDL module I first posted back in December.  At the end of that post I mentioned several things I was considering adding, and that made me want to add some of them.

  • You can now specify users that should always be excluded even though they match the filter:
    Set Filter with Exclude Option

    The ability to exclude users from membership is now an option.

    The Set-AutoDLFilter cmdlet has also been updated with the AlwaysExclude parameter.

  • LDAP syntax checking has been added and occurs when the focus leaves the filter’s text box:
    Example of bad syntax prompt

    LDAP syntax will be validated and present a warning if it does not pass.

    The OK and Preview buttons will be disabled if the syntax check fails.  I looked around for ways to simply and efficiently validate syntax, but I could not find anything besides what is provided via the LDAP interface to Active Directory.  When performing an LDAP search, a specific error is returned if the syntax is not valid, so the module passes the entered filter to AD and checks the response.  (For efficiency with filters that pass, I use the FindOne() method so the search will stop as soon as the first matching object is found.)

  • A preview window has been added:
    AutoDL MemberShip Preview

    Clicking the Preview button opens a new window to show who will be in the DL.

    The preview window will display a sorted list of member display names.  You can copy this list to the clipboard to be able to, for example, paste pending membership into an email of the person requesting the DL for confirmation.  There is also a field that provides the membership count.  If the filter results in no members, the count field will contain a hyphen and the Copy button will be disabled.

  • A parameter, VerboseMembershipChanges, has been added to Update-AutoDL if you want to include in the output (screen and log file) the individual members that were added and removed:

    Detailed Membership Changes

    You can get detailed membership changes in the screen output and log file.

  • The Verbose parameter can be used with Set-AutoDLFilter and Update-AutoDL to get more details of activity.  I added a bunch of Write-Verbose commands while I was debugging these recent changes and I left them in for those that want more information about what is happening.
  • The module has been updated to conform with the recommendations of the PSScriptAnalyzer module.
  • The module now requires PowerShell version 3 or higher and will check for that.
  • Several (well, more than several) minor fixes as a result of previous changes made that weren’t discovered until I was testing in a new lab and had to create new DLs and provision them and do other tasks that are less common once you are only maintaining automated DLs.  This is what modules like Pester are for, I suppose.

You can download the updated module with this link:

  AutoDLManagement.psm1 (55.4 KiB)


(I have started exploring the possibility of hosting a Nuget repository or publishing my modules on the PowerShell Gallery so PowerShell v5+ users can simply install them with a command like Install-Module AutoDLManagement -force.)

PowerShell module for managing automated distribution groups

Posted on by — No Comments ↓

Articles in the "AutoDL management module" series

  1. PowerShell module for managing automated distribution groups [This article]
  2. AutoDL module updated yet again

Years ago I used a third-party application for creating, managing, and updating automated DLs in Exchange.  When I moved to another company that didn’t have that application, nor would spend the money to buy it, I wrote a script as a poor-man’s version of that application.  Since then, it has been updated with more features, and I recently updated it again for a customer who wanted features I had never needed.

You might wonder why not use dynamic distribution groups?  There are a number of limitations when using DDGs that automated DGs (née DLs) overcome:

  • Use the group as a security principal
  • Allow static members (those that should be members even though they don’t match the filter)
  • Use any LDAP attribute in the schema in the search filter, not just those exposed via OPATH
  • View the group membership (when an end user)

The way the module works is it marks an existing DG as automated and updates the Note property stating that it is an automated group; you enter an LDAP filter to use for membership, optionally specifying any static members and domains to restrict membership to, and save it; and the membership criteria are saved as a binary value in the extensionData attribute.  Then you can run Update-AutoDL to update the membership of that one group or all (or all in a specified domain), or schedule a task to run that cmdlet to update membership, say, daily.  If you want to download it without reading all of the details:

  AutoDLManagement.psm1 (55.4 KiB)

Prerequisites:

  • Permission to manage the group objects in Active Directory
    The module uses ADSI to manage the groups, search for members, and to update membership.
  • A shell with Exchange cmdlets loaded
    A few of the Exchange cmdlets are used to validate that a group exists, and to enable it for automation.

Installation:

  • Put the module in an appropriate directory
    If you want the module to be implicitly loaded, or explicitly loaded without specifying a path, put the module inside a directory of the same name inside the Modules directory of your profile, e.g., C:\Users\<username>\Documents\Windows PowerShell\Modules\AutoDLManagement\AutoDLManagement.psm1.  You can put it anywhere if you want to load it manually and specify the full path to it.
  • For interactive or scheduled execution, load the Exchange cmdlets
    Whether using implicit remoting or a shell loading the Exchange snap-in locally, you’ll want to have the Exchange cmdlets loaded in the session before running any of the AutoDL cmdlets.

Features of the AutoDLManagement module:

  • Uses an LDAP filter as membership criterion
    Note: There is no syntax checking for the filter, so be sure you enter a valid one.
  • Membership criteria are stored in the group itself
    There is no external dependency except for the module to update the group membership.
  • Group can contain static members
    Include recipients that do not match the search filter.  The attribute used to match static members can be customized.  (The default is sAMAccountName.)
  • Restrict membership to specified domains
    The default configuration is to include all matching recipients in the forest, but you can specify one or more domains in the forest that membership is restricted to.
  • Global filter that applies to all groups
    This is a filter that is applied in addition to a group’s filter, such as not including terminated users.
  • Customize which custom attribute (extensionAttributeN) and what text string are used to indicate a group is automated
  • Email notification when a group update fails or membership is zero when it wasn’t previously
    If the module is unable to add members after clearing membership, resulting in no members, you are notified so you don’t end up with a group that people are emailing and no one gets the message.
  • Supports all versions of Exchange, including Exchange Online
    Note: Exchange Online support requires groups to be authoritative on-premises, syncing to Exchange Online via AAD Connect.
  • Display membership criteria in both “raw” and “pretty” formats
    When you run Get-AutoDLFilter, the result is an IE window that displays the LDAP filter, the static members, and domains for membership.  The filter is displayed as both a raw string that you can copy and paste for editing, and as a formatted filter (based on my LDAP-formatting script described here) that can be easier to read and is useful when sending a filter to a user.

  • Mirror membership from other groups
    Sometimes a group owner wants the membership to mirror a security group because it isn’t mail-enabled.  You can mirror membership of one or more groups by setting the LDAP filter to be “guid:<GUID of group>”.  Separate multiple GUIDs with a semicolon.
  • Logging of each group that is updated, including old and new count
  • Ability to update the membership of one group, all groups, or groups in a specified domain
    When running Update-AutoDL, you can provide the identity of a group, use the UpdateDomain parameter to specify a single domain’s groups to update, or no arguments to update all groups in the forest.  (If you update all groups interactively, it will ask for confirmation.  If updated via a scheduled task, the confirmation is suppressed.)
  • Set/Update a group’s criteria via command line or GUI
    Run Set-AutoDLFilter (alias sadl) with the identity of a group.  If the group is not already managed/automated, it will ask if you want to enable it:
    After entering ‘y’, the GUI will display so you can enter membership criteria:
    The form performs basic checks in order to enable the OK button: if static members is checked, then the text box has to be populated; if domain restrictions is checked, then at least one domain must be checked.When you run the cmdlet with a group that already has membership criteria, the form will be populated with the appropriate values, so you only have to modify what is to be changed about the criteria.For command-line updating, use the appropriate parameters for the filter, static members, and domain restrictions.  (The parameters are documented in the cmdlet’s help.)  If you update from the command-line, you need to include all values even if it isn’t changing.  For example, if you are modifying the LDAP filter for a group that has static members, you need to specify those members even if they aren’t changing.  Otherwise, the static members (or domain restrictions) will be lost.
  • Pipeline support
    For bulk-enabling and -changing groups, you can pipe groups to the Set-AutoDLFilter cmdlet.  This allows you to bulk-enable groups for automation and set their membership criteria if you already have that information in, say, a CSV.  You can use a switch parameter with the cmdlet to suppress the confirmation prompt to enable automation for a group.

Functionality that I am considering adding in the future:

  • Static exclusions (Now Added)
    If there are recipients you don’t want to be in the group even though they match the criteria, you can specify them.  (You can do this now by adding the username to the filter with a NOT operator, e.g., (!samaccountname=johndoe).)
  • Add object picker for selecting groups for mirrored membership
    Currently, you need to manually enter the source group’s GUID in the LDAP filter field (preceded by guid:).  The object picker would let you browse the forest and select a group to have its GUID automatically entered.
  • Perform syntax checking of an LDAP filter (Now Added)
  • Preview the pending membership of a group (Now Added)
    In the criteria editor, you could click a button to preview the membership based on the criteria currently specified (and without having to save it).
  • Add/remove only changed members (Module already does this)
    Currently, group membership is cleared and then updated with all objects returned in the search.  This has an element of risk because if a failure occurs between clearing and adding membership, the group is left with no members.  This feature would compare the current and pending memberships and only add/remove the necessary objects.

If these or other features would be of interest to you, let me know in the comments.  Download the module:

  AutoDLManagement.psm1 (55.4 KiB)

| Next in series

Search mailboxes for large items that may impede migrations to Exchange Online

Posted on by — No Comments ↓

I have a customer that will be enabling hybrid mode soon and moving mailboxes to Exchange Online. One part of the project entails finding mailboxes that will not have a successful migration because they contain items over 150 MB. I referred them to the script on the TechNet Gallery that does exactly that. When that script was run against an admin’s mailbox, it took 10 minutes to complete. Extrapolating that single mailbox’s time for all 80,000 mailboxes (555 days) is far from accurate, but it does indicate that the process would likely take far longer than they have available to complete that part. (The extrapolation also doesn’t factor running multiple threads of the script.) So I looked at the script to see how it is doing what it does. It enumerates every folder, searches every folder for every item in it, then looks at the size of each item so it can report how many total items there are in each folder and how many are over the size limit.

So they could get results in far shorter time, I wrote a script that uses the EWS Managed API and leverages the hidden AllItems search folder created by Outlook 2010+ when it connects to a mailbox. Since it isn’t a well-known folder name, you have to find the folder first. The following code searches the root of the mailbox for a search folder (folder property type is 2) whose display name is AllItems:

What if the folder doesn’t exist? If Outlook 2010+ for Windows hasn’t been used against the mailbox, the folder won’t exist. If this is the case, the folder needs to be created. To determine the search restriction used for the folder when created by Outlook, I used MFCMAPI and saw that there is only one: the item has the message class property populated. To create the same search folder with EWS:

When you create a search folder you need to specify the folders to search and whether subfolders are included, the name of the folder, the search parameters (restrictions), and where to put it. In this case, the folder to search is the well-known folder MsgFolderRoot, which is the visible root folder in the IPM subtree, and subfolders are included by specifying a deep traversal. (This means the recoverable items folders are not included. If you want to include them, you can add to RootFolderIds with the well-known folder for Recoverable Items.) The search parameter is that the ItemClass property exists. (This translates to PR_MESSAGE_CLASS when viewed with MFCMAPI.) The folder is then saved in the root of the mailbox. The folder search can be run again to get the newly created folder.

To get a count of any items that are over 150 MB, do a search against that search folder using a query string. This type of search leverages the content index and is faster than using a search filter with a restriction. This search returns the count of any items over 150 MB, where $maxSize is an integer representing the limit in MB:

Putting this all together, the script takes an email address (or mailboxes or email addresses from the pipeline), looks for the search folder, creates it if missing and looks for it again, searches for any items in the search folder over a given size, and outputs an object with the email address, the number of items found, and any errors. Running it in the customer’s environment went from 10 minutes per mailbox to 14 mailboxes per minute. You can pipeline the output to CSV to use a source with the large item script from TechNet to get more details of which folder has the oversize items, etc.

For performance, the autodiscover URL of the first mailbox in the pipeline will be cached and used for subsequent mailboxes. Or you can specify a URL to use instead. The default item limit is 150 MB, but you can specify any size you want. There is a switch to use impersonation; otherwise, full mailbox access is needed. I found that you don’t need any permission to a mailbox in order to bind to the root folder, so if you then do a search for the search folder, you get the same result when there isn’t a folder or when you don’t have permission. Therefore, the script checks the account’s permission to the root folder (which is contained in the bind response). Depending on what you want to do with the output, such as feed it to the TechNet script, you can choose to not include mailboxes with 0 large items in the output with the appropriate switch. Lastly, since creating a search folder and waiting for it to initially be populated can take a little time, when a mailbox needs the search folder created and you want to know that it is doing so, use the Verbose switch to see that in the console.

You can download the script via the link below or expand and copy code:

  Get-MailboxLargeItemCount.ps1 (8.3 KiB)

Delete all empty subfolders of Deleted Items

Posted on by — 2 Comments ↓

At my company we use a retention policy to delete items in the Deleted Items folder that are older than 14 days. (This is implemented via Managed Folders, a.k.a. MRM 1.0.) Managed Folders, and even Retention Policies (MRM 2.0), act on items, not folders. A common scenario, though, is that users will soft-delete a folder as an easy way to move all of its items to the Deleted Items folder. After 14 days, the managed folder policy will delete the items in that folder, but it leaves the folder itself. Over time, users may have dozens of empty folders in the Deleted Items folder, and the only way they will ever be deleted is if users delete them one by one. It goes without saying (though I am obviously saying it anyway) that users don’t do this, so lots and lots of empty folders build up over time.

I don’t like this, so I wrote a script that deletes the empty subfolders of Deleted Items for all the mailboxes in the organization. (I actually wrote two scripts, one for the Deleted Items, and one for any specified parent folder.) The script uses the EWS Managed API. You can have any version installed, and it will check for one, starting with the most recent version. The intent of the script is to process all mailboxes in the org, so it retrieves all mailboxes and pipes them into a ForEach-Object loop:

You can always change this line to restrict the returned mailboxes, or use the other script in the download, which supports pipelined input. Note the use of the parentheses when getting the collection of mailboxes. This is to avoid the pipeline constraint when using implicit remoting. The first function called is to get the folder ID of the Deleted Items folder for a given mailbox:

The FolderScope parameter is set to the Deleted Items folder, which is piped to Select-Object to limit the result to the top-level folder, ignoring any subfolders that will also be returned. The FolderId property is the function’s output.

The next step is to convert the folder ID from the management shell to one that is usable by Exchange Web Services:

Any plus (+) signs are transposed to hex since they get in the way. A typical connection to EWS is then established. The AlternateId class is used to specify an ID and what format it is in. Folder IDs in the management shell’s cmdlets are in the OwaId format. To convert them to the EwsId format needed by EWS when referencing an item, the ConvertId() method of the ExchangeService class (the base class of the EWS connection) is used. The UniqueId property is the function’s output.

The last function called is to actually delete any subfolders of the folder with the ID the previous function returned:

You can see this function redundantly establishes a new EWS connection. This is because I was adapting functions used elsewhere for this script and I didn’t optimize it. The function may take less time to complete by using a session that has already been established, but I didn’t determine that or change the previous function to create the session variable in the parent scope so it could be reused. After the session initialization, the function connects to the mailbox and binds to the Deleted Items folder using the folder ID from the previous function.

In order to get the list of empty subfolders, a search is performed for folders that contain no items. This is done by creating a view that will hold the results and defining a search filter whose only restriction is that it contains no items. The search is executed and then looped through each returned folder. Because the search is recursive, the loop will ignore any folder whose parent folder is not Deleted Items. The folder is then deleted.

Write-Output is used, not only to save puppies, but so that you can see what is happening while still allowing you to save the output to file. This can be accomplished with Tee-Object, e.g.:

One caveat of the script as written is that a subfolder that also contains subfolders, which may contain items, will still be deleted. Since I am deleting subfolders of Deleted Items, whose items are deleted after 14 days anyway, I am not too concerned about inadvertently deleting items. If this will be a problem for you, you will need to modify the script accordingly. (I may do so in the future.) Another workaround if this is a problem, is to use the second script (explained next) which has a parameter for skipping folders that have subfolders, regardless of their content.

While writing the script, I thought it might be beneficial to be able to run this process against any parent folder. So I converted the Deleted Items script into one that does just that. You can run this script with parameters that will do the exact same thing as the first script, but it also lets you specify any of the other folder scopes or a custom folder path. You can also indicate that you don’t want subfolders that have subfolders to be included (to avoid the potential issue using the first script). Lastly, this script requires that you provide a mailbox to act on. This can be manually specified with the Identity parameter, or you can pipe any number of mailboxes into it.

The FolderScope and FolderPath parameters are mutually exclusive. Use FolderScope when you want to easily specify one of the inbuilt scopes available to the Get-MailboxFolderStatistics cmdlet. This could be, for example, Inbox, RssSubscriptions, or SentItems. Use FolderPath when you want to specify a literal path to the parent folder, such as /Inbox/Subfolder1. The path to the folder is validated to ensure it starts with a forward slash and does not end with one, which matches the output of the Get-MailboxFolderStatistics cmdlet.

The functions in this script are broken down to be inline (not using functions) in order to support pipelining. The other change is supporting the exclusion of subfolders that have subfolders. This is done by modifying the search to use an additional restriction:

If the parameter to exclude subfolders is used, a search filter collection has to be used because of the need for multiple criteria. You define the search filters the same (using a unique name for each variable), but then you create a search filter collection object, add the search filters to the collection, and then use that object as the search filter in the search execution.

Both scripts are included in the download file, or you can copy the code below.

  Remove-EmptySubfolders.zip (3.4 KiB)

Remove-EmptyDeletedItemsSubfolder

Remove-EmptySubfolder

How to clear the mail attribute using PowerShell

Posted on by — 4 Comments ↓

I have been struggling to delete the value in the mail attribute after a mailbox has been deleted. Exchange populates the mail attribute when a mailbox is created (even though Exchange has no use for the attribute), but doesn’t clear the attribute when the mailbox is deleted. With ADUC integration removed in Exchange 2007, a quick way to know if an account has a mailbox is to look at the mail attribute. But if removing a mailbox no longer clears that attribute, it is difficult know (just by looking at a user account in ADUC) if the account still has a mailbox.

Since Exchange doesn’t use the mail attribute, you can’t use the Set-Mailbox attribute, especially if the mailbox is deleted anyway. I tried using Set-User with the -WindowsEmailAddress parameter, but because the data type is Microsoft.Exchange.Data.SmtpAddress, setting the value to "" or $null doesn’t work because those aren’t properly formatted SMTP addresses.

So, I figured I needed to get away from any Exchange cmdlet. I used PowerShell’s native support for ADSI to bind to the user object: New-Object DirectoryServices.DirectoryEntry "LDAP://UserDN". But you will get an error if you try to set the attribute to null ($user.mail = $null). You can set it to an empty value (""), but you will then get an error when you try to commit the change: $user.SetInfo().

How can you possibly clear this attribute, one that is so easy to do in ADUC just by deleting the value in it? It is necessary to fall back to the PutEx method. Using that will let you use the ADS_PROPERTY_CLEAR constant (indicated by the numeric one in the first argument). It has taken me days to finally get to this point, so hopefully this post will shorten that time for others trying to do the same thing.