AutoDL management module updated

I have spent a fair amount of time updating the AutoDL module I first posted back in December.  At the end of that post I mentioned several things I was considering adding, and that made me want to add some of them.

  • You can now specify users that should always be excluded even though they match the filter:
    Set Filter with Exclude Option

    The ability to exclude users from membership is now an option.

    The Set-AutoDLFilter cmdlet has also been updated with the AlwaysExclude parameter.

  • LDAP syntax checking has been added and occurs when the focus leaves the filter’s text box:
    Example of bad syntax prompt

    LDAP syntax will be validated and present a warning if it does not pass.

    The OK and Preview buttons will be disabled if the syntax check fails.  I looked around for ways to simply and efficiently validate syntax, but I could not find anything besides what is provided via the LDAP interface to Active Directory.  When performing an LDAP search, a specific error is returned if the syntax is not valid, so the module passes the entered filter to AD and checks the response.  (For efficiency with filters that pass, I use the FindOne() method so the search will stop as soon as the first matching object is found.)

  • A preview window has been added:
    AutoDL MemberShip Preview

    Clicking the Preview button opens a new window to show who will be in the DL.

    The preview window will display a sorted list of member display names.  You can copy this list to the clipboard to be able to, for example, paste pending membership into an email of the person requesting the DL for confirmation.  There is also a field that provides the membership count.  If the filter results in no members, the count field will contain a hyphen and the Copy button will be disabled.

  • A parameter, VerboseMembershipChanges, has been added to Update-AutoDL if you want to include in the output (screen and log file) the individual members that were added and removed:

    Detailed Membership Changes

    You can get detailed membership changes in the screen output and log file.

  • The Verbose parameter can be used with Set-AutoDLFilter and Update-AutoDL to get more details of activity.  I added a bunch of Write-Verbose commands while I was debugging these recent changes and I left them in for those that want more information about what is happening.
  • The module has been updated to conform with the recommendations of the PSScriptAnalyzer module.
  • The module now requires PowerShell version 3 or higher and will check for that.
  • Several (well, more than several) minor fixes as a result of previous changes made that weren’t discovered until I was testing in a new lab and had to create new DLs and provision them and do other tasks that are less common once you are only maintaining automated DLs.  This is what modules like Pester are for, I suppose.

You can download the updated module with this link:

  AutoDLManagement.psm1 (55.4 KiB)


(I have started exploring the possibility of hosting a Nuget repository or publishing my modules on the PowerShell Gallery so PowerShell v5+ users can simply install them with a command like Install-Module AutoDLManagement -force.)

PowerShell module for managing automated distribution groups

Articles in the "AutoDL management module" series

  1. PowerShell module for managing automated distribution groups [This article]
  2. AutoDL module updated yet again

Years ago I used a third-party application for creating, managing, and updating automated DLs in Exchange.  When I moved to another company that didn’t have that application, nor would spend the money to buy it, I wrote a script as a poor-man’s version of that application.  Since then, it has been updated with more features, and I recently updated it again for a customer who wanted features I had never needed.

You might wonder why not use dynamic distribution groups?  There are a number of limitations when using DDGs that automated DGs (née DLs) overcome:

  • Use the group as a security principal
  • Allow static members (those that should be members even though they don’t match the filter)
  • Use any LDAP attribute in the schema in the search filter, not just those exposed via OPATH
  • View the group membership (when an end user)

The way the module works is it marks an existing DG as automated and updates the Note property stating that it is an automated group; you enter an LDAP filter to use for membership, optionally specifying any static members and domains to restrict membership to, and save it; and the membership criteria are saved as a binary value in the extensionData attribute.  Then you can run Update-AutoDL to update the membership of that one group or all (or all in a specified domain), or schedule a task to run that cmdlet to update membership, say, daily.  If you want to download it without reading all of the details:

  AutoDLManagement.psm1 (55.4 KiB)

Prerequisites:

  • Permission to manage the group objects in Active Directory
    The module uses ADSI to manage the groups, search for members, and to update membership.
  • A shell with Exchange cmdlets loaded
    A few of the Exchange cmdlets are used to validate that a group exists, and to enable it for automation.

Installation:

  • Put the module in an appropriate directory
    If you want the module to be implicitly loaded, or explicitly loaded without specifying a path, put the module inside a directory of the same name inside the Modules directory of your profile, e.g., C:\Users\<username>\Documents\Windows PowerShell\Modules\AutoDLManagement\AutoDLManagement.psm1.  You can put it anywhere if you want to load it manually and specify the full path to it.
  • For interactive or scheduled execution, load the Exchange cmdlets
    Whether using implicit remoting or a shell loading the Exchange snap-in locally, you’ll want to have the Exchange cmdlets loaded in the session before running any of the AutoDL cmdlets.

Features of the AutoDLManagement module:

  • Uses an LDAP filter as membership criterion
    Note: There is no syntax checking for the filter, so be sure you enter a valid one.
  • Membership criteria are stored in the group itself
    There is no external dependency except for the module to update the group membership.
  • Group can contain static members
    Include recipients that do not match the search filter.  The attribute used to match static members can be customized.  (The default is sAMAccountName.)
  • Restrict membership to specified domains
    The default configuration is to include all matching recipients in the forest, but you can specify one or more domains in the forest that membership is restricted to.
  • Global filter that applies to all groups
    This is a filter that is applied in addition to a group’s filter, such as not including terminated users.
  • Customize which custom attribute (extensionAttributeN) and what text string are used to indicate a group is automated
  • Email notification when a group update fails or membership is zero when it wasn’t previously
    If the module is unable to add members after clearing membership, resulting in no members, you are notified so you don’t end up with a group that people are emailing and no one gets the message.
  • Supports all versions of Exchange, including Exchange Online
    Note: Exchange Online support requires groups to be authoritative on-premises, syncing to Exchange Online via AAD Connect.
  • Display membership criteria in both “raw” and “pretty” formats
    When you run Get-AutoDLFilter, the result is an IE window that displays the LDAP filter, the static members, and domains for membership.  The filter is displayed as both a raw string that you can copy and paste for editing, and as a formatted filter (based on my LDAP-formatting script described here) that can be easier to read and is useful when sending a filter to a user.

  • Mirror membership from other groups
    Sometimes a group owner wants the membership to mirror a security group because it isn’t mail-enabled.  You can mirror membership of one or more groups by setting the LDAP filter to be “guid:<GUID of group>”.  Separate multiple GUIDs with a semicolon.
  • Logging of each group that is updated, including old and new count
  • Ability to update the membership of one group, all groups, or groups in a specified domain
    When running Update-AutoDL, you can provide the identity of a group, use the UpdateDomain parameter to specify a single domain’s groups to update, or no arguments to update all groups in the forest.  (If you update all groups interactively, it will ask for confirmation.  If updated via a scheduled task, the confirmation is suppressed.)
  • Set/Update a group’s criteria via command line or GUI
    Run Set-AutoDLFilter (alias sadl) with the identity of a group.  If the group is not already managed/automated, it will ask if you want to enable it:
    After entering ‘y’, the GUI will display so you can enter membership criteria:
    The form performs basic checks in order to enable the OK button: if static members is checked, then the text box has to be populated; if domain restrictions is checked, then at least one domain must be checked.When you run the cmdlet with a group that already has membership criteria, the form will be populated with the appropriate values, so you only have to modify what is to be changed about the criteria.For command-line updating, use the appropriate parameters for the filter, static members, and domain restrictions.  (The parameters are documented in the cmdlet’s help.)  If you update from the command-line, you need to include all values even if it isn’t changing.  For example, if you are modifying the LDAP filter for a group that has static members, you need to specify those members even if they aren’t changing.  Otherwise, the static members (or domain restrictions) will be lost.
  • Pipeline support
    For bulk-enabling and -changing groups, you can pipe groups to the Set-AutoDLFilter cmdlet.  This allows you to bulk-enable groups for automation and set their membership criteria if you already have that information in, say, a CSV.  You can use a switch parameter with the cmdlet to suppress the confirmation prompt to enable automation for a group.

Functionality that I am considering adding in the future:

  • Static exclusions (Now Added)
    If there are recipients you don’t want to be in the group even though they match the criteria, you can specify them.  (You can do this now by adding the username to the filter with a NOT operator, e.g., (!samaccountname=johndoe).)
  • Add object picker for selecting groups for mirrored membership
    Currently, you need to manually enter the source group’s GUID in the LDAP filter field (preceded by guid:).  The object picker would let you browse the forest and select a group to have its GUID automatically entered.
  • Perform syntax checking of an LDAP filter (Now Added)
  • Preview the pending membership of a group (Now Added)
    In the criteria editor, you could click a button to preview the membership based on the criteria currently specified (and without having to save it).
  • Add/remove only changed members (Module already does this)
    Currently, group membership is cleared and then updated with all objects returned in the search.  This has an element of risk because if a failure occurs between clearing and adding membership, the group is left with no members.  This feature would compare the current and pending memberships and only add/remove the necessary objects.

If these or other features would be of interest to you, let me know in the comments.  Download the module:

  AutoDLManagement.psm1 (55.4 KiB)

Delegate management module updated to 1.4.6

A small update has been made so that comparing delegates to users with Full Access and Send As works for Exchange Online accounts. The code to translate a user ID to a SID returns an error when used against Exchange Online. If the connection mode is for Exchange Online, it will now use SMTP address to match a delegate to a user with Full Access and Send As permissions. Download the updated module here:

  DelegateManagement.zip (9.2 KiB)

Default folder retention tag script updated to 1.3

I spent some time figuring out why calendar items in the Deleted Items folder that would be immediately expired could not be updated with a tag. I found that Exchange wants to send an update to the organizer, regardless whether the calendar item is even a meeting or one that is in the past. If the item is in a different folder when the tag is applied, the error is that there is no recipient on the message, whereas the error when in the Deleted Items is that it can’t update an item that is already deleted. The latter error being misleading is why I originally couldn’t figure it out what they meant or why it was happening.

The solution is to use a second argument in the Update method to tell Exchange not to send an update message for invitations or cancellations. You have to do this even if it isn’t a meeting. Presumably, this is a bug where Exchange is running through logic against all IPM.Appointment-class items, rather than skipping those that aren’t meetings.

The script has been updated to account for this, plus some minor updates since the last version posted.

  Set-DefaultFolderItemsTag.ps1 (9.4 KiB)

Search mailboxes for large items that may impede migrations to Exchange Online

I have a customer that will be enabling hybrid mode soon and moving mailboxes to Exchange Online. One part of the project entails finding mailboxes that will not have a successful migration because they contain items over 150 MB. I referred them to the script on the TechNet Gallery that does exactly that. When that script was run against an admin’s mailbox, it took 10 minutes to complete. Extrapolating that single mailbox’s time for all 80,000 mailboxes (555 days) is far from accurate, but it does indicate that the process would likely take far longer than they have available to complete that part. (The extrapolation also doesn’t factor running multiple threads of the script.) So I looked at the script to see how it is doing what it does. It enumerates every folder, searches every folder for every item in it, then looks at the size of each item so it can report how many total items there are in each folder and how many are over the size limit.

So they could get results in far shorter time, I wrote a script that uses the EWS Managed API and leverages the hidden AllItems search folder created by Outlook 2010+ when it connects to a mailbox. Since it isn’t a well-known folder name, you have to find the folder first. The following code searches the root of the mailbox for a search folder (folder property type is 2) whose display name is AllItems:

What if the folder doesn’t exist? If Outlook 2010+ for Windows hasn’t been used against the mailbox, the folder won’t exist. If this is the case, the folder needs to be created. To determine the search restriction used for the folder when created by Outlook, I used MFCMAPI and saw that there is only one: the item has the message class property populated. To create the same search folder with EWS:

When you create a search folder you need to specify the folders to search and whether subfolders are included, the name of the folder, the search parameters (restrictions), and where to put it. In this case, the folder to search is the well-known folder MsgFolderRoot, which is the visible root folder in the IPM subtree, and subfolders are included by specifying a deep traversal. (This means the recoverable items folders are not included. If you want to include them, you can add to RootFolderIds with the well-known folder for Recoverable Items.) The search parameter is that the ItemClass property exists. (This translates to PR_MESSAGE_CLASS when viewed with MFCMAPI.) The folder is then saved in the root of the mailbox. The folder search can be run again to get the newly created folder.

To get a count of any items that are over 150 MB, do a search against that search folder using a query string. This type of search leverages the content index and is faster than using a search filter with a restriction. This search returns the count of any items over 150 MB, where $maxSize is an integer representing the limit in MB:

Putting this all together, the script takes an email address (or mailboxes or email addresses from the pipeline), looks for the search folder, creates it if missing and looks for it again, searches for any items in the search folder over a given size, and outputs an object with the email address, the number of items found, and any errors. Running it in the customer’s environment went from 10 minutes per mailbox to 14 mailboxes per minute. You can pipeline the output to CSV to use a source with the large item script from TechNet to get more details of which folder has the oversize items, etc.

For performance, the autodiscover URL of the first mailbox in the pipeline will be cached and used for subsequent mailboxes. Or you can specify a URL to use instead. The default item limit is 150 MB, but you can specify any size you want. There is a switch to use impersonation; otherwise, full mailbox access is needed. I found that you don’t need any permission to a mailbox in order to bind to the root folder, so if you then do a search for the search folder, you get the same result when there isn’t a folder or when you don’t have permission. Therefore, the script checks the account’s permission to the root folder (which is contained in the bind response). Depending on what you want to do with the output, such as feed it to the TechNet script, you can choose to not include mailboxes with 0 large items in the output with the appropriate switch. Lastly, since creating a search folder and waiting for it to initially be populated can take a little time, when a mailbox needs the search folder created and you want to know that it is doing so, use the Verbose switch to see that in the console.

You can download the script via the link below or expand and copy code:

  Get-MailboxLargeItemCount.ps1 (8.3 KiB)

Remove text messaging settings on behalf of users

1/24/17 Update:  The script has been updated to v1.3, fixing support for mailboxes on Exchange 2010.  There are also some minor fixes, such as correctly using autodiscover when neither an EWS URL nor the Exchange Online switch is used.

4/18/16 Edit: I was reviewing this script for an unrelated reason when I discovered that I had used incorrect construction in the begin block since you cannot access parameter values in it.  I have updated the script in the download and the inline code at the end of the post that any any code that references parameters has been moved to the process block.

Exchange has the ability to send text messages to specific carriers in a few countries, and is enabled by default. This allows users to configure calendar notifications (such as changes to meetings that are occurring in the next three days) and rules to forward email as a text message. Users have to use OWA (or if you prefer the new name, Outlook on the web) to configure this. But what if your users do this before you realize it is enabled by default and now you want to disable it?

If you modify the role assignment policy to remove MyTextMessaging or modify OWA Mailbox policy to remove Text Messaging, it hides this feature from users, but it doesn’t disable anything already in place. You then decide to use PowerShell to run Clear-TextMessagingAccount for someone, but it says the user cannot be read. You can run it for your own account, but nobody else, even as an admin. This is because the write scope of the role that contains the cmdlet is Self. So how to remove the settings for another user?

I wrote a script that uses the EWS Managed API modify the hidden messages that contain the settings and delete any inbox rules that are forwarding to a mobile device. I should point out that doing it this way is unsupported, but I have used it successfully for mailboxes on Exchange 2013 and in Exchange Online.

The calendar notification settings and text messaging configuration are stored in folder associated items (FAI) in the root folder of the mailbox, in the roaming XML property of a user configuration message. Because of this, you can use the Microsoft.Exchange.WebServices.Data.UserConfiguration class to easily get messages with a specific subclass and retrieve this property without having to define a property set with the extended MAPI property. The subclass for the calendar notification settings is CalendarNotification.001 and text messaging configuration is TextMessaging.001. If you already have a service object created, you can get the message for calendar notification with these two lines:

The roaming properties of a user configuration message are stored in the Dictionary, XmlData, and BinaryData properties of the search result object. The property for the calendar notification settings (PR_ROAMING_XMLSTREAM as the XmlData property) is a binary value returned as a byte array, so it needs to be converted to a string cast as an XML object so it can be manipulated with XML methods:

The three notification types have their own node and contains an element whose value indicates whether it is enabled. Since I don’t care what the other options are, only that they are disabled, this can be done by directly setting the value for the element:

To write the data back to the XmlData property and save it in the mailbox, it needs to be converted back to a byte array. This isn’t done with a one-liner like converting from a byte array. The XML data is converted to a string, which is then converted to a byte array. There could be a more efficient way of doing this, but I don’t know it at the time of this writing. The first line is the one-liner to take the XML data and store it as a byte array in the property, the second saves the message back to the mailbox, and the two functions that convert XML to a string and a string to a byte array follow:

For the text messaging configuration, it is in the same property of its message. Once converted to XML, devices are stored in the MachineToPersonMessagingPolicies node, with a PossibleRecipient node for each device that has ever been configured. To simply delete any devices, you can remove all sub-nodes since there aren’t any others:

Then convert the XML data back to a byte array and save the message the same as before.

What remains are any inbox rules that may have been created that forward to a text messaging device. As an admin, you can use PowerShell to get rules, but you won’t see any rules that have been disabled in Outlook. Even if a rule is visible because it is enabled or has been disabled via OWA, and so you are able to see if a given rule is forwarding to a text messaging device, if you delete the rule, you will also delete any rules that are currently disabled via Outlook. What’s worse, you won’t even know if there are disabled rules that will be deleted because the warning is presented for every mailbox regardless of the existence of any applicable rules.

So the script will get all FAI messages that are rules and delete any that are forwarding to a device configured via the text messaging feature. The first step is to get the rules by searching for all FAIs in the inbox whose class is that of a rule:

After getting the rules, we need to retrieve the property that contains a rule’s actions, which is PR_EXTENDED_RULE_ACTIONS (0x0E990102) when on Exchange 2013+ or 0x65EF0102 when on Exchange 2010, a binary property:

Parsing the binary data is not easy (for me) because it includes pieces of variable-length information. If the entire value is converted to a string, however, an action that forwards to a configured text messaging device contains the string MOBILE: followed by the E.164-formatted phone number. So, all that needs to be done is to get the rule’s actions, convert it to string, check for MOBILE, and delete the rule:

The script supports on-premises and Exchange Online, autodiscover or specified URL, pipelining mailboxes into it, impersonation and specifying credentials. The output will contain what actions it took on a mailbox, including whether any of the features were not configured in the first place. You can run it multiple times against a mailbox without it having an issue that any or all features are not configured. The full script can be expanded below, and it can also be downloaded via the following link:

  Remove-TextMessagingConfiguration.zip (3.1 KiB)