Issue: new BHOLD Access Management Connector doesn’t import all objects

Hi all,

Earlier this week, I had an issue with the new BHOLD Access Management connector (AMC), which imports users, groups and org.units from Microsoft BHOLD Suite (SP1). It appeared that not all groups that were present in BHOLD Core, were imported in FIM (even during a full import)!

To fully understand the problem (and solution), here’s some details about how the connector fetches the objects from BHOLD. The connector is built using the ECMA 2.0 API for building custom management agents. As most of you will probably know, this API enables you to specify a batch size. The idea is that the connector will retrieve the amount of records as specified in the batch size configuration value. After that, FIM Sync imports the records in the connector space. After importing, it calls the import-function again, until it returns that all records have been imported.

To get information from BHOLD, the AMC uses these stored procedures:

[GetFullImportKeysAndInitializeWatermark]
This stored procedure is executed during the initialization. Basically, it returns all the object types and the maximum id value of that object type.

[GetGroupBaseAttributes], [GetOrganizationalUnitBaseAttributes], [GetUserBaseAttributes]
This one retrieves the groups (called ‘permissions’ in BHOLD Core), org. units and users. It takes the following arguments:
@BatchSize; The size of each ‘page’ (as set in the FIM Sync run profile)
@ObjectKey; Only objects with an id higher than this value, will be imported. If the procedure is called the first time, the value ‘0’ is supplied.
@NextObjectKey; This is an ‘out’ parameter, that (unlike what the name suggests) contains the id of the last object that is part of the result.

So, the following workflow is followed:

  1. Execute ‘GetFullImportKeysAndInitializeWatermark’
  2. Execute ‘Get<object>BaseAttributes’, replacing ‘<object>’ with the retrieved object in step 1.
  3. Reformat the retrieved values to a format that FIM Sync understands.
  4. See if the value of @NextObjectKey (the out-parameter of the ‘Get<object>BaseAttributes’ procedure) is equal to the max id value as retrieved in step 1. If it is not, return ‘true’ to FIM Sync, indicating that there are more objects to be imported. If it is: return ‘false’, indicating that everything is imported.

According to how the AMC calls the stored procedures, this all looks implemented very well. Except: I’ve noticed that the AMC doesn’t retrieve all the groups. Furthermore: the number of imported objects is dependent on the batch size value.

Here’s the catch: to make all the above work, it is CRITICAL that the ‘Get<object>BaseAttributes’ procedure contains an ‘ORDER BY ‘ObjectIdentifier’ (id) clause! If not, it will miss objects.

Let’s say this is the content of your table that contains the groups:

PermissionId;PermissionName;ApplicationID
4;PermissionA;0
5;PermissionB;0
6;PermissionC;0
3;PermissionD;0
8;PermissionE;0

And, for this example, we use a batch size of ‘3’.

  1. ‘GetFullImportKeysAndInitializeWatermark’ will be executed. The max id of the groups object type returned is ‘8’.
  2. Next, ‘GetGroupBaseAttributes’ is executed with parameters:
    @BatchSize: 3
    @ObjectKey: 0
  3. The result, without the ‘ORDER BY’ clause, will be:
    PermissionId;PermissionName;ApplicationID
    4;PermissionA;0
    5;PermissionB;0
    6;PermissionC;0
    The output parameter will contain: ‘6’, because this is the max id of the returned objects.
  4. As the value ‘6’ is not equal to ‘8’ (step 1), FIM Sync gets the message that not all records have been retrieved yet.
  5. ‘GetGroupBaseAttributes’ is executed again, but now with the following attributes:
    @BatchSize: 3
    @ObjectKey: 6
  6. The result will be:
    8;PermissionE;0
    The output parameter will contain ‘8’.
  7. This is equal to the max id, so import will stop.

As you will notice: PermissionD, with ID ‘3’, has been missed!!

The solution

Search for the following line in the ‘GetGroupBaseAttributes’ stored procedure…
WHERE [p].[PermissionID] > @ObjectKey
…and add the following to it:
    ORDER BY [p].PermissionId

Finally, I would like to say thanks to my colleague Kenny. We worked on this issue together, having a hard time finding out what the problem was. In the end, he was the one that noticed the ORDER BY clause. Thanks!

As always, if you have any questions or remarks, let me know!!

Posted in Uncategorized | Tagged | 3 Comments

Issue when using the model generator when not running a local database

The model generator is a useful tool to import your existing data in BHOLD Core. You can use csv or excel-files in a certain format.

I’ve been using the model generator to import data in the Five-file set (as described in the link above). This is the environment I’m using:
Image

As you can see, the BHOLD Suite modules are not installed on the same server as the server where SQL Server is installed. This shouldn’t be a problem. When installing the Model Generator, everything worked fine, but once running it, I came across this error:
Image

 

This shows up in the event log:

System.Data.SqlClient.SqlException: A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 – Could not open a connection to SQL Server)

What happens here is that the model generator tried to export the contents of the text-files to the BHOLD database. According to the error, the SQL Server is unavailable. It turns out that the Model Generator is trying to access the database on the local server. This is caused by an error in the installation file.

To fix this, open the registry editor (‘regedit’) and navigate to: ‘HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\bhold\ModelGenerator’. The first thing you’ll notice is that you are not permitted to access this key.
Image
To solve this, open the regedit mmc using the service account you used during the model generator setup. You can do the modifications from there or change permissions so that your account gets the right permissions.

Once you gained access to the ‘ModelGenerator’-key, you’ll see two important values: ‘BackupSQLServer’ and ‘SQLServer’. You’ll notice that the model generator installer did not use the sql server hostname you provided but has instead used the local machine name. Change both values to make model generator work.

As always, if you need any help in getting the model generator to work, please let me know. It took me a while to get this tool working, but it’s quite useful once you figure out how to make it work :)!

Posted in Uncategorized | Tagged | 14 Comments

Provisioning permissions using the new BHOLD Access Management MA

A couple of weeks ago, Microsoft released Service Pack 1 of FIM 2010 R2 and the Microsoft BHOLD Suite. With that release, a new FIM Sync management agent was introduced, which makes it much easier to provision users, org.units and permissions to BHOLD. In previous versions, we had to create three separate sql ma’s to provision intermediate tables. The new management agent, which was built using the ECMA 2.0 framework simplifies this process.

To enable people to implement this management agent, Microsoft published a test lab guide that describes how to install and setup the MA and how to provision users and org.units. Unfortunately (at the time of writing), this guide does not specify how to provision permissions, which is quite an essential step.

I adapted the code that was provided to provision users and org.units to also provision permissions (known as  ‘groups’ in the MA). Setting the ‘bholdDescription’ was not enough. It resulted in an ‘unknown error’. To solve this, I added flows for ‘bholdMaxRoles’ and ‘bholdMaxUsers’, which I’ve both set to a constant value of ‘0’. Also, I’ve added a flow for the ‘ApplicationDescription’ field. This field should contain the same value as the ‘Description’-field of the Application you want the permission to be a part of (in my case: ‘Active Directory’).

Image

 

Running an export on my BHOLDMA resulted in exactly the same behavior (unexpected-error). Next step was to add the ‘bholdTaskName’. While taking a look at the other imported groups, I noticed that, for the B1 roles, it always contains the same value as the bholdDescription field. I did some research on this and figured out that when I flowed the permission name to bholdTaskName, the exported was successful. While experimenting with this, I noticed that this value needs to be unique. It cannot even contain the value of a deleted permission in BHOLD Core. Altering the value (for example: when changing the name of an AD group) also resulted in some unwanted behavior: the object needed to be rejoined because FIM Sync does not detect this as a change, but as an entirely different object.

My solution for this? It turns out that that BHOLDTaskName is visible as the ‘Permission’ field in BHOLD Core, but apart from that, isn’t really a very useful field: an end user would not really see the value. To guarantee uniqueness, I altered my provisioning code to generate a guid and set it initially. Here’s how you see that value in BHOLD Core:
Image

As you can see in the below screenshot, BHOLD Core shows the description and doesn’t use the BHOLDTaskName value while searching a permission:

Image

 

Later on, I found out that there is only one inconsistency: viewing the permissions of a user shows the guid:
Image

But viewing it in any other location shows the correct permission description:
Image

 

Although, I must admit, I don’t feel comfortable by setting a non-descriptive value to this attribute, I think it’s the best option for now. It’s just too error-prone to use the description, because the field is not changeable.

Here’s my provisioning code:

public void Provision(MVEntry mventry)
{
try
{
if ((mventry.ObjectType.Equals(“person”)))
ProvisionToADUsers(mventry);
if ((mventry.ObjectType.Equals(“person”)))
ProvisionToAMCUsers(mventry);
if ((mventry.ObjectType.Equals(“user”)))
ProvisionToADUsers(mventry);
if ((mventry.ObjectType.Equals(“organization”)))
ProvisionToAMCOrgunits(mventry);
if ((mventry.ObjectType.Equals(“group”)))
ProvisionToAMCGroups(mventry);
}
catch (Exception ex)
{
throw ex;
}
}

 

 

private void ProvisionToAMCGroups(MVEntry mventry)
{
try
{
int numberofConnectors = 0;
ConnectedMA myMA = mventry.ConnectedMAs[“BHOLDMA”];
numberofConnectors = myMA.Connectors.Count;
if (0 == numberofConnectors)
{
CSEntry obCS = default(CSEntry);
obCS = myMA.Connectors.StartNewConnector(“Group”);
ReferenceValue DN = default(ReferenceValue);
DN = myMA.EscapeDNComponent(System.Guid.NewGuid().ToString());
obCS[“bholdTaskName”].Value = Guid.NewGuid().ToString();
obCS.DN = DN;
obCS.CommitNewConnector();
}
}
catch (Exception ex)
{
throw ex;
}
}

 

And my export flows:Image

 

 

If you have any questions on provisioning permissions (or should I say ‘groups’ :)?) to BHOLD Core, let me know!

 

Posted in Uncategorized | Tagged | 6 Comments

Upgrading to FIM 2010 R2 Service Pack 1

Today, I updated my FIM 2010 R2 sandbox environment to service pack 1. Updating FIM Sync and the FIM Portal was very easy. This can be done by either using Windows Update (which sees this as an important update) and two simple wizards. The most important thing is that you backup your database before performing the installation. Here’s some screenshots:

Image

 

Image

 

The upgrade of the BHOLD suite is a different story. The reason behind this is that there are some significant improvements to BHOLD since the last version. To perform the upgrade, I followed the instructions on this site.

Before installing the new service pack, all previously installed modules need to be uninstalled. This uninstall process does not remove your existing database. After doing this, you can execute a SQL script (as shown on the page I linked) that updates your current database to the new version. Note that the execution of the script might throw an error the first time you run it. Once you run it a second time, the error should be gone.

The next step is to delete the Management Policy Rules (MPR) that were previously created during the installation of the FIM Integration module. To do this, open the FIM Portal and delete every MPR that contains the word ‘BHOLD’ and ‘__ADMINISTRATORS CAN WORK WITH Workflow parts’.

When trying to re-install the BHOLD Core, I got an error. The type of error already shows the first improvement: it contains a lot of useful data!!

Image

According to the linked page, it’s ok to click ‘Ignore’ here. After that, the installation was successful!

Image

 

In my next blog post, I will explain how I installed the new Access Management Connector.

 

Posted in Uncategorized | Leave a comment

Whatever you do: don’t use ‘function’ as the name of your attribute.

I’ve seen before that you should be very careful in choosing the naming of attributes in your management agent configuration. Choosing the wrong name could result in unexpected behavior.

For instance: if you choose ‘Function’ as an attribute name, it is not possible anymore to define an Inbound Attribute Flow from the portal that makes use of the ‘Advanced Function’ functionality. Clicking the Function attribute from the dropdownlist displays the function controls. Clicking one of the available functions from the list is not possible. I recorded the behavior here:

If you change that attribute name in the management agent to something else, everything works again. At this time, I’m not aware if service pack 1 of FIM 2010 R2 solves this issue.

Posted in Uncategorized | Tagged | Leave a comment

Export and delete the FIM Sync run history using powershell

I know, this sounds rather old: Searching the web for a powershell script that deletes the run history has plenty of hits. Then again, I haven’t really found a script that does exactly what I want, so I combined a few scripts I found on the web.

The script below has 2 parameters:

  • dayDiff; If this parameter is passed, the number supplied will be subtracted from the current date. The script will only delete history records that are older than the result. If no value is given, all history will be deleted.
  • exportDirectory; If you pass a valid directory here, the script will export the full run history to that directory. Note: the directory should end with ‘/’. If no directory is supplied, the run history will not be exported.

Here’s the script:

PARAM([int]$dayDiff, [string]$exportDirectory)

$dateDelete = Get-Date

#if the dayDiff parameter is passed, subtract it from the current day.
If($dayDiff -ne 0)
{
$dateDelete = $dateDelete.AddDays(-$dayDiff)
}
Else
{
$dateDelete = $dateDelete.AddDays(1)
}

#if the exportDirectory parameter is passed, save the run history to that directory
If($exportDirectory -ne “”)
{
Write-Host “Exporting full run history.”

$lstManagementAgent = @(get-wmiobject -class “MIIS_ManagementAgent” -namespace “root\MicrosoftIdentityIntegrationServer” -computer “.”)
$runDetails = $lstManagementAgent[0].RunDetails().ReturnValue

$doc = New-Object System.Xml.XmlDocument
$doc.LoadXml($runDetails)
$dateNow = Get-Date -format “yyyyMMddHHmm”

$filePathName = $exportDirectory + $dateNow + “.xml”
$doc.Save($filePathName)

Write-Host “Successfully exported run history to: ” $filePathName
}

#finally, delete the run history:
Write-Host “Deleting run history earlier than:” $dateDelete.toString(‘MM/dd/yyyy’)

$lstSrv = @(get-wmiobject -class “MIIS_SERVER” -namespace “root\MicrosoftIdentityIntegrationServer” -computer “.”)
Write-Host “Result: ” $lstSrv[0].ClearRuns($dateDelete.toString(‘yyyy-MM-dd’)).ReturnValue

#——————————————————————————————————————–
Trap
{
Write-Host “Error: $($_.Exception.Message)” -foregroundcolor white -backgroundcolor darkred
Exit
}
#——————————————————————————————————————–

Here’s the source for the original script that clears the run history (without exporting it first).
http://identityandbeyond.com/2010/05/17/clearing-run-history-from-fim-2010-with-powershell/

There’s still room for improvement: it might be better if the script would export just the records that it is about to delete, but it’s a great start to come forward to the average wishes of a FIM administrator :)!

 

Posted in Uncategorized | Tagged | 2 Comments