Issue: new BHOLD Access Management Connector doesn’t import all objects

Hi all,

Earlier this week, I had an issue with the new BHOLD Access Management connector (AMC), which imports users, groups and org.units from Microsoft BHOLD Suite (SP1). It appeared that not all groups that were present in BHOLD Core, were imported in FIM (even during a full import)!

To fully understand the problem (and solution), here’s some details about how the connector fetches the objects from BHOLD. The connector is built using the ECMA 2.0 API for building custom management agents. As most of you will probably know, this API enables you to specify a batch size. The idea is that the connector will retrieve the amount of records as specified in the batch size configuration value. After that, FIM Sync imports the records in the connector space. After importing, it calls the import-function again, until it returns that all records have been imported.

To get information from BHOLD, the AMC uses these stored procedures:

[GetFullImportKeysAndInitializeWatermark]
This stored procedure is executed during the initialization. Basically, it returns all the object types and the maximum id value of that object type.

[GetGroupBaseAttributes], [GetOrganizationalUnitBaseAttributes], [GetUserBaseAttributes]
This one retrieves the groups (called ‘permissions’ in BHOLD Core), org. units and users. It takes the following arguments:
@BatchSize; The size of each ‘page’ (as set in the FIM Sync run profile)
@ObjectKey; Only objects with an id higher than this value, will be imported. If the procedure is called the first time, the value ‘0’ is supplied.
@NextObjectKey; This is an ‘out’ parameter, that (unlike what the name suggests) contains the id of the last object that is part of the result.

So, the following workflow is followed:

  1. Execute ‘GetFullImportKeysAndInitializeWatermark’
  2. Execute ‘Get<object>BaseAttributes’, replacing ‘<object>’ with the retrieved object in step 1.
  3. Reformat the retrieved values to a format that FIM Sync understands.
  4. See if the value of @NextObjectKey (the out-parameter of the ‘Get<object>BaseAttributes’ procedure) is equal to the max id value as retrieved in step 1. If it is not, return ‘true’ to FIM Sync, indicating that there are more objects to be imported. If it is: return ‘false’, indicating that everything is imported.

According to how the AMC calls the stored procedures, this all looks implemented very well. Except: I’ve noticed that the AMC doesn’t retrieve all the groups. Furthermore: the number of imported objects is dependent on the batch size value.

Here’s the catch: to make all the above work, it is CRITICAL that the ‘Get<object>BaseAttributes’ procedure contains an ‘ORDER BY ‘ObjectIdentifier’ (id) clause! If not, it will miss objects.

Let’s say this is the content of your table that contains the groups:

PermissionId;PermissionName;ApplicationID
4;PermissionA;0
5;PermissionB;0
6;PermissionC;0
3;PermissionD;0
8;PermissionE;0

And, for this example, we use a batch size of ‘3’.

  1. ‘GetFullImportKeysAndInitializeWatermark’ will be executed. The max id of the groups object type returned is ‘8’.
  2. Next, ‘GetGroupBaseAttributes’ is executed with parameters:
    @BatchSize: 3
    @ObjectKey: 0
  3. The result, without the ‘ORDER BY’ clause, will be:
    PermissionId;PermissionName;ApplicationID
    4;PermissionA;0
    5;PermissionB;0
    6;PermissionC;0
    The output parameter will contain: ‘6’, because this is the max id of the returned objects.
  4. As the value ‘6’ is not equal to ‘8’ (step 1), FIM Sync gets the message that not all records have been retrieved yet.
  5. ‘GetGroupBaseAttributes’ is executed again, but now with the following attributes:
    @BatchSize: 3
    @ObjectKey: 6
  6. The result will be:
    8;PermissionE;0
    The output parameter will contain ‘8’.
  7. This is equal to the max id, so import will stop.

As you will notice: PermissionD, with ID ‘3’, has been missed!!

The solution

Search for the following line in the ‘GetGroupBaseAttributes’ stored procedure…
WHERE [p].[PermissionID] > @ObjectKey
…and add the following to it:
    ORDER BY [p].PermissionId

Finally, I would like to say thanks to my colleague Kenny. We worked on this issue together, having a hard time finding out what the problem was. In the end, he was the one that noticed the ORDER BY clause. Thanks!

As always, if you have any questions or remarks, let me know!!

Posted in Uncategorized | Tagged | 3 Comments

Issue when using the model generator when not running a local database

The model generator is a useful tool to import your existing data in BHOLD Core. You can use csv or excel-files in a certain format.

I’ve been using the model generator to import data in the Five-file set (as described in the link above). This is the environment I’m using:
Image

As you can see, the BHOLD Suite modules are not installed on the same server as the server where SQL Server is installed. This shouldn’t be a problem. When installing the Model Generator, everything worked fine, but once running it, I came across this error:
Image

 

This shows up in the event log:

System.Data.SqlClient.SqlException: A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 – Could not open a connection to SQL Server)

What happens here is that the model generator tried to export the contents of the text-files to the BHOLD database. According to the error, the SQL Server is unavailable. It turns out that the Model Generator is trying to access the database on the local server. This is caused by an error in the installation file.

To fix this, open the registry editor (‘regedit’) and navigate to: ‘HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\bhold\ModelGenerator’. The first thing you’ll notice is that you are not permitted to access this key.
Image
To solve this, open the regedit mmc using the service account you used during the model generator setup. You can do the modifications from there or change permissions so that your account gets the right permissions.

Once you gained access to the ‘ModelGenerator’-key, you’ll see two important values: ‘BackupSQLServer’ and ‘SQLServer’. You’ll notice that the model generator installer did not use the sql server hostname you provided but has instead used the local machine name. Change both values to make model generator work.

As always, if you need any help in getting the model generator to work, please let me know. It took me a while to get this tool working, but it’s quite useful once you figure out how to make it work :)!

Posted in Uncategorized | Tagged | 14 Comments

Provisioning permissions using the new BHOLD Access Management MA

A couple of weeks ago, Microsoft released Service Pack 1 of FIM 2010 R2 and the Microsoft BHOLD Suite. With that release, a new FIM Sync management agent was introduced, which makes it much easier to provision users, org.units and permissions to BHOLD. In previous versions, we had to create three separate sql ma’s to provision intermediate tables. The new management agent, which was built using the ECMA 2.0 framework simplifies this process.

To enable people to implement this management agent, Microsoft published a test lab guide that describes how to install and setup the MA and how to provision users and org.units. Unfortunately (at the time of writing), this guide does not specify how to provision permissions, which is quite an essential step.

I adapted the code that was provided to provision users and org.units to also provision permissions (known as  ‘groups’ in the MA). Setting the ‘bholdDescription’ was not enough. It resulted in an ‘unknown error’. To solve this, I added flows for ‘bholdMaxRoles’ and ‘bholdMaxUsers’, which I’ve both set to a constant value of ‘0’. Also, I’ve added a flow for the ‘ApplicationDescription’ field. This field should contain the same value as the ‘Description’-field of the Application you want the permission to be a part of (in my case: ‘Active Directory’).

Image

 

Running an export on my BHOLDMA resulted in exactly the same behavior (unexpected-error). Next step was to add the ‘bholdTaskName’. While taking a look at the other imported groups, I noticed that, for the B1 roles, it always contains the same value as the bholdDescription field. I did some research on this and figured out that when I flowed the permission name to bholdTaskName, the exported was successful. While experimenting with this, I noticed that this value needs to be unique. It cannot even contain the value of a deleted permission in BHOLD Core. Altering the value (for example: when changing the name of an AD group) also resulted in some unwanted behavior: the object needed to be rejoined because FIM Sync does not detect this as a change, but as an entirely different object.

My solution for this? It turns out that that BHOLDTaskName is visible as the ‘Permission’ field in BHOLD Core, but apart from that, isn’t really a very useful field: an end user would not really see the value. To guarantee uniqueness, I altered my provisioning code to generate a guid and set it initially. Here’s how you see that value in BHOLD Core:
Image

As you can see in the below screenshot, BHOLD Core shows the description and doesn’t use the BHOLDTaskName value while searching a permission:

Image

 

Later on, I found out that there is only one inconsistency: viewing the permissions of a user shows the guid:
Image

But viewing it in any other location shows the correct permission description:
Image

 

Although, I must admit, I don’t feel comfortable by setting a non-descriptive value to this attribute, I think it’s the best option for now. It’s just too error-prone to use the description, because the field is not changeable.

Here’s my provisioning code:

public void Provision(MVEntry mventry)
{
try
{
if ((mventry.ObjectType.Equals(“person”)))
ProvisionToADUsers(mventry);
if ((mventry.ObjectType.Equals(“person”)))
ProvisionToAMCUsers(mventry);
if ((mventry.ObjectType.Equals(“user”)))
ProvisionToADUsers(mventry);
if ((mventry.ObjectType.Equals(“organization”)))
ProvisionToAMCOrgunits(mventry);
if ((mventry.ObjectType.Equals(“group”)))
ProvisionToAMCGroups(mventry);
}
catch (Exception ex)
{
throw ex;
}
}

 

 

private void ProvisionToAMCGroups(MVEntry mventry)
{
try
{
int numberofConnectors = 0;
ConnectedMA myMA = mventry.ConnectedMAs[“BHOLDMA”];
numberofConnectors = myMA.Connectors.Count;
if (0 == numberofConnectors)
{
CSEntry obCS = default(CSEntry);
obCS = myMA.Connectors.StartNewConnector(“Group”);
ReferenceValue DN = default(ReferenceValue);
DN = myMA.EscapeDNComponent(System.Guid.NewGuid().ToString());
obCS[“bholdTaskName”].Value = Guid.NewGuid().ToString();
obCS.DN = DN;
obCS.CommitNewConnector();
}
}
catch (Exception ex)
{
throw ex;
}
}

 

And my export flows:Image

 

 

If you have any questions on provisioning permissions (or should I say ‘groups’ :)?) to BHOLD Core, let me know!

 

Posted in Uncategorized | Tagged | 6 Comments

Upgrading to FIM 2010 R2 Service Pack 1

Today, I updated my FIM 2010 R2 sandbox environment to service pack 1. Updating FIM Sync and the FIM Portal was very easy. This can be done by either using Windows Update (which sees this as an important update) and two simple wizards. The most important thing is that you backup your database before performing the installation. Here’s some screenshots:

Image

 

Image

 

The upgrade of the BHOLD suite is a different story. The reason behind this is that there are some significant improvements to BHOLD since the last version. To perform the upgrade, I followed the instructions on this site.

Before installing the new service pack, all previously installed modules need to be uninstalled. This uninstall process does not remove your existing database. After doing this, you can execute a SQL script (as shown on the page I linked) that updates your current database to the new version. Note that the execution of the script might throw an error the first time you run it. Once you run it a second time, the error should be gone.

The next step is to delete the Management Policy Rules (MPR) that were previously created during the installation of the FIM Integration module. To do this, open the FIM Portal and delete every MPR that contains the word ‘BHOLD’ and ‘__ADMINISTRATORS CAN WORK WITH Workflow parts’.

When trying to re-install the BHOLD Core, I got an error. The type of error already shows the first improvement: it contains a lot of useful data!!

Image

According to the linked page, it’s ok to click ‘Ignore’ here. After that, the installation was successful!

Image

 

In my next blog post, I will explain how I installed the new Access Management Connector.

 

Posted in Uncategorized | Leave a comment

Whatever you do: don’t use ‘function’ as the name of your attribute.

I’ve seen before that you should be very careful in choosing the naming of attributes in your management agent configuration. Choosing the wrong name could result in unexpected behavior.

For instance: if you choose ‘Function’ as an attribute name, it is not possible anymore to define an Inbound Attribute Flow from the portal that makes use of the ‘Advanced Function’ functionality. Clicking the Function attribute from the dropdownlist displays the function controls. Clicking one of the available functions from the list is not possible. I recorded the behavior here:

If you change that attribute name in the management agent to something else, everything works again. At this time, I’m not aware if service pack 1 of FIM 2010 R2 solves this issue.

Posted in Uncategorized | Tagged | Leave a comment

Export and delete the FIM Sync run history using powershell

I know, this sounds rather old: Searching the web for a powershell script that deletes the run history has plenty of hits. Then again, I haven’t really found a script that does exactly what I want, so I combined a few scripts I found on the web.

The script below has 2 parameters:

  • dayDiff; If this parameter is passed, the number supplied will be subtracted from the current date. The script will only delete history records that are older than the result. If no value is given, all history will be deleted.
  • exportDirectory; If you pass a valid directory here, the script will export the full run history to that directory. Note: the directory should end with ‘/’. If no directory is supplied, the run history will not be exported.

Here’s the script:

PARAM([int]$dayDiff, [string]$exportDirectory)

$dateDelete = Get-Date

#if the dayDiff parameter is passed, subtract it from the current day.
If($dayDiff -ne 0)
{
$dateDelete = $dateDelete.AddDays(-$dayDiff)
}
Else
{
$dateDelete = $dateDelete.AddDays(1)
}

#if the exportDirectory parameter is passed, save the run history to that directory
If($exportDirectory -ne “”)
{
Write-Host “Exporting full run history.”

$lstManagementAgent = @(get-wmiobject -class “MIIS_ManagementAgent” -namespace “root\MicrosoftIdentityIntegrationServer” -computer “.”)
$runDetails = $lstManagementAgent[0].RunDetails().ReturnValue

$doc = New-Object System.Xml.XmlDocument
$doc.LoadXml($runDetails)
$dateNow = Get-Date -format “yyyyMMddHHmm”

$filePathName = $exportDirectory + $dateNow + “.xml”
$doc.Save($filePathName)

Write-Host “Successfully exported run history to: ” $filePathName
}

#finally, delete the run history:
Write-Host “Deleting run history earlier than:” $dateDelete.toString(‘MM/dd/yyyy’)

$lstSrv = @(get-wmiobject -class “MIIS_SERVER” -namespace “root\MicrosoftIdentityIntegrationServer” -computer “.”)
Write-Host “Result: ” $lstSrv[0].ClearRuns($dateDelete.toString(‘yyyy-MM-dd’)).ReturnValue

#——————————————————————————————————————–
Trap
{
Write-Host “Error: $($_.Exception.Message)” -foregroundcolor white -backgroundcolor darkred
Exit
}
#——————————————————————————————————————–

Here’s the source for the original script that clears the run history (without exporting it first).
http://identityandbeyond.com/2010/05/17/clearing-run-history-from-fim-2010-with-powershell/

There’s still room for improvement: it might be better if the script would export just the records that it is about to delete, but it’s a great start to come forward to the average wishes of a FIM administrator :)!

 

Posted in Uncategorized | Tagged | 2 Comments

FIM R2 and BHOLD Suite Service Pack 1

Just a quick update on my entry about the bug in BHOLD Suite concerning permission deprovisioning. I mentioned that Service Pack 1 would contain a fix for this issue. Yesterday, Andreas Kjellman announced that Service Pack 1 is released. This is however a so-called ‘soft release’, which means that the update is only available on MSDN and that there isn’t much documentation available yet. It also means that you are not allowed to apply the update to your production environment just yet.

I’ll try to install this new version on my sandbox environment and let you know if the issue is resolved. Of course, I’m also curious about any other changes. Dave Nesbit already mentioned on his blog that the BFPC and BFSS services are replaced by the new ‘Access Management Connector’, so I’ll see if I can bring any news on that as well.

I’ll keep you posted on any progress!

Posted in Uncategorized | Tagged | 1 Comment

Interesting bug in Microsoft BHOLD Suite concerning permission deprovisioning

While working on a BHOLD implementation, I found an interesting bug:

If you’re getting to know BHOLD, you’ll soon find out that it consists out of (among others) these object types:

  • User; An employee in your organization, mostly imported to BHOLD by the FIM Synchronization engine through the ‘FIMEmployees’ table.
  • Org.Unit; An organizational unit. Your BHOLD environment will probably contain an entire tree structure of org.units. An Employee should have a reference to an Org.Unit object.
  • Role; A role that can be assigned to an employee. It can be assigned to an employee based on his/her position in the Org.Unit structure, or based on an attribute of the user (basically, the assignment of roles is one of the main features of BHOLD).
  • Permission; A role consists out of 1 or more permissions. Linked to Active Directory, this would be the same as a ‘Group’. Other applications might have different names for this.

Basically, through his/her role membership, an employee is entitled to a set of permissions. To export these permissions to your target system, you will need to synchronize them using the FIM Synchronization service. FIM Sync imports the permissions and memberships through the ‘tblObjects’ and ‘tblReferences’ tables, where ‘tblObjects’ contains the users and permission objects and ‘tblReferences’ contains the link between the two objects.

That’s it for the technical explanation, let’s continue to the details of the bug: The tblReferences table is filled by a BHOLD service (the ‘BFPC’) once a user is entitled to a certain permission, which works as intented. Apparently, it doesn’t do a good job of deleting a record once the user is not entitled to a permission anymore.

I’ve contacted someone from BHOLD, who says this is a known bug. He tells me there is a patch available to fix this issue, but this patch is not generally available. You can however request the patch by creating a service call at Microsoft. Service Pack 1 of the BHOLD suite will contain a fix for the issue, but this service pack is not yet released for production environments.

If you’re experiencing the issue, or if you want to know more technical details about it, let me know. Hang in there: a production-suitable fix is on the way!!

Posted in Uncategorized | Tagged | Leave a comment

Microsoft BHOLD Suite Logging

For the last couple of weeks, I started to work with Microsoft BHOLD suite. If you’re new to the product, you’ll soon find out that the logging of the BFSS and BFPC services is a good starting point in trying to solve issues. Here’s some information on the logging configuration of these services.

By default, all information is logged to the files ‘BFSS.log’ and ‘BFPC.log’ in the ‘C:\temp\’ folder. To change the directory in which these files are stored, create a registry key in ‘HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\bhold\b1Core\LogfileDir‘, containing the directory (without the filename) where you want the logfiles to be stored.

You can also add the ‘BFSSLogLevel’ and ‘BFPCLogLevel’ keys. This suggests that it is possible to control the amount of messages logged, but it doesn’t. If the value is set to ‘verbose’, all available information is logged. Any other value (like ‘none’ or ‘warning’ / ‘error’) stops all logging for the BFSS and limits the logging for BFPC to things like ‘BFPC Initialising’ and ‘BFPC Ended’.

To summarize: there aren’t much possibilities to configure logging, it’s basically all or nothing. The information above could prove value in changing the default log location and disabling logging in your production environment. This is most recommended because if verbose logging is enabled (which it is by default), this will result in gigabytes of logdata per day!

Posted in Uncategorized | Tagged | Leave a comment