How to bring any content from Microsoft Graph to Microsoft Search with the CLI for Microsoft 365

28/02/2024

CLI for Microsoft 365

With the CLI for Microsoft 365 you can manage Microsoft 365 tenants and SharePoint Framework projects on either Linux, MacOS or Windows.

The CLI runs on any shell like Azure Cloud Shell, bash, cmder, PowerShell, and zsh and supports authentication methods like Azure Managed Identity, certificate, client secret, device code, and username/password.

Install

To use the CLI for Microsoft 365 you need Node.js v18+.

npm install -g @pnp/cli-microsoft365

By default, the CLI uses the PnP Management Shell multitenant application to sign users in and device code authentication.

m365 login

For authentication with client secret, use the authType option and set the value to secret. To sign-in using a custom Entra app, use the appId option together with the id of the custom app and the tenant option with the id of your tenant.

m365 login --authType secret --appId {entra_app_id} --tenant {tenant_id}

For logout, use the m365 logout command.

Run commands

List all commands using the help option.

m365 --help

Similar for subcommands

m365 external --help

Connectors

The Graph API exposes endpoints that enable you to bring data from external sources and index data to be available in Microsoft Search.

What's important, you are not limited to external data only, but you can index data already available in the Microsoft Graph.

You can index any data you want.

To bring data, you need to build a custom Microsoft Graph connector. The whole process consists of several steps:

create a custom Entra application with limited permissions to manage a connector
create an external connection
create a schema for the external connection to describe the content
add items to the external connection

Create custom Entra app

A Entra app should be able to create an external connection and add items. Based on these requirements, the following application permissions must be granted to the app

ExternalConnection.ReadWrite.OwnedBy - create a connection, create a schema
ExternalItem.ReadWrite.OwnedBy - add item

The M365 CLI has command m365 entra app add for registering a new app.

m365 entra app add --name 'MyDataExternalConnection' --withSecret --apisApplication 'https://graph.microsoft.com/ExternalConnection.ReadWrite.OwnedBy,https://graph.microsoft.com/ExternalItem.ReadWrite.OwnedBy' --grantAdminConsent

The option withSecret will create a default secret with the expiration 1 year. Secret value will be returned in the response.

The option grantAdminConsent will grant permissions through the admin consent.

The response will contain appId, tenantId and client secret.

Create connection

To create a new external connection, you need to set a unique ID, name, and a description of the connector.

m365 login --appId {appId} --tenant {tenantId} --secret {clientSecret} --authType secret

Create a new external connection:

m365 external connection add --id {connectorId} --name {connectorName} --description {connectorDescription}

Create schema

Schema defines a set of properties and their attributes, labels, and aliases.

The table below describes a schema property definition

Property	Description
name	The name of the property
type	The type of the property
aliases	A set of alternate names
labels	A set of well-know tags
isSearchable	Specifies whether the property is indexed and can searchable
isRetrievable	Specifies whether the property is returned in the result set
isQueryable	Specifies whether the property is used for filtering the result set
isRefinable	Specifies whether the property supports aggregation

The possible types are

string, stringCollection
int64, int64Collection
double, doubleCollection
dateTime, dateTimeCollection
boolean

Use command m365 external connection schema add to create a schema with the CLI

m365 external connection schema add -i {connectorId} --schema {schema} --wait

Creating a schema is a long running process and may take several minutes. The wait option ensures that the command will periodically check the status and exit after the schema is created.

Add items

Once the schema is created, you can add items to the connector.

When adding an external item, the options you specify will map to the item properties, eg. --userName 'John Doe' will set the userName property to John Doe.

m365 external item add --externalConnectionId {connectorId} --id {itemId} --userName 'John Doe'

That's all of you need to be able to bring data to Microsoft Search.

Example

As an example, I'll show how to get data that is already exposed by the Graph API.

Users endpoint allows you to retrieve a collection of users and perform server side filtering to retreive a subset of users. You can define which properties will be included in the response returned by API.

Some properties can't be returned within a user collection. Properties like aboutMe, birthday, hireDate, interests, mySite, pastProjects, preferredName, responsibilities, schools, skills, or mailboxSettings can be retrieved only for a single user:

GET /v1.0/users/{user_id}?$select=aboutMe,mailboxSettings,mySite

If your organization has thousand of users and you need to periodically check some of the properties above, it's something when the external connectors and Microsoft Search can be useful. Instead of retrieving users one by one, we can create a custom connector.

Schema

The table below describes schema of custom connector

Property	Name	Type	Queryable	Retrievable	Searchable	Labels	Aliases
`user.id`	id	string	true	true	true	title	-
`user.userPrincipalName`	userPrincipalName	string	true	true	true	-	upn
`user.mailboxSettings.userPurpose`	userPurpose	string	true	true	false	-	userType
`user.mailboxSettings.language.locale`	locale	string	true	true	false	-	-
`user.mailboxSettings.timezone`	timezone	string	true	true	false	-	-
`user.employeeType`	employeeType	string	true	true	false	-	-
`user.mySite`	personalSite	string	true	false	false	-	site
`user.manager.userPrincipalName`	manager	string	true	true	true	-	-
`user.country`	country	string	true	true	false	-	-
`user.city`	city	string	true	true	false	-	-
`user.department`	department	string	true	true	false	-	-

The CLI has command to read user properties

m365 entra user get --id {userId} --properties 'id,userPrincipalName,mailboxSettings,employeeType,mySite,country,city,department' --withManager

Full code

PowerShell script

# create an Entra app

m365 login

$permissions = 'https://graph.microsoft.com/ExternalConnection.ReadWrite.OwnedBy,https://graph.microsoft.com/ExternalItem.ReadWrite.OwnedBy,https://graph.microsoft.com/User.Read.All,https://graph.microsoft.com/MailboxSettings.Read'

$appResponse = m365 entra app add `
--name 'UserDataExternalConnectorApp' `
--withSecret `
--apisApplication $permissions `
--grantAdminConsent | ConvertFrom-Json

m365 logout

# login to a new app 
# wait for a while until the Entra app is provisioned
Start-Sleep -Seconds 10

m365 login --tenant $appResponse.tenantId --appId $appResponse.appId --secret $appResponse.secrets[0].value --authType secret

# connector id
$externalConnectionId = 'UserDataConnector'

# create connector
m365 external connection add --id $externalConnectionId --name 'User data' --description 'User data connection'

# define and create schema

$params = @{
	baseType = "microsoft.graph.externalItem"
	properties = @(
		@{
			name = "id"
			type = "String"
			isRetrievable = "true"
            isQueryable = "false"
            isSearchable = "true"
			labels = @(
				"title"
			)
		}
		@{
			name = "userPrincipalName"
			type = "String"
			isRetrievable = "true"
            isQueryable = "false"
            isSearchable = "true"
            aliases = @(
                "upn"
            )
		}
		@{
			name = "userPurpose"
			type = "String"
			isRetrievable = "true"
            isQueryable = "true"
            isSearchable = "false"
            aliases = @(
                "userType"
            )
		}
        @{
			name = "locale"
			type = "String"
			isRetrievable = "true"
            isQueryable = "true"
            isSearchable = "false"
		}
        @{
			name = "timeZone"
			type = "String"
			isRetrievable = "true"
            isQueryable = "true"
            isSearchable = "false"
		}
        @{
			name = "employeeType"
			type = "String"
			isRetrievable = "true"
            isQueryable = "true"
            isSearchable = "false"
		}
        @{
			name = "mySite"
			type = "String"
			isRetrievable = "true"
            isQueryable = "false"
            isSearchable = "false"
            aliases = @(
                "site"
            )
		}
        @{
			name = "manager"
			type = "String"
			isRetrievable = "true"
            isQueryable = "true"
            isSearchable = "true"
		}
        @{
			name = "country"
			type = "String"
			isRetrievable = "true"
            isQueryable = "true"
            isSearchable = "false"
		}
        @{
			name = "city"
			type = "String"
			isRetrievable = "true"
            isQueryable = "true"
            isSearchable = "false"
		}
        @{
			name = "department"
			type = "String"
			isRetrievable = "true"
            isQueryable = "true"
            isSearchable = "false"
		}
	)
}

$schemaJson = $params | ConvertTo-Json -Compress -Depth 3

m365 external connection schema add -i $externalConnectionId --schema $schemaJson --wait

# read all users ids

$usersWithId = m365 entra user list --properties id | ConvertFrom-Json

Foreach ($userWithId in $usersWithId) {
  # read details of each user
  $user = m365 entra user get --id $userWithId.id --properties 'id,userPrincipalName,mailboxSettings,employeeType,mySite,country,city,department' --withManager | ConvertFrom-Json

  $locale = $user.mailboxSettings.language.locale -eq $null ? '' : $user.mailboxSettings.language.locale
  $timeZone = $user.mailboxSettings.timeZone -eq $null ? '' : $user.mailboxSettings.timeZone
  $employeeType = $user.employeeType -eq $null ? '' : $user.employeeType
  $mySite = $user.mySite -eq $null ? '' : $user.mySite
  $manager = ($user.manager -eq $null -or $user.manager.userPrincipalName -eq $null) ? '' : $user.manager.userPrincipalName
  $country = $user.country -eq $null ? '' : $user.country
  $city = $user.city -eq $null ? '' : $user.city
  $department = $user.department -eq $null ? '' : $user.department
  $content = $user.userPrincipalName -eq $null ? '' : $user.userPrincipalName
  #
  m365 external item add --externalConnectionId $externalConnectionId `
  --id $userWithId.id `
  --userPrincipalName $user.userPrincipalName `
  --userPurpose $user.mailboxSettings.userPurpose `
  --locale $locale `
  --timeZone $timeZone `
  --employeeType $employeeType `
  --mySite $mySite `
  --manager $manager `
  --country $country `
  --city $city `
  --department $department `
  --content $content `
  --acls 'grant,everyone,everyone'
}

m365 logout

Microsoft 365 admin center

Now, the connector should be visible in Microsoft 365 admin center.

Go to Settings → Search & intelligence → Data sources tab

Searching

When the connector is created, you can use search API to query data

POST https://graph.microsoft.com/v1.0/search/query
{
    "requests": [
        {
            "entityTypes": [
                "externalItem"
            ],
            "contentSources": [
                "/external/connections/UserDataConnector"
            ],
            "query": {
                "queryString": "*"
            },
            "from": 0,
            "size": 100,
            "fields": [
                "id",
                "userPurpose",
                "userPrincipalName",
                "manager",
                "department",
                "city",
                "country",
                "employeeType"
            ]
        }
    ]
}

Search for	Query string
shared mailboxes	`"queryString": "userPurpose:shared"`
rooms	`"queryString": "userPurpose:room"`
equipment	`"queryString": "userPurpose:equipment"`
users from US	`"queryString": "userPurpose:user AND country:\"United States\""`
contractors from CZ	`"queryString": "employeeType:contractor AND country:\"Czech Republic\""`

Conclusion

With the custom Microsoft Graph connector you can bring external or internal data into Microsoft Search. The CLI for Microsoft 365 simplify the management of the external connectors and adding items.

The PowerShell script can be found in the GitHub repository.