Confluence connector for Simpplr Enterprise Search

Introduction

The Confluence connector allows Simpplr Enterprise Search to index Documents and Spaces that you have created in Confluence, making it easily discoverable and searchable directly within Simpplr.

With this connector, you can (use cases):

• Bring Confluence content into Simpplr Enterprise Search so users can find Confluence spaces and documents, alongside intranet content in one place.

• Respect Confluence permissions so users only see files they already have access to in Confluence.

• Use advanced features like autocomplete, hybrid ranking, and Smart Answers on top of Confluence content.

Indexed content from Simpplr Enterprise Search is available in:

• In main search listing

• Smart answers

Capabilities at a glance

Objects and content supported

1. Objects - List the object types that are indexed, for example:

• Space

• Page

2. Metadata - For each indexed item, Confluence captures based on the type of Object : 

• Name

• Type

• Created time and last modified time

• Owner,

• Author,

• Ancestors

• URL

• Space Details

  • Space Name

  • Space URL

  • Space ID

• Status

• Permissions (users and groups level access)

3. Permissions model - Permissions are read from Confluence and enforced in Simpplr Enterprise Search. Include: 

   1. How user and group permissions are synchronized

• Confluence user and group memberships are fetched and stored in the ACL index.

• When a user is added to or removed from a Confluence group, the ACL index is updated the next time the ACL sync runs (by default, every hour).

2. How public or link-shared content is handled

• Content that is only available via anonymous or public shared links is not indexed in the current version.

3. What happens when access is removed for a specific document

• When a user loses access to a Space or Page in Confluence , the updated permissions are applied during the next Incremental / ACL sync.

• The file will no longer appear in that user’s Simpplr search results after the Incremental / ACL sync completes.

Versions and editions supported 

• Supported Confluence account editions: All.

• Not supported: N/A 

Prerequisites

Before you begin, ensure the following:

1. Source system permissions

• You need access to the Confluence cloud Developer Console to create and configure your application.

2. Application / service account

• The user should be having Product Admin access ( to Sync Access Control Information)

• Ability to generate API token from Security Section of the Above User’s Profile 

Confluence documentation:

• Create Confluence API Token

• Assign product admin access to the user

Authentication and security

1. Authentication mechanism
   Describe how  Simpplr Enterprise Search connects to Confluence:

• Auth type: Api token

• Scopes or permissions required (examples):

  • User needs to have product Admin access

2. Data security

   1. Data storage and residency: Indexed content and ACLs from Confluence are stored within your Simpplr Enterprise Search environment, in the same region as your Simpplr tenant.

   2. Encryption in transit: Server-side encryption with Amazon S3 managed keys (SSE-S3), TLS encryption in Kafka.

   3. Encryption at rest: SSL (TLS 1.2 or higher), Auth: OAuth 2.0 Bearer tokens (client-credential).

   4. Permission enforcement: Jita access controls (users and groups) are stored in the ACL index and applied at query time. Search results are always filtered by the signed-in user’s identity and Confluence group memberships.

Setup and Configuration

There are two steps to setup this connector:

1. Setup credentials in Connector Source - This step yields an API Token that will be used in the next step.

2. Setup the connector in Simpplr Intranet

Step 1 - Prepare Confluence source

Step 1a - Setup Prerequisites

• Log in to Confluence Cloud using the Admin Account.

• Make sure the user has Product Admin access. (Required to sync Access Control related information)

• Here are the steps to add Product Admin access for a new user or if the user does not have Product Admin access:

  • Go to Jira Administration by clicking the tiles on the Top Left of the Screen

1. Go to Directory and click on the User.

2. Under Product Access, click on Grant access and add Product Admin

Step 1b - Configuring the Api Token

1.  Navigate to Security → API Tokens → Create and Manage API Tokens
   There are two token options available: The Jira Connector supports both Scoped Tokens and General Tokens. You may choose either type based on your access requirements.

   1. Scoped Token:
      Scoped API tokens allow you to specify what actions a token can perform in Confluence Cloud. Unlike classic tokens, which grant all permissions available to the user, scoped tokens restrict access to only the selected scopes, reducing risk if a token is compromised.
      Scoped tokens have specific permissions and are restricted by selected scopes. They use the api.atlassian.com domain instead of your site-specific URL.

2. General Token (Classic):
   Classic tokens often grant all permissions available to the user who generated them across an entire account or organization. For example, a classic token with "repo" scope has access to all private repositories the user can access.

Due to their broad access, Classic tokens pose a greater security risk if compromised. Revealing a classic token could potentially expose all of an organization's projects and data.

You can setup the connector in two ways:

1. Scoped Api Token: Complex, gives you more control

2. Classic Api Token: Quick & simple

Scoped Api Token

1. Select Create API token with scopes.

2. Give your API token a name that describes its purpose.

3. Select an expiration date for the API token.
   Token expiration must be between 1 and 365 days.

4. Select the Confluence app for the API token to access.

5. Select the below mentioned scopes

Note:

    Total 12 permissions are required   

Filter the Scope Actions: Read, Read Only, Search

Scope Type: Granular Permissions (1):

1. read:content-details:confluence

Tip: Search by scope name read:content-details:confluence

Scope Type: Classic Permissions (11):

1. read:account

2. read:confluence-content.all

3. read:confluence-content.permission

4. read:confluence-content.summary

5. read:confluence-groups

6. read:confluence-props

7. read:confluence-space.summary

8. read:confluence-user

9. read:me

10. readonly:content.attachment:confluence

11. search:confluence

Tip: Filter Scope Type: Classic

6. Select Next, review the token and then select Create.

2. Select Copy to clipboard, then paste the token to your script, or save it somewhere safe.
   You can't recover the API token after you’ve completed this step. We recommend saving it in a password manager.

   

Classic Api Token

1. Generate and copy the API Token.

Step 1c - Extracting the Confluence Cloud ID

Cloud ID can be accessed by navigating to URL:

 <you-tenant-url>/_edge/tenant_info

For Example: If your Confluence application URL looks like this: https://simpplr.atlassian.net/wiki/home,  your tenant URL would be: https://simpplr.atlassian.net

Finally, your cloud ID can be accessed using this URL:
https://simpplr.atlassian.net/_edge/tenant_info

Step 1 Output

2. Api Token

3. Confluence Cloud ID

Step 2 - Create the connector in Enterprise Application

Step 2a - Setting up the configuration

1. In Simpplr, go to: Manage Features → Enterprise Search → Add Source.
   
   

2. Search and Select “Atlassian Confluence Cloud”.
   
   

3. Enter basic information:

   • Connection Name: (Connector Name for this instance)

4. Provide authentication details:

   • Confluence Cloud account email (email)

   • Confluence Cloud API token (copied from the earlier step)

   • Confluence URL label: Host URL should be in the format (use the cloud Id from the earlier step)
     https://api.atlassian.com/ex/confluence/<Cloud ID>

5. Click “Save Configuration” and “confirm”.

Step 2b - Setting up the Data & Audience Filters

• Configure inclusion rules:

  • Not configurable in the current version

• Configure exclusion rules:

  • Exclude Spaces

• Configure Audience based filtering.

  • Include audiences

  • Exclude audiences

Step 2c - Save & Sync

• Default schedule: Full crawl at first setup and once in a week, incremental sync every 4 hours, ACL runs every hour

• Configuration options:

  • No option to configure the sync schedule, however sync can be paused and resumed manually

Step 2d - Monitor

1. Monitor the initial full sync status (starts automatically) in the connector dashboard.

Crawling and sync behavior

How the connector works over time:

1.  Initial full crawl

• All the content present in the storage account is indexed during the first run

• How long it may take: Depends on the size of the content.

2. Incremental updates

• Mechanism: Based on Timestamp of previous sync.

• What changes trigger reindexing:

  • New docs created

  • Existing doc updated

  • Permissions changed

  • Docs moved or renamed

  • Docs deleted

3. Deletion and permission changes

• Deleted items are removed from index at next sync.

• Permission changes are updated at the next sync cycle.

4. Expected latency

• With the default schedule (incremental sync every 4 hours and ACL sync every hour), changes made to Confluence content are generally reflected in Simpplr search results within 4 hours of the update, and the permission lag in the system can be up-to 4hours. (as the incremental Sync every 4 hours). On top of that, there can be certain cases, where the permission sync can take up-to 7days (When the full sync is run), subject to content volume and system load.

Field mapping and search experience

1.  Search experience - How content from Confluence connector appears in search:

• Result layout: (Icon, Connector Type (Confluence), Space, Title (name) as link, Body(excerpt), owner, Updated Date, Content Type, Access Detail)

• Available filters and facets: 

  • Sources = Confluence Cloud

  • File type

  • Owner

  • Created Date

• Participation in advanced features:

  • Smart Answers / Q&A: Yes

  • Autocomplete: Yes

  • Recommendations / “Suggested for you”: N/A

  • Trending / popular results: N/A

  • Semantic / hybrid ranking: Yes

2. Limits and known limitations

Monitoring and troubleshooting

1. Connector health and monitoring - Describe where admins can see status information:

• Enterprise Search -> Connector name

• Available metrics:

  • Last sync status (Success / Warning / Failed)

  • Last sync time

  • Next scheduled sync

  • Sync Type

  • Total items indexed count

2. Common issues and resolutions. Example pattern:

   • Issue: Authentication failed, Failed to authorize with the given Api token (invalid token)
     Possible causes:

     • Incorrect Token

     • User does not have the access of Product admin

Resolution:

3. Verify and re-enter credentials

4. Confirm if the user has the access of Product admin

• Issue: 404 error, URL not found:
  Possible causes:

  • Incorrect Host URL

  • Incorrect Cloud ID

Resolution:

5. Verify the cloud ID and Host URL Format.

6. When to contact support. 

1. Authentication error persists even after trying the above-mentioned resolutions

2. Sync is stuck in the Pending state,

3. Sync is in progress but no documents are getting ingested.

4. Sync failure with cancelled error (when not cancelled manually)

5. Incomplete or Partial sync.

6. When contacting Support, include:

• Connector name and instance ID (if available)

• Organization URL

• Approximate time and date of the issue

• Error messages or screenshots

• Steps you already tried

Frequently asked questions (FAQ)

Q1. Can I connect multiple Confluence tenants or domains?
A. Functionality to connect multiple domains into the single connector has not been implemented yet.

Q2. How often does Confluence sync data?
A. The connector runs a full crawl on first setup and then once per week. Incremental sync runs every hour and ACL (permission) sync runs every hour by default.

Q3. Are comments, revisions, or version history indexed?
A. Comments and individual versions are not indexed as separate items. The connector indexes the latest file metadata, including the last updated time and updated-by user.

Q4. Does the connector index content from external guests or shared links?
A. No. Connector indexes the confluence documents and spaces only from the configured tenant. 

Q5. What happens when a user loses access to an item in Confluence Storage ?
A. The updated access permissions will be indexed during the next sync.

Note:

The permission lag in the system can be up-to 1-2hours. (As the incremental sync every hour). On top of that, there can be certain cases, where the permission sync can take up-to 7days (When the full sync is run).   

Q6. Can I exclude certain sites/teams/folders from being indexed?
A. Documents can be excluded based on the Confluence spaces. Documents can also be excluded based on file extension, size, and age. Additionally, documents can be included or excluded based on audiences.

Q7. How are deletions handled?
A. Objects deleted from the source are permanently deleted from the index.