Arranger Documentation¶

Note: This project is undergoing refactoring work, for what will become version 3+. Covering great feedback we’ve received from the community, there will be some breaking changes, and an upgrade guide is already in the works.

Introduction¶

What is Arranger?¶

Arranger is a collection of reusable components for creating “-centric” search portals with Elasticsearch. Arranger consists of the following components:

Arranger Search API provides a layer that sits above your Elasticsearch cluster to expose a data-model aware GraphQL API, generated from your own Elasticsearch index mapping.
Arranger Components provides a rich set of UI components that are configured to speak to the search API.
Arranger Admin provides the API and UI for configuring the search API and content management for the search portal.

Arranger is one of many products provided by Overture and is completely open-source and free for everyone to use.

Features¶

GraphQL API for query flexibility.
SQON query filter notation balances between human-interpretability and machine-readability to simply search.
Admin UI for API configuration and content management.
Configuration import and export for easy migration.

License¶

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License along with this program. If not, see https://www.gnu.org/licenses.

Getting Started¶

The easiest way to understand Arranger, is to simply use it!

Below is a description of how to get Arranger quickly up and running, as well as a description of how Arranger works and some important terms.

Quick Start¶

The goal of this quick start is to get a working application quickly up and running.

Using Docker:

Download the latest version of Arranger.
From the Arranger root directory, run docker-compose:

$ docker-compose up -d

Arranger should now be deployed locally.

Alternatively, see the Installation instructions.

How Arranger Works¶

1. Starting with some Elasticsearch (ES) indices with mappings.

Arranger makes no assumption about your data model.
Model your index mappings and index them.
For demo convenience, you can follow a tutorial bellow to index some test data from our Kids First project.

Architecture¶

Indexing Demo Data¶

From your browser, visit the locally running Kibana at http://localhost:5601 and go to Dev Tools
Creating a file_centric index:
- Run these commands to create a file_centric index and add a mapping then these commands to index some demo documents into the index
- Run these commands to create a participant_centric index and add a mapping then these commands to index some demo documents into the index
You can run GET file_centric/_mapping and GET participant_centric/_mapping to confirm that the mapping has been created successfully

Arranger for Administrators¶

Tutorial¶

To administer Arranger, the admin must:

1. Install Arranger.

View the installation instructions.

2. Have an Elasticsearch mapping and data indexed to search.

View the Indexing Demo Data for a demo setup.

3. Admin registers the indices with arranger through the admin UI and apply configurations.

Using the Admin UI¶

The arranger UI reflects the following pseudo entity relationship:

1) Projects:

This page lists the available projects and provides an interface for registering new projects

Available functionalities:

Adding a new project

Removing existing project

Export configuration data (exported data can then be imported into new projects to migrate data).

Clicking on a project id will navigate to that project’s list of indices.

2) Indices:

This page lists the indices registered to Arranger under the selected project.

Clicking on an index name will navigate to the configuration page for the index. The following configurations are available:

a) Fields configurations

This lists all fields available in the index and allows configuration of Arranger metadata for these fields, including:

Display Name: how the field should be displayed to user.

Aggregation Type: lets the search portal know how to display aggregation filters for the field.

Active: this field is DEPRECATED

Quicksearch enabled: whether the field is enabled for quicksearch using the @arranger/components’s QuickSearch component.

Is primary key: check if the field is the unique identifier for the index’s main entity.

Is array: check if the field is an array. Elasticsearch’s mapping does not specify this information.

For convenience, filtering on the fields can be done through the inputs above the header.

b) Facet panel configurations

This lists all available aggregations on the fields mentioned. On Arranger’s default portal UI, this list is rendered as a facet panel. Each entry on Supported configutations:

Ordering the facets: drag the facet on its “hamberger menu icon” to place the facet at the desired position. Alternatively, the position can also be set through the select menu beside the icon.

Shown: displays the facet in the portal’s facet panel.

Active: enables this facet for search. An Active facet will appear in the AdvancedFacetView component in @arranger/components. Only facets that are both Active and Shown will be shown in the portal’s facet panel.

c) Data table configurations

This contains configuration for the data table in the default portal. Each entry in the list represents a column in the data table. Available configurations:

Column order: positioning can be done by dragging or using the select, similar to the facet panel.

Active: enables this column to be viewed in the table. Does not show by default.

Default: shows this column by default. Can only be checked if Active is checked.

Sortable: enables sorting of the table on this field.

d) Quick search configurations

This contains configuration for the portal’s quick-search feature, which allows users to filter indexed entities by text. Currently, Arranger only supports exact match on quicksearch, but free-text search is in our roadmap to support. This feature can be exposed to end-users throught the QuickSearch UI component from @arranger/components.

Only entities (in other words, the root object and its “nested” fields in Elasticsearch) are available for quick search.

Available configurations:

Display Name: the name to display this field as.

Active: check to enable search for this entity.

Key Field: the unique field that identifies each instance of this entity.

Search Field: the properties of the entity to enable search on.

SQON Filters¶

Arranger uses a custom JSON object format for filtering that is called SQON (Serializable Query Object Notation, pronounced like “Scone”). SQON provides a flexible system for combining many different filters.

A SQON object consists of nested objects of two types: Operations and Values.

Operation objects apply boolean logic to a list of operation objects. They are of the form:

Combination Operation (aka, Boolean Operation)

which groups one or more filters

{
  "content":[] // List of Operation objects that the boolean operation will apply to
  "op":"", // Operation to apply to content ["and", "or", "not"]
}

OR

Field Operation

that applies to a filter to Value Object

{
  "content":{} // Value object specifying the field and list of values that the field must be "in" or "not-in"
  "op":"", // Operation to apply to content ["in", "<=", ">="]
}

Value objects

specify a list the field name and values for it that the wrapping . This filter can specify to include or exclude fields with any of the listed values. It will have the following format:

{
  "field":"", // name of the field this operation applies to
  "value":[] // List of values for the field if using the "in" operation, or a scalar value for ">=" and "<=" operations
}

The top level of a SQON must always be a Combination Operation, even if only a single filter is being applied.

Sample¶

{
  content: [
    {
      content: [
        {
          content: {
            field: "id",
            value: [
              "id123"
            ]
          },
          op: "in"
        }
      ],
      op: "or"
    },
    {
      content: {
        field: "id",
        value: [
          "id123"
        ]
      },
      op: "in"
    }
  ],
  op: "and"
}

Contribute¶

If you’d like to contribute to this project, it’s hosted on github.

See https://github.com/overture-stack/arranger