Migrate from Sensu Core and Sensu Enterprise to Sensu Go

This guide includes general information for migrating your Sensu instance from Sensu Core and Sensu Enterprise to Sensu Go. For instructions and tools to help you translate your Sensu configuration from Sensu Core and Enterprise to Sensu Go, review the Sensu Translator project.

Sensu Go includes important changes to all parts of Sensu: architecture, installation, resource definitions, the observation data (event) model, check dependencies, filter evaluation, and more. Sensu Go also includes many powerful commercial features to make monitoring easier to build, scale, and offer as a self-service tool to your internal customers.

Sensu Go is available for Debian- and RHEL-family distributions and Docker. The Sensu Go agent is also available for Windows.

Aside from this migration guide, these resources can help you migrate from Sensu Core or Sensu Enterprise to Sensu Go:

• Sensu Community Slack: Join hundreds of other Sensu users in our Community Slack, where you can ask questions and benefit from tips others picked up during their own Sensu Go migrations.
• Sensu Community Forum: Drop a question in our dedicated category for migrating to Go.
• Sensu Go workshop: Download the workshop environment and try out some monitoring workflows with Sensu Go.
• Sensu Translator: Use this command-line tool to generate Sensu Go configurations from your Sensu Core config files.

We also offer commercial support and professional services packages to help with your Sensu Go migration.

Configuration management with Ansible, Chef, and Puppet

Configuration management integrations for Sensu Go are available for Ansible, Chef, and Puppet:

• Ansible collection for Sensu Go and documentation site
• Chef cookbook for Sensu Go — contact us for more information
• Puppet module for Sensu Go

Packaging

Sensu Go is provided as three packages: sensu-go-backend, sensu-go-agent, and sensu-go-cli (sensuctl). This is a fundamental change in Sensu terminology from Sensu Core: in Sensu Go, the server is now the backend.

Clients are represented within Sensu Go as abstract entities that can describe a wider range of system components such as network gear, a web server, or a cloud resource.

Read Sensu concepts and terminology to learn more about new terms in Sensu Go.

Architecture

In Sensu Go, an embedded transport and etcd datastore replace the external RabbitMQ transport and Redis datastore in Sensu Core.

Single Sensu Go backend or standalone architecture

In Sensu Go, the Sensu backend and agent are configured with YAML files or the sensu-backend or sensu-agent command line tools rather than JSON files. Sensu checks and pipeline elements are configured via the API or sensuctl tool in Sensu Go instead of JSON files.

The Sensu backend is powered by an embedded transport and etcd datastore and gives you flexible, automated workflows to route metrics and alerts. Sensu backends require persistent storage for their embedded database, disk space for local dynamic runtime asset caching, and several exposed ports:

• 2379 (gRPC) Sensu storage client: Required for Sensu backends using an external etcd instance
• 2380 (gRPC) Sensu storage peer: Required for etcd cluster members to communicate directly with their peers
• 3000 (HTTP/HTTPS) Sensu web UI: Required for all Sensu backends using a Sensu web UI
• 8080 (HTTP/HTTPS) Sensu API: Required for all users accessing the Sensu API
• 8081 (WS/WSS) Agent API: Required for all Sensu agents connecting to a Sensu backend

Sensu agents are lightweight clients that run on the infrastructure components you want to monitor. Agents automatically register with Sensu as entities and are responsible for creating check and metric events to send to the backend event pipeline.

The Sensu agent uses:

• 3030 (TCP/UDP) Sensu agent socket: Required for Sensu agents using the agent socket
• 3031 (HTTP) Sensu agent API: Required for all users accessing the agent API
• 8125 (UDP) StatsD listener: Required for all Sensu agents using the StatsD listener

The agent TCP and UDP sockets are deprecated in favor of the agent API.

Agents that use Sensu dynamic runtime assets require some disk space for a local cache.

Read the backend, agent, and sensuctl reference docs for more information.

Entities

Clients are represented within Sensu Go as abstract entities that can describe a wide range of system components such as network gear, a web server, or a cloud resource.

Sensu Go includes agent entities that run a Sensu agent and the familiar proxy entities. Sensu Go also includes service entities, which represent business services in the business service monitoring (BSM) feature.

Read the entities reference and the guide to monitoring external resources for more information about Sensu Go entities.

Checks

In Sensu Go, checks work with Sensu agents to produce observability events automatically. The Sensu backend coordinates check execution by comparing the subscriptions specified in check and entity definitions to determine which entities should receive execution requests for a given check.

Subdue

Sensu Go checks include a subdues attribute that allows you to set specific periods of time when Sensu will not execute the check. Read Subdues in the checks reference for more information and examples.

You can also use cron scheduling in Sensu Go checks to specify when checks should be executed.

Standalone checks

Sensu Go does not include standalone checks. Read Self-service monitoring checks in Sensu Go to learn more about using role-based access control (RBAC), dynamic runtime assets, and entity subscriptions to achieve similar functionality to Sensu Core’s standalone checks in Sensu Go.

Check hooks

Check hooks are a distinct resource type in Sensu Go, which allows you to create, manage, and reuse hooks independently of check definitions. You can also execute multiple hooks for any given response code.

Default handler

Sensu Go does not try to run a default handler when executing checks whose definitions do not specify a handler name. In Sensu Go, you explicitly add the name of a handler in a pipeline and reference the pipeline in your check definition.

Events

In Sensu Go, all check results are considered events and are processed by pipelines, which include event filters, mutators, and handlers.

Use Sensu Go’s built-in is_incident filter to recreate the Sensu Core behavior in which only check results with a non-zero status are considered events.

Handlers

Sensu Go includes pipe and TCP/UDP handlers, but not transport handlers. To create similar functionality to transport handlers in Sensu Go, create a pipe handler that connects to a message bus and injects event data into a queue.

Sensu Go also includes streaming handlers, such as the Sumo Logic metrics handler, to provide persistent connections for transmitting Sensu observation data to remote data storage services to help prevent data bottlenecks.

Filters

In Sensu Go, JavaScript expressions replace the Ruby eval logic in Sensu Core, opening up powerful ways to filter events based on occurrences and other event attributes. As a result, Sensu Go does not include the built-in occurrence-based event filter in Sensu Core. To replicate the Sensu Core occurrence-based filter’s functionality, use Sensu Go’s repeated events filter definition.

Sensu Go includes three built-in event filters: is_incident, not_silenced, and has_metrics. Sensu Go does not include a built-in check dependencies filter, but you can use the sensu/sensu-dependencies-filter dynamic runtime asset to replicate the built-in check dependencies filter functionality from Sensu Core.

Sensu Go event filters do not include the when event filter attribute. Use Sensu query expressions to build custom functions that provide granular control of time-based filter expressions.

Fatigue check filter

The sensu/sensu-go-fatigue-check-filter dynamic runtime asset is a JavaScript implementation of the occurrences filter from Sensu Core. This filter looks for check and entity annotations in each event it receives and uses the values of those annotations to configure the filter’s behavior on a per-event basis.

The Sensu Translator version 1.1.0 retrieves occurrence and refresh values from a Sensu Core check definition and outputs them as annotations in a Sensu Go check definition, compatible with the fatigue check filter.

However, the Sensu Translator doesn’t automatically add the sensu/sensu-go-fatigue-check-filter dynamic runtime asset or the filter configuration you need to run it. To use the sensu/sensu-go-fatigue-check-filter dynamic runtime asset, you must register it, create a correctly configured event filter definition, and add the event filter to the list of filters on applicable handlers.

Dynamic runtime assets

The sensu-install tool in Sensu Core is replaced by dynamic runtime assets in Sensu Go. Dynamic runtime assets are shareable, reusable packages that make it easier to deploy Sensu plugins.

You can still install Sensu Community plugins in Ruby via sensu-install by installing sensu-plugins-ruby. Read Install plugins for more information.

Role-based access control (RBAC)

Role-based access control (RBAC) is a built-in feature of the open-source version of Sensu Go. RBAC allows you to manage and access users and resources based on namespaces, groups, roles, and bindings. To set up RBAC in Sensu Go, read the RBAC reference and Create a read-only user.

Silencing

Silencing is disabled by default in Sensu Go. You must explicitly enable silencing by creating silencing resource definitions with sensuctl, the Sensu web UI, or core/v2/silenced API endpoints. Read the Sensu Go silencing reference for more information.

Token substitution

The syntax for token substitution changed to double curly braces in Sensu Go (from triple colons in Sensu Core).

Aggregates

Sensu Go supports check aggregates with the sensu/sensu-aggregate-check dynamic runtime asset.

API

In addition to the changes to resource definitions, Sensu Go includes new versioned APIs. Read the API overview for more information.

Step-by-step migration instructions

Step 1: Install Sensu Go

1. Install the Sensu Go backend

The Sensu backend is available for Debian- and RHEL-family distributions and Docker. Read the installation guide to install, configure, and start the Sensu backend according to your deployment strategy.

2. Log in to the Sensu web UI

The Sensu Go web UI provides a unified view of your observability events with user-friendly tools to reduce alert fatigue and manage your Sensu instance. After starting the Sensu backend, open the web UI by visiting http://localhost:3000. You may need to replace localhost with the hostname or IP address where the Sensu backend is running.

To log in, enter your Sensu user credentials or use Sensu’s default admin credentials (username: admin and password: P@ssw0rd!).

3. Install sensuctl on your workstation

Sensuctl is a command line tool for managing resources within Sensu. It works by calling Sensu’s HTTP API to create, read, update, and delete resources, events, and entities. Sensuctl is available for Linux, Windows, and macOS. Read the installation guide to install and configure sensuctl.

4. Set up Sensu users

Use Sensu’s built-in RBAC to manage and access users and resources based on namespaces, groups, roles, and bindings. To set up RBAC in Sensu Go, read the RBAC reference and Create a read-only user.

In Sensu Go, namespaces partition resources within a Sensu instance. Sensu Go entities, checks, handlers, and other namespaced resources belong to a single namespace. The Sensu translator places all translated resources into the default namespace — we’ll use the translater in a moment.

In addition to built-in RBAC, Sensu Go’s commercial features include support for authentication using Microsoft Active Directory (AD) and standards-compliant Lightweight Directory Access Protocol tools like OpenLDAP.

5. Install agents

The Sensu agent is available for Debian- and RHEL-family distributions, Windows, and Docker. Read the installation guide to install, configure, and start Sensu agents.

If you’re doing a side-by-side migration, add api-port (default: 3031) and socket-port (default: 3030) to your agent configuration (/etc/sensu/agent.yml or C:\ProgramData\sensu\config\agent.yml.example). This prevents the Sensu Go agent API and socket from conflicting with the Sensu Core client API and socket.

api-port: 3031
socket-port: 3030

You can also disable these features in the agent configuration using the disable-socket and disable-api configuration options.

Sensu should now be installed and functional. The next step is to translate your Sensu Core configuration to Sensu Go.

Step 2: Translate your configuration

Use the Sensu Translator command line tool to transfer your Sensu Core checks, handlers, and mutators to Sensu Go.

1. Run the translator

Install dependencies:

yum install -q -y rubygems ruby-devel

Install the Sensu translator:

gem install sensu-translator

Run the Sensu translator to translate all configuration in /etc/sensu/conf.d to Sensu Go and output to /sensu_config_translated:

sensu-translator -d /etc/sensu/conf.d -o /sensu_config_translated

As an option, you can also translate your configuration in sections according to resource type.

If translation is successful, you should receive a few callouts followed by DONE!, similar to this example:

Sensu 1.x filter translation is not yet supported
Unable to translate Sensu 1.x filter: only_production {:attributes=>{:check=>{:environment=>"production"}}}
DONE!

Combine your config into a sensuctl-readable format.

find sensu_config_translated/ -name '*.json' -exec cat {} \; > sensu_config_translated_singlefile.json

Most attributes are ready to use as-is, but you’ll need to adjust your Sensu Go configuration manually to migrate some of Sensu’s features.

2. Translate checks

Review your Sensu Core check configuration for the following attributes, and make the corresponding updates to your Sensu Go configuration.

Translate metric checks

The Sensu Core type: metric attribute is not part of the Sensu Go check spec, so you’ll need to adjust it manually. Sensu Core checks could be configured as type: metric, which told Sensu to always handle the check regardless of the check status output. This allowed Sensu Core to process output metrics via a handler even when the check status was not in an alerting state.

Sensu Go treats output metrics as first-class objects, so you can process check status as well as output metrics via different event pipelines. Read the guide to metric output to update your metric checks with the output_metric_handlers and output_metric_format attributes and use output_metric_tags to enrich extracted metrics output.

Translate proxy requests and proxy entities

Read Monitor external resources to re-configure proxy_requests attributes and update your proxy check configuration. read the entities reference to re-create your proxy client configurations as Sensu Go proxy entities.

Translate hooks

Check hooks are now a resource type in Sensu Go, so you can create, manage, and reuse hooks independently of check definitions. You can also execute multiple hooks for any given response code. Read the guide and hooks reference docs to re-create your Sensu Core hooks as Sensu Go hook resources.

Custom attributes

Instead of custom check attributes, Sensu Go allows you to add custom labels and annotations to entities, checks, dynamic runtime assets, hooks, filters, mutators, handlers, and silences. Review the metadata attributes section in the reference documentation for more information about using labels and annotations (for example, metadata attributes for entities).

The Sensu Translator stores all check extended attributes in the check metadata annotation named sensu.io.json_attributes. Read the checks reference for more information about using labels and annotations in check definitions.

3. Translate event filters

Ruby eval logic used in Sensu Core filters is replaced with JavaScript expressions in Sensu Go, opening up powerful possibilities to combine filters with filter dynamic runtime assets. As a result, you’ll need to rewrite your Sensu Core filters in Sensu Go format.

First, review your Core handlers to identify which filters are being used. Then, follow the event filters reference and guide to using filters to re-write your filters using Sensu Go expressions and event data. Check out the blog post on filters for a deep dive into Sensu Go filter capabilities.

Sensu Core hourly filter:

{
  "filters": {
    "recurrences": {
      "attributes": {
        "occurrences": "eval: value == 1 || value % 60 == 0"
      }
    }
  }
}

Sensu Go hourly filter:

---
type: EventFilter
api_version: core/v2
metadata:
  name: hourly
spec:
  action: allow
  expressions:
  - event.check.occurrences == 1 || event.check.occurrences % (3600 / event.check.interval) == 0
  runtime_assets: null

{
  "type": "EventFilter",
  "api_version": "core/v2",
  "metadata": {
    "name": "hourly"
  },
  "spec": {
    "action": "allow",
    "expressions": [
      "event.check.occurrences == 1 || event.check.occurrences % (3600 / event.check.interval) == 0"
    ],
    "runtime_assets": null
  }
}

4. Translate handlers

In Sensu Go, all check results are considered events and are processed by event handlers. Use the built-in is_incident filter to recreate the Sensu Core behavior, in which only check results with a non-zero status are considered events.

Review your Sensu Core check configuration for the following attributes and make the corresponding updates to your Sensu Go configuration.

5. Upload your config to your Sensu Go instance

After you review your translated configuration, make any necessary updates, and add resource definitions for any filters and entities you want to migrate, you can upload your Sensu Go config using sensuctl.

sensuctl create --file /path/to/config.json

Access your Sensu Go config using the Sensu API.

Set up a local API testing environment by saving your Sensu credentials and token as environment variables. This command requires curl and jq.

export SENSU_USER=admin && SENSU_PASS=P@ssw0rd!
export SENSU_TOKEN=`curl -XGET -u "$SENSU_USER:$SENSU_PASS" -s http://localhost:8080/auth | jq -r ".access_token"`

Return a list of all configured checks:

curl -H "Authorization: Bearer $SENSU_TOKEN" http://127.0.0.1:8080/api/core/v2/namespaces/default/checks

Return a list of all configured handlers:

curl -H "Authorization: Bearer $SENSU_TOKEN" http://127.0.0.1:8080/api/core/v2/namespaces/default/handlers

You can also access your Sensu Go configuration in JSON or YAML using sensuctl. For example, sensuctl check list --format wrapped-json. Run sensuctl help To view available commands. For more information about sensuctl’s output formats (json, wrapped-json, and yaml), read the sensuctl reference.

Step 3: Translate plugins and register dynamic runtime assets

Sensu plugins

Within the Sensu Plugins org, review individual plugin READMEs for compatibility status with Sensu Go. For handler and mutators plugins, review the Sensu plugins README to map event data to the Sensu Go event format. This allows you to use Sensu plugins for handlers and mutators with Sensu Go without re-writing them.

To re-install Sensu plugins onto your Sensu Go agent nodes (check plugins) and backend nodes (mutator and handler plugins), read the guide to installing the sensu-install tool for use with Sensu Go.

Sensu Go dynamic runtime assets

The sensu-install tool in Sensu Core is replaced by dynamic runtime assets in Sensu Go. Dynamic runtime assets are shareable, reusable packages that make it easier to deploy Sensu plugins.

Although dynamic runtime assets are not required to run Sensu Go, we recommend using assets to install plugins where possible. You can still install Sensu Community plugins in Ruby via sensu-install by installing sensu-plugins-ruby. Read Install plugins for more information.

Sensu supports dynamic runtime assets for checks, filters, mutators, and handlers. Discover, download, and share dynamic runtime assets with Bonsai, the Sensu asset hub.

To create your own dynamic runtime assets, read the asset reference and guide to sharing an asset on Bonsai. To contribute to converting a Sensu plugin to a dynamic runtime asset, read Contributing Assets for Existing Ruby Sensu Plugins at the Sensu Community Forum on Discourse.

Step 4: Translate Sensu Enterprise-only features

Integrations

Most Sensu Enterprise integrations as available as Sensu Go assets. Read the guide to installing plugins with assets to register assets with Sensu and update your Sensu Go handler definitions.

• Chef
• Email
• Graphite
• InfluxDB
• IRC
• Jira
• PagerDuty
• ServiceNow
• Slack
• VictorOps

Contact routing

Contact routing is available in Sensu Go woth the sensu/sensu-go-has-contact-filter dynamic runtime asset. Read Route alerts with event filters to set up contact routing in Sensu Go.

LDAP

In addition to built-in RBAC, Sensu includes license-activated support for authentication using Microsoft Active Directory and standards-compliant Lightweight Directory Access Protocol tools like OpenLDAP.

Step 5: Sunset your Sensu Core instance

When you’re ready to sunset your Sensu Core instance, stop the Sensu Core services according to the instructions for your platform — these instructions are listed under Operating Sensu on each platform’s page.

After you stop the Sensu Core services, follow package removal instructions for your platform to uninstall the Sensu Core package.