How to monitor external resources with proxy requests and entities
Proxy entities allow Sensu to monitor external resources
on systems or devices where a Sensu agent cannot be installed, like a
network switch or a website.
You can create proxy entities using sensuctl, the Sensu API, or the proxy_entity_name
check attribute. When executing checks that include a proxy_entity_name
, Sensu agents report the resulting event under the proxy entity instead of the agent entity.
This guide requires a running Sensu backend, a running Sensu agent, and a sensuctl instance configured to connect to the backend as a user with read and create permissions for entities, checks, and events.
Using a proxy entity to monitor a website
In this section, we’ll monitor the status of sensu.io by configuring a check with a proxy entity name so that Sensu creates an entity representing the site and reports the status of the site under this entity.
Installing an HTTP check script
First, we’ll install a bash script, named http_check.sh
, to perform an HTTP
check using curl.
sudo curl https://raw.githubusercontent.com/sensu/sensu-go/5.1.0/examples/checks/http_check.sh \
-o /usr/local/bin/http_check.sh && \
sudo chmod +x /usr/local/bin/http_check.sh
PRO TIP: While this command may be appropriate when running a few agents, you should consider using Sensu assets or a configuration management tool to provide runtime dependencies.
Creating the check
Now that our script is installed, we’ll create a check named
check-http
, which runs the command http_check.sh https://sensu.io
, at an
interval of 60 seconds, for all entities subscribed to the proxy
subscription, using the sensu-site
proxy entity name.
Create a file called check.json
and add the following check definition.
{
"type": "CheckConfig",
"api_version": "core/v2",
"metadata": {
"name": "check-http",
"namespace": "default"
},
"spec": {
"command": "http_check.sh https://sensu.io",
"interval": 60,
"proxy_entity_name": "sensu-site",
"publish": true,
"subscriptions": [
"proxy"
]
}
}
Now we can use sensuctl to add this check to Sensu.
sensuctl create --file check.json
sensuctl check list
Name Command Interval Cron Timeout TTL Subscriptions Handlers Assets Hooks Publish? Stdin? Metric Format Metric Handlers
──────────── ──────────────────────────────── ────────── ────── ───────── ───── ─────────────── ────────── ──────── ─────── ────────── ──────── ─────────────── ─────────────────
check-http http_check.sh https://sensu.io 60 0 0 proxy true false
Adding the subscription
To run the check, we’ll need a Sensu agent with the subscription proxy
.
After installing an agent, open /etc/sensu/agent.yml
and add the proxy
subscription so the subscription configuration looks like:
subscriptions:
- "proxy"
Then restart the agent.
sudo service sensu-agent restart
Validating the check
Now we can use sensuctl to see that Sensu has created the proxy entity sensu-site
.
sensuctl entity list
ID Class OS Subscriptions Last Seen
────────────── ─────── ─────── ─────────────────────────── ───────────────────────────────
sensu-centos agent linux proxy,entity:sensu-centos 2019-01-16 21:50:03 +0000 UTC
sensu-site proxy entity:sensu-site N/A
And that Sensu is now monitoring sensu-site
using the check-http
check.
sensuctl event info sensu-site check-http
=== sensu-site - check-http
Entity: sensu-site
Check: check-http
Output:
Status: 0
History: 0,0
Silenced: false
Timestamp: 2019-01-16 21:51:53 +0000 UTC
NOTE: It might take a few moments for Sensu to execute the check and create the proxy entity.
We can also see our new proxy entity in the Sensu dashboard.
Using proxy requests to monitor a group of websites
Now let’s say that, instead of monitoring just sensu.io, we want to monitor multiple sites, for example: docs.sensu.io, packagecloud.io, and github.com.
In this section of the guide, we’ll use the proxy_requests
check attribute, along with entity labels and token substitution, to monitor three sites using the same check.
Before we get started, go ahead and install the HTTP check script if you haven’t already.
Installing an HTTP check script
If you haven’t already, install a bash script, named http_check.sh
, to perform an HTTP
check using curl.
sudo curl https://raw.githubusercontent.com/sensu/sensu-go/5.1.0/examples/checks/http_check.sh \
-o /usr/local/bin/http_check.sh && \
sudo chmod +x /usr/local/bin/http_check.sh
PRO TIP: While this command may be appropriate when running a few agents, you should consider using Sensu assets or a configuration management tool to provide runtime dependencies.
Creating proxy entities
Instead of creating a proxy entity using the proxy_entity_name
check attribute, we’ll be using sensuctl to create proxy entities to represent the three sites we want to monitor.
Our proxy entities need the entity_class
attribute set to proxy
to mark them as proxy entities as well as a few custom labels
that we’ll use to identify them as a group and pass in individual URLs.
Create a file called entities.json
and add the following entity definitions.
{
"type": "Entity",
"api_version": "core/v2",
"metadata": {
"name": "sensu-docs",
"namespace": "default",
"labels": {
"proxy_type": "website",
"url": "https://docs.sensu.io"
}
},
"spec": {
"entity_class": "proxy"
}
}
{
"type": "Entity",
"api_version": "core/v2",
"metadata": {
"name": "packagecloud-site",
"namespace": "default",
"labels": {
"proxy_type": "website",
"url": "https://packagecloud.io"
}
},
"spec": {
"entity_class": "proxy"
}
}
{
"type": "Entity",
"api_version": "core/v2",
"metadata": {
"name": "github-site",
"namespace": "default",
"labels": {
"proxy_type": "website",
"url": "https://github.com"
}
},
"spec": {
"entity_class": "proxy"
}
}
PRO TIP: When creating proxy entities, you can add whatever custom labels make sense for your environment. For example, when monitoring a group of routers, you may want to add ip_address
labels.
Now we can use sensuctl to add these proxy entities to Sensu.
sensuctl create --file entities.json
sensuctl entity list
ID Class OS Subscriptions Last Seen
─────────────────── ─────── ─────── ─────────────────────────── ───────────────────────────────
github-site proxy N/A
packagecloud-site proxy N/A
sensu-centos agent linux proxy,entity:sensu-centos 2019-01-16 23:05:03 +0000 UTC
sensu-docs proxy N/A
Creating a reusable HTTP check
Now that we have our three proxy entities set up, each with a proxy_type
and url
label, we can use proxy requests and token substitution to create a single check that monitors all three sites.
Create a file called check-proxy-requests.json
and add the following check definition.
{
"type": "CheckConfig",
"api_version": "core/v2",
"metadata": {
"name": "check-http-proxy-requests",
"namespace": "default"
},
"spec": {
"command": "http_check.sh {{ .labels.url }}",
"interval": 60,
"subscriptions": [
"proxy"
],
"publish": true,
"proxy_requests": {
"entity_attributes": [
"entity.entity_class == 'proxy'",
"entity.labels.proxy_type == 'website'"
],
"splay": true,
"splay_coverage": 90
}
}
}
Our check-http-proxy-requests
check uses the proxy_requests
attribute to specify the applicable entities.
In our case, we want to run the check-http-proxy-requests
check on all entities of entity class proxy
and proxy type website
.
To make sure that Sensu runs the check for all applicable entities, we need to set the splay attribute to true
with a splay coverage percentage value of 90
.
This gives Sensu 90% of the check interval
, 60 seconds in this case, to execute the check for all applicable entities.
Since we’re using this check to monitor multiple sites, we can use token substitution to apply the correct url
in the check command
.
Now we can use sensuctl to add this check to Sensu.
sensuctl create --file check-proxy-requests.json
sensuctl check list
Name Command Interval Cron Timeout TTL Subscriptions Handlers Assets Hooks Publish? Stdin? Metric Format Metric Handlers
─────────────────────────── ───────────────────────────────── ────────── ────── ───────── ───── ─────────────── ────────── ──────── ─────── ────────── ──────── ─────────────── ─────────────────
check-http http_check.sh https://sensu.io 60 0 0 proxy true false
check-http-proxy-requests http_check.sh {{ .labels.url }} 60 0 0 proxy true false true false
Validating the check
Before validating the check, make sure that you’ve added the proxy
subscription to a Sensu agent if you haven’t already.
Now we can use sensuctl to see that Sensu is monitoring docs.sensu.io, packagecloud.io, and github.com using the check-http-proxy-requests
.
sensuctl event list
Entity Check Output Status Silenced Timestamp
─────────────────── ─────────────────────────── ──────── ──────── ────────── ───────────────────────────────
github-site check-http-proxy-requests 0 false 2019-01-17 17:10:31 +0000 UTC
packagecloud-site check-http-proxy-requests 0 false 2019-01-17 17:10:34 +0000 UTC
sensu-centos keepalive 0 false 2019-01-17 17:10:34 +0000 UTC
sensu-docs check-http-proxy-requests 0 false 2019-01-17 17:06:59 +0000 UTC
Next steps
You now know how to run a proxy check to verify the status of a website, as well as using proxy requests to run a check on two different proxy entities based on label evaluation. From this point, here are some recommended resources:
- Read the proxy checks reference for in-depth documentation on proxy checks.
- Read the guide to providing runtime dependencies to checks with assets.
- Read the guide to sending alerts to Slack with handlers.