Elasticsearch

Connect an Elasticsearch cluster to let CloudThinker agents list indices, read field mappings, run DSL searches and ES|QL queries, and check shard allocation while they investigate an incident. The connection signs in with an API key or a username and password, and every tool it exposes is a read.

Prerequisites

An Elasticsearch cluster with an HTTPS endpoint CloudThinker can reach over the network, such as an Elastic Cloud deployment or a public DNS name. localhost and private addresses aren’t reachable from CloudThinker Cloud.
If a firewall or Elastic Cloud traffic filter guards the cluster, allow CloudThinker’s egress IPs. These docs don’t publish CloudThinker’s egress IP list: ask your CloudThinker contact, or contact us. See Reaching private services for other options.
Permission to create an API key (Kibana Stack Management → Security → API keys, or the _security/api_key API) or a dedicated user.
The index patterns agents should read, such as logs-* or filebeat-*.

Setup

Create a read-only API key

In Kibana, open Dev Tools and run the request below. Replace logs-* with your own index patterns.

POST /_security/api_key
{
  "name": "cloudthinker",
  "expiration": "90d",
  "role_descriptors": {
    "cloudthinker_read": {
      "cluster": ["monitor"],
      "indices": [
        {
          "names": ["logs-*"],
          "privileges": ["read", "view_index_metadata", "monitor"]
        }
      ]
    }
  }
}

Copy the encoded value from the response. Elasticsearch shows it once.You can also create the key in Stack Management → Security → API keys by clicking Create API key, turning on Control security privileges, and pasting the same role_descriptors object.

Copy the cluster endpoint

Use the full URL with scheme and port, such as https://elasticsearch.example.com:9200. On Elastic Cloud, copy the Elasticsearch endpoint from the deployment page.

Add the connection in CloudThinker

Open More → Workspace settings → Connections and select Elasticsearch. Enter the endpoint in ES_URL, choose API key under Authentication, and paste the encoded key. Leave TLS certificate on Verify unless the cluster serves a self-signed certificate. Click Connect.Success state: the connection shows Connecting and Verifying, then Connected. If the test fails, the connection shows an error with the message the cluster returned. Match it against Troubleshooting.

Connection details

Field	Required	Description	Example
ES_URL	Yes	Cluster endpoint, including scheme and port	`https://elasticsearch.example.com:9200`
Authentication	Yes	API key (default) or Username & password	API key
API key	API key only	The `encoded` value from the create response	`VnVhQ2ZHY0JDZGJr...`
Username	Username & password only	A dedicated Elasticsearch user	`cloudthinker`
Password	Username & password only	That user’s password
TLS certificate	No	Verify (default) or Allow self-signed	Verify

Fill in one method only. If you enter both an API key and a username or password, the connection is rejected with “Choose either an API key or username and password.”

Required permissions

Agents need to	Grant
Check cluster health	Cluster privilege `monitor`
List indices and read shard allocation	Index privilege `monitor` on your patterns
Read field mappings	Index privilege `view_index_metadata` on your patterns
Run DSL searches and ES\|QL queries	Index privilege `read` on your patterns

If you use Username & password, create a role with the same privileges and a user that holds it. Run each request separately in Dev Tools:

POST /_security/role/cloudthinker_read
{
  "cluster": ["monitor"],
  "indices": [
    {
      "names": ["logs-*"],
      "privileges": ["read", "view_index_metadata", "monitor"]
    }
  ]
}

POST /_security/user/cloudthinker
{
  "password": "<a long random password>",
  "roles": ["cloudthinker_read"]
}

List the index patterns agents need instead of *. A key scoped to logs-* can’t read a customers index, even if a search asks for it.

Agent capabilities

Capability	What agents can do
Index inventory	List indices with their status and document count, and spot naming patterns such as `logs-app-*`
Mappings	Read field names and types before filtering, so queries use fields that exist
Search	Run bounded DSL queries with a time range, sort, and result size
ES\|QL	Aggregate with ES\|QL, such as error counts per service, always with a `LIMIT`
Shards and health	Find unassigned shards and report cluster health as green, yellow, or red

The connection has no tool that writes, deletes, or changes cluster settings.

Verify the connection

Check Elasticsearch cluster health and list the five largest indices by document count

Expect a health status, a shard count with any unassigned shards, and five index names with document counts.

Example prompts

Count ERROR log lines per service in logs-app-* over the last hour
Search logs-* for authentication failures in the last 24 hours and group them by source IP
Why is the cluster yellow? List the unassigned shards and the indices they belong to

Troubleshooting

Connection fails with a timeout or connection refused

ES_URL points at an address CloudThinker can’t reach. Use a public endpoint, include the scheme and port, and check that your firewall or Elastic Cloud traffic filters allow CloudThinker’s egress IPs (see Prerequisites).

401 Unauthorized

The API key is wrong, expired, or was invalidated, or the password changed. Create a new key and replace the credential with its encoded value, not the id or the api_key field alone.

security_exception on one index

The key or user has no privileges on that index. Add its pattern to names in a new key, then replace the credential on the connection.

Certificate errors on a self-hosted cluster

The cluster serves a self-signed certificate. Set TLS certificate to Allow self-signed only if you trust that cluster.

A search returns nothing

The index pattern doesn’t match, the field name differs from what the prompt assumed, or the time range is empty. Ask the agent to list indices and read the mapping for the index first.

Security

Least privilege — grant only the permissions the agents need for your use case; start read-only and widen later.
Read-only by default — use read-only credentials unless you want agents to make changes through this connection.
Rotate credentials — rotate keys and tokens on your normal schedule; CloudThinker picks up the new value when you update the connection.
Revoke on offboarding — remove the credential at the provider when you delete a connection or a teammate leaves.

Set an expiration on the key: an expiration such as 90d limits how long a leaked key works. Create a new key before it expires, then open the connection, click the pencil icon on its row, and in Edit Instance click Replace…. Paste the new key and click Save and test. If the test fails, CloudThinker keeps the old key, so agents don’t lose access during the swap.
Treat indexed content as sensitive: logs can carry tokens or personal data that an application wrote. Scope the key to the indices you want agents to read.

Root cause analysis

See how agents use log searches to find what caused an incident.

Connections overview

Rotate credentials, reach private services, and see every connection.

Connections

Customize

Approvals

Workspace settings

Admin Settings

Artifacts and sharing

Prerequisites

Setup

Connection details

Required permissions

Agent capabilities

Verify the connection

Example prompts

Troubleshooting

Security

Root cause analysis

Connections overview

​Prerequisites

​Setup

​Connection details

​Required permissions

​Agent capabilities

​Verify the connection

​Example prompts

​Troubleshooting

​Security

​Related

Root cause analysis

Connections overview

Prerequisites

Setup

Connection details

Required permissions

Agent capabilities

Verify the connection

Example prompts

Troubleshooting

Security

Related