Skip to content

Conversation

jsoriano
Copy link
Member

@jsoriano jsoriano commented Jan 13, 2025

This change moves some methods from the serverless provider to the clients and adds some additional methods that can be useful to extend the support of existing stacks.

Fleet Server:

  • New client.
  • Method to request the status of a Fleet Server.

Elasticsearch:

  • Method to request cluster info like build information.

Kibana:

  • Support deployments that don't report any version, assuming it is the latest available one.
  • Method to list agents filtered by kuery.
  • Method to get a valid enrollment token for a given policy.
  • Methods to support enrollment of Fleet servers: creation of Fleet token, creation and modification of Fleet Server hosts.
  • Method to delete Fleet outputs.
  • Build flavor is obtained when available when requesting version information.

This is part of #2298.

@jsoriano jsoriano requested a review from a team January 13, 2025 19:16
@jsoriano jsoriano self-assigned this Jan 13, 2025
@jsoriano jsoriano marked this pull request as draft January 13, 2025 19:28
@jsoriano jsoriano force-pushed the enrollment-token-apis branch from ddc3a23 to 4e1ccea Compare January 13, 2025 21:02
@jsoriano jsoriano marked this pull request as ready for review January 13, 2025 21:05
outputID := ""
if settings.LogstashEnabled {
outputID = serverless.FleetLogstashOutput
}

logger.Infof("Creating agent policy")
err = project.CreateAgentPolicy(ctx, sp.kibanaClient, options.StackVersion, outputID, settings.SelfMonitor)
_, err = createAgentPolicy(ctx, sp.kibanaClient, options.StackVersion, outputID, settings.SelfMonitor)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not for this PR, probably this method could be reused in system/tester.go to create the agent policies too.

policyToEnroll, err = r.kibanaClient.CreatePolicy(ctx, policyEnroll)

policyToTest, err = r.kibanaClient.CreatePolicy(ctx, policy)

Comment on lines -272 to -276
project, err = sp.currentProjectWithClientsAndFleetEndpoint(ctx, config)
if err != nil {
return fmt.Errorf("failed to retrieve latest project created: %w", err)
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! One less request (or two requests depending if there is default a fleetserver or not).

Comment on lines +93 to +94
Name: id,
ID: id,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this fail if tests are run in parallel? Or maybe this will not fail but it will use an unexpected output. Or maybe it is good to reuse the same output when running in parallel.

Or well, the caller is the one to take into account assigning the right/expected IDs.

Copy link
Member Author

@jsoriano jsoriano Jan 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this can be an issue if we use the same stack for different environments, for example if a serverless cluster with two environments, one with logstash and the other one without it. But it should not be an issue though while running tests in parallel with a single environment.

For reusing a stack for different executions we have more conflicts. I would leave fixing them for future improvements. One issue we have is that we are relying on default Fleet configurations (output, fleet server, fleet server policy...), and I think we could avoid it.

I think that ideally each environment should use its own resources, and they should be explicitly set in agent policies, and don't rely on defaults.

if err != nil {
return nil, fmt.Errorf("failed to parse Kibana version (%s): %w", c.versionInfo.Number, err)
// Version info may not contain any version if this is a managed Kibana.
if c.versionInfo.Number != "" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this happen in Serverless projects ? Should we set a default version in that case or just not set it at all ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this happens in serverless.

Not sure of what would be better.

In my other branch I was setting 9.0.0 (here). But it can be tricky to chose a proper version, and remember to update it if it is not something like 99.99.99.

So here I am opting by no setting it at all, what indicates that it is a deployment with an unknown version. The only known case for this is serverless, so it is in principle safe to assume that is the latest version. The problem with this approach is that we have to remember to do null checks.

An alternative I considered was to create an interface with the features that are dependant of the version, and set the proper implementation depending on the info here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So here I am opting by no setting it at all, what indicates that it is a deployment with an unknown version. The only known case for this is serverless, so it is in principle safe to assume that is the latest version.

It looks reasonable to assume that, ok 👍

jsoriano and others added 3 commits January 14, 2025 15:21
Co-authored-by: Mario Rodriguez Molins <marrodmo@gmail.com>
Co-authored-by: Mario Rodriguez Molins <marrodmo@gmail.com>
@jsoriano jsoriano force-pushed the enrollment-token-apis branch from 6cadd80 to cc789a8 Compare January 14, 2025 14:38
@jsoriano jsoriano requested a review from mrodm January 14, 2025 14:40
@elasticmachine
Copy link
Collaborator

elasticmachine commented Jan 14, 2025

💛 Build succeeded, but was flaky

Failed CI Steps

History

cc @jsoriano

@jsoriano jsoriano enabled auto-merge (squash) January 14, 2025 22:58
@jsoriano jsoriano merged commit 48f74c9 into elastic:main Jan 14, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants