[Enhancement]: Allow setting up ElasticsearchContainer with HTTP

[Jsehahn]

Problem

The Elasticsearch container is setup with HTTPS by default, which forces you to do clunky stuff like this:

/// <summary>
/// Get the Elasticsearch SSL-certificate file from the container, so it can be passed along
/// to dependent container(s) required to connect to Elasticsearch
/// </summary>
private static async Task<byte[]> ExportCertificateAsync(ElasticsearchContainer container)
{
    return await container.ReadFileAsync("/usr/share/elasticsearch/config/certs/http_ca.crt")
        .ConfigureAwait(false);
}

We can work around this behaviour by disabling xpack security, like so:

  private ElasticsearchContainer BuildElasticSearchContainer()
  {
      return new ElasticsearchBuilder()
          .WithImage(ElasticsearchImageVersion)
          .WithNetwork(_network)
          .WithNetworkAliases(ElasticsearchContainerName)
          .WithEnvironment("xpack.security.enabled", "false")
          .WithEnvironment("xpack.security.http.ssl.enabled", "false")
          .Build();
    }

But because ElasticsearchContainer.GetConnectionString() always created an HTTPS endpoint, and this is a sealed class, we're left with needing work-arounds.

  public string GetConnectionString()
  {
      var endpoint = new UriBuilder(Uri.UriSchemeHttps, Hostname, GetMappedPublicPort(ElasticsearchBuilder.ElasticsearchHttpsPort));
      endpoint.UserName = _configuration.Username;
      endpoint.Password = _configuration.Password;
      return endpoint.ToString();
  }

This is one of the workarounds:

public static class ElasticsearchContainerExtensions
{
    public static string GetHttpConnectionString(this ElasticsearchContainer container) =>
        container.GetConnectionString().Replace("https://", "http://");
}

...

var clientSettings = new ElasticsearchClientSettings(new Uri(_elasticsearchContainer.GetHttpConnectionString()));

Solution

My preferred solution would be to Implement an extension method on the ElasticsearchBuilder, called WithHttp or something. This disables the relevant xpacks and sets the correct connection string.

Benefit

Easily set up an HTTP connection without tricky workarounds that I don't fully trust. A secure HTTPS connection is hardly required for a test container, and this saves you from having to pass certificates around (which is also hacky).

Alternatives

As mentioned, there is a workaround.

Would you like to help contributing this enhancement?

Yes

[HofmeisterAn]

Thanks for creating the issue. But can't you just configure the Elasticsearch client the same way we do in our tests? Or are you using a different client?

testcontainers-dotnet/tests/Testcontainers.Elasticsearch.Tests/ElasticsearchContainerTest.cs

Lines 24 to 25 in b1244cc

	var clientSettings = new ElasticsearchClientSettings(new Uri(_elasticsearchContainer.GetConnectionString()));
	clientSettings.ServerCertificateValidationCallback(CertificateValidations.AllowAll);

[Jsehahn]

I've tried that, but can't get it to work. The other containers in the cluster aren't able to communicate with the Elastic Client.

[HofmeisterAn]

So my idea is to check whether SSL is enabled in the GetConnectionString() method, e.g. by checking the environment variables, and then return the appropriate connection string.

Just keep in mind that for container to container communication, you cannot use this method, you need to manually build the connection string using the network alias.

WDYT?

[jessehahn]

Sorry for the late reply. When you say environment variables, do you mean these two?

 .WithEnvironment("xpack.security.enabled", "false")
 .WithEnvironment("xpack.security.http.ssl.enabled", "false")

And if so, why not add an extension method like WithHtpp() that sets those variables for you?

[HofmeisterAn]

And if so, why not add an extension method like WithHtpp() that sets those variables for you?

We don't offer APIs for every possible configuration because there are simply too many, and it's too difficult to support them across different versions. Our modules are opinionated and follow best practices to support most common use cases. They won't cover everything, but developers can override the configuration or fall back to the generic container builder if needed.

For this issue, I'll update the string GetConnectionString() method to support configurations where security is disabled. That said, I still think it's better to extract the certificate as suggested in the documentation (something we can also build in OOB).

[jessehahn]

We don't offer APIs for every possible configuration because there are simply too many, and it's too difficult to support them across different versions.

Alright, I understand. Thanks for implementing the other solution!