Reuse containers

[tisonkun]

Like https://java.testcontainers.org/features/reuse/

I currently use bollard directly to reuse containers between tests, while noticed that testcontainers encapsulation is valuable and ergonomic, like all the code to support get_host_port_ipv4.

So if testcontainers can support reuse containers, users like me can benefit from both features.

[tisonkun]

My current code in gist - https://gist.github.com/tisonkun/c2ac3154bd6bea003c6097b195a9e32d

[DDtKey]

Hi 👋
Thank you for raising the issue!

Yes, it makes sense to support and shouldn't be hard

[gurinderu]

How do you want to make a reuse ID? Should it be calculated automatically from the source code & call site meta or provided by hand?

[gurinderu]

Also, it is not clear to understand when to stop the container, lets say we run nextest with the same reuse id in a serial way. Nextest use process-per-test strategy and we don't have ryuk

[DDtKey]

Reusable containers are not expected to be cleaned.

Documentation from dotnet impl:

Enabling reuse will disable the resource reaper, meaning the resource will not be cleaned up

and also from Java:

Reusable containers are not suited for CI usage and as an experimental feature not all Testcontainers features are fully working (e.g., resource cleanup or networking).

You can check java and dotnet docs, because it describes how it works. And we can rely on their impls of identifiers to a certain extent.

Btw, ryuk will be supported soon I think, just need to find spare time 🙂

[tisonkun]

Reusable containers are not suited for CI usage

Just a comment to this statement: We reuse containers in the CI environment, and the runner will destroy everything once the job is finished. So it works well for our CI usage :D

[DDtKey]

Yes, actually it should work if your runners are properly configured and don't store the state between different jobs. E.g DnD approach will work

[the-wondersmith]

@DDtKey I've actually been working on implementing support for this. Would you accept a PR for the feature?

[DDtKey]

@the-wondersmith I will be happy to review the PR.

Feel free to open 🙂

[the-wondersmith]

@the-wondersmith I will be happy to review the PR.

Feel free to open 🙂

Will do, as soon as I'm able 😁

[the-wondersmith]

@DDtKey Looked over what I had and realized it would probably be better to break it up into two PRs. The implementation I've been working on uses labels on the testcontainers-managed containers to figure out if a container should be reaped at drop or at exit, so support for applying the labels to begin with is a prerequisite but also a pretty useful feature all on its own.

tl;dr I just opened a precursor PR that adds support for labels 😅, once we're able to get that merged I'll follow up with the full reusable feature

[DDtKey]

Actually I'd like to see how exactly the labels are going to be used, before merging this part. For now it's more like black box without justification (don't get me wrong)

I mean this PR has references(in documentation) to some internal usage which does not exist yet and it's harder to review without full context

Also I can provide some guidance or highlight something while having the whole context

We still can merge them as 2 separate PRs if agreed, but you can open second one as soon as you are ready.

Thank you!

[the-wondersmith]

... this PR has references(in documentation) to some internal usage which does not exist yet

Ah! Apologies for the confusion. The note in the docs is an attempt to make it as clear as possible that the com.testcontainers namespace should be treated as reserved for internal use by the testcontainers crate, and therefore users shouldn't try to use labels like com.testcontainers.my-label. Nothing is actually using anything in the com.testcontainers namespace, the idea is just to establish the "standard" or pattern, so to speak, that nothing in the testcontainers crate itself should / will use any label outside of the com.testcontainers namespace with the exception of the managed-by label.

If you look at the code, you'll see that the managed-by: testcontainers label is unconditionally applied to all containers started by testcontainers, and it's explicitly done in a way that it cannot be overridden by user code. The idea is that future features or even expansions / refactors of existing ones will be able to use labels to determine appropriate behavior by doing simple checks like:

// ...

if labels.get("managed-by").is_some_and(|value| value == "testcontainers")
   && labels.get("com.testcontainers.some-feature").is_some_and(|value| value == "true") {
  // feature code
}

// ...

Its uh... similar to Kubernetes' well-known labels concept 😅.

... I'd like to see how exactly the labels are going to be used, before merging this part.

That's totally fair.

We still can merge them as 2 separate PRs if agreed, ...

Of course, naturally happy to do it in whatever way works best 😁. Thank you for taking the time to look it over! I'll have the next PR up as soon as time allows