#40: Simplify Kubernetes examples with offline registration #69

Merged
dachary merged 2 commits from gmem/runner:40-kubernetes-examples into main 2023-08-17 13:16:30 +00:00
Contributor

The examples in their current state use a persistent volume and would be unable to be scaled up as needed. This approach instead leverages offline registration to generate and persist configuration with an initContainer, opening the door to scaling runners as high as needed.

General behavior seems to be "first" pod to come up will pick things up first - there's no round robin or anything. Pods pick up jobs in the order the pods are created.

Job 1 created -> Pod 1 -> Finishes -> Job 2 created -> Pod 1
Job 1 created -> Post 1 -> Job 2 created -> Pod 2

The examples in their current state use a persistent volume and would be unable to be scaled up as needed. This approach instead leverages offline registration to generate and persist configuration with an `initContainer`, opening the door to scaling runners as high as needed. General behavior seems to be "first" pod to come up will pick things up first - there's no round robin or anything. Pods pick up jobs in the order the pods are created. Job 1 created -> Pod 1 -> Finishes -> Job 2 created -> Pod 1 Job 1 created -> Post 1 -> Job 2 created -> Pod 2
gmem added 1 commit 2023-08-17 09:27:20 +00:00
Simplify Kubernetes examples with offline registration
All checks were successful
checks / check and test (pull_request) Successful in 45s
70d17228df
gmem added 1 commit 2023-08-17 09:43:24 +00:00
Update Kubernetes example README
All checks were successful
checks / check and test (pull_request) Successful in 31s
af87ebbe7c
Member

I assume you tested that with a set of manual steps. How difficult would it be to reproduce these manual steps in a workflow? Note that with runs-on: self-hosted the run steps are executed on a Debian GNU/Linux bullseye machine that is systemd capable, which makes it possible to install k3s and do pretty much all you can do on a physical machine. The workflow is not constrained by the limitations of a docker container.

See this example of usage of self-hosted.

I assume you tested that with a set of manual steps. How difficult would it be to reproduce these manual steps in a workflow? Note that with `runs-on: self-hosted` the `run` steps are executed on a Debian GNU/Linux bullseye machine that is systemd capable, which makes it possible to install k3s and do pretty much all you can do on a physical machine. The workflow is not constrained by the limitations of a docker container. See [this example](https://code.forgejo.org/forgejo/runner/src/branch/main/.forgejo/workflows/integration.yml#L11-L13) of usage of self-hosted.
Member

The reason I'm suggesting that is that having the CI run these examples would ensure that they work now and also guard against future regressions.

The reason I'm suggesting that is that having the CI run these examples would ensure that they work now and also guard against future regressions.
Author
Contributor

I was wondering what the best way of going about this could be - whether it be a dry run kubectl apply to test them or spinning up a small cluster. I'll take a look around and see if there are some existing best practices for this :)

My two main concerns are how do we verify that the pods are actually ready and running properly (we could grep and compare the output of kubectl get), and do we also want to validate jobs are picked up in some way? (probably too much complexity and just verifying it runs would be enough)

I was wondering what the best way of going about this could be - whether it be a dry run `kubectl apply` to test them or spinning up a small cluster. I'll take a look around and see if there are some existing best practices for this :) My two main concerns are how do we verify that the pods are actually ready and running properly (we could grep and compare the output of `kubectl get`), and do we *also* want to validate jobs are picked up in some way? (probably too much complexity and just verifying it runs would be enough)
Member

probably too much complexity and just verifying it runs would be enough

That's a good first step.

do we also want to validate jobs are picked up in some way?

That's what setup-forgejo tests are doing and it is not very complicated. The general idea is to allow for automatically creating a project with actions enabled just by pushing a repository. And then poll for the status of the commit to be "success".

> probably too much complexity and just verifying it runs would be enough That's a good first step. > do we also want to validate jobs are picked up in some way? That's what [setup-forgejo tests](https://code.forgejo.org/actions/setup-forgejo/src/branch/main/.forgejo/workflows/integration.yml#L44) are doing and it is not very complicated. The general idea is to allow for [automatically creating a project with actions enabled](https://code.forgejo.org/actions/setup-forgejo/src/branch/main/forgejo.sh#L42-L44) just by pushing a repository. And then [poll for the status of the commit](https://code.forgejo.org/actions/setup-forgejo/src/branch/main/forgejo-test-helper.sh#L58-L66) to be "success".
Owner

That will not be possible because of #55

It is not much work really but doing it properly requires a significant learning curve.

That will not be possible because of https://code.forgejo.org/forgejo/runner/issues/55 It is not much work really but doing it properly requires a significant learning curve.
earl-warren approved these changes 2023-08-17 13:08:10 +00:00
earl-warren left a comment
Owner

I'm in favor of merging this as is. And maybe create an issue to remember such tests would be welcome.

I'm in favor of merging this as is. And maybe create an issue to remember such tests would be welcome.
Member

I forgot about this roadblock when suggesting the tests. Not such a good idea after all.

I forgot about this roadblock when suggesting the tests. Not such a good idea after all.
dachary approved these changes 2023-08-17 13:16:13 +00:00
dachary merged commit 58f8538eb9 into main 2023-08-17 13:16:29 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: forgejo/runner#69
No description provided.