Codeberg/Community

Fork 11

Codeberg-hosted Forgejo-Actions #1702

New issue

Open

opened 2024-11-13 23:08:54 +00:00 by fnetX · 3 comments

fnetX commented

2024-11-13 23:08:54 +00:00

Owner

Comment

Hi everyone! Forgejo Actions has matured a lot over the past year, and we think it's time to approach a hosted offer by Codeberg, next to our existing Woodpecker CI. We are looking forward to your feedback and suggestions, and we'll use this issue to communicate plans and progress.

The current CI/CD situation

Unfold to read more about the status quo

Codeberg offers hosted CI/CD using Woodpecker CI. Access is granted per user on request. It is a powerful CI/CD solution, based on an early version of Drone, but successfully enhanced by the community over the past years. The service is generously maintained by some Woodpecker developers on Codeberg's owned hardware.

Forgejo Actions is a new CI/CD implementation and integrated into the software. It is similar to GitHub Actions and a migration is usually easy (although they are not fully compatible!). It was initially developed with some controversy by Gitea Ltd. The Forgejo developers have picked up the effort and enhanced it over time, using it daily for powering their work (about one third of all tasks run at Codeberg.org originate from the Forgejo development).

Forgejo Actions makes it easy to connect own runners, and so many users already made use of the feature by connecting their own runners to Codeberg. Codeberg does not yet provide computing power for Forgejo Actions users.

Why hosted Forgejo Actions?

We have a working CI/CD offering with Woodpecker, which is used by a lot of projects on Codeberg and exists for several years now. However, there is an increasing number of people who are familiar with GitHub Actions and are in need for a libre replacement for it. Offering something similar allows for a more convenient migration out of GitHub.

Forgejo Actions is "baked" into Forgejo and has nice UI integration. Users of Codeberg find the option in their repository settings. Some are confused about our "official" CI offering having nothing to do with the integrated CI.

And, last but not least, having a second offer allows users to decide which to prefer and honours the work of the Forgejo developers.

What about the existing Woodpecker CI offer?

Both services will be maintained by independent teams, and there are no plans to retire the existing service. It depends on the availability and commitment of volunteers in the end, so we can't do any promises for either of the services.

I will follow up with more details about the concept.

### Comment Hi everyone! Forgejo Actions has matured a lot over the past year, and we think it's time to approach a hosted offer by Codeberg, next to our existing Woodpecker CI. We are looking forward to your feedback and suggestions, and we'll use this issue to communicate plans and progress. ## The current CI/CD situation <details><summary>Unfold to read more about the status quo</summary> Codeberg offers hosted CI/CD using [Woodpecker CI](https://woodpecker-ci.org/). Access is granted per user [on request](https://codeberg.org/Codeberg-e.V./requests). It is a powerful CI/CD solution, based on an early version of Drone, but successfully enhanced by the community over the past years. The service is generously maintained by some Woodpecker developers on Codeberg's owned hardware. Forgejo Actions is a new CI/CD implementation and integrated into the software. It is similar to GitHub Actions and a migration is usually easy (although they are not fully compatible!). It was initially developed with some controversy by Gitea Ltd. The Forgejo developers have picked up the effort and enhanced it over time, using it daily for powering their work (about one third of all tasks run at Codeberg.org originate from the Forgejo development). Forgejo Actions makes it easy to connect own runners, and so many users already made use of the feature by connecting their own runners to Codeberg. Codeberg does not yet provide computing power for Forgejo Actions users. </details> ## Why hosted Forgejo Actions? We have a working CI/CD offering with Woodpecker, which is used by a lot of projects on Codeberg and exists for several years now. However, there is an increasing number of people who are familiar with GitHub Actions and are in need for a libre replacement for it. Offering something similar allows for a more convenient migration out of GitHub. Forgejo Actions is "baked" into Forgejo and has nice UI integration. Users of Codeberg find the option in their repository settings. Some are confused about our "official" CI offering having nothing to do with the integrated CI. And, last but not least, having a second offer allows users to decide which to prefer and honours the work of the Forgejo developers. ## What about the existing Woodpecker CI offer? Both services will be maintained by independent teams, and there are no plans to retire the existing service. It depends on the availability and commitment of volunteers in the end, so we can't do any promises for either of the services. I will follow up with more details about the concept.

fnetX added the

service

Codeberg

labels 2024-11-13 23:08:55 +00:00

fnetX commented

2024-11-13 23:33:55 +00:00

Author

Owner

The concept for hosted Codeberg runners

Forgejo makes it easy to connect numerous runners and distribute them.
It is also trivial to distribute jobs across runners, but it is not trivial to benefit from caching if jobs are distributed (this can be configured, but it is not a widely tested setup yet).

We assume that we can have a somewhat efficient resource usage if we dedicated runners to the different use cases using labels and letting users choose which runner to use exactly.

Currently, we do not have effective abuse detection and prevention methods, so we'll start with strict timeouts. That said:

The initial setup will focus on lightweight and short convenience jobs. This means, we'll start hosting Forgejo Actions for things like

building static pages and automating website deployments
automating metadata checks for your repositories (automating things like issue labelling or other quick checks)
spellchecks and (small) linters
other helper tools such as release notes assistants etc

Especially for the small jobs, we expect it will save a lot of effort by not requiring everyone to set up a dedicated runner just for some tiny status checks. If you require more powerful CI, you can bring your own runner. You can also have them coexist, e.g. run status checks all the time on Codeberg's runner and launch your runner overnight when your server is otherwise idle.

Runner labels

Forgejo runners each bring one or multiple labels. They will have a prefix to avoid clashing with your own runners. They can be used in pipelines using a syntax like:

jobs:
  my-job:
    runs-on: codeberg-tiny

My idea is to have a set of labels which represent

the exact specs of the machine (in case it is required to launch specific hardware specs)
optional: -lazy variants to indicate a job does not need to run in realtime (they could be scheduled during times of low load or on offsite hardware during sunny days)
optional: architecture (once we offer multiple, default is amd64)

Labels could be aliased for convenience and if the exact specs are not super important. For example, codeberg-tiny could be guaranteed to run at least on machines with 1 cpu, 1 GB of memory and with a timeout of at least one minute, but it might also run on more powerful hardware.

Open questions

We would like to learn more about your needs. Let us know things like

what you would probably do with the offer
how much resources your jobs need
how much use of caching you would want to make
how many runs you would spawn
the maximum delay you'd like to wait for your jobs for
...

This will help figuring out the final concept for labels and runner setups.

One big issue might be abuse. Unlike with Woodpecker CI, there is no way to restrict access to CI to some users only. We think that starting with a "convenience" runner for small Actions, which will have tight limits, is a good start and unlikely to get abused. Something like 2 CPU cores, 2 GB of memory and a timeout of only a few minutes is probably not attracting everyone for abuse. At least I hope so.

Timeline

We will probably start first tests with Forgejo runners soon. A global runner might appear in a few days already.
We will ask you to test it, but it might go down or disappear from time to time.
Once proven somewhat stable, we'll offer a first runner and announce a set of available labels for you.
The first runner will be capped in resources and execution time to prevent abuse.
Once the platform and tooling matures, we will slowly expand capacity and available resources to match your needs. The future is open, and you can help shaping it.

## The concept for hosted Codeberg runners Forgejo makes it easy to connect numerous runners and distribute them. It is also trivial to distribute jobs across runners, but it is not trivial to benefit from caching if jobs are distributed (this can be configured, but it is not a widely tested setup yet). We assume that we can have a somewhat efficient resource usage if we dedicated runners to the different use cases using labels and letting users choose which runner to use exactly. Currently, we do not have effective abuse detection and prevention methods, so we'll start with strict timeouts. That said: The initial setup will **focus on lightweight and short convenience jobs**. This means, we'll start hosting Forgejo Actions for things like - building static pages and automating website deployments - automating metadata checks for your repositories (automating things like issue labelling or other quick checks) - spellchecks and (small) linters - other helper tools such as release notes assistants etc Especially for the small jobs, we expect it will save a lot of effort by not requiring everyone to set up a dedicated runner just for some tiny status checks. If you require more powerful CI, you can bring your own runner. You can also have them coexist, e.g. run status checks all the time on Codeberg's runner and launch your runner overnight when your server is otherwise idle. ## Runner labels Forgejo runners each bring one or multiple labels. They will have a prefix to avoid clashing with your own runners. They can be used in pipelines using a syntax like: ~~~yaml jobs: my-job: runs-on: codeberg-tiny ~~~ My idea is to have a set of labels which represent - the exact specs of the machine (in case it is required to launch specific hardware specs) - optional: `-lazy` variants to indicate a job does not need to run in realtime (they could be scheduled during times of low load or on offsite hardware during sunny days) - optional: architecture (once we offer multiple, default is amd64) Labels could be aliased for convenience and if the exact specs are not super important. For example, `codeberg-tiny` could be guaranteed to run at least on machines with 1 cpu, 1 GB of memory and with a timeout of at least one minute, but it might also run on more powerful hardware. ## Open questions We would like to learn more about your needs. Let us know things like - what you would probably do with the offer - how much resources your jobs need - how much use of caching you would want to make - how many runs you would spawn - the maximum delay you'd like to wait for your jobs for - ... This will help figuring out the final concept for labels and runner setups. One big issue might be abuse. Unlike with Woodpecker CI, there is no way to restrict access to CI to some users only. We think that starting with a "convenience" runner for small Actions, which will have tight limits, is a good start and unlikely to get abused. Something like 2 CPU cores, 2 GB of memory and a timeout of only a few minutes is probably not attracting everyone for abuse. At least I hope so. ## Timeline 1. We will probably start first tests with Forgejo runners soon. A global runner might appear in a few days already. 2. We will ask you to test it, but it might go down or disappear from time to time. 3. Once proven somewhat stable, we'll offer a first runner and announce a set of available labels for you. 4. The first runner will be capped in resources and execution time to prevent abuse. 5. Once the platform and tooling matures, we will slowly expand capacity and available resources to match your needs. The future is open, and you can help shaping it.

jornfranke commented

2024-11-14 23:15:25 +00:00

How do you isolate the different runners? Are they on a shared machine or each runner is an own machine?
Is it using (rootless) Podman in the background?

How do you isolate the different runners? Are they on a shared machine or each runner is an own machine? Is it using (rootless) Podman in the background?

MichaelAgarkov commented

2024-11-15 10:12:37 +00:00

Will the Forgejo Actions runner support large projects, and if so, will it have a Windows-based runner?

No milestone

No project

No assignees

3 participants

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Codeberg/Community#1702

No description provided.