Our internal needs are extremely simple, it's literally, we have a folder on disk and we want its content to be available through an S3 API. We don't really care about the more complex S3 management APIs, we don't care about replication or any of that stuff.

So my preference would be to rip the minio stuff out and replace it with just having our built-in web server implement the few bits of the S3 API needed to handle file upload and retrieval. I believe we already have our own copy of the authentication logic anyway so hopefully not too much work to get something like that done.

@stgraber If you just want to expse a local folder via s3, rclone server s3 might be an option.

You can run somting like rclone serve s3 /data --addr :9000

The idea is actually not to use an external tool, rather stay within the Go runtime. That minimizes non-Go dependencies and makes binary packaging easier.

VersityGW is also written in Go, so it could also be an option. It's also Apache licensed so it could be used as a starting point to eventually embed into the web server.

I also really like the idea to implement the s3 functionality directly inside incus, instead of running a subprocess for it 👍🏼

VersityGW is also written in Go, so it could also be an option. It's also Apache licensed so it could be used as a starting point to eventually embed into the web server.

https://github.com/eteran/silo seems also rather simple in regards of features, maybe also a good starting point/reference? It doesnt have a licence, but i think we could ask the author to clarify.

That may fit our need: https://github.com/amwolff/awsig
The author gives a reference implementation of an S3 server which can be pretty useful. However, I can only find a single downstream user, so judging the maturity is a little tough.

Do we want full compatibility with existing local buckets, including their current on-disk layout and metadata, or are we willing to introduce a migration step? MinIO uses its own internal metadata format called xl.meta, and that is tied to features like file versioning. Preserving full compatibility with existing bucket contents might require a lot of work.

A reduced initial scope focused on basic operations would make a native implementation much simpler, but it could be a regression for existing users who rely on current MinIO behavior. Maybe a good solution would be to add a parallel implementation that has a simple feature set at first, and offer a migration path.

My expectation is that folks primarily care about being able to push and pull files to and from the bucket. Any other fancy S3 feature which was enabled by Minio was never documented as supported in our API so I'm not too concerned about losing some metadata during migration.

We'll obviously want to try and play nice, so if the data is easily parsable and can be used, great (kinda doubt it though), otherwise, I think we should just move the existing metadata into a hidden folder or something on disk should the user want to do something with it and otherwise just move on.

Documentation will be important for sure and we'll want to really have folks be aware of the different backend when moving to the new release.

Speaking of, I've now milestoned this for 7.0 as it would be nice not to have to depend on a dead project for the next 5 years...