i often talk about ethics in software design, and it's become apparent to me that i've made a mistake of my own.

any user can scrape any other user's posts*. this is how activitypub works, and it would be kinda weird from a technical perspective if it didn't.

i've written two programs that scrape people's posts; mstdn-ebooks and fedibooks. both of them do not require the scraped user to give consent, because this is simply how AP works. if authentication isn't needed, why bother asking for it?

of course, this is a naive answer. a given user might not consent to their posts being scraped, either by the program as a whole or in a specific instance. presently, there's no way for this permission to be denied by the scraped user.

i could very much take the pleroma approach and argue that the posts are publicly available, anyone can scrape them, and having my software require authentication is just giving users a false sense of security. but i think that's the wrong answer.

fedibooks NEEDS to ask the scraped users for permission. not technically, but ethically. the authentication tokens aren't necessary and can even be thrown away immediately. what IS necessary is that users are asked to consent to their posts being scraped.

i'll be implementing this feature soon. i'm sorry that i didn't think to implement it sooner, and i feel as though i've personally let the fediverse community down on this front. this is my fault.

*barring networks with authenticated fetches