Transformers as Support Vector Machines https://arxiv.org/abs/2308.16898
I'd say Postgres + pgvector is even simpler if you're doing small scale document search (eg. internal knowledge bases, documentation sources, codebase indexing, etc.).
