GitLab

This fixes an issue with RAG that stops loading documents as soon
as it reaches a file with unsupported mimetype.

Endpoint role-checking was redundantly applied but FastAPI provides a nice abstraction mechanic...so I applied it. There should be no logical changes in this code; only simpler, cleaner ways for doing the same thing.

Changed RAG parser to prefer file extensions over MIME content types. If the type of file is not recognized assume it's a text file.

Enables RAG for golang, python, java, sh, bat, powershell, cmd, js, css, c/c++/c#, sql, logs, ini, perl, r, dart, docker, env, php, haskell, lua, conf, plsql, ruby, db2, scalla, bash, swift, vue, html, xml, and other arbitrary text files.