Show HN: GPTBanker – map/reduce over documents of any length with OpenAI - Klaput News
Show HN: GPTBanker – map/reduce over documents of any length with OpenAI
2 by aicharades | 0 comments on Hacker News.
Hi all! I created a app that can map/reduce over documents of an arbitrary length with OpenAI completions. You can enter in your OpenAI key and try it out. The backend is built with LangChain, which made it easy to manage the models, workflows (chains), and troubleshoot pipelines with buffer issues. Learnings: - Tokens was the most helpful key for calls rather than characters or words. LangChain also just implemented a token splitter, which made it easier for me to adapt to documents of any type. - Davinci performed much better than Curie for summarizing long documents over multiple chunks - Oftentimes a map step was the only thing needed and the reduce/combine introduced more errors instead of improving context.
RELATED POST :
2 by aicharades | 0 comments on Hacker News.
Hi all! I created a app that can map/reduce over documents of an arbitrary length with OpenAI completions. You can enter in your OpenAI key and try it out. The backend is built with LangChain, which made it easy to manage the models, workflows (chains), and troubleshoot pipelines with buffer issues. Learnings: - Tokens was the most helpful key for calls rather than characters or words. LangChain also just implemented a token splitter, which made it easier for me to adapt to documents of any type. - Davinci performed much better than Curie for summarizing long documents over multiple chunks - Oftentimes a map step was the only thing needed and the reduce/combine introduced more errors instead of improving context.
RELATED POST :
Memuat...
Comments
Post a Comment