Post
250
π’ The second part of the tutorial series on tiny integration LLMs integration in web with streaming and batching support.
π¬οΈ https://youtu.be/yNKYJzlKxh0
With the second part we making step from sending data chunks β‘οΈ integration of LLM through the third-party providers This would be helpful to understand the minimalistic concept that is supposed to be done in order to support the streaming of large language models.
π Powered by:
π§βπ» bulk-chain (inference framework): https://github.com/nicolay-r/bulk-chain
βοΈ nlp-thirdgate (providers): https://github.com/nicolay-r/nlp-thirdgate
π¬οΈ https://youtu.be/yNKYJzlKxh0
With the second part we making step from sending data chunks β‘οΈ integration of LLM through the third-party providers This would be helpful to understand the minimalistic concept that is supposed to be done in order to support the streaming of large language models.
π Powered by:
π§βπ» bulk-chain (inference framework): https://github.com/nicolay-r/bulk-chain
βοΈ nlp-thirdgate (providers): https://github.com/nicolay-r/nlp-thirdgate