Together is revolutionizing the world of AI by developing the first-ever distributed cloud designed specifically for handling large-scale foundation models. Their innovative platform combines data, models, and computing to help AI researchers, developers, and businesses advance and harness AI technology.

The Together team firmly believes that open-source models for philanthropic purposes have the potential to be more democratic, open, strong, and adaptive. To further this cause, they have recently launched OpenChatKit 0.15, an open-source chatbot framework that enables the development of domain-specific and general-purpose chatbots. OpenChatKit offers a robust foundation for chatbots, with the code, model weights, and training datasets freely available to the public under the Apache-2.0 license.

The chatbot framework was developed in collaboration with LAION and Ontocord and offers support for multi-turn discussions, knowledge reasoning, and answer generation. The chat model boasts 20 billion parameters and was trained on 43 million instructions. The framework includes a generic chatbot and the necessary components to create specialized bots.

OpenChatKit offers several features that make it a useful tool for building chatbots. Firstly, it includes a large language model called GPT-NeoX-20B from EleutherAI, which is specifically tuned for chat and has been trained on 100% carbon-negative compute. Secondly, the framework provides a set of customization recipes that allow users to fine-tune the model to achieve high accuracy on specific tasks. These recipes are documented and available open-source on GitHub. Thirdly, the framework includes a retrieval system that allows information from a document repository, API, or live-updating source to be added to a bot’s responses at inference time. Finally, OpenChatKit provides a moderation model derived from GPT-JT-6B, which selects which queries the bot answers.

While GPT-NeoXT-Chat-Base-20B is a powerful tool for building chatbots, it does have some limitations. For example, the model may not return an accurate or relevant answer when asked something novel, unclear, or outside of its training data. However, the Together team is dedicated to building a more robust and inclusive chatbot, and they invite participation from a wide range of groups and individuals.

OpenChatKit is an excellent resource for those interested in studying models of conversation and language, exploring and comprehending the flaws and biases of chatbot models, creating works of art, and applying them to design and other creative tasks, and developing tools for learning. The framework is an example of Together’s commitment to democratizing AI and making it accessible to everyone.