6/9/2023 0 Comments Open source soundplant“Language models will form the backbone of our digital economy, and we want everyone to have a voice in their design. They demonstrate how small and efficient models can deliver high performance with appropriate training,” Stability AI web designer Anel Islamovic wrote in a blog post about Stable LM. Our StableLM models can generate text and code and will power a range of downstream applications. “With the launch of the StableLM suite of models, Stability AI is continuing to make foundational AI technology accessible to all. The result gives what Stability AI calls a “surprisingly high performance in conversational and coding tasks” relative to its parameter size. Stability used the Pile open-source dataset and its amalgamation of Wikipedia, Stack Exchange, and PubMed to train StableLM, but an expanded version with triple the number of tokens, 1.5 trillion tokens all told. StableLM’s condensed parameter size is counterbalanced by its training, according to the company. For comparison, OpenAI built GPT-3 with 175 billion parameters and GPT-4 with an unknown but larger number than that. The number of parameters comes across as relatively limited in some ways. You can see an example of what it can do on the right. Those interested can experiment with a fine-tuned version of StableLM as a chatbot through Hugging Face as well. The company released both a 3 billion and a 7 billion-parameter version of StableLM on GitHub and promised the imminent release of 15 billion and 65 billion-parameter options soon. Stability AI has been quietly working on LLMs with EleutherAI, but StableLM is far more comprehensive and widely available. StableLM widens Stability’s portfolio beyond its popular Stable Diffusion text-to-image generative AI model and into producing text and computer code, setting itself up as a direct rival for OpenAI, Google, and other generative AI model developers. There’s also an edition optimized specifically to generate Python code.Synthetic media startup Stability AI shared the first of a new collection of open-source large language models (LLMs) named StableLM this week. Alongside the core version, there is an edition of the model that has been trained on additional Python, Java, JavaScript code samples, which should translate into improved support for the three languages. StarCoderBase is available in multiple versions. Codex powers GitHub Copilot, the AI coding assistant offered by Microsoft Corp.’s GitHub unit. ServiceNow and Hugging Face claim that the AI can also outperform an early version of OpenAI LLC’s Codex model. They determined that the AI outperforms all other open-source code generation models with built-in support for multiple programming languages. During an internal test, the companies compared StarCoderBase with multiple open-source alternatives. The companies claim that the AI model can not only generate code in multiple languages, but also do so more efficiently than many rival models. According to the companies, the server cluster they used to train StarCoderBase included 512 graphics cards. The A100 was Nvidia Corp.’s flagship data center AI accelerator until the chipmaker introduced its newest H100 chip last year. ServiceNow and Hugging Face trained StarCoderBase using a cluster of 64 servers equipped with A100 graphics cards. A token is a unit of data that comprises a word, a word fragment or a few digits. In total, the AI model was trained on about one trillion tokens. ServiceNow and Hugging Face didn’t use the entire dataset, but only code samples written in 86 of the supported programming languages.ĭuring training, the companies also supplied StarCoderBase with software documentation and related technical information. StarCoderBase was trained on a dataset called The Stack that includes code written in 358 programming languages. Those are the settings that determine how an AI model goes about performing tasks such as generating code. The core edition, StarCoderBase, features 15.5 billion parameters. StarCoder is available in multiple versions. “This endeavor is a testament to the potential of open‑source as we work toward democratizing AI.” “The joint efforts led by Hugging Face and ServiceNow enable the release of powerful base models that empower the community to build a wide range of applications more efficiently than a single company could come up with,” said BigCode co-lead Leandro von Werra. The project, which is called BigCode, drew contributions from not only the two companies’ engineers but also hundreds of other AI experts. It was developed through a research project that ServiceNow and Hugging Face launched last year. The companies claim that StarCoder is the most advanced model of its kind in the open-source ecosystem. today introduced StarCoder, an open-source artificial intelligence model model that can generate code in multiple programming languages.
0 Comments
Leave a Reply. |