Established for 2 years, each employee is worth $21 million. Why did MosaicML sell for $1.3 billion?
In recent times, there has been a wave of investment and acquisitions in the AI field. Renowned global company Salesforce has invested $450 million in Anthropic, while Runway has successfully raised $141 million in funding. Additionally, Snowflake has announced the completion of the acquisition of Neeva, and Chinese domestic giant Meituan has acquired AI company Lightyear for $2.065 billion.
However, the most notable deal is undoubtedly the acquisition of the startup MosaicML. It is reported that Databricks, a big data giant, has acquired MosaicML for approximately $1.3 billion, with its valuation soaring six times in this transaction, making it the largest acquisition in the first half of this year. What has supported MosaicML's high valuation in just 2 years of existence with more than 60 employees?
1. Databricks Acquires MosaicML, Accelerating Democratization of Generative AI Technology
Databricks recently announced the acquisition of generative AI startup MosaicML for approximately $1.3 billion, aiming to provide services for building ChatGPT-like tools for enterprises.
Following the acquisition, MosaicML will become part of Databricks' Lakehouse platform, integrating the entire MosaicML team and technology. This will provide enterprises with a unified platform to manage data assets and enable them to build, own, and protect their own generative AI models using proprietary data.
MosaicML is a very young generative AI company. It was established in San Francisco in 2021 and has only disclosed one round of financing, with only 62 employees. In the previous financing round, its valuation was 220 million US dollars, which means that the valuation of the acquisition of MosaicML has directly increased sixfold. This transaction is the largest acquisition announced in the generative AI field this year. Just recently, cloud computing giant Snowflake announced the acquisition of another generative AI company, Neeva. After months of investment frenzy, large companies seem to be starting a wave of large-scale acquisitions of generative AI startups.
Databricks originated from UC Berkeley and participated in the development of the Apache Spark project. As a data storage and analytics giant, its valuation reached 31 billion US dollars by 2022, helping large companies such as AT&T, Shell, Walgreens process data. Recently, it open-sourced its large-scale model, Dolly, aiming to achieve similar effects as ChatGPT with fewer parameters. With the increasing popularity of cloud computing, the "Data Lakehouse" concept proposed by Spark has deeply influenced a group of big data startups. Since its establishment in 2013, Databricks has rapidly grown into the world's hottest Data Infra company. Last year, Databricks announced annual revenue exceeding 1 billion US dollars, and after completing its latest financing round in August 2021, its latest valuation reached 38 billion US dollars.
2. Advantages of MosaicML MPT Series Models
MosaicML's MPT series models are subclassed from the HuggingFace PretrainedModel base class and fully compatible with the HuggingFace ecosystem. The MPT-7 B model is one of MosaicML's most popular models, with billions of parameters and the ability to handle over 2,000 natural language processing tasks. The optimization layers of MPT-7 B, including FlashAttention and low-precision layer norm, enable the model to be 2-7 times faster than traditional training methods, and the near-linear scalability of resources ensures that models with billions of parameters can be trained in a matter of hours instead of days. MosaicML has also released a new commercially available open-source large language model, MPT-30 B, with 300 billion parameters and better performance than GPT-3.
Source: Evaluation of mainstream MosaicML models by MT-Bench
The advantage of the MPT series models lies in their efficiency and low cost. The complexity of artificial intelligence models "trained" with large amounts of data has skyrocketed, and it now costs millions of dollars to train a model, which is beyond the reach of small and medium-sized enterprises, except for large companies. However, MosaicML's MPT series models enable companies to train their own language models at a lower cost and with higher efficiency, making it easier to apply generative AI technologies and achieve better business performance. Most open-source language models can only handle sequences with a maximum of several thousand tokens (see Figure 1). However, with the MosaicML platform and a single node of 8 x A100-40 GB, users can easily fine-tune MPT-7 B to handle a context length of up to 65k. The ability to handle such extreme context lengths comes from ALiBi, which is one of the key architectural choices in MPT-7 B.
For example, the complete text of "The Great Gatsby" is less than 68k tokens. In one test, the StoryWriter model read "The Great Gatsby" and generated an ending. One of the endings generated by the model is shown in Figure 2. The StoryWriter finished reading "The Great Gatsby" in about 20 seconds (approximately 150,000 words per minute). Due to the longer sequence length, its typing speed is slower than other MPT-7 B models, at around 105 words per minute. Despite the fine-tuning of StoryWriter on a context length of 65k, ALiBi allows the model to infer longer inputs than those used for training: in the case of "The Great Gatsby," up to 68k tokens, and up to 84k tokens in testing.
Figure 2: MPT-7 B-StoryWriter-65k+ wrote an ending for "The Great Gatsby". The result of the ending is the full text of "The Great Gatsby" (approximately 68k tokens) provided as input to the model, followed by the word "Ending," allowing the model to continue generating.
3. Popularization of Generative AI Technology
Generative AI technology is a branch of artificial intelligence that uses large amounts of data and deep learning algorithms to automatically generate original text, images, computer code, and other content. This technology enables people to process and analyze data more conveniently, better serving human needs. With the rapid development of big data and artificial intelligence technology, generative AI technology has been widely applied in areas such as natural language processing, image recognition, and virtual reality. For example, in the field of natural language processing, GPT-4 has become one of the most popular generative AI models, used for tasks such as generating articles, translating languages, and answering questions. In the field of image recognition, StyleGAN 2 can generate high-quality images, used in game development, film production, virtual reality, and other fields.
MosaicML's CEO Naveen Rao previously stated that since 2018, the complexity of AI models trained with massive amounts of data has risen sharply. Training a model now costs at least millions of dollars, making it unaffordable for small and medium-sized enterprises, except for large companies. However, after this acquisition, the joint product of Databricks' Lakehouse platform and MosaicML's technology will allow companies to train and build generative AI models using their proprietary data in a simple, fast, and cost-effective manner, giving users control and ownership of the data for custom AI model development. According to Databricks, with the support of their platform and technology, the cost of training and using LLMs for enterprises will significantly decrease, estimated to be around a few thousand dollars. This provides convenience for the popularization of generative AI.
4. The Significance of Databricks' Acquisition of MosaicML
The main purpose of Databricks' acquisition of MosaicML is to accelerate the development and democratization of generative AI technology. By integrating the technologies and resources of the two companies, Databricks can better meet customer needs and provide more efficient and convenient solutions. Specifically, this acquisition will bring the following changes:
1. More Efficient Large Language Models
After the acquisition of MosaicML, Databricks can integrate the MPT series models into its Lakehouse platform, providing customers with more efficient and cost-effective large language models. This will help enterprises better handle natural language processing tasks, improving business efficiency and accuracy.
2. Faster Model Training Speed
MosaicML's MPT series models have the feature of fast training, which will help Databricks provide faster model training services. This is particularly important for businesses that need to respond to market demands quickly and can help them better meet customer needs.
3. Higher level of democratization
The acquisition of MosaicML by Databricks also means that the democratization level of generative AI technology will be further enhanced. MosaicML's MPT series models make it easier for small and medium-sized enterprises to train their own language models, enabling better application of generative AI technology and achieving better business performance. This will contribute to the development and application of generative AI technology, promoting the popularization and development of artificial intelligence.
Summary
Generative AI applications aim to generate raw text, images, and computer code based on natural language prompts from users. Since the launch of the online generative AI chatbot ChatGPT by the AI startup OpenAI in November last year, there has been a surge of interest in this technology. "Every organization should be able to benefit from the AI revolution and have more control over the way they use their data. Databricks and MosaicML have an incredible opportunity to democratize artificial intelligence and make the Lakehouse the best place to build generative AI," said Ali Ghodsi, Co-Founder and CEO of Databricks.
Databricks acquisition of MosaicML not only accelerates the development and democratization of generative AI technology but also integrates the technologies and resources of both companies to provide customers with more efficient and convenient solutions. As artificial intelligence technology continues to rapidly advance and be applied, generative AI technology will play an increasingly important role. Databricks' acquisition of MosaicML reflects the attention and investment of various companies in this direction. Companies like Anthropic and OpenAI license ready-made language models to enterprises, which then build generative AI applications on top of them. Driven by strong commercial demand for these models, opportunities have been created for startups like MosaicML. The consecutive acquisitions by Snowflake and Databricks indicate that large technology companies are gradually transitioning from independent research and strategic investments to mergers and acquisitions in generative AI technology.
Reference sources:
https://www.databricks.com/company/newsroom/press-releases/databricks-signs-definitive-agreement-acquire-mosaicml-leading-generative-ai-platform
https://mattturck.com/mosaic/
https://twitter.com/lmsysorg/status/1672077353533730817/photo/1
https://www.mosaicml.com/blog/mpt-7b#appendix-eval
https://www.mosaicml.com/blog/mpt-30b
Copyright statement: For reprinting, please add WeChat Assistant for communication. Unauthorized reprinting, plagiarism, and rewriting are subject to legal liability.
Disclaimer: The market has risks, and investing requires caution. Readers should strictly comply with local laws and regulations when considering any opinions, viewpoints, or conclusions in this article. The above content does not constitute any investment advice.
