10/6/2023 0 Comments Chimp test human benchmark![]() ![]() These two together drive the use of flops and the increasingly emergent behavior of the models. Let’s talk about scale and scope for a minute, and specifically the parameter and token counts used in the training of the LLMs. It remains to be seen how any of this can be monetized, but if training and inference costs come down, as we think they can, it is reasonable to assume that generative AI will be embedded in damned near everything. Demand is too high and supply is too low for GPUs for commercial entities to not be paying a hefty premium.Īfter a deep dive on Inflection AI a few weeks ago, we promised that we would circle back and see how its Inflection-1 LLM stacked up against the other large models that are vying to be the underpinnings of generative AI additions to applications large and small – and here and there and everywhere. ( We compared these machines to the “El Capitan” supercomputer being built by Hewlett Packard Enterprise and AMD for Lawrence Livermore National Laboratory recently in terms of price and various kinds of performance.) Let’s just say the US government is getting one hell of a deal for its exascale systems, and we do not think AI startups or Microsoft can boast about this. The hardware costs for this machine alone, we have calculated, would cost somewhere around $1.35 billion, a little bit more than what we think Microsoft is shelling out to build a rumored cluster with 25,000 GPUs to train what we assume is the GPT-5 model from partner OpenAI. ![]() What we do know is that Nvidia and CoreWeave are building a cloudy cluster with 22,000 H100s, which presumably will be used to create and train the Inflection-2 LLM. The company’s homegrown Inflection-1 LLM was trained on 3,500 Nvidia “Hopper” H100 GPUs as part of its recent MLPerf benchmark tests, and we think that many thousands more Nvidia GPUs were added to train the full-on Inflection-1 model. Inflection AI, the startup behind the Pi “personal intelligence” chatbot service, has raised $1.53 billion in venture capital and will end up spending a lot of that money on AI training capacity to CoreWeave, an AI cloud compute provider, and its partner, Nvidia. To be fair, Meta Platforms is rumored to be considering a commercial variant of its LLaMA foundation model, which we talked about in detail back in February. Some, more than one, and despite all of the lip service, they are reluctant to open source the foundation models they have developed over the past several years. We learn a lot of lessons from the hyperscalers and HPC centers of the world, and one of them is that those who control their own software control their own fates.Īnd that is precisely why all of the big hyperscalers, who want to sell AI services or embed them into existing services to maintain their products against fierce competition, have their own models.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |