Everything was great and very open when progress began in the development of practical artificial intelligence systems a few years ago. Companies like OpenAI weren’t just releasing amazing engines like DALL-E 2 or GPT-3, they were publishing extensive studies detailing how they were created. That ended.
GPT-4. This week OpenAI announced the new version of its conversational AI model, GPT-4, and although in the official announcement it showed the advantages and its capacity, no internal details of its development were given. It is not known how much data the system has been trained with, what the energy cost is, or what hardware and methods have been followed to create it.
I think we can call it shut on ‘Open’ AI: the 98 page paper introducing GPT-4 proudly declares that they’re disclosing *nothing* about the contents of their training set. pic.twitter.com/dyI4Vf0uL3
— Ben Schmidt / @[email protected] (@benmschmidt) March 14, 2023
OpenAI is anything but ‘Open’. Ben Schmidt, an engineer at a mapping AI company called Nomic, pointed out an important detail of the 98 page technical report about GPT-4: those responsible declared that they were not going to reveal anything at all about how they had trained the model.
The reason? competition. On the second page of that report they talked about the scope and limitations of that report, warning that although they would give data about their capabilities, they would not reveal internal details of the development so as not to make it easier for other competitors to match their development:
“Considering the competitive landscape and security implications of large-scale models such as GPT-4, this report does not contain further details on the architecture (including model size), hardware, training computation, data set construction, training method or other similar data, data set construction, training method or the like”.
Companies get serious. Ilya Sutskever, one of the co-founders of OpenAI, explained on the verge that “there is a lot of competition out there. (…) A lot of companies want to do the same thing, so from a competitive point of view, you can see this as the maturation of this field.” From sharing knowledge as happened years ago, companies have moved to a much more protectionist approach towards their advances.
But there is a danger. The problem with this new secretive position lies in the security of these models, which without this transparency lose the ability to be audited by other experts or independent organizations. For Sutskever, the approach was precisely the opposite: these models can end up causing “a lot of damage,” and as capabilities improve, “it makes sense that you don’t even want to reveal (how they work on the inside)” so bad actors can’t take advantage of it.
“We were wrong”. OpenAI’s attitude in this regard has totally changed, and in the past they shared extensive information about their models. According to Sutskever “we were wrong”. For him, if the AI ends up being powerful, “there is no point in opening the source code. It’s a bad idea… I hope that in a few years it will be completely obvious to everyone that open source AI is not a good idea.”
Opinions for all tastes. While OpenAI closes in band with GPT-4, Meta has just launch FLAME, an essentially open competitor that we can actually install on our laptop. The approach is currently the one that OpenAI originally had, but for others the decision of the creators of GPT-4 makes sense from a business point of view. That is what William Falcon, creator of the PyTorch Lightning tool, confessed, who in VentureBeat explained how “If this model goes wrong, and it will go wrong, you have already seen it with hallucinations and giving you false information, how is the community supposed to react? ?”
And then there is copyright. There is also a legal aspect to all this: the data sets or ‘datasets’ with which these models are trained are gigantic and much of this information is collected from the web. Some of that content is likely to be copyrighted, so not saying how they’ve been trained protects them (initially) from potential copyright lawsuits.
In Xataka | I have no idea about programming but thanks to GPT-4 I have created a clone of Flappy Bird
