“Attention is all you need”. That scientific study released seven years ago cited the Beatles to explain a new architecture of neural networks linked to artificial intelligence. They called it “transformer” and it would have given life to the latest breakthrough in AI, generative, to which OpenAI’s ChatGpt, among others, belong.
No one in the spring of 2017 could imagine that those eleven pages would have a similar impact, an evolution of technology that in economic terms would become a revolution. Just look at the stock market share of Nvidia, which designs the processors on which most AIs operate and are trained: it was worth around 44 dollars then, today it reaches 900. At the end of last week, the co-founder and CEO Jensen Huang, in his usual black leather jacket, he presented the latest platform for chips dedicated to artificial intelligence: Blackwell, with 208 billion transistors, is a big leap forward compared to the 80 billion of the previous one. It’s not only four times faster, but it uses a quarter of the energy. And to think that these are processors that have their roots in the graphics of the video cards used mostly in the world of video games.
Returning to the 2017 scientific study, over time it has been cited more than 100 thousand times in other scientific papers. The record still belongs to “The Lowry protein assay” by the American biochemist Oliver Lowry published in 1951, with over 300 thousand citations. But “Attention Is All You Need” fully entered the top ten. It was signed by eight Google researchers: Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, ?ukasz Kaiser, Illia Polosukhin. They all later left the web giant to join or found other companies, from OpenAI to EssentialAI. Above all, they created a turning point that reduced the leading role of Google itself in this situation, incapable of understanding what it had in its hands, of fully exploiting it and valorising its researchers. Things that unfortunately happen when there is a lack of attention.
#happen #lack #attention
– 2024-05-11 22:28:47
