If you read the previous article, you probably thought that Alice is a scientific fraudster. But the truth was not like that.
In her second paper “Attention, the vortex you’re caught in,” Bob (Former Alice) reflects on the idea that “attention is all you need” and how it became a popular misconception in the field of artificial intelligence. Bob argues that while attention mechanisms are important, they can also lead to a narrow focus on certain aspects of information, missing the broader perspective.
Bob explains that attention can be deceiving, leading researchers down a path of oversimplification and ignoring other crucial elements. Just like a vortex can trap and hold things in a tight space, the exclusive focus on attention can lead AI researchers to miss important information.
Bob had a golden saying. Working on the attention mechanism is like being stuck in a swamp, the more you wade, the sooner you sink.
Bob mentioned five attention problems:
Narrow focus: The exclusive focus on attention can lead to a narrow and limited perspective, missing the bigger picture.
Oversimplification: Attention mechanisms can lead to oversimplification, causing researchers to ignore other important elements.
Limited view: The singular focus on attention can be like being trapped in a swamp, limiting the view and ability to understand the full picture.
Inaccurate predictions: By missing important information, AI systems that rely solely on attention mechanisms may make inaccurate predictions.
Lack of comprehensiveness: A comprehensive approach to AI development that incorporates additional factors beyond attention is necessary to create truly intelligent machines.
Bob’s article was published in a not so important journal. His article did not make much noise.
Indeed, why this critical article was not noticed by researchers? Bob, or former Alice, knew that all formers attacks were under the supervision of his ex-husband, George Petrovitch, he was widely regarded as the godfather of the Attention Mechanism Mafia.
Petrovitch had long been a staunch advocate of the “attention is all you need” philosophy and was vehemently opposed to any challenges to this view. He saw Alice’s paper as a direct attack on his life’s work and was determined to discredit it.
Adding fuel to the fire was the fact that Petrovitch was Alice’s ex-fiancé. It was revealed that the two had a tumultuous breakup, and Petrovitch was still harboring a grudge against Alice. He saw this as the perfect opportunity to exact his revenge.
Petrovitch launched a smear campaign against Alice, claiming that the paper was based on fraudulent research and that Alice was behind it all. He went so far as to start a poisoning plot against Alice, trying to discredit her and undermine the credibility of the paper.
Despite the opposition, Alice stood by her findings, and her paper continued to generate discussion and debate in the AI community. The conflict between the two sides was intense, with Petrovitch and his followers on one side and Alice on the other. The “attention vs. context and relationships” debate divided the community, with no clear winner in sight.
Although this matter was told in the form of a story and was not a scientific fact. But my experience says that attention is not all you need. Do not pay too much attention to this story 😊
Indeed, this video of mine is about attention, don’t miss it. https://www.youtube.com/watch?v=-gk0oHPCvAw
I think this post can be interesting for you. :)
Despite the monetization of my account, Medium does not pay me due to my geographical location. Make me hope to continue writing by buying coffee. Thanks.
Tip link is below :)
