If You Don’t Like the Question, Ask a Different One
Forget about the title; this is actually about “I” 😁
So …
The Questions I Got
In a research talk that I gave in early 2018, I got one question from the audience, “is NLP still worth studying, given the amazing performance of deep learning on many benchmarks?”
I admit it was a tough question at that moment. To a researcher whose daily job was building machine learning models to improve prediction performance back then, it did look hopeless to some extent.
What I did not expect was the situation could get “worse” quickly. Later that year, the EMLo paper won the best paper award at NAACL. Then, in October, BERT was uploaded on arXiv. People realized you don’t need to build individual models for different benchmark datasets. One model tackles them all.
I am not sure whether we have hit rock bottom about this hopeless situation of working on NLP, because I started to get some questions because of ChatGPT/GPT-4. And this time, not only from academia but also from my family and friends — I don’t think some of them were interested in ChatGPT. Their major concern is probably I am going to lose my job.
The Questions I Asked
Luckily, one thing I learned from a TV show is that if you don’t like the question, ask a different one.¹
Five years ago, my answer to the question was another question, “Is NLP research just about evaluating model performance on benchmark datasets?”
Of course not. Even a simple data-driven research pipeline would consist of several components, such as
- Problem definition
- Data collection
- Modeling
- Evaluation
If we see that a model performs well on a well-defined problem with an existing dataset on some standard evaluation metrics, then we should be cautious to say there is nothing left for us to work on. (I am reluctant to give a specific example here, but certainly you can find one by yourself.)
Fast forward to today, another question but sharing the same spirit could be, “ With ChatGPT (or GPT-4), what else can you do?”
So, I will use the same trick again and ask a different question, “what do you want to do?”
- Do you want to build NLP systems for high-stakes applications (e.g., in the medical domain)?
- Do you want to test whether NLP models satisfy certain privacy and fairness constraints (instead of trusting a black-box model)?
- Do you want to run NLP models with good prediction performance on edge devices (e.g., smartphones)?
- Do you want to facilitate users’ information needs with precise information (e.g., instead of inflating the answer with something irrelevant)?
- …
This list can go on forever, and I am sure GPT-x may get even better. But I have already found something for my group to work on :)
¹ To be precise, the actual line is “if you don’t like the answer, ask a different question.”
