NLP Problems Overview — Understanding Perspective(2/2)

(You can find the Korean version of the post at this link.)

This is the 3rd post in a series covering various problem types in Natural Language Processing (NLP). Previously, we examined how NLP problems can be broadly categorized into two types:

1. Problem types that can directly measure the ‘performance’ of applied systems
2. Problem types that measure the ‘language understanding ability’ of foundational language models
   - ‘Classification’ type understanding ability of language models
   - ‘Classification/generation’ type understanding ability of (large) language models (← this post)

Among these, the field of evaluating the ‘language understanding ability’ of foundational language models has been particularly heavily researched in recent years.

This post introduces several recent attempts to comprehensively evaluate large language models’ understanding of language, culture, ethics, and practical functions, including both classification and generation types.

The entire content covered in this post can be summarized briefly in the table and graph below.

Evaluation benchmarks for large language models