Kerne, A., Smith, S.M., Webb, A., Linder, Lupfer, N., Qu, Y., R., Moeller, J., Damaraju, S., Information-based Ideation Metrics show that Mixed-Initiative Information Composition Provokes Creativity, ACM Transactions on Computer-Human Interaction (ToCHI), 21(3), June 2014, 48 pages.
This article introduced ideation metrics of curation for evaluating information-based ideation(IBI). This artical is very well written, it is a good example of how to illustrate research problems, make hypothesis, doing study and evaluating the research. If we have read this paper before the final project purposal, we would have saved a lot of time figuring out how to write the purposal and we might have come up with a better purposal.
Anyway, in this artical, the author first raised two challenges for IBI:
One challenge is overcoming fixation — for instance, when a person gets stuck in a counterproductive mental set.
The other challenge is to bridge information visualization’s synthesis gap, by providing support for connecting findings.
To address the challenges, the author developed a mixed-initiative information composition(MI2C), integrating human curation of information composition with automated agents of information retrieval and visualization.
The artical is built through 3 episodes:
- IBI evaluation methodology
- Case study
I think the most valuable part is the evaluation. IBI support environment is a creativity support environment, and in this artical, for the evaluation part, the author designed four steps. Currently, I am developing Fizz, which is a creativity support environment that allow users design their own cocktail recipes. And for the evaluation, I can do it according to the 4 steps mentioned in this artical.
(1)engage people perform IBI task: we design some experiments that is constrained yet involves curation. For example, design a cocktail that use ingredients “Gin” and “Cherry liquor”, and is suitable for official business party.(fyi, official business party is a category of cocktails, and there are typical recipes and design princples that are suitable for business party cocktails, so it is possible to evaluate if the cocktailis are suitable or not )
(2)elicit their curation product: we can ask the users to design the cocktail using the creation function of Fizz.
(3)measure creativity with ideation metrics of curation:
Fluency: how many cocktails designed in a given amount of time.
Flexibility: througn observation, we can tell how many different combination a user have tried during creation. And how many ingredients he have tried to add.
Novelty: To measure the novelty, we need to first classify the existing recipes. We have already crawled 4000+ existing recipes, and we will cluster these recipes according to the base liquor and the ingredients. So there might be several cluster centers. And for the new recipe, the further it lies from the center of the clusters, the higher the novelty is.
(4)compare ideation metrics distributions across conditions: We can carry out the experiments under different control conditions: 1) edit a cocktail recipe to make it more suitable for chrismas family party; 2) create a recipe that can be classified a a new era drink; 3) create a recipe that is red; and so on.
This way we can quantify the evaluation of Fizz.