Diary study for ShopBack OMO affiliate product — SBMart (part 3/3)
中文版 — ShopBack 日誌研究: OMO聯盟行銷 — 發票回饋 (Part 3/3)
Click to check out Part 1 and Part 2!
Intro
In the last article on SBMart research, we talked about how we collect, analyze data by using tools like Line, Excel, and Figjam board to help us visualize data. In this chapter, I am going to share more about my learning from this journey.
Learning from the Diary study
As a researcher with 7 years of experience, the Diary study is a research method I learned many years ago but never had a chance to practise and apply to product cases before. Compared to many Diary studies I have seen before in ethnographic conferences, this project is considered a much smaller scale one, with shorter timeframes. I’d like to share some of my learnings from my first diary study experience.
Let go of control
As qualitative researchers, we often watch users in action and follow up with questions in real-time, but not in Diary studies. Though we did a couple of pilot tests in advance, I was still anxious if I explained the study clearly enough to get quality data from our users.
A Diary study is not an in-depth interview eventually, many activities and unknown information are collected by a self-reported process, so we can’t expect every user to deliver the same quality of data.
It requires practise to let go of some control and enable users to freely explore on their own without a researcher watching. In the end, it’s important to remember that “it’s okay to collect imperfect data” because you can always follow up later during 1-on-1 interviews.
Allow extra days to collect data if possible
During the study, we learned that non-routine incidents can interrupt users’ behaviour, and impact data collection — we observed that users who had travel plans were less motivated to complete tasks in a limited timeframe. The delayed-report behavior can also be interpreted as — ‘there’s no explicit desire for users to access our app when they are in a non-routine situation.’
Thus, if we want to observe routine behaviour over the course of a week, allowing an extra 3–4 days for users to complete the task will significantly reduce stress for users, and the data would be much closer to their natural behaviours.
Research outcome & Validation
This research surprisingly provides a new perspective and problem area for the team to solve. Here are the top three key insights we learned from the research:
Designated entry VS Merchant logos
There’s a designated entry point to SBMart on the ShopBack home screen, but we found that displaying merchant logos is more attractive, and a quicker identifier for new users.
Ability to use receipt carrier impacted new user’s motivation
In Taiwan, people generally keep their receipts because of the bi-monthly receipt lottery held by the Ministry of Finance. Receipts in Taiwan can be in two forms — digital or printed. If anyone chooses to use a digital receipt, it requires an in-app carrier to store the receipts. Receipt carriers have been widely adopted in Taiwan due to environmental awareness and the convenience to enter the lottery.
Providing a receipt is one of the key actions SBMart users have to take to prove their purchase of an item. We discovered that there’s a big denial from digital-receipt-users because they didn’t observe digital-receipt-options in SBMart. Users also misunderstood SBMart as being available only for printed receipts, due to the visual hints on our marketing campaign flyers.
“I don’t want to use it (SBMart) because it requires printed receipts. Printed receipts are printed with toxic inks on non-recyclable paper, and also occupy space in your wallet, so it’s not a good thing,” said the user.
Search results may lead to drop-offs for search-oriented user
We discovered there’s a type of user who shows search-oriented behavior — they search because they want to minimize time spent on shopping, and focus on specific products with high buying intent while they shop.
This led to the learning that search result accuracy, or how we handle cases where there are no search results can directly impact search-oriented users’ decision to continue using SBMart.
Triangulation of quantitative data
In recent practice, the research team tends to provide quantitative data to validate qualitative findings in the sharing session. We observe it generates more discussion and action items generated in the sharing session, which is favorable to see. There are three ways we leverage quantitative data
Survey results being investigated through diary study:
Existing users prefer Merchant logos to shortcuts: A recent survey we conducted shows existing users prefer to enter SBMart through merchant logos as well. We can further confirm the advantage of showing merchant logos for both types of users.
Diary study results are validated through analytics data.
The majority of our users are digital-receipt users: Existing digital-receipt users can be acquired in an earlier phase when the receipt carrier entry was still on the home screen. We identified the concern from digital-carrier users and the impact when the receipt carrier entry is unclear to users.
Funnel data results are being investigated through a diary study.
Conversion rate is low after search in SBMart: We discovered that the search conversion rate is not doing well. The qualitative finding helps us to learn why and who would care most in the search process. If we’d like to acquire more search-oriented users, we would need to address and improve problems like search result accuracy, and empty states for searches.
Summary
I am hoping this learning will be helpful for research practitioners to reference and learn how to conduct Diary studies for fast-paced projects. Like my reflection mentioned above, there are still many things that can be modified. I am thankful for this opportunity and thankful for the research team allowing researchers to experiment with new methods and quickly learn from them.
If you’re a researcher with past experiences in diary studies and would like to share your thoughts and inputs, we would love to hear them! Feel free to get in touch with us at Research@Shopback.com, follow us on ShopBack Design or Instagram for a chat!
Credit to Naning Utoyo, Iju Hsu, Helen Huang, 陳亭勻 (Selena) for supporting the research process and feedback, and Yishan and Shanty for polishing this article to make it better :)
❗️ Interested in what else we work on?
Follow us (ShopBack Tech blog | ShopBack LinkedIn page) now to get further insights on what our tech teams do!
❗❗️ Or… interested in us? (definitely, we hope 😂)
Check out here how you might be able to fit into ShopBack — we’re excited to start our journey together!
中文版 — ShopBack 日誌研究: OMO聯盟行銷 — 發票回饋 (Part 3/3)
Click to check out Part 1 and Part 2!
前情提要
上一篇文章中我們討論到發票回饋的研究中，我們如何在搜集還有分析過程中，利用不同的工具像是:Line, Excel, Figjam 來幫助我們視覺化資料。這個章節，我們將會去分享我在這個研究中的一些學習還有建議。
關於日誌研究的學習
當一個七年經驗的研究員，日誌研究一直是我學習多年但從未有機會使用的研究方法。過去我在研討會中看見其他研究員所使用的日誌研究，相較起來我的日誌研究的尺度是相對小的專案，時程也比較短。我想要分享一些關於我在這個經驗中的學習。
放手
身為一個質性研究員，我們經常會觀察用戶即刻發生的行為，以及我們經常會有追問問題的辦法，但是日誌研究無法追問問題。雖然我們已經在事前做過幾次的測試，我依然是體驗到很高的焦慮感，焦慮的內容不外乎：我有沒有解釋的夠仔細好讓我拿到足夠好的品質的資料。
就像其他研究方法一樣，日誌研究也是有他的極限，不像是訪談可以隨時追問問題，了解用戶深入的想法。許多用戶的未知行為和活動會是讓用戶自己主動匯報的方式去採集的，所以我們沒辦法期待用戶的能力如同我們一樣，搜集和我們ㄧ樣品質的資料。
然而身為一個研究員，這是需要練習的心法，我們的安心還有信任，可以讓用戶在不被看照下自由自在的探索，做他們自己。最終，研究員依然要記得最重要的一件事情理所當然的事情就是“收集不到完美的資料是可以接受的”，因為我們永遠都有機會去在未來的一對一訪談中去釐清資料採集中不清楚的事情。
準備充裕的時間採集日誌資料
搜集資料途中，我們發現非慣性的事件可能會發生並且影響資料採集的時程。我們觀察到用戶有旅行計畫又被限制在期限中完成任務的時候，他們完成日誌研究的動力就會降低許多。這種拖延性回報的行為當然也可以被理解作為資料的一種 — 當新用戶在非慣性情況中，他們更沒有動力去使用我們的服務。
因此，當我們想要觀察一個超過一週的慣性行為的話，有個額外的三到四天好讓用戶可以完成任務，會大大地減低用戶的壓力，增加他們完成的意願，資料的成果也會更趨近於他們真實的原貌。
研究成果＆驗證
這個研究很有趣的提供了團隊新的視角還有問題去探討和解決。在此和大家分享我們看到的幾個有趣的行為發現：
設計的入口 v.s 商家Logo
即使發票回饋在ShopBack上有個設計的入口，但我們發現商家的入口可以讓新用戶更快的找到發票回饋，而且也更加吸引用戶去點擊。
“能不能使用載具” 會影響新用戶的決定
在台灣多數的人會為了兩個月一次的統一發票兌獎而保存他們的發票，而發票的形式有兩種：可以被印出來，也可以被數位化的存放; 選擇數位存放的人會需要一個數位載體去存放發票，我們稱之為載具。載具在近幾年被廣泛地使用，除了減少印刷還大大增加了結帳效率兌獎效率等等。
用發票提供購買證明，是發票回饋中重要的一個步驟。我們發現現有的數位發票/載具用戶對於發票回饋有強烈的反抗感，而這些反抗感來自於他們“以為”發票回饋只能用紙本發票。然而他們為什麼會有這些誤會，原來來自於我們的傳單印刷，還有載具登入的入口過度隱藏。
”我不想要用發票回饋，因為他要我印出發票。發票上面的油墨很不健康，而且發票的紙不能回收，它還會佔據我錢包很多位置，這不是一個好東西！“用戶說。
搜尋結果會影響“搜尋為主的用戶”
研究當中我們發現有一種用戶只會用搜尋的方式去找到他要的產品，他們搜尋源自於他們想要減少他們在店內消費的時間，並且專注於他們心中已經想要的那個產品。
我們學習到對這種用戶而言，搜尋的正確率，甚至搜尋不到東西都會大大的影響他們繼續使用發票回饋的可能性。
連結量化分析
在ShopBack研究團隊中我們試著提供更多的量化數據來輔佐我們看到的質化數據。我們發現當這兩種數據放在一起並且呼應我們看到的用戶行為的故事時，他可以創造更多的討論。而這也是研究團隊欣然樂見的，這裡我想列出研究團隊如何使用量化數據和質化數據的三種方式：
利用問卷結果去了解現在的新用戶行為
現有用戶喜歡用商家Logo來當成捷徑：之前的問卷研究中，從中了解用戶的入口是來自於商家Logo，因此我們不但從新用戶行為中看到這個行為，現有用戶也是如此。
利用數據分析來輔佐日誌研究的結果
現有的主要長期用戶是載具用戶：現有的載具用戶可能是更早時候我們將綁定載具的入口放在更明顯處。當我們看到“紙本發票”的訊息大過於“可以使用載具”影響到用戶接受度，我們了解了我們必須增加載具的訊息溝通。
Funnel 數據來輔佐日誌研究結果
搜尋後的轉換率偏低：我們發現在發票回饋中搜尋後的轉換率偏低，質化研究的成果告訴我們原因來自於搜尋的結果還有準確率。如果我們想要得到更多”搜尋為主“的用戶，我們會需要讓我們的搜尋準確率更進步，還有避免搜尋結果空白。
總結
希望我的這些學習可以對UXR有興趣的人，正在用UXR工具練習的同行們有幫助。就像我上面有的自省一樣，這過程中還有好多的細節可以進步可以更好。非常感激研究團隊 Naning Utoyo Iju Hsu，發票回饋團隊 陳亭勻 (Selena) Helen Huang 的大力支持，讓我主持這個研究並在這個研究中學習許多。
如果你也做過日誌研究也想要和我們分享您的想法，歡迎和我們聊聊，你可以透過 Research@Shopback.com聯繫我們, 或是追蹤我們 ShopBack Design 或是這裡Instagram！
再次謝謝 Naning Utoyo, Iju Hsu, Helen Huang, 陳亭勻 (Selena) 在專案中的協助還有反饋。謝謝Yishan & Shanty 幫忙我校稿讓這篇文章更好。
❗️ 對我們的工作感興趣嗎?
記得關注我們的 ShopBack Tech blog 還有 ShopBack LinkedIn page: 進一步瞭解我們的技術團隊所做的工作。
❗❗️ 或者… 對我們感興趣？(對～我們希望你是! 😂)
在這裡看看你怎樣加入我們ShopBack 的團隊 — 我們很期待可以和你一起有個新的開始!
