Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
public:papers:cacm2018 [2017-11-25 22:24] – [Summarizing video] xukroppublic:papers:cacm2018 [2020-05-14 11:38] (current) xukrop
Line 2: Line 2:
 ~~NOTOC~~ ~~NOTOC~~
  
-//The study of online security software explains the importance of knowing who the beta testers are.//+<grid> 
 +<col xs="12" sm="8" lg="8"> 
 +<TEXT size="large">
  
-  * **Authors:** Vlasta Stavova, Lenka Dedkova, Martin Ukrop and Vashek Matyas +\_{{fa>user}}\_\_//Authors:// [[:publications:authors:vlasta-stavova|Vlasta Stavova]], Lenka Dedkova, [[:publications:authors:martin-ukrop|Martin Ukrop]] and [[:publications:authors:vashek-matyas|Vashek Matyas]]
-  * **Primary contact:** Vlasta Stavova <vlasta.stavova@mail.muni.cz> +
-  * **Journal:** [[https://cacm.acm.org/ Communications of the ACM]], issue XX +
-  * **Author pre-print:** //To be added later// +
-  * **Citation:** {{fa>file-code-o|BibTeX}} ++ BibTeX |<code>@Article{stavova2017study, +
-  title = {A large-scale comparative study of beta testers and standard users}, +
-  author = {Vlasta Stavova and Lenka Dedkova and Martin Ukrop and Vashek Matyas}, +
-  journal = {To appear in Communications of ACM}, +
-  year = {2018}, +
-  keywords = {usablesec, ESET}, +
-  organization = {ACM}, +
-+
-</code>+++
  
-----+{{fa>user-circle-o}}\_//Primary contact:// Vlasta Stavova %%<%%<vlasta.stavova@mail.muni.cz>%%>%%
  
-===== Key insights =====+{{fa>bullhorn}}\_//Journal:// [[https://cacm.acm.org/|Communications of the ACM]]
  
-  * Use data you can collect to find out who your users and beta testers areThe fewer testers you have, the pickier you should be about their selection. +\_{{fa>id-badge}}\_\_//DOI:// [[https://doi.org/10.1145/3173570|10.1145/3173570]] 
-  * Testers should be representative of standard users. Keep checking that this is the case – or remediate with additional analyses and more carefully reached conclusions.+</TEXT> 
 +</col>
  
-===== Summary =====+<col xs="12" sm="4" lg="4"> 
 +<TEXT align="right">
  
 +<button type="warning" icon="fa fa-file-pdf-o">[[https://crocs.fi.muni.cz/_media/publications/pdf/2018-cacm-stavova.pdf|Pre-print PDF]]</button>
 +
 +<button type="info" icon="fa fa-external-link">[[https://dl.acm.org/citation.cfm?id=3173570|Final version (ACM DL)]]</button>
 +
 +<button collapse="bibtex" icon="fa fa-file-code-o">BiBTeX</button>
 +</TEXT>
 +</col>
 +</grid>
 +
 +<collapse id="bibtex" collapsed="true">
 +  @Article{2018-cacm-stavova,
 +    Title = {A Large-scale Comparative Study of Beta Testers and Regular Users},
 +    Author = {Vlasta Stavova and Lenka Dedkova and Martin Ukrop and Vashek Matyas},
 +    Journal = {Communications of the ACM},
 +    Year = {2018},
 +    Volume = {61},
 +    Number = {2},
 +    Pages = {64--71},
 +    Publisher = {ACM},
 +    Doi = {10.1145/3173570},
 +  }
 +</collapse>
 +
 +<panel type="default" title="Abstract">
 Beta testers are the first end users outside a software company to use its product. They have been used for decades and are rightly credited not only with finding and reporting bugs, but also with improving general product usability through their feedback and/or the ways they use the product. Beta testers are the first end users outside a software company to use its product. They have been used for decades and are rightly credited not only with finding and reporting bugs, but also with improving general product usability through their feedback and/or the ways they use the product.
  
Line 34: Line 50:
  
 We conclude that - at least in our study - beta users represent standard users well in terms of hardware and operating system in large scale beta testing. However, populations differ significantly in the distribution of users and testers between countries. This may cause a problem when a testing includes localization and usability issues that may be influenced by regional differences. We conclude that - at least in our study - beta users represent standard users well in terms of hardware and operating system in large scale beta testing. However, populations differ significantly in the distribution of users and testers between countries. This may cause a problem when a testing includes localization and usability issues that may be influenced by regional differences.
 +</panel>
  
 ===== Summarizing video ===== ===== Summarizing video =====
Line 39: Line 56:
 {{ youtube>H7ik9SeAexQ?900x520 |A large-scale comparative study of beta testers and standard users}} {{ youtube>H7ik9SeAexQ?900x520 |A large-scale comparative study of beta testers and standard users}}
  
-===== How did we make the research video =====+===== Key insights =====
  
-Some textevetually "making-of" video will be embedded.+Our research produced the following actionable takeaways for software developers: 
 +  * **Using data**\\ Data you can collect can help you learn who your users and beta testers are. Consider country of originsoftware and hardware configuration, and basic demographics. 
 +  * **Selecting testers**\\ The fewer testers you have, the pickier you should be about their selection. 
 +  * **Identifying usability issues**\\ When testing international products, ensure beta testers are culturally representative of regular users to help identify potential localization and cultural usability issues. 
 +  * **Ensuring representation**\\ Most important, testers should be representative of regular usersKeep checking that this is the case or pursue additional rigorous analyses to reach the most credible and applicable conclusions possible. 
 + 
 +===== How did we make the research video =====
  
 +{{ youtube>xIrJtydnjmk?900x520 |Making of: "A large-scale comparative study of beta testers and standard users"}}
 ===== Other materials ===== ===== Other materials =====
   * [[https://is.muni.cz/th/410304/fi_m/ |Diploma thesis]] based on localization data from this project (in czech only). It compares free available online GeoIP databases on a large sample of available IP adresses.    * [[https://is.muni.cz/th/410304/fi_m/ |Diploma thesis]] based on localization data from this project (in czech only). It compares free available online GeoIP databases on a large sample of available IP adresses.