Usability Requirements

Motivation

Electronic Voting (e-voting) systems continue to be used in different countries and contexts around the globe, enabling governments to obtain information on citizens’ preferences more quickly and efficiently. Systems in use are both e-voting machines and remote e-voting systems. Four reasons can be stated why usability is important in e-voting systems. First, due to the election principle of universal suffrage, anyone who meets the voting age requirement should be able to use these systems to cast his vote. This includes first time voters, elderly persons, and even those who do not frequently interact with technology. Second, a voter should be able to easily express his wishes. The interface design should neither cause him to make mistakes nor influence his decision. Poor interface design can easily cause a voter not to cast a vote for his desired candidate. Third, voters remain novices due to a lack of training and irregular interaction with these systems, since elections are held infrequently in many countries and contexts. As such, the learning that occurs from continuous interaction with systems over a period of time is less likely to occur. Finally, if the usability of e-voting systems is not considered, frustration is likely to occur, reducing acceptance among voters, and thus decreasing voter turnout. Usability issues are especially important in e- voting systems that provide verifiability and in particular when using cryptographic verifiability.

There is a lot of literature available on usability studies of e-voting systems. Researchers and developers of future e-voting systems should take this into account, especially if they address the most complex systems, namely verifiable and cryptographically-verifiable remote e-voting systems. 

 

Results

The research group reviewed existing literature on usability of e-voting systems. From the literature review, the lessons learned  for e-voting system interface design, user studies, and usability criteria were summarised, from which relevant recommendations were extracted. 

The reviewed literature was identified by searching scientific literature repositories, such as www.scholar.google.com, and the digital libraries of the Institute of Electrical and Electronics Engineers (IEEE), and the Association for Computing Machinery (ACM). Search criteria included ‘usability and e-voting’, ‘usability and remote e-voting’ and ‘ballot design’. The search was not limited to a specific time period. From the results of the electronic repository search, we reviewed conference papers and journal articles. Further papers and reports were obtained from the references of the identified papers. The papers reviewed were published between the years 1998 and 2012.

The conference papers were proceedings of HCI, usability, e-voting, security, democracy, and governance conferences and workshops. We reviewed these to identify papers that focused on user studies and usability of e-voting systems. These conferences are the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI), the International Conference on Human Computer In- teraction (HCII), the IADIS International Confer- ence Interfaces and Human Computer Interaction, the Symposium on Usable Privacy and Security (SOUPS), the USENIX Security Symposium, the International Conference on E-Voting and Identity (VOTE-ID), the Electronic Voting Technology Workshop/Workshop on Trustworthy Elections (EVT/WOTE), the International Workshop on Requirements Engineering for Electronic Voting Systems (REVOTE), the International Confer- ence on Electronic Governance (EGOV), the International Conference for E-Democracy and Open Government (CEDEM), the International Conference on Theory and Practice of Electronic Governance (ICE-GOV), the Workshop on Socio- Technical Aspects in Security and Trust (STAST), and the International Conference on Information Integration and Web-based Applications and Services (IIWAS). We included a paper (Everett, Byrne, & Greene, 2006) that was published in the proceedings of the Annual Meeting of the Human Factors and Ergonomics Society (HFES). References from the identified papers were reviewed and further relevant papers identified and selected for inclusion.

Relevant journals were from the fields of usability, user studies, computing, electronic governance, information design, and political science. Here we reviewed articles from the Journal of Usability Studies, International Journal of Human- Computer Studies, IEEE Transactions on Informa- tion Forensics and Security, International Journal Universal Access in the Information Society, the European Journal of Information Systems, the Social Science Computer Review, the International Journal of Electronic Governance, Information Design Journal, American Politics Research, the Journal for the American Political Science Association, Perspectives on Politics, and Public Opinion Quarterly, to identify relevant papers. Once again, the references from the selected papers were reviewed to identify other relevant articles.

The resulting recommendations aim at three groups of researchers: those interested in designing e-voting system interfaces, those interested in carrying out user studies in e-voting, and those seeking for further research in this field. The recommendations adress different types of electronic votin systems, such as voting machines with paper audit trails or cryptographically verifiable systems, and include further recommendations on future work in these domains. Other recommendations focus on the evaluating the usability of the voting systems, such as designing and conducting user studies. The list of all the recommendations is as follows:

Designing electronic voting systems

  • R-ID-BD-1: Ballot design should be standardized such that the ballots are familiar to voters, for example, imitating the pa- per ballot design on the e-voting system interface.

  • R-ID-BD-2: The ballot should indicate to voters when their vote has been success- fully cast and should clearly indicate if the vote casting process has been completed.

  • R-ID-BD-3: The interface should inform voters if their vote is invalid based on the selections they have made.

  • R-ID-BD-4: Researchers should use the bubble ballot as a standard design where the ballots and candidate listing supports it.

  • FW-ID-BD-1: Future research should investigate the design of large ballots on e- voting systems and their usability.

  • R-ID-BI-1: Simple and clear instructions should be designed for ballots.

  • R-ID-BI-2: The consequences of an action in an instruction should precede the call to act.

  • R-ID-BI-3: Each instruction sentence on the ballot should give the result of the ac- tion before stating the action to be carried out.

  • R-ID-BI-4: The instructions should use words that are familiar to voters, and ad- ditionally should match the order of tasks on the ballot, in a logical sequence.

  • R-ID-BI-5: Instructions should be placed at the upper left-hand corner of the ballot, and should be given before the task to be carried out, for instance, having instruc- tions how to mark a candidate correctly, just above where this action will be carried out.

  • R-ID-RS-1: Review screens should be im- plemented in e-voting systems.

  • R-ID-RS-2: Voters should be instructed to pay attention to the review screen, in or- der to ensure that they notice any changes made to their votes.

  • R-ID-RS-3: Techniques such as additional coloring or highlighting should be used to draw voters’ attention to races where they have not yet voted.

  • FW-ID-RS-1: Future research should in- vestigate how to instruct voters to pay more attention to the review screen especially for them to notice when their candidate selec- tions are modified.

  • R-ID-TS-1: Researchers should aim to reduce the amount of time and effort that voters need to take in order to cast their vote, that is, the number of voting tasks should be reduced, in particular regarding poll workers and voters enabling e-voting machines or in checking the voter’s right to vote in Internet voting.

  • R-ID-TS-2: Care should be taken in facilitating voters to vote quickly, as in direct ac- cess DREs, since faster voting may lead to more voter errors.

  • FW-ID-TS-1: Further research should in- vestigate how much time voters would be willing to spend in carrying out voting tasks, and how much time they would con- sider to be too much.

  • R-ID-HF-1: It is recommended that de- signers integrate help facilities to give vot- ers information when they need it and to guide them on what next steps are required. In Internet voting, for example, the help fa- cilities should be placed on every page the voter will access on the e-voting system, as well as next to tasks that are likely to be confusing for voters.

  • FW-ID-HF-1: Future research should in- vestigate appropriate help for voters given the different voting contexts, for example hotlines and email in Internet voting.

  • R-ID-ED-1: Voters should be educated be- fore introducing new e-voting technology.

  • R-ID-ED-2: The techniques used for edu- cating voters, for example, videos or handouts, should take into account the diversity of voters, in terms of age, experience with voting, and education.

  • FW-ID-ED-1: Future research should in- vestigate effective means of voter educa- tion, particularly to introduce voters to, and enable them to use, new e-voting systems and new approaches to voting, such as verifiable e-voting systems.

  • FW-ID-ED-2: As poll workers also re- quire education for new e-voting systems and approaches, and as this has not yet been addressed comprehensively in the lit- erature, future research should investigate poll worker education.

  • R-ID-MM-1: Voters’ mental model should be investigated as voter confusion may be due to differences between how voters think an e-voting system should work, and how it has been designed and implemented.

  • R-ID-MM-2: Voters should be educated about verifying their vote to deal with the problem of confusion and secrecy concerns.

  • FW-ID-MM-1: Future research should investigate specific approaches to extending voters’ mental model to take into ac- count new e-voting systems and voting approaches, as the literature surveyed does not give information on how to close the gap identified.

  • R-ID-VV-1: Voters should be provided with clear instructions that are easy to understand and follow, guiding them on how to check their vote selection when VVPATs are provided.

  • FW-ID-VV-1: Further research is neces- sary to identify the best approach for instructing voters, as well as to further inves- tigate voter challenges with VVPATS.

  • FW-ID-VV-2: Voters’ mental model regarding VVPATS should be investigated in future research.

  • R-ID-CV-1: Voters should be provided with clear instructions that are easy to understand and follow to verify whether their vote is properly encrypted (cast as intended) and stored (stored as cast).

  • R-ID-CV-2: Appropriate help features should be integrated for cryptographically- verifiable e-voting, taking into account the different types of voters, ranging from first- time voters, to frequent voters.

  • R-ID-CV-3: Voter education is recommended in cryptographically-verifiable e-voting as voters may not understand the need for verifiability, and instead expect a secure system that does not require one to carry out verifiability steps.

  • FW-ID-CV-1: It is recommended that vot- ers’ mental model and voter education be investigated in future research, with the aim of eliminating confusion and the re- luctance to carry out the necessary verifi- ability steps.

  • FW-ID-CV-2: Further research should investigate voters’ view of the amount of time it takes to verify votes in the case of cryptographically-verifiable e-voting systems, whether they find it too long or acceptable.

Evaluating usability of the electronic voting systems

  • R-US-RM-1: We recommend the following methodology in conducting usability studies of electronic voting systems: A usability study should begin with an evaluation involving experts, after which changes can be made to the e-voting aspect under study based on the feedback received. As a second step, a user study should be carried out. Note that be- fore the user study is carried out, pilot studies are necessary to identify any difficult or unclear issues in the study design. If a lack of understanding of the e-voting system aspect under study is observed, the vot- ers’ mental model should be investigated. Based on feedback from participants after the user study, the e-voting should be re- designed. The re-design should be tested in subsequent user studies, and several iterations at this stage may be necessary, switching between re-design and small user studies for user feedback. Finally, field studies should be carried out, where the re- designed e-voting system can be tested in a real election with real voters. Exit polls should accompany the field studies, to ob- tain voters’ feedback on the e-voting sys- tem, and related aspects being studied.

  • R-US-EV-1: Ballots used in user studies should be similar to those used in real elec- tions to minimize confusion and the num- ber of errors from participants.

  • R-US-EV-2: Researchers should select one of the following approaches to provide ecological validity: a) The ballot used in the study should be similar to a real bal- lot, for example based either on the candi- dates listed, the design of the ballot, or the number of races provided. b) The voting environment should be similar to that of a real election, either by the voting machines used, or holding the election in a location where real elections are held, for example in a town hall. c) Giving voters tasks simi- lar to tasks in a real election, for example, picking a ballot paper, marking their bal- lot in the voting booth, and dropping their ballot in a ballot box. Though this example is for polling station and paper-based elec- tions, the same can be used for Internet vot- ing where voters are made aware that they are in the voting booth (marking candi- dates) and dropping their ballot in the bal- lot box (submitting the vote). d) Running an election for which participants are more likely to be interested in the results, for ex- ample a charities’ election. 

  • R-US-EV-3: This recommendation includes suggestions that are optional for the researcher. Where mock elections are set up for user studies, they should give a realistic feel to the participants, for ex- ample, in the design of the ballot, or in the location of the study. Fictitious candidates can be included in ballots for user studies. As argued in the literature surveyed, this increases the life span of the research in- struments used, and they do not become obsolete in a short period of time. User studies can also be set up in the participants’ natural environment, or in the case of Internet voting, should use the partici- pant’s equipment where possible, in order to be realistic.

  • R-US-VS-1: Where possible, researchers should preserve vote secrecy of the study participants, or inform participants when it will not be preserved. As an example, researchers can direct vot- ers which candidates they should vote for, but should not link the vote to the partici- pants (for example, by randomly giving voter slates in the study). Researchers can then compare expected votes with actual votes to identify error the rate.

  • R-US-IP-1: Researchers should offer financial or in-kind incentives to participants in user studies.

  • FW-US-IP-1: Future research should in- vestigate if incentives will improve partici- pants’ engagement in e-voting studies.

  • FW-US-IP-2: A second question should explore the role of intrinsic motivation in engaging participants in e-voting user studies, for example, by motivating them to participate in studies in order to be part of improving the usability of e-voting systems.

  • R-US-NP-1: It is recommended that researchers determine the number of participants for their e-voting studies based on the resources available, the study design, and whether statistically significant results are required. For statistically significant results, statistical techniques can be used, based on the desired degree of confidence, to identify the number of participants from the sample population. If statistically significant results are not required, studies should have, as a minimum, 15-20 participants, depending on the goals of the study.

  • R-US-NP-2: Field studies should have a large number of participants, as the results will be representative of a larger population.

  • R-US-VIP-1: Researchers should not require participants to remember voting tasks, and instead should provide both written and verbal instructions on what tasks participants are expected to carry out.

  • R-US-EI-1: Usability study design as well as tasks for participants should be re- viewed to ensure that they do not violate ethical requirements.

  • R-US-EI-2: Participants should be informed about the goals of the study either before or after the study.

  • R-US-EI-3: Participants should sign consent forms before participating in e-voting usability studies.

  • R-US-EI-4: Where a university ethics board or institutional review board is available, these should check that the study de- sign and materials consider ethical issues. Where these bodies are not available, or cannot offer necessary guidance, researchers should take the responsibility of separately reporting how they have met standard ethical requirements.

  • FW-US-EI-1: Future research in usability of e-voting systems should investigate how to handle ethical issues, besides requiring full disclosure to participants as this may not match with e-voting study objectives.

  • R-US-ET-1: Researchers should use equipment whose development has been completed and tested, in order to avoid er- rors arising in actual user studies.

  • R-UC-BD-1: Usability evaluations of traditional e-voting systems should be carried out with the specific objective of providing baseline data to allow for comparison between traditional and new e-voting systems.

 

Publications

E-Voting System Usability: Lessons for Interface Design, User Studies, and Usability Criteria: Olembo, M.; Volkamer, M. 2013. Human-centered system design for electronic governance. Ed.: S. Saeed, 172-201, Information science reference, Hershey, PA. doi:10.4018/978-1-4666-3640-8.ch011