Pearson VUE @ ATP

Session Details

Monday, March 18, 2019
11:30 a.m. – 12:30 p.m.  |  Room: Bayhill 21 & 22

Industry Standards for the Implementation of Automated Scoring Technologies

Erin Yao, ACT
Andre Rupp, ETS
Scott Wood, ACT
Susan Lottridge, American Institutes of Research
Cathy Wendler, ETS
Karen Lochbaum, Pearson

Session Description:
The use of automated scoring technology is becoming more and more prevalent in today's testing programs. As this technology evolves and becomes more frequently used in large-scale testing programs, it is crucial that robust standards for its implementation exist and are used. Four organizations who are industry leaders in automated scoring technologies have collaborated to create an initial version of industry-wide standards that can be used as a guideline for testing programs to consider as they implement automated scoring in their programs. Standard topics to include: evaluating the performance of automated scoring models, monitoring operational automated scoring systems, maintaining automated scoring models, and how to communicate and demonstrate the validity of automated scoring to assessment stakeholders.


Monday, March 18, 2019
11:30 a.m. – 12:30 p.m.  |  Room: Celebration 9 & 10

A New Way to Test: Creating and Using Animated Scenarios for a Licensure Exam

Cheryl Paullin, HumRRO
Matt Turner, Director of Examination Services, ASPPB
Tina Riner, Pearson VUE

Session Description:
Animated Scenarios are changing the way we assess a candidate's true knowledge and skills. In this session, we'll dive into the initial launch of a new psychology exam, showing you how Animated Scenarios put a candidate in a simulation of a licensed psychologist and assess whether they use their skills to react appropriately. This secondary skills-based exam allows for more item types than just a basic multiple choice item and will better test candidates' skills. The Animated Scenario may present multiple stimuli and multiple items associated with the animation, and items are non-branching in nature.

We'll discuss how items and animations from the initial launch were created, including timelines and the processes used. In addition, we'll show how we identified areas of the blueprint where Animated Scenarios might apply and we'll discuss our initial successes and opportunities for improvement in the future.


Monday, March 18, 2019
2:00 p.m. – 3:00 p.m.  |  Room: Bayhill 18

SPONSOR SESSION (Pearson VUE): The Computer-Based Testing Industry is Constantly Evolving. Are You?

Joe Brutsche, Pearson VUE
Bryan Friess, Pearson VUE
Casey Kearns, Pearson VUE
Peter Pascale, Pearson VUE

Session Description:
While the industry is constantly evolving, one thing is constant: our commitment to advancing the computer-based testing industry by continually investing in core computer-based testing technologies. From cloud technology and security enhancements to dynamic exam development and delivery options, join Pearson VUE for an exciting conversation about where our industry — and our business — is going. We’ll help you cut through the noise so you understand the new technologies available to you to diversify and expand your program. Considering online proctoring but wondering whether its security features are stringent enough? Curious about how performance-based testing might add value to your exam, but not sure where to start? Want to know what we’re considering for the latest enhancements to ID verification? We’re leading the way in the evolution of modern computer-based testing. Join us for a focused discussion to help you navigate the latest options available to you and support you in making strategic decisions about where your program is going next.


Monday, March 18, 2019
4:00 p.m. – 5:00 p.m.  |  Room: Bayhill 25 & 26

Performance-Based Testing: Microsoft’s Journey from Desktop to Cloud

Dan Tanedo, Microsoft
Liberty Munson, Microsoft
Casey Kearns, Pearson VUE
Kishore Vakati, Pearson VUE

Session Description:
Performance-based testing is emerging as an effective way of assessing candidates' skills, allowing them to demonstrate their breadth of knowledge in a particular area. Although incorporating scored lab activities into assessments provides a higher fidelity to the real-world task that candidates will perform, doing so often introduces challenges in integrating multiple live systems and applications during exam delivery.

This presentation will showcase several innovative solutions for delivering live-in-the-app testing for a diversified set of products. We'll discuss the different challenges and solutions Microsoft applied to a spectrum of performance-based exams from desktop to cloud applications. We'll also navigate considerations for implementation, such as leveraging different lab providers and modality of deliveries for lab exams, and provide ideas and suggestions for simplifying performance-based exams.


Tuesday, March 19, 2019
8:30 a.m. – 10:00 a.m.  |  Room: Celebration 3 & 4

FEATURED SPEAKER SESSION: Meeting the Data Privacy Challenge: Complying with Multiple Laws in a Global Testing Environment

Marc Weinstein, Caveon
Gary Behrens, FifthTheory, LLC
Allison Mulford, Prometric Inc.
Steven Stransky, Thompson Hine LLP
Cordell Jung, Pearson PLC

Session Description:
Data privacy specialists within and outside the testing industry will discuss the challenges faced by assessment programs in complying with a proliferation of data privacy laws around the country and around the world. Panelists will offer pointers and ideas for achieving and maintaining compliance based on experience and insights. Attendees will be invited to ask questions and share perspectives as we seek consensus on the similarities of, and conflicting requirements between, privacy laws in different jurisdictions that may help inform best practices.


Tuesday, March 19, 2019
10:30 a.m. – 12:00 p.m.  |  Room: Bayhill 23 & 24

Guerrilla Tactics for Prototyping and User Testing

Ann Evers, GED Testing Service
Brittany Johnson, Independent Consultant
Cynthia Parshall, Touchstone Consulting
Ruth Ramstad, Pearson

Session Description:
Rapid item prototyping and user testing, like agile development, help you to learn quickly, discover quickly, and pivot quickly. Guerrilla tactics means doing these things as rapidly and inexpensively as possible. For example, an item screen can be prototyped on paper, to save on resources and budget early in the user testing process.

Prototyping and user testing are important activities because any change to the item/test interface will impact the candidate's experience and has the potential to affect validity. To ensure that your innovations make sense to your candidates and support good measurement, you should try things out early and iterate often, leading to valuable improvements.

The workshop will include a heavy emphasis on user testing activities and how-to approaches. Session participants will get hands-on experience creating a paper prototype, observing how people interact with it, and extracting findings to brainstorm improvements for a sample project.


Tuesday, March 19, 2019
10:30 a.m. – 12:00 p.m.  |  Room: Bayhill 29 & 30

Shifting to a Design Science Paradigm to Develop Desirable, Feasible, and Viable Tests

Alex Casillas, ACT Inc
Ellen Forte, edCount, LLC
Emily Lai, Pearson

Session Description:
In the testing industry, product design and development can experience a number of challenges (e.g., high development costs, dated technology, poor UX, lack of coordination among stakeholders, insufficient validity data) while trying to address market problems and move at the speed of business. In this workshop, we present an introduction to design science, three tenets of design (desirability, feasibility, viability), and the tool of theory of action to guide the design and development of the next generation of testing products. This workshop will provide both "translational" and "hands-on" opportunities for testing professionals to gain exposure for how design science principles and tools can be used effectively by assessment designers. We will conclude by discussing attendees' mini theories of action, as well as pain points in design and how iteration and feedback can be used to address them.


Tuesday, March 19, 2019
1:15 p.m. – 2:15 p.m.  |  Room: Celebration 9 & 10

Back to the Future: Insights from Delphi on the Testing Industry

Joshua Goodman, National Commission on Certification of Physician Assistants
Kirk Becker, Pearson Assessments

Session Description:
The Delphi method provides an iterative process in which participants answer questionnaires in multiple rounds and feedback is presented from the previous round. This allows participants to discuss and modify their positions, and typically results in more accurate forecasts than large questionnaires or in-person meetings. In 2010, a Delphi study was conducted to explore the likelihood of different trends in the testing industry, and results were presented at ATP. In 2019, we will present a follow-up study on future directions in the testing industry, which will be presented and discussed by a panel of industry experts.


Tuesday, March 19, 2019
1:15 p.m. – 2:15 p.m.  |  Room: Celebration 14 & 15

Increasing Test Program Efficiency Using Computerized Adaptive Testing (CAT)

Casey Marks, (ISC)2
Anthony Zara, Pearson VUE

Session Description:
All test users are under growing pressure for increased efficiency, exam security, improved test formats, flexible scheduling, better measurement, and more efficient administrative processes. Although meeting each of these requirements seems like a daunting task, there is one proven testing methodology has the capability to make inroads to improving all of them: Computer Adaptive Testing (CAT). This session will discuss the programmatic efficiencies that CAT enables. This session will also provide concrete examples of how one program was able to accomplish changing program modalities from paper-and-pencil to CAT. We will describe the psychometric, technical, and business considerations that resulted in a successful conversion. Attendees will gain an understanding of the key issues that lead to success and failure in a computer-based testing program conversion and they will have an opportunity to have their conversion-related questions answered by experienced practitioners.


Wednesday, March 20, 2019
9:45 a.m. – 10:45 a.m.  |  Room: Bayhill 17 & 18

Why Do We Need to Talk about Threatening Test Taker Behavior?

Kendra Johnson, Graduate Management Admission Council
Debbie Bergtholdt, Pearson VUE

Session Description:
Unfortunately, in the world we live in, threatening behavior is a reality. People are angry and they are taking it out on everyone. In this session, we will discuss how to identify and deal with threats from test takers when they are in the process of applying for accommodations or processing through the test center.

We will look at threats from candidates that call because they aren't happy about their accommodation determination or their test scores. Inappropriate or disruptive behavior at the test center will be discussed. Whether you do your own testing, or your candidates test somewhere else, if there is misbehavior, you will likely be asked to weigh in on the situation.

There will be a discussion of the legal ramifications of how we deal with threatening behavior. We will explore a threat assessment policy. Do you or should you have one? Join us and share.


Pearson VUE

Visit us at ATP booth #207

For more information on ATP:
Click here

Innovations in Testing
Exhibit Hall Hours & Events

Sunday, March 17

6:00 p.m. - 7:00 p.m.


Opening Reception


Monday, March 18

7:30 a.m. - 8:30 a.m.

Breakfast with Exhibitors

11:15 a.m. - 11:30 a.m.

Coffee Break with Exhibitors

12:30 p.m. - 2:00 p.m.

Lunch with Exhibitors & Product Demos

3:00 p.m. - 4:00 p.m.

Dessert with Exhibitors

5:30 p.m. - 7:00 p.m.


Reception with Exhibitors and ePoster Sessions


7:30 p.m. - 10:00 p.m.

Pearson VUE party


Party information


Tuesday, March 19

7:30 a.m. - 8:30 a.m.

Breakfast with Exhibitors

10:00 a.m. - 10:30 a.m.

Break with Exhibitors

12:00 a.m. - 1:15 p.m.

Lunch with Exhibitors

2:15 p.m. - 4:15 p.m.

Dessert with Exhibitors