Innovating for exam programs facing an ever-evolving industry.

Join us at ATP 2024!

We’re proud to sponsor this year’s Innovations in Testing conference, and we’ve put together a slate of sessions with something for everyone. Get all the details below. And be sure to join us on the roof at this year’s Surf Shack-themed party!
 


Session details

Monday, March 4

Peas in a Pod
10:00 a.m. – 10:50 a.m.

How Can We Measure How Well Testing Programs are Doing in DE&I?


Speaker(s):
Tamika Hughes, Pearson VUE
John Kleeman, Learnosity

Session Description:
Equity, Accessibility, and Diversity (DEI): Methods and practices for ensuring fairness, accessibility, and accommodations for diverse stakeholders.

 

Panel Discussion (Featured Speaker Session)
11:10 a.m. – 12:00 p.m.

Generative AI in Test Development (Continuing the Conversation)


Speaker(s)
Eric Meyer, Pearson VUE
Marc Weinstein, Marc J. Weinstein, PLLC and Caveon, LLC.
Pat Ward, ITS
Liberty Munson, Microsoft

Session Description:
We have learned so much since our first conversation at ATP in 2023. Much has changed over the past year, and this panel is ready to continue the conversation. This panel will incorporate all the impacts of the test development process from legal, human, program, and technology. What has changed? What are the current views? How are programs adapting? How has the legal landscape changed? Are there new outcomes or tools to look for?

 

Panel Discussion (Featured Speaker Session)
1:20 p.m. – 2:30 p.m.

Navigating Change in the Assessment World: CEO Insights on Impact on the Industry


Speaker(s)
Gary Gates, Pearson VUE
Brodie Wise, ITS
Pat Ward, ITS
Janet Garcia, PSI Services
Tim McClinton, Meazure Learning
Stuart Udell, Prometric

Session Description:
In an era of dynamic change, the assessment industry is undergoing a profound transformation. Hear from a few C-Level leaders in our industry that will discuss the industry's evolution, dissect its trajectory, and confront imminent challenges. These leaders grapple with diverse concerns, including the disruptive influence of emerging AI technologies, the imperative to expand their businesses, and the necessity of reimagining credentialing processes in a global context marked by political and economic fluctuations.

Chief among their priorities is the protection against sophisticated security threats. Simultaneously, they prioritize strategic investments to address future needs while maintaining growth commitments. These leaders are committed to addressing the evolving landscape of testing, tackling challenges such as introduction to generative AI, redefining the value of standardized assessments in academic admissions, and striking a balance between user experience and security. The role of online platforms (OP) and test centers is central to their discussions, as they contemplate how to invest in new products and offerings while upholding their growth commitments.

In an industry marked by rapid change, these leaders are poised to navigate the future with foresight and innovation.

 

Peas in a Pod
4:10 p.m. – 5:00 p.m.

Never Mind the MCQ: The Current State of Alternative Items Research


Speaker(s)
Kirk Becker, Pearson VUE
Cynthia Parshall, Touchstone Consulting
Belinda Brunner, Assessment Systems Corporation

Session Description:
Alternative item types (sometimes called innovative or technology-enhanced items) encompass a wide range of formats for presenting and collecting assessment information. In 2023, a special issue of the Journal of Applied Testing Technology compiled research and practical guidance on the use of alternative item types. Articles in that special issue include construction and use of clinical judgment tasks, the current state of automated scoring, digital performance assessments, effective analysis techniques, and more.

Join this session to take part in a conversation between attendees and authors, with insights and practical advice on the use of new item types for specific purposes. Session attendees will be able to ask the authors detailed questions about their choices, methods, and findings. In addition, attendees are encouraged to bring their experiences, challenges, and questions with implementing alternative item types into their exam programs.

 

Presentation
4:10 p.m. – 5:00 p.m.

Crafting Good AI-Generated Test Items:  Prompt Engineering for the Rest of Us


Speaker(s)
Paul Jones, Pearson VUE
Vinita Talreja, AWS
Mamiko Ouazzani, IIA

Session Description:
The availability of generative artificial intelligence (AI) has produced both excitement and apprehension in the testing community, especially as a potential tool to aid in item development. While simple requests for items may produce flawed and relatively low-level content, it is possible to get well-constructed items across a range of cognitive levels with the right prompt. Join this session to discover the steps taken by several organizations to produce effective prompts and evaluate the items generated by those prompts. We’ll also explore the potential to move from drafting large numbers of specific items to instead editing, evaluating, and improving items, as well as examining new formats and ways of asking questions.

 

Presentation
4:10 p.m. – 5:00 p.m.

Introducing a Virtual Open Book Experience on Microsoft Certification Exams


Speaker(s)
Kimberly Clausen, Pearson VUE
Pat Hughes, ITS
Liberty Munson, Microsoft

Session Description:
As part of Microsoft’s ongoing efforts to increase the accessibility and real-world relevance of our certification exams, an open-book experience was incorporated into our high-stakes exams, giving candidates the ability to access Microsoft Learn (https://learn.microsoft.com) during the exam. This was added as another resource that could be leveraged during the exam; nothing else about the experience, including exam content and time, changed.

To maintain the integrity of the exam, implementation of this resource required a secure browser solution that limited web access to ONLY the Microsoft Learn domain. To complicate matters, we needed to block access to Microsoft Learn’s Q&A and profile experiences to ensure that people couldn’t leverage either to cheat, and the experience needed to meet Microsoft’s strict accessibility requirements.

This was a complex integration that required a close partnership between Microsoft, ITS, and Pearson VUE and careful testing to ensure that there were no holes in the experience that would allow candidates to access unauthorized websites. During this session, we will discuss the reason for adding Microsoft Learn as a resource as well as the design, development, testing, implementation, lessons learned, and feedback from key stakeholders and candidates. We will also share data on usage, impact on passing rates, and other key psychometric results that will help programs determine if adding a similar resource to their exams should be considered.

 

Presentation
4:10 p.m. – 5:00 p.m.

Guardians of the Galaxy… a.k.a. Your Certification Program


Speaker(s)
Joe Brutsche, Pearson VUE
Jonathan MacKesson, Amazon
Pete Van Dyke, Amazon

Session Description:
For most organizations, implementing a traditional web monitoring program can be daunting and time consuming. Then, there are the constant alerts to investigate, organize, triage… and then determine if they’re something you should spend time reviewing. And once you’ve confirmed it is something to dig into, you suddenly realize the adventure has just begun and the threat it poses. What you need is a web-monitoring program and team to be the “guardians” of your certification program—a rag-tag group that can help you focus your attention and maybe save the universe (or at least your exam program).

In this session, join two well-known IT certification programs to learn about how they assembled their own group of “avengers” to build and execute their web-monitoring programs. Each of the programs has taken non-traditional approaches toward launching their programs, gaining internal support, and seeing them through to success. Join us to learn how their approaches can be applied to your program.

 

Tuesday, March 5

Peas in a Pod
8:30 a.m. – 9:20 a.m.

Envision the Possibilities:  Using Virtual Reality in High-Stakes Testing


Speaker(s)
Erin Raw, Pearson VUE

Session Description:
A new, emerging technology for the assessment space, Virtual Reality (VR) presents exciting opportunities to assess candidates from all kinds of fields. For example, those who hope to become teachers could be given scenarios on classroom management. Aspiring doctors tested on their bedside manner. Prospective lawyers challenged to present opening arguments in a trial scenario. With VR, the possibilities are truly endless. But designing exams with VR, along with making decisions on scoring/validity and ensuring a good candidate experience all require innovative thinking. In this Peas in a Pod session, we’ll tackle all of it, as we discuss the use cases, benefits, and challenges for VR in assessment.

 

Panel Discussion
8:30 a.m. – 9:20 a.m.

Managing Test Security Challenges for High Stakes Languages Assessment in the Post Pandemic Era


Speaker(s)
Joe Brutsche, Pearson VUE
Ramond Nicosia, ETS
Ardeshir Geranpayeh, Cambridge Computational Psychometrics
Fiona Mitchell, British Council
André Kenji Horie, Duolingo

Session Description:
Join a panel of test security experts from major international testing organizations to discuss how they managed test security challenges during and post pandemic. Learn how embracing change and new technology has allowed testing organizations to not only manage unforeseen disruptions to standardized testing administrations but also allowed them to enhance their test security and fairness in the process.

 

Workshop
9:40 a.m. – 10:50 a.m.

Exam Cheating – Workshop on Tools and Techniques to Catch the Bad Guys


Speaker(s)
Harry Samit, Pearson VUE
Brent Morris, Cisco
Jonathan MacKesson, Amazon

Session Description:
This workshop will provide success stories from different sponsors that have produced hundreds of invalidations and exam enforcements that have protected the integrity of our certifications programs. Through these success stories, our different sponsors will show you under-the-hood examples of different security techniques used. We will provide an engaging workshop that will involve the audience in hands-on security crafts that have been so successful.

 

Presentation (Sponsor Session)
3:00 p.m. – 3:50 p.m.

The Measurement Mandate: Balancing the Promise of AI with the Indispensable Value of Psychometric and Assessment Validity


Speaker(s)
Kellie Britten, Pearson VUE
Joe Brutsche, Pearson VUE
Eric D'Astolfo, Pearson VUE
Tracy Sartaj, Pearson VUE

Session Description:
The immense promise of artificial intelligence (AI) is being fulfilled a bit more every day. And by now it’s clear that AI is here to stay — its use quickly and increasingly becoming an expectation rather than a novelty. In the testing industry, AI’s potential means it will be used more frequently, for everything from capacity planning and proxy tester identification to customer service and automated scoring.

 

Presentation
4:10 p.m. – 5:00 p.m.

Why You Need an AI Governance Team


Speaker(s)
Tracy Sartaj, Pearson VUE
Cordell Jung, Pearson VUE
Eduardo Ustaran, United Kingdom’s Hogan Lovells Office

Session Description:
As artificial intelligence (AI) continues to permeate and disrupt the testing industry, it’s more important than ever for organizations to establish robust, informed AI governance to help ensure that AI practices are transparent, responsible, and ethical.

Join Pearson VUE and the International Association of Privacy Professionals for a discussion on the function and purpose of the global AI Governance Center, with updates on its formation and ongoing work, including the provision of a centralized repository of resources, best practices, and research for professionals who aim to navigate the multifaceted legal, ethical, technological, and practical challenges of AI implementation.


Attendees will also learn how to approach AI governance from an organizational perspective and how best to leverage internal resources to meet the complex legal challenges around the growing use of AI in their work and in the services their organization provides.

 

Wednesday, March 6

Presentation
8:30 a.m. – 9:20 a.m.

Comparison of Three-Option vs. Four-Option Multiple Choice Items on Psychometric Properties and the Impact on Test-Taker Performance


Speaker(s)
Brad Wu, Pearson VUE
Huaping Sun, ABA
Isabelle Ebert, U.S. Airforce and Avery Garcia, U.S. Airforce

Session Description:
Multiple choice (MC) items are widely used in large-scale assessments due to their efficiency and objectivity. Traditionally, four or five options have been the norm, but recent research suggests that three-option items may have certain advantages, such as reduced burden on item development and reduced item response time, while item statistics derived from the three-option MC items are as robust as four- or five-option versions.

The three studies featured in this session aim to examine and compare three- and four-option MC items on psychometric properties, such as item difficulty, item discrimination, and test reliability; assess the impact on test-taker performance, such as pass rate, means, and response time; and review practical implementations in assessment.

This session will appeal to attendees across the certification and licensure, education, and industrial and organizational fields.

 

Hang loose with us at the Pearson VUE Surf Shack!
 

It's a Party With a VUE at ATP 2024

Monday, March 4 | 7:00 p.m. PST
The Westin Anaheim Resort | RISE Rooftop Lounge

Leave your board at home, but bring your dancing flip-flops! We’ll have a DJ spinning surf tunes, a spread of all your fave beach food, plus a photo booth to capture all the excellent happenings.