We’re proud to sponsor this year’s Innovations in Testing conference, and we’ve put together a slate of sessions with something for everyone. Get all the details below. And be sure to join us on the roof at this year’s Surf Shack-themed party!
Monday, March 4
Peas in a Pod
10:00 a.m. – 10:50 a.m.
Speaker(s):
Tamika Hughes, Pearson VUE
John Kleeman, Learnosity
Session Description:
Equity, Accessibility, and Diversity (DEI): Methods and practices for ensuring fairness, accessibility, and accommodations for diverse stakeholders.
Panel Discussion (Featured Speaker Session)
11:10 a.m. – 12:00 p.m.
Speaker(s)
Eric Meyer, Pearson VUE
Marc Weinstein, Marc J. Weinstein, PLLC and Caveon, LLC.
Pat Ward, ITS
Liberty Munson, Microsoft
Session Description:
We have learned so much since our first conversation at ATP in 2023. Much has changed over the past year, and this panel is ready to continue the conversation. This panel will incorporate all the impacts of the test development process from legal, human, program, and technology. What has changed? What are the current views? How are programs adapting? How has the legal landscape changed? Are there new outcomes or tools to look for?
Panel Discussion (Featured Speaker Session)
1:20 p.m. – 2:30 p.m.
Speaker(s)
Gary Gates, Pearson VUE
Brodie Wise, ITS
Pat Ward, ITS
Janet Garcia, PSI Services
Tim McClinton, Meazure Learning
Stuart Udell, Prometric
Session Description:
In an era of dynamic change, the assessment industry is undergoing a profound transformation. Hear from a few C-Level leaders in our industry that will discuss the industry's evolution, dissect its trajectory, and confront imminent challenges. These leaders grapple with diverse concerns, including the disruptive influence of emerging AI technologies, the imperative to expand their businesses, and the necessity of reimagining credentialing processes in a global context marked by political and economic fluctuations.
Chief among their priorities is the protection against sophisticated security threats. Simultaneously, they prioritize strategic investments to address future needs while maintaining growth commitments. These leaders are committed to addressing the evolving landscape of testing, tackling challenges such as introduction to generative AI, redefining the value of standardized assessments in academic admissions, and striking a balance between user experience and security. The role of online platforms (OP) and test centers is central to their discussions, as they contemplate how to invest in new products and offerings while upholding their growth commitments.
In an industry marked by rapid change, these leaders are poised to navigate the future with foresight and innovation.
Peas in a Pod
4:10 p.m. – 5:00 p.m.
Speaker(s)
Kirk Becker, Pearson VUE
Cynthia Parshall, Touchstone Consulting
Belinda Brunner, Assessment Systems Corporation
Session Description:
Alternative item types (sometimes called innovative or technology-enhanced items) encompass a wide range of formats for presenting and collecting assessment information. In 2023, a special issue of the Journal of Applied Testing Technology compiled research and practical guidance on the use of alternative item types. Articles in that special issue include construction and use of clinical judgment tasks, the current state of automated scoring, digital performance assessments, effective analysis techniques, and more.
Join this session to take part in a conversation between attendees and authors, with insights and practical advice on the use of new item types for specific purposes. Session attendees will be able to ask the authors detailed questions about their choices, methods, and findings. In addition, attendees are encouraged to bring their experiences, challenges, and questions with implementing alternative item types into their exam programs.
Presentation
4:10 p.m. – 5:00 p.m.
Speaker(s)
Paul Jones, Pearson VUE
Vinita Talreja, AWS
Mamiko Ouazzani, IIA
Session Description:
The availability of generative artificial intelligence (AI) has produced both excitement and apprehension in the testing community, especially as a potential tool to aid in item development. While simple requests for items may produce flawed and relatively low-level content, it is possible to get well-constructed items across a range of cognitive levels with the right prompt. Join this session to discover the steps taken by several organizations to produce effective prompts and evaluate the items generated by those prompts. We’ll also explore the potential to move from drafting large numbers of specific items to instead editing, evaluating, and improving items, as well as examining new formats and ways of asking questions.
Presentation
4:10 p.m. – 5:00 p.m.
Speaker(s)
Kimberly Clausen, Pearson VUE
Pat Hughes, ITS
Liberty Munson, Microsoft
Session Description:
As part of Microsoft’s ongoing efforts to increase the accessibility and real-world relevance of our certification exams, an open-book experience was incorporated into our high-stakes exams, giving candidates the ability to access Microsoft Learn (https://learn.microsoft.com) during the exam. This was added as another resource that could be leveraged during the exam; nothing else about the experience, including exam content and time, changed.
To maintain the integrity of the exam, implementation of this resource required a secure browser solution that limited web access to ONLY the Microsoft Learn domain. To complicate matters, we needed to block access to Microsoft Learn’s Q&A and profile experiences to ensure that people couldn’t leverage either to cheat, and the experience needed to meet Microsoft’s strict accessibility requirements.
This was a complex integration that required a close partnership between Microsoft, ITS, and Pearson VUE and careful testing to ensure that there were no holes in the experience that would allow candidates to access unauthorized websites. During this session, we will discuss the reason for adding Microsoft Learn as a resource as well as the design, development, testing, implementation, lessons learned, and feedback from key stakeholders and candidates. We will also share data on usage, impact on passing rates, and other key psychometric results that will help programs determine if adding a similar resource to their exams should be considered.
Presentation
4:10 p.m. – 5:00 p.m.
Speaker(s)
Joe Brutsche, Pearson VUE
Jonathan MacKesson, Amazon
Pete Van Dyke, Amazon
Session Description:
For most organizations, implementing a traditional web monitoring program can be daunting and time consuming. Then, there are the constant alerts to investigate, organize, triage… and then determine if they’re something you should spend time reviewing. And once you’ve confirmed it is something to dig into, you suddenly realize the adventure has just begun and the threat it poses. What you need is a web-monitoring program and team to be the “guardians” of your certification program—a rag-tag group that can help you focus your attention and maybe save the universe (or at least your exam program).
In this session, join two well-known IT certification programs to learn about how they assembled their own group of “avengers” to build and execute their web-monitoring programs. Each of the programs has taken non-traditional approaches toward launching their programs, gaining internal support, and seeing them through to success. Join us to learn how their approaches can be applied to your program.
Tuesday, March 5
Peas in a Pod
8:30 a.m. – 9:20 a.m.
Speaker(s)
Erin Raw, Pearson VUE
Session Description:
A new, emerging technology for the assessment space, Virtual Reality (VR) presents exciting opportunities to assess candidates from all kinds of fields. For example, those who hope to become teachers could be given scenarios on classroom management. Aspiring doctors tested on their bedside manner. Prospective lawyers challenged to present opening arguments in a trial scenario. With VR, the possibilities are truly endless. But designing exams with VR, along with making decisions on scoring/validity and ensuring a good candidate experience all require innovative thinking. In this Peas in a Pod session, we’ll tackle all of it, as we discuss the use cases, benefits, and challenges for VR in assessment.
Panel Discussion
8:30 a.m. – 9:20 a.m.
Speaker(s)
Joe Brutsche, Pearson VUE
Ramond Nicosia, ETS
Ardeshir Geranpayeh, Cambridge Computational Psychometrics
Fiona Mitchell, British Council
André Kenji Horie, Duolingo
Session Description:
Join a panel of test security experts from major international testing organizations to discuss how they managed test security challenges during and post pandemic. Learn how embracing change and new technology has allowed testing organizations to not only manage unforeseen disruptions to standardized testing administrations but also allowed them to enhance their test security and fairness in the process.
Workshop
9:40 a.m. – 10:50 a.m.
Speaker(s)
Harry Samit, Pearson VUE
Brent Morris, Cisco
Jonathan MacKesson, Amazon
Session Description:
This workshop will provide success stories from different sponsors that have produced hundreds of invalidations and exam enforcements that have protected the integrity of our certifications programs. Through these success stories, our different sponsors will show you under-the-hood examples of different security techniques used. We will provide an engaging workshop that will involve the audience in hands-on security crafts that have been so successful.
Presentation (Sponsor Session)
3:00 p.m. – 3:50 p.m.
Speaker(s)
Kellie Britten, Pearson VUE
Joe Brutsche, Pearson VUE
Eric D'Astolfo, Pearson VUE
Tracy Sartaj, Pearson VUE
Session Description:
The immense promise of artificial intelligence (AI) is being fulfilled a bit more every day. And by now it’s clear that AI is here to stay — its use quickly and increasingly becoming an expectation rather than a novelty. In the testing industry, AI’s potential means it will be used more frequently, for everything from capacity planning and proxy tester identification to customer service and automated scoring.
Presentation
4:10 p.m. – 5:00 p.m.
Speaker(s)
Tracy Sartaj, Pearson VUE
Cordell Jung, Pearson VUE
Eduardo Ustaran, United Kingdom’s Hogan Lovells Office
Session Description:
As artificial intelligence (AI) continues to permeate and disrupt the testing industry, it’s more important than ever for organizations to establish robust, informed AI governance to help ensure that AI practices are transparent, responsible, and ethical.
Join Pearson VUE and the International Association of Privacy Professionals for a discussion on the function and purpose of the global AI Governance Center, with updates on its formation and ongoing work, including the provision of a centralized repository of resources, best practices, and research for professionals who aim to navigate the multifaceted legal, ethical, technological, and practical challenges of AI implementation.
Attendees will also learn how to approach AI governance from an organizational perspective and how best to leverage internal resources to meet the complex legal challenges around the growing use of AI in their work and in the services their organization provides.
Wednesday, March 6
Presentation
8:30 a.m. – 9:20 a.m.
Speaker(s)
Brad Wu, Pearson VUE
Huaping Sun, ABA
Isabelle Ebert, U.S. Airforce and Avery Garcia, U.S. Airforce
Session Description:
Multiple choice (MC) items are widely used in large-scale assessments due to their efficiency and objectivity. Traditionally, four or five options have been the norm, but recent research suggests that three-option items may have certain advantages, such as reduced burden on item development and reduced item response time, while item statistics derived from the three-option MC items are as robust as four- or five-option versions.
The three studies featured in this session aim to examine and compare three- and four-option MC items on psychometric properties, such as item difficulty, item discrimination, and test reliability; assess the impact on test-taker performance, such as pass rate, means, and response time; and review practical implementations in assessment.
This session will appeal to attendees across the certification and licensure, education, and industrial and organizational fields.
It's a Party With a VUE at ATP 2024
Monday, March 4 | 7:00 p.m. PST
The Westin Anaheim Resort | RISE Rooftop Lounge
Leave your board at home, but bring your dancing flip-flops! We’ll have a DJ spinning surf tunes, a spread of all your fave beach food, plus a photo booth to capture all the excellent happenings.