Taking Student Learning Outcomes Assessment to the Next Level

The Kirwan Center for Academic Innovation and the USM Advisory Board for Graduate Education co-sponsored a system-wide convening on student learning outcomes assessment. National experts Natasha Jankowski (NILOA) and Daniel Denecke (Council of Graduate Schools) joined USM faculty, staff, and administrators for discussions on the “state of the art” of learning outcomes assessment across general education, undergraduate degree programs, graduate programs, and the co-curriculum. Workshops and roundtables featured useful tools and strategies for improving outcomes assessment at student, course, program, and institution levels and for building a culture of assessment at institutions.

To access materials from the sessions (PowerPoints, handouts, and other resources), go to the Program tab.


Bowie State University, 14000 Jericho Park Rd, Bowie, MD 20715


April 19, 2017



Program Overview

Resources from the sessions are linked to their session descriptions below.


8:00 - 8:45 am


8:45 - 9:00 am

Joann Boughman, Senior Vice Chancellor for Academic Affairs, University System of Maryland (USM)

Mickey L. Burnim, President, Bowie State University

Weldon Jackson, Provost and Vice President for Academic Affairs, Bowie State University


MJ Bishop, Director, William E. Kirwan Center for Academic Innovation, USM

Charles Caramello, Senior Advisor for Graduate Education, USM


9:15 - 10:30 am

Student Learning Outcomes Assessment: Past, Present, and Future | Natasha Jankowski, Director of the National Institute for Learning Outcomes Assessment (NILOA) and Research Assistant Professor, University of Illinois at Urbana-Champaign, and Daniel Denecke, Vice President, Best Practices and Strategic Initiatives, Council of Graduate Schools

In this plenary session, Dr. Jankowski and Dr. Denecke will consider learning outcomes assessment as a field, with its own history, conceptual frameworks, research bases, and best practices. After briefly examining the past and present of the field, they will look to the future of learning outcomes assessment from general education through doctoral training. Drawing on their experiences with multiple national projects, they will lay out challenges and opportunities for next steps in outcomes assessment as a process that helps us to learn about student learning.

Resources: Keynote Video Recording, N. Jankowski ppt., D. Denecke ppt.


10:30 - 10:40 am


10:40 - 11:40 am

Administrative Models for the Assessment Process | Kara Siegert, Special Assistant to the President for Institutional Effectiveness & Assessment, Salisbury University; Janet Rutledge, Vice Provost and Dean of the Graduate School, University of Maryland, Baltimore County; and Catherine Andersen, Vice Provost for Institutional Effectiveness, Planning and Academic Quality, University of Baltimore. Moderated by Charles Caramello, USM

This session will feature panelists discussing the ways in which outcomes assessment is organized at their respective institutions, how structures and roles have evolved, and lessons learned about making the process manageable and meaningful for those involved.

Closing the Loop: Using Assessment Data to Improve Practice | Tab Uhrich, Director of Academic Assessment and Associate Professor, Department of Kinesiology, Towson University and Lakeisha Mathews, Director, Career and Professional Development Center, University of Baltimore. Moderated by Nancy O’Neill, Associate Director, Kirwan Center for Academic Innovation, USM

Using assessment results to inform change is perhaps the most crucial, yet challenging, aspect of this work. In this session, panelists will describe how they have used findings from student learning outcomes assessment for the purposes of improvement. Examples will vary and include changes to classroom practices, course sequencing, co-curricular offerings, administrative processes and policies, and even the student learning outcomes under consideration.

Building a Robust Culture of Assessment | Cristi Ford, Associate Vice Provost, Center for Innovation in Learning and Student Success, University of Maryland University College; Doris Santamaria-Makang, Interim Associate Provost, Frostburg State University; and Becky Verzinski, Assistant Vice President for Assessment, Bowie State University. Moderated by MJ Bishop, USM

What does it take to create a context in which assessment is driven less by compliance and more by inquiry, engagement, and the pursuit of shared understanding about student learning? This panel will focus on strategies to create a robust culture of assessment. Included in the discussion will be strategies for how to engage departments and faculty as drivers of assessment, how to position assessment as responsive to questions about student learning, how to connect pockets of assessment activity, and how to assuage concerns and navigate resistance.


11:40 am - 12:30 pm

Table 1: Developing Effective General Education Student Learning Outcomes | Fiona Glade, Assistant Provost, University of Baltimore
In this roundtable, the facilitator will share “before” and “after” versions of general education student learning outcomes and talk through lessons learned about how to streamline and simplify GE outcomes while still respecting the multiple disciplinary inflections contained within them.

Resources: F. Glade handout #1, F. Glade handout #2

Table 2: Using the Excellence in Assessment (EIA) Evaluation Rubric for Institutional Improvement | Natasha Jankowski, NILOA
The Excellence in Assessment (EIA) program recognizes institutions for efforts to intentionally integrate institution-wide learning outcomes assessment. At this roundtable, participants will discuss using the EIA evaluation rubric, built from NILOA’s Transparency Framework, as a tool for institutional self-reflection and improvement.

Table 3: Aligning Undergraduate and Master’s Degree Student Learning Outcomes | Maureen McCarthy, Associate Director, Advancement and Best Practices, CGS
This roundtable will explore strategies for using transdisciplinary frameworks to align master’s-level student learning outcomes (SLOs) with undergraduate SLOs. Examples of frameworks to be discussed include the Degree Qualifications Profile (DQP) and European Qualifications Framework.

Resources: Degree Qualifications Profile (US; Lumina Foundation); Connecting Credentials: A Beta Credentials Framework (US; Lumina Foundation); European Qualifications Framework (EU; European Commission)

Table 4: A Rubric to Evaluate Rubrics: A Tool for Rubric Review and Refinement | Nancy O’Neill, USM
This roundtable will introduce participants to a rubric that is designed to help users review and refine assessment rubrics. The tool is versatile and can be used with newcomers to rubrics development as well as with more experienced rubric developers and users.

Resources: N. O'Neill handout

Table 5: Developing Effective Student Learning Outcomes Assessments for Doctoral and Professional Degrees | Daniel Denecke, CGS
Regional accreditors often provide guidelines for developing SLOs for doctoral degrees, and professional accreditors often specify SLOs for professional doctoral degrees. This roundtable will focus on how universities can develop SLOs to advance their academic goals while also responding to accreditation requirements.

Table 6: Put Your Assessment Program Under the Microscope | Cindy Herzog, Associate Dean and Professor of Psychology, College of Liberal Arts and Sciences, Frostburg State University
The Assessment of Student Learning Outcomes Process Rubric is a tool that can be used by campus assessment teams, department chairs, and program coordinators for review and discussion of assessment plans and implementation of those plans.  This “assessment of assessment” process helps programs to identify strengths and weaknesses and to make improvements.

Resources: C. Herzog handout #1, C. Herzog handout #2, C. Herzog handout #3, C. Herzog handout #4

Table 7: Bridging Student Learning and Student Success | MJ Bishop, USM
This roundtable discussion seeks to bridge the conversations happening at our institutions around outcomes assessment and those happening around analytics and student success. What value is there in connecting these efforts? What are the points of intersection?


12:30 - 2:00 pm

Building a Robust Culture of Assessment: A Case Study from Bowie State University | David Abrahams, Chair and Associate Professor, Department of Management, Marketing, Public Administration and Entrepreneurship; Dawn Johnson-Tate, Accreditation Coordinator, College of Education; Charla McKinzie Bishop, Associate Professor, Department of Behavioral Sciences and Human Services; Gail Medford, Chair and Professor, Department of Fine and Performing Arts; Diarra Robertson, Associate Professor and former USLAC Chair, Department of History and Government; and Patricia Westerman, Professor and inaugural USLAC Chair, Department of Psychology and Chair, Faculty Senate—all of Bowie State University. Moderated by Debra Salsi, Coordinator of Assessment and Accreditation, College of Business and Graduate Studies; Becky Verzinski, Assistant Vice President for Assessment, Office of the Provost; and C. Jenise Williamson, Associate Professor and USLAC Chair, Department of English and Modern Languages—all of Bowie State University
Assessment processes can be put in place, and results can even be used to inform changes, yet assessment practice can still veer toward compliance rather than toward inquiry, engagement, and shared understanding. In this case study from Bowie State University, panelists will discuss how academic program outcomes assessment has been transformed since 2011-12 through the efforts of faculty assessment coordinators and their service on the University Student Learning Assessment Committee (USLAC).

Student Learning Outcomes Assessment of Co-Curricular Experiences | Nancy O’Neill, USM; Sara Ousby, Director, Rosenberg Center for Student Involvement, University of Baltimore; and Emily Tipton, Internship Coordinator/Career Advisor, Career Center, Towson University
In this session, the presenters will describe efforts to assess student learning outcomes in student affairs, touching on the evolution of this work over time and lessons learned along the way. The session will also include time for participants to meet others engaged in these efforts and develop and discuss action steps related to “next level” SLO assessment in student affairs at their own institutions.

Resources: N. O'Neill, S. Ousby, and E. Tipton ppt., N. O'Neill UB handout, and S. Ousby UB handout

Assignment Design: A Faculty-Centered Approach to Improving Teaching, Learning, and Assessment | Natasha Jankowski, NILOA
Since 2013, NILOA has organized a series of national assignment design “charrettes”—collaborative processes where faculty give and receive feedback on their assignments within a concentrated time frame—to develop a library of assignments linked to Degree Qualifications Profile (DQP) proficiencies. The goal was to promote an “embedded” approach to assessment that was integral to the teaching and learning process and that would empower faculty to make improvements. In this workshop, participants will learn about NILOA’s initiative and take part in a mini-assignment charrette, with tips and tools on how to organize a charrette process at one’s home institution. The workshop will conclude with discussions on how assignment design work can dovetail with broader efforts to align courses and curricula and create clearer learning pathways for students.

Resources: N. Jankowski ppt., Catalyzing Assignment Design Activity on Your Campus: Lessons from NILOA’s Assignment Library Initiative (2014) and Organizing Assignment-Design Work on Your Campus: A Tool Kit of Resources and Materials.

Graduate Learning Outcomes Assessment, Part 1: The State of Play Nationally and in the USM | Maureen McCarthy, CGS
In this session, the presenters will provide an overview of the evolving national state of play in learning outcomes assessment in master’s and doctoral education. The presentation and subsequent discussion will be followed by a roundtable on the state of play of graduate assessment at USM institutions, with particular attention to both challenges and successes.

Resources: M. McCarthy ppt


2:00 - 2:10 pm


2:10 - 3:30 pm

Creating a Sustainable Process for Rubrics Assessment: Tips and Strategies | Fiona Glade, UB and Nancy O’Neill, USM
This session will offer tips and strategies for organizing rubrics assessment processes, highlighting different approaches taken at the University of Baltimore in general education and writing, and reviewing best practices from the field. Participants will have the opportunity to begin planning their own rubrics assessment event (whether in GE, in a degree program, or in the co-curriculum), considering issues of time, resources, training, opportunities, and challenges.

Resources: F. Glade and N. O'Neill ppt., F. Glade UB handout, N. O'Neill UB handout

Connecting Outcomes Assessment with Accreditation and Program Review Processes | Michael Bowden, Assistant Vice President, Planning and Accreditation, Coppin State University; Kathleen Angeletti, Assistant Dean and Executive Director of Teacher Education, College of Education, University of Maryland, College Park; and Terry Smith, Associate Professor, Director of the Writing Center, and Chair of the General Education Committee, University of Maryland Eastern Shore. Moderated by Jane Neapolitan, Assistant Provost for Academic Innovation, Towson University
In this session, panelists will discuss strategies for integrating assessment of student learning with external reporting required by regional and professional accreditors. They will also discuss the academic program review process and other program-level reporting as mechanisms for ensuring that outcomes assessment is part of the regular work of departments.

Exploring Assessment Technologies | MJ Bishop, USM and Sherri Braxton-Lieber, Director, Instructional Technology and Jennifer Harrison, Assistant Director for Assessment, Faculty Development Center, both of University of Maryland, Baltimore County
In this workshop, participants will explore assessment technologies at both the macro (e.g., assessment management systems, e-Portfolio platforms) and micro (e.g., clickers, card swipe technologies) levels through two hands-on activities: (a) the initial development of a taxonomy to make sense of the array of assessment-focused technologies available to institutions and (b) the development of evaluation criteria for judging the appropriateness of a particular technology for use by an institution or a specific division or department. Additionally, the session will focus on the process of identifying and engaging stakeholders at various stages of technology adoption: evaluation/vetting, selection, roll-out, and ongoing use.

Resources: M. Bishop, S. Braxton-Lieber, and J. Harrison ppt., J. Harrison handout

Graduate Learning Outcomes Assessment, Part 2: Engaging Faculty in Building a Culture of Assessment, and Using Outcomes Assessment for Program Improvement | Erin Golembewski, Senior Associate Dean, Graduate School, University of Maryland, Baltimore and Janet DeLany, Dean of Graduate Studies, Towson University
This session will begin with the presentation and discussion of case studies on two distinct and focused topics related to graduate learning outcomes assessment: (1) engaging faculty and programs and building a culture of graduate assessment (University of Maryland, Baltimore as a case study), and (2) using assessment data effectively to improve graduate programs (Towson University as a case study). Breakout groups will then workshop strategies for achieving these goals.

Resources: J. DeLany ppt.


3:30 - 3:40 pm


3:40 - 4:00 pm

MJ Bishop and Charles Caramello, USM

Keynote Speakers

Natasha Jankowski
Director of the National Institute for Learning Outcomes Assessment (NILOA) and Research Assistant Professor, University of Illinois at Urbana-Champaign

As Director of NILOA, Dr. Jankowski regularly presents at national conferences on the subject of outcomes assessment and has written numerous reports and occasional papers, including “Making Student Learning Evidence Transparent: The State of the Art” with Staci Provezis and “Using Assessment Results: Promising Practices of Institutions at Do It Well” with Gianina R. Baker, Staci Provezis, and Jillian Kinzie. Her main research interests include assessment and evaluation, organizational evidence use, and evidence-based storytelling. Dr. Jankowski holds a Ph.D. in Higher Education from the University of Illinois, an M.A. in Higher Education Administration from Kent State University, and a B.A. in philosophy from Illinois State University. She previously worked for GEAR UP Learning Centers at Western Michigan University and with the Office of Community College Research and Leadership studying community colleges and public policy.

Daniel Denecke
Vice President, Best Practices and Strategic Initiatives, Council of Graduate Schools (CGS)

Since joining CGS in 2012, Dr. Denecke has led projects and authored publications on a range of graduate education topics including research and scholarly integrity, international collaboration, professional development for non-academic careers, Ph.D. completion and attrition, and the integration of undergraduate outcomes assessment into Preparing Future Faculty programs. Dr. Denecke received his Ph.D. from the Johns Hopkins University and previously served as faculty member at Georgetown University and the University of Maryland, College Park.


For more information, contact Nancy O'Neill, Associate Director, Kirwan Center for Academic Innovation, at noneill@usmd.edu