Plan for Implementing Campus-wide Course and Program Assessment

Guiding Philosophy: 

A plan to train faculty for course and program assessment should be

bulletinstitutionally and culturally appropriate
bulletcharacterized by dialogue
bulletfaculty-led
bulletexemplary - modeling good assessment principles
bulletcommitted to valid and reliable methods
bulletsustainable and realistic concerning budget and workload
bulletfocused on closing the loop to improve education

Note: While I advocate classroom-based embedded assessment, there are campuses that successfully implemented assessment beginning with institutional effectiveness, program level assessment, or general education assessment. Starting at the classroom level will intrinsically motivate faculty and provide a safe environment to learn about and implement assessment. Several key features in this plan are a result of analyzing data and feedback from over 300 faculty and administrators attending SLO and Assessment workshop training. Features heavily emphasized in the feedback from community faculty and administrators throughout California are marked with an asterisk*.

Step 1: Select a Steering Committee as a Means of Training

Determine the component of campus governance that is effective and appropriate to the existing institutional culture that could be used as a steering committee to initiate training. Use this committee or a subcommittee as the core for campus training. Assessment relates to many different aspects of campus life and crosses a variety of institutional processes; therefore, nearly any effective governing body can be used as an implementation vehicle.

Sample Effective areas:

bulletCurriculum committee
bulletProgram Review
bulletExisting Assessment committee
bulletInstitutional effectiveness committee
bulletStaff development
bulletTeaching and Learning center
bulletDepartment chairs council
bulletBudget and program review committees

Ineffective areas:

bulletThe senate as a whole
bulletCollege council, President’s Council (campus-wide committees)
bulletPredominately or exclusively administrative committees

 

 

These may be ineffective because oversight alone is not enough.

Indeterminate area:

bulletCreating an all new Assessment committee

 

Some campuses have elected to create an all new committee. All successful and sustained assessment projects have an assessment committee steering them eventually but committee roles are very diverse.

If there is no existing committee, consider training a core and creating a long-term committee after training with interested participants.

 

Step 2: Develop a Steering Committee Structure and Budget

Determine whether this will be a subcommittee of another standing committee or whether an entire existing committee can reorient time and energy for the initial learning curve. (For instance, the Institutional Effectiveness or Program Review Committee might set aside one year of duties (e.g. reviews) to organize and carry out training)

 *Have broad multidisciplinary representation with adequate administrative members and try to include adjunct faculty. (In addition if classified staff or students can participate this is can be helpful.)

 Elect a faculty chair or create a co-chair position with a faculty member and administrator if this is a subcommittee or new committee.

 Train the core team using the training material available in the workbook or online at http://online.bakersfieldcollege.edu/courseassessment/ .

 Develop a budget immediately. Anticipate and fund campus-wide events, discipline (department) training events, guest speakers, a set of library resources, or books for faculty. A few examples are enumerated below. 

Potential Budget Expenses                               

Estimated Costs

Library of Assessment Texts

Sample texts:

Classroom Assessment Techniques, Angelo and Cross, 1993

Assessment for Excellence, Astin, 1993

Learner-Centered Assessment on College Campuses, Huba and Freed, 2000

How People Learn: Brain, Mind, Experience, and School, NRC, 2002

Knowing What Students Know, NRC, 2001

A Practitioner’s Handbook for Institutional Effectiveness and Student Outcomes Assessment Implementation, Nichols, 1995

A Learning College for the 21st Century, O’Banion, 1997

Assessment Essentials: Planning, Implementing, and Improving Assessment in Higher Education, Palomba and Banta, 1999

Student-Centered Classroom Assessment, Stiggens, 1994.

Effective Grading, Walvoord and Anderson, 1998

Assessment Clear and Simple, Walvoord, 2004

The Art of Changing the Brain, Zull, 2002

$400 - $600

Outside Speakers – e.g. Tom Angelo, Virginia Anderson, Peter Ewell, Douglas Eder, Terry O’Banion, WASC representatives, local experts, RP group workshop leaders

$1000-2000-5000 (depending on speaker)

Outside Assessment Conferences –

AAHE, AACU, IUPUI, and others

$250-2000 each

Training Workshops on Campus (multidisciplinary & by disciplines)

$1000 - 3000

Remunerations, stipends, reassign time – This should be consistent with campus policy and union contracts. This is instructional work and can be used as numerator expenses for the 50% Law.

Depends on campus and budget.

Step 3: Conduct an Assessment Audit

Have department chairs, deans, and managers document assessment activities currently taking place on campus. RP developed an assessment checklist which BC adapted. Use and modify these forms to conduct your own assessment audit.

Determine what processes already align with the WASC standards.

Catalogue types of evidence and assessment data easily available on campus (i.e. in vocational areas or for grants) or through the state chancellor’s office. Make this available campus-wide.

Identify faculty with expertise (cognitive measurement, statistical or research skills, learning specialists) for example:

·         English and Math faculty are often experienced in pre-collegiate assessment or placement testing - some of these tools can be used again as post tests.

·         Vocational faculty are very skilled in meeting professional outcomes and preparing for board examinations.

·         Psychology faculty are often skilled in cognitive assessment methods.

·         Math and psychology faculty, as well as institutional researchers, have expertise in statistics.

·         Science faculty and institutional researchers have skills in research project design.

·         Education and Basic Skills faculty have training in articulating student outcomes.

·         Faculty with rubric experience.

 

Step 4: Determine Campus-wide Logistics

 

While the assessment audit is being conducted, look at logistics from a campus-wide perspective. Plan how the steering team will educate the campus at-large.

Create a training dissemination plan. Because assessment is new to many instructors, you must start at the lower levels of the cognitive domain, vocabulary and general recall about assessment, and allow time for reflection and application.

Determine whether the training will begin with a small pilot group (general education committee), a dedicated group of leaders (department chairs council), or a particular discipline area (science, English, etc)?

*Think about the most effective grouping for training. Mixing disciplines in the initial stages can be very effective (e.g. art and agriculture, history and Spanish). However, training later on should concentrate within similar disciplines. Discipline groups are important when finalizing SLOs and assessment tools. Will you use the department or division chairs and disseminate the training via disciplines? Will you use the staff development and flex days for voluntary participation?

Map out opportunities for dialogue. Design and create opportunities for multidisciplinary training early in the process. Plan for discipline specific training later concerning course and program outcomes and assessment facilitated by those from the core training group that are not members of that discipline. While this may sound odd, it levels the playing field allowing younger faculty to have input as well as older faculty. Not always, but occasionally, older and powerfully outspoken faculty can be very resistant to assessment (anything “new” for that matter). Having an outside facilitator that will recognize and encourage participation is very helpful. In addition, instructional assistants and classified members should be encouraged to participate and share their thoughts and observations of student learning. Some campuses have created a specific weekly lunch time where faculty meet to discuss assessment initiatives, others have campus-wide dialogue hour to share data and get new ideas.

Step 5: Select and Train Core leaders (These should be from the steering team and may include other volunteer leaders. This training process will take from 6 months to a year and will go through all sections of the training materials.)

 

Select and train the steering team – these people will serve as mentors and consultants. Invest in this process, emphasize one-on-one feedback and dialogue amongst the team.

bullet

Use appropriate assessment training materials and methods

bullet

Use and modify the training material provided at the website                     http://online.bakersfieldcollege.edu/courseassessment/ 

bullet

*MODEL everything you want the participants to do

bullet

*Make numerous resources and samples available from a wide variety of disciplines.

bullet

*Make it fun, be candid and share your own failures and successes. When possible work with student artifacts.

bullet

Emphasize dialogue at every meeting and in every aspect of the training. Encourage them to share both positive and negative feedback with each other - ingrain the need for dialogue.

bullet

If possible take these trainees to other campuses to observe and to share what they are doing.

Have this core of trained faculty write and revise SLOs and begin assessment practices in their courses. The steering team must have experience with classroom and program assessment to be effective. Allow time to implement and reflect on assessment.

 

Develop, modify, or review a curriculum, course, program, or service.

 

Develop Student Learning Outcomes (SLOs)

 

Design & Measure Student Learning  

Collect, discuss, and analyze data.

 

Determine refinements based on outcomes data.

  Closing the Assessment Loop

 

 

 

 

 

Communicate the importance of closing the loop on a single project or SLO versus trying to do everything and closing the loop on nothing.

 Provide fair and reasonable remuneration to faculty and staff members that are implementing this process – the learning curve is steep and time consuming. If assessment is important, value it; don’t treat it as another add on. Do it right. Later as assessment practices become a normal part of professional life and new faculty are trained when hired, this remuneration may be unnecessary. Invest in this cultural shift of perspective; this indicates it is not just another fad. Investing in assessment implementation will set a precedent for budgetary decisions based on evidence of improved learning.

Remuneration or Incentive Possibilities

Adjust the number of office hours required for a year – instead of 5 office hours per week, require only 4 for student conferences and make one hour for assessment preparation, implementation, & analysis.

Provide meals for workshops and retreats.

Send faculty and staff to assessment conferences, such as IUPUI, AAHE, AACU.

Purchase and provide assessment texts for individual faculty that request them.

Update personal computers or provide laptops to facilitate assessment procedures.

Recognize and officially appoint assessment mentors.

Provide reassign time for mentoring or assessment projects.

Create assessment grants – have faculty and staff apply for specifically funded assessment projects.

Count training as FLEX time and pay hourly wages for training beyond required FLEX time.

Provide stipends for faculty involved in guiding the projects, as you would for faculty involved in accreditation.

Create financial incentives for completed assessment products (for instance $50 for each set of complete class SLOs, $100 for assessment plans, and $150 for the first assessment report). Writing SLOs and developing assessment plans are easier once the initial learning curve for a single course is completed.

Encourage team teaching (and pay both faculty members adequately) that targets assessment practices in key courses.

Refer to the AAHE 9 Principles of Good Practice for Assessing Student Learning (1992)

and Tom Angelo’s Guiding Principles for Assessing as if Learning Mattered Most.
Use the criteria for good assessment training attached as Appendix B.

Step 6: Train the Campus

 Use an all-campus meeting and other communications to disseminate the rationale behind assessment and communicate the change in accreditation standards

 *Keep training groups small 10-20 or less.

 *Do not force participation; voluntary participation is important. Those that won’t cooperate - never will; those that are on the fence will be encouraged to participate when the results validate the effort. Trust that improvement will occur and the effects of the process will be contagious.

 *Emphasize the benefit for students and faculty and provide some background concerning external requirements.

 *Group training time must be used in a valued way – craft the training to include hands-on activities, collaboration and creation of a product. (For example: Session one – create a draft SLO, Session 2 - Modify and improve a set of course SLOs, Session 3 - Create an assessment instrument).

 To the greatest extent possible modify the training to the specific needs and interests of the group (vocational board requirements, math standards, related professional standards).

 MODEL good classroom embedded assessment techniques in the training.

Tell participants what the outcomes of the training will be, evaluate and improve every training session, incorporating a method to help participants verbalize what they have learned.

Identify leaders and build the pool of trained mentors during the training.

Encourage dialogue, sharing of results, ideas, and concerns.

Remunerate faculty that produce a product (e.g. stipend or recognition after creating a product course SLOs, assessment plans, and first assessment reports).

*Provide opportunities for collaborative teaching (with appropriate workload) where the focus is creating, implementing and reporting assessment – e.g. capstone classes, honors classes, majors courses, basic skills.

Incorporate Student Services units with appropriate academic areas.

Emphasize the learning paradigm and use assessment to create processes that value, support, and improve learning.

*Set realistic goals, such as assessing one or two SLOs every year. Develop a timeline that considers all components – one or two course SLOs, plus one or two program SLOs, plus one or two institutional SLOs represent a heavy load for teaching faculty with multiple courses. One or two SLOs per year should complete and entire assessment cycle in 4 or 5 years approximating an accreditation cycle.

Step 7: Close the LOOP and Adjust Campus Processes

Incorporate SLOs in the curriculum documents of record.

 Modify program review to include assessment reporting.

 Communicate the effects (improved learning, curriculum, pedagogy) at key campus events.

 Require documentation of strategies to or evidence of improved learning each year (unit reports). Writing things down is the hardest part. Provide templates or models. This creating a summary report is essential to the reflective aspect of closing the loop. The reflective aspect of the report is guaranteed and validated by robust dialogue with the areas and faculty involved.

Step 8: Sustaining the Assessment Process

 Evaluate the training. Revise and modify the training using feedback and outcomes results.

Make training and updating available every year; catch those that want to begin after they have seen the outcomes.

Make improving learning an institutional goal, recognizing and communicating its importance.

Link future budgeting to those areas that close the loop and show improvement. Improvement is the major purpose for assessment, realistically funding improvement based upon evidence is a part of the process, but not the purpose behind assessment.

Share improvement, documenting and publicizing it to the community, students, faculty, and staff.

 Document the steps used to implement assessment – create a history of what has been done; share it at a conference   and with other institutions.

Appendix A

Instructional Assessment Audit Form

Administrative Assessment Audit Form

Appendix B

Criteria for a Training Plan