This poster session reviews faculty perceptions of the outcomes of using online technology in their courses. Most thought it improved their teaching and improved student learning, and some helped identify future steps they might take to make these improvements more quantifiable. After developing 29 online projects at CLU, we conducted post-project evaluations and interviews to determine faculty impressions of the projects' effects on teaching and learning. Faculty reported that the implementation of their projects prompted them to envision new and more effective ways to teach and that they observed improved student learning. The post-project evaluations also demonstrated, however, that in order to enhance and quantify these improvements, faculty needed to identify and incorporate new assessment strategies. Carefully constructed measures of teaching and learning will therefore be integrated into the organization and interface of the project revision process.
User Interface and Interactions, Design, and Online Teaching and Learning
During the implementation of a two-year faculty development grant from the Charles E. Culpeper Foundation, California Lutheran University (CLU) has explored technology's potential for improving teaching and learning. Twenty-nine online course redesign projects were developed by teams who used combinations of five major technologies: course management system (WebCT) projects (14); slide digitization and database development (2); video production (7); custom web sites (6); and CDRom's . The total student development time in hours was greatest for video (840 hours) followed by slide digitization (750 hours). The fourteen WebCT projects required 560 student development hours followed by the custom web projects with 240 student hours. The digital video and slide projects are not scaleable; WebCT, with its difficult user interface, appears to be scaleable.
Post-project evaluations demonstrated that implementation of their online projects prompted faculty were to envision new and more effective ways to teach and that they had observed improved student learning. Some also reported, however, that in order to enhance and quantify these improvements, new assessment strategies need to be identified and incorporated into subsequent versions of the projects. This paper reviews faculty perceptions of how integrating online technology improved their teaching, improved student learning, and generated ideas that would help quantify these improvements.
Faculty who noted that delivering course content online prompted them to improve their teaching strategies include the following.
A History professor who digitized his collection of 5000 photographs of Southeast Asian art and architecture commented that his project provided the "chance to better grasp the possibilities of how my images can be used more easily and more creatively in my teaching." He can now quite readily reorganize his images for specific instruction purposes. A reluctant convert to putting course materials online, this professor has finally changed his mind about teaching and technology: "This project has definitely been beneficial to both my professional development and my teaching . . . I can now combine images, text, and maps much more smoothly than I could using overhead transparencies, slide projectors, and wall maps. I'm more comfortable with the technology in the classroom and learning to use it more effectively."
A Drama professor created short video vignettes to teach acting fundamentals. The online video clips bring a three-dimensional aspect to the teaching and learning of character development and of various acting techniques. This professor says that: "Actually, implementing my Thespian Survivor project did force me to think differently, perhaps in more visual terms, about how to present the same material I had used in class for the last few years." Similar comments were offered by other faculty who developed video projects: a Biology professor whose marine biology video about marine life around Catalina Island supplements a course text and printed images; and an Education professor who uses online videos of real classroom settings to demonstrate teaching methods.
A Physiology professor reported that his web project had a significant impact both "on how I teach the course and on student learning." As a resource for images, graphics and animations, the custom web page he created for a human anatomy and physiology course enhanced and improved his lectures. Use of the web site in the lab was part of the original project's design, but the instructor was "surprised and pleased to discover how valuable an asset this has become. Use of the course web site in the lab has allowed me to expand the use of group learning and critical thinking exercises." Other faculty who reported that using web sites in face-to-face classroom instruction improved teaching and learning want to build on these results:
Some of the most convincing instances of improved student learning came from faculty who posted online examples of student work. This approach allows students to compare their own work with the exemplary, online samples of their peers. Many students begin to self-critique as they see quite readily how their work compares with the samples. In addition, the posted "standards give a more objective way to support grading decisions. Teachers can display and discuss the work in class, and students who follow the discussion are engaged because they have had to respond to the same assignment. Faculty who want to further exploit the positive benefits of posting online student work include:
In the post-project evaluation process, faculty reported they were teaching (and students were learning) differently and more effectively. Many of them wanted to quantify this awareness by identifying and incorporating new assessment strategies. All assessment of the effectiveness of online instruction would presume that students are engaging with the online materials. Like some of the other faculty, a Geology professor who provided digitized images of field trips thought that because there were not enough on-line assignments incorporated into the course, it was difficult to evaluate the effects of his project. To do so he said, "I need to design more assignments that force them (students) to use the web page and not to circumvent it.
An Education professor who did build assessment tools into her project noted that she had thought she would get good assessment data by using a survey provided by the Flashlight Project group. After administering the survey, however, she concluded that she would have to carefully reconstruct it in order to measure the learning outcomes in her online project. "The Flashlight survey, the professor said, "needs to be better adapted to the specific teaching and learning conditions of the online course. Modifying Flashlight evaluation tools appealed to other faculty too, who agreed that this modification needed to be an intregal part of their project revision process.
Other faculty want to explore models of assessment other than Flashlight. In evaluating the project of a Physics professor who used PRS, for example, we determined we should incorporate pre/post testing to measure learning. In fact, some assessment models have been developed for this purpose. The research of Richard R. Hake is a case in point. Hake has conducted a comprehensive study using pre/post test data of about 6000 students in high schools, colleges, and universities. The analysis, using Halloun-Hastenes Mechanics Diagnostic test (MD) or Force Concept Inventory (FCI) showed that use of interactive engagement (IE) strategies in the classroom can increase effective learning as much as 50% over traditional instructional methods. (American Journal of Physics [Vol. 66 (1998) pp. 64-74]).
Other faculty want to design their own evaluations, customizing them to reflect specific teaching and learning strategies and outcomes. A Psychology professor is using control groups to test learning outcomes by comparing the results of traditional teaching approaches with on-line presentation methods.
Although it is clear that developing new assessment strategies to quantify teaching and learning outcomes will require more time and resources, our University is willing to support such an effort. In part, this willingness is a result of our recent awareness that we need to strengthen upper division courses. Online course materials and assessments promise to help address that need by increasing student engagement and preparation time and bringing more rigor to their courses' areas that have now been formally identified as institutional goals.
http://www2002.org/cfp-posters.html