It recently occurred to me that very little has been written about the Schwartz Communication Institute’s most ambitious and potentially most promising project, our Video Oral Communication Assessment Tool, or VOCAT. I have presented on VOCAT a number of times over the years (most recently at the 2009 Computers and Writing conference in June), but have not yet written about it in any kind of real detail. So it’s high time to remedy that.

VOCAT is a teaching and assessment web application. It is the fruit of a collaboration between the Schwartz Institute and mad genius code-poets at , Cast Iron Coding, Zach Davis and Lucas Thurston. It is still very much in development (perpetually so) but is already in use in introductory speech communication and theater courses as well as a number of assessment projects. Our career center used it effectively a few semesters ago as well. To date, approximately 3200 Baruch students have used the tool.

VOCAT was developed in recognition of the principle that careful, guided review of video recordings of their oral presentations (or of any performance, for that matter) can be remarkably effective for aiding students in becoming confident, purposeful and effective speakers. It serves as a means for instructors to easily provide feedback on student presentations.  It enables students to access videos of their performances as well as instructor feedback and to respond to both. It likewise aggregates recorded presentations and instructor feedback for each user and offers an informative snapshot of a student’s work and progress over the course of a given term or even an entire academic career. Presentations can be scored live, as students perform, or asynchronously once the videos have been uploaded. (Our turnaround time at this stage is between one and seven days depending on how many sections are using the tool at once — once some of the key steps happen server-side, turnaround time will not be as much of a concern.) Built on the open source TYPO3 content management system, it is a flexible, extensible and scalable web application that can be used at once as a teaching tool and as a means of data collection for research or other assessment purposes. (Screenshots are available here. I am also happy to share demo login info with anyone who would like to take a look — please email me at mikhail [dot] gershovich [at] baruch [dot] cuny [dot] edu.)

While VOCAT is quite feature-rich at this early stage, especially when it comes to reporting, data export, and rubric creation, we are always thinking about ways in which the tool can be made more robust and flexible. Currently, we are playing around with adding a group manager feature for group presentations, tagging for non-numeric assessment, moving from QT to Flash video, video annotation, as well as server-side video processing and in-line video and audio recording. We are also considering allowing users to choose to enable social functionality to take advantage of web 2.0 tools for sharing and commenting on one another’s work. And since, at its core, VOCAT is a tool for aggregating and responding to anything that can be uploaded, we’re thinking about other uses to which it could be put. It could easily, for example, be adapted for writing assessment. And someone once suggested that it could be useful for teaching bedside manner for medical students. Adapting VOCAT for these purposes is hardly a big deal.

The platform on which VOCAT is built is open source but the tool itself is not yet open. Right now, it is Baruch’s alone. Whether it should stay that way is a question much discussed around here. Here at the Institute we face several critical issues around open education, not the least of which is conflicting views on student access of Blogs@Baruch. In regards to VOCAT, however, the one thing constantly on my mind is the tension between an internal drive to share the tool as an open-source web application and build a community around it (there are no shortage of interested parties) and the pressures (or maybe a pernicious institutional common sense) that seem to compel us to keep VOCAT proprietary and use it to generate as much revenue as possible. I have heard arguments that VOCAT should be Baruch’s alone — that we should charge for its use and seek private funding for its deployment and development. This is a business school, after all, and I’m sure promoting and marketing VOCAT could be an interesting project for an upper division Marketing course.

Yet, I am inclined to believe that VOCAT should be shared freely and widely with other institutions and that other developers should be encouraged to develop for it.  A great many more students would benefit and development would certainly be accelerated as more and more schools add features they need that could then be adopted for use here. Were VOCAT open, in other words, it would evolve quickly and probably in ways we haven’t even imagined. And that is very exiting.

In the coming months, I hope to continue to present on VOCAT and to gain insights into the roles it can play in communication intensive courses or in a communication-focused curriculum of any sort. More importantly, I would like to move towards opening it up and will work with our developers on the features and functionality that facilitate sharing. I hope also to draw upon the tremendous expertise of my friends and colleagues involved in the open education movement and learn from those who have worked with and developed various open source tools for teaching and learning. Listening to others’ ideas for VOCAT has been invaluable to thinking through what this web app could ostensibly do with the right sort of development.  could be and how to best realize its full potential as a teaching tool — both in terms of deployment, training, and development.

Have you noticed that textbook publishers are promoting web-based homework systems such as Prentice Hall’s Grade Assist (PHGA), McGraw-Hill’s Homework Manager and Wiley’s eGrade?

All 20 sections of Finance 3000 are using the McGraw-Hill product. Students do homework online and receive instantaneous feedback (with solutions), professors enjoy automated grading, and the coordinator appreciates bolstered grading fairness across sections. No two students get the same question due to randomized seed numbers (e.g., student 1: “solve X + 219 = 567″; student 2: “solve X + 98 = 673″). If a student doesn’t like his/her score, the entire problem set may be redone, with new seed numbers, and the professor’s grade report includes the score of every attempt.

I trialed PHGA with 80 MGT 3121 students, spring 2008. Students complained that they often reasoned correctly, but made errors inputting numeric answers in the software, and thus redid entire assignments (with new seed numbers) to get the points they felt they deserved. In some cases I had to agree with the students—the software is not perfect. My larger concern is that none of the types of questions that promote deep learning are available in the software. Rather, standard “textbook” questions—questions with a single correct answer such as “determine the reorder point and reorder quantity” or “forecast demand on day 150″—lulled students into deep comas. It’s about as exciting as the computerized SAT test.

Worried that web-based homework trades richness of student thinking for my convenience, I stopped using the software.

While some studies find that web-based homework leads to improvement in grades (see Heizer et al. [1] for a survey), I’m worried that improvement in grades is not the appropriate metric. Has anyone seen a study that shows that web-based homework promotes profoundly deep learning or the ability to apply the course content in unfamiliar contexts—the type of learning that is more likely with open-ended questions?

Andrea Pascarella [2] found that web-based homework encourages college physics students to abandon careful reasoning in favor of a guess-and-check approach due to the multiple-try option. She says

“Based on the feedback literature, it is known that corrective feedback combined with multiple tries can lead students to adapt a trial-and-error strategy with a focus on completing the assignment versus learning the fundamental principles underlying the material. … One way to help achieve the goal of making web-based homework systems more valuable learning tools would be to cut down the number of tries each student has to get the correct answer.” (p. 7)

Jay Heizer (the lead author of [1]) sees multiple tries as an advantage. He told me he now sees the majority of his students redoing entire homework assignments multiple times (some as many as 4-5 times) to obtain a high score. Naturally, we want our students to rework the problems they get wrong… or is this further evidence of Pascarella’s concern? Not sure.

Another study failed to show any web-based homework benefits. After controlling for teacher experience and student “academic competence” in a college statistics class, Palocsay and Stevens [3] found that “the technique used to deliver homework makes little difference in student success.” At least web-based homework doesn’t hurt our students.

I want to excite students about my discipline but perceive web-based homework as counterproductive in this regard. Am I being stubborn, ignorant, or both for refusing to adopt?

—

[1] Heizer, J., B. Render, K. Watson. 2008. Web Based Instruction Improves Learning. Decision Line, forthcoming.

[2] Pascarella, A.M. 2004. The Influence of Web-Based Homework on Quantitative Problem-Solving in a University Physics Class. Proceedings of the NARST 2004 Annual Meeting, Vancouver.

[3] Palocsay, S.W., S.P. Stevens. 2008. A Study of the Effectiveness of Web-Based Homework in Teaching Undergraduate Business Statistics. Decision Sciences J. of Innovative Ed. 6(2) 213-232.

As I’ve moved along in my teaching, I ask myself, “What is it that these students should know? What do they care about? What can bring them closer to the music and to the historical period?” At this point, I can honestly say that some of the information I now convey to my students is . . . uh . . . less than factual. At best, I suppose, it can pass as historical speculation. I’ve moved far away from factual detail in my lectures to a gray area of broad generalization and rhapsodic rambling.

Is it important that Mozart wrote a certain number of operas, or is it better that students know that he and his Italian librettist (Lorenzo da Ponte) were Masonic proto-revolutionaries? Should we call Mozart a “lofty genius” or rather think of him more as a musical Rainman with a touch of dyslexia perhaps and a smidgen of ADHD, as a contemporary scientific account alludes? When covering the 19th century, I want my students to think of Robert Schumann struggling with bi-polar disorder (and/or was it with his symptoms of late-stage syphilis?). I offer the class Schubert’s hypothetical pedophilia as explanation of a 17-year-old’s sensitive setting of “Erlking.”

There is certainly some evidence for these notions, but what do we really know? We really know dates for events and numbers of compositions, however, this is not the substance of a lecture; these facts are not worthy of study at anything but the graduate level. This is not history. All the trashy little tidbits that I spew paint a broader picture of Enlightenment and Romantic ideals. I want my students to identify with these historical figures as real people perhaps with lives similar to their own. Or so what if I have to sketch out characters as olde timey pop stars whose every move would today be covered by ET (Entertainment Tonight)? So, over time, I’ve moved away from a fastidious accuracy that marked my youthful scholarship. The broad sweep, the hyperbolic–that’s what works for me in the classroom. The factual details have become for me the dust of history.

*I just made this up, btw. This is becoming very easy to do.

Christoforatou and Freedman have achieved an inter-disciplinary collaboration that eludes most of us – even those of us charged to collaborate by the mission of the Learning Communities program. Most of us try, but cross-disciplinary course coordination is tough. It’s difficult to pick up a new discipline over the summer. But Christoforatou and Freedman have discovered another way to coordinate their courses, and neither had to train in the other’s specialty. In fact, their approach embraced the differences among their disciplines even as it reinforced particular ways of thinking and doing. So, how did they do it?

Rather than searching for terminological, conceptual, or other content overlaps between Art 1000 and English 2100 (good luck finding them!), the innovative instructors decided to weld their courses at an epistemological point. When they realized during their first meeting that they both embraced the concept of knowing the world via abstraction (abstract art, abstract linguistic concepts), they ran with the idea. Since then, they have crafted a cross-disciplinary collaboration complete with field trips and complementary assignments. Christoforatou has embarked on a quest to learn about abstract, modern art (alongside her students), while Freedman (and the rest of the LC) has been introduced to creative approaches to the text employed by Medievalists (hint: they are often surprising, ironic, and highly abstract). Rather than trying to incorporate graphic design into English 2100 or composition assignments into Art 1000, the two have achieved crossover in a more meaningful way. They teach to their strengths while reinforcing each other’s attempts to inculcate habits of critical thinking in their first-year students. The result? A partnership that has proved professionally fulfilling… and never-ending lines of students outside their offices.

I decided to try PowerPoint (known by its .ppt file extension). I slogged through the program, determined to make slides for three lectures. I compiled some nice illustrations of sound waves and musical instruments, discovered pleasant fonts and colors, added some zippy effects, and spent a lot of time trying out the various bells and whistles. In the end, I found the product ossifying. Something about the slides made them impossible to “riff off of,” as musicians might say. I was disappointed in the software, but figured, “It must be me,” since I was age 50, and this was a new technology that demanded a more malleable brain and a youthful receptiveness to the “new media” I suppose I had lost.

Over the last few years, I’ve had the opportunity to see two, much younger colleagues teach on straightforward music-historical topics using PowerPoint. The first instructor presented a mess of slides, filled with typos and formatting inconsistencies, quite an unstructured piece of work that was hard for me to follow. The second instructor was worse. Besides using a historically inappropriate illustration (a saccharine, late 19th-century rendering of Martin Luther), the second colleague provided too much information. Each of the slides comprised long lists of seven or eight bullet points with a mass of detail. Furthermore, he was incapable of reading the muddle of facts. The student seated next to me was furiously copying the useless text (e.g., “Luther married a nun, Katherine von Bora when he was 42 and she was 26″–remember, this is a music class). My other experiences viewing ppt presentations have not been much better than these.

Hmm. At that point I knew something was wrong. I knew it wasn’t me.

At the end of last semester, I came upon John Medina’s book Brain Rules. In it he discusses ways of keeping your brain operating at maximum efficiency. In the online chapter on “Vision” (his Rule #10), he states the following:

Toss your PowerPoint presentations. It’s text-based (nearly 40 words per slide), with six hierarchical levels of chapters and subheads—all words. Professionals everywhere need to know about the incredible inefficiency of text-based information and the incredible effects of images. Burn your current PowerPoint presentations and make new ones.

The online film for Rule #10 does make reference to the business applications of pictures and points up the potential failure of wordy ppt slides. This is not to say, of course, that illustrations and graphs shouldn’t be used. On the contrary, Medina makes it clear that the use of pictures increases our understanding and reinforces our memory. Therefore ppt pictures should be primary, superseding any verbal information on the slides.

My own feelings about ppt include the lack spontaneity it engenders. The instructor can’t very well follow the flow, the lead of the class, since the slides and their order have to be determined beforehand. To my mind, there’s something substantially different between underlining or circling a handwritten word with a marker and highlighting a term with a mouse or making the words flash. The intimate contact with the class that an instructor tries to achieve (especially in a large class) is rendered cold and distant when mediated by ppt technology. I think it’s much more useful, in a classroom situation, to write out words or sketch out graphs and symbols by hand, with the dynamism of a high school geometry teacher rather than rely on last night’s (or last year’s) bright idea.

I suppose I have three main problems with ppt: 1) it’s sometimes (often?) sloppily done; 2) it’s too frequently wordy; and 3) its use often further separates the instructor from the students.

So I ask you, is there a proper use for ppt? In my opinion, pfft!

[youtube http://www.youtube.com/watch?v=HLpjrHzgS...[/youtube]