It recently occurred to me that very little has been written about the Schwartz Communication Institute’s most ambitious and potentially most promising project, our Video Oral Communication Assessment Tool, or VOCAT. I have presented on VOCAT a number of times over the years (most recently at the 2009 Computers and Writing conference in June), but have not yet written about it in any kind of real detail. So it’s high time to remedy that.

VOCAT is a teaching and assessment web application. It is the fruit of a collaboration between the Schwartz Institute and mad genius code-poets at , Cast Iron Coding, Zach Davis and Lucas Thurston. It is still very much in development (perpetually so) but is already in use in introductory speech communication and theater courses as well as a number of assessment projects. Our career center used it effectively a few semesters ago as well. To date, approximately 3200 Baruch students have used the tool.

VOCAT was developed in recognition of the principle that careful, guided review of video recordings of their oral presentations (or of any performance, for that matter) can be remarkably effective for aiding students in becoming confident, purposeful and effective speakers. It serves as a means for instructors to easily provide feedback on student presentations.  It enables students to access videos of their performances as well as instructor feedback and to respond to both. It likewise aggregates recorded presentations and instructor feedback for each user and offers an informative snapshot of a student’s work and progress over the course of a given term or even an entire academic career. Presentations can be scored live, as students perform, or asynchronously once the videos have been uploaded. (Our turnaround time at this stage is between one and seven days depending on how many sections are using the tool at once — once some of the key steps happen server-side, turnaround time will not be as much of a concern.) Built on the open source TYPO3 content management system, it is a flexible, extensible and scalable web application that can be used at once as a teaching tool and as a means of data collection for research or other assessment purposes. (Screenshots are available here. I am also happy to share demo login info with anyone who would like to take a look — please email me at mikhail [dot] gershovich [at] baruch [dot] cuny [dot] edu.)

While VOCAT is quite feature-rich at this early stage, especially when it comes to reporting, data export, and rubric creation, we are always thinking about ways in which the tool can be made more robust and flexible. Currently, we are playing around with adding a group manager feature for group presentations, tagging for non-numeric assessment, moving from QT to Flash video, video annotation, as well as server-side video processing and in-line video and audio recording. We are also considering allowing users to choose to enable social functionality to take advantage of web 2.0 tools for sharing and commenting on one another’s work. And since, at its core, VOCAT is a tool for aggregating and responding to anything that can be uploaded, we’re thinking about other uses to which it could be put. It could easily, for example, be adapted for writing assessment. And someone once suggested that it could be useful for teaching bedside manner for medical students. Adapting VOCAT for these purposes is hardly a big deal.

The platform on which VOCAT is built is open source but the tool itself is not yet open. Right now, it is Baruch’s alone. Whether it should stay that way is a question much discussed around here. Here at the Institute we face several critical issues around open education, not the least of which is conflicting views on student access of Blogs@Baruch. In regards to VOCAT, however, the one thing constantly on my mind is the tension between an internal drive to share the tool as an open-source web application and build a community around it (there are no shortage of interested parties) and the pressures (or maybe a pernicious institutional common sense) that seem to compel us to keep VOCAT proprietary and use it to generate as much revenue as possible. I have heard arguments that VOCAT should be Baruch’s alone — that we should charge for its use and seek private funding for its deployment and development. This is a business school, after all, and I’m sure promoting and marketing VOCAT could be an interesting project for an upper division Marketing course.

Yet, I am inclined to believe that VOCAT should be shared freely and widely with other institutions and that other developers should be encouraged to develop for it.  A great many more students would benefit and development would certainly be accelerated as more and more schools add features they need that could then be adopted for use here. Were VOCAT open, in other words, it would evolve quickly and probably in ways we haven’t even imagined. And that is very exiting.

In the coming months, I hope to continue to present on VOCAT and to gain insights into the roles it can play in communication intensive courses or in a communication-focused curriculum of any sort. More importantly, I would like to move towards opening it up and will work with our developers on the features and functionality that facilitate sharing. I hope also to draw upon the tremendous expertise of my friends and colleagues involved in the open education movement and learn from those who have worked with and developed various open source tools for teaching and learning. Listening to others’ ideas for VOCAT has been invaluable to thinking through what this web app could ostensibly do with the right sort of development.  could be and how to best realize its full potential as a teaching tool — both in terms of deployment, training, and development.

Question: A student gives a presentation. He has a heavy foreign accent and is at times incomprehensible. Overall, the speech seems well researched and on target. What do you do?

a. Give him an A.

b. Subtract points for incomprehensibility and give him a B.

c. Tell him that the presentation was unacceptable and that he should improve his oral communication proficiency.

Instructors cite a variety of reasons (often with a kernel of truth) why they let incomprehensibility slide:

1. “Asking a student to reduce his/her accent is embarrassing and discouraging.” — It is true that accents are windows to our identity, and that a student changing his/her accent may experience a tangible sense of loss or feel repercussions from home culture friends and family.

2. “It’s not possible to reduce accents in adults. Native accents can be achieved only when we learn the language before puberty.” — It is true that, after the brain hemispheres separate at puberty, a native-sounding accent tends to be more difficult to achieve.

3. “People who don’t understand foreign accents are prejudiced. They should try harder to understand.” — It is true that sometimes the problem lies with the listener. Don Rubin (University of Georgia) conducted a study some years back, in which a recording of a native English speaker was played to undergraduate students. They were told that the speaker was a college instructor and asked to rate his comprehensibility. Some groups were shown a photo of a Caucasian, others a photo of an Asian-looking individual while the recording was played. Although the recording was the same, the Asian-looking individual was ranked less comprehensible than the Caucasian.

Despite these valid objections, I believe it is paramount that we encourage students to improve their oral language skills to a level where a benevolent native speaker can understand without straining. We are doing our students a disservice if we don’t.

Our nonnative students are sometimes being ostracized by native-speaking teammates, who don’t want them to participate in presentations for fear that their grade will be jeopardized. And our nonnative Baruch graduates are sometimes dismissed at the early stages of the job interviewing process, due to heavy accents. The first practice often happens under the radar; the second when it’s too late, and students have left the college. In my view, both practices are unethical. We have a responsibility to provide students with an education that promises workplace success.

How to approach the issue? One way would be to adopt a threshhold model by which we evaluate presentations (as well as papers) only if a basic standard has been met? Any student falling below this standard is asked to revise. A threshold model would require a common standards on what constitutes acceptable form (e.g., more than five errors per page or five incomprehensibilities per minutes of speech, and additional work is indicated).

In my experience, most students welcome the opportunity to improve. When I told a Russian student in a public speaking class some years back that he was not going to pass the course unless he worked on his comprehensibility, he committed himself to weekly tutorials and almost daily language lab work. The change in this student in only one semester was extraordinary. He changed from a low-proficiency speaker to one that was 100% comprehensible.

We are lucky that Baruch offers free services to assist the students: SACC has a number of professional speech tutors (not students, but trained professionals) who work with students one-on-one. We also have a new and well-equipped ESL speech lab that is open 10 hours most days, and even on Saturdays. The lab features tons of excellent materials not only on pronunciation, but also grammar, vocabulary development, conversation management, and more. For more information, please see http://www.baruch.cuny.edu/esllab

Considering that communication skills are consistently ranked at the top of skills desired by employers of college graduates (and oral skills usually outranking written skills at that), we need to make our students aware of these services and make adequate communication skills an integral part of our evaluation procedures.

While Edward Deci’s presentation was geared towards college teaching, he stated how motivation is very broadly relevant, for example, in parenting, sports, etc. He talked about the three basic psychological needs (autonomy, competency, and relatedness) and their importance in extrinsic and intrinsic motivation. For this post, I will focus a only a small portion of his thought-provoking presentation – the usefulness of tests.

I liked what he said about the usefulness of tests. He explained that tests can be useful when we focus on the “primary function… to assess whether students have learned and can perform.” Therefore, “tests can provide meaningful feedback to students, teachers, and administrators.” One key point was to “minimize rigidity” in testing, for example, having students grade their own quizzes for feedback on how well they are learning the course material. He emphasized the importance of being respectful and responsive to students and to provide a choice whenever possible.

I have discussed with students how well the class is doing as a group on the tests and even changed the format of the final exam based upon those discussions. For example, in one class, I noticed that my students performed best on short essay questions in my assessing their knowledge and understanding of the course material. We came to an agreement that the final exam would be all short essay questions – students had to choose 20 out of 30 short essay questions to answer. I felt the outcome from this change was a better measure of what students had learned.

This leads me to ask:

Are you open to renegotiating the learning contract (the syllabus if we’re focusing on the explicit part) with your students?

What other adjustments have you made in your courses based upon students’ input in order to enhance learning and assessment?

VIRAL VIDEOS ARE A NEW FORM OF FALLOUT

Though colleges have had to manage external criticism in the past, the viral video phenomenon is a different beast. Consider the issues our campus faced a couple of years ago with the fresh(wo)man text War is a Force that Gives Us Meaning…(New York Sun article: “Baruch Requires Students Read Book Some Are Labeling Anti-Semitic“).

Though the issue received prominent attention in the print press, the back-and-forth was short-lived, the college had time to craft a response (i.e., freedom of speech), and the exchange was largely print-based. The story reached thousands – not millions. The story lacked compelling oral and visual content (e.g., yelling, crying – mooning). It paled in comparison to the storm surrounding the viral debate video (e.g., print and television stories, a rumored Chronicle investigation, a 100% funding cut for one program and potentially related cuts at other colleges). Comparatively, the War controversy was tame. Importantly, it did not result in financial fallout…

OUTSIDER OPINION AFFECTS THE BOTTOM LINE (AND POOR STUDENTS)

College costs are rising, tax levy and financial aid moneys are in flux, and increasingly we need donor/investor money to bridge the gaps. Their money enables poor, working, and middle class students to enjoy the privilege of post-secondary education (aside: thank you for subsidizing my B.A., M.A., and Ph.D. Ohio/national taxpayers!) If they respond to controversy by curtailing their support, students can be deprived of programs, perspectives, professors… To the extent that most students cannot afford the “true” costs of their schooling (i.e., a 100% tuition-funded institution…), we have to consider/manage how their underwriters perceive our campus. Viral video makes us more vulnerable… financially and intellectually…

VIRAL VIDEO DESTROYS/ENHANCES FREEDOM OF SPEECH

The viral video phenomenon presents unique freedom of speech/academic integrity issues. In the debate coach case, viral video was used to normalize and silence. The video’s post-er provided no description of background/context (e.g., these coaches haven’t slept in days, community norms do not prohibit obscenities) and encouraged viewers to censure participants (e.g., contact the coaches’ department chairs – related videos contained names and addresses!) The message was clear: “monitor yourself more vigilantly, take few risks, you are being watched.” The video complemented a growing disciplinary regime that rewards professors for conforming to ever-stricter prescriptions (e.g., professors at a number of institutions have been told: “you will teach X, you will seek Y outcomes, your Z must not offend donors…”). To me, these prescriptions are often anti-intellectual (e.g., they over-emphasize civility, stifle dissent, give outsiders veto power over academic content…). Yet, viral videos also seem incredibly empowering (I’m a post-modernist – multi-facets are my thing)…

Students can now reveal what goes on behind classroom doors. To the extent that teachers create undemocratic environments and stifle students’ growth, they can be “outed” to the outside world. Students are no longer the captives of their professors! This is exciting! And dangerous! And it deserves our attention…

So… please contribute your thoughts on this issue… both here and beyond the confines of cyberspace…

A well-written goal simply states an outcome or end result to be achieved. In other words, where do we want to go? While goals should be specific, they are often phrased in broader terms that need to be operationally defined (called “fuzzies” by Robert Mager). Now that we know where we want to go, how do we get there? This is where objectives come in. They should be specific and measurable and state what must be done to achieve the goal. In the case of learning objectives, they should be phrased from students’ perspective, not teachers’.

From an instructional design perspective, learning objectives have three purposes:

• Serve as a guide in designing a course
• Communicate to students what they are expected to achieve
• Assist in evaluating instruction

I found a good article summarizing Robert Mager’s approach to writing learning objectives: “How to Write Great Learning Objectives.” I don’t adhere to Robert Mager’s approach as a strict formula to follow, especially when it comes to less tangible subjects – instead I use his approach as a guideline in writing more specific and therefore clearer learning objectives. I have found his approach in writing learning objectives very useful in guiding and improving instruction. The place for us to start, though, is clearly defining learning goals and objectives and using these terms consistently.