Cartoon from Paul Silli's blog post "Why Should School Districts Invest in Technology."

James Hoff, a Fellow at the Schwartz Communication Institute, just posted to the Institute’s blog a provocative argument against teaching with technology entitled “Back to Basics: Resisting the Allure of Web Technology in the Classroom.” Bellow is a snippet.

As a profession we seem to have thoughtlessly embraced the idea of technology precisely because we see it as a way of making learning easier and more accessible for more of our students. Obviously—the logic goes—our students are comfortable using the Internet and social networking tools, so why not allow them to use those skills to learn? This kind of thinking is common among instructors who embrace popular culture because they think it will help their students “relate” to the course material. These are the same teachers who spend class time screening Hollywood versions of Shakespeare because students are supposedly incapable of understanding Elizabethan English or who teach rap lyrics or song lyrics as poetry, because it’s easier for students to get the difference between a tenor and a vehicle when it’s Tupac or Bob Dylan speaking than when it’s Dylan Thomas or Langston Hughes. But our calling as educators extends beyond merely providing our students with opportunities to learn material. As educators we are also responsible for providing our students with experiences which they would not otherwise have access to, such as the experiences that result from finding solutions to difficult problems, engaged and thoughtful conversation, and collegial argument. But even more than this, it is important that we offer our students alternatives to the kinds of experiences provided by the technology of mass media. If we are going to insist on teaching them how to get by in the corporate world they’ve been given, we need to at least teach them that other worlds are still possible.

You can read the entire post and comment on it here.

The idea of the FRO blogging project, a collaboration with our colleagues at Advisement and Orientation, is to provide first-year students with an online, public space for reflecting on a number of required projects and activities as well as their experiences of acclimating to college life — to give our incoming students yet more curricular opportunities to write and, in the process, to increase engagement and deepen their thinking about what they are learning and experiencing in their first year at Baruch.

Luke’s post details some of the nitty gritty of the project, which is one big experiment (a full-scale pilot, if you will) that we hope will teach us quite a bit more about the pedagogical potential of online personal publishing in introductory programs and courses. We hope, for example, to look closely at the tremendous variety of writing we have seen in the FRO blogs so far (from well articulated, impressively developed posts resembling mini-essays to brief, informal missives written in like SMS text messages) and explore ways in which to better teach students the conventions of college-level written discourse. For now, we’re focused fairly heavily on logistics and mechanics and look forward to building on and refining the programmatic and pedagogical aspects in coming semesters.

Student blogging seems to be a natural fit for typical Freshman Seminars, so I would expect that other schools have tried something like this though it does appear as though we at Baruch are blazing new trails. If you know of other schools doing something similar, please let us know.

Like most case study teaching notes, there is a recommended teaching plan. Immediately after suggesting that instructors ask, “Given what you know from the case, what would you recommend…?” Spear says, “Wait! Give students a chance to offer responses. Instructor silence is a powerful tool!”

If you read my 26-Nov-2008 post “Understanding ‘The Pause’,” hopefully Spear’s remark puts a smile on your face.

Spear offers other advice uncommon in most teaching notes. For example, he later suggests, “A key objective is to teach them [the students], through experience, to be specific both in terms of what they have observed and also in terms of what they would recommend. Therefore it is the responsibility of the instructor to challenge students.” And a little later in the lesson plan he advises, “Don’t let students off the hook. Whatever their response, ask…”

I appreciate these comments because case studies are hard work. They require significant student reading and digesting time as well as prep time on the part of the professor. However, when they work well, even exhausted executives have lively discussions at 8 pm at night. A little silence and challenge do go a long way.

(For those interested in learning more about the art of case teaching, please allow me to plug Baruch’s fall 2009 workshops.)

It recently occurred to me that very little has been written about the Schwartz Communication Institute’s most ambitious and potentially most promising project, our Video Oral Communication Assessment Tool, or VOCAT. I have presented on VOCAT a number of times over the years (most recently at the 2009 Computers and Writing conference in June), but have not yet written about it in any kind of real detail. So it’s high time to remedy that.

VOCAT is a teaching and assessment web application. It is the fruit of a collaboration between the Schwartz Institute and mad genius code-poets at , Cast Iron Coding, Zach Davis and Lucas Thurston. It is still very much in development (perpetually so) but is already in use in introductory speech communication and theater courses as well as a number of assessment projects. Our career center used it effectively a few semesters ago as well. To date, approximately 3200 Baruch students have used the tool.

VOCAT was developed in recognition of the principle that careful, guided review of video recordings of their oral presentations (or of any performance, for that matter) can be remarkably effective for aiding students in becoming confident, purposeful and effective speakers. It serves as a means for instructors to easily provide feedback on student presentations.  It enables students to access videos of their performances as well as instructor feedback and to respond to both. It likewise aggregates recorded presentations and instructor feedback for each user and offers an informative snapshot of a student’s work and progress over the course of a given term or even an entire academic career. Presentations can be scored live, as students perform, or asynchronously once the videos have been uploaded. (Our turnaround time at this stage is between one and seven days depending on how many sections are using the tool at once — once some of the key steps happen server-side, turnaround time will not be as much of a concern.) Built on the open source TYPO3 content management system, it is a flexible, extensible and scalable web application that can be used at once as a teaching tool and as a means of data collection for research or other assessment purposes. (Screenshots are available here. I am also happy to share demo login info with anyone who would like to take a look — please email me at mikhail [dot] gershovich [at] baruch [dot] cuny [dot] edu.)

While VOCAT is quite feature-rich at this early stage, especially when it comes to reporting, data export, and rubric creation, we are always thinking about ways in which the tool can be made more robust and flexible. Currently, we are playing around with adding a group manager feature for group presentations, tagging for non-numeric assessment, moving from QT to Flash video, video annotation, as well as server-side video processing and in-line video and audio recording. We are also considering allowing users to choose to enable social functionality to take advantage of web 2.0 tools for sharing and commenting on one another’s work. And since, at its core, VOCAT is a tool for aggregating and responding to anything that can be uploaded, we’re thinking about other uses to which it could be put. It could easily, for example, be adapted for writing assessment. And someone once suggested that it could be useful for teaching bedside manner for medical students. Adapting VOCAT for these purposes is hardly a big deal.

The platform on which VOCAT is built is open source but the tool itself is not yet open. Right now, it is Baruch’s alone. Whether it should stay that way is a question much discussed around here. Here at the Institute we face several critical issues around open education, not the least of which is conflicting views on student access of Blogs@Baruch. In regards to VOCAT, however, the one thing constantly on my mind is the tension between an internal drive to share the tool as an open-source web application and build a community around it (there are no shortage of interested parties) and the pressures (or maybe a pernicious institutional common sense) that seem to compel us to keep VOCAT proprietary and use it to generate as much revenue as possible. I have heard arguments that VOCAT should be Baruch’s alone — that we should charge for its use and seek private funding for its deployment and development. This is a business school, after all, and I’m sure promoting and marketing VOCAT could be an interesting project for an upper division Marketing course.

Yet, I am inclined to believe that VOCAT should be shared freely and widely with other institutions and that other developers should be encouraged to develop for it.  A great many more students would benefit and development would certainly be accelerated as more and more schools add features they need that could then be adopted for use here. Were VOCAT open, in other words, it would evolve quickly and probably in ways we haven’t even imagined. And that is very exiting.

In the coming months, I hope to continue to present on VOCAT and to gain insights into the roles it can play in communication intensive courses or in a communication-focused curriculum of any sort. More importantly, I would like to move towards opening it up and will work with our developers on the features and functionality that facilitate sharing. I hope also to draw upon the tremendous expertise of my friends and colleagues involved in the open education movement and learn from those who have worked with and developed various open source tools for teaching and learning. Listening to others’ ideas for VOCAT has been invaluable to thinking through what this web app could ostensibly do with the right sort of development.  could be and how to best realize its full potential as a teaching tool — both in terms of deployment, training, and development.

“During the 1980s, I taught United States history to American soldiers stationed in South Korea. In one fairly typical class, not one of the students had heard of the Battle of Gettysburg. Perhaps their high school teachers considered the subject a mere matter of military history. But most of the people who fought and died at Gettysburg were ordinary Americans, and our lives would be very different now if the ones wearing blue had stayed home.”

A number of assumptions here are worth questioning. What first leaps out at me is the notion that if students seem not to recognize a topic, or don’t respond to questions about it, it is ipso facto clear that they don’t know anything about it and that they weren’t taught about the subject. I’m really quite flummoxed by these assumptions.

Students refrain from expressing familiarity with a topic for a great many reasons. In addition to the straightforward proposition that they indeed don’t know anything about it, there are oh so many other possibilities. It may be because they’re shy about participating in discussions; because they don’t remember clearly what they were taught; because they’re uncertain about how what they recall will be received by the instructor or fellow students; because there’s tension between what they learned and what they believe; or because of any other of a host of social and/or interpersonal dynamics in the classroom. To conclude that students don’t know about something simply because they don’t readily respond to questions about it is to misunderstand the social milieu of one’s own classroom.

It equally the case that students often don’t recall what they’ve learned until a context for the topic is fleshed out. As the subject is discussed and elaborated upon, gradual recognition and recollection may bring it slowly back to mind.

Students, of course, do forget much they’ve been taught. It’s unlikely that most students could recapitulate everything that’s been covered in a single class period, let alone an entire semester. The fact that students claim not to recall something is a very poor guide to whether they’ve ever been taught it.

The Battle of Gettysburg is remembered primarily because of the speech President Lincoln made to consecrate the memorial there. Without downplaying its size and significance, I nevertheless point out that it was only one battle out of many fought during the Civil War, and that ordinary Americans fought in them all. It strikes me as odd that knowledge about a single battle would be used as evidence of ignorance about the meaning of an entire war.

Finally, my own operating principle is that if students don’t know (or remember) something that I think is important, it’s my job to figure out how to teach it to them in such a way that they will remember it, rather than rail at them for not knowing it.

Question 1: On average, how intellectually challenging are courses at Baruch College? (5 = very challenging; 1 = not challenging at all)

Question 2: On average, how satisfied are you with this level of intellectual challenge?

Question 3: What does “intellectual challenge” mean to you?

Question 4: Which wording would you prefer on the course evaluation form?

Selected Comments:

1. Since English is my second language, courses are very challenging for me.

2. The courses at Baruch College are very challenging. The time frame in which professors needed assignments are not enough, due to the fact that we, the students, have other classes.

3. Although some professors manage to make course work challenging, they still keep students interested.

4. i believe the “intellectual challenge” is subjective to the instructor teaching the course. In some cases they do present a real challenge; on the other hand, some are as easy as it can get!!!

5. I’m glad to see this kind of survey. Most Baruch courses are definitely lacking on the stimulating side. Sometimes it feels like a waste of time–even if the course content seemed promising. Professors seem to be dumbing down a great deal.

6. Some of the courses are stimulating, but for the most part it is professors teaching to get a paycheck (a small one at that) and students participating to get a grade, but there does not seem to be a high level of engagement from either students or instructors.

7. In my experience at Baruch College, the majority of the courses are intellectually stimulating and enforce learning. However, there are some courses that are too difficult, and the work load should be lightened (i.e. Calculus).

8. I love Baruch–their educators, the academics, the social environment, the diversity. I am so proud to be a graduate of Baruch.

Conclusion: Students seem to feel reasonably challenged and find the level of challenge appropriate (Questions 1 and 2). When it comes to the definition of “intellectual challenge,” however, the picture gets somewhat confusing. Some see challenge as stimulation, and others see it as stressful and not essential for learning. (Question 3). The comments echo the confusion: Variably, challenge is seen as coming from lack of language proficiency or lack of time (rather than intellectual difficulty), or defined as positive (related to stimulation and engagement), or negative (likely to reduce interest, not enforcing learning).

Should the last item on the Baruch evaluation form be changed to something less fraught with conflicting interpretations . . . for example, “The course stimulated me intellectually”?

I take a deep breath and write that I find I have deep and progressively more distressing doubts about the worth and efficacy of teaching evaluations, at least as they exist at Baruch College. There, I’ve said it.

I lay no claim to having systematically studied Baruch’s evaluations as a whole. But I carefully scrutinize every evaluation of every member of my department every term, and as a member of the School of Liberal Arts & Sciences’ Personnel and Budget Committee I see the evaluations of every member of the arts and sciences faculty who comes before the committee for personnel actions, including reappointments, tenure and promotion, and sabbaticals. I’m probably as familiar with the college’s teaching evaluation patterns as anyone. And among the things I see are several consistent tendencies that trouble me. Trouble, as in “Why do we put so much emphasis on such imperfect instruments.” Let me quickly note that I wish the college paid a whole lot more attention to the importance of teaching in tenure and promotion processes than it does; what I’m talking about here are the evaluations, not the teaching. And, because this is a blog, I’m not going to go into detail; I’m merely pointing out some of the issues that concern me.

First, it is my considered opinion that our evaluation format serves primarily as a popularity contest. Because I’ve observed all my faculty (except some of the very newest GTFs and adjuncts) in the classroom, I have a sense of the relationship between what I can actually see of their skills and how students rate them. There is some consistency at the lower end, I think; people who are in my opinion less than skilled teachers do tend to get lower ratings, but I’m not sure there’s a strong correlation here. What I do find consistently, though, is that nice guys tend to finish first. I’ve seen accomplished teachers get lower scores because of their personalities. And I’ve seen at least one colleague consistently receive 5.0s while teaching not much differently than anyone else in our department. I have come to the conclusion that students tend to rate their profs by how much they like them, not by how skillful or effective their teaching is.

When I examine the entire range of items on a single evaluation printout, I find very little variation from item to item (though I would exclude from this generalization the final items about improvements in communication skills, etc., on the new forms). That is, if students approve of their instructors on the whole, they tend to rate them highly on every individual item. They tend to discriminate minimally among the different qualities the form asks them to evaluate. The fundamental idea behind our format is for us to receive useful feedback about our performances on specific aspects of our teaching (i.e., commentary on where we could use improvement), but I have little sense that this is what is actually happening.

The scores of a significant majority of my department’s faculty cluster rather closely together. While a few get scores in the mid to high 3s, and thus lower the department’s mean a bit, most get scores between 4.2 and 4.6. On the one hand, this isn’t a bad thing; it’s nice to see so many fine teachers in my department. On other hand, how do we make sense of a 5-point scale when almost everyone’s scores lie within a 4 decimal point range near the top? I’d like to be able to take teaching evaluation scores into account in my decision-making, but I can’t seriously make discriminations between people when nearly all the scores are bunched together this way, nor does this pattern incline me to think that students are employing their critical faculties when they evaluate us.

There is much more to be said about this. Most of you are familiar with many of the other criticisms that are regularly leveled against the evaluation process and format. I’ve opened the valve just a bit to release some of the pressure. I repeat, my comments here are in no way meant to downplay the importance of teaching. To the contrary, I so value good teaching that I’m troubled by how poorly we evaluate it. We are using an inherently flawed process as a guide to making critical tenure and promotion decisions. Experience tells me that it is next to impossible to change our evaluation procedures (it took 25 years or so to achieve the last changes to our format, changes I see as mostly cosmetic, though there are those who disagree with me about this), and I reluctantly acknowledge that I don’t have any alternative clearly in mind. I just thought I’d use this occasion to rant a bit.

In the course of a casual conversation with our Provost after I had written this, I mentioned some of these thoughts. He was surprised at my conclusions and told me research shows that evaluations are valid instruments. But as we pursued the topic a bit, it became clear that what he’s talking about-at least as I understood him-is the difference between someone with 2.5 and someone with a 4.5. Yes, I recognize that these evaluations can be used to spot the occasional complete incompetent. But that’s not how we use them most of the time; it’s actually quite rare to find someone consistently below 3.5 and there really aren’t all that many people consistently below 4.0, at least in the School of Liberal Arts & Sciences.

Most of you are probably familiar with the old saw: Those who can, do; those who can’t, teach. I once heard a coda: Those who can’t teach, teach pedagogy. I used to find the notion funny, but as I’ve observed new faculty beginning their careers over the years I’ve come increasingly to appreciate just how much craft goes into teaching. Good teachers may make it seem effortless, but it’s not. This perhaps explains why many folks think that teaching doesn’t call for as much a skill as other occupations. One antidote to this tendency to overlook the techniques we’re employing in the classroom is to devote a bit of time to pointing out to our students just what it is we’re doing. This can serve both to make them aware of the cues and signals we’re sending them, and to get them to understand how they can put this awareness to work in the rest of life. Here are a couple of the very simple things I point out to my students.

One is the way I use the whiteboards. I’ve never adopted PowerPoint because for me it seems to constrict spontaneity, creative flow, and opportunities to let students’ questions and arguments shape the direction of the class. I can write something on the whiteboard and then come back to it as often as I find myself needing to in the course of a lecture or discussion. Sometimes I return again and again to a key concept. At some point early in the term, I stop and point out to students that if they pay attention to what I’ve been doing, they will see that a particular term or phrase or illustration on the board has gradually acquired a halo of surrounding emphases, underlining, circling, stars, etc. “If you see a concept on the board that’s been well marked-up,” I tell them, “you should be sure to mark it up in your notes. Highlight it, draw big arrows pointing to it. I can assure you that when you’re writing your essays it’s a concept you’re going to want to include, to explain, and to emphasize.”

Likewise, when we’re reading in class, going over crucial passages in whatever text has been assigned, I explain to them that they should be highlighting the passages I’m dwelling on. And if I repeat a phrase or passage several times, slowing down and reading it dramatically, I tell them they should be marking it up accordingly, because it’s something that’s bound to be useful when they’re writing about the reading. (I note in passing here that that despite all the problems with the high costs of books, I urge my students to purchase their own used copies so that they can bring them to class and read and mark them along with me.)

One more example. I occasionally stop and exhort my students to be clever enough to figure out just what it is that Prof. Petersen wants from them, and then give it to him—it’s what we used to call “psyching out” the professor. College, I tell them, is real life, not simply an interlude before it begins. “When you get a job,” I explain, “one of your primary tasks is to figure out what your bosses want from you, and then give it to them. The classroom’s no different, really. I’m trying to teach you things I think are important, and I’m doing all I can to get you to pay attention to them. So pay attention to me, figure out—based on these cues, subtle or direct—what it is I think is most important, learn it, and then write essays that demonstrate to me that you’ve learned it. (This doesn’t mean rote memory, just a focus on key ideas.) Voila! You’ve figured out how to get promoted at your first job.” Simplistic, perhaps. But these are all small ways of showing students how to observe what’s going on around them, how to gauge what’s important, and how to find out what they’re being expected to know and do.

(Glenn’s caveat: I’m writing this for new teachers, folks still struggling to find their way in the classroom, not for seasoned professionals, though the old salts among you are welcome to it.)

Students’ speaking in class is highly valued and rightly so. Those of us who practice student-centered instruction don’t want to be the only one speaking during the class session. We also don’t want only a handful of our students participating in discussions. Therefore, I appreciated when Mel Silberman, author of several books on active training, conducted a session at the Baruch College Faculty Orientation in August in which he offered some tips on how to increase participation – tips included “pre-discussion” and students’ calling on the next speaker. And I have to say his methods worked; he increased participation in the session. My concern is the narrow focus on speaking without giving students an alternative to expressing themselves. An alternative that may embolden students to speak in class later in the course or down the road in other courses.

When I first started teaching, I was particularly sensitive to students who are not comfortable speaking in class regardless of the reason. I wanted to give students an alternative way of participating. While I was looking at sample syllabi, I came across a syllabus that incorporated another method of participation, a response sheet. I am sorry to say I do not have a record of the source of this way of using response sheets and have only one copy of a syllabus with my directions to students- with a few years away from work as a full-time mom, crashed hard drives and flooded basements – more than a few things have been lost. As I have stated, participation in class discussion is highly valued and often represents a portion of each student’s final grade. So why do we appear to only value oral participation? Could we not have written participation? The response sheet is an alternative avenue of participation for students that has worked well in my classes.

Here is my sample from the syllabus for a course that had weekly class sessions:

Class Participation

To receive 100% on class participation, you will need to speak in class on average about once per week. There will be opportunities to speak every session during lectures and classroom exercises. For students uncomfortable with speaking in class, you can participate by completing a one-page reaction sheet (2 paragraphs at a minimum) for each class session. State what you learned or how the session impacted you as an individual. It could be a reaction, an insight, an opinion, but NOT a summary of the material covered. Reaction sheets are not required but strongly recommended. All students are encouraged to do them, even if you participate regularly in class. The additional credit may help compensate for other areas of weakness such as test taking. Reaction sheets are due the next class session and will not be accepted beyond that. E-mail submissions are accepted and encouraged.

Some guidelines for the response sheet are as follows:

• For the first 2 response sheets received in a semester from a student, I give written feedback and a chance to resubmit for participation credit if the response sheet does not adhere to guidelines or is lacking in substance. I also read one or two samples out loud to the class as models of good response sheets (when I feel strongly that the anonymity of the student can be maintained). On all subsequent response sheets, I indicate whether the participation point has been earned or not and include other comments as I am inspired to.
• You can also keep the practice of reading a few response sheets out loud at the next class session and spend a few minutes discussing them with the class.
• I recall requiring response sheets or in-class participation in discussions for 10 weeks in the semester (for however many class sessions that works out to) in order for students to earn the maximum participation points.

In my experience, less than 25% of students submit response sheets. Some students started with response sheets then abandoned them because they were participating in class discussions. Perhaps they just needed time to feel more comfortable participating.

I really see this as one small way we can provide a more welcoming learning environment. Response sheets enable us to engage quiet students in a way that can lead to their speaking in class later in the course. If not, we still communicate that we value their input, and we too can benefit from the insights students express in writing. Imagine from the quiet student’s perspective of feeling encouraged as opposed to pressured to speak in class. I believe this creates a learning environment in which class participation will actually increase.