Has the Work Literacy Course Been "Successful"? How Do We Know?

I'm on my way out the door, but wanted to post this question. . .

Ken Anderson has been participating in the Connectivism and Connective Knowledge (CCK08) course and on Friday asked if the course was failing:

I wonder how many are left actively participating and/or monitoring the CCK08 course.  Some 2100+ allegedly originally registered, with 18 seeking credit. The Moodle forums have been drying up over the last 3 weeks, and the reported blogs trickle to a few postings during the week and some repetitious posting of the same in the Daily.  Elluminate sessions run at less than 30 participants, down to the teens on one occasion.  I don’t know the status of the 2nd Life group nor whether there is participation in any other areas.

As we're ending the Work Literacy course this week (don't worry--all the resources will be online for the forseeable future), the issue of defining "success" in an open, social media-enhanced course like this is definitely on my mind.

Should we define it by the number of people actively posting to the site? As with the CCK08 course, we've also seen a drop in activity in the forums and blog postings, but since we specifically gave people the opportunity to participate only as spectators, it's hard to know if a lack of forum postings should be our measure of success--or failure. I will say that when you see fewer discussions happening, there's a tendency to think that something's happening, but what that is, I can't say for certain.

Should we be looking at how many people visit the site? According to our Analytics, the visits each week to the site have been fairly steady, peaking early in the week and then falling down as we approach the weekend. This we expected, because we post each week's activities on Monday. There is also the option of following forum discussions via RSS, so we don't know how many people are participating that way.

Should "success" be defined by individual learners? That is, since this was an open course that allowed people to set their own levels of participation, should the measure of success really be a matter of individual opinion?

And none of this is getting at deeper levels of evaluation, like did people learn new skills and are they applying them to their personal work and their work with learners? We have some anecdotal evidence from participants that people were learning and trying out new things,  but this was a very informal course, so what "proof" we have is pretty informal too.

I guess my big question is, how do we know if this was something we should do again? What should we be thinking about and how should we be measuring what we accomplished over the past several weeks?

I'd love to hear your thoughts on this, either in comments here or, if you're a member of the Work Literacy course, in the forum I set up to discuss the issue.

Comments

[Carla Arena]

Hi, Michelle,

I was a spectator in the course and really profited from it. I accessed links, read, bookmarked resources of interest. I couldn't participate the way I wanted because of professional commitments, but I'd say the course benefited many in different levels. I really liked the idea of you setting those levels of participation, for I didn't feel any kind of guilt of just being around, but not really actively engaging with others. Thanks for one more wonderful endeavor! Your questions are really relevant as I face the same issues with some online sessions I'm giving to my institution. It's hard to tell them that people have a choice and their own goals on how they want to engage. For some, they want to learn, but are not worried about a paper. They want to do things their own way, and the online environment is much more flexible in this sense.Even with surveys, contact with participants, we never really know how we're reaching each one of the people there, which is also the case for any learning endeavor, isn't it?

[Michele Martin]

Hi Carla--I'm glad that the course worked for you. I've heard from several people that they liked the levels of participation, although on person wanted to know if I thought that we'd have had more people in the forums if we hadn't had the spectator level. I suspect that it would have been the same though. This way we basically legitimized something that people would have done anyway. I'll be curious to know what conclusions you reach about your own online sessions, Carla.