Thursday evening my colleague Heilyn and I had the final session in the PBBL course and we had asked the Learning Teams to prepare evaluations of the course. We have actually designed several evaluation activities, mainly because reflection-on-action is an important component is our pedagogical philosophy, but also because the course is part of the EU funded research project COMmunity of integrated Blended Learning in Europe (COMBLE) and as such we need to document our findings. Friday Heilyn and I did our weekly meta-reflection on the course as a whole, and I think Heilyn was a bit surprised, since I was not willing to say that the course has been a complete success. Judging from the various types of feedback we have received from the participants, there is no doubt that the course has been a success. Both the participants and Heilyn and I as designers/facilitators/researchers have learned so much and all in all it has been a very engaging, rewarding and fun experience. As part of the evaluation process we asked the participants to look at their individual expectations they all stated in the application form before entering the course and comment on whether the course had meet their initial expectations or not. Here are some quotes from their responses:

It confirms what I maybe knew before: No one can be learning by a system alone. The systems are just supporters for all the good intentions and the heavy work done by the facilitators. And it is probably not the time alone, it is the timing too, and the whole spirit from the facilitators which inspires the learners to show up and participate every time.

Now I know: I was wrong in the assessment of my previous knowledge about BL – thank you very much for the new insights.

My expectations, on the other hand, were more than fulfilled. Also, I like fulfilling other people’s expectations. I did learn some theory which will be useful, and the collaboration in our virtual team was wholly enjoyable. I did get re-started nicely on teaching in second life and I am taking a bag full of ideas home with me. Especially on PBL and how to deepen my application of this theory, but also with respect to how much time this takes when you take it seriously as an educator.

The work in our team has pleased me very much. It was surprising to see that we as a small group already had so many different approaches and we had longer discussions than I thought before, but we have reached a very cooperative process of discussion of how we ultimately manage and present our mini project. Working in our team was really fun!

In short – all these expectations are met and are even exceeded. In a little longer – at work we used a bit “americanized” AL in some high potential training courses. That is basically all I knew about this kind of adult learning method and now I feel I have a good first insight what PBL means and what you can achieve with it. This will very much help me to influence what we do in the future with such programs at work. Also seeing what online learning opportunities beside just simple WBTs are out there was eye opening for me, however I am not so sure what I am going to do with that new knowledge in regards to me current work environment.

Also thank you very much for an interesting course – you have done a great job, and also shown us, how much time we have to be willing to invest, if we want to implement BL courses in our practice. A very important information for the institution before the boot. I wanted to get better assumptions to transform f2f courses into BL and get knowledge about PBL and technology to have a better foundation to choose the right method and technology for the target groups. I do think I got it all. It has been very interesting to learn about the methodology and explore the virtual environment. The team process has been excellent with lots of good discussions.

For me this course was more than I expected. I liked BL, when something was difficult in audio version, ppt gave the missing parts. I learned lot from Comble members in SL. Very useful was reading forums. The most boring for me was my computer, movements in SL were slow, arrows buttons were slow, there was kind of break or time between me to press the button and me to move in SL. Then I pressed the button again and again and once, when I started to move in SL I went and went … and got lost. This course gave me a new dimension of teaching. From flat Internet to 3 D in SL, from one tool lecturing to BL. Thank you all, it was fun.

I learned very much – but unfortunately also something I didn´t care to know about myself: I am not any longer resilient enough for such a giant course. I didn´t visit each optional meeting and I’ve only been able to read about a third of the requested literature efficiently. After some days full of daily life I couldn’t manage to stay fully concentrated trough the evening sessions. On the other hand it has been great to see the advantages of SL, to gain a lot of experiences and to develop my aims to handle PBL and AL. Thank you for this very intensive but great experience.

I really enjoyed learning in SL which was a complete novelty to me. I regret that I wasn’t able to turn up during all SL optional meetings as I really liked them. For someone like me, to whom getting around in SL (broadly speaking) was really completely new, the tours gave many hints how to let yourself being educated in the virtual world and maybe, in the future, how to organize the learning process there. Before the course I had just theoretical basis of PBL/AL (quite superficial, I believe). Now, I know much more, have a lot of materials and some experience to use.

When entering the course I was hoping to experience SL as e-learning platform on my own skin and it happened on the skin of my avatar ;) But seriously I really had a chance to see how education process may look like in SL. I think it is too early to state that now I will also use it in my courses but I do really have something to think about. If in the future I had opportunity to provide similar course I think I would be ready from instructional point of view but there is still a lot of things I should learn about PBL… In fact during that course I realized how complex and difficult it is and how hard it is to implement it into the teaching/learning process.

Like everybody else, I want to thank you for the fabulous work, you have been doing during this course. I will forever be inspired and have your doing in my mind working with BL. It is hard to find words to express my experience during this course. I am amazed, and I really feel sad, that it’s now over, even though it has been hard work  :-) I feel connected of course mostly to the danish team but also to the rest of the participants in a way, that I never thought possible after 6 weeks working together only online and never meeting in RL. I have really learned a lot too.

Now looking at these comments and the rest of the feedback we have received, it may come as a surprise why I’m somewhat reluctant to label the course a complete success, so I’ll try to elaborate a bit on that as a) course designer/facilitator and as b) researcher.

As course designer and facilitator my responsibility together with Heilyn has been to design for optimal conditions for learning to happen in order to meet the promised outcome. Even though taking responsibility for one’s own learning is very much part of the PBL pedagogy, the facilitator still has a huge responsibility to try to ensure that learning actually can happen (through the design), and when evaluating a course it thus makes sense to look at the expected outcome and review the final learning result. In this PBBL course the general course objective and the outcome was the following:

The main objective of the course is to provide the participants with a combination of conceptual, theoretical and practical strategies with regards to designing, implementing and teaching/training courses of different duration in blended modality using an overall PBL approach.

The participants will learn how to design blended learning courses/learning units based on a PBL approach.

There is absolutely no doubt that the participants learned a great deal about Blended Learning (BL) and PBL and that they got a lot of inspiration and new knowledge with regards to designing, implementing and teaching/training. But I would not feel comfortable in saying that they actually learned how to design, implement and teach/train – and how to are the operative words here. Heilyn and I discussed this and agreed that the course outcome actually should have been differently stated with an emphasis on learn about instead of learn how to, since we both agree that learning how to is way too ambitious for a short 6 week part time course, where the majority of the participants had little or no prior experience with BL, PBL and the 2 chosen course technologies, Moodle and Second Life (SL). We cannot propose to prolong the course, since our 3 European COMBLE partners find longer courses unrealistic to implement in their local settings. Consequently we will suggest changing the course outcome, when we do our final report, but even so I find it necessary to review the design and see if anything could or should have been done differently.

The most important course component aimed at learning how to design, implement and teach courses based on a PBL approach was the Mini-projects we asked each Learning Team to produce. As mentioned in a previous post only half of the 6 Learning Teams managed to hand-in their projects in due time, 2 teams handed in later and 1 team resigned. And in my book that is not satisfactory. Insufficient time to do the projects has been stated as the main explanation, and so we need to look at the general workload and consider carefully if the project description, incl. assessment criteria can be done differently. Another issue pointing at a revision is the quality of the projects. When looking at the quality of the 5 project reports, 3 of them were quite good content wise, whereas the 2 final reports did not in a convincing way show that the teams were able to identify characteristic and attributes of PBBL, discuss the advantages and challenges, align goals with local standards and design a course.  The course has been offered by Aalborg University at master level, and as such we have been assessing the projects according to our local standards for academic reports at this level.  The course has shown that there are cultural differences in what is to be considered a good academic report, and if we had more time in the COMBLE project, this would be something I would want to investigate further. For the final report we will recommend assessment criteria based on local standards and this would also influence the content of the projects.

Other ways of ensuring a higher quality would be to;

• Form the Learning Teams in the first week and get them started earlier
• Ask the Learning Teams to submit their problem formulations earlier
• Ask the Learning Teams to submit an example of their writing before handing in the final reports
• Have a synchronous supervision session with each Learning Team

Given that project work is a core element in the way we do PBL at Aalborg University I would hesitate to reduce the Mini-projects further, but would prefer to reduce some of the other course activities. We have had 2 lectures and 1 optional meeting pr. week, and in our final report we will suggest only 1 lecture and 1 optional meeting pr. week. In this course we also included Action Learning (AL), and even though there are similarities between PBL and AL, Heilyn and I agree that in a future course we would not include AL, since it seemed to create some confusion among the participants and bringing in AL in the projects did not improve them quality wise.

Heilyn and I have also discussed the use of especially SL. It is evident that the steep learning curve in SL has been very time consuming for the majority of the participants. Depending on course objectives one could chose a different environment for synchronous communication and probably gain some time, but it would be at the expense of some other important factors, which leads us to the research perspective.

As mentioned above this course has been part of a research project and Heilyn and I had the task of designing, implementing and evaluating a Blended Learning course based on innovative technology and Problem Based pedagogy. Besides this we decided to formulate an investigation question more targeted at our respective PhD interests and we came up with this question:

How can deliberate pedagogical design promote community creation and motivation among participants?

In order to investigate this question we decided to apply an action research approach meaning that we worked in (small) cycles of planning, acting, observing and reflecting throughout the course. We also decided to share our findings with the participants inviting them to comment on our meta-reflections, and this really turned out to be a great learning experience for all of us. Based on the feedback we got from the participants, we were also able to act proactively and in this way I think our research perspective supported our functions as designers/facilitators a lot. We planned many different activities targeted at promoting the community creation and motivation, and we both agree that the course has been a huge success in this sense.  Many of the activities we planned were carried out in SL, and I’m convinced that the immersive and engaging character of SL contributed greatly to the community creation and the motivation.  I was happily surprised to see so many participants show up for especially the lectures in-world, and I’m not convinced they all would have been as motivated if the synchronous lectures had been in a different platform.  This part of the project is something Heilyn and I will investigate further in 2 papers for the PAN PBL2010 Conference and the 7th Networked Learning Conference 2010. For now I’ll finish with a quote from one of the participants, who had no prior experience with SL:

In the beginning I though SL is only some funny tool/place and more gaming than really learning something seriously. However after getting over the voice issue I had in the beginning I really feel that his is somehow better than just telecons with sharing PowerPoint presentations. Somehow it’s more fun and more engaging to join in for sessions and being able to write within different groups in parallel to speaking is very interactive and useful. …. Someone that normally favors f2f against any sort of technology meeting.

Summing up the course has been a success, but there still is room for improvement, and this is actually what I appreciate most about being a course designer, facilitator and researcher – it is a lifelong learning experience :-)

/Mariis