Month: January 2010

Evaluation, Literacy and Transliteracy

Mariis2 Comments

As previously described I’ll be using my Connective Model for ICT-remediated Didactic Design for the general analysis of my PhD data. A first step in this process is working with the 9 basic elements in the model and in this post I’ll focus on some preliminary work I’ve done on the element of Evaluation as depicted in the model below.

Given that definitions shape the way we think about and practice particular phenomena the very act of defining something should not be done without prudence. I’m quite confident that working with the different elements in the model will refine the way I’ll end up describing them, so for now I settle for working definitions and as such I’ve found inspiration in UNESCO’s definition of Evaluation and after modifying it so that it fits better into my study field of Didactic Design it would read as follows:

Evaluation means arriving at a value judgment on the basis of measures (qualitative or quantitative) considered to be valid and reliable, which compare the actual results of a Didactic Design with its anticipated results.

The element of Evaluation is to some degree connected to all the other elements, but according to the working definition there is a particular strong connection to the element of Goals, since this is where we can derive the criteria for evaluating the results. When dealing with Didactic Design there will always be a least two major perspectives from which we can look upon certain elements, namely the teaching perspective and the learning perspective, and in what follows I’ll present some initial reflections on the particular part of evaluation that concerns the evaluation of learning outcome/results. To do so I want to dwell a bit on the concept of Literacy, which I consider to be vital when discussing the purpose and goals of especially formal education.

In its most narrow sense literacy refers to the ability to read and write, but used as a more general concept literacy refers to being knowledgeable or educated within a particular field. In an interesting UNESCO report on the plurality of literacy and the concept’s connection to the right to education as stated in article 26 of  The Universal Declaration of Human Rights of 1948, the evolving notion of literacy is discussed and defined: “Literacy is the ability to identify, understand, interpret, create, communicate and compute, using printed and written materials associated with varying contexts. Literacy involves a continuum of learning in enabling individuals to achieve their goals, to develop their knowledge and potential, and to participate fully in their community and wider society.” (UNESCO.2004:13). Above all the plurality of literacy refers to the many fields in which literacy can be employed – specific literacies denominated by prefixes such as information, computer or media to name a few.

In my research on literacy I recently came across a very interesting article in First Monday that explores the concept of Transliteracy, which the authors Thomas; Joseph; Laccetti; Manson; Mills; Perril & Pullinger. 2007 define like this:

Transliteracy is the ability to read, write and interact across a range of platforms, tools and media from signing and orality through handwriting, print, TV, radio and film, to digital social networks. (Thomas et al. 2007)

What interests me the most is actually not the definition, but rather the idea and purpose of developing some sort of meta-literacy concept. According to the authors Transliteracy can be characterized as:

  • a possible unifying perspective on what it means to be literate in the twenty-first century
  • an extension of transliteration that also includes the increasingly wide range of communication platforms and tools at our disposal
  • a concept that calls for a change of perspective away from the battles over print versus digital, and a move instead towards a unifying ecology not just of media, but of all literacies relevant to reading, writing, interaction and culture, both past and present
  • a concept that doesn’t replace but contains both media and digital literacy
  • a possible literacy for (media) convergence
  • a concept of not just computer–based materials, but about all communication types across time and culture
  • a concept that insists on a lateral approach to history, context and culture, an interest in lived experience and a focus on interpretation via practice and production
  • an inclusive concept which bridges and connects past, present and, hopefully, future modalities
  • a concept that pays attention to the whole range of modes and to the synergies between them to produce a sense of a ‘transliterate lifeworld’ in constant process
  • both a concept and a practice productively situated in a liminal space between being a new cognitive tool and the recovery of an old one
  • a concept that deliberately refuses to presuppose any kind of offline/online divide
  • the kind of literacy we require to be able to simultaneously attend to multiple media and modes of communication as well as the kind of literacy we use to apply the literacies of one mode or medium to another one

Based on these characteristics I would interpret Transliteracy as a meta-literacy and I do find the characteristics both relevant and much needed in trying to define some sort of unifying literacy. The authors describe their work with the concept as a work in progress and “a good example of open source thinking between diverse collaborators” and they encourage further discussion and development of the concept. First author, Sue Thomas and her co-writers are all involved in the Production and Research in Transliteracy (PART) Group at the Institute of Creative Technologies (IOCT) at De Montfort University, UK and they’ll be hosting a conference on Transliteracy on February 9th 2010, which can be followed via several social media. In the video lecture below Sue Thomas explains the concept of Transliteracy based on the above mentioned article:

Returning to my interest in Literacy and especially new forms of literacy, I believe that there is a strong need to consider and develop new ways/methods of evaluation of learning outcome – not least when the learning processes and products have been facilitated by ICT-remediated Didactic Design. In this process of developing new evaluation methods, I think concepts like Transliteracy can prove quite valuable in giving indications of what criteria to focus on. Especially in Academia we seem to be stuck in using evaluation criteria and methods based on traditional literacy giving primacy to old media and modalities. Quoting Yancey. 2004: 90 “we use the frameworks and processes of one medium to assign value and to interpret work in a different medium”, which obviously is not the most appropriate way of accommodating the use of multiple and/or new media. In my third research cycle in the MIL case, I experimented with both criteria and methods of evaluation, and the results from this experiment will form the basis of a forthcoming post on evaluation of new media productions/compositions…

/Mariis

Connective models for Didactic Design

Mariis13 Comments

As previously described my PhD-project is aimed at improving Blended Learning within Higher and Further Education through remediation and redidactization. Through a process of designing and redesigning two specific Blended Learning courses within 6 research cycles the aspiration is to enhance learner experience and learning outcome by using new augmented/immersive 3D media and a learner centered Problem Based pedagogical approach. In both cases the target group is adult teachers/ trainers from the educational and the private/industrial sector from different countries. Having teachers/trainers as target group has made it quite natural to situate my work within the field of Didactics.

Especially in Northern Europe Didactics refers to a field of research and practice concerned with reflections and actions related to teaching and learning. Historically the field has been teacher-, goal- and/or content-centered, but since the mid 1970’ies we have – at least in Scandinavia – seen an almost paradigmatic shift to a more learning and learner-centered approach.  In Denmark this shift was above all initiated by the establishment of two new universities, in 1972 Roskilde and in 1974 Aalborg (where I work) that were founded in clear opposition to the “Old(fashioned)” universities by using an overall pedagogical approach based on Problem Based Learning and Project Organization in an attempt to amplify student motivation, engagement and learning with higher relevance for the surrounding society.

Within teacher/trainer education Didactic Analysis, as a means to learn how to plan, act, observe and reflect on didactic practices, has been a core component of the curriculum, and especially one model for didactic analysis has gained widespread use, namely the so called “Didactical Relationship Model” by Norwegian education researchers, Hilde Hiim and Else Hippe. Building on the work of fellow countrymen, Bjørndal and Lieberg (whose original model was more teacher-centered), Hiim and Hippe developed the model to show some important relations between different elements in Didactics using a learning theoretical approach. An English description of the model and the use of it in developing an online tutorial for Information Literacy can be found here.

In my PhD-project I currently have data from 4 completed research cycles and I’ve decided to use modified versions of the Hiim & Hippe model as part of my analytical strategy, which will consist of several phases progressing from a general to a more specific focus on didactic elements I find relevant in my particular case. Throughout the different phases I will be using different models, but as I find Hiim & Hippe’s model useful in depicting important relations and elements for general analysis, I’ll start by presenting this model briefly.

As mentioned above the model shows 6 important elements in a teaching/learning situation, these elements are interrelated and so influence each other in various ways and to various degrees. Even though I find the concept of depicting interrelated elements valuable, I don’t agree on the chosen elements, the description/content of the elements and the semantics in general. In my dissertation I will of course elaborate on this, but for now I will turn to my own revised models.

At the Master Programme in ICT and Learning (MIL), where I conduct most of my teaching and research, my colleagues, Bo Fibiger (1945-2008) and Birgitte Holm Sørensen originally conceived the concept of Didactic Design and combined with the use of Information and Communication Technology (ICT), Sørensen today defines it as: “The process by which the purpose, the goals and the content is determined, and where the planning, the organization and the arena for teaching and learning is shaped based on theories and in relation to ICT-based practice in a context.” I agree on the essence of the definition, but I also see Didactic Design as a result/product and sometime down the line I will also work on revising this definition. For now, the important point is that I consider my work to be part of the emerging field of Didactics combined with ICT and as a consequence my revised model is aimed at Didactic Design as depicted below:

In line with Hiim & Hippe’s model, my model also portrays important didactic elements, but I have chosen to add a few more elements, substitute one and rename some of them. I also prefer to speak of connections instead of relations, while the latter to me implies some sort of personalized bond that I don’t see between all the elements that are interconnected. I suspect that the major reason as to why Hiim & Hippe’s model has gained such popularity has to do with the fact that the elements are quite generic and thus enable the user of the model to define sub-elements depending on own needs and purpose.  One could argue that the elements I’ve added already are part of Hiim & Hippe’s model as sub-elements, but by highlighting them I believe greater emphasis can be obtained. While I do consider the elements in my model to be generic too and that my work with the model will refine the content/sub-elements, I do have some preliminary reflections.

  • ICT – in Hiim & Hippe’s model part of the setting. In my point of view the use of ICT has the potential of changing the Didactic Design quite dramatically and should as such not just be a sub-element. Furthermore the use of ICT has been written into the curricular of most educational practices from pre-schools to HE in Denmark.
  • Teacher(s) & Learner(s) – in Hiim & Hippe’s model people are absent at first glance. The Teacher is considered to be part of the setting and I guess that since the model is aimed at describing learning conditions and learning processes the student is somehow inherent. Based on my own teaching experience I, however consider the people involved in the Didactic Design to be the most influential element. This does not mean that I don’t consider the conditions (e.g. prerequisites) for teaching and learning to be important. On the contrary, but I think there is an acute need to focus on teachers’ conditions separately – especially when we combine Didactics with ICT.
  • Goals – in formal education goals are dominated by curriculum, but depending on theoretical foundation they can be formulated and attained more or less teacher-driven. One of the major advantages of a Problem Based approach is exactly the possibility of sharing the responsibility for this element between all participants in the Didactic Design.
  • Content – another element typically determined by curriculum and goals, but again within a Problem Based and especially Project Oriented approach this element can be based on collective decisions.
  • Contexts – teaching and learning doesn’t happen in a vacuum. Didactic Design is always situated, but not restricted to physical buildings or formal settings.
  • Activities – another very important element that shapes the teaching and learning processes possible and therefore also potential outcome.
  • Evaluation – Hiim & Hippe use the word assessment, which I think mainly relates to the learners and I do believe that a critical review on the teacher(s) and the teaching also is an important part of sustaining quality and I think that evaluation better covers this.
  • Time – is a crucial element, but is often missing in models and theories of teaching and learning despite the fact that there seems to be consensus on the fact that learning at a certain level actually takes a lot of time. Besides the time available for learners another sub-element could be time available for the teacher(s). In my experience many teachers/trainers find especially ICT-integration difficult and frustrating precisely because they don’t feel that they have sufficient time to learn to master the technologies and subsequent practices.

Besides revising the number and to some extend the content of the elements, I’ve also chosen to place the elements within a frame illustrating the point that Didactic Designs within formal education generally function as quite closed systems with very little permeability. Usually the influence from external factors (e.g. political, economical and other societal factors) is much greater than the other way.

In my PhD-project I’ve been working with 2 different cases. The MIL course (3 research cycles) and the COMBLE course (1 research cycle). In the MIL course the majority of the activities have been online, whereas the COMBLE course was 100% online, and I would describe the Didactic Design in both cases as having been ICT-remediated. In Bolter & Grusin’s original concept remediation refers to the process whereby new media refashions older media, but when we start to rely more and more on ICT/new media in our practices, I would argue that not only the media are refashioned, but there is a potential and in many cases a need to also reconsider and most likely revise the other elements in the model. These considerations have led to the next model:

It may come as a surprise that the model doesn’t appear that different, but that’s actually an important point of mine. ICT-remediation constitutes a potential for change, but it doesn’t happen automatically, and changes will depend on the various types of ICT. Walled Garden technology – like conventional LMS’/VLEs – is never pedagogical neutral. Different types of technology have different kinds of affordances and the user’s possibility to change or modify intrinsic ways of communication and content creation is usually very limited. As long as the majority of formal educational institutions choose to rely on conventional technology for remediating their practices, I personally see little prospect of real change. There are, nonetheless, some positive aspects in all of this. Regardless of the rest of the elements in the model ICT-remediation – especially based on Web 2.0 – will force the system to open up and connect more with the outside world and as both learners and teachers become more ICT literate as a consequence of ICT permeating our daily practices, I do expect changes to occur.

At the MIL education ICT is part of the curriculum and even though we also could do with more change, we do try to keep an eye on new media and their teaching and learning potentials. This was also the reason why my PhD-project became concerned with new augmented/immersive media in the shape of the 3D virtual world, Second Life (SL). Based on my experience with remediating existing practice into SL, this kind of medium clearly has the potential of changing the Didactic Design. Without having gone thoroughly through my data, I do see some changes regarding especially teacher(s), learner(s), contexts and activities. These four elements will be foci points in my analysis of SL and are highlighted in the model below:

It is quite deliberate that I’ve maintained the ICT element in this version of the model, because the use of SL doesn’t diminish the need to consider ICT in general. Several kinds of 2D technologies are at play in-world, and as I still consider SL to be an emerging, and sometimes very unstable technology, I wouldn’t at this point in time recommend using SL as a stand-alone technology.

These models all focus on traditional didactic elements and I will use them (most likely in revised versions) for my general Didactic Analysis. The last version has a clear connection to another model I’ve developed, which focuses on People (teachers/learners), Places (contexts) and Practices (activities). Based on that PPP-model I’ll be able to focus on topics that are less common in Didactics and in this way I think the models will complement each other profitably.

/Mariis

Case MIL: preliminary thoughts based on 3 research cycles

MariisLeave a comment

Background
The primary case in my PhD project is a course in the Masterprogramme in ICT and Learning (MIL), which I have redesigned and run three times so far. Though the course has been quite different from research cycle to research cycle, there are still some common traits.

  • The course is the first part of a module on ICT and Didactic Design and according to the curriculum the course should focus on theory and analysis, whereas the second course in the module should focus on concept and implementation. Though separate, the two courses should be regarded as connected, in the sense that the learning outcome of the first course should be more or less applied in the second course. In the first course the students usually are provided with 2-3 optional virtual teaching and learning environments between which they are asked to choose one as analytical object. Throughout the MIL programme the students are introduced to different virtual teaching and learning environments covering a wide range of mainly conventional 2D asynchronous and synchronous examples. Therefore the environments chosen for this course always represent the more unconventional trends, since it is our experience that these often provide more rich and radical settings, which can stimulate reflections.
  • For the last three years the students have been able to choose between the 3D virtual game, Global Conflicts or the 3D virtual world, Second Life as their analytical object.
  • Regardless of choice, the students are expected to discuss and analyze the teaching and learning environment on the basis of the 5 following mandatory topics; Didactics and target groups, Orientation and navigation, Interaction, Learning processes and Audio-visuals.
  • The goal of the course is to teach the students to analyze and reflect based on theoretical foundations – not to teach them about the analytical objects per se.
  • Officially the course runs for 6 weeks from November 1st – until approx.  December 15th.  However, the first two weeks are regarded as preparation period, the students finish the previous module in this period, so they are mainly expected to prepare by reading and the period ends with a 2 ½ day f2f seminar where we introduce the upcoming module.

Research cycles – short overview
Regardless of the fact that I ran the first in-world course in the fall 2007 before I started on my PhD project (January ’08), that pilot course provided valuable data and I consider it to be the first research cycle.


The main research purpose of this first research cycle was to explore SL as medium and quite deliberately I chose not to change the existing course design, which meant that by and large the students were expected to explore the environment by themselves with very little teacher facilitation. Even though MIL students are used to self paced learning and in general can be regarded as being quite tech-savvy SL turned out to be too complex and too unfamiliar for this strategy and interventions proved necessary. I arranged 5 optional activities where we explored different educational designs, met some of my in-world colleagues and engaged in didactic discussions. Since these activities were optional only about half of the students participated, but it resulted in a strong community feeling among those who did.

Based on the findings from this research cycle and in particular the ongoing feedback from the students I decided to focus on different pedagogical in-world activities  and a different overall organization in the second research cycle.


In the first research cycle the students chose to work in their regular study groups meaning that these groups would continue in the second course of the module and the design of the asynchronous environment supported this organization. In the second research cycle the students chose more based on their individual interests and since there were significantly less students, I encouraged them to act like one large group or community and changed the design of the asynchronous environment accordingly (for an elaboration on the asynchronous environment have a look at this post).

The emphasis on different activities proved valuable and especially the preliminary meetings focusing on learning basic SL skills, the Didactic Design Discussions and the students’ own tours seemed worth preserving.  Even though the students did a great job in the first research cycle, the quality of the student analysis and reflections in the second research cycle clearly showed a better understanding of the medium and the relation between theory and practice became more nuanced and critical. The activity of the students in the second cycle rose to an unprecedented level (also compared to other MIL courses) and in general the changes in the didactic design seemed to be successful when judging their learning outcome.  Still, the downside was that we were all exhausted afterwards and the activity level just did not seem realistic to keep up in a forthcoming course. Once again the students provided valuable feedback on the course and they pointed to one very important issue that in their opinion needed to be changed; the assessment method.

In these two research cycles I had maintained the assessment method (write a min. of 3 posts in the asynchronous environment) as stated in the curriculum – above all because I believe asynchronous reflection and writing in general to be one of the greatest learning activities, but also because I didn’t want the choice between the two analytical objects to be based on a difference in this. Furthermore the students at MIL are accustomed to working asynchronously for the main part as it is considered to provide the most flexibility, which naturally is very important when you deal with adults in further education. I asked the students in the first cycle if they would have been willing to do some sort of in-world activity instead of writing posts, but they were reluctant. In hindsight, I think the reluctance was closely connected to the little facilitation and the task of doing something in-world probably seemed overwhelming, whereas the students in the second cycle attained a more profound knowledge of the medium itself. As a result of these considerations, I decided to experiment with the assessment method and some of the other pedagogical activities in the third research cycle.

In the second research cycle all scheduled activities were optional except one (of their own choice). In the third cycle I chose to make two in-world activities (one “Newbie Night” and one Didactic Design Discussion – again of their own choice) mandatory. In between the 2nd and the 3rd research cycle I ran another in-world course (in another setting than MIL) and based on that experience, I decided that learning the basic skills should be mandatory and teacher facilitated. In the second cycle the Didactic Design Discussions proved valuable not least in showing how teaching and learning actually can happen in-world, and I wanted to make sure that all students experienced what it was like to participate in activities like this. And then I also decided to give the students the opportunity to do a synchronous in-world presentation of their analysis and reflections on SL as teaching and learning environment instead of doing it asynchronously.

In my experience there are always a few students who struggle very hard with SL (for various reasons – but some due to technical difficulties alone), and I didn’t want to inflict extra pressure on these students, so I chose to make the assessment method optional.  7 out of 8 students chose to do their presentation in-world. Unfortunately one of the students gave up on doing her presentation in-world because her set-up was removed shortly before she was scheduled to present and she ended up doing it asynchronously, but besides this the in-world presentation idea seemed successful.  I’ve  covered these presentations in the following four posts (part 1, part 2, part 3 and part 4), and the learning outcome was definitely high both theoretically and practically. Obviously the experience of being in charge of an in-world session (all presentations included learning activities) gave the students some important knowledge on the teaching and learning potential of the medium.

Looking back all three courses have been successful in the sense that the students undoubtedly have learned a lot, but there is an unsolved discrepancy between the huge amount of time both the students and I have been spending and the flexibility that is needed in an educational setting like this. MIL students are used to a workload of 15-20 hrs. pr. week, but in general they control and manage this time on their own, and this changes noticeably when you choose to remediate the majority of the activities into a complex, synchronous medium like SL. And so I’m wondering; is it worth it? Does it make sense to use SL given these particular circumstances (4 effective weeks, a curriculum other than the medium itself, full time employed students and recurring technical challenges)? I honestly can’t say for the time being.

At MIL we have recently rewritten the curriculum and we are currently in the process of implementing the new structure and the new courses. This means that if I choose to run a SL course next fall the conditions will be better (more weeks, more curricular focus on SL), but I will remain concerned with this dilemma of asynchronous vs.  synchronous activities in flexible, distance education …

/Mariis

2009 in retrospect

MariisLeave a comment

2009 was undoubtedly the worst year of my life both privately and professionally. The illness and death of my mother clouded the entire year, but as I’m not used to life being easy it is not a complaint, but rather an explanation as to why I didn’t reach all the goals I had set for my work.  Looking back on some of the posts I wrote early in 2009, I had hoped to enter a phase where I would be able to dig deeper, analyze and reflect more critically on my findings and start working on my strategy for remediating people, places and practices, which essentially is the goal of my PhD. And so the question is, what did I achieve professionally in 2009?

  • I managed to write 56 posts – on average they consist of more than 500 words, so even though it was less than the previous year (72 posts), I think it is ok
  • I managed to write a few posts, where I compared some of the findings from my research cycles, and even though I clearly need to dig deeper, it is satisfactory that I started this part of the work
  • I managed to start the work on a model for remediation that I will use as part of the strategy, and even though this also needs much more work, the important point is that I got off to a good start on this
  • In January I participated in a PhD Master class on Knowledge Media where I had the opportunity to meet one of my professional heroes, Terry Anderson. This later lead to an invitation to collaborate with both Anderson, Ross McKerlich and Brad Eastman, and this collaboration is something I’m really looking forward to continuing in 2010
  • In June I was invited to participate in a research group on Dialogic Education and Research. Unfortunately, I only managed to find the time to participate in one meeting with this group, but it did influence my thinking very much, so the work with this group has not been futile
  • Together with co-facilitators, Heilyn and Roland, I ran two very successful 6-week courses in-world in the fall and they both provided valuable data for my work – this is the work I’m most satisfied with … and proud of
  • Together with co-editors, Ulla, Søren and Lone I managed to finish the work and publish an anthology on ICT and Learning. The anthology has been well received and this is also something I’m quite satisfied with
  • I managed to write a Danish article on my first research cycle (Fall 2007), which was set to be published in 2009 in an anthology on Knowledge Media. The publication has been postponed, but it was nevertheless satisfying to wrap-up that particular research cycle
  • Once again I participated in the SLCC, which I consider to be highly relevant and important for my work as a means to strengthen my invaluable network. In SF I also had the opportunity of meeting Professor Tringham and later I received a scholarship to visit her at UC Berkeley spring 2010. I’m convinced that my work with Tringham will strengthen my PhD, so of course this is also something I’m very satisfied about
  • Besides these achievements that are directly related to my PhD, I also managed together with especially educational coordinator of the Master in ICT and Learning (MIL), Winnie to rewrite the curriculum and write an accreditation report of the education
  • Again together with Winnie, I managed to get a rather huge grant for the MIL education that will enable MIL to further develop and research its use of especially video-tools
  • In June I took over the responsibility of managing the local part of a research project I have been participating in since 2008 together with several colleagues from e-Learning Lab. Even though we have managed to deliver what was expected of us, I personally found this work dissatisfying – mainly because I really do not care for management tasks
  • In June I also took over the responsibility of daily management of the MIL education in close collaboration with Winnie. Even though MIL plays an important role in my work and I care deeply for the education, I don’t think daily management is compatible with being a PhD Candidate, and one of my best decisions in 2009 was if fact to discharge myself from this responsibility in the future. Starting this month the daily management will be conducted by Winnie, my PhD colleague Ulla and the rest of the steering committee – and I will return to being a full time PhD Candidate for the remaining of my project period.

In retrospect, I think I did ok in 2009 given the circumstances, but there’s plenty of room for improvement.  I had hoped to find the time and energy to immerse myself into theoretical and methodological aspects of my PhD, and I didn’t. I did, however manage to start on some of this work, and so I enter this New Year with an optimistic sense of being on the right track…

/Mariis