This blog post is a summary of the “Effective Learning” stream of the Learning Cafe UnConference which was led by Bob Spence.
This stream, which was well attended all day, was concerned with how we can make learning more effective and to this end five separate Sessions dealt with some of the many and various aspects of effectiveness.
Evaluation and Measurement – Bob Spence
It was agreed that evaluating the success of learning is something we know should be done but, in reality, it’s rarely completed well.
The Session participants noted that the most common form of “evaluation” was the completion of “smile sheets” by participants which typically provide some vague and thinly veiled opinion about the training session just attended. Even less effort seemed to be taken to conduct any worthwhile analysis of this data. This was not unexpected.
Two fundamental issues were discussed:
- The first concerned the problems associated with evaluating the effectiveness of learning in situations when the development of a learning intervention has been ”forced” upon the L&D function even though a different intervention (such as say job re-design or better communication) would have been more appropriate. The discussion wove its way around the L&D function having the fortitude to say “No” to such demands and sufficient know-how to offer clear alternatives.
- The second concerned better understanding a client (business unit for example) requirements. The discussion centred around asking the client ”How will you know the learning has been successful?” It was readily agreed that it is the answer to this question, hearing it and acting on it, that sets the ground rules and the activity around evaluation and measurement.
Not all participants at the Session were familiar with the Kirkpatrick Model (4 levels – Reaction; Learning; Application; Impact) of Evaluation, or the 5th level (ROI) discussed by Phillips and the 6th (Sustainability) and 7th (Sharing the Benefits) levels put by Rylatt. An explanation of these levels was covered and this lead to some more detailed discussion about using a basic measure of “performance” as a way to approach learning evaluation. It was agreed this could be achieved by evaluating at the 3rd level (Application). In other words, ensuring that learning obtained in say the classroom or from an e-learning course, is actually applied in the on-job situation. This probably means a trainee being “observed” by a manager to confirm learning has taken place. It was agreed that although the 7 Level Model mentioned above may not be always necessary or appropriate, it was important that L&D professionals understand the concepts and be able to discuss the pros and cons.
Bob argued that the accountability for learning effectiveness does not rest with the learning and development function solely. He based this argument on the fact that managers have a responsibility to ensure the people they look after are engaged with their work because without that in place, learning will not occur effectively. He challenged the learning and development function to face this full on by looking for opportuities associated with fostering engagement with a view to expanding the range of interventions it can provide managers.
Rather than evaluation being something that happens at the conclusion of a learning development cycle, the benefits of developing and growing an evaluation plan as the course development activity takes place/progresses was discussed vigorously until time ran out!
Designing Learning for Work – Bob Spence
Bob put forward an opinion that with the lines separating learning and work becoming blurred, together with increased focus on people performance, that learning designers of the future will need to work much closer with those designing work processes and work-flow. Further, that this multidisciplinary approach to design needs to focus on dynamic flexible learning and performance interventions integrated with work itself.
The use of the term “Instructional Design” was debated and it was agreed that the term depicts a strict, in-flexible, disciplined (one-way the only-way) approach to “instruction” whereas a term such as “Learning Design” depicts learning as a more inclusive and collaborative process. It was argued that learning design should, in many cases, be focussed on work performance rather than remembering facts, figures, process steps etc. and include rich case studies and simulations as preparation for on-job learning.
The discussion expanded to the need for many L&D professions to re-visit the basics of learning design: to work more thoroughly with learning objectives, test/assessment item choice and development and the employment of teaching strategies. There are many learning theories out there but without understanding the basics, it is very difficult to make judgements about the appropriateness of one over the other.
Blended Learning Design Approach – Jeevan Joshi
Jeevan led this Session and opened by questioning the general current approach to “blended learning”. He likened it to the mixing of water and oil; they mix but don’t blend. Jeevan asked the group to consider if there is a need for a different design paradigm which ensures integration of off-line and on-line teaching methods.
The group responded overwhelming with a “yes” – that there was too much of merely adding say e-learning or a webinar at the back-end of a traditional classroom event without considering the design, and consequently the degree of effectiveness involved. The discussion traverssed the use of differing media types and also the use of formal and informal learning activities in the “blend” and it was highlighted that a good blend could be a combination of all these things.
Jeevan’s next point concerned current instructional design approaches and he questioned if they catered sufficiently for blended learning development. The group responded that overall instructional design approaches were sound but it was really the individual designer’s skill that came to the fore when making decisions about media and formal/informal interventions. Some of the group argued that many decisions about media use is made for them (not necessarily through informed decision making) and in many cases it was a matter of using what they had. There was certainly a sentiment running through the discussion that designers need to be better prepared and better informed to make the right decisions about the design of blended learning.
An interesting aspect to the discussion was around cost and whether this was a driving factor in the way blended learning was designed and developed. It was readily agreed that the cost of the development and learning and the cost associated with the delivery of learning were important factors that the L&D team had to deal with. Some said that these costs were far more important to some management groups than the effectiveness of what was produced, but that some management group reserved the right to question effectiveness after implementation.
Again, it was interesting to note that this Session also highlighted the need for the acceptance and respect of the professionalism of the learning and development function to be elevated.
Neuroscience of Learning – Michael Parker
Michael Parker put forward an explanation about neuroscience for learning and led discussion about the topic which, not surprisingly, concluded that the jury is still out, and while evidence about the application of neuroscience for learning to determine learning effectiveness was still being gathered and debated, it was a field that learning and development professionals of the future will need to monitor. In keeping with many of the Sessions conducted at the Unconference, this led to an underlying issue flowing from Michael’s Session; the expanding scope for learning and development professionals, the sophistication of the future skill base necessary to better understand how we learn, the conditions in which we learn best, and how we prove it.
Here is a simple explanation of neuroscience for learning. . .
In the overall quest to ensure learning effectiveness, particularly in the fields of reading and mathematics, researchers of neuroscience have been investigating if the manner in which the human brain processes information, can be taken into account to improve the way learning is delivered, comprehended and utilised.
In the past learning in cognitive psychology and neuroscience has centred on how humans (and other species) have evolved to extract useful information from the natural and social worlds around them. By contrast education, and especially modern formal education, focuses on descriptions and explanations of the world that learners cannot be expected to learn by themselves. And so, learning in a scientific sense and learning in an educational sense can be seen as complimentary concepts.
This creates a new challenge for cognitive neuroscience to adapt to the real world practical requirements of educational learning. (Frith, C (2007). Making Up the Mind: How the Brain Creates Our Mental World. Oxford: Blackwell.) Conversely, neuroscience creates a new challenge for education, because it provides new characterisations of the current state of the learner – including brain state, genetic state, and hormonal state – that could be relevant to learning and teaching. By providing new measures of the effects of learning and teaching, including brain structure and activity, it is possible to discriminate different types of learning method and attainment. For example, neuroscience research can already distinguish learning by rote from learning through conceptual understanding in mathematics. (Ischebeck, A.; Zamarian, L; Siedentopf, C; Koppelstätter, F; Benke, T; Felber, S; Delazer, M (2006). “How specifically do we learn? Imaging the learning of multiplication and subtraction”. Neuroimage (Elsevier) 30 (4): 1365–1375.)
Although an increasing number of researchers are seeking to establish educational neuroscience as a productive field of research, debate still continues with regards to the potential for practical collaboration between the fields of neuroscience and education, and whether neuroscientific research really has anything to offer educators. Some researchers feel that linking neuroscience and education is not possible without a third field of research is introduced to facilitate the linking. Other researchers disagree.
From a different perspective, some researchers claim that neuroimaging could be used to assess the impact of particular training programmes and that some brain imaging research is beginning to show that for children with dyslexia who receive targeted educational interventions, their brain activation patterns begin to look more like those of people without reading disorders, and in addition, that other brain regions are acting as compensatory mechanisms.
Informal Learning – Andrea McDonough
Andrea explained that informal learning, such as learning from peers and colleagues, including on-job training, learning by trial and error, individual reading and research in order to be able to perform better, are all important aspects of on-going development. She referred to the Australian Bureau of Statistics which has found that informal learning is by far the most prevalent form of adult learning and cited further research that claims 70% of learning is learn through work, 20% through couching and mentoring and 10% through formal training.
Andrea’s first challenge to the group concerned what L&D can do to create an environment that encourages learners to acquire and accumulate knowledge, skills, attidudes and insights from daily experiences (Coombs and Ahmed, 1974). The conversation entered the area of having in place a supportive working environment; an environment that encourges learning and an environment that people enjoy working in. In essence, if people are not engaged in their job they are unlikely to learn effectively, or go out of their way to learn new things or explore new or different approaches to work. It was agreed the role of L&D was to encourage this environment, to lead by example and to champion the development of techniques and interventions that enhance engagement. Some refernce was made to Bob’s opening remarks about the Effectiveness Stream where he stated that learning effectiveness is not the sole repsonsibility of the L&D function; it’s a shared responsibility with management who have clear accountability to provide the best possible “conditions for learning” in form of pleasant, productive, supportive work places.
Interestingly, and again inkeeping with much of the discussion across the Streams and Sessions during the day, the question about whether L&D professionals being adequately equipped, well enough informed and competent to add value to informal learning was raised within the contaxt of technology and the “trendiness” of social media. In the time available it was not possible to have in-depth discussion but there was a clear message from the group that while some L&D people had the opportunity to utilise and try out new media/methods, some organisations were cautious. It was generally agreed that (generally) L&D professionals have some way to go to become proficient at linking formal/structured learning with informal/unstructured learning. It was suggested participants refer to the following link within the Learning Cafe for more information. http://www.learningcafe.com.au/2011/09/blended-learning-whats-in-a-name-2/
Lively discussion folded in to the perplexing issue about recording and measuring informal learning. Not surprisingly, no consensus was found for this one! But, the counter argument could be “why record it?” and indeed “why measure it?” The rationale could be that informal learning is going to happen anyway, wouldn’t it make more sence to encourage it? provide the best possible working environment to facilitate it? rather than worry about recording that someone has “done” it? Further, if we accept that learning and work are merging (have merged?) isn’t the best measure simply that a person can complete the tasks assigned to them – in other words that they can perform!
The last point here is an area ripe for much more discussion at the next Unconference.