Learning Metrics

This half-day event arranged jointly with the E.E.P. considered the impact of digital systems on learning activities. Experts from digital and learning fields looked at current lessons to be learned and the potential knowledge streams flowing from learning data. Read the full event report....

true

On 8th December 2004, NMK and EEP hosted a discussion on Learning Metrics, looking at how new technology might improve learning methods, and how we might try to measure its effect on learning. <a href="http://www.nmk.co.uk/me/add/">Register</a> and post your own comments on this article below... <a href="#Roger">Roger Broadie</a> of the <a href="http://www.eep-edu.org/">European Education Partnership</a> introduced the event. He said that when presented with new educational situations, we have to start from scratch, feel our way forward. Academia is unhelpful, as it focuses on long-term research programs. Education facilitators cannot wait that long. He mentioned usage of the site <a href="http://www.skoool.ie/" target="BLANK">skoool.ie</a> in 2003 - in the run-up to the national exams, 60% of all students accessed the site. The busiest hours were in the evening, and usage did not substantially decrease at the weekend. Clearly this gives a strong message that politicians can understand, but it must be crystallised and refined. <a href="#Graham">Graham Taylor</a> of <a href="http://www.sawtrycc.com/" target="NEW">Sawtrey Community College</a> and the <a href="http://www.e-learningfoundation.com/" target="NEW">e-Learning Foundation</a> discussed an e-learning scheme his college had run, where students who volunteered were given laptops with internet access. How does this affect the transactional quality of teaching, and does it have the potential to completely transform the nature of education? National research findings suggest that students with access to this technology collaborate more readily, and that their research and analytical skills, their problem-solving skills and higher-order critical thinking, are enhanced. Teachers themselves become more facilitative and less directive. But these conclusions were based on anecdotal evidence, and for his doctorate Martin put them to the test using empirical observation over a number of years. Results showed students with laptops spent less time on low-level traditional learning activities and more time on higher-level evaluation. Laptops increased motivation in the students who had them. Teachers spent more time as facilitators and enablers than as leaders and information sources. In conclusion, although Graham felt that technology could successfully change the transactional nature of teaching, he didn’t feel it would replace traditional schooling methods - he believes that any teacher who could be replaced by a computer deserves to be. <a href="#James">James Blomfield</a> of <a href="http://www.intuitivemedia.com/">Intuitive Media</a> talked about <a href="http://www.gridclub.com/" target="BLANK">GridClub</a>, a website devoted to providing learning to 7-11 year olds in an enjoyable way. A tension exists between formal and informal education in these environments - the government always wants evaluation. In this case Manchester Met provided it through tracking usage, sending out questionnaires etc. The site demonstrated six different types of learning: Learning to learn (GridClub is a positive environment where children can demonstrate resilience, reflection, resourcefulness). Developmental assets (relationships with adults, empowerment, positive peer interaction, willingness to take responsibility, positive self-identity). Communities of Practice (interaction free of adults, adult moderators as role models, a sense of belonging, learning from one another). Fun and play (children perceive GridClub as fun, and they, along with teachers and parents, make the explicit link between fun on the site and learning). Being and Becoming (children act with independence, are treated with respect). Flow (flow experiences are common for children using computers, and GridClub led to many educational flow experiences). This research has allowed them to make changes to the site, for instance boosting the collaborative nature of certain areas. <a href="#Jayne">Jayne Solomon</a> of <a href="http://www.samlearning.com/" target="BLANK">SAM</a> Learning talked about SAM learning, a web-based revision and exam practice tool. It extends and personalises learning, with 45% of usage taking place outside school hours. Students using the site have a username and password, so their behaviour can be monitored. They can use it at their own pace and involve parents or others. Students are responsible for their own work, so it treats them like adults. Learner profiles can be created, so students can see how they are progressing in a particular subject. Teachers can also access these profiles, can set various tasks for students and can create groups of students, for instance all those who need extra work in a certain area of the curriculum. Data extracted from SAM suggests that using SAM in the run-up to exams for only ten hours can produce significant improvements in GCSE results. Medium ability students are the most likely to benefit, and boys more so than girls. <a href="#Phil">Phil Hemmings</a> OF <a href="http://www.rm.com/" target="BLANK">RM Plc</a> believes that measurement is a good thing; it drives improvement. There’s a lot of debate about metrics in education - students are taking more and more tests. These tests aren’t good enough, ignoring processes and just examining results. But if the results are bad, we must look at the processes. In industry, quality control is put in place at every stage of the process - the same must be done in schools. Using ICT may well produce better results, but measuring this is not as helpful as measuring how it affects the entire process. He identified three key points: (1) Output measures are necessary but not sufficient. (2) Measures that allow change are needed. (3) It’s not necessary to completely understand the entire process in order to improve it. ICT gives out very helpful diagnostic data that is useful for teachers and can be embedded in an unobtrusive way, unlike traditional tests. <a href="#Chris">Chris Yapp</a> of <a href="http://www.microsoft.com/uk/" target="BLANK">Microsoft</a> discussed the ways ICT can change teaching methods. It can lead to evolutionary changes - localised exploitation, e.g., use of computers in revision. Or integration - if everyone in the school is online, it could save a lot of administrative time, freeing up teachers to do more teaching. It can also lead to revolutionary steps: process redesign (putting the learner at the heart of the system - for instance having video link ups between language classes in France and the UK). Or process change - he gave an example of a school in Mexico City where students researched different aspects of a subject and then the whole class collaborated in creating an essay. Or network redesign - PDAs showing specific, relevant content as a student wanders around a site if historical interest, for instance. Scope redefinition - if every school in the country had to teach Chinese, it would be cheaper and faster to use new technology to facilitate this, rather than relying on traditional methods, which would take decades to implement. <a href="#Peter">Peter O’Hagan</a> of <a href="http://www.serco.com/" target="BLANK">Serco</a> began with an old Chinese proverb, ‘You can’t make the pig fatter by weighing it everyday’, which mirrors the government’s current attitude to learning metrics. He feels that education has to move away from data-driven decision making and be more about individuals learners. If assessment could be described as the DNA of improvement, it’s essential that we don’t get the wrong DNA by measuring the wrong things. We must move forward sensibly, not keep going through the same processes. We must seek out and encourage experimentation, learn by doing, and value failure for what it teaches us. We must search for new metrics, new standards, recognise the value of new networks. Small, iterative refinements of the current system are damaging - we need bold, visionary steps. We must work out what we want to measure, and why. Should schools be more modular, like universities? We must develop a lateral strategy, not a hierarchical one, and ICT has a part to play in that. <a href="#Ben">Ben Gammon</a> of the <a href="http://www.sciencemuseum.org.uk/" target="BLANK">Science Museum</a> talked about metrics the Science Museum developed in order to show funders that visitors were getting genuine educational value from exhibits. The museum needed metrics that were useful and could give guidance on improving practice. They had to be valid (genuinely measuring learning) and practical (effective within the time and resources available). A huge challenge was to attempt to educate funders, who typically want to know numbers of visitors, rather than what those visitors actually learn. The problems faced were: a lack of an agreed model of learning among professionals, and a perceived negative connotation of learning and education. Learning is often assumed to be the acquisition of facts. But it also includes consolidation of what is already known and the chance to put that knowledge to use. It can be affective, challenging beliefs and attitudes, can develop observation and deduction skills, and improve social interaction. It was just as important for the museum to evaluate where things went wrong as when they worked. A combination of observation of visitors at exhibits and conversation with them after they had finished with an exhibit was used to show how well the exhibit did what it was supposed to do. <a href="#Mike">Mike Bostock</a> of <a href="http://www.new-media-learning.com/4M_index.htm" target="BLANK">4Matrix</a> said the traditional model of schooling is inefficient and expensive. We need a 21st century vision. Why, for instance, do people need to physically go to school for purely knowledge-based subjects? Instead of fine-tuning the existing system, we need to look at radical shake-ups of the whole way we approach schooling. How we build schools for the future needs a bolder approach - we mustn’t just build more ‘new old’ schools. The traditional method of measuring success is based on the outward signs. What’s going on in students’ minds is just as important. Improving the climate for learning is not enough in itself, we must improve students’ disposition towards learning. Students’ determination to learn must be improved, their learning preferences studied. Data speaks the language of business professionals - metrics systems must transform the figures so that schools can use them. All interested parties must work in lock-step, otherwise the government will blunder in and create a mess. <a href="#Martin">Martin Owen</a> of <a href="http://www.nestafuturelab.org/" target="BLANK">Nesta Futurelab</a> began by asking the audience a difficult geography question, which no one could answer. This illustrated the redundancy of old-style rote learning, as anyone with web access could have looked it up in a second. Technological improvements help us, they change for the better the ways we do things. Computer games can be used to address physics questions, and are more relevant to many students than more traditional methods. Teachers must adjust, and so must metrics. Martin feels that teacher judgement is probably still the best way of evaluating students in most situations. Peer assessment is also useful. He concluded by identifying what he believes are the three most important issues. (1) The need for a common language - what are we looking for? (2) The need to develop activities for learners that allow them to display their creativity. (3) The development of ways of gathering evidence of this (peer review, developing games, videoing themselves being creative etc.) Tools are needed that can manage the assessment of this extremely complex data without simplifying it. There is good data mining in other areas of life - it is urgently needed in education, too. -------- Roger Broadie concluded the session by stressing how great a challenge this was going to be. This is an area where something needs to be done, but no one has yet decided what it should be. It’s important that all interested parties keep talking to each other, keep the debate alive, and hopefully arrive at some kind of consensus before one is imposed from above by less sympathetic forces. About the Chair <a id="#Roger" name="#Roger">Roger Broadie</a> is Chief Executive of the <a href="http://www.eep-edu.org/" target="BLANK">European Education Partnership</a>. The EEP helps European, national, regional and city-level ICT-for-learning leaders find selected guidance and training, which will practically help educators to adopt ICT-based approaches that increase the effectiveness of education and learning. The EEP promotes to these people the companies and organisations providing practically useable guidance and training on innovative use of ICT-for-learning. About the speakers <a id="#James" name="#James">James Blomfield</a>, Director, Online Education, <a href="http://www.intuitivemedia.com/" target="BLANK">Intuitive Media</a>. James has been heavily involved in the development and operation of <a href="http://www.gridclub.com/" target="BLANK">GridClub</a>, which has been extensively evaluated by the Manchester Metropolitan University Centre for ICT, Pedagogies and Learning. James will highlight and discuss the wide range of learning activities that Gridclub stimulates, including games, forums and other activities, and learners' reactions to them. He will propose possible methods of gaining qualitative and quantitative measure of the learning processes involved and their impact. <a id="#Mike" name="#Mike">Mike Bostock</a>, Director of <a href="http://www.new-media-learning.com/4M_index.htm" target="BLANK">New Media Learning Ltd</a>. New Media Learning is an education consultancy specialising in the use of ICT in education. Mike was an LEA ICT Adviser for 15 years, is an Ofsted-accredited inspector, and is currently Chair of the Executive Committee of Naace. Mike is the author of several publications on ICT in education including ‘Implementing ICT’ (Naace). He contributed to ‘Learning in the 21st century: the vision and practice of e-learning in local government’ (Naace, Socitm, Becta) and to John Davitt’s book ‘New Tools for Learning’. Mike is the author of Logicator, the leading computer control software used in schools. New Media Learning has developed ‘4Matrix’, a performance analysis toolbox for schools. <a id="#Ben" name="#Ben">Ben Gammon</a>, Head of Learning and Audience Development, <a href="http://www.sciencemuseum.org.uk/" target="BLANK">Science Museum</a>. Ben leads a team of approximately 100 staff who develop and deliver educational programmes and resources for families, adults and schools, both in the museum and as part of community outreach initiatives. The team is also responsible for running the interactive galleries and for conducting audience research. <a id="#Peter" name="#Peter">Peter O'Hagan</a>, , Director of Research and Innovation, <a href="http://www.serco.com/" target="BLANK">Serco</a>. Peter is responsible for answering the question, where next for Serco Learning? He is looking beyond the current policy horizon at what education may look like and what Serco need to be doing as an organisation. <a id="#Martin" name="#Martin">Martin Owen</a>, Director of Learning, <a href="http://www.nestafuturelab.org/" target="BLANK">Nesta Futurelab</a>. Martin's role at Futurelab is to think strategically about the ways technology transforms learning. He helps to build new partnerships and bring new ideas into the organisation, and is involved in forming and evolving these ideas by taking them out to learners. <a id="#Phil" name="#Phil">Phil Hemmings</a>, Director of Corporate Affairs, <a href="http://www.rm.com/" target="BLANK">RM Plc</a>. RM plc is one of the UK's leading suppliers of software, services and systems to the UK eduaction sector. Founded in 1973, from the very beginning RM has concentrated on making industry standard technologies accessible and appropriate in an educational environment. Looking beyond the technology, in the 1990s the Group formed long-term partnerships with both educationalists and other learning technology companies. Phil is responsible for RM's relationships with education policy makers and other influential education community stakeholders. <a id="#Jayne" name="#Jayne">Jayne Solomon</a>, National Sales Manager, <a href="http://www.samlearning.com/" target="BLANK">SAM Learning</a>. Jayne is responsible for the SAM Learning sales team nationally, including - raising awareness of SAM Learning service and its impact at a government level; formation of user groups regionally; liaising with senior educationalists to keep abreast of current trends. Jayne's areas of interest are - extended & personalised learning whereby students can study at appropriate times and locations; Learner profiles which can be shared with teachers, learning mentors parents and used to show progess and set targets/goals; data: capturing and interpreting data from an individual user level to LEA level whereby the impact of initiatives can be measured. <a id="#Graham" name="#Graham">Graham Taylor</a> of the <a href="http://www.e-learningfoundation.com/" target="BLANK">e-Learning Foundation</a> is Vice-Principal of Sawtry Community College and Assistant Director, Innovation Unit/NPDT National Programme for raising Boys’ Achievement. E-learning is an embedded practice at <a href="http://www.sawtrycc.com/">Sawtrey Community College</a>, a Leading Edge Technology College in Cambridgeshire. The college was a pilot school for the Anytime Anywhere Learning Project, a Microsoft sponsored initiative exploiting the potential of one-to-one dedicated student access to connective laptop technology for the transformation of traditional pedagogy and the organisation of formal education. From 1999 to 2001 Graham conducted doctoral research on the impact of dedicated laptop use on the quality of learning and teaching in schools. Not an exploration of the application of e-metrics as such, this research nontetheless demonstrates that ICT impacts significantly students motivation, and on the quality of their learning and the development of a range of critical/analytical skills in the classroom. <a id="#Chris" name="#Chris">Chris Yapp</a>, Head of Public Sector Innovation, <a href="http://www.microsoft.com/uk/" target="BLANK">Microsoft UK</a>. A Google search on 'Chris Yapp' produces an astonishing 19,100 matches - and even cursory scan of the first several pages reveals these to be largely matches to the past 20 years of Chris's activity in promoting the use of ICT for learning, on the UK, European and International stage. A useful blog of this event can be found on David Jennings' weblog <a href="http://alchemi.co.uk/archives/ele/measuring_learn.html" target="BLANK">DJ Alchemi</a>. <a href="http://www.nmk.co.uk/me/add/">Register</a> and post your own comments on this article below...

2004-12-14T16:36:06+00:00

317 default.jpg 2004/12/14/learning-metrics

learning metrics

Learning Metrics

2007-11-19T09:18:45+00:00

true