Translate

Monday, December 29, 2014

Evaluation & Innovation: Dirk's View

The last post for 2014 belongs to Dirk Ifenthaler. Dirk and I ran into each other in Jacksonville, FL at AECT. In the midst of a hectic schedule, he took the time to share his thoughts. Although he would have liked to provide more insights, he says he does have many arguments ready for us, as necessary. Ready to hear from Dirk?
Dr. Dirk Ifenthaler

Dirk Ifenthaler’s previous roles include Professor and Director, Centre for Research in Digital Learning at Deakin University, Australia, Manager of Applied Research and Learning Analytics at Open Universities Australia, Professor for Education and Interim Department Chair at the University of Mannheim, Germany.

He was a 2012 Fulbright Scholar-in-Residence at the Jeannine Rainbolt College of Education, at the University of Oklahoma, USA. Dirk’s research outcomes include numerous co-authored books, book series, book chapters, journal articles, and international conference papers as well as successful grant funding in Australia, Germany, and USA . He is the Editor-in-Chief of the Springer journal Technology, Knowledge and Learning.
Website: http://www.ifenthaler.info
Technology, Knowledge and Learning: http://www.springer.com/10758/

How do we recognize innovation in teaching and learning with technology?
The major lesson to be learned from fifty years of research in the area of teaching and learning with technology is that what happens in learning environments is quite complex and multi-faceted. Students cannot learn from technology, rather, technology is a vehicle to support the processes of learning.
Due to the often very slow publication process, reports and results of innovative projects are published years later which does not facilitate the often discussed link between research and practice. The recent activities of researchers in social media help to disseminate some aspects of innovation, however, important results are often held back for publications which again slows down the innovation process.

What challenges might practitioners and researchers face when evaluating innovation? Do you have ideas for how to overcome these challenges?
There is a clear lack of empirical evidence focusing on innovation in teaching and learning with technology. We need to conduct multidisciplinary and multimethods research within innovative learning environments involving researchers, education providers, and international institutions. Clearly, practitioners and researchers need to talk to each other. One place is AECT (www.aect.org) which brings together these two groups at their annual International Convention.

Can you point to some promising innovations in teaching and learning?
The current debate focusing on learning analytics is very promising. However, most often learning analytics is misunderstood as being a tool for reporting educational data. The real innovation of learning analytics is the real-time support and feedback for all stakeholders involved in teaching and learning with technology. This will result in facilitation of learning processes, optimization of teaching practices and materials, as well as improved curricular designs and administrative processes.

Are there some effective research initiatives or studies our readers should examine? If not, why do you think that is the case?
Check out Technology, Knowledge and Learning (www.springer.com/10758). There, I am supporting Work-in-Progress Studies and Emerging Technology Reports. These two publication types shall help to disseminate results from innovative projects already in their early phase.

Previous posts in the series:

Wednesday, November 19, 2014

Evaluation & Innovation: Anissa's View


Thanks to an ITForum nomination, I introduce to some and present to others this month's post by 
Dr. Anissa Likey-Vega. As faculty at Kennesaw State University, Anissa serves as the coordinator for the Online Teaching Endorsement and Certificate programs for the Department of Instructional Technology.  She is also founder and lead consultant at Curriculum Research and Development (CR&D), an educational technology program evaluation firm that specializes in serving the K12 sector. She has her Ph.D. in Instructional Design and Technology from Georgia State University. Curriculum Research & Development: http://www.curriculumrd.com/
Department of Instructional Technology at Kennesaw State University http://bagwell.kennesaw.edu/departments/itec

How do we recognize innovation in teaching and learning with technology?
I’m sure you’ve heard this before: If you look for a problem, you will find one; however, near that problem you will also likely find innovation. Whether K12, higher education, or industry training, the teaching and learning process is in itself an ill-structured problem complicated by unique contexts, limited resources, and (those pesky) humans. People will be innovating to solve the problems that serve as the biggest barriers to them in their context. This doesn’t mean innovation will always work or will always be adopted to saturation, but each innovation will stir debate and nay-sayers. This tension occurs because once an innovation has proven itself effective enough to be widely adopted in the environment, it is no longer innovative. It becomes the new status quo.

What challenges might practitioners and researchers face when evaluating innovation? Do you have ideas for how to overcome these challenges?
Each evaluation is unique. The combination of context, time, problem, and innovation is unique every single time. Yes, there are general principles and models that can guide an evaluation plan; however, flexibility is critically important to the process and knowing when it is appropriate to change course can be challenging. Evaluation expertise is not always valued as much as it should be and this can result in evaluation outcomes that offer little direction to organizational managers. Practitioners should read up on evaluation practices in their fields and knowledgably look for evaluation expertise within their organization. If such expertise is not readily available, then it is wise to consider outsourcing the process. While it is not always possible, the best evaluations are pursued early in the innovation process, or maybe even before. This allows for the data to drive the change process and optimize the outcomes.

Can you point to some promising innovations in teaching and learning?
I’m probably a bit biased on this one since I coordinate the online teaching certificate at Kennesaw State, but I think blended and online learning practices and tools have the most potential to disrupt teaching and learning in a positive and meaningful way.  Of course, if you teach at a fully-online school, these tools and practices are now the status quo; however, many settings particularly in the K12 sector are not there, yet. Online instruction grants the learner more power in the differentiation of instruction and it puts a teacher’s instructional design on display for self-reflection. This self-review is not as easily achieved in a real-time unrecorded class session. Also, their best designs are more easily replicated when stored in a digital format rather than gone with the passing of time. This allows for teachers to put their best foot forward even on an off day.

Are there some effective research initiatives or studies our readers should examine? If not, why do you think that is the case?
Well, this is a bit challenging given that evaluation is context and problem-type specific.  This means it would be difficult to curate a research journal on the topic of innovation evaluation alone.  The audience would likely be scattered in context and problem-type, resulting in few articles to warrant devoted readership for each individual. I read about evaluation in journals that cross all disciplines. In fact, when I wrote my dissertation there was no published method to evaluate one or more intended curricula. I had to develop that process on my own. That said, my evaluation-related books that are resting closest to me right now are also the ones that show the most wear. These are the works of Thomas Guskey and Joellen Killion. I have to modify their designs as expected for each situation, but they both promote excellent planning and structure in the evaluation process.  Each time I start to wrap my brain around a new program or innovation that I need to evaluate, their works serve as my evaluation muses.

Previous posts in the series:

Monday, October 20, 2014

Evaluation & Innovation: Bryan's Perspective

Dr. Bryan Alexander

I met Bryan in Atlanta at a NITLE symposium in April of 2013, but as those who know Bryan understand, it seems like I've known him forever.  I am a regular reader of his posts and enjoy his sense of humor. He was the first to volunteer to help with this blog. Although I started this series with my Lehigh students and AECT in mind, I'd like to hear from my fellow evaluation instructors and professors about how this series helps you with your students. 
Bryan Alexander is a futurist, researcher, writer, speaker, consultant, and teacher, working in the field of how technology transforms education. He completed his English language and literature PhD at the University of Michigan in 1997, with a dissertation on doppelgangers in Romantic-era fiction and poetry. Learn more about this NITLE fellow here http://bryanalexander.org/bio/


How do we recognize innovation in teaching and learning with technology? 
Not well, and that's a problem.  Teaching in general is something difficult to track for a variety of reasons, including the way course management systems lock away classes from observers..  For now we hear about innovative teaching through publications (statistically very rare), social media (a bit better), and personal contact.  Social media may be the best route for discovery innovation, as its ease of use lets practitioners share thoughts, reactions, observations on the fly.
What challenges might practitioners and researchers face when evaluating innovation? Do you have ideas for how to overcome these challenges? 
There are many challenges, depending on the background of would-be evaluators.  Professional training can make it difficult to appreciate innovation, either when trying to perceive new work in someone else's field, or in seeking to understand changes to one's own.  Observers can also focus too much on the first iteration of a project, rather than looking to its development and maturation over time.  It is also important to distinguish between achievement and learning versus other student responses (i.e., approval of change).
Overcoming these challenges is implicit in each one - pay more attention to innovation over time, pay less attention to non-learning responses.  Additionally, evaluators would do well to use social media to develop their reflections.
Can you point to some promising innovations in teaching and learning? 
The flipped classroom is perhaps the most notable.  It depends on no specific technology (usually Web video or podcasting), and is clearly focused on improving learning through enhancing the classroom experience.
Open education is growing steadily, as the amount of open material builds and the number of practitioners increases.
Mobile learning continues to transform teaching and learning, rendering nearly all locations potential connections to vast amounts of content and collaboration.
Gaming offers potentially huge and deep gains for learning.
Are there some effective research initiatives or studies our readers should examine? If not, why do you think that is the case? 
Columbia's Teachers College keeps publishing useful material.
There are a good number of books on mobile technology worth reading, even as that tech's rapid advances dates them quickly: Rheingold's Smartmobs, Katz and Aakhus's Perpetual Contact, Ito'sPersonal, Pedestrian, Portable.  

Let me return to social media.  Readers should curate sources on Twitter, the blogosphere, podcasting, etc,, as innovators often inhabit this space to share their thoughts.

Previous posts in the series:

Saturday, September 13, 2014

Evaluation & Innovation: Rick's Perspective

Drs. Shearer and Amankwatia at DTL
Dr. Rick Shearer and I at DTL
    

     
      Penn State University has been a good PA neighbor and friend to me  and other practitioners in the distance education field. At the Distance  Teaching and Learning Conference, I ran into Rick Shearer and asked  him to lend his voice to our series. Rick is Director of Penn State's  World Campus Learning Design. Dr. Shearer has been involved in the  distance education field for over 25 years.
 

      How do we recognize innovation in teaching and learning with technology?
I believe the best description of innovation in teaching and learning is related to the idea that innovation is a process where we use new technologies to solve existing problems in our learning environments. So in essence we likely have innovation happening everywhere and we want it to happen organically and not in a managed way. Thus instructional designers daily are using new tools to solve instructional problems in our online courses. What we need is a way to capture these innovations and better disseminate information so folks do not have to re-invent the wheel.

What challenges might practitioners and researchers face when evaluating innovation? Do you have ideas for how to overcome these challenges?
The biggest challenge is finding a way to pilot an innovation and then to also be able to walk away from it if it doesn’t work. Thus, how can you run small pilot studies on technology innovations in courses and then evaluate the effectiveness without getting into long-term contractual relationships with vendors. Often it is easy to get an innovative idea into a course, but much harder to stop using it if it does not meet the need in helping students reach the stated learning outcomes. Also, in piloting these innovations and technologies we must allow time over several semesters to evaluate the impact and we cannot get caught up in running out to adopt the next shinny object that has come along. We must take a more measured approach to our testing and evaluation of the innovations.

Can you point to some promising innovations in teaching and learning?
If we know of innovative solutions they probably are not innovative anymore, thus a better question may be about problems where we need technologies to address them.

Key areas where I see a lot of innovative approaches happening in the future will be around learning analytics (not predictive analytics) where we explore ideas of personalized learning paths and mastery-based approaches. What do these look like in today’s connected world, where we value the social aspect of learning.

Also, how will the distance education community move in order to provide verification of our students for the DOE and possibly for upcoming requirements of the reauthorization of the higher education act.

Another area that is already emerging, but needs more work is the integration of social presence type tools in our courses that are seamless, but also protect the privacy and rights of our students.

Are there some effective research initiatives or studies our readers should examine? If not, why do you think that is the case?
Unfortunately there are few places for practitioners to publish their work. Most journals are research oriented and require full studies that are difficult for practicing IDs to take on.
Although AECT has recently provided a venue for practical type research/reports, we need more options available for these types of practical studies. 

Previous posts in the series:

Friday, August 22, 2014

Evaluation & Innovation: Michael's Perspective

I recently returned from my regular trip to the Distance Teaching and Learning Conference in Madison, WI. One of my highlights was a pre-conference workshop on developmental evaluation with Michael Q. Patton. I asked Michael to lend his voice to our series.

I am the Evaluator- Patton
Michael Q. Patton, Ph.D.


Michael Quinn Patton is an independent evaluation consultant based in Minnesota. He is former President of the American Evaluation Association (AEA).  He has authored six evaluation books including a 4th edition of Utilization-Focused Evaluation and 3rd edition of Qualitative Research and Evaluation Methods. He is recipient of the Alva and Gunnar Myrdal Award for "outstanding contributions to evaluation use and practice" and the Paul F. Lazarsfeld Award for lifetime contributions to evaluation theory, both from AEA.  His latest book is Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use.  
http://www.guilford.com/books/Developmental-Evaluation/Michael-Quinn-Patton/9781606238721

     

How do we recognize innovation in teaching and learning with technology?
Innovation is contextual not absolute.  What is accepted practice in one context may be different and innovative in another.  Thus, innovation is a socially constructed notion which has to be understood and interpreted within a context. 


What challenges might practitioners and researchers face when evaluating innovation? Do you have ideas for how to overcome these challenges?
The challenge is that traditional ways of doing evaluation (logic models, SMART goals, predetermined outcome indicators, mandated rubrics and metrics) inhibit innovation.  The solution is developmental evaluation which accompanies innovators on the emergent journey of innovation, adapting evaluation to emergent issues.  For details about developmental evaluation see: http://www.guilford.com/books/Developmental-Evaluation/Michael-Quinn-Patton/9781606238721

Can you point to some promising innovations in teaching and learning?
Digital portfolios for online learners to archive their learning and products of their learning. These are accessible to the learners on a cumulative, long-term basis and can be shared for special purposes (e.g., a job opportunity).

Are there some effective research initiatives or studies our readers should examine? If not, why do you think that is the case?
Online learning, like all effective interventions, is most effective when geared to specific subgroups with identifiable needs.  For example, homeless youth have special needs.  Here is a study that identifies how to work effectively with homeless youth: http://www.terralunacollaborative.com/wp-content/uploads/2014/03/9-Evidence-Based-Principles-to-Help-Youth-Overcome-Homelessness-Webpublish.pdf

Previous Post in Series: Steve Ehrmann's Perspective