bronwynhegarty doctorate meanderings

meeting 24 march and progress stunted by Womad

2009-03-23T20:27:00.000-07:00

Image: Moana & the Tribe - WOMAD Taranaki 2009
At the moment I am chugging along with the individual cases. Womad sort of got in the way but now I am on the wagon again with the thesis. I am almost finished the third case. My supervisors have the draft general results and the draft methodology. They are on the shelf at the moment as the priority is to finish the individual cases so I can prepare a discussion about them.

Meeting 24 March 24, 2009

S & L are not sure about adding in extra description for case three. I have added more description about the style of writing in the section called: use of the reflective framework. The reason is because the reader/examiners will need to compare all the cases by headings which need to be consistent – if extra headings and extra writing may confuse examiners. However, I may have discovered a new heading which is needed in all the cases even if no data is available for some because they did not write their reflections using the template headings.

B to do

1. Send third case when done.

2. Put all three cases together (first one with feedback) and send to S & L.

3. Delete results chapter with S’s feedback and leave until later. Tempted to look.

4. Start on next case in a different doc. Add to finished cases and send as draft doc – this will help keep everything together and enable comparisons to be made more easily, plus control versioning of the document a bit better. S & L will keep the same version with their feedback on it.

5. Aim for two weeks to have all cases finished and sent to S & L.

6. Work on a discussion about themes across all cases.

Doctorate overview of findings presented to Spotlight Tertiary Teaching and Learning Colloquium for the Southern Region

2008-11-25T15:02:00.000-08:00

On 20 November I gave a presentation about my Doctorate research to the Spotlight on Tertiary Teaching and Learning Colloquium for the Southern Region. The presentation is below or you can link to all functions for the presentation - pdf download, full screen etc - on Myplick. The Three-Step Reflective Framework diagram and template can be accessed here.

I have a Creative Commons by attribution licence on my work so as long as you ensure you give attribution to me, you are free to use my work and modify it to suit your situation.

BHSpotlight presentation three step framework2

Data analysis: how to measure critical reflection?

2008-09-08T02:04:00.000-07:00

The process is slow but I finally feel as if I am getting somewhere with the qualitative analysis part of my thesis. I have been analysing participants' reflection assignments using the Levels of Reflection Taxonomy - descriptive, explanatory, supported, contextual and critical using NVIVO - which I developed way back in August 07.

Since pulling out the textual references at each category and sub-category for the five levels of reflection, I have annotated them and put them in tables so I can pull out the themes and patterns both for each participant and across the group. I have also calculated frequencies for each category and sub-category for each level of reflection so I can compare participants' writing.
A dilemma came up when trying to look for the higher levels of reflection - contextual and critical - it is impossible to gauge these levels by using a reduction methodology i.e. coding each sentence. The unit of a sentence I have used for coding in NVIVO works for descriptive, explanatory and supported reflection. To ascertain whether contextual or critical reflection is occurring, I will need to analyse the data more holistically, i.e. look for paragraphs and chunks in each assignment and rate them for contextual or critical reflection using criteria. So to do this I need to go back to the theoretical principles I used to create the taxonomy in the first place.

It was based on the following people's work:

Allen, J. & Jeffers, G. (2000). Teaching the dialectic process to preservice teachers in an educational psychology class. Annual meeting of the American Educational Research Association. New Orleans, LA, April 24 – 28.

Hatton, N. & Smith, D. (1995). Reflection in teacher education: Towards definition and implementation. Teaching and Teacher Education, Vol 11 (1), p33-49.

The five level approach developed by Hatton and Smith (1995) has reflection-in-action placed at the highest level. In summary, the model has the following components:

1. technical - decision-making about skills

2. descriptive - seeking best possible practice

3. dialogical - exploring alternative solutions

4. critical reflection - goals and ethical practices in wider context

5. reflection-in-action - contextualisation of multiple viewpoints - drawing on

the other four levels.

Sparks-Langer, G., Simmons, J., Pasch, M., Colton, A (1990). Reflective pedagogical thinking: How can we promote it and measure it. Journal of Teacher Education, Vol. 41, No. 5, 23-32.

I have also started organising the findings to see how I can answer the main research question as below:

1. What type of reflective strategies support practitioners when they develop and present an electronic portfolio?

a. What sort of approaches do participants take when writing reflectively?

My next step is to explore critical reflection further. Here is a paragraph I like about critical reflection:

“Critical reflection involves thinking and problem solving (Copeland, Birmingham, De La Cruz, & Lewin, 1993; Dieker & Monda-Amaya, 1997). Problem solving is a process in which capable individuals attempt to make sense of a challenging situation, identify areas of practice needing scrutiny, define goals for improvement, and pursue actions to accomplish them.

Reflective practitioners use this process to modify and enhance their understanding of professional practice. The end result of problem solving is reconstruction of knowledge. Ginsburg (1988) notes that critical reflection must involve practical experience as well as a knowledge base for thinking: Critical praxis is the process of combining critical thinking and critical practice. The issue here is to retain a critical stance toward society, schooling, and teacher education, while developing and refining strategies for action, for intervening in political, economic, and ideological arenas. It is not just a question of theory informing practice, but also practice informing theory (p. 202).” An Examination of the Construct of Critical Reflection: Implications for Teacher Education Programming in the 21st Century."

Author(s):Deborah S. Yost, Sally M. Sentner and Anna Forlenza-Bailey.

Source:Journal of Teacher Education 51.1 (Jan 2000): p39. (6488 words)

looking ahead in 2008

2008-01-24T01:38:00.000-08:00

The pic is a reminder of my wonderful camping holiday on the west coast and Golden Bay - the holiday memories will keep me going in 2008. It certainly cleared my mind ready for the onslaught this year - now I am ready to focus.

This is a pic I took of Kaihaihai beach from the zig zag track at the start of the Heaphy track near Karamea - January 2008.

Data collection and analysis

To date data has been collected from seven participants:
1. survey questions - demographic, professional role and previous experience with reflective writing;
2. assignments - reflections prepared using the RF and supporting statements which accompanied the learning objects;
3. interviews with seven participants and the subject lecturer.

Survey responses were used to prepare semi-structured interview questions.

To date the following analysis of the data has occurred:
1. Survey data has been entered on a spreadsheet;
2. Reflections have been coded in hard copy to develop a hierarchy and categories for coding in NVIVO;
3. Levels of reflection have been coded using NVIVO;
4. Content coding analysis has begun.

All participant interviews have been transcribed.

The plan for 2008 is to have a draft of my thesis by the end of this year! Here is the list of milestones:
Autumn Session 2008
31 January - data analysis - all codes checked for assignments and investigation of relationships between data commenced, themes and patterns investigated.
29 February - interviews coded. Methodology written up.
31 March - interview codes checked and relationships, themes and patterns extracted.
30 April - results ready to write up more formally from project notes.
31 May - results completed.
30 June - literature review added to and compiled for final thesis. Reference list continues to be added to.

Spring Session 2008:
31 July - Discussion of results commenced.
31 August - Discussion continues to be written.
30 September - Discussion completed.
31 October - Introduction commenced.
30 November - Introduction continues.
31 December - Introduction completed, draft thesis.

For the next meeting with my supervisors, I need to have finished the code checking of the reflections - levels of reflection and also content coding if possible AND prepare a short synopsis. The annual report has been completed and signed off. Still waiting to find out if an ethics progress report is required. This year we will be having fortnightly meetings to keep the progress happening. Each day I need to spend a concentrated amount of time on the work.

In preparation for the 3/12 write up time in the second half of the year, I need to organise my work life carefully so any projects are neatly tied up when I go on leave apart from the research project which I can keep ticking over.

Free nodes and more categories

2007-10-21T20:40:00.000-07:00

Analysis of the data is going to be around not only levels of reflection but also categories and sub-categories illustrated in the mind map here. For example, multimedia design, challenges, professional. These were some of the words which jumped out at me when I was working through the participants' reflections. They were mentioned frequently and in relation to words such as skills, technology, roles, goals and learning. If you click on the diagram you can view a larger size.

A month of qualitative analysis

2007-10-01T20:09:00.000-07:00

October is the month of qualitative analysis. I have developed some codes to use in NVivo on the levels of reflection and have set up tree nodes using the software. I also had a play with free nodes. NVivo uses a hierarchy system with tree nodes.

I found it easiest to create a concept map first of the main categories which came up when I worked through the data in hard copy e.g. learning, MM design, professional, teaching, ed tech tools, challenges, readings. Watch this space!

The intention at present is to focus on the levels of reflection in five categories:
1. descriptive reflection - superficial;
2. explanatory reflection - with rationale;
3. supported reflection - with evidence;
4. contextual reflection - different perspectives;
5. critical reflection - applied.

The levels of reflection apparent in the data don't necessarily directly match the categories in the three-step reflective framework though I have incorporated them. So far, I am working through the documents and coding levels of reflection in NVivo. Once this is done for one data set, I will code the other categories and sub-categories. All the reflections are now coded with codes for levels of reflection. I have to run some coding queries and see how the text I have coded looks regarding levels of reflection. I also have to figure out how to measure the instances of reflection and range of levels. At the moment the highest levels of reflection - contextual and critical are not very common. Explanatory reflection (where rationale is given) and supported reflection (evidence from theory & the literature) and descriptive reflection (no rationale) are the most common.

Some questions:

What is critical reflection really for the purposes of this research?
Has the reflective framework helped with critical reflection?
Is it okay to code one paragraph or sentence with more than one code?

So far I have defined critical reflection as: Writing demonstrates awareness of multiple theories and principles and multiple perspectives – ideas which go beyond the subject, the project and the self. The application to professional practice is evident. What was learned and how it can be used is outlined in the writing. There are a few examples of critical reflection emerging from analysis of the reflections.

To do: The idea of setting up a matrix to look for relationships is something I need to explore further.

The next thing to do is run some queries on the reflections using the free nodes I have set up. Plus explore them to make sure I have coded thoroughly. some of the free nodes will become tree nodes eventually. I am still not entirely sure about the hierarchy in this system.

NVivo tutorials
Luckily I found some free tutorials on the web for NVivo as the tutorials which come with the software are pretty slim pickings. They can be found from Lyn Richards home page. She has spent years running NVivo workshops and has developed some pretty comprehensive tutorials.

Research articles
When pondering where to code goals as a sub-category in NVivo, I found an interesting looking conference paper called: Nicholls, H. (2003). Cultivating “The Seventh Sense” – metacognitive strategising in a New Zealand secondary classroom. International Education Research Conference AARE - NZARE, Auckland, New Zealand. Available at: http://www.aare.edu.au/index.htm as a pdf

I will read and annotate as it may help me get some clarity about the whole coding thing.

self-regulation and teacher feedback

2007-06-05T01:04:00.000-07:00

It is a while since I made an entry. At present I am busy collecting data via interviews and written reflections. One of the things I am interested in, apart from how the three-step reflective framework was used, is how participants have been reflecting in their studies and professional lives normally and how they may have changed their approach since using the reflective framework. One of the variables in the reflective process may also be the amount of feedback they were given on their assignments.

As part of the interview, I asked participants whether lecturer feedback was helpful in helping them reflect - so far they have said yes. Then I was listening to a keynote from the REAP assessment conference I found some interesting material which suggests how important feedback is in the process of reflection. In the keynote called: Principles of good assessment and feedback: Theory and practice, David Nicol, University of Strathclyde- the emphasis was on self-assessment and feedback. The following statement relates closely to the way in which teacher feedback can assist reflection:

..."the act of using teacher feedback implies that self assessment is present. To use teacher feedback students must decode the message, internalise it and use that feedback to make judgements about their own work and then act on those judgements – but this already implies internal processes of comparison and self-assessment. Hence self-assessment is actually embedded in the use of teacher feedback. And if the use of feedback involves active self-assessment would it not be better if we strengthened this capability rather than only focus on improving teacher feedback. So one question for me was: How can we scaffold students’ learning so that they become better at self-regulation?". In the article accompanying the keynote I found some principles which were linked to feedback and reflection.

Four principles relate to feedback as an assessment tool. But note that only one specifically mentions reflection. I would suggest that feedback of any kind stimulates some sort of reflection anyway.

The principles 3 to 6 are as follows:
3. Deliver high quality feedback information that helps learners self-correct.
What kind of teacher feedback do you provide – in what ways does it help
students self-assess and self-correct?
4. Encourage positive motivational beliefs and self-esteem.
To what extent do your assessments and feedback processes activate your
students’ motivation to learn and be successful?
5. Encourage interaction and dialogue around learning (peer and teacherstudent.
What opportunities are there for feedback dialogue (peer and/or tutor-student)
around assessment tasks in your course?
6. Facilitate the development of self-assessment and reflection in learning.
To what extent are there formal opportunities for reflection, self-assessment or
peer assessment in your course?

Some of the resources associated with the REAP project can be found on the project site.

This self-assessment idea and feedback also relates to e-Portfolios and blogs - think about how we reflect on what we have written when someone gives us feedback on our blogs. Feedback on e-Portfolios, however, could be a more spurious thing because they are rarely set up to receive dynamic and ongoing feedback - generally the feedback is more finite i.e. whether you get a pass grade or the promotion, or the practising certificate or the job for which you created the e-portfolio. So the question is..the reflective framework is designed to assist people reflect as they prepare evidence for an e-Portfolio or blog, but how can it assist in the feedback process...this may be something I need to look into further....feedback well positive feedback can be very motivating. Negative feedback on the other hand can drive us underground or into retaliation...ah but that is another story.

Does feedback on a reflective blog help people reflect even further and alter the course of their learning?

progress on data collection in 2007

2007-04-11T22:48:00.000-07:00

I now have seven participants taking part in my research. They are associated specifically with a course of study into which the reflective framework has been woven to ensure it is used as part of several assignments.

The workshop in March was very useful both for talking to prospective participants and also for trialling the reflective framework template. I presented a short overview about reflection and explained how the reflective framework and template could work.

Basically there are three levels as you can see on the diagram: 1. Take notice and describe the experience - free writing stage; 2. Analyse the experience - why the actions and decisions, what reactions and 3. Take action - what was learned and how will the learning be used.

During the workshop, participants practised doing some reflection about a holiday experience and used the template to write in. They all found the first stage relatively easy, once they had decided what to write about - reflection in itself, but found stage two and three more problematic. Once they used the framework once, however, they could see how they actually reflected on different levels.

After the initial practice using the framework about a personal experience, they then wrote a reflection relating to the type of work they do and the professional development they were undertaking as part of their study.

I was very happy with the first results of trialling the framework. It became obvious during the workshop how important support and guidance in "teaching" reflection actually was to ensure that practitioners did it well and to a level which was satisfactory for them.

During the workshop, participants also filled out a survey. The questions gathered information about both their professional experience and about their experience with using reflective techniques. Following the workshop I recorded my reflections about how it all went. Now I feel as if I am on my way with regard to data collection. I will be collecting the reflective assignments of the participants and analysing them to see how they are using the reflective framework.

The plan for 2007

2006-10-15T17:59:00.000-07:00

The eureka moment idea I had was not regarded as suitable by my supervisors. With some new strategies in place, they think I will get more participants next time.

Next year I will actually go to Wollongong and spend some time with potential participants. This will happen on 3rd March 2007 at a Saturday workshop when the next design subject starts.
I will talk about my experiences doing the subject and how I am using what I learned in my work as an educational designer and teacher. I can also introduce the research in particular the reflective framework and talk about how they will be able to use it. I will also get them to fill out the questionnaire in class. I prefer online questionnaires to save time on the analysis but it would be silly not to take advantage of the f2f opportunity. Unless I can get a computer lab for the purpose.

In the meantime:

1. review the reflective framework and prompts in light of the three design objects and supporting statements, and possible project log. - BH

2. hear re arrangements for the Saturday workshop on 3 March 2007

- I will outline some ideas for what I will cover (introduce research, my experience with the design subject and how I am now using it in my practice, focus group re reflection, get them to fill in questionnaire) for sue to critique

3. receive the new subject outline i.e. explanation for the assessments and dates for submission from sue later on this year.

4. develop some specific criteria that can be used with the supporting statements to bring out reflection re professional practice, and send for critique e.g. how well did you achieve your design goals?

5. develop some criteria for analysing the quality of the reflections

6. support statement instructions to be rewritten so it is more reflective

7. pilot report to be reviewed by supervisors to decide which bits to include in methodology -

8. Prepare timeline for data collection and analysis

Thoughts about how to attract participants

2006-10-07T17:20:00.000-07:00

I had a eureka moment the other day about what I could do to attract participants for my research using the reflective framework I have developed. Since the initial idea to embed it in a course at the University of Wollongong was not lucrative re getting enough participants.

It makes sense to ask my networks and students in the design course to have a look at the framework to see both what they think about it and also to ask them if they'd be interested in using the framework with their blogs and learning logs to re-examine them reflectively.

Reflective framework

I will have to re-apply for ethics approval of course providing my supervisors like the idea but it would be better to undertake the research more openly I think away from a restriction such as having the framework tied to a class assignment. That is of course if people think the framework is useful and they will take the time to use it. I have started a wiki called brondocresearch where interested people can contribute.

More detail about how to use the framework on my wiki.

1. Analyse the experience

3.1 Why these decisions?

Analyse your actions and decisions. Think about why they were useful and how they helped. This process will assist you to become a more active learner.

Prompts:

§ Comment on what worked and what didn’t.

§ Think about how your actions and decisions may help your teaching.

§ Consider the links between learning theory and your teaching/design practice.

§ Indicate what you learned.

Tips: Be honest with yourself, and use the information to get a clearer picture of how you learn.

Use diagrams etc to illustrate your point.

3.2 What was your reaction?

Brief notes on your emotional and physical responses will help you better understand the way you learn. This will help you in future to choose strategies which suit your learning style.

Prompts:

§ Comment on what you liked or did not like about your ideas.

§ Think about what could have been done better.

§ Indicate some strategies which may help next time.

§ Did it cause you to rethink your ideas about teaching/design?

Tips: Keep an open mind and seek assistance from a mentor or peer as necessary.

the main study has started

2006-09-05T21:34:00.000-07:00

This week my first participant in the main study sent in a consent form and the survey sheet. Not sure what has happened to the rest of them. Maybe they are coming by camel and ship from Wollongong. I am steadily working on writing up some detail about the different reflective models from the literature which I have selected to fit the framework I am using.

starting out

2006-04-03T16:33:00.000-07:00

Ah the doctorate!. This year I will be sampling for the main research study. I have made quite a change in my plan of attack since running the pilot last year.

The strategies I had in place for the pilot were too complex. Since writing up the pilot and meeting with a critical friend, I have moved between two areas of approach. I was going to try out some metacognitive strategies - plan, monitor, evaluate with self-questioning....but now I have been persuaded to stick with reflection. apparently the literature on metacognition is too fragmented...the research on reflection is much more consolidated.

So today I have been doing lots of reading about reflection, so I can link the reflective framework I am sketching out to some previous research. My critical friend suggested using three overarching questions about decisions made during an assignment ...my participants will be students in an teacher education course at the university I am studying at: University of Wollongong. They will be asked to keep a journal as they prepare an assignment.

In the reading today, I refreshed myself about Jennifer Moon and Donald Schon's models.
Moon has five levels:
1. noticing - what is known, perceptions of purpose, emotional responses, presentation of material - superficial
2. making sense - organise and order material - superficial
3. making meaning - assimilate, link to what is known, understand material - deep
4. working with meaning - reflection, generate ideas, review how knowledge influences actions - deep
5. transformative learning - cognitive adjustments increase further, awareness of learning process, judgements about usefulness.

Schon - reflection-in-action and reflection-on-action (easier and most common).

Hatton and Smith (1995) have developed a five level approach based on Schon. I need to read more about this.
1. technical
2. descriptive
3. dialogical
4. critical reflection
5. reflection-in-action

I particularly liked the reflective cycle put forward by Rodgers ( 2002) and have started to explore this. see diagram:
Rodgers compared to Moon (1999)
1. presence in experience - similar to noticing and making sense (Moon) also mindfulness
2. description of experience - free writing - making sense (Moon)
3. analysis of experience - making meaning and working with meaning (Moon)
4. experimentation - working with meaning & transformative learning

Rodgers does not really fully have an equivalent to transformative learning but I guess the transformation occurs when the learner changes their ideas and actions following a period of reflection. Zepke (2003) refers to Kolb's experiential cycle in the book called reflection to transformation (Eds) (Zepke, N, Nuent, D. & Leach, L.) lent to me by a dear colleague. I also found a very practical model for reflective journaling in R's study book Burns, T. & Sinfield, S. (2003). Essential study skills: the complete guide to
success @ university. London: Sage Publications Ltd.

The trouble is the reflective framework I want to use needs to be suitable for when they actually write their reflective journal while doing the assignment and also when they formulate their supporting statement to go with the product they are designing and developing. A discussion with my daughter made me realise that both will be reflection-on-action.....ah but which model will I align with? More tomorrow.....