Taking this opportunity to drop in another paper I wrote for ED603. Let me know if you find it helpful!
Introduction
This paper discusses the evaluation of methods for a research project that involves attempting to measure building community interaction among first-time users of Blackboard while capturing the participants’ opinions of their experience.
Preparing for my research project I evaluated both the qualitative and quantitative research methods. Prior to understanding about the mixed method, I determined to incorporate portions of both types. The quantitative research method is appealing because of the larger number of research subjects, and what I feel is straightforward reporting on factual data.
The qualitative method has great benefits in the opportunity to dig deeper and get to an understanding of why or how a student might arrive at success; how the student perceives success in online education could be explored; why a student chooses to stay in and complete a course could be examined. All of these are worthwhile endeavors. Furthermore the qualitative method can put a voice to the research, a strong component in swaying middle administrative individuals and faculty to make changes.
Methods
I intend to follow a mixed methods approach to my project. According to Creswell,
you use mixed methods when you want to provide an alternative perspective in a study. An example of this would be an experimental study in which the experiment yields useful information about outcomes, but the additional collection of qualitative data develops a more in-depth understanding of how the experimental intervention actually worked. Another example would be when a policymaker wants both the “numbers’ and the “stories’ about an issue. (Creswell, J.W., 2012, p. 535)
Project questions
Will asking new Blackboard users perform basic tasks in a fun environment prepare them to excel in their every day use of the learning management system? I’d like to harness collateral learning to help students succeed. More importantly, I’d like to impact student satisfaction due to peer interaction. Student satisfaction is key in the social marketing heavy world where students have increasingly more choices for online education.
Can UAF eCampus help students build a peerage they might leverage to find a job or advance in their career? “When students finish their online classes, their situational relationships… evolve into productive professional networks.’ (Luo W., 2010, p. 96)
Project groundwork. I intend to sit in on a meeting with the individuals responsible for maintaining and updating the Blackboard Orientation shell. To this end, I’ve spoken to one of the individuals who maintain the product, as well as my immediate supervisor.
- The pre and post survey and the discussion board prompts designed to get student interacting with each other while learning some basic skills would be offered to the full number of participants loaded into the fall 2014 Blackboard shell: “UAF eCampus BLACKBOARD ORIENTATION.” If it is possible to create the survey instruments, get them approved, and build the prompts–as well as sample responses–prior to the start of the semester, this will work.
Nonprobability sampling is the avenue that is primarily available to me. I see it as akin to convenience sampling, but in reality the audience that I’m looking at is available and they will self-select. I have access to a pool of students via a LMS shell. My work will refresh the product regardless of how many students choose to interact with the new prompts and material. (p. 145)
Quantitative instruments. The first survey would be primarily closed ended questions. The numbers that could be captured on how many students participate in the discussion board tasks come from a series of reports. The post survey would be both closed and open-ended questions.
- After the students take the introductory survey they would receive an invitation to participate in a specific series of Blackboard Discussion threads to walk them through three tasks that are designed to do three things: be fun, assist the student in building connections to other students who are also just beginning their online learning journey, and practice three simple tasks that are helpful to Blackboard users: creating links, uploading images, and attaching files.
Qualitative instrument. In the post survey I will ask whether the student a. took the pre survey and b. whether they participated in one or more of the discussion threads as closed ended questions. I will prepare a few Likert-scale questions regarding their experience. The qualitative portion of the survey will seek to unearth attitudinal measures. Did they create one or more connections to other students whether in their field? How the student feels their participation impacted their Blackboard experience. I will write my own questions versus using an existing instrument. (p. 152) My goal is to keep the survey as short as possible.
Data collection. I intend to collect both types of data via surveys presented to the participants. The closing survey will have open-ended and closed-ended questions including qualitative data. Additional qualitative data–information about the number of student who chose to complete or attempt my proposed Blackboard tasks–will come via Blackboard reports. This data will not be tied to students responding to the survey. It will be the numbers of students who access or look at particular items as well as when these items are accessed. While I may consider at this point that two different types of data will be collected at the same time, it is not strictly the case. Reading through the explanation of convergent design analysis it is clear the qualitative data I collect occurs after the individual participates in the discussion board to explore attitudinal measures. In this way I would describe the data analysis to be explanatory. I will seek further guidance and assistance from my instructors prior to creating the survey instruments in order to make sure the data I am seeking to collect and the analysis I think I will perform will be in alignment.
Data analysis. The information I’ve read on descriptive statistics makes sense to me. In following research articles that were the easiest for me to read and understand–those that resonated with what I consider would be my approach–I would like to use as many visual representations, charts, Venn diagrams, simple tables, as possible and keep the statistics as clear and easy to follow as I can.
Reporting. The chapter on Mixed Methods Designs covers ways mixed methods results may be presented. For example
This design … is an explanatory mixed methods design; […] consists of first collecting quantitative data and then collecting qualitative data to help explain or elaborate on the quantitative results. The rationale for this approach is that the quantitative data and results prove a general picture of the research problem; more analysis, specifically through qualitative data collection, is needed to refine, extend, or explain the general picture. (p. 542)
Use of a notation system to clearly layouts out the “concurrent collection of quantitative and qualitative data’ as well as showing how I give priority to the variables, either quantitative or qualitative will be essential. (p. 538) At present I do not think that they will have the same weight, therefore expressing which has greater weight will be necessary. (p. 549)
Future Impact
Timeframe. Given my desire to complete the ONID program within the next four semesters, I believe I will only be able to show whether or not additional research should be done in a similar area. However, the side benefit of this research improves the online orientation–making it better perceived, received, and utilized. Should this occur, the lessons we learn can be extended to faculty teaching online as an example of how to help their students build a cohort they can take forward into the work arena.
Sequential design. Future researchers may wish to collect data via exploratory sequential mixed methods. This research project may lay groundwork for another student to collect quantitative data and then qualitative data. From my reading it is clear that this takes considerable time and planning; “a mixed methods researcher might collect quantitative and qualitative information sequentially in two phases, with one form of data collection following and informing the other.’ (p. 542)
Reflection
I am aware I cannot limit who chooses to participate in the discussion threads. Furthermore, I do not feel it would be beneficial to limit the individuals that participate in the threads to only those who take the pre survey. It may be interesting to collect information about those who perform the tasks and the complement, but I do not see how I can track the individuals to that level of granularity given the current Blackboard reporting to which I have access.
Regardless of the method I use I will need informed consent. Since I am now considering using human subjects, I need to review the rules and make sure my paperwork is in compliance and my committee is involved prior to anything beginning.
One aspect I want to think about even before beginning is how I will display information in the final project write up. Because I truly enjoy working with data I would like to include a lot of it. However, I understand through my article review that many reports either speak at a level that is too difficult to understand, or include information that is difficult to absorb. For the most part the data, tables, and Venn diagrams in Challenges for educators using distance and online education to prepare students for relational professions are easy to comprehend. (Hockridge, D., 2013)
References
Creswell, J.W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research. Boston: Pearson Education.
Hockridge, D. (2013). Challenges for educators using distance and online education to prepare students for relational professions. Distance Education, 34(2), 142-160.
Lu, W. (2010). How Social Network Position Relates to Knowledge Building in Online Learning Communities?. Frontiers Of Education In China, 5(1), 4-25.