I did mentioned about meta-analysis once in my blog entry last year. While it is still fresh in my mind, I would like to share about this. I have recently completed two meta-analysis of pair programming (PP) effectiveness for the PP studies conducted in higher education settings. First of all, you need to know what is meta-analysis, what is it for, and whether you need to do it or not. Once you understand the major purpose of doing a meta-analysis, you should then identify how to carry it out. Well, here is a brief explanation about meta-analysis (refer to references for further info):
In the simplest way, meta-analysis can be defined as a method or approach to combine findings from several independent (but homogeneous) studies for the purpose to estimate which intervention is more effective. Meta-analysis has been used extensively for research in medicine to confirm about the effectiveness of certain healthcare intervention normally thru several randomised controlled trials. As for my case, I am studying about how effective has PP been within a higher education setting. In particular, how effective has PP been in helping students achieving better scores in their assignments and final exams, compared with solo programming.
Meta-analysis (MA) involves 3 major steps:
- Decides which studies to be included in the MA.
- Estimate an effect size for each an individual study.
- Combine the effect size from the individual studies
So, another terminology that you must understand: effect size. Here is the definition from the Wikepedia:
In statistics, effect size is a measure of the strength of the relationship between two variables. In scientific experiments, it is often useful to know not only whether an experiment has a statistically significant effect, but also the size of any observed effects. In practical situations, effect sizes are helpful for making decisions. Effect size measures are the common currency of meta-analysis studies that summarize the findings from a specific area of research.The effect size can be measured in two ways:
- by a standardized difference (if the study has comparison) OR
- by a measure of association (i.e. correlation coefficient)
The choice of an indicator of effect size depends on the type of studies included in your MA. For my case, I am using the standardized difference of means to calculate the effect size because I'm having two different group of studies: experimental group (PP) Vs Controlled group (solo).
Using the Cohen's formula, effect size can be calculated as:
d = M
1 - M2 / s where s = Ö[å(X - M)² / N]
where X is the raw score, M is the mean, and N is the number of cases.
While using Hedges formula,
Hedges g = M
1 - M2 / S
pooled where S = Ö[å(X - M)² / N-1]
and, Spooled = ÖMSwithin
No worries about this, as long as you know which calculation of effect size is relevant to your case. Usually MA software have the abilities to generate the effect size and what more important is you must be able to interpret the overall effect size produced by the software :-)
- So, what software can I use?
From my little knowledge and observation, there are several MA software available out there, some of which are publicly accessible :-D and surely some are commercial ($$$ maah)
The one that I'm using now is MIX version 1.7, a MA software developed by Lion Bax and his fellows at Kitasato Clinical Research Center, Kitasato University, Japan. I highly recommend the use of MIX, not only because it is FREE :-), I found that it was easier to be used when compared with other software. Other available MA software (commercial use) : Comprehensive Meta Analysis 2.0, MetAnalysis, MetaWin, WEasyMA. Others open source MA sofware: EpiMeta, Meta-Stat, RevMan etc.
The following forest plot was generated using MIX1.7. The interpretation of the graph is available in my paper, which is still in the writing stage. I will share the write up of the results when I finish with the paper. insya'Allaah.
References:
[1] http://www.cemcentre.org/renderpage.asp?linkID=30325015
[2] L.M. Pickard et al. "Combining Empirical Results in Software Engineering", Information & Software Technology, vol. 40, pp. 811-821, 1998.
[3] http://wilderdom.com/research/effectsizes.html
[4] http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2048970
[5] http://www.mix-for-meta-analysis.info/index.html
2 comments:
Salam achik, nnti dtg blog jajai amik 'Bear' yer...sedap peluk time sejuk2 ni...hehehehe, take care...
Salam jajai,
hehe okie dokie nnt acik g amik ye. Selamat berpuasa :-)
Post a Comment