Reliability in Meta-Literature Reviews: Pressure and Resistance in Research Pathways

Introduction

Literature reviews are important in academic studies to elaborate information, collect and investigate existing publications. Systematic literature reviews are important in increasing the reliability of the literature review. In general, literature reviews can be structured according to several different criteria. Suggesting an absolute 'best practice' is impossible as different reviews require a different approach. When literature reviews were analysed, it was found that there are literature reviews that respond to different research questions, follow a different systematic and are referred to by different names.

The literature reviews must result in accurate analyses for science. The studies should be criticisable, have powerful data, and be free from political approaches and pressures. Meta-analysis (Glass, 1976), also called analysis of analyses, involves summarising the results of a large number of quantitative studies and performing various analytical tests to show whether a particular variable has an effect. This type of analysis serves as a means by which the results of many quantitative studies on a particular topic can be summarised and compared. This approach aims to determine whether a certain variable has a certain effect by comparing the results of different studies (Bryman, 2016, pp.16-17). Like all studies, it is crucial to fully understand the quality of the data, the rigour applied in the coding and analysis stages, and the statistical assumptions made in meta-analysis. Quantitative meta-analyses require the utmost care and an approach that clarifies decision processes, decision criteria, data, analysis and interpretation procedures (Gretzel & Kennedy-Eden, 2012).

As in all research, it is important to fully understand the quality of the data, the rigour applied in the coding and analysis stages, and the statistical assumptions made in meta-analysis. Quantitative meta-analyses require maximum attention and an approach that makes decision processes, decision criteria, data, analysis and interpretation procedures clear (Gretzel & Kennedy-Eden, 2012). This point is important because, as in other research, there are critical aspects in the meta-analysis system. These criticisms usually say that the information is under political influence. This study aims to discuss the meta-literature method used to examine the effect of social media on mental health in detail and to evaluate the possibilities and limitations of this method. In addition, the political reflections of this method in the field of social sciences and its contribution to contemporary debates will also be analysed.
 

 

Meta-analysis is the systematic summarisation of a group of studies on a particular topic with the help of statistical methods (Paul & Barari, 2021, p. 1099). Meta-analysis, which is a systematic summarisation of individual studies on a particular subject with the help of statistical methods, is seen as a detailed literature review method in the most basic sense. This calls the meta-analysis method the analysis of analyses (Glass, 1976). Based on the lexical meanings of the method, it is clear that the word meta carries a more comprehensive, comprehensive meaning. It can be stated that meta-analysis, which is a systematic literature review method, is a serious process provided that the individual studies that are evaluated are taken into consideration to the greatest extent possible according to the suitability of the individual studies and subjected to a detailed screening and classification. The meta-analysis method, whose awareness and application is gradually increasing, first emerged in the early 1900s as a result of Pearson's development of formulas with the average values of individual studies previously conducted on the same subject for the study he conducted in 1904 to examine the relationship between vaccination and typhoid fever. Quantitative methods of combining the results of previous individual studies were first defined in the early 1930s. In 1932, the probabilities found in the studies conducted by Fisher represent the definitions mentioned above. In 1954, the algorithms prepared by Cochran by combining the parameters obtained from the results of different individual studies and the methods he proposed are a turning point for the meta-analysis method (Egger & Smith, 2001). Glass, McGaw, & Smith (1981), Hedges & Olkin (1985), Hunter, Schmidt, & Jackson (1982), and Rosenthal (1984) have conducted studies in many fields since the 1980s with the new techniques they developed.

With the increasing interest in the meta-analysis method in the 1970s, applications were made especially in the fields of health sciences. Glass (1976) named the research conducted in this way as meta-analysis method. In the following years, the 1980s, the meta-analysis method started to make further progress thanks to the intensive, detailed and patient work of Peto and his colleagues at Oxford. Hedges and Olkin (1985) and Petitti (1994) developed and defined the statistical methods of meta-analysis, and Greenland (1987) defined in much more detail the statistical methods required for the suitability of individual non-experimental studies for meta-analysis.

Meta-analysis requires the expression of research results as the degree of effect. The concept of "effect size" with Cohen's "d" was first introduced to the literature in 1978. As proposed by Glass, meta-analysis is the summarisation of different research results on a topic by using quantitative research synthesis methods. However, this method may have political potential in certain research steps.
 

 

The meta-analysis technique is a statistical method that has been frequently used in recent years to eliminate contradictions in different subjects due to the idea that it is the method that provides the best evidence. Meta-analysis is a very powerful method in terms of eliminating contradictions in researches, obtaining a strong result and creating a final valid result because it combines the researches conducted by researchers on the same subject for many years. Compared to all other research methods, it is at the top of the pyramid of studies that provide the best evidence in terms of evidence generation (Cook & Leviton, 1980; Eysenck, 1978).

Although it systematically combines research, it also analyses all the data. Therefore, it provides an original view and the publication it produces is an original publication (Cordes, 1985, p. 16). It is an important issue to pay attention to whether the article is internally unbiased in the selected researches and to select it after providing evidence that it is unbiased. The adequacy of the evidence as an external assumption depends on the quality of the relevant studies and whether they are fully published (Glass, 1976). One of the biggest factors in the preference of meta-analysis is that it is faster, easier and can be applied without the need for another method.

When we consider the reasons for preferring meta-analysis, we can say some things: With meta-analysis, researchers estimate the lowest variance, reliable and valid parameters related to the subject they are investigating. Meta-analysis helps to eliminate the inconsistency between the results obtained from different studies. Meta-analysis is done to increase the sample size, to increase the precision and power of parameter estimates and thus to reduce the probability of making errors in estimates. If there is heterogeneity between studies, it is done to prevent this. It is done to prevent bias due to the researcher. It is done to find solutions to questions that do not come to mind at the beginning of the study.

One of the most important issues to be considered in meta-analysis is the bias of the publication (Harrison, 2011; Dubben & Beck-Bornholdt, 2005). This is due to the presence of positive significance effects in clinical studies and the high tendency of these situations to be published in journals with high impact factors, which leads to publication bias. The fact that the articles with publication bias are likely to be cited by others because they find a positive effect increases the bias even more. Since the combinations made in meta-analysis require some decisions, personal judgement and expertise, some difficulties may be encountered. Some of these may be as follows; The fact that some of the groups are small and have obtained biased results negatively affects the meta-analysis. Meta-analysis should be done with the results obtained from the studies conducted respectively, not from random sampling. One of the expected results of the meta-analysis is to include all evidence obtained from multiple independent sources in the analysis to evaluate the hypothesis. It is impossible to identify all available evidence from a small number of sources. For this reason, it is necessary to search most of the databases to find relevant publications on the subject.
 

 

After determining the studies to be included in the analysis in meta-analysis, the model and statistical methods suitable for these studies are determined and the results are combined. Selecting the model to be used in meta-analysis is the most important stage of the analysis. The biggest problem in combining more than one study is that the sample numbers are different. There are two statistical models used in meta-analysis; fixed effects model or random effects model.
 

 

According to this model, all studies included in the analysis have a true effect size, and differences in observed effects are due to sampling error. A possible effect of a variable is not affected by the study criteria and remains constant in each study (Torgerson, 2003).

This model assumes that all studies estimate the same effect. The difference between effect sizes is due to sampling error. Narrower confidence intervals are obtained. Since variance is not taken into account, it does not give precise information about the homogeneity of studies (Finfgeld, 2003, p. 895).
 

 

According to this model, the effect size of the included studies are different from each other. The effect sizes of the studies added to the meta-analysis define the sample. The overall effect size obtained as a result of the meta-analysis is the average value obtained from the effect sizes of all studies (Noah, 2017, p. 201). In the fixed effect model, a single effect size is calculated, but in the random effect model, an average effect size is calculated from the effect sizes of the studies. Since the effect size of the studies is different, all studies are included in the general effect size calculations. In the fixed effect model, sampling or estimation error in the studies may cause uncertainty. In the random effect model, an error may also occur due to variance between studies. In the random effect model, the variance, standard error and confidence interval values for the summary effect size are always larger than the fixed effect model (Bryman, 2016). Since the variance of the studies is also combined, wider confidence intervals are obtained. It gives an idea about the homogeneity of the studies. This model is more sensitive in small studies.
 

 

In meta-analyses, effect size estimates differ from each other. It is important whether these differences should be taken into account. Before using the combined findings in a meta-analysis, statistical tests should be performed to detect heterogeneity and the findings obtained should be visually examined.

When the p-value is found to be low in statistical tests to detect heterogeneity, the differences observed between the study findings cannot be neglected. However, the power of the tests for heterogeneity is low, and there is no definite and clear level of significance. Therefore, when the p value is not too high, heterogeneity should also be reviewed graphically.

To detect heterogeneity in meta-analysis, statistical hypothesis testing is performed first. The results are reported according to whether the statistical results are significant or not. The H0 hypothesis, which assumes homogeneity, is tested. When heterogeneity is detected according to the p-value obtained according to the statistical analysis results, the random effect model is used in the meta-analysis, and when homogeneity is found, the fixed effect model is used. Different statistical analyses can be performed to determine heterogeneity between studies.
 

 

The effects of social media on mental health may result differently with the meta-analysis method. Therefore, the limits of meta-analysis management are important. The difference between studies, bias of publications, heterogeneity, misinterpretation of effect size and the potential for pressure on the study directly affect the outcome of the study. The objectivity of the studies is important. For this reason, the meta-analysis technique is risky because the conditions under which the studies were conducted can be ignored.

It is not possible to understand whether some researches are biased by a social media platform. Any social media platform can have a study result published in line with its own wishes and it can be concluded that social media is good for mental health. This also applies to political pressures. Studies involving the political system can also be published biased. Even if it is clear by whom such studies are conducted, it is also important by whom they are conducted. For this reason, transparency, sources of data, details of reports, and sources used are important for meta-analysis results to be more reliable.
 

 

Meta-analysis method is a method that provides the opportunity to make a common and healthy decision by combining individual studies previously conducted in the field in order to eliminate the lack of confidence due to insufficient data, inconsistent and contradictory individual study results. However, in some cases, the results of the research may not be of high quality and low quality publications negatively affect the result. The results of researches using faulty methods also affect the quality negatively. In this case, the result of meta-analysis is negatively affected. The reliability of the data within the scope of the research is important. Faulty methodology and biased content in unreliable studies directly affect the results of research. In research on the use of social media, how the data is collected is as important as the objectivity of the research. The meta-analysis technique, which is based on the collection of the results, may reveal different results according to these results. A researcher who investigates the effect of social media on mental health, it is important how deep he/she deepens this research.

How the researcher prepares and diversifies the research questions is also important. Even gender, culture, education level and geography can be influential for the outcome of the research. It is also important how which platform is evaluated. If the researcher analyses all social media tools under a single heading, he may have reached wrong conclusions. For example, while those who use platforms such as twitter and facebook are negatively affected, there may be those who spend more productive time on platforms such as youtube. For this reason, it is necessary not to ignore the details in the research conducted by the researcher. Since the Meta Analysis technique collects this data and reveals new data, the result may be misleading.

On the other hand, data collection and objectivity are also important. It is important in which language or geography the researcher conducts his/her research. Collecting data only in the English language may produce different results for speakers of other languages and ignore others. The method of data collection is also important for research to provide quality results. In sample studies, Huang (2022) used 123 studies measuring the effect of social media on mental health in his research with meta-analysis and reached 244,676 people and reached the diversity mentioned above. Again, as mentioned above, multiple social media use was emphasised and focused only on facebook. In order to reach the correct correlation, research was carried out in more than one country in a wide geography. The findings obtained include the issues criticised in terms of meta-analysis and are considered sufficient to draw a safe conclusion. Here, the size of the correlation is important to measure the effect of social media on mental health. We reach another finding in the research: it is important to have a wide geography and more countries for correlation findings, but we should not ignore the number of publications in each country. The number of studies published in countries is as important as the data obtained from different countries. Even if researchers want to reach as many research results as they can, countries with insufficient number of studies may negatively affect the results. The language of publications is also important. In this study, only studies conducted in the English language were included. To better understand the difference between cultures and to reach clearer results, the question arises whether local languages should also be looked at and this has to be included in the correlation. On the other hand, the findings in the research should also be known in terms of age range and gender. For the idea that social media affects mental health, the addition of age ranges would make a positive contribution.
 

 

It is important to emphasise that we need to reveal a wide variety of problems before the meta-analysis technique is applied. As shown in the above-mentioned example, it is important to consider how broadly researchers conduct their research. Details such as the number of users, geography, country, language, age range, gender, how many hours a day social media use is important for the result of the research. The number of questions sought in the studies is as important as the study area and the number of studies examined.

In future studies, more comprehensive and systematic meta-analysis methods will reveal healthier results. Researchers should keep the language, geography and dates of publications wider and determine their criteria impartially. It is important that the researches are quality publications and have criteria supported by official institutions. Reports created by reliable institutions eliminate manipulation and bias. Researchers should prefer reliable publications instead of data commissioned by certain social media platforms (any institution that may create pressure and political suspicion). The use of multiple data in research will positively affect the results.
 

(ACADEMIA)