Using network analysis for rapid, transparent, and rigorous thematic analysis: A case study of online distance learning

Yosep Dwi Kristanto; Russasmita Sri Padmi

doi:10.21831/pep.v24i2.33912

Using network analysis for rapid, transparent, and rigorous thematic analysis: A case study of online distance learning

Yosep Dwi Kristanto, Universitas Sanata Dharma, Indonesia
Russasmita Sri Padmi, SEAMEO QITEP in Mathematics, Yogyakarta, Indonesia

10.21831/pep.v24i2.33912

Abstract

In thematic analysis, themes construction can be performed manually by the researcher or automatically by a computer. Both methods have strengths and weaknesses. This article introduces a strategy that involves the role of both researcher and computer to construct themes from qualitative data in a rapid, transparent, and rigorous manner. The strategy uses network analysis and is demonstrated by employing a case study on students’ perceptions of online distance learning they experienced during the COVID-19 pandemic. The themes-construction strategy consists of four systematic phases, namely (1) determining unit of analysis and coding; (2) constructing the code co-occurrence matrix; (3) conducting network analysis; and (4) generating, reviewing, and reporting the themes. The strategy is successfully demonstrated in generating themes from the data with modularity value Q = 0.34. The application of network analysis in this strategy allows researchers to automatically generate themes from qualitative data using mathematical algorithms, represent these themes visually using network graph, and interpret the themes to answer the research questions.

Keywords

qualitative data analysis; thematic analysis; network analysis; online distance learning; covid-19

Full Text:

PDF

References

Aguinis, H., & Solarino, A. M. (2019). Transparency and replicability in qualitative research: The case of interviews with elite informants. Strategic Management Journal, 40, 1291–1315. https://doi.org/10.1002/smj.3015

Attride-Stirling, J. (2001). Thematic networks: An analytic tool for qualitative research. Qualitative Research, 1(3), 385–405. https://doi.org/10.1177/146879410100100307

Bastian, M., Heymann, S., & Jacomy, M. (2009). Gephi: An open source software for exploring and manipulating networks. International AAAI Conference on Weblogs and Social Media. http://www.aaai.org/ocs/index.php/ICWSM/09/paper/view/154

Blondel, V. D., Guillaume, J.-L., Lambiotte, R., & Lefebvre, E. (2008). Fast unfolding of communities in large networks. Journal of Statistical Mechanics: Theory and Experiment, 2008(10), P10008. https://doi.org/10.1088/1742-5468/2008/10/P10008

Brandes, U., Delling, D., Gaertler, M., Gorke, R., Hoefer, M., Nikoloski, Z., & Wagner, D. (2008). On modularity clustering. IEEE Transactions on Knowledge and Data Engineering, 20(2), 172–188. https://doi.org/10.1109/TKDE.2007.190689

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Braun, V., & Clarke, V. (2012). Thematic analysis. In H. Cooper, P. M. Camic, D. L. Long, A. T. Panter, D. Rindskopf, & K. J. Sher (Eds.), APA handbook of research methods in psychology, Vol 2: Research designs: Quantitative, qualitative, neuropsychological, and biological. (pp. 57–71). American Psychological Association. https://doi.org/10.1037/13620-004

Brennan, L., Cusack, T., Delahunt, E., Kuznesof, S., & Donnelly, S. (2019). Academics’ conceptualisations of the research-teaching nexus in a research-intensive Irish university: A dynamic framework for growth & development. Learning and Instruction, 60, 301–309. https://doi.org/10.1016/j.learninstruc.2017.10.005

Bringer, J. D., Johnston, L. H., & Brackenridge, C. H. (2006). Using computer-assisted qualitative data analysis software to develop a grounded theory project. Field Methods, 18(3), 245–266. https://doi.org/10.1177/1525822X06287602

Burgess-Allen, J., & Owen-Smith, V. (2010). Using mind mapping techniques for rapid qualitative data analysis in public participation processes. Health Expectations, 13(4), 406–415. https://doi.org/10.1111/j.1369-7625.2010.00594.x

Burns, S., & Schubotz, D. (2009). Demonstrating the merits of the peer research process: A Northern Ireland case study. Field Methods, 21(3), 309–326. https://doi.org/10.1177/1525822X09333514

Chandra, Y., & Shang, L. (2019). An overview of R and RQDA: An open-source CAQDAS platform. In Qualitative research using R: A systematic approach (pp. 47–51). Springer Singapore. https://doi.org/10.1007/978-981-13-3170-1_4

Chen, K., Zhang, Y., & Fu, X. (2019). International research collaboration: An emerging domain of innovation studies? Research Policy, 48(1), 149–168. https://doi.org/10.1016/j.respol.2018.08.005

Clarke, V., & Braun, V. (2013). Teaching thematic analysis: Overcoming challenges and developing strategies for effective learning. The Psychologist, 26(2), 120–123. https://uwe-repository.worktribe.com/output/937596/teaching-thematic-analysis-overcoming-challenges-and-developing-strategies-for-effective-learning

Drijvers, P., Grauwin, S., & Trouche, L. (2020). When bibliometrics met mathematics education research: the case of instrumental orchestration. ZDM. https://doi.org/10.1007/s11858-020-01169-3

Farrell, O., & Seery, A. (2019). “I am not simply learning and regurgitating information, I am also learning about myself”: learning portfolio practice and online distance students. Distance Education, 40(1), 76–97. https://doi.org/10.1080/01587919.2018.1553565

Fielding, N. G., & Lee, R. M. (2002). New patterns in the adoption and use of qualitative software. Field Methods, 14(2), 197–216. https://doi.org/10.1177/1525822X02014002005

Frandsen, T. (2017). Evolution of modularity literature: A 25-year bibliometric analysis. International Journal of Operations & Production Management, 37(6), 703–747. https://doi.org/10.1108/IJOPM-06-2015-0366

Galdas, P. (2017). Revisiting bias in qualitative research. International Journal of Qualitative Methods, 16(1), 160940691774899. https://doi.org/10.1177/1609406917748992

Good, B. H., de Montjoye, Y.-A., & Clauset, A. (2010). Performance of modularity maximization in practical contexts. Physical Review E, 81(4), 046106. https://doi.org/10.1103/PhysRevE.81.046106

Hannes, K., & Macaitis, K. (2012). A move to more systematic and transparent approaches in qualitative evidence synthesis: Update on a review of published papers. Qualitative Research, 12(4), 402–442. https://doi.org/10.1177/1468794111432992

Harary, F. (2018). Graph theory. CRC Press.

Haskins Lisle, A., Merenda, C., & Gabbard, J. (2020). Using affinity diagramming to generate a codebook: a case study on young military veterans and community reintegration. Qualitative Research, 20(4), 396–413. https://doi.org/10.1177/1468794119851324

Holloway, I., & Todres, L. (2003). The status of method: Flexibility, consistency and coherence. Qualitative Research, 3(3), 345–357. https://doi.org/10.1177/1468794103033004

Jackson, K., & Bazeley, P. (2019). Qualitative data analysis with NVivo. SAGE.

Jackson, K. M., & Trochim, W. M. K. (2002). Concept mapping as an alternative approach for the analysis of open-ended survey responses. Organizational Research Methods, 5(4), 307–336. https://doi.org/10.1177/109442802237114

Jamieson, S. (2016). Analyse qualitative data. Education for Primary Care, 27(5), 398–402. https://doi.org/10.1080/14739879.2016.1217430

John, W. S., & Johnson, P. (2000). The pros and cons of data analysis software for qualitative research. Journal of Nursing Scholarship, 32(4), 393–397. https://doi.org/10.1111/j.1547-5069.2000.00393.x

Kessler, M. M. (1963). Bibliographic coupling between scientific papers. American Documentation, 14(1), 10–25. https://doi.org/10.1002/asi.5090140103

Kiger, M. E., & Varpio, L. (2020). Thematic analysis of qualitative data: AMEE Guide No. 131. Medical Teacher, 1–9. https://doi.org/10.1080/0142159X.2020.1755030

King, A. (2010). ‘Membership matters’: Applying membership categorisation analysis (MCA) to qualitative data using computer‐assisted qualitative data analysis (CAQDAS) software. International Journal of Social Research Methodology, 13(1), 1–16. https://doi.org/10.1080/13645570802576575

King, N., & Brooks, J. (2018). Thematic analysis in organisational research. In C. Cassell, A. L. Cunliffe, & G. Grandy (Eds.), The SAGE handbook of qualitative business and management research methods: Methods and challenges (pp. 219–236). SAGE Publications Ltd. https://doi.org/10.4135/9781526430236.n14

Kristanto, Y. D., & Santoso, E. B. (2020). Towards a mathematics textbook for supporting 21st century learning: The student perspective. Journal of Physics: Conference Series, 1657, 012037. https://doi.org/10.1088/1742-6596/1657/1/012037

Kristanto, Y. D. (2018). Technology-enhanced pre-instructional peer assessment: Exploring students’ perceptions in a Statistical Methods course. REiD (Research and Evaluation in Education), 4(2), 105–116. https://doi.org/10.21831/reid.v4i2.20951

Kristanto, Y. D., Utomo, B., & Sulistyani, N. (2020). Towards instructional design principles for online distance learning. https://doi.org/10.17605/OSF.IO/S2CN8

LeCompte, M. D., & Schensul, J. J. (1999). Analyzing & interpreting ethnographic data. AltaMira Press.

Lee, V., Coombe, L., Mahoney, R., Allen, C., & Robinson, P. (2018). Incorporating indigenous and non-indigenous worldviews through innovative text analysis: An evaluation of indigenous public health curricula. International Journal of Qualitative Methods, 17(1), 160940691881937. https://doi.org/10.1177/1609406918819377

Littenberg-Tobias, J., & Reich, J. (2020). Evaluating access, quality, and equity in online learning: A case study of a MOOC-based blended professional degree program. The Internet and Higher Education, 47, 100759. https://doi.org/10.1016/j.iheduc.2020.100759

Lund, K., Chen, B., & Grauwin, S. (2018). The potential of interdisciplinary in MOOC research. Proceedings of the Fifth Annual ACM Conference on Learning at Scale, 1–10. https://doi.org/10.1145/3231644.3231661

McCain, K. W. (1990). Mapping authors in intellectual space: A technical overview. Journal of the American Society for Information Science, 41(6), 433–443. https://doi.org/10.1002/(SICI)1097-4571(199009)41:6<433::AID-ASI11>3.0.CO;2-Q

Newman, M. E. J. (2004). Analysis of weighted networks. Physical Review E, 70(5), 056131. https://doi.org/10.1103/PhysRevE.70.056131

Newman, M. E. J. (2006). Modularity and community structure in networks. Proceedings of the National Academy of Sciences, 103(23), 8577–8582. https://doi.org/10.1073/pnas.0601602103

Newman, M. E. J. (2010). Networks: An introduction. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199206650.001.0001

Nowell, L. S., Norris, J. M., White, D. E., & Moules, N. J. (2017). Thematic analysis: Striving to meet the trustworthiness criteria. International Journal of Qualitative Methods, 16(1), 1–13. https://doi.org/10.1177/1609406917733847

Opsahl, T., Agneessens, F., & Skvoretz, J. (2010). Node centrality in weighted networks: Generalizing degree and shortest paths. Social Networks, 32(3), 245–251. https://doi.org/10.1016/j.socnet.2010.03.006

Pokorny, J. J., Norman, A., Zanesco, A. P., Bauer-Wu, S., Sahdra, B. K., & Saron, C. D. (2018). Network analysis for the visualization and analysis of qualitative data. Psychological Methods, 23(1), 169–183. https://doi.org/10.1037/met0000129

Rose, J., & Johnson, C. W. (2020). Contextualizing reliability and validity in qualitative research: toward more rigorous and trustworthy qualitative social science in leisure research. Journal of Leisure Research, 51(4), 432–451. https://doi.org/10.1080/00222216.2020.1722042

Ryan, G. W., & Bernard, H. R. (2003). Techniques to identify themes. Field Methods, 15(1), 85–109. https://doi.org/10.1177/1525822X02239569

Salmona, M., Lieber, E., & Kaczynski, D. (2019). Qualitative and mixed methods data analysis using Dedoose: A practical approach for research across the social sciences. SAGE.

Scott, J., & Carrington, P. (2014). The SAGE handbook of social network analysis. SAGE Publications Ltd. https://doi.org/10.4135/9781446294413

Small, H. (1973). Co-citation in the scientific literature: A new measure of the relationship between two documents. Journal of the American Society for Information Science, 24(4), 265–269. https://doi.org/10.1002/asi.4630240406

Stopar, K., & Bartol, T. (2019). Digital competences, computer skills and information literacy in secondary education: mapping and visualization of trends and concepts. Scientometrics, 118(2), 479–498. https://doi.org/10.1007/s11192-018-2990-5

Thompson, R. (2002). Reporting the results of computer-assisted analysis of qualitative research data. Forum: Qualitative Social Research, 3(2), Art. 25. https://doi.org/10.17169/fqs-3.2.864

Tuval-Mashiach, R. (2017). Raising the curtain: The importance of transparency in qualitative research. Qualitative Psychology, 4(2), 126–138. https://doi.org/10.1037/qup0000062

Watkins, D. C. (2017). Rapid and rigorous qualitative data analysis. International Journal of Qualitative Methods, 16(1), 160940691771213. https://doi.org/10.1177/1609406917712131

Zhao, D., & Logan, E. (2002). Citation analysis using scientific publications on the web as data source: A case study in the XML research area. Scientometrics, 54(3), 449–472. https://doi.org/10.1023/A%3A1016090601710

DOI: https://doi.org/10.21831/pep.v24i2.33912