ChatGPT in Social Sciences: A literature review on the use of OpenAI artificial intelligence (AI) in qualitative and quantitative research
DOI:
https://doi.org/10.3145/thinkepi.2024.e18a04Keywords:
Artificial intelligence, ChatGPT, Research Methodologies, Systematic review, Qualitative methods, Quantitative methods, OpenAI, AIAbstract
This research note examines the impact of artificial intelligence (AI), particularly the ChatGPT model, on Social Science research. To this end, an exploratory systematic review is carried out, identifying the main academic articles published in 2023 on the OpenAI chatbot, and more specifically, on its use in research methods. The results confirm the existence of three types of thematic groups: (1) studies on ChatGPT and education, (2) studies on the use of the chatbot to support research, and (3) research on ChatGPT, information, and health. This work concludes by recognizing the importance of the responsible integration of AI into scientific research practices, emphasizing human-machine collaboration and critical thinking as fundamental oversight mechanisms.
References
Adams, Donnie; Chuah, Kee-Man; Devadason, Edward; Azzis, Muhammad-Shamshinor-Abdul (2023). “From novice to navigator: Students’ academic help-seeking behaviour, readiness, and perceived usefulness of ChatGPT in learning”. Education and Information Technologies, pp. 1-18. https://doi.org/10.1007/s10639-023-12427-8
Alanzi, Turki M. (2023). “Impact of Chatgpt on teleconsultants in healthcare: perceptions of healthcare experts in Saudi Arabia”. Journal of Multidisciplinary Healthcare, n. 16, pp. 2309–2321. https://doi.org/10.2147/JMDH.S419847
Allehyani, Sabha-Hakim; Algamdi, Mohammed-Abdullah (2023). “Digital competences: Early childhood teachers’ beliefs and perceptions of ChatGPT application in teaching English as a Second Language (ESL)”. International journal of learning, teaching and educational research, v. 22, n. 11, pp. 343-363. https://doi.org/10.26803/ijlter.22.11.18
Alqasham, Fahd-Hamad (2023). ChatGPT in the Saudi EFL classroom: A study of learner usage patterns and possibilities in learning optimization. Migration letters, v. 20, n. S7, pp. 1251-1263. https://doi.org/10.59670/ml.v20iS7.4828
Alshami, Ahmad; Elsayed, Moustafa; Ali, Eslam; Eltoukhy, Abdelrahman E. E.; Zayed, Tarek (2023). “Harnessing the power of ChatGPT for automating systematic review process: Methodology, case study, limitations, and future directions”. Systems, v. 11, n. 7. https://doi.org/10.3390/systems11070351
Al-Shboul, Mohammad-Khaled-Issa; Alwreikat, Asma; Alotaibi, Faiz-Abdullah (2023). “Investigating the use of ChatGpt as a novel method for seeking health information: A qualitative approach”. Science & technology libraries, pp. 1-10. https://doi.org/10.1080/0194262X.2023.2250835
Burger, Bastian; Kanbach, Dominik; Kraus, Sascha; Breier, Matthias; Corvello, Vincenzo (2023). “On the use of AI-based tools like ChatGPT to support management research”. European journal of innovation management, v. 26. n. 7, pp. 233-241. https://doi.org/10.1108/EJIM-02-2023-0156
Codina, Lluís; Lopezosa, Carlos; Freixa, Pere (2022). “Scoping reviews en trabajos académicos en comunicación: frameworks y fuentes”. En: Larrondo-Ureta, Ainara; Meso-Ayerdi, Koldo; Peña-Fernández, Simón (eds.). Información y Big Data en el sistema híbrido de medios-XIII Congreso Internacional de Ciberperiodismo. 15-17 nov 2021; País Vasco. [Leioa]: Universidad del País Vasco; pp. 67-85 Universidad del País Vasco. http://hdl.handle.net/10230/53155
Cox, Rachel; Hunt, Karen; Hill, Rebecca (2023). “Comparative analysis of NCLEX-RN questions: A duel between ChatGPT and human expertise”. Journal of nursing education, v. 62, n. 12, pp. 679-687. https://doi.org/10.3928/01484834-20231006-07
Chubb, Laura-Ann (2023). “Me and the machines: Possibilities and pitfalls of using artificial intelligence for qualitative data analysis”. International journal of qualitative methods, v. 22. https://doi.org/10.1177/16094069231193593
Chung, Eric M.; Zhang, Samuel C.; Nguyen, Anthony T.; Atkins, Katelyn M.; Sandler, Howard M.; Kamrava, Mitchell (2023). “Feasibility and acceptability of ChatGPT generated radiology report summaries for cancer patients”. Digital Health, v. 2023, n. 9. https://doi.org/10.1177/20552076231221620
Dengel, Andreas; Gehrlein, Rupert; Fernes, David; Görlich, Sebastian; Maurer, Jonas; Pham, Hai; Großmann, Gabriel; Eisermann, Niklas (2023). “Qualitative research methods for large language models: Conducting semi-structured interviews with ChatGPT and BARD on Computer Science Education”. Informatics, v. 10, n. 4. https://doi.org/10.3390/informatics10040078
D’Oria, Marika (2023). “Can AI language models improve human sciences research? A phenomenological analysis and future directions”. Encyclopaideia, v. 27, n. 66, pp. 77-92. https://doi.org/10.6092/issn.1825-8670/16554
ElSayary, Areej (2023). “An investigation of teachers’ perceptions of using ChatGPT as a supporting tool for teaching and learning in the digital era”. Journal of computer assisted learning, v. 2023, pp. 1–15. https://doi.org/10.1111/jcal.12926ELSAYARY15
Esmaeil, Ahmad-Azmi-Abdelhamid; Matanluk, Ovelyn-Oy; Marshall, Sanen (2023). “Understanding student perception regarding the use of ChatGPT in their argumentative writing: A qualitative inquiry”. Jurnal komunikasi: Malaysian journal of communication jilid, v. 39, n. 4, pp. 150-165. https://doi.org/10.17576/JKMJC-2023-3904-08
Giannakopoulos, Kostis; Kavadella, Argyro; Aaqel-Salim, Anas; Stamatopoulos, Vassilis; Kaklamanos, Eleftherios (2023). “Evaluation of the performance of generative AI large language models ChatGPT, Google Bard, and Microsoft Bing Chat in supporting evidence-based dentistry: Comparative mixed methods study”. Journal of medical internet research, v. 25, e51580. https://doi.org/10.2196/51580
Gilson, Aidan; Safranek, Conrad; Huang, Thomas; Socrates, Vimig; Chi, Ling; Taylor, Richard; Chartash, David (2023). “How does ChatGPT perform on the United States medical licensing examination? The implications of large language models for medical education and knowledge assessment”. JMIR medical education, v. 9, n. 1, e45312. https://doi.org/10.2196/45312
Gil de Zúñiga, Homero; Goyanes, Manuel; Durotoye, Timilehin (2023). “A scholarly definition of artificial intelligence (AI): advancing AI as a conceptual framework in communication research”. Political communication, v. 41, n. 2, pp. 317-334. https://doi.org/10.1080/10584609.2023.2290497
Hamilton, Leah; Elliott, Desha; Quick, Aaron; Smith, Simone; Choplin, Victoria (2023). “Exploring the use of AI in qualitative analysis: A comparative study of guaranteed income data”. International journal of qualitative methods, v. 22. https://doi.org/10.1177/1609406923120150
Jangjarat, Kris; Kraiwanit, Tanpat; Limna, Pongsakorn; Sonsuphap, Rattaphong (2023). “Public perceptions towards ChatGPT as the robot-assistant”. Online journal of communication and media technologies, v. 13, n. 3, e202337. https://ssrn.com/abstract=4480718
Khlaif, Zuheir; Mousa, Allam; Hattab, Muayad; Itmazi, Jamil; Hassan, Amjad; Sanmugam, Mageswaran; Ayyoub, AbdulKarem (2023) “The potential and concerns of using artificial intelligence in scientific research: The case of ChatGPT”. JMIR medical education, v. 9. http://doi.org/10.2196/47049
Kim, Pyoung-Won (2023). “A framework to overcome the dark side of generative artificial intelligence (GAI) like ChatGPT in social media and education”. IEEE transactions on computational social systems. https://doi.org/10.1109/TCSS.2023.3315237
Morgan, David L. (2023). “Exploring the use of artificial intelligence for qualitative data analysis: The case of ChatGPT”. International journal of qualitative methods, v. 22. https://doi.org/10.1177/16094069231211248
Mukred, Muaadh; Asma’mokhtar, Umi; Hawash, Burkan (2023). “Exploring the acceptance of ChatGPT as a learning tool among academicians: A qualitative study”. Jurnal komunikasi: Malaysian journal of communication Jilid, v. 39, n. 4, pp. 306-323. https://doi.org/10.17576/JKMJC-2023-3904-16
Niloy, Ahnaf; Akter, Salma; Sultana, Nayeema; Sultana, Jakia; Rahman, Sayed (2023). “Is Chatgpt a menace for creative writing ability? An experiment”. Journal of computer assisted learning, v. 2023, pp. 1-12. https://doi.org/10.1111/jcal.12929
Parker, Jessica L.; Richard, Veronica M.; Becker, Kimberly (2023). “Flexibility & iteration: Exploring the potential of large language models in developing and refining interview protocols”. The qualitative report, v. 28, n. 9, pp. 2772-2791. https://doi.org/10.46743/2160-3715/2023.6695
Parker, David; Mancini, Karen; Abram, Marissa (2023). “Natural language processing enhanced qualitative methods: An opportunity to improve health outcomes”. International journal of qualitative methods, v. 22. https://doi.org/10.1177/1609406923121414
Preiksaitis, Carl; Rose, Christian (2023). “Opportunities, challenges, and future directions of generative artificial intelligence in medical education: scoping review”. JMIR medical education, v. 9, e48785. https://doi.org/10.2196/48785
Ruiz-Estrada, Mario; Park, Donghyun; Staniewski, Marcin (2023). “Artificial intelligence (AI) can change the way of doing policy modelling”. Journal of policy modeling, v. 45, n. 6, pp. 1099-1112. https://doi.org/10.1016/j.jpolmod.2023.11.005
Remoto, Jocelyn (2023). “ChatGPT and other AIs: Personal relief and limitations among mathematics-oriented learners”. Environment and social psychology, v. 9, n. 1. https://doi.org/10.54517/esp.v9i1.1911
Švab, Igor; Klemenc-Ketiš, Zalika; Zupanič, Saša (2023). “New challenges in scientific publications: Referencing, artificial intelligence and ChatGPT”. Slovenian journal of public health, v. 62, n. 3, pp. 109-112. https://doi.org/10.2478/sjph-2023-0015
Yun, Ji-Young; Kim, Dong-Jin; Lee, Nara; Kim, Eun-Key (2023). “A comprehensive evaluation of ChatGPT consultation quality for augmentation mammoplasty: A comparative analysis between plastic surgeons and laypersons”. International journal of medical informatics, v. 179. https://doi.org/10.1016/j.ijmedinf.2023.105219
Villan, Fabiano; Santos, Renato P. (2023). “ChatGPT as Co-advisor in scientific initiation: Action research with project-based learning in elementary education”. arXiv preprint arXiv:2311.14701. https://doi.org/10.48550/arXiv.2311.14701
