ChatGPT Shows Promise as Teaching Tool
Artificial intelligence has been used as a tool and a toy, but could it be a great teacher, too? That is the hypothesis of researchers at the University of Iowa (UI) College of Engineering, who set out to study ways that AI could be a facilitator of learning and teaching.
In one test of AI’s ability to handle complex information, the researchers adapted the Fundamentals of Engineering (FE) exam to see how well ChatGPT, a popular chatbot program, would perform. According to the researchers, who have a paper outlining their results awaiting peer review, they “cannot claim for a fact that ChatGPT is or is not able to pass the FE Exam.” But their results do lead them to believe that not only can AI help optimize learning experiences for human students, it can become a valuable assistant for professional engineers.
“The recent developments and success around artificial intelligence and conversational chatbots like ChatGPT [spurred us to] test the current level of AI agents in supporting students in preparation for the exam,” said Ibrahim Demir, civil and environmental engineering associate professor at UI. “Our research lab focuses on novel applications of artificial intelligence in engineering and earth sciences, and next generation communication and visualization systems coupled with AI.”
The lab’s previous work, “AI-Enabled Intelligent Assistant (AIIA) for Personalized and Adaptive Learning in Higher Education,” shows AI's ability to engage with students interactively, the researchers noted. This prompted them to question if the technology could extend to understanding and solving complex engineering problems, which led to the new study, “Performance of ChatGPT on the US Fundamentals of Engineering Exam: Comprehensive Assessment of Proficiency and Potential Implications for Professional Environmental Engineering Practice,” which is currently under peer review ahead of publication.
Have A Listen: How AI Could Impact Manufacturing
The experiment found that ChatGPT-4 performed more impressively than earlier models of the system, scoring a 75.37 percent after modest prompt modifications—such as telling it to only pick from give multiple-choice answers or to explain its reasoning—so that the AI would answer in the expected format while genuinely understanding the questions. Without modifications, it scored 66.42 percent.
Because the FE test asks questions in a variety of formats, the team had to adapt the questions for ChatGPT’s text-based limitations. For point-and-click and drag-and-drop questions, they translated visual elements into text descriptions, effectively converting them into a format that ChatGPT could interpret and to which it could respond.
Though not exactly like the FE test, these modifications let the team assess ChatGPT's “reasoning and problem-solving capabilities within its current modality,” said graduate research assistant Vinay Pursnani. “Engineering encompasses a distinct blend of theoretical knowledge and applied mathematics. The absence of training prior to the study was a deliberate choice to benchmark the base model's performance. Comparatively, engineering disciplines may present more of a challenge due to the necessity for precise calculations and the application of complex principles, which differs from the broader knowledge recall and reasoning often assessed in fields like medicine or law.”
Become a Member: How to Join ASME
Broadly, the team’s work could help integrate other engineering tools and software with AI, develop and integrate AI study partners into the classroom, enhance accessibility and inclusion, and prevent cheating on engineering exams.
“Within educational settings, integrating AI into learning management systems like Canvas could unlock new heights of functionality and impact,” added assistant research scientist Yusuf Sermet. This could “create a cohesive ecosystem where AI not only complements but significantly amplifies the capabilities of traditional educational and engineering tools.”
AI can serve as a virtual tutor, Sermet said, “providing personalized feedback and enabling interactive learning experiences that adapt to individual student needs. In practical settings, such as internships or labs, AI can offer real-time guidance and support, simulating scenarios that students may encounter in the field and preparing them for real-world challenges.”
And AI can also help engineers long after they’ve passed the FE exam.
The team is also working on a National Science Foundation project to develop AI Hubs that integrate real-time hydrological data sources and flood risk maps to create specialized AI agents that can also mirror the roles and expertise of professionals such as floodplain managers, scientists, and researchers, Demir explained.
Energy Blog: AI Propels Sustainable Solutions
“The interplay between these AI agents facilitates a new frontier of decision-making and policy simulation, enabling complex, multidisciplinary trade-offs and social experimentation,” Demir said. “This approach exemplifies how AI models can evolve to become more effective and applicable over time, ensuring that their integration into real-world scenarios remains valid and impactful, irrespective of advancements in underlying AI technology.”
Demir added that while AI can automate certain tasks, it remains a tool that must be directed and interpreted by human experts. "Engineering is a profession deeply rooted in innovation, problem-solving, and creativity—qualities that AI is currently unable to replicate to the full extent of a human engineer,” he said.
Therefore, AI should be viewed “as a collaborative tool that can enhance the engineer's capabilities, not replace them,” Demir said. “It's the synergy between human expertise and AI's computational power that will define the future of the profession.”
Eydie Cubarrubia is an independent writer in New York City.
In one test of AI’s ability to handle complex information, the researchers adapted the Fundamentals of Engineering (FE) exam to see how well ChatGPT, a popular chatbot program, would perform. According to the researchers, who have a paper outlining their results awaiting peer review, they “cannot claim for a fact that ChatGPT is or is not able to pass the FE Exam.” But their results do lead them to believe that not only can AI help optimize learning experiences for human students, it can become a valuable assistant for professional engineers.
“The recent developments and success around artificial intelligence and conversational chatbots like ChatGPT [spurred us to] test the current level of AI agents in supporting students in preparation for the exam,” said Ibrahim Demir, civil and environmental engineering associate professor at UI. “Our research lab focuses on novel applications of artificial intelligence in engineering and earth sciences, and next generation communication and visualization systems coupled with AI.”
The lab’s previous work, “AI-Enabled Intelligent Assistant (AIIA) for Personalized and Adaptive Learning in Higher Education,” shows AI's ability to engage with students interactively, the researchers noted. This prompted them to question if the technology could extend to understanding and solving complex engineering problems, which led to the new study, “Performance of ChatGPT on the US Fundamentals of Engineering Exam: Comprehensive Assessment of Proficiency and Potential Implications for Professional Environmental Engineering Practice,” which is currently under peer review ahead of publication.
Have A Listen: How AI Could Impact Manufacturing
The experiment found that ChatGPT-4 performed more impressively than earlier models of the system, scoring a 75.37 percent after modest prompt modifications—such as telling it to only pick from give multiple-choice answers or to explain its reasoning—so that the AI would answer in the expected format while genuinely understanding the questions. Without modifications, it scored 66.42 percent.
Because the FE test asks questions in a variety of formats, the team had to adapt the questions for ChatGPT’s text-based limitations. For point-and-click and drag-and-drop questions, they translated visual elements into text descriptions, effectively converting them into a format that ChatGPT could interpret and to which it could respond.
Though not exactly like the FE test, these modifications let the team assess ChatGPT's “reasoning and problem-solving capabilities within its current modality,” said graduate research assistant Vinay Pursnani. “Engineering encompasses a distinct blend of theoretical knowledge and applied mathematics. The absence of training prior to the study was a deliberate choice to benchmark the base model's performance. Comparatively, engineering disciplines may present more of a challenge due to the necessity for precise calculations and the application of complex principles, which differs from the broader knowledge recall and reasoning often assessed in fields like medicine or law.”
Become a Member: How to Join ASME
Broadly, the team’s work could help integrate other engineering tools and software with AI, develop and integrate AI study partners into the classroom, enhance accessibility and inclusion, and prevent cheating on engineering exams.
“Within educational settings, integrating AI into learning management systems like Canvas could unlock new heights of functionality and impact,” added assistant research scientist Yusuf Sermet. This could “create a cohesive ecosystem where AI not only complements but significantly amplifies the capabilities of traditional educational and engineering tools.”
AI can serve as a virtual tutor, Sermet said, “providing personalized feedback and enabling interactive learning experiences that adapt to individual student needs. In practical settings, such as internships or labs, AI can offer real-time guidance and support, simulating scenarios that students may encounter in the field and preparing them for real-world challenges.”
And AI can also help engineers long after they’ve passed the FE exam.
The team is also working on a National Science Foundation project to develop AI Hubs that integrate real-time hydrological data sources and flood risk maps to create specialized AI agents that can also mirror the roles and expertise of professionals such as floodplain managers, scientists, and researchers, Demir explained.
Energy Blog: AI Propels Sustainable Solutions
“The interplay between these AI agents facilitates a new frontier of decision-making and policy simulation, enabling complex, multidisciplinary trade-offs and social experimentation,” Demir said. “This approach exemplifies how AI models can evolve to become more effective and applicable over time, ensuring that their integration into real-world scenarios remains valid and impactful, irrespective of advancements in underlying AI technology.”
Demir added that while AI can automate certain tasks, it remains a tool that must be directed and interpreted by human experts. "Engineering is a profession deeply rooted in innovation, problem-solving, and creativity—qualities that AI is currently unable to replicate to the full extent of a human engineer,” he said.
Therefore, AI should be viewed “as a collaborative tool that can enhance the engineer's capabilities, not replace them,” Demir said. “It's the synergy between human expertise and AI's computational power that will define the future of the profession.”
Eydie Cubarrubia is an independent writer in New York City.
Discover and explore content covering the latest trends and developments in digital engineering including artificial intelligence, IoT, Big Data, digital twin, simulation, and computer aided design.