Rubrics for Engineering Education

The awareness of incorporating global competency within the engineering programme is gaining increasing popularity. Global competency is the awareness and interest in learning about the world and its function. Although assessment of such competency within engineering programmes has brought attention among scholars, studies featuring exhaustive methods for assessing competencies with respect to professional practice within the academic discipline have been minimal. Aspects of global competency have also been specified in the ABET outcome criteria.

ABET Program Outcomes:

1. an ability to apply knowledge of mathematics, science and engineering
2. an ability to design and conduct experiments, as well as to analyze and interpret data
3. an ability to design a system, component, or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability
4. an ability to function on multidisciplinary teams
5. an ability to identify, formulate, and solve engineering problems
6. an understanding of professional and ethical responsibility
7. an ability to communicate effectively (both oral and written)
8. the broad education necessary to understand the impact of engineering solutions in a global, economic, environmental, and societal context
9. a recognition of the need for, and an ability to engage in life-long learning
10. a knowledge of contemporary issues
11. an ability to use the techniques, skills, and modern engineering tools necessary for engineering practice.

Given the importance of incorporating global competency within engineering programs, the following study conducted by a professor from California Polystate University (DeTurris, 2012) aims to develop rubrics that assess global competency in relation to the ABET outcomes namely (c), (h), (j) and (k). The proposed rubrics encompass both the technical and professional skills essential in assessing global competency with respect to the four outcomes. It also includes a spectrum for attitudes, knowledge and skills, an examination of an internal frame of reference and behavioral observation. The skills in relation to global competency are categorized in terms of awareness, perspectives, and participation for each of the four outcomes with an expanding scale of capability.

References:

• DeTurris, J. D. (2012, April). Assessment rubrics for global competency in engineering education. Paper presented at the Proceedings of the 2012 ASEE PSW Conference, San Luis Obispo, California.

The study conducted at Iowa State University's Faculty of Aerospace Engineering and Engineering Mechanics (Kellog, Mann, & Dieterich, 2001) discusses the process of developing and refining the rubrics for engineering design courses and laboratory courses. Apart from the development and refinement of the rubrics, the discussion also covers observations and feedback from faculty and teaching assistants using the rubrics and the results from the student summative survey data, which includes the implemented changes that address student concerns.

Developing rubrics are never an easy task because the process involves a lot of trial and error, which challenges the developer's patience. Besides developing rubrics, refinement is also crucial. A good indicator suggesting rubrics need refinement is when the teacher feels that the best piece of work is not receiving the best grades. Although developing and refining rubrics are exhaustive, the results from the study revealed that faculty and teaching assistants all appreciate the use of the rubrics as a way to ensure that the grading are unified and to describe standards for completing assignments. However the summative survey results from students revealed a mixed response about the use of rubrics. The seniors were positive about the rubrics in which they even asked for rubrics, whereas sophomores were less pleased with the rubrics.

The hypothesis for such varied response is because the seniors have developed certain familiarity with rubrics, thus the direction offered by the rubrics could be easily interpreted into their actions. In addition, the seniors considered the rubrics invaluable because it provided them guidance on how to document their work. In contrast with the seniors, the sophomores had the tendency to perceive the rubrics to be a checklist for their laboratory report that was used to punish them. Many of the responses were very performance-oriented. They felt that the rubrics aimed at providing them a guideline, which directs them how to do something rather than providing them with examples of what they should do. Some responses stated that they believed they had met the criteria in the rubrics, yet they still received poor grades.

The following are summative observations from the faculty in their development and implementation of the rubrics.

(1) It is observed that the key factor to the success or failure of the rubrics used in the laboratory course depends on how the teacher applies the rubrics and how well students were educated on the use of the rubrics. Seniors did not need much guidance and discussion about the rubrics compared to the sophomores.

(2) Another key to its success is students' experience with the material. Students who have not had the experience of writing an assignment like technical reports should be offered materials like sample reports or checklists along with the rubrics so that they can understand what is expected with an assignment.

(3) The faculty refined the rubrics as the semester progressed by changing the weighting of the objectives. This emphasized the higher-level skills, the quality of content as the semester proceeded and the students' mastery of the "mechanical aspects of reporting technical information" (Kellog et al., 2001, p.8).

(4) Collaboration among teachers in the development and implementation of the rubrics seems to be important in standardizing grading. For design courses, the teachers and the teaching assistant discussed the rationale behind the objectives and criteria for the rubrics. Examples of evaluated reports using the rubrics were given to the teaching assistants. However for the laboratory courses, such measure has not been adopted. Thus students have commented about the inconsistency of grading in the laboratory courses with different teachers using the rubrics.

(5) When students become accustomed to the use of rubrics, they can provide invaluable feedback in the refinement process of the rubrics. The students in each of the courses provided sufficient feedback and opinions from the summative survey results. Some of which facilitated the refinement process. For instance it was evident, that sophomore students needed extra support and detail on the use of rubrics.

References:

• Kellog, R. S., Mann. J. A., & Dieterich, A. (2001, June). Developing and using rubrics to evaluate subjective engineering laboratory and design projects. Paper presented at the 108^th ASEE Annual Conference and Exposition, Albuquerque, New Mexico.

McMartin, McKenna, & Youseffi (2000) describe the use of a scenario assignment in teaching non-freshman students in a Mechanical Engineering course at the University of California, Berkeley. The scenario assignment is basically a qualitative performance assessment tool created to assess students' knowledge of teamwork, engineering practices, and problem solving. Students were offered the scenario to describe a "day in the life" problem faced by engineers. Students were asked to describe the process or plan they would adopt in finding the solution to a technical or design problem as a team instead of just solving the problem presented in the scenario in terms of analyzing appropriate models, running simulations, and converging on a correct recommendation.

Analytic and holistic rubrics were developed to assess students' knowledge with respect to the learning outcomes associated with the scenario assignment. Initial findings suggest that the scoring of the scenarios using analytic rubrics facilitated faculty in figuring out students' strengths and weaknesses quickly. In addition to figuring out the strengths and weaknesses, the analytic rubrics can also assist the faculty in adapting their course to address the areas where students need attention. For holistic rubrics, the rubrics can be easily used to assess the changes in students' learning and development over time and across a curriculum. However, the holistic rubrics fail to provide definitive details regarding the achievement of learning outcomes i.e. the ability to solve open-ended problems or the ability to work in an inter-disciplinary team. Therefore the creators of the rubrics took the initiation of developing analytic rubrics to resolve the problems in holistic rubrics. Figures 1 and 2 presented below shows the rubrics the creators have developed for the scenario assignment.

Figure 1: An example of holistic rubrics for the scenario assignment (Accessed from McMartin et al., 1999)

Scoring guide:
"1" = incompetent: fails to meet criteria;
"2" = limited: meets few criteria to a limited degree;
"3" = adequate: meets some criteria to an adequate degree;
and "4" = sophisticated: meets most of all criteria to an exceptional degree.

Figure 2: An example of analytic rubrics for the scenario assignment with a focus on criteria (d) (Accessed from McMartin et al., 1999)

References:

• McMartin, F., McKenna, A., & Youssefi, K. (1999, November). Establishing the trustworthiness of scenario assignments as assessment tools for undergraduate engineering education. Paper presented at the 29^th ASEE/IEEE Frontiers in Education Conference, San Juan, Puerto Rico.

• McMartin, F., McKenna, A., & Youssefi, K. (2000). Scenario assignments as assessment tools for undergraduate engineering education. Education, IEEE Transactions, 43(2), 111-119.