Pressure because of AACSB accreditation

The third development initiator was that the university applied for AACSB accreditation, which required us to be very serious about the operations in education and the quality assurance system. In each program, we needed to look through the training and outcome requirements and a system was established to measure at the end of the MA whether or not students acquired the expected knowledge and skills. It was a huge and very complex task, and we had to rethink the main goals of the whole program. We have identified the four most important learning goals of the program and assigned two or three additional objectives to each learning goal. This has ultimately led to a quality assurance system that is capable of testing the fulfilment of training and outcome requirements. We have assigned two observable criteria to each of the nine objectives, for which a data collection and measurement system had to be built up. By continuously collecting the data on the 18 observable criteria (Figure 2) during the program and analysing the measures at the end, we could determine the performance of each student and the extent to which the outgoing students meet the TOR. This new system required a rethinking of the program to determine exactly what the goal of the education was and to assign clear measurement to it.

Figure 2 Assurance of learning system at SCM MA

Source: own edition

As a result of the process, the definition of student performance, the goals of the training and the measure of efficiency have become more transparent. The first test of the measurement was carried out in the spring of 2018 by examining the results of 20 randomly selected MA students, and based on these, we initiated developments. At the next measurement, in the summer of 2018, we used the improved system, e.g. we have fine-tuned the system and also modified some objectives. Using the summer experiences, we

have started new developments that will be cyclically performed (continuous development, Slack, Lewis, 2002) and delivering results to education development.

3. Results

In this section, we review the results of the development process. First, we summarize the achievement of the two Pilot courses. During the Logistics Services and Distribution courses, the development process has primarily helped to understand the relationship between training and outcome requirements, student activities and the assessment system: how important are they, how they can be built upon each other, interrelationship, and that transparency significantly increases student acceptance and effectiveness. The results of the pilot developments can be demonstrated both in the increase in student performance (final grade) and in the students' opinions on the course (Table 1).

Student performance was measured on a 0-100% scale, while the University organizes the course evaluation in a centralized way (HALVEL, 2017, 2018). The course evaluation process is carried out at the end of the semesters, and students are requested to fill in an online questionnaire about the course. They evaluate not only the teacher, the teaching material, but their attitude and student activities, too. The method of evaluation is to assess aspects on a 5-point Likert-scale, where 1 means totally disagree or bad answer, and 5 means absolutely agree or excellent opinion.

Table 1 Results of course development in course evaluation and student performance

2017 spring 2017 fall 2018 spring (PILOT)

* average of 4 aspects on a 5-point scale (where 1=bad, 5=excellent) Source: Own edition based on HALVEL data for 2017 and 2018

The experience was so favourable that, in the light of this, the development of other compulsory SCM MA specialized courses has also been carried out, and significant educational material development will be realized by 2020. The EU grant offers us to develop nine traditional case studies, eight cooperative cases with videos and interviews at companies, fourteen illustrations videos and 11 online and offline test bank and exercise book collections.

In the Assurance of Learning system, which measures the realization of TOR, we set out four substantial learning goals. The first of these is that students completing our program should have a comprehensive knowledge of the theoretical and practical aspects of supply chain management, and secondly that they can identify problems and formulate solutions and to implement a system-based analysis. Thirdly, students graduating in our program should be critical of their own and their colleagues' work and strive for quality work. Fourth, graduates must be able to formulate their own independent thoughts both in text and presentation form. These four goals were broken down into a further nine objectives, which were measured by two metrics. It can be stated that 36% of our students are over the expectation, 56% are average, and only 8% are below expectations.

4. Discussion

In the discussion, we would like to evaluate the results of the analysis carried out and to summarize the strengths and weaknesses of the development and the opportunities to exploit and threats to avoid. Our main strength during the development process was definitely that we were able to work in a very thoroughly structured course structure, so we do not have to think about whether our subjects are in the right place. The development team had a great deal of belief in the work, and the Pilot subjects’ instructors were both committed to the improvement. The pedagogical support and training provided by the Department of Pedagogy to the team to guide the development in the right direction were essential. In such a development, it is crucial to have committed management, engaged to change. During the development, the views and feedback of the dual partners were very useful as they are the ones who employ our students and their expectations fundamentally influence the students' success in work and the labour market.

The attitudes of colleagues in the second phase of the development have been revealed as a weakness because unfortunately, not everyone is thinking service-oriented and student-oriented, when redefining a course. Although development has been done in every SCM MA-special compulsory courses, in practice, it is not realized as it should because many colleagues stick to their old teaching methods. We also formulated as a

weakness that the colleagues, who teach management, does not have pedagogical qualifications (just like in many other management education fields) that are very lacking when it comes to thinking about student activities and evaluation systems, or the amount of credit equivalent to work.

We see an as opportunity that, due to AACSB accreditation, these developments had to be carried out, and so it was organized with pedagogical support. It is also a great opportunity to provide a framework to use EU funding effectively to develop high-quality teaching materials. Another possibility is that the students gave excellent feedbacks after the courses, so they experienced the benefits of the developments.

We have identified as a threat that there is always a danger that the ourse leader will return to his/her well-established, old teaching routine and use obsolete, not necessarily student-oriented, and not the most modern methods. Another threat is that these new learning outcomes oriented teaching methods, class materials, and most importantly, the evaluation system requires much more effort from the instructor, which may not be convenient for all instructors.

5. Conclusion

Our achievements are that we could develop the MA program based on comprehensive research and could find developing methods and tools that best serve our students' knowledge and skills. The success of this was examined through an Assurance of Learning quality assurance system, which we develop continuously. The outlined model can be a useful example for other Universities to analyse and develop their higher education programs systematically.

The study concludes with the following conclusions. An education program is essential to meet the training and outcome requirements, and it is necessary to harmonize these and the courses performed during the education program, regularly. It is crucial to define the learning activities through which we pass knowledge, develop skills, and an evaluation system that meets these training and outcome requirements. It is also true, however, that such a development process takes a long time and requires very close cooperation between the teachers involved in the higher education program to become a continuous development activity. Learning outcome-based education requires a new approach and attitude from the instructors: this is service orientation and student orientation. Managerial support, reconciliation of different perspectives and the internal need for development are essential to the success of the development. Corporate feedback was also a great help; it broadcasted market impulses, which, in our case, was

the feedback from dual partners. We also started to develop in a lucky time, because we were able to do this with financial support, and therefore we have the opportunity to develop an abundant teaching material that can significantly increase the quality of our education program. The most important lesson of development was that this development never ends; it is a cyclical process that we will have to do over and over again, every few years.

References

HALVEL Student course evaluation (2017 and 2018). Corvinus University of Budapest, internal database. Available at:

http://halvel.uni-corvinus.hu/index.php?lang=en&page=kezdo&PHPSESSID=a32d2979ee3d48cd2d 663daffe33c4a9

Slack, N. and Lewis, M. (2008): Operations strategy. Harlow: FT Prentice Hall Training and Outcome Requirements (TOR, 2016): Decree of the Ministry of Human

Resources. Available at:

https://net.jogtar.hu/jogszabaly?docid=A1600018.EMM&timeshift=20160813&txtref erer=00000001.txt

Subjective factors of course evaluation. Can we rely on

undergraduates’ opinion?