The existing research findings from four state universities (see Pecakovska, 2015 for details) showed partial compliance of the QA related policies and practices with all seven ENQA ESG standards (ESG 2005) on internal quality assurance, as well as the incompliance with two and partial compliance with six out of eight ESGs on external QA.
The findings also showed incompliance with 3 and partial compliance at 5 out of 8 ENQA standards on QA of the Higher Education Accreditation and Evaluation Board (HEAEB).
The matrix given below summarizes the level of compliance with the first version of ESG 2005 against the four descriptors that have been used by the ENQA review panels in their final reports on the compliance of the Agencies with ENQA membership criterion / ESG standards: full compliant, substantial compliant, partially compliant or no compliant.
Notwithstanding, the situation has not been changed much, and the findings remain equally relevant to date.
The findings related to internal quality suggested excessive bureaucracy and time-consuming procedures that lack mechanisms for internal approval, monitoring and periodic review of study programs and qualifications. The students are insufficiently involved in QA processes, particularly not consulted for the content of the program and curriculum design, whereas the universities very little or insufficiently take into consideration the opinion of students for improving the conditions of student services and their opinion on the performance of the teaching staff sought through students’ surveys. Over half of the respondents agree that the students are assessed by publicly announced criteria opinion is sought through students’ surveys on the performance of the teaching staff. The self-evaluation is applied in a very legalistic manner and not as an essential I instrument to achieve better quality, while the universities have not gone far beyond formal and obligatory responses to the requirements of external quality assurance.
Table 1.1. : Matrix of compliance with ENQA european standards and guidelines for quality assurance (ESG 2005)
Standard Conclusion
Part 1. European Standards for the Internal QA of Higher Education Institutions
ESG 1. 1. Policy and procedures for quality assurance Partially compliant ESG 1.2. Approval, monitoring and periodic review of programs and awards Partially compliant
ESG 1.3. Assessment of students Partially compliant
ESG 1.4. Quality assurance of teaching staff Partially compliant ESG 1.5. Learning resources and student support Partially compliant
ESG 1.6. Information systems Partially compliant
ESG 1.7. Public information Partially compliant
Part 2: European Standards for the External QA of Higher Education
ESG 2.1. Use of internal quality assurance procedures Partially compliant ESG 2.2. Development of external quality assurance processes Partially compliant
ESG 2.3. Criteria for decisions Partially compliant
ESG 2.4. Processes fit for purpose Partially compliant
ESG 2.5. Reporting Partially compliant
ESG 2.6. Follow-up procedures Partially compliant
ESG 2.7. Periodic reviews Non-compliant
ESG 2.8. System-wide analyses Non-compliant
Part 3: European Standards for Quality Assurance Agencies
ESG 3.1. Use of External QA Procedures for HE Partially compliant
ESG 3.2. Official status Non-compliant
ESG 3.3. Activities Partially compliant
Standard Conclusion
ESG 3.4. Resources Partially compliant
ESG 3.5. Mission statement Partially compliant
ESG 3.6. Independence Partially compliant
ESG 3.7. External QA Criteria and Processes used by the Agencies Non-compliant
ESG 3.8. Accountability procedures Non-compliant
Source: Adapted from Pecakovska, 2015
These findings and the above developments suggest that Macedonian HE is far away from having an efficient system of QA. What are the possible ways out?
• Rethinking the Macedonian QA system in line with country strategy priorities on HE and ENQA standards and guidelines on QA. In line with the country long term strategic priorities for higher education, a national consensus should be achieved on why and what type of QA system do we need. An extensive professional discussion is needed on the real purpose of QA by including all stakeholders and by encouraging the academia to walk from a reactive to a proactive stance. The current national accreditation and evaluation guidelines and procedures need to be adjusted and revised in conformity with the European Standards and Guidelines on QA developed by ENQA (ENQA, 2015). Attention should be paid on international cooperation, demonstration of consistency and rigor in external evaluation and accreditation particularly for study programs on regulated professions (including the compliance with EU directives 2013), investment in development of teaching and research competences of the academic staff, further development of National Qualification framework, improvement of data information systems etc.
• External evaluation of all universities by ENQA accredited and EQAR registered QA agency: The curse of a small country like Macedonia where people knows each other brings lack of objective judgement in accreditation, re-accreditation and evaluation of HEIs providers. The new Government should therefore seriously consider commencing an ad-hock external evaluation of all accredited universities and HEIs (both public and private) under equal conditions by the independent ENQA accredited, and ENQAR registered QA agency. This will ensure credible and unbiased assessment on what is really going on at our HEIs. It will provide trustworthy baseline data on HE and evidence for improved quality regulations and informed decisions on optimization of the existing diverse network of institutions, study programs and profiles. This will be beneficial to all stakeholders and on a long run will have many positive effects in the society, preventing degree meals and protecting
students and their parents. It may also urge shutting down of underperforming programs and institutions and may initiate positive change at other, encouraging revision of degrees and study programs.
• Supporting Macedonian HEIs for the introduction of efficient internal system for quality assurance. Since “the primary responsibility for quality assurance lies within the HEIs” (European Ministries, Berlin, 2003), it is of utmost importance for the country to invest in introduction of efficient internal QA system aimed at continuous improvement of HEIs. QA should be perceived as an integral part of everyday activities and the long term strategic plans of the universities, and it is therefore important the managerial university structures to understand and support it. The HEIs need to ensure regular review of the content of study programs, pedagogical approaches, the workload and assessment of students, quality and effectiveness of the teaching staff, public information, learning resources and supporting systems for students.
Participation of students in quality assurance processes needs further improvement. The integration of the learning outcomes and their use into teaching, learning and assessment of students should become part of the compulsory training of the university teaching staff. Human resource policies of the universities/HEIs should be developed, enacted autonomously and accompanied by proper financial support by the state. An electronic database for the university teaching staff will help gather necessary data for HE policies and ease the process of professional development.
• Ensuring the necessitate preconditions for HEAEB operational independence and efficiency: The state must ensure that all preconditions for making the QA Agency independent and fully operational are in place. The Agency should develop the necessary internal policies and accountability procedures in line with the ENQA ESG for QA agencies. A new organizational structure consisted of a Managing Board, the Appeals committee, ERIC-NARIC Center, law department, HE analytics etc. could be workable with the determined number of positions and staff competences required for better efficiency. The Agency should receive the necessary funding for systemic training of its members in accreditation and especially in external evaluation.
The staff and the evaluators should be given permanent re-training. Further efforts are needed in equipping the Agency with the necessary technical equipment and electronic data supporting system. It should take over the responsibility to manage the funds from its own revenues account. It is critically for the Agency to get involved in its self-evaluation, which will help the administrative staff and members to reflect on its work.
• Pausing the national university ranking: The current indicators have many methodological, technical and data accuracy limitations which brought illogical
outcomes in the ranking of universities, while the three subsequent national rankings did not bring any surprises in the top 5 ranked universities. This puts in question the purpose and the real needs for ranking. In the absence of an efficient (internal and external) system of QA, the national university ranking performed “by an independent provider preferably from abroad”, should be looked into very carefully. Since the ranking does not contribute to the enhancement, but only to the accountability function of quality assurance, it should not be seen as a quality assurance tool (Costes et al., 2011). European University Association in its external evaluation report for the Univerisity Goce Delcev in Shtip (EUA 2014:18) concludes that “Shanghai rankings seemed to attract national interest and was mentioned several times in discussions with the team. The team believes that such rankings do not add to the meaningful development of universities and supplemental indicators should be used”.
• Fostering cooperation on quality assurance: Reliance exclusively on domestic reviewers can compromise the impartiality of judgment in the process of accreditation and external evaluation. It may foster collegial solidarity or emerges quasi-competition among the HEIs. The involvement of external reviewers outside Macedonia will require of the HEIs to conduct the self-evaluations and reviews in English, which includes additional translation costs.
HEIs should consider accepting this extra burden as a step towards increased credibility of the interview panels and the external evaluation process.
Networking and cooperation with the experts from the Balkan region might be a viable option because of the language similarities.