ENIC and NARIC Networks, main tasks of national information centre – European experience

International Legal Instruments. Lisbon Recognition Convention and Best Practices in Europe

Recognition Tools and Practices – Criteria and Procedure

Information provision and information sources

Making use of Qualification Frameworks

Recognition types and policies Higher education reforms in Europe

Mission report of Activity 4.5 Pilot evaluations

TWINNING AZERBAIJAN

Support to the Ministry of Education of the Republic of Azerbaijan for Further Adherence of the Higher Education System to the European Higher Education Area (AZ-ad-EHEA)

AZ/14/ENP/OT/31

EU Short Term Expert Mission Report 

1. Basic Information

Component and Activity:

Component: 4. Standards and Guidelines for QA in HE

Activity: 4.5 Pilot Evaluations

Name of the Expert: Ms Hilla Aurén

Dates of the Mission: 20 – 24 February 2017

Contractor: Finnish Education Evaluation Centre (FINEEC) /

      Estonian Quality Agency for Higher and Vocational Education (EKKA)

2. Relevant Background Information/State of Affairs

International cooperation in quality assurance has been an essential element of the Bologna process aiming to create a European Higher Education Area. A central tool in this work has been the European Standards and Guidelines for Quality Assurance in the European Higher Education Area (also known as the ESG). The Twinning project offers an opportunity for applying the ESG in Azerbaijani higher education. One of the mandatory results of the project is to develop Standards and Guidelines for Quality Assurance in Higher Education in Azerbaijan (AzSG) in line with the ESG and to test them with three higher education institutions.

In April 2016 a Drafting Group was appointed by the Ministry of Education to work on a proposal for AzSG. A draft manual for the pilot evaluations was discussed with the Advisory Group in a seminar in June 2016. The draft was published on the Twinning project’s website in order to get feedback on the assessment areas and criteria. Amendments were made to the manual based on the feedback. The capacity of the pilot institutions to conduct a self-evaluation was supported through trainings and events in several previous missions. The self-evaluation reports were submitted by the three pilot universities in January-February 2017 and have been translated or are currently undergoing translation to English.  

The main aims of the pilot evaluations have been set as: supporting the strategic management of institutions, providing external feedback to the institutions’ own internal quality assurance procedures as well as informing the internal and external stakeholders of the compliance of the institutions’ quality assurance with the ESG. The pilot evaluations will have an institutional approach with the focus on teaching and learning. The evaluation report will provide the pilot institutions with information regarding their strengths and good practices as well as recommendations for the institutions’ further development.

3. Objectives and Tasks of the Mission

The overall aim of the Component 4 is to develop Standards and Guidelines for Quality Assurance in Higher Education in Azerbaijan, in line with the European Standards and Guidelines for Quality Assurance (ESG) and test them with three higher education institutions.

In Activity 4.5, the Standards and Guidelines are tested in pilot evaluations at three higher education institutions – Azerbaijan State Economic University, Azerbaijan State Pedagogical University and Azerbaijan Technical University. The objective of the first mission of Activity 4.5 was to prepare for the pilots by visiting the three pilot evaluation universities and meeting with the relevant staff to provide information on the arrangements of the pilots.

4. Time Schedule of the Mission

5. Achievement of the Expected Results

The meetings at the pilot evaluation universities - Azerbaijan State Economic University on February 21^st, at Azerbaijan State Pedagogical University on February 22^nd and at Azerbaijan Technical University on February 23^rd - were carried out successfully. The STE was accompanied by the MS Project Leader, the MS Junior Project Leader, staff from the Accreditation and Nostrification Office as well as the RTA’s team. The delegation were welcomed by the Rectors and Vice-Rectors of the universities, who assured their satisfaction with participating in the pilot evaluations. The meetings were attended by the staff involved in the self-evaluation process and the contact persons from each university. During the meetings, the STE reviewed the aims of the evaluation, the ethical principles (code of conduct) of the evaluation group, and provided information regarding the schedule, programme, interviews, composition of the evaluation group and the reporting process. The STE also informed the universities of what is expected from them in preparing and conducting the site visits. Discussions took place at each university regarding who should be invited to the interviews. The universities also had an opportunity to comment and ask questions regarding the evaluation process. Some of the questions included who to consider as external stakeholders for the university, and how syllabi are reviewed in the evaluation.

In addition, several organisational issues regarding the pilot evaluations were discussed and decided during the mission between the Project Leaders, STE, RTA’s office and in consultation with the MoE and ANO staff. These issues included the compensation for local participants in the evaluations groups, the attendance of observers and the training provided for the evaluation groups at the beginning of the site visit weeks.

6. Unexpected Results

An issue that rose during the mission was that the same student representative had been assigned to all three evaluation groups, causing an unfair workload for the student. The RTA’s office will try to recruit other students for two of the evaluation groups. 

7. Issues Left Open After the Mission

The participation of some of the local experts in the evaluation groups still needs to be confirmed, including the student representatives for the Azerbaijan State Economic University and the Azerbaijan Technical University.

8. Recommendations for Future Missions

The evaluation groups have a considerable amount of work ahead of them, and the role of the project managers as well as chairs is crucial. It is necessary to coordinate the process among the FINEEC and EKKA project managers, in order to implement the three pilot evaluations in a similar manner in all three universities.

9. Conclusions and General Remarks Concerning the Project

The mission provided a good opportunity to discuss the next steps in the pilot evaluations, and arrangements for the pilot evaluations are underway. The pilot evaluations will provide an excellent learning opportunity for all.

Mission report of Activity: 1.2 Revisions of the Legislative Framework

TWINNING AZERBAIJAN

Support to the Ministry of Education of the Republic of Azerbaijan for Further Adherence of the Higher Education System to the European Higher Education Area (AZ-ad-EHEA)

AZ/14/ENP/OT/31

EU Short Term Expert Mission Report 

1. Basic Information

Component and Activity:

Component: 1. Legal and Regulatory

Activity: 1.2 Revisions of the Legislative Framework

Name of the Expert: Ms Heli Mattisen

Dates of the Mission: 20-24 February 2017

Contractor: Finnish Education Evaluation Centre (FINEEC) /

   Estonian Quality Agency for Higher and Vocational Education (EKKA)

2. Relevant Background Information/State of Affairs

As a result of the previous mission of Activity 1.2 that took place on 7 – 11 November 2016, the STEs Lagle Zobel and Helka Kekäläinen proposed a number of recommendations regarding the new draft of accreditation standards and accreditation rules developed by ANO. Based on the recommendations, ANO drafted new versions of standards and rules which was translated by Twinning office and sent to the STE’s of the Legal Component.

The final version of the Accreditation Rules of Higher Education Institutions (Accreditation Rules) and Standards determining the compliance of the activity of institution with the requirements of state education standards (Accreditation Standards) should be forwarded to the Cabinet of Ministers for approval in March 2017.

3. Objectives and Tasks of the Mission

The main objectives of the Mission were:

• Comprehensive discussion of ANO’s regulations in the light of the Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG) and identification of areas for improvement
• Formulation of concrete recommendations for the improvement of the drafts of Accreditation Standards and Accreditation Rules.

4. Time Schedule of the Mission

5. Achievement of the Expected Results

The expected results of the mission were achieved. The new draft of Accreditation Rules and Accreditation Standards, as well as recommendations were discussed. Due to the active participation of ANO, it was possible to critically evaluate the impact and applicability of recommendations.

Remarkable efforts have been made by ANO in reviewing the existing Accreditation Rules and Accreditation Standards.  In order to provide further support ANO regarding the improvement of their regulatory framework, we have the following recommendations:

Regarding the Accreditation Standards

• Some positive modifications regarding stakeholder involvement and use of student feedback in updating study programmes, the use of modern teaching materials and creative methods in teaching and learning, as well as a reference to ECTS have been made in the Accreditation Standards regarding part 4 (study programmes).

However, the concept of learning outcomes has not yet been introduced in the Standards. According to clause 1.2 of the ESG, institutions should have processes for the design and approval of their programs. The programs should be designed so that they meet the objectives set for them, including the intended learning outcomes. This should serve the overall objective of student-centred learning established in clause 1.3 of the ESG.

We strongly recommended adding to part 4 of the Draft Standard also explicit requirements for learning outcome based programme development as well as assessment of achieved learning outcomes. For example, following criteria could be added / reviewed:

• (Standard 4) Objectives, intended learning outcomes, admission and graduation requirements of the programmes are clearly defined; qualification resulting from the programme is clearly specified, communicated and referred to the appropriate level of the national qualifications framework (Remark: as the AzQF is not approved yet, the European QF or reviewed State Standards of Higher Education of the Republic of Azerbaijan could be referred to)
• (Standard 5)Practical work/internship supports the achievement of the learning outcomes of the programme.(in current draft: Institution shall organize internships for learners)
• (Standard 5) Student assessment shall be objective, consistent, transparent and support the achievement of learning outcomes.(in current draft: Correct, objective and transparent assessment of learners’ knowledge shall correspond to existing rules as well as to the international best practice)

• We also recommend reconsidering some criteria in the current draft version focused on checking the quantity rather than focusing on the quality of education, g. 2.8. The office work shall be run in line with the requirements of existing legislation; 3.2. Number of employees on the payroll (full-time, part-time etc.), as well as of those working on a substitution and hourly basis and their workload shall correspond to existing norms;

6.9. Number of doctoral and dissertation students assigned to one academic supervisor shall correspond to norms etc. The investment of expert-time requested for checking the fulfilment of the aforementioned requirements will not in balance with the level of potential impact for quality improvement.

• The inclusion of requirements for further sustainability of teaching staff cannot be detected in ANO regulations. It would still be advisable to add this requirement to the Accreditation Standards. The STEs recommended adding to the criteria for teaching staff also requirements for future sustainability. An example might be taken from the Accreditation Manual, which includes a following requirement: The distribution of full-time teaching staff by age and qualifications facilitates the sustainability of studies in a certain study area.

• Current drafts include the definition of accreditation introduced in Education Law. We recommend adding to the definition the purpose of improving the quality of higher education in Azerbaijan, as to ANO’s mission statement – Improve quality in higher education to increase local and international competitiveness of higher education.

Regarding the Accreditation Rules

• According to clause 3.1 of the revised ESG, the quality assurance activities undertaken by the agencies should have clear and explicit goals and objectives that are made available to the stakeholders. At the moment, the object of accreditation is not clear – is institutional accreditation combined with programme accreditation or should all programmes be accredited separately. Due to the voluminous amount of study programs, the current approach (accrediting the institutions together with study programs) does not allow devoting sufficient attention to the overall quality of the programs and thus, remains inevitably formalistic. We recommend clarifying the approach towards combining institutional and programme accreditation during the first period of accreditations in accreditation rules.

• As there is still no legal framework regarding the possible use of foreign agencies in the accreditation process, no further developments can be detected regarding the use of foreign agencies in the accreditation process in ANO regulations. Presumably, changes should first be made on legislative level. Thus, we strongly recommend including the conditions and procedure for the use of foreign agencies in the Statute of HEIs and mention this possibility also in the Accreditation Rules. Whether or not to officially recognize the results of any kind of cross-border evaluation is obviously a policy decision for the MoE, i.e. Accreditation Council headed by the Minister. However, we believe that allowing these kinds of activities would create a positive incentive for further internationalization of the HEIs and also ANO. It would also help to disseminate best practices of other agencies.

• According to clause 3.3 of the ESG, agencies should be independent and act autonomously. This includes organizational and operational independence and independence of formal outcomes. Currently, ANO enjoys a fair amount of operational independence. However, as ANO is a part of the MoE and the final decisions are made by the Accreditation Council chaired by the minister of education, organizational independence is clearly lacking.

As soon as the full trust of the Ministry has been gained, we recommend giving ANO full organizational and operational autonomy regarding its procedures and formal outcomes of the quality assurance processes. Implementing this recommendation definitely requires some fundamental changes on legislative level.

• As a remarkable development, changes have been made in clause 6.1 of the Accreditation Rules regarding the election of Accreditation Council on a permanent basis for four years. Although the inclusion of student members in the Council has not been specifically mentioned in the Rules, an explanatory statement has been added to the Rules (although it is not quite clear whether this will also remain in the final version of the Rules). It would still be advisable to make the inclusion of a student member in the Council obligatory and also specify the amount of members in the Council. We also recommend defining the proportion of independent experts in the Accreditation Council and agree upon the criteria for their election. An example of composition rules for the council applied by EKKA can be found here.

The reviewed principles of election and composition of the Council, including possible limitations, should be included also in the Statute of HEIs.

• According to clause 2.5 of the ESG, any outcomes or judgments made as the result of external quality assurance should be based on explicit and published criteria that are applied consistently, irrespective of whether the process leads to a formal decision. In other words, to ensure equal and fair treatment of the HEIs, the margin of discretion of the Accreditation Council should be limited by some basic decision-making principles.

A worksheet has been prepared for the Accreditation Commission (in the Final Review of the Accreditation Commission) for determining whether different accreditation areas are in line, partially in line or not in line with the Accreditation Standards.

The Accreditation Rules (see clauses 6.4 – 6.7) give the Accreditation Council an option to make one of three decisions based on the Accreditation Review: to fully accredit the institution/study programme if they meet the standards; to accredit the institution/study programme for 1 year if they partially meet the standards; and not to accredit the institution/study programme if the standards are not met.

However, the principles, pursuant to which the Council determines whether the standards are fully/partially/not met have not been elaborated in the Rules, providing the Council with a very wide margin of discretion in decision-making. In order to ensure equal treatment of all the HEIs and guarantee the consistency and predictability of accreditation results, it would be advisable to develop some sort of a framework for decision-making for the Council. E.g, it could be clarified  - either in the Accreditation Rules or some other ANO regulation made available to the public - how the assessments given by the Accreditation Commission to different assessment areas affect the final outcome of accreditation process.

• Pursuant to clause 2.4 of the ESG, external quality assurance should be carried out by groups of external experts that include (a) student member(s). As a very positive development in line with the ESG, the possibility of involving student members as well as foreign experts in the Commission have been added to clause 4.4 of the Accreditation Rules. However, according the ESG, the involvement of a student member in mandatory.

• According to clause 2.6 of the ESG, full reports by the experts should be published, clear and accessible to the academic community, external partners and other interested individuals. According to clause 6.8.1 of the reviewed version of the Accreditation Rules, the outcome of accreditation of the institution and its separate study programmes is posted on the official homepage of the Office. This is in line with the recommendation given by the STEs, but it should be clear that not only the accreditation decisions but also the accreditation reports should be available to the public.

• At the moment, follow-up activities have not been described in the ANO regulations. If ANO is planning to organize such activities, it would be advisable to consider elaborating this also in the regulations. According to clause 2.3 of the ESG, as a part of implementing process of external QA activities, agencies should have a consistent follow-up process for considering the action taken by the institution.

6. Unexpected Results

There were no unexpected results.

7. Issues Left Open After the Mission

There were no issues left open after the Mission.

8. Recommendations for Future Missions

There are 2 missions left in the Component 1 and it is important that the revision of the State Standards of Higher Education and the Statute of Higher Education Institution planned for the last mission week will and support the development of quality in higher education in Azerbaijan. 

9. Conclusions and General Remarks Concerning the Project

The sound and constructive cooperation between the higher education department and ANO in developing the legal framework for quality assurance in higher education is crucial.

Pilot Evaluation Guidelines

Mission report of Activity: 3.5 Enhancing Practices and Procedures

TWINNING AZERBAIJAN

Support to the Ministry of Education of the Republic of Azerbaijan for Further Adherence of the Higher Education System to the European Higher Education Area (AZ-ad-EHEA)

AZ/14/ENP/OT/31

EU Short Term Expert Mission Report 

1. Basic Information

Component and Activity:

Component: 3. Developing AzQF

Activity: 3.5 Enhancing Practices and Procedures

Name of the Expert: Mr Gunnar Vaht; Ms Baiba Ramina

Dates of the Mission: 27 February – 3 March 2017

Contractor: Finnish Education Evaluation Centre (FINEEC) /

   Estonian Quality Agency for Higher and Vocational Education (EKKA)

2. Relevant Background Information/State of Affairs

The Republic of Azerbaijan signed the Lisbon Recognition Convention (LRC) in 1997 and was first country that ratified (1998) the Convention. The preliminary system of recognition of foreign qualifications was introduced in 1992, but the Government Regulation on recognition process according to the principles of the LRC was adopted in 2003.

Until 2016 the competent authority for nostrification was Ministry of Education, since April 2016 the recognition and nostrification tasks were transferred to the ANO when this new institution was established for accreditation and nostrification/recognition.

The Republic of Azerbaijan is highly interested in qualified specialists studied abroad, and for this purpose there is a strict and detailed assessment of study programmes and foreign qualifications. The recognition authority often meets the problem that the quality of some individual qualifications from some countries of former Soviet Union is not at appropriate level to the labor market in Azerbaijan.

About 70% of foreign qualifications are recognised in practice, 20% are not recognised on the reason that knowledge of holders of foreign qualifications are at lower level than requirements in national standards and 10% of applicants present incomplete documentation for nostification.

The ANO is the only competent authority for recognition of foreign qualifications in the Republic of Azerbaijan. The holder of qualification should present the application together with all documentation on-line to the ANO. According to the national legislation, all documents must be translated and certified by apostille. In case the country is not party to the Hague Convention, the legalisation is required. The time limit is usually one month and the process is not free of charge to the applicant.

The procedure to the nostrification has three steps: 1) checking application and documents submitted for nostrification, and assessing whether the qualification is awarded by a recognized education institution; 2) assessing and giving equivalency to the qualification, and 3) inviting the applicant to the interview, and making final nostrification decision.

The assessment of foreign qualification for nostrification is based on seeking equivalency of the study programme. During the assessment of programme it is compared whether the programme completed abroad correspond to the national standard of the same or similar programme. The assessment of equivalency takes place at universities where the same or similar programmes are offered. The comparison of equivalency includes the list of courses and number of hours of each course. The programme content should be the same or very similar to the programme provided by the universities in Azerbaijan. In case the programme is not offered by the universities in Azerbaijan, the foreign qualification should be not recognized.

If a foreign qualification is not recognized, the ANO will explain the reason of rejection. The applicant has right to appeal the nostrification decision.

At the end of nostrification process an applicant receive two certificates – one for general recognition and one for nostrification.

3. Objectives and Tasks of the Mission

The objective of the mission was to facilitate the elaboration of a proposal for reflecting higher education of the AzQF in Azerbaijani recognition procedures and practices. The objective includes one-day national seminar on academic recognition of foreign qualifications and periods of study in Europe and Republic of Azerbaijan, and workshop on recognition practices and assessment criteria to the staff of ANO. The main task of the mission was to share information about best practices in Europe and to facilitate recognition criteria and procedure in the Republic of Azerbaijan.

4. Time Schedule of the Mission

5. Achievement of the Expected Results

The expected results of the mission were proposals for updating recognition procedures and practice.

All expected results were achieved. Seminar on assessment and recognition of foreign qualifications for the representatives of the Ministry of Education, ANO and higher education institutions, and workshop together with the staff of ANO were organised.

6. Unexpected Results

No unexpected results occurred.

7. Issues Left Open After the Mission

No issues regarding the mission were left open. The purpose of the mission was fulfilled.

8. Recommendations for Future Missions

No missions foreseen with the project.

9. Conclusions and General Remarks Concerning the Project

Since the ANO was established in 2016 there have been successful developments for assessment and recognition of foreign qualifications in the Republic of Azerbaijan. The staff of the recognition division is open to introduce new developments according to the LRC and its subsidiary texts.

The ANO receives a lot of foreign qualifications for recognition, the important amount of applicants are coming from the countries of former Soviet Union, mostly from Russian Federation, Ukraine and Georgia, where fake degrees and studies are problems to the office work. The ANO is putting a lot of efforts to check the authenticity of educational documents, and quite a lot of time is spent for verification and interviews of the holders of qualifications. Verification of evidence of studies is important part of recognition, however it could be clearly stated in operational charter of recognition, that interviews is not a tool for all applicants, and that it is used only when evaluator have some doubts about evidence of studies. Hereby it is suggested to create schematic outline as it is used in European Area of Recognition (EAR) Manual http://ear.enic-naric.net/emanual/ .

The workshop with the staff of ANO was focused on recognition criteria, and special cases and practices with the qualifications from different countries.

Current system of recognition of foreign qualifications is focusing on the content of the programme, the content of each course, number of working hours and other input elements that makes the recognition too detailed and complex. According to the national regulation: definition of the equivalence of higher education specialties is carried out through: Examination – recognition of higher education specialties of foreign countries and definition of compliance of the education acquired in the foreign country with educational standards of the Republic of Azerbaijan  https://www.nostrifikasiya.edu.az/frontend/en/content/item/9-%C3%BCmumi-qaydalar.

According to the subsidiary texts to the LRC the recognition of foreign qualifications should be based on five key elements of the qualifications – level, workload, profile, quality and learning outcomes, and the function of a qualification (formal right of the holder of qualification in home system of education). The ANO works for simplification of the assessment process, but the draft guidelines for new recognition system are not yet adopted by the Government. Therefore it can be suggested that the Ministry of Education will improve the recognition procedures moving from equivalence to the recognition unless substantial differences can be shown between the foreign and national qualification. It is suggested to implement the assessment and recognition system in the light of guidelines approved by the ENIC (European Network of Information Centres on Academic Recognition) – European Area of Recognition (EAR) manual, which is a practical tool for recognition authorities, credential evaluators and admission officers.

Also it can be suggested to use and implement subsidiary texts to the LRC to improve fair recognition of qualifications concerning higher education. Recommendation on Criteria and Procedures for Assessment of Foreign Qualifications (revised, adopted in 2010), and Recommendation on the Use the Qualifications Frameworks in the Recognition of Foreign Qualifications (adopted in 2013) are the fundamental tools for this process to introduce recommended criteria and procedure towards fair recognition based on learning outcomes.

According to the Recommendation on the Use the Qualifications Frameworks in the Recognition of Foreign Qualifications, the competent recognition authorities should also refer to the National Qualifications Framework, European Qualifications Frameworks and other similar Qualifications Frameworks as part of the assessment process. Therefore the development of Azerbaijan Qualification Framework will be a good transparency tool for the recognition of qualifications awarded by the education institutions in Azerbaijan.

LRC states that authorities shall provide adequate information on any institution belonging to its higher education system, and on any programme operated by these institutions. Higher education institutions in Azerbaijan are recommended to provide minimum information about their higher education programmes and qualification on their websites. Minimum information should be provided not only on national, but also on commonly used language, preferable in English.

It is very important to provide information about the educational system of Azerbaijan and on recognition issues on the website of ANO.

ENIC Network is useful in collecting information about education systems and lists of recognised education institutions. ANO looking forward to cooperate with the activities of the network, including the participation on recognition listserv (ENIC listserv). The ENIC network is established to exchange information and support assessment and recognition of foreign qualifications as well to exchange information on higher education systems and recognized institutions. Therefore a suggestion is also to improve information provision about Azerbaijani higher education system and recognition in the network´s website www.enic-naric.net.

Effort should be put to improve language skills of the recognition staff of ANO. The working language of ENIC Network is English, also all projects introduced by the European Commission and working documentation presented by the Council of Europe and UNESCO are in English. Capacity building of the staff of ANO should be part of development strategy of ANO. Therefore the suggestion is to organise regular staff training in the forms of study visits to other countries, on-line training courses, participation in conferences and seminars organised by the ENIC offices or international organisations in the field of recognition.

To improve cooperation with universities and disseminate information on recognition practices it is suggested to organize regular seminars on recognition for higher education audience.

ANO recognition terminology is recommended to use in light of Lisbon Recognition Convention. Therefore the use of term “nostricifation”, which is common term for old recognition approaches, at least on English translations is recommended to change to term “recognition”.

Higher education institutions have interest to develop internationalisation activities and therefore have interest on recognition procedures for admission of students holding a foreign qualification. Seminar for higher education institutions on recognition issues was held in Baku State University on 1 March 2017. There were a total of 34 participants mainly from the universities as Azerbaijan State Oil and Industry University, Azerbaijan State Arts and Culture University, Azerbaijan Technical University, Khazar University, Azerbaijan State Economic University, Azerbaijan University of Languages, Baku Engineering University, Sumgait State University, Nakhchivan State University, Azerbaijan State Pedagogical University, but also participants from Ministry of Education and ANO. The agenda of the seminar was focused on the presentations on international legal instruments; reforms in the European Higher Education Area; assessment recognition practices in Europa; assessment criteria and procedure; recognition networks; information provision; and recognition system, principles and procedure in the Republic of Azerbaijan.