The Interim Assessment

PRESENTER: Thanks. For those who don't know me I'm a researcher working in university of Niš, but also for some time already involved in many different EURAXESS activities, among which the HRS4R process. I was working as an assessor for some time already but I was also working as a lead researcher for drafting and implementing the strategy of my own organisation, University of Niš. I was also involved in some other projects in supporting the organisations from the region in developing their own HRS4R strategies making their own action plans.

Now, let me just share my screen. Today it's time to talk about the internal review for interim assessment. I have the pleasure to present the most boring topic.You know, this stage of implementing your strategy, in which you are not actually challenged, at least you are not challenged by the assessors, by the European Commission, we will see shortly why did I say that, but in fact I still believe that it's quite an important phase, you know, because it's a milestone in which you are having a lot of responsibility and responsibilities even greater because the power of judgment is on you. It is on you to decide whether your progress is good enough to continue with a commitment and to continue to work on the implementation of your action plan until the site visit review, or you need to implement some corrective measures. Of course, you will be helped by assessors in doing that but as I said before you will not be actually a challenged organisation.

This is the diagram you have already seen a lot of times and it somehow illustrates the life cycle of the strategy, of the work you implement and you carry out in building the strategy and implementing the action plan. After these first two years in which you're supposed to implement the initially proposed action plan endorsed by European Commission, eventually with some corrections after the assessment of your initial plan, then you're facing this self-assessment or interim assessment or inter-actual interval review in which you are always to self-assess the progress, to report about this progress eventually, if there are some changes in your plan to propose the revised action plan. Then this report is reviewed by one assessor, before that the review is assessed for administrative eligibility by the European Commission. Then the assessor does the assessment of the quality of the action implementation and fulfilment by looking at the targets indicators, timeline..., basically everything you did in this stage of self-assessment.

As I said before the review is considered as the recommendation only so there is no way for your HR Award to be revoked at this stage, so the only thing that you can get is a kind of slap on your wrist, kind of warning that something is wrong, but also we sincerely hope that you will not come to this and that the review you get from assessors will be let's say much more optimistic.

So there are three possible review outcomes: the first one is the encouragement to continue along the path and this is the best possible scenario obviously. However if everything is okay that's great, however have in mind that you're still doing self-assessment that you have a lot of bias in your work that even though you engage possibly and most desirably researchers from your organizations in this self-assessment stage as you are doing that also in the initial phase still be very careful about this review outcome you know, because

If there is not enough data at this point and you have only one assessor's eyes are let's say weaker than the three assessor size so this assessment do not hold that strength, do not hold I would say that you should take it very seriously but you still should continue look at what you are doing and you are the outcomes of your work.

The second possible outcome is encouragement to undertake specified corrective actions to improve already sufficient performance so here is theorem still on a good path but reviewer had some comments related to some problems that have been identified in your report and the third one the not desirable one if review have seen some major problems in your report, some major discrepancies, some major inconsistencies between your current action plan in the previous one or current action plan and the gap analysis from the initial phase that you will be warned to take strong corrective actions in case of judgment of insufficient progress or low quality of the action plans implementation that may happen also if in the previous initial phase you did not get maybe assessors missed something that can be eventually seen in the internal review stage by this additional assessor.

So basically, as I said before, you cannot fail you're not challenged but this is a very important milestone for you because this is very good opportunity for you to improve, to look at what you have been doing in past two years but don't know not only with your eyes but also with your researchers because you are supposed to also implement some kind of exercise in which they will be engaged and they will be asked to somehow assess the progress of your actions. So this is not the threat to have the label revoked, you are much more relaxed but still you will need to stay very responsible about this and the assessment report you can consider it as a kind of friendly advice because all of us working as an assessors are very committed to the values described in the Charter and Code document and many of us are very experienced in reviewing the other reports the other applications for the initial phase, so we have seen a lot of different strategies, we definitely so far had some measure of the expectations we could have from different organizations. Most of us are very tough of the comments but very easy on the marks so you can definitely take this report in a good way that it was made in a good faith and as a useful suggestion on how to continue the process.

One thing you have to be also aware of is that even if you are not satisfactory with your progress in the past two years and you're happy that you can pass this internal view without failing that's not a good place to be you know because HRS4R process is playing a long game because if you are not happy now when you cannot fail, imagine what could happen in three years at the site visit because at that

time your label can be revoked so you should definitely very seriously take any warnings you get from assessors but again I come back to my initial point that you should be very serious and very responsible about judging your own work, no matter if this judgment was made by some kind of a work group or steering committee, or the pool of researchers you involved in some kind of survey in or any other engagement activity. So the internal review is performed in this way so you basically review the fulfillment of the English action plan, you look at what you have promised initially to your researchers, to your organization, to your ecosystems, to European Commission and then you see where you are with each of the individual actions. There are some statuses that you can assign to each action, we will see later about that. There are some indicators and targets you proposed in the initial phase so basically you need to look at those targets and see if those targets have been met it's as simple as that. Then based on that you will revise the action plan so you will have to extend some of the actions because you did not finish them which is quite okay, this is not really a big deal. The big problem is that if you have to extend all the actions, so if you don't have any performance or vast majority of the actions and then again of course t's all only oh this is also a relative thing because there are some actions which take much more effort than some others, so you will have also to take this into account when assessing your progress because assessor will also consider the weight of the individual action when looking at the current status of this action if it was extended or finished or whatever. In any case you will have to make some revisions maybe you will have some other circumstances involved you will have some national strategy for research that was adopted in the meantime. Based on that, the strategy will maybe to plan some other actions, they will be added to the initial action plan, maybe you will get more ambitious when you see that your action plan is progressing well so you have to kind of raise the bar and maybe involve some new things you want to address some of the identified gaps within the initial phase.

Whatever you do you will draft a revised action plan and we have to publish it. Publishing the action plan is a kind of expression of commitment to your ambition to implement this action plan, so it's a kind of public commitment, it's commitment to your researchers, they are all now aware of your activities with regard to this topic and also the other researchers that would like to be employed in your organizations that compete for your job ads will see that and they will be aware of your ambition to improve in HR management in organization.

Basically you will have the forms at your disposal which are similar to the forms you get in an initial phase we will shortly get into it also, this is a kind of overview what you need to do you if you are a person with a signed case for HRS4R process in your organizations you will see in your access area the HRS4R link to access the HRS4R tool, then you will click on the number of your case in the next page and then you will simply start adding data and the data you enter is pretty similar to the one you entered before in initial phase, there are very similar outlook of forms at this for preparing internal review to the one in the initial phase you will have to put some organizational information that you will enter strengths and weaknesses of the current practices actions including a OTM-R policy and again some of those actions, after your work on the internal review will be new the actions you want to add to your action plan, some of the others will be the actions which you are assigning the status in progress. Then for some of them and hopefully for many of them you will mark them as a completed and for as few as possible you will have to assign the extended status, meaning that for some reason this action has not been implemented in the previous period and now you have to continue working on it, with the extended deadline, and finally some remarks related to implementation.

The form you will see will be populated with the data entered in the initial phase in the HRS4R tool so it will be easier for you to follow up on those remarks you have made at that time. However, this will be the case only if your organization used HRS4R tool in this previous stage, if not the forms will be empty. So you will have to start basically working on your internal review without insight into what you have been doing before but this initial review of course you will have on your hand in some other format which was used at that time. So now I want to just shortly show you the pages, this is the demo account so basically this organization is not the one and the data is dummy data but it’s just sufficient for the illustration. Also this is not a form you will actually see this is the form as it will be seen by the assessor, but basically the form will be almost the same, with the same outlook, just that you will be having options to add data at this page. I can only see them but you will be able to add information about all the aspects of your internal review.

The first aspect is organizational information, this is important for assessors because every application or every internal review assessed in some context, first of all this context is defined by the size of your organization and the structure of its employees, so what is typically looked at and what gets the most attention here in this data is how many international researchers you have, meaning that if you have a lot of them that means that assessors then expect to have many actions addressing them, because they have many administrative problems typically, we know about it. So we expect for the organization to take a special care about them the similar stands for also the external funded researchers, also when we look about the number of women among researchers we can say Okay there is a gender balance or not and if these figures are not similar there is some kind of imbalance. Then we will look at the actions related to gender situation to improving the situation, to ensuring that men and women have the equally opportunities for career advance also what we will be looking at is the number of the early stage researchers or post-docs R1 and R2, because also this structure, this ratio of researchers could have some impact on deciding what kind of actions you will be implemented. I mean if you have lots of R1 researchers then we will look especially the actions related to training because those actions will be the most relevant for them then. Regarding research funding not so important let's say for us but I would say that it would be interesting to see at the competitive funding for the organization because in a way this could imply the ambition of the oganizations to play well on the market

and typically a very competitive organization is the one that attracts the most talents so this is again something that's quite important for us to know in order to get a better context information for assessing the actual action plan.

Then a little bit about the organizations and then we go to strengths and weaknesses of the current practices, it's very difficult to do this kind of analysis you know because you have a lot of bias you're judging about your own strengths and weaknesses even though we have found a lot of very important information here that cannot be put in some other field defining the context for the action plan on some other aspects of the application. So at this stage for each of the topics like ethical and professional aspects recruitment selection working conditions and training in development you will be seeing your remarks from the initial phase and then you will be able to enter the strengths and weaknesses that are actually relevant for interim assessment you will be able to put some remarks. For those remarks we expect to have some kind of description of changes so what happened in the meantime why your perception of strengths and weaknesses have changed in the meantime if they did that would be interesting to know because by that we can assess the ambition that the direction organization is taking the ambition which is developing solving addressing its own weaknesses and promoting its strengths also with regard to that you will be able to specifically enter some information or data or perception about your perception on if your some of your priorities for short and medium term change. So this is pretty similar to what you can also enter above when considering strengths and weaknesses.

Then information about some changes related to the circumstances in which your organization operates. I also mentioned some major national strategies for research or action plans that put something new on the table that took some new requirements you have to meet sometimes. Those requirements can be in line and most of the times it will be in line with the Charter & Code principles so they will have basically positive impact on your HR strategy, but maybe in some cases they won't be in line and it will make your work in meeting those Charter and Code standards let's say more difficult and of course you will be having opportunity to add if there are any strategic decisions underway that may influence the action plan. Those strategic decisions can be made on the organizational level but also on the national level.

And then we go to the most important part. Basically as you can see all these talk is pretty similar to what you expected to do also in initial phase because the form is very similar to the one in initial phase and you have probably already seen this form for entering the actions. It is basically exactly the same like the one in the initial phase so you need to put the some statement of action to list the Charter principles that will be addressed or gaps that will be met by this action, gaps within some of the principle. Here is the opportunity to add indication the status of the action if it was new one, if you added the new action or this is extended one or something else. Then you can add the timing, you can indicate who is responsible unit what is the indicator that you are measured for, successfully talking about that later on and what are the targets. At the end after you finish your revised action plan you will be also getting automatically the unselected principles that you have not selected initially. Basically this list needs to correspond to the list of the principles for which you did not identify any gap in the initial phase. There is no way to revise your gap analysis at this point so basically you are doing the work based on the actual action plan for the initial phase, you don't take into account the gap analysis from the initial phase. So then you put this in appropriate format, whatever you want you print that as pdf and then you upload this pdf to the webpage you have previously created in the initial phase where your action plan is, so you're replacing the existing action plan with a new one before the assessment.

Before the assessment takes place also you put some of the comments on the implementation of the OTM-R principles you can see pre-populated text area from the initial phase here and of course if you have some revisions to the OTM-R checklist and of course if you have some accomplishments or achievements or new documents or by-laws procedures or whatever you will be uploading those on the dedicated OTM-R webpage on your website which can be also subpage of this HRS4R, Or it can be completely different page maybe even more accessible because this is something that's very important for the candidates for the jobs you advertise.

Then a few words about the implementation: general overview but also some answers to some specific questions, how have you prepared internal review, very general notes about the process and then some details about your rate or way to involve the research community in the implementation process. Did you implement the survey, did you implement some other way of communication with them in order to get evidences on your progress so the progress you assess is not the progress you as a member of work group but the progress independent actors stakeholders, the key stakeholders, the employees for whom you are doing all this are seeing or perceiving. So do you have an implementation committee or steering group regularly overseeing progress, is there any alignment of organization policies with HRS4R, organizational top management support… It’s a kind of a nice way to reflect the strength of the commitment of the organization to this work. So if your organizational policies are also embedded with the actions or with the principles from your HR strategy then it's also a very good indicator of strong commitment of the organization. How has the organization ensured that the proposed actions would be also implemented so this is a kind of risk management, how you will be mitigate eventual risk, what you will do to ensure hat one that the things that you have promised will be delivered at the end, how are you monitoring progress, timeline, do you have some meetings, some milestones in which you are kind of assessing what actions have been completed and what actions need to be extended, how will you measure progress in view of the next assessment, how do you expect to prepare for the external review… Lots of important questions they are all basically product project management related.

This is something for which I cannot suggest any kind of direction in which you will be going when preparing for this assessment or the external review because this is part of your organizational policy, organizational procedures. I can give you just some tips in the continuous of this presentation.

Let me just come back to the slides. So after you submit your internal review you will be undergoing administrative eligibility check, it will be carried out by the European Commission, you could expect to get feedback within four weeks after submission. Don't panic if you don't get it there are difficult times now I guess European commission is really buried with lots of questions, lots of initial phase applications, lots of internal reviews, so please be patient with this. There is a possibility to get rejection because of some administrative problem you have, some lack of data or whatever then you're expected to update and resubmit within two months and then after, when you pass this administrative eligibility check then your application will be undergoing the assessment carried out by the individual external expert.

So this part I would like to stress some of the things that are also relevant for the initial phase because they're also assessed in the interim assessment. Those are let's say the four factors. I would have to also stress that this is kind of personal perception, I could not say that all reviewers will be looking at the application by using this perspective but I believe it's very close and very aligned with the values of the charter and code declaration. I have seen in many organizations towards providing really nice career development opportunities for their researchers, excellent working conditions and other benefits of this wonderful profession and I would like to discuss shortly about the four aspects of your action plan, namely the commitment, the ambition outreach and the quality in no particular order.

So the commitment as you know and you are reading in all the important documents is important. This is something that will be helping your action plan to be developed in the smoothest way. You know the commitment means that it will be easy to get a consensus about some gaps, consensus about the ambition of the particular actions to pursue those gaps and the commitment also means that you will be seeing a lot of top management support, that your directors or deans of research will be involved in your process by overseeing this process but also by giving valuable remarks. And also the commitment means that what you do within this HRS4R process will have an impact organizational policies although this is also organizational policy but let's say at the top level visionary long-term or short-term documents, action plans, strategies that are built within the organization that address the human resources or anything that's in a way related to human resources of your organization. So basically if you can provide a clear evidence on the abandonment in the organizational policies by providing short statements about some particular articles of the strategies or some particular actions, in other actions adopted by your organization in some domain than nature management, this is the clearest evidence of the commitment of the organization, so assessors love to see that in the initial phase but as well in the phase of internal review. This is something that's a little bit more difficult to find out but as I said this is quite important.

Also on time delivery is the indicator of the commitment so if you in your internal review wrote that most of your actions are completed as planned that means that there is a high level of commitment in the organization to pursue the compliance to charter and code principles. I will be using a lot of impact words, so the impact originated in the actual action plan is especially appreciated so it's nice if you can demonstrate that some actions have been completed but it's wonderful if you can provide the evidence of the impact of those actions and two years is a lot of time and you can have that for many of the actions, you can ensure that there are some visible impact in the firm. Of course that means that maybe you will need to somewhat revise your approach to defining the indicators, so in a way some indicators will be goal oriented and the others will be impact oriented, we will talking about that shortly. Stick to the indicators, stick to the targets you have promised in the initial phase. Sometimes we see rationalization of failure I call this. We are trying to somehow make our failure relative and to present it as a kind of success. There is really no need to do that because as I said we are not assessing the excellence, we are assessing the progress and progress can be achieved if you have the commitment.

One way to demonstrate the commitment is to be honest and acceptance of failure is really a sign of the strong commitment to the values in the charter and code and trust me this will be appreciated by the assessors. The ambition is wonderful thing, it's really amazing when you see the action plans and when you for example when you look at the gap analysis and when you see a lot of problems found in each maybe sometimes even in each of those 40 principles, so I have seen in the applications initial phase with more than 30 principles with identified the gaps which is really a lot and then you have a lot of problems and then you say okay i will solve all those problems and not that I will only solve them but i will also go further with that and show that I will implement the risk mitigation, measures that some of gaps never reoccur. and this is also wonderful to see but sometimes this is not feasible to implement and assessors will be assessing that. So the context in which you need to position somehow your ambition and the context in which this ambition will be assessed is first of all gap width, what is your base line. When we assess the organization we do not assess its excellence, as an assessor our baseline is relative. I mean we determine the baseline based on your current state of play, the current state of play in your organization, current status of your organization. Based on that we assess the application based on the gap width, in general to the absolute compliance to charter and code principles, but basically in a relative way to the update then the size of the organizations, how many researchers you have, how many international researchers, I shortly explained that when I was showing the forms and also your playground size to which extended things you want to change depend on the external factors. I mean in some countries, here for example in Serbia for sure, the deadline for applying to the position advertised by the university is 15 days and this is prescribed very long, so no organization will be extending those 15 days to more and of course if you want to apply for the job if you're a foreigner you have to have your credentials and your diploma recognized. There is no way to do this in 15 days, so basically this is this disqualifying for us, it's quite difficult if not impossible to employ a foreign researcher here in a permanent position let's say, or let's say more long-term positions because there are no permanent positions. So this is our constraint so and basically someone taking the playground for us so we cannot achieve progress in this way we can maybe promise some actions, we can maybe plan some matches relating to lobbying maybe or for participation of our top management in different groups of think tanks or discussion groups or committees or whatever where we can push forward and ask for this change but this is the best we can do we cannot promise the progress in this term.

Also we are at the phase, although as well as the initial phase, we have to talk about the feasibility. I will tell you also one story from one of my experience for one site visits. Through a site visit we did some desk work before the actual site visit and all of the free assessors agreed that our organization was amazingly ambitious in addressing the gender gap, the actions, the strategies, the measures, the incentives, the benefits for female researchers,… It was really amazing, I mean something that neither of us have seen in other organizations. So this organization had quite big reputation of course I will not tell you what organization that was and imagine what the say when we came there at the city. We met before the meeting and one of the assessors showed us news article stating that for the previous day that there was a strike of female researchers at that organization because they were not happy about the teams their organization, the one we are visiting, the one which we were so enthusiastic with, they were not happy with the actions this organization is taking to address this gender gap. So what I’m saying is that is really great to have lots of ambitions but this submission has to be associated with the feasibility, I mean, there has to be some evidence that this ambition is founded, this ambition is supported by the

set of actions that are pushing the right button and targeting the actual target.

As for the quality we look at this traceability, traceability means that new action is somewhat associated to the actions, defined in the previous stage and then each of these sections are somehow associated to the gap analysis and actually two gaps and then gaps are of course associated in the relevant to the principles to which they are assigned so this relevance, this consistency needs to be visible and it will be checked. It is wonderful when you get this action plan with a very few words with a very explicit definition of the actions. We do not count the number of words in the action plan, in fact we appreciate where the actions are clearly defined with the explicit statements and then we can easily and without any vagueness we can easily understand what the action will be,and what action is supposed to achieve.

Then maybe the most difficult: weakness. The greatest weakness in average of the applications in the initial phase in the appropriateness of the indicators and targets, so basically in most cases targets are not provided at all, at least for the initial phase applications that I have been reviewing. But only the indicators in some cases there are some indicators saying that something will be achieved for example some by-law will be adopted this is kind of both indicator and target within one statement but in some other cases that cannot be seen. The example I will mention is not only related to the appropriateness of the capacity targets but also to approach and how those indicators are being chosen, for example in some career development action you are planning to organize some trainings about professional skills and you are planning to implement the serious structured program of professional skills training program for your R1 researchers, and then you say indicator is a plan for the program is developed. It's very good to have a plan, you cannot do anything without a plan but at least you could have that you will be organizing some number of trainings, the number of training would be indicator and the target would be for example five, but also this is very weak indicator because we don't know what dind of trainings are involved but more important we don't know anything about the potential impact of those trainings. So maybe better could be to say that your trainings will be attended by 500 participants, 49 in total for example, so the number of participants of the training would be indicator and the target would be 500.

But again what kind of professor would I be if the indicator of the quality of my lecture would be how many students attended it? This is not an indicator of the quality of the action, this is also the only indicator. How well you did communicate this section to the potential participants or your researchers, so I ask typically the organizations to look for the indicators which are more impact oriented, but let's talk about this in the next slide.

And also quality of the evidence that the actions are completed, in many cases we will see just the statement that something has been done but even though technically, formally this is sufficient, we would really like and we would really appreciate to see some kind of evidence that the actions are completed, some experience from the actions, some statements indicating some outcomes or results from the action, even some links to some documents that have been created as a result of this action.

Goal-oriented versus impact oriented strategy, I mean when you are setting up indicator you can you can say that your training have been attended by 500 researchers and you are achieving your goal and that's fine but what those researchers do with this training or with this knowledge and skills they gained during your training is exactly what the impact is and this is something that we will be looking at and your action plan those impact oriented indicators the use of it this is something that will be highly appreciated; not that the goal-oriented approach will be penalized, I'm just saying that impact oriented approach in defining the most desirable outcomes is something that will be greatly appreciated and the indicator of the commitment of the organizations to have to really have the outreach in assessing this approach.

We will also have to consider that work culture is a factor, there is a clear difference on average between the strategies coming for example from Mediterranean countries and the other coming from northern Europe and not that some strategies are better than the others, and not that some strategies are definitely having some different approach but you will be seeing a lot of these top down versus bottom-up approaches, so in some communities bottom-up approaches where you have a lot of engagement from researchers will be clearly visible but the other way you will on the other side you will see some top down strategies where the word of director is the most important, where the top management is the place for where you will get all the directions for implementing your strategies and that will be seen that is clearly visible when you assess the initial phase.

Again the thing that is related also to the above is engagement of early stage researchers in the process, this is the most important for us because with engagement of the early stage researchers in the process we will be let's say more convinced that what you have promised is evidence-based and thus it will reflect the realities of your organizations in the best rate.

Short-term and long-term impact of the actions towards early stage researchers, again I mentioned lots of early stage researches here because majority of the principles in the charter and code are addressing exactly this target group. They're in all the current political priorities of the European Union and strategies and also organizational policies. I would say that early stages are the most important for tne organization, so what happens next? After you get a positive or any kind of review from the assessors you will be implementing remarks, you will consider them for the revision of your interim internal review, actually for the revision of the revised action plan and again you will have to publish it as a kind of public commitment to the proposed actions and then you have three years continue implementation of your actions until the award renewal assessment.

Before we proceed to the questions you may have after this presentation I would also to address some of the questions that I have

received before the webinar. So there was one question saying what is the recommended time to start to write interim assessment to start doing the interim assessment? There is no correct answer to this I mean it depends a lot on the size of your organization, it depends a lot on the time it takes, for example to implement some kind of survey or other engagement exercises. It depends on the available frequencies to meet with your working group or steering committee on the meetings in which you will be making the decision related to internal review so there is no easy way to answer that. I can say that for my organization we did the interim internal review University of nis took some let's say month and a half to two months to do everything but because it was a long time ago I cannot say that it was a really busy one and a half month or two months. I guess for the size of our organization it was quite sufficient but this is something that you will have to decide by yourself, really nothing will explode if you break the deadline so i'm not suggesting that you should be doing that but i'm saying that it's much better to have a good well-written honest internal review than to meet the deadline and I hope that I haven't been too suggestive in this last statement.

Is it recommended to do the survey in every phase of the process and what is the recommended period for the survey? It is absolutely necessary to engage researchers in this process, so it's absolutely necessary to get some kind of, I would say external assessment of your work on implementing the action plan and this kind of assessment is being done best among the target groups of those actions, so among researchers. This will be the most honest assessment, whether you're doing it in a survey if you have a large organization or you’re doing it in a focus group or interviews if you have small organization. That's on you i mean it's much easier to implement the survey but also you will have to take care about the contents of this survey because it needs to be somewhat different than the one you typically do in initial review, so you are not maybe supposed to assess the compliance to each of the principles but you will have to assess the outcome of the actions you proposed. Basically some would say that it's essentially the same, but not necessarily; it's on you to decide that because it all depends on the contents of your action plan. Also when you do survey it's not just sufficient to just do the survey, this survey needs to demonstrate that you have a representative sample in the researchers among those respondents of the survey. The sample needs to be representative in terms of gender, in terms of career stage maybe, positions in different departments or whatever. And of course you will need somehow to take care also about this representatives relative to the survey you implemented in first initial phase because maybe you cannot compare the survey because you have a different target groups represented in another service so it's on you really, you have to take care about all those factors to get the most honest and the most representative opinion of your target groups. The recommended period depends on what you are doing is it service? is it a focus group? is it a series of interviews? in any case you have to plan it in advance, so basically again it depends a lot of factors.

We can maybe if you have some additional question with some special situation we can continue discussing in this direction. The action plan needs to be published before the assessment so you need to publish the action plan as soon as you revise it after your internal review has been completed and before your interim assessment started. The question was formulated like that, when we went to publish the updated action plan, gap analysis and what else are required to publish? The action plan is enough, I mean to publish action plan is quite enough, this is also the expression of commitment of your organization to meet thee charter and code principles and I have not seen organization who published the gap analysis, because the gap analysis should be something private to your organization, because you may be identifying some weaknesses that you don't want to reveal to the public. Maybe you are on the competitive market I don't know, if you are some RD organization maybe private competing for horizon 2020, occurs in Europe funds, so you need to implement this, to develop this strategy but again gap analysis is something that can reveal the things that you don't want to be revealed and want to be public. But also for some publicly funded organizations that could be some, I mean I don't see the benefits but I can see the reasoning behind the publishing gap on.

So thanks a lot for your attention, sorry for the hiccups I’m not the morning person actually, especially in these times of pandemics, but I will certainly I’m awaken now completely and I will be certainly happy to answer all the questions you may have.

MODERATOR: Thank you Milan. Okay so we do have a couple of questions from the chat area so I will read you the questions and if anyone wants to elaborate your questions please do so. We have two questions from Senna from Montenegro and she's asking: are we entering cumulative organization figures in the first table at the beginning for the period of review from initial review? to this review for example 2018 to 2020, number of international students, researchers…

PRESENTER: No, no, you put the absolute figures, so the current state of the play okay?

MODERATOR: and the second question that also came from Senna is does the system allow and this is also maybe a question that Michele could help us with, does the system allow to have the assessment questions in word of or pdf format beforehand to prepare the inputs for entering into system?

PRESENTER: you can you can start working on internal review you can see the form you don't have to enter the data immediately, you Can access the form I have shown before and you can see this form and then you can just start working, you can copy and paste questions from there and then you can start working on your internal review and then when you prepare what you need to enter in this form then you get back to this form then you enter it and submit it. You can also, and Michele can correct me if I’m wrong, you can also start working on the document and then save it and then when you think it's completely finished then you can submit it. So saving a form does not mean that you're doing submission, thank you.

EC: Thank you Milan, that's exactly what I wanted to say, that you can save the draft and come back to it afterwards.

MODERATOR Thanks Michele, good to hear from you, it's great to see you too.

EC: Thank you, thank you very much and thanks in particular for adding before don't panic if the competition has some delay, it's really a complicated period.

MODERATOR: Thank you. Okay, so we can go further on, thank you. We have a question from Yasmina from Sarajevo and she says good morning, many thanks to professor Zdravkovic who through our first HRS4R application and reapplication we are now preparing for the interim assessment and this webinar is very useful, thank you. To begin with could you please share with us the deadlines for submission of interim assessment? For example if we received our logo on 15th January, is it until this date two years after that we need to submit internal interim assessment? Michele, do you want to answer this?

EC: Yes sure, thank you for the question. The deadline starts counting in the e-tool from the moment you received a notification, so it's two years after you received the notification of the internal review feedback.

MODERATOR but they did not use the tool for the initial phase

EC: Yeah, Okay, first of all i want to recall the invitation with every occasion i keep repeating that the e-tool is now the only way to submit internal review, therefore if you were handling your case in a manual way you have to put everything in action to put yourself in the e-tool. In any case for the deadline it starts counting from the day you receive the email with the notification results of the intro review, yes? I hope this answers your question.

MODERATOR: Thank you okay further on we have a question from Reyes from Madrid, hi Reyes. She says is it mandatory to have a full OTM-R system implemented in the institution or only to show that we are on the good path?

PRESENTER: No, it's not mandatory, i mean You have to show that you're in good path, actually just to show that there is a progress in implementing the HR OTM-R system, I mean there is no other way to answer that absolutely not.

MODERATOR: Okay, so we have a question, do you want to maybe, hi! it's good to see you

PARTICIPANT: Hi, thank you so much, we had a doubt about the level of implementation of the OTM-R at this interim assessment point, so although we thought that it was not mandatory to have everything ongoing but it's always good to confirm with you, so thank you so much for the webinar and so nice to see you, bye.

MODERATOR: Okay we have a question from Isabel from Liège. Hi, Isabel, so she has a question regarding the IT tool, could you explain the difference between the extended and in progress options for reporting on the actions?

PRESENTER: My perception is that extended is something that means that somewhat the action was to some extent endangered in terms that if there isn't clear need to extend the deadline for its implementation even though that it should be completed. Because you know initial phase has been completed and in progress is something that like delayed just for some reason, I mean just the level of let's say, the seriousness of the reasons for extending the action, but of course Michelle is here, Isabel if you have something to add to this I would really be happy to share.

PARTICIPANT: Okay, nice to see you Milan and nice to be here. It's because we had the discussion with some people involved in this interim assessment and the question was what does extend it exactly refer to, so and the question was the extension the of this let's say of the original content so it means that we will go deeper so we will extend the scope or is it the question of not a delay but perhaps an extension of the duration? And we discussed with Barbara from Czech Republic and also with Giulliana Sabatini from Austria, and we were not sure of the definition of extension as compared to the question of let's say in progress we think that in progress is something that is running, you didn't reach yet your target, you need some time to go further perhaps because it was planned for five years and when do we decide to put our new action in one of the category or the other one. So use you speak in terms of time we spoke about the extension or the broadening of the original content of an original action so perhaps can we give the advice to the people to be clear on the description of what they intend to do in their new action plan in order to be sure that the assessor will well understand your own definition, if we have several definitions.

PRESENTER: But maybe the best approach would be to have one definition of what extended means, I mean Michele, do you have something to add on this?

EC: Oh, I managed to activate my video, now you can see me. Hello everybody, well I don't have much to add on this. You are the super experts on that. Well the extended, the way I see it is that there is a necessity for a supplement of information in order to coordinate at best the interim phase, especially if you go in view of the renewal assessment. When you are going to meet the assessors and then the picture should be clear and the suggestion should have been taken into account so an extended action in this sense means that there is a supplement of actually to be taken in order to maintain the award that the original phase but you and Isabel are the authorities on that.

PRESENTER: Okay, we can maybe leave it like that, I mean not to continue elaborating obviously there is a need for this to be explicitly defined, I mean if it's a subject of perception and obviously as we can see maybe even different perceptions so it's something that needs to be addressed in the future.

PARTICIPANT: For myself I will call to coherence for the participant if they would like to use in progress or extended action and what is important is to have something that makes sense, and if it makes sense for you to write extend it rather than in progress because you think that the scope or perhaps the duration of something in your intent of development and implementation makes sense so you have to use the words that is coherent with your perception and with your development. And by having some words that can help the assessor to assess your intention what you would like to do, that's the most important so do not hesitate and I give the advice to all the participants if you have something to say that can have some let's say ambiguous perception or that can explain what is your way of implementation use it in your description and in your file, it's very important for the assessor to understand what you have in mind and how you are evolving and progressing.

MODERATOR: okay thank you, I would just like to add, I'm not skipping the order, I’m sorry but there is one question that was asked the latest but it's pertinent for this, so Claire is asking from the presentation it sounded to me that in progress means undertaken not completed whereas extended means not undertaken yet, action deferred is that accurate?

PRESENTER: As you heard already we don't have accurate definition here but as Isabel mentioned I mean this is very good message, whatever you use you need to describe your actions in a way that is consistent, so this consistency can be recognized by the assessors in this process.

MODERATOR: Okay, so we can go back to the questions asked, so we have a question from Daria, she says what would be your sugestions to encourage universities to apply for the HRS4R? She's asking for just some principles or code phrases that she could maybe use.

PRESENTER: I mean it's a difficult question, this is a kind of uh we have this Article 32 as a kind of, not incentive, but something that relates appropriate HR policy with the implementation but I’m not sure that is a good way to promote it in some cultures. Again, I mean coming back to cultures, in some cultures, the way to say it is that I mean if you don't have this label then you will be difficult to see your first audit after your completed Horizon 2020 on Horizon Europe project, as simple as that. But of course in some other context this will not be sufficient. I mean, it's very difficult to use this for promotion but I can say that in many occasions I have seen through this embedment of HR process that it was intertwined with other organizational policies. I have seen a clear ambition to improve, clear ambition to have a major outreach and good strong relationships especially with early stage researchers and even in some cases i have already seen the benefits of it. It's very difficult to communicate with researchers they are pretty much unaware of anything that's not related to the domain of their research. Typically they go to the places where they can meet the best researchers, the best colleagues, so it's kind of difficult to show to them that working conditions is important, it's kind of difficult to demonstrate to them that having access to professional skills training is something that's good for them, so it's quite a difficult challenge also to make the case to them not only to the organizations. To be quite honest I’m not prepared for that kind of discussion so I’m not sure what to answer other than that. Isabel would you want to add maybe?

PARTICIPANT: Yeah, so for me and what i always say to those who would like to apply or hesitate to apply, I can say that we are really aware in our institution that performance and well-being are correlated and if we are working on the improvement of the researcher working condition at the level of an institution and we let it recognize and by the researcher and also listen to them and their needs, I think it's really of added value for the governance board. Secondly i think that it's really important to also provide an institutional tool to the researcher for helping them to feel good at work, perhaps to find some job and some application and to apply to some European project and so on. At the level of institution it's important because it's a help to better recreate and to let know to the worldwide community and the last thing that is of importance I think is that we are all working on that, so why not to be recognized by the European Commission? Because we are doing it professionally and it's really important also to be recognized and to let know about what we are doing so we are working on the HRS4R process at all in the university in the world because we are taking care of our let's say manpower and we try to have happy researchers and to let it recognize by an official international organization is really important. It requests some work at the level of the institution but the most important benefit we took in at my level as a hat of the administration is a nice project involving the researcher and the administration and i think it's really of added value for us. I never worked with my colleagues such hardly on a project all together at the level of the university and i think it's really important for that because not only the researchers have to feel happy at work, we have also to be recognized and to work at the level of the organization, so you embark the administration and the research together in an institutional project, I think it's one of the really important added value of this HRS4R process and it can be a good argument for the board so you will make it a great project for all the community and at the level of the university

PRESENTER: But it would be also great to have our own indicators of HRS4R process success and this is something between us, but I believe that it's a good place also to say that it would be, in my view, it would be great to have those indicators, in order to be to help have more tangible evidences of the benefits achieved by the HRS4R initiative and also it would be great to have some testimonials from the organizations who implemented their own action plans and to find out from their own perspective what benefits have been brought by this implementation process with them. But this is on us, I mean this is not the answer to your question but just the idea on what we could do to make these benefits clearer.

PARTICIPANT: Perhaps have some participant can testify on what was really important for them and what decided them to embark because I think that it's not so easy and we have many participants from different country and I think that it was not on media Article 32 that was let's say helping to embark the organization in this process.

MODERATOR: Michele, please you wanted to add?

EC: Yes, just wanted to add one small thing to complement what Milan and Isabelle said. Of course that there is also a huge increase in international visibility because of the HRS4R, do not forget the HRS4R, once you the institution obtains the logo it will be published on the EURAXESS network and it is something that increases the level of general visibility of the institution, also the recognition within the international institution like the European Commission is something that is extremely relevant, and then of course it's in between the lines of what Milan and Isabelle already said but the framework of the HRS4R in itself is of added value for the management of the HR within the institution who are working on it which are giving a clear framework of work And according to many institutions to the feedback provided by many institutions it would be very interesting to hear what our participants have to say. The feedback that we obtained in the past is very positive especially for the way the HRS4R provided the framework for concentrating the HR work on the institution participating, thank you.

MODERATOR: Thank you, okay, should we go further on with the questions? So we have two questions from Yasmina regarding the action plan, she asked do we write about indicators or just whether an action is new, completed in progress or extended? At what point do we justify that an activity took place? Is it just by stating that it was completed, or do we need to write numbers etc? That’s the first question.

PRESENTER: I mean you just enter the status completed in the form next to this section but also it wouldn't hurt to put some evidences. As i mentioned before that the action was completed, I mean for example if it was a training that you have organized and that training and that many people participate and stuff like that, so not extensive, I don't know, the document with the evidence, if you have a link to the document you can put it also in the description of action so there is no field. If you're asking that for elaboration on how the action was completed but you can put it in the description of action.

MODERATOR: Okay thank you. Yasmina also asked if we need to publish our revised action plan, does this mean that we need to have it adopted by the Senate the same way we adopted the strategy and the action plan in the application process? This is something that might be problematic because we need to wait for the monthly sessions.

PRESENTER: it depends on how the decisions are made in your organization, I mean sometimes I have a feeling you know that this document can be adopted at the level of the steering committee in the working group in charge for the organization. Especially if the top management is involved, engaged in the work of steering committee then you have all the decision power to make this decision at the level of steering committee. But in some organizations, the top management bodies of the university of the organization will be making the decision. Yasmina asked me in for this specific case, even though the senate has made the decision of adoption of the initial strategy, I would say that if you have a support of management and people sitting in the steering committee you don't have to go again through this adoption process but then again this all depends on your organization rules, procedures and how you did organize the work on the internal review.

MODERATOR: Okay thank you so this is actually all of the questions that we received through the chat I don't know if anyone else would like to comment on anything or ask any questions please do, this is the opportunity. Michele I think you raise your hand, if

you want to add something please

EC: i think Isabelle had raised the hand before me.

MODERATOR: Oh, I didn't see it

PARTICIPANT: so what i would like to say regarding the two last question is that what is important for you is to spend more time to Implement the HRS4R strategy, than to report on what you've done, what is what is really important is to provide evidence that you are in the way doing good things and progressing, so it's absolutely necessary for you to collect indicators, for you to report and to have also your HRS4R action plan embedded in the institutional strategic plan, it's really important to have the board commitment for what you are doing and what you plan to do. So it depends of your structure but what is important for the assessor is to have the opportunity to assess your report and to help you also to grow in your action so for me the worst is to have 200 pages to read on your interim assessment, what is important is to have a good idea that you are working and my advice is spend some time and pertinent time to write your interim assessment. What is more important is to have a good consultation of the researchers, to have the strong support of the institution and also to spend time to implement your your action rather than to report but the process is you have to write your interim report and it's a good opportunity to stop a little bit, but not too much time and on the action plan.

MODERATOR: Thank you Isabelle, Michele?

EC: Okay thank you, I thought of seizing this opportunity to Milan and Isabel too of course if she wants to a quest that I receive very often from institutions that goes more or less in this way: we underwent very many changes in the organization of our institution in the last year and now our action plan is completely different from what it was before, last time we submitted. Shall we reintroduce a new one, what shall we do? Sometimes I even have the institution saying we want to abandon the HRS4R and start from scratch. Of course we advise not to do that but how would you reply? Because we received this question at least once per week.

PRESENTER: I mean it's quite a difficult situation. To be quite honest, I would not have any understanding for someone that is changing completely the action plan and also it all depends on what do you mean by changing it completely? If you committed to the Action in the initial phase you cannot just discard it in the consequent phase, I mean this is the answer I can give and I can provide. What does it mean to go again from the process? I guess this is something that's very difficult to understand to again going from the initial application. But I from the formal point of view I guess it's possible I mean, I'm not sure what to say. The point is, the burden is also on the people who participated in the implementation of the initial application, so burden was on the members of steering committee and working group to make their case to the new management or in light of these new changes made. Of course as I mentioned before if there appears that some action from the initial stage proposed at initial stage cannot be implemented because there are some government imposed restrictions, like the example i mentioned then I guess i would consider agreeing with removing this section from the action plan. But the all the other situation I would consider that i cannot talk about because we don't have specific cases, I cannot generalize, but as an assessor I would consider this as a kind of interim assessment with a lot of changes, with the major warning.

PARTICIPANT: So from my point of view Michele, I think that the two first year of work are really important because you start to implement your action plan so if you want to change all the action the question is why? Is it because your board has different opinion or different priority? It can be the question but what about the point of view of the researchers? Because if you play the game directly it means that once you try to answer the needs of the researcher with your action plan, and the needs of the researcher they changed in the last two years? Why? Why did the priority changed, that's the real question. Is it because you saw that your implementation was not correct or your way of implementation or perhaps that you felt in your perception of the needs of the researchers and so on? I think that you can change everything but you have to explain why you decide to do that. If you say we have a new board, it’s not their priority to work on ethics, it's not their priority to work on open science, it's not their priority to make the researcher or to develop this kind of improvement of the working condition my answer will be, what's the point of view of the researcher in this sense? In my university our second action plan was completely different as compared to the first one, why? because at the early beginning it was really important to have very short-term implementation of actions because it was the start of the process, because we focused on also involving the researcher better in the action of the university. After two or three years we saw that our let's say action for trying to convince the people to work on some points was completely different, so our first action plan was expressed as having training for the researchers, political declaration of our board and administrative implementation of some actions. But today we changed our perspective, we have different packet of action related to improvement in ethics, improvement in doctoral supervision, giving the opportunity to train the mentors and the supervisor for the PhD candidates and for us it makes sense because we developed in parallel different new action plan related to new option of the institution. So it's coherent with what we did previously it's the continuation of what we did, but the expression of the action plan is completely different and I spent some time to explain that to the assessor and to write it in the narrative part of the action plan. What we changed, what is the continuity, and why we involve better this HRS4R action plan in our institutional plan.

PRESENTER: but let me make this clear. you changed the action plan by adding new actions but you've completed the actions from the initial action plan.

PARTICIPANT: I think that I reported also on some action that we are not supported by the board. So when you have one more thing yes so we have to be more international and so on, and the next one says it's not our priority we will change the things, I have not extended or in progress action but I’ve decided to free some action till we can have a better supportive context for doing that. And three, four year after when you change the board, when you have new president you can come back with this action by saying so okay it's time to rediscuss about the priority of this this action, or perhaps to express the action in another way and decide to drop it from the action plan for several years for instance. So it's not a question of continuing the action but changing over perspective, but we have to take care or you have to take care of the question of isn't it a shift to a more top-down approach? In this sense it's not good so you have always to refer to the researcher in order to know what are the needs, what are their priorities and is the action plan coherence with those gaps, because it's called gaps again.

PRESENTER: Yeah but, you know, as you mentioned at the beginning you know we have to be aware that in best case scenario the action is reflection of the attitude of the key target groups or the researchers towards the compliance with the principles. So basically the action plan is not the ownership, it's not the outhor of the working group but it's something that is forced bottom up, so it's pushed from up, so the arguments on revoking some of the actions or putting them on hold need to be really strong in my view as an assessor. Because you maybe changde the management or maybe the management to change the direction but you cannot change researchers, you cannot change your employees, so basically for me this is kind of very difficult topic and it's to be argued very strongly, especially if you are revoking some action but because when you mention you completely change the action plan, I mean introducing a completely new set of actions in the internal review stage is completely valid. I mean this is of course natural and this will happen a lot. In fact having the same set of actions which you had in the initial phase and just extending them to the second period is sloppy, this is lazy. This means that basically you don't recognize that something new happened in your organization to which you need to adapt.

MODERATOR: Okay thank you. Okay so we don't have any other questions that occurred in the meantime, I don't know if you have anything else you would like to comment on, Now this would be it. I see that Michele is raising his hand so either Milan or Michelem please.

EC: Okay thank you, you know this is a wonderful opportunity to remind participants of things that are usually forgotten or overlooked. There was a wonderful sentence before from Milan earlier in his presentation, it is better to delay your internal review rather than submitting a bad internal review. Please remember that you can ask for an extension in case you need additional time to complete your internal review, you can ask to the in the tool for an extension of the deadline so this is really crucial instead of submitting something that you are not satisfied with, just a reminder thank you.

MODERATOR: Thank you michele. Milan, you wanted to say something.

PRESENTER: i just wanted to say again don't panic, this is this is a stage witch you really can help yourself. Because you have three more years that seems like a lot of time you know, but it goes like that I mean, it can easily happen that you're getting close to the external review and that you are not happy with your progress so at this point, the point of internal review you can make those changes, you can review this progress and you can use this opportunity of external assessment by the expert to have additional explanations and help advices in how to proceed further. So good luck from my side and thank you really for this wonderful opportunity to share with you, thank you.

MODERATOR: thank you Milan, and thank you for your expertise, your time and especially in the morning time even if you don't like it, it was perfect as always so thank you so much and special thanks to Isabelle and to Michele who participated very much with their expertise in this area, it was a wonderful presentation, wonderful session. I hope we all learned a lot in it and we hope to have more of this in the future, in whichever format available. So please just a note to all the participants, go to the HRS4R e-learning module, it's very useful, it's a great guidance to the whole procedure you have all the webinars available there, this one will be also available in some time, so you can promote it and there will be more content to come in the coming year so stay tuned and thank you once again to everyone and to participants and hope to see you soon. Bye.