Employability and Sustainability of Young Graduates in the Slovak Labour Market: Counterfactual Approach

Introduction

An increase in the rate of employment and reducing the unemployment rate belong to the long-term general aims of the Slovak economic policy as well as the European Union (hereinafter “EU”) and its member states. The situation of the high level of unemployment of economically active people is in general an undesirable phenomenon. Employment and productive work are the condition of socio-economic development at the level of the region and state and are the source of regional as well international competitiveness [1]. As stated by Kabat et al. [2], relatively high unemployment undoubtedly leads to weakening the income situation of households and strengthening the social tension in society, including countries of the EU. Economic development has a dominant influence on employment development. This applies to Slovakia, too. Since 2013, the economies of the EU countries have been growing (annual improvements in GDP being around 2% on average) and this has reflected the improvement in the EU labour markets. Overall, the official average unemployment rate in the EU was below 7% in mid-2018, and in most countries of the EU, it was lower than the psychological level before the financial crisis in 2018 [3].

As far as Slovakia is concerned, unemployment became a clear problem immediately after the transformation of the Slovak economy from a centrally-planned to a market economy in 1989. Since that time, unemployment in Slovakia has been discussed from the economic, social, and political point of view. Historically (based on the official statistics of the Statistical Office of the Slovak Republic), the Slovak unemployment rate reached the highest level of 19.79% in January 1999 (that year the annual unemployment rate was 19.20%). Currently, in May 2019, the Slovak economy recorded an unemployment rate of 4.88%, the lowest rate ever since the establishment of independent Slovakia. The average value of unemployment from 1993 to 2018 was 12.74% [4]. Compared to other EU countries, Slovakia ranked among the top three countries with the highest unemployment rate during the years 1999–2008. However, in the first half of 2018, the unemployment rate was, for the first time since Slovakia joined the EU, slightly below the EU-28 average. This is a significant qualitative shift in the Slovak labour market, but compared with the labour markets of the Visegrad group (countries historically and economically related), Slovakia has long been showing the worst values of this indicator [3].

The main distinctive features of unemployment in Slovakia are regional disproportions, risk of increasing long-term unemployment, risk of structural unemployment, insufficient link of the education process with the needs of the labour market, burden of unemployment, insufficient innovation capacity that would support employment, and employability of excluded groups including the young. It is a general interest to tackle the unfavourable state of unemployment in all age groups. However, youth unemployment is a thorny problem as young people are at a stage of life when work habits are created and the starting working experiences and skills are gained. Failure to address this situation could lead to social erosion, undermining future prospects of individual careers [5], and from the macroeconomic point of view, raise the risk of long-term unemployment.

During the last financial crisis, the number of young unemployed people (20–29 years old) in the EU countries increased from 4.2 million in 2007 (10.8%) to more than 5.6 million in 2013 (17.7%). In Slovakia, the unemployment rate of this age group reached 13.8% in 2007 and 23.0% in 2013, the second highest level since 2002, when it peaked at 24.9%. In 2017, based on the official statistics, the youth unemployment rate in Slovakia was lower than in pre-crisis years and at the same time lower than the EU average (EU-28; 12.2%), which dropped to 12% [4]. Despite the fact that the situation in the labour market has improved and the current situation is a relatively favourable one, the unemployment of young people remains a matter of great social concern in Europe, as it is potentially menacing for the stability of democratic societies in the medium and long run [6]. Youth unemployment bears a tremendous burden for the social and economic future of the EU and its member countries [5]. In this context, specific target interventions of the active labour market policy were proposed with the aim to improve employability of the young not only in Slovakia but also in other EU countries [4]. Based on the European Council recommendation, the European Commission implemented the European Youth Guarantee (hereinafter “EU Guarantee”) in April 2013. According to the EU Guarantee, countries with regions at the NUTS 2 level with a youth unemployment rate more than 25% are entitled to receive financial support from the Youth Employment Initiative, the key European financial resource to support the implementation of the EU Guarantee, with a budget of 6 billion €. The EU Guarantee is composed of a package of possible active labour market policy measures for the young, as well as wage and recruitment subsidies for employers [5] for the 2014–2020 programming period [7]. It is assumed that in countries with pronounced active labour market policies, a larger proportion of young people is actually participating in these programs [8]. These measures are expected to reduce the risks of unemployment because their main aim is to reintegrate unemployed youths into work [9].

The Slovak labour market policy consists of active and passive parts. While the passive labour market policy ensures a sufficiently necessary income after a loss of a job, the active labour market policy is targeted to solve the tasks of unemployment. The legal framework supporting the implementation of current measures of the active labour market policy was set up in 2004. Since that time, its main features have undergone only minor changes. On the other hand, the context in which the measures are being implemented has changed substantially [3]. One of the frequently used measures within the active labour market policy in Slovakia is the intervention targeted at school-leavers—Graduate Practice. The intervention is targeted at young jobseekers who are not able to apply successfully for a job due to missing skills and practical experience. The aim of the intervention is to assess professional skills and practical experience, which graduates acquire at an employer corresponding to the level of their education. Other goals are making contact with potential employers, gaining working experience, working habits, and improving the sustainability of the jobs obtained.

The existence of this intervention is also important from the side of the replacement of the labour force, since the Slovak population is aging at a high pace, and the Slovak labour market cannot in the long term profit from older but higher skilled employees [3].

The Graduate Practise as a financial initiative in its current form has been used since 2014. Accepting the legal framework, including the conditions of participation, the aim of this study is to evaluate the effectiveness of this intervention considering employability and sustainability of the initiative’s participants by means of a caliper-matching technique, the technique of the propensity score-matching method. Except that in the discussion part, we present findings of the assessment of the impact of the intervention on the state budget. A contribution of this study is the use of the counterfactual approach in evaluating the selected intervention. The essence of the study is to reveal what would have happened if the intervention had not been provided, and thus, to quantify the effect of the intervention as the contrast between the difference in results of intervened and non-intervened jobseekers. Overall, despite the European Commission’s calls for implementation, the counterfactual evaluations of interventions provided to the excluded groups (in general meaning) are very rare in Slovakia. Authors consider the use of this evaluation methodology, in the case of one of the most frequently used interventions in Slovakia, significant, thus the contribution may help to fill the knowledge gap in conditions of the Slovak labour market.

Following the aim, the study is divided into four main parts. The first part introduces an issue of sustainability of young graduates in the labour market, presents a short literature review, and provides information on the Graduate Practice. The second part describes the chosen methodology that we used to analyse the effects of the Graduate Practice, namely the counterfactual method propensity score matching. The data used in the analysis are described in the third part. The last part presents results of the analysis including findings on the ability to place young jobseekers, who were intervened by the Graduate Practice, on the labour market and their sustainability, as well as the time impact of the intervention.

Literature Review

The interventions of the active labour policy are financed from the state budget, but the finances for the Graduate Practice are also drawn from the EU funds. It is, therefore, logical that the European Commission insists on carrying out rigorous evaluations of this intervention. As stated in the work of Guijarro [6], the political importance of active labour market policies for the young contrasts sharply with the low level of knowledge regarding its effectiveness. So far, there are several evaluations, whose results even indicate that some of the implemented interventions significantly reduce the employment probability of young people in the short to medium term, but for the creation of a design of future labour policy, it is urgently needed to get more evidence on the effectiveness of active labour market interventions for young people [6].

Evaluations of measures of the active labour market policy are carried out not only in the member states of the EU but also in other countries around the world. Within the EU, the European Commission requests and treats evaluations, as the EU is spending considerable funds on their implementation at the level of individual countries. Many evaluations of different measures of the active labour market policy have been undertaken in the last years in the EU countries. For instance, in Hungary, several projects aimed at evaluating the measures of the active labour market policy were implemented at the Budapest Institute for Policy Analysis [10,11]. The Institute found out that applied measures represented adequate ways of tackling long-term unemployment in the countries of Middle Europe. However, the adoption process of the analysed measures to market conditions was not as prompt as is required. In Macedonia, selected interventions, including the contribution to self-employment, were evaluated [12]. The authors of the study state that some of the state interventions in the Macedonian labour market were effective in tackling unemployment, but some of them, including the contribution to self-employment, were not able to generate expected effects [12]. In the studies of Guijarro [6] and Borra et al. [13], the authors analysed selected measures on the Spain labour market. The first study revealed that participating in active labour market initiatives had a positive impact on the probability of exiting unemployment in the Valencian region. Besides this, the results point to the fact that people aged 55 and older and women are the most vulnerable groups. The authors of the second study applied various techniques of propensity score matching and found out that the analysed measures were effective only in the short term and did not solve the long-term problems. Even in relatively economically stable EU countries, unemployment is a problem that is often addressed by measures of the active labour market policy. For example, Gonzales et al. [14] dealt with young unemployment in the United Kingdom. The problem was discussed also in the Scandinavian countries, for example, in Denmark [15], in Norway [16] and in Sweden [17].

As for the applied evaluation approach, counterfactual analysis, some authors have already used it to evaluate selected interventions of the active labour market policy, e.g., in Romania [18], the Czech Republic and other countries by Potluka et al. [19,20], Potluka and Spacek [21] and Kopecna [22]. As well, several experts have addressed the evaluation of the active labour market policy measures in Slovakia, e.g., Barosova et al. [23], Barosova [24], Borik and Caban [25], Harvan [26], Lubyova et al. [27,28], Stefanik [29], Tiruneh et al. [30] and Domonkos and Konig [31].

Graduate Practice and Legal Conditions of the Intervention

The Graduate Practice (Eurostat LML code 41_SK 7) is the intervention introduced under article 51 of the Act. No. 5/2004 Coll. on Employment Services and on amending certain laws (hereinafter “Act on Employment Services”) and currently, it is one of 18 active labour market policy measures in operation in Slovakia. The main aim of this intervention is to create conditions for acquiring relevant professional skills and practical experience, which would be appreciated and attractive for a potential employer in the labour market and that would be able to ensure higher employment of graduates. The intervention was suggested in accordance with the assumption that lower practical experience was a significant obstacle for graduates to enter the labour market in Slovakia [32]. Table 1 presents a log of the analysed intervention including expected outputs of the intervention, and Table 2 presents the expected effects of the intervention.

According to article 8, paragraph (1a) of the Act on Employment Services, a graduate is considered a disadvantaged jobseeker. The graduate is a person under the age of 26, who completed continuous preparation for a job with an appropriate level of education in full-time study less than 2 years ago and did not have a regular paid job for at least 6 consecutive months [32]. According to article 51 of the same act, the Graduate Practice is based on a written agreement on Graduate Practice between a graduate registered in the database of jobseekers for at least 1 month and the Central Office of Labour, Social Affairs and Family (hereinafter “COLSAF”), and on the basis of a written agreement between the COLSAF and the employer. A graduate practice is performed for a minimum of 3 months and maximum of 6 months without possibility to prolong it or repeat it in the scope of 20 h weekly [33]. Practically, a young jobseeker works part-time at the employer for 4 h daily and does the job that the employer assigns him/her. For this work, a graduate receives an allowance from COLSAF at the amount 65% of the subsistence minimum provided to one adult person in the given year. In 2014–2015, the time period analysed in this study, the allowance was 128.75 € [33]. In the following years, this amount rose to 129.66 [6] from the third quarter of 2017 to the second quarter of 2018, and 133.30 € from the third quarter of 2018 to the second quarter of 2019 [32].

The counterfactual evaluation of the Graduate Practice in this study covers the years 2014–2015. The effects of the intervention were followed up during a 24-month impact period (covering the years 2014–2017 depending on the individual date of the end of the intervention). During the impact period, the employability of the interveners and non-interveners, sustainability of their jobs quantified by the period of their placement in the labour market, and the height of their wages were analysed. In 2014, the analysed intervention treated 18,042 young jobseekers (approx. 1504 jobseekers monthly), and in 2015, 24,584 young jobseekers (approx. 2049 jobseekers monthly). Overall, the intervention database includes data on 42,626 participants forming continuous time series for the years 2014–2017.

Materials and Methods

The evaluation of the effects of the intervention on the employability of young jobseekers is carried out using a counterfactual approach. This approach is based on the following consideration that a young jobseeker, who is eligible to participate in the Graduate Practice and is interested in it, is further monitored during the 2-year post-intervention period (impact period). Participation in intervention should bring about an effect in the form of better employability of a treated jobseeker. However, in order to evaluate whether the employability of a jobseeker can be attributed to the intervention, it is necessary to create a counterfactual situation for him/her. Its main aim is to find out what would happen if the individual was not treated by the intervention [34]. As an unemployed individual at one time can only be in one situation (is or is not treated by the intervention), another individual (or more) who has not participated in the intervention (was eligible but did not apply for it) is matched to this treated jobseeker. This non-treated jobseeker forms a pair with him/her. This pair is designed to make two individuals as similar as possible. This similarity is measured by various techniques. e.g., it is possible to perform matching based on exactly the same values of matching characteristics of individuals (exact matching method). In the case of our study, we decided to use the results of the counterfactual evaluation gained by applying the propensity score-matching method (hereinafter “PSM”). This is a very popular statistical method for measuring the effects of interventions. Propensity score actually represents the probability that an individual will participate in the intervention. This probability is estimated based on his/her values of characteristics using a statistical model of logistic regression. This model contains all the characteristics of an individual and his/her environment as explanatory variables, and the dependent variable is the probability of participation in the Graduate Practice. Subsequently, treated and non-treated jobseekers are matched to have a very similar propensity score. This approach creates a counterfactual situation for each treated jobseeker being monitored: We have found another individual with a similar probability of participating in the intervention but who did not participate in it. Matched individuals are then measured during the 2-year period after the end of the intervention [30]. In doing so, we have measured outcome variables that expressed their course of employment. Subsequently, we have evaluated the average effect of the intervention as the difference between the mean values of outputs of these two groups in the labour market in the individual impact period [35]. Since pairs were designed to form the best match, the measured difference in the average values of the result variables is attributed to the intervention effect [36]. The difference in the results of these two groups represents the quantification of net intervention effects. In other words, the principle of PSM is to obtain an unobservable potential result of one individual through the observable result of similar individuals, who are in the opposite state [37].

One of the characteristics of matching is that it reduces the number of non-treated individuals, creating a sub-control group which has characteristics which are more homogeneous to treated individuals. Matching usually compares treated and non-treated individuals only in a so-called “common support sample”. This sample is created by the omission of control individuals, whose confrontational values in matching are either greater or less than the values of treated individuals [37].

In counterfactual evaluation, different approaches are usually used to match participants based on the propensity score. We used two methods in the evaluation, namely nearest neighbour matching (thereinafter “NN”) and caliper matching:

• Nearest neighbour matching is a frequently used matching technique. Each treated subject is matched with a non-treated one, one or more, who has the closest propensity score. The most common choices of the closest neighbours are from one to five [34]. The disadvantage of this approach is its data intensity. By applying this matching method, we obtained the smallest samples of common support compared to other applied evaluation methods (see following Table 3), and thus, we do not mention the results of the evaluation by this matching technique.
• Caliper matching is the most common implementation of the propensity score matching, where pairs of treated and non-treated subjects are formed whose propensity scores differ by at most a pre-specified amount (caliper width) [38]. Caliper matching based on the propensity score was made with a score, rounded to four decimal places. The rounding of the propensity score is in accordance with the rules of mathematical rounding. Rounding to four decimal places means that the maximum error we make is 5×10−5. Thus, the matching tolerance (caliper) is maximum 0.00005. This choice of the number of decimal places proved to be the most optimal because by using it we have obtained the largest sample of matched treated and non-treated individuals (so-called common support). Other options for rounding the propensity score led to a smaller common support sample (see the Table 3).

In addition, the overall results of the evaluation did not change by using a different choice of caliper for caliper matching; this option mainly affects the size of matched samples. By this approach, couples of the treated and controlled individuals were created and an impact period was then assigned to the control individuals, taken from the group of treated individuals.

Matching in both approaches can be done with or without replacement; we used the second option—matching without replacement. That is, each individual from the control group was matched with the individual from the treated group only once.

The evaluation of Graduate Practice using caliper matching of the propensity score was made based on the following steps:

• Creating a logistic regression model with predicting individual propensity score values for individuals in samples of treated and control jobseekers.
• Verification of fulfilment of the presumption of balance [37].
• Matching those individuals who have the same propensity score with a the matching tolerance maximum 5×10−5.
• Assigning an individual impact period for non-treated individuals based on the treated ones with who they were matched.
• Characteristics of employability of individuals during the assigned 2-year impact period by quantifying their values of result variables.
• Quantification of differences in labour market outcomes for non-treated individuals and those who participated in the Graduate Practice.
• Testing the significance of differences between the average values of result variables in the group of treated and non-treated individuals [37].

Data

The existence of relevant and credible data was a crucial aspect of the rigorous counterfactual evaluation of the Graduate Practice. All data came from the official sources. Primarily we used data from the COLSAF, the implementation institution of the analysed intervention and data provided by The Social Insurance Agency. In the study, we used the largest possible sample of young jobseekers (130,000) who were eligible potential participants of the intervention and comprehensive information about them was available thanks to data provided by both institutions. Each participant was characterized by information about: age, gender, family status, permanent residence or temporary residence, educational attainment, department of graduate school, type of school, code of study department, driving licenses, disadvantages in the labour market (under article 8 of the Act on Employment Services), profession (according to the International Classification of Occupations), date of registration in the database of jobseekers, date of removal from the database (variables to check eligibility of the intervention), registration period prior to evaluation, total time of registration of the applicant in the database of jobseekers, and removal from the database due to the departure of the applicant abroad.

In addition, we used other data to identify the group of treated and the group of matched non-treated individuals, time period of the treatment, and location where the intervention took place in the labour market (SK NACE of the employer, NUTS code of the region). The employability of young jobseekers, their placement in the labour market, and paid wages we monitored through other result variables. The needed data we got from the register of the Social Insurance Agency. The most important result data were:

• Placement in the labour market, including registration as an employee or voluntarily insured person or self-employed person.
• Average assessment basis representing the average monthly income of the individual in €.

We also used context data that informs about conditions in the local labour market in the regions, where young job seekers were looking for jobs. For instance: data on the unemployment rate based on NUTS location, average gross salary in the region of permanent residence of the participant, ratio of women in the region of permanent residence, expanse of the region of the permanent residence, and number of cities in the region of permanent residence.

Sample Description

Before matching the individuals based on their individual characteristics, we analysed some facts about the treatment and control groups of jobseekers in both years. The sample analysis we performed to ensure homogeneity of the intervention and the validity of counterfactual evaluation of impacts of the intervention. We considered the following number of applicants:

• 16,230 treated individuals and 7186 non-treated individuals in 2014 (First reference period).
• 18,092 treated individuals and 19,037 non-treated individuals in 2015 (Second reference period).

The map of Slovakia below (Figure 1) shows the total number of individuals included in the study. The greater the concentration of the red colour, the more participants from this region were included in the sample of young jobseekers. It is clear that most of the participants came from the parts characterized by a high level of unemployment. Application of the intervention made the biggest sense and suspected effect on sustainability of jobseekers in the labour market right in these regions. The biggest part of the participants in both analysed years came from the region of Presov (25% in the first period and 26% in the second period). The smallest part of the participants of the intervention came from Bratislava region (2.6% in the first period and 2.7% in the second period). As well, the smallest control sample of the non-treated individuals came from the Bratislava region.

To create a logistic regression model that predicts the propensity score of participation of each individual in the intervention, we used the chosen characteristics that we describe in the following text in more detail. Table 4 shows the gender-based distribution, marital status distribution and education level of jobseekers in each year included in this study and in the groups of treated and non-treated jobseekers.

The analysis shows that men were less interested in attending the Graduate Practice than women in both analysed years. The gender gap is about 23% in each year. The number of treated and non-treated men and women was approximately the same every year. If we look closer to the marital status distribution, most treated and non-treated jobseekers were single, which is logical given the age and the socio-demographic trend. Regarding the frequencies of individual levels of education of jobseekers, in both reference periods, secondary vocational school is the most represented among treated and non-treated individuals. The second most common level of education is vocational school. From the relative abundances, it is clear that lower degrees of education prevail among the non-treated. However, there is a visible difference in the case of university education, which is more common in the group of treated individuals than in the group of non-treated individuals (difference 22.1% in 2014 and 17% in 2015). A smaller difference in favour of treated individuals is also visible in comprehensive school education.

Table 5 lists descriptive characteristics of age of the groups of treated and non-treated young jobseekers. In both years, non-treated individuals were slightly older than the treated (difference of 1.83 year in 2014 and 1.64 year in 2015; differences in medians are 2 years in both periods). At the same time, treated individuals showed a higher age variance than non-treated jobseekers.

Selection Bias

The intervention Graduate Practice is intended for all eligible young jobseekers. It means that each jobseeker who meets the conditions of the intervention, can apply for it. That is why this intervention is influenced by a selection bias, which means that the impact of the intervention is potentially influenced by so-called self-selection effect. Therefore, we took into account the potential impact of unobservable characteristics that could affect the estimated average effects of the intervention during the process of evaluation. We tried to identify some variables to lower the effect of the self-selection. These variables could contribute to the most accurate estimation of an individual’s probability of participation in the intervention. We suggested the following instrumental variables:

• Number of habitants in the district from the last population census in Slovakia.
• Population change in the district over the last 15 years.
• Actual distance of permanent residence of jobseekers from the local Labour Office, where they registered.
• Density of settlement of the district.

We assumed that these suggested variables were able to describe the environment of each young jobseeker and the potential of the locality to create new jobs. We used them as instrumental variables to explain the self-selection effect and to lower the effect of selection bias in the logistic regression model.

Results

The analysis of employability and sustainability of young jobseekers in the Slovak labour market was performed using counterfactual evaluation of net effects of the intervention Graduate Practice.

The model of logistic regression is the first step of the entire process of evaluating the selected intervention. The model was estimated using all independent variables, which were in disposal in the case of both groups of individuals. In the modelling, we used all input and context data that were presented above as independent variables. As well, we used all presented instrumental variables. The final model of logistic regression was used for each individual in each year to estimate his/her value of the propensity score for the participation in the intervention Graduate Practice. Based on the model, we matched the participants of the intervention with the individuals from the control sample. The relative frequency (see Table 6) is a proportion of the sample remaining after the matching from the original sample of treated or non-treated jobseekers that we had at our disposal. This method is data-intensive because the loss of data that are not contained in the common support sample is significant. To count with so much data regarding all job seekers in Slovakia who met conditions of the intervention was a great advantage.

Measurement of the Employability

The employability of young jobseekers in the Slovak labour market was measured by using result variables. The most important variable, quantifying the process of employability, is the variable placement in the labour market. The following two figures (Figure 2 and Figure 3) depict the development of employability of the treated and non-treated individuals and their sustainability in the labour market during their 2-year impact period. The relative frequency of individuals, who were placed in the labor market, is depicted. The frequencies are recorded in percentage. As it can be seen in the figures, high frequencies of non-treated job seekers were revealed in both observed periods and this fact contributes to the improvement of the final net effect of the intervention on employability and sustainability of employment of young jobseekers.

In 2014 (Figure 2), approximately 53% of non-treated individuals were permanently unemployed during all 2 years of the impact period. In the case of treated individuals, only 22% of them were unemployed during the observed year.

As well, the figures present information about the development of employability and job sustainability. Based on these findings, 78% of the participants of the Graduate practice and only 47% of those who did not participate in the intervention, were working at least for 1 month. For a period of at least a half-year and more, 57% of treated participants and only 34% of non-treated participants were retained in the labour market. For a period of 1 year and longer, only every third treated individual and every fifth non-treated individual had a job.

The situation changed in favour of job sustainability of non-treated individuals after 1.5 years of a 2-year impact period of the intervention. Fifteen percent of the treated and 16% of the non-treated were working longer than 1.5 years. For the entire duration of the impact period, 14% of the non-treated individuals and only 4% of the treated individuals were employed.

Based on the shape of the graph, mainly its sharpness and slope, we can deduce that in the case that the non-treated individual is placed in the labour market, he/she is able to keep a job for a longer time and with higher probability in comparison to the treated individual. From the group of non-treated individuals who found a job, approximately every fourth was able to keep his/her job longer than 2 years. Considering the group of treated individuals, only every 20^th individual kept his/her job longer than 2 years. The sustainability of a job working position fell sharply in the case of the treated group, mainly during the first year after finishing the Graduate Practice.

In 2015 (Figure 3), only every fourth individual from the group of treated individuals was unemployed during the whole or almost the entire impact period. In the case of the non-treated group, it was 62% of individuals. Seventy-six percent of treated individuals and 38% of non-treated individuals were working for 1 month. As in the first studied year, the sustainability of a working position, mainly in case of the treated group, fell sharply during the whole year. After half a year, 61% of the treated individuals and 29% of the non-treated individuals were still employed, and after 1 year, it was 41% of the treated individuals and 21% of the non-treated ones.

The situation again chained in favour of non-treated individuals after 20 months. Only 5% of the treated and 11% of the non-treated were working during the entire impact period or longer. As we have detected, the participation in the Graduate Practice does not have a long-term effect on job maintenance. The positive effect is lost approximately 1.5 years after the participation in the Graduate practice. In that time, only 24% of the treated and 16% of non-treated individuals had a job. Thus, only every seventh individual, who found a job during the impact period from the group of treated individuals, remained employed also after 2 years. In the case of the non-treated group, about fewer than half of the individuals had a job compared to the treated group, but every fourth individual from this group was able to keep his/her job.

In the following Table 7, we present the average duration of the placement in the labour market (in months). The information relates to both groups—treated individuals and non-treated individuals during the 2-year impact period and includes both years 2014 and 2015. We quantified also the net effect of the intervention as the difference between the average values of the group of treated individuals and the group of non-treated individuals.

Based on the net effect of the intervention Graduate practice, the intervention had a positive impact—employability of young jobseekers increased by 10% in the first year and by 16% in the second year. It means that individuals who participated in the intervention in 2014 remained in the labour market on average 2.4 months longer than the non-treated individuals did, in the first year and for 3.84 months considering the second year.

Discussion and Conclusions

The general aim of the measures provided to long-term unemployed jobseekers is important and useful tools of the active labour market policy. Their aim is to stimulate excluded groups of jobseekers who find it difficult to get employed without any intervention. The measures are implemented in a way that they allow job seekers to be in touch with the labour market and with potential employers, to get or renew working habits and experience, increase working skills, and mainly help jobseekers to assert themselves in the labour market. A well-functioning and efficient intervention fulfills its main purpose, in the case of our study, it helps a graduate not only to find a job but also to keep it, thus, tackling long-term unemployment.

The main framework for the active labour market policy in Slovakia is the Act on employment services. Based on the Act, we currently have 18 registered measures in operation in Slovakia. Based on the study of Stefanik et al. [3], Slovakia ranked 19th among the 27 referring EU countries in 2016 regarding invested sources into measures of the active labour market policy.

The evaluation of the effectiveness and the efficiency of interventions is reasonable, as a lot of money is spent on the implementation of interventions from the state budget. One of the most used evaluation methods in this field is propensity score matching, the technique of the counterfactual evaluation. This method is able to quantify the probability of participation of an individual in an analysed intervention based on his/her individual characteristics and characteristics of the environment, where he/she lives and works. Then based on the creation of a counterfactual situation (non-participation in the intervention), the effects of the intervention are evaluated as the difference between the results of treated individuals in the labour market and individuals who were not treated during the individual impact period following the intervention.

In the presented article, we focused on the evaluation of the intervention Graduate Practice. We evaluated the intervention applying the propensity score caliper-matching method. The data that we used in the analysis, we obtained from different organizations in Slovakia and only the intervention database included information about 42,626 participants. The evaluation of the selected intervention was performed in Slovakia and in other member countries of the EU based on the call of the European Commission. The aim of the evaluation was to quantify and evaluate the effects of the Graduate Practice considering the positioning of jobseekers in the labour market and sustainability of their job position during a 2-year impact period.

The results of the counterfactual evaluation indicate that the Graduate Practice is a useful intervention for young jobseekers to get a connection with the labour market and an opportunity to get a job position. However, the impact of the intervention on a job sustainability is rather short-term. After half a year or a full year, the intervention effect is lost and thereby the final difference between the labour market sustainability of those individuals who were treated by the intervention and those individuals who were not treated is not so obvious. The findings therefore justify us in saying that the sustainability effect is generally short-term.

We also reached an interesting finding that concerns the efficiency of the analysed intervention—the financial costs of the participants of the Graduate Practice were on average less than those who did not participate in it. The evaluation of the financial effects of the intervention on the state budget indicates that despite the fact that the additional funding was needed for each jobseeker, the amount of funding was lower in the case of the participants of the intervention. The participation in the intervention saved approximately 20% of funds spent on each unemployed individual in 2014, and up to approximately 70% in 2015 [39]. Although it was necessary to provide a financial contribution to the participants of the Graduate Practice, on average, they were not able to return money to the budget. However, the financial impacts of their unemployment on the budget were mitigated. This fact we perceive as the crucial impact of the analysed intervention Graduate Practice.

It would be useful if the counterfactual evaluations of active labour market policy measures in Slovakia become a part of the creation of the labour market policy. The results of these evaluations indicate how necessary parameters of interventions should be set up to increase their efficiency and long-term impact. Such a properly targeted and adjusted intervention will bring about the desired effect not only for jobseekers and for the labour market, but also for the public finance.

One of the limits of this study is the availability of correct data provided by the Social Insurance Agency and COLSAF. The records regarding unemployed jobseekers in databases of both institutions were missing in many cases, which forced us to exclude a part of individuals from our analysis. Hence, the results of the evaluation can be slightly biased, but we do not expect diametrically opposed findings. It would also be useful to compare the results of the revaluation using the caliper-matching technique of propensity score with other matching techniques (e.g., nearest neighbour matching), other methods (e.g., exact matching) or by evaluation using the difference in the differences method. In the future, we would like to extend the study to assess the impact of the intervention on selected groups of individuals based on their gender, degree of education or region of permanent residence. This would be useful not only to obtain partial evaluation results, but it would also give policymakers the ability to target interventions to selected jobseekers, who the intervention has the strongest impact on.

Author Contributions

Conceptualization, L.S.; Data curation, M.D. Formal analysis, M.D. and K.V.; Investigation, K.J.; Methodology, M.D.; Project administration, L.S.; Software, L.S. and M.D.; Supervision, L.S. and K.V.; Validation, L.S. and M.D.; Writing—original draft, L.S., M.D. and K.K.; Writing—review & editing, L.S., M.D., K.K., K.V. and K.J.

Funding

This research was funded by the University of Zilina, Faculty of Operation and Economics of Transport and Communications, grant Institutional research 3/PEDAS/2019.

Acknowledgments

This article has benefited from constructive feedback on earlier drafts by the editors, the chief editor, and the anonymous reviewers.

Conflicts of Interest

The authors declare no conflict of interest.

References

1. Dvorský, J.; Betáková, J.; Havierníková, K. Regional competitiveness and specific issue of the regional management. Lect. Notes Manag. Sci. 2016, 58, 21–26. [Google Scholar]
2. Kabat, L.; Hampel, D.; Grochova, L.I.; Janova, J.; Strelec, L. Alternative approaches for assessing the European countries economic and social results. Proc. Econ. Financ. 2014, 12, 273–282. [Google Scholar] [CrossRef]
3. Štefánik, M.; Džambazovič, R.; Gerbery, D.; Karasová, K.; Lichner, I.; Miklošovič, T.; Polačková, Z.; Radvanský, M.; Rublíková, E.; Studená, I. Labour Market in Slovakia 2019+; ŠEVT: Bratislava, Slovakia, 2018. [Google Scholar]
4. Miera Evidovanej Nezamestnanosti. Available online: http://statdat.statistics.sk/cognosext/cgi-bin/cognos.cgi?b_action=cognosViewer&ui.action=run&ui.object=storeID(%22i24507BF5039C40238CBE3C03481C3B1A%22)&ui.name=Miera%20evidovanej%20nezamestnanosti%20%5Bpr3108rr%5D&run.outputFormat=&run.prompt=true&cv.header=false&ui.backURL=%2Fcognosext%2Fcps4%2Fportlets%2Fcommon%2Fclose.html&run.outputLocale=sk (accessed on 30 June 2019).
5. Dietrich, H.; Moeller, J. Youth unemployment in Europe—Business cycle and institutional effects. Int. Econ. Econ. Policy 2015, 13. [Google Scholar] [CrossRef]
6. Guijarro, F. Economic Recovery and Effectiveness of Active Labor Market Initiatives for the Unemployed in Spain: A Gender Perspective of the Valencian Region. Sustainability 2018, 10, 3623. [Google Scholar] [CrossRef]
7. What Is the Youth Guarantee? Available online: https://ec.europa.eu/social/main.jsp?catId=1079&langId=en#navItem-2 (accessed on 30 June 2019).
8. Gebel, M.; Giesecke, J. Does Deregulation Help? The Impact of Employment Protection Reforms on Youths’ Unemployment and Temporary Employment Risks in Europe. Eur. Sociol. Rev. 2016, 32, 486–500. [Google Scholar] [CrossRef]
9. Russell, H.; O’Connell, P.J. Getting a job in Europe: The transition from unemployment to work among young people in nine European countries. Work Employ. Soc. 2001, 15, 1–24. [Google Scholar] [CrossRef]
10. Varadi, B. How Can We Measure the Quality of Labor Market Related Decision-Making and Policy Design in Central- and Eastern European EU Member States and What Are Its Structural Determinants? GRINCOH Working Paper Series; Paper No. 4.05.2; The Budapest Institute: Budapest, Hungary, 2013; Available online: http://www.budapestinstitute.eu/grincoh_wp4.05.2_varadi.pdf (accessed on 10 August 2019).
11. Scharle, A.; Varadi, B. Barriers to Welfare Reform in the CEE: The Case of Employment Rehabilitation Services; GRINCOH Working Paper Series; Paper No. 5.06; The Budapest Institute: Budapest, Hungary, 2015; Available online: http://www.grincoh.eu/media/serie_5_social_cohesion_and_welfare/grincoh_wp5.06_scharle_varadi.pdf (accessed on 10 August 2019).
12. Mojsoska-Blazevski, N.; Petreski, M. Impact Evaluation of Active Labor Market Programs in FYR Macedonia: Key Findings; International Labor Oragnization: Budapest, Hungary, 2015; Available online: http://www.ilo.org/wcmsp5/groups/public/europe/rogeneva/srobudapest/documents/publication/wcms_384854.pdf (accessed on 10 August 2019).
13. Borra, C.; Palma, L.; Gonzalez, M.C.; Aguado, L.F. Evaluation of an Active Labor Market Programme in a Context of High Unemployment. Desarro. Soc. 2012, 70, 93–115. [Google Scholar] [CrossRef]
14. Gonzalez Carreras, F.J.; Kirchner Sala, L.; Speckesser, S. The Effectiveness of Policies to Combat Youth Unemployment; STYLE Working Papers, WP 3.2; CROME, University of Brighton: Brighton, UK, 2015; Available online: https://www.style-research.eu/wp-content/uploads/ftp/D_3_2_The_effectiveness_of_policies_to_combat_youth_unemployment_FINAL.pdf (accessed on 10 August 2019).
15. Jespersen, S.T.; Munch, J.R.; Skipper, L. Costs and benefits of Danish active labor market programmes. Labor Econ. 2008, 15, 859–884. [Google Scholar] [CrossRef]
16. Lorentzen Fafo, T.; Dahl, E. Active labor market programmes in Norway: Are they helpful for social assistance recipients? J. Eur. Soc. Policy 2005, 15, 27–45. [Google Scholar] [CrossRef]
17. Sianesi, B. An Evaluation of the Active Labor Market Programmes in Sweden in the 1990s; Working Paper WP02/01; The Institute for Fiscal Studies: London, UK, 2001. [Google Scholar]
18. Pirciog, S.; Ciuca, V.; Popescu, M.E. The net impact of training measures from active labor market programs in Romania—Subjective and objective evaluation. Procedia Econ. Financ. 2015, 26, 339–344. [Google Scholar] [CrossRef]
19. Potluka, O.; Bruha, J.; Spacek, M.; Vrbova, L. Counterfactual Impact Evaluation on EU Cohesion Policy Interventions in Training in Companies. Ekon. Cas. 2016, 64, 575–595. [Google Scholar]
20. Potluka, O.; Bruha, J.; Vozar, O.; Spacek, M.; Loun, J.; Polakovic, R. First results of the counterfactual impact evaluation of ESF assistance in the Czech Republic. In Proceedings of the 6th International Days of Statistics and Economics, Prague, Czech Republic, 13–15 September 2012; pp. 954–962. [Google Scholar]
21. Potluka, O.; Spacek, M. Procedures and Methods of Counterfactual Impact Evaluations for the Operational Program Employment in 2014–2020; Methodology of Impact Evaluations of the ESF Programming Document 2014–2020; The Czech Evaluation Society: Prague, Czech Republic, 2013; Available online: https://www.mpsv.cz/files-/clanky/17051/Metodika_CIE_MPSV_131015.pdf (accessed on 10 August 2019).
22. Kopecna, V. Counterfactual Impact Evaluation of the Project Internships for Young Job Seekers. Cent. Eur. J. Public Policy 2016, 10, 48–66. [Google Scholar] [CrossRef]
23. Barosova, M.; Dovalova, G.; Keselova, D.; Kosa, J. Applying Active Labor Market Measures in the Context of European Union Application Practice; Final report VÚ n. 2159; MPSVR SR, Institute for Labor and Family Research: Bratislava, Slovakia, 2012; Available online: http://www.ceit.sk/IVPR/images/IVPR/vyskum/2012/Barosova/2159_kolektiv.pdf (accessed on 10 August 2019).
24. Barosova, M. Monitoring of application practice of contribution for activation activity in the form of smaller municipal services for municipality or in form of smaller services for self-governing region. In Interdisciplinary Seminar the Concept and Practice of Application Activities in Relation to the Roma Social Inclusion; UN Development Program in Bratislava, Institute for Labor and Family RESEARCH: Bratislava, Slovakia, 2013. [Google Scholar]
25. Borik, V.; Caban, M. Pilot Impact Evaluation of Selected Active Labor Market Policy Measures; MILSAF, COLSAF: Bratislava, Slovakia, 2013. [Google Scholar]
26. Harvan, P. Evaluation of Efficiency and Effectiveness of Expenditures on Active Labor Market Policies in Slovakia; Institute of Financial Policy Ministry of Finance, Economic Analysis: Bratislava, Slovakia, 2011. [Google Scholar]
27. Baboš, P.; Gerbery, D.; Hvozdíková, V.; Karasová, K.; Lichner, I.; Lubyová, M.; Miklošovič, T.; Radvanský, M.; Rublíková, E.; Studená, I.; et al. Labor Market in Slovakia 2016+; Institute of Economic Studies SAS: Bratislava, Slovakia, 2015; ISBN 978-80-7144-255-4. [Google Scholar]
28. Lubyová, M.; Stefánik, M.; Baboš, P.; Gerbery, D.; Hvozdíková, V.; Karasová, K.; Lichner, I.; Miklošovic, T.; Radvanský, M.; Rublíková, E.; et al. Labor Market in Slovakia 2017+; Institute of Economic Studies SAS: Bratislava, Slovakia, 2017; ISBN 978-80-970850-4-9. [Google Scholar]
29. Stefanik, M. Estimating Treatment Effects of a Training Programme in Slovakia Using Propensity Score Matching. Ekon. Cas. 2014, 62, 631–645. [Google Scholar]
30. Bujňáková, T.; Domonkos, T.; Horvát, P.; König, B.; Lichner, I.; Radvanský;, M.; Štefánik, M.; Tiruneh, M.W. Labor Market in Slovakia: Analyzes and Forecasts; Institute of Economic Studies SAS: Bratislava, Slovakia, 2014; ISBN 978-80-7144-232-5. [Google Scholar]
31. Domonkos, T.; Konig, B. Odhad nákladov nezamestnanosti v podmienkach slovenskej ekonomiky. Polit. Ekon. 2015, 63, 498–516. [Google Scholar] [CrossRef]
32. COLSAF SR. Employment Services Section, Management Section; COLSAF: Bratislava, Slovakia, 2017; Available online: http://www.upsvar.sk/sluzby-zamestnanosti/nastroje-aktivnych-opatreni-trhu-prace/prispevky-pre-obcana/51-prispevok-na-vykonavanie-absolventskej-praxe.html?page_id=12940 (accessed on 10 August 2019).
33. Employment Institute. Available online: https://www.iz.sk/sk/projekty/zakon-o-sluzbach-zamestnanosti/paragraf-51?print=1 (accessed on 10 August 2019).
34. Khandker, S.R.; Koolwal, G.B.; Samad, H. A Handbook on Impact Evaluation: Quantitative Methods and Practices; The World Bank: Washington, DC, USA, 2010. [Google Scholar]
35. Caliendo, M.; Copeinig, S. Some Practical Guidance for the Implementation of Propensity Score Matching. J. Econ. Surv. 2008, 22, 31–72. [Google Scholar] [CrossRef]
36. Wang, Y.; Cai, H.; Li, C.; Jiang, Z.; Wang, L.; Song, J.; Xia, J. Optimal Caliper Width for Propensity Score Matching of Three Treatment Groups: A Monte Carlo Study. PLoS ONE 2013, 8, e81045. [Google Scholar] [CrossRef] [PubMed]
37. Cerulli, G. Econometric Evaluation of Socio-Economic Programs; Springer: Berlin/Heidelberg, Germany, 2015. [Google Scholar]
38. Austin, P.C. Optimal caliper widths for propensity-score matching when estimating differences in means and differences in proportions in observational studies. Pharm. Stat. 2011, 10, 150–161. [Google Scholar] [CrossRef] [PubMed]
39. Svabova, L.; Durica, M.; Kliestik, T. Modeling the cost of unemployment for young graduates in Slovakia: A counterfactual approach. Polit. Ekon. 2019. Forthcoming article. [Google Scholar]

---

This article is governed by: