Iuventa / Výskum mládeže / Data catalogue / 2009 / OECD PISA 2009 (Programme for International Student Assessment)

OECD PISA 2009 (Programme for International Student Assessment)

Research identification sheet ID: DAYR 034

Author of research: OECD, via the consortium lead by the Australian Council for Educational Research (ACER), expert groups for particular areas (reading literacy – REG, mathematics literacy – MEG, science literacy – SEG), national coordination centres and national project coordinators, as well as Governing Board for PISA study (PGB)
Data collection period: spring 2008 (field trial), 2. – 13. 3. 2009 (main study)
Data collection: Department for International Measurements located at ŠPÚ (until 31 August 2008)/NÚCEM (from 1 September 2008) was responsible for adaptation and translation of tools: tests, questionnaires, manuals for school coordinators and administrators, instructions for study administration and coding of student open constructed responses; sampling in cooperation with ÚIPŠ, WESTAT, and ACER; executed data entry procedures; and prepared databases and outcomes on the national level
ŠPÚ/NÚCEM research team: Paulína Koršňáková, PhD. (OECD PISA 2009 National Project Manager, in attendance to the PGB meetings, responsible for the mathematics literacy and science literacy domains), Mgr. Jana Kováčová (data entry manager), Mgr. Eva Ladányiová/Daniela Heldová, PhD. (filed trial/main study, responsible for the reading literacy domain) 

Abstract

65 countries over the world participated in OECD study PISA 2009: all 34 OECD member countries and 31 other, partner countries. Slovak Republic took part in the OECD study PISA for the third time.

PISA tasks are not derived from the curricula and they were not aimed to find out, how students in respective country mastered the content-based knowledge. PISA tasks were developed according to the model of lifelong learning to assess three domains of students´ literacy – reading (the core of the PISA 2009 study), mathematical and scientific literacy.

In the OECD PISA study, the literacy means ability to use knowledge and competencies gained during the compulsory education as well as ability to present own ideas and opinions.

PISA instruments allow for analysis of student achievements in the light of other factors related to their households and schools (for example the ICT).

The main survey of the PISA 2009 study was held from 2nd to 13th of March 2009 in all Slovakian regions. More than 4555 students (born in the 1993 year) from 189 schools (ISCED 2 and ISCED 3, out of them 11 with the Hungarian language of instruction, and 5 with two languages of instruction –Slovak & Hungarian) took part in the main survey.

Representative and stratificated selection of schools and students from the Slovak Republic offers a possibility to generalise obtained results within the Slovak educational system and to compare them with the results of the other participating countries. The study results on the national level are presented in the national report PISA 2009 – Slovensko available in the Slovak language on NÚCEM web site.

PISA field trial, organized one year ago in May 2008, involved 999 pupils. That survey was focused on verification of the testing tools like test items, tests, questionnaires, with the aim to obtaining all types of students´ responses and improve instructions on their evaluation.

1. Sample Construction

Required target population for the OECD study PISA consists in each country from 15-year-old students (more precise from the 15 years and three months to the 16 years and two months old students) attending at least the 7 grade (counted from the beginning of the school attendance).

Required target population is defined to provide a picture of educational system results at the end of compulsory school attendance. Definition of 15-year-old population depended also on the date of survey. In Slovakia was the main survey administrated within the period from 1 March to 31 May and the students born in 1993 were tested.

To get an image of the target population (population of 15-year-old students in Slovakia) we used data provided by ÚIPŠ. It was a database of all basic (ISCED 2) and secondary (ISCED 3) schools together with number of 15-year-old students in the school year 2007/2008 (actually 1 year older than the target population).

It was two-level stratified selection.

Explicit variables were:
region (8 regions), and type of school (basic school, secondary school – general, secondary school – other).

Implicit variables were:
school size – according to a number of students born in 1992 (large – 35 and more, small – 18 to 34, very small up to 17), language (Slovak language, Hungarian language, and Slovak & Hungarian), school subtype (basic school1 , special basic school, 8-year grammar school, 4-year (secondary) grammar school, secondary technical school, secondary vocational school, united secondary school).

Based on these variables 24 strata were created (PISA_STRAT).

_____________________________

1 Within 3 regions (Banskobystrický kraj, Košický kraj, and Prešovský kraj) the basic schools were split into two subgroups according the percentage of grade repeating students (less than 5 %, and more than 5 %).

Certain groups of schools and students could have been excluded from the target population; however criteria for exclusion were very strict. According the technical standards of the OECD PISA it was possible to exclude the whole schools all together attended by maximum of 2,5 % students and another 2.5% students could be excluded within schools.

The whole schools exclusion was possible from the technical reasons (0,5 %) – logistically, and/or financial too demanding survey administration. Another 2 % of students could have been excluded via exclusions of whole schools attended EXCLUSIVELY by students with special educational needs (severely challenged students) or limited knowledge of the test language. In the Slovak Republic were in such case excluded the schools with two different languages of instructions: Ukrainian language of instruction and Bulgarian language of instruction, as well as some special basic and vocational schools (all together 315 schools – 1.803 students).

As for the within school exclusions of students, it was allowed to exclude certain students with special needs or limited knowledge of the language of instruction.

The total target population was set on 70.651 students, which was 97,51 % from the whole 15-year-olds population. To define the sample for the Slovak Republic this number was used. Schools were ranked according to their size, proportionally towards the whole population. The defined number of sampled schools was 193 schools with 5.255 students.

To be successful in administration of the OECD survey PISA, 85 % from sampled schools had to participate. 35 students were sampled in each of those schools (in the case of the small and very small schools all students were sampled if they attended at least the grade 7). The participation of 80 percent of chosen students was required.
4555 students from 189 schools participated in the main survey of the OECD study PISA 2006.

2. Methodology of Data Collection

14 test booklets were administered in the main survey of the OECD study PISA 2006 to assess the reading, mathematical and scientific literacy. Tests were administered according the given procedure in the language of instructions by trained teachers (school associates – school coordinators and administrators). Each student had 120 minutes to solve his/her test and worked on four groups of units. In the year 2009, two to three of these groups assessed reading literacy. The other groups contained units assessing mathematical and science literacy of student. The PISA test contained items requiring open constructed responses as well as multiple choices or complex multiple choices. Items within one unit were related to a stimulus (a text and/or graphics).

After finishing tests students filled out also questionnaires (45 minutes) focused on students´ background, reading interests, metacognition-strategies and motivation in the field of formal education. ICT questionnaire was the last part of the Student Questionnaire.

Directors of participating schools filled out School Questionnaire focused on the context of education and school infrastructure.

3. Results

Outcomes of the SLOVAK SCHOOL SYSTEM ARE IN THE AVERAGE of OECD in the domain of mathematical literacy AND statistically significantly UNDER THE AVERAGE of OECD in all of the remaining PISA domains: reading and also scientific literacy.

Reading Literacy

The average score achieved by Slovakia in the year 2009 was 477 points (level 2 out of six possible levels). Compared to the year 2006 (466 points) was the score of Slovakia 11 points higher, compared to the year 2003 (469 points) was the score of Slovakia 8 points higher. However, the overall achievement did not change statistically significantly (the student achievement remained under the OECD average).

The final ranking of Slovakia was almost unchanged:

  • from 25th to 29th position among 34 OECD member countries,
  • from 32nd to 37th position among all 65 participated countries.

Mathematical Literacy

The average score achieved by Slovakia in the year 2009 was 497 points (level 3 out of six possible levels), which was enough to achieve the OECD average (496 points).

The overall achievement did not change statistically significantly at the national level, but the final ranking of Slovakia was slightly improved:

  • from 13th to 22nd position among 34 OECD member countries;
  • from 19th to 28th position among all 65 participated countries.

Scientific Literacy

The average score achieved by Slovakia in the year 2009 was 490 points (level 3 out of six possible levels). Compared to the year 2006 (488 points) was the score of Slovakia 2 points higher. However, the overall achievement did not change statistically significantly (the student achievement remained under the OECD average).

The final ranking of Slovakia was almost unchanged:

  • from 23th to 29th position among 34 OECD member countries;
  • from 29th to 37th position among all 65 participated countries.

4. Recommendations

The recommendations for the reading and mathematical literacy improvements formulated in relation to the OECD studies PISA 2003 & PISA 2006 results analyses are still valid (for details see DAYR 009 & DAYR 016, even more is available in Slovak language in an electronic version of the PISA thematic reports focused on mathematical and reading literacy – for details see www.nucem.sk).

5. Data Processing

The data obtained from questionnaires are presented in INT_Stu09_SVK_Dec10.sav and INT_Sc09_SVK_Dec10.sav. To use the students´ data, it is necessary to weight data with the final student weights (w_fstuwt). Except the final student weights there are also 80 replicate weights, which serve for computation of standard errors. This model is based on two-stage sampling and uses Fay’s modification of the BBR (Balanced Repeated Replication).

Cognitive data obtained from test booklets are presented in the file INT_Cogn_S_09_SVK_Dec10.sav (scored). Items included in assessment had different difficulties. In survey OECD PISA there is used so called Item response theory (IRT) for estimation the difficulty and competency. IRT is a mathematical model which provides the estimation of probability, that the answer of a student to a particular question is correct. This model creates a continuum (so called Rasch scale) on which both student performance and item difficulty will be located and a probabilistic function links these two components. On the basis of Rasch model five plausible values (PV1 – PV5) were attached to each student for each domain separately (they are included in the file INT_Stu09_SVK_Dec10.sav). To analyze the students´ data, which are concerned to performance, it is necessary to use the five sets of PVs.

More detailed information related to the theoretical background is available in English language:

PISA 2009 results are available in/on:


Data files for SVK in SPSS:

Registration
baner_registracia_komprax