Iuventa / Výskum mládeže / Data catalogue / 2003 / OECD PISA 2003 (Programme for International Student Assessment)

OECD PISA 2003 (Programme for International Student Assessment)

Research identification sheet ID: DAYR 009

Author of research: OECD, as a member of the consortium lead by the Australian Council for Educational Research (ACER), expert groups for particular areas (reading literacy - REG, mathematics literacy - MEG, science literacy - SEG), national coordination centres and national project coordinators, as well as Governing Board for PISA study (PGB)
Data collection period: spring 2002 (field trial), 7. - 15. 4. 2003 (main study)
Data collection: ŠPÚ (ŠPÚ was responsible for adaptation and translation of tools: tests, questionnaires, manuals for school coordinators, instructions for study administration and evaluation, sampling in cooperation with ÚIPŠ and ACER, data entry procedures, making  databases and outcomes on the national level)
ŠPÚ research team: Paulína Koršňáková, PhD. (OECD PISA national project manager from spring 2002), Ing. Alena Tomengová (PISA 2003 national project manager from 2001 to spring 2002 and in 2004 as a Slovak Republic representative in PGB), Ing. Martin Swan (data entry manager, responsible for statistical outcomes on the national level)

 

Abstract: 

In 2003 46 countries over the world participated in OECD study PISA and 7500 15-year-old students from 282 all types of schools were involved for the first time in this study from all Slovak regions. (PISA field trial, organized one year ago, was focused on verification of the testing tools like test items, tests, questionnaires, obtaining all types of students´responces and improving instructions on their evaluation). Representative and stratificated choice of our students gives us a possibility to generalise obtained results within our educational system and to compare with the results of the other participating countries.  PISA test items neither come out of syllables, nor find out content-based knowledge of students in particular countries. It is not possible for students to learn all the facts needed in their adulthood, but they must obtain an ability to use what they know in practice and to study. For this reason they need basis provided by reading literacy, mathematics literacy (domain of PISA 2003), science literacy and problem solving. The study results on the national level are discussed in several publications available on ŠPÚ web site (PISA SK 2003 - National Report,  PISA SK 2003 - Mathematics Literacy (report) (35.32 MB) ,  Test Items 2003 - Mathematics (items collection) (641.75 KB), PISA SK 2003 - Reading literacy (report), Test Items 2000 - Reading (items collection),  PISA SK 2003 - enclosure of Pedagogické rozhľady (1.27 MB)). 

1. Construction of a Chosen Sample: 

Required target population for OECD study PISA consists in each country from 15-year-old students at least from 7 grade.

Required target population is defined to provide a picture of educational system results at the end of compulsory school attendance. Definition of 15-year-old population depended on the date of testing. In Slovakia testing was administrated from 7 April 2003 to 18 May 2003, the tested students were born in 1987.

It was two-level stratificated choice.

Explicit variables: region (8 regions), programme (basic school, secondary school), school size -according to a number of students born in 1987 (big - 36 and more, small - 18 - 35, very small unto 17).

Implicit variables: language (Slovak language, Hungarian language), school type (basic school, special school, 8-year grammar school, 4-year grammar school, secondary technical school, secondary vocational school, vocational school), authority (state, private, church).

Based on these variables 20 strata were created (PISA_STRAT).

01-08 basic schools big according to a region, 09 - small basic schools from all regions, 10 - very small basic schools from all regions.

11-18 secondary schools big according to a region, 19 - small secondary schools from all regions, 20 - very small secondary schools from all regions.

Certain groups of schools could have been excluded from the target population, criteria for exclusion were very strict. In the Slovak Republic were excluded the schools without 15-year-old students (70 schools), a part of special schools (146 schools), as well as students with a language of instructions other than Slovak or Hungarian (55 students). The total target population was 81 132 students, which was 98,8 percents from the whole 15-year-old population. To define the sample for the Slovak Republic this number was used. Schools were ranked according to their size and proportion towards the whole population. The defined number of chosen schools was 193 with 5250 students. Because of ahead defined stratification according to regions the number of schools increased to 288 schools. It was a demand of the national centre to get from research.

To get an idea about the basic file (population of 15-year-old students in Slovakia) we used data provided by ÚIPŠ. It was a database of all basic and secondary schools together with number of 15-year-old students in 2002 (actually 1 year older than the target population).

To be successful in administration of testing, 85 percent from asked schools must have participated. 35 (or less) students were chosen from each of these schools by chance. The participation of 80 percent of chosen students was required.

Finally, 7346 students from 281 schools participated in the study.

2. Methodology of Data Collection: 

Students filled out test booklets and questionnaires, a director of school filled out school questionnaire. Test booklets contained questions focused on mathematics literacy (50 % of testing time), reading literacy and science literacy in the area of problem solving.

Student questionnaire and  ICT questionnaire (386.73 KB) consisted from parts about: a student and his family, education, school, studying mathematics, mathematics lessons and its teaching, school attendance and using information technologies.

External administration was supplied. Students had 120 minutes to solve test items and 45 minutes to answer the questionnaire.

3. Data Recording: 

Data stated in questionnaires are located in the files ziacky.sav and skolsky.sav. Students data are weighted (w_fstuwt), the files except of mentioned final weights also contains 80 replicated weights (from the reason of 2-level sampling closer towards weighting in document (2.78 MB)). 

Some indices from several test items were constructed by factor analysis and IRT (more details in document (3.05 MB) (Annex A1: Questionnaire indices), which are also included in mentioned files.

Cognitive data obtained from test booklets are presented in the file kognitiv.sav. These data were summarized on scales for particular domains with the average 500 and standard deviation 100. Each of participating students answered only a part of test items. Tasks have a various difficulty. A process used for OECD study PISA to reach difficulty and skills is called Item response theory (IRT). IRT is a mathematics model used for an estimate of probability, that a certain person will have the correct answer to the particular test item from the specified file. This probability is modelled on the whole continuum, which includes an individual performance and also item complexity, its difficulty. This difficulty continuum and performance is called „scale". Item difficulty and a student score are on the same scale. Items reaching students abilities are labelled in the file ziacky.sav as so called "plausible values" (more details in  document (2.78 MB)).  

  • Data file in SPSS:
    - ziacky.sav (13.3 MB)- skolsky.sav (180 kB) - kognitiv.sav (11 MB)

Reading Literacy in PISA 2003 study:

FINDINGS

  • The average of the Slovak Republic in this domain is statistically significantly bellow the OECD average;
  • Questions to incoherent text were very difficult for our students, probably at our schools they do not work enough with graphs, tables and forms;
  • Students had aproblem to form their own answer, they often did not answer open-ended questions;

Recommendations

  • Within the future teachers´preparation to implement a methodology of reading literacy skills into educational plan of all students;
  • To improve conditions for development of students reading literacy in smaller municipalities up to 15000 inhabitants, to provide them with more resources and easier materials availability;
  • To provide more help to schools with compulsory school attendance of children from social disadvantaged background (higher normative, food for free, etc.).
  • To implement more differentiated attitude towards students;
  • To create a space for individual work with students and care for students with special educational needs;
  • To enlarge space for written and oral expression of pupils;
  • To implement reading literacy levels among all general educational subjects aims and educational standards together with worked-out methodology of their evaluation;
  • To focus on boys as agroup while developing their concentration, consistency and patience within reading literacy.

Mathematics Literacy in PISA 2003 study:

FINDINGS

  • The average of the Slovak Republic is not statistically different from the OECD average;
  • Results obtained from our students in the areas stressed out in our curriculum were above the average;
  • Students reached poor results in solving tasks, which they did not practice on mathematics lessons, (e. g. from probability and statistics).

Recommendations

  • To provide more space for students argumentation and problems solving;
  • To create new items and items collections from real life, which provide some space for knowledge application;
  • To change the curriculum and revise educational standards for elementary schools;
  • To catch up gifted students;
  • To find reasons of differences between boys and girls performances.

Science literacy in PISA 2003 study:

FINDINGS

  • The final result of our students is average in the contrary with higher number of lessons in comparison with OECD countries average;
  • Students had aproblem to form answers to open-ended questions;
  • Students results have astrong relation to their reading literacy levels.

Recommendations

  • To focus on understanding, interpretation and evaluation of provided information, to form conclusions.

Problem Solving in PISA 2003 study:

FINDINGS

  • The Slovak Republic in testing of problem solving tasks was significantly bellow the OECD average;
  • It is aresult of a relatively low level of our students reading literacy.

Recommendations

  • To train children in reading literacy to understand various text types, core problems and in successful finding relevant information;
  • To train how to make experiments and learn from mistakes.
Registration
baner_registracia_komprax