New Arrivals Program (NAP)

Task 1:

Evaluation and Description:

The fresh arrivals into any community need help and support to settle down into their culture and traditions. This is by no means an easy task but it was undertaken by the Department of Education and Children’s Services (DECS) for new comers in South Australia. The New Arrivals Program (NAP) is a procedure devised for children who come to Australia for learning the native language and subsequently get prepared to walk abreast the locals in academic activities. It not only helps the children to learn English along with correct grammar and pronunciation, but also provides them with the basics to study in South Australia.

Organization:         

This program (NAP) has established centers all over South Australia to facilitate the new comers. They cater for students from the primary, secondary and higher secondary sections. There are separate centers available for students according to their age groups and prior qualifications.

Aims of the Program:

The program aims to achieve the following during its course,

1. To prepare students for achieving good grades in academic activities
2. Ensure that the best and most appropriate curriculum is selected for the students.
3. Develop the confidence to live in the Australian community.

Guiding Principles:

The guiding principles of the program have been set to ensure

1. Intensive support in language training with specialist teachers.
2. Personal involvement in terms of reviewing the syllabus periodically.
3. Generating various activities within the community and in school namely social events and/or excursions, trips etc.
4. Support in terms of bilingual exercises in classes.

Duration and Eligibility:

The course has been especially designed keeping in view the basic requirements of the newcomers from foreign countries. It caters to the deficiencies based on individual needs and works on them to build confidence and improve learning.

To be eligible for the program students need to enroll for the New Arrivals Program within 12 months of arrival in Australia. This may be extended to 18 months if the students have entered a reception of 1 year.

Some other specifics of the program include that all kinds of temporary visa holders are not eligible to enroll. A visa holder may also be required to pay for the services.

Evaluation Focus

The purpose of the evaluation is to identify the strengths and weaknesses of the program on new arrival students in primary schools in South Australia. It will be a formative assessment with qualitative research conducted on both the students and teachers. The results will be used to modify and enhance the design and content of the program and to assess its viability and performance so that it becomes more effective and efficient for the foreign students being enrolled in it.

TASK 2:

Program Theory Evaluation:

The prediction of the possible outcomes of a program or theory based on some underlying assumptions has been a very effective tool in the evaluation of that program. Research over the last few decades has not only suggested positive outcomes through such evaluations but also has improved the anticipated results.

Program theory evaluation is a principle or program which causes the intended results and offers a solution via evaluation guided by the program itself. It can be broadly divided in two portions: First is the conceptual component while the other is the empirical one.

Some other techniques or similar criteria have also been used to develop various evaluation based ideas and terms in which theory based evaluations (Weiss,1972) and also program logic (Funell, 1997) are some significant ones.

Central Idea:

The program theory of NAP goes with the following concepts,

1. A good and effective program will help new arrival students to excel in the fields as mainstream students and to make a good base for themselves and the community.
2. The program would help new arrivals to understand Australian culture, values and social norms in a more formal way.
3. The program would promote the idea of the hospitality and open-mindedness of the Australian community towards other races and their welfare.

Activities of the Program:

The main activities in the program can be divided into three main streams as per the qualification and age of the incoming students. The salient features of the programs according to the segregation are:

• Primary New Arrivals Program: This program caters children who are within the age group of 5 to 12 years. The focal points are:

1. Small classes and centers to focus individual attention.
2. Basic primary subjects are taught with special emphasis to English language.
3. Transportation facilities are provided on account of various factors.
4. Children are helped to build confidence and enter the mainstream of local schools as soon as possible.
5. Study opportunities in terms of effective transformation to citizens that can easily adjust to the new culture.

This new arrival program is focused on new arrival students so as to enable them to settle in Australian schools and to pursue further opportunities there during their studies. The primary school students at their young age will learn about Australian culture that will definitely help them to pursue their careers in a multitude of career fields.

Intended Outcomes of the Program:         

The goals for the intended outcome of this problem are:

• Short Term: the short term goals of the program are:

1. Ensure proper settlement of the new comers.
2. Help new comers to study and compete equally with local students.

• Long Term:

The long term goals of the plan are:

1. To benefit from the new comers in all aspects by settling them in the mainstream so that they are not a liability but an added advantage to economy etc.

Task 3

Framing the Evaluation:

Intended Users of the Program:

1. Primary:         The primary anticipated users of the program are the foreign immigrants who come to Australia either to study or to settle down permanently. Their smooth transition from diverse backgrounds to the Australian culture, traditions and social values is the primary aim of the program.
2. Secondary:     The secondary users of the program are the funding bodies which are the educational department of the government of South Australia. They are also the main stakeholders in the program because the more the number of graduates or ideally post graduates they can extract through this program, the better it would be for the overall economy and national interests.

Thirdly, this program has opened up a variety of job openings for many locals. The New Arrival Program Centers have several branches each having managerial and teaching opportunities for the locals that are related to educational background. The staff can hence be categorized as users of the program. 

Intended Users of the Program:

The user of the evaluation as the purpose is mainly the audit of the program is the government which has funded the program. Therefore the foremost requirement of the evaluation is of the government agencies who are auditing the public funds and whether they are being utilized in a proper way.

Purpose of the Evaluation:

A good evaluation covers the key aspects of the evaluand which are mainly based on efficiency, effectiveness and appropriateness. The purpose of this evaluation along with a clear concept of the above mentioned criteria are defined here.

Efficiency which can otherwise be defined as the value for money is the criteria of the investing body i.e. in this case the government whereby it would be decided whether the program is cost effective or not. Also the amount of inflow of finances must in some form be less than the intended outcome.

Effectiveness or the extent to which the goals are met is another main objective of the evaluation. The more the number of foreign students who benefit from this program, the more effective the program would be.

The third and the main criterion is defining how appropriate the program is. This may be called the extent of how compatible the outcome of the program and the public or individual needs are. In this case, the evaluation should aim at the procedures or curriculum of the program to decide whether it is designed appropriately or not and whether it covers all the aspects which need to be covered comprehensively or not. Also, whether in covering all the aspects, the level of difficulty has been kept to an appropriate level so as to cater to all kinds of students with diverse educational backgrounds.

Since the evaluation will be formative in nature, there will be room to alter the program as the need arises through the evaluation so that the program is able to cater to the needs of its primary and secondary users.

Key Evaluation Questions:

The main questions that need to be addressed by the program are:

1. Was the program effective in teaching some practical things to students?
2. How many students were able to enter the mainstream and compete with the local students?
3. What was the average performance of the attendees of the program in the local schools?
4. Were the students able to adjust in the community and understand the local customs/traditions?
5. How well do the students comprehend and write English after attending the program?
6. What is the main difference between the performances of students from different regions?
7. How is the program more effective for some students than others?
8. Why does the program have diverse effects on various students?
9. Has the program been efficient i.e. whether it is cost effective or not?
10. Is the curriculum appropriate to cater to students from all kinds of educational and cultural backgrounds?
11. Has the program brought about a significant change in the behavior and confidence of the attendees once they end up in the local schools? 

Establishing the Criteria for Success:

            The success of the program will be based primarily on whether the key evaluation questions are appropriately answered. This would be:

1. The number of students who have successfully completed the program and have adjusted in the mainstream with the local students.
2. The number of students who are able to speak and write English effectively after the program.
3. The children who adjust in the local community and are accepted there also.
4. The number of graduates and post graduates among these students who serve as a boost to the national economy in general.
5. The cost effectiveness of the program whether the inflow of finances balances the desired goals of the program.

TASK 4:

Evaluation:    Evaluation is a very diverse term which covers a lot of aspects. One classic definition of evaluation is explained by Sanders and Nafgizer as a planned and comprehensive procedure of assessing the success of a task or project in meeting its objectives and to give a clear picture of the lessons learnt.

As defined by Weiss (1972), evaluation is a measure of the extent to which goals have been achieved in a particular timeframe as compared to the inputs. 

The primary aim of the evaluation is to identify the strengths of the program and re-enforce them. Similarly it is also to identify the weaknesses and shortcomings of the program and make a planned effort to rectify them. The procedure keeps on repeating and the program keeps on getting filtered which improves it further.  

Evaluation Theory:

 The evaluation has to be based on some theoretical grounds for the support material to be realistic and helpful. Therefore for the evaluation the ‘Formative’ type can be used as it matches the outcomes desired from the evaluation the most. In this type of evaluation the intended outcome is aimed at slight improvements in the program or slight modifications for overall betterment.

Data is collected and is analyzed critically to ascertain which parts of the program are working up to the required standards while which ones are not. The key factor here would be addressing the ‘why’ part of the shortcomings. This will subsequently lead to the recommendations from the evaluation which may result in modification of procedures wherever required. Furthermore the data collection is based on primarily interactive sessions with the main stake holders of the program.

General Evaluation Theories:

Some general theories of evaluation are discussed as under:

1. Participatory Evaluation:

The participatory evaluation indicates that the stake holders and all the parties which are directly or indirectly involved participate in the evaluation. Their direct input in this regard not only helps the evaluation but also is fruitful for the outcomes and improvement.

2. Action Research:

This is another effective way of research based on cycles of action and good research. Apart from these a good plan of action is deemed as the base of action research.

3. Empowerment Evaluation:

This type of evaluation empowers the stake holders and encourages them to carry out their own evaluation. This self-analysis kind of activity is an effective way to analyze the mistakes and provide a realistic solution to the evaluation.

4. Realist Evaluation :

Another theory in terms of effective evaluation is that of realistic evaluation. Tilley and Pawson have introduced this method to address deficiencies in exercising of experimental designs in different programs which have a wide variety in terms of social and economic effects. The basic theme is to address what kind of actions to be taken in what kind of circumstances.

Data Collection:

The evaluation which is formative in nature has to be supported by facts and figures of the structured program. The inputs are required from every major stakeholder and have to be comprehensive to draw meaningful results. The authenticity of the data coming directly from the sources and the selection of information to be extracted needs to be done in a very professional way so as to carry out a justified evaluation.

The data that has to be collected will be accomplished in three main forms:

1. Surveys
2. Focus Groups
3. Structured Interviews

Surveys:

Surveys are one of the most effective ways for data collection. The surveys conducted will be through closed ended questionnaires emailed 50 participants. The following will be the respondents:

• Past participants: The first group would comprise of the past participants of the program who have gone through the activities and have a fair idea of how it works. The practical experience of this group would come in quite handy in the evaluation process.
• Present Participants:  The present attendees of the program would comprise of the second group for the survey who are presently attending the program. This would be a helpful insight to the real effectiveness of the program and how children are benefitting from it.
• Teaching Faculty: The teachers of the program would constitute the third group in the survey. The inputs of the teachers who are directly in contact with the participants and are fully aware of the ground realities would be of great value in evaluating the efficiency of the program.

Focus Groups:

The next step for data collection is the random selection of focus groups in a New Arrivals Program Center. This would be targeting mainly the teaching staff in 3 centers so that a healthy discussion may be generated. The inputs of different teachers and sharing of experience would lead to suitable solutions of mutual problems. This would further identify the main and most critical problems and short comings of the program.

Interviews:

The last tool for data collection is that of interviews. These would be conducted for mainly the heads of the centers and senior teaching staff. These interviews would be helpful in providing a rare insight in the problems faced in the program mainly from the administrative perspective. It would also help to resolve and identify the critical strengths and weaknesses of the program.

Analyzing the Collected Data: 

The next logical step, when the data has been collected, is to analyze the data correctly so that meaningful results can be extracted from it. This will be done by:

1. Arranging the data in tabular form and separating the inputs according to the broad categories as mentioned above.
2. Present the data of the survey conducted into a graphical form and analyze the trends.
3. Draw the main conclusions from the discussions generated in the focus groups and compile the descriptive results in tabular form.
4. Collect the salient features of the interviews and tabulate them in descriptive form.

A sample of the questionnaire of the interviews is attached as the general guideline for the interviews. A similar kind of discussion based on the same points would constitute the discussions in the focus groups.   

TASK 5:

Understanding the Causes:

The evaluation process is a very important tool in analyzing the health of a program. But where analysis procedures are implemented a great deal of responsibility also lies with the evaluators in understanding exactly what is required of them. As the evaluation could make or mar the image and overall reputation of the program, therefore, the evaluators must be very careful in their judgment. Correct analysis of available data, its correct interpretation and then logical steps for the improvement and efficiency of the program are the basic requirements from all evaluators. Therefore a lot of emphasis is laid on the fact that ‘understanding of causes’ is a prime objective.

Casual Attribution:   Sometimes the impacts of a program have not been caused by that particular program but by a sequential or disturbed combination of programs. It is also possible that this may as well be categorized as a ‘Casual Attribution’ whereby the outcomes are not in conjunction with the desired results (Better Evaluation website, 2012).

Induced Outcomes:   One other fact which needs to be addressed in evaluations and interventions is to monitor whether the evaluation itself has had an impact on the program. Sometimes the results, when analyzed, show a tendency which had not been present before the evaluation. So these outcomes may be attributed to the evaluation itself rather than the internal factors.

In order to assess any casual attribution between the defined goals of the New Arrivals Program and the outcomes of the evaluation, an effort was generated to find out any possible alternative explanations and to justify the counterfactual data. A reasonable conclusion can be reached by going over these exercises efficiently.

Asking Informants to Attribute to Causality:       

The survey technique is used to additionally ask participants of their specialized views and the results are used to refine the outcomes.

Compare the Outcomes with Results:        

The results of the surveys will be compared to the actual intended outcomes of the program and the deviations will be accordingly sorted out.

 Non-Experimental Techniques       

Non-experimental techniques like key informants who could be expert educationalists or experts within the community could be consulted to comment on the possible outcomes of the program.

General Elimination 

The ruling out of other or alternate explanations could very well be carried out on the basis of the surveys and the data collected. The general elimination rule could be applied to get rid of the counter arguments which are not supported by logic or facts.  

TASK 6

Synthesis:

After establishing the facts and the ground realities of the evaluation the next task is to synthesize the data in a meaningful pattern so that some logical conclusion can be extracted from it. In other words the summary of evidence collected in a coherent way.

Some techniques that can be applied to the collected data like:

Cost Effective Analysis:

A very important technique for judging the credibility of the data and evaluation is the cost-effective analysis of the program. If the students who benefit from the program are taken as a sample and the cost incurred on each individual is taken out separately as one unit then this can be called as the expenditure. It has to be compared to the number of students per course who end up performing on average grades in the mainstream educational system.

Value for Money:

The cost incurred may not be fully recoverable but here the concept of value for money comes into play which is the deciding factor for whether the program is effective or not. Even a very little percentage of individuals benefitting from the program reach important positions in the society at a later stage and prove to be extremely beneficial in diverse fields it can still be claimed that ‘the effort generated was worth the money invested in it.’

Statistical and Analytical Generalization:

The data collected for the evaluation in terms of surveys and discussions in a focal group can be synthesized in terms of a statistical analysis. This implies that a general point of view about the main concerns of the sample taken may be judged and suitable results can be extracted from them. Similarly the salient points and features of the discussions in focal groups may become the basis of the synthesized data and those points may be considered for future improvements in the program.

For Full Assignment:

[maxbutton id=”6″]

[maxbutton id=”1″]