Design and Development of Lifelong Skills-Enhancement e-Programmes Using Monitoring/Evaluation Tools: Exemplars with Policy Recommendations

. Developing ‘Future Ready’ learners in preparation of Industrial Revolution (IR) through implementation of lifelong skills-enhancement programmes collaborating with various sectors is the recent global governmental aspiration. In response to call for quality technology-enhanced ‘Science, Technology, Reading, Engineering, Arts, Mathematics’ (ST REAM) education, SEAMEO RECSAM initiated the ‘Learning Science and Mathematics Together’ [LeSMaT (Borderless)] project-based programme under the Golden SEAMEO Basic Education and Student Networking involving blended-mode lifelong education. This article reports SEAMEO Inter-Centre Collaboration (ICC) Education 4.0 project initiative as an offshoot programme of LeSMaT involving Design and Development Research (DDR) in developing lifelong skills-enhancement e-programmes integrating STREAM education with evidence-based output under sub-themes identified from LeSMaT(Borderless) involving ‘Analysis, Design, Development, Implementation, Evaluation’ (ADDIE) instructional model. ‘Analysis’ of these sub -themes were made to develop criteria as guiding focus for project teams to design technology-enhanced learning (TEL) output that could showcase the knowledge/skills required during IR4.0. Literature research was also made on existing e-programmes fulfilling SEAMEO’s priorities. During ‘Design and Development’ phases, e-surveys were developed as monitoring/evaluation tools for tracking of skills-enhancement e-programmes in line with Sustainable Development Goals (SDGs). During ‘Implementation/Evaluation’ phases, qualitative/quantitative data collection/analysi s methods were implemented involving case study and validation of e-survey entitled ‘Motivation towards STREAM education’ (MoToS). The qualitative analysis integrating ‘type 4’ multiple -case design includes analysing output illustrating curriculum innovation through transdisciplinary studies reflecting Education 4.0 and SDGs whereas quantitative method involved Rasch model to validate MoToS to monitor/evaluate participants’ engagement in 1 st Regional Workshop on SEAMEO-ICC Education 4.0. The findings usin g Rasch analysis in the ‘Evaluation’ phase revealed that MoToS is reliable with measure of CA 0.98 internal consistency and ‘feeling stressed on STREAM’ is the most difficult item. After the e -course series 2020-2022, participants' output was examined usi ng ‘Cross - Case Analysis’ (CCA), ‘Within/Exemplary - Case Analysis’ (WCA/ECA). The e-course series produced evidence-based SDG-related outputs with exemplars integrating SEAMEO Priority Areas No.7 and No.5. Policy recommendations and suggestions for future studies related to Education 4.0 are discussed including developing innovative programmes to improve transdisciplinary quality education (328 words) .


INTRODUCTION
Developing 'Future Ready' learners is the global governmental aspiration especially in the educational setings.In the advent of digital era preparing towards Industrial Revolution (IR) 4.0, it is essential for SEAMEO's Ministries of Education (MoEs) to invest on lifelong education in collaboration with various sectors including the industries to prepare younger generation for sustainable future through implementation of lifelong skills-enhancement programmes.
In response to call for quality technology-enhanced learning in particularly the emerging aspired 'Science, Technology, Reading, Engineering, Arts, Mathematics' (STREAM) education, SEAMEO RECSAM initiated the 'Learning Science and Mathematics Together' in a Borderless world [abbreviated as LeSMaT (Borderless)] project-based programme under the Golden SEAMEO Basic Education and Student Networking involving blended-mode lifelong education with ongoing Research and Development (R&D) activities in accordance to the 17 SDGs [1] [2] focusing on SEAMEO's Education Agenda 7 Priority Areas [3].
This article reports part of larger scale study on SEAMEO Inter-Centre Collaboration (ICC) Education 4.0 project initiative as an offshoot programme of LeSMaT involving Design and Development Research (DDR) [4] [5] in developing lifelong skillsenhancement e-programmes integrating STREAM education with evidence-based output under sub-themes identified from LeSMaT(Borderless).The DDR processes involved 'Analysis, Design, Development, Implementation, Evaluation' (ADDIE) instructional model.'Analysis' of these sub-themes were made to develop criteria as guiding focus for project teams to design technology-enhanced learning (TEL) output that could showcase the knowledge/skills required for potential career employability during Industrial Revolution (IR) 4.0.
Mixed-research study was conducted on students' motivation participating in STREAM related studies/career involving exemplary output demonstrating various models of skills under LeSMaT's sub-themes.These sub-themes were introduced during another offshoot programme entitled 'Learning Transdisciplinary Science Integrating Mathematics, Arts-Reading-Language-Culture, Engineering-Environmental Education-Economics-Entrepreneurship, Technology' (LearnT-SMArET) being implemented through e-course series since 2018 with output to be elaborated in this paper.The subthemes identified to promote STREAM are directly/indirectly related to Education 4.0 as guiding focus for project teams to develop TEL output that could showcase enhanced knowledge/skills required for their potential career employability.Hence, the following research questions are identified to guide this study: (1) What are the Design and Development Research (DDR) stages involved in developing lifelong skills-enhancement e-programmes with attributes of knowledge/skills identified to enhance 'Future Ready' learners' motivation and confidence on STREAM related capabilities required in the era of IR4.0? (2) Are there exemplary output with policy recommendations leveraging on the findings from 'Motivation towards STREAM' (MoToS) education instrument validation that may provide insights on the future direction of STREAM education in line with SDGs?

MATERIALS AND METHODS
Design and Development Research (DDR) [4][5] is the methodological framework of this study aiming at developing lifelong skills-enhancement e-programmes involving 'Analysis, Design, Development, Implementation, Evaluation' (ADDIE) instructional model integrating STREAM education with evidence-based output under sub-themes identified in LeSMaT(Borderless).The 'Product and Tool Research' of DDR's cluster is chosen involving the 'Specific Project Phases' through ADDIE with development of monitoring/evaluation tool [4] including learning module as one of the output of DDR [5].
'Analysis' of these sub-themes (including literature research conducted) were made with to develop criteria as guiding focus for project teams to design technology-enhanced learning (TEL) output that could showcase the knowledge/skills required during Industrial Revolution (IR) 4.0.During 'Design and Development' phases, e-surveys were developed as monitoring/evaluation tools for tracking of skills-enhancement eprogrammes in line with Sustainable Development Goals (SDGs).
During 'Implementation and Evaluation' phases, mixed-mode of qualitative data (mainly from documentary analysis, interviews, observation and open-ended responses) and quantitative data (mainly from survey questionnaires) collection/analysis methods [6] were implemented with validation of e-survey entitled 'Motivation towards STREAM education' (MoToS).'Multiple-case design' [7] of qualitative data collection approach was involved to overcome the issue of generalizability just based on a single case study, also to corroborate the evidence that enhances of validity of the study [8].
Cross-Case and Exemplary-Case analyses [9][10] were implemented to analyze case exemplars that are output of LeSMaT(Borderless) programme that is aspired to have regional integration and sustainability integrating transdisciplinary studies and assessment techniques including project/problem-based lifelong learning activities.This article reports on exemplary output (including prototype and/or publication) illustrating curriculum innovation in Education 4.0 through transdisciplinary studies reflecting SDGs and SEAMEO's priorities areas (abbreviated as PAs).The following are 4 (out of 9) subthemes that serve as guide for project teams to develop technology-enhanced learning (TEL) output that could showcase their enhanced knowledge and skills required for IR4.0 with potential career employability.To achieve curriculum innovation to promotes sustainability in preparation for IR4.0, literature review is made on the interlinkages among SDGs as adopted during the General Assembly by the United Nations with the 2030 Agenda for Sustainable Development [1] that emphasise on sustainable use of natural resources for basic needs and the importance to conserve these resources.SDGs are broadly framed as 17 separate goals and diverse elements including 169 targets as well as 230 indicators in which the types of interventions are governance (67%), environmental (21%) and social (12%); also the domains are education (17%), gender equality (17%), sanitation (12%) and climate change (12%) for Social Determinants of Health (SDH) actions (Pega, n.d.).For instance, SDGs No. 6, 13, 14, 15 were classified as Biosphere, SDGs No. 1, 2, 3, 4, 5, 7, 11, 16 under Society and SDGs No. 8, 9, 10, 12, 17 are also aspects of Economy [2].
Over the past few decades, numerous workshops and project-based programmes were conducted including the development of innovative curriculum that were mainly based on the K to 12 Science and Mathematics curriculum in SEAMEO region which reflect United Nations SDGs as well as Malaysian Education Blueprint (2013-2025) with 11 shifts to transform the educational system [17] supported by SEAMEO's Seven Priority Areas (PAs) [18].The efforts made with evidence-based findings included 'Human Values-based Water, Sanitation and Hygiene Education' (HVWSHE) reported by [11[12][13].Continuing efforts were made for educational settings to implement projectbased programmes with proper tools/prototypes for monitoring and evaluating participants from diverse backgrounds.It was expected that attitudes of learners towards conservation and wise use of resources also motivation towards STREAM related studies in line with SDGs could be enhanced and they are better prepared with knowledge/skills Page 146 of 155 necessary to increase career employability.

Lifelong Education integrating Technology-enhanced Learning through Transdisciplinary Studies
Promoting lifelong education is an imperative move in line with UNESCO's aspiration to enhance students' positive attitude towards environmental, sustainable issues and motivation for borderless learning (BL) through formal and informal settings (e.g.outdoor studies and presentation of investigative research project).This is because easily accessible and growing number of e-tools are available in increasingly globalised world in the digital IR era.Students have to be prepared to learn cross-cultural and transdisciplinary curriculum across the nation, not only they could develop their own language, but also other international languages.Transdisciplinary is operationally defined as a term goes beyond all or encompasses all the disciplines that are linked or connected to all the disciplines by a common theme, a unifying issue or topic/focus of inquiry/study.
In a study by [19], a hybrid system was developed to enhance BL with opportunitiesfor the underserved.The analysis of qualitative data revealed that generally the respondents were positive towards blended learning integrating Oxwall Social Networking Platform, the Koha [Integrated Library System (ILS)] and Google Apps (Gmail, Google calendar, Google drive) with the need to have online library that could better facilitate their learning in the countries of different culture as also concurs with the studies by [20].Promotion of wider and ongoing blended-mode participation of blendedmode project-based programmes was also made through e-course series initiated by RECSAM as reported by [21] [14].The following are the summary of four main activities conducted during Stage 3 as reported by [22] [23].(1) Drafted guide of implementation and reward plans; (2) Prepared two draft surveys and pilot them among SEAMEO participants through Telegram and RECSAM's Facebook groups.The respondents were from 10 countries (Brunei, Cambodia, Indonesia, Lao PDR, Malaysia, Myanmar, Philippines, Thailand, Singapore, Vietnam) for the 'Attitude towards conservation of energy and other resources' (AToCONEoR) survey and 8 countries (Brunei, Cambodia, Indonesia, Malaysia, Philippines, Singapore, Thailand, Vietnam) for the 'Motivation towards STEM Education' (MoToS) survey, as extracted from data in Excel files from Google Drive.(3) Conducted regional workshops on ICC-Education 4.0 were conducted in Phase 1 (March 15 to 19, 2021) and in Phase 2 (dates not specified).The Phase 1 workshop was followed up with a virtual session on July 26, 2021 entitled 'Workshop on follow-up trouble shooting activities on controller board in robotics kits: Issues, challenges and suggestions.Ministry of Education, Culture, Sports, Science and Technology (MEXT) sponsored the expert Assoc.Prof. (AP) Dr. Tairo as the facilitator for the workshop, and the coordinator prepared a planned schedule of activities with the allocation of hours per day with a total number of maximum 22 teaching hours (5 days).( 4) Upon approval granted by MEXT, more detailed schedules were planned to fill out the remaining time slots with invitation of experts to present the topics in line with the workshop theme and research objectives.After the regional workshop, evaluation report was also sent to MEXT.Exemplary Output with Policy Recommendations Leveraging on Findings from MoToS Instrument Validation Providing Insights on the Future Direction of STREAM (RQ2).This section illustrates output from qualitative and quantitative data analysis using 'multiplecase design' [7][9] [24].
'Cross-Case Analysis' (CCA)(Table 1) and 'Exemplary-Case Analysis' (ECA) on selected investigative research projects accepted for presentation revealed that the project output showed evidences of curriculum integration in support of SEAMEO Priority Areas (PAs) No.7 (Adopting a 21 st century curriculum), No. 5 (Revitalising teacher education), No. 2 (Addressing barriers to inclusion) and No. 3 (Resiliency in the face of emergencies)(Ng & Othman, 2020) as well as SDGs.The focus of CCA and WCA is based on 'What are attributes/characteristics of various models of knowledge/skills identified that may help learners developed confidence and STREAM related capabilities with potential career employability in various types of jobs'.Table 1 shows the CCA on SDG related output selected from the piloting of blended-mode curriculum developed through various phases by project coordinator, team leader/members of LeSMaT(Borderless) as well as its offshooting virtual learning programme during 2018-2022 e-course series.
'Exemplary-Case Analysis' was conducted through informal interviews, for example with the mentor of 'creative mathematics using e-tools to enhance STEM literacy'.The researcher was informed that the project students who had mastered the knowledge/skills in using the ICT tool such as Wolfram programming digital tool were later became engineers in a few American multinational technology companies that specialize in services and products related to Internet.One of them also became university lecturer specializes in STEM related education.The project entitled 'Using Minecraft as digital tool to promote healthy lifestyle in sustainable future city' [30] was a project presented during e-sport live event in conjunction with International Minecraft Championship (IMC) 2022 illustrating a detailed process on how to create sustainable food production system.
Apart from real time building in Minecraft world within stipulated timeframe, the project team members were supposed to communicate with judges how their project reflected the IMC's theme in line with SDGs.Hence, in addition to mastering the digital tools as summarised in Table 2, student preparing project involved in designing 'algorithms' to trigger food production system using Redstone and other digital tool so that simulation on the process of producing organic food can be viewed in Minecraft world.Hence, participating in project-based programme involving competition indirectly helped students acquire various STREAM related knowledge, skills and capabilities to prepare for their prospective career in future.
The MoToS instrument with 72 survey items was preliminarily piloted among 48 respondents from 8 countries [i.e.Malaysia (32), Brunei Darussalam (1), Cambodia (1), Indonesia (4), Philippines (6), Singapore (1), Thailand (2) and Vietnam ( 1 Rasch Model statistical tool was used to analyse data collected from the above 48 respondents and the summary of statistic is illustrated in Table 3.The analysis revealed reasonably good measure of internal consistency with Cronbach Alpha (CA) reliability value of 0.98.Hence, the findings indicated that the items are reliable and positive.The figures as summarised in Table 3 also suggested that the instrument acquired a high reliability and validity which denoted the replicability of MoToS instrument to be administered among other samples.Further discussions are made as below on the analysis as shown in Table 3 with reference to [23].
(1) 'Person separation' of 4.61 (that is more than 2 logit) implies that the instrument may be sensitive enough to distinguish between high and low performers.Thus, it is possible to show the profiling of respondents.(2) 'Item separation' is used to verify the hierarchy of the item.By comparing the means of item, the results implied that the questions in MoToS were reasonably easy to be answered by the participants.Item separation that is high with the value of 2.03 (that is more than 2) shows that the item difficulty hierarchy exists and the instrument is valid.Figure 2 illustrates the MoToS Item Wright Map. Figure 3 illustrates item Wright map indicating 3 items (A26, A6 and A24) that are the most difficult to agree with; 36 with moderate and 33 with easy items to incorporate technology-enhanced learning programmes on Values-based Sustainable Education (VaBSE) [i.e. one of the sub-theme of LeSMaT as aforementioned] in the classrooms involving STREAM education.In response to RQ2, the indication that can be obtained from the findings of MoToS instrument validation is that there is a high number i.e., 39 out of 72 items that may require some form of analysis to increase the motivation level to implement STREAM education.The analysis of item Wright map revealed that among all the items, the easiest item as agreed by respondents is A11 i.e. 'Girls and boys are both encouraged to take topics related to STREAM education'.However, the most difficult item to be responded is A26 'I feel stressed whenever I study a subject related to STREAM education'.The top 10 most difficult items responded are mostly from Part A except 1 item from Part B that is the 9 th most difficult response, i.e.B35 'I like to build abstract theories or models'.C45 'Listening to educational programme related to STREAM on radio' is the most difficult item in Part C to answer and it was placed as No.13 most difficult item out of all 72 items responded.Hence, the findings can help the researchers to improve the organisation of future training events.The areas of training to increase motivation may include managing stress levels when studying STREAM subjects also building abstract theories and models.

CONCLUSION
This study reports DDR process including instrument validation of MoToS as monitoring/evaluation tool for lifelong education in skills-enhancement e-course series conducted by RECSAM that reflect SEAMEO's Priority Areas and SDGs with reports on case exemplars also discussions on policy recommendations for future studies.[1] that emphasises on sustainable use of resources and the importance to conserve these resources for sustainability are reflected in various outputs from the project-based programmes initiated by RECSAM.Among the exemplars include SDGs No.6 (Clean Water and Sanitation), No.7, No.11 (Sustainable Cities and Community), No.12 [25].All the programmes and activities that were initiated towards achieving these SDGs would not have been possible without the provision of quality education (SDG No. 4) by Education for Sustainable Development (ESD) advocates in collaboration with educational institutions through smart partnerships in achieving goals (SDG No. 17) among all the stakeholders.These projects (that reflected LeSMaT sub-themes ensure sustainable living) were participated by both genders (SDG No.5 Gender Equality) with reduced inequalities (SDG No. 10) uncovering various aspects of SDGs that were fulfilled in the projects prepared by the SEAMEO participants.International networking activities (SDG No. 17) were seen during the implementation activities, e.g.institutional visit by Japanese students in Malaysian school for sharing and exchanging experience in projects related to energy conservation as well as creative Mathematics education.Some of the research/evidence-based findings were published on Learning Science and Mathematics (LSM) online journal, the This study is also important to provide feedback on rooms for improvement.For example, the findings in response to RQ2 revealed that respondents (who were mainly inservice science/maths educators) found that STREAM education is stressful and less motivated.Hence, fun-based science/TEL programmes or topics incorporating gamification, game-based active learning integrating Minecraft digital tool were introduced with opportunity for the project team members in Malaysia to obtain international co-curriculum marks.An example of initiative to promote transdisciplinary studies supported by technology-enhanced programmes that reflected SDG No.17 was the launching of the first 'Heritage Immortalised: A Nation Building MINECRAFT Championship 2021 [URL: https://youtu.be/UWYuNnuvzFM] and second International MINECRAFT Championship (IMC) 2022 [URL: https://youtu.be/v7og_neMcGQ] in which RECSAM served as strategic partner to support a LearnT-SMArET project teacher who led the organisation of these international events (scheduled from 8 May to 15 August 2021 and 27 March to 7 August respectively).Amazing responses with submission of project output were received for IMC 2021-2022, as well as LeSMaT offshoot programme LearnT-SMArET e-course series 2018 to 2022 as reported.Despite time constraints in school curriculum that were loaded with exams, students developed knowledge/skills while preparing project output reflecting the 9 LeSMaT subthemes.

Significance and Implications
In the emergence of the digital era and Education 4.0, the sustainability of quality education was also much dependent upon employing Information and Communication Technology (ICT) tools that promote student-centred learning anchoring on constructivist philosophy supported by technology-rich learning environments.The Gen Y or Millennial Generation who should succeed the roles vacated by Gen X to lead the future of digital era will gain higher values from independent learning abilities or thinking skills and enriched exposure if well guided through transdisciplinary approaches with awareness on sustainability as well as opportunities for blended learning.

Limitations and Lessons Learnt with Policy Recommendations for Future Studies
This study was limited only to reporting the analysis of LeSMaT project-based programme and its offshooting Education 4.0 and LearnT-SMArET e-course series initiated by the Centre within the SEAMEO region that were conducted fully online due to various constraints faced such as pandemic, time schedule, geographical area and finance.Due to the far flung region of SEAMEO with diverse socio-cultural backgrounds of many nations and the issue of digital divide that hinder the accessibility of Internet access, not all SEAMEO member countries participated in the study from 2020 to 2022.
More efforts should be made continuously to promote quality education for all with careful planning and collaborations involving regional and international educational partners, taking into account 'Gender Equality' (SDG No.5) and 'Reduced Inequalities' (SDG No.10) backed by e-tools, in order to reach a wider audience and create awareness of the SDGs.This is to ensure that all activities involves SEAMEO regions and beyond as well as implementation of policy and practices involving educational partners could fulfill all SDGs.Perhaps more interviews with project teachers participated in the lifelong learning programme or longitudinal study can be conducted to explore the career paths or employability of learners who were involved in submitting project output.
The authors will continue to be advocates with sustainable efforts to promote quality education supported by e-tools and e-platforms in raising awareness on SDGs through reaching out to all in the SEAMEO region and beyond.Moreover, the following topics are suggested for future work during Continuing Professional Development (CPD) programs based on item difficulty analysis using Rasch's model in response to RQ2: (1) Conduct more fun Science/TEL programmes to promote STREAM education since 'Learning subjects related to STREAM education through pedagogical approaches that promote active and fun learning' (item A26) was selected as priority area in e-survey; (2) Implement basic education integrating constructivist approach to introduce nature of science for better understanding since 'Constructing abstract theories and models through hands-on/minds-on/hearts-on learning engagement' (item B35) was identified as another difficult item selected; (3) Promote educational programmes related to STREAM education on communication channel (e.g.radio, telecommunications and computer networking) (item C4 as selected by respondent).

Stage 2 :
'Design and Development' Phases to Develop Monitoring/Evaluation Tools During Stage 2, planning was made with design and development processes as summarized in the following Figure 1.

Figure 1 .
Figure 1.Flow diagram from planning to implementation of the ICC-Education 4.0 activities During the 'Design and Development' phases, 'Motivation towards STEM Education' (MoToS) e-survey was developed as one of the monitoring/evaluation tool validated among 48 respondents or participants from a total of 8 countries (Figure 2) for the tracking of skills-enhancement e-programmes in line with Sustainable Development Goals (SDGs).

Figure 2 .
Figure 2. Respondents from 8 countries participated in the validation of MoToS e-survey

Figure 3 .
Figure 3.The MoToS Item Wright Map

Table 1 .
CCA on Exemplary SDG/PA Related Projects from e-Course Series (2018-22) with Research/ Evidence-based Articles Published and/or Prototype Presented in Regional Workshops on Education 4.0 (Phase 1 and 2) Showcasing IR4.0

Table 2 .
Exemplary-Case Analysis (ECA) of Selected Output Reflecting 'TeleHeal' Sub-theme of LeSMaT program with Knowledge/Skills Identified to Develop STREAM related Capabilities for IR4.0

Table 3 .
Summary of Statistical Analysis on the 72 items of MoToS with Data Obtained from 48 Respondents 134]findings revealed that the e-course series conducted by SEAMEO RECSAM provided excellent platforms for students' networking with learning output reflecting governmental aspiration to promote STREAM education and SEAMEO's Education Agenda Priority Areas especially No.7 (Adopting 21 st century curriculum), No. 3 (Resiliency in the face of emergencies) and No.5 (Revitalising teacher education).The 'Cross-Case Analysis' (CCA) as analysed in response to RQ1 reflecting LeSMaT's 4 subthemes as summarized in Table1revealed many aspects of interlinkages among SDG targets with one another making indivisible parts of sustainability from numerous perspectives[24].These include CADRRED (SDG No.13Climate Action), ConWUR and LoWBERI (SDG No.12 Responsible Consumption and Production), TeleHeal (SDG No.3 Good Health and Well-being), SE4ALL (SDG No.7 Affordable and Clean Energy), LearnT-SMArET and VaBSE (SDG No. 4 Quality Education), CIfE and TecVoTEE (SDG No. 9 Industry, Innovation and Infrastructure).The vision of SDGs 9 th International 'Conference on Science and Mathematics Education' (CoSMEd) 2021 or conferences organised by SRD, e.g. the 6 th 'International Conference on Management, Engineering, Science, Social Science and Humanities'