National Commission for Academic Accreditation and Assessment " Learning Outcomes" in program planning, delivery and evaluation based on National Qualifications Framework Facilitators: Prof Gina Wisker & Gabriel Jezierski Conveyor: National Commission for Academic Accreditation & Assessment & the British Council October 2012 : : Who are we? Professor Gina Wisker - Head of Centre for Learning and Teaching, Professor higher
education & contemporary literature, University of Brighton: National Teaching Fellow, Fellow Royal Society of Arts, Senior Fellow Staff &Educational Development Association, Fellow English Association; Universitys Academic Standards committee; Editor Innovations In Education and Teaching International journal . - . Gabriel Jezierski Worked in Scotland, Wales, Northern Ireland and England. National level Quality Improvement: Former Head of Policy and Partnerships (Wales and Northern Ireland) Higher Education Academy working with the QAA ; Head of Learning and Teaching; Fellow of the Higher Education Academy; Member of the University Academic Quality and Standards Board and Senior Management Group. Director Open Learning Foundation. Professional Coach. - J L : .. . Relationship? How do you build? How do YOU STRUCTURE and package these? i.e. specify the program - field of study? J :
Outcome ? SUBJECT MATTER EXPERT Start Year 2 Year 1 Program Course Any program Medical Sciences Engineering A subject
SUBJECT DOMAIN How do you describe what happens here? Year 3 WHAT attributes do you Finish expect a student to have as a result of studying and learning? How do you assess? How do you map? Postgraduate? Undergraduate?
Relationship? How do you build? Any program Level 5 A subject Level 4 Level 3 Start Level 2 Level 1 Finish How do they make informed decisions about .. ... ?(i) what to study at university ?(ii) the level they are aiming to achieve . BSc Bioinformatics
J MSc Opthalmology BA Mass Communication ?Their understanding .. .. How do you Manage the Quality of an Outcomes-based approach to Learning? The National Qualifications Framework How can you ensure consistency within the Kingdom in the standards of student learning outcomes regardless of institution attended? Benchmark?
LEARNING OUTCOMES: The Big Picture . Teaching and learning strategies Quality Learning ? LEARNING OUTCOMES AIMS PROGRAM (The curriculum) THE SCHOOL OF
Course Aims and Objectives . PROGRAM SPECIFICATION Mission PROGRAM DESIGN ASSESSMENT Practices Constructive alignment So .. .. What are we trying to achieve from this workshop? (1) )1( Participants will: : obtain a better understanding of the concept of learning outcomes learn how the use of learning outcomes relates to broader questions about student learning
have the opportunity to develop learning outcomes within their own institutional context )What are we trying to achieve from this workshop? (2 ()2 Learn how course LOs can be mapped to show the LOs for an entire programme Consider how students achievement of LOs can be assessed Examine the quality assurance questions related to the use of LOs with particular focus on NCAAA Standard 4 and related documents 4 L Reflect on the 5 themes and lessons of UK best practice from QAA Institutional Audit THE LEARNING OUTCOMES OF THE WORKSHOP
On successful completion of this workshop you will be able to: : Discuss and explain new theories, issues and best practices related to developing and approving ILOs at program and course levels for the different disciplines. f Refer to and use subject-benchmarks and professional standards and resources to develop, implement, and monitor program ILOs and curriculum considering NCAAA and international requirement. L Provide support and guidance for faculty, support groups and related committees in planning and developing ILOs for programs and courses in specified fields of study, mapping them across levels, skills, and courses. Design, implement and monitor T, L & A strategies for the different ILOs at program and course levels that are appropriate for specified disciplines using NCAAA specification and reporting templates. L
LEARNING OUTCOMES: Summary Overview DAY 22 Evidencing the achievement of Learning outcomes: : Assessment Use of the Domains in Program Planning and Student Assessment . Curriculum and Assessment Mapping . Quality assurance of the outcomes-based approach DAY 1 1 ?WHAT are they and WHY are they important
) (i.e. Outcomes-based education in Teacher Education Subject domains, knowledge construction and student learning J Verifying Consistency with the Qualifications Framework Action Plans - Moving to an Outcomes Based Model : 1. The rationale J
Summary/Conclusions 2. Key concepts . 3. The process and use of learning outcomes in Program and Course level J Planning )( NCAAA Standard 4 ][Program Specifications Principal Elements in the Qualifications Framework Activity 1: Your expectations
: ... Write down your objectives (outcomes) for the workshop ( ) By the end of this workshop I want to . J ........ Write down what you consider learning outcomes to be. . Learning outcomes are .......... How to link program and course outcomes and University wants How lecturer deals with students and resources/facilities and institutional regulations/structure; the institution must support and reflect NCAAA requirements How to manage new /old programs: transitions; Benchmarking when? How can LOs help students get into a different field/area of work
Gina SESSION 1 9.0 10.30 9:0 10:30 )What is Outcome-Based Learning (OBL and how is it being developed and used in ?KSA . What do Learning Outcomes look like? Example from statistics/mathematics course On successful completion of this course students will:
: be capable of selecting, carrying out and interpreting the results of appropriate statistical methods for describing and analysing data sets, in the context of their own research interests; . be able to make use of appropriate software packages to carry out statistical analysis, and to interpret their output in terms relevant to the research field; . demonstrate an understanding of the concepts of sampling variability, bias, sampling error and statistical significance, and their impact on the interpretation of research findings; J J . have an appreciation of a range of multivariate methods and their use and limitations in a research context. . ?What do Learning Outcomes look like Generic outcomes
be able to plan and improve their own performance J be able to research new areas of knowledge using both library and online resources show evidence of having worked in groups and developed group working skills be able to plan and deliver an effective presentation J What do Learning Outcomes look like? program-level outcomes apply.quantitative science and engineering tools to analyze problems (Engineering) ) ( L identify and assess central arguments, themes, perspectives, and theoretical frameworks of secondary sources (History, Humanities) .
( -) apply the principles, skills and knowledge of evidence-based medicine (Medicine) J L )( What do Learning Outcomes look like? Course-level outcomes in Humanities By the end of the course, students should have gained: : a) a broad experience of reading 18th-century and Romantic writing; J . b) a sense of major literary developments within the period in relation to their social and aesthetic contexts; J .
c) skills of analysis which are sensitive to the theoretical and critical issues emerging from the literature of the period. . (Module Restoration to Romantics, English Literature major) What do Learning Outcomes look like? Course-level outcomes in Medicine Upon completion of the course the student will be able to: : explain the meaning of key concepts in parasite epidemiology describe and apply techniques used in epidemiological surveys L critically analyze and interpret ecological and epidemiological data J (Module Parasite Epidemiology, School of Tropical Medicine, University of Liverpool) ,
So, why Learning outcomes are important: The Rationale Meaningful learning is most likely to be successful if it is goal oriented i.e., if the learner is aware of the goal (at least in a general sense) toward which he or she is working and possesses expectations appropriate for obtaining the desired outcome Shuell ,. ( . ) . .
Defn. Learning outcomes, naturally enough, are the performances made possible by learning that is, they are what the learner is able to do when learning is completed Wager and Gagne J : L J 5 Types ... ... Intellectual skills Verbal information
Cognitive strategies Motor skills Attitudes 1. 2. 3. 4. 5. )Gagne (1984, 1985) and Gagne and Briggs (1979 (()1979( 1985-1984 What is Outcomes-based Education (OBE)? An approach to student learning which specifies what the
student should be able to do in order to demonstrate that the intended learning has taken place Contrasts with the traditional model which focuses on the material which will be provided by the teacher. OBE vs traditional model OBE student centred output oriented (hence learning and )teaching ( ) traditional teacher centred input oriented (hence teaching )and learning - - ( ) Drivers for the growth of OBE ;increased pressure for transparency from students
employers requiring greater clarity as to what graduates can be ;expected to know/do requirement from governments and media for greater public accountability by universities. Advantages of the OBE approach help to guide students in their learning in that they explain what is expected of them, in turn helping them to succeed in their studies. help staff to focus on exactly what they want students to achieve in terms of both knowledge and skills. J provide a useful guide to inform potential candidates and employers about the general knowledge and understanding that a graduate will possess.
Advantages of the OBE approach improved communication between teacher and student shared understanding of course objectives; . more active student engagement; help to guide students in their learning in that they explain what is expected of them, in turn helping them to succeed in their studies. . better communication between teachers standards and expectations made explicit, interrelationship of courses better understood; improved student performance due to better understanding of what is expected/what will be assessed. , J Advantages of the OBE approach
The student learning experience There is ample evidence in reports that students valued the clarity that the use of learning outcomes had brought to the overall purpose of their programme, the interrelationship between parts of the programme and the nature and purpose of assessment tasks. QAA . J Perceived disadvantages for staff ; can be seen as onerous, involving bureaucracy, form-filling etc .. ; alters the balance of power between student and teacher can appear to produce excessive standardisation and stifle
;innovation may appear to reduce the scope for professional academic judgement. Criticisms of OBE reduces learning to the achievement of a long list of small outcomes fails to recognise that the whole may be greater than the sum of the parts J assumes that all desired learning can be assessed and measured Outcome Based Learning It is a design down approach to curriculum development. It starts with what students are expected to learn, then focuses on the creation of teaching and learning environments that will
assist students to reach the desired learning and finally involves the use of assessment that can provide feedback about the level of learning that has been achieved. * L . Outcome-based Education OBE is neither a new concept nor a passing phase in educational technology and is equally applicable throughout the educational continuum from primary school to postgraduate training. . OBE focuses on the end-product and defines what the learner is accountable for. It is not about telling teachers how to teach or students how to learn. J! LOs determine what is taught and assessed and can help to identify what is and is not essential. Having a clear idea of the desired outcomes does not necessarily have to be restricting as the methods of achieving the outcomes are still flexible.
. L OBE in KSA NCAAA Standard 4 - NQF: Learning and Teaching requires the adoption of an outcomes-based approach, with a range ;of associated requirements 4 : . Standard documents are provided for course planning, students evaluations, etc. L .. Exercise 1: Group discussion :1 Consider the NCAAA documents provided, particularly the Course Report, the National Qualifications Framework, and the Course Planning Matrix. L . ?1. To what extent are these already being used in your institution
.1 ?2. How would you assess awareness of this material in your institution .2 3. What changes need to be made in order to fully implement the NCAAA ?model in your institution .3 L 10.30 10:30 Orange in Princess Noora Faculty Pharmacy Have documents work in progress For 2 RED Seen as burden. Need strengthen awareness and importance. Evaluations are picking up things/issues. Need change perception in Faculty. Need put theory of quality into step by step practice SESSION 2 2 10.45 12.00
10.45 12.00 Some concepts related to Outcomes-based Learning How do you come to understand a subject ?your subject i.e. how does a student learn it and how do ?you teach it : We need look at learning theory OBE in context In planning a program / course we need to plan the sequence and structure of the knowledge but we also need to consider it in relation to the learner and
cognitive learning processes J For OBE to be fully effective, a change in the philosophy of learning and teaching is required; In particular, there are significant implications for assessment; It is important that the use of learning outcomes is not adopted in a mechanistic fashion. Organizing and structuring knowledge J Relevant concepts Models for the implementation of OBE L Constructive alignment Deep and surface learning L
Active learning and learning styles Models for OBE )(Based on Hager (1993 ()1993 1. Behaviourist or specific task approach 2 . Attribute or generic skills approach 3. Integrated or task attribute approach 1 2 3
Approach to outcomes Performance is defined as outcome . J General attributes predict future performance J Achievement of outcomes inferred from performance J Nature of
approach Can be reductive, mechanistic, routine Can be abstract, remote from practice L Holistic, captures richness of practice L
Effect on curriculum Leads to a doing curriculum ) ( Fragments curriculum into subjects Powerful device for improving all aspects of curriculum J
Number of outcomes Large number of very specific outcomes Small number of generic outcomes Manageable number of key outcomes Approach to
assessment Uniformity, one right way L Diversity, may be many right ways Diversity, may be many right ways Control of curriculum
Central Provider Joint (including input from students) ( ) Assessment Checklist approach Requirements may be vague, not
clearly specified Needs careful planning, diversity of methods L Constructive alignment Developed by Biggs (see Biggs 2007); (2007 ( Constructive refers to the way the learner constructs meaning by engaging in relevant activities; L Alignment refers to the way the teacher ensures that teaching
methods and assessment tasks are aligned with intended LOs. . Basic idea Set objectives Objectives and Learning Outcomes . Design learning activities Evaluate results )More detailed model (after Biggs, 2003 ( Assessment tasks )Teaching/learning 2003 activities Curriculum objectives
How well are the desired verbs ?displayed Highest level: hypothesise, apply to completely unfamiliar situations, etc : .. Designed to generate/develop desired verbs
Very good (explain, compare, )solve, analyse ( ) May be teacher-controlled, peer-controlled, student-controlled Quite satisfactory (classify, )cover topics x and y, etc ( ..) As appropriate to context Just satisfactory (list, ( )describe
) An Instructional Design perspective an intellectually disciplined step must be taken by the (instructional) designer to infer the nature of the performance that is expected to be attained as a result of learning i.e. determining what is to be learned is the first step in instructional design Wager and Gagne ... STEP 11 in designing Instruction
J ) ( : Identification of expected type of learning outcome Internal Learning Process J Alerting the learner to receive information Acquiring an expectancy of the results of learning Retrieval of items in long term memory to the working memory Selective perception of the patterns that enter into learning
1 2 3 4 . STEP 22 in designing instruction 9 events of instruction9 ) (Gagne & Briggs How to arrange the events that will make learning readily occur
1 Internal Learning Process J Alerting the learner to receive information Acquiring an expectancy of the results of learning Retrieval of items in long term memory to the working memory 3 Selective perception of the patterns that enter into learning 4 Semantic 5 encoding of presented material, to attain a form for long-term storage and ready retrieval 6
Responding with a performance that verifies learning J 2 STEP 2 in designing instruction 9 events of instruction (Gagne & Briggs 7 8 Reinforcement, by means of which the results of learning are established L Providing cues that are used in recall 9 Generalizing performance to new situations J How to arrange the events that will make learning readily occur
1 External Instructional Events Gaining attention 2 Informing learner of lesson objectives. 3 Stimulating recall of prior learning 4 Presenting stimuli with distinctive features 5 Providing learner guidance 6
Eliciting performance J 7 Providing feedback STEP 2 in designing instruction 9 events of instruction (Gagne & Briggs 8 Assessing performance J 9 Enhancing retention and learning transfer How to arrange the events that will make learning readily occur The hidden curriculum
Rowntree, 1987; 1987 ;Students may adopt instrumental approaches If overt curriculum and nature of assessment are not aligned, students will construct their own version of the curriculum to meet assessment needs. L What is Curriculum Alignment? Consistency and Intentionality /
Harden, R.M. (2001). AMEE Guide No. 21. Curriculum mapping: a tool for transparent and authentic teaching and learning. Medical Teacher, 23 (2), 123-137. Hobson, E.H. (2005). Changing pedagogy. Presentation at SACS-COC Institute on Quality Enhancement and Accreditation, Orlando, FL, July 24-27, 2005. 51 Deep vs surface learning Ramsden, 1988 and others; 1988 Some authorities also include strategic learning;
Not attributes of students the same student may adopt different approaches at different times. Surface Deep Focus on the surface signs Learning consists of unrelated parts J No reflective relationships Focus on what does this
?mean : Relates new knowledge to previous learning Relates theoretical ideas to everyday experience Deep Surface Can distinguish evidence from argument Structures content into a coherent whole
Driven internally by students wish to learn Basic concepts not differentiated from examples Matches content to imposed tasks Driven externally, by assessment Implications for Los LOs should be set in order to encourage deep learning; . L
Assessment methods should require deep approaches to be deployed. Concrete Concrete Experience Experience Active Active experimentation experimentation Kolb Learning Cycle
Reflection Reflection J J Conceptualization Conceptualization Kolb 1976 Kolbs learning styles Observers focus on facts, like practical examples, not so good with theory; L Thinkers like theory, concepts, influenced by expert opinion, may lack
common sense; Deciders like clear structures and rules, not so good with messy situations; Doers learn by trying things for themselves, by making mistakes, inclined to be uncritical of received wisdom J J Implications for Los Different LOs will suit/require different learning styles; . LOs should be selected to suit a range of learning styles; . Teaching should encourage students to explore their own learning styles and address their weaknesses; . . Assessment methods should cover a range to suit all types of learner.
Summing up This has been a very rapid overview of the inputs of some key thinkers into learning theory; These views are contested by others this is a developing field; J Do not neglect the importance of the subject element. Exercise 2: 2 : In your subject or specialization group, write four student learning outcomes; list the SLOs in order of progression. . 12.0
Example LOs (PN) To be able to handle medication and dosage Be able to gather, summarise, research information How to handle case related problems with patients Improve communication between patients and other health care workers Re design Student be able to imagine and create Create and implement in real life Be able to convert 2D into 3D Be able to communicate visually
SESSION 3 3 12.30 14.00 12:30 14:00 Setting Outcomes at Program and Course level . ?How to write LOs Questions of level and standard (with reference to the NCAAA Qualifications Framework). ) ( The language of Outcomes. What are learning outcomes?
Learning outcomes are the specific intentions of a program or course, written in specific terms. They describe what a student should know, understand, or be able to do at the end of that program or course. . A statement of learning outcome is a tool used to describe the learning to be achieved by a student in higher education. Learning outcomes are linked by definition to the credit that is attributed to courses. Credits, Levels and Domains of learning.. Relationship? How do you build? Outcome ?
Start Year 1 Year 2 How do you describe Year 3 what happens here? End How do you assess? How do you map? Course Any program .... A subject Standard 4: Learning and Teaching
:4 4.2 Student learning outcomes must be clearly specified, consistent with the National Qualifications Framework and requirements for employment or professional practice. 4.2 Standards of learning must be assessed and verified through appropriate processes and benchmarked against demanding and relevant external reference points. (e.g. NCAAA) J ( : ). Teaching staff must be appropriately qualified and experienced for their particular teaching
responsibilities, use teaching strategies suitable for different kinds of learning outcomes and participate in activities to improve their teaching effectiveness. . Teaching quality and the effectiveness of programs must be evaluated through student assessments and graduate and employer surveys with evidence from these sources used as a basis for plans for improvement. How do we use learning outcomes? A learning outcome statement is defined as: : an expression of what a student will demonstrate on the successful completion of a course. LOs are related to the level of learning; indicate the intended gain in knowledge and skills that a typical student will achieve and should be capable of being assessed. . . LOs should be written in a manner whereby the learning can be assessed through use of an assessment method. This link with assessment is crucial for quality assurance.
. . It is also important to note that learning outcomes are linked to level descriptors. The words used imply that the learning is at a particular standard. Linking LOs to level descriptors ensures that we build progression into programs. . . The Bologna Process designed to create a barrier free European Higher Education Area. The use of OBL in the context of the EU is regarded as a move towards a more integrated higher education system. L J The change in emphasis from the traditional teacher-centered to a student-centered learning produces a focus on the teaching learning assessment relationship and the fundamental links between the design, delivery and measurement of learning. Adam (2004) J
)2004( Harmonising Learning Outcomes across Europe: Medical Sciences Aims, objectives and outcomes (whats the difference?) ) ( What are Learning Outcomes? LOs are statements that describe . :
(i) what a learner will be able to do as a result of learning. (ii) the nature of the performance that is expected to be attained as a result of learning J LOs are also statements that describe what a learner will be able to do as a result of teaching. . Some definitions stress that LOs is a sort of contract that teachers make with learners that describes what they will be able to do after learning that they could not do before, the 'added value' of teaching. . ------------------------------------------------------------------------------------However the connection between teaching and learning is not a simple one. Just because knowledge or skills are taught does not mean that particular knowledge or skills are learned. Many factors can interfere with the achievement of outcomes: the existing knowledge of the learner, the relevance or usefulness of the material presented, the skills of the teacher. . : ! . ?Difference between Aims and Outcomes
Aims are general statements concerning the overall goals, ends or intentions of teaching. . Outcomes are the individual stages that learners must achieve on the way in order to reach these goals. . For example, to be able to take blood pressure using a sphygmomanometer, the student has to achieve a series of outcomes, e.g., to explain procedure to patient, to position cuff correctly, to inflate to correct pressure etc, etc. : J J .. Aims, objectives and outcomes
The difference aims are written in terms of teaching intention and indicate what it is that the teacher intends to cover in the block of learning (curriculum coverage). . ) ( . Learning outcomes are descriptions of what the learner is expected to learn in the period of learning defined. . : Aims are therefore more about teaching and the management of learning, and learning outcomes are more about the learning that is actually to be achieved by the learner. . )Aims, objectives and outcomes (Example () . () ) (
( ) . : . : : * . . . L.
We have course objectives, why do we need outcomes for individual teaching sessions? ... From a curriculum perspective the learning objectives from each taught session should fit together coherently, building towards the overall aims of each module and the whole curriculum. If learning objectives are not known for each session then it is impossible to see how the whole curriculum fits together. It becomes impossible for teachers in different phases of the curriculum to see what students have learned in other areas making managing and auditing the curriculum more difficult. . . . . . . Constructing Learning Outcomes (LO) An LO should ideally contain three parts that deal respectively with: J :
1. BEHAVIOUR: an action verb to describe what participants will be able to do as a consequence of a learning activity . : .1 2. CONDITION: an environment or situation in which the student will perform the behaviour or the tools/information they will be given when they demonstrate their learning / : .2 . 3. CRITERIA: describing the limits or range of an acceptable response, i.e. addressing the question of how well the learner has to perform for one to be able to say that the LO has been achieved? : .3 . Learning Outcome: Example 1 1 : Given a set of data the student will be able to compute the standard deviation. J . .
Condition - Given a set of data Behaviour - the student will be able to compute the standard deviation . Criterion (implied) - the number computed will be correct. ) ( Learning Outcome: Example 2 2 : At the end of this study-unit, students will be expected to be able to
analyse and interpret a text in detail, using different approaches from literary, rhetorical and/or linguistic theories. . Condition- a text is provided for analysis and interpretation Behaviour- analyse and interpret Criterion- detailed analysis/interpretation using different approaches / Can I modify LOs to make them more demanding?
LOs can be made more difficult or demanding depending on the degree of understanding or levels of experience of learners. You can change the active verb to a more complex one or you can add specific conditions or limits. . For example, simple LOs might be to list or state facts, formulae or definitions. More complex LOs might ask learners to apply or use knowledge in a particular context that might not have been met before. This is more cognitively demanding and really tests whether learners have deeply understood concepts. Finally the highest levels of LOs ask learners to solve complex problems by, for example evaluating or analyzing evidence or synthesizing information. To critically evaluate the causes of something involves not only a deep understanding of detailed factual information but also an ability to make complex judgments about the validity of evidence. L :
L . L L J . VERBS .... L )Examples of Intended SLOs (1 ()1 The Preliminary (Yr. 1) will enable you to: (1 ) : Select topics of interest from a pre-defined list
Devise a search strategy using library and the internet Access and search relevant databases and search engines used in clinical and scientific research Appraise data (understand strengths and limitations) from the literature ) ( ) Understand different forms of scientific literature (e.g. primary studies, review articles . ) : ( Produce a scientific written report of specified length that collates information gathered from the different sources Summarise the report clearly and concisely in lay terms Utilise appropriate software (including word processing, figures/tables and referencing) to produce a scientific written report ) ( Understand what constitutes plagiarism and how to avoid it Medicine Manchester Medical School, 2011 2011
Examples of Intended SLOs (2) Engineering ) 2( INTELLECTUAL QUALITIES I1 Apply proficiently engineering principles, quantitative methods and computer software to the analysis and solution of engineering problems taking appropriate account ant limitations of the approach. . J L . I2 Demonstrate an ability to identify, classify and evaluate the performance of systems and components using analytical methods and modelling techniques and apply a systems approach to the solution of mechanical engineering problems. J I3 Investigate and define an engineering problem and identify associated constraints - environmental and sustainability limitations, health and safety and risk assessment issues. I4 Be creative in the practical solution of problems, development of innovative designs and ability to work with
technical uncertainty in unfamiliar situations. I5 Manage competently the design process, appreciate customer and user needs, evaluate outcomes, assess commercial risk and manage cost drivers. Examples of LOs (3) )3( Medicine - Year 5 5
At the end of the year, you should: : Feel confident about your prospective role as a FY1 Doctor (Self & Tutor Assessment) ( ) Be able to recognise personal stress and know how to obtain support (Self & Tutor Assessment) ( ) . Be able to function as a team member (Self & Tutor Assessment) ( ) L Recognise your limitations in skills and knowledge and the importance of involving other people in these circumstances (Self & Tutor Assessment) ( ) . Understand how your attitudes to medical practice affect your approach and be able to explore these attitudes (Self & Tutor Assessment) . ) Be able to seek out information from a wide range of sources about clinical problems (Self & Tutor Assessment) ( ) Be able to analyse critically the information that is retrieved, and apply it to a clinical problem, using reasonable judgement when dealing with uncertainty. (PMP/ Self & Tutor Assessment)
( ) (Manchester University, Medical School, 2011) ( 2011 ) )Examples of LOs (4 ()4 Business Supervision and Management Identify the challenges and opportunities of being a manager in todays high tech global economy. . Demonstrate proficiency in using latest Project Management technology tools and software.
Discuss good business ethics, social responsibility, and the vital role in the establishment of trust and honesty expected of supervisory/managers and leaders today. f J The National Qualifications Framework Part of the system for accreditation and quality assurance in the Kingdom of Saudi Arabia J is designed to ensure that the quality of post secondary education and training is equivalent to high international standards, and is widely recognized as such in the international academic and professional communities. It is intended : to ensure consistency within the Kingdom in the standards of student learning outcomes regardless of institution attended, and to make clear the equivalence of those standards with those for equivalent qualifications granted by post secondary education institutions in other parts of the world.
The Framework will. help to provide appropriate points of comparison in academic and training standards for institutions in their planning and self review processes, J for external reviewers involved in program accreditation processes J and institutional reviews, and for employers, in understanding the skills and capabilities of graduates they may employ. Student Learning Outcomes and National Qualifications Framework
LOs must be consistent with the National Qualifications Framework, and with generally accepted standards for the field of study concerned including requirements for any professions for which students are being prepared. To satisfy these requirements: Should be consistent with the NQF. Should develop learning outcomes that meet requirements for professional practice in the KSA. L Any special student attributes specified by the institution for its graduates, should be incorporated as intended learning outcomes in all relevant courses. Use appropriate program evaluation mechanisms
Principal Elements in the Qualifications Framework The principal elements in the framework are: : Levels: Levels numbered and linked to qualification titles to describe the increasing intellectual demand and complexity of learning expected as students progress to higher academic awards. : Credits: Points allocated to describe the amount of work or volume of learning expected for an academic award or units or other components of a program. : J . Domains of Learning: The broad categories of types of learning outcomes that a program is intended to develop. :
Levels The qualifications framework begins at an entry level, which is the successful completion of secondary education, and culminates with the degree of doctor. : . The levels in the framework are: Entry. Completion of secondary education. Level 1. Associate Diploma .1 Level 2. Diploma
.2 Level 3. Bachelor .3 Level 4. Higher Diploma .4 5 Level 5. Master 6Level 6. Doctor Domains of Learning Outcomes The framework groups the kinds of learning expected of students into four domains and describes learning outcomes at each level in each of these groupings. The domains are: : knowledge, the ability to recall, understand, and present information, including: : knowledge of specific facts, L knowledge of concepts, principles and theories, and
J knowledge of procedures. J Cognitive skills, the ability to: : apply conceptual understanding of concepts, principles, theories J L apply procedures involved in critical thinking and creative problem solving, both when asked to do so, and when faced with unanticipated new situations J L J investigate issues and problems in a field of study using a range of sources and draw valid conclusions. . Interpersonal skills and responsibility, including the ability to:
: take responsibility for their own learning and continuing personal and professional development work effectively in groups and exercise leadership when appropriate act responsibly in personal and professional relationships act ethically and consistently with high moral standards in personal and public forums. Communication, information technology and numerical skills, including the ability to: :
communicate effectively in oral and written form use information and communications technology L use basic mathematical and statistical techniques Psychomotor skills involving manual dexterity are a fifth domain that applies only in some programs. They are extremely important in some fields of study. For example, very high levels of psychomotor skills are required for a surgeon, an artist, or a musician. . Writing Learning Outcomes Programme Specifications, benchmark statements and the National Qualifications
Framework define the student in terms of what they can do at the end of a programme or a particular level of study. . . This is a change from the more traditional approach where academics tend to define courses in terms of what is taught, rather than what the student can do at the end of the module or programme. . There is also much discussion of whether all articulated learning outcomes should be assessed, or whether it is acceptable that some outcomes must be achieved whilst others should be achieved. . There is no absolutely correct way of writing learning outcomes (Gosling and Moon, 2001) )2001 L ( L Good Learning Outcomes ?What are learning outcomes Learning outcome describes what a student should be able to do to complete a program
of study. Program of study might mean a whole degree; a year of study; or a course. . So, typically, from a program handbook: By the end of the course you should be able to.. .... 1. For a BSc in Computer Security: By the end of the degree you should be able to analyse risks to computers and computer systems, and recommend, develop, implement and. : 1. ..L 2. 2. From a BA Art, Event, Performance critical contexts course: You should be able to structure and communicate ideas effectively through both written and verbal, critical and creative outcomes. 2. J J : J .
A course learning outcome would be more specific: : 3. ...you should be able to advise parties to a case involving the sale of land of their rights and obligations, and justify your advice by reference to relevant regulations and cases. 3. .... .
... 4. ...you should be able to plan, undertake, write up and then present and defend to a client a small-scale survey to establish likely market demand for a new consumer product or service. 4. .... . J
The Learning Outcome Process Elements of the Program Specification Process informed by: : Whats the purpose of the programme? Aims of the program NCAAA NQF Level Descriptors
Learning Outcomes of the program including: Knowledge & understanding Cognitive Skills Interpersonal Skills and Responsibility Communication, IT & Numerical Skills Psychomotor Skills Subject Benchmarks Professional Body Requirements Outcomes for level attained through:
Program learning outcomes broken down Questions to ask yourself: Attainment verified by: What should students know and be able to do on completion?
Grades awarded according to: : Learning outcomes should: : be written in the future tense identify important learning requirements be achievable and assessable use clear language easily understandable to students When writing outcomes, it may be useful to use the following expression: At the end of this course you should be able to. : : ..... Then follow with a verb. Useful ones include: : Analyse ; appraise; applyL ; calculate; choose ; compare; contrast ; create;
criticise ; demonstrate; derive ; describe; design ; develop; differentiate ; discuss; explain ; evaluate; extrapolate; formulate ; identify; list; measure ; name; plan ; plot; postulate; predict ; present; propose; recall; recognise; use; utilise A well-written learning outcome is likely to contain the following components: : A verb that indicates what the learner is expected to be able to do at the end of the period of learning. Word(s) that indicate on what or with what the learner is acting. If the
outcome is about skills then the word(s) may describe the way the skill is performed. J . Word(s) that indicate the nature (in context or in terms of standard) of the performance required as evidence that the learning was achieved. J ( ) Good learning outcomes? A good learning outcome is, among other thingsJ : Active it describes what students can do : Attractive students want to achieve it : Comprehensible students know what it means : Appropriate to the students current goals and career plans . : Attainable most students will mostly meet it, with due effort
: Assessable we can see if it has been achieved L : Visible in the course booklet and on the VLE VLE : The Dearing Report recommended . that clear descriptions of programmers Should be developed so that students are able to compare different offerings And make sensible choices about the programmes they wish to take L 1987 . Dearing Pointers on knowledge and understanding outcomes Avoid learning outcomes which are too broad in scope, such as Recall the fundamental concepts of Structural, Mechanical and Electrical Engineering. , Avoid learning outcomes which are too narrow in scope, such as State the six categories in Blooms Taxonomy. L
Avoid overloading your modules with too much content: knowledge and understanding outcomes emphasize what your students will be able to comprehend and explain, but this isnt as important as being able to use the information through application, analysis, synthesis and evaluation. ,L , J . Cage or scaffolding or... ... Good learning outcomes are scaffolding on which you and your students build their studies and their learning. Outcomes do constrain. But they are visible, and so subject to review and improvement. Without visible outcomes, students and staff guess at or build their own constraints. And these are more confining because they are
invisible, implicit, secret. Its hard to improve the invisible. J Learning outcomes (Medicine) Exercise 3: Discipline-specific exercise Overnight task on writing learning outcomes; mapping the curriculum :3 , Science Draft three common programme-level outcomes which in your view would apply to all Bachelor-level Science programmes L
Humanities Can you agree three key skills which should be acquired by all Bachelor?level students of humanities, irrespective of their specialism L END DAY 1 DAY 2 SESSION 1 1 9.0 10.30 Evidencing the achievement of Learning Outcomes: Assessment :
Evidencing the achievement of Learning outcomes: :L Assessment Review day 1 Use of the Domains in Program Planning and Student Assessment Curriculum and Assessment Mapping Quality assurance of the outcomes-based approach L Verifying Consistency with the Qualifications Framework L
Action Plans - Moving to an Outcomes Based Model : Summary/Conclusions Feedback on overnight work - outcome mapping Overview Day 2 Evidencing the achievement of Learning outcomes: Assessment :
Assessment modes and their relationship to the different ;categories of LO, and to students learning styles ; Writing outcome-related assessment criteria ; using criteria to motivate students linking criteria to feedback on assessment; assessment mapping to ensure coverage of LOs. Student Assessment (1)) 1( Student assessment processes must be appropriate for the intended learning outcomes and effectively and fairly administered with independent verification of standards achieved. L To satisfy this requirement:
Student assessment mechanisms should be appropriate for the different forms of learning sought. Assessment practices should be clearly communicated to students at the beginning of courses. Appropriate, valid and reliable mechanisms should be used for verifying standards of student achievement in relation to relevant internal and external benchmarks. L Student Assessment (2) ) 2( The standard of work required for different grades should be consistent over time, comparable in courses offered within a program and college and the institution as a whole, and in comparison with other institutions. Grading of students tests, assignments should be assisted by the use of matrices or other means to ensure that the planned range of domains of student learning outcomes are addressed. L Arrangements should be made within the institution for training of teaching staff in the
theory and practice of student assessment. J J Policies and procedures should include action to be taken to deal with situations where standards of student achievement are inadequate or inconsistently assessed. J Student Assessment (3) )3( Effective procedures should be used to ensure that work submitted by students is actually done by the students concerned. J Feedback to students on their performance and results of assessments during each semester should be given promptly and accompanied by mechanisms for assistance if needed. Assessments of student work should be conducted fairly and objectively. Criteria and processes for academic appeals should be made known to students and administered equitably. J J .
Choosing assessment measures Use of the Domains in Program Planning and Student )Assessment (1 ()1 Learning outcomes included in the first two domains, knowledge and cognitive skills, are directly related to the occupation, field of study or profession for which students are being prepared. The framework describes the level of knowledge and skill expected in general terms that can be applied to any field, but in planning a program it is necessary to identify the specific knowledge and thinking skills that are expected in that field of study. Use of the Domains in Program Planning and Student Assessment (2) )2(
The third and fourth domains, interpersonal skills and responsibility, and communication, information technology and numerical skills, are general capabilities that all students should develop regardless of their field of study. Development of these abilities can be provided for in specially designed courses or integrated into a number of courses throughout a program. Linking Learning Outcomes to Assessment )(SLOs & curriculum mapping; assessment mapping ( ) Allocation of Responsibilities for Learning Outcomes to Courses Major Responsibility x Minor Responsibility
(Note: Add additional sheets if necessary to provide for all required courses in the program including any courses offered by other departments) Learning Outcomes Course Code and Number Knowledge FactsL Concepts, theories Procedures J Cognitive Skills Apply skills when asked Creative thinking and problem solving Interpersonal Skills and
Responsibility Responsibility for own learning Group participation and leadership Act responsibly-personal and professional situations Ethical standards of behavior Communication IT and Numerical Skills Oral and written communication Courses
117 Allocation of Responsibilities for Learning Outcomes to Courses Major Responsibility x Minor Responsibility Chem 223 Chem 323 Chem 201 & 202 Chem 303 Chem 212 & Chem 311 Chem 331 & Chem 332 Chem 479 Chem 471 & chem. 472
Chem 399 Math 101 &102 Phys 101 & 102 ICS 101 Engl 101&102 Engl 214 IAS 212 IAS 322 IAS 101 IAS 201 IAS 301 PE 101
Inst. Analysis Org. chem Qual org.chem Phys. Chem Inorg. Chem Chem seminar Chem Project Summer training Calculus Physics Computer prog.
English English Professional ethcs Human rights in Islam Practical Grammer Professional Writing Oral comm. skills Physical education
X X X X X X
X X X X X X X X X X X X X X X X X X X X X
X X X X X X Cognitive Skills Apply skills when asked Creative thinking and problem solving
X X X X X X X X X X X X X X X X X X X X X
X X X X X X X X X X X X X X X X X
X X X X X X X X
X X X X X X X X X X
X X X X X X X X X X X X X
X X X X X X X X X X X X
Course Code and Number Knowledge Facts Concepts, theories Procedures Interpersonal Skills and Responsibility Responsibility for own learning Group participation and leadership Act responsibly-personal and professional situations Ethical standards of behavior Communication IT and Numerical Skills Oral and written Chem 101&102
Anal chem Courses General Chem Learning Outcomes 118 X X
X X X X X X Curriculum Mapping and Learning Outcomes
Assessment Map Assessment Map introductory level Assessment Map intermediate level Strategies
Check the assessments against the domains of learning Checked which methods were appropriate- on the map eg practical.... .... **** Development of skills over time- in different disciplines early work case study then later years presentation early simple essay testlater advanced essays Lab work yr 1 yr 2 project yr 3 poster presentation 3 2 1 yr 4 by the OSCE 4 Exercise 4: 4 :
Using the LOs which you developed yesterday, consider how an appropriate assessment strategy might be developed. DAY 2 SESSION 2 10.45 = 12.00 Quality assurance of the Outcomes-based Approach The role of outcomes in course development, evaluation and monitoring. Assuring the quality of outcomes-based assessment.
Quality Assurance (UK): Demonstrating the achievement of Learning Outcomes: Assessment : :) ( In many cases, audit teams were unable to find an explicit linkage between learning outcomes and assessment L QAA Outcomes from Institutional Audit: The adoption and use of learning outcomes and this sometimes led to recommendations. Occasionally, this absence could be related to the lack of an institutional assessment policy or to guidelines that were in need of review . Program development
NCAAA Standard 4, paragraph 4.2: 4 4.2 Programs must be planned as coherent packages of learning experiences in which all courses contribute in planned ways to the intended learning outcomes for the program. Relationship? How do you build? How do YOU STRUCTURE and package these? i.e. specify the program - field of study? SUBJECT MATTER EXPERT Start Year 1 Year 2 Outcome ?
How do you describe what happens here? Year 3 Finish WHAT attributes do you expect a student to have as a result of studying and learning? Course Any program How do you assess? Medical Sciences A subject How do you map? SUBJECT DOMAIN Engineering
Postgraduate? Undergraduate? Evidencing the achievement of Learning outcomes: Assessment : all of the Learning Outcomes of all of compulsory modules of a programme ve been attained and demonstrated by a student . assessed), then it is judged that of the Programme Outcomes have been demonstrated ( ) This requires... ... Proper structured programme planning procedures, which include planning of teaching and learning and assessment strategies J
Use of programme specifications in a consistent format Use of course specifications in a consistent format L A shared understanding of terminology The National Qualifications framework SUBJECT DOMAIN PROGRAM DESIGN SPECIFICATION A Subject Course 1 Course 2 Course x Course team awareness of Learning Outcomes ? L Course 1
Course 2 Course x Course 1 Course 2 Course x and... ... input from professionals in the field (where a professional subject is involved) and/or from senior academics in other institutions to ensure LOs are appropriate ) ( mapping of course learning outcomes across the programme, and of LOs to assessments a defined approval process by a senior academic committee within the institution. J
Programme evaluation and review NCAAA standard 4, paragraph 4.3: 4 4.3 The quality of all courses and of the program as a whole must be monitored regularly through appropriate evaluation mechanisms and amended as required, with more extensive quality reviews conducted periodically. . Programme evaluation and review whose responsibility? - MODELS OF UNIVERSITY MANAGEMENT OF QUALITY
Program teams? Central function e.g. Quality Unit? This requires... ... systematic collection of feedback from students, which includes their views on the clarity of LOs and the extent to which they have been achieved . properly specified annual review processes, requiring reflection informed by evidence such as student feedback, assessment pass rates, changes in the professional environment, etc, J ... Resulting action plan and monitoring actions implemented
L and... .... ownership of the programme review by all staff involved in teaching L consideration of annual reports by a senior academic committee of the institution, with feedback to programme teams on key points including appropriateness and coverage of LOs L Periodic review NCAAA Standard 4, paragraph 4.3.8: 4 4.3.8 4.1.1 In addition to
annual evaluations a comprehensive reassessment of the program should be conducted at least once every five years. Procedures for conducting these reassessments should be consistent with policies and procedures established for the institution. 4.1.1 J
. This requires... ... systematic procedures to be set up by the institution, with a rolling programme which ensures that all programmes are regularly reviewed (5 or 6 years are common intervals) J ( ) It is good practice to involve external input into periodic reviews All LOs should be revisited, and where necessary revised, as part of the review J Assessment NCAAA Standard 4, paragraph 4.4:4.4
4 Student assessment processes must be appropriate for the intended learning outcomes and effectively and fairly administered with independent verification of standards achieved. J . This requires... ... that a range of assessment methods is used, so as to cover different LOs and different learning styles that specifications for assessment, and criteria for grading, are transparent and appropriate to the level of the course within the NQF that second-marking or moderation is routinely used that there is clarity about the grounds on which students may appeal a mark, and that students are aware of this
and... .... that deadlines for the return of marked work are published and adhered to . that full and constructive feedback on assessments is provided to students, linked to the LOs and to the assessment criteria that guidelines for dealing with plagiarism, and with requests for time extensions and mitigating circumstances, are clear and are adhered to . Quality assurance of assessment of Los May be achieved: : by direct observation inspection of assessments before they are set, and of samples of marked work
indirectly by examining the specification of assessment processes J via feedback from students, from employers, from professional bodies... L FEEDBACK FROM STUDENTS Teaching quality NCAAA Standard 4, paragraph 4.6: 4 4.6 Teaching must be of high quality with appropriate strategies used for different categories of learning outcomes.
Teaching and Learning Strategies NCAAA Domains of Learning Strategies Knowledge Facts Concepts, theories Procedures Cognitive Skills Apply skills when asked Creative thinking and problem solving Interpersonal Skills and Responsibility Responsibility for own learning
Group participation and leadership Act responsibly-personal and professional situations Ethical standards of behavior Communication IT and Numerical Skills Oral and written communication Use of IT Basic maths and statistics Psychomotor Skills 146 Assessment Strategies NCAAA Domains of Learning Strategies Knowledge Facts
Concepts, theories Procedures Cognitive Skills Apply skills when asked Creative thinking and problem solving Interpersonal Skills and Responsibility Responsibility for own learning Group participation and leadership Act responsibly-personal and professional situations Ethical standards of behavior Communication IT and Numerical Skills Oral and written communication Use of IT Basic maths and statistics Psychomotor Skills 147 This requires... ... structured training for new and inexperienced staff, which includes training in the use of LOs and their assessment
regular appraisal of staff performance J ideally, peer observation of teaching on a regular basis and... ... the identification of staff development needs, and provision of support for staff development a policy for quality enhancement which ensures that teaching strategies are regularly reviewed and that advances in pedagogic practice are disseminated L Programmes, Graduate attributes and Outcomes Characteristics of the Program
Characteristics of the Graduate Students Learning Outcomes: Program, Level and Course : 3.11 Verifying Consistency with the Qualifications Framework 3.1.1. Requirement 3. Page 11 11 3 The program objectives should develop learning outcomes in all of the required domains of learning. To provide evidence that this is done:
. : Learning objectives specified for the program should include outcomes in all of the domains. . Responsibility for achieving these learning outcomes should be distributed appropriately across the courses within the program and included in course objectives. L . . Program and course specifications should include methods of teaching and student activities that are appropriate for the learning outcomes in each of the domains. . Tests, examinations and other required assessment tasks should include appropriate forms of assessment of learning in each of the domains. . Program evaluations, including student, graduate or employer surveys and/or other mechanisms should include attention to learning outcomes in each of the domains.
L Characteristics of Programs and Expected Learning Outcomes at Each (Level in the Framework )Page 15 ( 15) Example Bachelor Degree 5.3 Level 3. Bachelor 5.3 Characteristics of Programs An award requiring a minimum of 120 credit hours, normally following four academic years of full time study or equivalent. There are differing expectations for length of programs in different fields of professional study and programs. 120 . . . A bachelor degree program is designed to develop a comprehensive understanding of a broad field of study, with some studies taken to considerable depth and involving critical
analysis of the latest developments and research. Students should be aware of relevant knowledge and theory in other related fields of learning. . L L A bachelor degree is the basic qualification for entry to a number of highly skilled professional fields and programs in these fields should develop both the knowledge and skill to practice in those professions, and the background in practical and theoretical knowledge and research to proceed to further study. . Characteristics of Graduates Holders of a bachelor degree should have demonstrated: : Knowledge of a comprehensive, coherent and systematic body of knowledge in a field of enquiry and of the underlying theories and principles associated with it; The ability to investigate complex problems and develop creative solutions with limited guidance, using insights from their own and other related fields of study; The ability to identify and use appropriate mathematical and statistical techniques in the analysis and resolution of complex issues, and select and use the most appropriate mechanisms for communicating the results to a
variety of audiences; Capacity to provide leadership and willingness to cooperate fully with others in joint projects and initiatives; L J L In the case of a professional program the full range of knowledge and skill required for effective practice in the profession concerned; . L In the case of an academic program not geared to professional practice, in depth knowledge and understanding of research literature in a field, and ability to interpret, analyze and evaluate the significance of that research in extending knowledge in the field. L 156 Issues we need to recognize: Outcomes can be defined at different levels for different purposes. Outcomes are complex, based on the integration of knowledge, skills and values.
Multiple outcomes need to be integrated and treated holistically The outcomes will provide the springboard for students to become active learners. The process for creating outcomes at different levels needs to be collaborative and open. It needs to involve a broad range of stakeholders. J Staff development and support will be necessary to assist individuals and teams at different levels to define learning outcomes that are significant. L Assessment needs to be in place at different levels to monitor students progress in their learning and provide constructive feedback. J J L Assessments primary role is to promote learning. .
Exercise 5: Taking account of what you have heard today, if you were to write a new learning, teaching and assessment strategy for your institution or department, what are the most important aspects of this strategy? . DAY 2 SESSION 3 - 12.45 14.00 Managing the move to Learning Outcomes -a possible approach
Reflections on 5 themes from UK QAA Institutional Audit J L QAA The adoption and use of learning outcomes 1. Institutional approaches to learning outcomes 2. The use of external reference points 3. Programme design 4. Assessment 5. Information provided to students Reflections on 5 themes from UK QAA Institutional Audit L 5 J Key messages * 1. Each institution must reflect and find its own approach (not emulate) ) ( 2. There is ample evidence in reports that students valued the
clarity that the use of learning outcomes had brought to the overall purpose of their programme, the interrelationship between parts of the programme and the nature and purpose of assessment tasks. Reflections on 5 themes from UK QAA Institutional Audit L 5 J Key messages * 3. Stimulated institutional approaches to looking closer at the relationship between teaching, learning and assessment practices and how these managed 4. Old processes not always fully explicit J . 5. Discipline self evaluation invariably used external reference points (FHEQ)
6. Weakness in differentiation between batchelors and masters levels (eg BEng and MEng) L Reflections on 5 themes from UK QAA Institutional Audit L J Key messages * 7. Mapping: the extent to which intended learning outcomes at module or unit level allow the learner to achieve programme level intended learning outcomes : 8. Relationship between learning outcomes and assessment not systematically applied L 9. Connections are not systematically made to the big picture and information is not communicated to students e.g. in handbooks. . Session (6) 12.30 14.00
DAY 2 2 ()6 Action Plan: Managing the move to Learning Outcomes a possible approach : Central University policy usually starts from Teaching and Learning Strategy, Assessment policy J Faculties/Schools then develop their T&L strategy and LOs for programmes Individual module teams develop their LOs in the light of Faculty and University policies . L J Mapping takes place to ensure that all programme-level LOs are being achieved In professional fields, due note is also taken of accreditation requirements L LOs published to students
Annual and periodic reviews ensure that policies are being maintained Managing the move to Learning Outcomes critical success factors Strict deadlines for completion of each stage need to be set and adhered to Commitment from the highest level of management is essential J Champions should lead development at faculty, programme, module level Staff development should support the project at every stage Proper reporting lines through committees should maintain oversight of the project, with regular updates Managing the move to Learning Outcomes the key point
The change is unlikely to be successful if it is cosmetic/driven by external requirements/minimalist The change is most likely to succeed if it is driven by the wish to enhance the student experience J Why LOs important? The student learning experience Do you have an institutional learning outcomes based approach to provision? (i.e. on all modules taken?) i.e. define student attainment in terms of achieved learning outcomes. (p5 QAA)
: (L . ) . ( 5 ) Activity: Action Plan: :: Reflecting on what you have learned in the workshop draft some key actions (i.e. how you will apply what you have learned and make use of it when you go back). L : ( ) . Activity 1: Your expectations ? .... :1 Have your objectives (outcomes) for the workshop been met? ()
In what way has your knowledge and understanding of LOs improved? Summary and conclusion .. .. We wish you every success with your work Thank You Useful references (UK) ) ( Quality Assurance Agency (QA)
Higher Education Academy (QI/QE) HEA Engineering Very generic learning outcomes for medicine are given at that Tuning Project website (seeking to find a level playing field across Europe) http://www.tuning-medicine.com/ The Scottish Doctor , very closely related to the Tuning Project but more UK focused and for Scottish institutions.