The Central Role of Commentary on Evidence in E-portfolios

Dr Terence Love
Curtin University, Australia
Lancaster University, UK
IADE Creative University, Portugal.

Dr Trudi Cooper
Edith Cowan University, Australia

Introduction

In this chapter we suggest that having a Commentary on Evidence section is central to successfully achieving the benefits of e-portfolios. We propose that Commentary on Evidence is necessarily a separate section of an e-portfolio in which an e-portfolio owner describes item by item exactly how each piece of evidence they present supports the claims they make about their skills, learning, competence, and experience. We observe that in the literature and discussions about e-portfolios, the role of this Commentary on Evidence section has been overlooked or problematically been assumed to be identical to e-portfolio owner’s ‘reflections’ on their experiences and learning. Previously, we have used the term Commentary for this section of an e-portfolio. The change to using the term Commentary on Evidence is to reaffirm the focus on ‘evidence’ to clarify meaning, and to differentiate the Commentary on Evidence from ‘reflective practice’ which a student undertakes as part of their learning activity.

There are many perspectives on the purpose(s) of e-portfolios, with implications for their assessment. For us, e-portfolios always have three roles: i) they make claims about the author’s skill, knowledge, experience; ii) they present evidence to support these claims; and iii) they document a learning process. The literature on portfolios is divided between two perspectives on assessment: the formative and the summative (see, for example, Barrett, 2008; Callele, 2008; Johnston, 2004; Murphy, MacLaren, & Flynn, 2009; Shavelson, Klein, & Benjamin, 2009; Strivens et al., 2009a, 2009b). In formative portfolios, the author documents their ‘learning journey’ and demonstrates that learning has occurred. On summative portfolios, the author documents their ‘learning outcomes’ and contains claims about the author’s particular skills, proficiency, knowledge or experience. A high quality portfolio presents sufficient evidence to convince the reader of the plausibility of claims made. In this chapter, we are primarily concerned with the summative portfolio, although some of the statements also apply to formative portfolios.

In the following sections, we explain in more detail the roles and purpose of a Commentary on Evidence in an e-portfolio and why and how we believe it has a central and essential role in successful e-portfolios.

 We explain the significance and benefits of including a Commentary on evidence on Evidence in e-portfolios and why:

From observation, e-portfolios (and traditional portfolios) without a Commentary on Evidence are problematic and difficult to assessment because of doubt about how the e-portfolio creators intended the evidence to be interpreted. The assessor, is left to infer significance of evidence, and may be left with unanswered questions about the status of ‘expert’ witnesses. Without a ‘Commentary on Evidence’, e-portfolios present problems in terms of readability, difficulties of assessment, and poor fitness of purpose, often reflected in excessive size.

E-portfolio platforms enable people to collect a single bank of evidence of documents and artifacts, which they can then use in different ways to produce selected views on their e-portfolio for different purposes: such as assessments, resumes’ and curriculum vitae (Hallam et al., 2008; JISC, 2008; Thornton, Johnson, & .Rees, 2009). The benefits of different types of e-portfolio are different for different stakeholders. It is important to understand and address the needs of all stakeholders to produce successful e-portfolio outcomes (Cooper & Love, 2007; JISC, 2008; Nottingham CETL for Integrative Learning, 2006). This is an important issue in view of the widely accepted understanding that there are many problems that can beset e-portfolio implementations (see, for example, Chou & Chen, 2009; Nagler, 2009; Shavelson et al., 2009). This chapter argues that an effective Commentary on Evidence is crucial to realization of the full benefits of e-portfolio platforms regardless of the types of e-portfolios that are produced. In the chapter, we discuss how to develop effective Commentaries on Evidence across e-portfolio platform implementations and their different stakeholders. The chapter also describes the relationship between Commentaries on Evidence; learners’ evidence of their learning, both direct and indirect; and the logic that determines whether and how items of direct and indirect evidence support the claims of expertise, competence, learning and personal attributes.

The Problems of Evidence in E-portfolios

A central role of e-portfolios is as a container of items of evidence of an e-portfolio owner’s learning, skills, experiences, competence and knowledge. Individual items of evidence, however, often relate to more than one aspect of learning, skill, experience, competence or knowledge. In addition, many individual items of evidence provide only partial support for claims of skill, experience, competence, knowledge or ability.

For many items of evidence it is not obvious or self-evident which learning outcomes they support. Problematically, this interpretation and judgment of evidence is frequently left to the e-portfolio reader to infer. In many e-portfolio implementations, interpretation of evidence into the assessment of learning and skill is left open to the e-portfolio reader, it being made explicit to the reader the intended relationships between evidence and learning. This can result in two kinds of problems for readers and failures of assessment. We refer to these as under-valuing and over-valuing.

In undervaluing the e-portfolio reader overlooks the ways that any particular item of evidence may imply claims about skills and knowledge. This happens typically in two situations: where the evidence in the e-portfolio does not explicitly direct the reader’s attention to the significance of particular pieces of evidence and information, or wher it deos not explain why they are important. A consequence is the reader may miss links between some pieces of evidence and the knowledge and skill claims made by the author. If a Commentary on Evidence is not provided, very often, the reader will miss some of these links.

In over-valuing, items of evidence are overvalued by the reader and the reader is more impressed by the evidence than the evidence merits. This may occur when a portfolio is well presented and visually attractive, and hence the reader assumes that the content must be of equal merit. Inexperienced students sometimes form a belief that a large volume of peripheral evidence well-presented, can compensate for a lack of substance. Most experienced assessors are not misled so easily.

Both over-valuing and under-valuing are problematic and can be avoided if a Commentary on Evidence is provided. A good Commentary on Evidence will explain clearly to the reader how the evidence presented supports the claims made by the author. The author can also acknowledge any weakness in their evidence.

The commentary on Evidence can be used in e-portfolios to improve accuracy of assessment and make portfolios easier for the reader to understand by:

·         Ensuring that items of evidence provide plausible support for the claims made in the e-portfolio (skills, experience, proficiency, competence, knowledge, abilities or personal attributes).

·         Ensuring the e-portfolio owner’s claims for learning, skills, experiences, competence, knowledge and abilities are clear and obvious to the e-portfolio reader

·         Ensuring that where multiple items of evidence contribute to plausibility of the claim, the contribution of each piece of evidence is made clear

·         Providing additional contextual information that strengthens the evidence.

·         Ensuring all the complex relationships between items of evidence and claims of the e-portfolio owner about their attributes, skills, experiences, competence, knowledge and abilities are made explicit to the e-portfolio reader

The UK expert body on ICT in education, JISC (2008), defined e-portfolios to include the above attributes,

An e-portfolio is the product, created by the learner, a collection of digital artifacts articulating [emphasis added] experiences, achievements and learning

We respectfully suggest that JISC’s documentation, and that of most of the literature of e-portfolios, has to date overlooked or ignored the importance of making explicit the process or activity of ‘articulation’ in the above definition. We suggest that all portfolios require making this articulation between evidence and claims of the e-portfolio owner (the learner) explicit. This is the role of the Commentary on Evidence section for e-portfolios described in this chapter.

Direct and Indirect Evidence

To make explicit that items of evidence provide proof of an e-portfolio owners claims of skills, knowledge and experience requires understanding how valid proof depends on whether evidence is direct or indirect (see, for example, Allen, 2008; Emden, Hutt, & Bruce, 2004; Mueller, 2008; Runshe, 2009; Terry, Wilding, Lewis, & Olsen, 2007). As a matter of course, educators, teachers, lecturers and all others responsible for the teaching of an e-portfolio owner must themselves have a very clear understanding of the relationships and roles of direct and indirect evidence and must be able to impart that understanding to the learner e-portfolio owner.

Indirect evidence is relatively worthless until it is associated with other indirect evidence that ensure it provides convincing support for claims made by the e-portfolio owners.

Example 1

Direct evidence of the competence of an apprentice electrician wiring the plug of an appliance is the observation by an approved examiner of that individual wiring an appliance plug.

In contrast, indirect evidence of the competence of an apprentice electrician wiring the plug of an appliance would be a report by a third party other than an approved examiner that they had observed the electrician wiring up an appliance plug. At that point, with only that element of indirect evidence, there is insufficient proof of the apprentice electrician’s competence at wiring the plug of an appliance:

Direct and indirect electronic evidence

Electronic records of observations are typically indirect evidence of a person’s learning and require support from other, appropriate, indirect evidence to come together into valid proof. The relationship between items of indirect evidence and claims must be made explicit to the e-portfolio reader; else their validity remains subject to question. Electronic representations, whether of hardcopies such as certificates and examination results, pictures and photos, and dynamic representations such as video and audio recordings can be easily modified via software. For example edited film or video recording of a professional intended to demonstrate an e-portfolio owner’s professional judgment might have been carefully edited to remove any evidence that might contradict the story presented in their portfolio. It is incumbent on the e-portfolio owner, e-portfolio reader and examiner and the educators guiding and supporting the e-portfolio owner’s learning to be aware that many forms of electronic evidence can be modified and thus by their nature are rendered potentially useless as direct evidence.

The consequences of overlooking evidence issues

Without effective Commentaries on Evidence, assessment of e-portfolios can become a very difficult subjective exercise where the assessor weighs complex evidence without guidance and sometimes without clear assessment criteria. This is one of the problems portfolio-based assessment was intended to solve. Recent concerns about inter-rater reliability of portfolio assessment indicate the importance of this issue (see, for example, Brzycki, Barker, & Lenze, 2003; Bures, Abrami, & Bentley, n.d.; Meyer & Latham, 2008; Yao & Aldrich, 2009). The implications of not addressing these problems of evidence issues (e.g. via a Commentary on Evidence or something similar) include that e-portfolios may be less efficient and effective than other assessment methods and may result in significant waste of time of assessors and e-portfolio owners. Perhaps more importantly, there is likely to be a reduction in the quality, standard and learning of the e-portfolio owner and the quality of assessment due to the wastage of time and other resources in creating and assessing e-portfolios that fail to reflect their owner’s learning and other personal attributes. Assessment is also open to contest, and successful appeal, if assessment cannot be moderated effectively.

The Role of the ‘Commentary on Evidence’

Portfolios contain evidence from a variety of sources. The credibility of claims made in an e-portfolio depends upon judgments about the validity of evidence presented. These judgments depend essentially on whether the reader is convinced that the evidence is reliable, comprehensive and credible. The purpose of the Commentary on Evidence section in an e-portfolio is to provide explanations to guide readers when they make these judgments. Assessment is made easier when explicit links are made between the evidence and the criteria for assessing a portfolio and there is clarity about the roles of direct and indirect evidence.

The Commentary on Evidence relates the claims made in the e-portfolio to the evidence presented in support of those claims and describes these relationships in an accessible manner for the reader or assessor, see Figure 1.

Figure 1: Centrality of Commentary on Evidence section in e-portfolios

The Commentary on Evidence makes clear for the reader exactly which elements of evidence support which of their claims and how. The Commentary on Evidence critically describes the validity and completeness of evidence presented and enables the e-portfolio owner to acknowledge and explain any limitations to the evidence.

Commentary on Evidence not ‘Reflection’

It is important to clearly distinguish and differentiate the role of the Commentary on Evidence from the activity of reflection. The activity of reflection is an important part of the e-portfolio owner’s learning in which the learner become aware of their thought processes when they make decisions and the links they make between theory and practice. Reflections form part of the evidence that an e-portfolio owner presents. Reflections help the learner explain to the reader why they acted as they did, and why they might act differently in future. Reflections are items of evidence that can be assembled with other evidence to support the portfolio-owner’s claim that they have achieved specified learning outcomes; that they possess particular attributes or have achieved a particular level of proficiency.

‘Reflection’ is an activity that results in the creation of an element of indirect evidence (describing the e-portfolio owner’s reflection). This is a different kind of entity in the e-portfolio to the Commentary on Evidence. The items of evidence that are the outcomes of reflection are the subject material on which the Commentary on Evidence reports. The process and differences in role are shown in Figure 1 above.

By contrast, the role of the Commentary on Evidence is to provide a descriptive link for the reader that explains item by item how single pieces of evidence presented in an e-portfolio justify the e-portfolio owner’s specific claims of competence, skill, understanding, experience and learning. The Commentary on Evidence does not form part of the evidence. Like, the Table of Contents, in an e-portfolio, the Commentary on Evidence is a comprehension aid.

Example 2: Commentary on Evidence of graduate-level professional skills in Youth Work

Frances is a youth worker. She is in the third year of a degree in Youth Work, has completed three supervised practica as part of her studies, and she has part-time employment as a relief worker in a youth refuge. She is putting together a career e-portfolio to demonstrate her skills so she can gain skilled professional employment when she graduates.

Frances’ e-portfolio will demonstrate many of the skills and attributes required in her chosen profession. Her claims might include a mix of achievements, personal characteristics, skills, habits, knowledge, and values statements. There are many different types of claims Frances could focus on, and, within each, there are many specific claims to expertise, learning, experience and personal skills that Frances could make. These can be further differentiated, for example Frances might make separate but related claims about her knowledge AND her ability to apply that knowledge skillfully.

Each of Frances claims must be justified by combinations of evidence, if her portfolio is effective. The potential diversity and complexity of the relation between claims and evidence in Frances’ e-portfolio is illustrated in Table 1. This example is simplified for brevity of explanation, but the complexity of the relationship between claims and evidence is illustrated even in this simplified example, see Table 1. A Commentary on Evidence provides a formal explanation of relationships between claims and evidence and removed the requirement for the e-portfolio reader to guess what the relationships might be between the evidence presented and the claims made.

Table 1: Commentary on Evidence

Types of claim

Claim

Type(s) of evidence

Personal characteristic

Trustworthiness

Two previous employment references indicating their confidence in Frances' trustworthiness

Skill (and knowledge)

Drug and alcohol counseling

Transcript of studies successfully completed
Statement from supervisor of second practicum affirming her skills

Achievement

First Aid Certificate

Electronic copy of certificate (with a statement affirming its authenticity). Original available on request.

Habit

Punctuality

Statement from current employer and two previous practicum supervisors

Knowledge (and skill to use knowledge)

Good knowledge of local referral agencies (referral skills)

Sample of her up-to-date referral file that details her contacts in a variety of local agencies to which she might refer clients .

A statement from her current employer about her referral skills

Value Statement

A statement of the values that underpin her approach to her work

A statement that explains her values, and examples of how she has applied these values in her work

Confirmation from supervisors and co-workers that confirm the veracity of the accounts and examples that Frances provides.

Knowledge / skill/ values/ personal characteristics

Sound Professional judgment

Examples of difficult situations she has faced, decisions she made, her reasoning, her outcomes and her reflections

Confirmation from supervisors and co-workers that confirm the veracity of the accounts and examples that Frances provides

When Frances creates her e-portfolio, she would include for each of her claims a paragraph to provide a commentary on her evidence to explain the relevance and context of the items of evidence she has attached for each claim.

Her Commentary on Evidence helps Frances avoid presenting superfluous, irrelevant or misleading items of evidence. In turn this improves the understanding and validity of Frances e-portfolio for readers and assessors.

Resolving Problems of Incomplete Evidence using a Commentary on Evidence

How can the reader or assessor best evaluate the materials presented in an e-portfolio? This is a question of central importance if, as we suggest, the primary purpose of the e-portfolio is to present the plausible claims by the e-portfolio owner to knowledge, skills, personal characteristics etc. It is a question that is often overlooked, especially where specific benefits offered e-portfolios and the assessment processes to elicit them are ignored. In the e-portfolio assessment literature it is often assumed that it is appropriate to use for summative assessment approaches more appropriate to formative assessment of single items of evidence such as projects or term papers. We suggested earlier in the chapter that the significance of accurate assessment is now emerging and is indicated by the increasing literature that foregrounds problems of inter-rater reliability of e-portfolio assessment (see, for example, Brzycki et al., 2003; Bures et al., n.d.; Meyer & Latham, 2008; Yao & Aldrich, 2009).

If e-portfolios are assessed in terms of the validity of complex evidence in justifying the claims of the e-portfolio owner (as we suggest is a key benefit of e-portfolio assessment over single item assessment) issues of inter-rater reliability and moderation are more easily resolvable. Central issues are how to identify and evaluate incomplete evidence and how to ensure that incomplete evidence is matched with other supporting evidence to plausibly support an e-portfolio owner’s claims of learning, skills and knowledge.

For example, in many instances, e-portfolios contain ‘witness statements’ from people who have observed the work of the e-portfolio owner, and who provide statements about their skills, habits or personal characteristics. How should the reader or assessor evaluate these testimonials and references if the individuals providing the witness statements are not known to them either personally or by reputation? The Commentary on Evidence can help identify such incomplete evidence and provide extra information to assist the reader.

Example 3: Using Commentary on Evidence to identify and resolve incomplete evidence in an e-portfolio

Jennifer is a childcare worker. Jennifer’s work colleague has provided her with a reference that attests that Jennifer has demonstrated the ability to work effectively under pressure; that she has been a good team worker; and that she is highly skilled in her work with young children in the context of childcare. This is an example of an incomplete item of evidence.

In some cases, such a ‘reference’ will provide information to help the reader make judgments about the plausibility of the reference, although often the ‘clues’ are incomplete. For example, the reference may be on letterhead paper, and this would indicate that Jennifer’s employer has provided the reference. The reference would be likely to give the name, date and role title of the referee. It may remain incomplete, however, in terms of validating the referee’s competence at making the assessment. For example, the referee may include their abbreviated qualifications, a BSocSci. This does not explain whether this is a reference from someone who is knowledgeable about professional expertise in child care, or perhaps a new business owner whose qualifications are not professionally relevant. Put simply, the reference may not contain sufficient information to enable the reader or assessor to make informed judgments about it. Much potentially useful information is not provided.

This is an example of where a full explanation by the e-portfolio owner in the Commentary on Evidence section can supply some of the missing information and increase the plausibility of the evidence.

Creating the Commentary on Evidence can help Jennifer as e-portfolio owner to identify weaknesses in the evidence she has collected and help her identify which additional evidence is necessary to address these weaknesses. In her Commentary on Evidence, Jennifer could add information about the length of her employment, the nature of her role, and information about the professional qualifications and experience of her referee. If she supplements the reference with additional evidence about her skills from other sources, she can use her Commentary on Evidence to explain how the separate pieces of evidence ‘fit together’ to provide plausible and convincing evidence for her claims, evidence that will convince readers because it is drawn from reliable independent sources.

Seven Functions of Commentary on Evidence in Effective E-portfolio Systems

The Commentary on Evidence clarifies and simplifies. For the e-portfolio owner, writing the Commentary on Evidence is the central part of the communication process with the reader. The e-portfolio owner can explain to the reader why they have included particular pieces of evidence and how they have combined evidence from different sources to compensate for limitations of single pieces of evidence.

An effective Commentary on Evidence performs multiple functions in a successful e-portfolio. Besides relating the evidence presented to the claims that are the purpose of the e-portfolio, the Commentary on Evidence can draw the reader’s attention to elements of evidence the reader might not otherwise notice and the significance of these elements in terms of the interest of the purpose of the e-portfolio and the reader’s purpose in reading it (e.g. assessment, job skills, recognition of prior learning etc).

The Commentary on Evidence also provides a filtering mechanism by which the e-portfolio owner selects only those elements and artifacts of evidence relevant to the specific purpose of their portfolio. In addition, the Commentary on Evidence provides a narrative that discusses the learning, skills, competencies, personal attributes and experiences of the e-portfolio owner in terms of the assumed interest of a particular readership. The Commentary on Evidence guides the reader and explains the structure, meaning and purpose of the evidence that is presented.

In short, the Commentary on Evidence has seven functional roles that are centrally important to the effectiveness of e-portfolio systems:

  1. As a location and a rationale for the e-portfolio owner to describe how the evidence they present in an e-portfolio validates the claims they make and aligns with the implicit or explicit criteria about the purpose of the e-portfolio and the interest of the reader and examiner of it.
  2. As a meta-reflection on the evidence presented in an e-portfolio and the strengths and limitations of that evidence.
  3. To improve the quality of assessment and reduce the time and resources needed for assessment by explaining to assessors how the evidence provides valid proof of the e-portfolio owner’s claims. The Commentary on Evidence on evidence significantly reduces assessors time commitment in marking each e-portfolio by removing the need to needs to search through each individual e-portfolio to try to interpret how skills, knowledge and experience might be validly inferred from an otherwise unstructured mess of evidence and how these would coherently contribute to individual marks in a way that moderates across all e-portfolios in a justifiable manner.
  4. As a section in which the e-portfolio owner can clarify, in terms of the evidence, the claims they are making in their e-portfolio
  5. As a location within the e-portfolio to provide additional information that enables the reader to assess the strength of the evidence presented
  6. As a learning tool for the e-portfolio owner by which they clarify their thoughts about what they are trying to present in a particular e-portfolio view or instance.
  7. As a theoretical and practical tool by which the e-portfolio owner can clarify the value of individual items of evidence in particular in identifying which potential elements of evidence are irrelevant and not worth including in a e-portfolio instance, o in the larger e-portfolio collection of evidence.

Different kinds of Commentary on Evidence are needed in different circumstances. For example, the Commentary on Evidence for a curriculum vitae for a specific job application would explain how elements of evidence relate to selection criteria for the job; how other skills, knowledge, or personal attributes will benefit the potential employer; why the candidate should be preferred over other applicants for the position.

Commentary on Evidence for Improving Problem E-portfolios

The benefits of using e-portfolios are offset by some characteristics problems of how some e-portfolios are developed. There are several common characteristics of problematic e-portfolios or e-portfolio implementations. From experience, these include e-portfolios that:

A well-written Commentary on Evidence provides a broad-based approach to improving problem e-portfolios. It does this by requiring the e-portfolio owner to explicitly focus on the purposes of their e-portfolio, the needs of the reader or examiner, carefully identifying the evidence they choose to include in the e-portfolio, the e-portfolio evaluation criteria, and how their evidence justifies their claims. When these issues are addressed, the problem characteristics are minimized and the e-portfolio outcomes are improved as a result.

In the context of assessment this is very significant. E-portfolios with an effective Commentary on Evidence can offer educational benefits of ready access to constructivist assessment methods without the time penalty often associated with e-portfolio assessment.

Commentary on Evidence and Lifelong Learning E-portfolios and Other Educational Modalities

‘Whole of Life’, lifelong learning, e-portfolio systems bring together all the evidence of an individual’s learning, skills, experiences and knowledge to be accessed through a single digital window(Grant, Rees Jones, & Ward, 2004; JISC, 2006; Perennes & Duhaut, 2009). The scope of lifelong learning e-portfolios are also likely to include competency-based education, negotiated learning, authentic learning, recognition of prior learning, problem-based learning, and self directed and auto-didactic learning. Informal education along with continuing professional and personal development is likely to include outcomes from any of these modalities of education at different points in most individuals’ lifelong learning pathways. In all of these educational modalities, there is a strong focus on accurate assessment tied to evidence – a strength of the Commentary on Evidence. In addition, one of the potential advantages of e-portfolios include the ability to remove many of the problems associated with the subjective and superficial assessment of complex higher level skills and knowledge because the Commentary on Evidence can help provide accurate authentic assessment of higher level learning.

An individual’s lifelong learning e-portfolio comprises a virtual representation of a pseudo-container of digitalized elements of evidence located physically in a variety of locations across many different electronic systems. The e-portfolio owner will select and present ‘views’ onto selected items of that evidence. A core problem is providing an integrated overview of the disparate evidence for a specific e-portfolio ‘view’ selected from multiple evidence sources within a larger ‘lifelong learning e-portfolio. Legal responsibilities for security and privacy of lifelong e-portfolios are another important consideration (see, for example, Charlesworth & Home, 2005).

The Commentary on Evidence offers a solution to many of the above issues by providing a central integrating role that is a single focus point to interpret the disparate evidence of the e-portfolio owner’s learning, knowledge, skills, competencies and personal attributes and experiences. There are software programming benefits in centralizing the e-portfolio computer interface around the Commentary on Evidence in order that the e-portfolio owner’s evidence can be easily accessed by the e-portfolio owner as author and can enable them to use the automation possibilities of virtual environments to draw elements of evidence from multiple data to bring it together into a coherent appearance. In addition, they offer the means to centralize addressing legal responsibilities.

Commentaries on Evidence and Stakeholders

The Commentary on Evidence has different benefits for different stakeholders:

·         The e-portfolio owner

·         The e-portfolio readers and assessors. This also includes e.g. examiners, employers, professional colleagues, teachers, lecturers, tutors, coaches, and friends.

·         Educators of whom the e-portfolio owner is a student. This includes lecturers, teachers, tutors

·         Education managers responsible for providing or improving education services

·         Information and Communication Technology professionals who are responsible for providing e-portfolio services

E-portfolio owners

For the e-portfolio owner, their use of a Commentary on Evidence helps them:

·         Develop their e-portfolio to show them in their best light by helping them identify the evidence they need to prove their claims about their learning, skills, knowledge, experience and personal attributes.

·         For e-portfolio creators of a ‘view’ on a lifelong e-portfolio, the Commentary on Evidence section offers a useful central focus for being able to quickly identify and insert evidence material from the larger whole of life e-portfolio resources.

Portfolio readers and assessors

A Commentary on Evidence offers e-portfolio readers and assessors improvements to efficiently and effectively review an e-portfolio. The Commentary on Evidence could be expected to list for their convenience the purposes, criteria and claims of an e-portfolio in a way that the reader can immediately tell whether the e-portfolio is aligned with their interests without having to delve deeply through the detail of evidence or trying to infer claims and justification from items of evidence. The Commentary on Evidence can also indicate whether the e-portfolio owner understands the purpose of their e-portfolio and any assessment criteria and can adequately identify which of their personal learning and experiences relates to them. This offers an understanding of how much the e-portfolio has value as a means of understanding the e-portfolio owners claims and evidence.

In practical terms in an online environment, user interface structures can make processes self-evident and efficient for the e-portfolio reader. For example, navigation and ‘click though’ tools can enable readers to review Commentary on Evidence discussions at the surface level and click through to get detail when they require it.

Educators

A variety of educators are likely to have interaction with e-portfolios: e.g. teachers, lecturers, tutors, coaches, mentors and advisers. The Commentary on Evidence offers four types of benefits:

·         A Commentary on Evidence provides a fast track to understanding the learner’s formation and their personal learning trajectory. This enables an educator to identify and tailor the best learning experiences for the e-portfolio owner.

·         E-portfolio owners may be their own primary educator in the new areas of lifelong learning, continuing professional development and where education is undertaken outside formal learning institutions. In these circumstances, the Commentary on Evidence aids reflective learning.

·         For educators in the role of e-portfolio ‘coach’, a Commentary on Evidence helps maximize the quality of output of e-portfolio. It provides a tool for the coach to be able to demonstrate modifications to the e-portfolio without undertaking major changes to its structure. This is especially useful in the context of creating an e-portfolio view onto a larger lifelong e-portfolio.

·         Where educators are the assessor, the Commentary on Evidence provides an efficient basis for formative and summative assessment of the e-portfolio owner/student in terms of their reflective understanding of their learning and the validity of the evidence they present to demonstrate their learning

Education managers

For education managers, the Commentary on Evidence offers benefits for improving learning and assessment across an organisation as described in earlier sections. In addition, it offers a useful focus for anonymous data mining for gaining insights into:

·         Whether the processes of education are proceeding satisfactorily

·         Whether the resources and other practical issues around the e-portfolio building and related education modalities are functioning well and efficiently.

·         The levels of learning and their distribution for quality management purposes;

·         Overarching management of curricula

·         The potential for gaining additional funding from government for example via their distribution of students in lower socio-economic demographics.

Information and Communication technology professionals

For ICT professionals charged with the implementation of e-portfolio applications there is potential to improve, services and systems by locating interfaces and e-portfolio services around the Commentary on Evidence:

·         Security/authentication process both for accessing the e-portfolio and for inputting marks, comments etc relative to an e-portfolio snapshot.

·         Different variants of the presentation of the Commentary on Evidence for learning guidance and assessment. There are significant advantages in keeping the two roles separate in software terms. This is usually addressed in multi role software by allowing the individual to log in separately as each of the roles that they possess.

·         As the locus of comments and advice.

·         As the locus of help and support in the form of FAQs, manuals, a forum, guides and links to other knowledge.

·         As to focus of opportunity to provide two parallel ‘instances’ of a e-portfolio view for an educator to be able to, modify and demonstrate changes to a copy of an e-portfolio owners work without modifying the original.

Conclusions and Summary

This chapter has described how the inclusion a Commentary on Evidence section in e-portfolios fulfills a missing central role and offers significant benefits for all stakeholders. In particular, we have described how a Commentary on Evidence section can offer improved assessment and learning.

Portfolios and e-portfolio systems including e-portfolio systems are intended to provide significant benefits over and above conventional educational modalities. We suggest that a Commentary on Evidence as the core of e-portfolio systems and individuals’ e-portfolios is a significant leverage point and differentiator that results in e-portfolios that offer significant teaching and learning benefits in educational outcomes, assessment efficiency and effectiveness and provides maximum opportunities for benefits in human, educational and financial resources – benefits to all stakeholders. In contrast, we suggest that e-portfolios and e-portfolio systems that do not use Commentaries on Evidence or use them peripherally risk producing underperforming e-portfolios that can perform less well than conventional education programs.

Centralizing e-portfolios around a Commentary on Evidence helps more usefully align e-portfolios with informal areas of education such as continuing professional development, lifelong learning, authentic learning, problem-based learning, negotiated learning, adult education, community education, recognition of prior learning, auto-didactic learning and andragogic education.

We suggest a practical way to improve e-portfolios for all stakeholders is to centralize the Commentary on Evidence in software in all aspects of e-portfolio related activities. Using Commentary on Evidence as the focal point offers simplifying usability benefits for connections between portfolio-building and assessment tools, different e-portfolio elements and data stores as well as the basis for meta-analysis of e-portfolios.

References

Allen, M. J. (2008). Strategies for Direct and Indirect Assessment of Student Learning,  Available from http://assessment.aas.duke.edu/documents/DirectandIndirectAssessmentMethods.pdf

Barrett, H. C. (2008). Using Electronic Portfolios for Classroom Assessment. Journal, 13(2), 4-6. Retrieved from http://electronicportfolios.com/portfolios/ConnectedNewsletter-final.pdf

Brzycki, D., Barker, W., & Lenze, J. (2003). A Study of Interrater Reliability in Assessing Teacher Preparation E-Portfolios. In C. Crawford (Ed.), Proceedings of Society for Information Technology & Teacher Education International Conference (pp. 107-110). Chesapeake, VA: AACE.

Bures, E. M., Abrami, P. C., & Bentley, C. C. (n.d.). Electronic Portfolios – Now That We Have them, How Can We Assess Them?,  Available from http://grover.concordia.ca/epearl/en/download/Bures_Electronic_Portfolios.pdf

Callele, M. F. (2008). An investigation of Formative and Summative Portfolio Assessment methods. University of Saskatechewam, Saskatoon.

Charlesworth, A., & Home, A. (2005). Data Protection, Lifelong Learner Record Systems & ePortfolios: A Short FAQ,  Available from http://www.elearning.ac.uk/mle/pdp/legalguide/Data_Protection_FAQ.pdf

Chou, P., & Chen, W. (Eds.). (2009). E-portfolio Use at Higher Education Institutions: Potential problems for Pedagogy (Vol. 3). Lisbon: Formatex.

Cooper, T., & Love, T. (2007). E-Portfolios in e-Learning. In N. A. Buzzetto-More (Ed.), Advanced Principles of Effective e-Learning (pp. 267-292). Santa Rosa, California: Informing Science Press.

Emden, C., Hutt, D., & Bruce, M. (2004). Portfolio Learning. Journal, 16(1-2), 124--132. Retrieved from http://www.contemporarynurse.com/archives/vol/16/issue/1-2/article/2047/portfolio-learning

Grant, S., Rees Jones, P., & Ward, R. (2004). E-portfolio and its relationship to personal development planning: A view from the UK for Europe and beyond,  Available from http://www.estandard.no/eportfolio/2004-11-03/terminologiavklaring_UK.pdf

Hallam, G., Harper, W., McCowan, C., Hauville, K., McAllister, L., Creagh, T., et al. (2008). Australian ePortfolio Project Final Project Report:2008. StrawBerry Hills, NSW: Australian Learning and Teaching Council.

JISC (2006). e-Portfolio Reference Model September 2006 Report,  Available from http://www.jisc.ac.uk/media/documents/programmes/elearningframework/ep4llfinalreport1b.pdf

JISC. (2008). Effective Practice with e-Portfolios: Supporting 21st century learning. Bristol, UK: HEFCE.

Johnston, B. (2004). Summative assessment of portfolios: an examination of different approaches to agreement over outcomes. Studies in Higher Education, 29(3), 395-412.

Meyer, B., & Latham, N. (2008). Implementing Electronic Portfolios: Benefits, Challenges, and Suggestions. Educause Quarterly, 31(1), 34-41.

Mueller, J. (2008). Authentic Assessment Toolbox. Naperville, Il: North Central College.

Murphy, T., MacLaren, I., & Flynn, S. (2009). Toward a Summative System for the Assessment of Teaching Quality in Higher Education. International Journal of Teaching and Learning in Higher Education, 20(2), 226-236.

Nagler, A. (2009). The Unintended Consequences of Portfolios in Graduate Medical Education. Academic Medicine, 84(11), 1522-1526.

Nottingham CETL for Integrative Learning. (2006). University of Nottingham JISC e-Portfolio Reference Model Project. Summary Report. Nottingham: Nottingham CETL for Integrative Learning.

Perennes, L., & Duhaut, D. (2009). E-portfolio for Lifelong Learning. Paper presented at the ICETC09, Singapore.

Runshe, D. D. (2009). Guide to the Learning Outcomes Assessment Process at the Program Level. Indianapolis: Ourdue University Indianapolis.

Shavelson, R. J., Klein, S., & Benjamin, R. (2009). The Limitations of Portfolios. Journal. Retrieved from http://www.insidehighered.com/views/2009/10/16/shavelson

Strivens, J., Baume, D., Owen, C., Grant, S., Ward, R., & Nicol, D. (2009a). The role of e-portfolios in formative and summative assessment practices - Final Report. Bristol, UK: JISC.

Strivens, J., Baume, D., Owen, C., Grant, S., Ward, R., & Nicol, D. (2009b). The role of e-portfolios in formative and summative assessment practices: Case Studies. Bristol, UK: JISC.

Terry, R. E., Wilding, W. V., Lewis, R. S., & Olsen, D. R. (2007). The use of direct and indirect evidence to assess university, program, and course level objectives and student competencies in chemical engineering. Paper presented at the American Society for Engineering Education, Honolulu, HI.

Thornton, L., Johnson, G., & .Rees, M. (2009). Assessing Impact: Eportfolios in Higher Educaiton. In 2009 Mid-Atlantic Educause Conference. Washington, DC: Educause.

Yao, Y., Foster, K., & Aldrich, J. (2009). Interrater Reliability of a Team-Scored Electronic Portfolio. Journal of Technology and Teacher Education, 17(2), 253-275.