//////////////////////////////////////////////////////////////////////////////////////////////// From: Kurt Maly Subject: focus group notes Date: Thursday, May 06, 1999 9:05 AM Teacher role Course material ? have we considered that major part is textbook, CD, IP rights, course packs Almost all have most on webpages and not in unix trees; some on ftp; Zipped software; software running on designated machines in directories ? license issues Use home grown program for grading, submitting material(?submit?) Archiving Very eager and willing to use But not willing to pay anything! to make effort to create course for first time OK if student TA does it Only if course has settled down Willing to make incremental changes Royalty issue Quite different opinion on usefulness of archiving due to changing material: Wild: 10% reuse only, almost all examples change from one to next, textbook changes Others: 80%,70%,70%, 50%,70-80% reuse Instantiating For first time scheduled to teach extremely useful; for new faculty extremely useful what current faculty are doing; different people have different perspective on same course Not share though material with faculty at other universities, at most within department. Big difference between 1st time and 2nd and thereafter creation Investment in material is high, e.g. powerpoint slides cost a lot to produce Focus library Provide also references lists (shopping basket approach, but tools exist to do that already with URLs); great idea; would use heavily, value judgement considered important; Cost effective: need to train students in how to use x different libraries vs teaching them to use one Is this solution seeking problem? Definitely no from new teacher; but maybe from teachers who change frequently. Comparing Stanford teacher who teaches 2 grad courses a year to community college teacher who teaches 5 different courses a semester. First would not use it; latter would love it. But [probably would not evolve it , just use canned approach. Student Role What would student like to see? First teacher wants to filter what goes in(e.g., not nasty e-mail to koch; or not koch?s e-mail address) Yes to performance stats; problems, FAQs ok but students probably will not go through Assignment ? could be confusing if similar to current assignment(they solve then the wrong problem and complain) Projects, exam ? very important(frat , companies have been doing this for long) Extremely important to students: level of effort required by specific teacher ? expectation on instructor Hide some details because they may give wrong impression Teacher needs to annotate material Most good courses have all this on Web! Portfolios Loved idea of portfolios , initially though of portfolios within course not within 4 years(Lotus notes); useful to othert students or whatever is given out Advisor would love this /////////////////////////////////////////////////////////////////////////////// From: Kurt Maly Subject: student focus group Date: Sunday, June 20, 1999 12:05 PM Notes on student focus group: Q1(previous offering useful): Syllabi and other info from catalogue usually outdated; would be much better for deciding which course to take Would be current on a particular Instructor Grade distribution and exam important Transferring to and from other universities (e.g. military move a lot) have difficulty finding about whats in what course, evaluating what they have Q2(what should be recorded for previous offerings): All of the listed, if vote for two: Grades -2, exam - 5, problems encountered -1, project info - 4 Q3(other info): how is course offered(teletechnet, iri, main campus,..) Detailed outline by date How much time to be spent per week as seen by teacher and actually reported by students in real courses Q3a(standard archival form): creates competition among universities; allows students to challenge teacher (see its not being offerd there why do we need to do it), Allows to see what others do Q4(portfolio): Security utmost important, would not want hackers to be able to get at their stuff; Series of different levels of permissions Would store projects, programs(which can be actually executed), papers Need easy tool for publishing, transparent to user Available for all students not just CS Students would use it for other purposes, space is commodity, would use it as stoarge place Content policy is important(no pornography), student would like simple policy not to have approval everytime they publish but rather here is whats allowed and then be responisble for adhering Very important that publishing is dynamic, see it more like their own directory or web page they can constantly change Would use for job Use would depend on major Expect lots of apathy by other than CS hackers Unless required in course would not be done Required is all that counts (students do what teachers want, e.g., 'briefcase' in windows 95 similar but not used) What does portability mean(can upload to your own directory and it still works because buckets have their own methods inside) To ensure use of portfolios give credit for it Q5(other use of portfolio): Temporary storage space Projected courses(planning tools for degree; advising) Alumni info Q6(Internet search vs library): like teacher preselected(give teacher what they want) Unguided search might get serendipitous results, provide both Q7Cobrowsing): definitely yes Need to change policy of some teachers and deptments against collaboration ////////////////////////////////////////////////////////////////////////////////// From: Steven Zeil Subject: Re: student focus group Date: Monday, June 21, 1999 2:48 PM Notes from my group: Q1: Yes, this would be useful. particularly like idea of getting grade distributions. Others argued: No, not useful. We generally have too little flexibility in when to take courses, so we take them when they are being offered. Counter-argument: useful for electives good to know up-front what will be required useful in trying to balance semester (e.g., not too many programming-intensive or writing-intensive courses at once) Would really like detailed lesson plans. Q2: Yes to all choices except "student evaluations". Strong consensus that these tend to be skewed by a few folks with axes to grind (one way or the other). Q3: Some people wanted a list of students who had taken the course. Others objected that this was a violation of privacy. Eventually compromised on idea of having students volunteer as references (raher like hiring a painter or contractor to work on your home!). Lesson plans (again) Reading lists, lists of related materials Q4: Some felt this was not important, but those who thought it was important at all thought it was extremely important - uniform interface is a major selling point. Especially across different University Depts. Q5: Yes to portfolios in general, especially if they did not require special expertise to create something useful. Aid in maintaining integrity of data over time seen as important - don't want to send a resume to potential employer and have links fail! Would a University portfolio library serve only current students or also alumni? Generally few qualms about leaving portfolio info behand at ODU after graduation, provided that they had control over terms and conditions at the beginning. Took it for granted that individuals would take initiative to protect/delete anything they did not want becoming public. Q6: Resume, courses taken - These are currently kept for employers at the Career Mgmt Center ($20 cost to students) Would be nice to be able to go beyond what's in the CMC database - papers, projects People were flattered by the notion that someone might be interested in reading their papers at a future date. Q7: Generally in favor of more structured search (but noted that the question was rather leading!) Can be had to find things on Web sometimes. Some counter-arguments: may limit research to instructor's point of view 9alothough with some instructors that may be the safest course of action!) , may discourage more independent research Q8: Perhaps because this group included many non Norfolk students, the idea of collaborative browsing was seen immediately as "neat", provided that it was integrated with a chat or other side-bar communication channel. /////////////////////////////////////////////////////////////////////////////////////// Focus Group Comments Topic: Digital Libraries Facilitator: jdm Q1: "Absolutely" "A wonderful idea, but some departments keep who is teaching what a secret, I wonder how they will deal with that" Q2: Grade Distributions and Course evaluations by students were Considered very important for course bucket inclusion. Assignment/examination/project details were thought to be useful but not so critical. Q3: The following were unanimously agreed upon by the focus group as items that should be included in all course buckets (mandatory inclusion): Grade distributions Syllabus of Course offering Student evaluations of the course Frequency that the course is offered (most emphatically) The following were also considered of value, but with less than 100% backing: Past examinations Sample projects Q4: The general consensus was that it was very desirable for archival information to be of a uniform structure. Some concern was the loss of practice in dealing with varying user interfaces. Others pointed out that it was still up to the individual as to whether or not to visit the virtual library outside of the university provided framework. Q5: They were worried about security and choice within the framework of student portfolios. They wanted to be able to manage the contents, and control read/write permissions. Comments about academic diversity were also raised in regard to the faculty/course buckets. One student stated, "I think a 80-20 rule: 80% consistency of content and format, with individual departments and faculty members still being given the freedom to customize not only bucket content, but presentation etc." This comment received wide support. Q6: Individual papers, projects, resume were all considered important. Also faculty recommendations were thought a good idea. Again, concern over control of the portfolio were voiced. Q7: The general consensus was that faculty buckets were a wonderful basis for a research project. They stated that having it would be invaluable in determining the direction of research OUTSIDE of the bucket.