Archive
This is a read-only archive of the BHL Staff Wiki as it appeared on Sept 21, 2018. This archive is searchable using the search box on the left, but the search may be limited in the results it can provide.
ErinThomas Jan 13, 2010Ah, now the particulars. not only do we need to decide what to ask, but HOW to ask it. off the bat, I'd like to say that it's a given that the particular phrasing of each question will have some participants who it doesn't quite make sense for. People are smart tho' (probably i won't ever say that again..lol) and they can figure out the info we want. There is no universal language for "what do you do?" Obviously, some folks are using BHL for their paid professions and some may be amatuer scientists..Keeping the language as simple yet DIRECT as possible sounds like a good way to go...even though there may be a way to phrase the question so it makes sense for a the broadest set of circumstances, let's defer to customary figures of speech. yeah? ErinThomas Jan 13, 2010this, as a "section" seems kinda problematic. What do we want to know besides profession? social demographics = unneccesary
dukeg1 Jan 14, 2010I believe that this section will probably be short, yes? We need simply to ask what they do, basically (like, are you a librarian, teacher, scientist, etc.). I wonder if it would be a good idea to ask about whether they use BHL for professional (like Erin said, as professional scientists) purposes, or whether they use it for "amateur" purposes, like citizen scientists. Does anyone think this would be a worthwhile question? Not really sure I know what we could get out of a question like that, and not really sure if we even care. Just thought I'd throw it out there. ErinThomas Jan 19, 2010I think it doesn't matter if its pro or amatuer. they need the same standard of quality access regardless...so, this is the complication, from my brainstems. this seems to be a one question "section".
thompsonkeri Jan 19, 2010 I think this is a 2 question section: Q1 is the what do you do, Q2 is how often do you use BHL portal
lipscombb Jan 19, 2010 Question suggestions: -- Have you used BHL before? -- How satisfied are you with BHL's services (1-5 scale)? -- Do you prefer Google to BHL (simple yes/no and if no then why)? -- Do you have regular access to these types of materials (define materials more exactly) in your local library? (-- What are you wearing? j/k!)
PilskThere is some new term besides "citizen scientist" but I can't remember what it is. 1) Check most closely identifies your work: Scientist (taxonomist) Scientist (other) Librarian (reference) Librarian (other) 2) Are you using BHL for work? for fun? for potential creative production?
matt707 Jan 20, 2010I agree -whether it be Citizen Scientist or a similar term, we need to reach out to general browsers so that we do not lose someone who is "Just surfing the web" , because our casual users help define us as a a physical library place in addition to a taxonomic data set. Browsing is what people complain does not happen in an online library, but if we can consider that as part of this library it would be positive. lipscombb Jan 20, 2010 Tom mentioned he was particularly interested in getting feedback from non-taxonomists, i.e. ecologists, evolutionary biologists, etc. thompsonkeri Jan 19, 2010Should we take these directly from the Dev. Priority document, e.g., ask at least one question about each priority, and/or have respondents rank our current dev projects? I think that could be our starting point, though I might leave off a question about digital preservation & Fedora. Also, I don't understand priority 5 - can someone explain that to me?? PilskBut we need to translate into Common Language thompsonkeri Jan 19, 2010 taken from the BHL-E survey Q1/2 and Q5/6, with slight modifications. Also, I think we need an open ended question about what services we aren't providing that they want.
h-scholz Jan 29, 2010 Francisco and myself have studied your questions here Chalkboard for BHL-US Survey. You changed a lot compared to our questionnaire. Comments of Francisco:
fwelter Feb 17, 2010 Default of scrollboxes shall always be "Please select one"| DRAFT Questions |
Suggestions |
Revised Qs |
|---|---|---|
| General Info |
- fwelter Feb 17, 2010 date of my Q comments was always 04 Feb 2010, not 18 Feb (the others were Henning's comments) |
|
| 1. Have you visited the Biodiversity Heritage Library before? [yes/no] IF YES THEN ALSO GIVE THEM Q3 |
- fwelter Feb 17, 2010 Q1: I don't see the sense in this question. BHL is known in the scientific world since several years ago. Is the survey not to improve the service for those who are using it? |
AGREED. Q1 Eliminated |
2. In general, how satisfied are you with the Biodiversity Heritage Library: [choose very sat/somewhat sat/neither sat nor dis/somewhat dis/very dis]
|
fwelter Feb 17, 2010 Q2: keep attention not to use terms or abbreviations that non-insiders don't understand. Offer a 6th option to answer "I don't know/I don't understand"."selection of publication, quality of scans" - should not appear lumped but with separated options to answer both items individually - fwelter Feb 17, 2010 instead of "poor" perhaps "not" or "insuffucient"? Poor sounds too good for zero agreement. Are these school grades in English countries? The lowest school grade in Germany translates "insufficient". - lipscombb Feb 18, 2010 "poor" is standard language, but I agree and made some changes.- lipscombb Feb 18, 2010 "APIs" is an insider term but appropriate and meaningful to those who know how to use our API services as explained here.See also my comments in Additional Considerations - fwelter Feb 17, 2010 link inappropriate(- lipscombb Feb 18, 2010 sorry, link updated)- fwelter Feb 18, 2010 still does not work, links invariably toBHL+Survey+2010#considerations which then goes to the headline Alignment of BHL-US & BHL-E Survey Goals- lipscombb Feb 19, 2010 ok i moved the anchor to just above my comments, which i touched on during our conversation yesterday.- fwelter Feb 20, 2010 all right, got it. Yes, that's the sperm whale example. Chris had given us a lecture about these new tools in one of the last meetings, I think it was in Prague in November. |
2. In general, how satisfied are you with each of the following components of the Biodiversity Heritage Library: [excellent, above avg, avg, below avg, very poor, no opinion/don't understand]
|
| 2.2 Please give us your comments about your satisfaction with the Biodiversity Heritage Library. |
- fwelter Feb 17, 2010 Q2.2: offer only one free text option in the whole survey. In the BHL-E survey we offered several text fields and people repeated the same ideas and arguments again and again in each free text field. - fwelter Feb 17, 2010 You also should offer this free text option at the end of the survey, not at the beginning. In the BHL-E survey they filled in detailed comments in free text fields right at the beginning and later saw that all these things were asked in Q8 or Q10. |
MOVE TO END 11. Please provide any additional comments you may have about the Biodiversity Heritage Library. - fwelter Feb 17, 2010 yes, good, that will be fine |
3. How often do you use the Biodiversity Heritage Library (http://www.biodiversitylibrary.org)?
Derived From: 11. How frequently would you visit the BHL-Europe Portal?
|
fwelter Feb 4, 2010 No, not too many options. This was the best question we had in the whole survey, and the only one I would not greatly modify. It gave exact figures and the result was excellent. US survey propsal Q3 is not so good because it is too much open for different interpretations ("weekly" and "1 single time per week" are not the same, if my English is sufficient to judge this). Giving a lower number of options does not necessarily reduce the time the participant needs for filling out the survey. The last 2 options should be lumped to "less frequently" (having asked these two separately was not harmful, but not necessary either), leaving us with 5 instead of 6 options. These 5 options gave a perfect Gaussian distribution of results!- fwelter Feb 17, 2010 In BHL-E we had asked "...would you visit the portal", while you have proposed to ask "...do you use the Biodiversity Heritage Library". There is a difference. Many users come in from archive.org, google, animalbase or elsewhere and don't use the portal, but they do use BHL. We have to discuss this, and the question had to make absolutely clear what is asked. Eventually we could ask 2 separate questions "how often do you work online with literature digitized by BHL" and "How often do you visit the BHL portal".- fwelter Feb 17, 2010 Q3: That was one of our best questions. Not too many options. That was really good to evaluate. You must explain exactly what you mean. If you use BHL 3 times a week - is this "daily" or "weekly"? You must avoid misunderstandings. |
AGREED. MAKE #1 1. How often do you use the Biodiversity Heritage Library (http://www.biodiversitylibrary.org)?
Need clarification on options. Having an option for a first time user would be useful. Option for "never" may not apply within targeted user groups. - fwelter Feb 17, 2010 options 1-4 fine, option 5 perhaps just say "less frequently", this would also include first time. I don't think "never" would be needed. Not a significant proportion of people would take the time to fill in a survey if they would never use this BHL service. Or do you think so? |
| Services and Use |
||
4. How often do you use the Biodiversity Heritage Library for the following? [select - often/sometimes/rarely/never ]
|
fwelter Feb 17, 2010 Q4 - as Henning said, give 5 options instead of 4. And add a 6th option for those who did not understand what was meant."interdisciplinary research" - too short and unclear what this means. Could be anything. You must explain more exactly what you mean. Yes, of course, this is not always easy. But if some participants misunderstand your questions, the results will be useless, or very difficult to interprete. And always keep in mind, the participants are not all insiders! same with "data mining / data reuse", "research species publication history", "citation/literature searches" and others. Be more precise. It will be much easier to answer questions if the participant will understand what exactly is meant. The sentences will be longer, but the time the participant will need, will be shorter. "verify species information" I want to verify statements on the morphology of a species. I want to verify if the correct spelling of the name of a species. I want to find information on the morphology of a species. "verify bibliographic citation information" - specify more exactly what you mean. We did not do this in the BHL-E survey and this was a big problem when I had to evaluate that survey. The term "citation" is used in various different senses in various disciplines. |
fwelter Feb 18, 2010 Wow, excellent help this one! Wish they had this in Spanish too.4. How often do you use the Biodiversity Heritage Library for the following? [almost always/often/sometimes/rarely/never/(I don't know, or don't understand)] - fwelter Feb 17, 2010 5-point scale options are fine
Need help clarifying language - fwelter Feb 17, 2010 Frank Wieland and I must understand as exactly as possible what shall be asked and then we can translate this into bioscientists' language |
5. How do you most commonly search for information in the Biodiversity Heritage Library? [choose multiple]
|
- fwelter Feb 17, 2010 Q5: not "choose multiple". Give options 1-5 also here. You will see that participants will rate their answers in a way you would possibly not expect. If you give only one option "yes" or "no", they will all click "yes", but some will select "no" if they think the other options are more important. But the whole question is unclear to me. I would be one of those participants who would take long thinking about "what do they mean with this question"? Answer 5.6 "Google" suggests that the search page in BHL is NOT meant? You must explain precisely what you like to know, in an easy language. Seems that "links from archive.org" should also be mentioned? "EOL" - not known to all |
Upon review, what we are actually trying to ask here is: 5. (When conducting your research) How do you find/come upon Biodiversity Heritage Library content?
fwelter Feb 17, 2010 my proposals for changes in bold.The archive.org option is separated out to see specifically if people use IA (but doubtful). - fwelter Feb 17, 2010 yes, good.I don't understand the last point, is the BHL blog not a BHL website and as such, not also BHL content? |
6.What feature(s) that are currently available in the Biodiversity Heritage Library do you find useful? [very useful/somewhat useful/not very useful/not at all useful not applicable/haven't used before]
|
fwelter Feb 4, 2010 Q6: auto-generated species bibliographies – I’m not 100% sure what it is, so it needs explanation for the user to give the right answer- fwelter Feb 4, 2010 The situation has changed since Nov 2009, browsing pages online is really fast now. The background for several points in this question has changed.It might be interesting to see how frequently and in which situations people download items, and why. Downloading has advantages (you can read the book when you are not online) - and shortcomings (you have to establish a system where to find your book again on your harddisk). An important point is that users seem to consult the online presentation to verify if it is the correct book they were looking for - if yes they start downloading. An important point for downloading PDFs is lack of confidence: nobody trusts BHL or any other provider in the internet to provide the same free service, stable URL and powerful search function in 5 or 10 years. If you talk to scientists and ask them, why do you download all this literature? instead of rwading it online or copy the URL, they answer "today it's free - who knows how long it will stay free?", and if you explain that BHL and European libraries are public services and that the items will stay free and online with stable URLs they smile... What I really would like to know is the development of usage preferences in the time line. Will users learn that BHL - in contrast to Google and almost all other internet providers - is trustworthy? How many read books online/download PDFs today, and to which extent would the proportions change in a few years? - fwelter Feb 17, 2010 Q6: As said above, give 5 true options and a 6th option "haven't used before". The middle option is important, I would use "moderately useful" as the middle option and replace "somewhat useful" by a term that lies more precisely between moderately and totally useful. This is important because you eventually want to evaluate the answers in a numeric way."open access to data, e.g., species names, URLs and citations" - not a good question. The term "citation" is likely to be misunderstood because scientists use it in a different sense than librarians. For a scientist the term "citation" as used here gives no sense. "stable URLs for linking to books, pages/articles, authors, subjects" - separate the points, and separate also pages and articles. APIs - insider term |
AGREED. Use 5 option scale. 6. What feature(s) that are currently available in the Biodiversity Heritage Library do you find useful? [very useful/quite useful/somewhat or moderately useful/slightly useful/not at all useful/not applicable-haven't used before] - fwelter Feb 17, 2010 options are fineBut conflict with Q2, should either be moved in one, or follow one another.
Need help making language more specific - fwelter Feb 19, 2010 In which extent and why do users download PDF files? |
| Development Priorities |
||
7. Help us plan the future of the Biodiversity Heritage Library! Please select your TOP 5 new features from the list below:
|
The wording should be very clear to allow non specialist to answer the questions properly (e.g. Q 4 and 7) - fwelter Feb 17, 2010 Q7: "Help us plan the future of the Biodiversity Heritage Library" - this comes late. As a participant I would have suspected from the beginning on that BHL has not designed this survey to plan the past...I would prefer to ask questions on the same topic one following the other directly. First block, how are you using it now, immediately next question, how would you like it to have. The way you have it now is not user-friendly. Participants will have to read an ugly long text (moreover, in an extremely difficult language), and after that select 5 features. If they like 7 features? Or 3? The survey must be designed in a way that the participant does not loose much time in thinking about the correct answers. "ability for users to submit and download journal articles" - insider knowledge needed to understand what is meant. "ability for the Biodiversity Heritage Library to index and present journal articles" - same. unclear to me what "to index" means. "data mining" - insider term, has no meaning to most scientists, less to non-English natives "improved search across all books, journals and articles" - this can mean anything. Be more precise. "faster downloads / smaller pdfs" - this is good! Everyone will understand this. And everyone will click on it because it is one of the few points they can actually understand "ability for users to download a single or selected page images" - good point, but I don't see the difference between "single" and "selected" "empower Biodiversity Heritage Library users to improve book information, e.g., correct OCR" - ask more precisely what you mean. Is "corrrect" used as a verb or as an adjective here? |
7. Help us prioritize developments for the future of the Biodiversity Heritage Library. Please rate importance of each new feature from the list below [6 opt. scale]: - fwelter Feb 18, 2010 Or 7 options:[Very important/Quite important/Fairly important/Slightly important/Not at all important/I have no opinion/I don't understand] We have to keep in mind: can we use this question also in 15 months? Place for this question perhaps immediately after Q2 (my bold comment in the middle row of the table)
fwelter Feb 18, 2010 Language should be included somewhere here:
|
8. How would you prefer to search the Biodiversity Heritage Library? [choose multiple]
|
fwelter Feb 17, 2010 Q8: Choose multiple is a bad move here. We experienced that in such a situation they just selected everything. If you ask them "would you like to have this?" they always like to have everything. Especially the Spanish participants. What you want to know is more about which one should be the default functions, and which features you should shift to the Advanced mode. For this purpose you need reliably and numerically evaluatable answers. (If evaluatable is an English word...)"Boolean terms" - insider term. "relevancy ranked keyword ("Google-like")" - good. Google-like is a very good term, everyone will understand what is meant. The shortcoming is that this (best understanding) will be a reason for this point to be ranked high. We had this term in the BHL-E survey, but I don't think I would like to repeat this next time, at least not without caution. If you tell them that if they search fro "Linnaeus Systema Naturae" and use a Google-like search function they will not get 10 but get 10,000 results, then they might not necessarily select "Google-like" as their favourite. "book or article title" - separate the points. Book title, article title, journal title. "book or article author" - author of a publication "searching full-text (search inside the book)" - good, can easily be understood. But will the readers also understand that they will get millions of results then? "species scientific names" - scientific names for species and genera "species common names" - common names for animals and plants "other" - as said above, avoid free text |
8. How would you prefer to search the Biodiversity Heritage Library? [choose top 5, use check boxes - how would this work in survey monkey?] - fwelter Feb 18, 2010 Perhaps best to use a 5-point + 1 scale and then do the evaluation of the ranking ourselves? And put the items in a form like "I would like to use this function" [very often, fairly often, sometimes, rarely, never, no opinion]
|
9. How would you prefer to sort your search results? [choose multiple]
|
- fwelter Feb 4, 2010 This was a very important question. As a scientist I am used to literature lists being sorted by author and year. Only a few US journals work with endnotes and footnotes to cite literature sources. Most international journals, in any language, have their literature lists at the end of a paper always sorted by author and year. This is a standard widely used for far more than 100 years, it was established in the mid-1700s. The results strongly suggested that the participants of the survey answered in accordance with current practice in scientific research. I had expected this, but not that the answer came in so brilliantly clearly. It is not sufficient to provide functionalities. If you want to provide a successful service it is important to have selected the most requested functionality as the default one. - fwelter Feb 17, 2010 Q9: extremely important question, but also here: offer 5 options and a 6th for a neutral answer"Reverse sort order (A-Z vs. Z-A)" - is misunderstood. We also had this, but I doubt they understood. You speak better English than we do - it must be possible to express more preceisely what is meant here. What you seem to mean is that the user can have an option to select individually in a search to show the results in a reverse order, not as the default way to sort all results generally. "subject/keyword" - can be anything, provides a wide range for misunderstandings |
9. What is your preferred default sort order? / How would you prefer to sort your search results? [choose multiple] Rank each...6 options - fwelter Feb 18, 2010 [Excellent choice, good choice, I don't care, no good choice, worst choice, no opinion]
|
| User Profile |
||
10. In the context of your research needs, what best describes your profession? [choose one]
|
fwelter Feb 17, 2010 Q10: not choose one, should be choose multiple. I can be a teacher/professor/educator and at the same time a researcher, a taxonomist and a scientist...The answers suggest that taxonomists are not scientists in the proper sense... "10.1 What is your field/discipline? [fill in blank]" - no blank field here, you will not be able to evaluate the results, or need many hours to do that. The question is necessary, but the answers too. Give a scrollbox, and let the audience select one term of a scrollbox. Otherwise you will get 100 answers from 100 participants and will stay with the work to sort out which family or genus name belongs to which plant or animal group... and believe me, you will get many answers like "Hesperiidae" or "Caryophyllacea" if you ask scientists about their discipline! In BHL-E we will certainly add some more questions here that BHL-US might not need: language will be important (native languages, preferred and used languages for the search options, languages of the consulted literature), I would also like to know the regions where the participants come from. |
- fwelter Feb 17, 2010 10. In the context of your research needs, what best describes your profession? [checkboxes... or not? I am not sure]
scrollbox 1 for bioscientists: Fishes Birds Other vertebrates Molluscs Coleopteran insects Dipteran insects Hymenopteran insects Lepidopteran insects Other insects Other arthropods Other animals Angiosperms Other higher plants Algae, Lichen, Fungi Bacteria, Archaea Other/various groups I have no special group checkboxes 2: taxonomy/systematics/nomenclature morphology physiology ecology nature protection evolutionary biology paleontology, geology history of science other
China, Korea, Japan Pakistan to Vietnam and Indonesia Turkey and Near East to Iran Russia and central Asia E and SE Europe (Slavic languages) N Europe (Scandinavian languages) British Isles Central Europe with France S Europe (Roman and other languages) Africa USA, Canada South America to Mexico, Caribbean
English French Spanish/Portuguese German Italian Russian Other Slavic Swedish, Danish etc. Dutch, Flamish Other European Chinese Japanese Other East Asian Arabic, Farsi, Hebrew etc. Turkish and central Asian Hindi and other south Asian Other
- lipscombb Feb 18, 2010 There has always been an anecdotal reference to "science historians" as being a particular BHL demographic and it would be useful to see if anyone would happen to select this. I have been contacted by a few people doing this kind of research, however, they could simply consider themselves under the student/scholar category but I'd prefer to separate them.- lipscombb Feb 18, 2010 Tom G. has asked that we try to see if BHL is being used by ecologists, evolutionary biologists, etc. and I'd like to discuss how best we can work this in. |
fwelter Feb 4, 2010 I have tried to summarize several kinds of topics for survey questions. h-scholz Feb 9, 2010 Some thoughts from my side here. The main goal of this first large scale BHL-E survey is the analysis of demand and service elements to identify the key components of BHL that support user needs – this is what our contract says. That means we have to find out how the users like and use the existing BHL Portal, what features they like, what features they not like. A wishlist based on use cases not realised so far would be also important. Is there a difference in their current use and their anticipated use? This will set the development priorities for our development team working on the German language prototype of BHL-Europe. For the questions it means that there must be a relation between the feature the user is working with and the technology behind. Just a simple example: If the user say “I prefer sort my search results alphabetically by publication author”, it needs to be clear for the development team how to implement that. fwelter Feb 9, 2010 6. User profile. h-scholz Feb 9, 2010 The questions currently are sometimes to generic and the language is more suitable for experts then for other interested users. However, it is important to not only involve the very active users (because they probably like BHL anyway). We need to find the sceptical users or new non-expert users as targets to get a fresh look. fwelter Feb 9, 2010 Good points. With options 1-5. Might be good to ask in addition a likewise question about the topics, langauge and time of the works the participants do currently read in BHL. lipscombb Feb 18, 2010 After discussion with Chris Freeland, we toyed around with the idea that at some point we would like to issue a Tech survey that would reach out to users who rely on BHL's APIs and Taxon name finding services to enhance their own data repositories. The stable URLs, while useful for librarians to some extent, and APIs that we offer are largely intended for use by repositories that want to link to any string in BHL (incl. taxa, subjects, authors, individual pages, etc.) and return results within the context of their own websites. IPNI would be an example of this kind of repository. There are other users that fall between the "repository user" and standard users (researchers, librarians, etc.) which are users that take BHL data en mass and re-purpose it into novel applications that display the data in new ways that are unique to BHL's current capabilities. One such example of this is through the work of Rod Page, see http://iphylo.blogspot.com/2009/10/biodiversity-heritage-library.html dukeg1 Jan 14, 2010This is basically a question of, "what do you use the BHL portal for," yes? I feel like this question is very leading, and if we keep it in any form, we need to modify how it is asked, or at least provide a text box for them to fill in their own answer. However, I'm not convinced this is the best way to ask this question to get the information we want. Won't, in the course of asking them what they use the BHL portal for, we'll come across the information that this question attempts to collect? lipscombb Jan 19, 2010 This question is very specific to taxonomists. We should include other options that are more general/cater to librarians. I think the point of this question is to get at the concepts behind people's use of BHL, i.e. people's research interests...perhaps we could add some points that differentiate between known-item searching and browsing actions but perhaps this is more in line with Q2, an information request, rather than a concept. Could also be transformed into a ranking Q "What research inquiries are you most likely to pursue in BHL, rank in order of... matt707 Jan 19, 2010 "Why are you performing a search in the Biodiversity Heritage Library?" dukeg1 Jan 20, 2010I still see a potential problem with giving users specific options to choose from, as though we assume we know all the uses users might have for BHL. I think this is leading and could potentially result in us missing uses and needs from users that we didn't even know existed. If we keep this question in the format of choices, I think we at least need to let them have an "other" box to provide open-ended comments. Pilsk "Why are you performing a search in the Biodiversity Heritage Library?" With choices that include: Looking for a book. Looking for a article. Looking for an species. Looking for anything written by a specific person. Looking for anything writting on my general interests. Other please tell us. lipscombb Jan 19, 2010] lipscombb Jan 19, 2010 lipscombb Jan 19, 2010 lipscombb Jan 19, 2010 dukeg1 Jan 14, 2010 I don't have a problem with this question - I think it would be interesting to know if more and more people are looking for illustrations. If we find this to be the case, it could influence development in terms of prioritizing some sort of mass pagniation interface that would at least allow users to tag illustrations. This question definitely needs to have an "other" box for users to enter more information/other options into. thompsonkeri Jan 19, 2010 In a way, I think this question is giving us the same information as Question 1, i.e., what do you use BHL Portal for, but is phrased slightly better,, and is more general. I think modifying this one slightly to make sure it is complete will obviate the need for asking Q1. matt707 Jan 19, 2010 "What information are you seeking by performing a search in the Biodiversity Heritage Library?" Pilsk If I was coming to read an article or to read book chapter or to read the original Linneaus writing, what would I check? lipscombb Jan 19, 2010) (- fwelter Feb 4, 2010 the background for this question had been that a digitized BHL page took average 30 seconds to load in and that browsing/reading a book online was almost impossible - this was substantially improved in November 2009) - lipscombb Feb 16, 2010 My (very unnecessary) sarcasm is b/c I took this question to mean that you could easily browse the collection online, which I still don't think one can. The only browsing options are alpha by title - not so robust. lipscombb Jan 19, 2010 lipscombb Jan 19, 2010 lipscombb Jan 19, 2010 dukeg1 Jan 14, 2010 I feel like this question isn't really asking what they're going to do with them, but how they're going to access/keep them. And the options of "looking for artwork" and "looking for taxanomic names" seems redundant with the above question of why they're trying to access information. I feel like, if we're asking what they're going to do with the information they get, it should be more along the lines of "for writing a paper," or "furthering research" or something. I feel like, with these answers, the question should be more like, "how will you interact with the information once you find them" or "how will you gather information from what you access" - print, read online, browse it, link from website, etc. lipscombb Jan 19, 2010 I agree w/ Grace re: redundancy. thompsonkeri Jan 19, 2010 ditto. If we deem this is useful at all, the question should be rephrased to answer the question of how they are using the information we provide access to. matt707 Jan 19, 2010 "In what way(s) do you intend to use the information you discover as a result of the search(s) you have performed?" dukeg1 Jan 20, 2010I really like the way Matt phrased this question. If we keep it at all, we should phrase it along these lines, I think. Pilsk I think it would be good to know if they plan on reading online, downloading, creating a link to a page or title or something. Wouldn't it? lipscombb Jan 20, 2010 Everyone wants PDFs that don't take forever to download. period. No really, we should look at this question next to Q8 and decide how we want to approach it. Sounds like there are 2 different things going on: Use, as Matt points out, and Access, as Suz points out. fwelter Feb 4, 2010 The situation has changed since Nov 2009, browsing pages online is really fast now. The background for several points in this question has changed. dukeg1 Jan 14, 2010 I don't feel like this question would give us anymore information than what we got from question 2. It's basically the same thing - what kind of information are you looking for? I don't think there's anything we can get out of this that we can't get out of 2. Maybe the wording of this question gets the point across better than the wording in question 2, though. PilskWhat do you think they thought they were going to get from this question? I'm confused! lipscombb Jan 20, 2010 This question did make it brutally obvious that users are really interested in full text (84.3%) and plates (70.6%). I think this sort of question is important for development priorities and QA = prioritize making full text searching available and prioritize making plates more discoverable for users, i.e. better pagination tools (and better quality but well...what can we do...). To elminiate this question requires that we work these concepts into some other question. Perhaps with a wider pool of participants plates aren't as important? fwelter Feb 4, 2010 This question was certainly not extremely useful. It was only asked "text", not (OCR'd) full text, and it was totally unclear what "text" should have meant (because it seemed to have stood in contrast to "abstract/conclusion"). Terms like "citations" and "indexes" were not understood. It was not at all clear what had actually been asked in this whole question. An outstanding example for the need to ask exact questions. dukeg1 Jan 14, 2010This might be useful in terms of guiding development, since I know the development team plans to work on search functionality. If we know how people want to search, we might better be able to determine what development regarding searching the development team should focus on. matt707 Jan 19, 2010 "Do the BHL searching functionalities fulfill your research needs? What other search functionality(s) would assist you with your research?" ...5 & 6 could be combined. dukeg1 Jan 20, 2010I agree with Matt - combine 5 & 6 and phrase as Matt suggested. lipscombb Jan 20, 2010 I think Matt should draft all our questions :) dukeg1 Jan 14, 2010Again, I think this might be useful for guiding development in terms of improving searching. thompsonkeri Jan 19, 2010 perhaps combine Q5 and Q6 to answer the search functionality development question. dukeg1 Jan 20, 2010agree! PilskAgree but wording needs to be non-librarianish. Nix the Metadata Field phrase! lipscombb Jan 20, 2010 dukeg1 Jan 14, 2010 we already have the functionality to sort results according to name, title, year, and subject. I'm not sure we're really looking to change anything that might come as a result of answers to this question, and I'm not sure answers to this question would result in any new information. thompsonkeri Jan 19, 2010 do we care about this? PilskI'm not caring unless we were going to offer something different than the above. Geographically? Color of binding? Location of physical book? or something odd. lipscombb Jan 20, 2010 I think we do care about this (perhaps this is a good thing to look into for CiteBank). This question revealed that people were most interested in having search results sorted by author, then year, then title. Our default sort is by title. Plus, you can't search by a title and have the results sort in order by author or some other combination - something that you can do in a library catalog, so I think this question is important for functionality. It would also be good to check in on the subject of relevance again -- we hear from EOL users that they want relevancy ranked titles that link into EOL pages, but the BHL-E survey showed relevance as the lowest - only 16/50 thought it was "very important" fwelter Feb 4, 2010 This was a very important question. As a scientist I am used to literature lists being sorted by author and year. Only a few US journals work with endnotes and footnotes to cite literature sources. Most international journals, in any language, have their literature lists at the end of a paper always sorted by author and year. This is a standard widely used for far more than 100 years, it was established in the mid-1700s. The results strongly suggested that the participants of the survey answered in accordance with current practice in scientific research. I had expected this, but not that the answer came in so brilliantly clearly. It is not sufficient to provide functionalities. If you want to provide a successful service it is important to have selected the most requested functionality as the default one. dukeg1 Jan 14, 2010Users already have the option to choose any of these different options. I feel like the only pertinent information we could get out of this question relates to how they most often access the content (with text, OCR download, PDF, etc.) and what they are looking for (if they want high resolution for download/print out, they probably are looking for illustrations). Earlier questions already asked how they want to access the content and what kind of content they are looking for. If they are looking for images, I think it stands to reason that they want high resolution print outs of those images. Thus, I don't think we can glean any new information from the results we would get from this question that we haven't already gleaned from earlier questions. matt707 Jan 19, 2010 "Do the search results and files you have used on the BHL website meet your technical needs?" dukeg1 Jan 20, 2010If we ask the question this way, I think we need to also ask, why or why not and leave it open ended so users can provide specific information. I know surveys I've done before where I just asked a yes or no question like this really didn't provide me with any additional information because, if they said no, I didn't know why it wasn't meeting their needs and what I could do to make it better. PilskDisplay and Download or two different needs. I want it to display to recognize and download high quality. OR I might want to display high quality for some online mash up and have a quick and dirty download for referencing or something. Where is the request for thumbnails? Is that somewhere else? lipscombb Jan 20, 2010 we need to ask if people still want to be able to download a single image (which they can't do anymore w/ the flippy book) or download all images at a high resolution. Thumbnails is a great question AND a sticky issue like the create-your-own PDF which both equate to us hosting our own images. These should be asked, I think, to make such things a priority w/ management. I know this isn't news to anyone that BHL needs these things but it's always good to point to user responses that echo these ideas. dukeg1 Jan 14, 2010We already have stable URLs. Won't the question of "what do you want to do with our content" or something of the like give us this information? For instance, if they want to mine the data for use with other programs, doesn't that tell us what additional services we need? I don't feel like we need this question. thompsonkeri Jan 19, 2010 agreed. this q is redundant. - matt707 Jan 19, 2010me too... PilskI think we should only ask for additional services we are already planning on having! HA! But I would like to ask if they want something that pulls together "famous" authors (the ones we have more than one book written by). Or better linking to species or something. lipscombb Jan 20, 2010opportunity to ask about crowd-sourcing services here -- edit the OCR, paginate the pages, approve/deny species name recognition (could get hairy w/ taxonomists but what I mean here is to have people help us differentiate run-of-the-mill Latin words that are recognized as spp. names from actual spp. names) dukeg1 Jan 14, 2010Wouldn't this be on a case to case basis, though, depending on what they want to report at the time? For instance, if they want to report an error, they'll want the bug reporting form, but if they want to submit a request, they'll want the general feedback form, etc. Maybe instead, for our purposes, we want to ask them to rate how we gather feedback and offer suggestions on how we can improve it, not what kind of feedback they think they might want to offer. lipscombb Jan 20, 2010 I think this is a question for a future survey. We're doing what we can with the feedback functionality we currently have. Also, feedback should be a single stream in my opinion - not different options for bug reporting vs. scan quality vs. general feedback vs. metadata correction. Giving people the option to track our progress fixing the error is opening up a HUGE can of flesh-eating worms! Do other sites offer this? dukeg1 Jan 14, 2010This would be nice to know, and I think maybe should be in the demongraphics section, since it's asking a particular question about the user themselves (though how often you visit the portal isn't really a demographic question...) matt707 Jan 19, 2010 too many options... lipscombb Jan 20, 2010 agreed. too many options and should be something quick to ask in the demongraphics section. fwelter Feb 4, 2010 No, not too many options. This was the best question we had in the whole survey, and the only one I would not greatly modify. It gave exact figures and the result was excellent. US survey propsal Q3 is not so good because it is too much open for different interpretations ("weekly" and "1 single time per week" are not the same, if my English is sufficient to judge this). Giving a lower number of options does not necessarily reduce the time the participant needs for filling out the survey. The last 2 options should be lumped to "less frequently" (having asked these two separately was not harmful, but not necessary either), leaving us with 5 instead of 6 options. These 5 options gave a perfect Gaussian distribution of results! dukeg1 Jan 14, 2010Again, this might be nice to know, but what would we do with this? They can already get RSS feeds, and they can access the portal themselves. So, really, what would we do with the information we get out of this question? thompsonkeri Jan 19, 2010 eliminate this q, and save room for questions that give us answers we need! matt707 Jan 19, 2010phrasing is too euro-lingo for us... "How would you like to receive updates about new BHL developments?" lipscombb Jan 20, 2010 as with the feedback question, this is one for a future survey. I think it is useful to ask using Matt's rephrasing. fwelter Feb 4, 2010 results gave 70 % website, 15 % RSS feeds. I regard this question as useless, should not be repeated. ErinThomas Jan 20, 2010Right. there are questions here that will be useful for us to keep and tweek. But, it also could be that these questions provide a false foundation for us to work from. I'd like to do some reorganizing from that perspective. if its not too late...So. which questions do we keep and tweek?