Dr. Dan Russell has been teaching people how to augment their cognition by becoming more effective online researchers for the past decade. In that time, he's taught thousands of people (think students, librarians, professional researchers, and just plain folks) how to find out what they seek through Google (and many other online resources and tools). This talk covers his experiences in learning how to teach these skills, and what he's learned from direct interactions with his students and from various studies he's run in the lab and with live search traffic. He'll discuss my MOOC (PowerSearchingWithGoogle.com), which has had over 4M students, his live classes, and various publications in paper, book, and video formats. He can tell you which methods work best, why, and how it changes the way people think and answer difficult research questions.
Daniel Russell is Google's Senior Research Scientist for Search Quality and User Happiness in Mountain View. He earned his PhD in computer science, specializing in Artificial Intelligence. These days he realizes that amplifying human intelligence is his real passion. His day job is understanding how people search for information, and the ways they come to learn about the world through Google. Dan's current research is to understand how human intelligence and artificial intelligence can work together to better than either as a solo intelligence. His 20% job is teaching the world to search more effectively. His MOOC, PowerSearchingWithGoogle.com, is currently hosting over 3,000 learners / week in the course. In the past 3 years, 4 million students have attended his online search classes, augmenting their intelligence with AI. His instructional YouTube videos have a cumulative runtime of over 350 years (24 hours/day; 7 days/week; 365 weeks/year). His new book, The Joy of Search, tells intriguing stories of how to be an effective searcher by going from a curious question to a reliable answer, showing how to do online research with skill and accuracy. (MIT Press)
Questions and Answers from Dan Russell’s talk
The Joy of Search: Augmenting Intelligence by Teaching People How to Search
These are questions that were asked at my talk. Hope you find the answers to be useful!
Q: How many results does Google supply to each query? Is there a mathematical/strategic place to stop viewing the results?
A: In general Google can provide up to 1000 results / query (that’s 100 pages of 10 results/page). The “Number of results” that you see at the top of the page are an estimate of the number of total results that could be found at Google. It’s important to realize that while there might be an estimated 3 million results, you can only see the best 1000.
In general, I rarely go beyond page 2 (that is, result #20). Google rank orders results by relevance, and the overall relevance quickly drops off after result 10 (or so). If you’re going beyond result number 20, I recommend reformulating your query. It will be closer to what you really want.
Q: How do you ask a question that has several descriptors of the same thing and many of the descriptors have more than one word. Eg, what is a plant with red thorny canes, no flowers, small green leaves. The google search came back with red roses. I don’t want red to describe the flowers I want red to describe the canes. How do I get red thorny canes to be considered together, and separately from, but in conjunction with no flowers? Sorry for the long example!
A: I’d do something like [ “red canes” “no flowers” ] Using double quotes keeps those words together. Of course, the word “canes” is a very specific term, often used to describe rose stems. So you might try something like this: [ “red branches” “no flowers” -roses ] …. The -roses means to exclude roses from consideration. When I do that, I get this, which might be what you seek:
Q: Is plant search on Google good? That is what I am hoping to gain. I teach plant ID to college students and using Google is an area we cover. Boolean operators are helpful, but any other ways to improve searches for ID purposes would be great!
A: You probably want to also teach how to do “search by image.” Here’s a quick summary on how to do that.
Q: How do you properly use "and" "or" "not", "both" , etc. It seem when I use "and" I get everything that individual has "both" and not thanks that are "red and blue" together. Does that make sense?
A: I don’t teach Boolean because Google doesn’t really use it. In particular, if you use the term AND, you end up searching for “and,” which is super common, so it’s not valuable. Note that all of the terms in a Google query are (by default) ANDed together. The one Boolean operator we support is OR. Example: [ Mary OR Elizabeth Browne poet ] “BOTH” isn’t an operator!
Q: I have struggled with how to help my students understand what kind of source they have found in answer to their search. I asked myself how I learned this and I traced it back to journalism class in high school. Do you have any advice on how to teach this skill in the digital era?
A: The answer is to take a course on fact-checking! This is a great, and huge topic, well-worth teaching in a classroom setting. Here are a couple of places to start: My talk on critical thinking in a post-fact world; Daniel Levitin on “How to think critically in the post-truth era” -- there are a few such courses online.
Q: When you type in a search, does the order of the words matter?
A: Yes! Compare [ dog chow ] and [ chow dog ] -- generally speaking, put words in the sequence in which they appear in real life.
Q: If things change/improve over time, is there a way to learn what the updates are or is it just repeated trial and error every so often? How do you know when to re-ask?
A: I recommend subscribing to a blog or newsletter to get the latest info on this. Naturally, I recommend mine, SearchResearch. You can sign up for the weekly version of the blog, which save a lot of time. Vsit the Google Group sign-up page and click on the blue button that says "Apply to Join Group" and you'll get the once-weekly emails from SRS.
Q: How do you limit google search to not give information that is not relevant to the question?
A: Basically, by giving Google excellent queries. What you ask determines what you find.
Q: Can he please show how he used Control F? I don't know how he does it.
A: It’s Command+F on a Mac:
And Control-F on a PC:
Q: In terms of finding text on a page - know about CTL F - but it does not always work within all of the "parts" of a web page are there any tricks to FINDING within all the parts, tables, etc. or do you just have to move your cursor to different parts of the page? just curious?
A: Yeah. This has changed over the past few years as web designers have taken to hiding text that’s ON the page, but not visible. This is especially a problem with scrolling interfaces (e.g., Facebook) where you see something, but once it scrolls off the page, it’s unfindable. Sorry… that’s not a Google problem, it’s a webpage design problem! (So yes, you’re right. You’ll have to move your cursor.)
One trick I use (but do this only if you’re comfortable with it) is to look at the web page in “Developer Mode” and do my text find in there. NOTHING is hidden in Developer mode. (You can try this on Chrome by doing View>Developer>View Source. If you don’t like what you see or it’s too scary, just close the window.)
Q: What do you see as the key link of this presentation (and book) with eXtension? Could you say something about how eXtension thinking is connected with “The Joy of Search” and “augmenting intelligence” E.g. what are the limits & possibilities of search (archives) for eXtension?
A: I think of eXtension programs as basically educational--they reach out to folks who are trying to learn about their world. As such, my book teaches people how to find answers for questions they have that aren’t commonly taught in the classroom setting. The skills in The Joy of Search are really about making you, the reader, a better and smarter searcher. These methods augment your intelligence by letting you go figure these things out for yourself.
Q: What are some tools for teaching web and search literacy to under-educated adults?
A: I would recommend my online course, PowerSearchingWithGoogle.com as a great way to start. Of course, there’s my book, The Joy of Search! After that, then you might want to check out other resources.
Q: How do you advise people within specific content areas about how to interact with Google/search? (i.e. Extension clientele/agriculture folks)
A: If you know the language of your speciality area (say, ag), then when you do your search, try to use terms that are specifically for that topic area. Example: You might want to search for terms like silage, or “disease resistance,” or “blight” rather than other, less technical terms. In general, use the terms that people would write in their web pages. Usually, that’s the terminology people in that content area would use.
Q: When trying to find people, why do all those background check things show up?
A: If you’re seeing ads for background checks, that’s what they do, so when you search for a person, they make a bid and (maybe) get an ad showing up. For the non-ad results, it’s because you’re implicitly asking Google to help find this person, and that’s what these services do.
Q: Do you think that the 80% number of student failures would get better if they worked in a group?
A: (This question refers to Sam Wineberg’s study finding that 80% of university students have difficulty in validating a reference they are given.) That’s a great question. It’s not clear to me that it would work, but a great teacher might be able to pull it off. If you try it, and find that it’s working, let me know!
Q: Odd question... but concerns about privacy in asking questions and searching for info - for example on patent searching - is that info available to others or Google who could use that to pursue a patent?
A: Privacy is super important to Google, so we don’t ever disclose the actual queries that people do (except as required by law enforcement). That information isn’t available to anyone. WRT patent searches--we’re especially careful about NOT having that kind of search behavior get out.
Q: Thinking about the terms people use, and how they vary by geography and context...How do you see the future of Google AI impacting small rural towns as broadband slowly spreads?
A: Google already does a lot of special processing of the query based on your location. (Example: A query for [pizza] in rural Montana will not tell you about pizza places in NYC.) In particular, the geolocation is taken into account for many queries in order to make location-based knowledge give the best possible results (e.g., that of a small rural town in the American West or in a small farming village in Croatia).
If you use quotes, doesn’t that mean you want that exact phase. What if the reference says The canes on this plant are red and covered in thorns. Would putting “red cane” in quotes exclude that reference?
Q: Will Command F work on a Mac?
A: Yes! Note that the different browsers (or apps) work slightly differently. CMD+F on Chrome isn’t quite the same as on Safari. You have to pay attention to how each find function works.
Q: Would google ever consider doing a “kids” search capacity for elementary to middle school, for example?
Q: I ALWAYS teach kids to qualify their queries with “For Kids” but this is not something I have seen taught by others. If this was a feature of google, in conjunction with Google Classroom, it might be helpful to narrow down some of the search results to more age appropriate and language appropriate content.
A: It might, but note that what you’re doing is searching for the string “For Kids” in the body of the web page. That often gives you kid-friendly content, but it will push other YA content farther down the page where the kids won’t find it. It’s not a perfect filter.
It’s worth noting that several years ago we DID have a filter that was specifically for different reading levels--Simple, Intermediate, Advanced. Nobody used it, so after a couple of years we discontinued it. (It costs real money to offer a service like that, and if nobody pays any attention, we have to figure out what to do that would be better.
Q: What you said about the percentage of answers being wrong after a certain period of time, is not unlike what happens in the classroom regarding good information.
A: Yes, this does seem to be a common pattern. The longer it takes to find an answer can be a signal that the question is very hard, or that it’s difficult to find high quality content.
Q: How do you determine the order of the words when you do a search?
A: Just put them in the most natural order you can. Usually, this means grouping terms into obvious sequences. For instance, if I was to search for flowers that are yellow and grow in serpentine soils near the San Andreas fault, I’d do this query: [ San Andreas yellow flowers serpentine soil ]. The important word order here is San Andreas, and then yellow flowers, and then serpentine soil. Those bigrams (two words next to each other) matter much more than whether you put serpentine soil before (or after) San Andreas.
Q: Just to reiterate, "AND" doesn't aid searches on Google? For instance yellow flower AND rosette.
A: That’s right. The word “AND” is just another word… and it’s such a common word that it really doesn’t add much to your search.