HathiTrust+Bookworm (HT+BW) visualizes word trends in 13.7 million works held by the HathiTrust. It is currently being developed under a two-year NEH Implementation Grant, but it can already be tried at https://analyticsbookworm.htrc.hathitrustillinois.orgedu/develop/bookworm.
The world's great research libraries have, over time, carefully assembled a rich body of metadata pertaining to the books in their collections. Since the HTRC has access to volume-level metadata as well as volume-level content, we have constructed a Bookworm of the HathiTrust corpus. We felt that setting up Bookworm with a HathiTrust corpus would provides scholarly researchers with the means of exploring trends.
John Unsworth has noted that a fundamental goal of the humanities is appreciation: "by paying attention to an object of interest, we can explore it, find new dimensions within it, notice things about it that have never been noticed before, and increase its value" (2004). Shifting from traditional close reading to a large-scale view of text presents a profound discomfort for humanities scholars, due to the difficulty in retaining the same sensitivity to what is actually contained in the works being studied. HTRC-Bookworm will function as a link between quantitative analysis (distant reading) and close reading. According to Frederick Gibbs and Daniel Cohen, "any robust digital research methodology must allow the scholar to move easily between distant and close reading, between the bird's eye view and the ground level of the texts themselves" (2011). This is what HTRC-Bookworm intends to accomplish (within the limitations of applicable copyright laws.)
Soda vs Pop
Soda or Pop pop? How do Americans in different places refer to their soft drinks? Besides all kinds a variety of scientific papers and journal articles arguing about it, the Pop vs Soda (http://popvssoda.com/) , a web-based project, Soda project plots the regional variations in the use of the terms "Poppop" and "Sodasoda" to describe soft drinks. Current statistics from the project are available here at http://popvssoda.com/statistics/USA.html. Their statistics and mappings are interesting to read. However, what if we want to look back into the history and find the hidden statistics? Where can we get historical evidence for our question? What will the results look like if the statistics are based on authorized publications rather than people's voting online? Try Bookworm! With data extracted from millions of publications from 1940 to 2015, we visualized the” Soda to Pop Ratio” by State state . YThe y-ray axis represents the publication states while Xthe x-ray axis shows the word ratio of “soda” to “Pop”. For example, we can see from the graph that publications in Massachusetts use Soda soda for the soft drinks almost ten times the frequency of using Pop pop.
Soda to Pop Ratio Graph
We can also visualize the Soda-to-Pop ratio with the statistics from the Pop vs Soda Page. Now you can look at a whole picture to find answers to more questions. What are the states where the word “Soda” always dominate in publications? Are publication language sharing the same words preferences for soft drinks as people’s oral language do? Start your exploration with Bookworm!.
Soda to Pop Ratio Graph (Based on the Pop vs Soda Page statistics)
What is the capital city of China? Interestingly, some would say Peking, while the others would say Beijing. Referring to the same city, Beijing is pretty close phonetically to the original Mandarin while Peking has been used for a longer time internationally. Some findings argue that the Chinese government is insisting insistent on the more modern transliteration Beijing rather than Peking. What’s more, they claim that with China’s rapid development and increasing power, the trend of replacing Peking with Beijing grows. To further investigate this argument, we used Bookworm to find out the the word usage in publications of six countries from the 1960s to 2010s. Then we generated a graph showing the log ratio of Peking to Beijing grouped by Country. YThe y-ray axis marks the publication country while Xx-ray axis shows the time of the publications. Blocks of different colors indicate different ranges of the ratio. Click on a block and you will find a list of related publications during a certain period in the country you pick. Try different settings and input various words, you will find more!
Word Popularity Lines of Inuit VS Eskimo in United States and Canada from 1981 to 2010
The word count lines of Canada showing Canadian publications is also dramatic, challenging the “common opinions” on the Inuit vs or Eskimo issue. As we zoom in, we can tell that for some reasons, the use of Inuit surged from 1968-1984 while the use of Eskimo dropped quickly at the same time. Why? What caused such changeover? Was it the same case in Canadian people‘s oral speaking? With such findings, we can raise more questions against the previous arguments and start our own research with new evidence.
Bookworm gives users a simple visualization platform that allow people to understand a large scale of data over many years in seconds. To make full use of this feature, we visualize the popularity of 6 single years: 1950, 1960, 1970, 1980, 1990, and 2000 with Bookworm to understand the varying popularity of these years. Learned from Bookworm, publications mentioned each of these years most frequently in 1 to 3 years after them. For example, the popularity of 1950 peaked in 1952. Once the popularity reached its peak, it kept delinking shrinking in the following years.
Popularity of Years: 1950, 1960, 1970, 1980, 1990, 2000 (smoothing: 0 years）
From this visual, we can see more clearly what the popularity of a single year is like. The popularity of year A always rockets up in 1-3 years after A itself. Then it decreases progressively. Decreasing rate is high at first and becomes lower laterthen tails off. Finally, the popularity maintains at a low level. Is this pattern universal? Does it reflect people’s memory and oblivion of history?
Currently, HT+BW supports one-word (unigram) entries as search terms, entered via the search field:
The facet fields display the options for faceted search. Initially, clicking "All texts" will pop up a menu that displays the possible facet fields. After a selection is made, you can modify it again by clicking on that label. Initially, on clicking "All" or inside this box, a popup menu displays a scrolling list of values that can be selected for the given facet. Multiple values for a given facet field can be chosen by clicking inside this box. Multiple values in a facet field constitute a logical OR for search purposes. So, for instance, in the example below, the search will be for "Genre:Biography OR Poetry". Different facet fields are in an implicit AND relationship with each other. In the figure below, three facet fields have been specified. The search criteria specified in this example translate into "(Class:unknown) AND (Genre: Biography OR Poetry) AND (Format: Book)."
Clicking on the Settings button, which looks like a gear, will display a menu for changing graph options to specify Time, Metric, Case and Smoothing.
Time range can be adjusted by dragging the "Time" slider for begin point and end point to the left and right as necessary.