* Here we introduce the Extracted Features functionality (currently under beta release) that have recently been developed by the HathiTrust Research Center. This functionality is one of the ways in which users of the HathiTrust Research Center can perform non-consumptive analysis of subsets of the HathiTrust Digital Library's corpus that they have custom-selected by means of the workset mechanism available through the HTRC. (Currently, this functionality is available only for the HathiTrust Digital Library's public domain corpus, consisting of slightly less than 5 million volumes.)
This document is a starting point for a user interested in downloading the json-format Extracted Features (EF) data files corresponding to the specific HathiTrust Digital Library volumes that constitute the user's custom workset (that the user has built with the HTRC Workset Builder). We will show, step by step, how you can create a custom workset and then how you can download the data files corresponding to the content of your workset.
2. Create your workset
This section shows you how to create a custom workset, for the volume(s) contained in which you will eventually download the corresponding advanced and basic EF data files. Your workset can contain as many volumes as you wish. However, the example workset for this section will consist, for the sake of simplicity, of a single volume from the HathiTrust Digital Library's public domain collection: a published-in-1920 edition of the book of poems titled Buch der Lieder by the German poet Heinrich Heine. Then we show you how you can download the EF data files corresponding to this workset. (One of the use cases for the EF approach to non-consumptive text analysis that we have posted also uses this particular book by Heine to make its point.)
2.1 Sign in to the HTRC
Navigate to “Create Virtual Machine” tab and fill in the form. You need to choose an image from the drop down list, provide username and password for the VNC session, and choose how many CPU and memory you want your virtual machine has. Finally you hit the "Create VM" button. The VM creation procedure usually takes about 1 minute to finish.
2.2 Show a Virtual Machine Status
Navigate to “Virtual Machines” page, you can see all the VMs and available operations associated with the VM.
You can click on the vm id link to see more details about the VM. The “VM Initial Logging User ID” and “VM Initial Logging Password” are the username and password you use to log into the VM. These are different from the ones you use to open your VNC session to the VM. The “Public IP” and “VNC port” are information you need to open a VNC session. You can also use “Public IP” and “SSH port” to log into VM through ssh but this is only allowed in maintenance mode.
2.3 Start a Virtual Machine
You can start a virtual machine in the “Virtual Machine” page. This operation usually takes 2 ~ 3 minutes. Once the VM starts successfully, you can see more available operations e.g., switch, stop, and delete.
2.4 Log into a Virtual Machine
You can use your favorite VNC client, e.g. Google VNC Client, to connect to the VM by providing the VNC password and VM login username as well as login password.
2.5 Switch Virtual Machine Mode
By default, VM starts in maintenance mode where you can have network access. To switch to secure mode, you can hit the “Switch to Security” button. Once you perform the mode switch, within the VNC session, your screen is frozen in a short time. After that, you can resume your work. To switch from secure mode to maintenance mode, make sure you eject/unmount the secure volume before switching out of secure mode to ensure that any changes made to the secure volume are made permanent.
In the portal, go to "HTRC Data Capsule -> Show Virtual Machines" for the page for switching between modes. In the maintenance mode, click on the "Switch To Secure Mode" button in the portal to switch to the secure mode.
In the secure mode, click on the "Switch To Maintenance Mode" button in the portal to switch to the secure mode.
2.6 Stop a Virtual Machine
You can stop a VM by pushing the “Stop VM” button. After that, the VM shutdowns and everything inside the VM remains.
2.7 Restart a Virtual Machine
Although we do not provide a reset button for you to restart the VM directly, you can always stop the VM and then start it again. This has the same effect of pushing a reset button on a machine.
2.8 Delete a Virtual Machine
You can delete a VM by pushing the “Delete VM” button. After that, the VM is wiped out and everything inside the VM is gone.
1) The source code
The code base has 3 parts, a web GUI, web service and backend scripts
You can download the code for web GUI from http://sourceforge.net/p/htrc/code/HEAD/tree/HTRC-UI-Portal2/.
You can download the code repository for web service and backend scripts from https://github.com/htrc/HTRC-Data-Capsules
2) The web interface
You can find the url for the web front end from the HTRC production portal https://htrc2.pti.indiana.edu