Child pages
  • HTRC Data Capsule Specifications and Usage Guide
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 10 Next »

Overview

The HTRC Data Capsule is a secure computing environment developed to facilitate non-consumptive text analysis research. Each capsule is a virtual machine (VM) that provides researchers a desktop they can use to perform their investigation of volumes in the HathiTrust Digital Library. 

Further information, check out the FAQs

Follow a detailed tutorial

Using an HTRC Data Capsule

Maintenance vs. Secure Mode

The capsules are configured with special security settings that allow you to interact with them in two modes:  maintenance mode and secure mode

  • In maintenance mode, you are allowed to access the network freely and install whatever software you want. 
  • In secure mode, general network access is restricted, but you can access the HTRC corpus repository, which is otherwise blocked. Any changes you make to the capsule in secure mode will not persist. To save data from your analysis, you'll need to save your results in the secure volume storage on your capsule. This storage option is not visible in maintenance mode. 

Administering a Capsule

Use the HTRC Portal interface to handle administrative tasks for your capsule:

  • Create - a capsule is created, but it is not yet running

  • Start - turn the capsule on in maintenance mode

  • Stop - shutdown a capsule

  • Delete - the capsule is deleted (including its data and settings) 

  • Switch modes - change the capsule from maintenance to secure mode, or vice-versa

  • See status - view your capsules and their statuses


Interacting with the Capsule Interface

Access and log-in to the interface for your capsule from your desktop using either a VNC client or through SSH. You can log in to a capsule using a VNC client if the capsule is in either maintenance or secure mode. You can only SSH into the capsule when it is in secure mode. 

Obtain the requisite information for the capsule under Capsule → Show Capsules → <capsule Id> 

  1. Access the capsule using a VNC client:
    • Install a VNC client on your computer if you do not already have one. There a several you can choose from, including RealVNC, Chicken, and Screen Sharing (on a Mac). 
    • Open your VNC client and input the  VNC URL 
  2. Access the capsule via SSH:
    • From your Linux terminal, use the following command
ssh -p <your capsule port> dcuser@thatchpalm.pti.indiana.edu 
dcuser@thatchpalm.pti.indiana.edu's password: dcuser


Generic Research Workflow

  1. Create and start a capsule in the HTRC Portal

  2. Log into the capsule using a VNC client

  3. Configure the software environment of the capsule as needed. Download the scripts or programs you plan to use in your analysis

  4. Switch capsule to secure mode through HTRC Portal

  5. Run your against the secure HTRC corpus repository

  6. Move your results to the secure volume storage on the capsule

  7. Switch capsule back to maintenance mode to regain normal network access

HTRC Data Capsule Configurations

Capsule Technical Specifications

You can set several parameters for their capsule during the creation process

  • Data Capsule Image: there are two images (versions) of the standard capsule desktop, one that comes pre-loaded with sample volumes from the HathiTrust and one that does not, for the researcher to choose between
  • VNC Login User Name: set by the researcher
  • VNC Login Password: set by the researcher
  • Virtual Machine CPUs (VCPUs): the number of virtual machine processors from 1-4 VCPUs for the capsule, set by the researcher
  • Memory: displayed in megabytes, between 2048 MB (2GB) and 16000 MB (16GB)

Pre-installed Software, Libraries, and Data

Each capsule comes pre-loaded with the following software, libraries, and data. For more information, consult the ReadMe file on the desktop of your capsule for more details about installed packages.

Software

NameVersionURLNote

Akka

2.4.14http://akka.io/
Anaconda 3 4.2.0https://www.continuum.io/anaconda-overviewSupports both Python 2.X and 3.X. See list below for the Python libraries pre-installed (some via Anaconda)

Ant

1.9.7http://ant.apache.org/

Hadoop

2.7.3http://hadoop.apache.org/

Mallet

2.0.8http://mallet.cs.umass.edu/

R

3.3.9https://www.r-project.org/

Sbt

0.13.13http://www.scala-sbt.org/

Scala

2.12.1https://www.scala-lang.org/

Spark

2.0.2http://spark.apache.org/

Python Libraries 

  • csvkit
  • dask
  • GenSim (currently running with warning)
  • htrc-feature-reader
  • nltk
  • numpy
  • pandas
  • pytables
  • regex
  • scipy
  • theano
  • toolz
  • ujson

Sample data and programs

  • 3 sample HTRC worksets of 1000 volumes each: U.S. Government Documents, German language volumes, 19th Century English Literature. 
  • Topic Explorer: http://inphodata.cogs.indiana.edu/


 

 

  • No labels