Documentation
Child pages
  • HTRC Data Capsule Tutorial
6 more child pages
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 62 Next »


Once you have your capsule running, you may find it useful to open this guide in an internet browser in your capsule so you can copy and paste commands. The short link for this page is: https://wiki.htrc.illinois.edu/x/TQFRAQ. 

  1. Register for an HTRC account if you do not already have one. Read the guide for extended directions, or follow the steps in the link below.

    Sign Up




    • On the Sign Up page, enter requested information together with username you intend to use and password. The password must meet these requirements:                                    
      • Password must be more than 15 characters long.
      • Password must contain characters from three of the following five categories:
        • Uppercase characters of European languages (A through Z, with diacritic marks, Greek and Cyrillic characters)
        • Lowercase characters of European languages (a through z, sharp-s, with diacritic marks, Greek and Cyrillic characters)
        • Base 10 digits (0 through 9)
        • Nonalphanumeric characters
        • Any Unicode character that is categorized as an alphabetic character but is not uppercase or lowercase. This includes Unicode characters from Asian languages.
      • Password must not contain any white spaces.
      • Password must not contain your user ID.

    Trouble shooting: if you can't successfully sign up

    You need to have an academic email to sign up. We maintain a growing list of allowed email domains, e.g. emails ending with .edu or edu.ac or edu.tw. I you find you can't successfully sign up with your academic email then it's probably your email domain is not on the list. In this situation, please submit an account request by clicking on the request an account button within the error message on the page.

    Sign In


    • You will receive an email in the email that you registered. Go to your email box, and follow the activation link in the email to activate your account.
    • Now you have created an HTRC Analytics account, you can sign in. On the top right of the page, sign in with your username and password.

     



  2. Install a VNC client on your computer to enable the communication between your computer and the capsule you will remote access into. You can choose any VNC client you prefer. We use VNC View for Google Chrome in the screenshots below.

    Unable to render {include} The included page could not be found.

  3. Make sure you are still logged in to the HTRC where you just created an account. 

  4. From the HTRC, create a capsule by clicking on Capsule →  Create Capsule on the top menu. You will be asked to provide information about the capsule you would like to create.

    Unable to render {include} The included page could not be found.

  5. Start the capsule you created by clicking the Start Capsule button on the Show Capsules page.

    On the Capsules page (found under Capsule on the top menu), click on the Start Capsule button.



     

  6. Interact with the capsule via the VNC client. 

    Unable to render {include} The included page could not be found.

  7. Alternatively, you can SSH into your capsule when it is in maintenance mode only. 

    Key information for accessing your capsule is located on the Capsule Status page.

    On a Linux terminal, enter the following command inserting your capsule's port number found on the Capsule Status page. Then enter the password dcuser when prompted.

    ssh -p <your capsule's port> dcuser@thatchpalm.pti.indiana.edu

    Note that if you want to interact with the Ubuntu desktop of your capsule, or if you want to use your capsule in secure mode, you will need to use a VNC client as described above.

  8. Switch between maintenance and secure mode.

    Each capsule is designed to have 2 modes: maintenance mode and secure mode.
    • In maintenance mode, the user is allowed to access network freely except for HTRC corpus repository and install whatever software they wants. 
    • In secure mode, network access is restricted. The user is only allowed to access a few network addresses e.g., HTRC corpus repository and search service. 

    Any changes user makes to their capsule in secure mode, such as data they download or create during their analysis, will not persist. To save data, you will need to save your data to a special storage area on your capsule called secure volume. The secure volume is invisible in maintenance mode. Follow the further steps in this tutorial to learn how to preserve your capsule between research sessions.

    To switch modes, on the Capsules page, click on the Switch to Secure Mode or Switch to Maintenance Mode buttons to switch to the other mode, as shown below. You can practice switching modes, but you'll need your capsule in maintenance mode to follow the rest of the tutorial.



     

  9. Fetch volume OCR content using the HTRC Data API. 

    Preferred method

    HTRC has developed a Python library for loading volumes into the Data Capsule environment: the HTRC Workset Toolkit. The Toolkit is standard in all capsules created after March 18, 2018. If you have an earlier-created capsule then you will need to install or update the Toolkit. 

    Make sure you are in secure mode in order to prepare to fetch content into your Data Capsule; it won't work in maintenance mode for security reasons.

    You can use the Workset Toolkit's "htrc download" command to transfer the volumes you would like to include in your dataset.

    For example, the following command will import the volumes in the HathiTrust collection 'Adventure Novels: G.A. Henty'.

    htrc download 'https://babel.hathitrust.org/cgi/mb?a=listis;c=464226859'


    You can also curate a list of volumes whose data you would like to import by creating a file containing a HathiTrust volume ID list that you're interested in, with one ID per line. Run the above command replacing the collection URL with your file name.

    For example, if you had a file called myvolumes.txt, you would run the following command.

    htrc download myvolumes.txt


    In the above examples, the data will be transferred to “/media/secure_volume/workset/”. If you want to specify an alternative location, provide an output by including -o and the file path in your command.

    Other functions of the Workset Toolkit

    You can also use a volume ID, collection URL, or catalog record ID to import volumes. Additionally, you have the option to concatenate files, remove folders, and retrieve metadata using the functions of the Workset Toolkit.

    For more examples, see the detailed guide.

    For the technical documentation, see: https://htrc.github.io/HTRC-WorksetToolkit/cli.html

    Optional method

    Researchers can use the HTRC Data API to bring text data into their capsule, and can refer to the HTRC Data API guide for more details.



  10. Run experiments. Follow the Use Case guides for examples.

  11. If you will need more than one session to complete your research, save your interim data.

    Save data to the Secure Volume

    Make sure your capsule is in secure mode (see directions above if needed).

    Open a terminal window in the capsule and navigate to the secure volume by typing:

    cd /media/secure_volume

    Suppose the file you'd like to release is at /home/demouser/demo/r/Rplots.pdf

    You can prepare the result data for release by adding it, which is done by typing the command:   

    releaseresults add /home/demouser/demo/r/Rplots.pdf

    Repeat using this command if you have other files to add.


  12. Between sessions, stop the capsule via the HTRC using the web browser on your personal desktop. The next time you log in, you can restart the same capsule and continue your work.

  13. When you are finished with your research, request to export your non-consumptive data.

    (This is the same as Release results)

    If you'd like to export results out of the capsule, you must release them from your virtual machine.

    First, switch the VM to secure mode in the Portal interface. 

    Second, open a terminal in the capsule, navigate to the secure volume by typing:

    cd /media/secure_volume

    Suppose the file you'd like to release is at /home/demouser/demo/r/Rplots.pdf

    You can prepare the result data for release by first adding it the the release list: 

    releaseresults add /home/demouser/demo/r/Rplots.pdf

    Repeat using this command if you have other files to add.

    Finally, to complete the release of your data, type: 

    releaseresults done


    The files will be delivered via email. You will receive them in the email that you registered for the HTRC Analytics account. The email link will be live for 12 hours.

    Shut Down the VM

    Go back to  HTRC Analytics, go to "Data Capsules" and click on the "Stop Capsule" button to shut down the VM you'd like to shut.

    Delete the VM

    If you do not need the capsule any more, you can delete it. On HTRC Analytics, navigate to "Data Capsules", click on the "Delete Capsule" button next to the virtual machine you want to delete.


     

     

  14. When you no longer need it, delete your capsule via the HTRC. 

Questions?

  • No labels