![]() ![]() ![]() Step 6: Confirm that the contents of your download are as expected (in terms of size, file count, etc). Step 5: Use 'downloadcmd' to download the package you identified in step 1. Step 4: Install software from, Skip to red bolded text if you don't wish to care about Python's virtual environments if you don't have to. Step 3: Confirm that this environment has all the requirements. Step 2: Locate or create a command line environment that has access to adequate storage space AND permission to install software, and then navigate to this environment (for example, with AWS in this tutorial, using computational credits for CCF users at the NDA). where it is located in S3, and how it is organized. Rather, it will contain metadata about the available image data files. If your purpose is to download a subset of data from the NDA (individual files you specify at the command line), then this package should NOT actually contain the image data (e.g. Step 1: Identify the package number you wish to download from the NDA. Willingness to locate or create a terminal to a Linux/MacOS operating system that affords you permission to install software and has access to a filesystem location with space for a download (downloads including imaging data for all HCA or HCD subjects can be hundreds of GB to 20TB, depending on which data you select).Step 7 of this tutorial will help you to locate and parse the datastructure_manifest.txt Using the downloadcmd tool to get subsets of a particular package of data currently depends on your ability to locate and parse the datastructure_manifest.txt file for S3 links. Note that the NDA appears to be in the process of rewriting the rules on datastructure_manifest.txt file usage and sharing in general, so the defaults for package generation might change (if this happens we will update these instructions). ![]() In particular, one that contains a 'datastructure_manifest.txt' file which is created as a part of the package creation process. a 'shared' package, a package we created that you added to your account, or a package you created yourself. A data package that you want to download from the NDA, i.e.The following steps should allow you to download any package to which your data use certification (DUC) grants you access. See Lifespan 2.0 Data Access & Download Instructions for full instructions on getting access. Access to data from your account at the NDA.Knowing this stuff will make using that stuff more intuitive. Secondary Purpose: Extend your exposure to AWS cloud computing from your desktop machine, whilst gaining exposure to the python virtual environment management tools that will make it possible to translate what you may be used to doing locally to more scalable, reproducible, and meta-analyzable compute infrastructure. Purpose Primary Purpose: Get data from the NDA using (mostly) command line tools. This tutorial should take about 1 hour to complete, not including download times. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |