Data Processing Documentation

Overview

Brief description of the data processing objectives and scope. Reminder to adhere to data ownership and usage guidelines.

Data Sources

List and describe data sources used, including links to cloud-optimized sources. Highlight permissions and compliance with data ownership guidelines.

CyVerse Discovery Environment

Instructions for setting up and using the CyVerse Discovery Environment for data processing. Tips for cloud-based data access and processing.

Data Processing Steps

Using GDAL VSI

Guidance on using GDAL VSI (Virtual System Interface) for data access and processing. Example commands or scripts:

gdal_translate /vsicurl/http://example.com/data.tif output.tif

Cloud-Optimized Data

Advantages of using cloud-optimized data formats and processing data without downloading. Instructions for such processes.

Data Storage

Information on storing processed data, with guidelines for choosing between the repository and CyVerse Data Store.

Best Practices

Recommendations for efficient and responsible data processing in the cloud. Tips to ensure data integrity and reproducibility.

Challenges and Troubleshooting

Common challenges in data processing and potential solutions. Resources for troubleshooting in the CyVerse Discovery Environment.

Conclusions

Summary of the data processing phase and its outcomes. Reflect on the methods used.

References

Citations of tools, data sources, and other references used in the data processing phase.