Projects‎ > ‎Solar activity‎ > ‎

1. Getting the data

Data files can be downloaded from the Joint Science Operations Center. We use the synoptic data, which are 1k x 1k images in FITS format, available at http://jsoc.stanford.edu/data/aia/synoptic/

To mass download files we write a script in R. In this example we fetch one image at one wavelength for each available date. The chosen wavelength is 211 Angstrom. The script below resulted in 760 files being downloaded (on July 3rd, 2012 - note that new files are added every day).

library(RCurl)
years = c("2010","2011","2012")
months = as.character(1:12)
months[nchar(months)==1] = paste("0",months[nchar(months)==1],sep="")
days = as.character(1:31)
days[nchar(days)==1] = paste("0",days[nchar(days)==1],sep="")

urlBase = "http://jsoc.stanford.edu/data/aia/synoptic"
locPath = "D:/data/SDOAIA"

for (yr in years) {
    for (mo in months) {
        for (da in days) {
            thisFile = paste("AIA",yr,mo,da,"_0000_0211.fits",sep="")
            thisPath = paste(yr,mo,da,"H0000",sep="/")
            print(thisFile)
            flush.console()
            myUrl = paste(urlBase, thisPath, thisFile, sep = "/")
            myDest = paste(locPath, thisFile, sep = "/")
            myBits = getBinaryURL(myUrl)
            if (length(myBits) > 500) { # if files don't exist we get a short reply from the server: skip these
                myFile = file(myDest, "wb")
                writeBin(myBits,myFile)
                close(myFile) 
            }
        }
    }
}

The downloaded files are in compressed FITS format. The Fitsio package for R cannot handle these. We use the funpack utility (see here) to decompress the files. Again, this can be done in batch using following R script. Make sure to set the directory with the FITS files as the working directory.

sdoFiles = dir()
sdoFilesFz = paste(sdoFiles,".fz",sep="") # rename the original files to have file extension .fz, as required by funpack
file.rename(sdoFiles,sdoFilesFz)
for (thisFile in sdoFilesFz) {
    system(paste("d://fpack//funpack",thisFile))
}
 
This will generate a lot of data, approx. 3 GB, make sure to have enough disk space available.
 
Information on the data use policy is given here: http://sdo.gsfc.nasa.gov/data/rules.php. In line with these guidelines we say the following about the data files used in this example: Courtesy of NASA/SDO and the AIA, EVE, and HMI science teams.