Multiresolution Fusion of Thermal Infrared Hyperspectral and VIS Data
Upon courtesy of the data owner, the data set of the 2014 Data Fusion Contest is kept available to the community for scientific purposes.
Goals and Organization of the Contest
The 2014 Data Fusion Contest, organized by the Image Analysis and Data Fusion (IADF) Technical Committee of the IEEE Geoscience and Remote Sensing Society, aimed at providing a challenging image analysis opportunity, including multiresolution and multisensor fusion, very high resolution imagery, and a completely new data type, which was never before considered in previous Data Fusion Contests.
The 2014 Contest involved two datasets acquired at different spectral ranges and spatial resolutions: a coarser-resolution long-wave infrared (LWIR, thermal infrared) hyperspectral data set and fine-resolution data acquired in the visible (VIS) wavelength range. The former was acquired by an 84-channel imager that covered the wavelengths between 7.8 to 11.5 µm with approximately 1-m spatial resolution. The latter was a series of color images acquired during separate flight-lines with approximately 20-cm spatial resolution. The two data sources covered an urban area near Thetford Mines in Québec, Canada, and were acquired and were provided for the Contest by Telops Inc. (Canada).
The 2014 Data Fusion Contest consisted of two parallel competitions:
- The Classification Contest, which was designed to promote innovation in classification algorithms, as well as to provide objective and fair comparisons among methods. The goal was to exploit coarser resolution thermal hyperspectral data and finer resolution color data to generate an accurate classification result at the finer of the two observed resolutions. Ranking was based on quantitative accuracy parameters computed with respect to undisclosed test samples. In addition to accuracy, another relevant aspect to assess a classification method is its computational burden, given the provided amount of training samples. In the Classification Contest, participants were given a limited time to submit their classification maps after the competition was started. For allowing the participants to effectively focus their methods on the proposed multiresolution task and type of input data, the Classification Contest consisted of two steps:
- Step 1: Participants were provided with a subset of the data, including ground truth to train their algorithms.
- Step 2: Participants received the full data set and submitted their classification maps by two weeks from the release of the full data set. In parallel, they submitted a description of the approach used.
- The Paper Contest, which aimed at promoting novel synergetic uses of multiresolution and multisensor data. Participants submitted 4-page IEEE-style manuscripts using the aforementioned data for fusion tasks. Each manuscript described the addressed problem, the proposed method, and the experimental results. The topic of the manuscript in the data fusion area was totally open and participants were encouraged to tackle open problems in multisensor and/or multiresolution data processing, as well as in the analysis of thermal hyperspectral imagery. Papers were evaluated and ranked by an Award Committee.
Schedule and Deadlines
- Step 1 of the Classification Contest: January 22, 2014
- Step 2 of the Classification Contest and opening of the Paper Contest: February 19, 2014 ► The full data set is kept available for download.
- Submission deadline for the Classification Contest: expired on March 5, 2014 ► The results of the Classification Contest are available HERE!
- Submission deadline for the Paper Contest: June 8, 2014 extended to and expired on June 15, 2014 ► The results of the Paper Contest are available HERE!
Questions and comments on the data and the Contest can be submitted to the Linkedin group of the IADF Technical Committee: http://www.linkedin.com/groups?home=&gid=3678437&trk=anet_ug_hm&goback=.gmr_3678437
The Data for the Contest
The data can be requested by registering for the Contest. Participants must read and accept the Contest Terms and Conditions.
|The thermal hyperspectral image was acquired using the “Hyper-Cam,” an airborne LWIR hyperspectral imager, which is based on a Fourier-transform spectrometer (FTS). The hyperspectral imager was integrated to a gyro-stabilized platform inside a fixed-wing aircraft. The airborne LWIR hyperspectral imagery consists of 84 spectral bands in the 868 to 1280 cm^-1 region (7.8 µm to 11.5 µm), at a spectral resolution of 6 cm^-1 (full-width-halfmaximum). It has been calibrated to at-sensor spectral radiance units, in W/(m^2 sr cm^-1). The spatial resolution is approximately 1 m.
A digital color camera (2 MegaPixels), mounted on the same airborne platform, was also used to acquire visible imagery data that consisted of multiple color sub-images associated with distinct flight-lines. The color imagery consists of RGB uncalibrated digital data. The spatial resolution of the color data used in the competition is 20 cm.
The two airborne data sources are georeferenced. The average height of both sensors above ground was 2650 ft (807 m). The two data sources were collected simultaneously on May 21, 2013, between 22:27:36 to 23:46:01 UTC.
Note that the airborne color data are composed of a series of spatially disjoint sub-images and that the data subset released for training purposes in Step 1 included only one of these color sub-images (one flight-line), together with the spatially corresponding sub-part of the airborne thermal hyperspectral image (see Fig. 1).
In the full data set, the airborne thermal hyperspectral image covers the whole considered ground area, while, in the airborne color data, the sub-images associated with distinct flight-lines are spatially separated by blank (missing) data (see Fig. 2).
Compared to the full data set, the data subset included approximately20% of the thermal hyperspectral samples and 30% of the available color samples.
The classification results to be submitted to the Classification Contest were expected to be at 20-cm spatial resolution and to cover the areas endowed with both thermal and color airborne observations (i.e., all pixels endowed with non-blank samples in the airborne color data).
A training map was provided in Step 1 along with the data subset. This map has the same spatial resolution as and is registered with the airborne color data subset. It has been made available for the Contest in both GeoTIFF and EXELISVIS-ENVI binary raster RAW formats. For both file formats, pixel values in the map are encoded through 8-bpp unsigned integers and represent seven thematic classes according to the following legend:
|Fig. 1. Example of the structure of the data subset: airborne color sub-image at 20-cm spatial resolution (bottom right); subset of the airborne thermal hyperspectral channels at 1-m spatial resolution (top); and training map at the same resolution of the color data (bottom left). Click to enlarge.||Fig. 2. Example of the structure of the full data set: airborne thermal hyperspectral channels covering the whole considered area at 1-m spatial resolution (top); and airborne color data at 20-cm spatial resolution, including multiple sub-images associated with distinct flight-lines and spatially separated by blank (missing) data (bottom). Click to enlarge.|
Only samples provided through this training map were allowed to be used for training purposes in the Classification Contest.
For the Paper Contest, additional data were also provided by Telops Inc., including:
- The 74 individual orthorectified hypercubes from which the airborne thermal hyperspectral image was generated through mosaicing.
- Ground spectral measurements obtained using a similar LWIR hyperspectral imager as the one used for the airborne acquisition. This hyperspectral imager was installed on a tripod on the ground, the line-of-sight of the instrument being approximately parallel to the horizon. A digital color camera was also taking visible ground data of the same scene, which were provided along with the ground spectral measurements. These imagers were installed at a geographic position located inside the ground area that was covered by the aforementioned airborne data.
To learn more about all data provided by Telops Inc. for the Contest, click here to download a description of the airborne and ground data.
Deadline for the submission of a classification map to the Classification Contest was March 5, 2014, i.e., two weeks after the release of the full data set.
Deadline for the submission of a manuscript to the Paper Contest was June 15, 2014.
For both competitions, submission consisted in sending the classification map or the manuscript as an attachment to an e-mail addressed to the contact e-mail address of the IADF Technical Committee.
Submissions were required to be compliant with the following formatting instructions:
- The subject line of the submission e-mail was IADFTC_2014_Classification_Contest and IADFTC_2014_Paper_Contest for the Classification and Paper Contests, respectively.
- The body of the submission e-mail for the Classification Contest included a short abstract (max 100 words), in which the classification algorithm was outlined.
- The classification map submitted to the Classification Contest had the same size and spatial resolution as the input color airborne data source in the full data set, was registered with these airborne color data, and was recorded as a TIFF or GeoTIFF file with pixel values (grey levels) encoded through 8-bpp unsigned integers. In the classification map, a thematic class label was expected to be assigned to all pixel locations corresponding to available (non-blank) samples in the airborne color data. These thematic class labels were represented through the following legend: 0 = unclassified; 1 = road; 2 = trees; 3 = red roof; 4 = grey roof; 5 = concrete roof; 6 = vegetation; 7 = bare soil.
- The manuscript submitted to the Paper Contest described the addressed problem, the proposed method, and the experimental results. The manuscript was written in English and was formatted as a PDF file following the guidelines and templates of the 2014 IEEE Geoscience and Remote Sensing Symposium (IGARSS-2014; details can be found at www.igarss2014.org). The manuscript specified the name(s), affiliation(s), and e-mail contact(s) of the participant(s).
- The name of the submitted file was formatted as lastname_firstname_email_address.tif and lastname_firstname_email_address.pdf for the Classification Contest and Paper Contest, respectively. The TIF file for the Classification Contest was ZIP-compressed before attaching it to the submission e-mail and the name of the compressed file was lastname_firstname_email_address.zip. For example, if John Doe, e-mail: email@example.com, participated to the Classification Contest, then his classification map was named doe_john_someid_at_somedept_dot_someuni_dot_edu.tif and the file attached to the submission e-mail was doe_john_someid_at_somedept_dot_someuni_dot_edu.zip.
Failure to follow these instructions automatically made the submission invalid, resulting in the classification map or manuscript not being evaluated.
For each of the two tracks of the Contest, one and only one submission was allowed per registered participant. Should multiple entries from any participant be received, then exclusively the first result submitted was considered.
Note that, regardless of the common format, submissions to IGARSS-2014 and to the Paper Contest were independent, i.e., a manuscript submitted to the Paper Contest was not considered for inclusion in the IGARSS-2014 technical program.
Award Committee for the Paper Contest
Each manuscript submitted to the Paper Contest was evaluated by an Award Committee on the basis of its novelty, scientific contribution, analysis and fusion methodology, experimental validation, and clarity of explanation. The Award Committee was composed of:
- Jon A. Benediktsson, University of Iceland (Iceland)
- Martin Chamberland, Telops Inc., Québec (Canada)
- Jenny Q. Du, Mississippi State University (USA)
- Paolo Gamba, University of Pavia (Italy)
- Gabriele Moser, University of Genoa (Italy)
- Fabio Pacifici, DigitalGlobe Inc. (USA)
- Michal Shimoni, Royal Military Academy (Belgium)
- DevisTuia, University of Zurich (Switzerland)
Results, Awards, and Prizes
The winning teams of both competitions were awarded at IGARSS-2014 (Québec, Canada) in July 2014. The award ceremony took place during the Technical Committees and Chapter Chairs Dinner. Each winning team was awarded an IEEE Certificate of Recognition and received a Nexus 7 tablet (one per team).
Furthermore, a paper summarizing the outcomes of both competitions will be submitted to the IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing (JSTARS) and in order to maximize impact and promote the potential of current multisensor remote sensing technologies, the open-access option will be used for this submission. The costs for open-access publication, for the winning teams’ participations to the Technical Committees and Chapter Chairs Dinner at IGARSS-2014, and for the prizes were supported by the GRSS.
The Contest was organized in collaboration with Dr. Michal Shimoni (Signal and Image Centre, Royal Military Academy, Belgium). The dataset was collected by Telops Inc. (Québec, Canada).
The IADF Technical Committee wish to express its greatest appreciation to Dr. Shimoni for her indispensable contribution to the organization of the contest, to Telops Inc. for acquiring and providing the data used in both competitions, to the Centre de Recherche Public Gabriel Lippmann (CRPGL, Luxembourg) and to Dr. Martin Schlerf (CRPGL) for their contribution of the Hyper-Cam LWIR sensor, to Dr. Michaela De Martino (University of Genoa, Italy) for her contribution to the preparation of the Classification Contest, and to the IEEE GRSS for continuously supporting the annual Data Fusion Contest through funding and resources.
How to Get the Data
The data are provided only for the purpose of participation in the 2014 Data Fusion Contest. To request the full data set, please click here to proceed to registration.
Those who have already registered to get the data subset released in Step 1 do not need to register again to get the full data set and have received download information directly via e-mail. For the sake of completeness, the data subset is also enclosed with the full data set.
By submitting the registration form, you acknowledge that you register to the Contest, that you have read the following Contest Terms and Conditions, and that you agree to these terms and conditions:
- The owner of the data and of the copyright on the data is Telops Inc. (Québec, Canada).
- The data are only available for the scientific purposes of the 2014 IEEE GRSS Data Fusion Contest.
- Any dissemination or distribution of the data by any registered user is strictly forbidden.
- The data can be used in scientific publications subject to approval by the IEEE GRSS IADF Technical Committee and by Telops Inc. on a case-by-case basis. To submit a scientific publication for approval, the publication shall be sent as an attachment to an e-mail addressed to firstname.lastname@example.org and email@example.com.
- In any scientific publication using the data, the data shall be identified as “grss_dfc_2014” and shall be referenced as follows: “[REF. NO.] 2014 IEEE GRSS Data Fusion Contest. Online: http://www.grss-ieee.org/community/technical-committees/data-fusion/”
- Any scientific publication using the data shall include a section “Acknowledgement.” This section shall include the following sentence: “The authors would like to thank Telops Inc. (Québec, Canada) for acquiring and providing the data used in this study, the IEEE GRSS Image Analysis and Data Fusion Technical Committee and Dr. Michal Shimoni (Signal and Image Centre, Royal Military Academy, Belgium) for organizing the 2014 Data Fusion Contest, the Centre de Recherche Public Gabriel Lippmann (CRPGL, Luxembourg) and Dr. Martin Schlerf (CRPGL) for their contribution of the Hyper-Cam LWIR sensor, and Dr. Michaela De Martino (University of Genoa, Italy) for her contribution to data preparation.”