Dataset Overview

The SkyScapes dataset focuses on semantic understanding of airborne scenes. Understanding the complex urban infrastructure with centimeter-level accuracy is essential for many applications from autonomous driving to mapping, infrastructure monitoring, and urban management. Aerial images provide valuable information over a large area instantaneously; nevertheless, no current dataset captures the complexity of aerial scenes at the level of granularity required by real-world applications. To address this, we introduce SkyScapes, an aerial image dataset with highly-accurate, fine-grained annotations for pixel-level semantic labeling. SkyScapes provides annotations for 31 semantic categories ranging from large structures, such as buildings, roads and vegetation, to fine details, such as 12 (sub-)categories of lane markings. We have defined two main tasks on this dataset:

dense semantic segmentation
multi-class lane-marking extraction

We carry out extensive experiments to evaluate state-of-the-art segmentation methods on SkyScapes. Existing methods struggle to deal with the wide range of classes, object sizes, scales, and fine details present.

We therefore propose a novel multi-task model, which incorporates semantic edge detection and is better tuned for feature extraction from a wide range of scales. This model achieves notable improvements over the base-lines in region outlines and level of detail on both tasks.

In the following, we give an overview on the design choices that were made to target the dataset’s focus.

Features

Polygonal annotations

Dense semantic segmentation
Instance segmentation for vehicles
- (for buildings will be included)

Raster annotations

Raster semantic segmentation of lane-markings

Edge annotations

Lane semantic annotation of edge boundaries

Complexity

30 classes
See Class Definitions for a list of all classes and have a look at the applied labeling policy.

Diversity

Ranging from very large objects to very tiny objects
Daytime
Good/medium weather conditions
Manually selected frames
- Large number of lane-marking instances
- Large number of dynamic objects
- Varying scene layout
- Varying background

Volume

70 000 annotated images with fine annotations (examples)

Metadata

GPS coordinates
Ego-motion data; height, focal length and camera orientation

Metadata

SkyScapes-Dense: Pixel-level dense multi-class semantic labeling
SkyScapes-Lane: Multi-class lane-marking semantic labeling
SkyScapes-Dense-Category: Pixel-level dense multi-class semantic labeling (categorized)
SkyScapes-Dense-Edge-Multi : Multi-class edge semantic labeling
SkyScapes-Dense-Boundary-Multi: Multi-class boundary semantic labeling
SkyScapes-Lane-Binary: Binary lane-marking labeling
SkyScapes-Dense-Boundary-Binary : Binary boundary labeling

Type of annotations

Contained areas

Labeling policy

Several annotators worked on the creation of the ground truth, each focusing on a separate set of classes. To ensure annotation consistency, a list of rules was established and extended as special cases were discovered. These guide-lines relate to two aspects of the annotation work: target identification and boundary topology. For the former, the annotators referred to the comprehensive class definitions to assign every object in the image to asemantic category. The vertical ordering of classes (or class overlays) was based on the natural physical ordering found in the real world, and as also considered in transportation systems,i.e., vehicles were put on top of all road-like objects, etc.

Labeled foreground objects must never have holes, i.e. if there is some background visible ‘through’ some foreground object, it is considered to be part of the foreground. This also applies to regions that are highly mixed with two or more classes: they are labeled with the foreground class. Examples: tree leaves in front of house or sky (everything tree), transparent car windows (everything car).

Class definitions

Please click on the individual classes for details on their definitions.

Number of annotated pixels (filled) and instances (non-filled) per class in SkyScapes-Dense and SkyScapes-Lane for low-vegetation (LV), tree (T), building (B), paved-road (PR), paved-parking-place (PP), non-paved-parking-place (nPP), non-paved-road (nPR), lane-marking (LM),sidewalk (SW), bikeway (BW), danger-area (DA), entrance-exit (EE), car (Ca), van (V), truck (TK), trailer (TR), long-truck (LT), bus (Bu), impervious-surface (IS), clutter (Cl),long line (LL), dash line (DL), tiny dash line (TDL), zebrazone (ZZ), turn sign (TS), stop line (SL), other signs (OS),the rest of lane-markings (R), parking zone (PZ), no parking-zone (nPZ), crosswalk (CW), and plus sign (PS).

Category	Classes
nature	low vegetationIncludes all natural areas without large plants, e.g., lawns. , treeAreas covered by large plants, such as trees or large bushes.
residential	building ^*Structures with walls and a roof, such as houses, factories, and garages.
vehicle-area	paved-road ^* ⁺Includes all roads that are asphalted. , non-paved-roadAll roads that are not paved, e.g., forest roads, dirt roads, and unsurfaced roads. , paved-parking-placeIncludes all asphalted areas for parking vehicles, such as car parks. The parking area include the vehicle as well which has not been shown in the figure , non-paved-parking-placeUnsurfaced areas used for parking. The parking area include the vehicle as well which has not been shown in the figure.
lane-markings	long-line ⁺Thin solid lines, such as no passing lines or roadside markings , dash-line ⁺Any broken line with long line segments, e.g., lane separators. , tiny dash-line ⁺Any broken line with tiny line segments, e.g., lines enclosing pedestrian crossings. , zebra-zoneAreas with diagonal lines, e.g., restricted zones. , turn signArrows on the road, such as intersection arrows or merge arrows. , stop lineThick solid line across lanes that signal to stop behind the line. , parking zoneIncludes any lines that mark parking spots. , no parking zoneZig-zag lines next to the curb mark that indicate that stopping or parking is forbidden. , crosswalkZebra-striped markings across the roadway mark a pedestrian crosswalk. , plus signAll crossing tiny lines. , other signsIncludes all other signs, e.g., numbers that indicate the speed limit. , rest of lane-markingsAny other lane-marking.
human-area	sidewalkPath with a hard surface on one or both sides of a road for pedestrians. , bikewayIncludes all lanes or roads for bikes. , danger-areaThe intersection of bikeways with road marked with red, blue or green in Germany and some other countries.
shared	entrance/exitAll entrance and exit areas that are shared with pedestrians.
vehicle	car ^Includes all cars except vans. , van ^Any vehicles with box-like shapes. , truck ^Includes all small trucks such as delivery trucks. , long-truck ^All long trucks such as heavy goods vehicles. , trailer ^Includes all trailers that can be attached to any vehicle, e.g., trucks or cars. , bus ^Any buses including tourist coaches, school buses, and public buses.
other	imprevious surface ⁺Includes all other surfaces, such as construction sites, and nontemporary obstacles road users cannot go through (e.g., low wall, rocky terrain, river). , clutter,Includes all other human made structures, such as garbage bins, fences, or outdoor furniture.

^* instance-level annotations are available. In case, boundary between such instances cannot be clearly determined, the whole group is labeled together
⁺ this label is raster.

Dataset Overview

Features

Labeling policy

Class definitions

SkyScapes-Dense

SkyScapes-Lane

Interesting links

Categories

Archive