2024-12-06
Timeline
Reference Data Selection
Example detailed classification
Example coarser detailed classification
Lots of missing and undocumented processed data
Wrong and undocumented information - CRS, type, year, rate-limited APIs
Good coverage, but quality drops some places
Street processing
Algorithm Design and Theoretical Basis Description
Model protocol
Subregions
Enclosures
Nodes
Tessellation cells
Building area in Krakow
Evaluation setup
Spatial Split
Spatial Split zoom
Predictions with an accuracy of ~ .68
Classification vs Segmentation
Overall approach comparison
→ choose final model approach
Model adaptations (baseline approach)
Segmentation dataset
- 224 x 224 pixel tiles
- Train: 21,402 tiles; Test: 5,351 tiles
Classification dataset
- 56 x 56 pixel tiles
- Train: 342,648 tiles; Test: 61,074 tiles
Class imbalance
Shared train/test split
Pipeline
Fine-tuned three models
Recap
Metric | Satlas | Clay (Best) | Prithvi |
---|---|---|---|
Weighted Accuracy | 0.57 | 0.72 | 0.62 |
IoU | 0.33 | 0.58 | 0.41 |
F1 Score | 0.41 | 0.69 | 0.58 |
Training Time (epoch) | 9 min | 8 min | 20 min |
→ Clay model outperformed other models
Clay model
→ trained with focal loss
Pixel-level comparison
Approach | Global Ac. | Macro Acc. | F1 Score | IoU |
---|---|---|---|---|
A: Class. (embed.) | 0.76 (0.66) | 0.22 (0.13) | 0.23 | 0.63 |
A: Class. + H3 lvl 5 | 0.87 (0.82) | 0.42 (0.35) | 0.45 | 0.79 |
B: Seg. (Clay) | 0.73 | 0.31 | 0.30 | 0.58 |
C: Class. (Clay) | 0.59 (0.68) | 0.09 | 0.12 | 0.38 |
Comparison of approach B & C
Signatures are not strictly categorical (some are closer than other ones)
ordinal_mapping = {
'Wild countryside': 0,
'Countryside agriculture': 1,
'Urban buffer': 2,
'Open sprawl': 3,
'Disconnected suburbia': 4,
'Accessible suburbia': 5,
'Warehouse/Park land': 6,
'Gridded residential quarters': 7,
'Connected residential neighbourhoods': 8,
'Dense residential neighbourhoods': 9,
'Dense urban neighbourhoods': 10,
'Urbanity': 11,
}
Approach | Global Acc. | Macro Acc. | F1 Score | IoU |
---|---|---|---|---|
A: Class. (embed.) | 0.76 (0.66) | 0.22 (0.13) | 0.23 | 0.63 |
A: Class. + H3 lvl 5 | 0.87 (0.82) | 0.42 (0.35) | 0.45 | 0.79 |
A: Class. + H3 + ordinal | 0.80 (0.80) | 0.26 (0.26) | 0.26 | 0.69 |
Tile size | Model | Global Acc. | Macro Acc. | F1 |
---|---|---|---|---|
56x56 | Class. (embed.) | 0.76 | 0.22 | 0.23 |
56x56 | Class. (embed.) + H3 lvl 5 (cat) | 0.87 | 0.42 | 0.45 |
56x56 | Class. (embed.) + H3 lvl 5 (lat/lon) | 0.87 | 0.39 | 0.42 |
56x56 | Class. (embed.) + H3 lvl 5 ordinal | 0.80 | 0.26 | 0.26 |
25x25 | Class. (embed.) | 0.73 | 0.31 | 0.30 |
25x25 | Class. (embed.) + H3 lvl 5 (lat/lon) | 0.81 | 0.46 | 0.53 |
Random Sampling | H3 Split (Resolution 3) |
---|---|
Ensures diverse samples but risks spatial leakage, overestimating performance. | Reduces spatial leakage for realistic generalization but may under/over-sample signature types. |
Benefits training diversity but may inflate results due to proximity of train/test data. | Highlights spatial independence but may penalize heterogeneity within regions. |
Random sampling
Currenty investigating!
Goal
→ Deployment on all data in the end; (sampling only for reporting accuracy)
Consolidated Stakeholder Requirements Specification
Looking at urban challenges and innovative EO-integrated solutions, by bringing together urban policymakers, Earth Observation researchers, service providers and various end-users.
Convened by UN-Habitat, the Forum is a high level, open and inclusive platform for addressing the challenges of sustainable urbanisation.
Representing both public and private sectors.
Interest in addressing unmet data needs for energy and climate applications: interview to COM staff + either interview to selected members of CoM or survey.
Interest in classifying Lithuania into morphological types and being engaged also in the validation stage.
Comments