Mathematical analysis of

(1)

Mathematical analysis of satellite images

Part 1 István László*

István Fekete**

Summer School in Mathematics

Institute of Mathematics, Eötvös Loránd University, Budapest, Hungary, June 6 - 10, 2016

* Institute of Geodesy, Cartography and Remote Sensing

(2)

2

Summer School in Mathematics, ELTE, Budapest, 06-10 June 2016

•Part 1: An overview of remote sensing

• 1. Introduction: the evolution of remote sensing

• 2. Raw material: acquisition and pre-processing

• 3. Evaluation: only visual approach?

• 4. Practical applications

• 5. Education and university collaboration

•Part 2: Advanced methodology

• 1. Numerical evaluation of satellite images

• 2. The whole process of evaluation

• 3. Segment-based thematic classification

• 4. Data fusion

• 5. Object-based image analysis (OBIA)

(3)

1. Introduction: the evolution

of remote sensing

(4)

Needs?

- The exhausting of local and global resources (first: oil) - Global problems

- The extinction of species - Club of Rome

Therefore natural resources should be managed, based on exact survey!

Opportunities?

- Space technology - Sensors

- High speed data transfer

- 1972: launch of the first LANDSAT (ERTS) satellite - Fast computers with graphical capabilities

- Digital image processing

4

(5)

Remote sensing:

from data acquisition to thematic evaluation

The 3 main

components of remote

sensing system

(6)

The parts of optical wavelength interval

6

(7)

The first series of Landsat

satellites

(1, 2, 3)

(1972-1983)

(8)

SPOT 5 HRG sensor

8

(9)

Very high resolution satellite images

(0,5 m x 0,5 m – 4 m x 4 m-es ground resolution)

IKONOS satellite image

(10)

A part of IKONOS multispectral satellite image (4 m) Sapporo, Japan, 1999 October 6

Space Imaging

10

(11)

The first Hungarian satellite: MASAT-1

http://cubesat.bme.hu

2012 February 13 – 2015 January 9

(12)

Long-term European programme: Copernicus

…and Sentinel satellites

Sentinel-1A (2014. 04. 03.): weather independent, radar sensor

Sentinel-2A (2015. 06. 23.): high resolution multispectral optical sensor

12

(13)

2. Raw material:

data acquisition and pre-processing

(14)

• Passive, reflective systems:

- Sun is the source of electromagnetic (EM) radiation - Sampling “windows” of the whole EM spectrum

The physical background of remote sensing

 Different land covers reflect differently:

- crops, - water, - soil

 Sensors measure the intensity of electromagnetic radiation arriving from the Earth’s surface

14

(15)

Spectral response curves

What are the acquisition bands used for?

(16)

The diversity of remote sensing ^{- Carrier}

- Height - Time - Sensors - Wavelength

Satellites:

Geostationary orbit: 36 000 km (Near) polar orbit: 450-1000 km

Airplanes:

300 m-10 km

Drones:

UAV – Unmanned Aerial Vehicle or RPAS - Remotely

Piloted Aircraft Systems:

30-600 m

Gliders:

100-300 m

Terrestrial observation:

5 m

16

(17)

Scanning acquisition:

(18)

(19)

What does a remote sensing image contain?

Pixels <--> elementary pieces of surface

Band values within a pixel

(20)

The main parameters

of remote sensing systems

Spatial:

- Spatial coverage - Ground resolution Spectral:

- Spectral resolution

- Radiometric resolution Temporal:

- Temporal resolution (cycle length)

- Other factors: data access (availability, speed, price)

20

(21)

Scene ID Date

1333 2012.08.06.

8364 2012.08.18.

8481 2012.08.18.

4565 2012.08.25.

The mosaicking of image tiles

(22)

Geometric correction

It is clear that an accurate Digital Elevation Model is inevitable.

Scene 8481, RGB 423 Scene 8364, RGB 412

22

(23)

Two complete coverages of VHR images (Pléiades)

Pléiades, 2013.05.18. + Pléiades, 2013.07.17.

(24)

Pixel-based fusion („merge”)

- Pricipal Component Analysis (PCA)

- Modified Intensity-Hue-Saturation Merge (MIHS)

- In-house developed fusion, based on high pass filter (HPF)

24

(25)

3. Evaluation:

only visual approach?

(26)

The thematic evaluation of remote sensing images

2003. 03. 29., Landsat 7 ETM+

2003. 07. 27., Landsat 5 TM

A part of a crop map

Őszi búza Tavaszi árpa Őszi árpa Kukorica Silókukorica Napraforgó Cukorrépa

Lucerna Vízfelszínek

Nem mezőgazdasági területek Egyéb szántóföldi növények

26

(27)

The diversity of thematic categories

Őszi búza Tavaszi árpa Őszi árpa Kukorica Silókukorica Napraforgó Cukorrépa

Lucerna Vízfelszínek

Nem mezőgazdasági területek Egyéb szántóföldi növények

(28)

Texture: the regular changes of intensity values

28

(29)

Feladat Vizuális interpretáció (szem + agy rendszer)

Számítógépes kiértékelő rendszer

Geometriai összefüggések, struktúrák

felismerése

kitűnő gyenge

Textúra felismerése, azonosítása jó gyenge

Textúra mérése gyenge kitűnő

Tónusok elkülönítése közepes kitűnő

Megbízhatóság, objektivitás, reprodukálhatóság

közepes jó

Feldolgozási sebesség gyenge kitűnő

Bonyolult szakértelem, egyéb ismeretek alkalmazása

jó közepes

Több adatforrás vagy időpont együttes kiértékelése

gyenge kitűnő

The two basic methods of remote sensing data evaluation:

visual interpretation

and digital image processing

(30)

3.1. Numerical evaluation

30

(31)

The basic task of image processing

and the elementary solutions of classification

Pixels belonging to categories

The intensity vectors of categories show a typical probability distribution

in certain parts of intensity space

(32)

(33)

Clusters and thematic categories

(34)

3.2. Visual evaluation

34

(35)

Application in remote sensing subsidy control (CwRS)

Sensor Date

Ikonos 2013.06.21.

Ikonos 2013.07.02.

GeoEye 2013.07.03.

GeoEye 2013.07.06.

(36)

CwRS, Example#1: tree density counting

Ikonos, 2012.07.02. Pléiades, 2012.08.18.

36

(37)

CwRS, Example#2: the detection of sunflower

GeoEye, 2012.07.06. Pléiades, 2012.08.18.

(38)

CwRS, Example#3: cereal stubble (weed-free)

Ikonos, 2012.07.02. Pléiades, 2012.08.18.

38

(39)

3.3. Combined solutions

(40)

The detection of weed-infected areas

Weed infection in soybean parcels, detected with the quantitative evaluation of NDVI map. The grades of weed infection can be observed with Pleiades images on

soybean stubble. The extent of infection can be well measured within parcels.

40

(41)

Automatic delineation of forest areas

Object-based Image Analysis (OBIA)

(42)

The quantitative comparison of Pleiades images

2012.08.06.  2012.08.18.:

Left to right:

difference, 2012.08.06, 2012.08.18.

42

(43)

4. Practical applications

(44)

4.1. The National Operational

Crop Monitoring and Production Forecast Programme (CROPMON)

44

(45)

Data Flow in CROPMON

CROPMON INFORMATION

EXTRACTION SYSTEM Precalibration,

historical data

Reference data

Low resolution High resolution satellite data

Crop maps and area figures

Development assessment

Yield forecast

Production Reports

W in ter w heat L egen d

Spring ba rley W in ter ba rley M aize Sila ge m a ize Sunflow er Sug arbeet A lfalfa P ea s O ther cerea ls Spring fo dder crops

1. Megye Őszi búza

(ha) Őszi árpa

(ha) Tavaszi árpa

(ha)

2. Pest, Budapest 69 694 13 522 7 871

Közép-Magyarország 69 694 13 522 7 871

Fejér 82 809 8 603 4 659

Komárom-Esztergom 30 598 5 621 4 744

Veszprém 36 982 15 751 9 654

Közép-Dunántúl 150 389 29 975 19 057

Győr-Moson-Sopron 68 062 13 965 24 257

Vas 39 011 7 456 13 853

Zala 22 241 7 441 6 030

Nyugat-Dunántúl 129 314 28 862 44 140

Baranya 55 873 13 734 5 959

Somogy 50 241 11 666 2 018

Tolna 54 666 10 264 1 965

Dél-Dunántúl 160 780 35 664 9 942

Borsod-Abaúj-Zemplén 58 269 5 249 20 885

Heves 52 188 6 397 10 906

Nógrád 22 031 2 040 6 673

Észak-Magyarország 132 488 13 686 38 464

Hajdú-Bihar 68 156 8 238 8 487

Jász-Nagykun-Szolnok 116 323 14 016 15 198

Szabolcs-Szatmár-Bereg 36 212 5 087 3 284

Észak-Alföld 220 691 27 341 26 969

Bács-Kiskun 93 202 27 864 6 043

Békés 124 146 21 279 3 893

Csongrád 70 870 17 375 2 930

Dél-Alföld 288 218 66 518 12 866

1. Megye Őszi búza

(kg/ha) Őszi árpa

(kg/ha) Tavaszi árpa

(kg/ha)

2. Pest, Budapest 3 650 3 146 2 540

Közép-Magyarország 3 650 3 146 2 540

Fejér 4 180 3 802 3 298

Komárom-Esztergom 3 909 3 486 2 834

Veszprém 3 760 3 407 3 031

Közép-Dunántúl 4 022 3 535 3 047

Győr-Moson-Sopron 3 712 3 378 3 307

Vas 3 626 3 250 3 245

Zala 3 861 3 610 3 152

Nyugat-Dunántúl 3 712 3 405 3 266

Baranya 4 346 3 867 2 934

Somogy 3 718 3 572 2 959

Tolna 4 179 3 957 3 040

Dél-Dunántúl 4 093 3 796 2 960

Borsod-Abaúj-Zemplén 3 328 2 912 2 721

Heves 3 116 2 977 2 614

Nógrád 3 193 2 841 2 541

Észak-Magyarország 3 222 2 932 2 659

Drought Alert

Reports

(46)

26^th EARSeL Symposium May 29 – June 2, 2006, Warsaw

• Basics:

• combination of spatial with spectral/temporal information (high res. + AVHRR)

• NOAA AVHRR images and crop maps =>

crop specific temporal profiles

• Features:

• generic: works for several crops

• year independent

• area independent

• reliable, accurate, timely

The FÖMI RSC crop yield forecast model

Part of a crop map with 1.1 km grid overlay, corresponding to the NOAA

AVHRR pixel size

Corresponding subset of a NOAA AVHRR colour

composite (211 RGB)

good

bad

(47)

Area measurement: Crop maps –

detailed analysis at pixel level

Kukorica megyei területadatok 1991-2000 FÖMI TK (távérzékeléssel mérve) - KSH adat

R² = 0,91

20000 40000 60000 80000 100000 120000 140000 160000

KSH területadatok (ha)

Őszi búza megyei területadatok 1991-2000 FÖMI TK (távérzékeléssel mérve) - KSH adat

R² = 0,97

0 20000 40000 60000 80000 100000 120000 140000 160000 180000

KSH területadatok (ha)

(48)

4.2. Waterlog and flood monitoring

48

(49)

Monitoring the impact of waterlog

using satellite image time series

(50)

Waterlog maps derived from IRS WIFS medium resolution satellite data

• good overview at country and at county level

• frequent (3-4 days), good for change detection

• cover the whole country with low cost

• quick processing

50

(51)

Monitoring the change of waterlog

March 12, 1999 March 3, 1999

182 ha 86 ha 1384 ha 596 ha 38 ha 17 ha

The dynamics of waterlog and its impact can be quantified. The affected areas can be assessed by

villages as well.

standing water saturated soil crop in water no waterlog

natural water bodies

(52)

Real-time flood monitoring, 2001

52

(53)

4.3. The building up and maintenance

of Land Parcel Identification System

(LPIS; MePAR in Hungarian)

(54)

Orthophoto 2011, MePAR 2012

Review and change management of physical blocks Example #1

The reduction of eligible area because of road construction

54

(55)

Review and change management of physical blocks Example #2

The reduction of eligible area because of urban development

(56)

Review and change management of physical blocks Example #3

The reduction of eligible area because of the extension of industrial area (opencast)

56

(57)

The MePAR land cover system

(58)

4.4. Control with Remote Sensing of Agricultural Subsidies

58

(59)

The comparison of claims and real situation:

• Cultivated crop

• Parcel area

• Good Agricultural and Environmental Conditions (GAEC)

Remote sensing control of area-based agricultural subsidies

Claims

(electronic)

Control in GIS

using satellite images

Result: control

documents

(electronic)

(60)

Spring 1

Crop identification

Satellite images

High resolution (10-25m) time seriesVery high resolution (0,5-1m) Spring 2

Summer 1 Summer 2

SPOT 2 XS

SPOT 4/5/6/7 Xi

Landsat 5/7/8 (E)TM IRS-1C/D/P6/R2 LISS RapidEye

VHR

Area measurement

Ikonos QuickBird

Pléiades 1A/1B GeoEye

WorldView 1/2/3

Basic data of CwRS:

high and very high resolution satellite images (HR, VHR)

CwRS central database

60

(61)

Computer-aided Photo-interpretation (CAPI) with GIS software developed within FÖMI

Digitised parcel drawing

Claim database

High resolution satellite image time series for Very high resolution (VHR)

satellite images for area

(62)

Different crop types can be distinguished at parcel level using Very High Resolution images

rape seed cereal row crop

cereal rape seed

rape seed

62

(63)

The protection of permanent grassland: at least one mowing per year or regular grazing

Control of minimum cultivation practice

(64)

2009-04-22, SPOT4 2009-07-24, SPOT4 2009-08-17, SPOT5 2009-05-20, VHR

Encroachment of unwanted weeds must be prevented

Control of minimum cultivation practice

64

(65)

GAEC - the checking of prevention of soil erosion with DEM+GIS

(DEMsteep parcels, CAPIparcels with row crops, intersection: problem!)

(66)

5. Education and

university collaboration

66

(67)

• 1983-84: Joint development of a system evaluating satellite images

• 1985-2002: Occasional collaborations, joint publications

• 1999, 2005: Segment-based classification appears in PhD projects

• 2003: The establishment of Faculty of Informatics. Further joint research projects start

• 2004: The launching of Geospatial Information Systems educational module. It contains the subject Analysis of Remote Sensing Images (2+2), maintained jointly by ELTE and FÖMI

• 2006-2011: Twelve students attended at cooperative education in FÖMI

Collaboration in research and education

between ELTE and FÖMI

(68)

Education: the course

„Analysis of Remote Sensing Images”

- Started in 2005, 5th year MSc students. The curriculum contains a series of about 400 slides within 15 lectures.

- Presentation (I. László, FÖMI) includes the theoretical background of remote sensing (pre-processing, image analysis, statistical classification) and covers wide variety of practical applications.

- Lab seminars (R. Giachetta, ELTE): students implement programming tasks in connection with remote sensing (transformations, filtering,

segmentation, clustering, classification).

68

(69)

Mathematical analysis of