Handbook of Big Geospatial Data

Preface
	Overview of the Book
Contents
Part I Spatial Computing Systems and Applications
	1 IBM PAIRS: Scalable Big Geospatial-Temporal Data and Analytics As-a-Service
		1.1 Introduction
		1.2 PAIRS Architecture Overview
		1.3 Key-Value Store Design and Performance
		1.4 PAIRS User Experience
			1.4.1 Data Service
			1.4.2 Search or Discovery Service
			1.4.3 Analytics Platform Service
		1.5 Selected Industry Applications
			1.5.1 PAIRS Enabled Improvements in Weather Forecasting
			1.5.2 Vegetation Management
		1.6 Conclusion and PAIRS Resources
		References
	2 Big Geospatial Data Processing Made Easy: A Working Guide to GeoSpark
		2.1 Introduction
		2.2 Background
			2.2.1 Cluster Computing Systems
			2.2.2 Spatial Queries
		2.3 Overview
		2.4 Spatial RDD Layer
			2.4.1 Supported Spatial Data Sources
			2.4.2 Spatial RDD Built-In Geometrical Library
			2.4.3 Spatial RDD Partitioning
			2.4.4 Spatial RDD Index
			2.4.5 Spatial RDD Customized Serializer
		2.5 Spatial Query Processing Layer
			2.5.1 Spatial Range Query
			2.5.2 Spatial K Nearest Neighbors (KNN) Query
			2.5.3 Spatial Join Query
		2.6 Perform Spatial Data Analytics in GeoSpark
			2.6.1 Run Queries Using RDD APIs
			2.6.2 Run Queries Using SQL APIs
			2.6.3 Interact with GeoSpark via Zeppelin Notebook
		References
	3 Indoor 3D: Overview on Scanning and Reconstruction Methods
		3.1 Introduction
			3.1.1 Terminology
		3.2 Properties of Indoor Environments and Identification of Scanning and Reconstruction Problems
		3.3 Map Representations
		3.4 Development of Indoor Scanning Systems
			3.4.1 Single Sensor Methods and Multi-sensor Systems
				3.4.1.1 Carriable Systems
				3.4.1.2 Mobile Platforms
				3.4.1.3 Micro Aerial Vehicles
		3.5 Iterative Closest Point SLAM
			3.5.1 The ICP Algorithm
			3.5.2 Computing Optimal Poses
			3.5.3 Marker and Feature-Based Registration
			3.5.4 ICP-Based SLAM
			3.5.5 Assessing the SLAM Errors
		3.6 Indoor 3D Reconstruction
			3.6.1 Space Subdivision and Room Segmentation
			3.6.2 Reconstruction of Walls
			3.6.3 Grammar Approach
			3.6.4 Detection and Reconstruction of Openings
			3.6.5 Reconstructing Occluded Data by Machine Learning
		3.7 Applications
		3.8 Future Trends
		3.9 Exercises for Students
		References
	4 Big Earth Observation Data Processing for Disaster Damage Mapping
		4.1 Monitoring Disasters from Space
		4.2 Earth Observation Satellites
			4.2.1 Optical Satellite Missions
			4.2.2 SAR Satellite Missions
		4.3 Land Cover Mapping
		4.4 Disaster Mapping
			4.4.1 Flood Mapping
			4.4.2 Landslide Mapping
			4.4.3 Building Damage Mapping
		4.5 Conclusion and Future Lines
		References
	5 Spatial Data Reduction Through Element-of-Interest (EOI) Extraction
		5.1 Introduction
		5.2 Methods to Obtain EOI from Georeferenced Big Data
			5.2.1 Methods Commonly Used in the Remote Sensing and Mapping Fields
				5.2.1.1 Pixel-Based Methods
				5.2.1.2 Object-Based Methods
				5.2.1.3 Machine Learning
			5.2.2 Methods to Analyze Social Media and Location-Based Data
				5.2.2.1 Data Mining
				5.2.2.2 Data Analytics
				5.2.2.3 Machine Learning
		5.3 Use Cases in the Active and Passive Big Data Spatial Realms
			5.3.1 Active Use Cases
			5.3.2 Passive Use Cases
		5.4 Conclusion
		References
	6 Semantic Graphs to Reflect the Evolution of GeographicDivisions
		6.1 Introduction
		6.2 Context
			6.2.1 Not Fully Interconnected Data
			6.2.2 Broken Time-Series
			6.2.3 Removal of Changes
		6.3 Towards a Change in Representation with the Semantic Web
			6.3.1 Open Data
			6.3.2 Linked Data
			6.3.3 Semantic Data
			6.3.4 Linked Open Geospatial Data
		6.4 Modeling Geospatial Changes in the Semantic Web
			6.4.1 Identity and Changes
			6.4.2 Modeling Changes
				6.4.2.1 Standard Space and Time Ontologies
				6.4.2.2 Fundamentals for the Modeling of Evolving Geospatial Entities
			6.4.3 Ontological Approaches for the Modeling of Evolving Entities
				6.4.3.1 Versioning Approach
				6.4.3.2 SNAP and SPAN Approach
				6.4.3.3 Ontologies for Fluents Approach
			6.4.4 Ontological Approaches for the Modeling of Evolving Geospatial Entities
		6.5 Contributions
		6.6 Conclusion and Perspectives
		References
Part II Trajectories, Event and Movement Data
	7 Big Spatial Flow Data Analytics
		7.1 Introduction
		7.2 Flow Mapping & Geovisualization
			7.2.1 Flow Aggregation
			7.2.2 Edge Bundling
			7.2.3 Visual Analytics and Tools
		7.3 Spatial Data Mining Methods
			7.3.1 Spatial Outlier Detection
			7.3.2 Flow Clustering
		7.4 Spatial Statistical Methods
			7.4.1 Spatial Patterns Detection
			7.4.2 From Patterns to Spatial Interaction Models
		7.5 Conclusion
		References
	8 Semantic Trajectories Data Models
		8.1 Introduction
		8.2 Preliminaries
			8.2.1 Historical Perspective
			8.2.2 Spatial vs. Semantic Trajectories
		8.3 A Semantic Trajectory Meta-model
		8.4 Semantic Trajectory Data Models: A Purpose-Driven Taxonomy
			8.4.1 Conceptual Representation
			8.4.2 Database Logical Models
			8.4.3 Query Processing
			8.4.4 Data Analytics
		8.5 Final Remarks and Research Directions
		References
	9 Multi-attribute Trajectory Data Management
		9.1 Introduction
		9.2 Related Work
			9.2.1 Enriching Spatio-Temporal Trajectories
			9.2.2 Indexing Spatio-Temporal Trajectories
		9.3 Problem Definition
			9.3.1 Data Representation
			9.3.2 Queries
		9.4 Indexing Multi-attribute Trajectories
			9.4.1 An Overview of the Structure
			9.4.2 Packing Trajectories
			9.4.3 Partitioning Trajectories
			9.4.4 BAR
			9.4.5 Updating the Index
			9.4.6 The Generality
		9.5 Query Algorithms
			9.5.1 An Outline
			9.5.2 Processing RQMAT
			9.5.3 Processing CRQMAT
			9.5.4 Processing CkNN_MAT
		9.6 The System Development
			9.6.1 The Architecture
			9.6.2 A Tool for GPS Data Clean
			9.6.3 The Generation of Multi-attribute Values and Query Interface
			9.6.4 MDBF: A Tool for Monitoring Database Files
		9.7 Performance Evaluation
			9.7.1 Evaluation of RQMAT
			9.7.2 Evaluation of CRQMAT
			9.7.3 Evaluation of CkNN_MAT
		9.8 Future Directions
			9.8.1 Data Analytics
			9.8.2 Intelligent Trajectory Data Management
		9.9 Conclusions
		References
	10 Mining Colocation from Big Geo-Spatial Event Data on GPU
		10.1 Introduction
		10.2 GPU Computing
		10.3 Related Work
		10.4 Problem Statement
			10.4.1 Basic Concepts
			10.4.2 Problem Definition
		10.5 Approach
			10.5.1 Algorithm Overview
			10.5.2 Cell-Aggregate-Based Upper Bound Filter
			10.5.3 Refinement Algorithms
		10.6 Evaluation
			10.6.1 Results on Synthetic Data
				10.6.1.1 Effect of the Number of Instances
				10.6.1.2 Effect of Clumpiness
				10.6.1.3 Comparison on Filter and Refinement
			10.6.2 Results on Real World Dataset
				10.6.2.1 Effect of Minimum Participation Index Threshold
				10.6.2.2 Comparison of Filter and Refinement
		10.7 Discussion and Conclusion
		References
	11 Automatic Urban Road Network Extraction From Massive GPS Trajectories of Taxis
		11.1 Introduction
		11.2 Literature Review
			11.2.1 Density-Based Approaches
			11.2.2 Cluster-Based Approaches
		11.3 Methodology
			11.3.1 Trajectory Compression
			11.3.2 Identification of the Trajectory Points Along the Road
				11.3.2.1 Density-Based Spatial Clustering of Applications with Noise (DBSCAN)
				11.3.2.2 Anisotropic Perspective on Local Point Density
				11.3.2.3 Anisotropic Density-Based Clusters with Noise (ADCN) Algorithm
				11.3.2.4 ADCN Algorithm in Road Network Extraction
			11.3.3 Road Network Generation
				11.3.3.1 Road Density Surface Generation
				11.3.3.2 Collapse Surface to Centerline
		11.4 Case Study
			11.4.1 Data
			11.4.2 Experiment
				11.4.2.1 Evaluation Metrics
				11.4.2.2 Results
		11.5 Conclusion and Future Work
		References
	12 Exploratory Analysis of Massive Movement Data
		12.1 Introduction
		12.2 Movement Data Characteristics & Their Relation to Big Data Vs
			12.2.1 Variety
			12.2.2 Velocity & Volume
		12.3 Exploratory Data Analysis (EDA)
		12.4 EDA Tasks for Massive Movement Data
			12.4.1 Task 1: Spatio-Temporal Lookup or Range Queries
				12.4.1.1 Challenge 1: Trajectory Indexing
				12.4.1.2 Challenge 2: Spatio-Temporal Visualizations of Massive Movement Data
			12.4.2 Task 2: Similar Trajectory Search and Join
				12.4.2.1 Challenge 3: Building and Segmenting Trajectories
				12.4.2.2 Challenge 4: Moving Object Identifiers
			12.4.3 Task 3: Density Mapping and Other Grid-Based Summarizations
				12.4.3.1 Challenge 5: Representativeness & Bias
			12.4.4 Task 4: Extracting Events & Places
				12.4.4.1 Challenge 6: Data Quality or Veracity
			12.4.5 Task 5: Detection of Outliers and Anomalies
				12.4.5.1 Challenge 7: Anomaly Detection Performance
		12.5 Privacy
			12.5.1 k-Anonymity
			12.5.2 Differential Privacy
			12.5.3 Privacy by Design
		12.6 Recommended EDA Workflow for Massive Movement Data
			12.6.1 Establishing an Overview
			12.6.2 Putting Movement Records in Context
			12.6.3 Extracting Trajectories & Events
			12.6.4 Exploring Patterns in Trajectory and Event Data
			12.6.5 Analyzing Outliers
		12.7 Conclusions
		References
Part III Statistics, Uncertainty and Data Quality
	13 Spatio-Temporal Data Quality: Experience from Provision of DOT Traveler Information
		13.1 Introduction
		13.2 Example Data Quality Problems
		13.3 Data Quality Attributes
		13.4 Data Quality Assessment Methods
		13.5 Enhanced Methods
			13.5.1 General Definitions
			13.5.2 General Approach
			13.5.3 Interpolation to Model Ground Truth
			13.5.4 Our SMART Approach
			13.5.5 Artificial Data Set
			13.5.6 Evaluation
			13.5.7 Evaluation Using an Artificial Data Set
			13.5.8 December 2015 MADIS California Data
			13.5.9 December 2017 MADIS Montana Data
			13.5.10 December 2015–2017 USGS Streamflow Data
			13.5.11 Evaluation Summary
		13.6 Further Research and Development Topics
		Bibliography
	14 Uncertain Spatial Data Management: An Overview
		14.1 Introduction
		14.2 Discrete and Continuous Models for Uncertain Data
			14.2.1 Existing Models for Uncertain Data
			14.2.2 Discrete Models
			14.2.3 Continuous Models
		14.3 Possible World Semantics
		14.4 Existing Uncertain Spatial Database Management Systems
		14.5 Probabilistic Result Semantics
			14.5.1 Object Based Probabilistic Result Semantics
			14.5.2 Result Based Probabilistic Result Semantics
		14.6 Probabilistic Query Predicates
			14.6.1 Probabilistic Threshold Queries
			14.6.2 Probabilistic Topk Queries
			14.6.3 Discussion
		14.7 The Paradigm of Equivalent Worlds
			14.7.1 Equivalent Worlds
			14.7.2 Exploiting Equivalent Worlds for Efficient Algorithms
		14.8 Case Study: Range Queries and the Sum of Independent Bernoulli Trials
			14.8.1 Poisson-Binomial Recurrence
				14.8.1.1 Complexity Analysis
			14.8.2 Generating Functions
				14.8.2.1 Complexity Analysis
		14.9 Advanced Techniques for Managing Uncertain Spatial Data
		14.10 Summary
		References
	15 Spatial Statistics, or How to Extract Knowledge from Data
		15.1 Introduction
		15.2 Spatial Data
		15.3 Geostatistical Models
			15.3.1 Covariogram Estimation
			15.3.2 Modeling Approaches
			15.3.3 Dimensionality Reduction of the Spatial Covariance Matrix
		15.4 Spatial Regression Models
			15.4.1 Specification of Spatial Weighting Matrices
			15.4.2 Inferences on Parameter Estimates
			15.4.3 Estimation Procedures
		15.5 Case Study
		15.6 Conclusion
		15.7 Further Reading
		References
Part IV Information Retrieval from Multimedia Spatial Datasets
	16 A Survey of Textual Data & Geospatial Technology
		16.1 Introduction
		16.2 Research Questions & Different Notions of ``Where''
		16.3 Spatial Indexing
			16.3.1 Spatial Data Structures
			16.3.2 Spatially Enabled Database Management Systems
		16.4 Address Geocoding
		16.5 Geoparsing and Spatial Resolution
			16.5.1 Toponym Resolution
			16.5.2 Geospatial Expression Resolution
		16.6 Content Enrichment with Geospatial Metadata
		16.7 Hybrid Textual/Spatial Document Retrieval
		16.8 Geofencing
		16.9 Applications
			16.9.1 Location Search
			16.9.2 Crime Mapping, Hotspot Analysis and Forecasting
			16.9.3 Political Anaysis and Intelligence Applications
			16.9.4 Healthcare Applications
			16.9.5 Location-Based Services and Location-Aware Advertising
			16.9.6 Other Applications
		16.10 Summary, Conclusion and Future Work
		Appendix: Ancillary Tasks
		Augmenting Gazetteers via Web Mining
		Curating Gold Standard Data for Evaluation and Training
		Bibliography
		References
	17 Harnessing Heterogeneous Big Geospatial Data
		17.1 Introduction
		17.2 Geospatial Data Conflation
		17.3 Geospatial Data Integration
		17.4 Geospatial Data Enrichment
		17.5 Summary
		References
	18 Big Historical Geodata for Urban and Environmental Research
		18.1 Introduction
		18.2 Data Sources and Time Spans
		18.3 From the Data Source to Big Geospatial Data
		18.4 Potentials of Big Historical Geodata
			18.4.1 Human-Environment Interactions
			18.4.2 Land Change Model Calibration
			18.4.3 Data-Driven Geoscience and Geodata Science
			18.4.4 Digital Humanities and Cultural Heritage
			18.4.5 Urban Research and Spatial Planning
		18.5 Conclusion
		References
	19 Harvesting Big Geospatial Data from Natural Language Texts
		19.1 Introduction and Motivation
		19.2 Methods and Tools
			19.2.1 Toponym Recognition
			19.2.2 Toponym Resolution
			19.2.3 Developed Geoparsers and Tools
			19.2.4 Location Inference from Language Modeling
			19.2.5 Summary
		19.3 Applications of Geospatial Data Harvested from Texts
			19.3.1 Understanding Places and Human Experiences
			19.3.2 Situation Awareness for Emergency Response
			19.3.3 Place Relations in Virtual or Cognitive Space
		19.4 Summary and Future Directions
		References
	20 Automating Information Extraction from Large Historical Topographic Map Archives: New Opportunities and Challenges
		20.1 Introduction
		20.2 Digital Historical Map Archives
		20.3 Preprocessing Methods
			20.3.1 Automated Georeferencing
			20.3.2 Spatial Data Alignment
			20.3.3 Exploratory Methods
		20.4 Automated Map Content Recognition and Extraction
			20.4.1 Training Data Collection
			20.4.2 Recognition and Extraction Methods
		20.5 Conclusions and Outlook
		References
Part V Governance, Infrastructures and Society
	21 The Integration of Decision Maker's Requirements to Develop a Spatial Data Warehouse
		21.1 Introduction
		21.2 Overview of the Existing Approaches
		21.3 Overview of the Proposal
		21.4 GeoCIM Definition
		21.5 Classification of the GeoCIMs Models
		21.6 K == Random Number of the Clusters Containing Adjacent Objects
		21.7 From GeoCIM to GeoPIM
			21.7.1 GeoPIM Definition
			21.7.2 Formal Transformations from GeoCIM to GeoPIM
		21.8 Using Topological Relationships to Enrich Dimension Hierarchies
			21.8.1 Geo SM Definition
		21.9 Transformations from GeoPIM to GeoPSM
		21.10 Experimentation
			21.10.1 Transition from the Requirements Model to the Implementation Model of a SDW
		21.11 Case Study
		21.12 Evaluation of the Proposal
		21.13 Conclusion
		References
	22 Smart Cities
		22.1 Introduction
		22.2 History and Background: A Brief Review
		22.3 Defining Smart Cities in Practice
		22.4 Context Variables Affecting Smart Cities
			22.4.1 Structural Factors
			22.4.2 Economic Development
			22.4.3 Technology
			22.4.4 Effective Environmentally-Progressive Governance
		22.5 The Role of Data
			22.5.1 Smart City and Big Data
			22.5.2 Real-Time Data
			22.5.3 Open Government Data
			22.5.4 The Semantic Web and Linked Open Data
				22.5.4.1 OpenStreetMap
				22.5.4.2 GeoNames
		22.6 Examples of Smart Cities
		22.7 Future Directions
		22.8 Conclusion
		References
	23 The 4th Paradigm in Multiscale Data Representation: Modernizing the National Geospatial Data Infrastructure
		23.1 Access to Nationally Managed Spatial Data in the United States
		23.2 Chronology and Current Status of NSDI in the United States
			23.2.1 Geospatial Interoperability Reference Architecture (GIRA)
			23.2.2 Geospatial Platform
			23.2.3 Cloud Computing
			23.2.4 Multiagency Geospatial Acquisition
		23.3 The Role of the Fourth Paradigm
		23.4 Activities for Short- and Longer-Term NSDI Implementation
			23.4.1 Short-Term Goals: Integrate NSDI Across Spatial and Temporal Scales
			23.4.2 Longer-Term Goals: Aligning NSDI with User Needs and Demands
		23.5 Implications and Prospects of the Fourth Paradigm for the NSDI
		References
	24 INSPIRE: The Entry Point to Europe's Big Geospatial Data Infrastructure
		24.1 Introduction
		24.2 Big Data in the EU
		24.3 INSPIRE State of Play
			24.3.1 Legal, Technical and Organisational Framework
			24.3.2 INSPIRE Geoportal
		24.4 Inspire as a Big Data Infrastructure
			24.4.1 Characteristics of INSPIRE in Terms of Big Data
			24.4.2 Challenges from the User Perspective
				24.4.2.1 Discoverability of Datasets
				24.4.2.2 Combining National Datasets to Create Pan-European Products
				24.4.2.3 Data Access and Consumption by Clients
				24.4.2.4 Cloud Infrastructures
		24.5 Conclusions and Outlook
		References