Big Data: Techniques and Technologies in Geoinformatics

Cover
Half Title
Title Page
Copyright Page
Table of Contents
Editor
Contributors
Preface
Chapter 1: Distributed and Parallel Computing
	1.1 Introduction
	1.2 Distributed Computing
		1.2.1 Cluster Computing
			1.2.1.1 Architecture
			1.2.1.2 Data and Message Communication
			1.2.1.3 Task Management and Administration
			1.2.1.4 Example Geospatial Big Data Project on Cluster
		1.2.2 Grid Computing
			1.2.2.1 Architecture
			1.2.2.2 Types of Grid Architectures
			1.2.2.3 Topology
			1.2.2.4 Perspectives
				1.2.2.4.1 User
				1.2.2.4.2 Administrator
				1.2.2.4.3 Application Developer
			1.2.2.5 Example Geospatial Big Data Project on Grids
		1.2.3 Cloud Computing
			1.2.3.1 Taxonomies
			1.2.3.2 Cloud Service Models
			1.2.3.3 Cloud Deployment Models
				1.2.3.3.1 Cloud APIs
				1.2.3.3.2 Levels of Cloud APIs
				1.2.3.3.3 Categories of APIs
			1.2.3.4 Example Geospatial Big Data Project on Clouds
	1.3 Parallel Computing
		1.3.1 Classes of Parallel Computing
		1.3.2 Shared Memory Multiple Processing
		1.3.3 Distributed Memory Multiple Processing
		1.3.4 Hybrid Distributed Shared Memory
		1.3.5 Example Geospatial Big Data Project on Parallel Computers
	1.4 Supercomputing
		1.4.1 Supercomputing Worldwide
		1.4.2 Trend
		1.4.3 Future Supercomputer Research
		1.4.4 Example Geospatial Big Data Project on Supercomputers
	1.5 XSEDE: A Single Virtual System
		1.5.1 Resources
		1.5.2 Services
		1.5.3 Example Big Geospatial Data Project on XSEDE
	1.6 Choosing Appropriate Computing Environment
	1.7 Summary
	References
Chapter 2: GEOSS Clearinghouse Integrating Geospatial Resources to Support the Global Earth Observation System of Systems
	2.1 Introduction
	2.2 Catalog and Clearinghouse Research Review
		2.2.1 Metadata Repository and Standardized Metadata
		2.2.2 Catalog and Clearinghouse Based on Service-Oriented Architecture and Standard Services
		2.2.3 Semantic-Based Metadata Sharing and Data Discovery
	2.3 Technological Issues and Solutions
		2.3.1 Interoperability
		2.3.2 Provenance and Updating
		2.3.3 System Performance
		2.3.4 Timely Updating
	2.4 Design and Implementation of CLH
		2.4.1 Architecture
		2.4.2 Administration, User, and Group Management
		2.4.3 Harvesting
		2.4.4 Metadata Standards and Transformation
		2.4.5 User Interface and Programming APIs
			2.4.5.1 Search through Web Graphics User Interface
			2.4.5.2 Remote Search
	2.5 Usage and Operational Status
		2.5.1 System Operations
		2.5.2 System Metadata Status
		2.5.3 Usage
	2.6 Big Geospatial Data Challenges and Solutions
		2.6.1 Data and Database
		2.6.2 Distribution and Cloud Computing
		2.6.3 Searching Performance and Index
	2.7 Summary and Future Research
	Acknowledgments
	Notes
	References
Chapter 3: Using a Cloud Computing Environment to Process Large 3D Spatial Datasets
	3.1 Introduction
		3.1.1 Big Spatial Data
		3.1.2 Need for Cloud Computing Environment
	3.2 Methodology
		3.2.1 Iowa LiDAR Database
		3.2.2 CLiPS Design and Implementation
	3.3 Results
		3.3.1 Application Example: DEM Generation Using Large LiDAR Datasets
		3.3.2 Heuristic Models Development
	3.4 Conclusions
	Acknowledgment
	References
Chapter 4: Building Open Environments to Meet Big Data Challenges in Earth Sciences
	4.1 Introduction
	4.2 Technology Foundation and Methodology
		4.2.1 Interoperability
		4.2.2 Serviceability
		4.2.3 Infrastructure
	4.3 Discussions
	4.4 Summary
	References
Chapter 5: Developing Online Visualization and Analysis Services for NASA Satellite-Derived Global Precipitation Products during the Big Geospatial Data Era
	5.1 Introduction
	5.2 Overview of Global Precipitation Products and Data Services
		5.2.1 TRMM Background
		5.2.2 TRMM Products
		5.2.3 TRMM Data Services
		5.2.4 Global Precipitation Measurement Mission
	5.3 Big Data Challenges and Solutions
		5.3.1 Big Data Challenges
		5.3.2 Solutions
	5.4 A Prototype
		5.4.1 Data
		5.4.2 System Description
		5.4.3 Examples
	5.5 Conclusions
	Acknowledgment
	References
Chapter 6: Algorithmic Design Considerations for Geospatial and/or Temporal Big Data
	6.1 Motivation
		6.1.1 Challenges
			6.1.1.1 Algorithmic Time Complexity
			6.1.1.2 Algorithmic Space Complexity
	6.2 Geospatial Big Data Algorithms: The State of the Art
		6.2.1 Volume Algorithms
		6.2.2 Velocity Algorithms
		6.2.3 Variety Algorithms
	6.3 Analysis of Classical Geospatial and Temporal Algorithms
	6.4 Approaches to Algorithmic Adaptation for Geospatial Big Data
		6.4.1 Divide and Conquer
		6.4.2 Subsampling
		6.4.3 Aggregation
		6.4.4 Filtering
		6.4.5 Online Algorithms
		6.4.6 Streaming Algorithms
		6.4.7 Iterative Algorithms
		6.4.8 Relaxation
		6.4.9 Convergent Algorithms
		6.4.10 Stochastic Algorithms
		6.4.11 Batch versus Online Algorithms
		6.4.12 Dimensionality Reduction
		6.4.13 Example
	6.5 Open Challenges
	6.6 Summary
	References
Chapter 7: Machine Learning on Geospatial Big Data
	7.1 Motivation
		7.1.1 Supervised, Unsupervised, and Feature Learning
			7.1.1.1 Supervised Learning
			7.1.1.2 Unsupervised Learning
			7.1.1.3 Feature Learning
		7.1.2 Big Data Challenges
		7.1.3 Three Vs
			7.1.3.1 Volume
			7.1.3.2 Velocity
			7.1.3.3 Variety
	7.2 Geospatial Big Data Feature Learning
		7.2.1 Approaches to Big Data Feature Learning
	7.3 Reducing Dimensionality of Geospatial Big Data, Making Machine Learning Tractable
		7.3.1 Feature Construction
			7.3.1.1 Windowing in Raster Data
			7.3.1.2 Windowing in Time Series Geographic Data
			7.3.1.3 Big Data Feature Construction
		7.3.2 Dimensionality Reduction
			7.3.2.1 Feature Selection
			7.3.2.2 Feature Extraction
	7.4 Algorithmic Approaches to Machine Learning of Geospatial Big Data
		7.4.1 Space Complexity
			7.4.1.1 Online Learning
		7.4.2 Time Complexity
			7.4.2.1 Online Learning
			7.4.2.2 Ensemble Learning
	7.5 Conclusions
	Note
	References
Chapter 8: Spatial Big Data: Case Studies on Volume, Velocity, and Variety
	8.1 Introduction
	8.2 What Is Spatial Big Data?
	8.3 Volume: Discovering Sub-Paths in Climate Data
	8.4 Velocity: Spatial Graph Outlier Detection in Traffic Data
	8.5 Variety in Data Types: Identifying Bike Corridors
	8.6 Variety in Output: Spatial Network Activity Summarization
	8.7 Summary
	References
Chapter 9: Exploiting Big VGI to Improve Routing and Navigation Services
	9.1 Introduction
	9.2 What Is Big Data?
	9.3 VGI as Big Data
	9.4 Traditional Routing Services
	9.5 Routing Services using Big VGI/Crowdsourced Data
		9.5.1 Routing with Landmarks Extracted from Big VGI/Crowdsourced Data
		9.5.2 GPS Traces
		9.5.3 Social Media Reports
	9.6 Challenges for Exploiting Big VGI to Improve Routing Services
		9.6.1 Limitations of VGI and Crowdsourced Data
		9.6.2 Impact on the Development of Routing and Navigation Services
			9.6.2.1 Interoperability
			9.6.2.2 Finding the Right Data
			9.6.2.3 Analyzing and Interpreting Data
		9.6.3 Applicability of Big Data Solutions to Big VGI
	9.7 Summary
	References
Chapter 10: Efficient Frequent Sequence Mining on Taxi Trip Records Using Road Network Shortcuts
	10.1 Introduction
	10.2 Background, Motivation, and Related Work
	10.3 Prototype System Architecture
	10.4 Experiments and Results
		10.4.1 Results of BC on Original Sequences
		10.4.2 Results of Association Rule Mining on Original Sequences
		10.4.3 Results of the Proposed Approach
	10.5 Conclusion and Future Work
	Notes
	References
Chapter 11: Geoinformatics and Social Media: New Big Data Challenge
	11.1 Introduction: Social Media and Ambient Geographic Information
	11.2 Characteristics of Big Geosocial Data
	11.3 Geosocial Complexity
	11.4 Modeling and Analyzing Geosocial Multimedia: Heterogeneity and Integration
	11.5 Outlook: Grand Challenges and Opportunities for Big Geosocial Data
	Notes
	References
Chapter 12: Insights and Knowledge Discovery from Big Geospatial Data Using TMC-Pattern
	12.1 Introduction
	12.2 Trajectory Modeling
		12.2.1 TMC-Pattern
			12.2.1.1 Determining Meaningful Location
		12.2.2 Time Correlation
		12.2.3 Location Context Awareness
		12.2.4 Relevance Measures of a Region
		12.2.5 TMC-Pattern
			12.2.5.1 Determining Residence Mode of a Region
		12.2.6 Trajectory Extraction
	12.3 Trajectory Mining
		12.3.1 Frequent Locations from TMC-Pattern
		12.3.2 TMC-Pattern and Markov Chain for Prediction
			12.3.2.1 Markov Chains
			12.3.2.2 Markov Chain from TMC-Pattern
			12.3.2.3 Computation of Markov Chain Transition Probability
			12.3.2.4 Computation of Scores from TMC-Pattern
	12.4 Empirical Evaluations
		12.4.1 Experimental Dataset
		12.4.2 Evaluation of TMC-Pattern Extraction
			12.4.2.1 Single-User Data
			12.4.2.2 Multiuser Data
		12.4.3 Frequent Patterns
		12.4.4 Location Prediction
	12.5 Summary
	References
Chapter 13: Geospatial Cyberinfrastructure for Addressing the Big Data Challenges on the Worldwide Sensor Web
	13.1 Introduction
	13.2 Big Data Challenges on the Worldwide Sensor Web
	13.3 Worldwide Sensor Web Architecture
	13.4 GeoCENS Architecture
		13.4.1 OGC-Based Sensor Web servers
		13.4.2 Decentralized Hybrid P2P Sensor Web Service Discovery
	13.5 3D Virtual Globe-Based and 2D Map-Based Sensor Web Browsers
	13.6 Online Social Network
	13.7 Sensor Web Recommendation Engine
	13.8 Semantic Layer Service
	13.9 Applications Powered by GeoCENS
	13.10 Related Works
	13.11 Summary
	Acknowledgments
	Notes
	References
Chapter 14: OGC Standards and Geospatial Big Data
	14.1 What Is the OGC?
	14.2 Introduction
	14.3 What Is Observation (Sensor) Fusion?
	14.4 OGC Standards and Big Data
		14.4.1 OGC WMS Interface Standard
		14.4.2 OGC WCS Interface Standard
		14.4.3 OGC SOS Interface Standard
		14.4.4 OGC WPS Interface Standard
	14.5 Key Issues
		14.5.1 Issue One: Privacy
		14.5.2 Issue Two: Provenance
		14.5.3 Issue Three: Data Quality
	14.6 COBWEB: Sensor Fusion, Big Data, and Data Quality
	14.7 Summary
	Notes
	References
Chapter 15: Advanced Deep Learning Models and Algorithms for Spatial-Temporal Data
	15.1 Introduction
		15.1.1 Deep Learning
		15.1.2 Deep Learning for Spatial-Temporal Data
		15.1.3 Case Study: Climate Study
		15.1.4 Contribution
		15.1.5 Structure of Chapter
	15.2 Deep Learning Models for Spatial Patterns
		15.2.1 CNNs for Spatial Data
			15.2.1.1 Fundamentals of CNN Architecture
				15.2.1.1.1 Convolutional Layer
				15.2.1.1.2 Pooling Layer
				15.2.1.1.3 Fully Connected Layer
			15.2.1.2 Case Study
				15.2.1.2.1 Implementation Details
				15.2.1.2.2 Results
		15.2.2 GANs for Spatial Data
			15.2.2.1 Fundamentals of GANs Architecture
			15.2.2.2 Case Study
				15.2.2.2.1 Implementation Details
				15.2.2.2.2 Results
	15.3 Deep Learning Models for Spatial-Temporal Patterns
		15.3.1 Introduction
		15.3.2 Introduction to Temporal Pattern Models
			15.3.2.1 Recurrent Neural Networks
			15.3.2.2 Long Short-Term Memory Networks
			15.3.2.3 Gated Recurrent Units
		15.3.3 Combining Spatial and Temporal Models for Spatial-Temporal Patterns
			15.3.3.1 Fundamentals of ConvLSTM Architecture
			15.3.3.2 Case Study
				15.3.3.2.1 Implementation Details
				15.3.3.2.2 Data Sources
				15.3.3.2.3 Results
			15.3.3.3 Other Combinations
		15.3.4 Transformer for Spatial-Temporal Patterns
			15.3.4.1 Introduction
			15.3.4.2 Fundamentals of Transformer Architecture
			15.3.4.3 Case Study: Transformer for Spatial Patterns
			15.3.4.4 Case Study 2: Transformer for Spatial-Temporal Patterns
	15.4 Advanced Training Approach for Understanding Spatial-Temporal Patterns
		15.4.1 Transfer Learning
			15.4.1.1 Case Study
		15.4.2 Meta-Learning
			15.4.2.1 Case Study
		15.4.3 Contrastive Learning
			15.4.3.1 Case Study
	15.5 Summary
	Reference
Chapter 16: Deep Learning for Spatial Data: Heterogeneity and Adaptation
	16.1 Introduction
	16.2 Problem Definition
		16.2.1 Key Concepts
		16.2.2 Heterogeneity-Aware Learning: Two Sub-Tasks
		16.2.3 Sub-Task 1: Separation of Heterogeneous Processes via Space Partitioning
		16.2.4 Sub-Task 2: Adaptation to New Regions
	16.3 Related Work
	16.4 Separating Heterogeneous Processes: Spatial Transformation
		16.4.1 Representation: Hierarchical Multi-Task Learning
		16.4.2 Statistically-Guided Transformation
			16.4.2.1 Phase 1: Space-Partitioning Optimization with Prediction Error Distribution
			16.4.2.2 Phase 2: Active Significance Testing with Learning
			16.4.2.3 Spatial Transformation via a Dynamic and Learning-Engaged Hierarchy H
		16.4.3 Validation
	16.5 Adapting to New Regions
		16.5.1 Solution 1: A Spatial Moderator
		16.5.2 Validation for the Spatial Moderator
		16.5.3 Solution 2: A Context-Based Approach
			16.5.3.1 Modeling Spatial Contexts: A Point-to-Context Representation
			16.5.3.2 Co-Training Sequence
		16.5.4 Validation for the Context-Based Approach
	16.6 Conclusions
	16.7 Funding
	Notes
	References
Chapter 17: Assessing Multilevel Environmental and Air Quality Changes in Australia Pre- and Post-COVID-19 Lockdown: A Spatial Machine Learning Approach Utilizing Earth Observation Data
	17.1 Introduction
	17.2 Methods
		17.2.1 Study Area
		17.2.2 EO Satellite-Based Datasets
		17.2.3 Research Framework
		17.2.4 Spatial Analysis to Build Environmental Datasets
		17.2.5 Spatial Statistics and Spatial Machine Learning
	17.3 Results
		17.3.1 National Level Changes
			17.3.1.1 Distribution of Nationwide Land Surface and Air Quality Parameters Pre and Post Lockdown
			17.3.1.2 Associations between Environmental and Air Quality Parameters
		17.3.2 Regional Level Changes
			17.3.2.1 Comparison between Cities and Regional Areas (Rest of States)
		17.3.3 Local Level Changes
			17.3.3.1 Local Bivariate LST-NDVI Relation
			17.3.3.2 Spatially Clustering Analysis
	17.4 Discussions
	17.5 Conclusions
	Acknowledgment
	Appendix
	References
Chapter 18: Fairness-Aware Deep Learning in Space
	18.1 Introduction
	18.2 Key Concepts
	18.3 Fairness Enforcement through Bi-level Learning
	18.4 Fragility of Spatial Fairness under MAUP
		18.4.1 Key Instances
			18.4.1.1 Pure Fairness-Driven Learning
			18.4.1.2 Pure Bias-Injection Learning
			18.4.1.3 False Fairness-Preserving Learning
		18.4.2 Pure Fairness-Preserving Learning
		18.4.3 Pure Bias-Injection Learning
			18.4.3.1 Partitioning-Level Bias Injection
			18.4.3.2 Partition-Level Bias Injection
		18.4.4 False Fairness-Preserving Learning
			18.4.4.1 Partitioning Level
			18.4.4.2 Partition-Level
		18.4.5 Experiments
			18.4.5.1 Datasets
			18.4.5.2 Candidate Methods
			18.4.5.3 Results
				18.4.5.3.1 California Crop Mapping Dataset
				18.4.5.3.2 Palm Oil Plantation Mapping Dataset
	18.5 Time-Aware Spatial Fairness
		18.5.1 Problem Formulation and Preliminary
		18.5.2 Physics-Guided Sample Reweighting
		18.5.3 Fairness-Driven Model Refinement
		18.5.4 Experiments
			18.5.4.1 Dataset
			18.5.4.2 Experimental Design
			18.5.4.3 Results
	18.6 Conclusion
	Acknowledgment
	Notes
	References
Chapter 19: Integrating Large Language Models and Qualitative Spatial Reasoning
	19.1 Introduction
	19.2 Background
	19.3 Approach
		19.3.1 Motivating Example
		19.3.2 Datasets
		19.3.3 The GPT3.5 Large Language Model
	19.4 Experiment 1: Answering Spatial Queries Directly with GPT
		19.4.1 Discussion
	19.5 Experiment 2: GPT3.5’s Performance on SpRL Tasks
		19.5.1 Zero-Shot Learning
		19.5.2 Few-Shot Learning
		19.5.3 Evaluation
		19.5.4 Results
			19.5.4.1 Zero-Shot Learning
			19.5.4.2 Few-Shot Learning: Internal Validity
			19.5.4.3 Few-Shot Learning: External Validity
		19.5.5 Discussion
	19.6 Conclusion
	Note
	References
Chapter 20: Toward a Spatial Metaverse: Building Immersive Virtual Experiences with Georeferenced Digital Twin and Game Engine
	20.1 Introduction
		20.1.1 Motivation to Develop a Realistic, High-experiential City Model with Digital Twin
		20.1.2 New Opportunity to Build a Spatial Metaverse with Digital Twin, and VR Using Game Engine
		20.1.3 Aims and Objectives
	20.2 Methodology
		20.2.1 Conceptual and Methodological Framework of Spatial Metaverse Development
		20.2.2 3D Photorealistic Campus Model Reconstruction
			20.2.2.1 3D Photomesh Tiles Generation
			20.2.2.2 Creating an Integrated Mesh Scene Layer
		20.2.3 Building a Campus Digital Twin Using Game Engine
			20.2.3.1 Prototyping a Digital Twin in a Gamification Environment
			20.2.3.2 Connecting IoT Real-Time Data to Unity Environment
		20.2.4 Building Immersive Virtual Campus Experiences
			20.2.4.1 Game Avatar Character and Experience Development
			20.2.4.2 VR Experience Development
	20.3 Results
		20.3.1 3D Campus Photorealistic Model for Digital Twin
		20.3.2 3D Campus Photomesh and Integrated Scene Layer
			20.3.2.1 RMIT Campus 3D and Its Real-World Counterpart
				20.3.2.1.1 Composition and Specifics of the Campus 3D Model
		20.3.3 RMIT Campus Digital Twin User Interface
		20.3.4 Campus Immersive Virtue Experiences with Virtual Navigation and Wayfinding
	20.4 Discussion
		20.4.1 Paradigm Shift towards Realism and Experiential Richness
		20.4.2 Development and Usability of the RMIT Metaverse
		20.4.3 How Immersive Virtual Campus Experience Was Achieved
		20.4.4 Unlocking the Potential of Geospatial Data in the Spatial Metaverse
	20.5 Conclusions
	Acknowledgements
	References
Chapter 21: A Topological Machine Learning Approach with Multichannel Integration for Detecting Geospatial Objects
	21.1 Introduction
	21.2 Background
		21.2.1 Geospatial Object Detection
			21.2.1.1 ML-Based Object Detection
			21.2.1.2 Topological Data Analysis
		21.2.2 Topological ML
	21.3 Methods
		21.3.1 Extracting Topological Information from PH and Mapper
		21.3.2 Transforming Topological Information into a Multichannel Image
	21.4 Experimental Setup
	21.5 Results
	21.6 Discussion
	21.7 Conclusions
	References
Index