DevOps for Databases: A practical guide to applying DevOps best practices to data-persistent technologies

Cover
Copyright
Contributors
Table of Contents
Preface
Part 1:Database DevOps
Chapter 1: Data at Scale with DevOps
	The modern data landscape
		How do we generate data today?
	Why speed matters
	Data management strategies
	The early days of DevOps
	SRE versus DevOps
	Engineering principles
	Objectives – SLOs/SLIs
	Summary
Chapter 2: Large-Scale Data-Persistent Systems
	A brief history of data
		The early days of computing
		The rise of relational databases
		Conclusion
	Database evolution
		Hierarchical database models
		Network database model
		Relational databases
		OO databases
		NoSQL database paradigms
	Data warehouses
		Architecture
		Data modeling
		Integration
	Data lakes
		Architecture
		Data ingestion and processing
		Storage and retrieval
		Security considerations
		Conclusion
	Summary
Chapter 3: DBAs in the World of DevOps
	The continuously evolving role of the DBA
		The rise of data architecture and integration
	The emergence of cloud computing and big data
		The shift to DevOps
		Conclusion
	DevOps and DBAs
	The role of the database expert in a DevOps-conscious team
		Designing and implementing databases
		Ensuring high availability and disaster recovery
		Performance tuning
		Security and compliance
		Automation
	A proven methodology with quantifiable benefits
		Faster time to market
		Improved quality
		Reduced downtime
		Increased scalability
		Improved security
	Summary
Part 2:Persisting Data in the Cloud
Chapter 4: Cloud Migration and Modern Data(base) Evolution
	What is cloud migration (and why are companies doing it)?
		The importance of cloud migration
		Steps to keep in mind before cloud migration
		Key milestones through cloud migration
	Types of cloud migrations
		Lift-and-shift migration
		Lift-and-reshape migration
		Refactoring migration
		Hybrid cloud migration
		Multi-cloud migration
	The process of cloud migration
		Monolithic or distributed database systems?
	What can a database expert help with during cloud migration?
	Data migration strategies and their types
		The Big Bang migration strategy
		The phased migration strategy
		The parallel migration strategy
		The hybrid migration strategy
		The reverse migration strategy
		The ETL strategy
		The replication strategy
	Why are data migration strategies important during a database cloud migration project?
		Taking your migration journey one step further
	Summary
Chapter 5: RDBMS with DevOps
	Embracing DevOps
		Provisioning and configuration management
		Monitoring and alerting
		Backup and disaster recovery
		Performance optimization
		DevSecOps
	Summary
Chapter 6: Non-Relational DMSs with DevOps
	Activities and challenges
	Data modeling
		Denormalization
		Nested and dynamic data
		Data denormalization
	Schema management
		Schemaless data modeling
		Dynamic schema evolution
		Consistency and concurrency control
	Deployment automation
		Deployment of multiple database engines
		Backup and disaster recovery
		Capacity planning and scaling
	Performance tuning
		Data modeling for performance
		Distributed query optimization
		Network latency and data transfer
	Data consistency
		Lack of transactions
		Eventual consistency
		Data sharding
	Security
		Limited access control
		Distributed denial of service attacks
		Lack of encryption
	Anti-patterns (what not to do…)
		Overusing or misusing denormalization
		Ignoring or underestimating data consistency
		Failing to secure a database
		Overlooking performance tuning
		Neglecting to plan for growth
	Summary
Chapter 7: AI, ML, and Big Data
	Definitions and applications of AI, ML, and big data
		The relationship between AI, ML, and big data
		The role of DevOps and engineering in AI, ML, and big data
		Challenges of AI, ML, and big data
	A deep dive into big data as a DevOps data expert
		Big data infrastructure
		Challenges with big data
	A deep dive into ML as a DevOps data expert
		How ML works
		How ML differs from traditional software applications
		Challenges with ML for a DevOps data expert
	A deep dive into AI as a DevOps data- expert
		Amazon SageMaker
		Google Cloud AI platform
		Microsoft Azure Machine Learning
		Challenges with AI for a DevOps data expert
	Summary
Part 3:The Right Tool for the Job
Chapter 8: Zero-Touch Operations
	Traditional versus zero-touch approaches
		Automated configuration management
		Automated release management
		Automated monitoring and alerting
	Increased operational efficiency
		Automated database provisioning
		Automated backup and recovery
	Improved reliability and consistency
		Automated configuration management
		Automated release management
	Accelerated deployment and time-to-market
		CI/CD pipelines
		IaC and orchestration
	Enhanced scalability and elasticity
		Automated resource provisioning
		Container orchestration
	Reduced downtime and faster recovery
		Automated monitoring and alerting
		Streamlined recovery processes
	Improved compliance and security
		Automated security configurations
		Automated compliance checks
	Sanity-checking our approach
		Conclusion on ROI
	Summary
Chapter 9: Design and Implementation
	Designing data-persistence technologies
		Database design principles
		RDBMS versus NoSQL versus NewSQL
	Implementing data-persistence technologies
		Installation, configuration, and management of database systems
		Practical example – PostgreSQL database server installation, configuration, and management
		Disaster recovery planning
		Practical example – MongoDB replication and automatic failover
	Database provisioning and Infrastructure as Code
		Practical example – using Terraform to script the setup of a SQL Server database
	Database version control and CI/CD
		Importance of database version control
		Practical example – using Liquibase to manage database schema changes
		Role of the DevOps DBA in CI/CD pipelines
		Practical example – Jenkins pipeline with database migrations using Flyway
	Database performance tuning
		Importance of performance tuning and common strategies
		Practical example – optimizing a poorly performing query in Oracle
	Security and compliance
		Importance of security in database management
		Common security measures
		Practical example – best practices for securing a MySQL database and ensuring GDPR compliance
	Collaboration and communication
	Summary
Chapter 10: Database Automation
	Autonomous database management
		Self-driving databases – a new horizon in DBMs
		Automation of database administration tasks
		The implications of self-driving databases
		Challenges and future directions
		Conclusion
	The revolution of performance tuning – from manual to autonomous
		Understanding performance tuning
		The need for automated performance tuning
		Technological foundations of automated performance tuning
		The mechanics of automated performance tuning
		The implications of automated performance tuning
		Challenges and future directions
		Conclusion
	Automated data lineage tracking – a new era of transparency in data management
		Understanding data lineage
		The evolution from manual to automated data lineage tracking
		The technological foundations of automated data lineage tracking
		The process of automated data lineage tracking
		The implications of automated data lineage tracking
		Challenges and future directions
		Conclusion
	Data privacy automation – advancing the frontier of privacy compliance in the digital age
		Understanding data privacy
		The challenges of data privacy
		Data masking and anonymization
		The advent of data privacy automation
		Technological underpinnings of data privacy automation
		The process of data privacy automation
		The benefits and implications of data privacy automation
		Conclusion
	Automated data discovery and cataloging – unveiling the hidden treasures in today’s data landscape
		Understanding data discovery and cataloging
		The growing need for automation in data discovery and cataloging
		What is automated data discovery and cataloging?
		The key features of automated data discovery and cataloging tools
		The process of automated data discovery and cataloging
		The benefits and implications of automated data discovery and cataloging
		Conclusion
	The ascendancy of DBaaS – transforming business efficiency and data utilization in the digital age
		Understanding DBaaS
		The mechanics of DBaaS
		The impact of DBaaS on business operations
		DBaaS – the future of database management
		Conclusion
	The emergence of serverless databases – revolutionizing DBaaS through on-demand scalability and cost efficiency
		Understanding serverless databases
		Why serverless databases? The driving forces
		How serverless databases work
		The impact of serverless databases on business operations
		The future of serverless databases
		Conclusion
	Summary
Part 4:Build and Operate
Chapter 11: End-to-End Ownership Model – a Theoretical Case Study
	End-to-end ownership – a case study
	Adoption of the end-to-end ownership model
	Setting the stage
		Project initiation
		Formation of cross-functional teams
		Defining end-to-end ownership
	Design and development phase
		Collaborative design and planning
		Agile development practices
		CI and continuous testing
	Deployment and release
		IaC
		CD pipelines
		Canary releases and feature flags
	Monitoring and IM
		Proactive monitoring and alerting
		IR and post-mortems
		Continuous improvement
	Feedback and iteration
		Gathering user feedback
		Prioritizing and implementing changes
		A/B testing and experiments
	Scaling and challenges
		Scaling the end-to-end ownership model
		Managing dependencies
		Balancing autonomy and alignment
	Summary
Chapter 12: Immutable and Idempotent Logic – A Theoretical Case Study
	Introduction to immutable and idempotent logic
	Immutable logic in data-persisting technologies
		Understanding immutability in the context of data storage
		Benefits and use cases of immutable data storage
		Examples of immutable data storage approaches
		Implementing immutable logic with data-persisting technologies
	Idempotent logic in data-persisting technologies
		Introduction to idempotent operations and their significance
		Examples of idempotent operations in data persistence
		Ensuring idempotency in data processing and transformations
	Practical examples and use cases
		Immutable and idempotent logic in relational databases
		Immutable and idempotent approaches in NoSQL databases
		Immutable and idempotent patterns in distributed storage systems
	Considerations and best practices
		Performance and scalability implications of immutable and idempotent approaches
		Data consistency and integrity considerations
		Handling failures and retries with idempotent logic
		Managing data evolution and schema changes with immutability
	Future trends and challenges
		Emerging technologies and advancements in data persistence
		Integrating immutable and idempotent logic with cloud-native architectures
		Addressing complexities and trade-offs in large-scale data persistence systems
	Summary
Chapter 13: Operators and Self-Healing Data Persistent Systems
	Self-healing systems
		Components of a self-healing system
		Importance of self-healing systems
		Risks and limitations
		Technical example of each core principle of a self-healing system
	Operators in Kubernetes
		Overview of Kubernetes and containerization
		Understanding operators
		Operator frameworks and ecosystem
		Benefits of operators in Kubernetes
		Use cases of operators in Kubernetes
	Self-healing databases
		Traditional database challenges
		Self-healing mechanisms in databases
		Benefits of self-healing databases
		Risks and limitations
	Factors influencing self-healing in different databases
		Relational databases
		NoSQL databases
		NewSQL databases
		Time-series databases
	Self-healing in Kubernetes – implementation and best practices
		Key components for self-healing in Kubernetes
		Implementing self-healing in Kubernetes – best practices
		Challenges and considerations
		Conclusion
	Case studies – self-healing databases in Kubernetes
		Case study 1 – MySQL Operator
		Case study 2 – MongoDB Operator
		Case study 3 – Cassandra Operator
	Benefits of self-healing databases in Kubernetes
	Challenges and future directions
		Challenges in self-healing systems
		Future directions
	Summary
Chapter 14: Bringing Them Together
	Alex’s AI journey
		Introduction and project assignment
		Software and infrastructure architecture decisions
		Relational versus non-relational databases
		Implementing caching, data lakes, and data warehouses
		Security concerns and solutions
		First status update
	Implementation
		Utilizing DevOps and SRE methodologies
		The power of immutable and idempotent logic
		Embracing zero-touch automation
		Update 2
		Implementing self-healing systems
		Implementing load balancers and scaling
		Update 3
	Observability and operations
		The art of canary deployments
		Database scaling
		Security and compliance in operations
		Update 4
		Version-controlled environment variables
		Update 5
	Lessons learned and future directions
	Summary
Part 5:The Future of Data
Chapter 15: Specializing in Data
	Mastering data – bridging the gap between IT and business
		Data-driven decision-making for business executives
		Building a data-driven culture – an enterprise perspective
	My first experience, Unix – 2009
	The first signs of DevOps – 2010s
		Support and software engineering in 2012
	My first SRE team – 2015
	Steep learning curves – 2017
	Putting it all into practice – 2019
	The landscape of 2023 – data and DevOps united
		Integration of DevOps with data engineering
		DataOps – revolutionizing data analytics pipelines
		MLOps – bridging the gap between ML development and operations
		AI-powered DevOps/SRE
		Application of SRE principles to data systems
		DevSecOps – security in the age of data
	Summary
Chapter 16: The Exciting New World of Data
	Part 1 – the future of data-persisting technologies
		The evolution of current data-persisting technologies
		Emerging data-persisting technologies
		Future use cases and challenges
	Part 2 – anticipated changes in AI/ML DevOps
		Advancements in MLOps
		Future use cases and challenges
		Career impacts and future skill requirements
	Part 3 – evolving trends in SRE
		Changes in the SRE landscape
		The role of SRE in future IT operations
	Part 4 – career outlook and emerging skill sets in SRE
		SRE careers – opportunities, skills, and preparation
		Career opportunities in SRE
		Skill set requirements
		Preparing for a career in SRE
		Innovations in data persistence technologies
		The current state of data persistence technologies
		Future outlook – next-generation data persistence technologies
		Preparing for the future – skills and strategies in data persistence technologies
	Part 5 – the future of designing, building, and operating cutting-edge systems
		Emerging technologies in system design and development
		Potential implications and challenges in designing, building, and operating cutting-edge systems
		Strategies for success – preparing for the future of designing, building, and operating cutting-edge systems
	Summary
Index
Other Books You May Enjoy