关于此课程： Data analysis has replaced data acquisition as the bottleneck to evidence-based decision making --- we are drowning in it. Extracting knowledge from large, heterogeneous, and noisy datasets requires not only powerful computing resources, but the programming abstractions to use them effectively. The abstractions that emerged in the last decade blend ideas from parallel databases, distributed systems, and programming languages to create a new class of scalable data analytics platforms that form the foundation for data science at realistic scales. In this course, you will learn the landscape of relevant systems, the principles on which they rely, their tradeoffs, and how to evaluate their utility against your requirements. You will learn how practical systems were derived from the frontier of research in computer science and what systems are coming on the horizon. Cloud computing, SQL and NoSQL databases, MapReduce and the ecosystem it spawned, Spark and its contemporaries, and specialized systems for graphs and arrays will be covered. You will also learn the history and context of data science, the skills, challenges, and methodologies the term implies, and how to structure a data science project. At the end of this course, you will be able to: Learning Goals: 1. Describe common patterns, challenges, and approaches associated with data science projects, and what makes them different from projects in related fields. 2. Identify and use the programming models associated with scalable data manipulation, including relational algebra, mapreduce, and other data flow models. 3. Use database technology adapted for large-scale analytics, including the concepts driving parallel databases, parallel query processing, and in-database analytics 4. Evaluate key-value stores and NoSQL systems, describe their tradeoffs with comparable systems, the details of important examples in the space, and future trends. 5. “Think” in MapReduce to effectively write algorithms for systems including Hadoop and Spark. You will understand their limitations, design details, their relationship to databases, and their associated ecosystem of algorithms, extensions, and languages. write programs in Spark 6. Describe the landscape of specialized Big Data systems for graphs, arrays, and streams
制作方： University of Washington
教学方： Bill Howe, Director of Research
Scalable Data Analytics
| 基本信息 | 课程 1（共 4 门，Data Science at Scale Specialization ） |
| 承诺学习时间 | 4周课程，6-8小时/周 |
| 语言 | English |
| 如何通过 | 通过所有计分作业以完成课程。 |
| 用户评分 |
授课大纲
第 1 周
Data Science Context and Concepts
Understand the terminology and recurring principles associated with data science, and understand the structure of data science projects and emerging methodologies to approach them. Why does this emerging field exist? How does it relate to other fields? How does this course distinguish itself? What do data science projects look like, and how should they be approached? What are some examples of data science projects?
22 视频, 4 阅读材料
- 视频: Appetite Whetting: Extreme Weather
- 视频: Appetite Whetting: Digital Humanities
- 视频: Appetite Whetting: Bibliometrics
- 视频: Appetite Whetting: Food, Music, Public Health
- 视频: Appetite Whetting: Public Health cont'd, Earthquakes, Legal
- 视频: Characterizing Data Science
- 视频: Characterizing Data Science, cont'd
- 视频: Distinguishing Data Science from Related Topics
- 视频: Four Dimensions of Data Science
- 视频: Tools vs. Abstractions
- 视频: Desktop Scale vs. Cloud Scale
- 视频: Hackers vs. Analysts
- 视频: Structs vs. Stats
- 视频: Structs vs. Stats cont'd
- 视频: A Fourth Paradigm of Science
- 视频: Data-Intensive Science Examples
- 视频: Big Data and the 3 Vs
- 视频: Big Data Definitions
- 视频: Big Data Sources
- Leyendo: Supplementary: Three-Course Reading List
- Leyendo: Supplementary: Resources for Learning Python
- 视频: Course Logistics
- Leyendo: Supplementary: Class Virtual Machine
- Leyendo: Supplementary: Github Instructions
- 视频: Twitter Assignment: Getting Started
已评分: Twitter Sentiment Analysis
第 2 周
Relational Databases and the Relational Algebra
Relational Databases are the workhouse of large-scale data management. Although originally motivated by problems in enterprise operations, they have proven remarkably capable for analytics as well. But most importantly, the principles underlying relational databases are universal in managing, manipulating, and analyzing data at scale. Even as the landscape of large-scale data systems has expanded dramatically in the last decade, relational models and languages have remained a unifying concept. For working with large-scale data, there is no more important programming model to learn.
24 视频
- 视频: From Data Models to Databases
- 视频: Pre-Relational Databases
- 视频: Motivating Relational Databases
- 视频: Relational Databases: Key Ideas
- 视频: Algebraic Optimization Overview
- 视频: Relational Algebra Overview
- 视频: Relational Algebra Operators: Union, Difference, Selection
- 视频: Relational Algebra Operators: Projection, Cross Product
- 视频: Relational Algebra Operators: Cross Product cont'd, Join
- 视频: Relational Algebra Operators: Outer Join
- 视频: Relational Algebra Operators: Theta-Join
- 视频: From SQL to RA
- 视频: Thinking in RA: Logical Query Plans
- 视频: Practical SQL: Binning Timeseries
- 视频: Practical SQL: Genomic Intervals
- 视频: User-Defined Functions
- 视频: Support for User-Defined Functions
- 视频: Optimization: Physical Query Plans
- 视频: Optimization: Choosing Physical Plans
- 视频: Declarative Languages
- 视频: Declarative Languages: More Examples
- 视频: Views: Logical Data Independence
- 视频: Indexes
已评分: SQL for Data Science Assignment
第 3 周
MapReduce and Parallel Dataflow Programming
The MapReduce programming model (as distinct from its implementations) was proposed as a simplifying abstraction for parallel manipulation of massive datasets, and remains an important concept to know when using and evaluating modern big data platforms.
26 视频
- 视频: A Sketch of Algorithmic Complexity
- 视频: A Sketch of Data-Parallel Algorithms
- 视频: "Pleasingly Parallel" Algorithms
- 视频: More General Distributed Algorithms
- 视频: MapReduce Abstraction
- 视频: MapReduce Data Model
- 视频: Map and Reduce Functions
- 视频: MapReduce Simple Example
- 视频: MapReduce Simple Example cont'd
- 视频: MapReduce Example: Word Length Histogram
- 视频: MapReduce Examples: Inverted Index, Join
- 视频: Relational Join: Map Phase
- 视频: Relational Join: Reduce Phase
- 视频: Simple Social Network Analysis: Counting Friends
- 视频: Matrix Multiply Overview
- 视频: Matrix Multiply Illustrated
- 视频: Shared Nothing Computing
- 视频: MapReduce Implementation
- 视频: MapReduce Phases
- 视频: A Design Space for Large-Scale Data Systems
- 视频: Parallel and Distributed Query Processing
- 视频: Teradata Example, MR Extensions
- 视频: RDBMS vs. MapReduce: Features
- 视频: RDBMS vs. Hadoop: Grep
- 视频: RDBMS vs. Hadoop: Select, Aggregate, Join
已评分: Thinking in MapReduce
第 4 周
NoSQL: Systems and Concepts
NoSQL systems are purely about scale rather than analytics, and are arguably less relevant for the practicing data scientist. However, they occupy an important place in many practical big data platform architectures, and data scientists need to understand their limitations and strengths to use them effectively.
36 视频
- 视频: NoSQL Roundup
- 视频: Relaxing Consistency Guarantees
- 视频: Two-Phase Commit and Consensus Protocols
- 视频: Eventual Consistency
- 视频: CAP Theorem
- 视频: Types of NoSQL Systems
- 视频: ACID, Major Impact Systems
- 视频: Memcached: Consistent Hashing
- 视频: Consistent Hashing, cont'd
- 视频: DynamoDB: Vector Clocks
- 视频: Vector Clocks, cont'd
- 视频: CouchDB Overview
- 视频: CouchB Views
- 视频: BigTable Overview
- 视频: BigTable Implementation
- 视频: HBase, Megastore
- 视频: Spanner
- 视频: Spanner cont'd, Google Systems
- 视频: MapReduce-based Systems
- 视频: Bringing Back Joins
- 视频: NoSQL Rebuttal
- 视频: Almost SQL: Pig
- 视频: Pig Architecture and Performance
- 视频: Data Model
- 视频: Load, Filter, Group
- 视频: Group, Distinct, Foreach, Flatten
- 视频: CoGroup, Join
- 视频: Join Algorithms
- 视频: Skew
- 视频: Other Commands
- 视频: Evaluation Walkthrough
- 视频: Review
- 视频: Context
- 视频: Spark Examples
- 视频: RDDs, Benefits
Graph Analytics
Graph-structured data are increasingly common in data science contexts due to their ubiquity in modeling the communication between entities: people (social networks), computers (Internet communication), cities and countries (transportation networks), or corporations (financial transactions). Learn the common algorithms for extracting information from graph data and how to scale them up.
21 视频
- 视频: Structural Analysis
- 视频: Degree Histograms, Structure of the Web
- 视频: Connectivity and Centrality
- 视频: PageRank
- 视频: PageRank in more Detail
- 视频: Traversal Tasks: Spanning Trees and Circuits
- 视频: Traversal Tasks: Maximum Flow
- 视频: Pattern Matching
- 视频: Querying Edge Tables
- 视频: Relational Algebra and Datalog for Graphs
- 视频: Querying Hybrid Graph/Relational Data
- 视频: Graph Query Example: NSA
- 视频: Graph Query Example: Recursion
- 视频: Evaluation of Recursive Programs
- 视频: Recursive Queries in MapReduce
- 视频: The End-Game Problem
- 视频: Representation: Edge Table, Adjacency List
- 视频: Representation: Adjacency Matrix
- 视频: PageRank in MapReduce
- 视频: PageRank in Pregel
常见问题解答
运作方式
Trabajo del curso
Cada curso es como un libro de texto interactivo, con videos pregrabados, cuestionarios y proyectos.
Ayuda de tus compañeros
Conéctate con miles de estudiantes y debate ideas y materiales del curso, y obtén ayuda para dominar los conceptos.
Certificados
Obtén reconocimiento oficial por tu trabajo y comparte tu éxito con amigos, compañeros y empleadores.
制作方
University of Washington
Founded in 1861, the University of Washington is one of the oldest state-supported institutions of higher education on the West Coast and is one of the preeminent research universities in the world.
价格
| 购买课程 | |
|---|---|
| 访问课程材料 | 可用 |
| 访问评分的材料 | 可用 |
| 收到最终成绩 | 可用 |
| 获得可共享的课程证书 | 可用 |
评分和审阅
A very good introduction to skills needed for applying data science ideas on large scale data problems.
Thanks for this course.True Parallel computing example would have made it even more awesome .
Very good class - the assignments were pretty uninteresting, though.
Very good course for understanding the underlying logic behind emerging big data technologies