7 Applications into Humanities and Social Sciences Fields

Loading...

来自 Nanjing University 的课程

Data Processing Using Python

93 个评分

Nanjing University

Data Processing Using Python

93 个评分

This course (The English copy of "用Python玩转数据" <https://www.coursera.org/learn/hipython/home/welcome>) is mainly for non-computer majors. It starts with the basic syntax of Python, to how to acquire data in Python locally and from network, to how to present data, then to how to conduct basic and advanced statistic analysis and visualization of data, and finally to how to design a simple GUI to present and process data, advancing level by level. This course, as a whole, based on Finance data and through establishment of popular cases one after another, enables learners to more vividly feel the simplicity, elegance and robustness of Python. Also, it discusses the fast, convenient and efficient data processing capacity of Python in humanities and social sciences fields like literature, sociology and journalism and science and engineering fields like mathematics and biology, in addition to business fields. Similarly, it may also be flexibly applied into other fields. The course has been updated. Updates in the new version are : 1) the whole course has moved from Python 2.x to Python 3.x 2) Added manual webpage fetching and parsing. Web API is also added. 3) Improve the content order and enrich details of some content especially for some practice projects.

从本节课中

Python Data Statistics and Visualization

Welcome to learn Module 04 “Python Data Statistics and Visualization”! In this module, I will show you, over the entire process of data processing, the unique advantages of Python in data processing and analysis, and use many cases familiar to and loved by us to learn about and master methods and characteristics. After learning this module, you can fast and effectively mine your desired or expected or unknown results from a large amount of data, and can also present those data in various images. In addition, the data statistics modes of all third party packages in Python are extraordinarily and surprisingly strong, but we, as average persons, can still understand and possess them.

2 Basics of Matplotlib Plotting7:40

3 Control of Matplotlib Image Attributes9:44

4 Plotting with pandas6:34

5 Data Access4:10

6 Applications of Python into Science and Engineering Fields7:03

7 Applications into Humanities and Social Sciences Fields7:45

与讲师见面

ZHANG Li
associate professor
Department of Computer Science

Apart from science and engineering problems,

Python may also solve problems of humanities and social sciences.

Let's take NLTK for example. It's a Python toolkit

related to natural language processing.

Let's see how to utilize functions

in Python and NLTK

to gather and analyze data in corpus.

Learning them, you may further utilize NLTK

or other similar toolkits to analyze and predicate

other language materials, especially big data

containing enormous important information.

Let's take Project Gutenberg and the US presidents' inaugural speech corpus for example.

Let's look at Project Gutenberg first.

Here, as we talked about before.

Use function "fileids()"

to calculate the books of Project Gutenberg,

that have been included into NLTK

like these.

Next, we may do some simple calculations.

The "words()" function here, say,

puts the numbers of all words in the book Hamlet

including punctuation marks,

into the variable "allwords",

and then we can count the total quantity of all the words, including the punctuation marks.

Well, how many of these words and punctuation marks are not repeated.

Can we first process them with a set

and then use the "len()" function

to get such a result?

Next, let's do some other statistics. For example,

how many times does the string of "Hamlet" appears in this story?

Can we use the "count()" function?

The result says, 99 times.

Besides we can do

some slightly complex statistics,

say, all the words whose length.

All words whose length is over 12.

Let's realize it through list comprehension

and then sort these words.

The output result is like this.

Look at another example.

What's the effect of this example?

Let's see.

We first use the "FreeDist()" function.

It is to create a frequency distribution of a given data.

What are the data here?

They first include all the words

and they are all converted into the lower case.

After acquiring such an object,

we use the dedicated methods "B()" and "N()" in NLTK

to count the numbers of different words and the numbers of all words, respectively.

Then, we use the "tabulate()" method

to display the first 20 datasets

in the form of table.

The effect is like this.

As we see.

The highest-frequency word appears for 993 times.

It's the word of "the".

So far, you might have already known

the effect of this program.

It is to gather statistics of the book Hamlet

count the top 20 high-frequency words,

regardless of upper or lower cases

and count their times of appearance.

Well, we might as well use the "plot()" method

to plot this result.

By the way,

when using the "plot()" method,

we often use the argument "cumulative"

i.e. to add data cumulatively.

Sure, in this case, this argument is meaningless.

Having explored some secretes in stories,

let's look at the inaugural speeches of US presidents.

What kind of secrets can be explored from corpus?

We know that, US presidents like to mention

freedom and democracy in their inaugural speeches.

Let's check the frequency of the word "freedom"

in the inaugural speech texts.

Gather statistics of all the inaugural speech texts,

and then calculate the frequency with the "freq()" method

As we see, the result is like this:

it's approximately one in a thousand.

Then, let's look at the word length habits

of US presidents in their inaugural speeches

during certain time intervals.

Here, we use such a function.

It's a function for conditional frequency distribution.

When corpus texts are divided into several types,

it can separately calculate the frequency distribution of each type.

What can it be used to research

into the systematic difference among types?

We gather statistics for such a time interval

based on what,

based on the word length.

Let's check its result.

As we see, in the inaugural speech made by Barack Obama in 2009,

there are 599 words whose length is 3, the most favored type in his speech.

So far, have you found

any problem?

As we guess,

of these words,

does this word appear for so many times as well?

However, as we know,

this word is of no significant meaning

in statistics.

Would it be better to remove such a word in advance?

For this kind of statistics,

it really is,

That's the concept of "stop word".

We must first remove such stop words from the raw text,

i.e. some words of no significant meaning.

In NLTK,

there's a section dedicated to downloading of stop word corpus.

You may have a look.

It's available in multiple languages.

After removing those stop words,

statistics will be more meaningful.

That's stop word list provided in NLTK.

This is the English version.

Here, as we see,

it contains many function words,

such as me, are, my

like these

quite common words

without any practical meaning.

It's worth noticing that

the stop word list is not universal,

it should be determined according to the research topic and field.

For researches into personal pronouns, for example,

words like "we" and "your" can not be regarded as stop words.

Those are what we have explored from the NLTK corpus.

You may explore more of what you're interested in

NLTK is a common

natural language processing toolkit.

It's easy to start

with abundant available resources and functions.

I strongly recommend you use it.

Sure, learners majoring in humanities may look for

some other corpuses related to your majors,

or scrape data by yourselves for analyses.

Besides, in our previous introduction,

have you found out that,

many operations are not so difficult?

Hope that you will be more skilled and free in using them after more practice.

探索我们的目录

免费加入并获得个性化推荐、更新和优惠。

Coursera 致力于普及全世界最好的教育，它与全球一流大学和机构合作提供在线课程。

© 2018 Coursera Inc. 保留所有权利。

通过 App Store 下载

通过 Google Play 获取