How to use FB API (part 2)

Today, I am going to crawl data under private authorities.

1. If posts or comments are only visible to friends, what authorities do we need?
2. If posts or comments are on a group, what authorities do we need?

And after these two work, we will try to deal with the comments.

1. Figure out the JSON structure of response
2. Learning how to use python to deal with it
3. Store the data into JSON file or CSV file

Alright, so let’s get started.

First, we just try to use one of our own private post and try if it works.

After several tests, we know:

1. we could successfully access our own posts with authorities to friends
2. we could successfully access posts and comments on a public fan page
3. we could not access public posts or comments on our friends’ pages

According to the documents, there are no way to get friends’ posts unless they authorize our app.

Ok, now we turn to crawl posts and comments from the group.

Frist, you have to know we don’t use posts here and change to feed, look like:

{BASE_URL}/{GROUP_ID}/feed?access_token={ACCESS_TOKEN}

1. [200 response] we could successfully get all the posts on a public group.
2. [200 response] although we recieved 200 response, we didn’t get any data from a private or closed group.
3. [200 response] You can successfully get all the posts on a private or closed group if you are the manager of the group. However, you have to open another authority called user_managed_groups

Until now, we could

1. Get data from a public fan page
2. Get data from your own page(Any authorities settings)
3. Get data from a public group
4. Get data from a private or closed group if you are group manager

You can find all the related codes on my github:

polyline/crawl.data

Now, we going to analyze the JSON response.

When we send a request from python, we can also verify it on the browser.

Hence, we can figure out the JSON structure by doing so.

In python, we need a passage called json to help us turn response into dictionary type.

j = json.loads(res.text)