A study into the credibility of Hong Kong Media Organisations

Part One — Key concepts and operationalisation of key terms

The aim of this study is to determine the credibility of five different articles on the land reclamation policy announced by Carrie Lam, Chief Executive of Hong Kong, during the Annual Policy Address on 10th October 2018. Lam announced the Hong Kong Government was to pursue the reclamation of land in Lantau to ease Hong Kong’s housing crisis. The policy was met with a fair degree of concern over cost, environmental concerns and debate over effectiveness.

Lam, 61, has been chief executive of the Hong Kong SAR since 2017.

By credibility, we aim to determine how useful the article is in giving the reader a good understanding of the issue at hand, there are several different aspects we will be using to measure this.

The headline

The heading at the top of the article is vitally important. This is particularly because people could from time to time, just read the headline of the article without looking further into the story.

As such, it is important the headline accurately reflects the story, meaning that even a quick glance at just the headline can give the reader a good basic understanding of the story.

Sensationalism

Media articles can occasionally over-exaggerate details. But these can drastically alter the reader’s perception of events and understanding of the issue.

For example, using exaggerated language such as “shocking” or “unbelievable” for events that are not really surprising can make a mundane event appear to be more significant than it is. It could also make the story look more positive or negative depending on the terminology used.

Bias

News reports should be objective. The aim should be too merely state the facts and explain an issue. Subjective journalism is an acceptable form of media but it should not be labelled as news, it should be labelled as opinion.

The story must also be willing to evaluate all sides equally, giving both a fair chance at public debate and discussion. Media organisations are often where people look first for information, so it must not dismiss an idea completely without letting the public think for themselves.

Balance

One of the ways it can allow the public to think for themselves is by just reporting the facts of what each side says about a policy or proposal.

This is fair to both sides as it puts the onus on them to convince the public, the journalist merely gives them the platform where they can do it if their argument is strong enough.

Poor balance could give the false impression that a certain viewpoint is not common when that may not be the case, though it must be acknowledged hearing one side is still worthwhile as it gives part of the picture.

5Ws1H

It is impossible to have a fair and reasonable understanding of the issues if the story lacks the basics of what has happened. Who? What? Where? Why? When? And How? May seem rudimentary questions but they require answering.

For example, if a senior politician made a comment about the cost of a policy, readers would need to know who said it as this shows them how seriously to take the story. They need to know what they said and the precise figure to understand what is actually being discussed. The where can be crucial — if the politician said it whilst debating an opponent or in a private memo would be important as it gives a clue to motives. The why is also important as the reader needs to know why they should care, is the policy too expensive, for example. The when, if the politician said it before the policy was announced, perhaps privately, that could raise important questions about the credibility of the politician, whilst how is a vital component, we need to know how they said it, it may have been a joke rather than stating an accurate figure and we must know how the figure was derived to be able to evaluate it sufficiently.

Accuracy

All information related to a story could potentially in some way alter the understanding of it, and the ability of the reader to properly evaluate things.

Leaving out certain information could drastically alter the perception of the reader. Going back to the example of a politician, if the politician was opposing a policy but had received a large donation from an organisation who hold it in their interests to see the policy not passed, that would obviously alter the perception of the story and result in readers being more likely to dismiss the claims.

But if this information is left out in an article, then it never gets in their thinking unless they read or hear it elsewhere.

This, of course, only applies to known information as a newspaper cannot be expected to report information it is not privy to.

Rudeness

The article should not use derogatory or vulgar language to describe a person or a group of people.

Personal attacks are unnecessary to support an argument and the individuals being attacked could feel they will be putting themselves under risk by openly stating their opinions, stifling further legitimate debate.

It is also not credible as it may lead to legitimate grievances being trivialized and dismissed by people who do not feel the same way. For example, those concerned by immigration could be attacked as being racists or conspiracy theorists, meaning those without those concerns think poorly of those on the other side and dismiss them and those with concerns do not wish to openly speak out.

Comments

Some articles have comments sections. These are another factor to consider. The article may do a really good job in being clear, accurate, fair, balanced, respectful but this could be undermined by comments. People will read comments and these can also unfairly alter their perceptions in the same way that the article could.

The ideal scenario would be the comments sharing the same idle traits as the article and facilitating further legitimate debate, that is thought-provoking and helps further understanding. This would have to be done through moderation, as it would be unreasonable to expect commenters to be self-moderating due to the rise of trolls.

A midpoint would be ban to not allow comments though this limits further discussion which may not have been considered in the article and makes it harder.

This category does not refer to sharing on social media and only on the comments section of the online article. This is because it would be impossible to moderate these comments with complete control as the media organisation does not host the site, nor would it be able to see some comments on shares for example.

The Hong Kong skyline is dominated by skyscrapers and apartment blocks.

Part Two — Coding Procedure

In order to code an article for evaluation, we have quantified the eight aspects of credibility listed above into a measurement scale.

Each category has a highest potential score of 12. This was done as this number could be factored into each aspect and give each an equal weighting. Though it can be argued some aspects are more important than others, this was perceived to be too subjective to justify as part of the analysis.

The higher the score, the better and more credible the article is. The criteria does involve some degree of subjectivity, but we are essentially dealing with attempts to quantify qualitative measures, so this is unavoidable.

Map of Lantau Island

Coding Scale

Headline

0 — Headline is vague and could be interpreted as referring to something completely different

6 — Headline requires prior knowledge of subject matter to be interpreted correctly

12 — Headline clearly refers to the topic in question.

Sensationalism

0 — News report greatly exaggerates all aspects of the story

3 — News report greatly exaggerates most aspects of the story

6 — News report greatly exaggerates half of the story

9 — News report greatly exaggerates some of the story

12 — News report does not exaggerate any of the story at all

Bias

0 — The news story clearly supports one viewpoint and does not acknowledge the possibility of an alternative viewpoint.

4 — News report supports one side and only applies evaluation to the opposing view.

8 — News report spends more time evaluating opposing view than its own.

12 — News report shows no support for one side over another.

Balance

0 — Only supporters from one side are quoted.

6 — News report gives substantially more time and/or weight to sources from one side.

12 — News report gives equal worth to sources from both sides.

Who? What? Where? Why? When? How?

0 — Answers none of the questions

2- Answers one of these questions

4 — Answers two of these questions

6- Answers three of these questions

8- Answers four of these questions

10 — Answers five of these questions

12 — Answers six of these questions

Accuracy

0 — Key information that would be extremely likely to alter the audience’s perception of the story is not included.

4 — Information that would be likely to slightly alter the audience’s perception of the story is left out.

8 — Information is left out but it is unlikely to alter the audience’s perception of the story.

12 — All relevant information is reported at some point in the news story.

Rudeness

0 — The article uses languages that personally attacks people for whatever reason.

12 — The article does not resort to personal insults, instead focusing on facts regarding the issue.

Comments

0 — Unmoderated comments appear on the site that unfairly undermines one side of the argument.

6 — No comments are allowed on the site.

12 — Comments do appear on the site but these are either moderated or naturally allow for reasonable debate on the topic at hand.

We will then calculate the mean of each index and use this to give the article a final credibility score.

Score Meaning

0.0–0.9

This article is not credible at all and should not be trusted to give information on the topic.

1.0–3.9

This article displays some credibility but is still not a reasonable source on the topic.

4.0–6.9

This article displays a reasonable amount of credibility but still suffers greatly.

7.0–9.9

The article is generally credible but it is perhaps advisable to use it in addition to other sources.

10.0–12.0

The article is credible and can be trusted to give a reasonable report on the issue.

It was considered to use a category which factored in the journalist writing the article and the credibility of the articles they had written prior. However, this would have been a case of induction, mislinking cause and effect. The perception of a journalist’s credibility is not the same as their actual credibility. It would also be impractical to factor in every single published work due to time and accessibility. It would be far easier if one wished to go down this route to measure the credibility of individual articles as they were published and use the average to work out the credibility of the journalist. In other words, it is more practical to use articles to determine the credibility of a journalist than using a journalist to determine the credibility of an article. After all, it is possible for a journalist who usually writes sensationalist, divisive and misleading articles to write one that is credible. Under the method of using articles as the basis, a good article will raise their credibility, rather than a bad journalist lowering the credibility of the article.

The Big Buddha statue is a popular tourist destination on Lantau.

Part Three — Findings

The articles used included a combination of English and Chinese language sources for comparison — South China Morning Post, The Standard, China Daily Hong Kong Edition, HK01 and Now. We choose them due to their popularity in Hong Kong and public perception of them, The Standard is the most popular English language newspaper, whilst South China Morning Post has the most paid subscribers. It also has been named in reports as the most trusted. China Daily Hong Kong Edition is owned by the Chinese Communist Party and has been described by the BBC as “state run”, HK01 is the youngest organisation but has been reported to have experienced 500% year-on-year growth. Meanwhile, Now’s TV division is the biggest in Hong Kong by subscriber numbers.

All the articles used for the study were the first articles published after the announcement of the policy. This is important as these articles would generally be the first to be consumed by people wishing to learn more about the issue and from which their initial judgements will be made. It is thus vital, the story is credible and gives them a good understanding of the story and the arguments behind the different sides. It is from that prism that people will then look at further articles, which may be less news based and more opinion based. It is vital they thus start from an opinion that has been generated by reasonable debate rather than misleading or deceptive content.

Table

Article Analysis

South China Morning Post

The article is generally credible but it is perhaps advisable to use it in addition to other sources.

SCMP scores a six on headline as “In tomorrow’s world, Hong Kong’s leader sees a massive HK$500 billion artificial island in middle of the sea, home to 1.1 million people” whilst giving a lot of the detail does not explain the context and requires prior knowledge, readers are not told it is part of the policy address. There is no exaggeration with all figures being precise and attributed to government or opposition sources. There is no indication of any support to either side so it gets a 12 for bias. However, it only receives a six for balance as Carrie Lam is quoted as saying “there is no such thing as consensus for whatever issue”, there is no quote from an opposition figure evaluating Lam’s argument that she has no choice. Lam responds to arguments, but the opposition do not in this article. It does, however, offer all answers to the 5Ws and 1H (“Carrie Lam”, “presses ahead with controversial project”, “Lantau Island”, “to ease Hong Kong’s chronic overcrowding”, “to be completed in three years from 2025”, “land reclamation”) and does cover all key information succinctly, giving it another 12 in these. There is no rudeness or comments sections resulting in scores of 12 and six respectively.

The Standard

The article is generally credible but it is perhaps advisable to use it in addition to other sources.

The headline, “(Policy address) ‘Lantau Tomorrow’ proposal revealed)”, requires prior knowledge about what the Lantau Tomorrow policy is supposed to contain, resulting in a score of six. With the article focusing almost exclusively on quotes, there is no sensationalism. The article does not openly support one side over the other so scores the highest for bias, however it does only quote Carrie Lam, so scores a zero for balance. It does not offer the timescale of the project so is docked two points for 5Ws and 1H (It does discuss how Carrie Lam announced a land reclamation policy in Lantau that seeks to use land on brownfield sites to ease the population crisis). In terms of accuracy, it does not acknowledge opposition and the concerns over cost, effectiveness and environment so scores a zero. But it does not contain any personal attacks or a comments sections so scores 12 and six respectively.

China Daily Hong Kong Edition

This article displays a reasonable amount of credibility but still suffers greatly.

Once more, the headline “‘Lantau Tomorrow Vision’ — home to 1.1m”, requires prior knowledge of the Lantau Tomorrow plan as and it does not mention the policy address or land reclamation, so it scores a six. The use of the word “ambitious” loses three points in sensationalism. Land reclamation in Hong Kong has been going on for centuries and there’s no explanation why this is any more ambitious than prior projects. China Daily scores zero for bias due to the paragraph: “An ambitious land reclamation planned to be built near Lantau Island will go a long way in addressing the city’s acute land shortage problem, as well as in spurring social and economic development.” The word “will” shows clear support for the policy and there’s no acknowledgement that other viewpoints could exist, it scores zero for balance as only supporters for land reclamation are quoted. Joseph Tsang comes closes to be being against the government but he is not opposed to the policy, merely priorities (stressing importance of looking at immigration and population) which is not entirely relevant to the story. It gives a good explanation of the 5Ws and 1H (It explains how Carrie Lam has announced a land reclamation policy which seeks to create artificial islands in Lantau to ease the population crisis that will be enacted in the next “two to three decades”) but does not include information about concerns over cost, effectiveness and environment, meaning it is not accurate. Like the two prior articles, it does not go into personal attacks or allow comments, meaning it takes 12 and six for these categories.

HKO1

The article is generally credible but it is perhaps advisable to use it in addition to other sources.

The headline “明日大嶼可住110萬人 較人口頂峰多33萬 工程師:咁大幅地做咩” once more requires prior knowledge about what the Lantau plan curtails. The headline mentions the name of the plan and the population size as well as opposition from an engineer but not what the plan actually involves — reclaiming land from the sea. It scores lowly on sensationalism as it places the engineer’s comment in the headline, this can be seen as exaggerating a negative aspect. A local engineer, though a worthy source, is not the main story here. Whilst it does not make a comment supporting either side of the debate, it only talks to the engineer who is against the policy, so it scores 12 for bias and zero for balance. It scores a 12 for 5W1H (Who: Ngai Hok Yan, What — Says land reclamation is waste of resource, Where- Lantau Island, Why- Because the population will decrease in the future, When — About 40 years later, How- When arriving to the population peak, the population will decrease). It scores highly for accuracy due to the fact it does list pro’s of the policy despite being relatively one sided. It avoids making personal attacks and though comments are enabled, at the time the analysis was taken none of the comments displayed any personal attacks or lies.

Now

The article is generally credible but it is perhaps advisable to use it in addition to other sources.

Once more, six points was given for the headline. Prior knowledge of the policy and the fact it involves land reclamation is required to fully understand the headline “[Policy Report] “Lantau Tomorrow Vision” can build 26 to 400,000 units” (translated from 【施政報告】「明日大嶼願景」可建26至40萬個單位). The source did not openly support one side and all statements made are backed up or cited. Carrie Lam was the only quoted source so Now scores lowly for balance. It also loses two points for 5W1H as it does not offer a time scale but does explain how Carrie Lam has announced a policy to reclaim land in Lantau to arrest the growing crisis facing Hong Kong. It fails to mention reasons why people may have concerns over the policy so does not score any points for accuracy. Like HK01, it avoids making personal attacks and though comments are enabled, at the time the analysis was taken none of the comments displayed any personal attacks or lies.

Conclusion

All the articles required prior knowledge in order to understand the headline, whilst a lot of people will have possessed this knowledge as land reclamation had been discussed in the lead up to the policy address, someone who had been completely oblivious to the process would have been unaware of the implications of the headlines.

The articles tended not to sensationalise or show open bias. It was generally poor balance and accuracy which brought down the credibility scores of the articles. The articles tended to quote Carrie Lam a lot more than opposition figures and gave more time and focus to her. South China Morning Post and HK01 were the only two sources who even mentioned any reasons anyone had to oppose such a policy. The credibility of the articles could all be improved if more time was given to opposition sources, and in the case of The Standard, China Daily Hong Kong Edition and Now, there was no acknowledgement of the existence of those dissenting views.

The information needs to be accessible to allow reasonable debate on the subject and allow readers to come to reasoned judgements about the merits of the policy. Not having information about the economic or environmental concerns does not allow this to happen.

The average for the credibility scores of the five articles reviewed came to 8.15, placing all sources with the exception of China Daily Hong Kong Edition above the average. This would equate to a conclusion of: “The article is generally credible but it is perhaps advisable to use it in addition to other sources.” None of the articles collectively did enough to break into the top category on the scale. Though the South China Morning Post came close, scoring 12 in five categories and six in the other three, if just one of the scores in the aspects the article scored six in was a twelve, then it would place its average over ten.

Conversely, it was bias in China Daily Hong Kong Edition which contributed to its low rating. Though it also scored zero for balance and accuracy, the zero for bias was a real outlier as only SCMP did not score zero for balance and only SCMP and HK01 did not score zero for accuracy.

Overall, improvements to the credibility of all articles could be made. The best articles of those in the sample are best viewed in accompaniment to another and considering the merits of each article and the relative scores to one another, in order to get a credible overview of the topic it is perhaps best to read and circulate South China Morning Post and at least one of Now, The Standard and HK01.

Whilst all the articles actually have to be read to be understood due to the headlines requiring prior knowledge, between them they possess a high level of credibility and all the information a reader could need to make a valued judgement about the topic in hand.

Limitations

The coding requires some subjective valuation. Whilst attempts have been made to reach these judgements through defined qualitative evaluation, this could be re-evaluated to give a more uniform scale that does not lead to disagreement.

The scale could also be defined differently. Whilst this scale avoids having to justify subjectively placing value open certain aspects, the fact that none of them scored less than 10 for 5Ws1H and none of them dropped any points for rudeness suggests these may be easy to score high in and could distort the ranking. It may be worth putting more weighting on aspects that can reasonably be considered to be harder to score well in.

Lastly, the final score for the China Daily Hong Kong Edition is 45, this is a whole 33 points lower than South China Morning Post, enough points for two whole aspects and just three short of a third. However, it is just one category below on the judgement scale, the ranking system could be altered to allow there to be more distance between high performing articles and low performing ones.

This study was written in collaboration with Anson Chung.