By: Kathryn Wallace
The ambiguity of social media content regulation has heaped opprobrium on well-known companies including Facebook, Twitter, and Instagram. Barbaric radical Islamist propaganda is sanctioned, while photos of breastfeeding mothers were banished from users’ timelines until 2014. “[S]leeping people with images drawn on their faces,” are prohibited, while content including (male/male and female/female) foreplay (kissing, groping, etc.) is deemed appropriate for Facebook’s 1.23 billion monthly active users. The inconsistent manner in which social media sites determine the appropriateness of their subject matter calls into question the validity of the content moderation process.
The offshorization of content management may provide an explanation for these polemic inconsistencies. When United States companies delegate culturally sensitive tasks to moderators with distinct conceptions of sexuality, violence, religion and power, the importance of cultural idiosyncrasies becomes paramount. By investigating the diverse cultural conceptions of gender, violence, religion, and authority, I will explore the validity of concerns that outsourcing of content moderation leads to inconsistent results. Additionally, I will posit that it is unlikely that these cultural distinctions have been the primary issue in content moderation, but rather that an inconsistent and problematic universal perception of these issues is to blame.
What is Content Moderation?
Simply put, content moderation is the “removal of offensive material,” from any online platform. With the exponential growth and increasing accessibility of the Internet, the demand for palatable communication has risen parallel to floods of violent, sexual, and uncensored content. It is what Wired calls “The Grandma Problem: now that grandparents routinely use services like Facebook to connect with their kids and grandkids, they are potentially exposed to the Internet’s panoply of jerks, racists, creeps, criminals and bullies.” Moderating content ensures that users of all ages will continue to visit their sites.
Originally, content moderation was handled by “nanny” software, which scans photos for large areas of flesh tones. Unsurprisingly, this form of supervision did not take into account the contextual complexities of the data only discernable by the human eye. Much like many other industries (i.e. telecommunications), as the need for human labor increased, the incentive to outsource did as well.
Outsourcing Content Moderation
This is how numerous international content moderation firms come into play. Companies like SSP Blue, Sykes, TaskUS, and Open Access BPO provide cost-effective and discrete solutions to mega-corporations and governmental entities like Microsoft, News Corporation, Myspace, Tagged, Backpage.com, United Parents Online, the Obama Administration, the United Nations, and various others. Primarily stationed in remote locations like India, the Philippines, and Costa Rica, these companies commence the first phase of content moderation: sifting through around 20 million images a week in order to discern which will be deemed appropriate for public viewing.
Each New Delhi, San José, or Manila worker has a grid of around 10 photos which he or she quickly flicks through – 8 straight hours of explicit images of bestiality, beheadings, child pornography, and hate speech which website users have flagged as inappropriate. Immediately after the initial diagnosis, the images speed to an American corporate office, where a considerably-higher paid moderator double-checks to ensure that it meets the company’s standards. This two-pronged process attempts to guarantee that no flagged image goes uncensored. However, content moderation standards are forced to be ambiguous in order to balance the varying interests of the people who participate in these online communities.
Within different cultural frameworks, these arbitrary standards allow for a wide range of interpretations. Dissimilar cultural conceptions of violence, sexuality and gender roles, political power and authority, and religion shape the ways in which moderators determine the appropriateness of certain content. The 1964 Supreme Court Justice Stewart quote, “I know it when I see it,” about the indefinability of pornography is not necessarily true as you transition across topics and between cultures. What is sexually explicit in one country can be considered artistic expression in another; hate speech can be perceived as harmless banter; violent radicalism can be considered free speech. The heterogeneity of international cultures poses problems for architects of content moderation guidelines.
Cultural Conceptions of Inappropriate Content
Dissimilar perceptions of gender, violence, religion, and authority could result in drastically different moderation standards between United States companies and contractors in other countries. Admittedly, the interpretations of these facts and their presumable effects on content moderation are assumptions and generalizations.
Gender and Violence
The 2013 WHO report on violence against women shows that the prevalence rates of intimate partner and non-partner sexual violence among all women of 15 or older in WTO defined regions are as follows:
|Eastern Mediterranean||36.4% (No data were available for non-partner sexual violence in this region)|
|High income countries||32.7%|
India, as a part of the South-East Asia region, has one of the highest global rates of violence against women. The Philippines, amid the Western Pacific region, has one of the lowest rates, with 27.9 percent, below the “high income countries.” The Americas region, which includes Costa Rica, has a prevalence rate remarkably close to the international average of 35 percent, and only slightly above the United States’ region. The regional prevalence of violence against women presumably has an effect on the perceptions of gender-based violence.
As an emerging global power, India is increasingly focused on human rights and equality. However, the enduring legacies of its human rights abuses and the slow implementation of policies which protect women and girls demonstrate the continued opportunities for improvement in the country.
One example of violence against women which has left an imprint upon the Indian society is the act of self-immolation:
Sati (also called suttee) is the practice among some Hindu communities by which a recently widowed woman either voluntarily or by use of force or coercion commits suicide as a result of her husband’s death. The best known form of sati is when a woman burns to death on her husband’s funeral pyre. However other forms of sati exist, including being buried alive with the husband’s corpse and drowning.
The Commission of Sati (Prevention) Act of 1987 and subsequent reforms have criminalized sati and the participation or encouragement of onlookers. However, widows’ self-immolation is still practiced throughout India. Vandana Shiva, an Indian feminist, argues that continued glorification of the victims demonstrates that “deeply held and deeply cherished norms cannot be changed simply by enacting laws.”
In relation to content moderation, Indian individuals have been explicitly and subconsciously influenced by the existence of the sati tradition. The recent criminalization of the self-immolation signifies that many generations of Indians were alive during the legal practice of sati. Whether there is a common approval for the practice, or merely a small portion of the population which promotes the devaluation of women after the loss of their husbands, it is important to take into consideration the potential consequences that longstanding traditions have on the current Indian population. The perception of women’s value as completely dependent upon that of her male counterpart is extremely important in determining the value of females in imagery and Internet content. For example, content moderators, particularly those who have been directly influenced by sati tradition, which are given an ambiguous instruction to remove photos which are sadistic towards women may have an established bias regarding female subordination and deserved forcefulness.
Due to the heavy Iberian influence during colonialism, the Philippines is a predominantly Catholic society. Rosemary Radford Ruether argues that Catholicism’s complex and problematic assumptions about women as subordinate and corporeal has led to a denial of decision-making abilities for predominantly Catholic societies. Women are viewed as virginal until the event of marriage, when like Virgin Mary, they are held in highest esteem for their reproductive and motherly capabilities. For example, in 2006, the Catholic Bishops Conference of the Philippines announced that it would deny baptism, communion, confirmation, weddings, and burials to all that supported the use of contraception. Additionally, Filipinos older than 15 were required to pass a course on Catholic sexual teachings in order to be eligible to receive sacraments. The strong influence of the Catholic Church on Filipino society would undeniably influence the decision making of content moderators. Women portrayed as non-virginal before marriage or as lacking puritanical values or procreative capabilities could be perceived immediately as offensive or stigmatized.
Costa Rica is neighbor to some of the most violent nations in the world: Honduras has the highest international homicide rate, with 90 homicides per 100,000 population, compared to the region’s rates of 44.7 in Belize, 41.2 in El Salvador, 21.5 in Mexico and 8.5 in Costa Rica. Trapped between the countries which produce and consume drugs, Central America is burdened with the dual challenge of economic development and policing drug trafficking. Gang violence, weak institutions, and political corruption have consumed the region, elevating the persistent rates of violence.
These incidents of violence bleed into households. According to Ryan Villarreal, only 5 percent of violent deaths of women have been investigated and prosecuted in the last decade in Costa Rica. Although difficult to document due to underreporting, Costa Rica is predicted to have extremely high rates of domestic violence. These rates, as well as femicide, or gender-selective mass killing, are reinforced by the ingrained culture of machismo (male dominance and expected female subordination) and the complex and violent regional circumstances. It could be postulated that high rates of violence in general, and violence against females would lead to a normalization or desensitization of violent content, decreasing the ability of the content moderators to determine what would be inappropriate for users in the United States.
Religion and Authority
India is a predominantly Hindu nation. Hinduism is a cultural, philosophical, and religious tradition practiced primarily in South Asia. It is the world’s oldest extant religion, characterized by the “belief of reincarnation, one absolute being of many manifestations, the law of cause and effect, following the path of righteousness, and the desire for liberation from the cycle of births and deaths.” An individual from India would be heavily influenced by the Hindu faith. For example, a content moderator would presumably find a cow slaughter video extremely offensive, no matter the context. According to the National Hindu Student’s Forum, Hindus respect, honor, and adore the cow, among all sacred creatures.
Another characteristic of Hindu society which may influence the subconscious decision-making of content moderators is the presence of the caste system. This socially stratified classification, which has been embedded in the Indian culture for the past 1,500 years is based on the fundamental idea that men are created unequal.
The ranks in Hindu society come from a legend in which the main groupings, or varnas¸ emerge from a primordial being. From the mouth come the Brahmas-the priests and teachers. From the arms come the Kshatriyas-the rulers and soldiers. From the thighs come the Vaisyas-merchants and traders. From the feet come the Sudras-laborers. Each varna in turn contains hundreds of hereditary castes and subcastes with their own pecking orders. A fifth group describes the people who are achuta, or untouchable. The primordial being does not claim them. Untouchable are outcasts.
Although untouchability was criminalized sixty-five years ago, more than one-fourth of Indians say they continue to practice it in some form in their homes, according to the Nation. This signifies an enormous sway of class perceptions in everyday life. This would indubitably influence content moderators in the manner in which they regulate online content.
The Philippines and Costa Rica
Costa Rica and the Philippines have an historical legacy of Spanish colonialism which explains the emergence and importance of the Catholic faith. Although these two have diametrically oppositional cultural vestiges, the prominence of the Catholic faith is significant in the development of the moral code of these societies. The hierarchical nature of the Catholic Church has been extremely influential in determining the societal structure of post-colonial Costa Rica and the Philippines. For example, both nations are democratic, but their governments have tended toward strong authoritative presidents. In a similar manner in which the Pope has omnipotence, the President is given full control. This has increased the incidence of corruption and overextension of power within these nations. Additionally, the citizens of these nations, as with religion, are often more willing to relinquish political and civil liberties so that the leadership to have more definitive decision-making power.
A Cross-Cultural Conundrum
Although these cultural distinctions weigh heavily on the decision-making skills of the respective societies, I speculate that the issues with content moderation are the result of explicit guidelines which promote problematic conceptions of what is right. For example, repeated leaks of Facebook’s content moderation standards have demonstrated that the instructions given to moderators are clear and precise, defining the exact body parts, particular word usage, and context for determining the appropriateness of content.
Additionally, the aforementioned two-pronged strategy, which employs American workers to ensure that the first round of content management was appropriately completed, confirms that the US, Facebook’s (and assumedly industry-wide) standards are flawed. In order to appease the diversity of clients which interface with the Internet, these standards reflect the imperfect conceptions of sexuality, violence, religion, and authority which are present throughout the world. Although it would be easy to blame the heterogeneity of the global culture, it is likely a universal issue rather than a cross-cultural one. The following examples demonstrate the pervasive nature of problematic conceptions of these issues in Western cultures.
Sexuality and Violence
Recently, extremely visible incidents of domestic violence associated with the National Football League and the high rate of college sexual assault have forced the United States to confront the pervasive nature of gender violence domestically. Interestingly though, when attempting to investigate the rates regarding gender-based violence in the United States, Google search tool failed to unearth any statistics of just the United States (Try searching ‘gender-based violence in US’). Instead, the results were related to global violence against women. This simple example demonstrates the denial of Western cultures that gender violence is still present within our societies. Although 1 in 5 college women are sexually assaulted on campus, Google fails to provide a search result within the first two pages which even resembles information that details college gender-based violence. Although domestic violence is a persistent issue in all cultures throughout the world, Google neglects to inform the researcher that the number of women who were murdered by intimate partners between 2001 and 2012 was 11,766, double the rate of the number of troops killed in Afghanistan in the same time period.
Although racially-biased sexual assault is pervasive throughout the United States, not until the bottom of the first page does Google allude to the UN Human Rights Commission Report on police brutality and (secondly) gender-based violence. Even with 22 percent of Black women and 50 percent of racially mixed women experiencing rape, and the continued devaluation of Black women as legitimate victims of rape, the majority of Google search results are about the “US Strategy to Prevent and Respond to Gender-Based Violence Globally.”
The inconsistencies continue with the Western conceptions of religion and authority. The United States’ legal and moral code is dominantly constructed through the Protestant faith. In Christianity, the God of the Bible is “a wrathful, jealous tyrant, swift to send terrible punishment to anyone who does not obey him absolutely and do exactly as he commands.” Although the Christian God exhibits possessive, controlling and misogynistic behavior, a Christian is encouraged to seek an intimate and personal relationship with him. This example of the mercurial nature of the God upon which many Western governmental systems are based calls into question the moral code which He promotes.
Additionally, as the “great American melting pot,” the United States contains all of the independent religious identities of the Philippines, India, Costa Rica, and many more, and the corresponding conceptions of religion and authority. The heterogeneity of this concomitant mixture is enough to create apparent inconsistencies in the manner in which different United States citizens would moderate the same content. Other Western nations struggle with cultural miscegenation and the resulting consequences. The recent Charlie Hebdo attacks were the product of the religious struggles and subsequent radical opposition which formed in lieu of a placid cultural combination. The presence of religious tension and incoherent societies’ identities result in arbitrary content moderation for any culture.
Struggles to define the middle class and to distribute income more equally among the citizens of Western cultures have been exemplified by the 99 percent movement, the discussion of “middle class economics” by President Obama, and the publication of the well-received critique of capitalist inequality, Thomas Piketty’s Capital in the 21st Century. In the United States, expanding influence of “big money” in the legislative, campaign, and policymaking processes has augmented concerns of corruption and misplaced power. The increased incidents of police brutality and racial violence and the “Black Lives Matter” movement exemplify the chronic resistance to the development of equal civil rights.
Moreover, the increased economic turmoil in the European Union and the resulting secessionist movements, including that of Scotland and Catalonia, has emphasized the unhappiness associated with the current societal status quo in the West. These cultural characteristics, as problematic as those apparent around the world, demonstrate that the West is just as likely to conceive of sexuality, violence, religion, and authority in an arbitrary and inconsistent manner.
Cultural Singularity: West versus the Rest?
The subject of content moderation addresses an interesting debate. Often we, among the Western cultures, assume that our ideology is superior, and that others are merely catching up. Upon learning that content moderation was outsourced, I immediately assumed that cultural discrepancies were to blame for the controversial and oftentimes horrendous mistakes related to content moderation. As an American, I assumed that it must be that other cultures just don’t understand yet, and that they had “other” ideas relating to sexuality, violence, religion, and authority. In reality, I discovered that although our cultures are distinct, there are limitations to blaming cross-cultural interactions for our failures. Content moderation is often deplorably inconsistent, misogynistic, and inconsiderate, but that is entirely the result of the guidelines created by Western companies in order to suit Western consumers.
Kathryn A. Wallace is a Master’s Candidate at the Patterson School studying diplomacy with a concentration in Latin America. She is particularly interested in issues relating to women’s and reproductive rights. Feel free to contact her for potential opportunities at www.linkedIn.com/in/kathrynawallace/.