Listening to the CrowdAnahi Ayala Iacucci – Media Innovation Advisor

Crowdsourcing information for elections

How to verify information collected online during elections

WHAT DO WE MEAN WITH VERIFICATION AND WHY WE DO IT?

Verification and falsification

The goal of verification of crowdsourced information is to assess whether the information reported is indeed referring to an actual fact that happened exactly in the same way it is reported. For this reason the verification process needs to happen as a multi-level approach, where all components are analyzed separately and then merged.

Types of Verification and Falsification of crowdsourced information

Falsifying/Verifying the content Falsifying/Verifying the sourceFalsifying/Verifying the context

Content5W1H

WHO who are the

parties involved. Who are the

victims and who are the

perpetrators. WHATwhat actually

really happened,

what violation has been made

WHERE the

location of the fact

reportedWHEN when the

fact reported happen

WHY why did the fact

happened

HOW the context

and the dynamics of the

actual fact

What is the content?

Falsifying the content

PARTIAL FALSIFICATION The

information is partially false. There is one (or more) of the 5W1H that is false while some others are true.

Verifying the content

The first step is to verify the content

of the information, looking at the

verification of the 5W1H

WHOwho are the

parties involved? Are really the

perpetrators the once mentioned

in the report? Who are the

victims?

WHEREwhere did the

fact happen? Is there any

reference to the actual place? Addresses? Landmarks

mentioned?

WHEN when did the

violation happen? Is the time of

reporting close or not to the time of

the actual fact reported?WHAT

what actually did really happen? Can we re-built

the entire dynamic of the

event?

WHY is there any

information on why the fact

happen or about the intention of

the perpetrators?

HOWhow did the fact happen? What is the connection in

between the different

components of the event?

Types of information

sources

Primary sources

the source is the

eyewitness of the fact

Secondary sourcesthe source is

reporting something based

upon evidence from primary

sources

Tertiary sourcesCompilation based upon primary and secondary

sources

Falsifying the sourceThe falsification of the source is often used to undermine the reliability

or reputation of a source or to gain trust from the audience

Falsifying to gain trust

This is often done to add credibility to the information reported by attributing it to a source that will be more trusted than the

reporter himself. For instance one common example of this is when secondary sources

present themselves as primary sources

Falsifying to undermine trust

On the contrary this type of falsification is done to attribute to a

source a certain information in order to make them look less credible, or

even personally involved in the events

Falsifying the context

The verification of the context is one of the most difficult things to do because certain information may be true in the content, and the source may be a reliable one, but the context may give to the same information a different angle and therefore a completely different meaning. See this picture as an example:

THE ONLINE VERIFICATION OF THE SOURCE

Online verification

of the source

Social Media ID ID of the

Content on the Social

media

ID of the Network

ID of the trusting network

InfluenceLocation

Timing

Media authentication

Direct engagement

The Social media ID

When getting an information from a social media source the first thing that we need to look at is the bio of the source (account) as provided in its home page. In this first step we will look only the Identity of the person to try to re-create his/her ID based on his/her social media bio. We can look at several different social medias: Twitter, Facebook, You Tube, Flickr, Linkedin, etc.

Bio on Social Network

Is there a name and surname?

Look for the name/surname on

Google

Is there a link to a blog or a website?

Check the sites and see what content

they have and what bios they have

Is there a link to the Facebook/Twitter/Li

nkedIn/You Tube/Flickr Page?

Check the pages and the bios there

Are there pictures? Videos? Lists of previous jobs or

schools attended?

Check all of those and see what they can tell you about

this person

Evaluate what do you find trying to understand as much as possible about this person identity: does he/she seems to be linked to any political party? Does he/she work for a specific company and if yes what is this company doing exactly? Does he/she have a picture, and if yes, does the picture tells you something about his/her political/religious affiliation? Also make sure that all information in blogs/websites/Twitter bio and so on report the same information: any discrepancies can be a sing of an identity falsification.

ID of the content on social media

The second step of the online verification of the source is to look at the content of the source as provided in its home page. After having checked his/her Bio from the previous section, now we will look more closely to his content on social media, what he is talking about, what his ideas may be regarding certain issues and so on. This process is done to add more information to the bio of this person and to try to get a broader picture of what this person is interested in. In this section the methodology is the same for all social media previously considered.

The first step here is to make a list of

his/her social media, or online

presence. An example can be:

Twitter

Facebook page

You tube channel

Flickr account

Linkdin page

Website/Blog

The second step is to closely look at the content.

Is the content relevant to the issue reported?

Does this person seems

to be committed to a certain cause or

particularly interested in a certain topic?

What other topics is he/she posting about?

Does the content seem reliable? Is

there content that does not seems to have

been supported by other sources?

Does he has strong opinions

related to certain topics?

And if yes, what his opinions

are?

Is this a one-issue tweeter?

In this process make sure that you take a look at all possible type of content posted on social media: videos, photos, comments to other people content, articles, re-tweets and forums. Most of us have themes running through our interests, but a tweet/messages history that is completely consumed with one or two issues without the occasional personal reference, humor or reference to other topics is suspect. Behind every social media account should be a personality.

In this third phase we are looking at the

social network that this person has. In this context we will look at the people he/she is interacting with and

the friends he/she has on social networks

Who are his/her friends?

Do they share the same type

of content?

Do they seem to be related or

linked to any particular

group/party or group?

Do they share his/her views

on specific topics or not?

ID o

f the

Soc

ial N

etw

ork

ID of the trusting network

This fourth step is related to the people this person trust and that he seems to be influenced by. What we are trying to see here is how to dissect the social network and look more closely to the network of people that this person trusts.

Twitter/Facebook

Who are the people that this

person is re-tweeting most

often?

Who is he/she following?

What are these people tweeting about and what are the topics he

interacts with them about?

Who are the people that

he/she is interacting with

most often?

Linkdin/Blog

Who are the people that have given

recommendations to this person?

Who are the people in his

network?

Who are the people that

he has worked for and that he knows?

Influence

In this phase of the verification process we will be looking at the people that this person influence and that trust him as a source. This information will give us a idea of what is the influence that this person has on his/her social media network.

Twitter

Who are the people that are re-tweeting this

person most often?

Who are the people

following him/her?

What are these people

tweeting about and what are

the topics they interacts with

him/her about?

Who are the people that are interacting with

this person most often?

Location

In this section of the verification phase we want to verify the location of this person in the real world. The reason why we do this is to be able to define if this person is reporting as a primary source, a secondary source or a tertiary source. The actual location of a person in the virtual world may be really tricky so we need to proceed by steps.There are two types of location services that are associated with social media:- Manual location settings- Smart phone location settings

Manual Location settings

Some social media have the location already set up in their settings, if the user has allowed it. For example, Twitter has this option in the profile page:

This function though is a manual function meaning that it is the user that manually set the place where he/she is based and where he/she is from. Most of the time this location is the home-town of the person, and not necessarily where he/she is tweeting from.

The same is valid for Facebook location settings, that you can find in the About page of the person:

When people are using their smart phones to use Social Media, they can unable the automatic location of their messages, both on Twitter and Facebook. In this case the location is the one of the mobile phone, so it does correspond to the actual location of the person at the moment when the message was sent.

Twitter

The Smart Phone Location settings

Facebook

Triangulation of the location

1. Check the location of the person on the manual

settings (if available)

Is the location the same of the location mentioned in

the message you are verifying?

NO: the person may be on a trip or may have moved

from the original place

YES: pass to the verification processes2,3,4

2. Check the location on the smart phone settings

(if available)

Is the location the same of the location mentioned in

the message you are verifying?

NO: this person is probably a secondary or tertiary

source

YES: most likely this person is a primary source. Pass to

step 3 and 4

3. Cross-reference locations from section 1

and 2 (if available)

Are the locations the same?

NO: the person may have been moving from the

original location

YES: The person is probably a primary source

4. Cross reference the location mentioned in the message you are verifying

with the location mentioned in other

messages from the same person

Are those location the same or in the same area?

NO: if the locations are pretty far, this person is probably a secondary or

tertiary source

YES: This person is most likely a primary source

Language/Location cross-reference

This is a second step to be done when looking for the verification of the location of your source. In this phase you can look at verifying whether the vocabulary, slang, accents are correct for the location that a source might claim to be reporting from. For instance, if the reports are coming from a very rural area and the language is a perfect English, you may want to further verify the source. The same if a certain source is always reporting in a certain language and then suddenly you start receiving reports in another language (this may indicate that those reports are copied or taken from someone else).

Timing

The time is an important characteristic of the information you are looking at. This will also tell you something about the possible location of the person sending the message out. In almost all social medias and blog/websites there is a time stamp of the publication, but as in the location, this stamp can also be misleading with regard to the actual event reported. For this reason we need to proceed with the triangulation of this information to get closer to an estimate of the real time of the event and therefore to the reliability of the source.

Check time stamp

Is the time close to the time of the event?

YES

The source may be a primary source

The source may be not a primary source but

someone very much following the event closely

NO

The source may still be a primary source but just had

no access to communication means

The source may be a secondary or tertiary

source

Time of account creation

This part of the verification process is intended to look at the existence of the account before the time of the reporting. The idea is to find out if the Twitter/Facebook handle is a new account, with only a few tweets. If so, this makes authentication more difficult. Arasmus notes that "the more recent, the less reliable and the more likely it is to be an account intended to spread disinformation." In general, the longer the Twitter/Facebook handle has been around and the more Tweets/posts and people linked to this handle, the better. This gives a digital trace, a history of prior evidence that can be scrutinized for evidence of political bias, misinformation, belonging to a certain group or another and so on.

Media authentication

Media authentication is the trace of this source in Media reports: the source in fact may be quoted by trusted/bias media outlines whether this be in the mainstream/local media or social media space. In this case the evaluation of the reliability of this media as a source, can tell us something about the person we are scrutinizing. For instance a media that is bias towards the government will use a source that is also bias towards the government, while an independent media will most likely rely on independent sources. Google or any other research engine will be useful to find out if the source you are looking for is quoted in any online medias or social network.

Direct engagement

This is the most difficult part of the verification process and it is suggested to use it only after a very careful considerations of the risks involved in a direct contact on line with the source. For example on Twitter, you can tweet the source back and ask them to identify the source of what they are reporting (suggested to use DM for this). NPR's Andy Carvin has employed this technique particularly well.This technique will force the source of the message to either give you more information to identify themselves or to find additional sources to validate his/her information

Crowdsourced verification of

the source

Social authentication

Triangulation

Social Authentication

The Social authentication refers to the use of social networks to identify or validate the source of the report. If you're still unsure about the source's reliability, you can use:- Your own social network (Twitter, Facebook, LinkedIn) -to find out if anyone in your network know about the source's reliability- The entire social network to find out the reliability of the source, for example by tweeting something like “ Does anyone know about this person/account?”

Triangulation

We have seen already how triangulation of information works. The main point is to put together all the information that we are able to gather by looking at the previous information, and then put them together trying to find discrepancies or compatibilities in between them.The triangulation of information needs to be based on 2 very important assumptions:- You will never have 100% certainty of the reliability of the sources unless

you know them personally- Each information can tell you everything and the contrary of everything. - All the technics you are using to verify information, may also be used by

someone else to falsify your information

For this reason there are several things that you need to keep in mind when applying all the methodologies described here:1. Ask yourself all the possible questions, and never give anything for granted2. As Arasmus notes, "remain skeptical about the reports that you receive.

Look for multiple reports from different unconnected sources." The more independent witnesses you can get information from, the better.

3. If after doing all the necessary verifications of the report, you find out that the user reporting an event is not necessarily the original source, can the original source be identified and authenticated? In particular, if the original source is found, you still need to go through the same steps highlighted here.

4. Run multiple verifications for the same user and, even better, ask others to run the same verification processes on the same account. Personal beliefs and state of mind can change how we interpret the same information.

5. In general, keep in mind that the less you find about someone the less reliable you can consider that source.

THE VERIFICATION OF THE CONTENT

Once the source of an information has been authenticated, we need to verify if the content is actually true. This process is very time consuming because the description of an actual fact can be presented in four different ways:- It can be accurate in all aspects- It can be not accurate but still referring to an actual fact- It can be accurate in all factual aspects but misleading in terms of the

context- It can be completely falseTo verify the content of an information, we need to look at the so called 5W and 1H.

Online verification of the content

Falsifying the content

PARTIAL FALSIFICATION The

information is partially false.

There is one (or more) of the 5W1H that is false while some others are

true.

Falsi

ficati

on o

f the

Con

tent

Online verification

of the content

Language

Pictures/Video

Follow up with source

Triangulation

Time and place

Content ID on Social Network

Content5W1H

WHO who are the

parties involved. Who are the

victims and who are the

perpetrators.

WHATwhat actually

really happened

WHERE the location of the fact reported

WHEN when the fact

reported happen

WHY why did the

fact happened

HOW the context

and the dynamics of

the actual fact

The verification of the content starts with the verification of the different components of the information. For each of the 5W1H you will need to look at that specific information and ask yourself several questions to make sure that each of those correspond to the information reported. Some of those questions will not be answered directly in the report, but will need from your part an online search for other sources reporting the same event. See the Triangulation slides for more on this.

The first step is to verify the content

of the information,

looking at the 5W1H

WHOwho are the

parties involved? Who are the

perpetrators? Who are victims? WHERE

where did the fact happen? Is

there any reference to the

actual place? Addresses? Landmarks

mentioned?

WHEN when the violation

happen? Is the time of reporting close to the time of the actual fact

reported?WHAT

what did actually happen? Can we

re-built the entire dynamic of the

event with all the details of the

event?

WHY is there any

information on why the fact

happen or about the intention of

the perpetrators?

HOWHow did the fact

happen? Is it possible to make

connections between the

different components of

the event?

Language is a very important factor of the information reported that you may want to look at. Some of the particular things related to the use of language that may give you some indication about the veracity of the information reported are:•Use of dialectal words•Use of official language like “breaking news”, “urgent”, “confirmed” etc. •The specific vocabulary, slang, accents, which needs to be cross checked, to see if they are correct for the location that a source might claim to be reporting from.

Language

Is the language used in the report neutral or the person is using lots of adjective and

qualitative judgments on the information

reported?

Is the language used a proper language or a dialect? Are there

particular inflections, slang words or

grammatical mistakes in the report?

Is the language used in the report technical. For

example using a lot of journalistic language

or human right language?

If the report you receive is also paired with a

photographic or video “evidence”, you need to look

into the photo/video

provided to search for any clues about

the 5W1H:

WHOIs there any indication of who are the

people in the video?

WHATWhat are they

doing in the video/picture? Can you assess

that with no doubts?

WHERE is there any sign

of where the video/picture

was taken?

WHENIs there any indication of

when the video/picture

was taken?

WHYIs there anything that can tell you

what is the motivation of

the people involved in the

action?

HOWWhat is the

context and the dynamics of the

actual fact?

Video• Is there any sound that you can connect with the place? For example sounds

of cars passing by, people talking in the background?What are the sounds on the

background?

• If they do, in which context? Do those places/people means something in the context of the action taking place?

Do people mention names of people or places in the

dialogue?

• Can you identify buildings, signs, cars, etc., in the background?Is there any indications of the

place based on the action taking place?

• For example, check weaponry against those known for the given country.Are there objects in the video that may give you additional

information?

• Examine weather reports to confirm that the conditions shown fit with the claimed date and time. Also looks for shadows to determine the possible time of day that a picture was taken.

What is the weather like in the video?

• Language used, tone of voice of the people in videos footages can also be a good indication of the place and context.

Which language is the dialogue taking place in?

Pictures

Is there an indication of the place where the

photo was taken? A sign, a particular

place marker?

If yes, look for other

pictures on line to make

sure that that place

really looks like that. Google

street view for example

is a good way to cross-

check streets and

public places

How are the people in the

photo dressed? Does this fits with the place where

the photo is supposed to be

take?

Search on line for the meaning of

those dresses:

does particular outfit tells

you who this person is or who he/she belongs to?

Where is the light coming from in the

picture? Does it seems artificial?

Cross-check that the

direction of the light is indeed in

line with the position of the sun at that time (like from above if

midday, etc)

Are there objects in the picture that

do not fit in the overall scene?

Look very carefully at the objects

on the background, the corners

of the image and the floor.

Is the picture very clear and

professional or does it looks like it was taken with a

phone?

There are indeed

phones that take very

clear pictures, but

in general double

check if a picture it’s too perfect

and too clear.

As explained before, this is the most difficult part of the verification process and it is suggested to do it only after a very careful considerations of the risks involved in a direct contact with the source. This type of verification can happen in two ways:-Online-Offline The best situation is when the verificators already have contacts in the geographic area of interest, so that they can ask them to follow up directly/in-person to confirm the accuracy of the report. Obviously this is not always possible, particularly in conflict zones. In using this technic it is very important to keep in mind that the people verifying the information on the ground may not be able to do so directly, but still they could gather first hand witnesses reports based on their local network.

Follow up with source

The verification of the information provided online is basically based on the creation of a dialogue with the sender of the information. If for example someone has reported something on Twitter or Facebook you can simply write back to them to ask for more details about the event reported. In this case it is suggested to do it privately (via Facebook mailing box or Twitter DM). Before engaging in any online conversation with anyone it is suggested also to inquiry as much as possible if this is safe and if the person reporting will not be put in danger by you contacting them.

On Line Contact

The idea of contacting personally someone is related to the possibility to have someone on the ground that is trusted and safe to get in contact with. If such a contact exists then this person (or network of persons) can be used to directly get in contact with the reporters to inquiry about the issue reported. Again, in this case very careful security and privacy considerations needs to be done. The use of this strategy to verify information is suggested as the last resource especially because both the trusted person on the ground and the reporter’s lives can be put in extreme danger.

Offline Contact

The time and place of the event normally present the most problematic issues to verify since they are the once that can be affected by the re-reporting of the information by different sources. The idea here is that most of the time we are in front of a “Chinese whispers” type of situation and we need to make sure that those two variables are reported correctly.

Time and place

Time and Plac

e

Spelling: if the original language of the reporter it is different than the one of the report, you can find that a place

may refer to several locations due to spelling

issues. One example is Arabic,

where the numbers of letter As in the alphabet are several, while in English there is

only one.

Edited Re-Tweets: when you find

tweets reporting something try to

do as many searched as

possible to find out what the

original tweet was saying. People tend to edit re-

tweets when they think there is

something wrong (like spelling of

names) and in this way they may

actually change a correct name.

Format of dates: the format of

dates are different form place to

place, so people may translate 09/08/12 as 9

August 2012, or as 08 of September 2012. If people

repost a message and decide to

change the format of the date, they

may not know which one of the two version they

are using.

The triangulation of information is basically the act of putting together all the information you can find about the issue reported and cross-check them against the fact as reported to you.To do this it is suggested to start filtering all possible information sources available: social media, websites, blogs, local and international media, TVs, and so on.In this tasks, make sure you always remain skeptical about the reports that you receive. Look for multiple reports from different unconnected sources, since the more independent witnesses you can get information from, the better and the less critical the need for identity authentication. If after doing all the necessary verification of the report, you find out that the fact is reported by other people but with different characteristics, like time and location for example, are not matching, do not discard immediately the fact as false but try to identify which version of the report is the most accurate. If you cannot find confirmation for any of the versions, do not chose the most likely one, but make sure you report all versions and highlight which difference they have. The verification of the context often can help in the identification of the those partial falsifications. Run multiple verifications for the same user and, even better, ask others to run the same verifications processes on the same account. Personal beliefs and state of mind can change how we interpret the same information. In general, keep in mind that the less you find about something that less reliable you can consider that information, and the more you have to rely on people on the ground to confirm it.

Triangulation of content

When you are crowdsourcing information about a specific event or place, try to look in the social network about that same event. If you do not find anything related to that event, search for events in the same area or for issues that may be related to that event. In the social networks normally events tend to be repeated and reported in several platforms (Twitter and Facebook for example) so this process will also help you identifying possible editing of the event reported done by different users.

Content ID on Social Network

Crowdsourced verification of the content

Social authentication

Trusted Network on the ground

The creation of Virtual Trusted

Networks (active and

passive)

When you are crowdsourcing information about a specific event or place, you should start finding an online social networks related to that specific issue. For example in Twitter, that specific social network will be the one twitting very on a specific hashtag (#).You can use that social network to ask information about how unusual the event reported is, to get a sense of how likely it is to happened in the first place. This online community, for example, can help you translate, triangulate, and track down key information. They can enable you to crowdsourced verification but you must always check and challenge what you are told. Ask people what they think, what do they know about that area, or about that event. You have also to remember that you must NEVER give an indication of what your source is, if it is not public already, and try not to give too many details of the information reported if that information is not public already.

Social authentication

When verifying information online the use of an offline network can be very helpful. As mentioned before, this network can be your ears and eyes on the ground, but they can also incur in considerable risks by doing so.For this reason those networks need to be very trusted, possibly people that you personally know and it should be open only to other people known and trusted personally.As already said in the module, security issues should be prioritize over everything else.

Trusted Network on the ground

In several occasion, the creation of a trusted virtual network has been done by several activists around the world to verify crowdsourced information. This network has been created by starting with a group of very trusted people on the ground (personally known and contacted) and then expanded only to people vetted by this same group of people, and only if personally met. To do this there are several tools that have been used, like Skype (better if encrypted since Skype is indeed vulnerable to monitoring by third parties), Google groups, encrypted emails mailing lists, etc.

The creation of Virtual Trusted Networks (active and passive)

It is also possible (and suggested) when doing crowdsourcing online to create a list of the sources that provide information online (directly to you or via social networks) that you can keep updated during your project. This list can also be compiled by adding comments to each of the accounts/people listed and in this way starting to create your reliable list of sources and “scoring” them according to your verification. This list will also be very useful for all the processes done before, and especially once you have done the source verification for those account and can attest the reliability of the source.

The creation of Virtual Trusted Networks (active and passive)

THE VERIFICATION OF THE CONTEXT

Context Matters

The understanding of the context of an information it is crucial to give a sense to what has really happened. Specifically in the context of crowdsourced information the context become extremely important because we may not know the source and we may have to find out if the information is actually true. On the other side though, even if the fact reported really did happened and the source is reliable, we may be in the situation where the context of the information can give us a completely different idea of what is the overall picture surrounding the event.

Why Context matters?

People develop their opinions based on the information available to them, and this is how we evaluate the verification processes of an information. Sometime though if the fact is not presented it in the right context, the same information may be evaluated in very different ways. The idea here is that once we have attested that a fact has happened, we need to make sure that every fact relevant to the story, which is a necessary part of the story, is also reported. In the context of this module we will look at two different ways that you can use contextualization technics to verify information:1. Find data that can further support or give you an idea of the

veracity of the information2. Find data that can give a meaning to the information reported

On line Verification the context

Factual context

Global Context

Time and

Location

Time and Location

We have already seen before how time and location can help us identify the veracity of an information, but in this case we will look at them as important factors for the contextualization of the event reported.Sometimes the time and location of an event make a huge different in considering the reliability of the information reported. What we can look at here is if the event reported makes sense considering the time and location reported.

Time

Does the time of the event make sense for the event reported?

For example: if an attack to a street vendor is reported, does the time of

the event reported make sense considering the type of economic

activity?

Also, does the time gives a specific meaning to the event? Like an

attack to a mosque on Friday? Or to a Synagogue on Saturday?

Location

For example, does the event take place in a location where it is very unlikely to find the actors involved?

Does the location have a specific meaning for the type of event

reported?

Does the location have access or closure times that do not fit with the time of the event reported?

Does the location of the even make sense for the type of event

reported?

Factual context

The first thing that we need to look at when looking at the context is to try to place the event we are analyzing in a greater picture. The picture in this page is an example. If you think about how you would describe the picture, the context of the event is giving you a completely different information depending on if you look at the right or left part of the picture. The event happening is always the same though. This is what we mean when we say that you have to look at the factual context.

Global context

The global context refer to the international context around that event. Sometimes political crisis outside a country can trigger events in a third country. One example can be the sudden explosion of tensions in between two neighboring countries that can push a third one to close its borders or move contingents of the army towards the borders. The global context is also something related to the important facts that are happening in the same country or in the same location of the event reported. The national context of a reported event in fact and when it does happen, can have a relationship with the reasons behind the event, that can help us understand if the event is likely to be true or if it can be triggered by other events happening in the same place or at the same time.

Is there any other event similar to this one that has happened in the same time

or area?

For example something very similar reported in the same

area

Like if an attack is reported, are there other attack been reported in the same area

before or after?

Or something that could have triggered that event

(like a retaliation or an attack)

For example, massive arrests of activists may be following

the dead of a police man during a demonstration

Or something that could have influenced the way

people described what they have seen (for example

something very shocking or inflammatory)

In this case people may report event to you exaggerating the

event because they are in a very emotional state of mind

following a very shocking event

Look at what is going on the region, but also globally in the world

Is there any political issues happening that can be related to

the country? Like important bilateral meetings? Or peace

treaties? Or important speeches being delivered?

Are there economical issues/ factors that can have an influence in

the country? Sudden rise in unemployment? Sudden rise in

price of oil?

There has been any events happening in neighboring countries

that may have lead to reaction in the place you are considering? For example sudden rise in tension in

between two countries? Or an economical agreement reached?

WHAT TO KEEP IN MIND

The 9 golden rules

1. Unless you see it and you hear it, it is not 100% verified.

2. After rule I, chose your verification criteria in advance and make them public (if possible)

3. Chose your trusted network and make clear rules for being part of it

4. Doubt and question everything, always

5. Be always ready to change your mind about the reliability

of a source6. Be always ready to change

your mind about the veracity of an information

7. You don’t need to verify everything. Choose you battles

8. Always admit when you fail or when you do a mistake

9. What you know is also what your enemies know. Keep this always in mind when verifying information

Keep your reputation intact

Since we said that rule number one for the verification of crowdsourced information is that unless you see and hear it, it is not 100% verified, you need to protect your reputation and your credibility every time you use, republish, collect or advertise crowdsourced information. To this, you need to be very transparent about your verification criteria and about what degree of verification you can achieve with the methodologies you are using. This also means that you do not have to decide just in between verified and not verified (like 0-100) but you can also choose to use several degrees of verification in the middle.The main important things is that you make very clear to whoever is taking that information from you what criteria you used and how are you verifying your sources and information. A second very important point, as expressed before, is that you need to always be ready to admit when you did a mistake in your verification process and be ready to change your protocols according to the situation.

HOW TO CREATE VERIFICATION PROTOCOLS

You do not necessarily need to verify everything. Choose what are most important information by looking at the typology and amount of information that you received. Make a considerate evaluation of your capacity. It is better to have few very well verified information, then a lot of quasi verified information.

Step I: Chose what you need to verify

Create your criteria for verification in advance and by looking precisely at what means you have to verify information. Ask yourself:1. When is that you will consider an information

verified?2. When is that you will not consider it verified?3. What degrees of verification are you willing

to have in the middle?

Step II: Choose your verification criteria

Choose very carefully who you will consider a trusted source. This does not necessarily need to be mainstream media or known people.In addition to this you need to update your trusted sources list periodically (I would say on a daily bases) to make sure that you add new sources and review the trustfulness of old sources.

Step III: draft your trusted sources list

Once you have drafted your verification protocols, make sure you always apply them to all information you process. Do not allow yourself to do exceptions to it, and make sure that all your information are verified following the same procedures. This is very important to assure consistency and to be able to always get back to your verification process to refine where it is not working.

Step IV: Draft your verification protocols and stick to it

One of the main important things about your verification protocol is to make it as public as possible. The reason behind this is not to inform people how they can go around it, but to assure that people do know what sort of credibility your information have. You do not have to make the all details public, but people using your information need to know exactly how much they can trust your information or not. The more they are aware of the verification protocols you use, the more you will be able, if necessary to explain eventual mistakes or failure in verifying information, as well as managing expectations about the reliability of the information posted.

Step V: make it public

EXAMPLES

Blogging the Egyptian Election “BEE” 2010

U SHAHID PLATFORM

Step 1: what to verify

One of the main problem of verification of crowdsourced information is related to the fact that the overload of information can make it impossible to actually do a meaningful verification of the information received. For this reason, the U-Shahid team’s first step was to define concrete criteria for what types of reports required verification. They didn’t attempt to verify all the content that was available, but only the one that was important or critical. 1. Reports related to an immediate threat or act of violence2. “Grave electoral fraud” the gravity of which was assessed by the importance of the parties involved (celebrities, known PMs, government officials) or the gravity of the fraud (involving violence, or massive votes exchange) or it’s location (inside the pooling stations)

The U-Shahid team also created some criteria that

all reports need to be checked against to be

considered as verified. If one or more of the criteria was met, then the report

would have been considered verified

The report was supported by a

video or pictures that clearly

confirmed what was been reported

(especially location);

It was been reported by two or more

independent sources (the list of

independent sources was created in advance and

updated by the team) ;

Messages coming from social media (Twitter, Facebook

and blogs) needed to be confirmed by an SMS, a media report or a direct witness;

At least one of the sources of the

information was clear and known (i.e., 2 SMSs from unknown sources could not verify

each other)

For reports that had to be verified but did not

meet the criteria outlines before, the

verification team was engaging directly with

the source in 4 different ways

If the report was sent by SMS, that

number was called to verify the person’s identity. The witness

was asked if they observed the event

themselves or if they simply learned about

the event from someone else. More specifically, details on the 5w1h are

asked. If the event being reported was still unfolding, the

witness was asked if anyone else nearby

was able confirm the information.

By asking to provide a video or picture of the event but only if it was safe to do so.

If the report came from Twitter itself, the account of the tweeter was reviewed. Simple content analysis of previous tweets and the account holder’s bio was

carried out online, in addition to revised the Tweeter’s

followers.

By tweeting back to the original Twitter

user asking for more information—again using the “who did

what, to whom, how and where” format, asking followers to

confirm if the information was indeed correct. St

ep 3

: Dir

ect

cont

act

wit

h th

e so

urce

Step 4: In person verification

The second core strategy involves in-person verification via a trusted

network of monitors on the ground:

The U-Shahid team determines whether one of their election monitors (trained by them) was close to the area

referenced in a report that required verification. If a monitor was on site or close enough, that person was

asked to verify the report.

If U-Shahid did not had any monitors in the area, the team was activating their NGO partners who

may had monitors in the area. If so, those individuals were asked to confirm the validity of

the report being investigated.

If no one of the two possibilities was possible, U-Shahid was using a network of trusted journalists

in the area to ask them to go to the place to confirm the information for them.

Step 5: online triangulation

If no one of the previous possibilities was available, the team was using the mainstream media and online tools for confirmation. They used web-based research to look for any evidence that is specific to the event that was reported as well as that location. They looked for article, blogs, social media, video or pictures, mainstream media articles and TV shows that could confirm the information reported.

In addition to this the U-Shahid team was then triangulating the report being investigated with other

reports they had already received.

OUTCOMESUshahid received on election day November 28, 2010 during voting period 1252 reports, documented with 287 videos and 111 images. By the end of the project on Dec 28, 2010 a total of 2700 were uploaded on the platform with an average of reports per day of 27 and a percentage of verified reports of 91.26%.

Electua 2012

http://www.youtube.com/watch?v=fdFZYDf9AdA

http://www.youtube.com/watch?feature=player_embedded&v=LsOo3jzkhYA

http://www.youtube.com/watch?feature=player_embedded&v=UI8S5kO-vEM

THANK YOU

Anahi Ayala IacucciMedia Innovation Advisor

aayala@internews.org@anahi_ayala

@info_innovation