Listening to the CrowdAnahi Ayala Iacucci – Media Innovation Advisor
Crowdsourcing information for elections
How to verify information collected online during elections
WHAT DO WE MEAN WITH VERIFICATION AND WHY WE DO IT?
Verification and falsification
The goal of verification of crowdsourced information is to assess whether the information reported is indeed referring to an actual fact that happened exactly in the same way it is reported. For this reason the verification process needs to happen as a multi-level approach, where all components are analyzed separately and then merged.
Types of Verification and Falsification of crowdsourced information
Falsifying/Verifying the content Falsifying/Verifying the sourceFalsifying/Verifying the context
Content5W1H
WHO who are the
parties involved. Who are the
victims and who are the
perpetrators. WHATwhat actually
really happened,
what violation has been made
WHERE the
location of the fact
reportedWHEN when the
fact reported happen
WHY why did the fact
happened
HOW the context
and the dynamics of the
actual fact
What is the content?
Falsifying the content
PARTIAL FALSIFICATION The
information is partially false. There is one (or more) of the 5W1H that is false while some others are true.
Verifying the content
The first step is to verify the content
of the information, looking at the
verification of the 5W1H
WHOwho are the
parties involved? Are really the
perpetrators the once mentioned
in the report? Who are the
victims?
WHEREwhere did the
fact happen? Is there any
reference to the actual place? Addresses? Landmarks
mentioned?
WHEN when did the
violation happen? Is the time of
reporting close or not to the time of
the actual fact reported?WHAT
what actually did really happen? Can we re-built
the entire dynamic of the
event?
WHY is there any
information on why the fact
happen or about the intention of
the perpetrators?
HOWhow did the fact happen? What is the connection in
between the different
components of the event?
Types of information
sources
Primary sources
the source is the
eyewitness of the fact
Secondary sourcesthe source is
reporting something based
upon evidence from primary
sources
Tertiary sourcesCompilation based upon primary and secondary
sources
Falsifying the sourceThe falsification of the source is often used to undermine the reliability
or reputation of a source or to gain trust from the audience
Falsifying to gain trust
This is often done to add credibility to the information reported by attributing it to a source that will be more trusted than the
reporter himself. For instance one common example of this is when secondary sources
present themselves as primary sources
Falsifying to undermine trust
On the contrary this type of falsification is done to attribute to a
source a certain information in order to make them look less credible, or
even personally involved in the events
Falsifying the context
The verification of the context is one of the most difficult things to do because certain information may be true in the content, and the source may be a reliable one, but the context may give to the same information a different angle and therefore a completely different meaning. See this picture as an example:
THE ONLINE VERIFICATION OF THE SOURCE
Online verification
of the source
Social Media ID ID of the
Content on the Social
media
ID of the Network
ID of the trusting network
InfluenceLocation
Timing
Media authentication
Direct engagement
The Social media ID
When getting an information from a social media source the first thing that we need to look at is the bio of the source (account) as provided in its home page. In this first step we will look only the Identity of the person to try to re-create his/her ID based on his/her social media bio. We can look at several different social medias: Twitter, Facebook, You Tube, Flickr, Linkedin, etc.
Bio on Social Network
Is there a name and surname?
Look for the name/surname on
Is there a link to a blog or a website?
Check the sites and see what content
they have and what bios they have
Is there a link to the Facebook/Twitter/Li
nkedIn/You Tube/Flickr Page?
Check the pages and the bios there
Are there pictures? Videos? Lists of previous jobs or
schools attended?
Check all of those and see what they can tell you about
this person
Evaluate what do you find trying to understand as much as possible about this person identity: does he/she seems to be linked to any political party? Does he/she work for a specific company and if yes what is this company doing exactly? Does he/she have a picture, and if yes, does the picture tells you something about his/her political/religious affiliation? Also make sure that all information in blogs/websites/Twitter bio and so on report the same information: any discrepancies can be a sing of an identity falsification.
ID of the content on social media
The second step of the online verification of the source is to look at the content of the source as provided in its home page. After having checked his/her Bio from the previous section, now we will look more closely to his content on social media, what he is talking about, what his ideas may be regarding certain issues and so on. This process is done to add more information to the bio of this person and to try to get a broader picture of what this person is interested in. In this section the methodology is the same for all social media previously considered.
The first step here is to make a list of
his/her social media, or online
presence. An example can be:
Facebook page
You tube channel
Flickr account
Linkdin page
Website/Blog
The second step is to closely look at the content.
Is the content relevant to the issue reported?
Does this person seems
to be committed to a certain cause or
particularly interested in a certain topic?
What other topics is he/she posting about?
Does the content seem reliable? Is
there content that does not seems to have
been supported by other sources?
Does he has strong opinions
related to certain topics?
And if yes, what his opinions
are?
Is this a one-issue tweeter?
In this process make sure that you take a look at all possible type of content posted on social media: videos, photos, comments to other people content, articles, re-tweets and forums. Most of us have themes running through our interests, but a tweet/messages history that is completely consumed with one or two issues without the occasional personal reference, humor or reference to other topics is suspect. Behind every social media account should be a personality.
In this third phase we are looking at the
social network that this person has. In this context we will look at the people he/she is interacting with and
the friends he/she has on social networks
Who are his/her friends?
Do they share the same type
of content?
Do they seem to be related or
linked to any particular
group/party or group?
Do they share his/her views
on specific topics or not?
ID o
f the
Soc
ial N
etw
ork
ID of the trusting network
This fourth step is related to the people this person trust and that he seems to be influenced by. What we are trying to see here is how to dissect the social network and look more closely to the network of people that this person trusts.
Twitter/Facebook
Who are the people that this
person is re-tweeting most
often?
Who is he/she following?
What are these people tweeting about and what are the topics he
interacts with them about?
Who are the people that
he/she is interacting with
most often?
Linkdin/Blog
Who are the people that have given
recommendations to this person?
Who are the people in his
network?
Who are the people that
he has worked for and that he knows?
Influence
In this phase of the verification process we will be looking at the people that this person influence and that trust him as a source. This information will give us a idea of what is the influence that this person has on his/her social media network.
Who are the people that are re-tweeting this
person most often?
Who are the people
following him/her?
What are these people
tweeting about and what are
the topics they interacts with
him/her about?
Who are the people that are interacting with
this person most often?
Location
In this section of the verification phase we want to verify the location of this person in the real world. The reason why we do this is to be able to define if this person is reporting as a primary source, a secondary source or a tertiary source. The actual location of a person in the virtual world may be really tricky so we need to proceed by steps.There are two types of location services that are associated with social media:- Manual location settings- Smart phone location settings
Manual Location settings
Some social media have the location already set up in their settings, if the user has allowed it. For example, Twitter has this option in the profile page:
This function though is a manual function meaning that it is the user that manually set the place where he/she is based and where he/she is from. Most of the time this location is the home-town of the person, and not necessarily where he/she is tweeting from.
The same is valid for Facebook location settings, that you can find in the About page of the person:
When people are using their smart phones to use Social Media, they can unable the automatic location of their messages, both on Twitter and Facebook. In this case the location is the one of the mobile phone, so it does correspond to the actual location of the person at the moment when the message was sent.
The Smart Phone Location settings
Triangulation of the location
1. Check the location of the person on the manual
settings (if available)
Is the location the same of the location mentioned in
the message you are verifying?
NO: the person may be on a trip or may have moved
from the original place
YES: pass to the verification processes2,3,4
2. Check the location on the smart phone settings
(if available)
Is the location the same of the location mentioned in
the message you are verifying?
NO: this person is probably a secondary or tertiary
source
YES: most likely this person is a primary source. Pass to
step 3 and 4
3. Cross-reference locations from section 1
and 2 (if available)
Are the locations the same?
NO: the person may have been moving from the
original location
YES: The person is probably a primary source
4. Cross reference the location mentioned in the message you are verifying
with the location mentioned in other
messages from the same person
Are those location the same or in the same area?
NO: if the locations are pretty far, this person is probably a secondary or
tertiary source
YES: This person is most likely a primary source
Language/Location cross-reference
This is a second step to be done when looking for the verification of the location of your source. In this phase you can look at verifying whether the vocabulary, slang, accents are correct for the location that a source might claim to be reporting from. For instance, if the reports are coming from a very rural area and the language is a perfect English, you may want to further verify the source. The same if a certain source is always reporting in a certain language and then suddenly you start receiving reports in another language (this may indicate that those reports are copied or taken from someone else).
Timing
The time is an important characteristic of the information you are looking at. This will also tell you something about the possible location of the person sending the message out. In almost all social medias and blog/websites there is a time stamp of the publication, but as in the location, this stamp can also be misleading with regard to the actual event reported. For this reason we need to proceed with the triangulation of this information to get closer to an estimate of the real time of the event and therefore to the reliability of the source.
Check time stamp
Is the time close to the time of the event?
YES
The source may be a primary source
The source may be not a primary source but
someone very much following the event closely
NO
The source may still be a primary source but just had
no access to communication means
The source may be a secondary or tertiary
source
Time of account creation
This part of the verification process is intended to look at the existence of the account before the time of the reporting. The idea is to find out if the Twitter/Facebook handle is a new account, with only a few tweets. If so, this makes authentication more difficult. Arasmus notes that "the more recent, the less reliable and the more likely it is to be an account intended to spread disinformation." In general, the longer the Twitter/Facebook handle has been around and the more Tweets/posts and people linked to this handle, the better. This gives a digital trace, a history of prior evidence that can be scrutinized for evidence of political bias, misinformation, belonging to a certain group or another and so on.
Media authentication
Media authentication is the trace of this source in Media reports: the source in fact may be quoted by trusted/bias media outlines whether this be in the mainstream/local media or social media space. In this case the evaluation of the reliability of this media as a source, can tell us something about the person we are scrutinizing. For instance a media that is bias towards the government will use a source that is also bias towards the government, while an independent media will most likely rely on independent sources. Google or any other research engine will be useful to find out if the source you are looking for is quoted in any online medias or social network.
Direct engagement
This is the most difficult part of the verification process and it is suggested to use it only after a very careful considerations of the risks involved in a direct contact on line with the source. For example on Twitter, you can tweet the source back and ask them to identify the source of what they are reporting (suggested to use DM for this). NPR's Andy Carvin has employed this technique particularly well.This technique will force the source of the message to either give you more information to identify themselves or to find additional sources to validate his/her information
Crowdsourced verification of
the source
Social authentication
Triangulation
Social Authentication
The Social authentication refers to the use of social networks to identify or validate the source of the report. If you're still unsure about the source's reliability, you can use:- Your own social network (Twitter, Facebook, LinkedIn) -to find out if anyone in your network know about the source's reliability- The entire social network to find out the reliability of the source, for example by tweeting something like “ Does anyone know about this person/account?”
Triangulation
We have seen already how triangulation of information works. The main point is to put together all the information that we are able to gather by looking at the previous information, and then put them together trying to find discrepancies or compatibilities in between them.The triangulation of information needs to be based on 2 very important assumptions:- You will never have 100% certainty of the reliability of the sources unless
you know them personally- Each information can tell you everything and the contrary of everything. - All the technics you are using to verify information, may also be used by
someone else to falsify your information
For this reason there are several things that you need to keep in mind when applying all the methodologies described here:1. Ask yourself all the possible questions, and never give anything for granted2. As Arasmus notes, "remain skeptical about the reports that you receive.
Look for multiple reports from different unconnected sources." The more independent witnesses you can get information from, the better.
3. If after doing all the necessary verifications of the report, you find out that the user reporting an event is not necessarily the original source, can the original source be identified and authenticated? In particular, if the original source is found, you still need to go through the same steps highlighted here.
4. Run multiple verifications for the same user and, even better, ask others to run the same verification processes on the same account. Personal beliefs and state of mind can change how we interpret the same information.
5. In general, keep in mind that the less you find about someone the less reliable you can consider that source.
THE VERIFICATION OF THE CONTENT
Once the source of an information has been authenticated, we need to verify if the content is actually true. This process is very time consuming because the description of an actual fact can be presented in four different ways:- It can be accurate in all aspects- It can be not accurate but still referring to an actual fact- It can be accurate in all factual aspects but misleading in terms of the
context- It can be completely falseTo verify the content of an information, we need to look at the so called 5W and 1H.
Online verification of the content
Falsifying the content
PARTIAL FALSIFICATION The
information is partially false.
There is one (or more) of the 5W1H that is false while some others are
true.
Falsi
ficati
on o
f the
Con
tent
Online verification
of the content
Language
Pictures/Video
Follow up with source
Triangulation
Time and place
Content ID on Social Network
Content5W1H
WHO who are the
parties involved. Who are the
victims and who are the
perpetrators.
WHATwhat actually
really happened
WHERE the location of the fact reported
WHEN when the fact
reported happen
WHY why did the
fact happened
HOW the context
and the dynamics of
the actual fact
The verification of the content starts with the verification of the different components of the information. For each of the 5W1H you will need to look at that specific information and ask yourself several questions to make sure that each of those correspond to the information reported. Some of those questions will not be answered directly in the report, but will need from your part an online search for other sources reporting the same event. See the Triangulation slides for more on this.
The first step is to verify the content
of the information,
looking at the 5W1H
WHOwho are the
parties involved? Who are the
perpetrators? Who are victims? WHERE
where did the fact happen? Is
there any reference to the
actual place? Addresses? Landmarks
mentioned?
WHEN when the violation
happen? Is the time of reporting close to the time of the actual fact
reported?WHAT
what did actually happen? Can we
re-built the entire dynamic of the
event with all the details of the
event?
WHY is there any
information on why the fact
happen or about the intention of
the perpetrators?
HOWHow did the fact
happen? Is it possible to make
connections between the
different components of
the event?
Language is a very important factor of the information reported that you may want to look at. Some of the particular things related to the use of language that may give you some indication about the veracity of the information reported are:•Use of dialectal words•Use of official language like “breaking news”, “urgent”, “confirmed” etc. •The specific vocabulary, slang, accents, which needs to be cross checked, to see if they are correct for the location that a source might claim to be reporting from.
Language
Is the language used in the report neutral or the person is using lots of adjective and
qualitative judgments on the information
reported?
Is the language used a proper language or a dialect? Are there
particular inflections, slang words or
grammatical mistakes in the report?
Is the language used in the report technical. For
example using a lot of journalistic language
or human right language?
If the report you receive is also paired with a
photographic or video “evidence”, you need to look
into the photo/video
provided to search for any clues about
the 5W1H:
WHOIs there any indication of who are the
people in the video?
WHATWhat are they
doing in the video/picture? Can you assess
that with no doubts?
WHERE is there any sign
of where the video/picture
was taken?
WHENIs there any indication of
when the video/picture
was taken?
WHYIs there anything that can tell you
what is the motivation of
the people involved in the
action?
HOWWhat is the
context and the dynamics of the
actual fact?
Video• Is there any sound that you can connect with the place? For example sounds
of cars passing by, people talking in the background?What are the sounds on the
background?
• If they do, in which context? Do those places/people means something in the context of the action taking place?
Do people mention names of people or places in the
dialogue?
• Can you identify buildings, signs, cars, etc., in the background?Is there any indications of the
place based on the action taking place?
• For example, check weaponry against those known for the given country.Are there objects in the video that may give you additional
information?
• Examine weather reports to confirm that the conditions shown fit with the claimed date and time. Also looks for shadows to determine the possible time of day that a picture was taken.
What is the weather like in the video?
• Language used, tone of voice of the people in videos footages can also be a good indication of the place and context.
Which language is the dialogue taking place in?
Pictures
Is there an indication of the place where the
photo was taken? A sign, a particular
place marker?
If yes, look for other
pictures on line to make
sure that that place
really looks like that. Google
street view for example
is a good way to cross-
check streets and
public places
How are the people in the
photo dressed? Does this fits with the place where
the photo is supposed to be
take?
Search on line for the meaning of
those dresses:
does particular outfit tells
you who this person is or who he/she belongs to?
Where is the light coming from in the
picture? Does it seems artificial?
Cross-check that the
direction of the light is indeed in
line with the position of the sun at that time (like from above if
midday, etc)
Are there objects in the picture that
do not fit in the overall scene?
Look very carefully at the objects
on the background, the corners
of the image and the floor.
Is the picture very clear and
professional or does it looks like it was taken with a
phone?
There are indeed
phones that take very
clear pictures, but
in general double
check if a picture it’s too perfect
and too clear.
As explained before, this is the most difficult part of the verification process and it is suggested to do it only after a very careful considerations of the risks involved in a direct contact with the source. This type of verification can happen in two ways:-Online-Offline The best situation is when the verificators already have contacts in the geographic area of interest, so that they can ask them to follow up directly/in-person to confirm the accuracy of the report. Obviously this is not always possible, particularly in conflict zones. In using this technic it is very important to keep in mind that the people verifying the information on the ground may not be able to do so directly, but still they could gather first hand witnesses reports based on their local network.
Follow up with source
The verification of the information provided online is basically based on the creation of a dialogue with the sender of the information. If for example someone has reported something on Twitter or Facebook you can simply write back to them to ask for more details about the event reported. In this case it is suggested to do it privately (via Facebook mailing box or Twitter DM). Before engaging in any online conversation with anyone it is suggested also to inquiry as much as possible if this is safe and if the person reporting will not be put in danger by you contacting them.
On Line Contact
The idea of contacting personally someone is related to the possibility to have someone on the ground that is trusted and safe to get in contact with. If such a contact exists then this person (or network of persons) can be used to directly get in contact with the reporters to inquiry about the issue reported. Again, in this case very careful security and privacy considerations needs to be done. The use of this strategy to verify information is suggested as the last resource especially because both the trusted person on the ground and the reporter’s lives can be put in extreme danger.
Offline Contact
The time and place of the event normally present the most problematic issues to verify since they are the once that can be affected by the re-reporting of the information by different sources. The idea here is that most of the time we are in front of a “Chinese whispers” type of situation and we need to make sure that those two variables are reported correctly.
Time and place
Time and Plac
e
Spelling: if the original language of the reporter it is different than the one of the report, you can find that a place
may refer to several locations due to spelling
issues. One example is Arabic,
where the numbers of letter As in the alphabet are several, while in English there is
only one.
Edited Re-Tweets: when you find
tweets reporting something try to
do as many searched as
possible to find out what the
original tweet was saying. People tend to edit re-
tweets when they think there is
something wrong (like spelling of
names) and in this way they may
actually change a correct name.
Format of dates: the format of
dates are different form place to
place, so people may translate 09/08/12 as 9
August 2012, or as 08 of September 2012. If people
repost a message and decide to
change the format of the date, they
may not know which one of the two version they
are using.
The triangulation of information is basically the act of putting together all the information you can find about the issue reported and cross-check them against the fact as reported to you.To do this it is suggested to start filtering all possible information sources available: social media, websites, blogs, local and international media, TVs, and so on.In this tasks, make sure you always remain skeptical about the reports that you receive. Look for multiple reports from different unconnected sources, since the more independent witnesses you can get information from, the better and the less critical the need for identity authentication. If after doing all the necessary verification of the report, you find out that the fact is reported by other people but with different characteristics, like time and location for example, are not matching, do not discard immediately the fact as false but try to identify which version of the report is the most accurate. If you cannot find confirmation for any of the versions, do not chose the most likely one, but make sure you report all versions and highlight which difference they have. The verification of the context often can help in the identification of the those partial falsifications. Run multiple verifications for the same user and, even better, ask others to run the same verifications processes on the same account. Personal beliefs and state of mind can change how we interpret the same information. In general, keep in mind that the less you find about something that less reliable you can consider that information, and the more you have to rely on people on the ground to confirm it.
Triangulation of content
When you are crowdsourcing information about a specific event or place, try to look in the social network about that same event. If you do not find anything related to that event, search for events in the same area or for issues that may be related to that event. In the social networks normally events tend to be repeated and reported in several platforms (Twitter and Facebook for example) so this process will also help you identifying possible editing of the event reported done by different users.
Content ID on Social Network
Crowdsourced verification of the content
Social authentication
Trusted Network on the ground
The creation of Virtual Trusted
Networks (active and
passive)
When you are crowdsourcing information about a specific event or place, you should start finding an online social networks related to that specific issue. For example in Twitter, that specific social network will be the one twitting very on a specific hashtag (#).You can use that social network to ask information about how unusual the event reported is, to get a sense of how likely it is to happened in the first place. This online community, for example, can help you translate, triangulate, and track down key information. They can enable you to crowdsourced verification but you must always check and challenge what you are told. Ask people what they think, what do they know about that area, or about that event. You have also to remember that you must NEVER give an indication of what your source is, if it is not public already, and try not to give too many details of the information reported if that information is not public already.
Social authentication
When verifying information online the use of an offline network can be very helpful. As mentioned before, this network can be your ears and eyes on the ground, but they can also incur in considerable risks by doing so.For this reason those networks need to be very trusted, possibly people that you personally know and it should be open only to other people known and trusted personally.As already said in the module, security issues should be prioritize over everything else.
Trusted Network on the ground
In several occasion, the creation of a trusted virtual network has been done by several activists around the world to verify crowdsourced information. This network has been created by starting with a group of very trusted people on the ground (personally known and contacted) and then expanded only to people vetted by this same group of people, and only if personally met. To do this there are several tools that have been used, like Skype (better if encrypted since Skype is indeed vulnerable to monitoring by third parties), Google groups, encrypted emails mailing lists, etc.
The creation of Virtual Trusted Networks (active and passive)
It is also possible (and suggested) when doing crowdsourcing online to create a list of the sources that provide information online (directly to you or via social networks) that you can keep updated during your project. This list can also be compiled by adding comments to each of the accounts/people listed and in this way starting to create your reliable list of sources and “scoring” them according to your verification. This list will also be very useful for all the processes done before, and especially once you have done the source verification for those account and can attest the reliability of the source.
The creation of Virtual Trusted Networks (active and passive)
THE VERIFICATION OF THE CONTEXT
Context Matters
The understanding of the context of an information it is crucial to give a sense to what has really happened. Specifically in the context of crowdsourced information the context become extremely important because we may not know the source and we may have to find out if the information is actually true. On the other side though, even if the fact reported really did happened and the source is reliable, we may be in the situation where the context of the information can give us a completely different idea of what is the overall picture surrounding the event.
Why Context matters?
People develop their opinions based on the information available to them, and this is how we evaluate the verification processes of an information. Sometime though if the fact is not presented it in the right context, the same information may be evaluated in very different ways. The idea here is that once we have attested that a fact has happened, we need to make sure that every fact relevant to the story, which is a necessary part of the story, is also reported. In the context of this module we will look at two different ways that you can use contextualization technics to verify information:1. Find data that can further support or give you an idea of the
veracity of the information2. Find data that can give a meaning to the information reported
On line Verification the context
Factual context
Global Context
Time and
Location
Time and Location
We have already seen before how time and location can help us identify the veracity of an information, but in this case we will look at them as important factors for the contextualization of the event reported.Sometimes the time and location of an event make a huge different in considering the reliability of the information reported. What we can look at here is if the event reported makes sense considering the time and location reported.
Time
Does the time of the event make sense for the event reported?
For example: if an attack to a street vendor is reported, does the time of
the event reported make sense considering the type of economic
activity?
Also, does the time gives a specific meaning to the event? Like an
attack to a mosque on Friday? Or to a Synagogue on Saturday?
Location
For example, does the event take place in a location where it is very unlikely to find the actors involved?
Does the location have a specific meaning for the type of event
reported?
Does the location have access or closure times that do not fit with the time of the event reported?
Does the location of the even make sense for the type of event
reported?
Factual context
The first thing that we need to look at when looking at the context is to try to place the event we are analyzing in a greater picture. The picture in this page is an example. If you think about how you would describe the picture, the context of the event is giving you a completely different information depending on if you look at the right or left part of the picture. The event happening is always the same though. This is what we mean when we say that you have to look at the factual context.
Global context
The global context refer to the international context around that event. Sometimes political crisis outside a country can trigger events in a third country. One example can be the sudden explosion of tensions in between two neighboring countries that can push a third one to close its borders or move contingents of the army towards the borders. The global context is also something related to the important facts that are happening in the same country or in the same location of the event reported. The national context of a reported event in fact and when it does happen, can have a relationship with the reasons behind the event, that can help us understand if the event is likely to be true or if it can be triggered by other events happening in the same place or at the same time.
Is there any other event similar to this one that has happened in the same time
or area?
For example something very similar reported in the same
area
Like if an attack is reported, are there other attack been reported in the same area
before or after?
Or something that could have triggered that event
(like a retaliation or an attack)
For example, massive arrests of activists may be following
the dead of a police man during a demonstration
Or something that could have influenced the way
people described what they have seen (for example
something very shocking or inflammatory)
In this case people may report event to you exaggerating the
event because they are in a very emotional state of mind
following a very shocking event
Look at what is going on the region, but also globally in the world
Is there any political issues happening that can be related to
the country? Like important bilateral meetings? Or peace
treaties? Or important speeches being delivered?
Are there economical issues/ factors that can have an influence in
the country? Sudden rise in unemployment? Sudden rise in
price of oil?
There has been any events happening in neighboring countries
that may have lead to reaction in the place you are considering? For example sudden rise in tension in
between two countries? Or an economical agreement reached?
WHAT TO KEEP IN MIND
The 9 golden rules
1. Unless you see it and you hear it, it is not 100% verified.
2. After rule I, chose your verification criteria in advance and make them public (if possible)
3. Chose your trusted network and make clear rules for being part of it
4. Doubt and question everything, always
5. Be always ready to change your mind about the reliability
of a source6. Be always ready to change
your mind about the veracity of an information
7. You don’t need to verify everything. Choose you battles
8. Always admit when you fail or when you do a mistake
9. What you know is also what your enemies know. Keep this always in mind when verifying information
Keep your reputation intact
Since we said that rule number one for the verification of crowdsourced information is that unless you see and hear it, it is not 100% verified, you need to protect your reputation and your credibility every time you use, republish, collect or advertise crowdsourced information. To this, you need to be very transparent about your verification criteria and about what degree of verification you can achieve with the methodologies you are using. This also means that you do not have to decide just in between verified and not verified (like 0-100) but you can also choose to use several degrees of verification in the middle.The main important things is that you make very clear to whoever is taking that information from you what criteria you used and how are you verifying your sources and information. A second very important point, as expressed before, is that you need to always be ready to admit when you did a mistake in your verification process and be ready to change your protocols according to the situation.
HOW TO CREATE VERIFICATION PROTOCOLS
You do not necessarily need to verify everything. Choose what are most important information by looking at the typology and amount of information that you received. Make a considerate evaluation of your capacity. It is better to have few very well verified information, then a lot of quasi verified information.
Step I: Chose what you need to verify
Create your criteria for verification in advance and by looking precisely at what means you have to verify information. Ask yourself:1. When is that you will consider an information
verified?2. When is that you will not consider it verified?3. What degrees of verification are you willing
to have in the middle?
Step II: Choose your verification criteria
Choose very carefully who you will consider a trusted source. This does not necessarily need to be mainstream media or known people.In addition to this you need to update your trusted sources list periodically (I would say on a daily bases) to make sure that you add new sources and review the trustfulness of old sources.
Step III: draft your trusted sources list
Once you have drafted your verification protocols, make sure you always apply them to all information you process. Do not allow yourself to do exceptions to it, and make sure that all your information are verified following the same procedures. This is very important to assure consistency and to be able to always get back to your verification process to refine where it is not working.
Step IV: Draft your verification protocols and stick to it
One of the main important things about your verification protocol is to make it as public as possible. The reason behind this is not to inform people how they can go around it, but to assure that people do know what sort of credibility your information have. You do not have to make the all details public, but people using your information need to know exactly how much they can trust your information or not. The more they are aware of the verification protocols you use, the more you will be able, if necessary to explain eventual mistakes or failure in verifying information, as well as managing expectations about the reliability of the information posted.
Step V: make it public
EXAMPLES
Blogging the Egyptian Election “BEE” 2010
U SHAHID PLATFORM
Step 1: what to verify
One of the main problem of verification of crowdsourced information is related to the fact that the overload of information can make it impossible to actually do a meaningful verification of the information received. For this reason, the U-Shahid team’s first step was to define concrete criteria for what types of reports required verification. They didn’t attempt to verify all the content that was available, but only the one that was important or critical. 1. Reports related to an immediate threat or act of violence2. “Grave electoral fraud” the gravity of which was assessed by the importance of the parties involved (celebrities, known PMs, government officials) or the gravity of the fraud (involving violence, or massive votes exchange) or it’s location (inside the pooling stations)
The U-Shahid team also created some criteria that
all reports need to be checked against to be
considered as verified. If one or more of the criteria was met, then the report
would have been considered verified
The report was supported by a
video or pictures that clearly
confirmed what was been reported
(especially location);
It was been reported by two or more
independent sources (the list of
independent sources was created in advance and
updated by the team) ;
Messages coming from social media (Twitter, Facebook
and blogs) needed to be confirmed by an SMS, a media report or a direct witness;
At least one of the sources of the
information was clear and known (i.e., 2 SMSs from unknown sources could not verify
each other)
For reports that had to be verified but did not
meet the criteria outlines before, the
verification team was engaging directly with
the source in 4 different ways
If the report was sent by SMS, that
number was called to verify the person’s identity. The witness
was asked if they observed the event
themselves or if they simply learned about
the event from someone else. More specifically, details on the 5w1h are
asked. If the event being reported was still unfolding, the
witness was asked if anyone else nearby
was able confirm the information.
By asking to provide a video or picture of the event but only if it was safe to do so.
If the report came from Twitter itself, the account of the tweeter was reviewed. Simple content analysis of previous tweets and the account holder’s bio was
carried out online, in addition to revised the Tweeter’s
followers.
By tweeting back to the original Twitter
user asking for more information—again using the “who did
what, to whom, how and where” format, asking followers to
confirm if the information was indeed correct. St
ep 3
: Dir
ect
cont
act
wit
h th
e so
urce
Step 4: In person verification
The second core strategy involves in-person verification via a trusted
network of monitors on the ground:
The U-Shahid team determines whether one of their election monitors (trained by them) was close to the area
referenced in a report that required verification. If a monitor was on site or close enough, that person was
asked to verify the report.
If U-Shahid did not had any monitors in the area, the team was activating their NGO partners who
may had monitors in the area. If so, those individuals were asked to confirm the validity of
the report being investigated.
If no one of the two possibilities was possible, U-Shahid was using a network of trusted journalists
in the area to ask them to go to the place to confirm the information for them.
Step 5: online triangulation
If no one of the previous possibilities was available, the team was using the mainstream media and online tools for confirmation. They used web-based research to look for any evidence that is specific to the event that was reported as well as that location. They looked for article, blogs, social media, video or pictures, mainstream media articles and TV shows that could confirm the information reported.
In addition to this the U-Shahid team was then triangulating the report being investigated with other
reports they had already received.
OUTCOMESUshahid received on election day November 28, 2010 during voting period 1252 reports, documented with 287 videos and 111 images. By the end of the project on Dec 28, 2010 a total of 2700 were uploaded on the platform with an average of reports per day of 27 and a percentage of verified reports of 91.26%.
Electua 2012
THANK YOU
Anahi Ayala IacucciMedia Innovation Advisor
[email protected]@anahi_ayala
@info_innovation
Top Related