Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed...
-
Upload
darcy-bruce -
Category
Documents
-
view
214 -
download
0
Transcript of Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed...
![Page 1: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/1.jpg)
Virtual Scientific-Community-Based Foundations for Popperian e-Science
Karl LieberherrAhmed Abdelmeged
Northeastern University, CCIS, PRL, Boston
04/19/23 1
![Page 2: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/2.jpg)
04/19/23 2
Ontology
Computer Science
Programming
Socio-Technical SystemThe Global Brain
Mathematics
Game Theory
ExtensiveFormMetaGaming
Mathematical Logic
Dialog Games IF Logic
inspired by ScienceWISE
Organization
Mechanism Design
![Page 3: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/3.jpg)
A claim is …
• information about one’s performance when interacting with another clever being in a specific domain.
• information about the performance of one’s program.
4/24/2011 3Crowdsourcing
![Page 4: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/4.jpg)
Outline
Theory
Methods
Results
Conclusion
Introduction
04/19/23 4
![Page 5: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/5.jpg)
Introduction• Explanation: SCG as a general pattern behind many different
competitions: topcoder.com, kaggle.com, tunedit.org, Renaissance, …
• Make SCG a part of cyber-infrastructure (e-science) to support teaching and innovation in constructive domains.
• SCG usage for teaching– Innovation Success with Undergraduates using SCG on
piazza.com: Qualitative Data Sources & Analysis– Avatar competitions are not for teaching (but for competitive
innovation)• Theoretical Properties of SCG• Take Home: EASY USE WITH STANDARD TOOLS
• group research, teaching, (intelligent) crowd sourcing
04/19/23 5
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
SCG = Scientific Community Game = Specker Challenge Game
![Page 6: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/6.jpg)
Popper
• One of the philosophers of science who has had a big impact.
• Popper’s solution: Falsification: A claim is falsifiable if you can imagine an observation that would cause you to reject the claim.
• That a claim is "falsifiable" does not mean it is false; rather, that if it is false, then some observation or experiment will produce a reproducible result that is in conflict with it.
04/19/23 6
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 7: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/7.jpg)
What SCG helps with
• Build and maintain knowledge bases (sets of claims believed to be true).
• How to identify experts?• How to decide if an answer is worthwhile?
– Use scholars to choose the winners
• How to organize egoistic scholars to produce social welfare: knowledge base and know-how how to defend it.– The scholars try to reverse engineer the solutions
of winning scholars.
04/19/23 7
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 8: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/8.jpg)
Abstraction from 4 Examples
• From a CS journal paper• Insilico experiment• From kaggle.com: Facebook competition• From a calculus problem
04/19/23 8
![Page 9: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/9.jpg)
Example 1: From an Abstract of a 2005 Journal Paper
• An instance of a constraint satisfaction problem (CSP) is variable k-consistent if any subinstance with at most k variables has a solution.
• For a fixed constraint language L, r(k,L) is the largest ratio such that any variable k-consistent instance has a solution that satisfies at least a fraction of r(k,L) of the constraints.
04/19/23 9
![Page 10: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/10.jpg)
Example 1
• From a 2005 TCS paper: Locally Consistent Constraint Satisfaction Problems by Manuel Bodirsky and Daniel Kral.
• Example– L = CNF– k = 1– What is r(1,CNF)?– Claims: r(1,CNF) = 0.6, r(1,CNF) = 0.7
04/19/23 10
![Page 11: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/11.jpg)
Example 1: Making a game to determine r(1,CNF)
• Observation: claims are falsifiable playing a two person game.
04/19/23 11
![Page 12: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/12.jpg)
Example 2: Claim involving Insilico Experiment
Claim InsilicoExperimental(X,Y,q,r)I claim, given raw materials x in X,I can produce product y in Y of quality qand using resources at most r.
12Crowdsourcing4/24/2011
![Page 13: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/13.jpg)
Example 2: Making a game to determine InsilicoExperimental(X,Y,q,r)
• Observation: claims are falsifiable playing a two person game.
04/19/23 13
![Page 14: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/14.jpg)
Example 3: Data mining
• Facebook competition from Kaggle.com: – Given a social network graph x with deleted edges
and the original social network graph gs (secret, from a family X of social networks)
– guess the complete social network graph y– quality(x, gs, y) = mean average precision
(adapted from IR)– I claim I can achieve a mean average precision of q
• for social graphs in family X: DM1(X,q)• for a specific reduced social graph: DM2(x,q)
04/19/23 14
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 15: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/15.jpg)
Example 3: Making a game to determine the optimal claims
• Observation: claims DM1(X,q) and DM2(x,q) are falsifiable playing a two person game.
04/19/23 15
![Page 16: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/16.jpg)
Example 4: Specker• Claims:
– Specker(set X, set Y(X), function f(X,Y)->[0,1], constant c): ForAll x in X Exists y in Y(X): f(x,y)≥c
• Example 1– X = Conjunctive Normal Forms with various restrictions– Y(X) = Assignments to CNFs– f(x,y) = fraction of satisfied clauses in x under y– c in [0,1], e.g., c= 0.61
• Example 2 (a reduction of example 1)– X = [0,1]– Y(X) = [0,1]– f(x,y)=x*y+(1-x)(1-y^2))– c in [0,1], e.g., c=0.61
04/19/23 16
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 17: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/17.jpg)
Example 4: Specker
• Observation: claims Specker(X,Y,f,c) are falsifiable playing a two person game.
04/19/23 17
![Page 18: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/18.jpg)
What is the abstraction?
• Sets of claims• Claims are falsifiable• …
04/19/23 18
![Page 19: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/19.jpg)
04/19/23 19
RP1
PG1
claimsC11C12C13…
SC1SC2
SC1SC1
RP2
PG2
claimsC21C22C23…
SC3SC4
SC5SC1
SCG defines:• refutation protocol interface• generic rules for all playgrounds
Each playground defines:• domain• claims language• specific protocol • data exchanged• configuration data
Playgrounds
D1
D2
![Page 20: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/20.jpg)
Example 1: Making a game to determine r(1,CNF)
• Observation: claims are falsifiable playing a two person game. defendable = !refutable– propose r(1,CNF) = 0.7
• refutable
– propose r(1,CNF) = 0.6 can be strengthened to r(1,CNF) = 0.61 which is defendable (refutation attempts will be unsuccessful)
– propose r(1,CNF) = (sqrt(5)-1)/2 ~ 0.618 … optimum: defendable and cannot be strengthened
04/19/23 20
![Page 21: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/21.jpg)
Who are the scholars?
• Scientists• Students in a class room
– High school– University
• Members of the Gig Economy– Between 1995 and 2005, the number of self-
employed independent workers grew by 27 percent.• Potential employees (Facebook on kaggle.com)• Anyone with web access; Intelligent crowd.
04/19/23 21
![Page 22: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/22.jpg)
Kaggle.com Competitions2012
• Facebook recruiting competitions– Task: Data scientist– Reward: Job– Teams: 197
• Heritage Health Prize– Task: Hospital admissions– Reward: $ 3 million– Teams: 1118
• Chess ratings – Elo versus the Rest of the World– Task: Predict outcome of chess games– Reward: $ 617 – Teams: 257
04/19/23 22
![Page 23: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/23.jpg)
Kaggle.com Competitions2012
• Eye Movements Verification and Identification– Task: Identify people– Reward: Kudos– Teams: 51
• EMC Data Science Global Hackathon– Task: Air Quality Prediction– Reward $ 7030– Teams: 114
04/19/23 23
![Page 24: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/24.jpg)
What Scholars think about!
• If I propose claim C, what is the probability that– C is successfully refuted– C is successfully strengthened
• If I try to refute claim C, what is the probability that I will fail.
• If I try to strengthen claim C, what is the probability that I will fail?
• Scholars are free to invent; game rules don’t limit creativity!
2404/19/23
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 25: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/25.jpg)
Degree of automation with SCG(X)
25
no automationhuman plays
full automationavatar plays
degree of automation used by scholar
some automationhuman plays
0 1
more applications:test constructive knowledge
transfer to reliable, efficient software
avatar Bobscholar Alice
04/19/23
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 26: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/26.jpg)
Organizational Problem Solved
• How to design a happy scientific community that encourages its members to really contribute.
• Control of scientific community– tunable SCG rules– Specific domain, claim definition to narrow scope.
04/19/23 26
happy = can be creative, can thrive, have opportunity to learn, not ignored
![Page 27: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/27.jpg)
Playground
• defines – what is wanted, e.g., an algorithm S in a particular
domain (inputs/outputs)– evaluation, e.g., how S is evaluated (quality)– claims, e.g., what kind of claims can be made
about S (expression with quantifiers)
• A playground defines WHAT is desired and the scholars/avatars define the HOW.
![Page 28: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/28.jpg)
Theory
• Extensive Form Representation of Game• Properties
– Community Property: All faulty actions can be exposed.
– SCG Equilibrium– Convergence to optimum claim
04/19/23 28
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 29: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/29.jpg)
Extensive-form representation
1. the players of a game: 1 and 22. for every player every opportunity they have
to move3. what each player can do at each of their
moves4. what each player knows for every move5. the payoffs received by every player for
every possible combination of moves
04/19/23 29
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 30: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/30.jpg)
Large Action Spaces
• Thick arrows mean: select from a usually large number of choices
1
2
04/19/23 30
![Page 31: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/31.jpg)
Refutation Protocol
• Collects data given to predicate p. Alternates.
refute(C,proposer,other)
p(C, …)?(1,-1):(-1,1)
claim payoff for proposerif p true (defense)
payoff for otherif p true (defense)
payoff for otherif p false (refutation)
payoff for proposerif p false (refutation)
other tries to makep false while proposertries to make p true.
p false means successfulrefutation.
p true means successfuldefense.
04/19/23 31
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 32: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/32.jpg)
1
propose claim C from Claims
2
refute(C,1,2)
p(C, …)?(1,-1):(-1,1)
1 scholar2 scholar
strengthen attempt C’ => C
refute(C’,2,1)
agree attempt C
refute(C,2,1)
p(C’, …)?(1,-1):(-1,1) p(C, …)?(1,-1):(-1,1)
04/19/23
refute attempt C
refute(C, proposer,other)p(…)?(proposer,other): (proposer,other)
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
p(C’, …)?(-1,1):(1,-1) p(C, …)?(0,0):(1,-1)
32
SCG Core
![Page 33: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/33.jpg)
Game Rules for Playground
• All objects exchanged during protocol must be legal and valid.
• Each move must be within time-limit.• Scholar who first violates a playground rule,
loses.
04/19/23 33
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 34: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/34.jpg)
4/24/2011 Crowdsourcing 34
good bad
Logic with Soundness
claimssentences
not just true/false claims, but optimum/non-optimum claims:good: true/optimumbad: false/non-optimum
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 35: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/35.jpg)
bad
4/24/2011 Crowdsourcing 35
good
Scientific Community Game Logicwith Community Principle
agreed by two scholars disagreed by two scholars
there exists a two-party certificateto expose misclassification
claimssentences
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 36: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/36.jpg)
Comparison Logic and SCG
Logic• sentences
– true– false
• proof for being true– proof system, checkable– guaranteed defense
• proof for being false– proof system, checkable– guaranteed refutation
• Universal sentences
Scientific Community Game• sentences = claims
– good– bad
• evidence for goodness– defense, checkable– uncertainty of defense
• evidence for badness– refutation, checkable– uncertainty of refutation
• Personified sentences
4/24/2011 Crowdsourcing 36
![Page 37: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/37.jpg)
Community Property
• For every faulty decision action there exists an exposing reaction that blames the bad decision.– Reasons:
• We want the system to be egalitarian. – It is important that clever crowd members can shine and
expose others who don’t promote the social welfare of the community.
• Faulty decisions must be exposable. It may take effort.
04/19/23 37
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 38: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/38.jpg)
Methods of Exploration
• Developed Platform SCG Court = Generator of teaching/innovation playgrounds– http://sourceforge.net/p/generic-scg/code-0/110/tree/GenericSCG/– Developed numerous playgrounds for avatars.
• Developed Algorithms Course using Piazza based on SCG Court experience– role of scholar played by humans – piazza.com: encourages students to answer each
other’s questions.
04/19/23 38
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 39: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/39.jpg)
Avatar Interface
• AvatarI – public List<Claim> propose(List<Claim> forbiddenClaims); – public List<OpposeAction> oppose(List<Claim>
claimsToBeOpposed); – public InstanceI provide(Claim claimToBeProvided); – public SolutionI solve(SolveRequest solveRequest);
• from http://sourceforge.net/p/generic-scg/code-0/110/tree/GenericSCG/src/scg/scg.beh
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
04/19/23 39
![Page 40: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/40.jpg)
Instance Interface (Domain)
• InstanceI – boolean valid(SolutionI solution, Config config); – double quality(SolutionI solution);
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
04/19/23 40
![Page 41: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/41.jpg)
InstanceSet Interface (Domain)
• InstanceSetI – Option<String> belongsTo(InstanceI instance);– Option<String> valid(Config config); }}
04/19/23 41
![Page 42: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/42.jpg)
Protocol Interface
• ProtocolI – double getResult(Claim claim, SolutionI[]
solutions, InstanceI[] instances); – ProtocolSpec getProtocolSpec(); – boolean strengthenP(Claim oldClaim, Claim
strengthenedClaim);
04/19/23 42
![Page 43: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/43.jpg)
Claim Class, for all playgrounds
• Claim – public Claim(InstanceSetI instanceSet, ProtocolI
protocol, double quality, double confidence)
04/19/23 43
![Page 44: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/44.jpg)
Protocol Library
• ExistsForAll.java• ForAllExists.java• Renaissance.java• AsGoodAsYou.java• Survivor.java
04/19/23 44
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 45: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/45.jpg)
Second Method: Piazza ExperienceGale-Shapley
• We propose that, for all integers n > 0, the maximum iterations the Gale-Shapely algorithm with n men and n women can produce is n(n-1)+1.
Note: Thus far, the inputs used for all other claims arrives at only (n(n+1))/2.
04/19/23 45
![Page 46: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/46.jpg)
Piazza Experience
• Leaf Covering: Improved running time from quadratic to constant time.
04/19/23 46
![Page 47: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/47.jpg)
Results
• Explanation: SCG as a general pattern behind many different competitions: topcoder.com, kaggle.com, Operations Research Competitions, tunedit.org, http://eterna.cmu.edu/ …
• SCG usage for teaching using forum– Innovation Success with Undergraduates using SCG on
piazza.com: Qualitative Data Sources & Analysis• Avatar competitions are not for teaching (but
good for competitive innovation)• Theoretical Properties of SCG
04/19/23 47
SCG = Scientific Community Game = Specker Challenge Game
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 48: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/48.jpg)
Competition tuning: minimum
• For each scholar– count claims that were successfully opposed
(refuted or strengthened)• encourages strong claims• gather information from competitors for free
– count claims that were not successfully agreed• Good for teaching
– students want minimum competition– good students want to build social capital and
help weaker students
04/19/23 48
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 49: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/49.jpg)
Piazza Results
• Do not give hints at solutions. This significantly decreased the amount of discourse taking place.
04/19/23 49
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 50: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/50.jpg)
Conclusions and Future Work
• We propose a systematic gamification of teaching STEM domains:– Design an SCG playground where the winning
students demonstrate superior domain knowledge.
04/19/23 50
STEM = Science, Technology, Engineering, and Mathematics
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 51: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/51.jpg)
Gamification of Software Development for Computational Problems
• Want reliable software to solve a computational problem? Design an SCG playground where the winning team will create the software you want.
• playground design = requirements– Programming the Global Brain
• socio-technical system (playground) will produce solution to requirements.
Crowdsourcing 514/24/2011
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 52: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/52.jpg)
Conclusions
• Flexible use of SCG using a forum environment with threads and replies using optimization playgrounds is productive:– teams took turns leapfrogging each other– reached state-of-the-art and even improved it
• SCG has desirable theoretical properties.– faulty decision –> exposing reaction– equilibria– convergence to optimum claim
04/19/23 52
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 53: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/53.jpg)
Future Work
• Make SCG part of cyber-infrastructure (e-science) both for avatars and human scholars.
• Polish SCG Court – The administrator software needs to be very reliable (to avoid
cheating by avatars).– Playground development and testing needs tool support.• Further develop SCG with forum software– Playground design defines requirements for know-how.– Hierarchical playgrounds: partitioning into balanced groups.– Restart playground after publishing all current ideas in
playground (if optimum is not yet reached).
04/19/23 53
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 54: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/54.jpg)
Links / Questions• SCG Home
– http://www.ccs.neu.edu/home/lieber/evergreen/specker/scg-home.html
• Piazza page for Algorithms– http://piazza.com/class#winter2012/cs4800/0
• Algorithms Home– http://www.ccs.neu.edu/home/lieber/courses/algorithms/cs4800/
sp12/course-description.html • Algorithms Feedback
– http://www.ccs.neu.edu/home/lieber/courses/algorithms/cs4800/sp12/feedback/
• SCG Court Source– http://sourceforge.net/p/generic-scg/code-0/110/tree/GenericSCG/
04/19/23 54
![Page 55: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/55.jpg)
The End
• More Questions?
04/19/23 55
![Page 56: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/56.jpg)
Extra slides
04/19/23 56
![Page 57: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/57.jpg)
Essence of Game Ruleswithout Payoff
• scholars: 1, 2• LifeOfClaim(C) = propose(1,C) followed by
(oppose(1,2,C)|agree(1,2,C)).• oppose(1,2,C) = (refute(1,2,C)|
strengthen(1,2,C,C’)), where stronger(C,C’).• strengthen(1,2,C,C’) = !refute(2,1,C’).• agree(1,2,C) = !refute(2,1,C)
04/19/23 57
blamed decisions:propose(1,C)refute(1,2,C)strengthen(1,2,C,C’)agree(1,2,c)
![Page 58: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/58.jpg)
1
propose claim C from Claims
2
refute(C,1,2)
p(C, …)?(1,-1):(-1,1)
1 scholar2 scholar
strengthen attempt C’ => C
refute(C’,2,1)
agree attempt C
refute(C,2,1)
p(C’, …)?(1,-1):(-1,1) p(C, …)?(1,-1):(-1,1)
04/19/23
refute attempt C
refute(C, proposer,other)p(…)?(proposer,other): (proposer,other)s: successfulu: unsuccessful
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
p(C’, …)?(-1,1):(1,-1)
u:1 2 s:1 2
s:1 2 u:1 2
p(C, …)?(0,0):(1,-1)s:1 2 u:1 2
58
![Page 59: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/59.jpg)
1
propose claim C from Claims
2
refute(C,1,2)
p(C, …)? (0,0) :(0,1)
1 scholar2 scholar
strengthen attempt C’ => C
refute(C’,2,1)
agree attempt C
refute(C,2,1)
04/19/23
refute attempt C
refute(C, proposer,other)p(…)?(proposer,other): (proposer,other)s: successfulu: unsuccessful
p(C’, …)?(0,1): (0,0)
u:1 2 s:1 2
s:1 2 u:1 2
p(C, …)?(0,0): (1,0)s:1 2 u:1 2
Low competition
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
59
![Page 60: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/60.jpg)
Conclusions for Teaching
• Transition– refute: (1,-1):(-1,1) -> (0,0) :(0,1)– strengthen: (-1,1):(1,-1) -> (0,1): (0,0)– agree: (0,0):(1,-1) -> (0,0): (1,0)
• creates better playgrounds for learning by lowering competition and increasing teaching between scholars.
04/19/23 60
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 61: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/61.jpg)
Claims
• Protocol. Defines scientific discourse.• Scholars make a prediction about their
performance in protocol.• Predicate that decides whether refutation is
successful. Refutation protocol collects data for predicate.
• As a starter: Think of a claim as a mathematical statement: EA or AE.– all planar graphs have a 4 coloring.
04/19/23 61
![Page 62: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/62.jpg)
More examples of Protocols
• Let f(x,y)=x*y+(1-x)(1-y^2)). Alice claims Math(0.61): Bob constructs an x in [0,1] and Alice constructs a y in [0,1], and Alice guarantees that f(x,y)> 0.61. True claim but can be strengthened to 0.618.
• Alice claims Solar(RawMaterials,m,0.61). Bob constructs raw materials r in RawMaterials and Alice constructs a solar cell s in Solution from r using money m and so that efficiency(s)> 0.61.
04/19/23 62
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 63: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/63.jpg)
Questions received
• In learning game, give credit to all contributors, not just final one (DARPA 10 ball challenge)
• Predicate logic -> SCG: make explicit• Playground design: involve competitors
04/19/23 63
![Page 64: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/64.jpg)
Questions
• Credit– first time the best claim is made– linear order by time– linear order by strength (quality)
04/19/23 64
![Page 65: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/65.jpg)
New insight
• Need to know very little about refutation protocol.– collect data, what is available when is not
important– evaluate predicate with collected data
04/19/23 65
![Page 66: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/66.jpg)
What is a loose collaboration?
• Scholars can work independently on an aspect of the same problem.
• Problem = decide which claims in playground to oppose or agree with.
• How is know-how combined? Using a protocol.– Alice claimed that for the input that Alice provides,
Bob cannot find an output of quality q. But Bob finds such an output. Alice corrects.
– Bug reports that need to be addressed and corrections.
04/19/23 66Playground = Instantiation of Platform
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 67: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/67.jpg)
Example: Independent Set
• Alice = proposer, Bob = other.• Protocol / claim: AtLeastAsGood. Alice claims
to be at least as good as Bob at IS.– Bob provides undirected graph G.– Bob computes independent set sB for G (secret).– Alice computes independent set sA for G.– Alice wins, if size(sA) >= size(sB) (= p(sA,sB)).
04/19/23 67
![Page 68: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/68.jpg)
Specker• Claims:
– Specker(set X, set Y(X), function f(X,Y)->[0,1], constant c): ForAll x in X Exists y in Y(X): f(x,y)≥c
• Example 1– X = Conjunctive Normal Forms with various restrictions– Y(X) = Assignments to CNFs– f(x,y) = fraction of satisfied clauses in x under y– c in [0,1], e.g., c= 0.61
• Example 2 (a reduction of example 1)– X = [0,1]– Y(X) = [0,1]– f(x,y)=x*y+(1-x)(1-y^2))– c in [0,1], e.g., c=0.61
04/19/23 68
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 69: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/69.jpg)
Kaggle.com
• Facebook competition: – X = Social Network Graph with deleted edges,
Original Social Network Graph (secret)– Y(X) = estimated complete Social Network Graph– quality(x,y) = mean average precision adapted
from IR
04/19/23 69
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 70: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/70.jpg)
Simpler talk
• Introduction: parameterized models of scientific communities
• Theory
04/19/23 70
![Page 71: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/71.jpg)
Abstraction from 4 Examples
• From a CS journal paper• Insilico experiment• From kaggle.com: Facebook competition• From a calculus problem
04/19/23 71
![Page 72: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/72.jpg)
Example 1: From an Abstract of a 2005 Journal Paper
• An instance of a constraint satisfaction problem (CSP) is variable k-consistent if any subinstance with at most k variables has a solution.
• For a fixed constraint language L, r(k,L) is the largest ratio such that any variable k-consistent instance has a solution that satisfies at least a fraction of r(k,L) of the constraints.
04/19/23 72
![Page 73: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/73.jpg)
Example 1
• From a 2005 TCS paper: Locally Consistent Constraint Satisfaction Problems by Manuel Bodirsky and Daniel Kral.
• Example– L = CNF– k = 1– What is r(1,CNF)?– Claims: r(1,CNF) = 0.6, r(1,CNF) = 0.7
04/19/23 73
![Page 74: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/74.jpg)
Example 1: Making a game to determine r(1,CNF)
• Observation: claims are falsifiable playing a two person game.
04/19/23 74
![Page 75: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/75.jpg)
Example 2: Claim involving Insilico Experiment
Claim InsilicoExperimental(X,Y,q,r)I claim, given raw materials x in X,I can produce product y in Y of quality qand using resources at most r.
75Crowdsourcing4/24/2011
![Page 76: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/76.jpg)
Example 2: Making a game to determine InsilicoExperimental(X,Y,q,r)
• Observation: claims are falsifiable playing a two person game.
04/19/23 76
![Page 77: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/77.jpg)
Example 3: Data mining
• Facebook competition from Kaggle.com: – Given a social network graph x with deleted edges
and the original social network graph gs (secret, from a family X of social networks)
– guess the complete social network graph y– quality(x, gs, y) = mean average precision
(adapted from IR)– I claim I can achieve a mean average precision of q
• for social graphs in family X: DM1(X,q)• for a specific reduced social graph: DM2(x,q)
04/19/23 77
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 78: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/78.jpg)
Example 3: Making a game to determine the optimal claims
• Observation: claims DM1(X,q) are falsifiable playing a two person game. Claim DM2(x,q) is falsifiable when the secret is revealed.
04/19/23 78
![Page 79: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/79.jpg)
Example 4: Specker• Claims:
– Specker(set X, set Y(X), function f(X,Y)->[0,1], constant c): ForAll x in X Exists y in Y(X): f(x,y)≥c
• Example 1– X = Conjunctive Normal Forms with various restrictions– Y(X) = Assignments to CNFs– f(x,y) = fraction of satisfied clauses in x under y– c in [0,1], e.g., c= 0.61
• Example 2 (a reduction of example 1)– X = [0,1]– Y(X) = [0,1]– f(x,y)=x*y+(1-x)(1-y^2))– c in [0,1], e.g., c=0.61
04/19/23 79
IntroductionIntroduction TheoryTheory MethodsMethods ResultsResults ConclusionConclusion
![Page 80: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/80.jpg)
Example 4: Specker
• Observation: claims Specker(X,Y,f,c) are falsifiable playing a two person game.
04/19/23 80
![Page 81: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/81.jpg)
What is the abstraction?
• Sets of claims• Claims are falsifiable• …
04/19/23 81
![Page 82: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/82.jpg)
Example 1: Making a game to determine r(1,CNF)
• Observation: claims are falsifiable playing a two person game. defendable = !refutable– propose r(1,CNF) = 0.7
• refutable
– propose r(1,CNF) = 0.6 can be strengthened to r(1,CNF) = 0.61 which is defendable (refutation attempts will be unsuccessful)
– propose r(1,CNF) = (sqrt(5)-1)/2 ~ 0.618 … optimum: defendable and cannot be strengthened
04/19/23 82
![Page 83: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/83.jpg)
What we get
• Engaged software developers– let them produce software that models an
organism that fends for itself in a real virtual world while producing the software we want. Have fun. Focus them.
– let them propose claims about the software they produce. Reward them when they
• defend their claims successfully or • oppose the claims of others successfully.
Crowdsourcing 83
Clear Feedback Sense of Progress
Possibility of Success
Authenticity
4/24/2011
![Page 84: Virtual Scientific-Community-Based Foundations for Popperian e-Science Karl Lieberherr Ahmed Abdelmeged Northeastern University, CCIS, PRL, Boston 9/17/20151.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649e575503460f94b4fe86/html5/thumbnails/84.jpg)
Reinterpret Refutation
• Refutation leads to successful strengthening or successful agreement.
04/19/23 84