Reflexive Metaprogramming Reflexive Metaprogramming in Rubyin Ruby

H. Conrad CunninghamH. Conrad CunninghamComputer and Information ScienceComputer and Information Science

University of MississippiUniversity of Mississippi

22

MetaprogrammingMetaprogramming

MetaprogrammingMetaprogramming: writing programs : writing programs that write or manipulate programs as that write or manipulate programs as data data

Reflexive metaprogramming: Reflexive metaprogramming: writing writing programs that manipulate themselves as programs that manipulate themselves as data data

33

Reflexive Metaprogramming Reflexive Metaprogramming LanguagesLanguages

EarlyEarly– LispLisp– SmalltalkSmalltalk

More recentMore recent– RubyRuby– PythonPython– GroovyGroovy

44

Basic Characteristics of RubyBasic Characteristics of Ruby(1 of 2)(1 of 2)

InterpretedInterpreted Purely object-orientedPurely object-oriented Single inheritance with mixinsSingle inheritance with mixins Garbage collectedGarbage collected Dynamically, but strongly typedDynamically, but strongly typed ““Duck typed”Duck typed” Message passing (to methods) Message passing (to methods)

55

Basic Characteristics of Ruby Basic Characteristics of Ruby (2 of 2)(2 of 2)

Flexible syntaxFlexible syntax– optional parentheses on method callsoptional parentheses on method calls– variable number of argumentsvariable number of arguments– two block syntax alternativestwo block syntax alternatives– symbol data typesymbol data type

String manipulation facilitiesString manipulation facilities– regular expressionsregular expressions– string interpolationstring interpolation

Array and hash data structuresArray and hash data structures

66

Why Ruby Supportive of Reflexive Why Ruby Supportive of Reflexive Metaprogramming (1 of 2)Metaprogramming (1 of 2)

Open classesOpen classes Executable declarationsExecutable declarations Dynamic method definition, removal, Dynamic method definition, removal,

hiding, and aliasinghiding, and aliasing Runtime callbacks forRuntime callbacks for– program changes (e.g. program changes (e.g. method_addedmethod_added))– missing methods (missing methods (missing_methodmissing_method))

77

Why Ruby Supportive of Reflexive Why Ruby Supportive of Reflexive Metaprogramming (2 of 2)Metaprogramming (2 of 2)

Dynamic evaluation of strings as codeDynamic evaluation of strings as code– at module level for declarations (at module level for declarations (class_evalclass_eval))

– at object level for computation (at object level for computation (instance_evalinstance_eval)) Reflection (e.g. Reflection (e.g. kind_of?, methods)kind_of?, methods) Singleton classes/methods for objectsSingleton classes/methods for objects Mixin modules (e.g. Mixin modules (e.g. EnumerableEnumerable)) Blocks and closuresBlocks and closures ContinuationsContinuations

88

Employee Class HierarchyEmployee Class HierarchyInitializationInitialization

class Employeeclass Employee @@nextid@@nextid = 1 = 1

def def initializeinitialize(first,last,dept,boss) (first,last,dept,boss) @fname@fname = first.to_s = first.to_s @lname = last.to_s@lname = last.to_s @deptid = dept @deptid = dept @supervisor = boss @supervisor = boss @empid = @@nextid @empid = @@nextid @@nextid = @@nextid + 1@@nextid = @@nextid + 1 endend

99

Employee Class HierarchyEmployee Class HierarchyWriter MethodsWriter Methods

def def deptid=(dept)deptid=(dept) # deptid = dept # deptid = dept @deptid = dept@deptid = dept endend

def supervisor=(boss)def supervisor=(boss) @supervisor = boss@supervisor = boss endend

1010

Employee Class HierarchyEmployee Class HierarchyReader MethodsReader Methods

def def name name # not an attribute# not an attribute @lname + ", " + @fname@lname + ", " + @fname endend

def def empid;empid; @empid; end @empid; end

def deptid; @deptid; enddef deptid; @deptid; end

def supervisordef supervisor @supervisor@supervisor end end

1111

Employee Class HierarchyEmployee Class HierarchyString Conversion ReaderString Conversion Reader

def to_sdef to_s @empid.to_s + " : " + name +@empid.to_s + " : " + name + " : " + @deptid.to_s + " (" + " : " + @deptid.to_s + " (" + @supervisor.to_s + ")" @supervisor.to_s + ")" endend

end # Employee end # Employee

1212

Employee Class HierarchyEmployee Class HierarchyAlternate InitializationAlternate Initialization

class Employeeclass Employee

@@nextid = 1@@nextid = 1

attr_accessor :deptid, :supervisorattr_accessor :deptid, :supervisor attr_reader :empidattr_reader :empid

def initialize(first,last,dept,boss) def initialize(first,last,dept,boss) # as before# as before endend

1313

Employee Class HierarchyEmployee Class HierarchyOtherOther Reader MethodsReader Methods

def namedef name @lname + ", " + @fname@lname + ", " + @fname end end

def to_sdef to_s @empid.to_s + " : " + name +@empid.to_s + " : " + name + " : " + @deptid.to_s + " (" + " : " + @deptid.to_s + " (" + @supervisor.to_s + ")" @supervisor.to_s + ")" endend

end # Employee end # Employee

1414

Employee Class HierarchyEmployee Class HierarchyStaff SubclassStaff Subclass

class class Staff < EmployeeStaff < Employee attr_accessor :titleattr_accessor :title

def initialize(first,last,dept,def initialize(first,last,dept, boss,title) boss,title) supersuper(first,last,dept,boss)(first,last,dept,boss) @title = title @title = title endend

def to_s def to_s supersuper.to_s + ", " + @title.to_s.to_s + ", " + @title.to_s end end end # Staffend # Staff

1515

Employee Class HierarchyEmployee Class HierarchyUsing Employee ClassesUsing Employee Classes

class TestEmployee class TestEmployee def def TestEmployee.do_testTestEmployee.do_test @s1 = @s1 = Staff.newStaff.new("Robert", "Khayat",("Robert", "Khayat", "Law", nil, "Chancellor")"Law", nil, "Chancellor") @s2 = Staff.new("Carolyn", "Staton",@s2 = Staff.new("Carolyn", "Staton", "Law", @s1,"Provost")"Law", @s1,"Provost") puts "s1.class ==> " + @s1.puts "s1.class ==> " + @s1.classclass.to_s.to_s puts "s1.to_s ==> " + @s1.to_sputs "s1.to_s ==> " + @s1.to_s puts "s2.to_s ==> " + @s2.to_sputs "s2.to_s ==> " + @s2.to_s @s1.deptid =@s1.deptid = "Chancellor" "Chancellor" puts "s1.to_s ==> " + @s1.to_sputs "s1.to_s ==> " + @s1.to_s puts "s1.methods ==> " + puts "s1.methods ==> " + @s1@s1.methods.methods.join(", ").join(", ") endendend # TestEmployeeend # TestEmployee

1616

Employee Class HierarchyEmployee Class HierarchyTestEmployee.do_testTestEmployee.do_test Output Output

irbirbirb(main):001:0> load "Employee.rb"irb(main):001:0> load "Employee.rb"=> true=> trueirb(main):002:0> TestEmployee.do_testirb(main):002:0> TestEmployee.do_tests1.class ==> Staffs1.class ==> Staffs1.to_s ==> 1 : Khayat, Robert : Law (), s1.to_s ==> 1 : Khayat, Robert : Law (),

ChancellorChancellors2.to_s ==> 2 : Staton, Carolyn : Law (1 : s2.to_s ==> 2 : Staton, Carolyn : Law (1 :

Khayat, Robert : Law (), Chancellor), ProvostKhayat, Robert : Law (), Chancellor), Provosts1.to_s ==> 1 : Khayat, Robert : Chancellor (), s1.to_s ==> 1 : Khayat, Robert : Chancellor (),

ChancellorChancellors1.methods ==> to_a, respond_to?, display, s1.methods ==> to_a, respond_to?, display,

deptid, type, protected_methods, require, deptid, type, protected_methods, require, deptid=, title, … kind_of?deptid=, title, … kind_of?

=> nil=> nil

1717

Ruby MetaprogrammingRuby MetaprogrammingClass MacrosClass Macros

Every class has Every class has ClassClass object where object where instance methods resideinstance methods reside

Class definition is executable Class definition is executable Class Class ClassClass extends class extends class ModuleModule Instance methods of class Instance methods of class ModuleModule

available during definition of classesavailable during definition of classes Result is essentially “class macros”Result is essentially “class macros”

1818

Ruby MetaprogrammingRuby MetaprogrammingCodeCode String EvaluationString Evaluation

class_eval class_eval instance method ofinstance method of classclass Module Module– evaluates string as Ruby code evaluates string as Ruby code

– using context of class using context of class ModuleModule– enabling definition of new methods and constantsenabling definition of new methods and constants

instance_evalinstance_eval instance method of class instance method of class ObjectObject– evaluates string as Ruby codeevaluates string as Ruby code

– using context of the objectusing context of the object

– enabling statement execution and state changesenabling statement execution and state changes

1919

Ruby MetaprogrammingRuby MetaprogrammingImplementingImplementing attr_reader attr_reader

# Not really implemented this way# Not really implemented this wayclass Moduleclass Module # add to system class # add to system class defdef attr_reader attr_reader(*syms)(*syms) syms.each do |sym|syms.each do |sym| class_evalclass_eval %{def #{sym}%{def #{sym} @#{sym}@#{sym} end}end} end # syms.eachend # syms.each end # attr_readerend # attr_readerend # Moduleend # Module

2020

Ruby MetaprogrammingRuby MetaprogrammingImplementingImplementing attr_writer attr_writer

# Not really implemented this way# Not really implemented this wayclass Module # add to system classclass Module # add to system class def attr_writer(*syms)def attr_writer(*syms) syms.each do |sym|syms.each do |sym| class_eval %{def #{sym}class_eval %{def #{sym}=(val)=(val) @#{sym} @#{sym} = val= val end}end} endend end # attr_writerend # attr_writerend # Moduleend # Module

2121

Ruby MetaprogrammingRuby MetaprogrammingRuntime CallbacksRuntime Callbacks

class Employee # class definitions executableclass Employee # class definitions executable def def Employee.inheritedEmployee.inherited(sub) # class method(sub) # class method puts "New subclass: #{sub}" # of Classputs "New subclass: #{sub}" # of Class endend class Faculty < Employeeclass Faculty < Employee endend class Chair < Facultyclass Chair < Faculty endend

OUTPUTSOUTPUTS

New subclass: FacultyNew subclass: Faculty New subclass: Chair New subclass: Chair

2222

Ruby MetaprogrammingRuby MetaprogrammingRuntime CallbacksRuntime Callbacks

class Employeeclass Employee def def method_missing(meth,*args) method_missing(meth,*args) # instance method# instance method mstr = meth.to_s # of Objectmstr = meth.to_s # of Object last = mstr[-1,1]last = mstr[-1,1] base = mstr[0..-2]base = mstr[0..-2] if last == "="if last == "=" class_eval("attr_writer :#{base}")class_eval("attr_writer :#{base}") elseelse class_eval("attr_reader :#{mstr}")class_eval("attr_reader :#{mstr}") endend endendendend

2323

Domain Specific Languages Domain Specific Languages (DSL)(DSL)

Programming or description language Programming or description language designed for particular family of problemsdesigned for particular family of problems

Specialized syntax and semanticsSpecialized syntax and semantics Alternative approachesAlternative approaches– External language with specialized interpreterExternal language with specialized interpreter

– Internal (embedded) language by tailoring a Internal (embedded) language by tailoring a general purpose languagegeneral purpose language

2424

Martin Fowler DSL ExampleMartin Fowler DSL ExampleInput Data FileInput Data File

#123456789012345678901234567890123456#123456789012345678901234567890123456

SVCLFOWLER 10101MS0120050313SVCLFOWLER 10101MS0120050313

SVCLHOHPE 10201DX0320050315SVCLHOHPE 10201DX0320050315

SVCLTWO x10301MRP220050329SVCLTWO x10301MRP220050329

USGE10301TWO x50214..7050329USGE10301TWO x50214..7050329

2525

Martin Fowler DSL ExampleMartin Fowler DSL ExampleText Data DescriptionText Data Description

mapping SVCL dsl.ServiceCallmapping SVCL dsl.ServiceCall 4-18: CustomerName4-18: CustomerName 19-23: CustomerID19-23: CustomerID 24-27 : CallTypeCode24-27 : CallTypeCode 28-35 : DateOfCallString28-35 : DateOfCallString

mapping USGE dsl.Usagemapping USGE dsl.Usage 4-8 : CustomerID4-8 : CustomerID 9-22: CustomerName9-22: CustomerName 30-30: Cycle30-30: Cycle 31-36: ReadDate31-36: ReadDate

2626

Martin Fowler DSL ExampleMartin Fowler DSL ExampleXML Data DescriptionXML Data Description

<ReaderConfiguration><ReaderConfiguration> <Mapping Code = "SVCL" TargetClass = "dsl.ServiceCall"><Mapping Code = "SVCL" TargetClass = "dsl.ServiceCall"> <Field name = "CustomerName" start = "4" <Field name = "CustomerName" start = "4" end = "18"/>end = "18"/> <Field name = "CustomerID" start = "19" end = "23"/><Field name = "CustomerID" start = "19" end = "23"/> <Field name = "CallTypeCode" start = "24" <Field name = "CallTypeCode" start = "24" end = "27"/>end = "27"/> <Field name = "DateOfCallString" start = "28" <Field name = "DateOfCallString" start = "28" end = "35"/>end = "35"/> </Mapping></Mapping> <Mapping Code = "USGE" TargetClass = "dsl.Usage"><Mapping Code = "USGE" TargetClass = "dsl.Usage"> <Field name = "CustomerID" start = "4" end = "8"/><Field name = "CustomerID" start = "4" end = "8"/> <Field name = "CustomerName" start = "9" <Field name = "CustomerName" start = "9" end = "22"/>end = "22"/> <Field name = "Cycle" start = "30" end = "30"/><Field name = "Cycle" start = "30" end = "30"/> <Field name = "ReadDate" start = "31" end = "36"/><Field name = "ReadDate" start = "31" end = "36"/> </Mapping></Mapping></ReaderConfiguration></ReaderConfiguration>

2727

Martin Fowler DSL ExampleMartin Fowler DSL ExampleRuby Data DescriptionRuby Data Description

mapping('SVCL', ServiceCall) domapping('SVCL', ServiceCall) do extract 4..18, 'customer_name'extract 4..18, 'customer_name'extract 19..23, 'customer_ID'extract 19..23, 'customer_ID'extract 24..27, 'call_type_code'extract 24..27, 'call_type_code'extract 28..35, 'date_of_call_string'extract 28..35, 'date_of_call_string'

endendmapping('USGE', Usage) domapping('USGE', Usage) doextract 9..22, 'customer_name'extract 9..22, 'customer_name'extract 4..8, 'customer_ID'extract 4..8, 'customer_ID'extract 30..30, 'cycle'extract 30..30, 'cycle'extract 31..36, 'read_date‘extract 31..36, 'read_date‘

endend

2828

Martin Fowler DSL ExampleMartin Fowler DSL ExampleRuby DSL Class (1)Ruby DSL Class (1)

require 'ReaderFramework'require 'ReaderFramework'

class BuilderRubyDSLclass BuilderRubyDSL

def initialize(filename)def initialize(filename) @rb_dsl_file = filename@rb_dsl_file = filename endend

def configure(reader)def configure(reader) @reader = reader@reader = reader rb_file = File.new(@rb_dsl_file)rb_file = File.new(@rb_dsl_file) instance_eval(rb_file.readinstance_eval(rb_file.read, @rb_dsl_file), @rb_dsl_file) rb_file.close rb_file.close endend

2929

Martin Fowler DSL ExampleMartin Fowler DSL ExampleRuby DSL Class (2 of 3)Ruby DSL Class (2 of 3)

def mapping(code,target)def mapping(code,target) @cur_mapping = ReaderFramework::ReaderStrategy.new(@cur_mapping = ReaderFramework::ReaderStrategy.new( code,target)code,target) @reader.add_strategy(@cur_mapping)@reader.add_strategy(@cur_mapping) yieldyield endend

def extract(range,field_name)def extract(range,field_name) begin_col = range.beginbegin_col = range.begin end_col = range.endend_col = range.end end_col -= 1 if range.exclude_end? end_col -= 1 if range.exclude_end? @cur_mapping.add_field_extractor(@cur_mapping.add_field_extractor( begin_col,end_col,field_name)begin_col,end_col,field_name) endend

end#BuilderRubyDSLend#BuilderRubyDSL

3030

Martin Fowler DSL ExampleMartin Fowler DSL ExampleRuby DSL Class (3 of 3)Ruby DSL Class (3 of 3)

class ServiceCall; endclass ServiceCall; end

class Usage; endclass Usage; end

class TestRubyDSLclass TestRubyDSL def TestRubyDSL.rundef TestRubyDSL.run rdr = ReaderFramework::Reader.newrdr = ReaderFramework::Reader.new cfg = cfg = BuilderRubyDSL.newBuilderRubyDSL.new("dslinput.rb")("dslinput.rb") cfg.cfg.configureconfigure(rdr)(rdr) inp = File.new("fowlerdata.txt")inp = File.new("fowlerdata.txt") res = rdr.process(inp)res = rdr.process(inp) inp.closeinp.close res.each {|o| puts o.inspect}res.each {|o| puts o.inspect} endendendend

3131

Using Blocks and Iterators Using Blocks and Iterators Inverted Index (1)Inverted Index (1)

class InvertedIndexclass InvertedIndex @@wp = /(\w+([-'.]\w+)*)/ @@wp = /(\w+([-'.]\w+)*)/ DEFAULT_STOPS = {"the" => true, "a" => true, DEFAULT_STOPS = {"the" => true, "a" => true, "an" => true}"an" => true}

def initialize(*args)def initialize(*args) @files_indexed = []@files_indexed = [] @index = Hash.new@index = Hash.new @stops = Hash.new@stops = Hash.new if args.size == 1if args.size == 1 args[0].each {|w| @stops[w] = true}args[0].each {|w| @stops[w] = true} elseelse @stops = DEFAULT_STOPS@stops = DEFAULT_STOPS endend endend

3232

Using Blocks and Iterators Using Blocks and Iterators Inverted Index (2)Inverted Index (2)

def index_file(filename)def index_file(filename) unless @files_indexed.index(filename) == nilunless @files_indexed.index(filename) == nil STDERR.puts("#{filename} already indexed.")STDERR.puts("#{filename} already indexed.") returnreturn endend unless File.exist? Filenameunless File.exist? Filename STDERR.puts("#{filename} does not exist.")STDERR.puts("#{filename} does not exist.") returnreturn endend unless File.readable? Filenameunless File.readable? Filename STDERR. puts("#{filename} is not readable.")STDERR. puts("#{filename} is not readable.") returnreturn endend @files_indexed << filename@files_indexed << filename

3333

Using Blocks and Iterators Using Blocks and Iterators Inverted Index (3)Inverted Index (3)

inf = File.new(filename)inf = File.new(filename) lineno = 0lineno = 0 inf.each do |s|inf.each do |s| lineno += 1lineno += 1 words = s.scan(@@wp).map {|a| a[0].downcase}words = s.scan(@@wp).map {|a| a[0].downcase} words = words.reject {|w| @stops[w]}words = words.reject {|w| @stops[w]} words = words.map {|w| words = words.map {|w| [w,[filename,[lineno]]]}[w,[filename,[lineno]]]} words.each do |p| words.each do |p| @index[p[0]] = [] unless@index[p[0]] = [] unless @index.has_key? p[0]@index.has_key? p[0] @index[p[0]] = @index[p[0]].push(p[1])@index[p[0]] = @index[p[0]].push(p[1]) endend endend inf.closeinf.close

3434

Using Blocks and Iterators Using Blocks and Iterators Inverted Index (4)Inverted Index (4)

@index.each do |k,v| # k => v is hash entry@index.each do |k,v| # k => v is hash entry @index[k] = v.sort {|a,b| a[0] <=> b[0]}@index[k] = v.sort {|a,b| a[0] <=> b[0]} endend @index.each do |k,v|@index.each do |k,v| @index[k] =@index[k] = v.slice(1...v.length).inject([v[0]]) v.slice(1...v.length).inject([v[0]]) do |acc, e|do |acc, e| if acc[-1][0] == e[0]if acc[-1][0] == e[0] acc[-1][1] = acc[-1][1] + e[1]acc[-1][1] = acc[-1][1] + e[1] elseelse acc = acc + [e]acc = acc + [e] endend accacc endend end#@index.each's blockend#@index.each's block selfself end#index_fileend#index_file

3535

Using Blocks and Iterators Using Blocks and Iterators Inverted Index (5)Inverted Index (5)

def lookup(word)def lookup(word) if @index[word]if @index[word] @index[word].map {|f| @index[word].map {|f| [f[0].clone, f[1].clone] } [f[0].clone, f[1].clone] } elseelse nil nil endend endend # …# …endend

3636

QuestionsQuestions