Preon (J-Fall 2008)
-
Upload
wilfred-springer -
Category
Technology
-
view
3.745 -
download
4
Transcript of Preon (J-Fall 2008)
![Page 1: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/1.jpg)
Err....
QuarksPreon
![Page 2: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/2.jpg)
Preon
Adeclarativedatabindingframeworkforbinaryencodeddata
![Page 3: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/3.jpg)
Binary Encoded Data?
Anyfileforwhich'file‐bi{filename}'doesnotreturnamimetypestartingwith'text/'
Non‐binary:text/plain(*.txt)text/html(*.html)text/xml(*.xml)
Binary:application/pdf(*.pdf)application/octet‐stream(*.mp4)image/png(*.png)
![Page 4: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/4.jpg)
Not only byte stream
Alwaysoctects(8bitvalues)
8bits8bits
![Page 5: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/5.jpg)
But also bit stream
Alwaysoctects(8bitvalues)
5bits 5bits
![Page 6: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/6.jpg)
Compressed Data
2kg
1021pages
![Page 7: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/7.jpg)
Network Traffic
![Page 8: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/8.jpg)
TomTom Map Files
Approx.300pagesofC++sourcecode,justfordecodingonly...
![Page 9: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/9.jpg)
Why?
Binary45.71%
Non‐binary54.29%
Binaryvs.non‐binarydistributiononarandomdirectoryonmysystem
![Page 10: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/10.jpg)
Why?
...andonmywife'ssystem
Binaryfilesonly100%
![Page 11: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/11.jpg)
Challenges
● Decodingfrombitstreamnotforthefaint‐hearted
● Encodingtobitstreamnotforthefaint‐hearted
● Hardtomaintain
● Hardtoextend
● Javadoesn'thelp(Bug4504839)
● Decoderandencodereasilygooutofsync
● Documentationandsoftwareeasilygooutofsync
![Page 12: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/12.jpg)
What if....
..this could all be solved easily?
![Page 13: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/13.jpg)
Preon Ambitions
● Declarativelymapdatastructuretoencodingformat
● Getthedecoder/encoder/documentation,freeofcharge
![Page 14: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/14.jpg)
5-Second User Guide
01Filefile=newFile(...);02Codec<BitMap>codec=Codecs.create(BitMap.class);03BitMapbitmap=Codecs.decode(codec,file);
DecodingaBitMap:
![Page 15: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/15.jpg)
What just happened?
01Filefile=newFile(...);02Codec<BitMap>codec=Codecs.create(BitMap.class);03BitMapbitmap=Codecs.decode(codec,file);
CreateaCodecforinstancesofBitMap
...anduseittodecodeaBitMap.
![Page 16: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/16.jpg)
Find the specification
ThedatastructureISthespecification:classBitMap{@Boundintwidth;@Boundintheight;@BoundintnrColors;@BoundList(size=”nrColors”)Color[];@BoundList(size=”width*height”)byte[]pixels;}
classColor{@Boundintred;@Boundintgreen;@Boundintblue;}
![Page 17: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/17.jpg)
Demo Decoding
![Page 18: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/18.jpg)
But what is actually happening?
CodecisnothingbutafacacetoachainofCodecs.
EachCodeconlyunderstandsitsowntask.
CodecswilldelegatetootherCodecs
![Page 19: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/19.jpg)
Codecs
Encapsulateeverythingthereistoknowaboutthemappingbetweenin‐memoryrepresentationandencodedrepresentation
![Page 20: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/20.jpg)
Demo Documentation
![Page 21: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/21.jpg)
So, now....
ForgeteverythingIjusttoldyou
![Page 22: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/22.jpg)
Preon just works
Annotations
● @Bound● @BoundList● @BoundString● @BoundNumber● @BoundObject● @BoundExplicitly● @If● @LazyLoading● @LengthPrefix● @TypePrefix● @ByteAlign● @Slice
Types
int,byte,short,long,Integer,Byte,Short,Long,boolean,Boolean,String,List<T>,T[],type‐safeenums,Object
Expressions
attributes:.{name}items:[{number}],or[{expr}]arithmetic:+,‐,/,*,^combinatorial:&&,||relational:>=,<=,==,<,>literals:'foobar',0x0f,0b01101
![Page 23: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/23.jpg)
Convention over Configuration
//DefaultsettingsforintCodec://32‐bits,littleendian@Boundintfoo;
//Readanint,butconstructtheint//from5bitsonly@BoundNumber(size=”5”)intfoo;
![Page 24: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/24.jpg)
Expressions (1)
/***Aniconofmaximal15*15pixels.Eachcolor*hasacolorintherange0‐255.*/publicclassIcon{@BoundNumber(size=”4”)intheight;@BoundNumber(size=”4”)intwidth;@BoundList(size=”height*width”)byte[]pixels;}
ByacceptingStringsinsteadofbooleansandintegers,Preonallowsyoutopassinexpressions.
![Page 25: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/25.jpg)
Expressions (2)
@BoundString(size=”{expr}”,...)
@BoundList(size=”{expr}”,offset=”{expr}”,...)
@BoundNumber(size=”{expr}”,...)
@Slice(“{expr}”)
...
![Page 26: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/26.jpg)
Boolean expressions
@BoundprivateintmapVersion;
@If(“mapVersion>=700”)@BoundprivateMapFlagsflags;
//Butalso://mapVersion+1>=700//mapVersion>3&&mapVersion<300//etc.
![Page 27: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/27.jpg)
References
Backwardreferencesonly
Referencethe“outer”object
addresses[1].streetouter.driversLicenses[1].statedriveresLicenses[nrDriverLicenses‐1].state
![Page 28: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/28.jpg)
Variable Introductions
@Boundint[]offsets;@BoundList(offset=”offsets[index]”,...)List<Node>nodes;
● PreonwillinjectListimplementation
● ListimplementationwillloadNodeslazily,ondemand
● Callingnodes.get(3)willcausetheListimplementationto
– Calculatethenode'sposition:offsets[index=3]=120
– CallCodec<Node>tostartdecodingfromthatpoint
Introduction
![Page 29: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/29.jpg)
Inheritance
JavaClasses PreonPerspective
![Page 30: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/30.jpg)
Codecs class is your friend
static <T> T decode(Codec<T> codec, byte[] buffer)static <T> T decode(Codec<T> codec, ByteBuffer buffer)static <T> T decode(Codec<T> codec, File file)
static <T> Codec<T> create(Class<T> type) static <T> Codec<T> create(Class<T> type, CodecFactory...
factories)static <T> Codec<T> create(Class<T> type, CodecDecorator...
decorators)
static <T> void document(Codec<T> codec, ArticleDocument document)
static <T> void document(Codec<T> codec, DocumentType type, OutputStream out)
static <T> void document(Codec<T> codec, DocumentType type, File file)
![Page 31: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/31.jpg)
Preon Layers
Anexpressionlanguagecapableofrenderingitselftohumanreadabletext.(limbo.sourceforge.net)
Afluentinterfaceforgeneratingdocuments.(pecia.sourceforget.net)
BitBufferabstractions
Databinding
![Page 32: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/32.jpg)
Preon License
Apache2.0
Apache2.0
GPL+ClasspathException
GPL+ClasspathException
![Page 33: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/33.jpg)
If not Preon, then what else?
Declarativeapproachisnotnew:– BSDL(BitstreamSyntaxDescriptionLanguage)– Flavor(http://flavor.sourceforge.net/)– XFlavor– BFlavor(http://multimedialab.elis.ugent.be/bflavor/)
Noneofthemwouldworkedinourcase:– Overlycomplicated– Nosolidimplementation– Assumptionsaresurreal:
● “everythingwilljustfitinmemory”● “therewillonlybeasinglethread”● “ourcurrentfeaturesetisallyoueverneed”
![Page 34: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/34.jpg)
The Preon Answer
“everythingwilljustfitinmemory”
Preoniscapableofusingmemory‐mappeddata.(DefaultBitBufferimplementationwrapsaroundByteBuffer,andhencealsoMappedByteBuffer.)
“therewillonlybeasinglethread”
InPreon,everythreadcanhaveitsownreferencetothecurrentpositionintheBitBuffer.
“ourcurrentfeaturesetisallyoueverneed”
Preonhasincrediblyopen:implementCodecFactoryorCodecDecoratortocreateyourowncodecsbasedontypeinformationorannotations.
![Page 35: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/35.jpg)
CodecFactory
interface CodecFactory { <T> Codec<T> create(AnnotatedElement metadata, Class<T> type, ResolverContext context);}
AnnotationsontheobjectexpectingdatatobedecodedbytheCodec.
Thetypeofobjecttobedecoded.
Theobjectforconstructingreferences.
returnsnullifitdoesnothaveawaytoconstructaCodec
![Page 36: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/36.jpg)
CodecFactory Usage
● CodecFactoriescanbepassedtoCodecs.create(...)
● ...whichwillcausetheCodecsclasstoconsiderthesefactorieswhenconstructingthevariousCodecs.
● Internally,abigchainofcommands
![Page 37: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/37.jpg)
Current Status
● http://preon.sourceforge.net/
● Interfacesfairlystable
● Currentversion1.0‐SNAPSHOT
● CompleteJavaclassexamplebeforefirstreleasecandidate
● Bugs...
● Iwantyou
![Page 38: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/38.jpg)
Future work
● Theencodeoperation(after1.0)
● Betterdebugging
● Annotationdrivensupportforothercompressiontechniques
● Morehyperlinkinginthedocumentation
● Betteralgebraicsimplificationofexpressions
● DescriptionsfromJavaDoc
![Page 39: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/39.jpg)
If there is only a couple of things...
● XMLisjustalamewayofbinaryencoding
● PreoncutsouttheInfosetmodel,preventingunnecessarytransformations
● Preonmakesbinaryencodingeasy
● Preonisextensible
● Preon(probably)scalesquitewell
● PreonisfriendlytoJavadevelopers
![Page 40: Preon (J-Fall 2008)](https://reader033.fdocuments.in/reader033/viewer/2022042614/5560d029d8b42a08088b4ded/html5/thumbnails/40.jpg)
Confused?