MediaQ - University of Southern Californiashin630/Youngmin/files/MediaQ_Android.pdf · MediaQ is an...
Transcript of MediaQ - University of Southern Californiashin630/Youngmin/files/MediaQ_Android.pdf · MediaQ is an...
MediaQ1 Introduction
MediaQisanonlinemediamanagementframeworkthatprovideservicestocollect,organizeandshareuser-generatedmobilevideos/imageswithautomaticallytaggedgeo-spatialmetadata.Itcouldbeusedindisasterresponsesituationsuchasearthquakeorfireplace.Thegovernmentandanyhelpingassociationunitcangetthefirstin-placelooktohelpmaketheaccuratedecision.Consideringthebroadcastofnetworkincrowdedplace,afterusergeneratesvideo/photo,MediaQalwaysuploadthesmall-sizemetadatafirstwhichcontaincapturetime,cameralocation,trajectoryandviewingdirection.Thenasanyoneinterestedinthatvideo/photo,weuseGoogleCloudMessagetomanagethecontentqueryfromotheruserandsendnotificationtothecontentowner.Therefore,thegoalofourfinalprojectistodevelopsuchfeaturetogetherwithphotographyuploadinginAndroidphones.
Figure1MediaQHomeScreen
2 BasicFunctions2.1 Recordavideoandtakeapicture
Userscanrecordavideoandtakeapicturelikeothercameraapp.Whiledoingit,ourappsavesthemetadatasuchasGPS,Compassdata,Internetinformationanddeviceinformationinthebackgroundside.
Figure2.Screenshotofcamera2.2 Automaticmetadatauploading
Whenuserscreatecontentbyrecordingavideoortakingapicture,metadataassociatedwithitisautomaticallyuploadedtotheserver.Contentitselfisonlyuploadedwhenusertriggersittoavoidunwantednetwork/datausageforuploadinglargefiles(metadatafilesizeisverysmall,atmostafewkBs).Westoredthedeviceid,userId,andlocationinformationandsoontobothshowvideoorphotoiconsintheMediaQwebsiteandidentifythefileholderlateron.2.3 Uploadvideosandpictures
Whenuserspushtheuploadbutton,thepictureandvideocontentsareuploadedtotheserverincludingtheirmetadata.2.4 ViewTranjectory
Whilerecordingavideoandtakingapicture,ourappsavesthepathdescribedbyanandroiddevicemovingbyusingGPSinformation.Userscanseethepathongooglemaps.2.5 TasksInquiriesandPendingTasks
Userscanasksometaskstotheserver.Theservercanreceivetheuser’srequestsandgivesometaskslikequeststousers.Afterreceivingthetasks,usercanuploadthevideosandpicturestakenfromthespecificplacewhichtheservergave.
2.6 PushnotificationsSincemetadataisautomaticallyuploaded,serverknowswhereusersrecorded
videos/photos,eveniftheydidn’tuploadthecontent.Theservercansendoutpushnotificationstotheclientstorequestthattheyuploadspecificmedia.2.7 Minorfunctions
Userscanviewlistofrecordedvideos,changethemood,previewthevideos,seewheretheyweretaken,uploadthem,deletethem,viewthetrajectoryongooglemaps,andetc,
Figure3.PendingTasks
Figure4.Videopreview/upload/delete
3 Details3.1 OverallApplication
Figure5.MediaQArchitecture
TherearethreepartitionofthisoverallsystemshowninFigure5.Androidapplication,GCM
serveranddataserver.AndroidapplicationuploadmetadatatodataserverandalsocommunicatewithGCMserver.AnotificationsentbyGCMservernoticeusertouploadtherecordedvideo/photocontent.Formorespecific,theMediaQapplicationnotonlyuploadthemetadata,theclientalsoregistertheirdeviceIDonserver.Thentheserverkeeptrackallclient'sIDanduseittodistinguisheachdevice.Sowhenuserwanttoparticipatewithvideowhichisnotuploadedyet,theserverisabletosentthenotificationtothecontentownerthroughGooglecloudmessage,askingthemuploadtherealdata.
3.2 AnalyzingaJsonfile(Thedatasendingtotheserver)
{ "format_version":"0.4", "video_id":"1lwpl9ky2c21o_2016_3_23_Videotake_1458763979183.mp4", "owner_properties":{ "tid_type":"email", "id":"[email protected]" }, "device_properties":{ "SIM_id":"1lwpl9ky2c21o", "OS":"android", "OS_version":"5.0", "firmware_version":"5.0", "model_name":"SM-N900K", "camera_direction":180 }, "UploadType":"request", //request:taskinquiries,personal:personalupload "TaskId":"4341029c-8095-4e", "TaskInquiryId":"f001dd75-49ec-4b", "TaskLat":"null", "TaskLong":"null", "UserId":"2832bc0e-cfc8-48", "Username":"user190", "TaskType":"video", "keywords":[ ], "LocationType":"outdoor", "LevelOfVideoPrivacy":"Public", "Mood":2, "UploadDate":"03/23/2016/08:13:07/PM/UTC-07:00", "RecordDate":"03/23/2016/08:12:59/PM/UTC-07:00", "sensor_data":[ { "location_array_timestamp_lat_long_accuracy":[ //GPSdata [1458763979926,03/23/2016/08:12:59/PM/UTC-07:00,0.000000,0.000000,0.000000,0.0], [1458763979960,03/23/2016/08:12:59/PM/UTC-07:00,34.019416,-118.288382,17.08,22.092] ] }, { "sensor_array_timestamp_x_y_z":[ //Compassdata [1458763979926,03/23/2016/08:12:59/PM/UTC-07:00,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0], [1458763980155,03/23/2016/08:13:00/PM/UTC-07:00,220.60396,-56.323544,343.79306,252.4256,-
1.5382746,5.2923827,8.271368,92.8,-121.7,-30.7,-0.0892159,-0.004261058,0.021837922], [1458763980329,03/23/2016/08:13:00/PM/UTC-07:00,217.96803,-57.671223,341.94247,254.1888,-
1.638831,5.0266266,8.353969,92.8,-121.1,-31.4,-0.090281166,-0.09853696,-0.0061252704], [1458763980532,03/23/2016/08:13:00/PM/UTC-07:00,217.61182,-61.34031,342.43127,255.716,-
1.5532384,4.9057193,9.414599,92.8,-119.9,-31.4,-0.1536644,-0.24021713,-0.09773801] ] } ,{ "wifi_array_timestamp_BSSID_level_frequency":[ [1458763979659,"00:24:6c:b4:1e:19",-58,"5745"] ] }]}
3.2.1 Location_array_timestamp_lat_long_accruacy(1)Timestamp(2)Recordingdate(3)Latitude(4)Longitude(5)Altitude(6)Accuracy
3.2.2 Sensor_array_timestamp_x_y_z
(1)Timestamp(2)Recordingdate(3)Orientationz(manipulatedazimuth)(4)Orientationx(manipulatedpitch)(5)Orientationy(manipulatedroll)(6)Light(7)Accelerometerx(8)Accelerometery(9)Accelerometerz(10)MagneticFieldx(11)MagneticFieldy(12)MagneticFieldz(13)Gyroscopex(14)Gyroscopey(15)Gyroscopez
3.3 Pushnotificationstorequestuploadingofmediacontents
Figure6.flowofgettingGCMRegistrationID
Toidentifyeachintent,weretrieveGCMregistrationIDbycommunicatewithGCMserverasFigure6shown.InthebeginningwegetthegoogleAPIkeyatGoogleDevelopersConsole.WiththeAPIkey,MediaQapplicationcommunicatewithgooglecloudmessageandgetRegistrationID.Finally,theclientshareRegIDwithserverandpostnotificationtoGCM.
4 Theroleoffiles4.1 Serverupload[_pictures].php
Ourapplicationsupportbothpictureandvideorecordinganduploading.AfterreceivednotificationfromGCM,wedeployedthisAPItouploadthevideo/photocontenttoMediaQDatabaseserver.uploadstatus[_picture].php
Toensureweknowthefilecontentormetadataiscorrectlystoredintheserver,especiallyallthemetadataisuploadedsuccessfullyontheserver,werequestthefilestatusthroughuploadstatus[_picture].phpanditreturnsusboththemetadatalength,andcontentlengthoftherequestedfile.Ifbothlengthmatchesthelocalfilelengths,wemarkedthisfileascompletedintheclientapplication.Otherwise,ifthemetadatasizeislessthelocalfile,weuploadedmetadata.Thisisaservicetohelpclientapplicationsknowthefilestatusintheserver,andclientwilltrytouploadmetadatawheneveritcanwhenserverdoesnotacquirethemetadatabecauseofsomeinterruptionofinternetorserverbreakdownatthemomentthatuserjusttakesvideosorphotos.GCMDeviceRegistration.php
Onthedataserver,therearenotonlystoredmediacontentandmetadata,butalsoserverservicesrunonit.OneoftheseisGCMdeviceregistration.
Foreveryintent,GooglewillassignauniqueIDwhichisdependsontheApplication.Inthisphpprocess,wecollectanyidenticalinformationfromdevice,userandregistrationIDgettingfromgoogleserverandproducethemtostoreonMediaQserverforlaterGCMnotificationusage.
4.2 Client(Android)DashboardActivity
FirstscreenshownasFigure7isthepageuserseesafterloggingin.Inthispage,usercanaccessTaskInquiries,PendingTasksandvideos/photoscollection.Whilethispageload,itimmediatelyretrievesregistrationtokenfromGCMserverandsendingdevicetokentoMediaQdataserver,toregisterforpushnotificationsservice.
Figure7.DashboardpageGeoVidDashboardActivity
AsFigure8shown,thedashboardsupportusertoaccessthatcontentrecordpageandnavigatestolistofvideos/photos.Thisdashboardisalsoautomaticallyswitchedtorightafteruserjusttakenavideo/photos.
Morespecific,thevideotakencontentalsocontainthetrajectoryandthedirectionofcamera.Soit’sgeo-spatialmediacontent.
Figure8.DataCollectionincludingVideoandPhoto
GeoVidActivityThisistheactivitythatshowalistofvideo/photocontents.Itdisplaysalistofcontents,
eachwithitsinformation,screenshotpicture,takendateandtimeandtheuploadstatus.Eachelementconnectstoitsvideo/photo.Whenauserselectsanitem,thatcorrespondvideo/photoisreadyforuploadingbyoneclickonitandchoose“upload”.GeoVidDialog
Thisdialogmenuactivitywillbeshownwhenuserselectsoneofvideos/photosinGeoVidActivity.GeoVidActivityretrievethecontentidandpopupandialog.Fromhere,usercanupload,changemood,preview,etc.
Figure9.Dialogtomakechangeofanyvideointhelist
GeoRecorderActivityThisactivityallowstheusertotakeapictureforanewvideo/photocontent.Itutilizesthe
Androidphone’scameratotakeapictureorvideoandsavesitontothephoto’slocalstoragepriortosendingthemetadatatotheMediaQdataserver.Afterfinishing,theactivityflowreturntoGeoVidActivity.GeoVideo
ThisserviceEncapsulatestheAndroidCameraAPIs,andtherecordingofmetadata(GPS,sensors)alongwithrecordingvideo.Afterthethevideo/phototaken,themetadatawillbeproducedandwillbeuploadtoMediaQdatabaseserverafteritbycallingUploadService.UploadService
Thisserviceisusedtouploadmetadataand/orcontenttotheserver.Inourapplicationthemetadataisimmediatelyuploadafterthevideo/photocontenthasbeentakenandthecontentwillbeaskedtouploadwhentheuserisnotifiedbyaninquiryanduploadthevideo/photocontent.Andtheseuploadingprocessisservedwiththisactivity.Weuploadfileschuckby
chuck,foreachchuck,theUploadServiceretriesatmaximal10timestoavoidsomesuddeninternetinterruptionstoppinguploading.MetadataTimedUploadService
Thisserviceisusedtotryuploadingmetadataforfileswhichhaven’thadmetadatauploadedyet.Incaseoffailure,retriesafter2minutes,foramaximumof10times.GcmBroadcastReceiver
ThisserviceisusedtoreceiveGCMmessages(i.e.pushnotificationsfromserver).LaunchesGcmMessageHandler.TheapplicationperiodicallysubscribestheGCMservertocheckifanynewinquiryhasbeensentwhichthesubjectcontentowneristheselfdevice.Iftheserverrespondswithnewvideo/photocontentinquiryandthisdeviceanduseristhecontentowner,theuserwillbenotifiedviaGooglecloudmessagingandthenotificationwillinformthedetailinformationofthedesirevideo/photocontent,filenameashelpingusertouploadassignedcontent.GcmMessageHandler
HandlesGCMmessagesbyshowingnotificationinthesystemnotificationbarwiththemessagethatwasreceivedfromtheserver.Iftheserverrespondswithnewvideo/photocontentinquiryandthisdeviceanduseristhecontentowner,therewillbeanmessageshownatnotificationbarandclearlyinformtheaskedfiletobeupload.
Figure10.ReceiveGCMNotification
5 Classdiagram
Figure11.Theflowofprocess
6 Futureworks6.1 GCMnotification
Fornow,theGCMnotificationonlyshowwhichfileisrequestedtoupload.Inthefuture,itmightdirectlylinktotheuploadingactivityofspecificfileforusertouploadthefile.6.2 GeoRecorderActivity
Fornow,whenrecordingavideoandtakingapicture,wegetrotationsensordatafromtheandroiddevice.Whilerotatingthedevicetorecordavideoforlandscapeorportraitmode,somelagsoccur.Itmightbefixedbyusingorientationsensordatainsteadofrotationsensordata.6.3 Compassdata
Intheserverside,onlyanazimuthvalue(orientation[0])isusedamongthe15compassdata.Wewillthinkabouthowremainingdatawilluse.6.4 Resolution
Currently,theandroiddevicesendsthephotoandvideocontentswiththehighestresolutiontotheserver.Wewillanalyzewhataretheprosandconsontheresolutionandmakebuttonswhichuserscanchoosetheresolution.