DELIVERABLE...o DNG o FFF o MEF o NEF o ORF o PEF o PSD o RAF o TIFF o TIF o RAW o RW2 FotoInMotion...

FotoInMotion D3.1 - Final

H2020 Contract No. 780612 v1.0, 30/09/2018

Page | 1 Dissemination level: PU - Public

Project Acronym: FotoInMotion

Grant Agreement number: 780612 (H2020-ICT-20-2017-1, RIA)

Project Full Title: Repurposing and enriching images for immersive storytelling

through smart digital tools

Project Coordinator: INTRASOFT International

DELIVERABLE

D3.1 - 2D image processing requirements and mechanisms

Dissemination level PU- Public

Type of Document Report

Contractual date of delivery M9 – 30/09/2018

Deliverable Leader QdepQ Systems BV

Status - version, date V1.0, 28/09/2018

WP / Task responsible WP 3

Keywords: Image processing, post production, toolbox,

architecture, API

Ref. Ares(2018)5554358 - 30/10/2018


H2020 Contract No. 780612 v1.0, 30/09/2018


Executive Summary This deliverable will contain the analysis of the requirements in terms of 2D image processing

and design specifications for the first demonstrator of the FotoInMotion application.

These technical requirements are derived from the functional requirements (as described in

D1.2). We also briefly describe the processes and methodology that we will use for our

services as guidelines for integration.

Only a preliminary version of our API is included in this document, as it will be a living

document that updates during implementation of functionality. To keep the integration

smooth and agile, the API will be designed in consultation with WP4.


H2020 Contract No. 780612 v1.0, 30/09/2018


Deliverable Leader: QDEPQ

Contributors: QDEPQ, ATC, NOOR

Reviewers: INESC, NOOR

Approved by: All Partners

Document History

Version Date Contributor(s) Description

0.1 14/09/2018 QDEPQ Initial Draft version

0.2 21/09/2018 NOOR, ATC Added input

0.3 25/09/2018 QDEPQ Feedback on first draft

1.0 28/09/2018 QDEPQ Final version


H2020 Contract No. 780612 v1.0, 30/09/2018


Table of Contents EXECUTIVE SUMMARY ..................................................................................................................................... 2

DEFINITIONS, ACRONYMS AND ABBREVIATIONS ............................................................................................. 6

1 INTRODUCTION ...................................................................................................................................... 7

PURPOSE OF THIS DOCUMENT AND RELATIONS WITH OTHER WPS .......................................................................... 7 DOCUMENT STRUCTURE ................................................................................................................................. 7

2 2D IMAGE PROCESSING .......................................................................................................................... 8

INPUT/OUTPUT ............................................................................................................................................ 8 File Format ...................................................................................................................................... 8 Debayering ...................................................................................................................................... 9 White balancing .............................................................................................................................. 9

COLOR MANIPULATION .................................................................................................................................. 9 Color Balance .................................................................................................................................. 9 Brightness/Contrast ........................................................................................................................ 9

GEOMETRY .................................................................................................................................................. 9 Cropping .......................................................................................................................................... 9 Flipping ........................................................................................................................................... 9 Rotation .......................................................................................................................................... 9

IMPLEMENTATION ....................................................................................................................................... 10 Image manipulation workflow ...................................................................................................... 10 Data storage policy ....................................................................................................................... 11 Tools .............................................................................................................................................. 11

3 TIMELINE MANIPULATION .................................................................................................................... 12

TIMELINE SPECIFICATION ............................................................................................................................... 12 EVENTS ..................................................................................................................................................... 12

Image events ................................................................................................................................. 13 Viewport events ............................................................................................................................ 13 Sound events ................................................................................................................................. 13

TRANSITION EFFECTS .................................................................................................................................... 13 Fade in/out .................................................................................................................................... 13 Slide in/out .................................................................................................................................... 13 Curl effect ...................................................................................................................................... 13

2D NARRATIVE EFFECTS ............................................................................................................................... 14 Panning ......................................................................................................................................... 14 Zooming ........................................................................................................................................ 14 Rotation ........................................................................................................................................ 14 Special Effects ............................................................................................................................... 14

4 SOUND PROCESSING............................................................................................................................. 15

INPUT / OUTPUT ......................................................................................................................................... 15 TIMELINE ................................................................................................................................................... 15 EFFECTS ..................................................................................................................................................... 15

Volume .......................................................................................................................................... 15 Compression.................................................................................................................................. 15 Fade in/out .................................................................................................................................... 15

5 RENDERING .......................................................................................................................................... 16


H2020 Contract No. 780612 v1.0, 30/09/2018


TRANSPORT ................................................................................................................................................ 16 PREVIEW ................................................................................................................................................... 17 RENDERING ................................................................................................................................................ 17 OUTPUT / PUBLISHING ................................................................................................................................. 17

6 CONCLUSION ........................................................................................................................................ 18

7 APPENDIX: PRELIMINARY API DOCUMENTATION ................................................................................. 19

INTRODUCTION ........................................................................................................................................... 19 Scope ............................................................................................................................................. 19

DATA STRUCTURES....................................................................................................................................... 19 RAW IMAGE IMPORTATION ............................................................................................................................ 19

Image reading ............................................................................................................................... 19 RAW IMAGE CONVERSION ............................................................................................................................. 20

User feedback ............................................................................................................................... 20 White balance ............................................................................................................................... 20 Brightness ..................................................................................................................................... 21 Contrast ........................................................................................................................................ 21 Loading ......................................................................................................................................... 21

STANDARD IMAGE IMPORTATION .................................................................................................................... 21 COLOR MANIPULATION ................................................................................................................................. 22 GEOMETRY MANIPULATION ........................................................................................................................... 22


H2020 Contract No. 780612 v1.0, 30/09/2018


Definitions, Acronyms and Abbreviations Acronym Title

API Application Programming Interface

HTTP Hypertext Transfer Protocol

SSL Secure Sockets Layer

FIM FotoInMotion

Term Definition

Beneficiary EC term used to designate the legal entity which has signed the

Grant Agreement. This term is often substituted by the common

language term ‘partner’.

RESTful API A RESTful API is an application program interface (API) that

uses HTTP requests to GET, PUT, POST and DELETE data. A RESTful API -

- also referred to as a RESTful web service -- is based on representational

state transfer (REST) technology, an architectural style and approach to

communications often used in web services development.

Consortium Group of beneficiaries that have signed the Consortium Agreement

and the Grant Agreement (either directly as Coordinator or

by accession through the Form A).

Deliverable

Leader

Responsible for ensuring that the content of the deliverable meets the

required expectations, both from a contractual point of view and in

terms of usage within the project. Is also responsible for ensuring that

the deliverable follows the deliverable process and is delivered on time.

https://searchmicroservices.techtarget.com/definition/application-program-interface-API

https://searchwindevelopment.techtarget.com/definition/HTTP

https://searchmicroservices.techtarget.com/definition/REST-representational-state-transfer

https://searchmicroservices.techtarget.com/definition/Web-services-application-services


H2020 Contract No. 780612 v1.0, 30/09/2018


1 Introduction Purpose of this Document and Relations with other WPs

The purpose of this document is to specify the 2D functionalities of the FotoInMotion

application. This includes basic image manipulation: image decoding, color manipulation,

geometry manipulation. 2D narrative effects and the core tools needed to enable the creation

of a FIM video, including timeline and soundtrack edition, up two the rendering of the final

video.

This document summarizes the required functionalities and gives guidelines towards their

implementation and integration inside the FIM application.

The image manipulation part has no direct links to other work packages. The integration of

this system in the FIM application is done in cooperation with WP4.

Document Structure This document is structured as follows:

The second section is dedicated to image manipulation functionalities and file decoding.

Next in the third section the timeline manipulation functionalities are described as well as the

first set of 2D narrative effects.

Section 4 describes the sound manipulation tools that will be proposed and finally the last

section describes the different rendering and output options.

The functionalities described in the document will be implemented in the first version of the

application in WP3 (tools) and WP4 (integration).


H2020 Contract No. 780612 v1.0, 30/09/2018


2 2D Image Processing

Input/Output The first step of the creation of a FIM video shall be the selection of an existing photo.

Oriented towards professionals, the application shall be able to read standard file formats as

well as raw camera file formats. For the later the application provides a basic set of tools for

loading and converting the file.

File Format

The application will be able to handle both raw and non-raw formats. A set of conversion tools

will be provided for the later. When a raw image is loaded, the raw image data will be used

for 2D to 3D conversion in the later phases of the project (task 3.2).

Here is a minimal list of file formats that must be handled by the application.

• Standard image file, those files are loaded directly. Only a minimal set of editing tools

is available:

o JPEG

o TIFF

o PNG

o BMP

• Raw file formats. Raw formats require some conversion prior to be loaded. For this,

some additional tools will be provided (see under):

o 3FR

o ARW

o CRW

o CR2

o CR3

o CAP

o IIQ

o DNG

o FFF

o MEF

o NEF

o ORF

o PEF

o PSD

o RAF

o TIFF

o TIF

o RAW

o RW2


H2020 Contract No. 780612 v1.0, 30/09/2018


Debayering

The first step of raw image conversion is the reconstruction of the image colors from the red-

green-blue pixel matrix from the sensors. The application shall support commonly available

filter configurations.

White balancing

The application will provide a color correction tool that will allow the user to select either

camera white balance either from a set of standard white balance and finally an interactive

tool to manually set the white balance from an area in the image.

Color Manipulation These tools are available for both raw and standard image formats.

Color Balance

Manipulation of color in the RGB and HSV (HSL) color space. For raw files this affect directly

the transfer curve during image conversion.

Brightness/Contrast

For raw files this affect directly the transfer curve during image conversion.

Geometry These tools are available for both raw and standard image formats.

Cropping

Interactively reframe the image. For raw files this affect directly the transfer curve during

image conversion.

Flipping

Flip the image left/right or upside/down. For raw files this affect directly the transfer curve

during image conversion.

Rotation

Rotate the image per 90/180/270 degrees.


H2020 Contract No. 780612 v1.0, 30/09/2018


Implementation

Image manipulation workflow

All image processing will be executed on the QdepQ server cluster. The workflow for any of

the operations listed is quite easy:

1. The user uploads its image, the system returns a unique ID.

2. The user sends the preferred operation with parameters and the image ID, the

systems acknowledges.

3. The user calls a render operation with parameters of the quality of the render (e.g. for

preview purposes or as a final full resolution render), the system executes the

processing and returns a link to the result.

During step 2 of the process, there is a possibility of sending multiple image operations to the

system that will be stored in an instruction list on the server. When the preview or full render

is requested, this list will be executed all together.

Figure 1: Image manipulation workflow


H2020 Contract No. 780612 v1.0, 30/09/2018


Data storage policy

All data on the QdepQ server cluster will be temporarily stored. To ensure privacy, every

datafile will be stored under a unique identifier, making an orphaned file untraceable to a

user.

The duration of a file being stored for processing will depend on a server-side setting.

Depending on storage space available, legal requirements and processing status of an image,

the storage time may be minutes or days. The intention is to keep the final rendering time of

an image under 1 minute, so unless a project work-in-progress, the storage of files should be

minimum.

During development the storage will be set to a longer time period for analysis of the system

and the results.

Tools

As a part of the Image Processing & Narration Toolbox (IPNT), 2D image processing

functionalities will be presented via a set of RESTful services. Internally, the raw file

manipulation tools will be provided by the libRaw library around which an interface will be

built to match the specifications of the RESTful services.


H2020 Contract No. 780612 v1.0, 30/09/2018


3 Timeline manipulation These are the basic tools to build a FIM video from the decoded photography. The timeline

describes the different events that occur during the video and how they are organized over

the duration of the video.

Timeline specification The first version of the application will focus on the process of creating a video from single

photography. Events are the main concept used for the specification of the timeline: A

photography being selected for the construction of the video, the user creates the

FotoInMotion by adding event to the timeline. Events will affect either the way the

photography is displayed, either the associated soundtrack. Events have an effect for a limited

duration of time, from zero to the entire length of the video, that is to be specified by the

user.

Once an event has ended, its effect remains, this means that events are mainly used to specify

transitions from one state to another (e.g. zooming in over a period of time then remaining

focused on the zoomed in area).

There is no limitation to how many events can occur simultaneously, however, some events

might conflict with each other. Specific rules then apply that will be specified on a per case

basis.

The timeline is the conjunction of an image and a list of events, to be triggered at specific

moments, that will affect the way the photography shall be displayed. As well as the

mechanisms to trigger the actions described by the events and strategies to solve

inconsistencies between effects.

Creating a FotoInMotion video is therefore the task of adding events to a timeline.

Events Events introduce an effect in the video over a certain period of time, itself resulting in a

change in the final rendering. Events affect either the image itself, either the camera either

the sound.

Therefor events have a target. A first rule to apply to avoid conflicts is to prevent events

having the same target to happen at the same time.

Events can also be linked together so as to create more complex effects (eg. link a fade out

with a fade in).

Events affecting the 3D map will be added later, however the general model remains.


H2020 Contract No. 780612 v1.0, 30/09/2018


Image events

Image events affect the pixel of the image itself (but not the geometry of the image). Image

events, as part of the conflict avoidance rules, are divided into further subcategories, such as

color and texture events.

Viewport events

These event describe the general motion of the camera for rendering. In this first version,

only 2D motions are considered. The effects triggered by viewport events are part of the

narrative effects toolbox described below.

There is no limitation into combining viewport events together as long as they have different

targets.

Sound events

These effects trigger effects that target the soundtrack of the video (see below). Only one

sound vent can be triggered at the same time but they have no effect on other events.

Transition effects Transition effects are duration effects that reveal the image in a gradual way. These effects

are initially made for a single image, but if there is a demand for multiple images, this

functionality will be added for a later demonstrator. Like other effects, transitions can be

combined

Fade in/out

A fade in/out effect on an image will be a linear gradual increase/decrease of the image

opacity in respect to a certain background color.

Properties of the fade will be the duration and the direction (in/out) as well as the color of

the background.

Slide in/out

The slide in/out effect makes use of a change in panning to reveal / hide the image.

The properties of the slide effect are a destination (slide-out) / source (slide-in) of the image,

as well as the partition of the image that will be in-/outside the viewport and the duration of

the event.

Curl effect

This transition is a special effect that curls over an image and folds it out of the viewport,

revealing the underlying background.


H2020 Contract No. 780612 v1.0, 30/09/2018


2D Narrative Effects The first version of the application will provide a basic set of narrative effects. They are mainly

camera motion effect that will allow to move a virtually camera over the photography. The

first demonstrator(D3.3) is limited to basic narration tools, though in the second

demonstrator(D3.4) the annotation data of WP2 will be used to suggest narrative effects.

Panning

Defines the translations of the camera in a plane parallel to the image.As a part of an event,

this effect is defined over a period by a vector defining the displacement of the camera during

this period. The speed of the camera remains constant over this period. This effect can be

used in conjunction with zooming or rotation.

Zooming

Zooms over the image in and out. It is specified over a duration of time and by a single number

as zoom factor. The speed of the transformation remains constant. Alone, it magnifies the

center of the field of view. It can be used in conjunction with Panning and Rotation.

Rotation

Rotates the image by a given angle. The effect is specified over a duration and the speed of

rotation remains constant over that period. When used alone, the center of rotation can be

set interactively. Used in conjunction with panning or zooming, the center of rotation is the

center of the field of view.

Special Effects

Some general purpose effects will be proposed in this first version. These are part of the

rendering pipeline and therefore affect the final image after all other effects have been

applied.

The first set of special effect includes:

Motion blur, 2D motion blur the images is affected by a 2D blur along a direction specified by

the motion of the camera.

Spotlight, the image is darkened except for a circular area defined by the user and,

automatically tracked when the camera moves.

Selective blur, simulate depth of field effects by selectively blurring areas of the image.

Intelligent tools will be provided to helps select areas to be blurred of not. The blur can be

applied progressively for a given time period.


H2020 Contract No. 780612 v1.0, 30/09/2018


4 Sound Processing This section describes the sound manipulation functionalities of the software. The application

provides a basic set of tools that allows to build the soundtrack of the FIM video.

Input / Output The tool shall be able to read commonly available sound files such as wav, mp3, ogg, etc….

The volume of each sound file is normalized to the same range at load time. Sound file

reading and decoding capabilities will be provided via ffmpeg and made available via a RESTful

interface.

Sound output will be provided by a master bus which outputs the mix of all video files audio

effects can be applied to either audio sample or the master output bus

Timeline In a way similar to video effects, sound effects are placed on the timeline and work in a similar

way. However, it is possible to load successively or simultaneously several sound files and to

play the simultaneously.

Effects Sound effects either act on a specific audio sample either on the master bus.

Volume

Set the volume of a sound sample. The effect is applied immediately. This effect cannot be

used while a fade in or out is playing.

Compression

Dynamics compression, low volume sounds are amplified while high volume ones are not.

This effect is effective over an entire sample (or the whole video when applied on the master

bus).

Fade in/out

Bring in/out a sound sample by progressively changing its volume until the desired setting, or

zero, is reached. The final (or start) volume is set using the volume effect while it is

progressively brought in/out over a user defined duration at beginning, resps. end, of the

sample.


H2020 Contract No. 780612 v1.0, 30/09/2018


5 Rendering This paragraph describes the functionalities of the rendering pipeline whose role is to process

the selected media (image and audio samples) to produce the final video. The rendering

pipeline is double: first oriented towards the user interface to allow the user to preview its

works and a final rendering pipeline to a video file or media platform.

Transport HTML5 video streaming enables the manipulation of multimedia content using the <video>

HTML tag. Using the <video> tag creates a native HTML5 video player, and allows additional

options to customize and display the content. The standard <video> tag provides the

functionality for displaying and hiding video controls including: play/pause, volume

adjustment, full screen toggle as well as seek slider. Either a complete video file or an ever-

updating video stream as a source can both be supported. Technically speaking, in case of a

video stream, the HTML5 specification is agnostic of which protocols, transports, or codecs

are used with HTML5 video. Though, the combinations supported by most browsers are

described below:

HLSi:

HLS is h264-compressed video with aacii- or mp3-compressed audio, transported with MPEG-

TS. The stream is divided into chunks described in m3u8iii playlists and is transmitted over

HTTP.

MPEG-DASHiv:

MPEG-DASH typically is h264/h265-compressed video with aac audio transported with mp4,

or vp8/vp9v transported with WebMvi. The standard however is not bound to any particular

codecs, protocols, or transports. As with HLS, the stream can be divided into chunks but it’s

optional in this case.

i https://en.wikipedia.org/wiki/HTTP_Live_Streaming ii https://en.wikipedia.org/wiki/Advanced_Audio_Coding iii https://en.wikipedia.org/wiki/M3U iv https://en.wikipedia.org/wiki/Dynamic_Adaptive_Streaming_over_HTTP v https://en.wikipedia.org/wiki/VP9 vi https://en.wikipedia.org/wiki/WebM

https://en.wikipedia.org/wiki/HTTP_Live_Streaming

https://en.wikipedia.org/wiki/Advanced_Audio_Coding

https://en.wikipedia.org/wiki/M3U

https://en.wikipedia.org/wiki/Dynamic_Adaptive_Streaming_over_HTTP

https://en.wikipedia.org/wiki/VP9

https://en.wikipedia.org/wiki/WebM


H2020 Contract No. 780612 v1.0, 30/09/2018


Preview The preview pipeline is intended to provide the user with a real time impression of a work in

progress. Therefore performance is crucial for the preview pipeline and only a downscaled

version of the final video is processed. Further, frame rate can also be reduced if necessary.

The rendered video and audio streams are then sent to an ffmpeg server that encodes and

streams it to the client. The output format and codec for the preview pipeline are yet to be

defined depending on the requirements of te video player component.

Rendering The final video is finally rendered at its full size and resolution, or a a resolution constrained

by the chosen output format or platform. Ffmpeg will be the tool of choice for this task and a

selection of the available output formats and options will be proposed. Calls to ffmpeg shall

be made via an API providing RESTful services.

Output / Publishing The final rendered video will temporary be stored on the QdepQ server clusters and the user

gets notified with a download link via a secured connection (SSL). Once the user downloads

the video and concludes the project, it is immediately deleted from the server (this setting

may be overruled during development to acquire data on performance).

Alternatively, the results can directly be pushed to an online platform. The application shall

provide at least access to the most important ones (Youtube, Dailymotion, Vimeo). Also, the

user shall be able to provide a custom server where the final video is to be sent. The work can

then be published automatically once the rendering done with no further user intervention,

even if the user is not connected.

In any case, the rendered video will not be kept on the server longer that the time necessary

for the user to download or publish it.


H2020 Contract No. 780612 v1.0, 30/09/2018


6 Conclusion This document summarizes the basic functionalities of the first version of the IPNT service,

with a focus on 2D image processing. 2D functionalities include basic image preprocessing

and raw file manipulation functions. Definition of a timeline upon which narrative effects can

be arranged and the definition of a basic set of narrative effects. And finally, the rendering

pipelines and publishing functionalities.

Together these features will form the backbone of the first version of the IPNT service to be

implemented in the first viable version of the application (D3.3)


H2020 Contract No. 780612 v1.0, 30/09/2018


7 Appendix: Preliminary API documentation Introduction

The 2D API of the FotoInMotion application is intended to provide the necessary functionalities for

opening and pre-processing image files. In the case of raw files, this includes the conversion of the raw

image to standard 8 bits images for rendering and the manipulation of the conversion parameters. For

standard images, basic color and image geometry manipulation is also provided.

These functionalities are accessible via actions the user interface that trigger processing on the server

side. Each of them are therefore specified from the front end and the back end side.

In this appendix we focus on the image processing functionalities and leave aside the effects based

around the time line, as specification of the later is out of the scope of this deliverable.

Scope

This document a preliminary work that intends to clarify the basic needs in terms of functionalities as

well as the general mechanisms underlying the client/server communication and the feedback to the

user. It also enlightens the basic data structures that will be needed to perform these tasks. These

specifications are subject to constant evolution in accordance with the Agile methodology used in the

project (see D4.1).

Data structures In terms of image data, the application relies on three main image data classes:

• RawImage

a RawImage class instance is used to manipulate the raw image conversion tasks. It contains

the raw sensor data, EXIF data and a thumbnail 8 bits image. Instance of RawImage only

exist on the back-end side.

• FIMImage

Internal image data class, this class embeds the converted (or read) 8 bits image data at full

resolution. Floating point image data consisting of either the debayered raw image data or the

floating-point conversion of the 8 bits data if the former is not available. A low resolution 8

bits thumbnail image. Instances of a FIMImage only exist on the back en side.

• FIMThumbnail

This is the class for providing feedback to the user interface. It consists of a downscaled

version of the image in its 8 bits version. Manipulations to either RawImage conversion

parameters or to FIMImage data are applied correspondingly on the corresponding

FIMThumbnail instance. FIMThumbnail instances are mirrored automatically on the front-

end side when modified on the back end.

Raw image importation

Image reading

These functions allow the importation of raw images and their conversion to standard 8 bits images, as

well as the manipulation of the conversion parameters. The image is read on the front-end device and

sent in a binary format to the back end server.


H2020 Contract No. 780612 v1.0, 30/09/2018


Raw image manipulations are widely based on the libRaw libraryvii and functionalities described her

are essentially an abstraction of the libRaw API tailored to the needs of the FIM application.

• FIM_Front_ReadRawImage(String path)

Opens the image file in path, establish a connect to the back end server and sends a stream of

binary data from the file.

• FIM_Back_ReadRawImage(InputStream stream, out RawImage img)

Read the data sent by FIM_Front_ReadRawImage and build a RawImage instance that

gathers the EXIF metadata, Raw image data, conversion parameters as well as a the data for a

thumbnail image used to provide feedback to the user (see 7.4.1) .

Raw image conversion Once the raw data has been made available, it is possible to manipulate the conversion parameters.

Depending on the limitations of the libRaw, not all parameters might be available for all kind of

cameras.

User feedback

In order to provide feedback to the user, each function call leads to an update of the thumbnail image.

That in return will be sent back to the front end. The thumbnail image is a preview of the imported

image, converted to 8 bits with the current settings and downscaled so that to insure smooth interaction.

These functionalities are handled internally so that each call to a front-end function will trigger an

update of the preview image in the UI. For these purpose two functions are made available:

• FIM_Front_UpdatePreview()

Make a request to the back end to send the up-to-date thumbnail image.

• FIM_Back_SendPreview()

Update thumbnail image with the current conversion parameters and sends it back to the front

end.

White balance

• FIM_Front_RawImageSetWhiteBalancePreset(int preset)

Send a flag corresponding to a white balancing preset to the back end.

• FIM_Back_RawImageSetWhiteBalancePreset(int preset)

Set image white balance to a predefined value (Camera, Automatic, …). The exact set is

depending of the camera and defind by the libRaw interface (see:

https://www.libraw.org/docs/API-datastruct.html#libraw_output_params_t).

• FIM_Front_RawImageSetWhiteBalancePoint(point2D point)

Send the coordinates of a user selected point to set white balance parameters to the back end.

• FIM_Back_RawImageSetWhiteBalancePoint(point2D point)

Compute white balance parameters in a small window around the designated point.

vii https://www.libraw.org/

https://www.libraw.org/docs/API-datastruct.html#libraw_output_params_t)int


H2020 Contract No. 780612 v1.0, 30/09/2018


• FIM_Front_RawImageSetWhiteBalanceTemperature(int temp)

Send a user defined white balance temperature value to the back end.

• FIM_Back_RawImageSetWhiteBalanceTemperature(int temp)

Compute white balance parameters according to a given temperature setting.

Brightness

• FIM_Front_RawImageSetBrightness(float brightness)

Send a user defined brightness value to the back end. Brightness is a floating-point value

within [0,1].

• FIM_Back_RawImageSetBrightness(float brightness)

Set the brightness setting to the given value.

Contrast

• FIM_Front_RawImageSetContrast(float contrast)

Send a user defined contrast value to the back end. Contrast is a floating point value within

[0,1].

• FIM_Back_RawImageSetcontrast(float contrast)

Set the contrast setting to the given value.

Loading

• FIM_Front_ConvertRawImage()

Trigger the conversion of the full resolution image on the back end.

• FIM_Back_ConvertRawImage(out FIMImage img)

Apply the conversion parameters to the whole RawImage and produce a FIMImage instance.

FIMImage is the internal image format used for further processing. It embeeds te converted 8

bits data as an openCV image, the raw data converted to floating point that will later be used

for 3D conversion as well as thumbnail images for user feedback.

Standard image importation The mechanisms for reading a standard 8 bits image are similar to the ones for raw images. The main

difference being that openCV will be the background library of choice for those files. Also there is no

need for conversion and related functions. The functions for reading standard images are:

• FIM_Front_ReadImage(String path)

Opens the image file in path, establish a connect to the back end server and sends a stream of

binary data from the file.

• FIM_Back_ReadImage(InputStream stream, out FIMImage img)

Read the data sent by FIM_Front_ReadImage and directly build a FIMImage instance

where the raw data is a floating point version of the image data.


H2020 Contract No. 780612 v1.0, 30/09/2018


Color manipulation These operations apply to the 8 bits version of the image contained in a FIMImage and the thumbnail.

Preview mechanisms for user feedback are identical to the ones ones for setting the conversion

parmeters. The raw data is left untouched.

• FIM_Front_FIMImageSetColorBalance(float r,float g, float b)

Send color balance parameters to the back end. Color balance is defined by three

multiplication factors within [0,1].

• FIM_Back_FIMImageSetColorBalance(float r,float g, float b, in/out FIMImage img, )

Apply the color balance to the FIM image by multiplying each color channel by the

corresponding factor. Previously, these factors have been rescaled so to match the range of

values in the image from [0,1] to [0, min(img.r/img.g/img.b)/255].

• FIM_Front_FIMImageSetBrightness(float brightness)

Send a user defined Brightness value to the back end. Brightness is a floating-point value

within [0,1].

• FIM_Back_FIMImageSetBrightness(float brightness, in/out FIMImage img)

Apply the Brightness setting to the image

• FIM_Front_FIMImageSetContrast(float Contrast)

Send a user defined Contrast value to the back end. Contrast is a floating-point value within

[0,1].

• FIM_Back_SetContrast(float Contrast, in/out FIMImage img)

Apply the Contrast setting to the image.

Geometry manipulation The functions apply to the FIMImage instance and modify both the 8 bits and the raw image as well as

the thumbnail.

• FIM_Front_FIMImageCrop(Point2D topLeft, Point2D bottomRight)

Send the top left and bottom right coordinates of a cropping rectangle. The coordinates are

sent relative to the dimensions of the thumbnail image

• FIM_Back_FIMImageCrop(Point2D topLeft, Point2D bottomRight, in/out FIMImage img)

Compute the coordinates of the full size rectangle and crop the FIMImage instance

accordingly.

• FIM_Front_FIMImageRotateLeft()

• FIM_Front_FIMImageRotateRight()

Send to the back end a message for rotating the image by 90 degrees to the left or to the right.

• FIM_Back_FIMImageRotateLeft(in/out FIMImage img)

• FIM_Back_FIMImageRotateRight(in/out FIMImage img)

Rotate img 90 degrees to the left or to the right.

• FIM_Front_FIMImageFlipHorizontal()


H2020 Contract No. 780612 v1.0, 30/09/2018


• FIM_Front_FIMImageFilpVertical()

Send to the back end a message for flipping the image left/right or upside/down.

• FIM_Back_FIMImageFlipHorizontal(in/out FIMImage img)

• FIM_Back_FIMImageFilpVertical(in/out FIMImage img)

Flip img left/right or upside/down.

DELIVERABLE...o DNG o FFF o MEF o NEF o ORF o PEF o PSD o RAF o TIFF o TIF o RAW o RW2 FotoInMotion...

Documents

Transcript of DELIVERABLE...o DNG o FFF o MEF o NEF o ORF o PEF o PSD o RAF o TIFF o TIF o RAW o RW2 FotoInMotion...