Barcode application innovation for smartphones

Gerald Eichler*, Karl-Heinz Lüke

#, Aykan Aydin

*, Roland Schwaiger

#

Innovation Development, Convergent Services and Platforms

Deutsche Telekom AG, Laboratories *Deutsche-Telekom-Allee 7, D-64295 Darmstadt

#Ernst-Reuter-Platz 7, D-10587 Berlin

gerald.eichler@telekom.de

karl-heinz.lueke@telekom.de

aykan.aydin@telekom.de

roland.schwaiger@telekom.de

Abstract: Modern smartphones provide a high sophisticated camera and a high

resolution display. Therefore, they can be used to enable various application

scenarios for 1D and 2D barcode recognition as well as barcode presentation

device. The device positioning against the barcode allows innovative interaction:

e.g., additional product information or “rail information” on paper tickets. In the

future, the challenges lie on applications that are “easy to use” with one-click: e.g.,

book auctions with one key press. Beside usability approaches, the technical

challenges cover fast and robust algorithms under bad light conditions, different

angel orientation and parallax calculation.

1 Motivation – new technology of smartphone cameras

Over the recent years, smartphone cameras have reached the level of compact cameras.

Manual macro switch or additional lenses were replaced by autofocus systems while the

resolution increases. Vendor dependent APIs to access picture or video stream mode are

available and the device processor allows real-time calculations which are essential for

object detection even under bad light conditions (brightness, colors, reflexes).

(a)

Shop 1Shop 2 Shop 3

Shop 6Shop 4 Shop 5

Mall

Shop 1Shop 2 Shop 3

Shop 6Shop 4 Shop 5

Mall

(b)

Figure 1: (a) fixed orientation angle ⇒ activation of different functions on key press; (b) square

parallax calculation ⇒ detection of camera sight point out of the 2D barcode.

If the object type – in our case various barcode types – is known, object deficits like

knitter, scraper, bulge can either be corrected or used for additional interaction by

orientation detection. New challenges lie on the usability of new applications that are

easy to use by extending the limited smartphone interaction options.

2 Barcode application overview

Barcode applications for smartphones can be divided in to receptive (recognition) and

productive (generation) services (see fig. 2). Beside a static barcode, scan technology

that covers angle orientation and parallax detection, dynamic scan methods will become

more relevant for further approaches. If the device with the camera or the object is in

motion, the applications should be usable in an intuitive way. Dedicated device motion

can be used as interactive function enabler. Therefore, the object has to be recognized

very fast under bad light conditions, surface damages and different orientation relation

between barcode and camera.

Barcodes & Smartphones

Barcode Generation

Barcode Recognition

Static Scan Dynamic Scan

Angle Orientation Parallax Detection Device Motion Object Motion

Barcodes & Smartphones

Barcode Generation

Barcode Recognition

Static Scan Dynamic Scan

Angle Orientation Parallax Detection Device Motion Object Motion

Figure 2: Classification of smartphone barcode applications.

2.1 Barcode detection and database/URL linking

Principle: Barcodes include either a short numeric (mostly 1D code) or more extensive

alphanumeric (mostly 2D code) information. In most cases, only an additional public or

private database (numeric ID) or the Internet by URL coding (alphanumeric) leads to the

final data source, which requires an online data connection for further service invention.

Appropriate barcodes: 1D EAN, ISBN as EAN, 2D QR-code

Deutsche Telekom Laboratories application prototypes:

• “Allergy Warner” for super market products [Contigo IV]. The Allergy Warner

gives an alert message if a product contains special ingredients when scanning

the product barcode and having activated the personal allergy profile during

shopping.

• “ebay Book Auction” for books per one key press [Contigo IV]. This

application gives the user a one click solution for book auctions by checking the

ISBN barcode. The customers benefit from the high usability.

• “Movie Information” and cinema trailer delivery [Contigo IV]. At special

locations in a city, e.g. bus tops, additional advertising information to movies,

can easily be added on posters. The barcode scan

triggers a trailer preview and leads to the nearest

cinema.

• “Additional rail information” on paper tickets

[LME+09]. The scan of barcode on paper tickets can

improve the allocation of real-time travel information

(e.g. delays, connections and platform information).

• QR-code for news and advertisement with direct URL coding [SSM+08]. Print

media are linked to web resources for further details and background

information.

2.2 Barcodes 2D with exact shape and orientation detection

Principle: Analysis of square parallax of a known barcode shape allows calculating the

exact point the camera center is aimed at.

Appropriate barcodes: square codes: 11x11 code, QR-code

Deutsche Telekom Laboratories application prototypes:

• “Mall Navigator” or “Interactive Fashion Catalogue”

with indirect object focusing [Contigo II]. Multiple

additional real-time shop (opening hours) or product

(sizes on stock) information bundled one page can

improve the user experiences.

• “Function Activation” by angle scan [Contigo III]. Scan and click initiates

different action e.g., show details, mail me, print info.

2.3 Location based barcode generation on mobile screens

Principle: Using location information as trigger of additional

services and direct reading from displays by scanner. In

specific situations, e.g. traveling or shopping, the customer

can receive a dynamic barcode coupon depending on his local

position or context.

Appropriate barcodes: EAN, QR-code

Deutsche Telekom Laboratories application prototypes:

Kaufen M

ehr

Kaufen M

ehr

• “Mobile Couponing” for location based discount campaigns when entering a

defined area [BEH07, EK06]. At selected places, like airports or stations, the

customer can get special mobile coupons that can be recognized by a barcode

reader directly from the display of the mobile phone. Campaign, time, area and

profile management is supported.

• “Mobile Ticketing” for ticket verification [LME+09]. Barcodes at platforms can

improve the accuracy of the GSM route tracing. The location-based mobile

ticketing solution Ring&Ride can be extended by using bar codes with platform

information. In comparison to ticketing solutions that uses NFC (Near filed

communication), cameras in mobile phones are much more widely spread on

the market than NFC-chip.

• “Context aware Adaptive Barcodes” generation for refunding. Context

information that are used on mobile phones, e.g. location information and

calendar data, can be used as a significantly improvement of the user

experience. Depending on the context, the user can receive a special barcode

which is useful in his current situation e.g., coupons or additional information.

3 Summary and Outlook

Barcode supported applications on smartphones are a powerful enabler for efficient user

interaction, although generic, device independent APIs to access the camera are still rare.

One click applications with a high usability will be the challenges for further work.

Beside the aspect of usability, new technological challenges cover robust and fast

algorithms with different angle orientation even under bad light conditions.

References

[BEH07] Böhm, A.; Eichler, G.; Haffner, M.: Location-based Couponing mit MMS Barcode.

Erfahrungen per Feldversuch, BDOA: „Bonus-/Gutscheinsysteme und Partnering”,

Hamburg, 2007, URL: http://www.bdoa.de/download/BDOA_TSY_1.1.pdf

[Contigo II], [Contigo III], [Contigo IV] Contextual Information to Go (Contigo) is a series of

innovation projects of Deutsche Telekom Laboratories in the project field Convergent

Services and Platforms, Berlin, Bonn, Darmstadt, 2001 – 2008

[EK06] Eichler, G.; Karge, R.: A Location and Privacy Middleware for Context-aware Mobile

Applications, ICIN, Convergence in Services, Media and Networks, Bordeaux, 2006

[LME+09] Lüke, K-H.; Mügge, H.; Eisemann, M.; Telschow, A.: Integrated Solutions and

Services in Public Transport on Mobile Devices, In: Gesellschaft für Informatik (GI),

I2CS conference proceedings, Jena, 2009

[SSM+08] Sapira, B.; Shoval, P.; Meyer, J.; Tractinsky, N.; Mimram, D.: ePaper - the

Personalized Mobile Newspaper, ReColl 2008: International Workshop on

Recommendation and Collaboration 2008 at International ACM Conference on

Intelligent User Interfaces (IUI 2008), 2008