Augmented Reality Retail Environment

8/14/2019 Augmented Reality Retail Environment

1/14


2/14


3/14

11. The system of claim 1, the monitoring component, the recommendation-makingcomponent, and the broadcast device are preservable, programmable, and transferable to

other users in need of assistance.

12. A method that facilitates recommendation-making, comprising the following acts:

identifying a product via a user request; analyzing the observations of the user withpreferences unique to the user; determining an action from the analysis that addresses the

issue; automatically implementing the action; and transmitting information regarding the

implemented action to other service providers as may be identified by the user.

12. The method of claim 12, further comprising: encrypting the information when

communicated beyond the user; and verifying the authentication and authorization

properties of recipients before communicating the information to the recipients.

13. A system that facilitates recommendation-making, comprising: means for

concurrently observing multiple aspects of product information; means for filtering outirrelevant information from the information services; means for analyzing relevant

information from the information services; and means for creating a suggestion based on

the analysis.

14. The system of claim 13, further comprising: means for applying behavior economics

in improving subsequent suggestions and visual information; and means for applying data

mining from prior suggestions when formulating the items to be purchased. In this systema receipt documents the evaluation of multiple product decisions that end up in a

shopping basket. Each selection can in effect be a POS entry upon validation at the POS

of items for close out of payment. In this manner the receipt tracks the decisions andrational for the recommendations. The person comes away with both the products and

the rational for the advice.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is a continuation-in-part of [M.I.T. Case No. 13143 "Personal

Healthcare Record (pHR) in an nfc mobile phone"] the entirety of which is incorporated

herein by reference as an example of how this invention can be applied to a specificindustry application in health care data exchange at the Point of Sale.

BACKGROUND

[0002] Individuals interact with retail environments in various manners. For any given

person, unique circumstances that may depend on social and market norms of a particularsituation can affect his/her behavior. For example, in some settings, such as a unique


4/14


5/14

preferences and characteristics, and available information resources.

[0006]The A.R.E. can continuously monitor a variety of factors in a user's environment,including aspects related to: location, time, local commerce/retail structures, temperature,

physical characteristics, application/relevancy/usage, and Internet/email configuration.

Mechanical and/or electromagnetic devices may be used to monitor a particularenvironment.. The A.R.E. combines such environmental factors with past, present, and/or

future preferences as well as attributes of the user to suggest options in a decision-making

process. A behavior economics based analysis can be performed in connection withmaking an automated recommendation as a function of expected benefit versus cost of

making an incorrect decision.

[0007]Any of the above information can be shared or held private as desired by the user.Shared information can be especially useful when applied to building ongoing service

relationships. Information sharing can be based on trust and/or restricted to a specific

window of time. In addition, a user may be hesitant to share his information to unfamiliar

services and limit such exposure accordingly. The A.R.E. can also evaluate the reputationof those services to assist the user in a determination of whether or not he wants to

interact with a particular service.

[0008]Furthermore, the A.R.E. can manage multiple personas for a given user. In an

example, a user may want to have a default or core persona for work, but maintain

separate personas for school, extracurricular activities, and personal life. Each personamay have distinct preferences, security defaults, subscriptions, memory settings, etc.

[0009]To the accomplishment of the foregoing and related ends, certain illustrativeaspects are described herein in connection with the following description and the annexed

drawings. These aspects are indicative, however, of but a few of the various ways in

which the principles of the claimed subject matter may be employed, and such subjectmatter is intended to include all such aspects and their equivalents. Other advantages and

novel features may become apparent from the following detailed description when

considered in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0010]FIG. 1 is a block diagram of an augmented reality evaluation system (A.R.E.).

DETAILED DESCRIPTION

[0011]The claimed subject matter is now described with reference to the drawing,

wherein like reference numerals are used to refer to like elements throughout. In thefollowing description, for purposes of explanation, numerous specific details are set forth

in order to provide a thorough understanding of the claimed subject matter. It may be

evident, however, that such subject matter may be practiced without these specific details.

In other instances, well-known structures and devices are shown in block diagram form in


6/14

order to facilitate describing the claimed subject matter.

[0012]As used in this application, the terms "component" and "system" are intended torefer to a computer-related entity, either hardware, a combination of hardware and

software, software, or software in execution. For example, a component may be, but is

not limited to being, a process running on a processor, a processor, an object, anexecutable, a thread of execution, a program, and/or a computer. Transponder (short-for

Transmitter-responder) refers to an automatic device that transmits a predetermined

message in response to a predefined received signal., in the case of an NFC phone,identifying the source of the communication. Interrogator refers to an automatic device

that transmits a predetermined message in response to a predefined received signal device

and which, in the case of NFC, may be on the same platform

[0013]FIG. 1 is a block diagram of an Augmented Reality Environment evaluation

system (A.R.E.) that provides suggestions and assistance in decision-making processes

for an individual. The A.R.E. 140 receives and analyzes information with respect to a

surrounding environment in the context of unique characteristics of a user and offers oneor more suggestions regarding the product at hand. The A.R.E. 140 comprises a

monitoring component 110 that collects and filters the information, and arecommendation-making component 120 that evaluates relevant environment information

in view of a user-attribute store 130 of preferences specific to the user. The evaluation

can result in a suggestion or automatic action that provides assistance, implements a

recommendation, or suggests a solution to the issue in question. The A.R.E. is not limitedto a particular device, area of information, or monitoring scheme, but rather is a flexible

and customizable tool to help regulate, monitor, and advise a user in the context of his/her

environment.

[0014]The A.R.E. 140 can detect a wide variety of states and extrinsic information, such

as, for example, location, time of day, and person or product identification . Not onlydoes the A.R.E. 140 monitor an environment, but also a user (e.g., physical

characteristics, etc.) to visualize how products will look and feel.. A variety of features of

a A.R.E. may be controlled by the user, such that the user can influence, set preferences,automate, script, upgrade, allow/deny permissions and turn on/off one or more aspects of

the A.R.E. at any time as he/she chooses.

[0015]The automated data acquisition device, may include a combination of AutoIdentification technologies such as NFC/GenII/2D barcode reader components 110

receives a product identifier (e.g. a product type, time of day, location, etc.), as well as

the identity of a user in the retail environment (e.g., a member of customer loyaltyprogram, etc.). A user may restrict the monitoring component 110 to observations that are

specifically permitted in advance by the user. The monitoring component 110 sends

relevant information for current or future recommendations to the recommendation-making component 120 that analyzes the information within the context of personal

preference data stored in the user-attribute store 130 in order to make a suggestion or

implement a recommendation. Such recommendation is made consistent with helping the

user, as well as avoiding situations where the user would be harmed. The
http://en.wikipedia.org/wiki/Messagehttp://en.wikipedia.org/wiki/Responsehttp://en.wikipedia.org/wiki/Messagehttp://en.wikipedia.org/wiki/Response


7/14

recommendation-making component 120 stores potentially relevant information for

future use in the user-attribute store 130. For example, the recommendation-making

component 120 can receive environment information regarding time of day, the locationof the user, and analyzes such information in connection with preferences saved in the

user-attribute store 130 setting forth the user's schedule and favored regimen. The

recommendation-making component 120 can suggest a list to the user or automaticallymake electronic purchases or prescriptions at a selected retailer. The recommendation-

making component 120 may concurrently manage multiple suggestions and actions that

span a wide range of services.

[0016]The user-attribute store 130 is a comprehensive container of personalized data

regarding a user. The user-attribute store 130 can be configured and updated directly by a

user as an explicit process and continually updated with information as received by therecommendation-making component 120 as an implicit process. In addition, the user-

attribute store 130 can be updated with information received or implied from

circumstances after a decision is made. For instance, the user can initially load and return

to update his/her schedule and list of preferred dry cleaning services, retailers, health careproviders etc. as well as corresponding contact phone numbers, addresses, menus, and

ratings. As the user travels through a new environment, the user-attribute store 130 iscontinually updated with data relating to the new environment that will likely apply to

his/her preferences. The information in the user-attribute store 130 may be supplemented,

deleted, and modified at any time by multiple parties, but the owner (user) of the user-

attribute store 130 can limit access to certain areas or times (e.g., modification by anothersource is authorized to update a restaurant phone number, but not authorized to change

the user's schedule).

[0017]It is to be appreciated that embodiments described herein can employ various

machine learning-based schemes for carrying out various aspects thereof. For example,

recommendation-making analysis can involve using an automatic classifier system andprocess. The classifiers can be employed to determine and/or infer a need for action, to

assist with what information should be analyzed to generate a suggestion and determine if

the suggestion should be implemented, and to automatically update stored informationrelating to stated or implied preferences of a user. The classifiers can also apply a utility-

based analysis that considers the cost associated with implementing a suggested course of

action against the expected benefit to the user, in view of costs suffered in the event the

action was not desired. Moreover, current user state (e.g., amount of free time, displaydevice capabilities . . ) can be considered in connection with recognition in accordance

with the embodiments described herein.

[0018] Machine Learning or statistical approaches are supported., including., static and

dynamic Bayesian networks, decision trees, and probabilistic graphical models providingdifferent patterns of independence can be employed. Classification as used herein also is

inclusive of statistical regression that is utilized to develop models of priority.

[0019]As will be readily appreciated from the subject specification, the subject invention


8/14

can employ supervised classifiers that are explicitly trained (for example, by a generic

training data) as well as semi-supervised or unsupervised classifiers that are implicitly

trained (for example, by observing user behavior, receiving extrinsic information). Forexample, SVMs are configured by a learning or training phase within a classifier

constructor and feature selection module. Thus, the classifier(s) can be used to

automatically perform a number of functions as described herein. Accordingly, themonitoring component 110, recommendation-making component 120, and user-attribute

store 130 can optionally employ classifiers in connection with affecting the

functionalities associated therewith.

[0020]In FIG. 1, the block diagram of a recommendation-making component 120 and

user-attribute store 130 that facilitates analysis of information and generation of a

suggestion or automated implementation of the suggestion is depicted. Therecommendation-making component 120 receives relevant information regarding a user

and his/her environment and produces a suggestion or implements the action, by way of

an environment processor 210, analysis component 220 (with reference to the user-

attribute store 130), and recommendation generator 230.

[0021]The environment processor 210 obtains information relevant to a user and his/herenvironment and processes such information to identify one or more situations where

recommendations should be made. For example, information about a local car accident

that just occurred can contribute to a recommendation to change a driving route. The

analysis component 220 evaluates the information in view of personal preferences storedin the user-attribute store 130.

[0022]While the user-attribute store 130 can be updated with relevant information about aretail environment, as it is received by the environment processor 210 and passed through

to the analysis component 220, the user-attribute store 130 can also track

recommendations made by the recommendation generator 230 in order to assist inautomated recommendation-making for future situations. Other information that may be

tracked includes user satisfaction with respect to the suggestion or automated action,

consequences that occur as a result of an action, and reactions of others in response to anaction. At any moment in time, a user can explicitly add, modify, or delete a specification

in the user-attribute store 130.

[0023]As illustrated in FIG. 1, a block diagram of a recommendation-making componentin connection with multiple services from different points in the supply chain 150 is

presented. In particular, an A.R.E. can proceed under one of many user profiles, such as

separate data stores for retailer preferences, home preferences, and school preferences.For example, a user at work may have different lunch preferences than the same user at

home or at school. Therefore, the recommendation-making component 120 can flexibly

access the proper personality for each recommendation.

[0024]The A.R.E. 140 monitors, supports, and regulates what information a user chooses

to share. The A.R.E. contains information directly acquired from an outside source (e.g.,


9/14

contact information for area retailers, demographic information, and other observable

information), as well as additional inferred information based on analysis of the acquired

information (e.g., level of preference for each recipe, predicted list of ingredients, etc.).Other users and devices may be interested in at least some of this information, and

therefore may offer payment in exchange for desired information. A user may also restrict

identifying information if he/she chooses to maintain anonymity. The A.R.E. facilitatesprotection of information through user preferences with respect to negotiations, sales,

cost-benefit analysis, etc. In the event of a security breach (e.g., a scam, virus, e-mail

spam, etc.), the OTA service provider 160 may immediately erect a barrier to preventfurther damage to the A.R.E..

[0025]The OTA service 160 manages permissions for access to the A.R.E. 140.

Restrictions can be explicitly set by the user or implicitly determined by the broadcastdevice 410 based on nature of the information, reputation of the source requesting access,

vulnerability of the user, etc. Such restrictions can be adjusted according to changing

circumstances or as desired by the user. The broadcast device 160 can concurrently allow

access for all users, some users, or no users and may limit the amount of informationavailable to each user. The A.R.E. can facilitate real time interaction between the retailer

and other service providers. . For example, the A.R.E. can release real time healthinformation with the user's doctor or caretaker via a personal health record (pHR) as

provided by Google Health and/or Microsoft HealthVault, and shared from that

infrastructure with the service provider of choice.

[0026]Likewise, the broadcast device such as an NFC mobile phone reader 110 can

acquire information from other devices such as the point of sale (PoS) (420-440) for the

A.R.E. 140. Since such information may be sensitive, the information may be encryptedbefore the broadcast device 110 commences transmission to other users/devices (170).

For example, the broadcast device 110 can provide aspects, such as the identity and

location of users/devices requesting access and the properties they are looking for. Thisinformation can assist the A.R.E. 140 in recommendation-making or permission setting.

[0027]In view of the example systems shown and described above, methodologies thatmay be implemented in accordance with the embodiments will be better appreciated with

reference to the flow charts of FIG. 1. While, for purposes of simplicity of explanation,

the methodologies are shown and described as a series of acts, it is to be understood and

appreciated that the methodologies are not limited by the order of the acts, as some actsmay occur in different orders and/or concurrently with other acts from that shown and

described herein. Moreover, not all illustrated acts may be required to implement the

methodologies described herein.

[0029]The methodologies may be described in the general context of computer-

executable instructions, such as program modules, executed by one or more components.Generally, program modules include routines, programs, objects, data structures, etc., that

perform particular tasks or implement particular abstract data types. Typically, the

functionality of the program modules may be combined or distributed as desired.


10/14

[0030]Referring specifically to FIG. 1, a method 500 for generating a recommendation

based on user-retailer environment interaction is illustrated. The method 500 identifies

and provides suggestions for one or more issues for a user based on current environmentinformation, current user information, and stored individual preferences, including

empirical situations and predefined settings. The method 500 simply assists the user by

presenting the suggestion as a hint or reminder.

[0031]The method 500 begins at 510, where aspects of the current user and environment

are monitored. Such aspects include areas relevant to the weather, location, direction oftravel, movement surrounding the user, health, and interaction among the user and

different elements of the environment, as well as the rate of change of such information.

At 520, an issue is identified, which may have been specified by a user (e.g., through

instructions to retrieve directions to a location). Continuing to 530, relevant informationis analyzed in view of stored preferences specific to the user. Such analysis presents, for

instance, a comparison or look-up that addresses a particular purchase item (e.g.,

shopping list), preference list (e.g., favorite foods), or command (e.g., always update

meal plan when seasons change). A suggestion based on the analysis is generated at 540,which includes identifying a shopping list, providing directions, and calling for financial

and/or healthcare analysis by the agent.

[0032]However, if the issue was not based on predefined settings that correspond with

instructions at 550, the method 500 proceeds to 570, where the suggestion is simply

proposed, but not implemented without a specific command. The user may choose toimplement the suggestion, ignore the suggestion, or alter settings to address the issue in

the future.

[0033]FIG. 1 also presents a method 600 for updating stored preferences. The preferences

pertain to settings, routines, schedules, and commands specific to a user in which

recommendations are made. The method 600 continuously updates such preferences toprovide for a more sophisticated evaluation scheme, where changes automatically and

seamlessly take into effect.

[0034]The method 600 starts by monitoring a customer shopping basket at 610. Since all

aspects observed may not be stored for the sake of time, resources, simplicity, etc. (e.g.,

certain time-sensitive information, unnecessarily detailed aspects, etc.), the method 600

identifies just the relevant information for storage at 620. At 630, if the updatedinformation relates to an existing stored preference, that stored preference is modified or

deleted in view of the newly acquired information at 640. Returning to 630, if the

updated information does not relate to an existing stored preference, then at 650, thenewly acquired information is transformed into an additional preference and stored as

such. A single environment observation can provide for various presentations of

information to be integrated as preferences. For example, one environment observationcan provide information relating to weather, location, health, etc. In addition, a single

aspect of information can affect multiple preference settings. For instance, a visit to the

in-store clinic can affect numerous preferences relating to location, scheduling, and

relation to other activities (e.g., a preference to enter into the deli waiting line or check on


11/14

e-prescription order status on arrival at retail in order to prioritize shopping route.).

[0035]In FIG. 1, a method 800 for broadcasting information is illustrated. An A.R.E.

contains information and preferences specific to a user that may be useful to other

devices. A user can more easily interact and coordinate with other devices when they canaccess a portion of his/her information, such as contactless payment information.

However, since some information may be sensitive, the permissions for certain

information for certain outside users/devices depend on individual circumstances.

[0036]Beginning at 810, the method 800 sets protections on private information that

should not be disclosed. The level of privacy can be globally defined for all outside

users/devices or applied to a subset of all outside users/devices and may be changed atany time. Such determination can originate from the user or be inferred by analysis of the

nature of the information and existing settings. In one example, a user may not want to

share his/her schedule regarding a particular day and thus can set a global rule to protect

information from that day for all outside users/devices. In another example, a user mayonly want to share personal information with designated family and friends. Proceeding

to 820, the method 800 determines whether access should be allowed for a particularuser/device, by considering explicit user instructions or analysis of the situation, such the

identity of the user, reason for access, past permissions for access, nature of the

information, etc. At 830, if the particular user/device is safe, then access to information in

the A.R.E. is allowed for that user/device. For example, depending on the reputation of astore, the user may not want the store to learn information other than his/her current

purchase. Otherwise, at 840, access to information in the A.R.E. is denied for that

user/device. After access is allowed or denied for a particular user/device, the method 800returns to 820, to repeat the determination for another user/device.

[0037]FIG. 1 also illustrates a method 1000 of a specific example facilitating overallenvironment evaluation and service suggestions or implementations, as well as a specific

mobile handset 110 to Point of Sale 140 data exchange. This particular example

emphasizes the dynamic flexibility of the A.R.E. in the situation of a retail customer atPoint of Purchase. The level of assistance is not restricted to a particular area, but rather

spans across a variety of requests and observations, as expected in a new environment.

The value of suggestions to a customer is higher than if he/she were in a familiar

environment; thus the customer is more likely to depend on the suggestions of the A.R.E..

[0038]Starting at 1010, the method 1000 monitors an environment and the interactionsrevolving around a user. Based on analysis of a location unfamiliar to the user and/or

confirmation from the user that he/she is shopping or at the clinic, at 1020, the profile

corresponding to retail settings is applied.

[0039]The method 1000 proceeds to 1050 and provides suggestions to the user in view of

the preferences found in the leisure profile. Such suggestions may include directions to a

specific product or section of the retailer. In addition, the method 1000 may guess that the


12/14

user will require a prescription refill and suggests products that match the preferred

pharmacy in his/her profile. The suggestions also take into account price range,

reputation, wait time, etc. At 1060, since the user may want a heightened sense of privacyand protection while in an unfamiliar place, all permissions for access to the user's

information are denied.

[0040]With reference to FIG. 1, an example environment 1110 for implementing various

aspects disclosed herein includes a mobile wireless internet phone 110 (e.g., hand held,programmable consumer or industrial electronics . . . ). equipped with identification

technology such as a SIM and/or an NFC chip to that enables both contactless

identification, payment and receipt of information from the PoS.140

[0041]A user enters commands or information into the computer 140 through input

device(s) 110. Input devices 110 include, but are not limited to, an NFC transponder and

interrogator, an HF and/or UHF GenII reader, a 1 or 2D bar code readers, a pointingdevice such as a mouse, trackball, stylus, touch pad, keyboard, microphone, joystick,

,digital camera, digital video camera, web camera, and the like. These and other inputdevices connect to the internal mobile phone processing unit 140 directly of via a

Bluetooth interface port(s). Interface port(s) may also include, for example, a serial port,

a parallel port, a game port, and a universal serial bus (USB). Output device(s) 140 use

some of the same type of ports as input device(s). Thus, for example, a USB port may beused to provide input to the mobile phone processor 140 and to output information from

mobile phone processor to an output device 120. Output adapter interface specifications

180 are provided to illustrate that there are some output devices 180 like displays (e.g.,flat panel and CRT), speakers, and printers, among other output devices that require

special adapters. The output adapters 180 include, by way of illustration and not

limitation, video and sound cards that provide a means of connection between the outputdevice 140 and the system.

[0042]Mobile phone processor140 can operate in a dual networked environment usinglogical connections to one or more remote computers, such as point of sale 160. The

remote computer/point of sale device(s) 160 can be a personal computer, a server, a

router, a network PC, a workstation, a microprocessor based appliance, a peer device or

other common network node and the like, and typically includes many or all of theelements described relative to computer 140. Network interface encompasses

communication networks such as local-area networks (LAN) and wide-area networks

(WAN). LAN technologies include Ethernet/IEEE 902.3, WiFi 802.11 and the like. WANtechnologies include, but are not limited to, 3G mobile data networks as well as WiFi

dual mode capabilities where supported.. Finally interface specifications 190 are provided

to connect identiers from one system another, subject to client confirmation.

[0043]What has been described above includes examples of the claimed subject matter. It

is, of course, not possible to describe every conceivable combination of components or


13/14

methodologies for purposes of describing such subject matter, but it can be recognized

that many further combinations and permutations of the embodiments are possible.

Accordingly, the claimed subject matter is intended to embrace all such alterations,modifications, and variations that fall within the spirit and scope of the appended claims.

Furthermore, to the extent that the term "includes" is used in either the detailed

description or the claims, such term is intended to be inclusive in a manner similar to theterm "comprising" as "comprising" is interpreted when employed as a transitional word

in a claim.

Fig. 1.

Patent Applications in related categories: NFC mobile phone retail display

US Patent 7172112 - Public/private dual card system and method; A systemand method for providing a multiple-service card is disclosed. A card member 108 is

provided with a single card that serves as both an open transaction instrument and aprivate retailer transaction instrument. This multiple-service card may have the traditional

credit card data on one side of the card, including, for example, the account number,name of the account holder, and the expiration date. The other side of the card may

include a magnetic stripe that contains the account information in machine readable form

as well as private retailer transaction instrument information. In the system, the primaryparty and the service partner participants cooperate to complete the processes associated

with the provision of the combined card services, including a new account process, card


14/14

replacement and renewal processes, a service partner cancellation process, and a process

for cancellation and/or transfer by a primary party.

US Patent 4258478 - Fashion previewing system; US Patent Issued on March 31, 1981; A

fashion previewing system having improved optical and color

characteristics for allowing the previewing of various hair styles andcolors, and other fashion items. The previewing system utilizes a slide

projector for projecting a hair style in realistic color onto a rear

projection screen. Forward of the screen is a lens, coated on the projector side of the lenswith an anti-reflective coating, and coated on the opposite side thereof with a partially

reflective coating. The outer surface of the lens with the partially reflective coating

thereon is convex so as to reflect the facial image of the viewer back to the viewer. The

inner surface of the lens is flat or curved in cooperation with the outer surface to makethe hair style projected by the projector visible to the viewer in proper proportion to the

reflected facial image.

US Patent 6944981 - Mobile dispensing display, method for conversion of shopping cart

into dispensing display and method of advertising and dispensing products; A mobiledispensing display, method for conversion of shopping cart into dispensing display and a

method of advertising and dispensing products involves the use of a conventional

shopping cart. The shopping cart is provided from a store where the promotional activity

onto the basket and mounting a product sample support panel on the shopping cart overthe child seat. The mobile display may be moved intermittently to advantageous locations

throughout the store where shoppers congregate. Product to be sold and/or sampled, as

well as coupons or other promotional materials may be placed in the cart, whereby thecart becomes a mobile promotional and sales display as the user functions as a shoppers

assistant providing information, samples or couponsis to be conducted and converted into

a mobile dispensing display by mounting an advertising display

US Patent 5878339 - Cellular radiotelephone system with remotely programmed mobile

stations; A cellular radiotelephone system includes mobile stations which may beremotely programmed from a customer activation system to effect activation and other

programming needs. Mobile stations are manufactured in a blank form that causes them

to operate only in an inactive state. During activation, information describing the mobile

station's electronic serial number (ESN) is collected along with area of use information.

Augmented Reality Retail Environment

Documents

Transcript of Augmented Reality Retail Environment