Image Sharpness

UNITED STATES PATENT AND TRADEMARK OFFICE

APPLICATION NUMBER

FILING or 371 (c) DATE FIL FEE REC'D

12/908,161 10/20/2010

24739 CENTRAL COAST PATENT AGENCY, INC 3 HANGAR WAY SUITE D WATSONVILLE, CA 95076

682

Ul\TfE]) STI\TES ])EPA RTME'IT OF COMMERCE United States Patent and Trademark Office Adill",. COMMISSIO'JER FOR PATENTS

PO Box 1450 Alexandria, Virgmia 22313-1450 \VVi\V.uspto.gOY

ATTY.DOCKET.NO

P882 18 5 CONFIRMATION NO. 3367

FILING RECEIPT

111111111111111111111111]~!I]~~I~~I~~11~~~~Jj ~llllllllllllllllllllllllll

Date Mai led: 11/04/201 0

Receipt is acknowledged of this non-provisional patent application. The application will be taken up for examination in due course. Applicant will be notified as to the results of the examination. Any correspondence concerning the application must include the following identification information: the U.S. APPLICATION NUMBER, FILING DATE, NAME OF APPLICANT, and TITLE OF INVENTION. Fees transmitted by check or draft are subject to collection. Please verify the accuracy of the data presented on this receipt. If an error is noted on this Filing Receipt, please submit a written request for a Filing Receipt Correction. Please provide a copy of this Filing Receipt with the changes noted thereon. If you received a "Notice to File Missing Parts" for this application, please submit any corrections to this Filing Receipt with your reply to the Notice. When the USPTO processes the reply to the Notice, the USPTO will generate another Filing Receipt incorporating the requested corrections

Appl icant( s) Rodney Shaw, Aptos, CA;

Power of Attorney: The patent practitioners associated with Customer Number 24739

Domestic Priority data as claimed by applicant

Foreign Applications

Permission to Access - A proper Authorization to Permit Access to Application by Participating Offices (PTO/SB/39 or its equivalent) has been received by the USPTO.

If Required, Foreign Filing License Granted: 11/01/2010

The country code and number of your priority application, to be used for filing abroad under the Paris Convention, is US 12/908,161

Projected Publication Date: 04/26/2012

Non-Publication Request: No

Early Publication Request: No ** SMALL ENTITY **

page 1 of 3

Title

Sharpness in Digital Images

Preliminary Class

382

PROTECTING YOUR INVENTION OUTSIDE THE UNITED STATES

Since the rights granted by a U.S. patent extend only throughout the territory of the United States and have no effect in a foreign country, an inventor who wishes patent protection in another country must apply for a patent in a specific country or in regional patent offices. Applicants may wish to consider the filing of an international application under the Patent Cooperation Treaty (PCT). An international (PCT) application generally has the same effect as a regular national patent application in each PCT-member country. The PCT process simplifies the filing of patent applications on the same invention in member countries, but does not result in a grant of "an international patent" and does not eliminate the need of applicants to file additional documents and fees in countries where patent protection is desired.

Almost every country has its own patent law, and a person desiring a patent in a particular country must make an application for patent in that country in accordance with its particular laws. Since the laws of many countries differ in various respects from the patent law of the United States, applicants are advised to seek guidance from specific foreign countries to ensure that patent rights are not lost prematurely.

Applicants also are advised that in the case of inventions made in the United States, the Director of the USPTO must issue a license before applicants can apply for a patent in a foreign country. The filing of a U.S. patent application serves as a request for a foreign filing license. The application's filing receipt contains further information and guidance as to the status of applicant's license for foreign filing.

Applicants may wish to consult the USPTO booklet, "General Information Concerning Patents" (specifically, the section entitled "Treaties and Foreign Patents") for more information on timeframes and deadlines for filing foreign patent applications. The guide is available either by contacting the USPTO Contact Center at 800-786-9199, or it can be viewed on the USPTO website at http://www.uspto.gov/web/offices/pac/doc/general/index.html.

For information on preventing theft of your intellectual property (patents, trademarks and copyrights), you may wish to consult the U.S. Government website, http://www.stopfakes.gov. Part of a Department of Commerce initiative, this website includes self-help "toolkits" giving innovators guidance on how to protect intellectual property in specific countries such as China, Korea and Mexico. For questions regarding patent enforcement issues, applicants may call the U.S. Government hotline at 1-866-999-HAL T (1-866-999-4158).

GRANTED

LICENSE FOR FOREIGN FILING UNDER

Title 35, United States Code, Section 184

Title 37, Code of Federal Regulations, 5.11 & 5.15

The applicant has been granted a license under 35 U.S.C. 184, if the phrase "IF REQUIRED, FOREIGN FILING LICENSE GRANTED" followed by a date appears on this form. Such licenses are issued in all applications where the conditions for issuance of a license have been met, regardless of whether or not a license may be required as

page 2 of 3

set forth in 37 CFR 5.15. The scope and limitations of this license are set forth in 37 CFR 5.15(a) unless an earlier license has been issued under 37 CFR 5.15(b). The license is subject to revocation upon written notification. The date indicated is the effective date of the license, unless an earlier license of similar scope has been granted under 37 CFR 5.13 or 5.14.

This license is to be retained by the licensee and may be used at any time on or after the effective date thereof unless it is revoked. This license is automatically transferred to any related applications(s) filed under 37 CFR 1.53(d). This license is not retroactive.

The grant of a license does not in any way lessen the responsibility of a licensee for the security of the subject matter as imposed by any Government contract or the provisions of existing laws relating to espionage and the national security or the export of technical data. Licensees should apprise themselves of current regulations especially with respect to certain countries, of other agencies, particularly the Office of Defense Trade Controls, Department of State (with respect to Arms, Munitions and Implements of War (22 CFR 121-128)); the Bureau of Industry and Security, Department of Commerce (15 CFR parts 730-774); the Office of Foreign AssetsControl, Department of Treasury (31 CFR Parts 500+) and the Department of Energy.

NOT GRANTED

No license under 35 U.s.C. 184 has been granted at this time, if the phrase "IF REQUIRED, FOREIGN FILING LICENSE GRANTED" DOES NOT appear on this form. Applicant may still petition for a license under 37 CFR 5.12, if a license is desired before the expiration of 6 months from the filing date of the application. If 6 months has lapsed from the filing date of this application and the licensee has not received any indication of a secrecy order under 35 U.S.C. 181, the licensee may foreign file the application pursuant to 37 CFR 5.15(b).

page 3 of 3

To: From:

[email protected],officeactions@C ENTRALCOASTPA TENT. COM, plambuth@centralcoastpatent.< [email protected]

Cc: [email protected] Subject: Private PAIR Correspondence Notification for Customer Number 24739

Nov 04, 201005:49:26 AM

Dear PAIR Customer:

CENTRAL COAST PATENT AGENCY, INC 3 HANGAR WAY SUITE D WATSONVILLE, CA 95076 UNITED STATES

The following USPTO patent application(s) associated with your Customer Number, 24739 , have new outgoing correspondence. This correspondence is now available for viewing in Private PAIR.

The official date of notification of the outgoing correspondence will be indicated on the form PTOL-90 accompanying the correspondence.

Disclaimer: The list of documents shown below is provided as a courtesy and is not part of the official file wrapper. The content of the images shown in PAIR is the official record.

Application 12908161

Document Mailroom Date APP.FILE.REC 11/04/2010

Attorney Docket No. P882

To view your correspondence online or update your email addresses, please visit us anytime at https:llsportal.uspto.gov/secure/myportal/privatepair.

If you have any questions, please email the Electronic Business Center (EBC) at [email protected] with 'e-Office Action' on the subject line or call 1-866-217-9197 during the following hours:

Monday - Friday 6:00 a.m. to 12:00 a.m.

Thank you for prompt attention to this notice,

UNITED STATES PATENT AND TRADEMARK OFFICE

PATENT APPLICATION INFORMATION RETRIEVAL SYSTEM

PTO/SB/14 (07-07) Approved for use through 06/30/2010. OMB 0651-0032

U.S. Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE

Under the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it contains a valid OMB control number.

Attorney Docket Number P882 Application Data Sheet 37 CFR 1.76

Application Number

Title of Invention Sharpness in Digital Images

The application data sheet is part of the provisional or non provisional application for which it is being submitted. The following form contains the bibliographic data arranged in a format specified by the United States Patent and Trademark Office as outlined in 37 CFR 1.76. This document may be completed electronically and submitted to the Office in electronic format using the Electronic Filing System (EFS) or the document may be printed and included in a paper filed application.

Secrecy Order 37 CFR 5.2 D Portions or all of the application associated with this Application Data Sheet may fall under a Secrecy Order pursuant to

37 CFR 5.2 (Paper filers only. Applications that fall under Secrecy Order may not be filed electronically.)

Applicant Information: Applicant 1 I Remove I Applicant Authority (!)Inventor I OLegal Representative under 35 U.S.C. 117 I OParty of Interest under 35 U.S.C. 118

Prefix Given Name Middle Name Family Name Suffix

Rodney Shaw

Residence Information (Select One) (!) US Residency 0 Non US Residency o Active US Military Service

City Aptos State/Province I CA I Country of Residence i I US

Citizenship under 37 CFR 1.41(b) i US

Mailing Address of Applicant:

Address 1 281 Ventana Way

Address 2

City I Aptos I State/Province I CA

Postal Code 95003 I Countryi I US

All Inventors Must Be Listed - Additional Inventor Information blocks may be I Add I generated within this form by selecting the Add button.

Correspondence Information: Enter either Customer Number or complete the Correspondence Information section below. For further information see 37 CFR 1.33(a).

D An Address is being provided for the correspondence Information of this application.

Customer Number 24739

Email Address [email protected] I I Add Email I IRemove Emaill

Application Information:

Title of the Invention Sharpness in Digital Images

Attorney Docket Number P882 I Small Entity Status Claimed ~ Application Type Nonprovisional

Subject Matter Utility

Suggested Class (if any) I Sub Class (if any)1

Suggested Technology Center (if any)

Total Number of Drawing Sheets (if any) 9 I Suggested Figure for Publication (if any) I

EFS Web 2.2.2


U.S. Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE Under the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it contains a valid OMB control number.


Application Number


Publication Information: D Request Early Publication (Fee required at time of Request 37 CFR 1.219)

Request Not to Publish. I hereby request that the attached application not be published under 35 D U.S.C. 122(b) and certify that the invention disclosed in the attached application has not and will not be the

subject of an application filed in another country, or under a multilateral international agreement, that requires publication at eighteen months after filing.

Representative Information:

Representative information should be provided for all practitioners having a power of attorney in the application. Providing this information in the Application Data Sheet does not constitute a power of attorney in the application (see 37 CFR 1.32). Enter either Customer Number or complete the Representative Name section below. If both sections are completed the Customer Number will be used for the Representative Information during processing.

Please Select One: C!J Customer Number 1 0 US Patent Practitioner 10 Limited Recognition (37 CFR 11.9)

Customer Number 24739

Domestic Benefit/National Stage Information: This section allows for the applicant to either claim benefit under 35 U.S.C. 119(e), 120, 121, or 365(c) or indicate National Stage entry from a PCT application. Providing this information in the application data sheet constitutes the specific reference required by 35 U.S.C. 119(e) or 120, and 37 CFR 1.78(a)(2) or CFR 1.78(a)(4), and need not otherwise be made part of the specification.

Prior Application Status I Remove I Application Number Continuity Type Prior Application Number Filing Date (YYYY-MM-DD)

Additional Domestic Benefit/National Stage Data may be generated within this form I I Add by selecting the Add button.

Foreign Priority Information: This section allows for the applicant to claim benefit of foreign priority and to identify any prior foreign application for which priority is not claimed. Providing this information in the application data sheet constitutes the claim for priority as required by 35 U.S.C. 119(b) and 37 CFR 1.55(a).

I Remove I

Application Number Country i Parent Filing Date (YYYY-MM-DD) Priority Claimed

0 Yes C!J No

Additional Foreign Priority Data may be generated within this form by selecting the I I Add button. Add

Assignee Information: Providing this information in the application data sheet does not substitute for compliance with any requirement of part 3 of Title 37 of the CFR to have an assignment recorded in the Office.

Assignee 1 I Remove I

EFS Web 2.2.2


U.S. Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE Under the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it contains a valid OMB control number.


Application Number


If the Assignee is an Organization check here. D Prefix Given Name Middle Name Family Name Suffix

Mailing Address Information:

Address 1

Address 2

City State/Province

Country il Postal Code

Phone Number Fax Number

Email Address

Additional Assignee Data may be generated within this form by selecting the Add I I Add button.

Signature: A signature of the applicant or representative is required in accordance with 37 CFR 1.33 and 10.18. Please see 37 CFR 1.4(d) for the form of the signature.

Signature !Donald R. Boys! Date (YYYY-MM-DD) 2010-10-20

First Name Donald I Last Name I Boys Registration Number 35074

This collection of information is required by 37 CFR 1.76. The information is required to obtain or retain a benefit by the public which is to file (and by the USPTO to process) an application. Confidentiality is governed by 35 U.S.C. 122 and 37 CFR 1.14. This collection is estimated to take 23 minutes to complete, including gathering, preparing, and submitting the completed application data sheet form to the USPTO. Time will vary depending upon the individual case. Any comments on the amount of time you require to complete this form and!or suggestions for reducing this burden, should be sent to the Chief Information Officer, U.S. Patent and Trademark Office, U.S. Department of Commerce, P.O. Box 1450, Alexandria, VA 22313-1450. DO NOT SEND FEES OR COMPLETED FORMS TO THIS ADDRESS. SEND TO: Commissioner for Patents, P.O. Box 1450, Alexandria, VA 22313·1450.

EFS Web 2.2.2

Privacy Act Statement

The Privacy Act of 1974 (P .L. 93-579) requires that you be given certain information in connection with your submission of the attached form related to a patent application or patent. Accordingly, pursuant to the requirements of the Act, please be advised that: (1) the general authority for the collection of this information is 35 U.S.C. 2(b)(2); (2) furnishing of the information solicited is voluntary; and (3) the principal purpose for which the information is used by the U.S. Patent and Trademark Office is to process and/or examine your submission related to a patent application or patent. If you do not furnish the requested information, the U.S. Patent and Trademark Office may not be able to process and/or examine your submission, which may result in termination of proceedings or abandonment of the application or expiration of the patent.

The information provided by you in this form will be subject to the following routine uses:

1. The information on this form will be treated confidentially to the extent allowed under the Freedom of Information Act (5 U.S.C. 552) an the Privacy Act (5 U.S.C. 552a). Records from this system of records may be disclosed to the Department of Justice to determine whether the Freedom of Information Act requires disclosure of these records.

2. A record from this system of records may be disclosed, as a routine use, in the course of presenting evidence to a court, magistrate, or administrative tribunal, including disclosures to opposing counsel in the course of settlement negotiations.

3. A record in this system of records may be disclosed, as a routine use, to a Member of Congress submitting a request involving an individual, to whom the record pertains, when the individual has requested assistance from the Member with respect to the subject matter of the record.

4. A record in this system of records may be disclosed, as a routine use, to a contractor of the Agency having need for the information in order to perform a contract. Recipients of information shall be required to comply with the requirements of the Privacy Act of 1974, as amended, pursuant to 5 U.S.C. 552a(m).

5. A record related to an International Application filed under the Patent Cooperation Treaty in this system of records may be disclosed, a a routine use, to the International Bureau of the World Intellectual Property Organization, pursuant to the Patent Cooperation Treaty.

6. A record in this system of records may be disclosed, as a routine use, to another federal agency for purposes of National Security review (35 U.S.C. 181) and for review pursuant to the Atomic Energy Act (42 U.S.C. 218(c».

7. A record from this system of records may be disclosed, as a routine use, to the Administrator, General Services, or his/her designee, during an inspection of records conducted by GSA as part of that agency's responsibility to recommend improvements in records management practices and programs, under authority of 44 U.S.C. 2904 and 2906. Such disclosure shall be made in accordance with the GSA regulations governing inspection of records for this purpose, and any other relevant (i.e., GSA or Commerce) directive. Such disclosure shall not be used to make determinations about individuals.

8. A record from this system of records may be disclosed, as a routine use, to the public after either publication of the application pursuan to 35 U.S.C. 122(b) or issuance of a patent pursuant to 35 U.S.C. 151. Further, a record may be disclosed, subject to the limitations of 37 CFR 1.14, as a routine use, to the public if the record was filed in an application which became abandoned or in which the proceedings were terminated and which application is referenced by either a published application, an application open to public inspections or an issued patent.

9. A record from this system of records may be disclosed, as a routine use, to a Federal, State, or local law enforcement agency, if the USPTO becomes aware of a violation or potential violation of law or regulation.

EFS Web 2.2.2

DECLARATION AND POWER OF ATTORNEY FOR PATENT APPLICATION

ATTORNEY DOCKET NO.P882

As a below named inventor, I hereby declare that: My residence, post office address and citizenship are as stated below next to my name. I believe I am the original, first and sole inventor (if only one name is listed below) or an original, first and joint inventor (if plural names are listed below) of the subject matter which is claimed and for which a patent is sought on the invention entitled: Sharpness in Digital Images

the specification of which (check one) ~ is attached hereto. o was filed on: __ o Application Serial No. __ o and was amended on __ (If applicable)

I hereby state that I have reviewed and understand the contents of the above-identified specification, including the claims, as amended by any amendment referred to above. I acknowledge the duty to disclose information which is material to the examination ofthis application in accordance with Title 37, Code of Federal Regulations, s 1.56 (a). In the case that the present application is a continuation-in-part application, I further acknowledge the duty to disclose material information as defmed in 37 CFR s 1.56(a) which became available between the filing date of tile prior application and the filing date of the present application. I hereby claim foreign priority benefits under Title 35, United States Code sll9 of any foreign applications for patent or inventor's certificate listed below and have also identified below any foreign application for patent or inventor's certificate having a filing date before that ofthe application on which priority is claimed: Prior Foreign Application(s)

(Number) (Country) (Day/MonthIY ear Filed)

(Number) (Country) (Day/Month/Year Filed) I hereby claim the benefit under Title 35, United States Codes, 120 and 119 of any United States application(s) listed

below and, insofar as the subject matter of each of the claims of this application is not disclosed in the prior United States application in the manner provided by the first paragraph of Title 35, United States Code, 8112, I acknowledge the duty to disclose material information as dermed in Title 37, Code of Federal Regulations, 8156(a) which occurred betwecn the filing date of the prior application and the national or peT international filing date of this application.

(Application Serial No.): __ (Filing Date): __ (Statns): __ _ (Application Serial No.); __ (Filing Date): __ (Statns): __ _ (Application Serial No.): __ (Filing Date): __ (Statns): __ _ (Application Serial No.): __ (Filing Date): __ (Status): __ _ (Application Serial No.): __ (Filing Date): __ (Status): __ _

POWER OF ATTORNEY: As a named inventor, I hereby appoint:

[;gI Practitioners at customer number: 24739

OR

D Practitioners: Name: Registration number

to prosecute this application and transact all business in the Patent and Trademark Office connected therewith.

Please send all correspondence practitioners at:

~ The practitioners at the customer number indicated abovc

D Customer number: 24739

DECLARATION AND POWER OF ATTORNEY FOR PATENT APPLICATION

ATTORNEY DOCKET NO. PSS2

Authorization To Permit Access To Application by Participating Offices

The above-signed hereby grants the USPTO authority to provide the European Patent Office (EPO), the Japan Patent Office (JPO), the Korean Intellectual Property Office (KIPO), the World Intellectual Property Office (WIPO), and any other intellectual property offices in which a foreign application claiming priority to the above-identified patent application is filed access to the above-identified patent application. See 37 CFR 1.14(c) and (h).

In accordance with 37 CFR 1.14(h)(3), access will be provided to a copy of the above-identified patent application with respect to: 1) the above-identified patent application-as-filed; 2) any foreign application to which the above-identified patent app1ication claims priority under 35 U.S.C. 119(a)-(d) if a copy of the foreign application that satisfies the certified copy requirement of 37 CFR 1.55 has been filed in the above-identified patent application; and 3) any U.S. application-as-filed from which benefit is sought in the above-identified patent application.

In accordance with 37 CFR 1.14(c), access may be provided to information concerning the date of filing the Authorization to Permit Access to Application by Participating Offices.

I hereby declare that all statements made herein of my own knowledge arc true and that all statements made on information and belief are believed to be tme; and further that these statements were made with the knowledge that willful false statements and the like so made are punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that such willful false statements may jeopardize the validity ofthe application or any patent issued thereon.

I st inventor's signature: ____ -¥-_--'-'wt---"LO----=------,j'---L. __ -=--"""O::" ______ Dated: Oct J'I '"' Residence: 281 Ventana Way Aptos CA 95003 2. 0 10 Post Office Address: Same

Declaration and Power of Attorney- Page 2

~ ~ ~ ~

,........; Q 0

..... ~ ~

~

.~ ~

--u

\~ ~ ~ ~ ~

~ ~ ~ .~ I I I

~ M '--

Cl) 0 ~

...D ,..t:j \". 'e l ". .~ ~~ /

~ ~ ~

~ ~

I I

c!j

,~ \. ~ ~ ~

I I I

@ @ @ @ @ @

@ @ @ @ @ /

/ @ @ @

!

@ @ @ @ @ ( @ @ @

~)® @ @ @ \

\ \

\ \

('.l

@ @ @ @ @

i /

j /

0 ,......... ,......... ~ 0 ,.........

\ \)

@ @ @

@ @ @

@ @ /1 @ /

! /

@ @ I

\ @

\

@ @ @

@ @ @ / \

@ @ @ \

@ \ @ @ \ \ :

\ J \ \

@ @ @ \ \

\ ('.l 0 ,.........

) ,.........~.

o ,.........

,......... o ,.........

501 507 2048 2048

Original Image 1536 Final Image 1536

502 ~ Downsizing

Algorithm //-- 503 I

504 505 t Upsizing

Algorithm

Downsized Fi~D ~ Second Third "n"th

>- ----» Iteration 320 I Original ----» Iteration Iteration Iteration Chosen

480 506

Fig. 5

(601 602 603

Select an image to be enhanced for Sharpness

Downsize the image Apply a minimal- computation I--->'~I to a substantially convolution filter to image "0" I >'1

smaller resolution to produce an image "n" producing image "0"

(606 (605

Add the quotient for each pixel Divide the algebraic from step 605 to associated pixel ~ difference for each pixell~(--

value in image "0", to in step 604 by integer "n" produce an image "I".

607~ ~

Repeat process in step 606, incrementing the image no.

by 1 for each iteration, until the image no. is n-l.

(608

Present images "0" through "n" --~>,t>l1 in sequential order for a user

to select "best" image for sharpness

"" (604

Determine algebraic difference between

each pixel value of "n" with each pixel of "0"

"" ,.

~

( Upsize "best" image back to original larger resolution.

Fig. 6 610

(701 (702 (703

Select a plurality of Choose one of the Apply a minimal- computation digital images of ~ images as image "0" ~ convolution filter to "0" similar sharpness quality to be enhanced to produce an image "n"

for Sharpness

706~ ("705

Add the quotient for each pixel Divide the algebraic from step 605 to associated pixel <: difference for each pixel 141(--

value in image "0", to in step 604 by integer "n" produce an image "1".

\It ("707

Repeat process in step 606, incrementing the image no.

by 1 for each iteration, until the image no. is n-l.

(708

Present images "0" through "n" ---l)1Ii'I1 in sequential order for a user

to select "best" image for sharpness

..... ,.

'W' r 704

Determine algebraic difference between

each pixel value of "n" with same pixel of "0"

'" ,.

'W' (709

( Process each remaining image of I the plurality to produce just the .(--

Note the number of the image the user selects as best image

enhanced image of the same 710 number as in step 708

Fig. 7

M 1 1

.9

.8 .8

.7

.6 .6

.5 .5

.4 .4

.3 .3

.2 .2

.1 .1

o Local Pixel Value Average 255

Fig. 8

901 (902

Place filter and determine Determine algebraic new pixel value at first diffence between pixel position. original and new

pixel value

906\

Save the result of step 905 as new value for image "n"

\It ("907

Place filter at next position and repeat the process until all

positions are determined.

~

"

"" ,

(903

Select multiplier between o and I from graph using

segment preference

(905

Add result of step 904 back to original pixel 14:1(-

value

Fig. 9

"" --,

\It ,904

Multiply difference from step 902 by

multiplier from step 903

5

SHARPNESS IN DIGITAL IMAGES

CROSS-REFERENCE TO RELATED APPLICATIONS

N/A

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention is in the field of apparatus and techniques for enhancing the

10 apparent visual quality of images stored and presented digitally.

2. Description of Related Art

Many techniques exist for processing and filtering images displayed in a

computerized system as two-dimensional matrices of pixels, typically presented in a

15 rectangular matrix. A digital display presents pixels in color and brightness according to

values stored in memory for each pixel. For color, for example, there will be a separate

value in the RGB system for red (R), green (G) and blue (B). In an eight-bit computer

process, the values for each pixel for each attribute range from 0 to 255, which is 28. For

any of a wide variety of reasons, digital images may be less than optimal in image quality

20 as viewed by a person, and many commercial programs and techniques exist and are

available for altering the pixel values to improve the apparent quality of a digital image.

To improve the apparent quality of an image, an original image may be altered in one or

more of several attributes, such as brightness, contrast, color or what is known in the art

as sharpness. The present patent application is in the field of altering sharpness of digital

25 images to improve apparent quality to an observer.

To change apparent sharpness in a pixilated image requires changing individual

pixel values in relation to the values of surrounding (proximal) pixels. One technique

well known in the art for changing apparent sharpness is controlled application of what

are known in the art as convolution filters. References in the art to convolution filters

- 2 -

and their uses are numerous. For example RoboRealm at

http://www.roborealm.com/help/Convolution.php has a good description of convolution

filters and their uses.

Very generally, the way a convolution filter works is that a group of typically

5 adjacent pixels in the image to be enhanced is considered, the group having a central

pixel whose value is to be modified, dependent in some fashion on the value of the

adjacent pixels. The value of each pixel in the group is multiplied by a pre-determined

number, which theoretically may be different for each pixel in the group, the results are

added, the sum is divided by the number of pixels in the group (average), the determined

10 average value is divided by a number that is a function of the multipliers for each pixel,

and the final value is applied as a new value for the central pixel. The values for the

other pixels in the group are not changed. Next the filter is repositioned to have a

different central pixel, and asserted again just as above, to determine a new value for the

new center pixel, which may be a pixel adjacent to the pixel just previously altered. In

15 this way new values are determined for almost all pixels in the image Edge pixels may be

unchanged because of the geometry of the filter, and may be treated separately, such as

leaving with the original value, which in practice has little if any noticeable effect on the

enhanced image.

In the art of image enhancement, an important consideration is presenting

20 relatively small changes in an image to a person for determination of improvement,

because the optimum sharpness is a matter of opinion and viewing conditions. If changes

in an image presented to a person are quite large, it is difficult and time consuming to

select a preferred image. It is a good idea, therefore, to be able to present enhanced

images such that a newly enhanced image differs from an original or a previous

25 enhancement in what the present inventor chooses to call a "just appreciable visual

sharpness difference" (JA VSD).

An historic problem with convolution filters for enhancement of apparent

sharpness, is that the process is computation intensive, requiring in many cases

substantial computer power. Small filters are possible (minimum number of pixels

- 3 -

typically nine), and can be defined so the divisor for the last step is one, but the effect of

processing an image with such a minimal-computation filter is typically a very large and

unacceptable change in sharpness, far beyond what one might consider a JA VSD. To get

a small appreciable variation in sharpness, typically a JAVSD, larger filters with much

5 more computational power required have to be used. This is impractical for very large

images (many megapixels), or for cameras, iPods, cell phones, and other devices limited

in computational power.

What is critically needed in the art of image enhancement is a solution in which

minimal-computation filters may be used, and at the same time enhanced images may

10 still be presented to a user in just-appreciable visual differences. Also needed is a way to

process very large, high-pixel density (high resolution) with a minimum of computational

power, therefore in essentially real time. The present invention provides this much

needed solution.

15

BRIEF SUMMARY OF THE INVENTION

The inventor in the present case has considerable experience in image

enhancement technology, and has been less than satisfied with the time and computing

20 power necessary to enhance images visually, especially in the attribute of sharpness, as

known in the art. As a consequence, the inventor has developed a unique system and

process that accomplishes the desired end with a minimum in time and computing power.

In this invention, in one embodiment, a method for enhancing sharpness for a digital

image is provided, comprising the steps of ( a) in a display of a computerized appliance,

25 selecting an image to be enhanced in sharpness; (b) downsizing the selected image by a

standard downsizing algorithm executing on the computerized appliance to produce an

image 0 at resolution substantially less than resolution of the original image selected in

step (a); (c) applying a convolution filter to image 0 to produce an image n with enhanced

sharpness, where n is an integer; (d) subtracting pixel values for pixels of image n from

- 4 -

corresponding pixels for image 0, saving the differences; (e) dividing the differences in

step (d) by integer n, and saving the quotients; (f) adding the quotients from step (e) to

values for corresponding pixels in image 0 to produce an image 1, then to values of pixels

for image 1 to produce an image 2, and repeating until an image n-l is produced; (g)

5 presenting images 0 through n to a user for selection of a best image for sharpness; and

(h) upsizing the user-selected image by a standard upsizing algorithm back to the

resolution of the image selected in step (a).

In one embodiment the convolution filter is a 3 x 3 filter with multipliers of -1 at

all cells but the center cell. Also in one embodiment multiplier at the center cell is 9,

10 producing a divisor of 1 for application of the filter. In some embodiments n = 10 or

greater.

In another aspect of the invention a system for enhancing sharpness for a digital

image is provided, comprising a computerized appliance having a digital display and

executing software from a machine-readable medium, the software providing a

15 mechanism enabling a user to select an image to be enhanced, a downsizing algorithm

enabling the user to downsize the selected image to a resolution substantially less than the

than resolution of the original image selected, a convolution filter and functions for

applying the convolution filter to stored images to produce images enhanced for

sharpness, and an up sizing algorithm enabling the user to upsize an image to a higher

20 resolution. The user selects an image to be enhanced in sharpness, the image is

downsized to produce an image 0 at resolution substantially less than resolution of the

original image selected, the convolution filter is applied to image 0 to produce an image n

with enhanced sharpness, where n is an integer, the pixel values for pixels of image n are

subtracted from corresponding pixels for image 0, saving the differences, the differences

25 are divided by n, saving the quotients, the quotients are added back to the pixel values for

image 0 to produce an image 1, and the process is repeated adding the quotients to pixel

values of image 1 to produce an image 2, and so forth, until an image n-l is produced,

then images 0 through n are presented to the user for selection of a best image for

- 5 -

sharpness, then the selected image is upsized back to the resolution of the original image

selected to be enhanced in sharpness.

In one embodiment of the system the convolution filter is a 3 x 3 filter with

multipliers of -1 at all cells but the center cell. Also in one embodiment the multiplier at

5 the center cell is 9, producing a divisor of 1 for application of the filter. In some

embodiments n = 10 or greater.

In another aspect of the invention a method for enhancing sharpness for a digital

image is provided comprising the steps of ( a) in a display of a computerized appliance,

selecting an image to be enhanced in sharpness; (b) downsizing the selected image by a

10 downsizing algorithm executing on the computerized appliance to produce an image 0 at

resolution substantially less than resolution of the original image selected in step (a); (c)

applying a convolution filter to image 0 to produce an image n with enhanced sharpness,

where n is an integer; (d) subtracting pixel values for pixels of image n from

corresponding pixels for image 0, saving the differences; (e) dividing the differences in

15 step (d) by integer n, and saving the quotients; (f) adding the quotients from step (e) to

values for corresponding pixels in image 0 to produce an image 1; (g) displaying image 1

to a user and asking for approval; (h) in case of no approval at step (g), adding the

quotients from step (e) to the pixel values for image 1 to produce an image 2; (i)

repeating building new images by process of steps (g) and (h) until the user selects one as

20 best image; and (j) up sizing the user-selected image by an upsizing algorithm back to the


In one embodiment of this method the convolution filter is a 3 x 3 filter with

multipliers of -1 at all cells but the center cell. Also in one embodiment the multiplier at

the center cell is 9, producing a divisor of 1 for application of the filter. In some cases n

25 = 10 or greater.

In yet another aspect of the invention a system for enhancing sharpness for a

digital image is provided, comprising a computerized appliance executing software from

a machine-readable medium, the software providing a mechanism enabling a user to

select an image to be enhanced, a downsizing algorithm enabling the user to downsize

5

- 6 -

the selected image to a resolution substantially less than the than resolution of the original

image selected, a convolution filter and controls for applying the convolution filter to

stored images to produce images enhanced for sharpness, and an up sizing algorithm

enabling the user to upsize an image to a higher resolution.

The user selects an image to be enhanced in sharpness, the image is downsized to

produce an image ° at resolution substantially less than resolution of the original image

selected, the convolution filter is applied to image ° to produce an image n with enhanced

sharpness, where n is an integer, the pixel values for pixels of image n are subtracted

from corresponding pixels for image 0, saving the differences, the differences are divided

10 by n, saving the quotients, the quotients are added back to the pixel values for image ° to

produce an image 1, image 1 is displayed to the user to approve or not as a best image for

sharpness, in the case of no approval the saved quotients are added to the pixel values of

image 1 to produce an image 2, which is displayed to the user for approval, and the

process is repeated until the user selects an image as the best image for sharpness, then

15 the selected image is upsized back to the resolution of the original image selected to be

enhanced in sharpness.

In one embodiment the convolution filter is a 3 x 3 filter with multipliers of -1 at

all cells but the center cell. Also in one embodiment the multiplier at the center cell is 9,

producing a divisor of 1 for application of the filter. In some embodiment n = 10 or

20 greater.

In still another aspect of the invention a method for producing a sequence of

images enhanced for sharpness is provided, comprising the steps of ( a) selecting an image

to be enhanced as image 0, (b) applying a convolution filter to image ° to produce an

image n with enhanced sharpness, where n is an integer, (c) subtracting pixel values for

25 pixels of image n from corresponding pixels for image 0, saving the differences, (d)

dividing the differences in step (c) by integer n, and saving the quotients, and (e) adding

the quotients from step (d) to values for corresponding pixels in image ° to produce an

image 1, then to values of pixels for image 1 to produce an image 2, and repeating until

an image n-l is produced.

5

- 7 -

In one embodiment there is a further step for presenting the images as a sequence

of images to a user for selection of one of the images as a best image for sharpness.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

Fig. 1 Fig. 1 is a representation of geometry and placement of a 3 x 3 convolution

filter 102 in the art.

Fig. 2 is an enlarged view of the filter of Fig. 1, with associated values, to

10 illustrate the computational procedure for asserting the filter at one position.

Fig. 3 is a representation of how the filter of Fig. 1 might be moved over an image

to produce heightened sharpness for the entire image.

Fig. 4 Fig. 4 illustrates a computerized appliance having Internet connection via a

wireless network that communicates with a station, thence through a gateway to the

15 Internet backbone, which represents all of the network interconnections in the Internet

network.

Fig. 5 illustrates a process for selectively enhancing sharpness of relatively large

images in a minimum amount of time, using devices of limited computational power.

Fig. 6 is a process flow diagram (flow chart) illustrating steps in a process

20 according to a preferred embodiment of the present invention.

Fig. 7 is a flow diagram illustrating a process for batch processing of similar

images in an embodiment of the present invention.

Fig. 8 is a diagram for use in preferential sharpening in different segments of an

image in an embodiment of the present invention.

25 Fig. 9 is a flow diagram depicting a process for preferential sharpening in

segments based on local pixel value averages indicating relative lightness or darkness in

the image in the local vicinity.

- 8 -

DETAILED DESCRIPTION OF THE INVENTION

Convolution filters are spatial filters. Spatial filtering is the filtering of an image

in the spatial domain. That is, the value of each pixel of the image is modified in

5 contextual relationship to values of neighboring pixels. Consider, for example, a digital

image of 320 rows and 480 columns, having a total of 1.536 x 105 pixels. The smallest

grouping of pixels which associates one pixel with all of its nearest neighbors is typically

a 3 x 3 matrix of nine pixels, in which a central pixel is seen surrounded by its eight

nearest neighbors.

10

15

Fig. 1 is a representation of geometry and placement of a 3 x 3 convolution filter

102 in the art, centered on a pixel 103 in the upper left comer of a portion of a pixilated

image 101. The rectangular geometry of filter 102 is seen to relate a central pixel 103 to

its eight surrounding closest neighbors. Only a small number of pixels at exaggerated

spacing distance is shown for image portion 101.

Fig. 2 is an enlarged view of the filter of Fig. 1, with associated values, to

illustrate the computational procedure for asserting the filter at one position. In this

representation each of the nine pixels associated in the 3 x 3 filter pattern has been given

a lower-case letter, a through i. Pixel e in this case is the center pixel 103 for which the

value will be changed in one assertion of the filter. Each of the nine positions in the

20 pattern has also been associated with a signed multiplier, which is +x for pixel e (103),

and -1 for each of the surrounding pixels. These numbers are multipliers in an algorithm

associated with the filter. Theoretically all of the multipliers may have unique values, but

for reasons of efficacy and by experience -1 is an appropriate choice for the eight

surrounding pixels. This multiplier is not the original pixel value at each position, but a

25 multiplier to be applied to the pixel value at that position each time the filter is asserted.

The procedure for the filter we are considering operates as follows:

(1) Determine the algebraic sum of the multipliers. This number is set aside as a divisor

for step (3) below. In the example of Fig. 2 this divisor is -8+x.

- 9 -

(2) Multiply each pixel value by the assigned multiplier and take the algebraic sum of the

results (-a-b-c-d+xe-f-g-h-i)

(3) Divide this result by the divisor determined in step (1) above.

(4) Replace the pixel value e by this new pixel value

5 (5) Move the filter to determine a new pixel value for another center pixel.

Fig. 3 is a representation of how the filter of Fig. 1 might be moved over an image

to produce heightened sharpness for the entire image. First the filter is applied at the

upper left comer of the image (A), over a center pixel that is the second pixel in the

second row. The filter procedure is applied for that center pixel, then the filter is moved

10 one pixel distance to the right (B), and applied to change the value for the third pixel in

the second row. This move and calculate procedure is repeated for all the pixels in the

second row, then the filter returns to the second column centered on the second pixel in

the third row (C). Then the filter is moved sequentially one pixel at a time through the

third row. This row-by-row procedure continues until all new pixel values are

15 determined that can be determined given the geometry of the filter. Now a new image is

stored that has enhanced sharpness compared to the original image.

The skilled artisan will recognize, of course, that the concept of a 3 x 3 filter with

multipliers assigned to each cell in the filter is just a convenient concept. What actually

happens is that a software routine, executing from a machine-readable medium coupled

20 to a computer appliance, consults a mapping of values in memory for an original image

that is to be modified by the algorithm, selects the appropriate values that are associated

with nine adjacent pixels, performs the steps of the algorithm, stores the new center pixel

value in memory for a new image, and then selects a new mapping of nine adjacent pixels

(moves the filter). If the pixel groups are selected in a manner that every pixel that may

25 be the center pixel of a 3 x 3 mapping, then new values will be determined a stored for all

pixels in the original image, except edge pixels.

Also a part of the algorithm is a step dictating that if a new value for a pixel is

determined to be zero or less than zero, zero is used; and if a new value is determined that

- 10 -

is greater than the maximum allowed (255 for an eight-bit computing machine), then 255

is used.

Given the procedure above for applying the convolution filter, it will be apparent

to the skilled person that in step (1), if one chooses 9 for the multiplier in the center cell

5 (x), than the divisor for step (3) is 1, and in effect step (3) may be skipped; a bonus in

computation efficiency.

It is rather well-known in the art that a 3 x 3 convolution filter with -1 as a

multiplier for the surrounding pixels is an appropriate choice to minimize computation

intensity, but to provide differences in sharpness in an altered image from an original that

10 is not too extreme, it is necessary to use a multiplier x that will require a division step

with a divisor greater than 1. Using 9 for x generates an amended image that has

dramatic enhanced sharpness. To produce an image enhanced for sharpness by a

JA VSD, it is necessary to use a much larger number for x. In practice it is seen that x

needs to be about 18, producing a divisor of 10, to produce a sharpness-enhanced image

15 at JAVSD. A typical user, however, will not be satisfied with viewing just a first

enhanced image. A user will typically want to see images enhanced step-by-step, until it

is apparent the image is too sharp. Then the user may back down to the just previous

image as the best choice.

To accomplish this in the art, assuming that x=18 produces a JAVSD, requires

20 that an enhanced image be produced by applying the filer with x= 18 (divisor 10) at all

positions that can be attained. Then a second enhanced image is produced with x= 17

(divisor 9), and so on (x = 16, 15, 14 ... ), until the user discovers the new image is too

sharp. This may require four or more image iterations with a new divisor greater than 1

for each. The skilled person will understand that the original image needs to be saved,

25 and each enhanced image produced from the original or from a previously enhanced

image also needs to be saved, and functionality needs to be provided for the user to select

anyone of the images as the preferred image for sharpness.

- 11 -

The process described above for producing and displaying enhanced images to a

viewer for selection to produce an image with preferred sharpness, is still quite

computationally intensive.

Fig. 4 illustrates a computerized appliance 401 having Internet connection via a

5 wireless network that communicates with a station 402, thence through a gateway 403 to

Internet backbone 405, which represents all of the network interconnections in the

Internet network. Two Internet-connected servers 406 and 407 are shown representing

all of the sites in the Internet network which may serve information and data to internet

connected appliances like appliance 401. Appliance 401 may be a cellular telephone, a

10 personal digital assistant, or any other Internet connectable appliance. In some cases

appliance 401 may be a laptop or desktop computer, or an iPad device. Internet

connection is represented in Fig. 4 as a means by which appliance 401 may receive

images, however, in some embodiments of the invention there may be no Internet

connection, and images may be loaded to the appliance by any other known data transfer

15 technique.

The skilled person will understand that appliance 401 will have a CPU and a

display, and will be capable of executing software 408 stored in local memory, without

this specification detailing the well-known components used in computerized appliances

for displaying images, and for executing software that may alter pixel values and display

20 altered images from original images stored in memory coupled to the device.

Fig. 5 illustrates a process for selectively enhancing sharpness of relatively large

images in a minimum amount of time, using devices of limited computational power. It

is well-known that the tendency in the art, due in part to the ever-descending cost of

memory and greater resolution in displays, is to images of higher resolution. Given the

25 descriptions above regarding the computational intensity of sharpness enhancement,

images of higher and higher resolution present an ever bigger problem in time and

computing power. There are, however, quite good downsizing algorithms available to

render high-resolution images at lower resolution. One site that deals with this issue is

http-.~L-:~:}Y~Jl1~p-l1Q.!Q.t}!!i~h~~::u~Q!nLl~11l:gg~Y1l:p-;;.i?;~,.htm .

- 12 -

So a first step in sharpness enhancement in an embodiment of the present

invention is to downsize the image desired to be enhanced. In Fig. 5 image 501 is an

image for which a user desires enhanced sharpness. This image is represented as 2048 x

1536, which is 3,145,728, or 3.146 mega-pixels. This particular size is used only for

5 exemplary purpose, and could be any image of high-resolution. A first step is

downsizing this image to a lower resolution, using a commercially available downsizing

algorithm.

In this example image 501 is downsized to 480 x 320, or 153,600 pixels, about

5% of the number of pixels in the larger image. The skilled person will recognize that the

10 visual quality of downsized image 502 will be essentially the same as the image 501, as

long as the display is presented without too much magnification. The smaller image will

be quite satisfactory for a user to make judgments as to the quality of sharpness. A very

big advantage is that application of a convolution filter to the smaller image will have to

deal with only one pixel in twenty of the larger image, and can operate either twenty

15 times quicker, or with far less computing power in the same time frame.

The next step in this unique process is running a minimal-computation

convolution filter over the smaller image in several steps to create a series of altered

images with just-appreciable visual difference from one image to the next, to create a

series of enhanced images 503, 504, 505 506. There is a unique difference in the way

20 this is done in this example than in the prior art. In this embodiment the 3 x 3 filter with

surrounding multipliers of -1 and x = 9 is used. this provides for the minimum

computation, because the divisor for step (3) on page 5 above will be 1, which allows us

to skip that step.

The result of the single pass of the minimal convolution filter, however is that the

25 sharpness change will be quite dramatic, as described previously above, beyond what

most users would select as a desirable improvement. In this embodiment this is handled

in a unique way. The pixel vales for the original image are saved as image 0, and the first

enhanced image is treated as image 10. Now our system takes, for each pixel, the

difference between the pixel value for image 10 and the pixel value for image 0, and

- 13 -

divides by ten. These differences are algebraically added back to image 0 to produce

image 1, an image with a just appreciable visual difference in sharpness from image O.

Adding the differences to the pixel values for image 1 produces image 2, an image with a

JAVSD from image 1, and a greater difference in sharpness from image O. The process is

5 repeated through image 9. Image 10 already exists as the result of applying the

convolution filter to image O.

We now have a series of ten images, each differing from its immediate neighbors

by JA VSD. A user may easily select the image that appears to be the best (in the eye of

the beholder) for sharpness.

10 It is not required that there be ten iterations. There may be five, or six, or four;

but there needs be several, so the user has a selection of several images from which to

choose. If the selection is too sparse, the best image to the user might well be between

two of the iterations presented. That is, one will appear to the user to be not sharp

enough, and an adjacent iteration will appear too sharp.

15 Once the "best" image is chosen by the user, it is needed to provide that result to

the larger resolution, which in this example is 2048 x 1536. In the prior art the process

would dictate that the filter be applied to the larger image. But in this embodiment of the

invention the best smaller image is simply upsized by a commercially-available algorithm

that has been determined to be appropriate. The result has been shown by the inventor to

20 be equal in quality to the prior art method of applying more computation-intensive filters

to the larger image, a process perhaps requiring orders of magnitude more power and

time.

Fig. 6 is a process flow diagram (flow chart) illustrating steps in a process

according to a preferred embodiment of the present invention, much as described above.

25 At step 601 an image is selected to be enhanced for sharpness. At step 602 the image is

downsized by applying a commercially-available downsizing algorithm, to a resolution

substantially less than the resolution of the original image selected in step 601, providing

an image 0 (502 in Fig. 5). At step 603 a convolution filter is applied to the reduced

resolution image to form a first image with enhanced sharpness (image 1 - 503 in Fig. 5).

- 14 -

At step 604 the algebraic difference in value between the pixel values for image 1 and the

original downsized image is determined. This algebraic difference is divided by an

integer n in step 605. At step 606 the quotient for each pixel is added to the pixel value

for image "0" to produce a second enhanced image 2. At step 607 the same quotient for

5 each pixel from step 605 is added to the pixels of image 2 to produce a third enhanced

image 3. Step 707 is repeated to add values to pixels of image 3, and so on, until an

image n-l is produced. At this point there are n enhanced images (0 to n), each

displaying a JA VSD with the one before. These may be displayed to a user, preferably in

order, and the user is invited to select the image judged to be the best for sharpness. This

10 is then upsized in the final act back to the original resolution of the larger image that was

first considered to be enhanced for sharpness. This final image may be saved.

The skilled person will recognize that the embodiments described herein may be

altered in several ways within the scope of the invention. The size of the "larger" image

is not a fixed value, for example, but can be anyone of a wide variety of resolutions. The

15 downsizing and up sizing algorithms are not fixed, but may be chosen from a variety of

readily-available and well-known algorithms. The size of the "smaller" image is not

fixed either, but may vary over a wide range. The smaller image is preferably

considerably smaller than the larger to effectively limit the number of pixels necessary to

recalculate in filter application. The number of iterations from image 0 to a final image,

20 each of which is produced by a single pass of the filter, is also not fixed, but is preferably

at least four, and more preferably eight or ten.

Another variation in the process might involve producing one alteration at a time,

and allowing the user to judge the new image before going on to a next. For example, the

system might present the first alteration to the user and wait for a signal to produce the

25 next, and then wait for a signal to produce another. The user may have access as well to

a "back" command, and to a command to compare the image with the original, so when

an image is presented that is slightly too sharp, the back command will revert to the just

previous image, and the user may then cause that image to be selected and upsized to the

original resolution. There are many similar possibilities.

- 15 -

Batch Processing

In another aspect of the present invention it may be desirable to do batch

processing, that is, to apply the process described above in different examples to a

5 plurality of digital images. A user may have, for example, a plurality of images of very

similar characteristics, such as a group of images captured by a digital camera in a

relatively short period of time, under similar circumstances of lighting, and without

changing settings on the camera, and displayed on the same monitor, perhaps a computer

display monitor.

10 Referring now to Fig. 6 and to the description of Fig. 6 above, and not considering

the downsizing or upsizing of an image, a process is described wherein a minimal

convolution filter is sequentially applied over the pixels of an image to be enhanced

(image 0), producing pixel values for an image "n". Sharpness is thus enhanced for

image "n", but the change (increase) in sharpness will typically be more than might be

15 desired. In this process the algebraic difference between the value for each original pixel

and the value for the pixel in the same position in the image for image "n" is determined,

and then the difference is divided by an integer. The integer may be theoretically any

integer, but the idea is to produce images between image 0 and image "n", in which there

is just an appreciable visual difference. So this integer value may be 1 0, for example, and

20 10 is set as "n".

The pixel value differences at each pixel position is divided by the integer, then

an image 1 is produced by adding to the pixel values at each pixel position for image 0

one-tenth of the difference between the pixel value for image 0 and the pixel value for the

same pixel in image 10. Similarly an image 2 is produced adding .2 times the difference

25 at each pixel position, an image 3 using .3 times the difference, and so on, producing

images 1 through 9 between image 0 and image 10, each successive image having a

sharpness increase of just an appreciable visual difference. These ten images are

displayed to a user, the user enabled to select the "best" image, that is, the one that seems

to have, for that user, the optimum quality of sharpness.

- 16 -

Assume now that this unique process is followed for one image of a plurality of

images of very similar characteristics by a user, and the user selects image 3 as the

optimum image sharpness. It may be assumed, then, that the image 3 for all of the

plurality of images will be, for this user, the image with the optimum sharpness. It will

5 not be necessary to produce image 1, 2, or 4 through 9 for any of the other images of the

plurality. Having selected the plurality of images and initiated a batch process, the batch

system in this embodiment will produce the ten images for the user, and enable selection

of the "best" image for the user, and then use the image selected (1st, 2nd, 3rd, etc.), to

produce a sharpness-enhanced image for all of the other images of the plurality.

10 Fig. 7 is a process flow diagram for the batch processing process described just

above, in which steps 702 through 708 repeat the process described with reference to Fig.

6 for one image. Step 701 is a first step for selecting the first image to be enhanced from

a plurality of similar images. Step 709 is for noting the number of the image chosen by

the user as the optimum image for sharpness, and step 710 repeats the process for every

15 other image of the plurality, but to produce only the image of the number selected. So it

is only necessary to produce all of the images for selection just once, then all of the other

enhanced images may be produced automatically.

Another example of batch processing in sharpness enhancement is in the area of

video technology. It is well-know that data streams for video are arranged to produce

20 successive frames in display, much in the manner of movie film presenting a rapidly

changing sequence of still images, each slightly altered from the previous. Typically all

of the frames in a video data stream will have very similar sharpness characteristics. If

one selects, then, just one frame, applies the process described above to the one frame,

and a user selects one of the candidate sharpness-enhanced images as the most

25 appropriate, then further processing may be truncated for all the other frames. Assume,

for example, that the process is applied to one frame, providing ten candidate images of

that frame, each with a JA VSD from the immediately preceding image, and the user

selects image four. One may safely assume that image four will be appropriate for all the

other frames of the video as well. and the original image for each frame may then be

5

- 17 -

processed to produce the fourth image, without producing all of the other candidate

Images.

Selective Segmentation in Image Sharpening

The processes described thus far in this specification apply the same process in

sharpness enhancement to every part of an image. The skilled artisan will understand

that in some cases a user may prefer to preferentially enhance sharpness in particular

segments of an image. In one instance, for example, a user may prefer to sharpen an

image preferentially in segments that are in shadow. In another instance a user may

10 prefer to sharpen an image preferentially in segments that are highlighted, that is in

brightness. In yet another instance a user may prefer to sharpen an image preferentially

in segments that are in midrange of brightness. Other similar preferences are possible.

Therefore, in another aspect of the present invention, a process is provided that

treats different segments of an image differently, according the local brightness

15 characteristics. Fig. 8 is a diagram that relates average local pixel value from 0 to 255

(assuming an eight-bit display system) for brightness to a vertical scale between 0 (at

origin) and 1. In Fig. 8 there is a straight line labeled S for use in a process to sharpen

preferentially in segments that are in shadow. A straight line labeled H is for use in a

process to sharpen preferentially in segments of an image highlighted. A curve M is for

20 use in a process to sharpen preferentially in midtone segments.

Assume for a first example that a user wants to sharpen preferentially in segments

that are in shadow. The process is very similar to that described above as conventional

art for producing an image n from an original image 0, in which a convolution filter is

applied sequentially to values for pixels of image 0 to produce image n. In this new and

25 non-conventional procedure, however, at each application of the filter, in addition to

producing a new pixel value for image n, the pixel values for each cell of the filter (nine

cells for a three by three filter) are added and divided by the number of the cells,

rendering a pixel value average for the cells in the vicinity of the center cell being altered

- 18 -

in value by the filter protocol. This average pixel value expresses the nature of the

segment in which the object pixel resides; that is, light, dark or midtone.

Now, in this preferential process for sharpening in shadow, the system utilizes the

graph of Fig. 8, and line S. Note that for a local pixel value average of zero (fully dark,

5 extreme shadow) the graph expresses 1, and for a local pixel value average of255 (fully

bright) the graph expresses zero, and for local averages in between, the value from the

graph is between 0 and 1 proportionally.

At every position for application of the filter the local average pixel value is

determined, and the new pixel value determined by the filter protocol is used to

10 determine the pixel value difference for that pixel position, and the difference is

multiplied by the value from the graph and added back for the new pixel value.

It will be apparent to the skilled person that this procedure, using the line S, will

preferentially sharpen in segments that a dark, and will sharpen less in segments that are

more light.

15 Ifit is desired to sharpen preferentially in areas that are light, then in the process

the line H is used from the graph of Fig. 8, and sharpness will be preferential for light

segments, and less for dark segments. If it is desired to sharpen in mid-tone segments,

the curve M will be used, so maximum effect will accrue for local averages near midtone

(halfway between 0 and 255), and there will be little effect near deep shadow or extreme

20 brightness, and the effect will demonstrate uniform differences, because the curves are

well-behaved.

It will be apparent to the skilled person that a graph may be created for just about

any segment enhancement. M may be inverted, for example, to sharpen preferentially in

both deep shadow or extreme brightness, but not at all at mid-tone. The skilled person

25 will also understand that the graph of Fig. 8 is used to illustrate the process, but that in

practice values will be picked from tables relating local pixel value averages to decimal

fractions between 0 and 1.

Fig. 9 is a flow diagram depicting the process just described for preferential

sharpening in segments based on local pixel values averages indicating relative lightness

- 19 -

or darkness in the image in the local vicinity. At step 901 the convolution filter is applied

for a first pixel in the original image "0". At step 902 the algebraic difference between

the original pixel value at the first position and the new pixel value is determined. At

step 903 a multiplier is selected according to the graph using the relationship for a

5 segment preference. At step 904 the difference from step 902 is multiplied by the

multiplier from step 903. At step 905 the result of step 904 is added back to the original

pixel value. At step 906 the result of step 905 is saved as the new pixel value for image

"n". At step 907 the filter is moved to a new position and the process is repeated until

new pixel values are determined for all of the pixel positions.

1 0 It will be apparent to the skilled person that there are many alterations that might

15

be made to embodiments described as examples herein, all within the scope of the

invention, which is limited only by the claims that follow.

5

10

15

- 20-

CLAIMS

1. A method for enhancing sharpness for a digital image, comprising the steps of:

(a) in a display of a computerized appliance, selecting an image to be enhanced in

sharpness;

(b) downsizing the selected image by a downsizing algorithm executing on the

computerized appliance to produce an image 0 at resolution substantially less than

resolution of the original image selected in step (a);

(c) applying a convolution filter to image 0 to produce an image n with enhanced

sharpness, where n is an integer;

(d) subtracting pixel values for pixels of image n from corresponding pixels for

image 0, saving the differences;

(e) dividing the differences in step (d) by integer n, and saving the quotients;

(f) adding the quotients from step (e) to values for corresponding pixels in image

o to produce an image 1, then to values of pixels for image 1 to produce an image 2, and

repeating until an image n-l is produced;

(g) presenting images 0 through n to a user for selection of a best image for

sharpness; and

20 (h) upsizing the user-selected image by an up sizing algorithm back to the

25


2. The method of claim 1 wherein the convolution filter is a 3 x 3 filter with multipliers

of -1 at all cells but the center cell.

3. The method of claim 2 wherein the multiplier at the center cell is 9, producing a

divisor of 1 for application of the filter.

4. The method of claim 1 wherein n = 10.

5

10

- 21 -

5. A system for enhancing sharpness for a digital image, comprising:

a computerized appliance having a digital display and executing software from a

machine-readable medium, the software providing:

a mechanism enabling a user to select an image to be enhanced;

a downsizing algorithm enabling the user to downsize the selected image to a

resolution substantially less than the than resolution of the original image selected;

a convolution filter and functions for applying the convolution filter to stored

images to produce images enhanced for sharpness; and

an upsizing algorithm enabling the user to upsize an image to a higher resolution;

wherein the user selects an image to be enhanced in sharpness, the image is




15 subtracted from corresponding pixels for image 0, saving the differences, the differences

are divided by n, saving the quotients, the quotients are added back to the pixel values for

image 0 to produce an image 1, and the process is repeated adding the quotients to pixel

values of image 1 to produce an image 2, and so forth, until an image n-l is produced,

then images 0 through n are presented to the user for selection of a best image for

20 sharpness, then the selected image is upsized back to the resolution of the original image


25

6. The system of claim 5 wherein the convolution filter is a 3 x 3 filter with multipliers


7. The system of claim 6 wherein the multiplier at the center cell is 9, producing a


8. The system of claim 1 wherein n = 10.

- 22-

9. A method for enhancing sharpness for a digital image, comprising the steps of:

(a) in a display of a computerized appliance, selecting an image to be enhanced in

sharpness;

5 (b) downsizing the selected image by a downsizing algorithm executing on the


resolution of the original image selected in step (a);

(c) applying a convolution filter to image 0 to produce an image n with enhanced


10 (d) subtracting pixel values for pixels of image n from corresponding pixels for


(e) dividing the differences in step (d) by integer n, and saving the quotients;

(f) adding the quotients from step (e) to values for corresponding pixels in image

o to produce an image 1;

15 (g) displaying image 1 to a user and asking for approval;

(h) in case of no approval at step (g), adding the quotients from step (e) to the

pixel values for image 1 to produce an image 2;

(i) repeating building new images by process of steps (g) and (h) until the user

selects one as best image; and

20 (j) upsizing the user-selected image by an up sizing algorithm back to the

25


10. The method of claim 9 wherein the convolution filter is a 3 x 3 filter with multipliers


11. The method of claim 10 wherein the multiplier at the center cell is 9, producing a


12. The method of claim 9 wherein n = 10.

5

10

- 23 -

13. A system for enhancing sharpness for a digital image, comprising:

a computerized appliance executing software from a machine-readable medium,

the software providing:

a mechanism enabling a user to select an image to be enhanced;

a downsizing algorithm enabling the user to downsize the selected image to a

resolution substantially less than the than resolution of the original image selected;

a convolution filter and controls for applying the convolution filter to stored

images to produce images enhanced for sharpness; and

an upsizing algorithm enabling the user to upsize an image to a higher resolution;

wherein the user selects an image to be enhanced in sharpness, the image is




15 subtracted from corresponding pixels for image 0, saving the differences, the differences

are divided by n, saving the quotients, the quotients are added back to the pixel values for

image 0 to produce an image 1, image 1 is displayed to the user to approve or not as a

best image for sharpness, in the case of no approval the saved quotients are added to the

pixel values of image 1 to produce an image 2, which is displayed to the user for

20 approval, and the process is repeated until the user selects an image as the best image for

sharpness, then the selected image is upsized back to the resolution of the original image


14. The system of claim 13 wherein the convolution filter is a 3 x 3 filter with multipliers

25 of -1 at all cells but the center cell.

15. The system of claim 14 wherein the multiplier at the center cell is 9, producing a


5

- 24-

16. The system of claim 13 wherein n = 10.

17. A method for producing a sequence of images enhanced for sharpness, comprising

the steps of:

(a) selecting an image to be enhanced as image 0;

(b) applying a convolution filter to image 0 to produce an image n with enhanced


(c) subtracting pixel values for pixels of image n from corresponding pixels for


10 (d) dividing the differences in step (c) by integer n, and saving the quotients; and

(e) adding the quotients from step (d) to values for corresponding pixels in image

o to produce an image 1, then to values of pixels for image 1 to produce an image 2, and

repeating until an image n-l is produced.

15 18. The method of claim 17 comprising a further step for presenting the images as a

sequence of images to a user for selection of one of the images as a best image for

sharpness.

20

- 25 -

ABSTRACT OF THE DISCLOSURE

A method for enhancing sharpness for a digital image follows this sequence: (a)

in a display of a computerized appliance, selecting an image to be enhanced in

5 sharpness; (b) downsizing the selected image by a downsizing algorithm executing on the


resolution of the original image selected in step (a); (c) applying a convolution filter to

image 0 to produce an image n with enhanced sharpness, where n is an integer; (d)

subtracting pixel values for pixels of image n from corresponding pixels for image 0,

10 saving the differences; (e) dividing the differences in step (d) by integer n, and saving the

quotients; (f) adding the quotients from step (e) to values for corresponding pixels in

image 0 to produce an image 1, then to values of pixels for image 1 to produce an image

2, and repeating until an image n-l is produced; (g) presenting images 0 through n to a

user for selection of a best image for sharpness; and (h) upsizing the user-selected image

15 by an upsizing algorithm back to the resolution of the image selected in step (a).

Electronic Patent Application Fee Transmittal

Application Number:

Filing Date:

Title of Invention: Sharpness in Digital Images

First Named Inventor/Applicant Name: Rodney Shaw

Filer: Donald Rex Boys/Sheri Beasley

Attorney Docket Number: P882

Filed as Small Entity

Utility under 35 USC 111 (a) Filing Fees

Description Fee Code Quantity Amount Sub-Total in

USD($)

Basic Filing:

Utility filing Fee (Electronic filing) 4011 1 82 82

Utility Search Fee 2111 1 270 270

Utility Examination Fee 2311 1 110 110

Pages:

Claims:

Independent claims in excess of 3 2201 2 110 220

Miscellaneous-Filing:

Petition:

Description Fee Code Quantity Amount Sub-Total in

USD($)

Patent-Appeals-and-Interference:

Post-Allowance-and-Post-Issuance:

Extension-of-Time:

Miscellaneous:

Total in USD ($) 682

Electronic Acknowledgement Receipt

EFSID: 8661904

Application Number: 12908161

International Application Number:

Confirmation Number: 3367

Title of Invention: Sharpness in Digital Images

First Named Inventor/Applicant Name: Rodney Shaw

Customer Number: 24739

Filer: Donald Rex Boys/Sheri Beasley

Filer Authorized By: Donald Rex Boys

Attorney Docket Number: P882

Receipt Date: 20-0CT-2010

Filing Date:

TimeStamp: 12:59:04

Application Type: Utility under 35 USC 111 (a)

Payment information:

Submitted with Payment yes

Payment Type Deposit Account

Payment was successfully received in RAM $682

RAM confirmation Number 9417

Deposit Account 500534

Authorized User

File Listing:

Document I Document Description I

File Name I

File Size(Bytes)/ I Multi ,I Pages Number Message Digest Part /.zip (ifappl.)

1347662

1 Application Data Sheet US_ADSJorm_SB_14a.pdf no 4 c7971 e9fOfaOc83791fddOc026d40016164d

bSf1

Warnings:

Information:

63858

2 Oath or Declaration filed POA882.pdf no 2 Ob9fa 1621 fd40468c987f14e3de664f2fd62

eOO

Warnings:

Information:

317943

3 Drawings-only black and white line

Drawings.pdf no 9 drawings

8e 1 b2 6b8a803 6ac4d Sa 7 5 03 bfOa 75 21289f 01321

Warnings:

Information:

156533

4 P882PA.pdf yes 25 44e37a7971 ed4ccc69ae201 dgec3923aa61

07078

Multipart Description/PDF files in .zip description

Document Description Start End

Specification 1 19

Claims 20 24

Abstract 25 25

Warnings:

Information:

35898

5 Fee Worksheet (PTO-875) fee-info.pdf no 2 40b9743f03edfb668cbf343835eaed938a7d

cf16

Warnings:

Information:

Total Files Size (in bytes) 1921894

This Acknowledgement Receipt evidences receipt on the noted date by the USPTO of the indicated documents, characterized by the applicant, and including page counts, where applicable. It serves as evidence of receipt similar to a Post Card, as described in MPEP 503.

New Applications Under 35 U.S.c. 111 If a new application is being filed and the application includes the necessary components for a filing date (see 37 CFR 1.53(b)-(d) and MPEP 506), a Filing Receipt (37 CFR 1.54) will be issued in due course and the date shown on this Acknowledgement Receipt will establish the filing date of the application.

National Stage of an International Application under 35 U.S.c. 371 If a timely submission to enter the national stage of an international application is compliant with the conditions of 35 U.S.c. 371 and other applicable requirements a Form PCT/DO/EO/903 indicating acceptance of the application as a national stage submission under 35 U.S.c. 371 will be issued in addition to the Filing Receipt, in due course.

New International Application Filed with the USPTO as a Receiving Office If a new international application is being filed and the international application includes the necessary components for an international filing date (see PCT Article 11 and MPEP 1810), a Notification of the International Application Number and of the International Filing Date (Form PCT/RO/l 05) will be issued in due course, subject to prescriptions concerning national security, and the date shown on this Acknowledgement Receipt will establish the international filing date of the application.

PTO/SB/06 (12-04) Date: 10/20/10 Approved for use through 713112006. OMB 0651-0032

U.S. Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE Under the Paperwork Reduction Act of 1995 no persons are required to respond to a collection of information unless it displays a valid OMB control number

, PATENT APPLICATION FEE DETERMINATION RECORD Application or Docket Number

Substitute for Form PTO-875 . 12/908,161 APPLICATION AS FILED - PART I OTHER THAN

(Column 1) (Column 2) SMALL ENTITY OR SMALL ENTITY

FOR NUMBER FILED NUMBER EXTRA RATE ($) FEE ($) RATE ($) FEE ($)

BASIC FEE N/A N/A N/A 82 N/A (37 CFR 1.16(a), (b), or(c» SEARCH FEE N/A N/A N/A 270 N/A (37 CFR 1.16(k), (i), or (m» EXAMINATION FEE N/A N/A N/A 110 N/A (37 CFR 1.16(0), (P), or (q» TOTAL CLAIMS 18 X$26= X$52 (37 CFR 1.16(i)) minus 20 = OR INDEPENDENT CLAIMS 5 .. 2 X$110= 220 X$220= (37 CFR 1.16(h» minus 3 =

If the specification and drawings exceed 1 00 APPLICATION SIZE sheets of paper, the application size fee due is FEE $270 ($135 for small entity) for each additional

(37 CFR 1.16(5» 50 sheets or fraction thereof. See 35 U.S.C. 41 (a)(l)(G) and 37 CFR

MULTIPLE DEPENDENT CLAIM PRESENT (37 CFR 1.160» N/A N/A

* If the difference in column 1 is less than zero, enter "0" in column 2. TOTAL 682 TOTAL

APPLICATION AS AMENDED - PART II OTHER THAN

(Column 1) (Column 2) (Column 3) SMALL ENTITY OR SMALL ENTITY

CLAIMS HIGHEST ADDI- ADDI-REMAINING NUMBER PRESENT

c:( AFTER PREVIOUSLY EXTRA RATE ($) TIONAL RATE ($) TIONAL I- AMENDMENT PAID FOR FEE ($) FEE ($) Z w Total

* Minus ** = OR :I!! X X = C (37 CFR 1.16(i)) Z Independent

* Minus *** = X = X = W (37 CFR 1.16(h» OR :I!! c:( Application Size Fee (37 CFR 1.16(s»

FIRST PRESENTATION OF MULTIPLE DEPENDENT CLAIM (37 CFR 1.160)) N/A OR N/A

TOTAL TOTAL ADD'T FEE OR ADD'T FEE

(Column 1) (Column 2) (Column 3) OR

CLAIMS HIGHEST ADDI- ADDI-REMAINING NUMBER PRESENT

III AFTER PREVIOUSLY EXTRA RATE ($) TIONAl RATE ($) TIONAL I- AMENDMENT PAID FOR FEE ($) FEE ($) Z w Total OR :I!! * Minus ** = X = X = C (37 CFR 1.16(i)) Z Inaependent

* *** W Minus = X = X = :I!! (37 CFR 1.16(h)) OR c:( Application Size Fee (37 CFR 1.16(s»

FIRST. PRESENTATION OF MULTIPLE DEPENDENT CLAIM (37 CFR 1.16(j)) N/A OR N/A

TOTAL TOTAL ADD'T FEE OR ADD'T FEE

* If the entry in column 1 is less than the entry in column 2, write 'a' in column 3. *. If the 'Highest Number Previously Paid For" IN THIS SPACE is less than 20, enter '20'. ... If the 'Highest Number Previously Paid For" IN THIS SPACE is less than 3, enter '3' .

The 'Highest Number Previously Paid For" (Total or Independent) is the highest number found in the appropriate box in column 1.

ThiS collection of Information IS reqUIred by 37 CFR 1.16. The information IS reqUIred to obtain or retain a benefit by the public which IS to file (and by the USPTO to process) an application. Confidentiality is governed by 35 U.S.C. 122 and 37 CFR 1.14. This collection is estimated to take 12 minutes to complete, including gathering, preparing, and submitting the completed application form to the USPTO. Time will vary depending upon the individual case. Any comments on the amount of time you require to complete this form and/or suggestions for reducing this burden, should be sent to the Chief Information Officer, U.S. Patent and Trademark Office, U.S. Department of Commerce, P.O. Box 1450, Alexandria, VA 22313-1450. DO NOT SEND FEES OR COMPLETED FORMS TO THIS ADDRESS. SEND TO: Commissioner for Patents, P.O. Box 1450. Alexandria, VA 22313·1450.

If you need assistance in completing the form. call1·80()''PTO·9199 and select option 2.

Image Sharpness

Documents

Transcript of Image Sharpness