CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The...

25
CCD Image Processing Prototyping Platform MDSP Group Senior Project Design Report Saar Drimer [email protected] 4/5/2002

Transcript of CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The...

Page 1: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

CCD Image Processing Prototyping Platform MDSP Group

Senior Project Design Report

Saar Drimer

[email protected] 4/5/2002

Page 2: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

2

Table of Contents

1. Introduction ................................................................................................................................ 4

1.1 General Info ........................................................................................................................... 4 1.2 General Overview .................................................................................................................. 4 1.3 General Requirements........................................................................................................... 5 1.4 Required Parts ....................................................................................................................... 5

2. Timeline ...................................................................................................................................... 5

2.1 Projected Timeline ................................................................................................................ 5 2.2 Actual Timeline..................................................................................................................... 6

3. Design Parts................................................................................................................................ 6

3.1 MicroController (µC) ............................................................................................................. 6 3.2 CCD (Charge-Coupled Device) ............................................................................................ 6 3.3 Analog Front End (AFE) ...................................................................................................... 7 3.4 Altera PLD............................................................................................................................. 7 3.5 Memory ................................................................................................................................. 7 3.6 MAX333 SPDT Switch .......................................................................................................... 8 3.7 Discrete Parts ........................................................................................................................ 8 3.8 Miniature Lens and PCB Mount........................................................................................... 8

4. System Design............................................................................................................................. 8

4.1 Power, Currents and Loads ................................................................................................... 8 4.1.1 Power Supply Unit .......................................................................................................... 8 4.1.2 Powering the CCD.........................................................................................................10 4.1.3 Current ...........................................................................................................................10 4.1.4 Capacitive Loads............................................................................................................10 4.1.5 Power Dissipation..........................................................................................................10 4.1.6 CCD Output Emitter Follower ......................................................................................10

4.2 Interface Design................................................................................................................... 11 4.2.1 RS232 Port and Transceiver........................................................................................... 11 4.2.2 Altera Flex10K Programming Port ................................................................................ 11 4.2.3 Enhanced Parallel Port (EPP)....................................................................................... 11

5. Dataflow, Controls and Waveforms ...........................................................................................12

5.1 Data Events Description ......................................................................................................12 5.1.1 Take Image ....................................................................................................................12 5.1.2 Reset CCD......................................................................................................................13 5.1.3 Integration Time............................................................................................................13 5.1.4 Parallel Transfer.............................................................................................................13 5.1.5 Serial Readout ................................................................................................................14 5.1.6 AFE operations ..............................................................................................................14 5.1.7 PLD Read.......................................................................................................................14 5.1.8 Memory Store.................................................................................................................14

5.1.9 Data Read from Memory and Transferred to µµµµC ..........................................................15 5.2 Putting it All Together .........................................................................................................16 5.3 Verilog ..................................................................................................................................20 5.4 Dynamic C............................................................................................................................20

6. Board Layout and PCB Fabrication ..........................................................................................20

Page 3: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

3

7. Conclusion .................................................................................................................................21

Sources .......................................................................................................................................22 Appendix A – Attached Materials..............................................................................................22 Appendix B – Acronyms ............................................................................................................22 Appendix C – Experience Notes and Tips................................................................................23 Appendix D – End of Quarter Demonstration..........................................................................25

Page 4: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

4

1. Introduction 1.1 General Info This project is designed to be a demonstration for the Multi-Dimensional Signal Processing (MDSP) research group headed by Dr. Peyman Milanfar at UC Santa Cruz. The design is a first step towards a hardware implementation of the super-resolution algorithms and other multimedia projects. The design presented in this paper may be used as a platform for many multimedia and image processing projects. Given the large size of the programmable logic device (PLD), image-processing algorithms in hardware description languages (HDL) could be easily downloaded and verified with this

prototyping board. In addition, the processing could be done on the RabbitCore microcontroller (µC) using Dynamic C. This project was designed to accommodate portions of the super-resolution algorithms. These algorithms take a series of low-resolution images and transform them into a single high-resolution image. The images must contain moving objects with respect to time or be taken from multiple cameras with sub-pixel offset (spatial). The intention of this project is to be the first step towards a 4-sensor camera that would produce 4 images with sub-pixel differences. This report, beyond describing the end product also outlines some of my experiences as an aspiring engineer and the personal learning process, which makes it a suitable read for anyone with technical background. Appendix C has many recommendations and notes addressed to other aspiring engineers working on their first design. The bulk of the work was done over the 10 weeks of fall quarter, 2001. Also note that most of the work was invested in the design. This report contains timing diagrams and specifications of the used components: they are all taken from their respective data sheets. 1.2 General Overview Figure 1 shows an overview of the project’s components. Light hits the lens in front of the CCD and comes out as analog output, the analog front end (AFE) digitizes the data and the PLD reads and stores it in the static RAM (SRAM). The data is then transferred to the computer through the RabbitCore2000 microcontroller. Figure 1. General outline of the system

PLD

RabbitCore

SRAM

CCD

AFE

Computer

Page 5: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

5

1.3 General Requirements The first phase was to design a CCD camera that would transfer raw pixel data to a computer for processing. The project was designed to expand to the integration of 4 CCDs; some choices along the way reflect these considerations. The current super-resolution algorithms are written in Matlab and can handle 8bit black and white raw data. In the 4 CCD application, the CCDs and lenses should be sufficiently close in order to achieve the sub-pixel differences required for the super-resolution. This requirement affected the choice of CCD with respect to package dimensions. 1.4 Required Parts Sensor: low-resolution black and white sensor with square pixels. Since the super-resolution algorithm enhances the quality of the images, the sensor needs to be sufficiently low in resolution to enable one to see the improvement. The sensor should also have square pixels to make the mechanical constraints easier for the 4-sensor board design. Using a Charge-Coupled Device (CCD) rather than a CMOS sensor was considered since CCDs have bigger well sizes and are considered to have better quality and accuracy. In addition, CCDs are much more readily available than any other type of sensor. AFE: The Analog Front End is a generalized name for a device that samples, amplifies and digitizes the output of a sensor. AFE ICs often have image processing capabilities and are usually referred to as digital signal processing (DSP) chips. Some companies offer a single chip that samples, amplifies and digitizes the analog data coming out of the CCD, but the processes may be carried out by multiple function-specific chips. The current super-resolution algorithm could handle 8bit of data so the ADC (Analog to Digital Converter) should be at least of that resolution.

Controller: a microcontroller (µC) to receive, store, process and send the data to a computer.

2. Timeline 2.1 Projected Timeline The projected timeline accounted for ample time at the end of the quarter to have the board debugged and presentable. Available were 10 weeks of fall quarter and some of the summer break for initial planning. During summer break I started researching parts and defined the goals of the project. By the beginning of fall quarter I had some schematics and a basic idea of the components to be used. The following is the timeline submitted with the project proposal:

Week1: Finish the majority of design on capture. Solve power and driving problems still pending. Week2: Finish details of design; including verification with experienced professionals Week3: PCB design in Orcad + verification. Week4: Send Gerber files to fabrication. Week5: Populate board. Week6-10: microcontroller programming and troubleshooting. Begin working on Phase 2.

The next section describes what the real timeline turned out to be.

Page 6: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

6

2.2 Actual Timeline It turned out that on many issues I was overly optimistic and underestimated the scope and time it takes to complete a project of this magnitude. Almost every step took longer than expected. The critical points were dealing with design integration, learning new software (Orcad Capture and Layout, MAX+), new language (Verilog) and designing the Printed Circuit Board (PCB). One major setback was the realization that the RabbitCore2000 was not fast enough. The RabbitCore2000 module runs at a published 25MHz. However, it uses a clock-doubler, which makes the clock duty cycle uneven. Also, the fastest square wave it can produce out of its ports is of 800KHz. Unfortunately, this was realized a few weeks into the quarter. The frame transfer CCD of the design has a sensitive area and a storage area without any means of blocking incoming light. The data from the sensitive area is transferred to the storage area while the sensitive elements still collect data. If the parallel transfer is not fast enough, the pixels would collect data while being transferred and the result would be a smeared image (see section 5.1.4). Therefore, the parallel transfer should be done with the highest speed possible (TI recommends 12.5MHz); 800KHz was not fast enough for this application. The solution was to use a PLD or FPGA. These devices can be programmed to perform any logic functions through any Hardware Description Language (HDL). This addition meant that I would have to learn Verilog and significantly change the design. I decided to undertake the challenge and therefore imposed further delays on the original timeline. The new design had promising results since the process of acquiring an image would be much faster. This is done by putting the slow RabbitCore out of the loop of the data acquisition flow. The PLD would interface with a SRAM and control the CCD lines and therefore be able to complete the process at the required speed. After the data is acquired, it is transferred to the RabbitCore. The design without the PLD would have been able to acquire and store an image every 25 seconds while the PLD design could do the same every 1/25 of a second. The data transfer to the RabbitCore would still be slow, but that could be overcome by adding an enhanced parallel port (EPP) port to bypass the microcontroller altogether (see section 4.2.3).

3. Design Parts All the design’s ICs are detailed in this section. 3.1 MicroController (µC) The first design choice was the controller: the RabbitCore2000 module. It was the easiest choice since it was supported in the university’s labs and the instructor knew it well. The RabbitCore was also an easy microprocessor to work with: it is C programmable, comes in a module with memory and has an easy programmable interface. In retrospect, the RabbitCore was overkill for the functions it performed but it was excellent for prototyping purposes. 3.2 CCD (Charge-Coupled Device) There are many variations of CCD and the whole design surrounds this IC making the choice crucial. I chose a frame transfer CCD in order to not deal with a shutter. Among other CCDs that met the

requirements I chose the TC237B 680x500 7.4µm x 7.4µm pixels frame transfer CCD. This CCD was still active, ready to be sampled and TI gave it support. Sony has a good variety of CCDs but was not willing to supply free samples for student projects.

Page 7: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

7

A feature worth mentioning of the TC237 is the antiblooming control. In the case of over exposure or very bright spots in the scene, the electrons can literally overflow from the CCD element to adjacent elements. Antiblooming protection has drains that funnel these overflowing electrons so they do not affect other pixels. In the TC237, the voltage on the ODB signal controls this feature and it can be also be disabled. 3.3 Analog Front End (AFE) The main functions of the AFE in dataflow order: Sampling: sample the analog data from the CCD. Many samplers offer Correlated Double Sampling (CDS) which sample twice: once a reference level and once the data. Those samples are subtracted to eliminate noise (dark current and die inconsistency). Amplification: most AFE’s have a Programmable Gain Amplifier (PGA), which lets the user adjust the gain according to the expected lighting of the scene. ADC: Analog to Digital Converter that converts the analog input to digital output. There were many possibilities for AFEs ranging from discrete, one-function devices to single chips that contained all functions. The main companies examined were Cirrus Logic, Analog Devices, TI, Maxim and Sony. In order to minimize the chip-count, I decided to use the function-bundled ICs. In earlier stages the use of the MAX1101 IC from Maxim was considered, which lacked the CDS function but was very easy to work with. The MAX1101 had a serial output, which was good for the 4 CCD application because then there would only be a need for one port to receive data from all four. Later changes in the project made me change the MAX1101 to a more robust device: TI’s VSP2230 36MHz (max), 10bit ADC, CDS, PGA AFE. This 48pin device performs all the functions of an AFE and made by TI, which also manufactures the TC237 CCD.

3.4 Altera PLD Upon deciding to use a PLD I had the Cypress, Xilinx and Altera prototyping boards available at days notice. I decided to use the Altera because their prototyping board (UP1) and software were readily at hand. I wanted to choose a part closest to the one on the Altera prototyping board. This led to the FLEX10K100A (EPF10K100ARC240-3): a 240pin fine pitch, RQFP PLD that has 189 general purpose I/O pins and ran at suitable speeds. The FLEX10K100A has 5000 logic cells from which only 400 logic cells are going to be used for the image acquiring process, leaving the rest for image processing applications. It was a hard task finding information about the non-I/O pins and how to terminate them. The answers were found in the application note discussing the programming of Altera devices. It turns out that many of the device’s pins are dedicated to various forms of programming. Since the Passive Serial method was implemented, many of those pins were not used. The Flex10K devices can operate at large range of clocks (up to 300MHz) and the user supplies the oscillator to two dedicated pins. The introduction of the PLD enabled me some flexibility with the design and layout since any general I/O pin could be mapped to any internal signal. 3.5 Memory Since the RabbitCore2000’s on-board SRAM was no longer in use, an addition of a memory chip was required. I looked for an asynchronous SRAM because it is easy to interface with. It was also required

Page 8: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

8

that there would be as much memory as possible on a single chip. The Toshiba’s TC55V8200 3.3V, 8bit, 2MByte CMOS static RAM was a match. Choosing a 3.3V device led me to another change of design: changing Vcc to 3.3V instead of 5V. All IC’s were compatible with 3.3V so it was not a major design change. The 8bit 2MB SRAM has 2,097,152 bytes. Each image had less than 340,000 bytes of information. The

result is that this chip could hold 2MB/340,000 ≅ 6 images. 3.6 MAX333 SPDT Switch The MAX333 is a Single Pole Double Throw (SPDT) switch. It is able to deal with high voltage swings and it was used on the ODB CCD input, which controls the antiblooming protection. 3.7 Discrete Parts The design includes many discrete parts that mainly work as filters and voltage dividers. A ferrite bead was added to the digital power of the ADC in order to get consistent, stable results. 3.8 Miniature Lens and PCB Mount The CCD needs a lens and a PCB mount to attach to. The lens chosen has a focal length of 6mm and an F-stop number of 2.8. It was crucial to know which PCB mount is to be used before fabrication in order to account for the space it is about to take.

4. System Design 4.1 Power, Currents and Loads A major portion of the design is integrating the parts into a functional design. One should make sure that all devices work within their specified range and are supplied with their power needs for normal operation. 4.1.1 Power Supply Unit The system is powered from a power supply unit shown in Figure 2. Table 1 shows the power distribution for the various ICs. This design involved 7 different voltage levels that are supplied from the power supply unit. The CCD alone requires 5 voltage levels.

Page 9: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

9

Figure 2. CCD power supply unit.

IC Vcc net

CCD → ODB (CCD reset) 26V

CCD → ADB (Amp. Drain Bias) 22V

MAX333A Switch 15V

CCD → ODB 15V

EL7547 Driver → Voltage supply, OE 15V

EL7547 Driver → VH (Voltage High) 12V

CCD → SAG, SRG, IAGx 12V

CCD → SUB (substrate) 10V

MAX232 5V Altera PLD 3.3V VSP2260 AFE 3.3V TC55V8200 3.3V

Table 1. Voltage supply nets

Page 10: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

10

4.1.2 Powering the CCD The substrate of the CCD can range from 0-10V and all other voltages are scaled to it. Choosing a

substrate of 10V to avoided negative voltages. In order to reset the CCD a 1µS pulse of 26V is required through the ODB pin, which normally supplies 15V for the antiblooming protection. For that switching function the MAX333 SPDT was used. The other control signals needs to be 2V above the substrate voltage (12V in this design) and they are supplied through the EL7457 high-speed CMOS switch. The ADB, which is the power- supply of the CCD needed to be 22V when using a 10V substrate. 4.1.3 Current The 15V net draws the most current since it supplies power to the device that drives the high input capacitance CCD pins. The RabbitCore2000 draws maximum of 130mA and the rest of the devices range from 10mA to 30mA. 4.1.4 Capacitive Loads The microprocessor and PLD can drive very small capacitive loads, in the order of up to 100pF. The TC237B CCD has 4 inputs that are 2 x 2000pF and 2x 4000pF. There was a need for a driver that would supply the current to drive these capacitive loads. After extensive research, the solution was found in Elantec’s EL7457C 40MHz 2Amp Quad CMOS driver. In order to drive the CCD 2 such devices were needed. However, the combination of high speed, high capacitance and large swings results in very high power consumption/dissipation described in the next section. 4.1.5 Power Dissipation Components have maximum ratings that indicate the threshold of proper operation without damage to the chip. As the power dissipation rises, the chip’s temperature rises and the designer must take care that the temperature will not exceed the maximum ratings. The EL7457C CMOS driver has a maximum power dissipation rating of 0.8 Watt. The application needed much more as shown below: V: Voltage swing (12V) f : Frequency (12.5MHz) C : Capacitance (4000pF, 2000pF) PD: Power Dissipation PD = C x V2 x f = (2000pF + 4000pF) x 122 x 12.5MHz = 10.8W That meant that these devices would dissipate power 12 times their maximum rating. Realizing that Watt is defined per second meant that if the device is not operating the whole second then there would be less power dissipated. Calculating the duty cycle of the driver showed that they would only be used for a fraction of a second, which would dramatically decrease the power dissipation and therefore the IC is suitable for this application. 4.1.6 CCD Output Emitter Follower The CCD output can drive a maximum capacitive load of 6pF. The input capacitance of the AFE is two 5pF capacitors in parallel, which make an effective capacitance of 10pF. Although marginal, an emitter follower was added in order to supply the needed current. The form of the emitter follower is a standard one and can be seen in the design schematics.

Page 11: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

11

4.2 Interface Design This section discusses how parts are programmed and interface with the outside world. 4.2.1 RS232 Port and Transceiver The RabbitCore daughter board needs an RS232 transceiver to communicate with the computer’s serial

port. The function of the transceiver is to convert the TTL 5V signals into ±10V signals in order for them to travel longer distances through the serial cable. The Maxim’s MAX233A was chosen since it does not require external capacitors like the MAX232. A 10-pin port for the RS232 cable was also added. 4.2.2 Altera FLEX10K Programming Port There are several ways to program the Altera Flex10K. The scheme implemented on the UP1 prototyping board was a good choice: passive serial programming (PS) with the ByteBlasterMV programming cable. It involves several control lines and one programming line. Figure 3 shows the PS programming scheme.

Figure 3. Passive Serial programming using the ByteBlasterMV [2] Each line is pulled up and extra care needs to be taken so the lines are not noisy. Altera warns that noisy line would result in programming failure. 4.2.3 Enhanced Parallel Port (EPP) One of the last things that were added to the board was an Enhanced Parallel Port (EPP). The reason was that transferring the pixel data to the computer through the RabbitCore microprocessor would take 23.6 seconds for every frame using the RS232: Transfer speed: 115200 bits per second

Page 12: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

12

Bits: 680x500x8 = 2720000 Seconds per image = 2720000 / 115200 = 23.6S This was unacceptable since the PLD could read more than 20 frames per second from the CCD. The EPP was a suitable solution providing up to 5Mbits per second of transfer rate. It was easy to implement in hardware and its handshaking scheme is fairly simple. For more information about EPP protocol refer to http://www.beyondlogic.org/epp/epp.pdf.

5. Dataflow, Controls and Waveforms This section describes the data path through the design. 5.1 Data Events Description This section describes the image acquiring steps and the data transfer to the microcontroller. For all CCD operation please review Figure 4.

Figure 4. Progressive-Scan Timing with Single Register Readout (TC237) 5.1.1 Take Image

µC raises the TAKE_IMAGE signal to instruct the PLD to start the image taking process.

Page 13: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

13

5.1.2 Reset CCD The CCD needs to be reset from all previous data and clean the slate for the next image. Raising ODB

signal to 26V for 1µS resets the CCD. 5.1.3 Integration Time The integration time is the length of exposure. A short integration time would result in less photons hitting the CCD elements and may yield an underexposed image. On the other hand, long exposure times would yield a bright image and may result in an overexposed image. Long integration times are used in very poorly lighted scenes like in astronomical photography. For this design capturing the scenes in a

controlled environment, 1µS to 30µS is typical. 5.1.4 Parallel Transfer This process takes the data from the sensitive area to a non-sensitive storage/memory area; hence, frame-transfer. As described before, this process needs to be done as fast as possible (recommended frequency of 12.5MHz) to avoid smear. For completeness, the following is a calculation of the smear percentage [1]: % smear = 10 * (tXFER / tINT) Where, tXFER = nIA / fXFER tXFER : time of parallel transfer

tINT: integration time (1µS - 30µS) nIA : number of image lines fXFER : parallel transfer frequency In this application: % smear = 0.25% The parallel Transfer is done a series of pulses of IAG1, IAG2, SRG and SAG as shown in Figure 4. Once all 500 rows were transferred the serial readout begins. After a single row has been read out from the serial readout register a pulse of SAG followed by SRG would transfer the next line to the readout serial buffer.

Page 14: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

14

Figure 5. Sensitive, storage and serial readout buffers of the TC237 CCD sensor 5.1.5 Serial Readout As shown in Figure 5, the CCD has 2 serial output registers. The design uses only one (OUT2, the lower register). The first 4 pixels of each line are called dummy pixels and are ignored. The next 22 are the black reference pixels, which tell the processor what is the actual black in the particular scene. Theses black reference pixels are not exposed to light and therefore are a good black reference. The next 658 pixels are read and processed. 5.1.6 AFE operations The output frequency of the AFE is determined by the SHP and SHD signals (refer to Figure 6 and the VSP2230 datasheet). In this diagram SHP and SHD are active low but the AFE could be programmed to make them active high. As mentioned before, the Correlated Double Sampling (CDS) samples a reference voltage, the data voltage and subtracts them to get an interference free sample. SHP samples the reference voltage and SHD samples the data. The output of the CDS is then amplified and digitized. The data is ready after tDO from the falling edge of SHD. ADCCK is an optional internal clock that is not used. In addition, this AFE outputs 10bit data, this design used the 8 most significant bits. 5.1.7 PLD Read The PLD reads the data from the AFE output. 5.1.8 Memory Store The data read from the AFE is stored onto the 2MB SRAM. The timing for the write cycle is shown in Figure 7. Note that nCE is fixed low in this design.

Page 15: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

15

Figure 6. AFE CDS timing diagram. Output frequency is determined by SHD and SHP

Figure 7. Write cycle timing for Toshiba’s TC55V8200 The TC55V8200 has 10ns access time, which is much less than needed. Therefore, there is no need to worry about data not being available on time.

5.1.9 Data Read from Memory and Transferred to µµµµC The read operation is fairly simple: the PLD reads data bytes from the memory starting at the base address.

Page 16: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

16

Figure 8. Read cycle timing for Toshiba’s TC55V8200

Figure 9. µC-PLD handshaking and data bus

Figure 9 shows the handshaking scheme between the PLD and the Rabbit µC. It is a simple asynchronous process to allow the systems that run at different clock frequencies to communicate. The process:

1. The µC raises uC_RTR (Ready To Receive) and waits for DATA_RDY 2. PLD acknowledges uC_RTR and puts data on the data bus 3. PLD raises DATA_RDY and waits for uC_RTR.

4. µC acknowledges DATA_RDY, lowers uC_RTR and reads data from data bus 5. Back to step 1.

The process ends when PLD raises DONE. 5.2 Putting it All Together The previous section described all the steps of taking one image and transferring the data to the microcontroller. This section will describe how everything is put together since some processes are done in a pipeline manner rather than sequentially.

PLD uC_DATA

DATA_RDY

uC_RTR

µC

PORT A

PLD_DATA_RDY

uC_RTR

8

Page 17: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

17

Figure 10. FSM diagram of the image taking process Figure 10 shows the Finite State Machine (FSM) of the process of taking one image. The following is the outline of each state (at this point you may want to look through the verilog code):

a. Reset CCD: ODB high for 1µS

b. Integration time: delay 1µS to 30µS c. Parallel transfer (2alternating states in the verilog code):

d. Clock line down x 500 (lines) > SAG, SRG (see Figure 4) begin

e. Pipeline process x 680 (pixels) begin

1. Clock pixel data from serial register 2. Sample 3. Digitize 4. PLD read 5. Memory store

end end

f. Transfer data to µC x (680x500) begin

1. Memory read

2. µC read end

g. Done

a

b

c

d e

f

g

680

500

340K

Page 18: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

18

Figure 11. Parallel transfer simulation; note the switching of states (MAXII+)

Figure 12. Simulation of handshaking and data transfer between µC and PLD (MAXII+)

Page 19: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

19

State ‘d’ is the most complicated one and involves many control signals. Figure 13. State ‘d’ pipeline: clock pixel out, sample twice and store digitized data in memory

Figure 15. Simulation of state ‘d’ (MAX+)

1 2 3 4 5 6 7 8 1 …

MEM

AFE

CCD

SRG

RST

SHP

SHD

ADDR

nWR

store AFE

Page 20: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

20

5.3 Verilog I have never programmed Verilog and thought this quarter is a good time to start. It took a while to get used to the parallel nature of HDL programming but once that was out of the way it was much easier. The attached Verilog code contains several finite state machines (FSM); it controls the various clocking signals and reads/writes to memory and communicates with the RabbitCore2000 microcontroller through a simple handshake scheme. 5.4 Dynamic C The microcontroller was at the top of the design, starting and ending processes. The programming cable of the RabbitCore daughter board contains an RS232 transceiver, which is used to communicate with the computer. The code is simple and in large contains the handshaking scheme and major controls to tell the PLD to start taking images. I used PORT A as the data port and PORT D as the general purpose control lines. PORT D is convenient since every bit of the port can be addressed and controlled individually. The following is PORT D’s pin assignment:

IN port D0 : PLD_DONE (PLD > uC) IN port D1 : PLD_DATA_RDY (PLD > uC) OUT port D4 : uC_TAKE_IMAGE (uC > PLD) OUT port D5 : uC_RTR (Ready To Receive, uC > PLD)

6. Board Layout and PCB Fabrication The board layout was to be fabricated and populated for the final demonstration. I started the task as soon as all the design details worked out and all the parts were on hand. For some time throughout the layout I was debating a 2-layer or 4-layer board. A 4-layer board means that there are 2 layers of routing, one layer for Vcc and one for GND. 4 layer boards are more stable since there are no long traces to ground and power, therefore eliminating daisy chaining and power distribution affects. On the other hand, 4-layer boards are more expensive. I finally decided on a 2-layer board with many of it covered with copper pours (areas which cover the surface of the board and are connected to a single net, typically Vcc or GND). Since the switching of the control and data signals would not exceed 20MHz, there was no emphasis on high-speed affects to the functionality of the design. Upon the completion of the layout, the Gerber files were produced and examined. They were then sent to fabrication; I received the boards the following week. I met with fellow engineers to populate the board. We made initial checks and started the population process. After soldering the major components it turned out that the 3.3V net was shorted. Debugging started and yielded no results. We started de-soldering the parts and found out that the problem was with the fabrication. The board had other issues, none of which had to do with the circuit design (see Appendix C and D). I decided later that week to concentrate on showing what I have using the prototyping boards and working on the second revision of the board after the quarter is done. The time pressure influenced this decision: if I had concentrated on fixing the problems of the board and ordered another revision it would arrive only days before the deadline. By now I knew that populating and debugging of the board takes at-least days. This situation had the potential of leaving me empty handed: no board and nothing to demo but simulated code.

Page 21: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

21

The goal for CE123 adjusted to having everything ready except for the PCB. This would satisfy the course requirements since PCB design is not part of the overall evaluation. Appendix C.5 describes in more detail personal experiences with laying out the board and dotted with recommendation to the beginner.

Figure 16. The PCB layout. Copper pours are not shown

7. Conclusion The design of this project is complete but has not yet been translated into working hardware. This has some advantages since in retrospect I would have chosen different paths and parts. I regard the microcontroller as a “nice-to-have” device; it was considered to be the main processor in early stages and was eventually replaced by the PLD. The reason it stayed in the design was because it was required under CE123 project class. Another issue is that the TC237 CCD needs at least 5 different voltage levels which complicate the design. In addition, its very high capacitive loads coupled with the high voltage swings made it far from perfect for this project. On a personal level this project was a major undertaking and through the many hours of hard work I have learned many things that would undoubtedly make me a better engineer. I am proud of this design and work along with a sense of disappointment that it has not been translated into working hardware. The layout and fabrication of the PCB was a major task. I am glad for this attempt since I was self taught and made errors that will not be repeated and therefore made the best out of these mistakes. As a learning experience this project cannot compare to any previous university activity I undertook. It made me confident in my abilities and talents as an engineer. I would like to thank the following people and companies for their valuable support and knowledge: Prof. Peyman Milanfar, Prof. Cyrus Bazeghi, Prof. Stephen Peterson, Dirk Robinson, Philip Hedges and Joe Downs. Altera, Maxim, Texas Instruments, Toshiba, Elantec, Analog Devices, Cirrus Logic and Philips Semiconductors.

Page 22: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

22

Sources [1] SOCA012, “TC237” Timing of Frame Transfer CCD Image Sensor”, Texas Instruments, June 1996. [2] “ByteBlasterMV Parallel Port Download Cable”, Altera, Nov. 2001 version 3.2 Appendix A – Attached Materials

1. Project notebook: this notebook was maintained throughout the project and contains all intermediate data that preceded this report. The notebook also contains first revisions of the design.

2. Data Sheets: data sheets of major IC’s of the design (EL7457, TC237, MAX333, VSP2230, TC55V8200). Complete data sheets are included in order for the reader to get a broader view of the design. Many small details of putting the parts together were not stressed in this report.

3. Design Revisions: all design revisions are included in the folder and notebook. 4. Verilog Code: the complete, working code for the Altera PLD. 5. Dynamic C Code: the complete, working code the RabbitCore2000. 6. PCB: the actual fabricated PCB. 7. Bill of Materials (BOM): contains all information about the IC’s and discrete parts in the

design and their reference IDs. 8. Altera Flex10K Pin Assignments: A table containing all the functional pins of the Flex10K

chip in the design. 9. Contact List: people I have been in touch with throughout the project.

Appendix B - Acronyms ADC. Analog to Digital Converter ADB. Amplifier Drain Bias BOM. Bill Of Materials CCD. Charge Coupled Device CDS. Correlated Double Sampling CMOS. Complimentary Metal-Oxide Semiconductor DIL. Dual In-Line DSP. Digital Signal Processing EPP. Enhanced Parallel Port FPGA. Field Programmable Gate Array FSM. Finite State Machine HDL. Hardware Description Language IAG. Image Area Gate IC. Integrated Circuit PCB. Printed Circuit Board PGA. Programmable Gain Amplifier PLD. Programmable Logic Device QSOP. Quarter Size Outline Package or Quality Small Outline Package SO. Small Outline SPDT. Single Pole Double Throw SRAM. Static Random Access Memory SRG. Serial Register Gate

Page 23: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

23

Appendix C – Experience Notes and Tips The following are tips and lessons I have learned through the progression of this project. It is intended for the engineers that attempt their first design; in a sense, these are notes I wish were available to me when I first started this project. C.1 Timeline

- When planning a project, at least double the time you think it’s going to take. - Engineers are typically optimistic when it comes to estimation of task difficulty.

- The learning curve of new software is steep, therefore takes much longer than one might think.

- Always account for unexpected complications/changes; they are bound to happen. C.2 Parts Choice Criteria Finding the right part for your design can save you a lot of time. If you need to replace a part after your design is complete, you will be set back significantly. Ample preparation and research is crucial for time sensitive designs like university projects. I created a criteria checklist that I followed for every chip considered. The list is ordered by importance; if a step fails, the next part is examined. The following is the criteria list and a short description of how to progress through the steps based on my experience:

a. Function: does the part fulfill all requirements? Examine datasheet closely, looking at every detail. To avoid future complications, it is best to read the whole data sheet and not skimp on the details because that is where parts fail to match the design. At this stage one needs to check if the device is still active. Discontinued parts would be harder to get support for and they are probably not sampled anymore.

b. Availability: is the part available in the relevant time period (i.e. now)? Now that you know the part you want you may either call the company/supplier/representative/distributor to check if the part is available. Beware of checking availability on-line; some companies do not update their web pages as often as they should.

c. Sampling: is the part available for sampling? First, ask if the part can be sampled, if it can, ask for more parts than you actually need incase something goes wrong (it will!). If the part cannot be sampled, ask if you may buy the low quantity you need. If no samples are provided (some companies do not provide samples to student projects, like Sony) and low quantity purchase is not available, move on to your next part choice.

C.3 Schematic Design Rules Schematics makes one realize the details of putting things together. I used Orcad Capture to lay out the design from the schematics capture tools. There are tens of revisions to the schematics due to the constant changing of the design. For each device and trace line, one should check the following: Device:

1. How much power is dissipated in the device and is it within maximum rating? 2. How much current does the device draw? Can the power supply handle this current?

Page 24: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

24

3. What is the maximum frequency the device operates at? Line/Wire:

1. Do the pins match? 2. Does the driving pin have enough power to drive the output (input/output capacitance and resistance)? 3. Do the voltage levels match (TTL/CMOS for example)? 4. How much power is dissipated through that line? How much current is flowing through the line?

For every new trace I drew and for every device added, I went through the above checklist. It helped making the devices work with each other and the whole design stable. C.4 Verilog The first concept to understand with regards to Verilog is that everything happens in parallel and the decision-making is controlled by simple or more elaborate if statements that are evaluated on every chosen signal cycle (usually, the clock). In order to figure out what a code does one should go from left to right and look for which statement matches the current state of the signals. An easy form of if statements is the case statement. case allows one to evaluate a certain variable, and according to its value decide which batch of statements to perform. This is how some FSM’s are constructed. A somewhat tricky issue is the bi-directional bus or pin. Bi-directional busses can receive and send data; a memory data bus, for example. The trick is to define the bus so there is no bus contention (where two devices try to drive the same line). A bi-directional bus is defined as an inout and must be driven by a continuous statement. The continuous statement will tristate the bus whenever it is driven by the “other” device. C.5 PCB Layout This portion of the project was a serious undertaking. I learned many things through readings and research. It took a couple of weeks to get familiar with Orcad Layout since it has quite a distinct interface. Since I was not able to receive help from university faculty and outside professionals, I gathered as much information as possible and decided to do my best and learn from the mistakes. The first step was to make the footprints of the IC’s that were not in Layout’s libraries. Data sheets can be confusing and give a range of dimensions. To be on the safe side I used the larger end of the range. In order to align the pads correctly, do not attempt to adjust the grids to match your desired distance; use the padstack spreadsheet to accurately set the spacing. Also make sure that through hole footprints have their hole through all layers. During placement one should place the parts so that the critical lines are as short as possible. Long lines add resistance and inductance that are typically undesired. One should also decide weather to put parts on both sides (solder-side or/and part-side) of the board before placement. In this design’s case, I thought it would be beneficial (and cheaper) to have parts only on one side. It took me 3 to 4 placement attempts to be satisfied with the result. Connecting the nets is the major portion of the work. I suggest coloring significant nets such as busses, Vcc and ground in different colors. This enables the designer to distinctly differentiate between nets. Then, using the spreadsheets, connect nets one by one. When choosing width of traces and the space between them, consider the following:

1. Power traces should be as wide as possible.

Page 25: CCD Image Processing Prototyping Platform - Saar … Image Processing Prototyping Platform ... The first phase was to design a CCD camera that would transfer raw pixel data to a computer

25

2. What width/space ratio doe the fabrication company support? Typical width/space ration is 8/8 (in mils = thousandth of an inch).

To conclude: - PCB layout is a serious undertaking. People do layout for a living and therefore a beginner

should have expect that it would take a long time. - NO layout is perfect on first revision.

- Solder mask is very important if you use fine-pitch parts

- Inspect all through-hole layers. - Orcad Layout may delete a net or trace every now-and-then.

- Inspect the Gerber files for as thoroughly as possible. Appendix D – End of Quarter Demonstration The projects needed to be demonstrated at the end of the quarter. Since the PCB was not working, I decided to demonstrate the code and functionality without it. I connected the RabbitCore2000 prototyping board and the Altera UP1 board and added a breadboard with a 64K SRAM, 16 LED indicators and 8bit switch to simulate the output of the AFE. In this demonstration, the only thing missing is the CCD and AFE, all the rest works as it would on the board. All output signals are visible through the LEDs and the clock is divided so the observer can notice and compare the signals to what they should be. The output of this demonstration is the AFE data on the computer’s monitor, same as if the CCD is there.