troubleshooting

download troubleshooting

of 88

description

repair

Transcript of troubleshooting

  • AADvance The Next Step in Automation

    AADvance Controller

    Troubleshooting and Troubleshooting and Troubleshooting and Troubleshooting and Maintenance ManualMaintenance ManualMaintenance ManualMaintenance Manual

    Issue: 08Issue: 08Issue: 08Issue: 08

    DOCUMENT: 553634DOCUMENT: 553634DOCUMENT: 553634DOCUMENT: 553634 (ICSTT(ICSTT(ICSTT(ICSTT----RM406_EN_P)RM406_EN_P)RM406_EN_P)RM406_EN_P)

  • ii

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    This page intentionally left blank

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    iii

    NoticeNoticeNoticeNotice

    In no event will Rockwell Automation be responsible or liable for indirect or consequential damages resulting from the use or application of this equipment. The examples given in this manual are included solely for illustrative purposes. Because of the many variables and requirements associated with any particular installation, Rockwell Automation does not assume responsibility or reliability for actual use based on the examples and diagrams.

    No patent liability is assumed by Rockwell Automation, with respect to use of information, circuits, equipment, or software described in this manual.

    Reproduction of this manual in whole or in part, without written permission of Rockwell Automation is prohibited.

    All trademarks are acknowledged.

    DisclaimerDisclaimerDisclaimerDisclaimer

    It is not intended that the information in this publication covers every possible detail about the construction, operation, or maintenance of a control system installation. You should refer to your own (or supplied) system safety manual, installation instructions and operator/maintenance manuals.

    Revision and Updating PolicyRevision and Updating PolicyRevision and Updating PolicyRevision and Updating Policy

    This document is based on information available at the time of its publication; however, the document contents are subject to change from time to time. You should contact Rockwell Automation Technical Support by e-mail [email protected] to check if you have the latest version of this publication.

    Copyright Notice, Rockwell Automation 2012

    This document contains proprietary information that is protected by copyright. All rights are reserved.

    Documentation FeedbackDocumentation FeedbackDocumentation FeedbackDocumentation Feedback

    Your comments will help us to serve your documentation needs better. If you discover any errors or have any suggestions on how to improve this publication send your comments to our product support group: [email protected]

    This technical manual applies to AADvance Release 1.3.

  • iv

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    Issue RecordIssue RecordIssue RecordIssue Record

    Issue Date Comments

    01 Dec 2008 First Issue

    02 Feb2009 Update with official product titles

    03 Apr 2009 Change title and add calibration procedures

    04 Nov 2010 Add fuse replacement

    05 Mar 2011 Release 1.2

    06 Aug 2011 Update Release 1.2 version for TUV review comments

    07 Apr 2012 Update Release 1.2 to add Analogue Output Module information

    08 June 2012 Update for Release 1.3 & 1.3.1

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    v

    Notes and Symbols used in this manual Notes and Symbols used in this manual Notes and Symbols used in this manual Notes and Symbols used in this manual

    This symbol calls attention to items which "must" be considered and implemented when designing and building an AADvance controller for use in a Safety Instrumented Function (SIF). It appears extensively in the AADvance Safety Manual.

    Note: Notes are used extensively to provide important information about the product.

    Standard Warnings and CautiStandard Warnings and CautiStandard Warnings and CautiStandard Warnings and Cautionsonsonsons

    WARNING

    ELECTRICAL ARCS AND EXPLOSION RISK IN HAZARDOUS AREAS

    If you connect or disconnect wiring, modules or communications cabling while power is applied, an electrical arc can occur. This could cause an explosion in hazardous location installations. Do not remove wiring, fuses, modules or communications cabling while circuit is energized unless area is known to be non hazardous.

    Failure to follow these instructions may result in personal injury.

    WARNING

    MAINTENANCE

    Maintenance must be carried out by people who are experienced in working on electronic equipment and in particular safety related systems.They should have knowledge and experience of local operating and safety standards. Failure to follow these recommendations may result in situations that can lead system damage and even personal injury.

    CAUTION

    RADIO FREQUENCY INTERFERENCE

    Most electronic equipment is influenced by Radio Frequency Interference. Caution should be exercised with regard to the use of portable communications equipment around such equipment. Signs should be posted in the vicinity of the equipment cautioning against the use of portable communications equipment.

    CAUTION

    HEAT DISSIPATION AND ENCLOSURE POSITION

    System and field power consumption by modules and termination assemblies is dissipated as heat. You should consider this heat dissipation on the design and positioning of your enclosure; e.g. enclosures exposed to continuous sunlight will have a higher internal temperature that could affect the operating temperature of the modules. Modules operating at the extremes of the temperature band for a continuous period can have a reduced reliability.

  • vi

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    ForewordForewordForewordForeword

    This technical manual describes how to maintain, troubleshoot and repair an AADvance Controller.

    Who ShoulWho ShoulWho ShoulWho Should Use this Manuald Use this Manuald Use this Manuald Use this Manual

    This manual is for plant maintenance personnel who need to trace and repair a fault in an AADvance system and perform routine maintenance tasks. You should be trained and experienced in the operation and maintenance of electronic equipment and in particular with safety systems.

    If the information contained in this manual does not assist you to discover the cause of the failure then contact technical support at [email protected].

  • viii

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    ContentsContentsContentsContents

    Chapter 1 Chapter 1 Chapter 1 Chapter 1 Introduction to Maintenance Activities ................................................ 1-1

    Making Repairs Promptly ................................................................................................................................. 1-1

    Resolving Multiple Faults .................................................................................................................................. 1-1

    Required Tools Standard AADvance ............................................................................................................ 1-2

    Required Test Equipment ................................................................................................................................ 1-2

    Return a Module ................................................................................................................................................ 1-3

    Conventions Used in Flow Charts ................................................................................................................ 1-3

    Chapter 2 Chapter 2 Chapter 2 Chapter 2 Preventive Maintenance ......................................................................... 2-1

    Preventive Maintenance Schedule .................................................................................................................. 2-2

    Check Fuses ........................................................................................................................................................ 2-2

    Check Wiring Terminals .................................................................................................................................. 2-3

    Check Seating of Plug-in Components ......................................................................................................... 2-3

    Check Physical Condition and Environmental Conditions ...................................................................... 2-3

    Check Ground Connection ............................................................................................................................. 2-3

    Check Analogue Input Module Calibration ................................................................................................. 2-3

    Check Digital Input Module Calibration ....................................................................................................... 2-4

    Perform the Manual Test ................................................................................................................................. 2-5

    Chapter 3 Chapter 3 Chapter 3 Chapter 3 About Troubleshooting .......................................................................... 3-1

    Prerequisites for Troubleshooting ................................................................................................................. 3-1

    Recovery Mode and How it Affects Controller Operation .................................................................... 3-2

    Internal Diagnostics ........................................................................................................................................... 3-2

    Actions of the Diagnostic Systems ................................................................................................................ 3-3

    Latching and Non-Latching Faults .................................................................................................................. 3-3

    Common Fault Alarm ....................................................................................................................................... 3-4

    Fault Indications .................................................................................................................................................. 3-4

    Fault Reporting Reference Information ........................................................................................................ 3-8

    Status Indicators on the T9110 Processor Module ............................................................................. 3-8 Status Indicators on the T94xx Series Input and Output Module ................................................. 3-10 Module Shutdown State and Possible Causes ..................................................................................... 3-12 I/O Module Channel Degradation and Shutdown .............................................................................. 3-13

    Understanding the State Variable (.STA) ............................................................................... 3-15

    Correlation of LEDs with State Variable for a Digital Input ............................................................ 3-15 Correlation of LEDs with State Variable for an Analogue Input .................................................... 3-16 Correlation of LEDs with State Variable for a Digital Output ....................................................... 3-16 Correlation of LEDs with State Variable for an Analogue Output ................................................ 3-17

    View Module Firmware Versions ................................................................................................................. 3-17

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    ix

    Chapter 4 Chapter 4 Chapter 4 Chapter 4 Troubleshooting and Rectifying Module Faults .................................... 4-1

    Troubleshooting System and Module Faults ............................................................................................... 4-2

    Rectify a Critical Firmware/Hardware Failure ............................................................................................ 4-3

    Install a T9110 Processor Module ................................................................................................................. 4-3

    Upgrade a Processor Module Firmware ................................................................................................ 4-4 Install a Processor Back-up Battery ......................................................................................................... 4-4 Processor Module Start Up Process ....................................................................................................... 4-7

    Install I/O Modules .......................................................................................................................................... 4-10

    I/O Module Start Up Process .................................................................................................................. 4-10

    Chapter 5 Chapter 5 Chapter 5 Chapter 5 Troubleshooting and Rectifying Channel/Field Faults ......................... 5-1

    Examine the State Variable .............................................................................................................................. 5-2

    Start Troubleshooting Channel/Field Faults ................................................................................................ 5-3

    Diagnose a Digital Input Channel ............................................................................................................. 5-3 Diagnose an Analogue Input Channel ..................................................................................................... 5-6 Diagnose a Digital Output Channel ......................................................................................................... 5-8 Diagnose an Analogue Output Channel ............................................................................................... 5-10

    Replacing Fuses ................................................................................................................................................. 5-12

    Replace Input Channel Fuse .................................................................................................................... 5-12 Replacing Digital Output Fuses ............................................................................................................... 5-13

    Install a New Termination Assembly .......................................................................................................... 5-14

    Chapter 6 Chapter 6 Chapter 6 Chapter 6 Operation and Maintenance Plan ........................................................ 6-15

    Input Module Calibration ............................................................................................................................... 6-16

    Planned Maintenance ....................................................................................................................................... 6-16

    Field Device Maintenance .............................................................................................................................. 6-17

    Module Fault Handling .................................................................................................................................... 6-17

    Monitoring ......................................................................................................................................................... 6-18

    Maintaining Functional Safety ........................................................................................................................ 6-18

    Input Module Calibration ......................................................................................................................... 6-18 Product Level Module and Firmware Updates ................................................................................... 6-19 Baselines ....................................................................................................................................................... 6-19 Modification Records ................................................................................................................................ 6-19

    Decommissioning ............................................................................................................................................. 6-20

    Chapter 7 Chapter 7 Chapter 7 Chapter 7 Parts List .................................................................................................. 7-1

    Chapter 8 Chapter 8 Chapter 8 Chapter 8 Glossary of Terms ................................................................................... 8-1

    Chapter 9 Chapter 9 Chapter 9 Chapter 9 Additional Resources .............................................................................. 9-1

    Regional Offices .................................................................................................................................................. 9-2

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    1-1

    Corrective maintenance activities for the AADvance controller embrace troubleshooting activities and the subsequent remedial work to rectify the problem and return the controller to normal operation. In addition, some preventive testing and maintenance should be done to make sure the system is available and healthy.

    In This ChapterIn This ChapterIn This ChapterIn This Chapter

    Making Repairs Promptly .................................................................................. 1-1 Resolving Multiple Faults ................................................................................... 1-1 Required Tools Standard AADvance ............................................................. 1-2 Required Test Equipment ................................................................................. 1-2 Return a Module ................................................................................................. 1-3 Conventions Used in Flow Charts ................................................................. 1-3

    Making Repairs PromptlyMaking Repairs PromptlyMaking Repairs PromptlyMaking Repairs Promptly

    A fault indication does not necessarily mean a vital part of the controller is not operational. Some faults have no immediate consequence for example, the fault may be within one of the diagnostic systems. Nevertheless, the problem should still be rectified in a timely manner.

    System repair must be done promptly, to ensure faults do not accumulate. This principle applies as much to a redundant arrangement, which needs to ensure continued fault-tolerant operation, as to a simplex controller. Multiple failures can cause a plant shutdown.

    SAFETY

    Never allow an AADvance controller used for a safety-critical function to operate for an extended period with a failed module. Replace the module within the MTTR assumed for PFD calculations to preserve the SIL level for the system. For PFD & PFH data refer to Doc No. 553847 PFH avg and PFD avg Data

    Resolving Multiple FaultsResolving Multiple FaultsResolving Multiple FaultsResolving Multiple Faults

    The fault diagnosis procedures described in this manual will locate and resolve a single fault. If you follow one of the processes to its conclusion but a fault indication persists, one or more additional faults remain. Quarantine any defective items you have removed during your first investigation, then start again at the beginning of the appropriate procedure.

    Chapter Chapter Chapter Chapter 1111 Introduction to MaintenancIntroduction to MaintenancIntroduction to MaintenancIntroduction to Maintenance Activitiese Activitiese Activitiese Activities

  • 1-2

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    Required Tools Standard AADvanceRequired Tools Standard AADvanceRequired Tools Standard AADvanceRequired Tools Standard AADvance

    The installation and maintenance of the AADvance controller requires the following tools:

    Standard ToolsStandard ToolsStandard ToolsStandard Tools

    Screwdriver, flat 0.8 x 9.0mm (1/25 c 3/8 inch), for the module clamp screws and blanking covers

    Screwdriver, cross head number 0, for battery cover on T9110 processor module

    screwdriver, flat 0.8 x 4.0mm (1/25 x 5/32 inch), for locking screws on extension cables

    Torque screwdriver, flat 0.6 x 3.0 mm(1/40 x 1/8 inch, for dc power wiring terminals

    Torque screwdriver, flat 0.4 x 2.0mm (1/64 x 5/64), for field wiring terminals

    2 x wrench, open end, 10mm, for ground stud nuts

    Allen key (hex wrench), 2.5mm, for plug and sockets assemblies used with extension cables

    Special ToolSpecial ToolSpecial ToolSpecial Tool

    Long nosed pliers to remove the fuses on termination assemblies.

    Required Test EquipmentRequired Test EquipmentRequired Test EquipmentRequired Test Equipment

    The maintenance of the AADvance controller requires the following test equipment.

    Note: All test equipment used should be calibrated and regularly tested for accuracy. Current measurement devices should be accurate to 0.01mA.

    1) Digital voltmeter

    DC voltage range 0 to 32V or better.

    Resistance resolution 0.01 or better. 2) Current simulation instrument

    Output range 4 to 20mA with an accuracy better than 0.05mA.

    3) Bench power supply

    Output range 0 to 32V dc.

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    1-3

    Return a ModuleReturn a ModuleReturn a ModuleReturn a Module

    If you need to return a module for any reason you should in the first instance contact your regional office (see list of regional offices at the back of this publication) to obtain an a Return Material Authorization (RMA) number.

    If for some reason no one is available in your regional office you may obtain an RMA number using the following procedure:

    1) Create an empty email and set the title to 'RMA request' (without the quote marks).

    2) Send the email to [email protected].

    You will receive an automated reply which includes instructions.

    3) Follow the instructions in the email.

    Conventions UseConventions UseConventions UseConventions Used in Flow Chartsd in Flow Chartsd in Flow Chartsd in Flow Charts

    This technical manual includes flow charts. The charts use solid and broken lines to denote different kinds of activities.

  • 1-4

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    This page intentionally left blank

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    2-1

    Safety systems are designed to run continuously without manual intervention. However, some preventive maintenance should be done to make sure the system is available and healthy. This chapter describes the preventive maintenance activities for an AADvance controller.

    WARNING

    ELECTRICAL ARCS AND EXPLOSION RISK IN HAZARDOUS AREAS

    If you connect or disconnect wiring, modules or communications cabling while power is applied, an electrical arc can occur. This could cause an explosion in hazardous location installations. Do not remove wiring, fuses, modules or communications cabling while circuit is energized unless area is known to be non hazardous.

    Failure to follow these instructions may result in personal injury.

    In This ChapterIn This ChapterIn This ChapterIn This Chapter

    Preventive Maintenance Schedule ................................................................... 2-2 Check Fuses ......................................................................................................... 2-2 Check Wiring Terminals ................................................................................... 2-3 Check Seating of Plug-in Components .......................................................... 2-3 Check Physical Condition and Environmental Conditions ....................... 2-3 Check Ground Connection ............................................................................. 2-3 Check Analogue Input Module Calibration .................................................. 2-3 Check Digital Input Module Calibration ....................................................... 2-4 Perform the Manual Test .................................................................................. 2-5

    Chapter Chapter Chapter Chapter 2222 Preventive MaintenPreventive MaintenPreventive MaintenPreventive Maintenanceanceanceance

  • 2-2

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    Preventive Maintenance SchedulePreventive Maintenance SchedulePreventive Maintenance SchedulePreventive Maintenance Schedule

    Table 1: Table 1: Table 1: Table 1: RRRRecommended Schedule for Preventive Maintenanceecommended Schedule for Preventive Maintenanceecommended Schedule for Preventive Maintenanceecommended Schedule for Preventive Maintenance

    Preventive Maintenance Task Interval

    Review status LEDs Daily

    Check fuses 3 months

    Check wiring terminals 3 months

    Check seating of plug-in components 3 months

    Check for contamination, general condition and adequate environmental protection

    3 months

    Check ground connection 3 months

    Check analogue input module calibration 2 years

    Check digital input module calibration 3 years

    Perform manual test Manual Test Interval

    Note: Testing of the logic solver and its associated field devices must be carried out according to the appropriate plant or process safety analysis and the safety integrity validation. The manual test interval is used by the Probability of Failure on Demand (PFD) data analysis as part of the certification process. Refer to the document "PFH and PFD Data for AADvance Controllers" - Doc No: 553847.

    Check FusesCheck FusesCheck FusesCheck Fuses

    To check the controller fuses do the following:

    1) Open the fuse cover on each termination assembly to inspect the fuses.

    2) Look for signs of overheated, damaged or incorrectly seated fuses.

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    2-3

    Check Wiring TerminalsCheck Wiring TerminalsCheck Wiring TerminalsCheck Wiring Terminals

    To check the wiring terminals do the following:

    Inspect the field, power and network wiring and look for any evidence of physical stress such as chafing.

    Tighten terminal screws to avoid open circuits causing trips.

    Note: For controller power and field wiring terminals, apply a minimum tightening torque of 0.5 Nm (0.37 ft lb) to the terminal screws. For serial connections and the processor alarm, apply a minimum tightening torque of 0.22 Nm (0.16 ft lb) to the terminal screws.

    Check Seating of PlugCheck Seating of PlugCheck Seating of PlugCheck Seating of Plug----in Componentsin Componentsin Componentsin Components

    To check the seating of plug-in components, do the following:

    Examine the AADvance controller assemblies and make sure all plug-in items, including cable assemblies and base unit bus connectors, are correctly seated.

    Check Physical Condition and Environmental ConditionsCheck Physical Condition and Environmental ConditionsCheck Physical Condition and Environmental ConditionsCheck Physical Condition and Environmental Conditions

    To check the physical condition of the controller and the environmental conditions, do the following:

    Examine the AADvance controller assemblies for contamination, corrosion, dampness and dust.

    Look for unauthorised modifications and observable deterioration such as missing covers, open wires and damaged insulation.

    Make sure that local ventilation and air conditioning systems are operating correctly.

    ChChChCheck Ground Connectioneck Ground Connectioneck Ground Connectioneck Ground Connection

    To check the controller ground connection, do the following:

    Measure the resistance of the connection to the ground stud on the 9100 processor base unit. It should be less than 0.2.

    Check Analogue Input Module CalibrationCheck Analogue Input Module CalibrationCheck Analogue Input Module CalibrationCheck Analogue Input Module Calibration

    The AADvance controller detects possible calibration drift by continually checking its measured input values. The controller uses diverse hardware to compare two measurements. It is recommended that you carry-out this calibration drift check every two years. You can verify that an analogue input is within the stated accuracy (0.05mA) without taking a module out of service.

    To check the analogue input module calibration, do the following:

    1) Use the AADvance Workbench to lock the input channel.

    The present input value freezes, allowing the process to continue operating.

  • 2-4

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    2) Disconnect the field device at the termination assembly and connect a calibrated current simulation instrument in its place.

    3) Set the current simulation instrument to provide 4mA, verify that the input value is in the range 3.95 to 4.05mA.

    4) Set the current simulation instrument to provide 12mA, verify that the input value is in the range 11.95 to 12.05mA.

    5) Set the current simulation instrument to provide 20mA, verify that the input value is in the range 19.95 to 20.05mA.

    6) Disconnect the current simulation instrument and reconnect the field device.

    Note: Apply a minimum tightening torque of 0.5 Nm (0.37 ft lb) to the terminal screws.

    7) Verify that the field device is reading an acceptable value.

    8) Unlock the input channel.

    The input is back in service.

    If desired, you can include this calibration check in the proof test for the loop instead. To do this, insert a calibrated current meter in series with the field device in step 2; use the field device to drive the input.

    Check Digital Input Module CalibrationCheck Digital Input Module CalibrationCheck Digital Input Module CalibrationCheck Digital Input Module Calibration

    The AADvance controller detects possible calibration drift by continually checking its measured input values. The controller uses diverse hardware to compare two measurements. It is recommended that you carry-out this calibration drift check every two years. You can verify that a digital input is within the stated accuracy (0.5V) without taking a module out of service.

    To check the digital input module calibration, do the following:

    1) Use the AADvance Workbench to lock the input channel.

    The present input value freezes, allowing the process to continue operating.

    2) Disconnect the field device at the termination assembly and connect a bench power supply and a calibrated digital voltmeter in its place.

    3) Set the bench power supply to provide 2V, verify that the input value is in the range 1.5 to 2.5V.

    4) Set the bench power supply to provide 16V, verify that the input value is in the range 15.5 to 16.5V.

    5) Set the bench power supply to provide 30V, verify that the input value is in the range 29.5 to 30.5V.

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    2-5

    6) Disconnect the test equipment and reconnect the field device.

    Note: Apply a minimum tightening torque of 0.5 Nm (0.37 ft lb) to the terminal screws.

    7) Verify that the field device is reading an acceptable value.

    8) Unlock the input channel.

    The input is back in service.

    Perform the Manual TestPerform the Manual TestPerform the Manual TestPerform the Manual Test

    The manual test checks for hidden failures of components which the AADvance controller alarms cannot reveal. To perform the manual test do the following:

    Transition each digital input to its opposite state and then back to its current state. Subject each analogue input to its full range (minimum to maximum) and check accuracy.

    At the same time, check each output operates as expected. Use the application software to force any outputs which cannot be seen to operate.

    Carry out a manual test to exercise every input and output.

  • 2-6

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    This page intentionally left blank

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    3-1

    Troubleshooting covers system faults, module and channel faults, termination assemblies and field fault. The corrective troubleshooting and maintenance regime is founded on a series of automatic diagnostic systems, fault warning through module status indicators (LEDs) or reported to the application through the Workbench and a principle of repair by replacement. All modules are replaceable.

    In a safety critical redundant configuration remove only one module at a time unless a system shutdown is planned.

    In This ChapterIn This ChapterIn This ChapterIn This Chapter

    Prerequisites for Troubleshooting ................................................................. 3-1 Recovery Mode and How it Affects Controller Operation ..................... 3-2 Internal Diagnostics ............................................................................................ 3-2 Actions of the Diagnostic Systems ................................................................. 3-3 Latching and Non-Latching Faults ................................................................... 3-3 Common Fault Alarm ........................................................................................ 3-4 Fault Indications .................................................................................................. 3-4 Fault Reporting Reference Information ......................................................... 3-8 Understanding the State Variable (.STA) ............................... 3-15 View Module Firmware Versions .................................................................. 3-17

    Prerequisites for TroubleshootingPrerequisites for TroubleshootingPrerequisites for TroubleshootingPrerequisites for Troubleshooting

    The troubleshooting procedures described in this manual make the premise that the System Healthy alarm is connected to a variable in the AADvance Workbench and the alarm can be used as a starting point for activities or, the processor module has defaulted to the Recovery Mode because of a critical firmware failure.

    The fault finding procedures make the following assumptions:

    The controller was fully operational before the fault arose.

    A serviceable spare module is available.

    There is a working network connection between the computer workstation and the AADvance controlle.r

    Note: You must fit the Program Enable Key (supplied with the T9100 processor base unit) in order to download application software onto the controller.

    Chapter Chapter Chapter Chapter 3333 About TroubleshootingAbout TroubleshootingAbout TroubleshootingAbout Troubleshooting

  • 3-2

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    Recovery Mode and How it Affects Controller OperationRecovery Mode and How it Affects Controller OperationRecovery Mode and How it Affects Controller OperationRecovery Mode and How it Affects Controller Operation

    Recovery ModeRecovery ModeRecovery ModeRecovery Mode

    Recovery Mode is a shutdown mode and uses a base level firmware. It is entered automatically when a critical firmware failure occurs or it can be entered manually by pressing the processor Fault Reset button immediately after the module has booted up.

    As an alternative firmware version it allows the following maintenance activities:

    Update the firmware using the ControlFLASH utility

    Program the processor IP Address with the AADvance Discover utility

    Extract diagnostic information

    In Recovery Mode the Ready, Run, Force and Aux LEDs go Amber and the Healthy and System Healthy LEDs stay Green. The System Healthy and Healthy LEDs may go Red if a fault is detected while in the Recovery Mode.

    Note: When in Recovery Mode the I/O communications are disabled and the Application code is not running.

    Internal DiagnosticsInternal DiagnosticsInternal DiagnosticsInternal Diagnostics

    The AADvance controller contains comprehensive internal diagnostic systems to identify faults that develop during operation and raise appropriate alarm and status indications. The diagnostic systems run automatically and check for system faults associated with the controller, and field faults associated with field I/O circuits.

    Serious problems are reported immediately, but faults on non-essential items are filtered to avoid spurious alarms. The diagnostic systems monitor such non-essential items only periodically, and need a number of occurrences of a potential fault before reporting it as a problem.

    The diagnostic systems use simple LED status indications to report a problem. The LED indications identify the module and can also identify the channel where the fault has occurred. There is also a summary system healthy indication for the whole controller.

    The application software uses its variable structures to report a problem; these variables proved status reports and are configured using the AADvance Workbench.

    A Fault Reset button on each processor module serves to clear a fault indication. However, the diagnostic systems will report a serious problem again so quickly there will be no visible change in the status indications. Pressing the Fault Reset button when no fault is indicated has no effect.

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    3-3

    Actions of the Diagnostic SystemsActions of the Diagnostic SystemsActions of the Diagnostic SystemsActions of the Diagnostic Systems

    The diagnostic systems filter potential but non-critical fault conditions by sampling at periodic intervals and requiring a number of matching error reports before reporting a problem. Typically the diagnostic systems maintain a counter for a particular fault. If an error is found, the counter increments. If an error is not found, the counter decrements, but by a smaller value. Once the counter reaches a threshold, the diagnostic systems latch the counter and raise alarm and status indications to report the fault.

    As an example, a non-essential item might be monitored every three hours and reported after 24 hours like this.

    Pressing the Fault Reset button resets every counter which has reached the fault threshold.

    Latching and NonLatching and NonLatching and NonLatching and Non----Latching FaultsLatching FaultsLatching FaultsLatching Faults

    Faults occurring in the T9110 processor module are non-latching. The controller will recover automatically, and the fault indication will clear, once the fault condition has been remedied.

    Faults occurring in the I/O subsystem are latched. In order to clear them, you must remedy the source of the fault (usually by replacing a module) and then press the Fault Reset button on the processor module. During the reset operation, the application software continues to run at its normal scan rate. There is no change in system performance and no additional vulnerability to faults.

    Indications of field faults (which appear as an amber Channel status indicator) are non-latching. You may not see some short term problems. The fault indication clears as soon you remedy the source of the problem.

  • 3-4

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    Common Fault AlarmCommon Fault AlarmCommon Fault AlarmCommon Fault Alarm

    A variable in the Workbench application program can be set up to provide a common fault alarm. The alarm is functionally equivalent to the System Healthy indicator on a processor module, and is usually the starting point for investigations into system faults.

    Fault IndicationsFault IndicationsFault IndicationsFault Indications

    To review status LEDs and rectify faults, do the following:

    1) Review the status LEDs on each module.

    Note: Some output faults are only visible when outputs are energised, so check for these while testing field devices.

    2) If the status LEDs reveal faults, use the procedures in this document to diagnose and rectify the problems.

    Note: There are no user-serviceable parts inside AADvance modules. A defective module must be returned to Rockwell Automation.

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    3-5

    Fault IndicationsFault IndicationsFault IndicationsFault Indications

    The diagnostic systems detect four classes of fault:

    System fault, pertaining to the AADvance system

    Module fault, pertaining to a particular module

    Channel fault, pertaining to the circuits inside an I/O module or TA serving a particular channel

    Field fault, pertaining to the field loop wiring outside the I/O module and TA serving a particular channel

    A channel fault produces a module fault and similarly a module fault produces a system fault.

    The diagnostic systems use some of the status indicators (LEDs) on the controller modules to show the presence of a fault. The indicators show the location and where possible the nature of a fault, and provide the information you need to locate the problem. The following groups of status indicators can show faults:

    The System Healthy LED on each processor module

    The Healthy LED on each processor and I/O module

    The Channel LED on each I/O module

    The relationships between the classes of faults and the status indicators are as shown.

  • 3-6

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    System FaultSystem FaultSystem FaultSystem Fault

    The controller indicates system fault when it detects a fault associated with itself rather than a field condition or a field device. Such a fault may be any of the following:

    A module fault (below)

    A fault the controller cannot isolate it to a single module. An example would be the absence of every I/O module in a termination assembly group.

    The System Healthy LED on each processor module will show red.

    Module FaultModule FaultModule FaultModule Fault

    The controller indicates a module fault when it detects a fault and can isolate it to the hardware of a particular module. The Healthy LED on the faulty module will show red and the System Healthy LED on each processor module will also show red.

    Channel FaultChannel FaultChannel FaultChannel Fault

    The controller indicates a channel fault when it detects a fault and can isolate it to a hardware fault on a specific channel of a single I/O module. The controller always reports a channel fault as a module fault as well. This means that the Channel LED will show red, the Healthy LED of the I/O module will show red and the System Healthy LED on each processor module will also show red.

    Field FaultField FaultField FaultField Fault

    The AADvance controller indicates a field fault when it detects a fault and can isolate it to a field condition or a field device. Examples are an open circuit field connection or an out-of-range signal. The Channel LED on the relevant I/O module will show amber.

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    3-7

    Recovery Mode IndicationsRecovery Mode IndicationsRecovery Mode IndicationsRecovery Mode Indications

    When the processor enters the Recovery Mode the following indications are displayed on the processor front panel when the module contains an application:

    Recovery Mode No Fault Present

    Recovery Mode Fault Present

  • 3-8

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    Fault Reporting Reference InformationFault Reporting Reference InformationFault Reporting Reference InformationFault Reporting Reference Information

    Each module has a set of front panel status indicators. The purpose and meaning of these indicators relevant to each module is as follows:

    Status Indicators on the T9110 ProcessStatus Indicators on the T9110 ProcessStatus Indicators on the T9110 ProcessStatus Indicators on the T9110 Processor Moduleor Moduleor Moduleor Module

    Table 2: Table 2: Table 2: Table 2: Status Indicators on the 9110 Processor ModuleStatus Indicators on the 9110 Processor ModuleStatus Indicators on the 9110 Processor ModuleStatus Indicators on the 9110 Processor Module

    Indicator Status Description

    Healthy Provides an indication of the module's fault status and power-on/operational status

    OFF No power

    RED Module has a fault

    Flashes RED briefly after being installed as the module is booting up or reset is in progress

    controller is off-line

    GREEN As the module is installed and receives power it flashes RED for a second then goes GREEN as the module boots up (10 to 20 seconds).

    When the module has booted up and is operational the LED stays GREEN and indicates that the module has no hardware faults.

    When in the recovery Mode and no faults are present the LED is GREEN

    Note:

    1. If Healthy is GREEN and all the other indicators are OFF then the module has failed to boot up

    2. If Healthy is GREEN and the Ready and Run are RED then the module is said to be in its shutdown state ( See topic - Module Shutdown

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    3-9

    State and Possible Causes in the Troubleshooting Manual Chap 3).

    Ready Provides an indication of the module's education and synchronisation status

    OFF No power

    RED Module is booting up (10 to 20 seconds) or not educated or synchronised with partners

    GREEN Module is educated and synchronised with partners

    Flashing GREEN Education or synchronisation is in progress (may be a very short period of flashing then Green)

    Amber Module is in the Recovery Mode

    Run Provides an indication of the module's application status. Should be the same for all educated and synchronised processors

    OFF No power and stays off while the module is booting up (10 to 20 seconds)

    RED Module is not educated/synchronised; No application loaded; The processor module is in the Recovery Mode and the base level firmware is running

    GREEN The application is present and running

    AMBER Module is in the Recovery Mode

    or

    The application is present but not running, press Fault Reset to start it running.

    System Healthy Provides an indication of the global health of the system, including all processors and I/O modules. Must be the same indication for all educated and synchronized processors

    OFF No power and stays off while the module is booting up (10 to 20 seconds)

    RED System or module faults present

    or the application has stopped running because the module has entered the Recovery Mode.

    GREEN No system or module faults present during normal operation and when in the recovery Mode.

    Force Provides an indication that variables are being locked/forced by the application. The same indication will show for all educated and synchronised processors

    OFF No power and stays off while the module is booting up (10 to 20 seconds)

    GREEN No variables are being locked/forced

    AMBER Module is in the Recovery Mode.

    or

    an operating controller has at least one variable being locked/forced

    Aux This LED is controlled by the application. The application can turn it on/off and any colour except when the processor is in the Recovery Mode and the application is stopped

    OFF No power and stays off while the module is booting up (10 to 20

  • 3-10

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    seconds), or under application control

    GREEN Under application control

    AMBER Module is in the Recovery Mode

    or

    under application control

    Serial 1 and 2 Provides an indication of serial port activity

    OFF No power and stays off while the module is booting up (10 to 20 seconds)

    RED Pulse stretched Tx

    GREEN Pulse stretched Rx

    AMBER Tx and Rx activity in close proximity

    Ethernet 1 and 2 Provides an indication of Ethernet port activity

    OFF No power and stays off while the module is booting up (10 to 20 seconds)

    GREEN Ethernet link present

    AMBER Tx or Rx activity

    Note: When inserting more than one processor module they MUST be inserted one at a time and the module allowed to educate (in the case of a 2nd and 3rd processor).

    Status Indicators on Status Indicators on Status Indicators on Status Indicators on the T94xx Series Input and Output Modulethe T94xx Series Input and Output Modulethe T94xx Series Input and Output Modulethe T94xx Series Input and Output Module

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    3-11

    Table 3: Table 3: Table 3: Table 3: Status Indicators on the 94xx Series Input and Output ModuleStatus Indicators on the 94xx Series Input and Output ModuleStatus Indicators on the 94xx Series Input and Output ModuleStatus Indicators on the 94xx Series Input and Output Module

    Indicator Status Description

    Healthy Provides an indication of the general status of the module

    OFF No power

    GREEN No module faults present

    RED One or more module faults are present

    Note.

    1. The Healthy indicator may turn RED immediately upon application of power to the module, before then turning GREEN

    2. If Healthy is GREEN and the Ready and Run are RED then the module is said to be in its "shutdown state" refer to the Troubleshooting Manual - Chap 3 for more information on the shutdown state.

    Ready Provides an indication of the module's ability to report channel values to a running application

    OFF No power or unlocked

    GREEN Locked and ready to report channel values

    RED Locked but not ready to report channel values

    Run Provides an indication that the module is reporting channel values to a running application

    OFF No power or unlocked

    GREEN Module is online and providing data to/receiving data from application

    AMBER Module is inserted into a running system but not online. Press the Fault Reset button on any processor module to enable the module to go online

    RED Module is ready to go online but no application is running

    Channel 1 - 8 Provides an indication of the status of each input or output channel

    OFF Input module: field switch is open Output module: output is in its de-energised state

    If the run indicator is not green (the module is not reporting channel values), all channel indicators will be off

    GREEN Input module: Channel input is on Output module: Output is in its energised state

    AMBER Field fault

    RED Channel fault

  • 3-12

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    Module Shutdown State and Possible CausesModule Shutdown State and Possible CausesModule Shutdown State and Possible CausesModule Shutdown State and Possible Causes

    Processor ModuleProcessor ModuleProcessor ModuleProcessor Module

    The processor module shutdown state is the Recovery Mode. If the application was running and it has been re-booted and the Fault Reset button pressed the module enters the Recovery Mode. Here the System Healthy LED will stay Green for a few seconds then go Red.

    Refer to the following topics:

    Recovery Mode and How it Affects Controller Operation

    Rectify a Critical Firmware/Hardware Failure

    I/O ModuleI/O ModuleI/O ModuleI/O Module

    A module (Processor or I/O Module) is in a shutdown state indicated by the following:

    Healthy: GREEN

    Ready: RED

    Run: RED

    Note: When the module is operational and before loading an application program the module is in the shutdown state. The module can also enter the shutdown state when the application stops running. This is not a fault but a normal state of operation.

    Possible Causes to be InvPossible Causes to be InvPossible Causes to be InvPossible Causes to be Investigatedestigatedestigatedestigated

    However, other faults or problems such as the following can cause a module to enter this state:

    The module is not in the processor's application control; i.e. the processor has not started the module and it stays in the shutdown state. Check: the application program to see if the I/O module is installed in the equipment and if so if it is installed into the correct I/O Bus and Slot. Check: the module PST value is it set to the correct value or has it been left at zero. Check: the communication link between the processor module and the I/O module (possible I/O base unit fault or loose bus cable)

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    3-13

    I/O Module Channel Degradation and ShutdownI/O Module Channel Degradation and ShutdownI/O Module Channel Degradation and ShutdownI/O Module Channel Degradation and Shutdown

    Process Safety TimeProcess Safety TimeProcess Safety TimeProcess Safety Time

    Each simplex module (or group of modules) has a defined process safety time (PST), which determines the maximum length of time that can elapse between successive reporting of channel values to the application before the channel is declared faulty. The module PST is usually the same as the system PST, however, it can be set to a different value within the AADvance Workbench when the controller is first configured, or later changed by an on-line update.

    The minimum PST value for a module is 20ms and the maximum 60,000ms (1 minute).

    Input Module NonInput Module NonInput Module NonInput Module Non----degraded Statusdegraded Statusdegraded Statusdegraded Status

    Degradation for input modules is on a channel-by-channel basis. A channel is considered not degraded when when all the modules in a group are on-line and none of them are reporting a fault for the particular channel. The Workbench reports the module status variable 'Discrepancy' as FALSE.

    A non-degraded simplex module configuration reports the channel values to the application.

    A non-degraded dual module configuration reports the channel values to the application based on the count value from either module.

    A non-degraded triple module configuration reports the values from any of the three channels in the group.

    Input Module Degraded StatusInput Module Degraded StatusInput Module Degraded StatusInput Module Degraded Status

    A channel is considered fully degraded when no modules in a group are reporting values for that channel, or there is a fault reported on that channel for all modules in the group and either condition exists for longer than the PST. The module status variable 'Discrepancy' is reported as TRUE and the module reports 'safe' values back to the application for the faulty channel.

    Output Module ShutdownOutput Module ShutdownOutput Module ShutdownOutput Module Shutdown

    When an output module does not receive updated command values from a running application within the PST, it automatically enters a shutdown mode. This action applies to all situations when commanded state updates are not received, including communication errors, lack of processing modules, and stopping an application.

    Degraded Input Channel Reporting ValuesDegraded Input Channel Reporting ValuesDegraded Input Channel Reporting ValuesDegraded Input Channel Reporting Values

    When a fault exists on a channel, the module will report safe values for that channel and the Discrepancy status variable reports TRUE.

    Simplex Module CoSimplex Module CoSimplex Module CoSimplex Module Configurationnfigurationnfigurationnfiguration

    When a simplex module channel fault exists for longer than the module PST the channel indicates a fault condition and the following status values are reported:

    Channel State = 7

    Channel Input State = FALSE

    Line fault = TRUE

  • 3-14

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    Discrepancy = TRUE

    Channel fault = TRUE

    Channel reports a voltage value = 0

    Dual Module ConfigurationDual Module ConfigurationDual Module ConfigurationDual Module Configuration

    When the reported values between modules in digital/analogue modules of a dual configurations diverge by more than twice the safety margin specification for a period of two application scans, that is by the following values:

    Digital input modules diverge by more than 2.0V dc

    Analogue input modules diverge by 400A.

    the lower of the two values will be reported and a discrepancy flag is set and the configuration degrades to a simplex operation.

    Triple Module ConfigurationTriple Module ConfigurationTriple Module ConfigurationTriple Module Configuration

    When the reported values between modules in digital/analogue triple configuration diverge by more than twice the safety margin specification for a period of two application scans, that is by the following values:

    Digital input module by more than 2.0V dc

    Analogue input module by 400A.

    the lower of the two values will be reported and a discrepancy flag is set and the configuration degrades to a dual operation.

    Output Channel ShutdownOutput Channel ShutdownOutput Channel ShutdownOutput Channel Shutdown

    In shutdown mode the output module drives its outputs to their configured shutdown settings; for example, de-energized or hold last state. The shutdown mode and channel drive states remain in place until new command states are received from a running application, or until the module loses power.

    On power up or module insertion, a module de-energizes all channels and they remain de-energized until command states are received from a running application.

    Group Output Module ShutdownGroup Output Module ShutdownGroup Output Module ShutdownGroup Output Module Shutdown

    As long as one module in a group continues to receive updated command state values from a running application within the PST, each channel is driven according to its commanded state. This covers the situation when only one module out of a pair goes into shutdown mode with some channels energized (from a hold last state setting). This makes sure that these channels do not get stuck energized, and that the remaining module is able to energize or de-energize these channels according to the commanded state received from a running application.

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    3-15

    Understanding thUnderstanding thUnderstanding thUnderstanding the State Variable (.STA)e State Variable (.STA)e State Variable (.STA)e State Variable (.STA)

    The state variable (.STA) is one attribute of the full structure for input and output channels. The variable reports a numeric value (from 1 to 7) which reflects the current state of the channel.

    States 2 and 4 represent normal operation for digital inputs and outputs; state 3 represents normal operation for an analogue input. The other states represent a fault.

    A digital input channel without line monitoring can detect and report states 2, 3, 4, 6 and 7, but cannot recognize an open circuit or short circuit. A line monitored input can detect and report every state, including an open circuit or short circuit. Analogue input channels are always line momitored.

    The state variable updates in real time. The fault diagnosis procedures for field faults use the value of the state variable.

    Correlation of LEDs with State Variable for a Digital InputCorrelation of LEDs with State Variable for a Digital InputCorrelation of LEDs with State Variable for a Digital InputCorrelation of LEDs with State Variable for a Digital Input

  • 3-16

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    Correlation of LEDs with State Variable for an Analogue InputCorrelation of LEDs with State Variable for an Analogue InputCorrelation of LEDs with State Variable for an Analogue InputCorrelation of LEDs with State Variable for an Analogue Input

    Correlation of LEDs with State Variable for a Digital OutpCorrelation of LEDs with State Variable for a Digital OutpCorrelation of LEDs with State Variable for a Digital OutpCorrelation of LEDs with State Variable for a Digital Outputututut

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    3-17

    Correlation of LEDs with State Variable for an Analogue OutputCorrelation of LEDs with State Variable for an Analogue OutputCorrelation of LEDs with State Variable for an Analogue OutputCorrelation of LEDs with State Variable for an Analogue Output

    View Module Firmware VersionsView Module Firmware VersionsView Module Firmware VersionsView Module Firmware Versions

    Using the AADvance Workbench you can view the module firmware information on screen and save this information with your project. Using an update function view the latest information and save it as an external text file.

    To do this proceed as follows:

    Note: To view the firmware version numbers of the modules you must be connected using Debug to a running controller.

    1) Select the Equipment View tab.

    2) Select the desired configuration node Config4(9000 Series Controller) in example shown below.

    3) Select the Version Information tab.

    The version information window appears. If the version information has previously been requested and saved (applied) then it will be visible in this window.

  • 3-18

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    4) Click the Update button.

    The window now shows your controller's current firmware version information.

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    3-19

    The information displayed is as follows:

    MAC Addresses - MAC addresses for the controllers. There are two addresses per controller set by the BUSP chip inserted into the processor base unit; However, 6 MAC addresses are displayed regardless if the system is a Single, Dual or Triple processor system.

    The module Information is as follows:

    Slot - the slot the module has been allocated

    Module - the module identity

    Serial - the module hardware serial number

    Versions - The firmware versions in the module

    1) Click Apply

    The information is saved with the project so that next time the project is opened you can view it.

    2) To save the information to a text file select the Save As button.

    3) A window opens with a default text file name Version_Report.txt; Enter your own file name and click Save.

    The saved text file can be viewed in Notepad.

  • 3-20

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    This page intentionally left blank

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    4-1

    The AADvance controller indicates a system fault through the common fault alarm for the system. This chapter explains how to troubleshoot and rectify a system and module fault.

    WARNING

    ELECTRICAL ARCS AND EXPLOSION RISK IN HAZARDOUS AREAS

    If you connect or disconnect wiring, modules or communications cabling while power is applied, an electrical arc can occur. This could cause an explosion in hazardous location installations. Do not remove wiring, fuses, modules or communications cabling while circuit is energized unless area is known to be non hazardous.

    Failure to follow these instructions may result in personal injury.

    In This ChapterIn This ChapterIn This ChapterIn This Chapter

    Troubleshooting System and Module Faults ................................................ 4-2 Rectify a Critical Firmware/Hardware Failure ............................................. 4-3 Install a T9110 Processor Module .................................................................. 4-3 Install I/O Modules ........................................................................................... 4-10

    ChChChChapter apter apter apter 4444 Troubleshooting and Rectifying Module FaultsTroubleshooting and Rectifying Module FaultsTroubleshooting and Rectifying Module FaultsTroubleshooting and Rectifying Module Faults

  • 4-2

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    Troubleshooting System and Module FaultsTroubleshooting System and Module FaultsTroubleshooting System and Module FaultsTroubleshooting System and Module Faults

    Use this flow chart to begin every system fault investigation.

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    4-3

    Rectify a Critical Firmware/Hardware FailureRectify a Critical Firmware/Hardware FailureRectify a Critical Firmware/Hardware FailureRectify a Critical Firmware/Hardware Failure

    When a single critical firmware failure occurs, such as a software watchdog failure, or self-test detects a hardware failure, the processor module will automatically reboot into the Recovery Mode. In Recovery mode the application is stopped so the processor displays the following changes to the status indications occurs:

    READY goes Amber

    RUN goes Amber

    FORCE goes Amber

    AUX goes Amber

    SYSTEM HEALTHY stays Green and after a short delay goes Red

    HEALTHY goes Red

    You should do the following:

    1) Collect the logs from the processor and contact Product Support at " [email protected] and submit them for analysis by support.

    alternatively

    2. You can replace the processor module and contact your regional office for an RMA number to return the faulty module.

    Install a T9110 Processor ModuleInstall a T9110 Processor ModuleInstall a T9110 Processor ModuleInstall a T9110 Processor Module

    Note: All new processor modules are delivered with the latest firmware and a base level Recovery Mode firmware build. You can enter the Recovery Mode by pressing the Fault Reset button immediately after the module has booted up.

    Inspect the module:

    Before inserting a new I/O module, inspect it for damage.

    The identification labels on the sides of the I/O module will be hidden once the module is installed. Therefore before installation record the location of the module and the details shown on the label.

    If you are installing more than one processor module make sure they all have the same firmware build.

    To install each T9110 processor module, do the following:

  • 4-4

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    1) Examine the coding pegs on the T9100 processor base unit and verify they complement the sockets on the rear of the processor module:

    2) Place the processor module on to the dowel pins on the processor base unit. Make sure the slot on the head of the module clamp screw is vertical and then push the module home until the connectors are fully mated.

    3) The module locking screw requires a quarter turn clockwise to lock. Use a broad (9mm) flat blade screwdriver to lock the locking screw.

    Note: The locking screw acts as a power interlock device. Therefore, the locking screw must be in the locked position after the power is applied otherwise the module will not boot up.

    Upgrade a Processor Module FirmwareUpgrade a Processor Module FirmwareUpgrade a Processor Module FirmwareUpgrade a Processor Module Firmware

    To upgrade ethe firmware use the Recovery Mode and follow the procedures using the ControlFLASH utility. Refer to the configuration Guide for detailed procedures on using the ControlFLASH utility.

    Note: If you need to downgrade the firmware to meet the requirements of your application and system contact Product support.

    Install a Processor BackInstall a Processor BackInstall a Processor BackInstall a Processor Back----up Batteryup Batteryup Batteryup Battery

    The T9110 processor module uses a battery to support its internal clock and memory when it is powered off. The battery condition is monitored by the module diagnostics every 24 hours. If the battery voltage is low, an application variable is set indicating there is a problem and the processor Healthy LED goes red.

    Note: The processor back up battery is supplied separately and must be installed into each new processor.

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    4-5

    To install a new battery, do the following:

    (Part No: T9905 Poly-carbonmonofluride Lithium Coin Battery, BR2032 (recommended type) , 20mm dia; Nominal voltage 3V; Nominal capacity (mAh) 190; Continuous standard load (mA) 0.03; Operating temperature -30C to 80C, supplied by Panasonic. The battery will last for approximately 10 years under normal operating conditions, or approximately six months if the module is not in use.

    1) Use a small cross head screwdriver to release the battery cover. Remove the cover.

    2) To remove an old battery pull on the ribbon in the battery holder and pull the battery out.

    3) To install a new battery orientate the new battery with the positive (+) terminal to the right. Trap the ribbon behind the new battery so it can be removed in the future and then push the battery into the holder.

    4) Reinstall the battery cover.

    5) Press the Fault Reset button on the processor module. The processor Healthy LED will go green (applies if the module is part of a running system).

    If the battery is replaced when only a single processor module is installed and the processor module is not under power the processor clock will need to be be reset to the current time. Correct the clock at the earliest opportunity. If it is replaced when more than one processor module is installed then the clock will be updated through synchronization.

    If you have previously set up SNTP when you set up your processor module then the clock will be reset to the current time automatically. If you haven't set up SNTP it is recommended that you do so, as this will not only reset the processor clock but will also keep the time accurately during normal operation. Refer to the AADvance Configuration Guide, Chapter 4 for the SNTP set up instructions.

    Alternatively use the following procedures to set the processor clock using the Processor Variables.

  • 4-6

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    Set the Processor clockSet the Processor clockSet the Processor clockSet the Processor clock

    To set the processor clock you can use the RTC variables:

    1) To monitor the time, wire variables to all the RTC Status points.

    2) To configure the time to be set, wire variables to the RTC Program points.

    Hours

    Minutes

    and Seconds

    3) Preset all RTC Program variables to the time that is to be programmed.

    It is recommended that you set the time to 03:00. Do not set the time to midnight or a slow clock will always be set back 24 hours.

    4) To control the time setting, wire variables to the RTC Control points:

    RTC Write

    RTC Read and

    5) Wire variables to RTC Control:

    Hours

    Minutes

    Seconds

    6) Set RTC Read to be always True (The time will not be written unless this point is also True)

    7) Set RTC Control elements Hours, Minutes and Seconds to be always True.

    8) Use an external trigger to change RTC Write from False to True at the right time.

    The time will be set into the Real Time Clock

    Note: On the very first setting it will be necessary to program all the time elements manually (Year, Month, Day, etc.)

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    4-7

    Processor Module Start Up ProcessProcessor Module Start Up ProcessProcessor Module Start Up ProcessProcessor Module Start Up Process

    Note: When inserting more than one processor module they MUST be inserted one at a time and the module be allowed to educate (in the case of a 2nd and 3rd processor).

    Table 4: Table 4: Table 4: Table 4: Single Processor Module Installation Procedure (New Processor Module)Single Processor Module Installation Procedure (New Processor Module)Single Processor Module Installation Procedure (New Processor Module)Single Processor Module Installation Procedure (New Processor Module)

    Step Task

    1. Install the processor into a processor base unit slot.

    2. All LEDs are off and after applying power the processor will display the following status indications:

    Healthy Flashes RED for a second then goes GREEN as the module boots up (10 to 20 seconds)

    Ready Will remain OFF as the module boots up (10 - 20 seconds) then goes RED

    Run Will remain OFF as the Module boots up (10 to 20 seconds) then goes RED

    System Healthy Will remain OFF as the Module boots up (10 to 20 seconds) then goes GREEN

    Force Will remain OFF as the Module boots up (10 to 20 seconds) then stays OFF until the module has educated.

    Aux Will remain OFF as the Module boots up (10 to 20 seconds) then is dependent upon data connection.

    Serial 1 Will remain OFF as the Module boots up (10 to 20 seconds) then is dependent upon data connection

    Serial 2 Will remain OFF as the Module boots up (10 to 20 seconds) then is dependent upon data connection

    Ethernet 1 Will remain OFF as the Module boots up (10 to 20 seconds) then is dependent upon data connection

    Ethernet 2 Will remain OFF as the Module boots up (10 to 20 seconds) then is dependent upon data connection

    You have to set up the processor IP Address so that the workstation can communicate with the processor module and download the latest firmware build and configure the processor variables.

    3. Set the Controller IP Address for all the processor modules using the "AADvance Discover Utility". (Refer to the Configuration Guide for detailed instructions on using the AADvance Discover Utility Doc. No: 553633.

    4. Install a Program Enable Key. Download a valid application and press the FAULT RESET button. When the application is downloaded and valid the module will display the following indications:

    Valid Application Downloaded

    Healthy GREEN

    Ready GREEN

    Run RED to GREEN (Flashes GREEN as the module educates)

    System Healthy GREEN

  • 4-8

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    Step Task

    Force GREEN

    Aux Off (Application Dependent)

    Serial 1 Dependent on data connection

    Serial 2 Dependent on data connection

    Ethernet 1 Dependent on data connection

    Ethernet 2 Dependent on data connection

    Table 5: Table 5: Table 5: Table 5: ProceProceProceProcedure for Installation of a 2nd and 3rd Processordure for Installation of a 2nd and 3rd Processordure for Installation of a 2nd and 3rd Processordure for Installation of a 2nd and 3rd Processor

    Note: The second and third processor modules must be programmed with the same firmware as the first processor. Check the firmware revision on the labels and if required use the procedure in the Configuration Guide. If the firmware revision is different to the first processor module you can download the latest firmware build to all the processor modules using the ControlFLASH utility.

    When inserting a second and third processor module they MUST be inserted one at a time and allowed to educate before inserting the next one.

    Step Task

    1. Place the processor module on slot B on the Processor Base Unit connectors and push the module home until the connectors are fully mated. Turn the locking screw with a flat bladed screwdriver to lock the module in position.

    All the Module LEDs are OFF until the module is installed. As soon as the module receives power it will boot up then educate and display the following indications:

    Healthy Flashes RED for a second then goes GREEN as the module boots up (10 to 20 seconds)

    Ready Will remain OFF as the module boots up (10 - 20 seconds) then goes RED for 10 secs then flashes GREEN as it educates and finally it goes to steady GREEN

    Run Will remain OFF as the module boots up (10 - 20 seconds) then goes RED until educated and then it goes AMBER

    System Healthy Will remain OFF as the Module boots up (10 to 20 seconds) then goes GREEN

    Force Will remain OFF as the Module boots up (10 to 20 seconds) then stays OFF until the module has educated and the application is running

    Aux Will remain OFF as the Module boots up (10 to 20 seconds) then is dependent upon data connection

    Serial 1 Will remain OFF as the Module boots up (10 to 20 seconds) then is dependent upon data connection

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    4-9

    Step Task

    Serial 2 Will remain OFF as the Module boots up (10 to 20 seconds) then is dependent upon data connection

    Ethernet 1 Will remain OFF as the Module boots up (10 to 20 seconds) then is dependent upon data connection

    Ethernet 2 Will remain OFF as the Module boots up (10 to 20 seconds) then is dependent upon data connection

    When the Run indicator goes AMBER press the Fault Reset button and the processor will display the following indications:

    2. Healthy Green

    Ready GREEN (can flash for a short time as the module educates)

    Run AMBER to GREEN (AMBER as the module educates)

    System Healthy GREEN

    Force Off to GREEN

    Aux Off (application dependent)

    Serial 1 Dependent on Data Connection

    Serial 2 Dependent on Data Connection

    Ethernet 1 Dependent on Data Connection

    Ethernet 2 Dependent on Data Connection

    3. To insert a 3rd processor module repeat step 1 and insert in Slot C.

  • 4-10

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    Install I/O ModulesInstall I/O ModulesInstall I/O ModulesInstall I/O Modules

    Inspect the module:

    Before inserting a new I/O module, inspect it for damage.

    The identification labels on the sides of the I/O module will be hidden once the module is installed. Therefore before installation record the location of the module and the details shown on the label.

    Examine the coding pegs on the termination assembly and verify they complement the sockets on the rear of the new I/O module.

    To install each I/O module, do the following:

    1) Examine the coding pegs on the termination assembly and verify they complement the sockets on the rear of the I/O module.

    2) Place the I/O module on to the dowel pins on the T9300 I/O base unit. Make sure the slot on the head of the module clamp screw is vertical and then push the module home until the module connectors are fully mated with the I/O base unit and termination assembly connectors.

    3) The locking screw requires a quarter turn clockwise to lock. Use a broad (9mm) flat blade screwdriver to lock the clamp screw.

    Note: The locking screw acts acts as a power interlock device and must be in the locked position when power is applied otherwise the module will not power up.

    I/O Module Start Up ProcessI/O Module Start Up ProcessI/O Module Start Up ProcessI/O Module Start Up Process

    To start up an Input/Output Module follow this procedure:

    Note: The start up sequence is different when a module is installed into an on-line system that is running compared to installing the module into a system that is off-line and has processor modules but no I/O modules installed. The first part of this procedure covers the initial start up of an off-line system, the second part covers a system that is on line and you are adding I/O modules.

    Table 6: Table 6: Table 6: Table 6: Single Module or FiSingle Module or FiSingle Module or FiSingle Module or First Module of a group Installation Procedurerst Module of a group Installation Procedurerst Module of a group Installation Procedurerst Module of a group Installation Procedure

    Step Task

    1. This procedure applies to a single module installed or the first module of a redundant group.

    2 Install the Input/Output Module and turn the locking screw to the lock position.

    3. The input module will provide the following status indications:

    Healthy GREEN

    Ready RED

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    4-11

    Run RED

    Channel 1 8 Off

    4. The input module will enter its start up sequence during which the module will educate. Wait for approximately 3 seconds.

    5. The module will now provide the following status indications:

    Healthy GREEN

    Ready GREEN

    Run AMBER

    Channel 1 8 Off

    6. Press the Fault Reset button on the processor module and the Run indication goes GREEN.

    7. The module will now be on-line with the following status indications:

    Healthy GREEN

    Ready GREEN

    Run GREEN

    Channel 1 8) Dependent on channel status

    8. If the module fails to educate and go on-line replace the module.

    Table 7: Table 7: Table 7: Table 7: Second or third Module of a Group Installation ProcedureSecond or third Module of a Group Installation ProcedureSecond or third Module of a Group Installation ProcedureSecond or third Module of a Group Installation Procedure

    Step Task

    1. This procedure applies to a second or third module of a redundant group.

    2 Install the Input/Output Module and turn the locking screw to the lock position.

    3. The input module will provide the following status indications:

    Healthy GREEN

    Ready RED

    Run RED

    Channel 1 8 Off

    4. The input module will enter its start up sequence during which the module will educate. Wait for approximately 3 seconds.

    5. The module will now provide the following status indications:

    Healthy GREEN

    Ready GREEN

    Run AMBER

    Channel 1 8 Off

    6. Press the Fault Reset button on the processor module and the Run indication goes GREEN.

    7. The module will now be on-line with the following status indications:

    Healthy GREEN

  • 4-12

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    Ready GREEN

    Run GREEN

    Channel 1 8) Dependent on channel status

    8. If the module fails to educate and go on-line replace the module.

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    5-1

    The AADvance controller indicates a channel fault that could be a faulty termination assembly or field fault by a Channel LED on an I/O module showing amber instead of the usual green. This chapter provides recommended approaches to troubleshoot and remedy channel/field faults. It should be read in conjunction with the operation and maintenance manual or equivalent documentation for the system.

    WARNING

    ELECTRICAL ARCS AND EXPLOSION RISK IN HAZARDOUS AREAS

    If you connect or disconnect wiring, modules or communications cabling while power is applied, an electrical arc can occur. This could cause an explosion in hazardous location installations. Do not remove wiring, fuses, modules or communications cabling while circuit is energized unless area is known to be non hazardous.

    Failure to follow these instructions may result in personal injury.

    In This ChapterIn This ChapterIn This ChapterIn This Chapter

    Examine the State Variable ............................................................................... 5-2 Start Troubleshooting Channel/Field Faults ................................................. 5-3 Replacing Fuses .................................................................................................. 5-12 Install a New Termination Assembly ........................................................... 5-14

    Chapter Chapter Chapter Chapter 5555 Troubleshooting and Rectifying Channel/Field FaultsTroubleshooting and Rectifying Channel/Field FaultsTroubleshooting and Rectifying Channel/Field FaultsTroubleshooting and Rectifying Channel/Field Faults

  • 5-2

    Document: 553634

    (ICSTT-RM406_EN_P) Issue: 08:

    Troubleshooting and Maintenance Manual (AADvance Controller)

    Examine the State VariableExamine the State VariableExamine the State VariableExamine the State Variable

    You will use the State variable (.STA) to trace field faults. To do this, you have to identify the tagname of the variable, and then add the variable to the Spy List. Do the following:

    1) Select the Equipment tab to use the Project Tree View to locate the channel you wish to investigate.

    2) Identify the reference for the State variable. This will be of one of two forms:

    %IBnnn.0.7 for an input module

    %QBnnn.0.7 for an output module (illustrated)

    Note: From Release 1.2, live data is available in the view shown above. For earlier releases, use the Spy List as described below.

    3) Select Debug Target and then open the Spy List.

  • Document: 553634 (ICSTT-RM406_EN_P) Issue: 08:

    5-3

    4) Double-click on the Name field.

    5) Select the tagname for the state variable from the drop down list.

    The State variable is added to the Spy list.

    The value of the State variable ('2' in this example) will update in real time to reflect the state of the I/O channel.

    Note: If the Spy List is empty, double-click on the ellipsis (...) to create the entry.

    Start Troubleshooting Ch