PR 50024 Phase II SCR 727 SCR 740 Web Services Jackie Ashbaugh Commercial Operations Data Integrity...

PR 50024 Phase II SCR 727

SCR 740 Web Services

Jackie AshbaughCommercial Operations

Data Integrity & Administration

August 18, 2006

August 18, 2006 2

Agenda

• Project Timeline Information• SCR740 System Flow Diagram Review• Project Assumptions• SCR740 Data Request Screens• Q & A Session

August 18, 2006 3

Project Information

• PR 50024 project phase: Execution– ESIID Web Services User Guide draft to be posted by 9/15/2006– WSDL & XSD drafts to be posted by 9/15/2006– Identify market training needs through DEWG 8/2006 & 9/2006– ITEST 10/2/2006– Production migration/stabilization 12/2/2006– Market release TBD (target 12/2006-01/2007; contingent on

successful production verification, stabilization, and blackout period)

August 18, 2006 4

SCR 740 Flow Chart

Texas Market

Link

CompositeWeb Service

(SOA)

XML SOAP

DDM ODS

MIRPayload

The Market

WSEPTIBCO

DC Verification

XML SOAP

August 18, 2006 5

SCR 740 Flow Chart (cont.)

• To make a data request, Market Participants will either interface with the “Schedule a Data Request” section of TML or submit a request programmatically via the API . Each data request on TML is a separate webpage form. Once the form is filled out with the requisite information and submitted, the information is converted into XML/SOAP and delivered to ERCOT. Data access validation is handled via digital certificate.

Texas Market

Link


(SOA)

XML SOAP

DDM ODS

MIRPayload

The Market

WSEPTIBCO

DC Verification

XML SOAP

August 18, 2006 6


• All requests external to ERCOT will be handled by a Composite Web Service (CWS) which will be a TIBCO service. All requests received by this system will be recorded in the DB with an associated status and various other metrics. The parameters of the request are evaluated and, if accepted, are routed to the corresponding Web Service End Point (WSEP).

Texas Market

Link


(SOA)

XML SOAP

DDM ODS

MIRPayload

The Market

WSEPTIBCO

DC Verification

XML SOAP

August 18, 2006 7


• WSEPs execute the specified data request function. Functions include Extract Initiation, Status, and/or Payload Requests. These end points will return an XML/SOAP message to the CWS containing information including ID for the request. The XML/SOAP message including receipt and metadata information is then returned to the Market Participant.

Texas Market

Link


(SOA)

XML SOAP

DDM ODS

MIRPayload

The Market

WSEPTIBCO

DC Verification

XML SOAP

August 18, 2006 8


• The Data Delivery Module (DDM) will handle WSEP work. The DDM places the WSEP actions in a queue to be processed on the ODS in a manner that optimizes ODS resources which allows multiple Market Participants to submit data requests at the same time without overloading the ODS.

Texas Market

Link


(SOA)

XML SOAP

DDM ODS

MIRPayload

The Market

WSEPTIBCO

DC Verification

XML SOAP

August 18, 2006 9


• The data is then pulled from the ODS by the DDM and converted into CSV or XML format. The DDM then zips the file and delivers it to the Market Information Repository (MIR). Once within MIR, the zipped files may be downloaded by the Market Participant via the TML interface.

Texas Market

Link


(SOA)

XML SOAP

DDM ODS

MIRPayload

The Market

WSEPTIBCO

DC Verification

XML SOAP

August 18, 2006 10

Project Assumptions

1. No point in time data requests may be made for dates prior to the stabilization date for the Lodestar ODS environment. This will date will be defined with implementation.

2. All ESIID level data requests will be validated based on the market participant ownership defined in the ESIIDSERVICEHIST table.

3. All ESIID extract record requests for SCR 727 ESIID Extract database refreshes may only be requested once per calendar year.

4. All requests for the ‘current’ system data will be based on Lodestar batch start time at SYSDATE - 1.

5. All requests using point in time selectivity (PITS) data will assume a timestamp equal to that of the Lodestar batch start time for the calendar date chosen.

6. NIDR and IDR data usage requests will be available. This does not guarantee this was the data type used during data aggregation/settlement.Note: The usage data type used for data aggregation/settlement is based on the

METERTYPE of the PROFILECODE at the time of aggregation/settlement.7. All requests allowing an upload file are limited to 1,000 records within the upload.

Note: This is not a limitation on the number of records returned.8. Limitation of two requests per day per Market Participant dunsnumber for the

request to provide aggregation detail data (Web Service #12). Note: Web Service questions 1 -11 (on following slides) do not have a request per day

limitation.9. Delete tables in SCR 727 ESIID Extract will not be available.

August 18, 2006 11

Data Request Home Page

August 18, 2006 12

Data Request Home Page (cont.)

August 18, 2006 13

1) Verify ESIIDs I own.

August 18, 2006 14

1) Verify ESIIDs I own (cont.)

• Business question answered:– What ESIIDs do I own for all of time as of a particular point in

time?• Note: Ownership is validated with ESIIDSERVICEHIST

• Delivery time frame expectation: <= 3 days to complete request.• Required inputs:

– Point In Time – a field box which will default to SYSDATE - 1– An upload file containing the ESIIDs/UIDESIIDs in question.

• Must contain <= 1,000 ESIIDs/UIDESIIDs.• Optional file format validation is available prior to submission.

– The DUNSNUMBER from the digital certificate will be used to access the service.

August 18, 2006 15

1) Verify ESIIDs I own (cont.)

• Optional inputs:– Trade date can be used to further filter the request.

• Output options: (all radio buttons)– 1. The whole ESIID record (from the ESIID table in LODESTAR).– 2. The whole ESH record (from the ESH table in LODESTAR).– 3. Both the whole ESIID and the whole ESH records.

• Output format options: (all radio buttons)– Output files can be delivered in either XML or CSV format.

• Output file names:– rpt.(MM-DD-YYYY HH:MM:SS)VERIFY_MMDDYYYY.csv– rpt.(MM-DD-YYYY HH:MM:SS)VERIFY_MMDDYYYY.xml

August 18, 2006 16

Show Upload File Contents

August 18, 2006 17

Show XML SOAP

August 18, 2006 18

2) Provide ESIIDs I own for a trade date.

August 18, 2006 19

2) Provide ESIIDs I own for a trade date (cont.)

• Business question answered:– What ESIIDs do I own for a particular trade date as of a particular

point in time with certain attributes?• Delivery time frame expectation: <= 3 days to complete request.• Required inputs:

– Point In Time – a field box which will default to SYSDATE - 1– Trade Date – a field box which defaults to current day– The DUNSNUMBER from the digital certificate will be used to

access the service.

August 18, 2006 20


• Optional inputs:– An upload file containing the ESIIDs/UIDESIIDs in question.


– LOSSCODE – field box– PROFILECODE – field box

• Any combination of the following can be supplied in substitution of complete profile code: PROFILETYPE, WEATHERZONE, WEATHERSENSITIVITY, METERTYPE, TIME OF USE.

– STATIONCODE – field box– ZIPCODE – field box– TDSPCODE/DUNSNUMBER – pull down menu– REPCODE/DUNSNUMBER – pull down menu– MRECODE/DUNSNUMBER – pull down menu– PGCCODE/DUNSNUMBER – pull down menu– STATUS – field box

August 18, 2006 21


• Output options: (all radio buttons)– 1. The whole ESIID record (from the ESIID table in LODESTAR).– 2. The whole ESH record (from the ESH table in LODESTAR).– 3. Both the whole ESIID and the whole ESH records.– 4. A list of distinct ESIIDs

• Output format options: (all radio buttons)– Output files can be delivered in either XML or CSV format.

• Output file names:– rpt.(MM-DD-YYYY HH:MM:SS)ESIIDTD_MMDDYYYY.csv– rpt.(MM-DD-YYYY HH:MM:SS)ESIIDTD_MMDDYYYY.xml

August 18, 2006 22

3) Provide CMZONE assignments for a trade date.

August 18, 2006 23

3) Provide CMZONE assignments for a trade date (cont.)

• Business questions answered:– What ESIIDs do I own for a particular trade date as of a particular

point in time with the provided CMZONE?• Delivery time frame expectation: <= 3 days to complete request.• Required inputs:


access the service.

August 18, 2006 24

3) Provide CMZONE assignments for a trade date (cont.)

• Optional inputs:

– An upload file containing the ESIIDs/UIDESIIDs in question.• Must contain <= 1,000 ESIIDs/UIDESIIDs.

• Optional file format validation is available prior to submission.

– CMZONE – field box

• Output options: (all radio buttons)

– 1. A record including ESIID, UIDESIID, CMZONE, STATIONCODE, STATIONNAME, TRADEDATE.

• Output format options: (all radio buttons)

– Output files can be delivered in either XML or CSV format.

• Output file names:

– rpt.(MM-DD-YYYY HH:MM:SS)CMZONETD_MMDDYYYY.csv

– rpt.(MM-DD-YYYY HH:MM:SS)CMZONETD_MMDDYYYY.xml

August 18, 2006 25

4) Provide NIDR transaction data for ESIIDs I own.

August 18, 2006 26

4) Provide NIDR transaction data for ESIIDs I own (cont.)

• Business questions answered:– For provided transaction IDs, what NIDR usage is loaded?

• Note: Cannot check for cancels, only originals.


– Point In Time – a field box which will default to SYSDATE - 1– An upload file containing ESIID, transaction ID, STARTTIME,

STOPTIME, METERTYPE.• Must contain <= 1,000 rows.• Optional file format validation is available prior to submission.


August 18, 2006 27

4) Provide NIDR transaction data for ESIIDs I own (cont.)

• Optional inputs:– None.

• Output options: (radio buttons)– 1. A record including ESIID, transaction ID, STARTTIME,

STOPTIME, METERTYPE.– 2. The whole ESIID usage record.

• Output format options: (radio buttons)– Output files can be delivered in either XML or CSV format.

• Output file names:– rpt.(MM-DD-YYYY HH:MM:SS)NIDRTRANS_MMDDYYYY.csv

– rpt.(MM-DD-YYYY HH:MM:SS)NIDRTRANS_MMDDYYYY.xml

August 18, 2006 28

5) Provide IDR transaction data for ESIIDs I own.

August 18, 2006 29

5) Provide IDR transaction data for ESIIDs I own (cont.)

• Business questions answered:– For provided transaction IDs or UIDCHANNELCUTs, what IDR

usage is loaded?• Note: Cannot check for cancels, only originals.


– Point In Time – a field box which will default to SYSDATE - 1– One of the following:

• An upload file containing ESIID, transaction ID, STARTTIME, STOPTIME.– Must contain <= 1,000 rows.– Optional file format validation is available prior to submission.

• An upload file containing UIDCHANNELCUTs– Must contain <= 1,000 rows.– Optional file format validation is available prior to submission.


August 18, 2006 30

5) Provide IDR transaction data for ESIIDs I own (cont.)


– None.

• Output options: (radio buttons)

– If ESIID upload provided:• 1. A record including ESIID, transaction ID, LSCCH STARTTIME, LSCCH STOPTIME.

• 2. LSCCH records

• 3. LSCCH and LSCCD records.

– If UIDECHANNELCUT upload provided:• 1. LSCCH records

• 2. LSCCH and LSCCD records

• Output format options: (radio buttons)


• Output file names:– rpt.(MM-DD-YYYY HH:MM:SS)IDRTRANS_MMDDYYYY.csv– rpt.(MM-DD-YYYY HH:MM:SS)IDRTRANS_MMDDYYYY.xml

August 18, 2006 31

6) Provide Non-IDR usage data for a trade date.

August 18, 2006 32

6) Provide Non-IDR usage data for a trade date (cont.)

• Business questions answered:– What ESIIDs have NIDR usage data loaded for a trade date as of

a particular point in time?• Delivery time frame expectation: <= 3 days to complete request.• Required inputs:

– Point In Time – a field box which will default to SYSDATE - 1– TRADE DATE – a field box which defaults to current day– The DUNSNUMBER from the digital certificate will be used to

access the service.– METERTYPE – radio button with default of kWh (KH)

August 18, 2006 33

6) Provide Non-IDR usage data for a trade date (cont.)


– An upload file containing the ESIIDs/UIDESIIDs in question.• Must contain <= 1,000 ESIIDs/UIDESIIDs.

• Optional file format validation is available prior to submission.

• Output options: (all radio buttons)

– 1. List of distinct ESIIDs.

– 2. ESIID usage records.

• Output format options: (all radio buttons)


• Output file names:– rpt.(MM-DD-YYYY HH:MM:SS)NIDRUSAGETD_MMDDYYYY.csv– rpt.(MM-DD-YYYY HH:MM:SS)NIDRUSAGETD_MMDDYYYY.xml

August 18, 2006 34

7) Provide ESIIDs where Non-IDR usage data not loaded for a trade date.

August 18, 2006 35

7) Provide ESIIDs where Non-IDR usage data not loaded for a trade date (cont.)

• Business questions answered:– What ESIIDs do not have NIDR usage data loaded for a trade

date as of a particular point in time?• Delivery time frame expectation: <= 3 days to complete request.• Required inputs:


access the service.– METERTYPE – radio button with default of kWh (KH)

August 18, 2006 36

7) Provide ESIIDs where Non-IDR usage data not loaded for a trade date (cont.)



• Output options: (radio buttons)– 1. List of distinct ESIIDs.


• Output file names:– rpt.(MM-DD-YYYY HH:MM:SS)NIDRNOLOAD_MMDDYYYY.csv

– rpt.(MM-DD-YYYY HH:MM:SS)NIDRNOLOAD_MMDDYYYY.xml

August 18, 2006 37

8) Provide IDR usage data for a trade date.

August 18, 2006 38

8) Provide IDR usage data for a trade date (cont.)

• Business questions answered:– What ESIIDs have IDR usage data loaded for a trade date as of a

particular point in time?• Delivery time frame expectation: <= 3 days to complete request.• Required inputs:


access the service.– Generation or Load – radio button with default of Load.

August 18, 2006 39

8) Provide IDR usage data for a trade date (cont.)



• Output options: (radio buttons)– 1. List of distinct ESIIDs.– 2. LSCCH records.– 3. LSCCH and LSCCD records.

• Note: only send the 1 trade date record for the LSCCH.


• Output file names:– rpt.(MM-DD-YYYY HH:MM:SS)IDRUSAGETD_MMDDYYYY.csv

– rpt.(MM-DD-YYYY HH:MM:SS)IDRUSAGETD_MMDDYYYY.xml

August 18, 2006 40

9) Provide ESIIDs where IDR usage data not loaded for a trade date.

August 18, 2006 41

9) Provide ESIIDs where IDR usage data not loaded for a trade date (cont.)

• Business questions answered:– What ESIIDs do not have IDR usage data loaded for a trade date

as of a particular point in time?• Delivery time frame expectation: <= 3 days to complete request.• Required inputs:


access the service.– Generation or Load – radio button with default of Load.

August 18, 2006 42

9) Provide ESIIDs where IDR usage data not loaded for a trade date (cont.)



• Output options: (radio buttons)– 1. List of distinct ESIIDs.


• Output file names:– rpt.(MM-DD-YYYY HH:MM:SS)IDRNOLOAD_MMDDYYYY.csv

– rpt.(MM-DD-YYYY HH:MM:SS)IDRNOLOAD_MMDDYYYY.xml

August 18, 2006 43

10) Provide NIDR usage data for ESIIDs I own.

August 18, 2006 44

10) Provide NIDR usage data for ESIIDs I own (cont.)

• Business questions answered:– What NIDR usage data is loaded for a list of ESIIDs as of a





– METERTYPE – radio button with default of kWh (KH)

August 18, 2006 45

10) Provide NIDR usage data for ESIIDs I own (cont.)


• Output options: (radio buttons)– 1. ESIID usage records.


• Output file names:– rpt.(MM-DD-YYYY HH:MM:SS)NIDRUSAGE_MMDDYYYY.csv

– rpt.(MM-DD-YYYY HH:MM:SS)NIDRUSAGE_MMDDYYYY.xml

August 18, 2006 46

11) Provide IDR usage data for particular ESIIDs I own.

August 18, 2006 47

11) Provide IDR usage data for particular ESIIDs I own (cont.)

• Business questions answered:– What IDR usage data is loaded for a list of ESIIDs as of a





– Generation or Load – radio button with default to Load.

August 18, 2006 48

11) Provide IDR usage data for particular ESIIDs I own (cont.)


• Output options: (radio buttons)– 1. LSCCH records.– 2. LSCCH and LSCCD records.


• Output file names:– rpt.(MM-DD-YYYY HH:MM:SS)IDRUSAGE_MMDDYYYY.csv

– rpt.(MM-DD-YYYY HH:MM:SS)IDRUSAGE_MMDDYYYY.xml

August 18, 2006 49

12) Verify records used in data aggregation for a trade date.

August 18, 2006 50

12) Verify records used in data aggregation for a trade date (cont.)

• Business questions answered:– Verify records used in data aggregation for ESIID relationship,

attributes, and IDR or NIDR data.• Delivery time frame expectation: <= 3 days to complete request.• Required inputs:

– Point In Time – a field box which will default to SYSDATE - 1– Trade Date – a field box which defaults to current day– LOSSCODE – field box– PROFILECODE – field box– STATIONCODE – field box– TDSPCODE/DUNSNUMBER – field box– The DUNSNUMBER from the digital certificate will be used to

access the service.

August 18, 2006 51

12) Verify records used in data aggregation for a trade date (cont.)


– None.

• Output options: (radio buttons)

– For NIDR data:• 1. ESIID records.

• 2. ESIIDSERVICEHIST records.

• 3. ESIID usage records. (default)

– For IDR data:• 1. ESIID records.

• 2. ESIIDSERVICEHIST records.

• 3. LSCCH and LSCCD records. (default)

• Output format options: (radio buttons)


• Output file names:– rpt.(MM-DD-YYYY HH:MM:SS)VERIFYAGG_MMDDYYYY.csv– rpt.(MM-DD-YYYY HH:MM:SS) VERIFYAGG_MMDDYYYY.xml

August 18, 2006 52

13) Provide ESIID extract records.

August 18, 2006 53

13) Provide ESIID extract records (cont.)


– Point In Time – a field box which will default to SYSDATE - 1– The DUNSNUMBER from the digital certificate will be used to

access the service.• Output formats:

– Output files can be delivered in either XML or CSV format.• Output options (only for MP tables):

– 1. Logical Keys for MP specific tables and full records for public tables.

– 2. Full records for MP specific tables and public tables.– 3. Record count for each table.

August 18, 2006 54


• Restrictions:– Can only be requested once a year per MP.

• MP Specific Tables:– ESIID, ESIIDSERVICEHIST, ESIIDUSAGE, LSCCH, LSCCH/LSCCD.

• Public Tables:– STATION - full records– STATIONSERVICEHIST - full records– CMZONE - full records– PROFILECLASS - full records– REP - full records– TDSP - full records– MRE - full records– PGC - full records

August 18, 2006 55


• Logical Key for logical keys output– ESIID – uidesiid, addtime– ESIIDSERVICEHIST – uidesiid, starttime, stoptime, addtime– ESIIDUSAGE – uidesiid, starttime, stoptime, metertype,

timestamp– LSCCH – uidchannelcut, LSCCD addtime– LSCCH/LSCCD – uidchannelcut, LSCCD addtime

August 18, 2006 56

14) Provide ESIID extract records for Logical Keys.

August 18, 2006 57

14) Provide ESIID extract records for Logical Keys (cont.)


– Upload file containing table name in question– The DUNSNUMBER from the digital certificate will be used to

access the service.• Output formats:

– Output files can be delivered in either XML or CSV format.• MP Specific Tables:

– ESIID, ESIIDSERVICEHIST, ESIIDUSAGE, LSCCH, LSCCH/LSCCD.

August 18, 2006 58

15) Provide public data extract.

August 18, 2006 59

15) Provide public data extract (cont.)


– Point In Time – a field box which will default to SYSDATE - 1• Output formats:

– Output files can be delivered in either XML or CSV format.• Restrictions:

– Public tables can be chosen without any limitations and any MP can request these tables.

• Public Tables:– STATION, STATIONSERVICEHIST, CMZONE, PROFILECLASS,

REP, TDSP, MRE, PGC.

August 18, 2006 60

Audit Return Screen

August 18, 2006 61

Questions ?Email: [email protected]

PR 50024 Phase II SCR 727 SCR 740 Web Services Jackie Ashbaugh Commercial Operations Data Integrity...

Documents

Transcript of PR 50024 Phase II SCR 727 SCR 740 Web Services Jackie Ashbaugh Commercial Operations Data Integrity...