Business Intelligence Portfolio

AllWorks, Inc

Business Intelligence Porfolio

Using the Microsoft Business Intelligence Suite

Contact InformationDeveloper: Karl Umaguing Phone: (626) 824-5062 Email: [email protected]

Business Intelligence Portfolio

Table of ContentsSSIS ETL Project Overview........................................................................................................................3

AllWorksOLTP Database......................................................................................................................4

List of AllWOrksOLTP Tables................................................................................................................5

Master Package...................................................................................................................................6

Process Employees..............................................................................................................................8

Process Employee Rates......................................................................................................................9

Process Overhead..............................................................................................................................10

Process Overhead Rates....................................................................................................................11

Process County Data..........................................................................................................................12

Process Client Data............................................................................................................................13

Process Project Data..........................................................................................................................14

Process Timesheet Data....................................................................................................................15

Process Material Purchase Data........................................................................................................16

Process Invoice Data..........................................................................................................................17

Process Invoice Job Data....................................................................................................................18

Process Receipt Data.........................................................................................................................19

Data Sources......................................................................................................................................20

Reporting Project Overview...................................................................................................................22

ScoreCard..........................................................................................................................................23

Materials............................................................................................................................................24

Overhead...........................................................................................................................................25

Labor..................................................................................................................................................26

Employee Labor Analysis...................................................................................................................27

Overhead Category............................................................................................................................28

Employee Jobs in Date Range............................................................................................................32

Basic Overhead by Date.....................................................................................................................33

Labor History Chart............................................................................................................................34

Job Profitability Chart........................................................................................................................35

2


SSIS ETL Project Overview

ScenarioAllWorks, Inc (a fictitious construction company) is experiencing significant growth. Project accounting information is currently maintained in various files; Excel Spreadsheets, an XML file and CSV files. A new approach is needed to better handle the increased influx in business and information.

Assignment Create a central repository to store all related information. Develop an ETL package in SSIS to load the data, scheduling it to run weekly.

3


AllWorksOLTP Database

4


ClientClientNo

AccountNo

ClientName

ProjectManager

CountyKey

ClientImage

CountyCountyKey

CountyDescription

FK_Client

EmployeeRatesEmployeeID

EffectiveDate

HourlyRate

EmployeesEmployeeID

FirstName

LastName

Employee

CellPhone

FullName

FK_EmployeeRates

EmpTimeEmployeeID

WorkDate

JobNo

RegHours

FK_EmpTime_EID

InvoiceInvoiceID

InvoiceNo

InvDate

ClientNo

FK_Invoice

InvoiceJobInvoiceID

JobNo

InvoiceNo

InvAmount

AddLabor

InvoiceJobIdentifier

FK_InvoiceJob_II

MaterialPurchaseMaterialPurchaseID

ProjectID

TypeID

PurchaseDate

PurchaseAmount

SalePoint

MaterialTypeTypeID

TypeDescription

FK_MaterialPurchase_TID

OverheadOverheadID

OverheadDescription

OverheadRatesOverheadID

EffectiveDate

HourlyRate

UseEmployee

UseContractors

FK_OverheadRates

ProjectProjectID

Description

ClientNo

ClientJob

MaterialMarkupPct

AdditionalOverheadPct

Closed

CloseDate

CreationDate

FK_Project

FK_EmpTime_JN

FK_MaterialPurchase_PID

FK_InvoiceJob_JN

ReceiptInvoiceID

JobNo

PaidDate

PaidAmount

ReceiptIdentifier

FK_Receipt_II

FK_Receipt_JN

List of AllWOrksOLTP Tables

Table Primary key Foreign key FK Reference Data SourceClient ClientNo CountyKey County.CountyKey ClientGeographies

Excel File : Client Listing Worksheet

County CountyKey ClientGeographies

5


Excel File : County Definitions Worksheet

Emptime EmployeeID,WorkDate,JobNo

EmployeeIDJobNo

Employees.EmployeeIDProject.ProjectID

EmpTime CSV File

EmployeeRates EmployeeID,EffectiveDate

EmployeeID Employees.EmployeeID Employees Excel File : Employee Rates Worksheet

Employees EmployeeID Employees Excel File : Employees Worksheet

Invoice InvoiceID ClientNo Client.ClientNo Invoices Excel File : Invoices Worksheet

InvoiceJob InvoiceIDJobNo

InvoiceIDJobNo

Invoice.InvoiceIDProject.ProjectID

Invoices Excel File : Invoices Worksheet

MaterialPurchase MaterialPurchaseID ProjectIDTypeID

Project.ProjectIDMaterialType.TypeID

MaterialPurchases XML File

MaterialType TypeID 1 - Regular Materials2 - Petty Cash3 - Fuel

Overhead OverheadID Overhead Excel File : Overhead Master Worksheet

OverheadRates OverheadID,EffectiveDate

OverheadID Overhead.OverheadID Overhead Excel File : Overhead Rates Worksheet

Project ProjectID ClientNo Client.ClientNo ProjectMaster Excel File : Project Master Worksheet

Receipt InvoiceID,JobNo,PaidDate

InvoiceID JobNo

Invoice.InvoiceIDProject.ProjectID

Invoices Excel File : Invoices Worksheet

6


Master Package

7


8


The Master package holds the ETL execution sequence and a series of data administration tasks. Each container within the execution sequence represents the processing for each data source; Employees, Overhead, ClientGeographies, ProjectMaster, EmpTime, MaterialPurchases and Invoices. Each process consists of an ETL to the associated table followed by disposition of the data source. Upon the successful completion of the entire ETL execution sequence, database administration tasks are performed. These tasks are shrinking of the database, rebuilding of indices, updating of database statistics and a backup of the database. An email reporting the successful completion of both the ETL execution process and the database administration tasks is sent to the appropriate groups. An email will also be sent should any of the database tasks fail. Note that emails are also sent within each ETL process indicating success or failure.

9


Process Employees

The Process Employees package does the following: Read the Employees Excel File – Employees Worksheet. Convert the data to SQL format. Create a Full Name column. Verify a match between the input employee id and the Employees table. For each Employees table “match”, compare input data with table data and update the

entire row if any change is detected. For each Employees table “no match”, insert a row.

10


Process Employee Rates

The Process Employee Rates package does the following: Read the Employees Excel File – Employee Rates Worksheet. Convert the data to SQL format. Find a match between the input employee id and the Employees table. For each Employees table “no match”, write an entry in the Invalid Employee Rates File. For each Employees table match, verify a match with the Employee Rates table based on

Employee Id and Effective Date. For each Employee Rates table “no match”, insert a row. For each Employee Rates table “match”, compare the incoming hourly rate with the current

hourly rate and update the entire row in the Employee Rates table if any difference is detected.

11


Process Overhead

The Process Overhead package does the following:

Read the Overhead Excel File – Overhead Master Worksheet. Convert the data to SQL format. Find a match between the input overhead and the Overhead table. For each input overhead without a match in the Overhead table, insert a row. For each input overhead with a match in the Overhead table, comparing the input

description with the current description and update the row when a change is determined.

12


Process Overhead Rates

The Process Overhead Rates package does the following: Read the Overhead Excel File – Overhead Rates Worksheet. Convert the data to SQL format. Find a match between the input overhead id and the Overhead table. For each input overhead id without a match in the Overhead table, write an entry in the

Invalid Overhead Rates File. For each input overhead id with a match in the Overhead table, verify a match with the

Overhead Rates table based on Overhead Id and Effective Date. For each input overhead rate without a match in the Overhead Rates table, insert a row. For each input overhead rate with a match in the Overhead Rates table, compare the

incoming data with the current data and update the entire row in the Overhead Rates table if any changes are detected.

13


Process County Data

The Process County Data package does the following: Read the ClientGeographies Excel File – County Definitions Worksheet. Convert the data to SQL format. Add an additional column based on Description change. Verify that input County is in the County table. For each County table “no match”, insert a row. For each County table “match”, compare the input description with the current description

and update the row when a change is detected.

14


Process Client Data

The Process Client Data Rates package does the following: Read the ClientGeographies Excel File – Client Listing Worksheet. Convert the data to SQL format. Verify that the associated County is in the County table. For each county not in the County table, write an entry in the Invalid Client Data File. For each county found in the County table, verify that a corresponding row based on County

Id and Client Number exists in the Client table. For each Client table “no match”, insert a row. For each Client table “match”, compare the incoming data with the current data and update

the entire row in the Client table if any changes are detected.

15


Process Project Data

The Process Project Data package does the following: Read the ProjectMaster Excel File – Project Master Worksheet. Convert the data to SQL format. Create a derived column to handle possible Null values in Client Number. Verify that the associated Client is in the Client table. For each client not in the Client table, write an entry in the Invalid Project Data File. For each client in the Client table, verify that the project exists in the Project table. For each Project table “no match”, insert a row. For each Project table match, compare the incoming data with the current data and update

the entire row in the Project table if any changes are detected.

16


Process Timesheet Data

The Process Timesheet Data package does the following: Read the EmpTime CSV File. Convert the data to SQL format. Verify that the employee exists in the Employee table. For each employee not in the Employee table, write an entry in the Invalid Employee File. For each employee in the Employee table, verify that the project exists in the Project table. For each Project table “no match”, write an entry to the Invalid Project File. For each Project table “match”, determine if a matching row exists in the EmpTime table. For each input record with a matching row in the EmpTime table, determine a change in

RegHours and update the EmpTime row if a change is detected. For each input record without a matching EmpTime table row, determine project status and

compare work date with project close date referencing the Project table. For each input record with an open project status or a work date less than or equal to

project close date, insert a row. For each input record with a closed project status and a work date greater than project close

date, write an entry to the Invalid Work Date File.

Additional notes: multiple EmpTime files can be processed at any one time.

17


Process Material Purchase Data

The Process Material Purchase Data package does the following: Read the MaterialPurchases XML File. Convert the data to SQL format. Create a derived column to handle possible Null values in SalePoint. Verify that the input project id exists in the Project table. For Project table “no match”, write an entry in the Invalid Project File. For each Project table “match”, verify that a corresponding row exists in the

MaterialPurchase table. For each input record with a matching row in the MaterialPurchase table, determine a

change in PurchaseAmount and update the MaterialPurchase row if a change is detected. For each input record without a matching MaterialPurchase table row, determine project

status and compare work date with project close date referencing the Project table. For each input record with an open project status or a work date less than or equal to

project close date, insert a row. For each input record with a closed project status and a work date greater than project close

date, write an entry to the Invalid Purchase Date File.

18


Process Invoice Data

The Process Invoice Data package does the following: Read the Invoices Excel File – Invoices Worksheet. Convert the data to SQL format. Verify that the input project id exists in the Project table. For each Project table “no match”, write an entry in the Invalid Project File. For each Project table “match”, verify that a corresponding row exists in the Invoice table. For each invoice without a corresponding entry in the Invoice table, insert a row. For each invoice with a corresponding entry in the Invoice table, update the entire row.

19


Process Invoice Job Data

The Process Invoice Job Data package does the following: Read the Invoices Excel File – Invoices Worksheet. Convert the data to SQL format. Verify that the input project id exists in the Project table. For each Project table “no match”, write an entry in the Invalid Project File. For each Project table “match”, verify that a corresponding row exists in the Invoice table. For each invoice with a corresponding Invoice row, verify that a row exists in the Invoice Job

table. For each invoice job without a corresponding Invoice Job row, insert a row. For each invoice job with a corresponding Invoice Job row, compare incoming data and row

data, updating the entire row if changes are detected.

20


Process Receipt Data

The Process Receipt Data package does the following:

Read the Invoices Excel File – Invoices Worksheet. Convert the data to SQL format. Verify that the associated invoice job exists in the Invoice Job table. For each invoice job not in the Invoice Job table, write an entry in the Invalid Invoice Job

File. For each invoice job found in the Invoice Job table, verify that a corresponding row exists in

the Receipt table. For each receipt with a corresponding Receipt row, compare the input paid amount with the

current paid amount and update the row if a change is detected. For each receipt job without a corresponding Receipt row, insert a row.

21


Data Sources

Excel Spreadsheets: Employees.xlsx

o First sheet (employees) contains roster of Employees and a flag to indicate whether the employee is a contractor or a regular employee (some overhead rates only apply to employees).

o Second sheet (employee rates) contains Employee Hourly Rates for each employee, along with an effective date.

ClientGeographies.xlsxo First sheet (Client Listing) contains each client for AllWorks, along with a County Key

to associate the client with a county.o Second sheet (County Definitions) contains the list of counties.

OverHead.xlsxo First sheet (Overhead master) lists each labor overhead category.o Second sheet (Overhead rates) lists the hourly rates for each overhead category, the

rate effective date, and whether the rate applies to employees and/or contractors. * IMPORTANT NOTE *

This relates to the employee flag in the employee table. Remember that a rate may change from applying to only employees, to applying to both employees and contractors (or vice-versa).

ProjectMaster.xlsxo First sheet (Project Master) contains one row for each Job work order. It contains a

reference to the client, material markup %, labor overhead markup %, whether the job is closed and the creation date. The markup % is optional and is used as follows: if $1,000 of material is purchased, and the Job markup % is 10 (10%...they are stored as whole numbers), then the total material cost is $1,100. The same concept applies to labor markup, based on the # of hours of labor for the job.

Invoices.xlsxo An Excel spreadsheet that contains client invoice data. Each row consists of job

number, invoice amount, invoice number, paid amount and any additional labor amount.* IMPORTANT NOTE *

Currently, the invoice spreadsheet only allows for one work order job per invoice. AllWorks wants to expand the functionality so that a single invoice can cover multiple jobs. Additionally, when AllWorks receives payment, they want to track how much was received for each job on each invoice. A onemany link/reference table will be needed, both for the invoice and the jobs associated with it. Another onemany table will also be needed for each invoice receipt and amount paid for each job on the invoice.

22


XML File: MaterialPurchases.xml

o This file contains the material purchase transactions for each job. Each record has a job number, type code, purchase amount and purchase date. AllWorks currently does not have a separate table for the Type Code. The codes are hardwired; 1 stands for Material, 2 stands for Petty Cash, and 3 stands for Fuel.

CSV Files: EmptimeXXXX.csv

o These files contain all the labor data transactions. Each record consists of employee id, work date, number of work hours, and job number.

23


Reporting Project Overview

ScenarioAllWorks, Inc (a fictitious construction company) is experiencing significant growth. A centralized database was created to contain all related project accounting information. Subsequently, an analytic database (OLAP) was created to provide for better analysis of data.

AssignmentCreate a varied group of reports using different reporting tools (Performance Point Server - PPS, SQL Server Reporting Services - SSRS and Excel Services) and making all reports accessible in Microsoft Office Sharepoint Server (MOSS) collection site. This will give business analysts a sampling of what’s available with reporting and presentation.

24


ScoreCard

This report was created and deployed in Performance Point Server as part of the AllWorks Dashboard. It consists of 2 scorecards; Overhead Trend, Client Job Financials.

The Overhead Trend scorecard shows the trend as the % change from the previous to current quarter for all Overhead types with current quarter as a selectable parameter. It uses Overhead perspective as its data source and KPIOverheadTrend for KPI.

The Client Job Financials scorecard shows Open Receivables As % Of Invoiced and Profit % for all clients. It also shows Overhead As % Of Total Cost for all jobs. It uses Summary perspective as its data source and KPIOpenReceivablesAsPctOfInvoiced, KPIProfitPercentage and KPIOverheadAsPctOfTotalCost for KPIs.

25


26


Materials

This report was created and deployed in Performance Point Server as part of the AllWorks Dashboard. This report charts total purchase amount by quarter for each material type for all clients or a specific client as selected by parameter. The report uses the Materials perspective as data source.

27


Overhead

This report was created and deployed in Performance Point Server as part of the AllWorks Dashboard. This report charts overhead cost by quarter for one or more overhead types as specified by parameter. It uses Overhead perspective as the data source.

28


Labor

This report was created and deployed in Performance Point Server as part of the AllWorks Dashboard. This report shows the top 10 labor jobs and the top 5 workers for a particular quarter as selected by parameter. It uses Labor perspective as data source.

The Top 10 Labor Jobs section shows the top 10 projects/jobs and labor cost in ascending order.

The Top 5 Workers section shows the top 5 workers, their hours and labor cost in ascending order.

29


Employee Labor Analysis

This report was created and deployed in Performance Point Server as part of the AllWorks Dashboard. It uses Labor Perspective as data source. It has 2 sections; % of quarterly labor dollars, % of labor dollars by project. Employee is a selectable parameter.

The % of quarterly labor dollars section charts the total labor cost for all employees only for jobs worked on by a specific employee. This is represented as a bar chart. It also charts a specific employee’s labor cost as a percentage of total labor cost for all employees. This is represented as a line chart using a 2nd y-axis.

The % of labor dollars by project section shows, for a specific employee, a list of projects he worked on, his labor cost for each project, the total labor cost for all employees for each project listed and the employee’s labor cost as a percentage of the total labor cost.

30


Overhead Category

This report was created and deployed in SSRS. It show total cost by Overhead Category for previous and current quarter and the calculated % change. Current quarter is a selectable parameter.

This report has been scheduled to run daily at 8 AM, generating a new version based on the most recent quarter.

31


This page shows the set-up for the shared schedule used by the Overhead Category report.

32


This page shows the set-up for the subscription to automatically generate a new version of the Overhead Category report on schedule.

33


This page shows set-up of stored credentials as part of automatically generating a new version of the Overhead Category report.

34


Employee Jobs in Date Range

This report was created and deployed in SSRS. It shows hours worked and labor cost by week end date and job for a particular employee. Employee name, from week end date and to week end date are selectable parameters. Overhead perspective is used as the data source.

35


Basic Overhead by Date

This report was created in Excel and deployed from PPS as part of the Excel Services dashboard. It shows overhead cost for each overhead type by quarter and year. Year is a selectable parameter. This report uses Overhead Perspective as the data source.

36


Labor History Chart

This report was created in Excel and deployed from PPS as part of the Excel Services dashboard. It charts labor cost by quarter for all contractors and employees for one to multiple clients. Year and client are selectable parameters. Labor perspective is used as the data source.

37


Job Profitability Chart

This report was created in Excel and deployed from PPS as part of the Excel Services dashboard. It charts job profit by quarter for one to multiple counties as represented in a bar chart. This report also charts profit percentage by quarter for one to multiple counties as represented by a line chart using a 2nd y-axis. Summary perspective is used as the data source.

38

Business Intelligence Portfolio

Documents

Transcript of Business Intelligence Portfolio