Zap application architecture-bi-2010

Zap BusinessIntelligence

ApplicATionARchiTecTuRe

www.zaptechnology.com

http://www.zapbi.com

http://www.zaptechnology.com

Application ArchitectureZap Business Intelligence 2010


PuRpose

The purpose of this document is to provide an overview of the architecture of Zap Business Intelligence 2010 to a relatively technical audience.

It is not expected that readers of this document will have an in-depth understanding of software development practices, methodologies or design principles; however, a reasonable understanding of enterprise-grade application architecture in general is assumed.

Scope

This document discusses the architecture of the Zap Business Intelligence application and how it interoperates with other components within the enterprise eco-system.

Specifically, this document does not cover deployment scenarios, network configuration or software and/or hardware requirements.

This document is not intended to provide a description of the feature set of Zap Business Intelligence 2010.

LAsT UpdATed

This document was last updated on 7/09/2010.


ConTenTs

1. Overview 7

2. Client Access Layer 8

3. Presentation Layer 9

4. Business Intelligence Components (Resources) 10

5. Business Intelligence Core 11 5.1 Analytics Engine 11 5.2 Workflow 11 5.3 Content Persistence 11 5.4 Authentication 11 5.5 Authorization 11 5.6 Licensing 12 5.7 Error and Usage Reporting 12

6. External Components 13 6.1 User Store 13 6.2 Content Persistence 13 6.3 OLAP Providers 13 6.4 Source Data 13

Application ArchitectureZap Business Intelligence 2010 7

1. OveRview

Zap Business Intelligence is a self-contained, multi-tiered application based on Microsoft’s .NET runtime and the ASP.NET framework.

Zap Business Intelligence utilises the Microsoft Windows Server platform and interoperates with Microsoft’s SQL Server and SQL Server Analysis Services.

As per Figure 1 below, Zap Business Intelligence comprises multiple internal components. All of these components are bundled together and must be installed on the same server, although they are logically separate and responsible for discrete pieces of functionality.

Authorization● Cube● Resource● Organization● Global

Zap Business Intelligence

Authentication● Forms (Multi-tenancy)● Windows Integrated

.NET Common Language Runtime

Windows Server

Presentation Layer Security

● Zap Business Intelligence Portal● Internet Information Services (IIS)● ASP.NET

Business Intelligence Components

● Dashboards● Analysis● Scorecards● KPIs● Report Packs● System Time & Date :: Aggregation :: Selection Types● Custom & 3rd party Example: vertical industry● Predictive - Price elasticity

!

User Store

.NET Role & Membership Providers

Windows Server Linux/Unix/Solaris/Other

ActiveDirectory

SQLServer

SQLDatabase

/etc /passwd

Custom Role &Membership Providers

SQL Server

Windows Server

LDAP (Integrated) | ADO.NET (Forms)

Business Intelligence Core

.NET Method Calls

.NET Method Calls

Data Warehouse Providers

SSAS SQL Server

Windows Server

Zap SSAS Extensions

AMO

SQL Server Database

Microsoft DynamicsCRM / NAV / AX

Other DataSources

WindowsServer

Linux / Unix /Solaris / Other

ADO

.NET

Zap CubeXpress

Client Access Layer

● XHTML● JavaScript● JQuery● AJAX.ASP.NET

HTTP (Get/Post), SOAP

Web Browser

Zap Business IntelligenceUser Interface

XMLA ● Analytics Engine

> Template System > Write-back > MDX Compiler > Core BI Logic● Work�ow > Rules > Events● Content Persistence● Licensing

Figure 1

The Zap Business Intelligence application comprises four main

components: a client-side (browser-based) user interface; a presentation

layer responsible for delivering content to the browser; the business

intelligence resources themselves; and finally a service layer responsible

for core business intelligence functionality.


2. ClienT Access LAyeR

The browser-based component of Zap Business Intelligence combines an XHTML- and JavaScript-based UI with jQuery and Microsoft’s AJAX.ASP.NET.

The browser component loads a basic application framework (“Portal”) into the browser on the first page load, and then dynamically modifies the page using SOAP-based web service calls, jQuery animations and custom JavaScript. A large proportion of the application’s user interface code is executed within the browser itself, requiring reduced communication with the Zap Business Intelligence server. The UI code makes extensive use of jQuery transitions and animations to enhance usability.

From a performance perspective, this means that network bandwidth and latency are much lesser issues than with a traditional web application and the individual user therefore experiences much faster performance. From a scalability perspective, this approach permits a single server to handle many more users without significant performance degradation than would otherwise be possible, and that performance degradation when it does occur is very gradual, as should be expected of an enterprise system.

Communication with the web server is performed using standard HTTP GET and POST operations for initial application load and printing, and SOAP-based web service calls for the remainder of the application’s lifetime.


Presentation Layer







● Analytics Engine > Template System > Write-back > MDX Compiler > Core BI Logic● Work�ow > Rules > Events● Content Persistence● Licensing


SSAS SQL Server

Windows Server

Zap SSAS Extensions

SQL Server Database


Other DataSources

WindowsServer


Zap CubeXpress

Client Access Layer


Web Browser


User Store



ActiveDirectory

SQLServer

SQLDatabase

/etc /passwd


Figure 2 Client Access (Browser) Layer


3. PResenTATion LAyeR

The server-side presentation layer is responsible for generating user interface code (XHTML and JavaScript) and delivering it to the browser. It sits on top of Microsoft’s Internet Information Server (IIS) and the ASP.NET framework.

The presentation layer knows about the current user’s application state (i.e. what they can currently see within their browser), which resources they have open and what state they are in. It also knows about the user’s permissions within an organization, and will dynamically show, hide or disable ribbon and menu options as appropriate.

Security within the presentation layer is focused largely on the prevention of scripting attacks and other browser-based exploits. The presentation layer does, of necessity, handle user authentication, but only to the extent of requiring the user to authenticate within that browser session and passing authentication tokens to the lower layers of the application.

User permissions (e.g. which resources a user can see, and what actions they may perform) are considered in this layer only insofar as keeping the UI courteous and generally not displaying options that a user is not permitted to select. Notably, role-based security is enforced lower down in the application’s stack and is not (and should not be) enforced in this layer.

The presentation layer communicates with the resource and core layers of Zap Business Intelligence using both .NET assembly references with direct method calls, and using internal web services where appropriate.


Presentation Layer









SSAS SQL Server

Windows Server

Zap SSAS Extensions

SQL Server Database


Other DataSources

WindowsServer


Zap CubeXpress

Client Access Layer


Web Browser


User Store



ActiveDirectory

SQLServer

SQLDatabase

/etc /passwd


Figure 3 Presentation Layer


4. Business InTelligence ComponenTs (ResouRces)

BI components (“resources”) in this context are the building blocks (dashboards, charts, analyses, KPIs etc.) that users generally associate with business intelligence. Those building blocks are themselves composed of smaller building blocks in the form of Zap Business Intelligence Functions, which allow shorthand access to everything from set selections (“Select all the leaf nodes in this hierarchy”) to statistical functions, time, date and current period filters, and even price elasticity.

A resource has a basic definition of what its designer desired it to display, and also has some additional state information; for example, the expansion state of particular cells within a cell-set.

The BI Components layer is also the layer in which third-party functions exist. One of the strengths of this approach is that third-party functions are treated the same way as Zap Business Intelligence’s internal functions are for the purposes of integration with the design experience, and users need neither know nor care about who actually provided the building blocks that enable them to design their resources quickly and efficiently. Further, commonly-used components within an organization can be extracted out into their own functions, allowing re-use and, hence, an even further-improved design experience.

Each and every resource has its own role-based permissions, allowing fine-grained control of access to individual resources. A folder is considered to be a type of a resource, so permissions on entire folders or tree structures can be managed in exactly the same way and with the same level of granularity as the permissions on a single, individual resource.


Presentation Layer









SSAS SQL Server

Windows Server

Zap SSAS Extensions

SQL Server Database


Other DataSources

WindowsServer


Zap CubeXpress

Client Access Layer


Web Browser


User Store



ActiveDirectory

SQLServer

SQLDatabase

/etc /passwd


Figure 4 BI Components Layer


5. Business InTelligence CoRe

The BI Core layer is responsible for the core business intelligence functionality of the system. It handles the actual analytics components with respect to compilation and generation of MDX (the query language analogous to SQL for multi-dimensional data sources), manages workflow around rules and events defined within Zap Business Intelligence, handles persistence (creating/loading/saving/deleting) of resources and the security around access to them, and manages communication with external components of the system. The BI Core layer is also responsible for the enforcement of security policies.

Obviously, the BI Core layer contains multiple components within itself. The details of these are outside the scope of this document as the inner workings of the application in this respect have little bearing on its interaction with external systems, so only a brief overview of each is provided herein.

5.1 Analytics Engine

The analytics engine handles all communications between Zap Business Intelligence and Microsoft’s Analysis Services (SSAS). The engine is responsible for generating MDX queries from BI resources, submitting the queries to SSAS, parsing the results and returning them to the presentation layer.

The analytics engine communicates with Analysis Services using the XMLA protocol with various security options around authentication, impersonation and other constraints on which users can see what data.

5.2 Workflow

The workflow engine in Zap Business Intelligence is responsible for triggering certain actions in response to user-defined conditions and events. A full description of the workflow capabilities of Zap Business Intelligence is outside the scope of this document.

5.3 Content Persistence

Content is preserved using a relational database stored on an instance of Microsoft’s SQL Server.

From the perspective of the application, almost every entity within Zap Business Intelligence is treated as a resource. Users, policies, organizations, data sources, folders and individual analytics components are all resources, and are stored in an XML-serialized (and XML-indexed) field within the application’s SQL Server database.

It should be noted that the content database is completely separate from any data warehouse providers, and also completely separate from the Analysis Services server(s) containing actual data. The content persistence layer manages only metadata.

Communication between the Zap Business Intelligence persistence layer and SQL Server uses Microsoft’s ADO.NET libraries wrapped by a LINQ abstraction layer. XML indexing is performed within the database on a subset of resource metadata in order to reduce search and retrieval time.

5.4 Authentication

User authentication is achieved using .NET’s authentication provider model. The default authentication provider uses Microsoft’s Active Directory as its backing store and issues queries via LDAP, but the design philosophy is that if a different user store were to be required then a different provider could be substituted without code changes.

Windows Integrated authentication is generally recommended for use within individual enterprise deployments, but forms-based authentication is also supported.

5.5 Authorization

Authorization is managed at several levels, from global (application-wide) policies, individual organization policies, resource permissions and even cube-level constraints down to individual tuples within an Analysis Services cube.

Global, organization and resource-level policy enforcement is the province of the Zap Business Intelligence Core application layer. Actions requested by a user (whether via web service calls, form posts or otherwise) are checked according to a user’s role memberships and are permitted only if that user has appropriate permissions.

Individual resources to which a user does not have access do not appear for that user, and if a resource to which a user does have access refers to one to which the user does not have access, permission is denied.


Presentation Layer









SSAS SQL Server

Windows Server

Zap SSAS Extensions

SQL Server Database


Other DataSources

WindowsServer


Zap CubeXpress

Client Access Layer


Web Browser


User Store



ActiveDirectory

SQLServer

SQLDatabase

/etc /passwd


Figure 5 Business Intelligence Core Layer


At the cube level, Zap Business Intelligence can be configured to impersonate the currently-logged-in user, which then allows Analysis Services to display only a sub-cube of the actual cube to that user.

5.6 Licensing

A license is composed of a set of restrictions, all of which must be satisfied in order for the license to be valid.

Licensing works on a very simple basis:

» A customer’s channel partner will create a license agreement for the customer within Zap Technology’s license-provisioning system and provide the customer with a product key.

» The customer’s system administrator installs Zap Business Intelligence and enters the product key.

» Zap Business Intelligence fetches a license from Zap Technology, validates it and permits access to the application accordingly.

Behind the scenes, a great deal of effort has gone into making the system simple and transparent. A license that is soon to expire, for example, will automatically and transparently trigger a re-fetch and require the involvement of only a system administrator if the customer’s license agreement has expired or is otherwise invalid.

All licensing communication is performed using standard SOAP web services over HTTPS, and stateful inspection of licensing traffic using Microsoft’s ISA Server or a similar trusted SSL proxy is supported.

For cases where a Zap Business Intelligence server is hosted within a DMZ that has no Internet connectivity, a manual licensing solution is also available.

5.7 Error and Usage Reporting

Zap Technology makes a sincere commitment to continuous improvement of software quality. To that end, there exists a facility within Zap Business Intelligence to automatically report errors and usage metrics back to the developers. This aids not only in prioritising the fixing of any bugs found, but also in the diagnosis of any bugs reported by a customer with difficult or incomplete reproduction steps.

This approach has already resulted in reduced turn-around times for fixes, and has enabled some other, non-reproducible behaviors that occurred only on specific systems to be identified and corrected.

This information is batched and sent to Zap Technology using a standard SOAP web service, again over HTTPS. Stateful inspection is supported and encouraged.

It must be made clear that any and all error or usage reporting is entirely voluntary and is via an opt-in process. In other words, the default behaviour of the application is to report zero information to Zap Technology and this setting may be changed only by a local system administrator.


6. eXTeRnAl ComponenTs

6.1 User Store

The default user store is Microsoft’s Active Directory. Other user stores (e.g. other SQL databases, RADIUS servers etc.) are feasible due to the use of the .NET authentication provider framework, but custom providers for non-Active Directory backing stores are not currently available.

As can be observed from the accompanying diagram, the .NET role and membership providers abstract authentication and membership away from Active Directory or an SQL database, and allow the application to be unaware of (and disinterested in) how its authentication is performed.

Although it is not currently supported, there is no reason why other authentication solutions such as Linux PAM, SQL databases on other operating systems or even RADIUS could not be, simply by the creation of appropriate authentication and role providers.

6.2 Content Persistence

BI content is persisted (saved) within a Microsoft SQL Server database. Content consists of the metadata defining an analytics resource, but notably does not include any actual data.

6.3 OLAP Providers

Zap Business Intelligence uses Microsoft’s SQL Server Analysis Services as its OLAP provider.

Connections to SSAS are defined as Data Sources within Zap Business Intelligence. There is no theoretical upper limit to the number of different Analysis Services instances or cubes for which data sources can be defined, although practical considerations such as server memory and processing power will limit the number of concurrent connections.

It should be noted that Microsoft PowerPivot uses SQL Server Analysis Services as its OLAP layer, so Zap Business Intelligence can easily query PowerPivot databases in the same way as it queries pure Analysis Services ones.

Third-party data warehousing components are also supported provided that there exists an extract/transform/load (ETL) tool or solution by which they can be transformed and loaded into Analysis Services.

There is no direct support for third-party OLAP providers.

6.4 Source Data

Zap Business Intelligence does not need to communicate directly with any source data systems. This permits the use of intelligent ETL products such as Zap Technology’s own Zap CubeXpress to import and transform data from arbitrary sources into Analysis Services OLAP databases.


Presentation Layer









SSAS SQL Server

Windows Server

Zap SSAS Extensions

SQL Server Database


Other DataSources

WindowsServer


Zap CubeXpress

Client Access Layer


Web Browser


User Store



ActiveDirectory

SQLServer

SQLDatabase

/etc /passwd



Presentation Layer









SSAS SQL Server

Windows Server

Zap SSAS Extensions

SQL Server Database


Other DataSources

WindowsServer


Zap CubeXpress

Client Access Layer


Web Browser


User Store



ActiveDirectory

SQLServer

SQLDatabase

/etc /passwd



Presentation Layer









SSAS SQL Server

Windows Server

Zap SSAS Extensions

SQL Server Database


Other DataSources

WindowsServer


Zap CubeXpress

Client Access Layer


Web Browser


User Store



ActiveDirectory

SQLServer

SQLDatabase

/etc /passwd


Figure 6 User Store

Figure 7 Data Warehouse Providers

Figure 8 Source Data

www.zaptechnology.com

Zap BusinessIntelligence

http://www.zaptechnology.com

http://www.zapbi.com

Zap application architecture-bi-2010

Business

Transcript of Zap application architecture-bi-2010