Post on 21-Dec-2015
description
CONNECTING TO DATASOURCES
Table of ContentsGeneral Information about Data Sources......................................................................................... 5
Understanding Data Sources ................................................................................................ 6Create New SQL Data Source .............................................................................................. 9Create a New Plug-in Data Source ..................................................................................... 13What does the Refresh Metadata button do? .................................................................... 17Metric Insights' Plugins........................................................................................................ 18Registering Your Server with Google to use their Plug-in Resources ................................. 21Populate a Data Source with an uploaded List of Reports.................................................. 26
How to Link to External BI Tools..................................................................................................... 30What are external reports? ................................................................................................. 31How to link to External report content ................................................................................. 33How to link to a Parameterized External report .................................................................. 41
Configuring Remote Data Collectors .............................................................................................. 48Overview of Remote Data Collection ................................................................................. 49Configure a Remote Data Collector ................................................................................... 50Installing Insightd on Windows Servers ............................................................................ 55Upgrading Insightd on Windows Servers ............................................................................ 62Installing Insightd on Linux Servers .................................................................................. 68Upgrading Insightd on Linux Servers .................................................................................. 70Troubleshooting a Remote Data Collector .......................................................................... 72
JDBC Connections ......................................................................................................................... 80Establishing a JDBC Connection ....................................................................................... 81Add a new JDBC Driver ..................................................................................................... 86Validation errors: "Value '0000-00-00' can not be represented as java.sql.Date" ............... 89
Sourcing data from 1010data ......................................................................................................... 92Establish connectivity to 1010data...................................................................................... 93Collect data from 1010data ................................................................................................. 97Creating 1010data macro code for your 1st Basic Metric ............................................... 102
Sourcing Data from Adaptive Planning ........................................................................................ 111How to establish connectivity to Adaptive Planning plugin ................................................112How to collect data from Adaptive Planning plugin ............................................................116
Sourcing Data from Basecamp .................................................................................................... 121Establish Connectivity to Basecamp ................................................................................. 122How to Collect Data using Basecamp Plug-in................................................................... 127
Sourcing data from Business Objects .......................................................................................... 131Establish Connectivity to Business Objects ................................................................ 132Collect Data from a Business Objects Document .................................................. 138
Sourcing Data using Elastic Search ............................................................................................. 146Establish Connectivity to Elastic Search ........................................................................... 147How to Collect Data using Elastic Search Plug-in............................................................. 151
Sourcing Data from Facebook...................................................................................................... 154Establish connectivity to Facebook ................................................................................... 155How to Collect Data from Facebook ................................................................................. 156
Sourcing Data from Fogbugz ....................................................................................................... 159Establish connectivity to FogBugz .................................................................................... 160How to collect data from Fogbugz..................................................................................... 164
Sourcing data from Google Analytics ........................................................................................... 169Establish connectivity to Google Analytics........................................................................ 170
Page 2Connecting To Data Sources
How to fetch data from Google Analytics .......................................................................... 175Collecting data from Google BigQuery ......................................................................................... 179
Establish connectivity to Google BigQuery ...................................................................... 180How to fetch data from Google BigQuery ......................................................................... 185
Sourcing Data from Google Calendar .......................................................................................... 192How to establish connectivity to Google Calendar............................................................ 193How to fetch data from Google Calendar.......................................................................... 197
Sourcing Data from a Google Spreadsheet ................................................................................. 203Establish connectivity to Google Spreadsheets ................................................................ 204Collect data from a Google Spreadsheet .......................................................................... 210
Sourcing Data from Graphite........................................................................................................ 215Establish Connectivity to Graphite .................................................................................... 216
Sourcing Data from IBM CoreMetrics........................................................................................... 220Establish connectivity to IBM Coremetrics ....................................................................... 221
Sourcing Data from Marketo ........................................................................................................ 225Establish Connectivity to Marketo ..................................................................................... 226
Sourcing Data from Mixpanel ....................................................................................................... 231How to Collect Data using Mixpanel Plug-in ..................................................................... 232
Sourcing Data using OLAP .......................................................................................................... 235Establish Connectivity to OLAP ........................................................................................ 236How to Collect Data using OLAP Plug-in .......................................................................... 241
Sourcing Data from Omniture....................................................................................................... 244Establish Connectivity to Omniture ................................................................................... 245
Sourcing Data from QlikView........................................................................................................ 250Getting ready for using Qlik with Metric Insights ............................................................... 251Establish Connectivity to QlikView .................................................................................... 253How to Collect Data using QlikView Plug-in...................................................................... 259
Sourcing Data using RSS............................................................................................................. 263Establish connectivity to RSS ........................................................................................... 264Collect data from RSS....................................................................................................... 268
Sourcing data from Salesforce ..................................................................................................... 272What is the difference between SalesForce plug-in and SalesForce SOQL plug-in? ....... 273Establish connectivity to Salesforce.................................................................................. 274How to collect data from Salesforce.................................................................................. 279
Sourcing data from Salesforce SOQL .......................................................................................... 287Establish connectivity to Salesforce SOQL....................................................................... 288Setting up Salesforce OAuth............................................................................................. 292How to collect data from Salesforce SOQL....................................................................... 296What is the difference between SalesForce plug-in and SalesForce SOQL plug-in? ...... 299
Sourcing Data from Tableau......................................................................................................... 300Hosting choices for Tableau Server .................................................................................. 301Hosting choices for Tableau Online................................................................................... 303Establish Connectivity to Tableau Server.......................................................................... 304Establish Connectivity to Tableau via a Remote Data Collector ........................................311Establish connectivity to Tableau Online........................................................................... 320How to collect data from Tableau ...................................................................................... 326How to create an External Report from Tableau ............................................................... 328Getting Started with Push Intelligence for Tableau ........................................................... 331Best practices for Tableau................................................................................................. 335Getting latest data from Tableau ....................................................................................... 341Embed database credentials in Tableau ........................................................................... 343Handling Date/Time columns in Tableau........................................................................... 345
Page 3Connecting To Data Sources
Sending a Tableau Workbook ........................................................................................... 350Using SSO on Tableau Server .......................................................................................... 351
Sourcing data from Treasure Data ............................................................................................... 354Installing the Treasure Data Plugin .................................................................................. 355Establish connectivity to Treasure Data ........................................................................... 356How to collect data from Treasure Data............................................................................ 363
Sourcing Data from Uptimerobot .................................................................................................. 368Establish Connectivity to Uptimerobot .............................................................................. 369How to Collect Data using Uptimerobot Plug-in ................................................................ 374
Sourcing Data using a Web Service............................................................................................. 377Connecting to a Web Service............................................................................................ 378
Page 4Connecting To Data Sources
General Information about Data Sources
Page 5Connecting To Data Sources
Understanding Data SourcesGENERAL:
There are two types of Data Sources:
Configurable: Are Data Sources for which the source database and access credentialsare user-defined. Each Metric Insights customer can establish and maintain SQL-basedaccess to internal databases and Plug-ins connections to external sources, including WebServices and a wide range of providers
Non-Configurable: Are received by all customers and require no customer maintenanceor configuration
A. Configurable Data Sources
You can use either of the following to pull data into Metric Insights:
SQL: Any source accessible through a JDBC driver is considered a SQL data source.This includes traditional databases as well as NoSQL sources with JDBC access; e.g.
Hadoop Hive. Plug-in: Are used to obtain Data that is not fetched using a JDBC driver and is collected
using a Plug-in. Plug-ins are special components, built by Metric Insights, that send anative fetch command to a Data Source. Results are fetched in a way that allows MetricInsights to consume the data. A list of supported Plug-ins can be found here.
NOTE: Web Services are considered to be a Plug-in configured to access data from a customWeb Service
B. Non-Configurable Data Sources available to all customers are:
Manual/CSV Data: Enter data manually or upload from a csv file Enter Data Manuallyand Upload Data from CSV provide additional information and links.
External Process: Push data to Metric Insights (does not require a credential set, butcan authenticate if desired); a Data Posting URL is provided See Source Metric Datafrom an External Process for more information
Existing Reports: Use data in one or more Reports to populate a new Report: See:Source a Report from another Report
Existing Metrics: Obtain data in one or more Metrics to populate a new Metric or Report:See: Source a Report from a Metric
Local Excel File: Upload data (on PC's only) from an MS Excel file
C. Remote Data Collectors
If a Data Source requires a Remote Data Collector, see Configure a Remote DataCollector for more information
1. Data Source Editors
Information on creating new Data Sources can be found at:
Page 6Connecting To Data Sources
SQL Data Sources
Plug-in Data Sources
2. Select a Data Source on an Element Editor
Once you have completed the Add Element pop-up and are directed to the Metric or ReportEditor, the browser automatically scrolls down to the Metric/Report Calculation section
1. The Data Source drop-down list contains all of the SQL and Plug in Data Sources thathave been defined for your instance as well as all non-configurable Data Sources
2. You still have to select a Data Collection Trigger that fetches the data3. Based on your Data Source selection, you either see a Plug-in Command or SQL
Statement text box into which you enter the fetch command
Page 7Connecting To Data Sources
3. Select a Data Source on a Functional Editor
The Event, Dimension and Data Dependency Editors all have data collection settings similar tothose on the Element Editors
The example above shows the Dimension Editor
Page 8Connecting To Data Sources
Create New SQL Data SourceYou can use a JDBC driver to pull data into Metric Insights using a SQL fetch command. Thisincludes traditional databases as well as NoSQL sources with JDBC access; e.g. Hadoop Hive.
This article describes the general process for creating a SQL data source.
Information on creating a data sources based on a plug-in can be found here.
For a description of Metric Insights overall approach to Data Sources, click here.
1. Open Data Source list
2. Create New Data Source
Page 9Connecting To Data Sources
2.1 Choose type
1. Select "SQL"2. Click Next step
Page 10Connecting To Data Sources
2.2 Complete all settings
1. Provide a descriptive Name for the connection.
2. Enter the Username.
3. Enter a Password.
4. Enter the Host Name.
5. The Port number will be set by default, based on your choice of JDBC Driver. Change it ifnecessary.
6. Enter the Database name.
Page 11Connecting To Data Sources
7. Select a JDBC Driver. (If you do not see the the driver that you wish to use in the drop-downprovided, contact Metric Insights for assistance.)
8. Optionally, specify the maximum number of concurrent Threads per Trigger Event to be usedin background processing when the system updates metrics and reports for this data source. Ifyou do not specify any value for this setting, batch data collection processing will be single-threaded.
9. If you select "Yes" to specify that this Data Source is remote; i.e., behind a firewall, you will berequired to select a Connector or create a new one. See Define a Remote Data Collector for moredetails.
10. Save.
11. Click Test Connection to ensure that settings are correct. (Some data sources, e.g., Hive,cannot be tested this way.)
Note: The JDBC string will be created automatically based on your other inputs. In some cases,however, it will not be possible to infer the correct string without additional inputs. If the ConnectionTest fails, please check the documentation for your JDBC driver.
2.3 Review and close the confirmation
1. If the connectivity is established, the confirmation message appears; click OK to continue
NOTE: It is not possible to directly test Hive connectivity from the connection editor since there isno standard query that can be run against a HiveQL instance. Contact Metric Insights forassistance in validating a HiveQL connection.
Page 12Connecting To Data Sources
Create a New Plug-in Data SourceData that is not fetched using a JDBC driver is collected using a Plug-in. Plug-ins are specialcomponents, built by Metric Insights, that send a native fetch command to a data source. It thenformats the results in a way that allows Metric Insights to consume the data. A list of supportedPlug-ins can be found here. It is also possible to access data from a custom web service by usinga Web Service Plug-in.
This article describes the general process for creating a Plug-in data source.
Information about how to create a new SQL data source is available here.
For a description of Metric Insights overall approach to Data Sources, click here.
1. Open Data Source list
2. Create New Data Source
Page 13Connecting To Data Sources
3. Choose type
1. Select "Plug-in"2. Click Next step
Page 14Connecting To Data Sources
4. Complete all settings
1. Select a Plug-in from the drop-down list of configured Plug-ins. If you do not find the oneyou need, contact your system Administrator or support@metricinsights.com
2. Create a unique name for this data source3. If you select "Yes" to Use Remote Data Collector?, you will be required to select a
collector or create a new one For more information, reference Define a Remote DataCollector
4. When the supported plug-in is configured, the required Plug-in Connection ProfileParameters are defined and are defaulted into the Parameters grid once you select thePlug-in setting
5. Any of these parameters may be edited by clicking the Edit link
Page 15Connecting To Data Sources
4.1 Edit Parameters
1. Use the Edit icon to enter/modify the parameter2. Change the parameter to be passed to the Data Source
Save
Repeat this process as necessary to update any/all parameters.
Save again before leaving the Editor so that your Data Source will be ready for use in defining anElement.
Page 16Connecting To Data Sources
What does the Refresh Metadata button do?Learn what the Refresh Metadata button does in the floating save bar in the SQL Data SourceEditor.
For SQL data sources, if using the visual editor (1), table/column metadata from the data base iscollected periodically and cached on the Metric Insights server. When users go into the VisualEditor from an Editor, the cached metadata is used instead of making a database hit to pull uptable/column information.
The Refresh Metadata button (2) allows you to force the refresh of this metadata.
Page 17Connecting To Data Sources
Metric Insights' Plugins
About Plugins
Metric Insights has a flexible plug-in infrastructure which allows us to connect to many differentdata sources to pull metrics into the KPI warehouse.
In order for Metric Insights to have access to your data source, however, in some cases it isnecessary to install Insightd (a thin remote data collection process) on a machine inside yournetwork. If a plug-in requires Insightd to be running on a machine in your network, it will be notedbelow.
Note: Even if a plug-in is listed as not requiring Insightd, if your network configuration is such thatyour data source is locked down and the Metric Insights server does not have access to it, you willstill need Insightd to run on a machine behind your firewall that has network access to your datasources. General stand-alone installations of Metric Insights should not require this, however
The list below includes our most popular plug-ins. If you don't see the the plug-in that you need inthe list below, please contact Metric Insights.
Supported Plug-in List
Software Description Insightd Required?
1010 Data No
AdapativeInsights
No
AmazonRedShift
No
BOBJ Business Objects Plugin Yes - on the businessobjects server
CoolaData No
Elasticsearch No
Facebook Facebook plugin No
Flat File Periodically import CSV files from a directory ona server
No
GoogleAnalytics
Google Analytics php plugin No
Page 18Connecting To Data Sources
GoogleBigQuery
No
GoogleCalendar
Google Calendar plugin for importing a googlecalendar as an event calendar
No
GoogleSpreadsheet
No
Graphite Graphite Plugin No
Hadoop Pig Pig Latin plugin that runs against Hadoop. Laststatement must be a DUMP statement
Yes. On a node configuredto run Pig against yourcluster
IBMCoremetrics
IBM Coremetrics plugin No
Marketo Marketo plugin No
MicroStrategy No
Mixpanel Mixpanel plugin No
MongoDB MongoDB Shell command plugin. Last statementmust include printjson() statement
No
OLAP Use the MDX query language to query OLAPcubes like those in Microsoft SSAS
No
OmnitureReporting
Omniture is an online marketing and webanalytics tool offered by Adobe Systems
No
QlikView Yes
QlikSense Yes
Salesforce Salesforce plugin to pull in metrics from thepopular online CRM software
No
Splunk No
SQLDatabases
Relational databases and other datastores with aJDBC driver, like Hadoop Hive, MySQL,Postgres
No
SugarCRM No
Tableau No
Teradata No
Page 19Connecting To Data Sources
TreasureData Treasure Data Plugin No
Web Service Periodically hit a webservice to get measurementvalues
No
Page 20Connecting To Data Sources
Registering Your Server with Google to use their Plug-inResourcesYou must register with Google for every application instance (domain) required access toBigQuery, Analytics, Calendar and other plug-in services that Google offers. The result allows youto obtain a client ID and secret code.
1. Google APIs Console
Go to: https://code.google.com/apis/console/?api=plus
1. You can use the current Google console as seen here. However, the steps in thisdocument use the "old console".
2. Choose the Go back link (yellow section) to use the "old console".
2. Register with Google
1. Click the Services tab and select services for instance2. Go to API Access tab in the left panel.
Page 21Connecting To Data Sources
3. Create a new Client ID
1. Click Create an OAuth 2.0 client ID
3.1 Create Client ID
Provide
Page 22Connecting To Data Sources
1. Product name2. Product logo3. Home Page URL
3.2 Create a new Web Application client id for your Metric Insights server
1. Select Web application
2. Enter site or hostname for your Metric Insights instance (in this example the hostname isdocument.metricinsights.com)
3. Click Create client ID
3.3 Record Client ID and Client Secret for use in application configuration file
1. Record Client ID
Page 23Connecting To Data Sources
2. Record Client secret
3. Click Edit settings...
3.4 Edit client settings
1. For Authorized Redirect URIs, enter /path/to/callback value as /editor/service/validategoogleoauth2 In this example where the hostname is document.metricinsights.com, thefull value becomes https://document.metricinsights.com/editor/service/validategoogleoauth2
Note: This value is referenced by Metric Insights in const.php config file via:
define('GOOGLE_OAUTH2_REDIRECT_URI', HOSTNAME.'/editor/service/validategoogleoauth2');
2. For Authorized JavaScript Origins, leave the default that it provides. In this example wherethe hostname is document.metricinsights.com, the default is https://document.metricinsights.com
4. Store the client ID credentials on the Metric Insights server
SSH to the Metric Insights server
1. Edit the file /var/www/iv/engine/config/const.php
# vi /var/www/iv/engine/config/const.php
2. Enter the values provided above for the variables shown below:
Modify the lines below inserting values obtained from Google for your host:
Page 24Connecting To Data Sources
define('GOOGLE_OAUTH2_CLIENT_ID', '761272366824.apps.googleusercontent.com');define('GOOGLE_OAUTH2_CLIENT_SECRET', 'yT4nDBBppanARIbWhU5WHNLG');
Page 25Connecting To Data Sources
Populate a Data Source with an uploaded List of ReportsA list of reports may be "stored" within a plug-in Data Source after being uploaded from a file,either csv or with some other type.
1. Create a Plug-in Data Source
Page 26Connecting To Data Sources
2. Upload List from a txt file
1. Click Load from CSV File2. Browse and select a file3. Click Import4. Review results of upload5. Review the list of External Reports in the grid
Page 27Connecting To Data Sources
3. Create an External Report using the Data Source
Create a new External Report, paying special attention to these settings:
1. Report Type: Choose a type consistent with the Data Source being used (In this example,Tableau)
2. Show Report in: For this example, choose "Viewer"3. Plug-in Connection Profile: Select an option consistent with the Data Source being
used (In this example, Tableau)4. External Report: Pick a report that has the desired data from the list of uploaded Reports5. External Report URL: Populated automatically once you choose a report6. Collect Image
Page 28Connecting To Data Sources
4. Review Image
The image reflects the Report selected from the list stored in the Data Source
Page 29Connecting To Data Sources
How to Link to External BI Tools
Page 30Connecting To Data Sources
What are external reports?This will describe how Metric Insights interfaces with other 3rd party BI Tools like Tableau,Microstrategy, etc.
Overview
When creating a new metric, the last two choices allow you to link to reports you've alreadydeveloped in other BI platforms, like Tableau, Crystal Reports, or Microstrategy. The differencebetween an 'External Report' and 'Other External Content' is that with an External Report youcan actually drill down from another Metric Insights tile into the specified external report. ExternalContent, however, is just any URL of your choosing that will not be passed in any dynamicparameters from Metric Insights.
URL Templates
For Other External Content, any old static url is defined for each report. However, for ExternalReports that you wish to drill down to, you can use a URL template that will use the followingvariable place holders
:measurement_start_time :measurement_end_time and, if the metric is dimensioned, by the dimension's bind_variable. So, if the metric is
dimensioned by sales channel, for example, you might also have the :channel templatevariable
Page 31Connecting To Data Sources
Thumbnails
Metric Insights tiles will actually auto update and generate values based on the value of the metric.In contrast, external report thumbnails will just be a static icon that the report developer sets.
Sometimes report developers will actually take a screenshot of the external report and store it withthe external report definition to allow for a more realistic and seamless transition from MetricInsights to the external BI tool.
External tool authentication
Each URL that Metric Insights will link too will, of course, require some form of login for thatsystem. Metric Insights will not be able to bypass authentication to any other system.
End result
Once an external report is created, a business user will be able to seamlessly link to a variety ofrelevant reports all from their own personal list of favorites or report categories they have accessto.
Page 32Connecting To Data Sources
How to link to External report contentThis article describes creating a new 'External Content' that will be linked from a Metric Insights. Ifyou wish to drill to External content based on dimension values, look at how to set up drill paths forexternal reports
1. On any page, use Admin Menu to begin creation of the element
Page 33Connecting To Data Sources
2. Create a new External Content report
Select Other External Content
Page 34Connecting To Data Sources
3. Set up the Metric Insights report metadata
Even though the report is already created in your external BI tool, for related metrics MetricInsights still need to know a couple of things about this particular content.
To facilitate this, set
1. Measurement Interval,2. Measure of,3. Category,4. and Topic
That way, if you're looking at another monthly report that happens to look at some measure, yourexternal content report will also show up to the user as a related report.
Page 35Connecting To Data Sources
4. Set up the external content URL
1. In this case, we'll just use a static web page that we've created just for examples:http://external.metricinsights.com/daily_sales_country.php .
2. Select 'SAVE' to display External Content Editor
Page 36Connecting To Data Sources
5. Select where to display the content
1. Select to display your content in either MetricInsights Viewer or on the External webpage2. Select 'Upload Image File' to display pop-up where you can add a Preview Image
Page 37Connecting To Data Sources
6. Set preview image
Take a screenshot of your report and upload it as the preview image. This image should be a .pngfile and to avoid distortion should have a width to height ratio of 1.8 (in other words, the width isnearly twice the height).
Page 38Connecting To Data Sources
7. Enable and Publish
1. After you're satisfied, save2. Make visible on the Home Page3. Enable and Publish
Page 39Connecting To Data Sources
8. Verify that the new external content
Now that it's enabled, make sure the new external content tile properly shows up in the HomePage. Double clicking on the report should take you to the url you expect.
Page 40Connecting To Data Sources
How to link to a Parameterized External reportThis article describes creating a new 'External Report' that will be linked from a Metric Insights.
1. Create a new External Content report
From the Add a new Element menu in Metric Insights, select External Report.
Page 41Connecting To Data Sources
2. Set up the Metric Insights report metadata
Even though the report is already created in your external BI tool, for related metrics MetricInsights still need to know a couple of things about this particular content.
To facilitate this, set the
1. Measurement Interval2. Measure of3. Category4. Topic
That way, if you're looking at another monthly report that happens to look at some measure, yourexternal content report will also show up to the user as a related report.
Page 42Connecting To Data Sources
3. Choose the report type or add a new one
A report type is used for organizational purposes. In addition, though currently unimplemented, itwill allow administrators to associate an icon url for each report type that allows users todifferentiate between report types when scrolling through your list of favorites.
NOTE: This will be auto-populated in future releases.
Page 43Connecting To Data Sources
Create the report type
Only Name field is required
4. Set up the external report URL template
External report URLs can by default take special parameters to help get relevant information. Forexample, a report may take a start_date and end_date parameters when running. To facilitatethis, Metric Insights has the following variables available to use in the report URL template.
:measurement_start_time - Metric Insights will automatically choose a reasonable startand end times based on the report frequency and drill down path. For example, if you'relooking at a Monthly Sales report, and you click on January 2012, the:measurement_start_time will be 'Jan 01, 2012 00:00:00' with :measurement_end_time ='Jan 31, 2012 23:59:59'
:measurement_end_time - see above. for dimensioned metrics only: a dimension name. For example, if your Dimension is
'Country' and you set up the Dimension to have a bind variable name of 'country_id', youwould be able to use :country_id in your URL template.
Page 44Connecting To Data Sources
4. Set up a screen shot for preview images
Take a screenshot of your report and upload it as the preview image. This image should be a .pngfile and to avoid distortion should have a width to height ratio of 1.8 (in other words, the width isnearly twice the height)
Page 45Connecting To Data Sources
5. Save and enable the metric
After you're satisfied, save and enable the external report so that it may be visible in thedashboard.
Page 46Connecting To Data Sources
6. Verify the new external content
Now that it's enabled, make sure the new external content tile properly shows up in the Tile Viewat Home Page. Double clicking on the report should take you to your external BI tool where you'llneed to login
Page 47Connecting To Data Sources
Configuring Remote Data Collectors
Page 48Connecting To Data Sources
Overview of Remote Data Collection
Overview
In order to safely connect a Metric Insights instance that is hosted on the cloud to data sourcesthat live behind your firewall, you will need to download Insightd, a thin, remote data collectionframework that can be used to run queries on Metric Insights behalf.
Insightd will run on a machine behind your firewall and will periodically poll Metric Insights for thequeries it needs to run. Once these queries are assigned and completed, the results are thensecurely posted back to the Metric Insights server to get turned into your visualizations.
Page 49Connecting To Data Sources
Configure a Remote Data CollectorIn order to safely connect a Metric Insights instance that is hosted on the cloud to data sourcesthat live behind your firewall, you will need to download Insightd, a thin, remote data collectionframework that can be used to run queries on behalf of Metric Insights.
Insightd will run on a machine behind your firewall and will periodically poll Metric Insights for thequeries it needs to run. Once these queries are assigned and completed, the results are thensecurely posted back to the Metric Insights server to get turned into your visualizations.
1. Select 'Remote Data Collectors' from Admin menu
Page 50Connecting To Data Sources
2. Add a new remote data collector
1. Give your new remote data collector a name2. Enter the hostname of your remote data source
3. Connect a data source to the new remote data collector
3.1 Click the name of the new remote data collector to edit it
Page 51Connecting To Data Sources
3.2 Add a SQL database or Plug-in data source to the remote data collector
Note: A single remote data collector can connect to multiple data sources if they are all accessiblethrough the same remote host.
Page 52Connecting To Data Sources
3.3 Choose the SQL database or Plug-in connection that you want to connectto
Note: We will connect to Tableau for demonstration purposes.
Page 53Connecting To Data Sources
4. Finally, download the Insightd installer on to the machine yourserver
As mentioned in the overview, the insightd installer will need to run on a server behind yourfirewall. Metric Insights allows you to download a pre-configured package of Insightd to install onyour machine.
5. Install insightd
Now, copy the downloaded zip file to the server that will be running Insightd.
Please refer to the following sections to install insightd on Windows and Linux servers.
Page 54Connecting To Data Sources
Installing Insightd on Windows ServersThis article assumes you have configured a remote data collector install, as described inConfiguring a Remote Data Collector
1. Download the Insightd installer from Metric Insights
On the Windows Server that you plan to install on, open a web browser and point it to your metricinsights server. Log in as admin, and go to Admin -> Remote Data Collectors.
2. Select the data collector that you wish to download
Page 55Connecting To Data Sources
3. Download the Insightd zip file
When downloading the zip file, you might get an error message like the following that warns youthat the insightd zip isn't normally downloaded. Tell the browser that you're sure you want the zipfile. We promise it's not dangerous.
Page 56Connecting To Data Sources
4. Extract the zip file and run the installer executable.
In a Windows Explorer window, right click on the downloaded zip file and extract all the insightdfiles to a directory.
Page 57Connecting To Data Sources
4.1
4.2
5. Configure the Business Objects SDK location if desired
If you wish to install the SAP Business Objects plugin, the location of the SDK is necessary. Thisshould be the directory where all the Business Object jar files live (usually something like
Page 58Connecting To Data Sources
C:\Program Files\Business Objects\Common\4.0\java\lib). If you are not using the BusinessObjects plugin, just use the default.
When done, click 'Close' to finish the installer.
Page 59Connecting To Data Sources
6. Start the Insightd Windows Service
Navigate to the Windows Services list and start the Metric Insights Daemon service
Page 60Connecting To Data Sources
7. Check the Windows Event log for any errors
If you notice that the service hasn't started for some reason, check the event log for moreinformation. There will be a special folder in the Event Log for the Metric Insights Daemon.
In the case below, Insightd failed to start up because it couldn't reach the Metric Insights server forsome reason. This can happen if port 8443 is not open on your Metric Insights instance.
Page 61Connecting To Data Sources
Upgrading Insightd on Windows ServersThis article walks you through the steps to upgrade your version of Insightd on Windows Servers.Insightd is the Windows service that runs the Remote Data Collector for your Metrics Insightsapplication. You can review more about what this is in the Overview of Remote Data Collection.
http://help.metricinsights.com/m/Connecting_to_Data_Sources/l/107311-overview-of-remote-data-collection
1. Get new version
To get a new version, follow the steps on how to download the new version in the article forInstalling Insightd on Windows Servers.
Start with Step 1 and continue to Step 3, the steps that explain how to download the Insightdinstaller.
http://help.metricinsights.com/m/Connecting_to_Data_Sources/l/107314-installing-insightd-on-windows-servers
2. Uninstall old version
After getting the new version of the Remote Data Collector in the previous step, the next steps areto uninstall the old version and then to install the new version.
The next several steps walk you through how to uninstall the old version of the Remote DataCollector.
Page 62Connecting To Data Sources
2.1 Backup config.ini file
Before uninstalling the old version of the Remote Data Collector, make a backup copy of yourconfig.ini file (located at "Metric Insights\Insightd\config\config.ini") and save it somewhere such asyour home directory.
Page 63Connecting To Data Sources
2.2 Stop Windows Service
Now stop the Remote Data Collector if it is running. Find the Metric Insights Daemon in theWindows Services control panel. If it is running then the panel will show you an option to Stop theservice and to Restart the service. If the service is not running, then it will only show you an optionto Start the service.
If the service is running, then Stop the windows Insightd service.
Click Stop the service.
Page 64Connecting To Data Sources
2.3 Uninstall Windows Service
Once the Insightd Windows Service is stopped, then it is safe to uninstall the service.
Uninstall the Insightd service.
Run the uninstall-insightd.exe file located at "Metric Insights\Insightd\bin\uninstall-insightd.exe". You can double click on it to run it.
3. Install new version
Finally, after completing the above steps of getting the new version of the Remote Data Collector,and then uninstalling the old version, the next step is to install the new version.
The next several steps walk you through how to install the new version of the Remote DataCollector.
Page 65Connecting To Data Sources
3.1 Install the Windows Service
To install the Insightd Windows Service, follow the steps in the article Installing Insightd onWindows Servers that explains how to extract the installer from the downloaded file and then howto install the service. Only go as far as running the installer. Do not start the Insightd WindowsService yet. Follow Step 4 through Step 5 in the article Installing Insightd on Windows Servers.
http://help.metricinsights.com/m/Connecting_to_Data_Sources/l/107314-installing-insightd-on-windows-servers
If you get "Error opening file for writing: ...Win32EventLogAppender.dll" just click Ignore andcontinue.
Page 66Connecting To Data Sources
3.2 Restore config.ini
Copy the config.ini file you saved in a prior step to the "Metric Insights\Insightd\conf\" directory.
3.3 Start the Windows Service
Finally, start the Insightd Windows Service. Continue at Step 6 in the article for Installing Insightdon Windows Servers.
http://help.metricinsights.com/m/Connecting_to_Data_Sources/l/107314-installing-insightd-on-windows-servers
Page 67Connecting To Data Sources
Installing Insightd on Linux ServersThis article assumes you have configured a remote data collector install, as described inConfiguring a Remote Data Collector
1. Download the Insightd installer from Metric Insights
On the Linux Server that you plan to install on, open a web browser and point it to your metricinsights server. Log in as admin, and go to Admin -> Remote Data Collectors.
2. Select the data collector that you wish to download
Select the remote data collector you wish to install.
Page 68Connecting To Data Sources
3. Download the Insightd zip file
When downloading the zip file, you might get an error message like the following that warns youthat the insightd zip isn't normally downloaded. Tell the browser that you're sure you want the zipfile. We promise it's not dangerous.
4. Extract the zip file and start Insightd
root@document:~# unzip remote_data_collector.zip
root@document:~# cd datacollector/bin
root@document:~# ./insightd start &
Page 69Connecting To Data Sources
Upgrading Insightd on Linux ServersThis article walks you through the steps to upgrade your version of Insightd on Linux Servers.Insightd is the Linux daemon that runs the Remote Data Collector for your Metrics Insightsapplication. You can review more about what this is in the Overview of Remote Data Collection.
http://help.metricinsights.com/m/Connecting_to_Data_Sources/l/107311-overview-of-remote-data-collection
1. Uninstall old version
The following steps walk you through how to uninstall the old version of the Remote DataCollector.
1.1 Stop Insightd
Stop the Remote Data Collector if it is running. Find the location where the remote data collector isrunning and stop it.
Stop the current remote data collector:
./datacollector/bin/insightd stop
If it's not running then you might get the following message:
"The insightd can not be stopped because it is not currently running."
1.2 Save Insightd (save current configuration)
Move files to a saved location. You can give the name of the new directory anything you want.
mv ./datacollector/ ./datacollector-todays-date
2. Get new version
To get a new version, follow the steps on how to download the new version in the article forInstalling Insightd on Linux Servers.
Start with Step 1 and continue to Step 3, the steps that explain how to download the Insightdinstaller. At this point do not extract the zip file. Do not start up Insightd.
http://help.metricinsights.com/m/Connecting_to_Data_Sources/l/107315-installing-insightd-on-linux-servers
Page 70Connecting To Data Sources
In rare cases you might receive the remote data collector via a different means. For example, fromMetric Insights company directly. For example you might get a patched version or one with a newfeature. In that case just follow the directions given for that situation.
3. Install new version
The next several steps walk you through how to install the new version of the Remote DataCollector.
3.1 Install Insightd
Install Insightd
unzip remote_data_collector.zip
This will install the remote data collector (i.e., extract the contents of the zip file) into the./datacollector directory. If you obtained this file via a different means then it might have a differentname.
3.2 Check if new configuration is correct
Check if new configuration is correct
cat ./datacollector/conf/config.ini
If you downloaded this from your Metric Insights system described in the above steps, then thisstep is not necessary. If you obtained the remote data collector via a different means then this stepis recommended.
3.3 Restore Configuration (if new configuration not correct)
If you downloaded this from your Metric Insights system described in the above steps, then thisstep is not necessary. However, if you obtained it via a different means and you need to restoreyour previous configuration for some reason then you can do the following
mkdir -p ./datacollector/confcp -p ./datacollector-todays-date/conf/config.ini ./datacollector/conf/
3.4 Start Insightd
Start insightd
./datacollector/bin/insightd start &
Page 71Connecting To Data Sources
Troubleshooting a Remote Data CollectorThis article provides some tips for troubleshooting problems with your Remote Data Collector
1. Check if remote data collector is active
From the Admin menu navigate to the Remote Data Collectors screen.
The Remote Collectors screen lists all your Remote Data Collectors and provides a visual queuewhether the remote data collector is active. It also lists the last time the collector successfullycommunicated with the system.
Page 72Connecting To Data Sources
1.1 Active remote data collector
If your remote data collector is active then it will show with white background, and the column for"Last Heartbeat Time" will have a value within a couple minutes of the current system time.
1.2 Inactive remote data collector
If your remote data collector is not active then it will show with pink background, and the columnfor "Last Heartbeat Time" will have a value with the last time it successfully connected. The imagebelow is for a remote data collector that is not active - the Last Heartbeat Time is old and thebackground is pink. You will need to troubleshoot more.
2. Restart the remote data collector
On the machine that the remote data collector is running, stop and restart the remote datacollector
Page 73Connecting To Data Sources
2.1 Windows - Restart remote data collector
Find the Services menu on the Windows machine and stop and restart the remote data collector.The service is named 'Metric Insights Daemon'
2.2 Linux - Restart remote data collector
Find the directory where the remote data collector is running and stop and restart the process. Seeinstructions for where you installed the program.
Stop the remote data collector:
/var/www/datacollector/bin/insightd stop
Start the remote data collector:
/var/www/datacollector/bin/insightd start
Page 74Connecting To Data Sources
3. Check error messages from the remote data collector on theremote machine
On the machine that the remote data collector is running, view the error messages from theremote data collector.
3.1 Windows - Check error messages from the remote data collector on theremote machine
Find the Event Viewer menu on the Windows machine and choose the events for the servicenamed 'Metric Insights Daemon'. Errors are flagged in red and provide information on problemsthat the remote data collector encountered.
3.2 Linux - Check error messages from the remote data collector on theremote machine
View the errors in the log.
cat /var/log/insight/insightd-error.log
Page 75Connecting To Data Sources
4. Check error messages on Metric Insights server
View the errors in the log.
Debian instance:
cat /var/log/apache2/pyinsight-error.log
CentOS instance:
cat /var/log/httpd/pyinsight-error.log
5. Verify correct settings in the remote data collector configurationfile
Page 76Connecting To Data Sources
5.1 Verify settings in the UI on Metric Insights
1. From Admin menu navigate to Data Collectors screen.2. From Data Collectors screen navigate to Remote Collector Editor screen by clicking on the
data collector entry in the list.3. Note the setting values for "Username for remote agent" and "Password for remote agent".
You will compare these to what is used on the remote data collector machine. In thisexample the values are "tableau.metricinsights.com" and "myPassword123".
4. Note the domain of your Metric Insights instance. In this example it is"demo.metricinsights.com" as seen in the web browser url.
Page 77Connecting To Data Sources
5.2 Windows - Verify settings in configuration file on remote machine
Navigate to the location of the configuration file used by the remote data collector
C:\Program Files (x86)\Metric Insights\Insightd\conf
1. View the configuration file "config.ini". You can view it with Notepad editor.2. Compare the setting values for "hostname" with "Username" in previous step. Do the same
for "password". In this example the values are "tableau.metricinsights.com" and"myPassword123".
3. Compare the setting value for "requestURL" with the domain in previous step. In thisexample the value is "demo.metricinsights.com".
5.3 Linux - Verify settings in configuration file on remote machine
Find the directory where the remote data collector is running and view the configuration file"config.ini". See instructions for where you installed the program.
View the configuration file "config.ini". You can use vi editor or print it to screen as in the examplebelow.
cat /var/www/datacollector/conf/config.ini
Page 78Connecting To Data Sources
[authentication]hostname = tableau.metricinsights.compassword = myPassword123requestURL = demo.metricinsights.com[parameters]FetchRestartWait = 300PollingInterval = 1PostingRestartWait = 300PostingTimeoutInterval = 10HeartbeatAlertInterval = 10PostingRestartsAllow = 3CacheDNSLookup = 0FetchRestartsAllow = 3FetchTimeoutInterval = 30
1. Compare the setting values for "hostname" with "Username" in previous step. Do the samefor "password". In this example the values are "tableau.metricinsights.com" and"myPassword123".
2. Compare the setting value for "requestURL" with the domain in previous step. In thisexample the value is "demo.metricinsights.com".
Page 79Connecting To Data Sources
JDBC Connections
Page 80Connecting To Data Sources
Establishing a JDBC ConnectionBefore you can run any queries, you will need to set up a connection to your data sources.
1. On any page, use Admin Menu to begin creation of the JDBCSource Database Connection
2. On Source Database Connections, open the pop-up
Page 81Connecting To Data Sources
3. Set type of new Data Source connection
Select SQL type
Page 82Connecting To Data Sources
4. Describe the Connection
1. Provide a descriptive Name for the connection.
2. Enter the Username.
3. Enter a Password.
4. Enter the Host Name.
5. The Port number will be set by default, based on your choice of JDBC Driver. Change it ifnecessary.
6. Enter the Database name.
Page 83Connecting To Data Sources
7. Select a JDBC Driver. (If you do not see the the driver that you wish to use in the drop-downprovided, contact Metric Insights for assistance.)
8. Optionally, specify the maximum number of concurrent Threads per Trigger Event to be usedin background processing when the system updates metrics and reports for this data source. Ifyou do not specify any value for this setting, batch data collection processing will be single-threaded.
9. Save.
Note: The JDBC string will be created automatically based on your other inputs. In some cases,however, it will not be possible to infer the correct string without additional inputs. If the ConnectionTest fails, please check the documentation for your JDBC driver.
5. Validate created connection from Data Source Editor
Page 84Connecting To Data Sources
6. New connection can be found in Data Sources list
NOTE: the test feature does not currently work for testing connections to Apache Hive.
1. Click the Test Link to Verify connectivity to the data source.
2. If you need to further edit the connection, click on the name link of the datasource to access thepop-up editor
7. Confirm success
1. If the connectivity is established, the confirmation message appears; click OK to continue
NOTE: It is not possible to directly test Hive connectivity from the connection editor since there isno standard query that can be run against a HiveQL instance. Contact Metric Insights forassistance in validating a HiveQL connection.
Page 85Connecting To Data Sources
Add a new JDBC DriverThe JDBC Driver drop-down in the Add/Edit a SQL Data Source Editor includes the list of JDBCdrivers that have been installed on the Metric Insights server.
You may add to this list of JDBC drivers by following the steps below.
PRE-REQUISITE:
You must have the driver JAR file stored locally on your computer
For the last step you will need to request Metric Insights to Register this driver.
1. Obtain the JAR file
1. Identify the proper JDBC driver JAR file for the driver to be added2. Download to your local machine
NOTE: A list of sources for JDBC drivers can be found at: http://www.sqlsummit.com/JDBCVend.htm
2. Open the list of Data Sources
1. Using the Admin drop-down list, choose Data Sources
Page 86Connecting To Data Sources
2. Click the ADD icon to the right on the Data Sources grid field
Optionally, you can Edit an existing SQL Data Source on its editor by clicking its Name link andadd your new JDBC driver from there; if this is your approach, skip to Step 4
3. Add the new Driver
Select Add New Driver Jar File
Page 87Connecting To Data Sources
4. Select the new driver
1. Use the Browse button to find and Open the driver found the .jar file that you downloadedand saved to your local hard drive
2. Once the Jar file is in the text box, click SAVE
5. Register Driver in Metric Insights
Contact Metric Insights and Metric Insights will register this Driver in the system. What MetricInsights will do is add info to the system (mysql jdbc_driver table) that includes your jdbc driverinformation. E.g., jdbc string template to use (e.g., jdbc:postgresql://:/) , sampleSQL statement to test connectivity and jdbc class name to invoke in the driver (e.g.,org.postgresql.Driver).
NOTE: Once complete you will then be ready to create a Data Source using this new JDBC driver
Page 88Connecting To Data Sources
Validation errors: "Value '0000-00-00' can not be representedas java.sql.Date"ANALYSIS:
This is a MySQL-specific problem that does not happen often. It is caused by storing zero dates('0000-00-00 00:00:00') in MySQL and trying to convert those into date objects in Java.Unfortunately, Java does not understand dates in this format, so the MySQL JDBC driver willthrow this error by default.
RECOMMENDED ACTION:
Apart from modifying the source data, the easiest solution is to modify the JDBC connection URLfor the affected Data Source to include ?zeroDateTimeBehavior=convertToNull as shown below.
REFERENCES:
For more information about this JDBC parameter, see the Datetimes bullet under section20.3.3.3.3 of the MySQL Manual.
1. Problem examples:
1.1 Example: "Value '0000-00-00' can not be represented as java.sql.Date"
Page 89Connecting To Data Sources
1.2 Example: "Cannot convert value '0000-00-00 00:00:00' from column N toTIMESTAMP"
2. Solution:
2.1 Navigate to the data source editor for the affected data source
Make sure the data source drop down is pointing to the appropriate data source, then click thegear icon to the right to edit the data source.
Page 90Connecting To Data Sources
2.2 Fix the SQL Data Source URL
The suggested fix is to append '?zeroDateTimeBehavior=convertToNull' to the end of the JDBCstring. So the original string "jdbc:mysql://localhost:3306/demo" would become"jdbc:mysql://localhost:3306/demo?zeroDateTimeBehavior=convertToNull"
Page 91Connecting To Data Sources
Sourcing data from 1010data
Page 92Connecting To Data Sources
Establish connectivity to 1010data
1. Go to Data Source Editor
2. Add New Data Source
Page 93Connecting To Data Sources
3. Select "Plug-in" as data source type
4. Select '1010data' from Plug-in picklist
Page 94Connecting To Data Sources
5. Additional fields will be display for entry
1. Enter a unique Name for your data Source2. Set Use Remote Data Collector to "No"3. Enter Authorized Username and Password
Page 95Connecting To Data Sources
6. Profile Parameters are generated automatically
Generated parameters may be modified/added via the gear icons
7. Why dont normal 1010data UIDs work with Metric Insights?
Metric Insights will run queries in the background, if a user is using their UID (or two metricsdecide to update at the same time) the result will be one will fail or kick the other off.
8. What are SAM Pools?
SAM pools are a mechanism used to associate multiple sessions on the 1010data backend with asingle set of credentials.
Using SAM credentials allow Metric Insights instances to keep metrics up to date withoutinterfering with the user or each other.
Page 96Connecting To Data Sources
Collect data from 1010dataThis article will show you how to create a Metric using a 1010data plug-in as a Data Source. Itassumes that you have already established connectivity to your 1010data server.
1. Add New Element to start
2. Create a simple metric without wizard
Page 97Connecting To Data Sources
3. Define Basic Metric Information
Page 98Connecting To Data Sources
4. Adjust Metric Calculation block
1. Set 1010data plug-in in Data Source field2. Enter Table Name3. Complete the Macro Code fetch command4. Validate Plug-in Command
Page 99Connecting To Data Sources
5. Collect Data
1. If validation is finished successfully you will see how many records are fetched. Informationmessage is displayed in green font. If any error occurs during validation you will see errormessage in red font.
2. Collect Data
Page 100Connecting To Data Sources
6. View Chart
Page 101Connecting To Data Sources
Creating 1010data macro code for your 1st Basic MetricThis will show you how to create a very simple Metric - a visualization that measures sales overtime.
1. Build your macro code
Page 102Connecting To Data Sources
1.1 Formatting date to mm/dd/yyyy
Insert a willbe in the Edit Actions screen to reformat the date to a mm/dd/yyyy format. Here is theexample code:
Page 103Connecting To Data Sources
1.2 Select Rows to show only recent data.
1. Use is greater than in the drop down - this will be used on the Metric Insights side.2. Regardless of the actual dates you are analyzing, choose a recent date to limit row count.
Page 104Connecting To Data Sources
2. Build a Tabulation
Page 105Connecting To Data Sources
3. Now we have Daily Sales for Total Month.
Page 106Connecting To Data Sources
4. Copy Macro Code to paste into Metric Insights.
Metric Insights takes raw macro code from 1010data. Copy the 1010data macro code out of theEdit Actions window to paste into Metric Insights.
To see how to build a Metric from scratch, click the following this link:
Page 107Connecting To Data Sources
5. Paste your code into Metric Insights Metric Editor
Paste the code into the Macro Code box in the Metric Editor.
Page 108Connecting To Data Sources
6. Remove table name note at the beginning of the code, and pastetable into table name field.
1. Paste Table Name into Table Name field & remove table note from code.
2. Click the Validate Plug-In Command button to verify code.
7. Validated Code
Once validated, the macro code window will convert from red to green. You will also see thenumber of records fetched from your code.
Page 109Connecting To Data Sources
8. Replace your date selection in the code with:last_measurement_time
Page 110Connecting To Data Sources
Sourcing Data from Adaptive Planning
Page 111Connecting To Data Sources
How to establish connectivity to Adaptive Planning pluginAn Administrator can use the process described in this article to create a new Plug-in DataSource that is required to allow Elements to fetch data from Adaptive Planning to create avisualization in Metric Insights.
1. Go to Data Source editor
1. Select Data Sources in Admin menu
Page 112Connecting To Data Sources
2. Add New Data Source
1. Select to Add New Plug-in
Page 113Connecting To Data Sources
3. Select "Plug-in" as Data Source Type
Choose Plug-in value
4. Select 'Adaptive Planning' from Plug-in picklist and inputmeaningful Name
Press Save
Page 114Connecting To Data Sources
5. Plug-in Data Source Editor will open
1. Enter Username2. Set Password
6. Save Basecamp Plug-in
Save plug-in
Page 115Connecting To Data Sources
How to collect data from Adaptive Planning pluginThis article will show you how to create an Element using a Adaptive Planning plug-in as a datasource. It assumes that you have already established connectivity to your Adaptive Conectivityaccount.
1. Add new Element
Select Add New Element in Admin the menu
Page 116Connecting To Data Sources
2. Select Report
Choose Report
Page 117Connecting To Data Sources
3. Choose Adaptive Planning plug-in as a Data Source for newelement
Page 118Connecting To Data Sources
4. Add XML Request and Validate Statement
Page 119Connecting To Data Sources
5. Publish your new element
Page 120Connecting To Data Sources
Sourcing Data from Basecamp
Page 121Connecting To Data Sources
Establish Connectivity to BasecampAn Administrator can use the process described in this article to create a new Plug-in DataSource that is required to allow Elements to fetch data from Basecamp to create a visualization inMetric Insights.
1. Go to Data Source editor
Page 122Connecting To Data Sources
2. Add New Data Source
3. Select "Plug-in" as Data Source Type
Page 123Connecting To Data Sources
4. Select 'Basecamp' from Plug-in picklist and input meaningfulName
Page 124Connecting To Data Sources
5. Plug-in Data Source Editor will open
1. Enter Username2. Set Password3. Click on gear icon to set User ID parameter.
6. Add User ID parameter to profile
Page 125Connecting To Data Sources
7. Save Basecamp Plug-in
Page 126Connecting To Data Sources
How to Collect Data using Basecamp Plug-inThis article will show you how to create an Element using a Basecamp plug-in as a data source. Itassumes that you have already established connectivity to your Basecamp account.
1. Add new Element
Page 127Connecting To Data Sources
2. Choose Basecamp plug-in as a Data Source for new element
1. Choose Basecamp as your Data Source.2. Input your Plug-in Command manually, OR3. Use Visual Editor.
Page 128Connecting To Data Sources
3. Basecamp Query Builder is called by Visual Editor link
You can select fields in Included Fields section, filter selection or group fields based on Filter onand Group By options.
Page 129Connecting To Data Sources
4. Publish your new element
1. Validate your plug-in command2. Then Collect data3. And Enable & Publish your Metric.
Page 130Connecting To Data Sources
Sourcing data from Business Objects
Page 131Connecting To Data Sources
Establish Connectivity to Business ObjectsMetric Insights uses a plug-in Data Source to connect to Business Objects (BOBJ). This allowsdata from an existing Business Objects report to be used to build an element using Metric Insightstools. In order to prepare the connection, you must first create the Data Source using the processdescribed below.
PRE-REQUISITE
Your Metric Insights instance must be configured to support Business Objects.
1. Create Business Objects plug-in
1. Find Data Sources item in Admin menu
2. Prepare to create a new Plug-in Data Source
1. Click Add New Data Source
Page 132Connecting To Data Sources
3. Set type of new Data Source
1. Select Plug-in
Click Next Step
4. Define the basics
1. Select "BOBJ Data Collection Plugin" from the Plug-in pick list (NOTE: If this option isnot found, contact your Administrator or support@Metricinsights.com for assistance sinceyour instance has not been configured for BOBJ
2. Enter a meaningful Name3. Optionally, enter the Username and Password required for this Plug-In
Page 133Connecting To Data Sources
Click Save
5. Continue the Data Source Definition
1. Click the Edit icon for auth.type parameter
5.1 Add auth.type parameter
1. Input auth.type value in this pop-up
Save
Page 134Connecting To Data Sources
6. Continue the Data Source Definition
1. Click the Edit icon for cms parameter
6.1 Add cms parameter
Input cms value in this pop-up
Save
Page 135Connecting To Data Sources
7. Add Remote Data Collector
Click Add New Remote Collector
7.1 Define Remote Data Collector
1. Select Remote Collector from the list
Click Save.
Page 136Connecting To Data Sources
8. Save Business Objects plugin
Page 137Connecting To Data Sources
Collect Data from a Business Objects DocumentBy fetching data from an existing Business Objects (BOBJ) report, you can use the data to createa Metric Insights element.
PRE-REQUISITE
You must have both the BOBJ plug-in configured in your Metric Insights instance and have createda Plug-in Data Source for BOBJ.
1. Create an element
1. Select Add an Element in the Admin menu
Page 138Connecting To Data Sources
2. Select type of the element
1. Choose Metric type2. Check Use step-by-step wizard
Click Next step
Page 139Connecting To Data Sources
3. Provide Basic Information
1. Set Measurement interval2. Set Measure of
Click Next:Collect data
Page 140Connecting To Data Sources
4. Prepare Metric for Data Collection
1. Find BOBJ plug-in in Data Source drop-down list2. Select a Data Collection Trigger from the pick list3. Select Business Objects Document from drop-down list4. Enter the Plug-in Command manually or using the Visual Editor link
Page 141Connecting To Data Sources
4.1 Select desired columns
You may select Field and set Expression for it, Fetch options are set below table with fields
5. Validate plug-in command
Page 142Connecting To Data Sources
6. Collect Data
1. If validation is finished successfully, you will see how many records are fetched displayedin green font; if an error occurs during validation, an error message is presented in redfont.
Click Collect Data
Page 143Connecting To Data Sources
7. Review results
1. You will get message how many values are recorded.
Click Next to generate Preview
Page 144Connecting To Data Sources
8. Review Chart and complete definition
1. Check "Make Visible...."
Click Enable metric to add tile on your Home Page
Page 145Connecting To Data Sources
Sourcing Data using Elastic Search
Page 146Connecting To Data Sources
Establish Connectivity to Elastic SearchAn Administrator can use the process described in this article to create a new Plug-in DataSource. It is required to allow Elements to fetch data using Elastic Search.
1. Go to Data Source editor
Page 147Connecting To Data Sources
2. Add New Data Source
3. Select "Plug-in" as Data Source Type
Page 148Connecting To Data Sources
4. Select 'ElasticSearch' from Plug-in picklist and input meaningfulName
5. Plug-in Data Source Editor will open
1. Click on gear icon to input your server's Endpoint (server can be local or remote).2. Provide Index parameter - the name of your Database.
Page 149Connecting To Data Sources
3. Optionally, provide Type parameter - the name of the table in your Database.
6. Save Elastic Search Plug-in
1. Save your changes.
Page 150Connecting To Data Sources
How to Collect Data using Elastic Search Plug-inThis article will show you how to create an Element using an Elastic Search plug-in as a datasource. It assumes that you have already established connectivity to Elastic Search.
1. Add new Element
Page 151Connecting To Data Sources
2. Choose Elastic Search plug-in as a Data Source for new element
1. Choose Elastic Search as your Data Source.2. Input your Plug-in Command.
Page 152Connecting To Data Sources
3. Publish your new element
1. Validate your plug-in command.2. You will see sample results under 'Validate' button.3. Save the Element to continue configuring.4. If configuring of the Element finished, Enable & Publish it.
Page 153Connecting To Data Sources
Sourcing Data from Facebook
Page 154Connecting To Data Sources
Establish connectivity to FacebookThe information required to set-up a Plug-in Data Source to be used to fetch data from Facebookto be used by an Element to create a Metric Insights visualization.
Required Variables
Include the following const.php variables:
1. FACEBOOK_OAUTH2_APP_ID2. FACEBOOK_OAUTH2_APP_SECRET3. FACEBOOK_OAUTH2_REDIRECT_URI4. FACEBOOK_OAUTH2_PERMISSIONS
The above are standard OAuth2 parameters required by Facebook API and are described inappropriate Facebook docs
http://developers.facebook.com/docs/reference/dialogs/oauth/
http://developers.facebook.com/docs/howtos/login/server-side-login/
Page 155Connecting To Data Sources
How to Collect Data from FacebookThis article will show you how to create an Element (Report in this case) using a Facebook plug-inas a data source. It assumes that you have already established connectivity to your Facebookaccount.
1. Add new Element
Page 156Connecting To Data Sources
2. Choose Facebook plug-in as a Data Source for new element
1. Choose Facebook as your Data Source.2. Input your Plug-in Command3. To see the Facebook Query Language (FQL) Reference click on appropriate link in hint
box.
NOTE: Do not forget, that The Facebook authentication token hat to be reissued every 60 days.
Page 157Connecting To Data Sources
3. Publish your new element
1. Validate your plug-in command2. After successful validation Save your Report to see Sample Result Set3. And if you've finished configuring of the Report Enable & Publish it.
Page 158Connecting To Data Sources
Sourcing Data from Fogbugz
Page 159Connecting To Data Sources
Establish connectivity to FogBugzThis article describes the process of creating a new Plug-in Data Source for accessing FogBugzso you can query the Data Source for a new or existing element.
FogBugz is an integrated web-based project management system with bug/issue tracking,discussion forums, customer relationship management, and evidence based scheduling.
1. Open Admin Menu
Click the Data Sources link to access Data Sources list
Page 160Connecting To Data Sources
2. Add new Data Source
1. Click the Add New Data Source button2. Select Plug-in as type3. Click Next step button
Page 161Connecting To Data Sources
3. Define main settings
1. Select 'FogBugz' plug-in in the drop-down list2. Enter Name3. Complete credential fields4. Click Save5. Edit Plug-in Data Source page opens
Page 162Connecting To Data Sources
4. Set required parameter
1. Click the gear icon to open the Edit Plug-in Data Source Parameter pop-up2. domain: Enter the domain name of your FogBugz account3. Click Save
Page 163Connecting To Data Sources
How to collect data from FogbugzThis article will show you how to create an Element using a Fogbugz plug-in as a data source. Itassumes that you have already established connectivity to your Fogbugz account.
1. Add new Element
Page 164Connecting To Data Sources
2. Choose Fogbugz plug-in as a Data Source for new element
1. Choose Fogbugz as your Data Source.2. Select one of the Filter from the drop-down.3. Input your Plug-in Command manually, OR4. Use Visual Editor.
Page 165Connecting To Data Sources
3. Fogbugz Query Builder is called by Visual Editor link
You can select fields in Included Fields section, filter selection or group fields based on Filter onand Group By options.
Page 166Connecting To Data Sources
4. Validate your Plug-in Command
1. Validate your Plug-in Command
Page 167Connecting To Data Sources
5. Enable and Publish Report
1. Upon successful validation, you will see Sample Result Set.2. Save the Report and continue to configure it OR3. If configuring of the Report completed, Enable and Publish.
Page 168Connecting To Data Sources
Sourcing data from Google Analytics
Page 169Connecting To Data Sources
Establish connectivity to Google AnalyticsThis article shows how to create a new plug-in connection to Google Analytics.
1. Open Plugin Connection Profile Editor
Start by creating your new metric or report and performing the following actions in the MetricCalculations section of the Metric Editor (or Report Editor):
1. Expand Data Source drop down list2. Select Add New Data Source
Page 170Connecting To Data Sources
1.1 Set plug-in type for new data source
2. Specify the name and credentials for the new profile
Next, perform the followings steps to define your new connection profile:
1. Pick 'Google Analytics' as the Plug-in2. Specify a name for your new Plug-in3. Specify your Google Analytics Username4. Provide the password for the specified username5. Optionally specify number of threads per trigger
Page 171Connecting To Data Sources
Save
3. Plugin Data Source Editor will display for additional input
Click on the gear icon next to the Profile ID variable
4. Enter your Profile ID from Google Analytics
Enter your Profile ID in the pop-up window.
Page 172Connecting To Data Sources
4.1 Finding your Google Analytics Profile ID: Step 1
Login to your Google Analytics account and click the Admin link (top right-hand corner) to accessyour Admin page. Then choose the profile that you want.
4.2 Finding your Google Analytics Profile ID: Step 2
Click the Profile Settings tab and find your Profile ID near the top of the tab
Page 173Connecting To Data Sources
4.3 Enter Profile ID into Plug-in Data Source parameter
5. Connecting the Data Source to your Elements
You are now ready to connect your new data source to a metric or report
Page 174Connecting To Data Sources
How to fetch data from Google Analytics
1. Assign the Profile to your New Element
Next, close the profile, back to Element Editor and set the newly defined profile as the source foryour metric/report.
Page 175Connecting To Data Sources
2. Define the Google Analytics Query
You can define your Google Analytics query by either:
1. Typing it into the Plug-In command field2. Clicking on the 'Query Builder' link to build the query using drop-downs
The Query Builder includes drop-downs and examples to facilitate the construction of your query.If you're not familiar with Google Analytics, we suggest that you start there.
Page 176Connecting To Data Sources
3. Query Builder makes it easy to define complex Google Analyticsqueries
The Query Builder provides an example for each entry. Just click on the desired box and theexample will appear on the right. The first three boxes (i.e., 'dimensions', 'metrics', and 'segment')include drop-down menus that will enable you to select entries by clicking your mouse.
Note: The 'segment' box will enable you to select or define a Google Analytics 'advancedsegment', which is a type of filter. It should not be confused with the dimensions that are used indimensioned metrics and reports.
You can fetch data incrementally from Google Analytics by referencing :last_measurement_timefor the start-date in the Query Builder. In the example above, the system will fetch visitors by dayincrementally by substituting the last date for which data was collected for the start-time variable inthe query expression.
IMPORTANT: By default, Google Analytics returns data for a relatively brief date range. If youwant a larger date range, a start-date must be specified.
Page 177Connecting To Data Sources
4. Handling dates in Google Analytics
As you know, Google identifies weeks and months by number, e.g., '11'. Metric Insights requiresdates, e.g., 11/1/2014. If you look in the 'Time' section of the Visual Editor, you will see threespecial Metric Insights functions which are designed to address this issue:
mi:hour returns a datetime value that is computed by concatenating ga:date and ga:hour(i.e., the native Google Analytics 'hour' function, which simply returns the hour of the day).
mi:week returns a date value corresponding to the first day of the week that is identified byga:week (i.e., the native Google Analytics 'week' function, which simply returns the week ofthe year).
mi:month returns a date value corresponding to the first day of the month that is identifiedby ga:month (i.e., the native Google Analytics 'month' function, which simply returns themonth of the year).
Page 178Connecting To Data Sources
Collecting data from Google BigQuery
Page 179Connecting To Data Sources
Establish connectivity to Google BigQueryThis article describes the process of creating a new Plug-in Data Source for accessing a GoogleBigQuery to be able to query the data source for a new or existing Metric.
PRE-REQUISITE
Before you use any Google plug-in resources, you must have registered your server with Google.See Registering Your Server with Google for more information.
1. Create BigQuery plug-in
Find Data Sources item in Admin menu
2. Add New Data Source
Page 180Connecting To Data Sources
3. Select the plug-in type of data source
4. Define the Profile
1. Specify "Google BigQuery" as the Plugin2. Provide a Plug-in name3. Save
Page 181Connecting To Data Sources
5. Obtain the "Token" parameter
Before you can complete this step, you must have registered your server with Google. SeeRegistering Your Server with Google for more information.
1. Enter a BigQuery Job Id for the "job_id_prefix" parameter. This is simply a tag that you willsee in your BigQuery logs so that you can tell which queries are coming from MetricInsights. You can leave that as the default value or change if you like. The value will notaffect your connection.
2. Click the Gear icon for "token" parameter
Page 182Connecting To Data Sources
5.1 Login using credentials for Google BigQuery account
1. Confirm by clicking Allow Access
5.2 Note the Token ID
1. Notice that a token ID appears in the Parameters grid in the Value column2. Save changes
Page 183Connecting To Data Sources
6. Check if profile is valid in the Metric Editor
Open Metric editor
1. Set BigQuery plug-in as Data Source for element2. Click the Refresh link next to the Project field and choose a project from the drop-down3. Enter statement into the Plugin Command box.4. Click the Validate Plug-in Command link to run it against Google BigQuery
Page 184Connecting To Data Sources
How to fetch data from Google BigQuery
1. Add a new Element
1. Open the Admin Menu on your Home Page2. Click "Add an Element"
2. Select type of the element and use wizard for creation
Page 185Connecting To Data Sources
3. Provide main information
Set Measurement interval (1) and Measure of (2) for metric. Then press Collect data (3).
If you don't already have a Measure, see Define Your First Measure
Page 186Connecting To Data Sources
4. Prepare metric for Data Collection
1. Find Google BigQuery plug-in in Data Source list
2. Set Trigger
3. Select Project from drop-down list
4. Input Plug-in Command in the text box
Page 187Connecting To Data Sources
5. Validate plug-in command
6. Enter Last Measurement Time to specify what data points shouldbe fetched
Page 188Connecting To Data Sources
7. Collect Data
1. If validation is finished successfully you will see how many records are fetched. Informationmessage is displayed in green font. If any error occurs during validation you will see errormessage in red font.
2. Set Time point from what data should be collected.
3. Press Collect Data
Page 189Connecting To Data Sources
8. Fetch is completed
You will get message how many values are recorded. Press Next button to generate Preview
Page 190Connecting To Data Sources
9. Metric creation is finished
Check "Make Visible" flag is set to true and click Enable metric to add tile on your Home Page
Page 191Connecting To Data Sources
Sourcing Data from Google Calendar
Page 192Connecting To Data Sources
How to establish connectivity to Google CalendarIn order to retrieve Event data from a Google Calendar, you must have a Plug-in Data Source.This article covers the creation process.
PRE-REQUISITE:
Before you use any Google plug-in resources, you must have registered your server with Google.See Registering Your Server with Google for more information.
1. Define a new Data Source
1. Select Data Sources from Admin menu to open the Data Sources grid2. Click Add New Data Source
Page 193Connecting To Data Sources
1.1 Choose Type
1