17 June 2019© MARKLOGIC CORPORATION

Establishing Real-World Data Integration Best Practices

CHRIS ATKINSONSolutions Architect

MarkLogic

T.J. TANGSoftware Architect

AbbVie

01 IntroductionAbbVie and T.J. Tang ─ Who are they?

02 First StepsTaking those first big decisions

03 EvolutionHow and why the platform changed

04 Establishing a COELearning from their own COE development

The AbbVie Journey to a COE

05 Help and Materials Useful resources when establishing your own COE

Key FactsAbbVie is an American publicly traded biopharmaceutical company founded in 2013. It originated as a spin-off of Abbott Laboratories.

8,800+ R&D Workers

175+ Countries

26M+ Patents

#1 Selling Drug World

Wide

Software Innovators

$4.8B R&D 2017

T.J. Tang – AbbVie MarkLogic COE

T.J. TangSoftware Engineer20 years experience

ArchitectEvaluates solutions and opportunities

ManagerHeads up the COE effort

The COETeam7 People

PurposeData integration best practices

Business DemandReduce costsAccelerate R&D

First Steps

Initial ImplementationModelled data to several smaller documents

CPF for processing

External content enrichment Raw Text Document

Source Metadata

Metadata from Named

Entity Recognition

Related Triples

Instance Data

Early Architecture Changes

Early Changes

Envelope pattern

Modelled data into larger documents

COE Takeaways

MarkLogic Experts often made available

Meetups for 1:1 discussions

Use SEs, SAs

Source Metadata

Metadata from Named Entity Recognition

Related Triples

Instance Data

Enveloped

Data Hub Implementation

COE Standards

V2 and later standards

Custom changes need migrating

From & to state needs mapping

COE Takeaways

Talk directly to PMs

SAs for roadmap

Request enhancements to MarkLogic

Evolution of the COE

The COE is Born

A natural evolution

Advice and expertise evolved

Servicing other LOBs

Share & Shepherd

Project Growth

Onboarded Consultant Expert

Advice from MarkLogic

Initial Solution for R&D Business

COE Takeaways

Experts are needed

Consider their role

COE Day-to-Day Activities

Educate

Qualify

Validate

COE Takeaways

Create standardized materials

Customize to need

Needs to be on-demand

Recent Innovations

NiFi Adoption

Less developer centric

Good performance

Some early challenges

COE Takeaway

One of a few options (e.g. MuleSoft)

Data Hub is innovating fast

MarkLogic relationship important

Request help/advice

Next Steps

Machine Learning

Avoids offloading data

Processing close to data

Establishing Your Own COE / Standards

Technical PatternsCode Examples

Data Integration GuidesArchitecture

Value

Avoiding Pitfalls

Internal ExpertsAn essential component

Expert HelpMarkLogic Experts

Solution Architects

Solution Engineers

Data Integration Guides

3.1 Modelling Entities in the Data Hub

3.3 Tracking Provenance & Lineage

2.2 Ingesting Different Data Types

2.1 Ingesting Multi-Model Data

4.5 Designing for Compliance Queries

4.4 Data Lifecycle Management

3.2 Orchestrating Data Integration

1. Agile Data Management

2. Ingestion

3. Curation

4. Delivery

1.0 Agile Data Management with

MarkLogic

4.3 Optimizing Delivery with Entities

Document

Topic

4.1 Designing for SQL Analytics

4.2 Designing for Operational Workloads

Integration Guide Example

Technical Patterns

Questions?

Thank you