1 v1.2
2 v1.2
Network Management & MonitoringModern Monitoring Architecture & Model Driven Programmability
bdNOG13Fakrul Alam | APNIC CT
3 v1.2
Table of Content• Basics of Network Monitoring
• Network Monitoring and Management– Monitoring – Why, What and How– Management – Why, What and How
• Network Monitoring and Management best practices
• Techniques and Tools
• Modern Monitoring Architecture
• Model Driven Programmability
4 v1.2
Module 1: Network Management & Monitoring
5 v1.2
Basics of Network Monitoring• Network have evolved from being a flat to a complex network with
lot more technologies:– Cloud– Wireless– Remote Users and VPN– Mobile Devices– IoT– VOIP
• Simple networks don’t meet the requirements of modern infrastructure anymore
6 v1.2
Basics of Network Monitoring• In spite of all the evolution that has occurred, one factor that has
been constant is the need for network monitoring
• For effective monitoring solution it’s critical to understand the– Major network components → Router, Switch, Firewall, Load Balancer, Server &
Services– Protocols → SNMP, ICMP, gRPC, Netflow– Monitoring Tools → LibreNMS, Nagios, Cacti etc
7 v1.2
Network Monitoring and Management• Monitoring
– Constantly checks/scans the status of a network– Collect statistics and performance metrics– Checking for error conditions notifies the network administrator for slow or failing
components
• Network Monitoring involves– What we should be looking for → Throughput, Latency, Packet loss, Bandwidth– How to find it → Ping, SNMP Trap, SNMP Poll, API– Where and how to store the values → Cloud, On-prem, RRD, Time Series Data– What thresholds indicate a problem situation → Performance metrics– How to notify → Email, SMS, Webhook
8 v1.2
Network Monitoring and Management• Management
– The processes, tools and applications used to administer, operate and maintain a network infrastructure
• Network Management involves– Administration → Tracking network resources– Operation → Network functions well– Maintenance → Upgrades and fixes to network resources– Provisioning → Network resource configuration
9 v1.2
Every Network is different
And
No single system will solve all your problems
Or
Meet all your requirements
10 v1.2
WHY do we Monitor• Track resource utilization and get historical data• Establish a baseline (what’s normal for the network)• Understand the performance matrices and do capacity planning• Helps network and systems administrators identify possible issues
before they affect business continuity• Find the root cause of problems when something goes wrong in
the network• Track configuration changes• Identify security threats
11 v1.2
WHAT do we Monitor• All the resources vs Critical resources
– Hardware → Network Devices & Servers– Services → DNS, DHCP, HTTP/HTTPS, SMTP– Application → On-prem and Cloud
• Underlay vs Overlay– IGP (OSPF, ISIS)– OMP – EVPN & VXLAN
Availability Reliability Capacity Performance
Uptime Jitter, Latency, RTT Traffic, Port Utilization CPU, Memory, Disk, Processes
12 v1.2
HOW to Monitor• Commonly used technologies:
– Ping– SNMP
• SNMP Trap• SNMP Poll
– Syslog– CDP/LLDP– Netflow– API
• There are tools which leverages these features to monitor network
13 v1.2
WHY do we Manage• Maximum efficiency and improve IT productivity
• Security and Threat detection
• Network upgrade and visibility
14 v1.2
WHAT do we Manage• Network resources (routers, switches, Firewall, Load Balancer)
• Systems (Servers, Applications)
• Power system devices
• Customer Premises Equipment (CPE)
• Storage
15 v1.2
HOW to Manage• Agent based – reside on mange network/system element
• Most common protocol is SNMP SET
• Few new protocols include NETCONF and RESTCONFSNMP NETCONF SOAP RESTCONF
Standard IETF IETF W3C IETFResources OIDs Paths URLsData Models Defined in MIBs YANG YANGManagement Operations
SNMP NETCONF XML HTTP Operations
Encoding BER XML XML XML, JSONTransport Stack UDP SSH
TCPSSLHTTPTCP
SSLHTTPTCP
16 v1.2
Network Monitoring and Management best practices
• Baseline network behaviour
• Escalation matrix
• Layered breakdowns
• Implement High Availability with failover options
• Capacity planning and growth
17 v1.2
Techniques and Tools• Agent-based and Agentless Monitoring
• Internal and External Monitoring
• Centralized vs Decentralized Network Management
18 v1.2
SaaS based Monitoring and Management• SaaS based monitoring and management tools
• Scalability, accessibility, upgradability, resilience and pay-as-you-go pricing options
• Work for both on-prem and cloud infrastructure• Few SaaS based Monitoring
tools• LogicMonitor• New Relic• Auvik• StatusCake
19 v1.2
Tools• Few Network Monitoring & Management toolsTool Function LinkIcinga Availability, Performance, Monitoring https://icinga.com/Nagios Availability, Performance, Monitoring https://www.nagios.org/LibreNMS Availability, Capacity, Discovery, Performance, Monitoring https://www.librenms.org/Zabbix Availability, Capacity, Discovery, Performance, Monitoring https://www.zabbix.com/Smokeping Availability, Latency, Monitoring https://oss.oetiker.ch/smokeping/Nfsen/Nfdump Traffic Analysis, Monitoring, Flow Collection http://nfsen.sourceforge.net/
https://github.com/phaag/nfdumpAS-Stats Traffic Analysis, Monitoring, Flow Collection https://github.com/manuelkasper/AS-StatsRancid Backup, Monitoring, Management https://shrubbery.net/rancid/Oxidized Backup, Monitoring, Management https://github.com/ytti/oxidizedRT/OSTicket Ticketing System https://osticket.com/NetDisco Discovery, Inventory, IPAM http://netdisco.org/Syslog-ng Log Management https://github.com/syslog-ng/syslog-ngGraylog Log Management https://www.graylog.org/products/open-sourceNetdot Documentation https://github.com/cvicente/Netdot/Nipap IPAM https://spritelink.github.io/NIPAP/
20 v1.2
Network Monitoring and Management – What Next?Legacy Way Modern Way
Network Monitoring SNMP GetSNMP TrapRRD
API (Webhook)Model Driven Telemetry (gRPC)Time Series Database (TSD)
Network Management SNMP Set NetConfRestConfOpenConfig
21 v1.2
Module 2: Modern Monitoring Architecture
22 v1.2
Time Series Database (TSDB)• Time series data is a set of data indexed by time
• A time series database (TSDB) is a database optimized for time-stamped or time series data
• This type of data describes the measurements of a measured subject at each time point of a time range
23 v1.2
Time Series Database (TSDB)• Time series data can be useful for:
– Tracking daily, hourly, or weekly weather data– Tracking changes in application performance– Medical devices to visualize vitals in real time– Tracking network logs
24 v1.2
Time Series Data Models• Modeling of time series data includes three important parts:
– Subject: The subject to be measured. Example: Taking server status monitoring. The measured subject is a server, and its attributes may include the cluster name, host name, and so on
– Time point: A subject may have one or more measurements, each corresponding to a specific metric. In the case of the server status monitoring, the measured metrics may include CPU usage and IOPS.
– Measurements: The measurement report is always attached with a timestamp attribute to indicate the time
timestamp cluster hostname cpu iops202-12-20T 17:50:00Z Cluster-A Host-a 10 10202-12-20T 17:50:10Z Cluster-A Host-b 20 30202-12-20T 17:50:20Z Cluster-A Host-a 5 9
Timestamp Subject Dimension Metrics and measurements
25 v1.2
Time Series Databases• Few popular Time-Series Databases:
– InfluxDB– Promethues– RRDtool– TimeScale– OpenTSDB– Graphite
26 v1.2
Monitoring Stacks• Popular open source solution stacks:
– Elastic Stack– TICK (Telegraf, InfluxDB, Chronograf and Kapacitor) – TIG (Telegraf, InfluxDB, and Grafana)
27 v1.2
A Modern Monitoring Architecture - TIG Stack• Open source tools built to make collection, storage, graphing, and
alerting on time series data easy• TIG Stack stand for Telegraf, InfluxDB, and Grafana
– InfluxDB is an open-source time series database written in Go– Telegraf is an agent for collecting, processing, aggregating, and writing metrics– Grafana is an open source data visualization and monitoring suite
A Modern Monitoring Architecture
Collects time-series data from different source
Store time-series data Visualizes and graphs the time series data stored in InfluxDB
CPU MEMORY
DISK PROCESS
LOAD NET
APPLICATIONS
Servers
28 v1.2
TIG Architecture
https://www.influxdata.com/products/influxdb/
Telegraf:• Telegraf is a data collection agent
that captures data from a growing list of sources and translates it into InfluxDB line protocol format for storage in InfluxDB
• Telegraf’s extensible architecture makes it easy to create plugins that both pull data (input plugins) and push data (output plugins) to and from different sources and endpoints
InfluxDB:• InfluxDB stores data for any use case
involving large amounts of timestamped data, including DevOps monitoring, log data, application metrics, IoT sensor data, and real-time analytics
Grafana:• Grafana is the user interface for the
TIG stack that provides customizable dashboards, data visualizations, and data exploration
29 v1.2
Telegraf• Telegraf is a plugin-driven server agent for
collecting and sending metrics and events from databases, systems, and IoT sensors
• It can collect metrics from a wide array of inputs and write them into a wide array of outputs
• With 200+ plugins already written by subject matter experts on the data in the community, it is easy to start collecting metrics from your end-points
• For all the plugins and integration– https://www.influxdata.com/products/integra
tionsInput Plugins Process
(transform, filter)Aggregate
(sum, count) Output Plugins
30 v1.2
InfluxDB• InfluxDB is an open-source time series database (TSDB)
developed by InfluxData
• Optimized for fast, high-availability storage and retrieval of time series data in fields such as operations monitoring, application metrics, Internet of Things sensor data, and real-time analytics
• Provides an SQL-like language, listening on port 8086
31 v1.2
Grafana• The open-source platform for monitoring and observability.• Grafana allows us to query, visualize, alert on and understand metric• Features:
– Visualize– Dynamic Dashboards– Explore Metrics– Explore Logs– Alerting– Mixed Data Sources
• Check Grafana in action– https://play.grafana.org/
• Import dashboard– https://grafana.com/grafana/dashboards
32 v1.2
Prometheus• Prometheus is an open-source monitoring tool and time-series
database
• Prometheus provides powerful query language, storage, and visualization features for its users
• InfluxDB vs Prometheus:
The most notable difference is between the scopes of these platforms. Both systems could be used for monitoring and time-series data storing. However, InfluxDB is more known as a time-series database, while Prometheus has a broader scope of monitoring purposes
Prometheus Demo:https://github.com/prometheus/demo-site
33 v1.2
Prometheus Architecture
source: https://prometheus.io/assets/architecture.png
34 v1.2
Monitoring StacksTime Series Database
Agent / Data Collector
Visualize Log Aggregators / Shippers
Logging Alerts Scripting/Query
Promethues PushgatewayExporters
Grafana, Promethues web UI
Promtail Loki AlertManager PromQLLogQ
Elasticsearch Beats → MetricbeatHeartbeat
Kibana Beats → FilebeatWinlogbeat
Logstash
InfluxDB Telegraf Chronograf Kapacitor FluxInfluxQL
OpenTSDB StasDCollectd
Graphite (Whisper & Carbon)
Graphite-web
35 v1.2
Module 3: Explore TICK Stack
36 v1.2
Explore TICK Stack – InfluxDB Cloud• To explore TICK stack we need to install following components
– Telegraf– InfluxDB– Capacitor– Kapacitor
• In this tutorial we will use InfluxDB Cloud. This will allow us to skip downloading and installing InfluxDB and jump into exploring InfluxDB 2.0 technology
• The Free Plan is designed for getting started with InfluxDB
https://www.influxdata.com/products/influxdb-cloud/
37 v1.2
InfluxDB CloudGot to https://www.influxdata.com/get-influxdb/ and signup
“Use it for free” Create account with valid email and verify
Choose where you like to store your data
Keep free plan
38 v1.2
Create a new bucket
Go to Data > Buckets and + Create Bucket
Name the bucket with your group name (example group10)
39 v1.2
Configure Telegraf Agent
Click on “Configure Telegraf Agent”
Choose “System” Name the config “system-monitor”
40 v1.2
Configure Telegraf Agent
Save the “API Token” & “Start Telegraf” command in notepad
41 v1.2
Install TelegrafLogin to you server and run the following commandswget https://dl.influxdata.com/telegraf/releases/telegraf_1.17.0-1_amd64.debsudo dpkg -i telegraf_1.17.0-1_amd64.deb
Run the “export INFLUX……” & “telegraf –config……” command; the one you have copied in notepad
Source: https://portal.influxdata.com/downloads/
42 v1.2
Verify the connection
“Listen for Data”
“Connection Found!”
43 v1.2
Explore Graph!1. Go to Explore and choose the
right bucket (group10 as an example)
2. Choose measurement (for example cpu)
3. Click Submit
44 v1.2
Module 4: Explore Grafana Cloud
45 v1.2
Grafana Cloud• Go to https://grafana.com/products/cloud/ and subscribe
• After login you will see services available in Grafana Cloud
46 v1.2
Grafana Cloud
47 v1.2
Grafana Cloud
Import dashboard ID 6126
48 v1.2
Grafana Cloud
49 v1.2
Module 5: Model Driven Programmability
50 v1.2
Table of Content• Device configurations - CLI & SNMP
• Model-driven programmability
• Data Model - YANG
• Protocols - NETCONF & RESTCONF
• Data Encoding - JSON & XML
• API Tools & Resources
51 v1.2
Device Configuration - Command-line Interface• There are many different ways to connect to and manage a
network
• The most commonly used method for the past 30 years has been by using the command-line interface (CLI)
• And one of the most glaring and biggest flaws with using CLI to manage a network is misconfiguration
• A majority of network outages are caused by human errors
52 v1.2
Device Configuration - Command-line Interface• CLI Pros and Cons
PROs CONsWell know and documented Difficult to scaleCommonly used method Large number of commandsCommand can be scripted Must known IOS command syntaxSyntax help available on each command Executing command can be slow
Can execute only one command at a timeCLI and command can change between software versions and platformsUsing CLI can pose a security threat if using Telnet (plaintext)Vendor-specific commandsComplex in parsing
53 v1.2
Device Configuration - SNMP• SNMP is widely used for fault handling and monitoring• In practice, SNMP is rarely used for configuration• Major problems of the traditional SNMP mode
– Insufficient Performance: Data configuration and reading are low, especially in the development of large-scale networks
– Difficult to deliver configurations: Only a few MIB objects support the write operation
– No support for the transaction mechanism: SNMP operations are stateless. Therefore, the operations cannot be interrupted in the case of a configuration timeout
– Poor programmability: Lack of composite data structures, few RPC interfaces and time-consuming commissioning
54 v1.2
A new API is needed?• In June, 2002 the IETF Internet Architecture Board (IAB) held a
Network Management Workshop to assess the state of network management and develop requirements for next generation network management protocol
• As a result, RFC 3535 was published that identified the need for a new NETwork CONfiguration protocol
• They outline some key requirements for this protocol– Easy to use for the operator– Provides Clear separation between Configuration state and operational state of the
device– Backup and restore capability– Human and Machine friendly– Provides Error checking
An API (Application Programming Interface) in its simplest form can be considered a Contract or Agreement between two end points, regarding what to send and what to expect in return.
It is used with Machines to request data and or to change data on the device
55 v1.2
The Rise of NETCONF• In 2006 NETCONF was born
– RFC 4741: NETCONF protocol v1.0– RFC 6241: NETCONF protocol v1.1
• The NETCONF standard define a communication channel between a Server (network Node) and client (NMS, SDN or Application)
• The end result has been a programmable device interface ideally suited for use in SDN and NFV
56 v1.2
Model-driven Programmability
source: https://blogs.cisco.com/getyourbuildon/model-driven-programmability
App1 App2 App3
Model-Driven APIsYANG Development Kit (YDK)
NETCONF RESTCONF gNMI
XML JSON GPB
SSH HTTP(S)
Data ModelsYANG (native, open)
Apps
Bindings
Protocol
Encoding
Transport
Models
Model-driven programmability inherits the power of models, making it easier to configure routers. It overcomes drawbacks posed by traditional router management
techniques
Model-Driven Configuration
Model-Driven Telemetry
57 v1.2
Model-driven Programmability - Benefits• Model based, structured, computer friendly
• Multiple model types (native, OpenConfig, IETF)
• Models decoupled from transport, protocol end encoding
• Choice of transport, protocol and encoding
• Model-driven APIs for abstraction and simplification
• Wide standard support while leveraging open source
58 v1.2
Data Models• Data models are used to describe
– whatever can be configured on a device, – everything that can be monitored on a device, – and all the administrative actions that can be executed on a device (example:
resetting counters or rebooting the device)
• The list of supported models includes – Native (Vendor)– OpenConfig and – IETF models
• YANG (Yet Another Next Generation) provides a modeling language optimized for network devices and with a growing number of tools and utilities
59 v1.2
Protocols• The separation of models from the choice of protocol provides a
high degree of flexibility
• We can control the network device using – NETCONF– RESTCONF or– gNMI (gRPC Network Management Interface)
• Your protocol choice will ultimately be influenced by your networking, programming and automation background, plus tooling available
60 v1.2
Encodings• The separation of encodings from the choice of model and
protocol provides additional flexibility
• Data can be encoded in – JSON– XML or – Google protocol buffers (GPB) format
• While some transports are currently tied to specific encodings (e.g. NETCONF and XML), the programmability infrastructure is designed to support different encodings of the same data model if the transport protocol supports it
61 v1.2
Model Driven Programmability - Summary
62 v1.2
Data ModelsModel Driven Programmability
63 v1.2
YANG• “Yet Another Next Generation” – A Data Modeling Language for NETCONF• YANG is a data modeling language used to
– model configuration– state data and – administrative actions manipulated by the NETCONF protocol
• Key YANG Capabilities– Human readable, easy to learn representation– Hierarchical configuration data models– Reusable types and groupings (structured types)– Extensibility through augmentation mechanisms– Supports the definition of operations (RPCs)– Formal constraints for configuration validation– Data modularity through modules and submodules– Versioning rules and development support
RFC 6020RFC 7950
64 v1.2
YANG vs Other Modeling Language• Other Modeling languages are already existed
– SMI (SNMP)– UML– XML Schema
• None of these languages were specifically targeted to the needs of configuration management
• The Network Modeling (NETMOD) working group is responsible for the YANG data modeling language
65 v1.2
YANG Structure• Yang files are called Modules• Each module is uniquely identified by a
namespace URI• YANG models data using a hierarchical,
tree-based structure with nodes• Main node types:
– Container : A grouping of other statements (which have no value)
– List : Multiple records containing at least one Leaf “Key” and an arbitrary hierarchy of other statements within the Module
– Leaf : A single Key/Value Pair– Leaf-List : Multiple Key/Value pairs of the same
type or commonality
• Each leaf is associated against a data type
66 v1.2
YANG: A Familiar ComparisonYANG Element Python ClassContainer DictionaryContainer name Dictionary keyLeaf name Dictionary keyLeaf Dictionary valueList ListString StringBool BoolInteger IntegerEmpty None
67 v1.2
YANG Model Types• Currently there are three main publishers for the YANG Modules
Vendor (native) IETF OpenConfig
Vendor specific
Example:“BGP” extensions on IOS-XE
Reference:https://github.com/YangModels/yang/tree/master/vendor/cisco
Standard definition from IETF, ITU etc)
Example:ietf-diffserv-policy.yangietf-diffserv-target.yang
Reference:https://github.com/YangModels/yang
Vendor-neutral data models
Example:openconfig-bgp-common-multiprotocol.yangopenconfig-mpls-rsvp.yang
Reference:https://www.openconfig.net/
OpenConfig and IETF models are mapped to native models
68 v1.2
YANG Tools - pyang• pyang is a YANG validator, transformation and code generator,
written in python
• It can be used to validate YANG modules for correctness, to transform YANG modules into other formats, and to generate code from the modules
• https://pypi.org/project/pyang/
$ pyang -f tree [email protected]: ietf-interfaces+--rw interfaces| +--rw interface* [name]| +--rw name string| +--rw description? string| +--rw type identityref| +--rw enabled? boolean| +--rw link-up-down-trap-enable? enumeration {if-mib}?| +--ro admin-status enumeration {if-mib}?| +--ro oper-status enumeration| +--ro last-change? yang:date-and-time| +--ro if-index int32 {if-mib}?| +--ro phys-address? yang:phys-address| +--ro higher-layer-if* interface-ref| +--ro lower-layer-if* interface-ref| +--ro speed? yang:gauge64
69 v1.2
YANG Tools - YANG Explorer• A GUI driven tool to test
NETCONF and RESTCONF interfaces defined by YANG models
• Features– Load YANG models from device– Browse YANG models– Execute NETCONF or RESTCONF
Operations– Generate self-contained Python scripts– Open Source https://github.com/CiscoDevNet/yang-explorer
70 v1.2
ProtocolsModel Driven Programmability
71 v1.2
Protocols• Set of rules that determine how data is transmitted between
different devices in the same network
• Common protocols for model-driven programmability– NETCONF– RESTCONF
72 v1.2
NETCONF• NETCONF is an IETF network management protocol designed
specifically for configuration management
• Features:– Makes a distinction between configuration and state data – Utilizes multiple configuration data stores (candidate, running, startup) – Configuration change transactions – Provides client-side configuration validation– Uses filtering mechanisms for selective data retrieval– Uses a client-server model and SSH as transport protocol
73 v1.2
NETCONF Communications• The NETCONF standard define a communication channel
between a Server (network Node) and client (NMS, SDN or Application)
74 v1.2
NETCONF Protocol Stack• The following are the key characteristics of NETCONF
– It uses SSH as Transport protocol– It is Connection oriented and Session Oriented– It uses XML for encoding and representing data– It defines multiple operations to interact with the network device– It uses RPC methods to invoke these operations on the remote network device
(4) Content Configuration / Operational Data
(3) Operations
(2) Messages
(1) Transport
Actions to Take
Remote Procedure Call (RPC)
TCP/IP
<data>
<get>, <get-config>, <edit-config>
<rpc>, <rpc-reply>
SSH (830)
XML
75 v1.2
NETCONF OperationsMain Operations Cisco Command Description<get> show ? Retrieve running configuration and device state
information<get-config> show run Retrieve all or part of specified configuration datastore<edit-config> con t Loads all or part of a configuration to the specified
configuration datastoreOther Operations Description<copy-config> Replace an entire configuration datastore with another <delete-config> Delete a configuration datastore<commit> Copy candidate datastore to running datastore (ex: XR)<lock> / <unlock> Lock or unlock the entire configuration datastore
system<close-session> Graceful termination of NETCONF session<kill-session> Forced termination of NETCONF session
76 v1.2
NETCONF Data Stores• The NETCONF standard defines an important concept Called
Data Stores to interact with the network devices, these Data Stores are:
– Running Data Store: Include the current active configuration on the device– Candidate Data Store: Holds that configuration changes before
committing/copying to the running configuration – Startup Data Stores: This is the configuration that the device use during boot up
• Not all data stores are supported by all devices
• Running config is the only required data store
77 v1.2
RESTCONF• It’s an implementation of a REST API
• Model-driven API
• Functional sub-set of NETCONF
• Exposes YANG models via a REST API (URL)
• Uses HTTP(S) as transport
• Uses XML or JSON for encoding
• Developed to use HTTP tools and programming libraries
78 v1.2
RESTCONF - Background on REST• Same HTTP Request Methods and Response Codes are used
Client Server
HTTP GET
HTML Response
API Client Web ServerOn a Network Device
HTTP GET
JSON/XML Response
79 v1.2
REST - HTTP Verbs
GET Retrieve / Read a resource show commandPOST Creates a new resource create logical interfacePUT Update/Replace a resource replace full interface config with what’s in
the body of requestPATCH Update/Modify a resource update (append) interface config with
what’s in the body of requestDELETE Removes a resource remove logical interface
• HTTP Verbs in the context of network devices
80 v1.2
YANG models over RESTCONF
81 v1.2
REST - HTTP Response Code• Common HTTP Response CodeSuccess (2xx) Description200 Request Succeeded201 The request has been fulfilled; new resource created204 The server fulfilled request but does not return a body
Client Error (4xx) Description400 Bad Request. Malformed syntax401 Unauthorized403 Forbidden. Server understood request, but refuses to full fill it.404 Not found
Server Error (5xx) Description500 Internal Server Error501 Not implemented
82 v1.2
REST API message formats - GETcurl -v https://ipwhois.app/json/8.8.8.8\?objects\=country,city,timezone
> GET /json/8.8.8.8?objects=country,city,timezone HTTP/1.1> Host: ipwhois.app> User-Agent: curl/7.58.0> Accept: */*>< HTTP/1.1 200 OK< Server: nginx/1.14.0< Date: Mon, 04 Jan 2021 12:43:10 GMT< Content-Type: application/json; charset=utf-8< Transfer-Encoding: chunked< Connection: keep-alive< X-Powered-By: PHP/7.4.6< Access-Control-Allow-Origin: *< Access-Control-Allow-Headers: *< X-Robots-Tag: noindex<* Connection #0 to host ipwhois.app left intact{"country":"UnitedStates","city":"Ashburn","timezone":"America\/New_York"}%
Request Headers
Response Headers
Response Body (Payload)
Status Code
Verb
83 v1.2
NETCONF vs RESTCONFNETCONF RESTCONF
Standard IETF IETFResources Paths URLsData Modeling Language YANG -Management Operation NETCONF HTTP OperationsEncoding XML XML, JSONTransport Stack SSH
TCPSSLHTTPTCP
84 v1.2
Data EncodingModel Driven Programmability
85 v1.2
Data Encoding• There needs to be structure behind what is communicated
between systemscore-router# show run interface GigabitEthernet 1 Building configuration...
Current configuration : 146 bytes ! interface GigabitEthernet1
vrf forwarding MANAGEMENT ip address 10.0.0.151 255.255.255.0 negotiation auto
end
• This is formatted text, not structured data• Standard data formats
• XML• JSON
86 v1.2
XML• XML stands for eXtensible Markup
Language
• Designed to store and transport data• In XML we have tags, elements, and
attributes• A tag is the text between <>, such as
<ip>
• An element is the starting tag, ending tag, and everything in between. Example from line 7-14
• An attribute is a key/value pair inside the starting tag of an element
<?xml version="1.0" encoding="UTF-8"?><GigabitEthernet>
<name>1</name><vrf>
<forwarding>MANAGEMENT</forwarding></vrf><ip>
<address><primary>
<address>10.0.0.151</address><mask>255.255.255.0</mask>
</primary></address>
</ip><negotiation xmlns="http://cisco.com/ns/yang/Cisco-
IOS-XE-ethernet" ><auto>true</auto>
</negotiation></GigabitEthernet>
123456789101112131415
161718
To start a comment, you use <!-- and then you end it with --> such as <!--This is a comment -->
87 v1.2
XML• Run the following command:curl -i -k -X "GET" "https://ios-xe-mgmt.cisco.com:9443/restconf/data/Cisco-IOS-XE-native:native/interface" -H "Accept: application/yang-data+xml" -u "developer:C1sco12345" -H "Content-Type: application/yang-data+xml"
<interface xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-native" xmlns:ios="http://cisco.com/ns/yang/Cisco-IOS-XE-native">
<GigabitEthernet><name>1</name><description>MANAGEMENT INTERFACE - DON'T TOUCH ME</description><ip><address><primary>
<address>10.10.20.48</address><mask>255.255.255.0</mask>
</primary></address>
</ip>
Output:
88 v1.2
JSON• JSON is based on a subset of the JavaScript programming language, as the name
implies• JSON is a syntax for storing and exchanging data• JSON has no requirement for indentation or white space. It is ideal to use white
space and spaces, most likely either two or four to make it human readable– Strings MUST use double quotes– Object literal names MUST be lowercase (null, false, true etc)– Special characters need to be escaped– { says “begin object”– } says “end object”– [ says “begin array”– ] says “end array”– : separates key and value in key/value pair– , separates key/value pair in an object or separates values in an array, think of it as “expect another one”
89 v1.2
JSON• JSON supports the following data
types:– Object– String– Number– Boolean– Null– Array
• It’s useful to use JSON formatter and validator
– https://jsonformatter.curiousconcept.com/
{"Cisco-IOS-XE-native:GigabitEthernet":{
"name":"1","vrf":{
"forwarding":"MANAGEMENT"},"ip":{
"address":{"primary":{
"address":"10.0.0.151","mask":"255.255.255.0"
}}
},"Cisco-IOS-XE-ethernet:negotiation":{
"auto":true}
}}
12345678910111213141516171819
90 v1.2
JSON• Run the following command:curl -i -k -X "GET" "https://ios-xe-mgmt.cisco.com:9443/restconf/data/Cisco-IOS-XE-native:native/interface" -H "Accept: application/yang-data+json" -u "developer:C1sco12345" -H "Content-Type: application/yang-data+json"
{"Cisco-IOS-XE-native:interface": {"GigabitEthernet": [{"name": "1","description": "MANAGEMENT INTERFACE - DON'T TOUCH ME","ip": {"address": {"primary": {"address": "10.10.20.48","mask": "255.255.255.0"
}}
},"mop": {"enabled": false,"sysid": false
},"Cisco-IOS-XE-ethernet:negotiation": {"auto": true
}},
Output:
91 v1.2
API Tools and Resources
92 v1.2
How to consume API• API documentation : Normally API contains documentation and examples, so
the first step is to read the doc to see auth and methods are supported, what format is expected, etc
– https://ipwhois.io/documentation– https://developers.facebook.com/docs/graph-api/overview/– https://sandbox-sdwan-1.cisco.com/apidocs– https://developer.cisco.com/meraki/api-v1/#!get-device
• Know API endpoint which consists of:– Server URL : Webserver. For example: NGNIX or Flask– Resource : Path like /device/interface/id or /client/payments/
• Client– Browser– CURL– Postman– Programming language
93 v1.2
CURL• Curl is a command-line tool for transferring data specified with URL syntax
• With curl, we can download or upload data using one of the supported protocols including HTTP, HTTPS, SCP, SFTP and FTP
• Install curl$ sudo apt install curl –y
• CURL CLI arguments-X --request - Custom request method-d --data - Sends the specified data
-H --header - Sends headers-i --include - Display response headers
94 v1.2
CURL - REST API• CURL verbose modecurl -v https://ios-xe-mgmt.cisco.com:9443
• CURL GET commandcurl -i -k -X "GET" "https://ios-xe-mgmt.cisco.com:9443/restconf/data/Cisco-IOS-XE-native:native/interface" -H "Accept: application/yang-data+xml" -u "developer:C1sco12345" -H "Content-Type: application/yang-data+xml"
95 v1.2
Postman• The Collaboration Platform for API Development
• Features– Create, send and save REST, SOAP or GraphQL requests– Edit URL– Select method or create and save custom method– Edit request headers and– Save preset headers– Manage cookies associated with various domains– Send multipart/form-data, url encoded, binary, or raw data in request body
96 v1.2
Postman• Download Postman
– https://www.postman.com/downloads/
• Postman Chrome Extension– https://chrome.google.com/webstore/detail/postman/fhbjgbiflinjbdggehcddcbncdd
domop?hl=en
97 v1.2
LabsModel Driven Programmability
98 v1.2
NETCONF• Model Driven Programmability NETCONF Labs
– Exercise 1: NETCONF get-config using SSH– Exercise 2 : NETCONF get-config using ncclient
99 v1.2
RESTCONF• Model Driven Programmability RESTCONF Labs
– Exercise 1: RESTCONF using Curl– Exercise 2 : RESTCONF using Postman
100 v1.2
References• YANG (https://tools.ietf.org/html/rfc6020)• NETCONF (https://tools.ietf.org/html/rfc6241)• RESTCONF (https://tools.ietf.org/html/rfc8040)• OpenConfig (https://github.com/openconfig/public)• Postman-for-AlwaysOn-Cisco-SD-WAN
(https://github.com/CiscoDevNet/Postman-for-AlwaysOn-Cisco-SD-WAN)
• YANG Catalog (https://yangcatalog.org/)• IOS XR Telemetry Collection Stack
(https://xrdocs.io/telemetry/tutorials/2018-06-04-ios-xr-telemetry-collection-stack-intro/)
101 v1.2
Thank You!
Top Related