ForwardJS 2017 - Fullstack end-to-end Test Automation with node.js
-
Upload
mek-srunyu-stittri -
Category
Technology
-
view
175 -
download
1
Transcript of ForwardJS 2017 - Fullstack end-to-end Test Automation with node.js
Full-Stack End-to-End Test Automation with
Node.jsOne year later
ForwardJS 2017Chris Clayman
Mek Stittri
2
First things first
Airware, we are a drone company.We build complete enterprise drone solutions; Cloud Platform, Ground Control Station, Drone Integration and etc
We are Hiring! - http://www.airware.com/careers● iOS Software Engineer● Javascript Software EngineerOr just come talk to us :)
3
A Bit About UsThe Automation team at Airware, we do drone stuffs.Continuous Deployment & Test Automation tools aka Quality Ops
Chris [email protected]/kidmillions
Mek [email protected]/mekdev
Overheard at the Monday workshops
"I don't write tests, testing sucks"
Let's talk About Test Automation
Let us give you a warm hug.
Why Node.js for Test Automation
QA, Automation engineers
Frontend engineers
Backend engineers
Java
Javascript
Java
Python
Javascript
Java
Ruby
Javascript
Node.js
Company A Company B Company C
Node.js
Javascript
Node.js
Airware
- Node.js adoption, more and more company moving to node.js- Share code with the whole engineering team- Play to the engineering team’s strength
True Full-Stack Delivery Team
Functional tests that interacts with the webapp as a user would.
Full-Stack End-to-End Automation
Visual
UI Interaction
Http Restful API
Javascript Unit tests
Microservice Integration
Unit tests
DB & Data Schema tests
Martin Fowler - Test Pyramid, Google Testing Blog2014 Google Test Automation Conference
Tools for tests● Visual Diff Engine ● Selenium-Webdriver ● REST API Client
SmallUnit tests
BigE2E tests
MediumIntegration tests
z’ Test Pyramid
Cost of test
Amount of tests
Client implementation that maps to your HTTP RESTful Backend API
Simulating browser AJAX calls and acts like a user.Via GET / POST / PUT / DELETE requests and etc..
Request libraries:- requests - https://github.com/request/request- request-promise - https://github.com/request/request-promise- etc...
REST API Client
Anyone here use selenium for automation ?
Industry standard for web browser automation
Language-neutral coding interface: C#, JavaScript , Java, Python, Ruby
Official Project - https://github.com/SeleniumHQ/selenium
Selenium Webdriver
$ npm i selenium-webdriver
Visual Diff Engine
Bitmap comparison tool for screenshot validation.
Validates a given screenshot against a baseline image
Available tools:- Applitools - https://applitools.com/ integrates with selenium JS- Sikuli, roll your own framework - http://www.sikuli.org/
Baseline Changes
Timeline
Q2 2015
The Early AgeRapid Prototyping Phase
The Middle Age“The Promise Manager”
Evaluating multiple test frameworks
Test Runner- Mocha / Karma- Ava / JestAPI - Http library- request- co-request- request-promise- Supertest / ChakramUI - Selenium bindings- Nightwatch- WebdriverIO- selenium-webdriver (WebdriverJS)
Q3 2015 Q3 2016 - Present
- ES6/ES2015- Promises- ESLint- Went with Mocha- Picked selenium-webdriveraka WebDriverJsBuilt-in Promise Manager(synchronous like behavior)- Went with co-requestGenerator calls, easier(synchronous like behavior)- Applitools visual diff
- ES7/2016+- Async Await- Babel- ESLint + StandardJS- Flowtype (facebook)- selenium-webdriver- Switched to request lib
Async / Await solves a lot of the previous problems, no more callbacks or thenables
The Modern Age“Async/Await” and beyond
Lessons Learnt
- Representation of Pages (Page Objects)- Readable tests- Frontend testability- Robust backend API
- Clients to interact with said API- De-async testing approach- Visual tests- Accurate and descriptive reporting
Automation Framework Ingredients
- Representation of a page or component.
- We use class syntax- Abstract UI interaction
implementation to a BasePage or UI utility outside of the pageobject
- Map page objects 1-to-1 to pages and components
Page Object Model (POM)
const CHANGE_PASSWORD_BTN = 'button.change-passwd'
class ProfilePage extends BasePage {
clickChangePassword () {
click(CHANGE_PASSWORD_BTN)
}
}
Readable Tests
clusterImageGallery.currentSelectedImageAnnotationIndicatorExistsInTrack()
Great work, Chip!
- Anyone should be able to read or write tests- Abstract low level webdriver implementation
outside of page objects, e.g. Navigation utility- Long-as-hell method names are actually OK
as long as they clarify the test
- Testability is a product requirement- Cook data models into your elements- Stop Xpath in its tracks. Use CSS selectors
Frontend Testability
Xpath Selector//div[@class='example']//a
CSS Selectordiv.example a
Frontend Testability- Data attributes for testing framework - image name, image uuid- Clear and concise selectors (ideally separate from styling concerns)
data attributes: image id, name
Concise selectors- class .main-image- Image track .slick-track- Selection status .selected
- UI tests should limit UI interaction to only the functionality being tested
- Build a client to speak to your backend through HTTP (still act as an ‘outsider’ user)
- Setup test data, simply navigate to it, and interact
- If you can’t expose your backend, bootstrap test data beforehand
Use the Backend API
it('Validate Bob\'s profile', function () {
const user = apiClient.getUser('bob')
// if you know Bob's user id,
// you can go straight to his profile
driver.goToProfile(user.id)
const title = profilePage.getTitle()
assert.isEqual(title, 'Bob\'s Killer Profile')
})
The gordian knot:
- Selenium actions and the WebDriver API are asynchronous
- JavaScript is built for asynchronicity- Tests should run in a synchronous
manner and in a wholly deterministic order
De-asyncing Tests
How to cleanly write async actions in a sync manner?
….sandwiches?
It’s like we finish each other’s….
- At first, we used the promise manager extensively- Heavily dependent on selenium-webdriver control flows
The promise manager / control flow● Implicitly synchronizes asynchronous actions● Coordinates the scheduling and execution of all commands● Maintains a queue of scheduled tasks and executes them in order
De-asyncing Via Promise Manager
driver.get('http://www.google.com/ncr');driver.findElement({name: 'q'}).sendKeys('webdriver');driver.findElement({name: 'btnGn'}).click();
driver.get('http://www.google.com/ncr') .then(function() { return driver.findElement({name: 'q'});}) .then(function(q) { return q.sendKeys('webdriver');}) .then(function() { return driver.findElement({name: 'btnG'});}) .then(function(btnG) { return btnG.click();});
test.it('Verify data from both frontend and backend', function() { var apiClient = new ApiClient(); var projectFromBackend; // API Portion of the test var flow = webdriver.promise.controlFlow(); flow.execute(function *(){ yield webClient.login(USER001_EMAIL, USER_PASSWORD); var projects = yield apiClient.getProjects(); projectFromBackend = projectutil.getProjectByName(projects, QA_PROJECT); }); // UI Portion of the test var login = new LoginPage(driver); login.enterUserInfo(USER001_EMAIL, USER_PASSWORD); var topNav = new TopNav(driver); topNav.getProjects().then(function (projects){ Logger.debug('Projects from backend:', projectsFromBackend); Logger.debug('Projects from frontend:', projects); assert.equal(projectsFromBackend.size, projects.size);});
- Not Readable / flat- Not testrunner-agnostic- Non-standard
API part of the testHTTP Requests
Issues with Promise Manager
Context switch to REST API calls
UI Part of the testSelenium & Browser
var test = require('selenium-webdriver/testing');
JobsPage.prototype.getJobList = function () { this.waitForDisplayed(By.css('.job-table')); const jobNames = []; const defer = promise.defer(); const flow = promise.controlFlow(); const jobList = this.driver.findElement(By.css('.job-table')); // get entries flow.execute(() => { jobList.findElements(By.css('.show-pointer')).then((jobs) => { // Get text on all the elements jobs.forEach((job) => { let jobName; flow.execute(function () { job.findElement(By.css('.job-table-row-name')).then((element) => { element.getText().then((text) => { jobName = text; });
// look up more table cells... }); }).then(() => { jobNames.push({ jobName: jobName }); }); }); }); }).then(() => { // fulfill results defer.fulfill(jobNames); }); return defer.promise;};
Bad complexity + promise chaining =
Async / Await
function notAsync () {
foo().then((bar) => {
console.log(bar)
});
}
async function isAsync() {
const bar = await foo();
console.log(bar)
}
Node > 7.6ES5
Given function foo() that returns a promise...
C’mon gang, let’s write a test!https://github.com/airware/webdriver-mocha-async-await-example
- When reporting, consider what information you need in order to be successful- Failures First- Flakiness vs. Failure- Assertion messaging - Screenshots- Video
- Screenshots very useful if done right- Integrate them into test reports- Use them for visual testing (shoutout to Applitools)
Reporting and Visual Testing
"I don't write tests, testing sucks"
"Test automation is easy with
JavaScript "
When You Talk About Test Automation
Airware github repo code example:https://github.com/airware/webdriver-mocha-async-await-example
ForwardJS OrganizersDave, Taylor, Tracy and team
Special Thanks
Saucelabs - browser CI infrastructureFeedback on selenium and node.js prototyping
● Neil Manvar● Kristian Meier● Adam Pilger● Christian Bromann
ApplitoolsAutomated Visual Diff Engine from Applitools
● Moshe Milman● Matan Carmi● Adam Carmi● Ryan Peterson
MochaJSGithub#mochajs
Test runner
Selenium WebdriverGithub
BrowserAutomation
WebdriverIOGithub@webdriverioEasy to useSelenium APIMature Mobile APIs
AppiumGithub@AppiumDevs
Mobile & Native Apps Automation
JavaScript Community and Open Source Projects. The world runs on OSS - Please help support these projects!