Post on 06-Jan-2017
getting touchyAN INTRODUCTION TO TOUCH AND POINTER EVENTS
Patrick H. Lauke / Version 1.4.14092015 / Smashing Conference Freiburg 2015
github.com/patrickhlauke/getting-touchy-presentation"evergreen" expanded version of this presentation
(and branches for specific conferences)
patrickhlauke.github.io/touch/tests/event-listener_mouse-only.html
patrickhlauke.github.io/touch/tests/event-listener_mouse-only.html
compatibility mouse events(mouseenter) > mouseover > mousemove* > mousedown >
(focus) > mouseup > click
* only a single “sacrificial” mousemove event fired
on first tap(mouseenter) > mouseover > mousemove >
mousedown > (focus) > mouseup > click
subsequent tapsmousemove > mousedown > mouseup > click
tapping awaymouseout > (blur)
focus / blur only on focusable elements in Firefoxmouseout not on iOS Safari/WebView (e.g. iOS Chrome)
Opera Mobile and
1. delayed event dispatch2. mousemove doesn't track
("evergreen" presentation also covers hover/mouseover)
patrickhlauke.github.io/touch/tests/event-listener_show-delay.html
patrickhlauke.github.io/touch/tests/event-listener_show-delay.html
touch events
introduced by Apple, adoptedin Chrome/Firefox/Opera
www.w3.org/TR/touch-events
patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html
patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.htmlBug 128534 - 'mouseenter' mouse compat event not fired...
events fired on taptouchstart > [touchmove]+ > touchend >
(mouseenter) > mouseover > mousemove > mousedown >(focus) > mouseup > click
(mouse events only fired for single-finger tap)
on first taptouchstart > [touchmove]+ > touchend >
(mouseenter) > mouseover > mousemove > mousedown >(focus) > mouseup > click
subsequent tapstouchstart > [touchmove]+ > touchend >mousemove > mousedown > mouseup > click
tapping awaymouseout > (mouseleave) > (blur)
• too many touchmove events prevent mouse compatibility events
after touchend (not considered a "clean" tap)
• too many touchmove events on activatable elements can lead to
touchcancel (in old Chrome/Browser versions)
• not all browsers consistently send the touchmove
some browsers outright weird...
Browser/Android 4.3(AppleWebKit/534.30)
mouseover > mousemove > touchstart > touchend >mousedown > mouseup > click
Browser/Blackberry PlayBook 2.0(AppleWebKit/536.2)
touchstart > mouseover > mousemove > mousedown >touchend > mouseup > click
patrickhlauke.github.io/touch/tests/event-listener_show-delay.html
patrickhlauke.github.io/touch/tests/event-listener.html
touch / mouse events delaytouchstart > [touchmove]+ > touchend >
[300ms delay]
(mouseenter) > mouseover > mousemove > mousedown >(focus) > mouseup > click
/* feature detection for touch events */
if ( 'ontouchstart' in window ) { /* some clever stuff here */}
/* older browsers have flaky support so more hacky tests needed...use Modernizr.touch or similar */
/* common performance “trick” */
var clickEvent = ( 'ontouchstart' in window ? 'touchend' : 'click' );
blah.addEventListener( clickEvent , function() { ... }, false);
/* if touch events are supported, only listen to touchend, not click */
patrickhlauke.github.io/touch/tests/event-listener_show-delay-naive-event-fix.html
Bug 888304 - touch-events on Firefox-desktop should be disabled until we can support themproperly
Issue 392584: Enable TouchEvent API all the time
Android + mouse – behaves like touchtouchstart > touchend > mouseover > mousemove > mousedown >
(focus) > mouseup > click
Windows 10 Mobile/Microsoft Edge + mouse - like desktopmouseover > mousedown > mousemove > mouseup > click
iOS + VoiceOver (with/without keyboard) – similar to touchfocus > touchstart > touchend > (mouseenter) > mouseover >
mousemove > mousedown > blur > mouseup > click
further scenarios?• desktop with external touchscreen
• desktop with external touchpad
• touchscreen laptop with non-touch second screen
• touchscreen laptop with trackpad/mouse
• ...and other permutations?
hacks.mozilla.org - Detecting touch [...]
/* feature detection for touch events */
if ('ontouchstart' in window) { /* browser supports touch events but user is not necessarily using touch (exclusively) */
/* it could be a mobile, tablet, desktop, fridge ... */}
/* doubled-up event listeners */
foo.addEventListener(' touchend ', someFunction, false);foo.addEventListener(' click ', someFunction, false);
/* but this would fire our function twice for touch? */
/* doubled-up event listeners */
foo.addEventListener('touchend', someFunction, false);foo.addEventListener('click', someFunction, false);
/* prevent mouse events + click as part of the common handler */
function someFunction(e) { ... if (e.type == 'touchend') { e.preventDefault(); } ...}
patrickhlauke.github.io/touch/fastclick/fastclick.htmlYouTube: iOS/Safari 300ms click delay: vanilla Bootstrap and using fastclick.js
<meta name="viewport" content="user-scalable=no">patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html
... content="minimum-scale=1, maximum-scale=1"patrickhlauke.github.io/touch/tests/event-listener_minimum-maximum-scale.html
"Force enable zoom" reintroduces delaypatrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html
Chrome 32+ / Android: content="width=device-width"suppresses double-tap-to-zoom, still allows pinch zoom
Google Developers: 300ms tap delay, gone away
Bug 941995 - Remove 300ms [...] on "responsive" pages[RESOLVED FIXED]
patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html
iOS/Safari designed themselves into a corner: “double-tap to scroll”Bug 122212 - Optimizations to remove 300ms touch > mouse events delay
events fired on taptouchstart > [touchmove]+ > touchend >
(mouseenter) > mouseover >mousemove* > mousedown > (focus) >
mouseup > click
* mouse event emulation fires only a single mousemovetoo many touchmove events prevent mouse compatibility events after touchend
var posX, posY;...function positionHandler(e) { posX = e.clientX ; posY = e.clientY ;}...canvas.addEventListener(' mousemove ', positionHandler, false);
var posX, posY;...function positionHandler(e) { posX = e.clientX ; posY = e.clientY ;}...canvas.addEventListener(' mousemove ', positionHandler, false);canvas.addEventListener(' touchmove ', positionHandler, false);
/* but this won't work for touch... */
interface MouseEvent : UIEvent { readonly attribute long screenX ; readonly attribute long screenY ; readonly attribute long clientX ; readonly attribute long clientY ; readonly attribute boolean ctrlKey; readonly attribute boolean shiftKey; readonly attribute boolean altKey; readonly attribute boolean metaKey; readonly attribute unsigned short button; readonly attribute EventTarget relatedTarget; void initMouseEvent(...);};
www.w3.org/TR/DOM-Level-2-Events/events.html#Events-MouseEvent
partial interface MouseEvent { readonly attribute double screenX; readonly attribute double screenY; readonly attribute double pageX ; readonly attribute double pageY ; readonly attribute double clientX; readonly attribute double clientY; readonly attribute double x ; readonly attribute double y ; readonly attribute double offsetX ; readonly attribute double offsetY ;};
www.w3.org/TR/cssom-view/#extensions-to-the-mouseevent-interface
interface TouchEvent : UIEvent { readonly attribute TouchList touches ; readonly attribute TouchList targetTouches ; readonly attribute TouchList changedTouches ; readonly attribute boolean altKey; readonly attribute boolean metaKey; readonly attribute boolean ctrlKey; readonly attribute boolean shiftKey;};
www.w3.org/TR/touch-events/#touchevent-interface
interface Touch { readonly attribute long identifier; readonly attribute EventTarget target; readonly attribute long screenX ; readonly attribute long screenY ; readonly attribute long clientX ; readonly attribute long clientY ; readonly attribute long pageX ; readonly attribute long pageY ;};
www.w3.org/TR/touch-events/#touch-interface
touches
all touch points on screen
targetTouches
all touch points that started on the element
changedTouches
touch points that caused the event to fire
patrickhlauke.github.io/touch/touchlist-objects
var posX, posY;...function positionHandler(e) { if ((e.clientX)&&(e.clientY)) { posX = e.clientX; posY = e.clientY; } else if (e.targetTouches) { posX = e.targetTouches[0].clientX; posY = e.targetTouches[0].clientY; e.preventDefault() ; }}...canvas.addEventListener('mousemove', positionHandler, false );canvas.addEventListener('touchmove', positionHandler, false );
www.splintered.co.uk/experiments/archives/paranoid_0.5
interface TouchEvent : UIEvent { readonly attribute TouchList touches ; readonly attribute TouchList targetTouches ; readonly attribute TouchList changedTouches ; readonly attribute boolean altKey; readonly attribute boolean metaKey; readonly attribute boolean ctrlKey; readonly attribute boolean shiftKey;};
www.w3.org/TR/touch-events/#touchevent-interface
/* iterate over touch array */
for (i=0; i< e.targetTouches .length; i++) { ... posX = e.targetTouches[i].clientX ; posY = e.targetTouches[i].clientY ; ...}
patrickhlauke.github.io/touch/tracker/multi-touch-tracker.html
/* iOS/Safari/WebView has gesture events for size/rotation,not part of the W3C Touch Events spec. */
gesturestart / gesturechange / gestureend
function(e) { /* e.scale e.rotation */}
/* not supported in Chrome/Firefox/Opera */
/* with some trigonometry we can replicate these from basic principles. */
var distance = Math.sqrt(Math.pow(...)+Math.pow(...));var angle = Math.atan2(...);
patrickhlauke.github.io/touch/picture-organiser
Issue 162757: Implement pointer events in Chrome behind experimental flag(however, more on this later...)
hacks.mozilla.org - Pointer Events now in Firefox Nightly(at the moment, only for mouse and quite basic)
Bug 105463 - Implement pointer events RESOLVED WONTFIX
Maciej Stachowiak - [webkit-dev] pointer events specification - first editors draft
@patrick_h_lauke paraphrasing Apple's stance on Pointer Events...
patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html
events fired on tap (Edge)mousemove* >
pointerover > mouseover >pointerenter > mouseenter >pointerdown > mousedown >
focusgotpointercapture >
pointermove > mousemove >pointerup > mouseup >lostpointercapture >
click >pointerout > mouseout >pointerleave > mouseleave
mouse events fired “inline” with pointer events(for a primary pointer, e.g. first finger on screen)
vendor-prefixed in IE10MSPointerDown etc
navigator.msMaxTouchPoints-ms-touch-action
unprefixed in IE11 (but prefixed versions still mapped for compatibility)
/* Pointer Events extend Mouse Events vs Touch Events and their completely new objects/arrays */
interface PointerEvent : MouseEvent { readonly attribute long pointerId; readonly attribute long width; readonly attribute long height; readonly attribute float pressure; readonly attribute long tiltX; readonly attribute long tiltY; readonly attribute DOMString pointerType; readonly attribute boolean isPrimary;}
/* plus all the classic MouseEvent attributes like clientX , clientY , etc */
/* detecting pointer events support */
if ( window.PointerEvent ) { /* some clever stuff here but this covers touch, stylus, mouse, etc */}
/* still listen to click for keyboard! */
/* detect maximum number of touch points */
if ( navigator.maxTouchPoints > 0 ) { /* device with a touchscreen */}
if ( navigator.maxTouchPoints > 1 ) { /* multitouch-capable device */}
patrickhlauke.github.io/touch/tests/event-listener.html
(IE11/WinPhone 8.1 Update no optimization for width=device-width )
patrickhlauke.github.io/touch/tests/event-listener.html(IE/Win8 has double-tap to zoom, so problem on desktop too)
patrickhlauke.github.io/touch/tests/event-listener.html(Microsoft Edge/Win10 has double-tap to zoom, so problem on desktop too)
• double-up pointerup and click listeners?
• prevent code firing twice with preventDefault ?
won't work: preventDefault() stops mouse compatibility events, butclick is not considered mouse compatibility event
CSS propertywhat action should the browser handle?
touch-action: auto | none | [ pan-x || pan-y ] | manipulation
www.w3.org/TR/pointerevents/#the-touch-action-css-propertyonly determines default touch action, does not stop compatibility
mouse events nor click
patrickhlauke.github.io/touch/tests/event-listener_touch[...]
patrickhlauke.github.io/touch/tests/event-listener_touch[...]
patrickhlauke.github.io/touch/tests/event-listener_touch[...]
patrickhlauke.github.io/touch/tests/event-listener_touch[...]
Bug 979345 - Implement touch-action:manipulation [...][RESOLVED FIXED]
Issue 349016: Add support for touch-action:manipulationchrome://flags/#enable-experimental-web-platform-features (Chrome 35)
[FIXED]
Bug 133114 - Implement " touch-action:manipulation " [...]Apple at least considering this, for once...
<meta name="viewport" content="user-scalable=no">patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html
... content="minimum-scale=1, maximum-scale=1"patrickhlauke.github.io/touch/tests/event-listener_minimum-maximum-scale.html
Microsoft Edge removes delay for width=device-widthpatrickhlauke.github.io/touch/tests/event-listener_width-device-width.html
patrickhlauke.github.io/touch/particle/2mousemove / pointermove fire, but browser scroll action takes over
patrickhlauke.github.io/touch/particle/2a(does not work in Microsoft Edge/Windows 10 Mobile
due to touch events support)
/* touch events: separate handling */
foo.addEventListener('touchmove', ... , false);foo.addEventListener('mousemove', ... , false);
/* pointer events: single listener for mouse, stylus, touch */
foo.addEventListener(' pointermove ', ... , false);
/* Pointer Events extend Mouse Events */
foo.addEventListener(' pointermove ', function(e) { ... posX = e.clientX ; posY = e.clientY ; ...}, false);
www.w3.org/TR/pointerevents/#pointerevent-interface
3D Rotator by Creative Punchcoded to only use mouse events
3D Rotator modified to use Pointer Eventsminimal code changes, as Pointer Events extend mouse events
foo.addEventListener('pointermove', function(e) { ... switch( e.pointerType ) { case ' mouse ': ... break; case ' pen ': ... break; case ' touch ': ... break; default : /* future-proof */ } ...} , false);
/* in IE11/Edge, pointerType returns a string in IE10, the return type is long */
MSPOINTER_TYPE_TOUCH: 0x00000002MSPOINTER_TYPE_PEN: 0x00000003MSPOINTER_TYPE_MOUSE: 0x00000004
MSDN: IE Dev Center - API reference - pointerType property
/* PointerEvents don't have the handy TouchList objects, so we have to replicate something similar... */
var points = [];switch (e.type) { case ' pointerdown ': /* add to the array */ break; case ' pointermove ': /* update the relevant array entry's x and y */ break; case ' pointerup ': /* remove the relevant array entry */ break;}
patrickhlauke.github.io/touch/tracker/multi-touch-tracker-pointer.html
(note multiple isPrimary pointers)
simultaneous use of inputs ishardware-dependent
(e.g. Surface 3 "palm blocking" prevents concurrent
touch/stylus/mouse, but not
touch/external mouse/external stylus)
YouTube: Pointer Events support multiple stylus/pen inputs simultaneously
/* like iOS/Safari, IE/Win has higher-level gestures , but these are not part of the W3C Pointer Events spec.
Replicate these from basic principles again... */
MSDN IE10 Developer Guide: Gesture object and events
/* Pointer Events - pressure*/
interface PointerEvent : MouseEvent { readonly attribute long pointerId; readonly attribute long width; readonly attribute long height; readonly attribute float pressure ; readonly attribute long tiltX; readonly attribute long tiltY; readonly attribute DOMString pointerType; readonly attribute boolean isPrimary;}
pressure : value in range [0,1]. if no hardware support,0.5 in active button state, 0 otherwise
patrickhlauke.github.io/touch/tracker/...YouTube: Touch tracker with Surface 3 pen
hovering stylus• hardware-dependant
• pointermove fires
• pressure == 0 (non-active button state)
• track pointerdown / pointerup to be safe
/* Pointer Events - tilt */
interface PointerEvent : MouseEvent { readonly attribute long pointerId; readonly attribute long width; readonly attribute long height; readonly attribute float pressure; readonly attribute long tiltX ; readonly attribute long tiltY ; readonly attribute DOMString pointerType; readonly attribute boolean isPrimary;}
tiltX / tiltY : value in degrees [-90, 90].returns 0 if hardware does not support tilt
patrickhlauke.github.io/touch/pen-trackerYouTube: ...pen with tilt, pressure and hover support
Issue 162757: Implement pointer events in Chrome behind experimental flag WontFix
1. performance issues (hit-testing)
2. unified event model not "mobile first"
3. difficult to implement "pull to refresh"
4. we already have touch events
1. performance issues (hit-testing)
2. unified event model not "mobile first"
3. difficult to implement "pull to refresh"
4. we already have touch events
5. Apple won't implement them...
Issue 404128: Meta: Extend TouchEvent API to have the power of PointerEvents...
W3C Touch Events Extensions WG Note
/* extension to touch objects */
partial interface Touch { readonly attribute float radiusX; readonly attribute float radiusY; readonly attribute float rotationAngle; readonly attribute float force;};
Google Developers: Using rotationAngle and touchRadius
W3C Touch Events - Level 2 (Editor's Draft)(merges errata, touch events extensions, fractional touch coordinates)
Chrome Canary (currently 47.0.2504.0) has rough experimentalsupport for Pointer Events (for desktop w/touch only)chrome.exe --enable-blink-features=PointerEvent
work now continues to enhance/expand Pointer Events...W3C Pointer Events - Level 2 (Editor's Draft)
MSDN IEBlog: The Mobile Web should just work for everyoneWindows Phone 8.1 Update now supports Pointer Events and Touch Events
touchstart > [touchmove]+ > touchend >[300ms delay] >
mouseover > mousemove > mousedown > mouseup > click
vs
pointerover > mouseover > pointerdown > mousedown >pointermove > mousemove > pointerup > mouseup >
[300ms delay] > click >pointerout > mouseout > pointerleave > mouseleave
patrickhlauke.github.io/touch/tests/event-listener.html
events fired on tap (IE11)mousemove* > pointerover > mouseover >
pointerenter > mouseenter > pointerdown >touchstart >
mousedown > gotpointercapture > focus >pointermove > touchmove > mousemove >
pointerup >touchend >
mouseup > lostpointercapture > pointerout > mouseout >pointerleave > mouseleave >
click
IE11/Windows Phone 8.1u1with "frankensteined" Pointer/Touch Events support
(but not on desktop)
patrickhlauke.github.io/touch/tests/event-listener.html
events fired on tap (Microsoft Edge)pointerover > pointerenter > pointerdown >
touchstart > gotpointercapture
pointermove > touchmove > pointerup >touchend >
mouseover > mouseenter > mousemove > mousedown >focus >
mouseup > click > lostpointercapture >
pointerout > pointerleave
Microsoft Edge/Windows 10 Mobile closer to touch events(note grouped mouse compatibility events)
W3C Touch Events WG mailing listJacob Rossi - Enabling Touch Events everywhere
/* cover all cases (hat-tip Stu Cox) */
if ('onpointerdown' in window) {
/* bind to Pointer Events: pointerdown, pointerup, etc */
} else {
/* bind to mouse events: mousedown, mouseup, etc */
if ('ontouchstart' in window) { /* bind to Touch Events: touchstart, touchend, etc */ }}
/* bind to keyboard / click */
GitHub - Polymer/PointerEvents deprecated
jQuery blog - Improving the Pointer Events Polyfill
/* adding jQuery PEP */
<script src="https://code.jquery.com/pep/0.3.0/pep.js"></script>
/* need to use custom touch-action attribute, not CSS (yet) */
<button touch-action="none" >...</div>
/* Hammer's high-level events example */
var element = document.getElementById('test_el');
var hammertime = new Hammer(element);
hammertime.on("swipe", function(event) { /* handle horizontal swipes */});
• Matt Gaunt – Touch Feedback for Mobile Sites
• Jonathan Stark – FastActive
• Stephen Woods – HTML5 Touch Interfaces
• YouTube: Stephen Woods – Responsive HTML5 Touch Interfaces
• Chris Wilson + Paul Kinlan – Touch And Mouse: Together Again For
The First Time
• Ryan Fioravanti – Creating Fast Buttons for Mobile Web Applications
• Boris Smus – Multi-touch Web Development
• Boris Smus – Generalized input on the cross-device web
• Boris Smus – Interactive touch laptop experiments
• Rick Byers + Boris Smus (Google I/O) – Point, Click, Tap, Touch -
Building Multi-Device Web Interfaces
• Grant Goodale – Touch Events
• W3C – Touch Events Extensions
• Mozilla Developer Network – Touch Events
• WebPlatform.org – Pointer Events
• Rick Byers – The best way to avoid the dreaded 300ms click delay is
to disable double-tap zoom
• Chromium Issue 152149: All touch-event related APIs should exist if
touch support is enabled at runtime
• Tim Kadlec – Avoiding the 300ms Click Delay, Accessibly
• David Rousset - Unifying touch and mouse [...]
• Microsoft – Pointer events updates (differences between IE10-IE11)
• Patrick H. Lauke – Webseiten zum Anfassen
• Patrick H. Lauke – Drei unter einem Dach: Pointer-Events für Maus,
Stylus und Touchscreen
• Patrick H. Lauke – Make your site work on touch devices
• Stu Cox – You can't detect a touchscreen
• Tilo Mitra – The State of Gestures
• YouTube: Tilo Mitra (YUI) – The State of Gestures
• Microsoft – Hover touch support (IE10/IE11)
• W3C Media Queries Level 4 – pointer
• Stu Cox – The Good & Bad of Level 4 Media Queries
• Peter-Paul Koch – Touch table
• Peter-Paul Koch – Preventing the touch default
• Peter-Paul Koch – Mouse event bubbling in iOS
• YouTube: Edge Conference (Feb 2013 London) – Panel: Input
• Edge Conference (Mar 2014 London) – Panel: Pointers and
Interactions
• Trent Walton – Device-Agnostic
• Stu Cox – The Golden Pattern for Handling Touch Input
• Matt Gaunt – ‘Focusing’ on the Web Today
• Mobiscroll – Working with touch events
• Peter-Paul Koch – The iOS event cascade and innerHTML
• Patrick H. Lauke – Make your site work on touch devices
• Scott Jehl (Filament Group) – Tappy: A custom tap event handler
• Yehuda Katz – Touch events on the web are fundamentally
unfinished ...
• Andrea Giammarchi – PointerEvents No More
• Google Developers – Web Fundamentals: Stateful Elements Respond
to Touch
• YouTube: Matt Gaunt (Google) - Touch in a Web App? No Problem
• Luke Wroblewski – How to Communicate Hidden Gestures
• David Washington - Designing custom touch interactions
• David Washington - Building pull-to-refresh
• Andy Peatling - JavaScript Pull to Refresh for the Web
• Rob Larsen - The Uncertain Web: Pointer Event Polyfill and Chrome
Fragmentation
• Detlev Fisher - Implications of new input modes (touch, speech,
gestures) for the Web Content Accessibility Guidelines
• Ralph Thomas - Towards declarative touch interactions
• Windows Dev Center: Windows desktop applications > Guidelines >
Interaction (touch, keyboard, mouse, pen)
• Microsoft Open Technologies - jQuery Adopts Pointer Events
• jQuery blog - Getting on Point
• IEBlog - Pointer Events W3C Recommendation, Interoperable Touch,
and Removing the Dreaded 300ms Tap Delay
• Microsoft Open Technologies - Pointer Events is now a W3C Standard
• Patrick H. Lauke (The Paciello Group) - Pointer Events advance to W3C
Recommendation
• Jacob Rossi - The Input Shouldn't Matter
get in touch@patrick_h_lauke
github.com/patrickhlauke/touchpatrickhlauke.github.io/getting-touchy-presentation
slideshare.net/reduxpaciellogroup.com
splintered.co.uk