Quality Assurance for WebRTC Services
-
Upload
tsahi-levent-levi -
Category
Technology
-
view
664 -
download
1
Transcript of Quality Assurance for WebRTC Services
![Page 1: Quality Assurance for WebRTC Services](https://reader034.fdocuments.in/reader034/viewer/2022042706/58714d5b1a28ab55588b7175/html5/thumbnails/1.jpg)
Quality Assurance for WebRTCServicesMarch 2016, Kranky Geek India
Tsahi Levent-Levi
![Page 2: Quality Assurance for WebRTC Services](https://reader034.fdocuments.in/reader034/viewer/2022042706/58714d5b1a28ab55588b7175/html5/thumbnails/2.jpg)
People don’t usually test their WebRTC implementationsSeriously
2
![Page 3: Quality Assurance for WebRTC Services](https://reader034.fdocuments.in/reader034/viewer/2022042706/58714d5b1a28ab55588b7175/html5/thumbnails/3.jpg)
And when they do, they test things manually
3
![Page 4: Quality Assurance for WebRTC Services](https://reader034.fdocuments.in/reader034/viewer/2022042706/58714d5b1a28ab55588b7175/html5/thumbnails/4.jpg)
Testing and automation are mandatory and widely usedMeanwhile, on other software fields…
4
![Page 5: Quality Assurance for WebRTC Services](https://reader034.fdocuments.in/reader034/viewer/2022042706/58714d5b1a28ab55588b7175/html5/thumbnails/5.jpg)
How do you go about automating your WebRTCtestbed?7 suggestions for you
5
W. Heath Robinson
![Page 6: Quality Assurance for WebRTC Services](https://reader034.fdocuments.in/reader034/viewer/2022042706/58714d5b1a28ab55588b7175/html5/thumbnails/6.jpg)
• WebRTC changes… monthly
• You have more than a voice callto test
Rely on the browser
6
4041
4243
4445
4647
4849
3536
3738
3940
4142
4344
45
Dec-14 Apr-15 Jul-15 Oct-15 Jan-16 May-16
Chrome
Firefox
Selenium is your friend
1
![Page 7: Quality Assurance for WebRTC Services](https://reader034.fdocuments.in/reader034/viewer/2022042706/58714d5b1a28ab55588b7175/html5/thumbnails/7.jpg)
Rely on the browser… and peer into the future
7
1
Stable What your customers are using
Beta What you should expect in the next release
DevWhere you can complain about breakage – and expect fixes in time
Canary A nightly build of whatever is available at that moment in time
Testing only against the Stable version of browsers means you find out issues only after the service breaks for your customers.
![Page 8: Quality Assurance for WebRTC Services](https://reader034.fdocuments.in/reader034/viewer/2022042706/58714d5b1a28ab55588b7175/html5/thumbnails/8.jpg)
> ffmpeg -i MYFILE.mp4 -pix_fmt yuv420p
> sed -i '0,/C420mpeg2/s//C420/' *.y4m
> google-chrome --use-fake-device-for-media-stream
--use-file-for-fake-video-capture=MYFILE.y4m
See http://testrtc.com/y4m-video-chrome/ for more details
Handle your media feeds
8
and think about the type of content you need2
![Page 9: Quality Assurance for WebRTC Services](https://reader034.fdocuments.in/reader034/viewer/2022042706/58714d5b1a28ab55588b7175/html5/thumbnails/9.jpg)
For some arcane reason (called AGC or something), browsers will loop your fake video but won’t look your fake audio
Loop an input 6 times (=expand 5 minutes to 30)> ffmpeg -f concat -i <(for i in {1..6}; do echo "file '$PWD/INPUT.wav'"; done) –c
copy OUTPUT.wav
> google-chrome --use-fake-device-for-media-stream
--use-file-for-fake-audio-capture=OUTPUT.wav
9
2 Handle your media feedsand make sure your audio recording is suitably long
![Page 10: Quality Assurance for WebRTC Services](https://reader034.fdocuments.in/reader034/viewer/2022042706/58714d5b1a28ab55588b7175/html5/thumbnails/10.jpg)
Take care of the network
10
Use netfilter.org iptables
Network Impairments
• Bitrate limitation• Packet loss• Jitter & latency
Firewall Configuration
• None• Force TURN relay• HTTP only• HTTPS only
Multiple Locations
• Virtualization sucks• Stability & Throughput
is what you need• Explore barebone
options
3
![Page 11: Quality Assurance for WebRTC Services](https://reader034.fdocuments.in/reader034/viewer/2022042706/58714d5b1a28ab55588b7175/html5/thumbnails/11.jpg)
• Caller / Callee
• Caller / Agent
• Caller / Expert
• Meeting point
• Conference
• Webinar
Handle synchronization
11
4
![Page 12: Quality Assurance for WebRTC Services](https://reader034.fdocuments.in/reader034/viewer/2022042706/58714d5b1a28ab55588b7175/html5/thumbnails/12.jpg)
Can you see the problem?
Visualize the results
12
5
![Page 13: Quality Assurance for WebRTC Services](https://reader034.fdocuments.in/reader034/viewer/2022042706/58714d5b1a28ab55588b7175/html5/thumbnails/13.jpg)
Visualize the results
13
Chrome 48 with VP8 Chrome 49 with VP9
5
![Page 14: Quality Assurance for WebRTC Services](https://reader034.fdocuments.in/reader034/viewer/2022042706/58714d5b1a28ab55588b7175/html5/thumbnails/14.jpg)
Are these real network issues with packet losses or bugs in these products and how they report packet loss?
Visualize the results (2)
14
5
![Page 15: Quality Assurance for WebRTC Services](https://reader034.fdocuments.in/reader034/viewer/2022042706/58714d5b1a28ab55588b7175/html5/thumbnails/15.jpg)
• How much time does it take to connect the session?
• Are the media channels you need really there?
• Are you sure there aren’t any “additional” media channels?
• Is the bitrate following your restrictions?
Don’t forget to set your expectations
15
6
![Page 16: Quality Assurance for WebRTC Services](https://reader034.fdocuments.in/reader034/viewer/2022042706/58714d5b1a28ab55588b7175/html5/thumbnails/16.jpg)
You never know when things will become handy…
Here are a few thoughts:
• chrome://webrtc-internals dump
• Browser console logs
• Media recordings
• Machine performance
Collect EVERYTHING
16
7
![Page 18: Quality Assurance for WebRTC Services](https://reader034.fdocuments.in/reader034/viewer/2022042706/58714d5b1a28ab55588b7175/html5/thumbnails/18.jpg)
Thank YouTsahi Levent-Levi
http://bloggeek.me
18