Learn Embedded Linux by Building
a Video Surveillance Device
Timesys University
2 2 Timesys Staff
With you today are:
Maciej Halasz
Product Management
412 325 6313
Theresa Kisha
Marketing
412 325 6362
Al Feczko
Technical Sales &
Field Engineering
412 325 6390
3 3 Timesys Staff
With you today are:
Maciej Halasz
Product Management
412 325 6313
Theresa Kisha
Marketing
412 325 6362
Al Feczko
Technical Sales &
Field Engineering
412 325 6390
4 4 Timesys Staff
With you today are:
Theresa Kisha
Marketing
412 325 6362
Al Feczko
Technical Sales &
Field Engineering
412 325 6390
Maciej Halasz
Product Management
412 325 6313
Aparna Nigam
Technical Sales
412 325 6333
5 5 We Are Building a … Video Surveillance Device!
More than one third of all shops are under video surveillance
6
7 7 Building a Video Surveillance Device with LinuxLink — Agenda
Module 1: BSP & Dev Environment/SDK How to assemble and deploy an initial BSP and setup development
environment with the matching SDK
Module 2: User Interface How to build a state-of-the-art User Interface using
Qt Embedded for Linux
Module 3: RTOS & MCC How to monitor a light sensor in RTOS and trigger video recording
under Linux
Module 4: Optimizations How to measure and optimize to boot more quickly
8 8 Module 1
Your Development Environment
Embedded Linux Primer
Where do I start my Linux design — Product
Requirements
Assemble a custom BSP using the
LinuxLink FREE Edition Web Interface
• Reflect product requirements
• Produce complete Linux design
• Learn how to find information
Application/System Development environment
setup with an SDK
Deploy the images on the Phytec Vybrid system
9 9
Your Development Environment:
Session Hardware/Software
Requirements
10 10 What We Need to Build Our Product
A host machine
• Linux is the best – any recent version is ok (Timesys recommends
Ubuntu LTS)
• Windows is fine, but you’ll need a virtual machine with a Linux desktop OS installation (Timesys recommends Oracle VirtualBox)
Cross-development environment / SDK
Linux source code (BSP) for the product
• Bootloader
• Linux kernel
• APIs
Various Linux utilities
e.g. mkimage
IDE with WYSIWYG for
faster development
A hardware development kit
11 11 Your Workstation Setup
Development
Environment
• Cross tools
• IDE
Exercises
USB
Ethernet/Serial
ETH
12 12 Having Fun Yet?
Question bucket
• We will ask some review questions at the end of each Module.
• In return for the first correct answer for each question asked, we will award a giveaway item.
13 13
Embedded Linux Primer
14 14
Repository Docs Support
Facto
ry - M
ulti O
S C
onfigura
tion &
B
uild T
ool
AR
M D
S5 - D
evelopm
ent &
D
ebug T
ools
Multi OS Infrastructure
Bootloader
Tower Modules
Cortex A5 Cortex M4
App on MQX
Linux Kernel
App on Linux
Message passing
MQX
App on Linux
AR
M D
ST
RE
AM
- JT
AG
Support
Complete Software Solution for Vybrid
Qt Application PEG Application
Device
Drivers
Device
Drivers
Demos
15
©2013 Timesys Corp.
Memory Model
A5 - Linux M4 - MQX
Shared SRAM chip
Hardware
Semaphore
Layer used for
Multi-Core Communication
(MCC)
DDR3 memory chip
Linux Address Space
MQX Address Space
16 16
U-Boot bootloader
Embedded Linux Reference Model
Middleware
RFS (rootfs)
shell
Vybrid
Cortex-A5 USB
Touch
Screen LCD
Linux kernel
Security Surveillance
Camera User Interface, Buttons, Stream sel, etc
SDIO XIP
Button
Network control
interface Auto-launch
Qt Embedded
Driver Driver
networking
Wireless tools
openssh
Driver Driver Shared
Memory Driver
sysfs
Splash
Screen
setup
scripts
Ethernet
Driver Driver
Serial
Sensor Control
LCD calibration
Memory
setup
script MCC
API
1
2
3
MQX image
17 17
Bootloader
MQX Reference Model
Functional
Blocks
Vybrid
USB
Touch
Screen LCD
MQX kernel
Application (s)
SDIO XIP
Button
Network control
interface
Driver Driver
networking
Driver Driver Shared
Memory Driver
File
system
Services
setup
scripts
Ethernet
Driver Driver
Serial
Sensor Control
Memory
setup
script MCC
API
1
2
18 18 Linux and MQX comparison
Linux MQX
Core footprint 3M <100k
Scheduler Priority, preemptive,
latency not guaranteed
Priority, preemptive,
latency guaranteed
Kernel Configurability Modular, fully
customizable
Modular, fully
customizable
APIs 40,000+ Limited
System protection Separate address spaces
Kernel/Application
separated
Single address space; no
protection
Kernel/Application
combined
19 19
Development Ready — When?
Linux kernel
1000s of Linux
packages
Custom software
gstreamer.com
Toolchain
Private plugins
GStreamer Ver. 0.22?
Open Source plugins
Ver.3.10?
Additional drivers?
GCC4.1?
uClibc or glibc?
Debug APIs/libs
optimizations
Embedded? Version?
Which modules?
Need HW acceleration?
Third party
mainline
Embedded Linux Challenges
For 2012, 57% of all projects finished late or were cancelled Source: EE Times Embedded Market Study, 2013
20 20
Vybrid Solution Approach:
LinuxLink + Tower + MQX
Linuxlink cloud development allows customers to rapidly deploy
pre-built or custom BSP/SDKs for Tower or other hardware platforms.
Linuxlink cloud development allows customers to rapidly deploy
pre-built or custom BSP/SDKs for Tower or other hardware platforms.
21 21
RTOS Challenges
Future Proofing new/future protocols (Stacks) • Bluetooth Light
Peripheral Support across product families Prototyping and Evaluation support Creating drivers to use with a scheduler Would like to spend your time developing application
code and not creating or maintaining a scheduling system
Multi-thread support with synchronization Portability of application code to other CPUs Resource handling Adding new features without affecting higher priority
functions Support for upper layer protocols such as:
• TCP/IP, USB, flash systems, web servers
• CAN protocols, embedded GUI, SSL, SNMP Determinism
22 22 MQX Solution
Established community of users and code examples • MQX is a 15-year-old RTOS
Solid RTOS with extensive code examples and community support
Standardized Driver creation
Support across all Freescale MCU Portfolio
Extensive Stack Library
Extensive third-party community development support
End User empowered to create custom BSP
Non Royalty RTOS
Ported Tower Systems for easy system lever evaluation
23 23
Product Requirements — Where
Do I Start My Linux Development?
24 24 Requirements – First Glance
GUI with the following
features:
• Live feed screen
• Recorded feed management screen
• Control panel screen
Capture video feed from USB
connected cameras
• Store on a local file system
Manage recordings
Utilize a video overlay
• See video feed and controls on a
single screen
• Display text on a video feed
Detect movement via light
sensor
Use Multi-core communication
(MCC) layer
• Trigger video recording
Video Surveillance Features:
25 25
U-Boot bootloader
Security Surveillance Device (Blueprint)
Middleware
shell
Vybrid
USB
Touchscreen LCD
Linux kernel
Security Surveillance Device
Video
playback User Interface, Buttons, Stream sel, etc
SDIO NAND
VPU
Network
control
interface Auto-launch
Qt Embedded
Driver Driver
networking
Wireless
tools
openssh
Driver Driver Driver
GStreamer
Splash
Screen
setup
scripts
Ethernet
Driver Driver
Serial
Camera
Capture
LCD calibration
IPU
setup script
Codecs
Shared
Memory
Light
Sensor
Memory
MCC
API
26 26
Exercise 1:
Assemble a Custom BSP & SDK
Using the LinuxLink Web Factory
27 27 LinuxLink Architecture
28 28 Exercise Details - BSP
Identify an initial set of requirements, sufficient to
boot the dev kit and develop and initial application
Build a custom BSP (and a matching SDK)
Start with a small Linux design, and
build it up as per your requirements
29 29 Requirements Helper
Feature What I need Where to select
Linux kernel Latest, 3.x kernel Linux kernel
Cross-Toolchain GCC, Binutils, GDB Toolchains
C Library glibc Toolchains
GStreamer gst-plugins-good Packages
Font freetype Packages
Qt qt-embedded-linux Packages
Networking openssh Packages
System initialization busybox Packages
30 30
Account Login info
We have accounts setup specifically for
Timesys University
• https://linuxlink.timesys.com
• Login with the following credentials:
U: labuserX where X is 1-20
P: lab123
If you’ve purchased a PHYTEC Vybrid kit,
you can register for a LinuxLink PRO account at:
• https://linuxlink.timesys.com/register/phytec/
If you’ve purchased a Freescale TOWER Vybrid kit,
you can register for a LinuxLink PRO account at:
• https://linuxlink.timesys.com/register/freescale/
For other processors and dev kits,
you can register for a LinuxLink FREE account at:
• http://www.timesys.com/register
31 31
Development Environment Setup with an SDK
32 32 Embedded Linux Reference Model
Middleware and Application Services
UI Networking Networking
Security Reliability
LCD HW
accel. SD USB more
Linux kernel Driver Driver Driver Driver more
Application 1 Application 2 Third-Party Application
Application (e.g. Base Station Control) App Features
SOC Board
Target Host
Hundreds of APIs from Open
Source and Proprietary
Reliability
Development Tools • Cross-Development Environment
(SDK)
• IDE
• Application Debugging
• Build System (e.g. Factory)
• RFS, Kernel, App
• Collaborative Development
• Profiling
• Tracing
• System Management
• Target Images
• Binaries Deployment
• Target Management
Ap
plic
ation
S
yste
m
Ha
rdw
are
S
oft
wa
re
Virtualization
• KGDB/ printk Debugging
• JTAG
• SMP support, RT enhancements
33 33 Embedded Linux Reference Model
Middleware and Application Services
UI Networking Networking
Security Reliability
LCD HW
accel. SD USB more
Linux kernel Driver Driver Driver Driver more
Application 1 Application 2 Third-Party Application
Application (e.g. Base Station Control) App Features
SOC Board
Target
Hundreds of APIs from Open
Source and Proprietary
Reliability
Ap
plic
ation
S
yste
m
Ha
rdw
are
S
oft
wa
re
Virtualization
Host
Development Tools
• Cross-Development Environment (SDK)
• IDE
• Application Debugging
• Build System (e.g. Factory)
• RFS, Kernel, App
• Collaborative Development
• Profiling
• Tracing
• System Management
• Target Images
• Binaries Deployment
• Target Management
• KGDB/ printk Debugging
• JTAG
• SMP support, RT enhancements
34 34 Development Environment Setup
Can find instructions
on how to setup the
board
BSP/SDK Summary
Software Development Kit
(SDK)
•Install on your host PC
Desktop Factory tools
•Install on your host PC
35 35
Exercise 2:
Setup Your Application
Development Environment
36 36 Exercise Details - SDK
Setup your own development environment
• Cross-toolchain
• IDE
• Libraries and header files
The goal
• Easily develop your value-add software
+
Answer:
TimeStorm IDE
37 37 Exercise Details
Install an SDK from a build output
Installation command (optional):
• sh <SDK name.sh>
For your convenience the BSP/SDK has been
downloaded for you and preinstalled under:
• /home/tsu-lab/LAB-5/BSP
Look at a directory structure
Verify toolchain with command:
• <fully qualified path to toolchain’s gcc> -###
38 38
Deploying Linux Using NFS
39 39 Is NFS useful?
Run the custom BSP on the target
• U-Boot bootloader
• Linux kernel
• Network mounted RFS
The goal
• Easily modify all aspects of your RFS
• Easily deploy and test your application
40 40 Network File System setup
Host Target
• NFS server
• DHCP server (local subnet)
• TFTP server (/tftpboot)
• Console
• U-Boot
• Console
Host RFS Target RFS Target RFS
• Ethernet
• Serial
Full directions can be found in the Getting Started Guide for the
Phytec Vybrid board. Access it through a LinuxLink dashboard.
42 42 Module 1 — Summary
Learned about the Vybrid LinuxLink – needed
for all exercises
Reflected initial product requirements in
Linux BSP and SDK
Built a custom BSP with LinuxLink FREE Edition
• Experiment with a pre-built starting point
Setup a development environment
• System level development and optimizations
• Development of a value-add software (applications)
Discussed NFS deployment
• Image transfer
• Bootloader configurations
43 43 Module 1 — Quiz
1. What do you need to know before you start your
initial design?
2. How is a Linux BSP different than an MQX RTOS
image?
3. What do you need in order to start developing your C
or C++ application?
44 44 Building a Video Surveillance Device with LinuxLink — Agenda
Module 1: BSP & Dev Environment How to assemble and deploy an initial BSP and setup development
environment with the matching SDK
Module 2: User Interface How to build a state-of-the-art User Interface using
Qt Embedded for Linux
Module 3: RTOS & MCC How to monitor a light sensor in RTOS and trigger video recording
under Linux
Module 4: Optimizations How to measure and optimize to boot more quickly
45 45 Module 2
Modify the underlying Linux image to
support Qt
Qt development tools installation/setup
Hello World application – GUI development
• Create a QMainWindow application
• Use a number of Qt widgets including
• QPushButton
• QLabel
• Layouts and more
• Create a resource file
• Test your application locally
• Cross-compile for the Vybrid Phytec target
• Debug on the target
46 46
U-Boot bootloader
Security Surveillance Device (Blueprint)
Middleware
shell
Vybrid
USB
Touchscr
een LCD
Linux kernel
Security Surveillance Device
Video
playback User Interface, Buttons, Stream sel, etc
SDIO NAND
VPU
Network
control
interface Auto-launch
Qt Embedded
Driver Driver
networking
Wireless
tools
openssh
Driver Driver Driver
GStreamer
Splash
Screen
setup
scripts
Ethernet
Driver Driver
Serial
Camera
Capture
LCD calibration
IPU
setup script
Freescale Codecs
Shared
Memory
Light
Sensor
Memory
MCC
API
47 47
Adding APIs/Libraries Using the
LinuxLink PRO Desktop Interface
48 48 Key Features
Advanced Customization (all aspects of your
Linux platform)
‘Factory’ – Only easy to use, innovative Build System
with:
• Advice engine — recommends software based on your hardware
and application requirements
• Update engine — automatic updates for your custom configuration
‘Build up’ vs. strip down
to small distribution
Build Repeatability
Easy Extensibility
It is easy to prototype and
to develop
Desktop Factory
50
©2013 Timesys Corp.
Factory Architecture
The Desktop Factory is the Timesys build engine
with the menuconfig UI.
The UI creates/edits the Factory “work-order” in a
.config file.
The Factory fetches source code for components
required by the work order, and follows recipes to
cross build those components.
Each component has its own recipe, embody in a
pair of files which drive the UI and the “make” build.
51
©2013 Timesys Corp.
Exercise 3: Basic UI
Basic navigation, Searching, Help, String entry
Source code locations
Running parallel make jobs
Package options
Templates
A peek at the .config file
52
©2013 Timesys Corp.
Factory: High-Level Flow
1.All sources, patches, and other files (e.g., startup
scripts) are fetched from the designated source.
2.The host tools are built including ccache, and the
gcc cross compiler suite (glibc or uclibc, binutils,
etc.). Any additional host utilities are built as well.
3.All target software is built. The kernel image
and RFS image are created here as well.
4.As requested (e.g., RPM, deb), binaries are
packaged.
5.Installers (e.g. SDK) are created.
1. Fetch Sources, Patches, Additional
files
1. Fetch Sources, Patches, Additional
files
2. Build Host Tools and Libraries
(toolchain)
2. Build Host Tools and Libraries
(toolchain)
3. Build Target Binaries (Root File
System)
3. Build Target Binaries (Root File
System)
4. Package binaries 4. Package binaries
5. Create Installers 5. Create Installers
At a high level, the Factory flows through these five stages.
However, due to dependencies that require components to be built
“out of order,” the Factory operates in a recursive manner.
53
©2013 Timesys Corp.
Factory: Directory Tree
./bin/ - contains scripts useful for extending Factory.
./doc/- contains license and version information.
./include/ - contains common recipes along with the KConfig library.
./src /– contains the downloaded sources.
./target/ - contains sub-directories containing recipes for the kernel, boot-loaders,
and RFS.
./toolchain/ - contains the recipes for each toolchain component as well as the
host utilities that are built with the toolchain.
./build/ – the ouput directory tree.
Makefile includes all other make rule files within the Factory tree
Config.in - includes all other Config.in files and sets global and default options.
.config - the generated work order used by the make rules.
bin doc include src target toolchain
Makefile
factory
build
Config.in
.config
54
©2013 Timesys Corp.
Factory: Build Output Tree
NOTE: “{ }” in names imply architecture or other build specific attribute.
toolchain-initial-{ }.tgz - This is a bare toolchain (cross compiler, etc.)
toolchain-final-{ }.tgz - Bare toolchain plus all selected package libraries.
{ }.sh - SDK installer
./images/ - contains kernel, bootloader, and RFS images
./packages/ – contains binary packages in RPM, DEB, IPKG, or TGZ.
./<package name>/ - each package has its own working directory.
./toolchain/ – toolchain working directory
./rfs/ - RFS working directory. This can be modified by hand and then the RFS
images can be recreated with the 'rfs-images' target. NOTE: this directory
cannot be directly NFS exported as it does not have valid device files (/dev).
The device files are populated during the RFS image creation process.
images packages rfs toolchain toolchain-
initial{ }tgz
Build-{ }
toolchain-
final-{ }tgz
{ }.sh
<pkg name>
55
©2013 Timesys Corp.
make Targets
Make Targets Functionality
checksystem, checktool-oprofile,
checktool-ltp, checktool-mpatrol,
checktool-gdb
System requirements and
compatibility checks.
advice, showpackages, showoutput Information commands
checkupdates, update, update-
noprompt, oldconfig
Updating
Clean, download-clean, distclean, rfs-
distclean, packages-distclean,
software-distclean, reset-toolchain
Clean commands
-clean, -distclean, -fetch, -unpack, -
patch, -configure, -build, -package, -
rfs-install, -host-install, -show
Per-application targets (all preceded
by application name without version
e.g. “busybox-clean)
menuconfig, config “workorder” (.config) editors
56 56
Exercise 4:
Enhance Your Design Locally
by Adding Required Packages
57 57 Exercise Details
Find Desktop Factory under:
• /home/tsu-lab/LAB-5/factory/factory-<string>
Look at the help screen by running make help
Bring up a configuration screen
• make menuconfig
Locate and select missing packages
58 58 Requirements Helper
Feature What I need Where to select
Linux kernel Latest, 3.x kernel Linux kernel
Cross-Toolchain GCC, Binutils, GDB Toolchains
C Library glibc Toolchains
GStreamer gstreamer Packages
gst-plugins-good Packages
MCC mcc-kmod, libmcc Packages
Video 4 Linux v4l-utils Packages
Font freefont Packages
Text display pango Packages
Qt qt-embedded-linux Packages
Networking openssh Packages
System initialization busybox Packages
59 59
Qt Development Tools
Installation/Setup
60 60 Qt Development Tools
Options
• DS5 with TimeStorm & Qt plugins http://linuxlink.timesys.com
– Qt Creator http://www.qt.nokia.com
Installation
• DS5 can be downloaded from ARM
• Download TimeStorm and Qt plugins from LinuxLink
• Preinstalled in your UBUNTU
Run
• Desktop icon for ARM DS5
61 61
Host PC/Linux
IDE and Cross-Environment
IDE
LinuxLink
Desktop Interface
Recognizes
Automatically
63 63
Security Surveillance Device –
GUI Development
64 64 The Qt Story
The Qt toolkit is a multi-platform C++ GUI toolkit
(class library)
Developed by Troll Tech AS, later acquired by Nokia
Qt can be used under one of the following licenses:
• LGPL v.2.1
• GPL v.3.0
• Commercial (mostly if you plan to extend Qt and sell it)
Around 1997, Qt was chosen as the code basis for the
KDE Linux desktop environment
Latest version — Qt 5.0 is available for Windows, Unix,
Linux, Embedded Linux, Mac OS X libraries and
Symbian OS
Recent additions include:
• Qt Quick (programming with QML)
• Qt Mobility
• Multi-touch and gestures support
66 66 Qt/Embedded (cont’d)
Modular approach
Each additional
functionality can be
enabled by specifying
module to be added
• Core classes provide the
baseline
• GUI building tools
• Can run with or without X
• Can be easily ported from other OSs
67 67 Qt — Flexible and Powerful UI API
Used in many applications in various markets
WYSIWYG approach
Can take advantage of Freescale hardware:
• OpenGL ES API
• Multiple frambuffers
68 68 Widgets
Base class for all UI widgets
• QPushButton, QLineEdit, QTabView, …
Properties
• width, height, backgroundColor, font, mouseTracking,
backgroundPixmap, etc.
Slots
• repaint, show, hide, move, setGeometry, setMainWidget, etc.
Signals
• mouseMoveEvent, keyPressEvent, resizeEvent, paintEvent,
enterEvent, leaveEvent, etc.
69 69 Event Handling
Qt approach to IPC: signals and slots
• A widget sends out various signals
• Object methods can be declared as slots
• Compatible signals and slots can be connected or plugged together (parameter types must match!)
Clear UI/Logic separation
• This separation between UI components and program elements
lends itself to component-based programming
70 70 Signals and Slots
http://doc.qt.nokia.com/latest/signalsandslots.html
71 71 Signals and Slots (cont.)
#include <QObject>
class Counter : public QObject {
Q_OBJECT
public:
Counter() { m_value = 0; }
int value() const { return m_value; }
public slots:
void setValue(int value);
signals:
void valueChanged(int newValue);
private: int m_value;
};
72 72 Events – Triggered by Signals
Signals: emit events
• declare as signals
• You don't implement them, you send them with the keyword “emit”
– e.g. emit(counterAtZero(100))
Slots: receive and handle events
• Normal member functions declared as slots
Connect: must connect signals to slots
• QObject::connect( Object 1, SIGNAL(activated(int)), Object 2,
SLOT(handlingMethod(int)) );
73 73 qmake
The qmake utility is typically invoked with the
following commands:
• qmake – project
• qmake
IDE tools invoke qmake automatically
• You can invoke it as well (Project->qmake)
Rules:
• Be sure to place code in its own directory.
• qmake scans all subdirectories for dependencies.
• Looks at the .pro file and generates Makefile structure
75 75 Qt Application Structure
.pro file
• Holds information on all files that will be used to build a predefined
executable(s)
• Used by qmake to generate Makefiles for your project
.ui file
• User Interface file
• Can open in a designer view
• Has canvas on which you can drag and drop widgets
• Its XML format is translated into C file (moc_) at compile time
.qrc file
• Holds information on all resources (non C++ code) used
• Referenced throughout the project e.g. QPushButton icons
.qss file
• Used to define a style sheet for a custom look and feel
• Loaded at runtime by your application
76 76
Exercise 5:
Create a GUI application using
ARM DS5 with TimeStorm & Qt plugins
77 77 Video Surveillance Application
Start DS5
• Desktop icon
Open a Workspace
• /home/tsu-lab/DS5-workspace-clean
Create a new Qt GUI Project
• UI Type – QMainWindow
• Don’t need to select any additional Qt frameworks (initially)
Create the following components
• Remove menu bar and status bar
• Use layouts
• Use a QTabView
• Preset the size of all UIs to selected LCD size (will help you see actual
design)
• Design UIs on all three screens
Example can be found
in Exercise 5 folder
in downloads.
78 78
Exercise 6:
Develop Logic Behind the GUI
79 79 Exercise Details: Video Surveillance
Qt Application
Define/Connect signals (Hide Button)
• Add code for Hide Button
• Capture the signal in a service class
• Implement the service routine
– Set alpha blending to 0
Switch the building layout graphic
• Utilize the border-image property of widgets’ style sheet
Compile and run your application on your host
Cross compile and debug your application on the
target
80 80 If we have time …
Compile the medical application example
Look under Exercise 6 solutions for full
implementation
Cross compile by switching Qt type in DS5 to cross
The design also includes the following:
• Custom Style Sheet
81 81 Module 2 — Summary
Used LinuxLink PRO Edition Desktop Factory
interface
• Added needed packages and rebuilt the runtime BSP and
SDK images
Used an IDE to design/develop Qt based UI
Developed a GUI with the following:
• Layers, several Widgets and Labels
• Implemented signals and handlers
Cross-compiled for Phytec’s Vybrid target
82 82 Module 2 — Quiz
1. Name at least one graphics package (other than Qt).
2. What are licensing implications for your proprietary
application software linking against LPGL version of
Qt?
83 83 Building a Video Surveillance Device with LinuxLink — Agenda
Module 1: BSP & Dev Environment How to assemble and deploy an initial BSP and setup development
environment with the matching SDK
Module 2: UI & Video Feeds How to build a state-of-the-art User Interface using
Qt Embedded for Linux
Module 3: RTOS & MCC How to monitor a light sensor in RTOS and trigger video recording
under Linux
Module 4: Optimizations How to measure and optimize to boot more quickly
84 84 Module 3
MQX overview
• Pingpong example
Multi-Core Communication (MCC) Layer
• Purpose
• Setup
• Add MCC communication in Security Surveillance Application
Add appropriate code to the UI application
85 85
U-Boot bootloader
Security Surveillance Device (Blueprint)
Middleware
shell
Vybrid
USB
Touchscr
een LCD
Linux kernel
Security Surveillance Device
Video
playback User Interface, Buttons, Stream sel, etc
SDIO NAND
Framebuffer
Network
control
interface Auto-launch
Qt Embedded
Driver Driver
networking
Wireless
tools
openssh
Driver Driver Driver
GStreamer
Splash
Screen
setup
scripts
Ethernet
Driver Driver
Serial
Camera
Capture
LCD calibration
IPU
setup script
Codecs
Shared
Memory
Light
Sensor
Memory
MCC
API
86 86
MQX & MCC Overview
87 87
Bootloader
MQX Reference Model
Functional
Blocks
Vybrid
USB
Touch
Screen LCD
MQX kernel
Application (s)
SDIO XIP
Button
Network control
interface
Driver Driver
networking
Driver Driver Shared
Memory Driver
File
system
Services
setup
scripts
Ethernet
Driver Driver
Serial
Sensor Control
Memory
setup
script MCC
API
1
2
88
©2013 Timesys Corp.
OS Comparison (for the purpose of training)
MQX • O/S components are libraries
linked with your application to
create a single bare-metal
execution image.
• Source distributed as DS-5
(Eclipse) projects in a single
source tree.
• Built with ARM bare-metal
compiler.
(advanced) • A single address space – no
memory protection, mapping,
or transformations between
hardware I/O, O/S components,
application.
• Programs are tasks each in their
own thread.
Linux
• Kernel is a bare-metal executable,
but isn’t enough.
• Your application (except possibly device
drivers) is built separately and runs
(launched) from a file system.
• Therefore a root file system (RFS) is a
required component of Linux.
• Timesys Factory (on web or desktop) builds
the RFS. It will build the Kernel and your
Application, but they can be also built
“outside” as well.
• Built with (Timesys) gcc compiler.
(advanced)
• Linux utilizes the memory management unit
(MMU) of the A5 core. All processes are
hardware protected from stepping on each
other’s memory.
• Programs run as a process – which can have
many threads.
89
©2013 Timesys Corp.
Overview of MCC
Multi-Core Communication
Communication layer between MQX and Linux
Lightweight, fast • API calls are simple send / receive between endpoints
• Endpoints are [core, node, port] triplets – Sort of IP host, address, port – but not really.
– Core is fixed (A5=0, M4=1]
– Linux can have multiple nodes, MQX only 1
– Each can have multiple endpoints, arbitrary numbering
• Uses shared SRAM and interrupts
• Received data can be passed by pointer or copied
• Variable timeouts
Configurable:
• Buffer sizes
• # of buffers
• Max # of endpoints
90
©2013 Timesys Corp.
MCC Architecture
A5 - Linux M4 - MQX
API shared library
Kernel Module Driver
User App (tasks)
API library
Shared SRAM
Endpoint 0,0,1
User App process(es)
Endpoint 0,2,3 Endpoint 1,0,1 Endpoint 1,0,2
Free buffer pool
Interrupts
Hardware
Semaphore
91
©2013 Timesys Corp.
Simple Example
A5 - Linux
1. “I will receive on endpoint [0,0,1]”
2. mcc_initialize(0)
3. mcc_create_endpoint([0,0,1], 1)
4. mcc_send([1,0,2], “hello”, 5, 5000) // <= 5 secs for
buffer
5. mcc_recv_copy([0,0,1], &buf, sizeof(buf), length,
0xffffffff)
M4- MQX
1. “I will receive on endpoint [1,0,2]”
2. mcc_initialize(0)
3. mcc_create_endpoint([1,0,2], 2)
4. mcc_recv_nocopy([1,0,2], &buf_p, length, 0xffffffff)
5. mcc_send([0,0,1], “hello”, 5, 5000) // <= 5 secs for
buffer
92
©2013 Timesys Corp.
Resource Sharing
X
Y
Z
Photosensor
MCC
A5 – Linux
• boa webserver
• Sensor demo
• Security
Surveillance App
• theater-vybrid
M4 – MQX
• Pingpong /
Photosensor
example
Vybrid SOC
Phytec-LCD
Touchscreen
I2C Bus
93
©2013 Timesys Corp.
Protecting I2C Sequences
MQX Application 1. Open the I2C driver
a) Disable and clear I2C
b) Clear out all I2C
events
c) Set the station
address
d) Set the frequency
e) Enable I2C
2. Set master mode
3. Set the device
address
4. Do read/write
sequence
5. Send out stop
6. Close the driver.
Must be protected in
application.
Linux Driver
1. Enable I2C controller
2. Set master mode,
Interrupts
3. Set the device address
4. Wait for interrupt
5. Do read/write sequence
including wait for
interrupt
6. Send out stop
7. Disable I2C controller
Protected in driver,
transparent to application
Hardware
semaphore
protection.
94 94
A5/M4 Boot Process
U-Boot
• Reset, copy U-Boot to SDRAM and jump to start address
• Basic System Init. (IMPORTANT)
• Copy the Linux kernel Image to SDRAM from SD
• Decompress the kernel if needed
• Jump to upload address and start the kernel
Kernel
• Run Kernel Init code
• Init kernel subsystems (device drivers)
• Init SD card
• Mount SD card partition with RFS
• Execute init script
MQX
• Load to specific address in memory with mqxboot utility
Application
• Depends on your specific requirements
Power On
U-boot Hardware Init
U-boot Copies kernel to
memory/uncompress
Kernel Boot Hardware/Subsystems
initialization
RFS Fetch/Mount/Boot into
User Application
Boot Process Sequence T
IME
MQX Load to addr & run
95 95
Exercise 7:
MQX & MCC
96 96 Exercise Details: MQX & MCC Components
Open DS5-Workspace-clean-preset 1. ~/MQX/FSLMQXOS_4_0_1_beta1-linux/mqx/build/ds5/bsp_twrvf65gs10_m4
2. ~/MQX/FSLMQXOS_4_0_1_beta1-linux/mqx/build/ds5/psp_twrvf65gs10_m4
3. ~/MQX/FSLMQXOS_4_0_1_beta1-linux/mcc/build/ds5/mcc_twrvf65gs10_m4
4. ~/MQX/FSLMQXOS_4_0_1_beta1-linux/mcc/examples/pingpong
Import the Linux side of the light sensor application
• Sensor demo or
• MCCDemo (already in the project)
Start pingpong/MCC/MQX component on M4 core
• run startmqx script on the target
• location: /home/ Note: Serial Console
will behave strange as
it is now shared
between M4 and A5
97 97 Integrate Application Components
Added a QThread based class
• Can handle independently MCC traffic
Leverage MCCDemo application
Create a custom Signal-Slot pair to trigger camera
feed when desired light threshold is reached
• Leverage start_camera() method and onStopButton() slot
Compile and deploy on Vybrid
• Place a breakpoint at start_camera method and debug on target
102 102 Module 3 — Quiz
1. Name one major difference between Linux and an
RTOS
2. What are MCC endpoints?
103 103 Building a Video Surveillance Device with LinuxLink — Agenda
Module 1: BSP & Dev Environment How to assemble and deploy an initial BSP and setup development
environment with the matching SDK
Module 2: UI & Video Feeds How to build a state-of-the-art User Interface using
Qt Embedded for Linux
Module 3: RTOS & MCC How to monitor a light sensor in RTOS and trigger video recording
under Linux
Module 4: Optimizations How to measure and optimize to boot more quickly
104 104 Module 4
How to measure boot time in our dev environment
Fast boot optimizations
• Boot process overview
• Bootloader optimizations
• Kernel level optimizations
• Filesystem optimizations
• Other options
105 105
Boot time Measurement
106 106 Measuring the boot time
Each embedded design is different
Instrumentation is available
• Helps measure boot time at all stages
• Most are open source based
• Can be easily removed when you are done
The trick behind boot time optimization is to know where your
system is spending time.
You may end up losing a lot of time with little results!
107 107 Available Instrumentation
Bootloader
• Logic analyzer (using GPIO pins)
Kernel Measurement
• Printk
– Compile kernel with: CONFIG_PRINTK_TIMES=y
– Switch on dynamically: “echo Y >/sys/module/printk/parameters/time”
• initcall_debug
– On boot command line, use: initcall_debug=1
User space measurement
• Bootchart (www.bootchart.org)
• Strace
– Run strace –cT 2>/tmp/strace.log bluetoothd
• Linux Trace Toolkit
System
• uptime
– Add (echo -n "uptime:" ; cat /proc/uptime) to an RC script
108 108
Fast boot Optimizations
109 109 Bootloader Optimizations
Low hanging fruit
• Set the bootdelay variable to 0 (time savings 4s)
• Preset the bootcmd; do not use setenv (time savings 0.5s)
• Disable console (time savings 2s)
– CFG_CONSOLE_INFO_QUIET
– CONFIG_SILENT_CONSOLE
– In our case- silent=yes
• Disable other tests (time savings 2-6s)
Additional modification/enhancements
• If possible, use uncompressed Linux kernel
• Optimize the NAND read operation, to shorten image copy time
• Rewrite/disable CRC32 checksum code
• Load the image directly to Entry point
– Set CONFIG_LOADADDR
• If NOR Flash is used, leverage XIP
• For large kernel image, use different compression algorithms
Boot Time <1s 75s
110 110 Linux Kernel Optimizations
Low hanging fruit (time savings: 2-15+s)
• Use uncompressed kernel
– Uncompressing takes time
• Remove unused kernel options
– Not used networking i.e. IPV6, multiple file systems
– Debug features and symbols (for final deployment)
• Build not initially used device drivers as Loadable Kernel Modules
– Keep the features needed at boot time built into the kernel
– Remaining drivers built as LKMs will make kernel smaller
• Consider various approaches for your RFS deployment
– JFFS2 with appended journal summary (skip flash scan)
– CRAMFS, UBIFS
• Suppress the console output
– Use “quiet” with your kernel command line
Boot Time <1s 75s
111 111 Making the Linux Kernel Small and Fast
Linux kernel options we will look at today
Kernel Option Comment
CONFIG_EMBEDDED Disables or tweaks a number of kernel options and
settings. Think uClinux
CONFIG_IKCONFIG Saves complete kernel configuration in the kernel
CONFIG_KALLSYMS Prints our symbolic crash information and
backtraces
CONFIG_BUG Disables BUG and WARN functions
CONFIG_HOTPLUG Can be disabled if no external devices will be
attached and if you use static device files
CONFIG_DNOTIFY File change notification to user space
CONFIG_EXT2 Disable if using jffs2 file system
CONFIG_PRINTK Makes kernel silent when disabled
CONFIG_PRINTK_TIME A way to track where time is spent at boot time
CONFIG_CC_OPTIMIZE_FOR_SIZE Will select –Os instead of –O2 resulting in a smaller
kernel
112 112 Userspace Optimizations
Filesystem modifications
• Optimize the RC Scripts (remove the ones you don’t use)
• If possible replace RC scripts with a single init
• Pre-link
• Link applications statically
• Avoid udev – create needed device nodes ahead of time
• Use static IP address
Filesystem modifications
• Staged booting
• Code level modifications
• Add a splash screen
Boot Time <1s 75s
113 113 Module 4 — Quiz
1. Why is fast boot useful?
2. What knowledge is required to get under 10s?
3. Can one get to sub 2s boot time?
114 114 Final Q&A and Training Surveys
Thanks for attending!
Please leave completed surveys at your stations
Our contact info:
Al Feczko Aparna Nigam (412) 325-6390 (o) (412) 325-6333 (o) (412) 897-2416 (m) [email protected] [email protected] @afeczko_ts
Top Related