Post on 31-Dec-2015
1
Huffman Codes
Drozdek Chapter 11
2
Objectives
You will be able to Construct an optimal variable bit length
code for an alphabet with known probability for each letter occuring in a message.
Huffman Code
Construct a tree for decoding messages encoded in a Huffman code.
Construct a tree for encoding messages encoded in a Huffman code.
3
Huffman Codes
Common character codes such as ASCII and EBCDIC use same size data structure for all characters. Eight bits per character.
Contrast Morse code Uses variable-length sequences.
Variable length codes can produce shorter messages than fixed length codes on average when applied to many messages
with given character probabilities.
4
Variable-Length Codes
Each character in such a codehas a weight (probability) and a length
The expected message length per character is the sum of the products of the code lengths and the probabilties for all the characters
(0.2*2) + (0.1*4) + (0.1*4) + (0.15*3) + (0.45*1) = 2.1
5
Immediate Decodability
When no sequence of bits that represents a character is a prefix of a longer sequence for another character Can be decoded without waiting for
remaining bits. Note how previous scheme is not
immediately decodable. And this one is
6
Immediate Decodability
Codes that are immediatly decodable are called prefix codes.
No valid code symbol is a prefix of another valid code symbol.
Perhaps better called prefix free codes.
7
Optimal Codes
We seek codes that are Immediately decodable. Average message length for a large
number of messages is minimal.
For a set of n characters { C1 .. Cn } with weights { w1 .. wn } We need an algorithm which generates
variable length bit strings representing the characters.
8
Huffman Codes
An optimal code scheme developed by David A. Huffman while a PhD student at MIT.
“A Method for the Construction of Minimum-Redundancy Codes” Proceedings of the I.R.E., Sept. 1952
http://en.wikipedia.org/wiki/David_A._Huffman
http://www.huffmancoding.com/david-huffman/scientific-american
9
Huffman's Algorithm
How to determine an optimal code for a set of N characters given their relative frequencies (or weights).
10
Huffman's Algorithm
Initialize a list of one-node binary trees One node for each character containing
the character and its weight.
While there is more than one tree in the list: Find two trees in the list having minimal weights. Remove those trees from the list and make them
the left and right subtrees of a new node having the sum of their weights as its weight.
Label the arc to the left subtree with 0. Label the arc to the right subtree with 1. Add the new tree to the list.
11
Huffman's Algorithm
The code for character Ci is the bit string along the path from the root to Ci in the final binary tree.
12
Example
Given characters and probabilities:
The end result isCharacter Huffman
Code
A 011
B 000
C 001
D 010
E 1
Note arbitrary choice for sibling of D.
13
Alternate Result
Average message length is the same.
14
Huffman Decoding Algorithm
Given a message as a string of 0's and 1's:Initialize pointer p to the root of Huffman tree.
While end of message string not reached:Let x be the next bit of the message string.
If x is 0 move p to the left childelse move p to the right child
If p points to a leaf Display the character at that leaf. Reset p to the root of the Huffman tree.
15
Huffman Decoding Algorithm
For message string 0001011010 Using Huffman Tree and decoding
algorithm
Click for answerClick for answer
000 1 011 010
B E A D
16
Implementing a Huffman Code Program
Let’s implement a program to build a Huffman code tree. Encode and decode text messages
using the resulting Huffman code.
Limit input to letters and spaces. Convert to letters to lower case.
17
Implementing a Huffman Code Program
In order to create a Huffman code for English text, we need weighting factors for the letters. Frequency tables are readily available.
To simplify testing and debugging, start with the a small example: Just the letters A, B, C, D, and E
18
Getting Started
Create a new empty C++ project in Visual Studio, Huffman_Code or a directory in Unix.
Add a C++ code file main.cpp
19
main.cpp
#include <iostream>
using namespace std;
int main(void)
{
cout << "This is the Huffman Code program" << endl;
cin.get();
cin.get();
return 0;
}
Build and test
20
Program Running
21
Class char_freq
We need a class to hold the elements of a Huffman tree. Data
Character Frequency (Probability of occurance)
Pointers Left child Right child
Add class Char_Freq
22
Char_Freq.h
#pragma once
#include <iostream>
using std::ostream;
class Char_Freq
{
private:
char ch;
double freq;
Char_Freq* left;
Char_Freq* right;
public:
Char_Freq(void);
Char_Freq(char c, double f);
Char_Freq(char c, double f, Char_Freq* Left, Char_Freq* Right);
char Ch() const { return ch;};
double Freq() const { return freq;};
bool operator<(const Char_Freq& rhs) const;
friend ostream& operator<< (ostream& os, const Char_Freq& cf);
};
23
Char_Freq.cpp
#include "Char_Freq.h"
Char_Freq::Char_Freq(void)
{}
Char_Freq::Char_Freq(char c, double f) :
ch(c), freq(f), left(0), right(0)
{}
Char_Freq::Char_Freq(char c, double f, Char_Freq* Left, Char_Freq* Right) :
ch(c), freq(f), left(Left), right(Right)
{}
bool Char_Freq::operator<(const Char_Freq& rhs) const
{
return this->freq < rhs.freq;
}
ostream& operator<< (ostream& os, const Char_Freq& cf)
{
os << cf.ch << " " << cf.freq;
return os;
}
24
The Huffman Tree
Add class Huffman_Tree
Will hold code to build and access the Huffman code for a specific set of characters and frequencies.
25
Starting the Huffman Tree
We will build multiple trees of Char_Freq elements.
Keep the roots in a list. Use Standard Template Library list class.
Initially one tree per character to be coded. Each tree consists of root only.
Method Add() will be used to add char-freq pairs to the list
26
Huffman_Tree.h
#pragma once
#include <list>
#include "Char_Freq.h"
class Huffman_Tree
{
public:
Huffman_Tree(void);
~Huffman_Tree(void) {};
// Add a single node tree to the list.
void Add(char c, double frequency);
void Display_List(void);
private:
std::list<Char_Freq> node_list;
};
27
Huffman_Tree.cpp
#include <iostream>
#include <string>
#include "Huffman_Tree.h"
using namespace std;
Huffman_Tree::Huffman_Tree(void)
{}
void Huffman_Tree::Add(char c, double frequency)
{
Char_Freq cf(c, frequency);
node_list.push_back(cf);
}
28
Huffman_Tree.cpp
void Huffman_Tree::Display_List(void)
{
cout << "Character frequency list:" << endl;
list<Char_Freq>::iterator itr;
for (itr=node_list.begin(); itr!=node_list.end(); ++itr)
{
cout << *itr << endl;
}
}
29
main.cpp#include <iostream>
#include <string>
#include "Huffman_Tree.h"
using namespace std;
Huffman_Tree huffman_tree;
int main(void)
{
cout << "This is the Huffman code program.\n\n";
huffman_tree.Add('a', 0.2 );
huffman_tree.Add('b', 0.1 );
huffman_tree.Add('c', 0.1 );
huffman_tree.Add('d', 0.15);
huffman_tree.Add('e', 0.45);
huffman_tree.Display_List();
cin.get();
cin.get();
return 0;
}
30
Program in Action
31
Implementing Huffman’s Algorithm
Huffman’s algorithm requires us to identify two trees with minimal total frequency.
To do this we can sort the list. The < operator for the char_freq class
compares the frequency values. So the sort method of the list template
class will sort the trees into increasing order by frequency.
32
Implementing Huffman’s Algorithm
Add function Make_Decode_Tree to class Huffman_Tree.
Repeatedly Sort the list of trees by frequency Remove the first two trees Create a new node with these trees as subtrees.
Frequency is sum of their frequencies Add the new node to the list.
Continue until there is only one node on the list.
33
Huffman_Tree.h
Add new public method: void Make_Decode_Tree(void);
34
Huffman_Tree.cpp
Start by sorting the list. Display the sorted list.
void Huffman_Tree::Make_Decode_Tree(void)
{
node_list.sort();
cout << "\nSorted list:\n";
Display_List();
}
35
main.cpp
Add call to make_decode_tree.
int main(void)
{
cout << "This is the Huffman code program.\n";
huffman_tree.Add('a', 0.2 );
huffman_tree.Add('b', 0.1 );
huffman_tree.Add('c', 0.1 );
huffman_tree.Add('d', 0.15);
huffman_tree.Add('e', 0.45);
huffman_tree.Display_List();
huffman_tree.Make_Decode_Tree();
cin.get();
cin.get();
return 0;
}
36
Program in Action
37
Huffman_Tree.cpp
Add to function Make_Decode_Tree()
while (node_list.size() > 1)
{
Char_Freq* cf1 = new Char_Freq(node_list.front());
node_list.pop_front();
Char_Freq* cf2 = new Char_Freq(node_list.front());
node_list.pop_front();
Char_Freq cf3(0, cf1->Freq()+cf2->Freq(), cf1, cf2);
node_list.push_back(cf3);
node_list.sort();
}
This is the essence of Huffman’s algorithm!
38
Huffman_Tree.h
Add a new private member variable to class Huffman_Tree to hold the root of the tree.
private:
std::list<Char_Freq> node_list;
Char_Freq decode_tree_root;
};
39
Huffman_Tree.cpp
In order to check our results we need to be able to display the tree. Also show the code as a list.
Add public functions to Huffman_Tree.h:
void Display_Decode_Tree(Char_Freq* cf, int indent) const;
void Display_Code(Char_Freq* cf, std::string prefix) const;
Add at top of Huffman_Tree.cpp:#include <iomanip>
40
Display_Decode_Tree()
void Huffman_Tree::Display_Decode_Tree(Char_Freq* cf,
int indent) const
{
if (cf->left != 0)
{
Display_Decode_Tree(cf->left, indent + 8);
}
cout << setw(indent) << " " << *cf << endl;
if (cf->right != 0)
{
Display_Decode_Tree(cf->right, indent + 8);
}
}
Note access of private members of cf. Make class Huffman_Tree a friend of class Char_Freq.
41
Char_Freq.h
Add at the end of Char_Freq.h:
bool operator<(const Char_Freq& rhs) const;
friend ostream& operator<< (ostream& os, const Char_Freq& cf);
friend class Huffman_Tree;
};
42
char_freq.cpp
Update << to handle merged nodes ch will be 0
ostream& operator<< (ostream& os, const Char_Freq& cf)
{
if (cf.ch > 0)
{
os << cf.ch << " " << cf.freq;
}
else
{
os << '*' << " " << cf.freq;
}
return os;
}
43
Huffman_Tree.cpp
Add at the end of function Make_Decode_Tree()
decode_tree_root = node_list.front();
cout << endl << "The Huffman Tree" << endl;
Display_Decode_Tree(&decode_tree_root, 0);
44
Program in Action