CodeGuru
Earthweb Search
Forums Wireless Jars Gamelan Developer.com
CodeGuru Navigation
Member Sign In
User ID:
Password:
Remember Me:
Forgot Password?
Not a member?
Click here for more information and to register.

jobs.internet.com

internet.commerce
Partners & Affiliates
Logo Design Custom
Prepaid Phone Card
Online Shopping
Cell Phones
Hurricane Shutters
Corporate Awards
Promotional Golf
Holiday Gift Ideas
Promotional Gifts
Best Price
Dental Insurance
Promos and Premiums
Web Hosting Directory
Laptop Batteries


RSS Feeds

RSSAll

RSSVC++/C++

RSS.NET/C#

RSSVB

See more EarthWeb Network feeds

Home >> Visual C++ / C++ >> C++ >> Algorithms & Formulas >> Compression/Decompression

Project Management Guide: Developing a Web Site. Best Practices, Tips and Strategies. Download Exclusive eBook Now.

Five Cents on Arithmetic Encoding
Rating:

Aliaksei Sanko (view profile)
October 28, 2005

Environment:  C++

While employing the classical arithmetic compression implementation [1], I found it possible to make some optimizations that significantly increase compression speed, especially on 32+-bit platforms. These optimizations do not alter the original algorithm, are absolutely compatible, and I have not found them mentioned so far. Considerations and changes proposed below are based on the original implementation [1] and hence some acquaintance with it is assumed.


(continued)



 Silverlight 2 SDK for Visual Studio 2008
This package is an add-on to the RTM release of Visual Studio 2008 to provide tooling for Microsoft Silverlight 2 Beta 1. It provides a Silverlight project system for developing Silverlight applications using C# or Visual Basic. »
 
 Article: What Does it Take to Build the Best RIA?
With the proliferation of Rich Interactive Application (RIA) platform choices out there, you no longer have to take a one-size-fits-all approach to developing your next RIA application. Knowing the strengths (and weaknesses) of each platform can help you to decide the best RIA for your next application. »
 
 Expression Blend 2.5 Preview
Use Expression Blend 2.5 to create and modify managed Silverlight 2-based applications. Expression Blend for Silverlight 2 includes all of the features in Expression Blend 2 but has not reached the quality level of Expression Blend 2 for WPF or Silverlight 1 development. »
 
 The Hottest Mobile Platform Meets the Hottest RIA Platform
With the Symbian OS now supporting Microsoft Silverlight, mobile developers can bring new and exciting capabilities to handsets all over the globe. Find out why developers now need to make mobile devices a core part of their RIA development strategy. »
 
 Article: Leveraging Your Flash Development with Silverlight
You're not giving up Flash any time soon (and we don't blame you.) But if you could get your Flash application working in Silverlight, why wouldn't you? We show you the tools and techniques required to have your rockin' Flash application rolled for Silverlight. »
 

It can be seen that, while looping inside encode_symbol() and decode_symbol(), as soon as the conditions high<Half and low>=Half are not met, they won't be met again. Moreover, a "bits plus follow" call is needed only once at the beginning; then "output bit" is enough. Here is the encode method structure changed:

do
{
   if (high<Half)
   {
      bit plus follow 0;
   }
   else if (low>=Half)
   {
      bit plus follow 1;

      low  -= Half;
      high -= Half;
   }
   else
      break;

   for (;;)
   {
      low  = 2*low;
      high = 2*high+1;

      if (high<Half)
      {
         output bit 0;
      }
      else if (low>=Half)
      {
         output bit 1;

         low  -= Half;
         high -= Half;
      }
      else
         break;
   }
}
while (false);

while (low>=First_qtr && high<Third_qtr)
{
   bits_to_follow++;
   low  = 2 * (low - First_qtr);    // Subtract offset to middle
   high = 2 * (high - First_qtr) + 1;
}

Because only 16 bits of low and high variables are used, these variables are substituted by the combined low_high one.

low_high = low + ((~high) << 16);

Variable high binary negation makes the next optimizations possible: As soon as ~X is equal to -X - 1, ~((~X) << 1) stands for 2 * X + 1. Then,

low  = 2*low;
high = 2*high+1;

can be substituted by just:

low_high <<= 1;

The cycle with checking

while (low>=First_qtr && high<Third_qtr)

can be reduced to (taking into account that conditions high<Half and low>=Half are met):

//while (low>=First_qtr && high<Third_qtr)
unsigned long buf = ~low_high;
if (!(buf & 0x40004000))
{
   do
   {
      bits_to_follow++;
      //low = 2 * (low - First_qtr);    // Subtract offset to
                                        // middle
      //high = 2 * (high - First_qtr) + 1;
      buf = (buf << 1) + 1 + (First_qtr << 1) -
            (First_qtr << 17);
   }
   while (!(buf & 0x40004000));
   low_high = ~buf;
}

Flushing bits to follow can be optimized as well. The flush_bits_to_follow helper is introduced; it simply puts the required amount of one or zero bits to the output:

void bit_plus_follow_0()
{
   output_bit_0();    /* Output the bit.          */
   while (bits_to_follow>0) {
      buffer >>= 1; buffer |= 0x80;
                      /* Put bit in top of buffer.*/
      if (--bits_to_go==0) {
                       /* Output buffer if it is   */
         flush_bits_to_follow(255);
         return;
      }
      bits_to_follow--;
                      /* opposite bits. Set       */
   }                  /* bits_to_follow to zero.  */
}

void output_bit_0()
{
   buffer >>= 1;    //if (bit) buffer |= 0x80;
      /* Put bit in top of buffer.*/
   if (--bits_to_go==0) {
      /* Output buffer if it is   */
      PutByte_(buffer);
      /* now full.                */
      bits_to_go = 8;
   }
}

void flush_bits_to_follow(int buffer_)
{
   for (;;)
   {
      PutByte_(buffer);    /* now full.                */
      buffer = buffer_;
      if (bits_to_follow <= 8)
         break;
      bits_to_follow -= 8;
   }
   bits_to_go = 9 - bits_to_follow;
   bits_to_follow = 0;
}

However, there is a drawback concerned with code value buffer capacity limitation. It can be fixed by changing the way conditions are checked. For maximal code value buffer size (32 bits for IA32 in this example), the algorithm encoding loop body fragment,

for (;;)
{
   if (high < Half)
   {
      bit plus follow 0;
   }
   else if (low >= Half)
   {
      bit plus follow 1;

      low  -= Half;
      high -= Half;
   }
   else
      break;

   low  = 2*low;
   high = 2*high+1;
}

with introduction of the special flags,

long half_flags = ~(low | not_high);

can be changed to:

int shift = num_clear_bits(half_flags);
if (shift != 0)
{
   bit_plus_follow(low & Half);
   // This loop can be definitely optimized
   unsigned long flag = Half;
   for (int i = shift; --i > 0;)
   {
      flag >>= 1;
      output_bit(low & flag);
   }
   low <<= shift;
   not_high <<= shift;
}

where num_clear_bits uses an algorithm similar to one of calculating an integer's logarithm [2]:

const char Table256[] = {
   8, 7, 6, 6, 5, 5, 5, 5, 4, 4, 4, 4, 4, 4, 4, 4,
   3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,
   2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
   2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
   1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
   1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
   1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
   1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
   0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
   0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
   0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
   0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
   0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
   0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
   0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
   0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
};

inline int num_clear_bits(unsigned long v)
{
   unsigned int t, tt;    // temporaries

   if (tt = v >> 16)
   {
      return (t = v >> 24) ? Table256[t] : 8 + Table256[tt];
   }
   else
   {
      return (t = v >> 8) ? 16 + Table256[t] : 24 + Table256[v];
   }
}

The decoding routine can be modified in a similar way. Optimizing multiple bits input/output can further accelerate it.

The complete modified sources can be downloaded as parts of the encoder/decoder code samples. They are implemented as template classes to ease their use with various data providers and consumers.

References

[1] Witten, I. H., Neal, R. M., and Cleary, J. G. (1987) "Arithmetic coding for data compression", Communications of the ACM, vol. 30, no. 6 (June).

[2] Bit Twiddling Hacks - Find the log base 2 of an integer with a lookup table

Downloads

  • arithmetic_demo.zip - Encoder/decoder code sample (36 k)
  • arithmetic_demo_ex.zip - Encoder/decoder code sample without buffer capacity limitation (43 k)

    Tools:
    Add www.codeguru.com to your favorites
    Add www.codeguru.com to your browser search box
    IE 7 | Firefox 2.0 | Firefox 1.5.x
    Receive news via our XML/RSS feed

    Data Sheet: IBM Information Server Blade
    Best Practices for Developing a Web Site. Checklists, Tips & Strategies. Download Exclusive eBook Now.
    Is it time to make your move to the multi-threaded and parallel processing world? Find out!
    Learn about expanding business opportunities for the reseller channel. Visit IT Channel Planet.
    Flash Demo: Learn how IBM Information Server Blade is easy to manage, highly scalable and efficient.


  • RATE THIS ARTICLE:   Excellent  Very Good  Average  Below Average  Poor  

    (You must be signed in to rank an article. Not a member? Click here to register)

    Latest Comments:
    Witten's code is dead - apolar (03/07/2007)
    its good - ramavathy (09/14/2004)

    View All Comments
    Add a Comment:
    Title:
    Comment:
    Pre-Formatted: Check this if you want the text to display with the formatting as typed (good for source code)



    (You must be signed in to comment on an article. Not a member? Click here to register)


    JupiterOnlineMedia

    internet.comearthweb.comDevx.commediabistro.comGraphics.com

    Search:

    Jupitermedia Corporation has two divisions: Jupiterimages and JupiterOnlineMedia

    Jupitermedia Corporate Info


    Legal Notices, Licensing, Reprints, & Permissions, Privacy Policy.

    Advertise | Newsletters | Tech Jobs | Shopping | E-mail Offers

    Solutions
    Whitepapers and eBooks
    Microsoft Article: Will Hyper-V Make VMware This Decade's Netscape?
    Microsoft Article: 7.0, Microsoft's Lucky Version?
    Microsoft Article: Hyper-V--The Killer Feature in Windows Server 2008
    Avaya Article: How to Feed Data into the Avaya Event Processor
    Microsoft Article: Install What You Need with Windows Server 2008
    HP eBook: Putting the Green into IT
    Whitepaper: HP Integrated Citrix XenServer for HP ProLiant Servers
    Intel Go Parallel Portal: Interview with C++ Guru Herb Sutter, Part 1
    Intel Go Parallel Portal: Interview with C++ Guru Herb Sutter, Part 2--The Future of Concurrency
    Avaya Article: Setting Up a SIP A/S Development Environment
    IBM Article: How Cool Is Your Data Center?
    Microsoft Article: Managing Virtual Machines with Microsoft System Center
    HP eBook: Storage Networking , Part 1
    Microsoft Article: Solving Data Center Complexity with Microsoft System Center Configuration Manager 2007
    MORE WHITEPAPERS, EBOOKS, AND ARTICLES
    Webcasts
    Intel Video: Are Multi-core Processors Here to Stay?
    On-Demand Webcast: Five Virtualization Trends to Watch
    HP Video: Page Cost Calculator
    Intel Video: APIs for Parallel Programming
    HP Webcast: Storage Is Changing Fast - Be Ready or Be Left Behind
    Microsoft Silverlight Video: Creating Fading Controls with Expression Design and Expression Blend 2
    MORE WEBCASTS, PODCASTS, AND VIDEOS
    Downloads and eKits
    Sun Download: Solaris 8 Migration Assistant
    Sybase Download: SQL Anywhere Developer Edition
    Red Gate Download: SQL Backup Pro and free DBA Best Practices eBook
    Red Gate Download: SQL Compare Pro 6
    Iron Speed Designer Application Generator
    MORE DOWNLOADS, EKITS, AND FREE TRIALS
    Tutorials and Demos
    How-to-Article: Preparing for Hyper-Threading Technology and Dual Core Technology
    eTouch PDF: Conquering the Tyranny of E-Mail and Word Processors
    IBM Article: Collaborating in the High-Performance Workplace
    HP Demo: StorageWorks EVA4400
    Intel Featured Algorhythm: Intel Threading Building Blocks--The Pipeline Class
    Microsoft How-to Article: Get Going with Silverlight and Windows Live
    MORE TUTORIALS, DEMOS AND STEP-BY-STEP GUIDES