Well, I finally got a working E-AC-3 encoder committed to Libav. The bitstream format does save a few bits here and there, but the overall quality difference is minimal. However, it will be the starting point for adding more E-AC-3 features that will improve quality.
The first feature I completed was support for higher bit rates. This is done in E-AC-3 by using fewer blocks per frame. A normal AC-3 frame has 6 blocks of 256 samples each, but E-AC-3 can reduce that to 1, 2, or 3 blocks. This way a small range can be used for the per-frame bit rate, but it still allow for increasing the per-second bit rate. For example, 5.1-channel E-AC-3 content on HD-DVDs was typically encoded at 1536 kbps using 1 block per frame.
Currently I am working on implementing AHT (adaptive hybrid transform). The AHT process uses a 6-point DCT on each coefficient across the 6 blocks in the frame. It basically uses the normal AC-3 bit allocation process to determine quantization of each DCT-processed "pre-mantissa" but it uses a finer resolution for quantization and different quantization methods. I have the 6-point DCT working and one of the two quantization methods. Now I just need to finish the other quantization method and implement mantissa bit counting and bitstream output.
