more FLAC parsing

As noted in my previous post, I was able to make a working FLAC parser, but I was not completely happy with it. I just had a feeling if I kept trying I could get it to work without buffering max_frame_size bytes and use a state variable instead.

Well, after many hours I finally got it working! At least it works for all samples I've tried so far. I still want to do more tests just to be sure though. Here is how it works:

• The FLACParseContext has a 31-byte state buffer and state size.
• I had to modify ff_combine_frame() to pad the parser buffer with 16 bytes instead of 8 bytes to accommodate negative values for start of frame less than -8.
  

1. If state buffer has data, copy data from current buffer to fill state buffer (or as much as possible). Search the first 16 bytes of the state buffer for start of frame and CRC of previous frame. If it passes, return start position within the buffer (will be negative).
   
2. Search the current buffer, up to 16 bytes before the end, for start of frame and CRC of previous frame. If the next frame is found, return the start position with the buffer. If none is found, check last 16 bytes for a frame sync word and if found, store in the state buffer. Return END_NOT_FOUND.
3. There are 2 exceptions to the CRC check. One is if an inline header is found since it will not have a CRC match. The other is if there are 10 CRC mismatches in a single frame. The reason for that is to abort the frame search for damaged files. It is extremely unlikely that a valid frame will have 10 CRC mismatches.
   

passing off work

I've been wanting to take some time to write an MPEG-4 ALS decoder for a while now, but haven't had the motivation. So this year I decided to propose the project for GSoC '09. Hopefully a good student will qualify and want to do the project.

FFmpeg Summer Of Code 2009 - MultimediaWiki

After writing a raw FLAC parser, then looking at the raw ALS format, I've decided to aggresively push for not touching raw .als files with a 10-foot pole. We just need to stick to MP4 first, and other containers like NUT and MKV second.

FLAC parsing

Let me start by saying that I very much dislike the raw FLAC format! After a lot of failed attempts I finally got a parser working. Part of the frustration was due to FLAC, and the other part was the way FFmpeg's parsing works.

First issue: reliable frame detection

• Using only the 16-bit frame header gives many many false positive frame start matches.
• Using the frame header + frame CRC-16 gives a handful of false positives.
• The only reliable way seems to be a full frame header validation including the header CRC-8, along with checking the CRC-16 for the whole frame.

Second issue: using FFmpeg's parser to combine pieces of frames from chunks of data

• Frame headers are variable-sized from 6 bytes to 16 bytes.
• The whole frame can be as small as 11 bytes, and silent frames are often less than 16 bytes.
  
• ff_combine_frame() only keeps up to 8 bytes of state information

So, the way I finally ended up with a working parser was to basically use the same strategy as the buffering that's done currently in the FLAC decoder, but do it with ff_combine_frame().

1. read first header to get starting point for block size/channels/bps
2. estimate maximum frame size
3. buffer enough data for maximum frame
4. detect next header
5. update maximum frame size with header info
   
6. update buffer
   
7. return frame start and size
8. goto 3

There are special cases for when there is no more data to read and for checking up to 11 bytes from the end of the buffer to make sure there isn't a small frame that's not being detected due to the 16-byte state size.