Way of the exploding head

QPSK abandoned for BPSK

The noise performance of the QPSK decoder wasn’t what I was hoping after several iterations. I pretty quickly threw out the Costas loop because loss of phase lock was the weakest link in the chain, and moved to a system with a tight bandpass filter at the carrier frequency, and measuring zero crossings of the carrier against a local oscillator. But still the noise performance was not impressive at 300baud / 3kHz carrier.

I had hoped to go back to a damaged QPSK code recovery and to correct the data phase on the broken code bits, because we know what they should be according to the code. But there was no visible commonality between what was happening on the phase information that carried the code and the phase information carrying the data.

So I have moved back for now to a two-phase BPSK system interleaving the code and the payload symbol by symbol, in order to understand better how far the code can be pushed with noise.

Improved noise performance

Currently I use a 4.8kHz carrier and 1200baud symbols. Here is the performance today:

The RED line is the best decode seen for the correct match offset, the BLUE line is the best decode for all other (wrong) offsets. The PURPLE line is the mean decode for the correct match offset over the 100 runs. GREEN is the “quality cutoff” basically keeping us from getting near the blue line’s false hits: because of the properties of the code it is extremely unlikely noise will get near the green line.

Here is a close-up on the noisy end of things:

So far in terms of detecting the code, we can on average do so when the input energy is 82% noise (-15dB SNR). We can still detect codes ~1% of the time even at 92% noise (-21dB SNR). Here is a further close-up (it is 1000 runs, not 100)

suggesting you can still get good recoveries ~0.1% of the time at 95% noise (-25.5dB SNR). And of course we are now measuring the whole system performance here including the demodulator part, not just damaging the code bits directly.

Current BPSK receiver

Here is the receiver for the current BPSK method:

I spent several days meddling with the QPSK version and arriving at the carrier zero-crossing method for phase detection. The original plan to have a symbol sampler running from a locked LO hung around causing lots of problems. The indications of a change in symbol — detected by zero crossings — were variously delayed by the phase itself and the filters used, making it difficult to convert their jittery indications into a guide for the symbol recovery clock. This resulted in double bits being sampled.

The code is the symbol clock

Because of this I eventually realized that no symbol clock was needed, by running the correlator at the sample frequency, and sampling symbols at a fixed period, because of the false match rejection properties of the code it would “discover” the correct phase and offset from the behaviour of the correlator output. And when the code was recovered best then the data interleaved with it will be recovered best too. This sounds power-unfriendly, but after the first offset is used, you don’t run the correlator until enough time has passed for 256 symbols to be acquired, and then you should still be locked from when you did the first 256 symbols: if not you run the correlator a few dozen times to find lock again.

Also of note is that what is stored in the ringbuffer is a weighted average of the last four phase results, this includes information from the phase of multiple carrier cycles for the same symbol, helping to reduce the effect of noise on the decode.

The code is the data!

Well recovering the code at high SNR is interesting, but how useful is it if the data bits interleaved with it have been subject to the same beating without the properties of the code to protect them? We can use the autosyncing properties of the code to help with trying to get some payload signal gain through averaging, but I think I have seen where this is headed now… the code IS the signalling system for the payload data. That means throwing out the interleaved data concept.

A ’0′ can be signalled as the normal code, and a ’1′ as a time-reversed code. Because it’s intended for low data rate communication at VERY low signal levels compared to noise, it’s okay if we are reduced to 1 byte/sec, which will be the end result of this at 1200 baud. Basically with this we carry over all of the great robustness qualities of the code to be attributes of the payload data.

So what is the point compared to just blasting 128 symbol times of the same phase carrier, which is a hell of a lot simpler?

• Three high accuracy results, “no result”, a ’0′ or a ’1′ detected. The carrier-only method will happily return a bogus result if there is noise energy at the carrier — if the code method ever claims a ’0′ or a ’1′ you can be almost certain it is genuine
• “Fuzzy” robust damage-tolerant signal detection, better performance than a simple threshold comparator
• Automatic bit sync (“bit clock recovery”) with almost no chance of wrong sync, sync recaptured each symbol; bit sync for the carrier-only version in high noise is unreliable
• Absolute phase polarity can be recovered from one symbol despite the 180 degree lock uncertainty for BPSK — you can even lose lock one or more times inside the symbol and the code with absolute phase can still be recovered; the carrier-only concept needs a coding at a higher level to determine absolute recovered phase, in turn needing multiple correct symbols recovered without losing lock

Well, the first cut of the QPSK demodulator, bit slicing and correlation code works, and this stochastic performance graph sums it up.

It shows 1,000 Monte Carlo runs each from 0% to 100% noise on the raw channel, considering correct correlator matches only, the blue line shows the WORST correlator match result at that noise level, the red line is the BEST correlator result seen at that noise level, and the purple line is the mean correlator match result seen at that noise level.

The Green line is the +64 threshold as a reference… below this we don’t consider the correlator to have matched, something we chose based from studying the code response to noise a couple of posts ago.

Basically it shows that up to about 20% channel noise there is a very strong probability we return perfect results (+128 result means that no bit errors were present in the recovered correlator code).

After that there is a region up to about 50% noise where perfect results are sometimes seen, normally there is some corruption, but on average we can still recover a corrupted but high probability correctly sync’d code (whether the attached data payload can survive that beating is another issue… we can at least have reasonably correct bit-sync to whatever is there, allowing payload averaging for example).

But after 60% noise, the probability of finding a usable recovered code is less than 1 in 1000.

If you consider the purple mean line as the overall average recovery capability, it’s clear that at the moment after 40% noise things stop being much fun.

These figures reflect the whole reception system: at the moment the data slicer is a primitive edge-triggered type thing, pretty sure that is the limiting factor here. When I eyeball the recovered data payload when it is challenged by heavy channel noise, It is often broken in a “sticky” way:

hello magic code
hello ma��������
hello magic code
gllo magic co$�
hello magic code
���oo magic code
hello magic code
Yello magic co�7
hello magic code
hello magic cod

I think the sequential bit errors will be harder to create if a more robust bit-slicer is figured out. Here is an idea of what 45% noise in the channel looks like:

and what happens to that nice clean, digital-looking demodulation when you give it that instead of the nice clean sine waves:

That last one is the input into the bit slicer, which has the job of choosing where the bit boundaries are… at the moment it looks at zero-crossings, if it were possible to do a more sophisticated job than that a lot more packets could be saved you would think. I have an extremely cool idea about this I will look into next.

Time for some practical experiments with the robustness of the magic correlator code. Rather than build any hardware, although some interesting RF hardware is available, I decided to first model the system in software, so I can change things around much easier while there is a lot uncharacterized about the performance and capabilities of the coding.

Testing plan

The general plan is to bind payload data to the magic correlator code, such that the correlator code alignment acts as both a frame sync and assurance that the recovered bit clock is correct (in fact it should act as a clock recovery synthesis pilot as well, since we know the sequence). Because of the properties of the correlator code, it should be possible to just add FEC-coded payload without further ado, either interleaved bit-by-bit or bound another way.

At the moment I am sending plain payload without FEC. Further I am going to initially do all the testing at baseband directly, in fact specifically at audio frequencies. This will eventually allow real world testing using a laptop “transmitting” the coding through its speakers and my moving another microphone-enabled laptop around the house seeing how far we can push the data recovery vs what I can hear myself. My house it pretty noisy, with cars going by, kids jumping around and so on, there is a good mix of whitish noise and short duration dropout crud in the audio spectrum. I am really interested to see how far it can be pushed, particularly if averaging is possible to get working.

Modulation strategy

Currently the test C code modulates the data and the magic correlator code bits together on QPSK. QPSK has four “modulation states” encoded as four phase angles of the carrier, so you are signalling two bits per symbol — one payload bit and one magic correlator code bit. Eventually, because we can recover even badly damaged correlator code quite often, it will be possible to score the likelihood of false payload recovery bit by bit based on looking back at the magic code bits that turned out to be wrong in a particular symbol: the payload and the coding bits were transmitted in the same symbol. This “distrust” indication per payload bit can open the door to some novel, if possibly expensive, error correction.

The current C code uses 44.1kHz sample rate and a 3kHz carrier modulated with QPSK carrying 300 baud symbols, ie, there are 10 carrier cycles per symbol. Each symbol contains two bits due to the QPSK coding. Maybe as we go on it will be necessary to drop the symbol rate to allow more cycles and better recovery, but this is a starting point. You can listen to the QPSK coded WAV file here, this has the magic code and a 16 byte ASCII message payload modulated on QPSK. The symbol transitions look like this:

So, the “transmitting” part is fine… don’t be fooled by those little spikes where it changes phase, those easily lost spikes are not carrying the information. The whole rest of the bit period carrier goes on at the new phase after that discontiguity, the next ten cycles of carrier phase encodes the information. The spike could and probably should be completely filtered out and not cause trouble on recovery.

QPSK symbol recovery

QPSK recovery requires a coherent oscillator in frequency and phase sync with the incoming carrier. You recover the symbols by watching the gyrations the local oscillator has to perform to keep the phase sync.

The RF guys were on to all this stuff back in ancient times, a version of a Costas loop capable of separately dealing the with quadrature “carriers” in QPSK was invented way back when. Its basic idea is to feed the coherent local oscillator to two mixers, one with the local oscillator via a 90 degree phase shift, then lowpass filter the result from the mixers and use that to feed back an error term to the local oscillator. The lowpassed mixer outputs are the “result” from the loop for the two bits encoded in the QPSK modulation.

Now described there and the literature one can find in Google, it’s simple, right? Some of the PDFs on QPSK recovery from Google even had beautiful smooth digital recovery pictures. But the actual implementation is a lot trickier. The problems come from the need to tune various filters in the Costas loop, they need to be tuned for the carrier and VCO lock performance considering the symbol rate. I got started by looking at the Costas loop implementation in GnuRadio, but this has no filters at all in it (I guess this is for flexibility you can add the filters outside it). No doubt someone somewhere has been through all this back in 1970 and written up all the equations, but I couldn’t find it. In the end I found some general advice about matching the lowpass filter 3db point to half the symbol rate and meddled around until it worked. Of course doing it in software I didn’t even have the nightmares of filter matching for the two mixers which bedevil an analogue implementation. I also found a lot of 2 x carrier noise in my loop, which I tried to notch out with some success. Anyway here is what the recovery looks like right now, being fed “perfect” noise-free signal… it looks okay but I am pretty uncertain about how it will react to noise

QPSK absolute phase uncertainty

The receiver locked phase compared to the transmitter 0 degrees phase is unknown, therefore the decoded bits can appear on either of the two output bitstreams and be inverted. This has to be taken into account when looking at what you’re getting. Initially at least you have to run the four possible correlations against the magic code to find out the effective phase offset / symbol coding you have locked at. I guess while the receiver does not lose lock (one can study the error term in the Costas loop I guess) you can just use the lock you previously determined.

Incidentally received absolute phase determination is yet another exploitation of the magic correlation code properties of robust matching.

Next stop

Next task is to recover the bit clock from the received bits and perform the correlation action, and to try to recover the message. All of will still be in pure software with no noise yet.