pq: eliminate corruption by forcing version byte to be persisted

When the PQ creates a new page and allocates a memory-mapped buffer, the
underlying file is zero'd out to full page capacity and the version byte is
written to the buffer.

If Logstash crashes or is shut down before any elements have been pushed into
the queue page, we have no guarantees that the version marker has been
persisted to the storage device. A subsequent attempt to load an all-zeros
queue page will result in an obscure error message and failure to load:

~~~
AbstractPipelineExt - Logstash failed to create queue.
org.logstash.ackedqueue.io.MmapPageIOV2$PageIOInvalidVersionException: Expected page version=2 but found version=0
~~~

By sending `MappedByteBuffer#force()` immediately after the version has been
added to the buffer, we can shrink the window in which a crash can leave the
queue on disk in a corrupt state.
This commit is contained in:
Ry Biesemeyer 2021-01-07 17:44:23 +00:00
parent c5941e6340
commit e2f2255fa5

View file

@ -190,6 +190,7 @@ public final class MmapPageIOV2 implements PageIO {
} }
buffer.position(0); buffer.position(0);
buffer.put(VERSION_TWO); buffer.put(VERSION_TWO);
buffer.force();
this.head = 1; this.head = 1;
this.minSeqNum = 0L; this.minSeqNum = 0L;
this.elementCount = 0; this.elementCount = 0;