org.apache.orc.impl.writer

Class TreeWriterBase

java.lang.Object

org.apache.orc.impl.writer.TreeWriterBase

All Implemented Interfaces:

TreeWriter

Direct Known Subclasses:

BinaryTreeWriter, BooleanTreeWriter, ByteTreeWriter, DateTreeWriter, Decimal64TreeWriter, DecimalTreeWriter, DoubleTreeWriter, FloatTreeWriter, IntegerTreeWriter, ListTreeWriter, MapTreeWriter, StringBaseTreeWriter, StructTreeWriter, TimestampTreeWriter, UnionTreeWriter

public abstract class TreeWriterBase
extends Object
implements TreeWriter

The parent class of all of the writers for each column. Each column is written by an instance of this class. The compound types (struct, list, map, and union) have children tree writers that write the children types.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.orc.impl.writer.TreeWriter

TreeWriter.Factory

Field Summary

Fields

Modifier and Type	Field and Description
protected BloomFilter	bloomFilter
protected OrcProto.BloomFilter.Builder	bloomFilterEntry
protected BloomFilterUtf8	bloomFilterUtf8
protected boolean	createBloomFilter
protected ColumnStatisticsImpl	fileStatistics
protected int	id
protected ColumnStatisticsImpl	indexStatistics
protected BitFieldWriter	isPresent
protected org.apache.orc.impl.writer.TreeWriterBase.RowIndexPositionRecorder	rowIndexPosition
protected ColumnStatisticsImpl	stripeColStatistics

Method Summary

Modifier and Type	Method and Description
void	createRowIndexEntry() Create a row index entry with the previous location and the current index statistics.
long	estimateMemory() Estimate how much memory the writer is consuming excluding the streams.
void	flushStreams() Flush the TreeWriter stream
protected OrcProto.RowIndex.Builder	getRowIndex()
protected OrcProto.RowIndexEntry.Builder	getRowIndexEntry()
protected ColumnStatisticsImpl	getStripeStatistics()
void	updateFileStatistics(OrcProto.StripeStatistics stats) During a stripe append, we need to update the file statistics.
void	writeBatch(ColumnVector vector, int offset, int length) Write the values from the given vector from offset for length elements.
void	writeFileStatistics(OrcProto.Footer.Builder footer) Add the file statistics to the file footer.
void	writeRootBatch(VectorizedRowBatch batch, int offset, int length) Handle the top level object write.
void	writeStripe(OrcProto.StripeFooter.Builder builder, OrcProto.StripeStatistics.Builder stats, int requiredIndexEntries) Write the stripe out to the file.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.orc.impl.writer.TreeWriter

getRawDataSize

Field Detail

id

protected final int id

isPresent

protected final BitFieldWriter isPresent

indexStatistics

protected final ColumnStatisticsImpl indexStatistics

stripeColStatistics

protected final ColumnStatisticsImpl stripeColStatistics

fileStatistics

protected final ColumnStatisticsImpl fileStatistics

rowIndexPosition

protected final org.apache.orc.impl.writer.TreeWriterBase.RowIndexPositionRecorder rowIndexPosition

bloomFilter

protected final BloomFilter bloomFilter

bloomFilterUtf8

protected final BloomFilterUtf8 bloomFilterUtf8

createBloomFilter

protected final boolean createBloomFilter

bloomFilterEntry

protected final OrcProto.BloomFilter.Builder bloomFilterEntry

Method Detail

getRowIndex

protected OrcProto.RowIndex.Builder getRowIndex()

getStripeStatistics

protected ColumnStatisticsImpl getStripeStatistics()

getRowIndexEntry

protected OrcProto.RowIndexEntry.Builder getRowIndexEntry()

writeRootBatch

public void writeRootBatch(VectorizedRowBatch batch,
                           int offset,
                           int length)
                    throws IOException

Handle the top level object write. This default method is used for all types except structs, which are the typical case. VectorizedRowBatch assumes the top level object is a struct, so we use the first column for all other types.

Specified by:

writeRootBatch in interface TreeWriter

Parameters:

batch - the batch to write from

offset - the row to start on

length - the number of rows to write

Throws:

IOException

writeBatch

public void writeBatch(ColumnVector vector,
                       int offset,
                       int length)
                throws IOException

Write the values from the given vector from offset for length elements.

Specified by:

writeBatch in interface TreeWriter

Parameters:

vector - the vector to write from

offset - the first value from the vector to write

length - the number of values from the vector to write

Throws:

IOException

flushStreams

public void flushStreams()
                  throws IOException

Description copied from interface: TreeWriter

Flush the TreeWriter stream

Specified by:

flushStreams in interface TreeWriter

Throws:

IOException

writeStripe

public void writeStripe(OrcProto.StripeFooter.Builder builder,
                        OrcProto.StripeStatistics.Builder stats,
                        int requiredIndexEntries)
                 throws IOException

Description copied from interface: TreeWriter

Write the stripe out to the file.

Specified by:

writeStripe in interface TreeWriter

Parameters:

builder - the stripe footer that contains the information about the layout of the stripe. The TreeWriterBase is required to update the footer with its information.

stats - the stripe statistics information

requiredIndexEntries - the number of index entries that are required. this is to check to make sure the row index is well formed.

Throws:

IOException

createRowIndexEntry

public void createRowIndexEntry()
                         throws IOException

Create a row index entry with the previous location and the current index statistics. Also merges the index statistics into the file statistics before they are cleared. Finally, it records the start of the next index and ensures all of the children columns also create an entry.

Specified by:

createRowIndexEntry in interface TreeWriter

Throws:

IOException

updateFileStatistics

public void updateFileStatistics(OrcProto.StripeStatistics stats)

Description copied from interface: TreeWriter

During a stripe append, we need to update the file statistics.

Specified by:

updateFileStatistics in interface TreeWriter

Parameters:

stats - the statistics for the new stripe

estimateMemory

public long estimateMemory()

Estimate how much memory the writer is consuming excluding the streams.

Specified by:

estimateMemory in interface TreeWriter

Returns:

the number of bytes.

writeFileStatistics

public void writeFileStatistics(OrcProto.Footer.Builder footer)

Description copied from interface: TreeWriter

Add the file statistics to the file footer.

Specified by:

writeFileStatistics in interface TreeWriter

Parameters:

footer - the file footer builder