org.apache.orc.impl.writer

Class StringBaseTreeWriter

java.lang.Object

org.apache.orc.impl.writer.TreeWriterBase

org.apache.orc.impl.writer.StringBaseTreeWriter

All Implemented Interfaces:

TreeWriter

Direct Known Subclasses:

CharTreeWriter, StringTreeWriter, VarcharTreeWriter

public abstract class StringBaseTreeWriter
extends TreeWriterBase

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.orc.impl.writer.TreeWriter

TreeWriter.Factory

Field Summary

Fields

Modifier and Type	Field and Description
protected StringRedBlackTree	dictionary
protected PositionedOutputStream	directStreamOutput
protected IntegerWriter	lengthOutput
protected DynamicIntArray	rows
protected boolean	useDictionaryEncoding

Fields inherited from class org.apache.orc.impl.writer.TreeWriterBase

bloomFilter, bloomFilterEntry, bloomFilterUtf8, createBloomFilter, fileStatistics, id, indexStatistics, isPresent, rowIndexPosition, stripeColStatistics

Method Summary

Modifier and Type	Method and Description
void	createRowIndexEntry() This method doesn't call the super method, because unlike most of the other TreeWriters, this one can't record the position in the streams until the stripe is being flushed.
long	estimateMemory() Estimate how much memory the writer is consuming excluding the streams.
void	flushStreams() Flush the TreeWriter stream
long	getRawDataSize() Estimate the memory used if the file was read into Hive's Writable types.
void	writeStripe(OrcProto.StripeFooter.Builder builder, OrcProto.StripeStatistics.Builder stats, int requiredIndexEntries) Write the stripe out to the file.

Methods inherited from class org.apache.orc.impl.writer.TreeWriterBase

getRowIndex, getRowIndexEntry, getStripeStatistics, updateFileStatistics, writeBatch, writeFileStatistics, writeRootBatch

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

lengthOutput

protected final IntegerWriter lengthOutput

dictionary

protected final StringRedBlackTree dictionary

rows

protected final DynamicIntArray rows

directStreamOutput

protected final PositionedOutputStream directStreamOutput

useDictionaryEncoding

protected boolean useDictionaryEncoding

Method Detail

writeStripe

public void writeStripe(OrcProto.StripeFooter.Builder builder,
                        OrcProto.StripeStatistics.Builder stats,
                        int requiredIndexEntries)
                 throws IOException

Description copied from interface: TreeWriter

Write the stripe out to the file.

Specified by:

writeStripe in interface TreeWriter

Overrides:

writeStripe in class TreeWriterBase

Parameters:

builder - the stripe footer that contains the information about the layout of the stripe. The TreeWriterBase is required to update the footer with its information.

stats - the stripe statistics information

requiredIndexEntries - the number of index entries that are required. this is to check to make sure the row index is well formed.

Throws:

IOException

createRowIndexEntry

public void createRowIndexEntry()
                         throws IOException

This method doesn't call the super method, because unlike most of the other TreeWriters, this one can't record the position in the streams until the stripe is being flushed. Therefore it saves all of the entries and augments them with the final information as the stripe is written.

Specified by:

createRowIndexEntry in interface TreeWriter

Overrides:

createRowIndexEntry in class TreeWriterBase

Throws:

IOException

estimateMemory

public long estimateMemory()

Description copied from class: TreeWriterBase

Estimate how much memory the writer is consuming excluding the streams.

Specified by:

estimateMemory in interface TreeWriter

Overrides:

estimateMemory in class TreeWriterBase

Returns:

the number of bytes.

getRawDataSize

public long getRawDataSize()

Description copied from interface: TreeWriter

Estimate the memory used if the file was read into Hive's Writable types. This is used as an estimate for the query optimizer.

Returns:

the number of bytes

flushStreams

public void flushStreams()
                  throws IOException

Description copied from interface: TreeWriter

Flush the TreeWriter stream

Specified by:

flushStreams in interface TreeWriter

Overrides:

flushStreams in class TreeWriterBase

Throws:

IOException