org.apache.orc.impl

Class PhysicalFsWriter

java.lang.Object

org.apache.orc.impl.PhysicalFsWriter

All Implemented Interfaces:

PhysicalWriter

public class PhysicalFsWriter
extends Object
implements PhysicalWriter

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.orc.PhysicalWriter

PhysicalWriter.OutputReceiver

Constructor Summary

Constructors

Constructor and Description
PhysicalFsWriter(FileSystem fs, Path path, OrcFile.WriterOptions opts)

Method Summary

Modifier and Type	Method and Description
void	appendRawStripe(ByteBuffer buffer, OrcProto.StripeInformation.Builder dirEntry) Appends raw stripe data (e.g.
void	close() Closes the writer.
org.apache.orc.impl.PhysicalFsWriter.BufferedStream	createDataStream(StreamName name) Create an OutputReceiver for the given name.
void	finalizeStripe(OrcProto.StripeFooter.Builder footerBuilder, OrcProto.StripeInformation.Builder dirEntry) Flushes the data in all the streams, spills them to disk, write out stripe footer.
void	flush() Flushes the writer so that readers can see the preceeding postscripts.
CompressionCodec	getCompressionCodec() Gets a compression codec used by this writer.
long	getFileBytes(int column) Get the number of bytes for a file in a given column by finding all the streams (not suppressed) for a given column and returning the sum of their sizes.
String	toString()
void	writeBloomFilter(StreamName name, OrcProto.BloomFilterIndex.Builder bloom, CompressionCodec codec) Write a bloom filter index in the given stream name.
void	writeFileFooter(OrcProto.Footer.Builder builder) Writes out the file footer.
void	writeFileMetadata(OrcProto.Metadata.Builder builder) Writes out the file metadata.
void	writeHeader() Writes the header of the file, which consists of the magic "ORC" bytes.
void	writeIndex(StreamName name, OrcProto.RowIndex.Builder index, CompressionCodec codec) Write an index in the given stream name.
long	writePostScript(OrcProto.PostScript.Builder builder) Writes out the postscript (including the size byte if needed).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

PhysicalFsWriter

public PhysicalFsWriter(FileSystem fs,
                        Path path,
                        OrcFile.WriterOptions opts)
                 throws IOException

Throws:

IOException

Method Detail

getCompressionCodec

public CompressionCodec getCompressionCodec()

Description copied from interface: PhysicalWriter

Gets a compression codec used by this writer.

Specified by:

getCompressionCodec in interface PhysicalWriter

getFileBytes

public long getFileBytes(int column)

Get the number of bytes for a file in a given column by finding all the streams (not suppressed) for a given column and returning the sum of their sizes. excludes index

Specified by:

getFileBytes in interface PhysicalWriter

Parameters:

column - column from which to get file size

Returns:

number of bytes for the given column

writeFileMetadata

public void writeFileMetadata(OrcProto.Metadata.Builder builder)
                       throws IOException

Description copied from interface: PhysicalWriter

Writes out the file metadata.

Specified by:

writeFileMetadata in interface PhysicalWriter

Parameters:

builder - Metadata builder to finalize and write.

Throws:

IOException

writeFileFooter

public void writeFileFooter(OrcProto.Footer.Builder builder)
                     throws IOException

Description copied from interface: PhysicalWriter

Writes out the file footer.

Specified by:

writeFileFooter in interface PhysicalWriter

Parameters:

builder - Footer builder to finalize and write.

Throws:

IOException

writePostScript

public long writePostScript(OrcProto.PostScript.Builder builder)
                     throws IOException

Description copied from interface: PhysicalWriter

Writes out the postscript (including the size byte if needed).

Specified by:

writePostScript in interface PhysicalWriter

Parameters:

builder - Postscript builder to finalize and write.

Throws:

IOException

close

public void close()
           throws IOException

Description copied from interface: PhysicalWriter

Closes the writer.

Specified by:

close in interface PhysicalWriter

Throws:

IOException

flush

public void flush()
           throws IOException

Description copied from interface: PhysicalWriter

Flushes the writer so that readers can see the preceeding postscripts.

Specified by:

flush in interface PhysicalWriter

Throws:

IOException

appendRawStripe

public void appendRawStripe(ByteBuffer buffer,
                            OrcProto.StripeInformation.Builder dirEntry)
                     throws IOException

Description copied from interface: PhysicalWriter

Appends raw stripe data (e.g. for file merger).

Specified by:

appendRawStripe in interface PhysicalWriter

Parameters:

buffer - Stripe data buffer.

dirEntry - File metadata entry for the stripe, to be updated with relevant data.

Throws:

IOException

finalizeStripe

public void finalizeStripe(OrcProto.StripeFooter.Builder footerBuilder,
                           OrcProto.StripeInformation.Builder dirEntry)
                    throws IOException

Description copied from interface: PhysicalWriter

Flushes the data in all the streams, spills them to disk, write out stripe footer.

Specified by:

finalizeStripe in interface PhysicalWriter

Parameters:

footerBuilder - Stripe footer to be updated with relevant data and written out.

dirEntry - File metadata entry for the stripe, to be updated with relevant data.

Throws:

IOException

writeHeader

public void writeHeader()
                 throws IOException

Description copied from interface: PhysicalWriter

Writes the header of the file, which consists of the magic "ORC" bytes.

Specified by:

writeHeader in interface PhysicalWriter

Throws:

IOException

createDataStream

public org.apache.orc.impl.PhysicalFsWriter.BufferedStream createDataStream(StreamName name)

Description copied from interface: PhysicalWriter

Create an OutputReceiver for the given name.

Specified by:

createDataStream in interface PhysicalWriter

Parameters:

name - the name of the stream

writeIndex

public void writeIndex(StreamName name,
                       OrcProto.RowIndex.Builder index,
                       CompressionCodec codec)
                throws IOException

Description copied from interface: PhysicalWriter

Write an index in the given stream name.

Specified by:

writeIndex in interface PhysicalWriter

Parameters:

name - the name of the stream

index - the bloom filter to write

codec - the compression codec to use

Throws:

IOException

writeBloomFilter

public void writeBloomFilter(StreamName name,
                             OrcProto.BloomFilterIndex.Builder bloom,
                             CompressionCodec codec)
                      throws IOException

Description copied from interface: PhysicalWriter

Write a bloom filter index in the given stream name.

Specified by:

writeBloomFilter in interface PhysicalWriter

Parameters:

name - the name of the stream

bloom - the bloom filter to write

codec - the compression codec to use

Throws:

IOException

toString

public String toString()

Overrides:

toString in class Object