eu.slipo.athenarc.triplegeo.utils

Class StreamConverter

java.lang.Object

eu.slipo.athenarc.triplegeo.utils.StreamConverter

All Implemented Interfaces:

Converter

public class StreamConverter
extends java.lang.Object
implements Converter

Provides a set of a streaming RDF triples in memory that can be readily serialized into a file.

Version:

2.0

Author:

Kostas Patroumpas

Field Summary

Fields

Modifier and Type	Field and Description
private static Configuration	currentConfig
private long	dt
private Assistant	myAssistant
private TripleGenerator	myGenerator
private FeatureRegister	myRegister
private int	numRec
private int	numTriples
private java.io.OutputStream	outFile
private java.io.BufferedWriter	registryWriter
private int	rejectedRec
private java.util.List<org.apache.jena.graph.Triple>	results
private org.apache.jena.riot.system.StreamRDF	stream
private long	t_start

Constructor Summary

Constructors

Constructor and Description
StreamConverter(Configuration config, Assistant assist, java.lang.String outputFile) Constructs a StreamConverter object that will conduct transformation at STREAM mode.

Method Summary

Modifier and Type	Method and Description
private void	collectTriples() Collects RDF triples generated from a batch of features (their thematic attributes and their geometries) and streamlines them to output file.
org.apache.jena.rdf.model.Model	getModel() Provides access to the disk-based model consisting of transformed triples.
java.lang.String	getTDBDir() Returns the local directory that holds the disk-based RDF graph for this transformation thread.
java.util.List<org.apache.jena.graph.Triple>	getTriples() Provides triples resulted after applying transformation against a single input feature or a small batch of features.
java.lang.String	getURItemplate4Classification() Provides the URI template used for all subjects in RDF triples concerning the classification hierarchy.
void	parse(org.geotools.feature.FeatureIterator<?> iterator, Classification classific, org.opengis.referencing.operation.MathTransform reproject, int targetSRID, java.lang.String outputFile) Parses each record from a FeatureIterator and streamlines the resulting triples (including geometric and non-spatial attributes).
void	parse(java.util.Iterator<org.apache.commons.csv.CSVRecord> records, Classification classific, org.opengis.referencing.operation.MathTransform reproject, int targetSRID, java.lang.String outputFile) Parses each record from a collection of CSV records and streamlines the resulting triples (including geometric and non-spatial attributes).
void	parse(OSMRecord rs, Classification classific, org.opengis.referencing.operation.MathTransform reproject, int targetSRID) Parses a single OSM record and streamlines the resulting triples (including geometric and non-spatial attributes).
void	parse(java.sql.ResultSet rs, Classification classific, org.opengis.referencing.operation.MathTransform reproject, int targetSRID, java.lang.String outputFile) Parses each record from a ResultSet and streamlines the resulting triples (including geometric and non-spatial attributes).
void	parse(java.lang.String wkt, java.util.Map<java.lang.String,java.lang.String> attrValues, Classification classific, int targetSRID, org.opengis.referencing.operation.MathTransform reproject, java.lang.String geomType, int partition_index, java.lang.String outputFile) Parses a Map structure of (key, value) pairs and streamlines the resulting triples (including geometric and non-spatial attributes).
void	parse(java.lang.String wkt, java.util.Map<java.lang.String,java.lang.String> attrValues, Classification classific, int targetSRID, java.lang.String geomType) Parses a single GPX waypoint/track or a single JSON node and streamlines the resulting triples (including geometric and non-spatial attributes).
void	parseWithRML(java.util.HashMap<java.lang.String,java.lang.String> row, be.ugent.mmlab.rml.model.dataset.RMLDataset dataset) Converts a row into triples according to the specified RML mappings.
void	setHeader(java.lang.String[] header) Retains the header (column names) of an input CSV file.
void	store(java.lang.String outputFile) Finalizes storage of resulting tuples into a file.
void	store(java.lang.String outputFile, int partition_index) Finalizes storage of resulting tuples into a file.
int	writeTriples(be.ugent.mmlab.rml.model.dataset.RMLDataset dataset, java.io.BufferedWriter writer, org.openrdf.rio.RDFFormat rdfFormat, java.lang.String encoding) Serializes the given RML dataset as triples written into a file.
int	writeTriples(be.ugent.mmlab.rml.model.dataset.RMLDataset dataset, java.io.OutputStream writer, org.openrdf.rio.RDFFormat rdfFormat) Serializes the given RML dataset as triples written into a file.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

currentConfig

private static Configuration currentConfig

results

private java.util.List<org.apache.jena.graph.Triple> results

myGenerator

private TripleGenerator myGenerator

myAssistant

private Assistant myAssistant

myRegister

private FeatureRegister myRegister

t_start

private long t_start

dt

private long dt

numRec

private int numRec

rejectedRec

private int rejectedRec

numTriples

private int numTriples

registryWriter

private java.io.BufferedWriter registryWriter

outFile

private java.io.OutputStream outFile

stream

private org.apache.jena.riot.system.StreamRDF stream

Constructor Detail

StreamConverter

public StreamConverter(Configuration config,
                       Assistant assist,
                       java.lang.String outputFile)

Constructs a StreamConverter object that will conduct transformation at STREAM mode.

Parameters:

config - User-specified configuration for the transformation process.

assist - Assistant to perform auxiliary operations.

outputFile - Output file that will collect resulting triples.

Method Detail

getTriples

public java.util.List<org.apache.jena.graph.Triple> getTriples()

Provides triples resulted after applying transformation against a single input feature or a small batch of features. Applicable in STREAM transformation mode.

Specified by:

getTriples in interface Converter

Returns:

A collection of RDF triples.

parse

public void parse(org.geotools.feature.FeatureIterator<?> iterator,
                  Classification classific,
                  org.opengis.referencing.operation.MathTransform reproject,
                  int targetSRID,
                  java.lang.String outputFile)

Parses each record from a FeatureIterator and streamlines the resulting triples (including geometric and non-spatial attributes). Applicable in STREAM transformation mode. Input provided via a FeatureIterator. This method is used for input from : Shapefiles, GeoJSON.

Specified by:

parse in interface Converter

Parameters:

myAssistant - Instantiation of Assistant class to perform auxiliary operations (geometry transformations, auto-generation of UUIDs, etc.)

iterator - FeatureIterator over spatial features collected from an ESRI shapefile of a GeoJSON file.

classific - Instantiation of the classification scheme that assigns categories to input features.

reproject - CRS transformation parameters to be used in reprojecting a geometry to a target SRID (EPSG code).

targetSRID - Spatial reference system (EPSG code) of geometries in the output RDF triples.

outputFile - Path to the output file that collects RDF triples.

parse

public void parse(java.sql.ResultSet rs,
                  Classification classific,
                  org.opengis.referencing.operation.MathTransform reproject,
                  int targetSRID,
                  java.lang.String outputFile)

Parses each record from a ResultSet and streamlines the resulting triples (including geometric and non-spatial attributes). Applicable in STREAM transformation mode. Input provided via a ResultSet. This method is used for input from a DMBS.

Specified by:

parse in interface Converter

Parameters:

myAssistant - Instantiation of Assistant class to perform auxiliary operations (geometry transformations, auto-generation of UUIDs, etc.)

rs - ResultSet containing spatial features retrieved from a DBMS.

classific - Instantiation of the classification scheme that assigns categories to input features.

reproject - CRS transformation parameters to be used in reprojecting a geometry to a target SRID (EPSG code).

targetSRID - Spatial reference system (EPSG code) of geometries in the output RDF triples.

outputFile - Path to the output file that collects RDF triples.

parse

public void parse(java.util.Iterator<org.apache.commons.csv.CSVRecord> records,
                  Classification classific,
                  org.opengis.referencing.operation.MathTransform reproject,
                  int targetSRID,
                  java.lang.String outputFile)

Parses each record from a collection of CSV records and streamlines the resulting triples (including geometric and non-spatial attributes). Applicable in STREAM transformation mode. Input provided by iterating over a collection of CSV records. This method is used for input from CSV.

Specified by:

parse in interface Converter

Parameters:

myAssistant - Instantiation of Assistant class to perform auxiliary operations (geometry transformations, auto-generation of UUIDs, etc.)

records - Iterator over CSV records collected from a CSV file.

classific - Instantiation of the classification scheme that assigns categories to input features.

reproject - CRS transformation parameters to be used in reprojecting a geometry to a target SRID (EPSG code).

targetSRID - Spatial reference system (EPSG code) of geometries in the output RDF triples.

outputFile - Path to the output file that collects RDF triples.

parse

public void parse(OSMRecord rs,
                  Classification classific,
                  org.opengis.referencing.operation.MathTransform reproject,
                  int targetSRID)

Parses a single OSM record and streamlines the resulting triples (including geometric and non-spatial attributes). Applicable in STREAM transformation mode. Input provided as an individual record. This method is used for input from OpenStreetMap XML/PBF files.

Specified by:

parse in interface Converter

Parameters:

myAssistant - Instantiation of Assistant class to perform auxiliary operations (geometry transformations, auto-generation of UUIDs, etc.)

rs - Representation of an OSM record with attributes extracted from an OSM element (node, way, or relation).

classific - Instantiation of the classification scheme that assigns categories to input features.

reproject - CRS transformation parameters to be used in reprojecting a geometry to a target SRID (EPSG code).

targetSRID - Spatial reference system (EPSG code) of geometries in the output RDF triples.

parse

public void parse(java.lang.String wkt,
                  java.util.Map<java.lang.String,java.lang.String> attrValues,
                  Classification classific,
                  int targetSRID,
                  java.lang.String geomType)

Parses a single GPX waypoint/track or a single JSON node and streamlines the resulting triples (including geometric and non-spatial attributes). Applicable in STREAM transformation mode. Input provided as an individual record. This method is used for input data from GPX or JSON files.

Specified by:

parse in interface Converter

Parameters:

myAssistant - Instantiation of Assistant class to perform auxiliary operations (geometry transformations, auto-generation of UUIDs, etc.)

wkt - Well-Known Text representation of the geometry

attrValues - Attribute values for each thematic (non-spatial) attribute

classific - Instantiation of the classification scheme that assigns categories to input features.

targetSRID - Spatial reference system (EPSG code) of geometries in the output RDF triples.

geomType - The type of the geometry (e.g., POINT, POLYGON, etc.)

parse

public void parse(java.lang.String wkt,
                  java.util.Map<java.lang.String,java.lang.String> attrValues,
                  Classification classific,
                  int targetSRID,
                  org.opengis.referencing.operation.MathTransform reproject,
                  java.lang.String geomType,
                  int partition_index,
                  java.lang.String outputFile)

Parses a Map structure of (key, value) pairs and streamlines the resulting triples (including geometric and non-spatial attributes). Applicable in STREAM transformation mode. Input provided as an individual record. This method is used when running over Spark/GeoSpark.

Specified by:

parse in interface Converter

Parameters:

myAssistant - Instantiation of Assistant class to perform auxiliary operations (geometry transformations, auto-generation of UUIDs, etc.)

wkt - Well-Known Text representation of the geometry

attrValues - Attribute values for each thematic (non-spatial) attribute

classific - Instantiation of the classification scheme that assigns categories to input features.

targetSRID - Spatial reference system (EPSG code) of geometries in the output RDF triples.

reproject - CRS transformation parameters to be used in reprojecting a geometry to a target SRID (EPSG code).

geomType - The type of the geometry (e.g., POINT, POLYGON, etc.)

partition_index - The index of the partition.

outputFile - Path to the output file that collects RDF triples.

collectTriples

private void collectTriples()

Collects RDF triples generated from a batch of features (their thematic attributes and their geometries) and streamlines them to output file.

store

public void store(java.lang.String outputFile)

Finalizes storage of resulting tuples into a file.

Specified by:

store in interface Converter

Parameters:

myAssistant - Instantiation of Assistant class to perform auxiliary operations (geometry transformations, auto-generation of UUIDs, etc.)

outputFile - Path to the output file that collects RDF triples.

store

public void store(java.lang.String outputFile,
                  int partition_index)

Finalizes storage of resulting tuples into a file. This method is used when running over Spark/GeoSpark.

Specified by:

store in interface Converter

Parameters:

myAssistant - Instantiation of Assistant class to perform auxiliary operations (geometry transformations, auto-generation of UUIDs, etc.)

outputFile - Path to the output file that collects RDF triples.

partition_index - The index of the partition.

setHeader

public void setHeader(java.lang.String[] header)

Retains the header (column names) of an input CSV file. Not applicable in STREAM transformation mode.

Specified by:

setHeader in interface Converter

Parameters:

header - A array of attribute (column) names.

getModel

public org.apache.jena.rdf.model.Model getModel()

Provides access to the disk-based model consisting of transformed triples. Not applicable in STREAM transformation mode.

Specified by:

getModel in interface Converter

Returns:

The model created for a dataset (RDF graph) stored on disk.

getTDBDir

public java.lang.String getTDBDir()

Returns the local directory that holds the disk-based RDF graph for this transformation thread. Not applicable in STREAM transformation mode.

Specified by:

getTDBDir in interface Converter

Returns:

Path to the local directory.

writeTriples

public int writeTriples(be.ugent.mmlab.rml.model.dataset.RMLDataset dataset,
                        java.io.BufferedWriter writer,
                        org.openrdf.rio.RDFFormat rdfFormat,
                        java.lang.String encoding)
                 throws java.io.IOException

Serializes the given RML dataset as triples written into a file. Not applicable in STREAM transformation mode.

Specified by:

writeTriples in interface Converter

Parameters:

dataset - RMLDataset that has collected the resulting triples.

writer - BufferedWriter to write triples in the output stream.

rdfFormat - Serialization format of triples to be written to the file.

encoding - Encoding for string literals.

Returns:

The number of triples written to the output file.

Throws:

java.io.IOException

writeTriples

public int writeTriples(be.ugent.mmlab.rml.model.dataset.RMLDataset dataset,
                        java.io.OutputStream writer,
                        org.openrdf.rio.RDFFormat rdfFormat)
                 throws java.io.IOException

Serializes the given RML dataset as triples written into a file. Not applicable in STREAM transformation mode.

Specified by:

writeTriples in interface Converter

Parameters:

dataset - RMLDataset that has collected the resulting triples.

writer - BufferedWriter to write triples in the output stream.

rdfFormat - Serialization format of triples to be written to the file.

Returns:

The number of triples written to the output file.

Throws:

java.io.IOException

parseWithRML

public void parseWithRML(java.util.HashMap<java.lang.String,java.lang.String> row,
                         be.ugent.mmlab.rml.model.dataset.RMLDataset dataset)

Converts a row into triples according to the specified RML mappings. Not applicable in STREAM transformation mode.

Specified by:

parseWithRML in interface Converter

Parameters:

row - Record with attribute names and their respective values.

dataset - RMLDataset to collect the resulting triples.

getURItemplate4Classification

public java.lang.String getURItemplate4Classification()

Provides the URI template used for all subjects in RDF triples concerning the classification hierarchy. Not applicable in STREAM transformation mode.

Specified by:

getURItemplate4Classification in interface Converter

Returns:

A URI to be used as template for triples regarding classification.