eu.slipo.athenarc.triplegeo.utils

Class RMLConverter

java.lang.Object

eu.slipo.athenarc.triplegeo.utils.RMLConverter

All Implemented Interfaces:

Converter

public class RMLConverter
extends java.lang.Object
implements Converter

Creates and populates a RML dataset so that data can be serialized into a file. LIMITATIONS: - No support for dynamically executed built-in functions (e.g., for calculating on-the-fly the area or length of geometries). - By convention, attribute names referring to input data must be specified in UPPER-CASE letters in the RML mappings (in .TTL files). - Currently, no support for transformation from OSM or GPX or on top of Spark.

Version:

2.0

Author:

Kostas Patroumpas

Field Summary

Fields

Modifier and Type	Field and Description
(package private) java.lang.String	attrAssignedCategory
(package private) java.util.Map<java.lang.String,java.lang.Integer>	attrStatistics
(package private) java.lang.String[]	csvHeader
private static Configuration	currentConfig
private long	dt
(package private) java.lang.String[]	exeTriplesMap
(package private) be.ugent.mmlab.rml.model.TriplesMap[]	maps
com.vividsolutions.jts.geom.Envelope	mbr
private Assistant	myAssistant
private FeatureRegister	myRegister
private int	numRec
private int	numTriples
(package private) java.lang.String	paramCategoryURI
(package private) java.util.Map<java.lang.String,java.lang.String>	parameters
(package private) java.lang.String	paramFeatureURI
(package private) be.ugent.mmlab.rml.performer.RMLPerformer[]	performers
private java.io.BufferedWriter	registryWriter
private int	rejectedRec
private long	t_start
(package private) java.lang.String	templateCategoryURI
(package private) java.lang.String	templateFeatureURI

Constructor Summary

Constructors

Constructor and Description
RMLConverter(Configuration config, Assistant assist, java.lang.String outputFile) Constructs a RMLConverter object that will conduct transformation at RML mode.

Method Summary

Modifier and Type	Method and Description
private void	collectTuples4Registry() Collects tuples generated from a batch of features (their basic thematic attributes and their geometries) and streamlines them to a file for the SLIPO Registry.
com.vividsolutions.jts.geom.Envelope	getMBR() Provides the MBR of the geographic dataset that has been transformed.
org.apache.jena.rdf.model.Model	getModel() Provides the disk-based model consisting of transformed triples.
java.util.Map<java.lang.String,java.lang.Integer>	getStatistics() Provides statistics collected for each attribute after transformation.
java.lang.String	getTDBDir() Returns the local directory that holds the disk-based RDF graph for this transformation thread.
java.util.List<org.apache.jena.graph.Triple>	getTriples() Provides triples resulting from transformation.
java.lang.String	getURItemplate4Classification() Provides the URI template used for all subjects in RDF triples concerning the classification hierarchy.
java.lang.String	getURItemplate4Feature() Provides the URI template used for all subjects in RDF triples concerning features.
void	parse(org.geotools.feature.FeatureIterator<?> iterator, Classification classific, org.opengis.referencing.operation.MathTransform reproject, int targetSRID, java.lang.String outputFile) Parses each record from a FeatureIterator and streamlines the resulting triples (including geometric and non-spatial attributes) according to the given RML mapping.
void	parse(java.util.Iterator<org.apache.commons.csv.CSVRecord> records, Classification classific, org.opengis.referencing.operation.MathTransform reproject, int targetSRID, java.lang.String outputFile) Parses each record from a collection of CSV records and streamlines the resulting triples (including geometric and non-spatial attributes) according to the given RML mapping.
void	parse(OSMRecord rs, Classification classific, org.opengis.referencing.operation.MathTransform reproject, int targetSRID) Parses a single OSM record and streamlines the resulting triples (including geometric and non-spatial attributes) according to the given RML mapping.
void	parse(java.sql.ResultSet rs, Classification classific, org.opengis.referencing.operation.MathTransform reproject, int targetSRID, java.lang.String outputFile) Parses each record from a ResultSet and streamlines the resulting triples (including geometric and non-spatial attributes) according to the given RML mapping.
void	parse(java.lang.String wkt, java.util.Map<java.lang.String,java.lang.String> attrValues, Classification classific, int targetSRID, org.opengis.referencing.operation.MathTransform reproject, java.lang.String geomType, int partition_index, java.lang.String outputFile) Parses a Map structure of (key, value) pairs and streamlines the resulting triples (including geometric and non-spatial attributes).
void	parse(java.lang.String wkt, java.util.Map<java.lang.String,java.lang.String> attrValues, Classification classific, int targetSRID, java.lang.String geomType) Parses a single GPX waypoint/track or a single JSON node and streamlines the resulting triples (including geometric and non-spatial attributes) according to the given RML mapping.
void	parseWithRML(java.util.HashMap<java.lang.String,java.lang.String> row, be.ugent.mmlab.rml.model.dataset.RMLDataset dataset) Converts a row (with several attribute values, including geometry) into suitable triples according to the specified RML mappings.
void	setHeader(java.lang.String[] header) Retains the header (column names) of an input CSV file.
void	store(java.lang.String outputFile) Stores resulting tuples into a file for the SLIPO Registry.
void	store(java.lang.String outputFile, int partition_index) Finalizes storage of resulting tuples into a file.
void	updateMBR(com.vividsolutions.jts.geom.Geometry g) Updates the MBR of the geographic dataset as this is being transformed.
private void	updateStatistics(java.lang.String attrKey) Update statistics for the given attribute during transformation.
int	writeTriples(be.ugent.mmlab.rml.model.dataset.RMLDataset dataset, java.io.BufferedWriter writer, org.openrdf.rio.RDFFormat rdfFormat, java.lang.String encoding) Serializes the given RML dataset as triples written into a file.
int	writeTriples(be.ugent.mmlab.rml.model.dataset.RMLDataset dataset, java.io.OutputStream writer, org.openrdf.rio.RDFFormat rdfFormat) Serializes the given RML dataset as triples written into a file.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

currentConfig

private static Configuration currentConfig

registryWriter

private java.io.BufferedWriter registryWriter

myAssistant

private Assistant myAssistant

myRegister

private FeatureRegister myRegister

performers

be.ugent.mmlab.rml.performer.RMLPerformer[] performers

exeTriplesMap

java.lang.String[] exeTriplesMap

parameters

java.util.Map<java.lang.String,java.lang.String> parameters

maps

be.ugent.mmlab.rml.model.TriplesMap[] maps

paramFeatureURI

java.lang.String paramFeatureURI

templateFeatureURI

java.lang.String templateFeatureURI

paramCategoryURI

java.lang.String paramCategoryURI

templateCategoryURI

java.lang.String templateCategoryURI

attrAssignedCategory

java.lang.String attrAssignedCategory

attrStatistics

java.util.Map<java.lang.String,java.lang.Integer> attrStatistics

csvHeader

java.lang.String[] csvHeader

t_start

private long t_start

dt

private long dt

numRec

private int numRec

rejectedRec

private int rejectedRec

numTriples

private int numTriples

mbr

public com.vividsolutions.jts.geom.Envelope mbr

Constructor Detail

RMLConverter

public RMLConverter(Configuration config,
                    Assistant assist,
                    java.lang.String outputFile)

Constructs a RMLConverter object that will conduct transformation at RML mode.

Parameters:

config - User-specified configuration for the transformation process.

assist - Assistant to perform auxiliary operations.

outputFile - Output file that will collect resulting triples.

Method Detail

updateStatistics

private void updateStatistics(java.lang.String attrKey)

Update statistics for the given attribute during transformation.

Parameters:

attrKey - Attribute name for which to update statistics.

getStatistics

public java.util.Map<java.lang.String,java.lang.Integer> getStatistics()

Provides statistics collected for each attribute after transformation.

Returns:

Count of transformed values per attribute.

updateMBR

public void updateMBR(com.vividsolutions.jts.geom.Geometry g)

Updates the MBR of the geographic dataset as this is being transformed.

Parameters:

g - Geometry that will be checked for possible expansion of the MBR of all features processed thus far

getMBR

public com.vividsolutions.jts.geom.Envelope getMBR()

Provides the MBR of the geographic dataset that has been transformed.

Returns:

The MBR of the transformed geometries.

parse

public void parse(org.geotools.feature.FeatureIterator<?> iterator,
                  Classification classific,
                  org.opengis.referencing.operation.MathTransform reproject,
                  int targetSRID,
                  java.lang.String outputFile)

Parses each record from a FeatureIterator and streamlines the resulting triples (including geometric and non-spatial attributes) according to the given RML mapping. Applicable in RML transformation mode. Input provided via a FeatureIterator (used with input formats: Shapefiles, GeoJSON).

Specified by:

parse in interface Converter

Parameters:

iterator - FeatureIterator over spatial features collected from an ESRI shapefile of a GeoJSON file.

classific - Instantiation of the classification scheme that assigns categories to input features.

reproject - CRS transformation parameters to be used in reprojecting a geometry to a target SRID (EPSG code).

targetSRID - Spatial reference system (EPSG code) of geometries in the output RDF triples.

outputFile - Path to the output file that collects RDF triples.

parse

public void parse(java.sql.ResultSet rs,
                  Classification classific,
                  org.opengis.referencing.operation.MathTransform reproject,
                  int targetSRID,
                  java.lang.String outputFile)

Parses each record from a ResultSet and streamlines the resulting triples (including geometric and non-spatial attributes) according to the given RML mapping. Applicable in RML transformation mode. Input provided via a ResultSet (used with input from a DMBS).

Specified by:

parse in interface Converter

Parameters:

rs - ResultSet containing spatial features retrieved from a DBMS.

classific - Instantiation of the classification scheme that assigns categories to input features.

reproject - CRS transformation parameters to be used in reprojecting a geometry to a target SRID (EPSG code).

targetSRID - Spatial reference system (EPSG code) of geometries in the output RDF triples.

outputFile - Path to the output file that collects RDF triples.

parse

public void parse(java.util.Iterator<org.apache.commons.csv.CSVRecord> records,
                  Classification classific,
                  org.opengis.referencing.operation.MathTransform reproject,
                  int targetSRID,
                  java.lang.String outputFile)

Parses each record from a collection of CSV records and streamlines the resulting triples (including geometric and non-spatial attributes) according to the given RML mapping. Applicable in RML transformation mode. Input provided by iterating over a collection of CSV records (used with input from CSV).

Specified by:

parse in interface Converter

Parameters:

records - Iterator over CSV records collected from a CSV file.

classific - Instantiation of the classification scheme that assigns categories to input features.

reproject - CRS transformation parameters to be used in reprojecting a geometry to a target SRID (EPSG code).

targetSRID - Spatial reference system (EPSG code) of geometries in the output RDF triples.

outputFile - Path to the output file that collects RDF triples.

parse

public void parse(OSMRecord rs,
                  Classification classific,
                  org.opengis.referencing.operation.MathTransform reproject,
                  int targetSRID)

Parses a single OSM record and streamlines the resulting triples (including geometric and non-spatial attributes) according to the given RML mapping. Input provided as an individual record (used with input format: OpenStreetMap XML). TODO: Implement for RML transformation mode.

Specified by:

parse in interface Converter

Parameters:

rs - Representation of an OSM record with attributes extracted from an OSM element (node, way, or relation).

classific - Instantiation of the classification scheme that assigns categories to input features.

reproject - CRS transformation parameters to be used in reprojecting a geometry to a target SRID (EPSG code).

targetSRID - Spatial reference system (EPSG code) of geometries in the output RDF triples.

parse

public void parse(java.lang.String wkt,
                  java.util.Map<java.lang.String,java.lang.String> attrValues,
                  Classification classific,
                  int targetSRID,
                  java.lang.String geomType)

Parses a single GPX waypoint/track or a single JSON node and streamlines the resulting triples (including geometric and non-spatial attributes) according to the given RML mapping. Input provided as an individual record (used with input format: GPX, JSON). TODO: Implement for RML transformation mode.

Specified by:

parse in interface Converter

Parameters:

wkt - Well-Known Text representation of the geometry

attrValues - Attribute values for each thematic (non-spatial) attribute

classific - Instantiation of the classification scheme that assigns categories to input features.

targetSRID - Spatial reference system (EPSG code) of geometries in the output RDF triples.

geomType - The type of the geometry (e.g., POINT, POLYGON, etc.)

parse

public void parse(java.lang.String wkt,
                  java.util.Map<java.lang.String,java.lang.String> attrValues,
                  Classification classific,
                  int targetSRID,
                  org.opengis.referencing.operation.MathTransform reproject,
                  java.lang.String geomType,
                  int partition_index,
                  java.lang.String outputFile)

Parses a Map structure of (key, value) pairs and streamlines the resulting triples (including geometric and non-spatial attributes). Input provided as an individual record. This method may be used when running over Spark/GeoSpark. TODO: Implement for RML transformation mode.

Specified by:

parse in interface Converter

Parameters:

wkt - Well-Known Text representation of the geometry

attrValues - Attribute values for each thematic (non-spatial) attribute

classific - Instantiation of the classification scheme that assigns categories to input features.

targetSRID - Spatial reference system (EPSG code) of geometries in the output RDF triples.

reproject - CRS transformation parameters to be used in reprojecting a geometry to a target SRID (EPSG code).

geomType - The type of the geometry (e.g., POINT, POLYGON, etc.)

partition_index - The index of the partition.

outputFile - Path to the output file that collects RDF triples.

collectTuples4Registry

private void collectTuples4Registry()

Collects tuples generated from a batch of features (their basic thematic attributes and their geometries) and streamlines them to a file for the SLIPO Registry.

store

public void store(java.lang.String outputFile)

Stores resulting tuples into a file for the SLIPO Registry. Used only for flushing data to the registry in RML transformation mode.

Specified by:

store in interface Converter

Parameters:

outputFile - Path to the output file that collects RDF triples.

store

public void store(java.lang.String outputFile,
                  int partition_index)

Finalizes storage of resulting tuples into a file. This method is used when running over Spark/GeoSpark. Not applicable in RML transformation mode.

Specified by:

store in interface Converter

Parameters:

outputFile - Path to the output file that collects RDF triples.

partition_index - The index of the partition.

setHeader

public void setHeader(java.lang.String[] header)

Retains the header (column names) of an input CSV file.

Specified by:

setHeader in interface Converter

Parameters:

header - A array of attribute (column) names.

getModel

public org.apache.jena.rdf.model.Model getModel()

Provides the disk-based model consisting of transformed triples. Not applicable in RML transformation mode.

Specified by:

getModel in interface Converter

Returns:

The model created for a dataset (RDF graph) stored on disk.

getTDBDir

public java.lang.String getTDBDir()

Returns the local directory that holds the disk-based RDF graph for this transformation thread. Not applicable in RML transformation mode.

Specified by:

getTDBDir in interface Converter

Returns:

Path to the local directory.

getTriples

public java.util.List<org.apache.jena.graph.Triple> getTriples()

Provides triples resulting from transformation. Not applicable in RML transformation mode.

Specified by:

getTriples in interface Converter

Returns:

A collection of RDF triples.

parseWithRML

public void parseWithRML(java.util.HashMap<java.lang.String,java.lang.String> row,
                         be.ugent.mmlab.rml.model.dataset.RMLDataset dataset)

Converts a row (with several attribute values, including geometry) into suitable triples according to the specified RML mappings. Applicable in RML transformation mode only.

Specified by:

parseWithRML in interface Converter

Parameters:

row - Record with attribute names and their respective values.

dataset - RMLDataset to collect the resulting triples.

getURItemplate4Classification

public java.lang.String getURItemplate4Classification()

Provides the URI template used for all subjects in RDF triples concerning the classification hierarchy. Applicable in RML transformation mode.

Specified by:

getURItemplate4Classification in interface Converter

Returns:

A URI to be used as template for triples regarding classification.

getURItemplate4Feature

public java.lang.String getURItemplate4Feature()

Provides the URI template used for all subjects in RDF triples concerning features. Applicable in RML transformation mode.

Returns:

A URI to be used as template for triples of features.

writeTriples

public int writeTriples(be.ugent.mmlab.rml.model.dataset.RMLDataset dataset,
                        java.io.BufferedWriter writer,
                        org.openrdf.rio.RDFFormat rdfFormat,
                        java.lang.String encoding)

Serializes the given RML dataset as triples written into a file. Applicable in RML transformation mode.

Specified by:

writeTriples in interface Converter

Parameters:

dataset - RMLDataset that has collected the resulting triples.

writer - BufferedWriter to write triples in the output stream.

rdfFormat - Serialization format of triples to be written to the file.

encoding - Encoding for string literals.

Returns:

The number of triples written to the output file.

Throws:

java.io.IOException

writeTriples

public int writeTriples(be.ugent.mmlab.rml.model.dataset.RMLDataset dataset,
                        java.io.OutputStream writer,
                        org.openrdf.rio.RDFFormat rdfFormat)

Serializes the given RML dataset as triples written into a file. Applicable in RML transformation mode only.

Specified by:

writeTriples in interface Converter

Parameters:

dataset - RMLDataset that has collected the resulting triples.

writer - BufferedWriter to write triples in the output stream.

rdfFormat - Serialization format of triples to be written to the file.

Returns:

The number of triples written to the output file.

Throws:

java.io.IOException