Package io.github.ajevans.dbcode.dbconsumers

Class HadoopConsumerPushTest

java.lang.Object
io.github.ajevans.dbcode.dbconsumers.HadoopConsumerPushTest
@TestInstance(PER_CLASS)
@TestMethodOrder(org.junit.jupiter.api.MethodOrderer.OrderAnnotation.class)
@DisplayName("Unit tests for DerbyConsumer, minus GUI")
public class HadoopConsumerPushTest
extends Object
/** Unit tests for HadoopConsumer, minus GUI, using the slower data push pathway.

This is just functional tests for the slow-loading large-file push pathway. See HadoopConsumerTest for the majority of tests against this class.

Reads defaults and first and last lines from application.properties to allow for simple replacement of data for fundamental tests.

JUnit 5.

Writes files to user directory.

Author:
Andy Evans
Version: 1.0 01 Mar 2021

  • Field Details

    • propertiesDirectory

      private String propertiesDirectory
      Directory for properties files.

    • defaultDatabaseDirectory

      private String defaultDatabaseDirectory
      Directory database is in - gained from properties.

    • defaultFileDirectory

      private String defaultFileDirectory
      Directory files are in - gained from properties.

    • defaultDataset

      private String defaultDataset
      Default database name - gained from properties.

    • defaultTable

      private String defaultTable
      Default table name - gained from properties.

    • defaultTableOriginal

      private String defaultTableOriginal
      Default table name without characters removed - gained from properties.

    • defaultSource

      private String defaultSource
      Default source file - gained from properties.

    • defaultDatasetOriginal

      private String defaultDatasetOriginal
      Default source file without characters removed - gained from properties.

    • defaultSource1stLine

      private String defaultSource1stLine
      Default source file - gained from properties.

    • defaultSourceLastLine

      private String defaultSourceLastLine
      Default source file - gained from properties.

    • store

      private String store
      Database location - constructed from default variables.

    • dataSupplier

      private CruTs2pt1Supplier dataSupplier
      Data supplier to use.

    • dataConsumer

      private HadoopConsumer dataConsumer
      Data consumer to use.

    • fileSystem

      org.apache.hadoop.fs.FileSystem fileSystem
      HDFS.

    • conf

      org.apache.hadoop.conf.Configuration conf
      Configuration.

  • Constructor Details

    • HadoopConsumerPushTest

      public HadoopConsumerPushTest()

  • Method Details

    • loadData

      @BeforeAll @DisplayName("Load data") void loadData()
      Runs the application to build data files. Uses default locations from the app's META-INF/application.properties file.

    • disconnect

      @AfterAll @DisplayName("Disconnect from file system") public void disconnect()
      Cuts file system connection.

    • dataLoadedTest

      @Order(1) @Test @DisplayName("Data is loaded into file system") void dataLoadedTest()
      Connects to the default data file and checks the first and last lines.

      Uses default names from the app's META-INF/application.properties file along with first and last line from the same file.

      Validates on first and last lines.

    • UStoISODate

      private String UStoISODate​(String line)
      Converts strings in the format "XX,YY,mm/dd/yyy,ZZ" to the format "XX,YY,yyyy-MM-dd,ZZ", ie, from US date format to ISO 8601.
      Parameters:
      line - Line to convert.
      Returns:
      String Converted line.

    • metadataLoadedTest

      @Order(2) @Test @DisplayName("Metadata is loaded into file system") void metadataLoadedTest()
      Checks the default file's metadata.

      Validates on title.

      Uses default names from the app's META-INF/application.properties file.