Package io.github.ajevans.dbcode.dbconsumers

Class HadoopConsumerTest

java.lang.Object
io.github.ajevans.dbcode.dbconsumers.HadoopConsumerTest
@TestInstance(PER_CLASS)
@TestMethodOrder(org.junit.jupiter.api.MethodOrderer.OrderAnnotation.class)
@DisplayName("Unit tests for DerbyConsumer, minus GUI")
public class HadoopConsumerTest
extends Object
Unit tests for HadoopConsumer, minus GUI.

Tests were built for edge conditions for a few specific methods, but the most significant tests run through a full file load and check metadata ('title') and the first and last lines of output data files. As data entry is checked with the supplier, these were largely implemented to check soundness during development.

Reads defaults and first and last lines from application.properties to allow for simple replacement of data for fundamental tests.

JUnit 5.

Writes files to user directory.

Author:
Andy Evans
Version: 1.0 01 Mar 2021

  • Field Details

    • propertiesDirectory

      private String propertiesDirectory
      Directory for properties files.

    • defaultDatabaseDirectory

      private String defaultDatabaseDirectory
      Directory database is in - gained from properties.

    • defaultFileDirectory

      private String defaultFileDirectory
      Directory files are in - gained from properties.

    • defaultDataset

      private String defaultDataset
      Default database name - gained from properties.

    • defaultTable

      private String defaultTable
      Default table name - gained from properties.

    • defaultTableOriginal

      private String defaultTableOriginal
      Default table name without characters removed - gained from properties.

    • defaultSource

      private String defaultSource
      Default source file - gained from properties.

    • defaultDatasetOriginal

      private String defaultDatasetOriginal
      Default source file without characters removed - gained from properties.

    • defaultSource1stLine

      private String defaultSource1stLine
      Default source file - gained from properties.

    • defaultSourceLastLine

      private String defaultSourceLastLine
      Default source file - gained from properties.

    • store

      private String store
      Database location - constructed from default variables.

    • dataSupplier

      private CruTs2pt1Supplier dataSupplier
      Data supplier to use.

    • dataConsumer

      private HadoopConsumer dataConsumer
      Data consumer to use.

    • fileSystem

      org.apache.hadoop.fs.FileSystem fileSystem
      HDFS.

    • conf

      org.apache.hadoop.conf.Configuration conf
      Configuration.

  • Constructor Details

    • HadoopConsumerTest

      public HadoopConsumerTest()

  • Method Details

    • loadData

      @BeforeAll @DisplayName("Load data") void loadData()
      Runs the application to build data files. Uses default locations from the app's META-INF/application.properties file.

    • disconnect

      @AfterAll @DisplayName("Disconnect from file system") public void disconnect()
      Cuts file system connection.

    • setStoreAsNullTest

      @Test @DisplayName("(setStore) \"\" -> null") public void setStoreAsNullTest()
      Tests setStore flags empty Strings as null.

    • setStoreSanitiseTest

      @Test @DisplayName("(setStore) unsanitisedString -> sanitisedString") public void setStoreSanitiseTest()
      Tests setStore and its sanitisation and separator add.

    • setRecordStoreNamesAsNullTest

      @Test @DisplayName("(setRecordStoreNames) null -> null") @Deprecated public void setRecordStoreNamesAsNullTest()
      Deprecated.
      Was needed, now of limited excitment.
      Just checks sending setRecordStoreNames null returns early leaving them as null.

    • setRecordStoreNamesSanitiseTest

      @Test @DisplayName("(setRecordStoreNames) unsanitisedStrings -> sanitisedStrings") public void setRecordStoreNamesSanitiseTest()
      Tests setRecordStoreNames and its sanitisation.

    • findTitleDefaultTest

      @Test @DisplayName("(findTitle) null -> DEFAULT") public void findTitleDefaultTest()
      Tests whether a missing title setup returns the default.

      Note that this is not the same as returning a default for an empty string (the starting point for Metadata), though maybe it should be.

    • findTitleTest

      @Test @DisplayName("(findTitle) title -> title") public void findTitleTest()
      Tests whether the consumer can find the database title in metadata.

    • sanitiseVigorousTest

      @Test @DisplayName("(sanitise(SANITISE_DIRPATH)) unsanitisedString -> sanitisedString") public void sanitiseVigorousTest()
      Test sanitise for a SANITISE_DIRPATH run.

    • sanitiseNameTest

      @Test @DisplayName("(sanitise(SANITISE_FILENAME)) unsanitisedString -> sanitisedString") public void sanitiseNameTest()
      Test sanitise for a SANITISE_FILENAME run.

    • sanitiseWeakTest

      @Test @DisplayName("(sanitise(SANITISE_DATA)) unsanitisedString -> sanitisedString") public void sanitiseWeakTest()
      Test sanitise for a SANITISE_DATA run.

    • dataLoadedTest

      @Order(1) @Test @DisplayName("Data is loaded into file system") void dataLoadedTest()
      Connects to the default data file and checks the first and last lines.

      Uses default names from the app's META-INF/application.properties file along with first and last line from the same file.

      Validates on first and last lines.

    • UStoISODate

      private String UStoISODate​(String line)
      Converts strings in the format "XX,YY,mm/dd/yyy,ZZ" to the format "XX,YY,yyyy-MM-dd,ZZ", ie, from US date format to ISO 8601.
      Parameters:
      line - Line to convert.
      Returns:
      String Converted line.

    • metadataLoadedTest

      @Order(2) @Test @DisplayName("Metadata is loaded into file system") void metadataLoadedTest()
      Checks the default file's metadata.

      Validates on title.

      Uses default names from the app's META-INF/application.properties file.