Test object becomes a case class

README.md

@@ -5,7 +5,7 @@
 ## Overview
 
 
-Version: 1.0.19
+Version: 1.0.20
 
 API Scaladoc: [SparkHelper](http://xavierguihot.com/spark_helper/#com.spark_helper.SparkHelper$)
 
@@ -131,7 +131,7 @@ With sbt, add these lines to your build.sbt:
 ```scala
 resolvers += "jitpack" at "https://jitpack.io"
 
-libraryDependencies += "com.github.xavierguihot" % "spark_helper" % "v1.0.19"
+libraryDependencies += "com.github.xavierguihot" % "spark_helper" % "v1.0.20"
 ```
 
 With maven, add these lines to your pom.xml:
@@ -147,7 +147,7 @@ With maven, add these lines to your pom.xml:
 <dependency>
 	<groupId>com.github.xavierguihot</groupId>
 	<artifactId>spark_helper</artifactId>
-	<version>v1.0.19</version>
+	<version>v1.0.20</version>
 </dependency>
 ```
 
@@ -161,7 +161,7 @@ allprojects {
 }
 
 dependencies {
-	compile 'com.github.xavierguihot:spark_helper:v1.0.19'
+	compile 'com.github.xavierguihot:spark_helper:v1.0.20'
 }
 ```
 

build.sbt

@@ -1,6 +1,6 @@
 name := "spark_helper"
 
-version := "1.0.19"
+version := "1.0.20"
 
 scalaVersion := "2.11.12"
 

docs/com/spark_helper/monitoring/Monitor.html

@@ -65,8 +65,8 @@ <h4 id="signature" class="signature">
   <span class="cmt">// Let's say you want to get some KPIs on your output before storing it:</span>
   <span class="kw">val</span> outputIsValid = monitor.updateByKpisValidation(
     <span class="std">List</span>(
-      <span class="kw">new</span> Test(<span class="lit">"Nbr of output records"</span>, processedData.count(), SUPERIOR_THAN, <span class="num">10</span>e6d, NBR),
-      <span class="kw">new</span> Test(<span class="lit">"Some pct of invalid output"</span>, your_complex_kpi, INFERIOR_THAN, <span class="num">3</span>, PCT)
+      Test(<span class="lit">"Nbr of output records"</span>, processedData.count(), SUPERIOR_THAN, <span class="num">10</span>e6d, NBR),
+      Test(<span class="lit">"Some pct of invalid output"</span>, your_complex_kpi, INFERIOR_THAN, <span class="num">3</span>, PCT)
     ),
     <span class="lit">"My pipeline descirption"</span>
   )
@@ -581,7 +581,7 @@ <h4 class="signature">
 validation and the success status of the monitoring is set to false if the
 KPI isn't valid.</p><p>If the validation is a failure then after that the status will never be
 success again, even if you update the report with success tasks.</p><p>Using this method like this:</p><pre>monitor.updateByKpiValidation(
-  <span class="kw">new</span> Test(<span class="lit">"pctOfWhatever"</span>, <span class="num">0.06</span>d, INFERIOR_THAN, <span class="num">0.1</span>d, PCT),
+  Test(<span class="lit">"pctOfWhatever"</span>, <span class="num">0.06</span>d, INFERIOR_THAN, <span class="num">0.1</span>d, PCT),
   <span class="lit">"Tests for whatever"</span>)</pre><p>will result in this to be appended to the report:</p><pre>[<span class="num">10</span>:<span class="num">35</span>-<span class="num">10</span>:<span class="num">37</span>] Tests <span class="kw">for</span> whatever: success
   KPI: pctOfWhatever
     Value: <span class="num">0.06</span>%
@@ -609,9 +609,9 @@ <h4 class="signature">
 if at least one KPI isn't valid.</p><p>If the validation of tests is a failure then after that the status will
 never be success again, even if you update the report with success tasks.</p><p>Using this method like this:</p><pre>monitor.updateByKpisValidation(
   <span class="std">List</span>(
-    <span class="kw">new</span> Test(<span class="lit">"pctOfWhatever"</span>, <span class="num">0.06</span>d, INFERIOR_THAN, <span class="num">0.1</span>d, PCT),
-    <span class="kw">new</span> Test(<span class="lit">"pctOfSomethingElse"</span>, <span class="num">0.27</span>d, SUPERIOR_THAN, <span class="num">0.3</span>d, PCT),
-    <span class="kw">new</span> Test(<span class="lit">"someNbr"</span>, <span class="num">1235</span>d, EQUAL_TO, <span class="num">1235</span>d, NBR)
+    Test(<span class="lit">"pctOfWhatever"</span>, <span class="num">0.06</span>d, INFERIOR_THAN, <span class="num">0.1</span>d, PCT),
+    Test(<span class="lit">"pctOfSomethingElse"</span>, <span class="num">0.27</span>d, SUPERIOR_THAN, <span class="num">0.3</span>d, PCT),
+    Test(<span class="lit">"someNbr"</span>, <span class="num">1235</span>d, EQUAL_TO, <span class="num">1235</span>d, NBR)
   ),
   <span class="lit">"Tests for whatever"</span>
 )</pre><p>will result in this to be appended to the report:</p><pre>[<span class="num">10</span>:<span class="num">35</span>-<span class="num">10</span>:<span class="num">37</span>] Tests <span class="kw">for</span> whatever: failed

docs/com/spark_helper/monitoring/package.html

@@ -115,8 +115,8 @@ <h4 class="signature">
   <span class="cmt">// Let's say you want to get some KPIs on your output before storing it:</span>
   <span class="kw">val</span> outputIsValid = monitor.updateByKpisValidation(
     <span class="std">List</span>(
-      <span class="kw">new</span> Test(<span class="lit">"Nbr of output records"</span>, processedData.count(), SUPERIOR_THAN, <span class="num">10</span>e6d, NBR),
-      <span class="kw">new</span> Test(<span class="lit">"Some pct of invalid output"</span>, your_complex_kpi, INFERIOR_THAN, <span class="num">3</span>, PCT)
+      Test(<span class="lit">"Nbr of output records"</span>, processedData.count(), SUPERIOR_THAN, <span class="num">10</span>e6d, NBR),
+      Test(<span class="lit">"Some pct of invalid output"</span>, your_complex_kpi, INFERIOR_THAN, <span class="num">3</span>, PCT)
     ),
     <span class="lit">"My pipeline descirption"</span>
   )
@@ -193,25 +193,27 @@ <h4 class="signature">
 /main/scala/com/spark_helper/monitoring/Monitor.scala">Monitor</a>
 </p></div><dl class="attributes block"> <dt>Since</dt><dd><p>2017-02</p></dd></dl></div>
     </li><li name="com.spark_helper.monitoring.Test" visbl="pub" data-isabs="false" fullComment="yes" group="Ungrouped">
-      <a id="TestextendsAnyRef"></a>
+      <a id="TestextendsProductwithSerializable"></a>
       <a id="Test:Test"></a>
       <h4 class="signature">
       <span class="modifier_kind">
-        <span class="modifier"></span>
-        <span class="kind">class</span>
+        <span class="modifier">final </span>
+        <span class="kind">case class</span>
       </span>
       <span class="symbol">
-        <a href="Test.html"><span class="name">Test</span></a><span class="result"> extends <span class="extype" name="scala.AnyRef">AnyRef</span></span>
+        <a href="Test.html"><span class="name">Test</span></a><span class="params">(<span name="description">description: <span class="extype" name="scala.Predef.String">String</span></span>, <span name="kpiValue">kpiValue: <span class="extype" name="scala.Double">Double</span></span>, <span name="thresholdType">thresholdType: <a href="ThresholdType.html" class="extype" name="com.spark_helper.monitoring.ThresholdType">ThresholdType</a></span>, <span name="appliedThreshold">appliedThreshold: <span class="extype" name="scala.Double">Double</span></span>, <span name="kpiType">kpiType: <a href="KpiType.html" class="extype" name="com.spark_helper.monitoring.KpiType">KpiType</a></span>)</span><span class="result"> extends <span class="extype" name="scala.Product">Product</span> with <span class="extype" name="scala.Serializable">Serializable</span></span>
       </span>
       </h4><span class="permalink">
-      <a href="../../../index.html#com.spark_helper.monitoring.package@TestextendsAnyRef" title="Permalink" target="_top">
+      <a href="../../../index.html#com.spark_helper.monitoring.package@TestextendsProductwithSerializable" title="Permalink" target="_top">
         <img src="../../../lib/permalink.png" alt="Permalink" />
       </a>
     </span>
       <p class="shortcomment cmt">A class which represents a KPI to validate.</p><div class="fullcomment"><div class="comment cmt"><p>A class which represents a KPI to validate.</p><p>This is intended to be used as parameter of Monitor.updateByKpiValidation
-and Monitor.updateByKpisValidation methods.</p><p>Some exemples of Test objects:</p><pre><span class="kw">new</span> Test(<span class="lit">"pctOfWhatever"</span>, <span class="num">0.06</span>d, INFERIOR_THAN, <span class="num">0.1</span>d, PCT)
-<span class="kw">new</span> Test(<span class="lit">"pctOfSomethingElse"</span>, <span class="num">0.27</span>d, SUPERIOR_THAN, <span class="num">0.3</span>d, PCT)
-<span class="kw">new</span> Test(<span class="lit">"someNbr"</span>, <span class="num">1235</span>d, EQUAL_TO, <span class="num">1235</span>d, NBR)</pre></div><dl class="attributes block"> <dt>Since</dt><dd><p>2016-12</p></dd></dl></div>
+and Monitor.updateByKpisValidation methods.</p><p>Some exemples of Test objects:</p><pre>Test(<span class="lit">"pctOfWhatever"</span>, <span class="num">0.06</span>d, INFERIOR_THAN, <span class="num">0.1</span>d, PCT)
+Test(<span class="lit">"pctOfSomethingElse"</span>, <span class="num">0.27</span>d, SUPERIOR_THAN, <span class="num">0.3</span>d, PCT)
+Test(<span class="lit">"someNbr"</span>, <span class="num">1235</span>d, EQUAL_TO, <span class="num">1235</span>d, NBR)</pre></div><dl class="paramcmts block"><dt class="param">description</dt><dd class="cmt"><p>the name/description of the KPI which will appear on the
+validation report.</p></dd><dt class="param">kpiValue</dt><dd class="cmt"><p>the value for this KPI</p></dd><dt class="param">thresholdType</dt><dd class="cmt"><p>the type of threshold (SUPERIOR_THAN, INFERIOR_THAN or
+EQUAL_TO).</p></dd><dt class="param">appliedThreshold</dt><dd class="cmt"><p>the threshold to apply</p></dd><dt class="param">kpiType</dt><dd class="cmt"><p>the type of KPI (PCT or NBR)</p></dd></dl><dl class="attributes block"> <dt>Since</dt><dd><p>2016-12</p></dd></dl></div>
     </li><li name="com.spark_helper.monitoring.ThresholdType" visbl="pub" data-isabs="true" fullComment="no" group="Ungrouped">
       <a id="ThresholdTypeextendsAnyRef"></a>
       <a id="ThresholdType:ThresholdType"></a>

docs/index.html

@@ -38,7 +38,7 @@
             <ol class="templates"><li title="com.spark_helper.DateHelper"><a class="tplshow" href="com/spark_helper/DateHelper$.html" target="template"><span class="object">(object)</span><div class="placeholder"></div><span class="tplLink">DateHelper</span></a></li><li title="com.spark_helper.HdfsHelper"><a class="tplshow" href="com/spark_helper/HdfsHelper$.html" target="template"><span class="object">(object)</span><div class="placeholder"></div><span class="tplLink">HdfsHelper</span></a></li><li title="com.spark_helper.SparkHelper"><a class="tplshow" href="com/spark_helper/SparkHelper$.html" target="template"><span class="object">(object)</span><div class="placeholder"></div><span class="tplLink">SparkHelper</span></a></li></ol>
             <ol class="packages"> <li class="pack" title="com.spark_helper.monitoring">
             <a class="tplshow" href="com/spark_helper/monitoring/package.html" target="template">com.spark_helper.monitoring</a>
-            <ol class="templates"><li title="com.spark_helper.monitoring.EQUAL_TO"><a class="tplshow" href="com/spark_helper/monitoring/EQUAL_TO$.html" target="template"><span class="object">(object)</span><div class="placeholder"></div><span class="tplLink">EQUAL_TO</span></a></li><li title="com.spark_helper.monitoring.INFERIOR_THAN"><a class="tplshow" href="com/spark_helper/monitoring/INFERIOR_THAN$.html" target="template"><span class="object">(object)</span><div class="placeholder"></div><span class="tplLink">INFERIOR_THAN</span></a></li><li title="com.spark_helper.monitoring.KpiType"><div class="placeholder"></div><a class="tplshow" href="com/spark_helper/monitoring/KpiType.html" target="template"><span class="trait">(trait)</span><span class="tplLink">KpiType</span></a></li><li title="com.spark_helper.monitoring.Monitor"><div class="placeholder"></div><a class="tplshow" href="com/spark_helper/monitoring/Monitor.html" target="template"><span class="class">(class)</span><span class="tplLink">Monitor</span></a></li><li title="com.spark_helper.monitoring.NBR"><a class="tplshow" href="com/spark_helper/monitoring/NBR$.html" target="template"><span class="object">(object)</span><div class="placeholder"></div><span class="tplLink">NBR</span></a></li><li title="com.spark_helper.monitoring.PCT"><a class="tplshow" href="com/spark_helper/monitoring/PCT$.html" target="template"><span class="object">(object)</span><div class="placeholder"></div><span class="tplLink">PCT</span></a></li><li title="com.spark_helper.monitoring.SUPERIOR_THAN"><a class="tplshow" href="com/spark_helper/monitoring/SUPERIOR_THAN$.html" target="template"><span class="object">(object)</span><div class="placeholder"></div><span class="tplLink">SUPERIOR_THAN</span></a></li><li title="com.spark_helper.monitoring.Test"><div class="placeholder"></div><a class="tplshow" href="com/spark_helper/monitoring/Test.html" target="template"><span class="class">(class)</span><span class="tplLink">Test</span></a></li><li title="com.spark_helper.monitoring.ThresholdType"><div class="placeholder"></div><a class="tplshow" href="com/spark_helper/monitoring/ThresholdType.html" target="template"><span class="trait">(trait)</span><span class="tplLink">ThresholdType</span></a></li></ol>
+            <ol class="templates"><li title="com.spark_helper.monitoring.EQUAL_TO"><a class="tplshow" href="com/spark_helper/monitoring/EQUAL_TO$.html" target="template"><span class="object">(object)</span><div class="placeholder"></div><span class="tplLink">EQUAL_TO</span></a></li><li title="com.spark_helper.monitoring.INFERIOR_THAN"><a class="tplshow" href="com/spark_helper/monitoring/INFERIOR_THAN$.html" target="template"><span class="object">(object)</span><div class="placeholder"></div><span class="tplLink">INFERIOR_THAN</span></a></li><li title="com.spark_helper.monitoring.KpiType"><div class="placeholder"></div><a class="tplshow" href="com/spark_helper/monitoring/KpiType.html" target="template"><span class="trait">(trait)</span><span class="tplLink">KpiType</span></a></li><li title="com.spark_helper.monitoring.Monitor"><div class="placeholder"></div><a class="tplshow" href="com/spark_helper/monitoring/Monitor.html" target="template"><span class="class">(class)</span><span class="tplLink">Monitor</span></a></li><li title="com.spark_helper.monitoring.NBR"><a class="tplshow" href="com/spark_helper/monitoring/NBR$.html" target="template"><span class="object">(object)</span><div class="placeholder"></div><span class="tplLink">NBR</span></a></li><li title="com.spark_helper.monitoring.PCT"><a class="tplshow" href="com/spark_helper/monitoring/PCT$.html" target="template"><span class="object">(object)</span><div class="placeholder"></div><span class="tplLink">PCT</span></a></li><li title="com.spark_helper.monitoring.SUPERIOR_THAN"><a class="tplshow" href="com/spark_helper/monitoring/SUPERIOR_THAN$.html" target="template"><span class="object">(object)</span><div class="placeholder"></div><span class="tplLink">SUPERIOR_THAN</span></a></li><li title="com.spark_helper.monitoring.Test"><div class="placeholder"></div><a class="tplshow" href="com/spark_helper/monitoring/Test.html" target="template"><span class="case class">(case class)</span><span class="tplLink">Test</span></a></li><li title="com.spark_helper.monitoring.ThresholdType"><div class="placeholder"></div><a class="tplshow" href="com/spark_helper/monitoring/ThresholdType.html" target="template"><span class="trait">(trait)</span><span class="tplLink">ThresholdType</span></a></li></ol>
             <ol class="packages"> </ol>
           </li></ol>
           </li></ol>

docs/index/index-a.html

@@ -19,5 +19,8 @@
     </div><div class="entry">
       <div class="name">appendHeaderAndFooter</div>
       <div class="occurrences"><a href="../com/spark_helper/HdfsHelper$.html" class="extype" name="com.spark_helper.HdfsHelper">HdfsHelper</a> </div>
+    </div><div class="entry">
+      <div class="name">appliedThreshold</div>
+      <div class="occurrences"><a href="../com/spark_helper/monitoring/Test.html" class="extype" name="com.spark_helper.monitoring.Test">Test</a> </div>
     </div></body>
       </html>

docs/index/index-d.html

@@ -31,5 +31,8 @@
     </div><div class="entry">
       <div class="name">deleteFolder</div>
       <div class="occurrences"><a href="../com/spark_helper/HdfsHelper$.html" class="extype" name="com.spark_helper.HdfsHelper">HdfsHelper</a> </div>
+    </div><div class="entry">
+      <div class="name">description</div>
+      <div class="occurrences"><a href="../com/spark_helper/monitoring/Test.html" class="extype" name="com.spark_helper.monitoring.Test">Test</a> </div>
     </div></body>
       </html>

docs/index/index-k.html

@@ -13,5 +13,11 @@
         <body><div class="entry">
       <div class="name">KpiType</div>
       <div class="occurrences"><a href="../com/spark_helper/monitoring/package.html" class="extype" name="com.spark_helper.monitoring">monitoring</a> </div>
+    </div><div class="entry">
+      <div class="name">kpiType</div>
+      <div class="occurrences"><a href="../com/spark_helper/monitoring/Test.html" class="extype" name="com.spark_helper.monitoring.Test">Test</a> </div>
+    </div><div class="entry">
+      <div class="name">kpiValue</div>
+      <div class="occurrences"><a href="../com/spark_helper/monitoring/Test.html" class="extype" name="com.spark_helper.monitoring.Test">Test</a> </div>
     </div></body>
       </html>