sampleclean.crowd

AsyncCrowdResult

class AsyncCrowdResult[O] extends Awaitable[RDD[(String, O)]]

An asynchronous result object that returns tuples as they are processed. Callbacks are registered and executed on completion of individual or batches of tuples.

O

type of output data

Linear Supertypes
Awaitable[RDD[(String, O)]], AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. AsyncCrowdResult
  2. Awaitable
  3. AnyRef
  4. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new AsyncCrowdResult()(implicit sc: SparkContext)

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  6. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  7. def batchProcessed(batch: Seq[(String, O)]): Unit

    Fires the processed batch event.

    Fires the processed batch event. This will cause the onBatchProcessed callback to execute.

    batch

    the newly processed batch of output tuples.

  8. var batchSize: Int

  9. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  10. def complete(): Unit

    Fires the completed event.

    Fires the completed event. This will cause the onComplete callback to execute.

  11. var completed: Boolean

  12. var curBatch: Seq[(String, O)]

  13. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  14. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  15. val executionPromise: Promise[Unit]

  16. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  17. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  18. def getCurrentResults: Option[RDD[(String, O)]]

    Returns all tuples processed so far.

    Returns all tuples processed so far. Can be called by users at any time during processing.

  19. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  20. def isCompleted: Boolean

    Returns true if processing is complete.

    Returns true if processing is complete. Can be called by users at any time during processing.

  21. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  22. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  23. final def notify(): Unit

    Definition Classes
    AnyRef
  24. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  25. def onBatchProcessed(f: (Seq[(String, O)]) ⇒ Unit, tupleBatchSize: Int = 10): AsyncCrowdResult[O]

    Registers a callback to be called when a new batch of tuples is processed.

    Registers a callback to be called when a new batch of tuples is processed. Should be called by users.

    f

    callback that the new tuples and returns nothing.

    tupleBatchSize

    size of batches after which to call the callback.

  26. var onBatchProcessedFunc: (Seq[(String, O)]) ⇒ Unit

  27. def onComplete(f: () ⇒ Unit): AsyncCrowdResult[O]

    Registers a callback to be called when processing is complete.

    Registers a callback to be called when processing is complete. Should be called by users. To access processed data, users should call the getCurrentResults() function.

    f

    callback that takes no arguments and returns nothing.

  28. def onTupleProcessed(f: ((String, O)) ⇒ Unit): AsyncCrowdResult[O]

    Registers a callback to be called when a new tuple is processed.

    Registers a callback to be called when a new tuple is processed. Should be called by users.

    f

    callback that the new tuple and returns nothing.

  29. var onTupleProcessedFunc: ((String, O)) ⇒ Unit

  30. def ready(atMost: Duration)(implicit permit: CanAwait): AsyncCrowdResult.this.type

    Blocks until processing is complete.

    Blocks until processing is complete.

    Definition Classes
    AsyncCrowdResult → Awaitable
    See also

    scala.concurrent.Awaitable

  31. def result(atMost: Duration)(implicit permit: CanAwait): RDD[(String, O)]

    Blocks until processing is complete, then returns the processed results.

    Blocks until processing is complete, then returns the processed results.

    Definition Classes
    AsyncCrowdResult → Awaitable
    See also

    scala.concurrent.Awaitable

  32. var results: RDD[(String, O)]

  33. implicit val sc: SparkContext

  34. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  35. def toString(): String

    Definition Classes
    AnyRef → Any
  36. def tupleProcessed(tuple: (String, O)): Unit

    Fires the processed tuple event.

    Fires the processed tuple event. This will cause the onTupleProcessed (and possibly the onBatchProcessed) callbacks to execute.

    tuple

    the newly processed tuple.

  37. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  38. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  39. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Awaitable[RDD[(String, O)]]

Inherited from AnyRef

Inherited from Any

Ungrouped