Oℕ
class collection[T: TraversableOnce] {}
or class collection[T[_]: Functor] {}
or class collection[T[_]: Functor] {}
this advice has numerous errors, beware
def map(data: T): T = data.map(_.length)
Size: a a a
2020 February 27
Oℕ
collection has a method
this definition is nonsensical
ZH
this definition is nonsensical
Ok then what is your opinion?
ZH
this definition is nonsensical
O have a dynamic computation model based on which I run these transformations
ZH
λ
Ok then what is your opinion?
you need just functor implementation for your Rdd
and than it is enought to have final class X[C: Functor](source: C)
and than it is enought to have final class X[C: Functor](source: C)
ZH
you need just functor implementation for your Rdd
and than it is enought to have final class X[C: Functor](source: C)
and than it is enought to have final class X[C: Functor](source: C)
But this is not just RDD I am talking about Iterator, Array, List here
Oℕ
O have a dynamic computation model based on which I run these transformations
Looks like "dymamic computation model" is "I write my code as if scala had no types"
We can help you with modelling, but we need simple but complete code samples about what your task is
We can help you with modelling, but we need simple but complete code samples about what your task is
ZH
Looks like "dymamic computation model" is "I write my code as if scala had no types"
We can help you with modelling, but we need simple but complete code samples about what your task is
We can help you with modelling, but we need simple but complete code samples about what your task is
Ok I can share that but this is isolated still you guys should be able to answer how to implement different collection types as generics in the same class right?
Oℕ
I think that you'll need to radically change your approach in order to get a typesafe solution.
Oℕ
For example RDD - is not a collection
ZH
I think that you'll need to radically change your approach in order to get a typesafe solution.
Ok now we are getting somewhere so my approach is wrong here right?
Oℕ
It simply can't be understood in terms of locally synchronous iterable container
ZH
How can I implement such methods that can implement transformations on any type?
λ
For example RDD - is not a collection
Are you shure ?
Here is explanation from documentation
A Resilient Distributed Dataset (RDD), the basic abstraction in Spark. Represents an immutable, partitioned collection of elements that can be operated on in parallel
Here is explanation from documentation
A Resilient Distributed Dataset (RDD), the basic abstraction in Spark. Represents an immutable, partitioned collection of elements that can be operated on in parallel
Oℕ
So one approach is to us various specific typeclasses such as mentioned Functor, FunctorFilter, MonoidK, etc.
But this way requires pretty advanced scala knowledge.
Any other way to unify distributed collections with local one is no simplier.
But this way requires pretty advanced scala knowledge.
Any other way to unify distributed collections with local one is no simplier.
ZH
So one approach is to us various specific typeclasses such as mentioned Functor, FunctorFilter, MonoidK, etc.
But this way requires pretty advanced scala knowledge.
Any other way to unify distributed collections with local one is no simplier.
But this way requires pretty advanced scala knowledge.
Any other way to unify distributed collections with local one is no simplier.
I am a beginner in scala
ZH
Are you shure ?
Here is explanation from documentation
A Resilient Distributed Dataset (RDD), the basic abstraction in Spark. Represents an immutable, partitioned collection of elements that can be operated on in parallel
Here is explanation from documentation
A Resilient Distributed Dataset (RDD), the basic abstraction in Spark. Represents an immutable, partitioned collection of elements that can be operated on in parallel
That was my understanding
Oℕ
Are you shure ?
Here is explanation from documentation
A Resilient Distributed Dataset (RDD), the basic abstraction in Spark. Represents an immutable, partitioned collection of elements that can be operated on in parallel
Here is explanation from documentation
A Resilient Distributed Dataset (RDD), the basic abstraction in Spark. Represents an immutable, partitioned collection of elements that can be operated on in parallel
It could be understood as a collection semantically, but not as a collection in scala stdlib sense
Oℕ
What RDD really is : a recipe of a distributed computation