我有这个Java代码:
public class TestMapper extends AppEngineMapper{ public TestMapper() { } // [... other overriden methods ...] @Override public void setup(Context context) { log.warning("Doing per-worker setup"); } }
...我转换为:
class TestMapper extends AppEngineMapper[Key, Entity, NullWritable, NullWritable] { // [... other overriden methods ...] override def setup(context: Context) { log.warning("Doing per-worker setup") } }
现在实际问题:
Context被定义为 org.apache.hadoop.mapreduce.Mapper类中的嵌套类:
public static class Mapper{ //[... some other methods ...] protected void setup(org.apache.hadoop.mapreduce.Mapper .Context context) throws java.io.IOException, java.lang.InterruptedException { /* compiled code */ } public class Context extends org.apache.hadoop.mapreduce.MapContext { public Context(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.mapreduce.TaskAttemptID conf, org.apache.hadoop.mapreduce.RecordReader taskid, org.apache.hadoop.mapreduce.RecordWriter reader, org.apache.hadoop.mapreduce.OutputCommitter writer, org.apache.hadoop.mapreduce.StatusReporter committer, org.apache.hadoop.mapreduce.InputSplit reporter) throws java.io.IOException, java.lang.InterruptedException { /* compiled code */ } }
所以我不能告诉我的Scala类实际上是什么/什么是Context.如果Mapper没有泛型,我可以引用Context via
Mapper#Context
但我如何判断Mapper是否有泛型?
Mapper[_,_,_,_]#Context
......没用
在您的情况下,您必须为您的类型投影提供确切的基本类型
Mapper[Key, Entity, NullWritable, NullWritable]#Context
所以压倒一切setup
都会写成
override def setup(context: Mapper[Key, Entity, NullWritable, NullWritable]#Context)
通过引入类型别名可以简化用法
class TestMapper extends AppEngineMapper[Key, Entity, NullWritable, NullWritable] { type Context = Mapper[Key, Entity, NullWritable, NullWritable]#Context override def setup(context: Context) = { // ... } }
如果您想编写多个映射器,您可以将其重构为可以混合到您的实现中的特征:
trait SMapper[A,B,C,D] extends Mapper[A,B,C,D] { type Context = Mapper[A,B,C,D]#Context } class TestMapper extends AppEngineMapper[Key, Entity, NullWritable, NullWritable] with SMapper[Key, Entity, NullWritable, NullWritable] { override def setup(context: Context) = { // ... } }
或者对于普通的hadoop:
class TestMapper extends SMapper[Key, Entity, NullWritable, NullWritable] { override def setup(context: Context) = { // ... } }