Hadoop giving error on avro tools concat
I'm trying to concatenate multiple avro files in an HDFS directory using avro tools. I'm following the approach as this question. However, I'm getting the following error:
Exception in thread "main" java.io.FileNotFoundException: /user/myuser/output/output.avro (No such file or directory)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
at org.apache.avro.tool.Util.fileOrStdout(Util.java:79)
at org.apache.avro.tool.ConcatTool.run(ConcatTool.java:58)
at org.apache.avro.tool.Main.run(Main.java:80)
at org.apache.avro.tool.Main.main(Main.java:69)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
- The directory /user/myuser/output/ does exist.
- The command I'm using is hadoop jar /home/myuser/avro-tools-1.7.4.jar concat ${IN} /user/myuser/output/output.avro
- $IN has the value /user/hive/warehouse/somedatabase.db/some_table/on_date=2016-05-01/for_date=2016-05-08/confidence=0.8/000000_0
- The hadoop version is Hadoop 2.0.0-cdh4.7.0.
Theoretically, it should work, however it does not. I feel as if avro-tools is checking the local filesystem for the file(directory?) and that is causing the error. Any pointers?