`
qun715715
  • 浏览: 11599 次
  • 性别: Icon_minigender_1
  • 来自: 北京
社区版块
存档分类
最新评论

hdfs追加文件

    博客分类:
  • hdfs
阅读更多
hdfs追加文件测试代表如下:
Configuration conf=new Configuration();
conf.setBoolean("dfs.support.append", true);
conf.set("dfs.client.block.write.replace-datanode-on-failure.policy","NEVER");
conf.set("dfs.client.block.write.replace-datanode-on-failure.enable","true");
FileSystem fs = FileSystem.get(new URI("hdfs://192.168.1.46:8020"), conf,"hdfs");

boolean isExist=fs.exists(new Path("/test/test.txt"));
if(isExist){
FSDataOutputStream out = fs.append(new Path("/test/test.txt"));
FileInputStream in = new FileInputStream(new File("d:/test.txt"));
IOUtils.copyBytes(in, out, 2048, true);
}else{
FSDataOutputStream out = fs.create(new Path("/test/test.txt"));
FileInputStream in = new FileInputStream(new File("d:/test.txt"));
IOUtils.copyBytes(in, out, 2048, true);
}

    conf.set("dfs.client.block.write.replace-datanode-on-failure.policy","NEVER");
    conf.set("dfs.client.block.write.replace-datanode-on-failure.enable","true");

    增加以止两个属性是为解决报错: Failed to replace a bad datanode on the existing pipeline due to no more good datanodes being available to try

分享到:
评论

相关推荐

Global site tag (gtag.js) - Google Analytics