p04896132
/
mindspore
forked from mindspore-Ecosystem/mindspore
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

!736 Skip blank line for TextFileDataset 

Merge pull request !736 from yanghaitao/yht_tftextfile_black_line
This commit is contained in:
mindspore-ci-bot 2020-04-27 19:02:08 +08:00 committed by Gitee
parent 353fa8d763 1cb3bead32
commit e6473f0765
2 changed files with 7 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
mindspore/ccsrc/dataset/engine/datasetops/source/text_file_op.cc
View File

@ -143,6 +143,9 @@ Status TextFileOp::LoadFile(const std::string &file, const int64_t start_offset,
std::unique_ptr<TensorQTable> tensor_table = std::make_unique<TensorQTable>(); std::unique_ptr<TensorQTable> tensor_table = std::make_unique<TensorQTable>();
while (getline(handle, line)) { while (getline(handle, line)) {
if (line.empty()) {
continue;
}
// If read to the end offset of this file, break. // If read to the end offset of this file, break.
if (rows_total >= end_offset) { if (rows_total >= end_offset) {
break; break;
@ -425,7 +428,9 @@ int64_t TextFileOp::CountTotalRows(const std::string &file) {
std::string line; std::string line;
int64_t count = 0; int64_t count = 0;
while (getline(handle, line)) { while (getline(handle, line)) {
count++; if (!line.empty()) {
count++;
}
} }
return count; return count;

1
tests/ut/data/dataset/testTextFileDataset/1.txt
View File

@ -1,3 +1,4 @@
This is a text file. This is a text file.
Be happy every day. Be happy every day.
Good luck to everyone. Good luck to everyone.